2024-09-21T00:15:27.701 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-21T00:15:27.778 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-21T00:15:27.865 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913327 branch: wip-pdonnell-testing-20240920.212106-debug description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/sessionmap} email: pdonnell@redhat.com first_in_suite: false job_id: '7913327' kernel: client: sha1: distro kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-pdonnell-testing-20240920.212106-debug ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd cluster-conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 2 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: client mount timeout: 30 debug client: 20 debug mgr: 20 debug ms: 1 mon warn on pool no app: false mon: mon op complaint time: 120 osd: osd op complaint time: 180 conf: client: pid file: '' global: ms die on skipped message: false mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - fs.*is degraded - filesystem is degraded - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - filesystem is offline - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - insufficient standby MDS daemons available - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - client session with non-allowable root sha1: 1843221895adccfb3b692060251dd6a0183bc410 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 1843221895adccfb3b692060251dd6a0183bc410 kclient: syntax: v1 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-pdonnell-testing-20240920.212106-debug sha1: 1843221895adccfb3b692060251dd6a0183bc410 owner: scheduled_pdonnell@teuthology priority: 90 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 7864 sha1: 1843221895adccfb3b692060251dd6a0183bc410 sleep_before_teardown: 0 subset: 67/512 suite: fs suite_branch: wip-pdonnell-testing-20240920.212106-debug suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 1843221895adccfb3b692060251dd6a0183bc410 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi112.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNzz9m/KbL0HE2cYqytRkGlqSbiyfkuxmJ+Op4aCHHqroao9pIx9+IirS4Nprb5VnEG6c8xr5OZ321BMAQYVIkU= smithi187.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKkHhPgTrEljf4AIYwRQx8iIF/4DMFO1KxevBC8kgwgIugNhsMVxP7c3BiWnPt6XvmvCuzkxgC3HLph9+odyhGs= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - kclient: null - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.test_sessionmap teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-20_23:48:22 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2902 2024-09-21T00:15:27.865 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa; will attempt to use it 2024-09-21T00:15:27.866 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa/tasks 2024-09-21T00:15:27.866 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-21T00:15:27.867 INFO:teuthology.task.internal:Checking packages... 2024-09-21T00:15:27.887 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '1843221895adccfb3b692060251dd6a0183bc410' 2024-09-21T00:15:27.887 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-21T00:15:27.887 INFO:teuthology.packaging:ref: None 2024-09-21T00:15:27.887 INFO:teuthology.packaging:tag: None 2024-09-21T00:15:27.887 INFO:teuthology.packaging:branch: wip-pdonnell-testing-20240920.212106-debug 2024-09-21T00:15:27.887 INFO:teuthology.packaging:sha1: 1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T00:15:27.888 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-pdonnell-testing-20240920.212106-debug 2024-09-21T00:15:28.028 INFO:teuthology.task.internal:Found packages for ceph version 19.3.0-5120.g18432218 2024-09-21T00:15:28.030 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-21T00:15:28.081 INFO:teuthology.task.internal:no buildpackages task found 2024-09-21T00:15:28.081 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-21T00:15:28.087 INFO:teuthology.task.internal:Saving configuration 2024-09-21T00:15:28.106 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-21T00:15:28.116 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-21T00:15:28.142 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi112.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913327', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-21 00:10:30.149100', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNzz9m/KbL0HE2cYqytRkGlqSbiyfkuxmJ+Op4aCHHqroao9pIx9+IirS4Nprb5VnEG6c8xr5OZ321BMAQYVIkU='} 2024-09-21T00:15:28.165 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi187.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913327', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-21 00:10:30.150663', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKkHhPgTrEljf4AIYwRQx8iIF/4DMFO1KxevBC8kgwgIugNhsMVxP7c3BiWnPt6XvmvCuzkxgC3HLph9+odyhGs='} 2024-09-21T00:15:28.165 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-21T00:15:28.181 INFO:teuthology.task.internal:roles: ubuntu@smithi112.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-09-21T00:15:28.181 INFO:teuthology.task.internal:roles: ubuntu@smithi187.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-09-21T00:15:28.181 INFO:teuthology.run_tasks:Running task console_log... 2024-09-21T00:15:28.287 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fe149d88160>, signals=[15]) 2024-09-21T00:15:28.287 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-21T00:15:28.312 INFO:teuthology.task.internal:Opening connections... 2024-09-21T00:15:28.312 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi112.front.sepia.ceph.com 2024-09-21T00:15:28.313 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T00:15:28.387 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi187.front.sepia.ceph.com 2024-09-21T00:15:28.388 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi187.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T00:15:28.462 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-21T00:15:28.472 DEBUG:teuthology.orchestra.run.smithi112:> uname -m 2024-09-21T00:15:28.489 INFO:teuthology.orchestra.run.smithi112.stdout:x86_64 2024-09-21T00:15:28.489 DEBUG:teuthology.orchestra.run.smithi112:> cat /etc/os-release 2024-09-21T00:15:28.546 INFO:teuthology.orchestra.run.smithi112.stdout:NAME="CentOS Stream" 2024-09-21T00:15:28.547 INFO:teuthology.orchestra.run.smithi112.stdout:VERSION="9" 2024-09-21T00:15:28.547 INFO:teuthology.orchestra.run.smithi112.stdout:ID="centos" 2024-09-21T00:15:28.547 INFO:teuthology.orchestra.run.smithi112.stdout:ID_LIKE="rhel fedora" 2024-09-21T00:15:28.547 INFO:teuthology.orchestra.run.smithi112.stdout:VERSION_ID="9" 2024-09-21T00:15:28.547 INFO:teuthology.orchestra.run.smithi112.stdout:PLATFORM_ID="platform:el9" 2024-09-21T00:15:28.547 INFO:teuthology.orchestra.run.smithi112.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-21T00:15:28.547 INFO:teuthology.orchestra.run.smithi112.stdout:ANSI_COLOR="0;31" 2024-09-21T00:15:28.547 INFO:teuthology.orchestra.run.smithi112.stdout:LOGO="fedora-logo-icon" 2024-09-21T00:15:28.547 INFO:teuthology.orchestra.run.smithi112.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-21T00:15:28.547 INFO:teuthology.orchestra.run.smithi112.stdout:HOME_URL="https://centos.org/" 2024-09-21T00:15:28.547 INFO:teuthology.orchestra.run.smithi112.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-21T00:15:28.547 INFO:teuthology.orchestra.run.smithi112.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-21T00:15:28.547 INFO:teuthology.orchestra.run.smithi112.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-21T00:15:28.548 INFO:teuthology.lock.ops:Updating smithi112.front.sepia.ceph.com on lock server 2024-09-21T00:15:28.569 DEBUG:teuthology.orchestra.run.smithi187:> uname -m 2024-09-21T00:15:28.588 INFO:teuthology.orchestra.run.smithi187.stdout:x86_64 2024-09-21T00:15:28.588 DEBUG:teuthology.orchestra.run.smithi187:> cat /etc/os-release 2024-09-21T00:15:28.646 INFO:teuthology.orchestra.run.smithi187.stdout:NAME="CentOS Stream" 2024-09-21T00:15:28.646 INFO:teuthology.orchestra.run.smithi187.stdout:VERSION="9" 2024-09-21T00:15:28.646 INFO:teuthology.orchestra.run.smithi187.stdout:ID="centos" 2024-09-21T00:15:28.646 INFO:teuthology.orchestra.run.smithi187.stdout:ID_LIKE="rhel fedora" 2024-09-21T00:15:28.646 INFO:teuthology.orchestra.run.smithi187.stdout:VERSION_ID="9" 2024-09-21T00:15:28.646 INFO:teuthology.orchestra.run.smithi187.stdout:PLATFORM_ID="platform:el9" 2024-09-21T00:15:28.646 INFO:teuthology.orchestra.run.smithi187.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-21T00:15:28.646 INFO:teuthology.orchestra.run.smithi187.stdout:ANSI_COLOR="0;31" 2024-09-21T00:15:28.646 INFO:teuthology.orchestra.run.smithi187.stdout:LOGO="fedora-logo-icon" 2024-09-21T00:15:28.647 INFO:teuthology.orchestra.run.smithi187.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-21T00:15:28.647 INFO:teuthology.orchestra.run.smithi187.stdout:HOME_URL="https://centos.org/" 2024-09-21T00:15:28.647 INFO:teuthology.orchestra.run.smithi187.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-21T00:15:28.647 INFO:teuthology.orchestra.run.smithi187.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-21T00:15:28.647 INFO:teuthology.orchestra.run.smithi187.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-21T00:15:28.647 INFO:teuthology.lock.ops:Updating smithi187.front.sepia.ceph.com on lock server 2024-09-21T00:15:28.669 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-21T00:15:28.684 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-21T00:15:28.788 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-21T00:15:28.788 DEBUG:teuthology.orchestra.run.smithi112:> test '!' -e /home/ubuntu/cephtest 2024-09-21T00:15:28.791 DEBUG:teuthology.orchestra.run.smithi187:> test '!' -e /home/ubuntu/cephtest 2024-09-21T00:15:28.807 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-21T00:15:28.817 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-21T00:15:28.817 DEBUG:teuthology.orchestra.run.smithi112:> test -z $(ls -A /var/lib/ceph) 2024-09-21T00:15:28.848 DEBUG:teuthology.orchestra.run.smithi187:> test -z $(ls -A /var/lib/ceph) 2024-09-21T00:15:28.935 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-21T00:15:29.013 INFO:teuthology.run_tasks:Running task kernel... 2024-09-21T00:15:29.026 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'} 2024-09-21T00:15:29.026 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}, 'client.2': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}, 'client.3': {'sha1': 'distro'}} 2024-09-21T00:15:29.026 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}}, timeout 300 2024-09-21T00:15:29.027 DEBUG:teuthology.orchestra.run.smithi112:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-21T00:15:29.027 DEBUG:teuthology.orchestra.run.smithi187:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-21T00:15:29.043 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T00:15:29.044 DEBUG:teuthology.orchestra.run.smithi112:> uname -r 2024-09-21T00:15:29.044 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T00:15:29.045 DEBUG:teuthology.orchestra.run.smithi187:> uname -r 2024-09-21T00:15:29.100 INFO:teuthology.orchestra.run.smithi112.stdout:5.14.0-503.el9.x86_64 2024-09-21T00:15:29.101 INFO:teuthology.task.kernel:Running kernel on smithi112: 5.14.0-503.el9.x86_64 2024-09-21T00:15:29.101 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum install -y kernel 2024-09-21T00:15:29.102 INFO:teuthology.orchestra.run.smithi187.stdout:5.14.0-503.el9.x86_64 2024-09-21T00:15:29.103 INFO:teuthology.task.kernel:Running kernel on smithi187: 5.14.0-503.el9.x86_64 2024-09-21T00:15:29.103 DEBUG:teuthology.orchestra.run.smithi187:> sudo yum install -y kernel 2024-09-21T00:15:31.645 INFO:teuthology.orchestra.run.smithi187.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.3 MB 00:00 2024-09-21T00:15:31.675 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - BaseOS 9.8 MB/s | 8.3 MB 00:00 2024-09-21T00:15:33.355 INFO:teuthology.orchestra.run.smithi187.stdout:CentOS Stream 9 - AppStream 32 MB/s | 20 MB 00:00 2024-09-21T00:15:37.630 INFO:teuthology.orchestra.run.smithi187.stdout:CentOS Stream 9 - CRB 13 MB/s | 6.5 MB 00:00 2024-09-21T00:15:39.566 INFO:teuthology.orchestra.run.smithi187.stdout:CentOS Stream 9 - Extras packages 27 kB/s | 19 kB 00:00 2024-09-21T00:15:40.369 INFO:teuthology.orchestra.run.smithi187.stdout:Extra Packages for Enterprise Linux 33 MB/s | 23 MB 00:00 2024-09-21T00:15:42.034 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - AppStream 2.2 MB/s | 20 MB 00:09 2024-09-21T00:15:45.791 INFO:teuthology.orchestra.run.smithi187.stdout:lab-extras 26 kB/s | 1.7 kB 00:00 2024-09-21T00:15:46.343 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - CRB 13 MB/s | 6.5 MB 00:00 2024-09-21T00:15:47.706 INFO:teuthology.orchestra.run.smithi187.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-21T00:15:47.706 INFO:teuthology.orchestra.run.smithi187.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-21T00:15:47.819 INFO:teuthology.orchestra.run.smithi187.stdout:Dependencies resolved. 2024-09-21T00:15:47.824 INFO:teuthology.orchestra.run.smithi187.stdout:================================================================================ 2024-09-21T00:15:47.824 INFO:teuthology.orchestra.run.smithi187.stdout: Package Architecture Version Repository Size 2024-09-21T00:15:47.824 INFO:teuthology.orchestra.run.smithi187.stdout:================================================================================ 2024-09-21T00:15:47.824 INFO:teuthology.orchestra.run.smithi187.stdout:Installing: 2024-09-21T00:15:47.824 INFO:teuthology.orchestra.run.smithi187.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-21T00:15:47.824 INFO:teuthology.orchestra.run.smithi187.stdout:Installing dependencies: 2024-09-21T00:15:47.824 INFO:teuthology.orchestra.run.smithi187.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-21T00:15:47.824 INFO:teuthology.orchestra.run.smithi187.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-21T00:15:47.825 INFO:teuthology.orchestra.run.smithi187.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-21T00:15:47.825 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:15:47.825 INFO:teuthology.orchestra.run.smithi187.stdout:Transaction Summary 2024-09-21T00:15:47.825 INFO:teuthology.orchestra.run.smithi187.stdout:================================================================================ 2024-09-21T00:15:47.825 INFO:teuthology.orchestra.run.smithi187.stdout:Install 4 Packages 2024-09-21T00:15:47.825 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:15:47.825 INFO:teuthology.orchestra.run.smithi187.stdout:Total download size: 79 M 2024-09-21T00:15:47.825 INFO:teuthology.orchestra.run.smithi187.stdout:Installed size: 126 M 2024-09-21T00:15:47.826 INFO:teuthology.orchestra.run.smithi187.stdout:Downloading Packages: 2024-09-21T00:15:48.135 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - Extras packages 41 kB/s | 19 kB 00:00 2024-09-21T00:15:48.442 INFO:teuthology.orchestra.run.smithi187.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 63 kB/s | 36 kB 00:00 2024-09-21T00:15:48.940 INFO:teuthology.orchestra.run.smithi112.stdout:Extra Packages for Enterprise Linux 33 MB/s | 23 MB 00:00 2024-09-21T00:15:49.376 INFO:teuthology.orchestra.run.smithi187.stdout:(2/4): kernel-core-5.14.0-508.el9.x86_64.rpm 10 MB/s | 16 MB 00:01 2024-09-21T00:15:49.934 INFO:teuthology.orchestra.run.smithi187.stdout:(3/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 17 MB/s | 35 MB 00:02 2024-09-21T00:15:50.701 INFO:teuthology.orchestra.run.smithi187.stdout:(4/4): kernel-modules-core-5.14.0-508.el9.x86_6 13 MB/s | 29 MB 00:02 2024-09-21T00:15:50.702 INFO:teuthology.orchestra.run.smithi187.stdout:-------------------------------------------------------------------------------- 2024-09-21T00:15:50.702 INFO:teuthology.orchestra.run.smithi187.stdout:Total 27 MB/s | 79 MB 00:02 2024-09-21T00:15:51.387 INFO:teuthology.orchestra.run.smithi187.stdout:Running transaction check 2024-09-21T00:15:51.734 INFO:teuthology.orchestra.run.smithi187.stdout:Transaction check succeeded. 2024-09-21T00:15:51.734 INFO:teuthology.orchestra.run.smithi187.stdout:Running transaction test 2024-09-21T00:15:52.707 INFO:teuthology.orchestra.run.smithi187.stdout:Transaction test succeeded. 2024-09-21T00:15:52.708 INFO:teuthology.orchestra.run.smithi187.stdout:Running transaction 2024-09-21T00:15:54.262 INFO:teuthology.orchestra.run.smithi112.stdout:lab-extras 18 kB/s | 1.7 kB 00:00 2024-09-21T00:15:55.165 INFO:teuthology.orchestra.run.smithi187.stdout: Preparing : 1/1 2024-09-21T00:15:55.808 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-21T00:15:55.901 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-21T00:15:56.144 INFO:teuthology.orchestra.run.smithi112.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-21T00:15:56.144 INFO:teuthology.orchestra.run.smithi112.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-21T00:15:56.246 INFO:teuthology.orchestra.run.smithi112.stdout:Dependencies resolved. 2024-09-21T00:15:56.256 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2024-09-21T00:15:56.256 INFO:teuthology.orchestra.run.smithi112.stdout: Package Architecture Version Repository Size 2024-09-21T00:15:56.256 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2024-09-21T00:15:56.256 INFO:teuthology.orchestra.run.smithi112.stdout:Installing: 2024-09-21T00:15:56.256 INFO:teuthology.orchestra.run.smithi112.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-21T00:15:56.256 INFO:teuthology.orchestra.run.smithi112.stdout:Installing dependencies: 2024-09-21T00:15:56.257 INFO:teuthology.orchestra.run.smithi112.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-21T00:15:56.257 INFO:teuthology.orchestra.run.smithi112.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-21T00:15:56.257 INFO:teuthology.orchestra.run.smithi112.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-21T00:15:56.257 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:15:56.257 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction Summary 2024-09-21T00:15:56.257 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2024-09-21T00:15:56.257 INFO:teuthology.orchestra.run.smithi112.stdout:Install 4 Packages 2024-09-21T00:15:56.258 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:15:56.258 INFO:teuthology.orchestra.run.smithi112.stdout:Total download size: 79 M 2024-09-21T00:15:56.258 INFO:teuthology.orchestra.run.smithi112.stdout:Installed size: 126 M 2024-09-21T00:15:56.258 INFO:teuthology.orchestra.run.smithi112.stdout:Downloading Packages: 2024-09-21T00:15:56.572 INFO:teuthology.orchestra.run.smithi112.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 128 kB/s | 36 kB 00:00 2024-09-21T00:15:58.665 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-21T00:15:58.803 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-21T00:16:03.611 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-21T00:16:03.653 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-21T00:16:03.997 INFO:teuthology.orchestra.run.smithi112.stdout:(2/4): kernel-modules-core-5.14.0-508.el9.x86_6 3.8 MB/s | 29 MB 00:07 2024-09-21T00:16:04.238 INFO:teuthology.orchestra.run.smithi112.stdout:(3/4): kernel-core-5.14.0-508.el9.x86_64.rpm 2.0 MB/s | 16 MB 00:07 2024-09-21T00:16:06.064 INFO:teuthology.orchestra.run.smithi112.stdout:(4/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 3.5 MB/s | 35 MB 00:09 2024-09-21T00:16:06.065 INFO:teuthology.orchestra.run.smithi112.stdout:-------------------------------------------------------------------------------- 2024-09-21T00:16:06.065 INFO:teuthology.orchestra.run.smithi112.stdout:Total 8.0 MB/s | 79 MB 00:09 2024-09-21T00:16:06.742 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction check 2024-09-21T00:16:07.079 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction check succeeded. 2024-09-21T00:16:07.080 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction test 2024-09-21T00:16:08.045 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction test succeeded. 2024-09-21T00:16:08.045 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction 2024-09-21T00:16:08.496 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-21T00:16:10.456 INFO:teuthology.orchestra.run.smithi112.stdout: Preparing : 1/1 2024-09-21T00:16:11.106 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-21T00:16:11.181 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-21T00:16:13.934 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-21T00:16:14.060 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-21T00:16:18.866 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-21T00:16:18.978 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-21T00:16:23.820 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-21T00:16:46.900 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-21T00:16:46.940 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-21T00:16:47.341 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-21T00:16:47.341 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-21T00:16:47.341 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-21T00:16:47.341 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-21T00:16:48.229 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-21T00:16:48.229 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:16:48.229 INFO:teuthology.orchestra.run.smithi187.stdout:Installed: 2024-09-21T00:16:48.229 INFO:teuthology.orchestra.run.smithi187.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-21T00:16:48.229 INFO:teuthology.orchestra.run.smithi187.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-21T00:16:48.229 INFO:teuthology.orchestra.run.smithi187.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-21T00:16:48.230 INFO:teuthology.orchestra.run.smithi187.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-21T00:16:48.230 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:16:48.230 INFO:teuthology.orchestra.run.smithi187.stdout:Complete! 2024-09-21T00:16:48.602 DEBUG:teuthology.orchestra.run.smithi187:> rpm -q kernel | sort -rV | head -n 1 2024-09-21T00:16:48.693 INFO:teuthology.orchestra.run.smithi187.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-21T00:16:48.693 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-21T00:16:48.693 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-21T00:16:48.693 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-21T00:16:48.693 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-21T00:16:48.694 INFO:teuthology.task.kernel:Installing distro kernel on client.1... 2024-09-21T00:16:48.694 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi187.front.sepia.ceph.com, path=None, version=distro) 2024-09-21T00:16:48.694 DEBUG:teuthology.orchestra.run.smithi187:> rpm -q kernel | sort -rV | head -n 1 2024-09-21T00:16:48.752 INFO:teuthology.orchestra.run.smithi187.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-21T00:16:48.752 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-21T00:16:48.752 DEBUG:teuthology.orchestra.run.smithi187:> sudo rpm -qi grub2-tools 2024-09-21T00:16:48.784 INFO:teuthology.orchestra.run.smithi187.stdout:Name : grub2-tools 2024-09-21T00:16:48.784 INFO:teuthology.orchestra.run.smithi187.stdout:Epoch : 1 2024-09-21T00:16:48.784 INFO:teuthology.orchestra.run.smithi187.stdout:Version : 2.06 2024-09-21T00:16:48.784 INFO:teuthology.orchestra.run.smithi187.stdout:Release : 82.el9 2024-09-21T00:16:48.784 INFO:teuthology.orchestra.run.smithi187.stdout:Architecture: x86_64 2024-09-21T00:16:48.785 INFO:teuthology.orchestra.run.smithi187.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-21T00:16:48.785 INFO:teuthology.orchestra.run.smithi187.stdout:Group : Unspecified 2024-09-21T00:16:48.785 INFO:teuthology.orchestra.run.smithi187.stdout:Size : 8274670 2024-09-21T00:16:48.785 INFO:teuthology.orchestra.run.smithi187.stdout:License : GPLv3+ 2024-09-21T00:16:48.785 INFO:teuthology.orchestra.run.smithi187.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-21T00:16:48.785 INFO:teuthology.orchestra.run.smithi187.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-21T00:16:48.785 INFO:teuthology.orchestra.run.smithi187.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-21T00:16:48.785 INFO:teuthology.orchestra.run.smithi187.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-21T00:16:48.785 INFO:teuthology.orchestra.run.smithi187.stdout:Packager : builder@centos.org 2024-09-21T00:16:48.785 INFO:teuthology.orchestra.run.smithi187.stdout:Vendor : CentOS 2024-09-21T00:16:48.785 INFO:teuthology.orchestra.run.smithi187.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-21T00:16:48.785 INFO:teuthology.orchestra.run.smithi187.stdout:Summary : Support tools for GRUB. 2024-09-21T00:16:48.785 INFO:teuthology.orchestra.run.smithi187.stdout:Description : 2024-09-21T00:16:48.785 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:16:48.785 INFO:teuthology.orchestra.run.smithi187.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-21T00:16:48.786 INFO:teuthology.orchestra.run.smithi187.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-21T00:16:48.786 INFO:teuthology.orchestra.run.smithi187.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-21T00:16:48.786 INFO:teuthology.orchestra.run.smithi187.stdout:hardware devices. 2024-09-21T00:16:48.786 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:16:48.786 INFO:teuthology.orchestra.run.smithi187.stdout:This subpackage provides tools for support of all platforms. 2024-09-21T00:16:48.787 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-21T00:16:48.787 INFO:teuthology.task.kernel:Updating grub on smithi187 to boot 5.14.0-508.el9.x86_64 2024-09-21T00:16:48.787 DEBUG:teuthology.orchestra.run.smithi187:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-21T00:16:49.409 INFO:teuthology.orchestra.run.smithi187.stderr:Generating grub configuration file ... 2024-09-21T00:16:50.757 INFO:teuthology.orchestra.run.smithi187.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-21T00:16:50.794 INFO:teuthology.orchestra.run.smithi187.stderr:done 2024-09-21T00:16:50.796 DEBUG:teuthology.orchestra.run.smithi187:> mktemp 2024-09-21T00:16:50.812 INFO:teuthology.orchestra.run.smithi187.stdout:/tmp/tmp.LoKtfoaXpy 2024-09-21T00:16:50.813 DEBUG:teuthology.orchestra.run.smithi187:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.LoKtfoaXpy 2024-09-21T00:16:50.878 DEBUG:teuthology.orchestra.run.smithi187:> sudo chmod 0666 /tmp/tmp.LoKtfoaXpy 2024-09-21T00:16:51.021 DEBUG:teuthology.orchestra.remote:smithi187:/tmp/tmp.LoKtfoaXpy is 6KB 2024-09-21T00:16:51.069 DEBUG:teuthology.orchestra.run.smithi187:> rm -fr /tmp/tmp.LoKtfoaXpy 2024-09-21T00:16:51.084 DEBUG:teuthology.orchestra.run.smithi187:> sudo /bin/ls /boot/loader/entries || true 2024-09-21T00:16:51.146 INFO:teuthology.orchestra.run.smithi187.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-21T00:16:51.146 INFO:teuthology.orchestra.run.smithi187.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-21T00:16:51.146 INFO:teuthology.orchestra.run.smithi187.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-21T00:16:51.146 INFO:teuthology.orchestra.run.smithi187.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-21T00:16:51.148 DEBUG:teuthology.orchestra.run.smithi187:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-21T00:16:51.305 DEBUG:teuthology.orchestra.run.smithi187:> sudo shutdown -r now 2024-09-21T00:17:01.649 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-21T00:17:01.690 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-21T00:17:02.069 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-21T00:17:02.069 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-21T00:17:02.069 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-21T00:17:02.069 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-21T00:17:02.890 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-21T00:17:02.890 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:17:02.890 INFO:teuthology.orchestra.run.smithi112.stdout:Installed: 2024-09-21T00:17:02.890 INFO:teuthology.orchestra.run.smithi112.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-21T00:17:02.890 INFO:teuthology.orchestra.run.smithi112.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-21T00:17:02.890 INFO:teuthology.orchestra.run.smithi112.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-21T00:17:02.890 INFO:teuthology.orchestra.run.smithi112.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-21T00:17:02.890 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:17:02.890 INFO:teuthology.orchestra.run.smithi112.stdout:Complete! 2024-09-21T00:17:03.255 DEBUG:teuthology.orchestra.run.smithi112:> rpm -q kernel | sort -rV | head -n 1 2024-09-21T00:17:03.337 INFO:teuthology.orchestra.run.smithi112.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-21T00:17:03.337 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-21T00:17:03.337 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-21T00:17:03.337 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-21T00:17:03.337 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-21T00:17:03.337 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-09-21T00:17:03.337 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi112.front.sepia.ceph.com, path=None, version=distro) 2024-09-21T00:17:03.337 DEBUG:teuthology.orchestra.run.smithi112:> rpm -q kernel | sort -rV | head -n 1 2024-09-21T00:17:03.396 INFO:teuthology.orchestra.run.smithi112.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-21T00:17:03.396 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-21T00:17:03.396 DEBUG:teuthology.orchestra.run.smithi112:> sudo rpm -qi grub2-tools 2024-09-21T00:17:03.434 INFO:teuthology.orchestra.run.smithi112.stdout:Name : grub2-tools 2024-09-21T00:17:03.435 INFO:teuthology.orchestra.run.smithi112.stdout:Epoch : 1 2024-09-21T00:17:03.435 INFO:teuthology.orchestra.run.smithi112.stdout:Version : 2.06 2024-09-21T00:17:03.435 INFO:teuthology.orchestra.run.smithi112.stdout:Release : 82.el9 2024-09-21T00:17:03.435 INFO:teuthology.orchestra.run.smithi112.stdout:Architecture: x86_64 2024-09-21T00:17:03.435 INFO:teuthology.orchestra.run.smithi112.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-21T00:17:03.435 INFO:teuthology.orchestra.run.smithi112.stdout:Group : Unspecified 2024-09-21T00:17:03.435 INFO:teuthology.orchestra.run.smithi112.stdout:Size : 8274670 2024-09-21T00:17:03.435 INFO:teuthology.orchestra.run.smithi112.stdout:License : GPLv3+ 2024-09-21T00:17:03.435 INFO:teuthology.orchestra.run.smithi112.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-21T00:17:03.435 INFO:teuthology.orchestra.run.smithi112.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-21T00:17:03.435 INFO:teuthology.orchestra.run.smithi112.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-21T00:17:03.435 INFO:teuthology.orchestra.run.smithi112.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-21T00:17:03.436 INFO:teuthology.orchestra.run.smithi112.stdout:Packager : builder@centos.org 2024-09-21T00:17:03.436 INFO:teuthology.orchestra.run.smithi112.stdout:Vendor : CentOS 2024-09-21T00:17:03.436 INFO:teuthology.orchestra.run.smithi112.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-21T00:17:03.436 INFO:teuthology.orchestra.run.smithi112.stdout:Summary : Support tools for GRUB. 2024-09-21T00:17:03.436 INFO:teuthology.orchestra.run.smithi112.stdout:Description : 2024-09-21T00:17:03.436 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:17:03.436 INFO:teuthology.orchestra.run.smithi112.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-21T00:17:03.436 INFO:teuthology.orchestra.run.smithi112.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-21T00:17:03.436 INFO:teuthology.orchestra.run.smithi112.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-21T00:17:03.436 INFO:teuthology.orchestra.run.smithi112.stdout:hardware devices. 2024-09-21T00:17:03.436 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:17:03.436 INFO:teuthology.orchestra.run.smithi112.stdout:This subpackage provides tools for support of all platforms. 2024-09-21T00:17:03.437 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-21T00:17:03.437 INFO:teuthology.task.kernel:Updating grub on smithi112 to boot 5.14.0-508.el9.x86_64 2024-09-21T00:17:03.437 DEBUG:teuthology.orchestra.run.smithi112:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-21T00:17:04.052 INFO:teuthology.orchestra.run.smithi112.stderr:Generating grub configuration file ... 2024-09-21T00:17:05.315 INFO:teuthology.orchestra.run.smithi112.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-21T00:17:05.335 INFO:teuthology.orchestra.run.smithi112.stderr:done 2024-09-21T00:17:05.338 DEBUG:teuthology.orchestra.run.smithi112:> mktemp 2024-09-21T00:17:05.353 INFO:teuthology.orchestra.run.smithi112.stdout:/tmp/tmp.m3SUgu9yxq 2024-09-21T00:17:05.354 DEBUG:teuthology.orchestra.run.smithi112:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.m3SUgu9yxq 2024-09-21T00:17:05.430 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod 0666 /tmp/tmp.m3SUgu9yxq 2024-09-21T00:17:05.576 DEBUG:teuthology.orchestra.remote:smithi112:/tmp/tmp.m3SUgu9yxq is 6KB 2024-09-21T00:17:05.624 DEBUG:teuthology.orchestra.run.smithi112:> rm -fr /tmp/tmp.m3SUgu9yxq 2024-09-21T00:17:05.639 DEBUG:teuthology.orchestra.run.smithi112:> sudo /bin/ls /boot/loader/entries || true 2024-09-21T00:17:05.701 INFO:teuthology.orchestra.run.smithi112.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-21T00:17:05.702 INFO:teuthology.orchestra.run.smithi112.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-21T00:17:05.702 INFO:teuthology.orchestra.run.smithi112.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-21T00:17:05.702 INFO:teuthology.orchestra.run.smithi112.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-21T00:17:05.703 DEBUG:teuthology.orchestra.run.smithi112:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-21T00:17:05.846 DEBUG:teuthology.orchestra.run.smithi112:> sudo shutdown -r now 2024-09-21T00:17:21.322 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2024-09-21T00:17:21.323 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi187.front.sepia.ceph.com' 2024-09-21T00:17:21.324 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi187.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T00:17:35.862 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-09-21T00:17:35.863 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi112.front.sepia.ceph.com' 2024-09-21T00:17:35.863 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T00:17:39.827 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.187 2024-09-21T00:17:48.834 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi187.front.sepia.ceph.com' 2024-09-21T00:17:48.835 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi187.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T00:17:51.891 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.187 2024-09-21T00:17:54.415 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.112 2024-09-21T00:18:03.418 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi112.front.sepia.ceph.com' 2024-09-21T00:18:03.419 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T00:18:03.893 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi187.front.sepia.ceph.com' 2024-09-21T00:18:03.894 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi187.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T00:18:22.319 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.187 2024-09-21T00:18:37.330 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi187.front.sepia.ceph.com' 2024-09-21T00:18:37.331 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi187.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T00:18:37.944 DEBUG:teuthology.orchestra.run.smithi187:> true 2024-09-21T00:18:38.501 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi187.front.sepia.ceph.com' 2024-09-21T00:18:38.501 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "5.14.0-508.el9.x86_64"... 2024-09-21T00:18:38.501 DEBUG:teuthology.orchestra.run.smithi187:> uname -r 2024-09-21T00:18:38.549 INFO:teuthology.orchestra.run.smithi187.stdout:5.14.0-508.el9.x86_64 2024-09-21T00:18:38.550 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-21T00:18:38.550 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-21T00:18:38.550 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-21T00:18:39.551 DEBUG:teuthology.parallel:result is None 2024-09-21T00:19:03.421 DEBUG:teuthology.orchestra.remote:timed out 2024-09-21T00:19:15.422 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi112.front.sepia.ceph.com' 2024-09-21T00:19:15.423 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T00:19:15.731 DEBUG:teuthology.orchestra.run.smithi112:> true 2024-09-21T00:19:16.080 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi112.front.sepia.ceph.com' 2024-09-21T00:19:16.080 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-508.el9.x86_64"... 2024-09-21T00:19:16.080 DEBUG:teuthology.orchestra.run.smithi112:> uname -r 2024-09-21T00:19:16.096 INFO:teuthology.orchestra.run.smithi112.stdout:5.14.0-508.el9.x86_64 2024-09-21T00:19:16.096 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-21T00:19:16.096 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-21T00:19:16.096 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-21T00:19:17.097 DEBUG:teuthology.parallel:result is None 2024-09-21T00:19:17.097 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-21T00:19:17.111 INFO:teuthology.task.internal:Creating test directory... 2024-09-21T00:19:17.112 DEBUG:teuthology.orchestra.run.smithi112:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-21T00:19:17.114 DEBUG:teuthology.orchestra.run.smithi187:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-21T00:19:17.134 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-21T00:19:17.151 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-21T00:19:17.172 INFO:teuthology.task.internal:Creating archive directory... 2024-09-21T00:19:17.172 DEBUG:teuthology.orchestra.run.smithi112:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-21T00:19:17.175 DEBUG:teuthology.orchestra.run.smithi187:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-21T00:19:17.210 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-21T00:19:17.216 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-21T00:19:17.216 DEBUG:teuthology.orchestra.run.smithi112:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-21T00:19:17.248 DEBUG:teuthology.orchestra.run.smithi187:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-21T00:19:17.338 INFO:teuthology.orchestra.run.smithi112.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T00:19:17.349 INFO:teuthology.orchestra.run.smithi187.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T00:19:17.362 INFO:teuthology.orchestra.run.smithi187.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T00:19:17.373 INFO:teuthology.orchestra.run.smithi112.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T00:19:17.375 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-21T00:19:17.382 INFO:teuthology.task.internal:Configuring sudo... 2024-09-21T00:19:17.382 DEBUG:teuthology.orchestra.run.smithi112:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-21T00:19:17.384 DEBUG:teuthology.orchestra.run.smithi187:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-21T00:19:17.412 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-21T00:19:17.421 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-21T00:19:17.421 DEBUG:teuthology.orchestra.run.smithi112:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-21T00:19:17.453 DEBUG:teuthology.orchestra.run.smithi187:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-21T00:19:17.469 DEBUG:teuthology.orchestra.run.smithi112:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T00:19:17.548 DEBUG:teuthology.orchestra.run.smithi112:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T00:19:17.608 DEBUG:teuthology.orchestra.run.smithi112:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T00:19:17.691 DEBUG:teuthology.orchestra.run.smithi112:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T00:19:17.730 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T00:19:17.731 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-21T00:19:17.795 DEBUG:teuthology.orchestra.run.smithi187:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T00:19:17.841 DEBUG:teuthology.orchestra.run.smithi187:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T00:19:17.891 DEBUG:teuthology.orchestra.run.smithi187:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T00:19:17.975 DEBUG:teuthology.orchestra.run.smithi187:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T00:19:18.017 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-09-21T00:19:18.017 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-21T00:19:18.083 DEBUG:teuthology.orchestra.run.smithi112:> sudo service rsyslog restart 2024-09-21T00:19:18.085 DEBUG:teuthology.orchestra.run.smithi187:> sudo service rsyslog restart 2024-09-21T00:19:18.128 INFO:teuthology.orchestra.run.smithi112.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T00:19:18.176 INFO:teuthology.orchestra.run.smithi187.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T00:19:18.455 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-21T00:19:18.463 INFO:teuthology.task.internal:Starting timer... 2024-09-21T00:19:18.463 INFO:teuthology.run_tasks:Running task pcp... 2024-09-21T00:19:18.472 INFO:teuthology.run_tasks:Running task selinux... 2024-09-21T00:19:18.480 DEBUG:teuthology.orchestra.run.smithi112:> sudo service auditd rotate 2024-09-21T00:19:18.528 INFO:teuthology.orchestra.run.smithi112.stdout:Rotating logs: 2024-09-21T00:19:18.530 DEBUG:teuthology.orchestra.run.smithi187:> sudo service auditd rotate 2024-09-21T00:19:18.582 INFO:teuthology.orchestra.run.smithi187.stdout:Rotating logs: 2024-09-21T00:19:18.584 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-21T00:19:18.584 DEBUG:teuthology.orchestra.run.smithi112:> /usr/sbin/getenforce 2024-09-21T00:19:18.607 INFO:teuthology.orchestra.run.smithi112.stdout:Permissive 2024-09-21T00:19:18.607 DEBUG:teuthology.orchestra.run.smithi187:> /usr/sbin/getenforce 2024-09-21T00:19:18.633 INFO:teuthology.orchestra.run.smithi187.stdout:Permissive 2024-09-21T00:19:18.633 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi112.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi187.front.sepia.ceph.com': 'permissive'} 2024-09-21T00:19:18.633 DEBUG:teuthology.orchestra.run.smithi112:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-21T00:19:18.673 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T00:19:18.674 DEBUG:teuthology.orchestra.run.smithi187:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-21T00:19:18.698 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T00:19:18.699 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-21T00:19:18.699 DEBUG:teuthology.orchestra.run.smithi112:> sudo /usr/sbin/setenforce permissive 2024-09-21T00:19:18.743 DEBUG:teuthology.orchestra.run.smithi187:> sudo /usr/sbin/setenforce permissive 2024-09-21T00:19:18.772 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-21T00:19:18.785 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-09-21T00:19:18.788 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-09-21T00:19:18.912 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-21T00:19:18.922 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-21T00:19:18.923 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi112.front.sepia.ceph.com,smithi187.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-21T00:23:02.015 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi112.front.sepia.ceph.com'), Remote(name='ubuntu@smithi187.front.sepia.ceph.com')] 2024-09-21T00:23:02.017 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi112.front.sepia.ceph.com' 2024-09-21T00:23:02.018 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T00:23:02.096 DEBUG:teuthology.orchestra.run.smithi112:> true 2024-09-21T00:23:02.182 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi112.front.sepia.ceph.com' 2024-09-21T00:23:02.185 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi187.front.sepia.ceph.com' 2024-09-21T00:23:02.186 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi187.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T00:23:02.261 DEBUG:teuthology.orchestra.run.smithi187:> true 2024-09-21T00:23:02.358 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi187.front.sepia.ceph.com' 2024-09-21T00:23:02.358 INFO:teuthology.run_tasks:Running task clock... 2024-09-21T00:23:02.372 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-21T00:23:02.372 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-21T00:23:02.372 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T00:23:02.375 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-21T00:23:02.375 DEBUG:teuthology.orchestra.run.smithi187:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T00:23:02.402 INFO:teuthology.orchestra.run.smithi112.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-21T00:23:02.417 INFO:teuthology.orchestra.run.smithi112.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-21T00:23:02.427 INFO:teuthology.orchestra.run.smithi187.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-21T00:23:02.442 INFO:teuthology.orchestra.run.smithi187.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-21T00:23:02.442 INFO:teuthology.orchestra.run.smithi112.stderr:sudo: ntpd: command not found 2024-09-21T00:23:02.453 INFO:teuthology.orchestra.run.smithi112.stdout:506 Cannot talk to daemon 2024-09-21T00:23:02.466 INFO:teuthology.orchestra.run.smithi187.stderr:sudo: ntpd: command not found 2024-09-21T00:23:02.467 INFO:teuthology.orchestra.run.smithi112.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-21T00:23:02.477 INFO:teuthology.orchestra.run.smithi187.stdout:506 Cannot talk to daemon 2024-09-21T00:23:02.481 INFO:teuthology.orchestra.run.smithi112.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-21T00:23:02.491 INFO:teuthology.orchestra.run.smithi187.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-21T00:23:02.505 INFO:teuthology.orchestra.run.smithi187.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-21T00:23:02.537 INFO:teuthology.orchestra.run.smithi112.stderr:bash: line 1: ntpq: command not found 2024-09-21T00:23:02.540 INFO:teuthology.orchestra.run.smithi112.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T00:23:02.541 INFO:teuthology.orchestra.run.smithi112.stdout:=============================================================================== 2024-09-21T00:23:02.541 INFO:teuthology.orchestra.run.smithi112.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T00:23:02.541 INFO:teuthology.orchestra.run.smithi112.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T00:23:02.541 INFO:teuthology.orchestra.run.smithi112.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T00:23:02.541 INFO:teuthology.orchestra.run.smithi112.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T00:23:02.562 INFO:teuthology.orchestra.run.smithi187.stderr:bash: line 1: ntpq: command not found 2024-09-21T00:23:02.565 INFO:teuthology.orchestra.run.smithi187.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T00:23:02.566 INFO:teuthology.orchestra.run.smithi187.stdout:=============================================================================== 2024-09-21T00:23:02.566 INFO:teuthology.orchestra.run.smithi187.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T00:23:02.566 INFO:teuthology.orchestra.run.smithi187.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T00:23:02.566 INFO:teuthology.orchestra.run.smithi187.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T00:23:02.566 INFO:teuthology.orchestra.run.smithi187.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T00:23:02.566 INFO:teuthology.run_tasks:Running task install... 2024-09-21T00:23:02.575 DEBUG:teuthology.task.install:project ceph 2024-09-21T00:23:02.575 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410'}} 2024-09-21T00:23:02.576 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410'} 2024-09-21T00:23:02.576 INFO:teuthology.task.install:Using flavor: default 2024-09-21T00:23:02.581 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-21T00:23:02.581 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-21T00:23:02.582 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'tag': None, 'wait_for_package': False} 2024-09-21T00:23:02.582 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T00:23:02.584 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'tag': None, 'wait_for_package': False} 2024-09-21T00:23:02.584 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T00:23:02.736 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/ 2024-09-21T00:23:02.736 INFO:teuthology.task.install.rpm:Package version is 19.3.0-5120.g18432218 2024-09-21T00:23:02.749 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/ 2024-09-21T00:23:02.749 INFO:teuthology.task.install.rpm:Package version is 19.3.0-5120.g18432218 2024-09-21T00:23:02.842 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-21T00:23:02.843 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-09-21T00:23:02.843 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-21T00:23:02.856 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-21T00:23:02.856 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T00:23:02.856 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-21T00:23:02.868 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-21T00:23:02.869 DEBUG:teuthology.orchestra.run.smithi187:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1843221895adccfb3b692060251dd6a0183bc410/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-21T00:23:02.883 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-21T00:23:02.884 DEBUG:teuthology.orchestra.run.smithi112:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1843221895adccfb3b692060251dd6a0183bc410/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-21T00:23:02.934 DEBUG:teuthology.orchestra.run.smithi187:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-21T00:23:02.949 DEBUG:teuthology.orchestra.run.smithi112:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-21T00:23:03.009 DEBUG:teuthology.orchestra.run.smithi187:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-21T00:23:03.025 DEBUG:teuthology.orchestra.run.smithi112:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-21T00:23:03.073 INFO:teuthology.orchestra.run.smithi187.stdout:check_obsoletes = 1 2024-09-21T00:23:03.074 DEBUG:teuthology.orchestra.run.smithi187:> sudo yum clean all 2024-09-21T00:23:03.089 INFO:teuthology.orchestra.run.smithi112.stdout:check_obsoletes = 1 2024-09-21T00:23:03.090 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum clean all 2024-09-21T00:23:03.301 INFO:teuthology.orchestra.run.smithi187.stdout:45 files removed 2024-09-21T00:23:03.324 INFO:teuthology.orchestra.run.smithi112.stdout:45 files removed 2024-09-21T00:23:03.330 DEBUG:teuthology.orchestra.run.smithi187:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-21T00:23:03.350 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-21T00:23:03.889 INFO:teuthology.orchestra.run.smithi187.stdout:ceph packages for x86_64 308 kB/s | 84 kB 00:00 2024-09-21T00:23:03.897 INFO:teuthology.orchestra.run.smithi112.stdout:ceph packages for x86_64 321 kB/s | 84 kB 00:00 2024-09-21T00:23:04.173 INFO:teuthology.orchestra.run.smithi112.stdout:ceph noarch packages 104 kB/s | 24 kB 00:00 2024-09-21T00:23:04.182 INFO:teuthology.orchestra.run.smithi187.stdout:ceph noarch packages 98 kB/s | 24 kB 00:00 2024-09-21T00:23:04.440 INFO:teuthology.orchestra.run.smithi112.stdout:ceph source packages 8.3 kB/s | 1.9 kB 00:00 2024-09-21T00:23:04.457 INFO:teuthology.orchestra.run.smithi187.stdout:ceph source packages 8.0 kB/s | 1.9 kB 00:00 2024-09-21T00:23:04.949 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - BaseOS 17 MB/s | 8.3 MB 00:00 2024-09-21T00:23:05.924 INFO:teuthology.orchestra.run.smithi187.stdout:CentOS Stream 9 - BaseOS 5.8 MB/s | 8.3 MB 00:01 2024-09-21T00:23:07.794 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - AppStream 11 MB/s | 20 MB 00:01 2024-09-21T00:23:09.025 INFO:teuthology.orchestra.run.smithi187.stdout:CentOS Stream 9 - AppStream 10 MB/s | 20 MB 00:02 2024-09-21T00:23:12.103 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - CRB 13 MB/s | 6.5 MB 00:00 2024-09-21T00:23:13.521 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - Extras packages 107 kB/s | 19 kB 00:00 2024-09-21T00:23:13.818 INFO:teuthology.orchestra.run.smithi187.stdout:CentOS Stream 9 - CRB 6.6 MB/s | 6.5 MB 00:00 2024-09-21T00:23:14.383 INFO:teuthology.orchestra.run.smithi112.stdout:Extra Packages for Enterprise Linux 31 MB/s | 23 MB 00:00 2024-09-21T00:23:15.552 INFO:teuthology.orchestra.run.smithi187.stdout:CentOS Stream 9 - Extras packages 57 kB/s | 19 kB 00:00 2024-09-21T00:23:16.440 INFO:teuthology.orchestra.run.smithi187.stdout:Extra Packages for Enterprise Linux 30 MB/s | 23 MB 00:00 2024-09-21T00:23:19.847 INFO:teuthology.orchestra.run.smithi112.stdout:lab-extras 34 kB/s | 1.7 kB 00:00 2024-09-21T00:23:21.846 INFO:teuthology.orchestra.run.smithi187.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2024-09-21T00:23:22.010 INFO:teuthology.orchestra.run.smithi112.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T00:23:22.011 INFO:teuthology.orchestra.run.smithi112.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T00:23:22.025 INFO:teuthology.orchestra.run.smithi112.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-21T00:23:22.026 INFO:teuthology.orchestra.run.smithi112.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-21T00:23:22.027 INFO:teuthology.orchestra.run.smithi112.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-21T00:23:22.028 INFO:teuthology.orchestra.run.smithi112.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-21T00:23:22.030 INFO:teuthology.orchestra.run.smithi112.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-21T00:23:22.033 INFO:teuthology.orchestra.run.smithi112.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-21T00:23:22.034 INFO:teuthology.orchestra.run.smithi112.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-21T00:23:22.099 INFO:teuthology.orchestra.run.smithi112.stdout:Dependencies resolved. 2024-09-21T00:23:22.124 INFO:teuthology.orchestra.run.smithi112.stdout:======================================================================================= 2024-09-21T00:23:22.124 INFO:teuthology.orchestra.run.smithi112.stdout: Package Arch Version Repository Size 2024-09-21T00:23:22.124 INFO:teuthology.orchestra.run.smithi112.stdout:======================================================================================= 2024-09-21T00:23:22.124 INFO:teuthology.orchestra.run.smithi112.stdout:Installing: 2024-09-21T00:23:22.124 INFO:teuthology.orchestra.run.smithi112.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-21T00:23:22.124 INFO:teuthology.orchestra.run.smithi112.stdout: ceph x86_64 2:19.3.0-5120.g18432218.el9 ceph 6.6 k 2024-09-21T00:23:22.124 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-base x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.5 M 2024-09-21T00:23:22.124 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 873 k 2024-09-21T00:23:22.125 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T00:23:22.125 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr x86_64 2:19.3.0-5120.g18432218.el9 ceph 1.5 M 2024-09-21T00:23:22.125 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 154 k 2024-09-21T00:23:22.125 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-dashboard noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 14 M 2024-09-21T00:23:22.125 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 7.4 M 2024-09-21T00:23:22.125 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-rook noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 49 k 2024-09-21T00:23:22.125 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-radosgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 10 M 2024-09-21T00:23:22.125 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-test x86_64 2:19.3.0-5120.g18432218.el9 ceph 40 M 2024-09-21T00:23:22.125 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-volume noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 285 k 2024-09-21T00:23:22.125 INFO:teuthology.orchestra.run.smithi112.stdout: cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 784 k 2024-09-21T00:23:22.125 INFO:teuthology.orchestra.run.smithi112.stdout: cephfs-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 226 k 2024-09-21T00:23:22.125 INFO:teuthology.orchestra.run.smithi112.stdout: cephfs-shell x86_64 2:19.3.0-5120.g18432218.el9 ceph 27 k 2024-09-21T00:23:22.125 INFO:teuthology.orchestra.run.smithi112.stdout: cephfs-top noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 18 k 2024-09-21T00:23:22.126 INFO:teuthology.orchestra.run.smithi112.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-21T00:23:22.126 INFO:teuthology.orchestra.run.smithi112.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-21T00:23:22.126 INFO:teuthology.orchestra.run.smithi112.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-21T00:23:22.126 INFO:teuthology.orchestra.run.smithi112.stdout: libcephfs-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 32 k 2024-09-21T00:23:22.126 INFO:teuthology.orchestra.run.smithi112.stdout: libcephfs2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 766 k 2024-09-21T00:23:22.126 INFO:teuthology.orchestra.run.smithi112.stdout: librados-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 127 k 2024-09-21T00:23:22.126 INFO:teuthology.orchestra.run.smithi112.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-21T00:23:22.126 INFO:teuthology.orchestra.run.smithi112.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-21T00:23:22.126 INFO:teuthology.orchestra.run.smithi112.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-21T00:23:22.126 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-21T00:23:22.126 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-21T00:23:22.126 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-21T00:23:22.127 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cephfs x86_64 2:19.3.0-5120.g18432218.el9 ceph 161 k 2024-09-21T00:23:22.127 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rados x86_64 2:19.3.0-5120.g18432218.el9 ceph 321 k 2024-09-21T00:23:22.127 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 307 k 2024-09-21T00:23:22.127 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 99 k 2024-09-21T00:23:22.127 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 86 k 2024-09-21T00:23:22.127 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.1 M 2024-09-21T00:23:22.127 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-nbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 170 k 2024-09-21T00:23:22.127 INFO:teuthology.orchestra.run.smithi112.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-21T00:23:22.127 INFO:teuthology.orchestra.run.smithi112.stdout:Upgrading: 2024-09-21T00:23:22.127 INFO:teuthology.orchestra.run.smithi112.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-21T00:23:22.127 INFO:teuthology.orchestra.run.smithi112.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-21T00:23:22.127 INFO:teuthology.orchestra.run.smithi112.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-21T00:23:22.127 INFO:teuthology.orchestra.run.smithi112.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-21T00:23:22.127 INFO:teuthology.orchestra.run.smithi112.stdout: librados2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.4 M 2024-09-21T00:23:22.128 INFO:teuthology.orchestra.run.smithi112.stdout: librbd1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.2 M 2024-09-21T00:23:22.128 INFO:teuthology.orchestra.run.smithi112.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-21T00:23:22.128 INFO:teuthology.orchestra.run.smithi112.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-21T00:23:22.128 INFO:teuthology.orchestra.run.smithi112.stdout:Installing dependencies: 2024-09-21T00:23:22.128 INFO:teuthology.orchestra.run.smithi112.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-21T00:23:22.128 INFO:teuthology.orchestra.run.smithi112.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-21T00:23:22.128 INFO:teuthology.orchestra.run.smithi112.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-21T00:23:22.128 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 20 M 2024-09-21T00:23:22.128 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-grafana-dashboards noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 34 k 2024-09-21T00:23:22.128 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mds x86_64 2:19.3.0-5120.g18432218.el9 ceph 2.3 M 2024-09-21T00:23:22.128 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-modules-core noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 292 k 2024-09-21T00:23:22.129 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mon x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.0 M 2024-09-21T00:23:22.129 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-osd x86_64 2:19.3.0-5120.g18432218.el9 ceph 18 M 2024-09-21T00:23:22.129 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-prometheus-alerts noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 16 k 2024-09-21T00:23:22.129 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-selinux x86_64 2:19.3.0-5120.g18432218.el9 ceph 25 k 2024-09-21T00:23:22.129 INFO:teuthology.orchestra.run.smithi112.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-21T00:23:22.129 INFO:teuthology.orchestra.run.smithi112.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-21T00:23:22.129 INFO:teuthology.orchestra.run.smithi112.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-21T00:23:22.129 INFO:teuthology.orchestra.run.smithi112.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-21T00:23:22.129 INFO:teuthology.orchestra.run.smithi112.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-21T00:23:22.129 INFO:teuthology.orchestra.run.smithi112.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-21T00:23:22.129 INFO:teuthology.orchestra.run.smithi112.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-21T00:23:22.129 INFO:teuthology.orchestra.run.smithi112.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-21T00:23:22.129 INFO:teuthology.orchestra.run.smithi112.stdout: libcephsqlite x86_64 2:19.3.0-5120.g18432218.el9 ceph 158 k 2024-09-21T00:23:22.130 INFO:teuthology.orchestra.run.smithi112.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-21T00:23:22.130 INFO:teuthology.orchestra.run.smithi112.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-21T00:23:22.130 INFO:teuthology.orchestra.run.smithi112.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-21T00:23:22.130 INFO:teuthology.orchestra.run.smithi112.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-21T00:23:22.130 INFO:teuthology.orchestra.run.smithi112.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-21T00:23:22.130 INFO:teuthology.orchestra.run.smithi112.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-21T00:23:22.130 INFO:teuthology.orchestra.run.smithi112.stdout: libradosstriper1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 507 k 2024-09-21T00:23:22.130 INFO:teuthology.orchestra.run.smithi112.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-21T00:23:22.130 INFO:teuthology.orchestra.run.smithi112.stdout: librgw2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.3 M 2024-09-21T00:23:22.130 INFO:teuthology.orchestra.run.smithi112.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-21T00:23:22.131 INFO:teuthology.orchestra.run.smithi112.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-21T00:23:22.131 INFO:teuthology.orchestra.run.smithi112.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-21T00:23:22.131 INFO:teuthology.orchestra.run.smithi112.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-21T00:23:22.131 INFO:teuthology.orchestra.run.smithi112.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-21T00:23:22.131 INFO:teuthology.orchestra.run.smithi112.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-21T00:23:22.131 INFO:teuthology.orchestra.run.smithi112.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-21T00:23:22.131 INFO:teuthology.orchestra.run.smithi112.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-21T00:23:22.131 INFO:teuthology.orchestra.run.smithi112.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-21T00:23:22.131 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-21T00:23:22.131 INFO:teuthology.orchestra.run.smithi112.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-21T00:23:22.131 INFO:teuthology.orchestra.run.smithi112.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-21T00:23:22.131 INFO:teuthology.orchestra.run.smithi112.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-21T00:23:22.131 INFO:teuthology.orchestra.run.smithi112.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-21T00:23:22.132 INFO:teuthology.orchestra.run.smithi112.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-21T00:23:22.132 INFO:teuthology.orchestra.run.smithi112.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-21T00:23:22.132 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-21T00:23:22.132 INFO:teuthology.orchestra.run.smithi112.stdout: python3-ceph-argparse x86_64 2:19.3.0-5120.g18432218.el9 ceph 45 k 2024-09-21T00:23:22.132 INFO:teuthology.orchestra.run.smithi112.stdout: python3-ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T00:23:22.132 INFO:teuthology.orchestra.run.smithi112.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-21T00:23:22.132 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-21T00:23:22.132 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-21T00:23:22.132 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-21T00:23:22.132 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-21T00:23:22.132 INFO:teuthology.orchestra.run.smithi112.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-21T00:23:22.132 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-21T00:23:22.132 INFO:teuthology.orchestra.run.smithi112.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-21T00:23:22.133 INFO:teuthology.orchestra.run.smithi112.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-21T00:23:22.133 INFO:teuthology.orchestra.run.smithi112.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-21T00:23:22.133 INFO:teuthology.orchestra.run.smithi112.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-21T00:23:22.133 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-21T00:23:22.133 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-21T00:23:22.133 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-21T00:23:22.133 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-21T00:23:22.133 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-21T00:23:22.133 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-21T00:23:22.133 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-21T00:23:22.133 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-21T00:23:22.133 INFO:teuthology.orchestra.run.smithi112.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-21T00:23:22.133 INFO:teuthology.orchestra.run.smithi112.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-21T00:23:22.133 INFO:teuthology.orchestra.run.smithi112.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-21T00:23:22.134 INFO:teuthology.orchestra.run.smithi112.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-21T00:23:22.134 INFO:teuthology.orchestra.run.smithi112.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-21T00:23:22.134 INFO:teuthology.orchestra.run.smithi112.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-21T00:23:22.134 INFO:teuthology.orchestra.run.smithi112.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-21T00:23:22.134 INFO:teuthology.orchestra.run.smithi112.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-21T00:23:22.134 INFO:teuthology.orchestra.run.smithi112.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-21T00:23:22.134 INFO:teuthology.orchestra.run.smithi112.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-21T00:23:22.134 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-21T00:23:22.134 INFO:teuthology.orchestra.run.smithi112.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-21T00:23:22.135 INFO:teuthology.orchestra.run.smithi112.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-21T00:23:22.135 INFO:teuthology.orchestra.run.smithi112.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-21T00:23:22.135 INFO:teuthology.orchestra.run.smithi112.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-21T00:23:22.135 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-21T00:23:22.135 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-21T00:23:22.135 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-21T00:23:22.135 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-21T00:23:22.135 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-21T00:23:22.135 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-21T00:23:22.135 INFO:teuthology.orchestra.run.smithi112.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-21T00:23:22.135 INFO:teuthology.orchestra.run.smithi112.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-21T00:23:22.135 INFO:teuthology.orchestra.run.smithi112.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-21T00:23:22.135 INFO:teuthology.orchestra.run.smithi112.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-21T00:23:22.136 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-21T00:23:22.136 INFO:teuthology.orchestra.run.smithi112.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-21T00:23:22.136 INFO:teuthology.orchestra.run.smithi112.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-21T00:23:22.136 INFO:teuthology.orchestra.run.smithi112.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-21T00:23:22.136 INFO:teuthology.orchestra.run.smithi112.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-21T00:23:22.136 INFO:teuthology.orchestra.run.smithi112.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-21T00:23:22.136 INFO:teuthology.orchestra.run.smithi112.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-21T00:23:22.136 INFO:teuthology.orchestra.run.smithi112.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-21T00:23:22.136 INFO:teuthology.orchestra.run.smithi112.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-21T00:23:22.136 INFO:teuthology.orchestra.run.smithi112.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-21T00:23:22.136 INFO:teuthology.orchestra.run.smithi112.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-21T00:23:22.136 INFO:teuthology.orchestra.run.smithi112.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-21T00:23:22.136 INFO:teuthology.orchestra.run.smithi112.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-21T00:23:22.136 INFO:teuthology.orchestra.run.smithi112.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-21T00:23:22.137 INFO:teuthology.orchestra.run.smithi112.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-21T00:23:22.137 INFO:teuthology.orchestra.run.smithi112.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-21T00:23:22.137 INFO:teuthology.orchestra.run.smithi112.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-21T00:23:22.137 INFO:teuthology.orchestra.run.smithi112.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-21T00:23:22.137 INFO:teuthology.orchestra.run.smithi112.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-21T00:23:22.137 INFO:teuthology.orchestra.run.smithi112.stdout:Installing weak dependencies: 2024-09-21T00:23:22.137 INFO:teuthology.orchestra.run.smithi112.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-21T00:23:22.137 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:23:22.137 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction Summary 2024-09-21T00:23:22.137 INFO:teuthology.orchestra.run.smithi112.stdout:======================================================================================= 2024-09-21T00:23:22.137 INFO:teuthology.orchestra.run.smithi112.stdout:Install 149 Packages 2024-09-21T00:23:22.137 INFO:teuthology.orchestra.run.smithi112.stdout:Upgrade 8 Packages 2024-09-21T00:23:22.137 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:23:22.138 INFO:teuthology.orchestra.run.smithi112.stdout:Total download size: 228 M 2024-09-21T00:23:22.138 INFO:teuthology.orchestra.run.smithi112.stdout:Downloading Packages: 2024-09-21T00:23:22.915 INFO:teuthology.orchestra.run.smithi112.stdout:(1/157): ceph-19.3.0-5120.g18432218.el9.x86_64. 49 kB/s | 6.6 kB 00:00 2024-09-21T00:23:23.149 INFO:teuthology.orchestra.run.smithi112.stdout:(2/157): ceph-fuse-19.3.0-5120.g18432218.el9.x8 3.7 MB/s | 873 kB 00:00 2024-09-21T00:23:23.224 INFO:teuthology.orchestra.run.smithi112.stdout:(3/157): ceph-base-19.3.0-5120.g18432218.el9.x8 12 MB/s | 5.5 MB 00:00 2024-09-21T00:23:23.250 INFO:teuthology.orchestra.run.smithi112.stdout:(4/157): ceph-immutable-object-cache-19.3.0-512 1.4 MB/s | 148 kB 00:00 2024-09-21T00:23:23.341 INFO:teuthology.orchestra.run.smithi112.stdout:(5/157): ceph-mds-19.3.0-5120.g18432218.el9.x86 20 MB/s | 2.3 MB 00:00 2024-09-21T00:23:23.417 INFO:teuthology.orchestra.run.smithi112.stdout:(6/157): ceph-mgr-19.3.0-5120.g18432218.el9.x86 9.1 MB/s | 1.5 MB 00:00 2024-09-21T00:23:23.900 INFO:teuthology.orchestra.run.smithi112.stdout:(7/157): ceph-common-19.3.0-5120.g18432218.el9. 18 MB/s | 20 MB 00:01 2024-09-21T00:23:24.039 INFO:teuthology.orchestra.run.smithi187.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T00:23:24.040 INFO:teuthology.orchestra.run.smithi187.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T00:23:24.052 INFO:teuthology.orchestra.run.smithi187.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-21T00:23:24.053 INFO:teuthology.orchestra.run.smithi187.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-21T00:23:24.054 INFO:teuthology.orchestra.run.smithi187.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-21T00:23:24.054 INFO:teuthology.orchestra.run.smithi187.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-21T00:23:24.056 INFO:teuthology.orchestra.run.smithi187.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-21T00:23:24.059 INFO:teuthology.orchestra.run.smithi187.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-21T00:23:24.059 INFO:teuthology.orchestra.run.smithi187.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-21T00:23:24.123 INFO:teuthology.orchestra.run.smithi187.stdout:Dependencies resolved. 2024-09-21T00:23:24.145 INFO:teuthology.orchestra.run.smithi187.stdout:======================================================================================= 2024-09-21T00:23:24.145 INFO:teuthology.orchestra.run.smithi187.stdout: Package Arch Version Repository Size 2024-09-21T00:23:24.145 INFO:teuthology.orchestra.run.smithi187.stdout:======================================================================================= 2024-09-21T00:23:24.145 INFO:teuthology.orchestra.run.smithi187.stdout:Installing: 2024-09-21T00:23:24.145 INFO:teuthology.orchestra.run.smithi187.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-21T00:23:24.145 INFO:teuthology.orchestra.run.smithi187.stdout: ceph x86_64 2:19.3.0-5120.g18432218.el9 ceph 6.6 k 2024-09-21T00:23:24.145 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-base x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.5 M 2024-09-21T00:23:24.145 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 873 k 2024-09-21T00:23:24.145 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T00:23:24.145 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr x86_64 2:19.3.0-5120.g18432218.el9 ceph 1.5 M 2024-09-21T00:23:24.146 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 154 k 2024-09-21T00:23:24.146 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-dashboard noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 14 M 2024-09-21T00:23:24.146 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 7.4 M 2024-09-21T00:23:24.146 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-rook noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 49 k 2024-09-21T00:23:24.146 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-radosgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 10 M 2024-09-21T00:23:24.146 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-test x86_64 2:19.3.0-5120.g18432218.el9 ceph 40 M 2024-09-21T00:23:24.146 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-volume noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 285 k 2024-09-21T00:23:24.146 INFO:teuthology.orchestra.run.smithi187.stdout: cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 784 k 2024-09-21T00:23:24.146 INFO:teuthology.orchestra.run.smithi187.stdout: cephfs-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 226 k 2024-09-21T00:23:24.146 INFO:teuthology.orchestra.run.smithi187.stdout: cephfs-shell x86_64 2:19.3.0-5120.g18432218.el9 ceph 27 k 2024-09-21T00:23:24.146 INFO:teuthology.orchestra.run.smithi187.stdout: cephfs-top noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 18 k 2024-09-21T00:23:24.146 INFO:teuthology.orchestra.run.smithi187.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-21T00:23:24.146 INFO:teuthology.orchestra.run.smithi187.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-21T00:23:24.147 INFO:teuthology.orchestra.run.smithi187.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-21T00:23:24.147 INFO:teuthology.orchestra.run.smithi187.stdout: libcephfs-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 32 k 2024-09-21T00:23:24.147 INFO:teuthology.orchestra.run.smithi187.stdout: libcephfs2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 766 k 2024-09-21T00:23:24.147 INFO:teuthology.orchestra.run.smithi187.stdout: librados-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 127 k 2024-09-21T00:23:24.147 INFO:teuthology.orchestra.run.smithi187.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-21T00:23:24.147 INFO:teuthology.orchestra.run.smithi187.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-21T00:23:24.147 INFO:teuthology.orchestra.run.smithi187.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-21T00:23:24.147 INFO:teuthology.orchestra.run.smithi187.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-21T00:23:24.147 INFO:teuthology.orchestra.run.smithi187.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-21T00:23:24.147 INFO:teuthology.orchestra.run.smithi187.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-21T00:23:24.147 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cephfs x86_64 2:19.3.0-5120.g18432218.el9 ceph 161 k 2024-09-21T00:23:24.147 INFO:teuthology.orchestra.run.smithi187.stdout: python3-rados x86_64 2:19.3.0-5120.g18432218.el9 ceph 321 k 2024-09-21T00:23:24.147 INFO:teuthology.orchestra.run.smithi187.stdout: python3-rbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 307 k 2024-09-21T00:23:24.148 INFO:teuthology.orchestra.run.smithi187.stdout: python3-rgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 99 k 2024-09-21T00:23:24.148 INFO:teuthology.orchestra.run.smithi187.stdout: rbd-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 86 k 2024-09-21T00:23:24.148 INFO:teuthology.orchestra.run.smithi187.stdout: rbd-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.1 M 2024-09-21T00:23:24.148 INFO:teuthology.orchestra.run.smithi187.stdout: rbd-nbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 170 k 2024-09-21T00:23:24.148 INFO:teuthology.orchestra.run.smithi187.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-21T00:23:24.148 INFO:teuthology.orchestra.run.smithi187.stdout:Upgrading: 2024-09-21T00:23:24.148 INFO:teuthology.orchestra.run.smithi187.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-21T00:23:24.148 INFO:teuthology.orchestra.run.smithi187.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-21T00:23:24.148 INFO:teuthology.orchestra.run.smithi187.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-21T00:23:24.148 INFO:teuthology.orchestra.run.smithi187.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-21T00:23:24.148 INFO:teuthology.orchestra.run.smithi187.stdout: librados2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.4 M 2024-09-21T00:23:24.148 INFO:teuthology.orchestra.run.smithi187.stdout: librbd1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.2 M 2024-09-21T00:23:24.148 INFO:teuthology.orchestra.run.smithi187.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-21T00:23:24.148 INFO:teuthology.orchestra.run.smithi187.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-21T00:23:24.149 INFO:teuthology.orchestra.run.smithi187.stdout:Installing dependencies: 2024-09-21T00:23:24.149 INFO:teuthology.orchestra.run.smithi187.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-21T00:23:24.149 INFO:teuthology.orchestra.run.smithi187.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-21T00:23:24.149 INFO:teuthology.orchestra.run.smithi187.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-21T00:23:24.149 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 20 M 2024-09-21T00:23:24.149 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-grafana-dashboards noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 34 k 2024-09-21T00:23:24.149 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mds x86_64 2:19.3.0-5120.g18432218.el9 ceph 2.3 M 2024-09-21T00:23:24.149 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-modules-core noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 292 k 2024-09-21T00:23:24.149 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mon x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.0 M 2024-09-21T00:23:24.149 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-osd x86_64 2:19.3.0-5120.g18432218.el9 ceph 18 M 2024-09-21T00:23:24.149 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-prometheus-alerts noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 16 k 2024-09-21T00:23:24.149 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-selinux x86_64 2:19.3.0-5120.g18432218.el9 ceph 25 k 2024-09-21T00:23:24.149 INFO:teuthology.orchestra.run.smithi187.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-21T00:23:24.149 INFO:teuthology.orchestra.run.smithi187.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-21T00:23:24.149 INFO:teuthology.orchestra.run.smithi187.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-21T00:23:24.150 INFO:teuthology.orchestra.run.smithi187.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-21T00:23:24.150 INFO:teuthology.orchestra.run.smithi187.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-21T00:23:24.150 INFO:teuthology.orchestra.run.smithi187.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-21T00:23:24.150 INFO:teuthology.orchestra.run.smithi187.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-21T00:23:24.150 INFO:teuthology.orchestra.run.smithi187.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-21T00:23:24.150 INFO:teuthology.orchestra.run.smithi187.stdout: libcephsqlite x86_64 2:19.3.0-5120.g18432218.el9 ceph 158 k 2024-09-21T00:23:24.150 INFO:teuthology.orchestra.run.smithi187.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-21T00:23:24.150 INFO:teuthology.orchestra.run.smithi187.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-21T00:23:24.150 INFO:teuthology.orchestra.run.smithi187.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-21T00:23:24.150 INFO:teuthology.orchestra.run.smithi187.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-21T00:23:24.150 INFO:teuthology.orchestra.run.smithi187.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-21T00:23:24.150 INFO:teuthology.orchestra.run.smithi187.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-21T00:23:24.150 INFO:teuthology.orchestra.run.smithi187.stdout: libradosstriper1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 507 k 2024-09-21T00:23:24.151 INFO:teuthology.orchestra.run.smithi187.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-21T00:23:24.151 INFO:teuthology.orchestra.run.smithi187.stdout: librgw2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.3 M 2024-09-21T00:23:24.151 INFO:teuthology.orchestra.run.smithi187.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-21T00:23:24.151 INFO:teuthology.orchestra.run.smithi187.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-21T00:23:24.151 INFO:teuthology.orchestra.run.smithi187.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-21T00:23:24.151 INFO:teuthology.orchestra.run.smithi187.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-21T00:23:24.151 INFO:teuthology.orchestra.run.smithi187.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-21T00:23:24.151 INFO:teuthology.orchestra.run.smithi187.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-21T00:23:24.151 INFO:teuthology.orchestra.run.smithi187.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-21T00:23:24.151 INFO:teuthology.orchestra.run.smithi187.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-21T00:23:24.151 INFO:teuthology.orchestra.run.smithi187.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-21T00:23:24.151 INFO:teuthology.orchestra.run.smithi187.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-21T00:23:24.151 INFO:teuthology.orchestra.run.smithi187.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-21T00:23:24.151 INFO:teuthology.orchestra.run.smithi187.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-21T00:23:24.152 INFO:teuthology.orchestra.run.smithi187.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-21T00:23:24.152 INFO:teuthology.orchestra.run.smithi187.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-21T00:23:24.152 INFO:teuthology.orchestra.run.smithi187.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-21T00:23:24.152 INFO:teuthology.orchestra.run.smithi187.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-21T00:23:24.152 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-21T00:23:24.152 INFO:teuthology.orchestra.run.smithi187.stdout: python3-ceph-argparse x86_64 2:19.3.0-5120.g18432218.el9 ceph 45 k 2024-09-21T00:23:24.152 INFO:teuthology.orchestra.run.smithi187.stdout: python3-ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T00:23:24.152 INFO:teuthology.orchestra.run.smithi187.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-21T00:23:24.152 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-21T00:23:24.152 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-21T00:23:24.152 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-21T00:23:24.152 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-21T00:23:24.152 INFO:teuthology.orchestra.run.smithi187.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-21T00:23:24.152 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-21T00:23:24.153 INFO:teuthology.orchestra.run.smithi187.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-21T00:23:24.153 INFO:teuthology.orchestra.run.smithi187.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-21T00:23:24.153 INFO:teuthology.orchestra.run.smithi187.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-21T00:23:24.153 INFO:teuthology.orchestra.run.smithi187.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-21T00:23:24.153 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-21T00:23:24.153 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-21T00:23:24.153 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-21T00:23:24.153 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-21T00:23:24.153 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-21T00:23:24.153 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-21T00:23:24.153 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-21T00:23:24.153 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-21T00:23:24.153 INFO:teuthology.orchestra.run.smithi187.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-21T00:23:24.153 INFO:teuthology.orchestra.run.smithi187.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-21T00:23:24.154 INFO:teuthology.orchestra.run.smithi187.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-21T00:23:24.154 INFO:teuthology.orchestra.run.smithi187.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-21T00:23:24.154 INFO:teuthology.orchestra.run.smithi187.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-21T00:23:24.154 INFO:teuthology.orchestra.run.smithi187.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-21T00:23:24.154 INFO:teuthology.orchestra.run.smithi187.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-21T00:23:24.154 INFO:teuthology.orchestra.run.smithi187.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-21T00:23:24.154 INFO:teuthology.orchestra.run.smithi187.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-21T00:23:24.154 INFO:teuthology.orchestra.run.smithi187.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-21T00:23:24.154 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-21T00:23:24.154 INFO:teuthology.orchestra.run.smithi187.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-21T00:23:24.154 INFO:teuthology.orchestra.run.smithi187.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-21T00:23:24.154 INFO:teuthology.orchestra.run.smithi187.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-21T00:23:24.154 INFO:teuthology.orchestra.run.smithi187.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-21T00:23:24.154 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-21T00:23:24.155 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-21T00:23:24.155 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-21T00:23:24.155 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-21T00:23:24.155 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-21T00:23:24.155 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-21T00:23:24.155 INFO:teuthology.orchestra.run.smithi187.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-21T00:23:24.155 INFO:teuthology.orchestra.run.smithi187.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-21T00:23:24.155 INFO:teuthology.orchestra.run.smithi187.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-21T00:23:24.155 INFO:teuthology.orchestra.run.smithi187.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-21T00:23:24.155 INFO:teuthology.orchestra.run.smithi187.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-21T00:23:24.155 INFO:teuthology.orchestra.run.smithi187.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-21T00:23:24.155 INFO:teuthology.orchestra.run.smithi187.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-21T00:23:24.155 INFO:teuthology.orchestra.run.smithi187.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-21T00:23:24.156 INFO:teuthology.orchestra.run.smithi187.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-21T00:23:24.156 INFO:teuthology.orchestra.run.smithi187.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-21T00:23:24.156 INFO:teuthology.orchestra.run.smithi187.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-21T00:23:24.156 INFO:teuthology.orchestra.run.smithi187.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-21T00:23:24.156 INFO:teuthology.orchestra.run.smithi187.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-21T00:23:24.156 INFO:teuthology.orchestra.run.smithi187.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-21T00:23:24.156 INFO:teuthology.orchestra.run.smithi187.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-21T00:23:24.156 INFO:teuthology.orchestra.run.smithi187.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-21T00:23:24.156 INFO:teuthology.orchestra.run.smithi187.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-21T00:23:24.156 INFO:teuthology.orchestra.run.smithi187.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-21T00:23:24.156 INFO:teuthology.orchestra.run.smithi187.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-21T00:23:24.156 INFO:teuthology.orchestra.run.smithi187.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-21T00:23:24.156 INFO:teuthology.orchestra.run.smithi187.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-21T00:23:24.156 INFO:teuthology.orchestra.run.smithi187.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-21T00:23:24.157 INFO:teuthology.orchestra.run.smithi187.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-21T00:23:24.157 INFO:teuthology.orchestra.run.smithi187.stdout:Installing weak dependencies: 2024-09-21T00:23:24.157 INFO:teuthology.orchestra.run.smithi187.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-21T00:23:24.157 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:23:24.157 INFO:teuthology.orchestra.run.smithi187.stdout:Transaction Summary 2024-09-21T00:23:24.157 INFO:teuthology.orchestra.run.smithi187.stdout:======================================================================================= 2024-09-21T00:23:24.157 INFO:teuthology.orchestra.run.smithi187.stdout:Install 149 Packages 2024-09-21T00:23:24.157 INFO:teuthology.orchestra.run.smithi187.stdout:Upgrade 8 Packages 2024-09-21T00:23:24.157 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:23:24.158 INFO:teuthology.orchestra.run.smithi187.stdout:Total download size: 228 M 2024-09-21T00:23:24.158 INFO:teuthology.orchestra.run.smithi187.stdout:Downloading Packages: 2024-09-21T00:23:24.302 INFO:teuthology.orchestra.run.smithi112.stdout:(8/157): ceph-radosgw-19.3.0-5120.g18432218.el9 26 MB/s | 10 MB 00:00 2024-09-21T00:23:24.384 INFO:teuthology.orchestra.run.smithi112.stdout:(9/157): ceph-mon-19.3.0-5120.g18432218.el9.x86 4.8 MB/s | 5.0 MB 00:01 2024-09-21T00:23:24.401 INFO:teuthology.orchestra.run.smithi112.stdout:(10/157): ceph-selinux-19.3.0-5120.g18432218.el 251 kB/s | 25 kB 00:00 2024-09-21T00:23:24.585 INFO:teuthology.orchestra.run.smithi112.stdout:(11/157): ceph-osd-19.3.0-5120.g18432218.el9.x8 15 MB/s | 18 MB 00:01 2024-09-21T00:23:24.610 INFO:teuthology.orchestra.run.smithi112.stdout:(12/157): cephfs-mirror-19.3.0-5120.g18432218.e 1.1 MB/s | 226 kB 00:00 2024-09-21T00:23:24.627 INFO:teuthology.orchestra.run.smithi112.stdout:(13/157): cephfs-shell-19.3.0-5120.g18432218.el 648 kB/s | 27 kB 00:00 2024-09-21T00:23:24.644 INFO:teuthology.orchestra.run.smithi112.stdout:(14/157): libcephfs-devel-19.3.0-5120.g18432218 952 kB/s | 32 kB 00:00 2024-09-21T00:23:24.686 INFO:teuthology.orchestra.run.smithi112.stdout:(15/157): libcephfs2-19.3.0-5120.g18432218.el9. 13 MB/s | 766 kB 00:00 2024-09-21T00:23:24.696 INFO:teuthology.orchestra.run.smithi187.stdout:(1/157): ceph-19.3.0-5120.g18432218.el9.x86_64. 47 kB/s | 6.6 kB 00:00 2024-09-21T00:23:24.728 INFO:teuthology.orchestra.run.smithi112.stdout:(16/157): libcephsqlite-19.3.0-5120.g18432218.e 1.8 MB/s | 158 kB 00:00 2024-09-21T00:23:24.753 INFO:teuthology.orchestra.run.smithi112.stdout:(17/157): librados-devel-19.3.0-5120.g18432218. 1.9 MB/s | 127 kB 00:00 2024-09-21T00:23:24.778 INFO:teuthology.orchestra.run.smithi112.stdout:(18/157): libradosstriper1-19.3.0-5120.g1843221 9.9 MB/s | 507 kB 00:00 2024-09-21T00:23:24.837 INFO:teuthology.orchestra.run.smithi112.stdout:(19/157): python3-ceph-argparse-19.3.0-5120.g18 775 kB/s | 45 kB 00:00 2024-09-21T00:23:24.920 INFO:teuthology.orchestra.run.smithi112.stdout:(20/157): python3-ceph-common-19.3.0-5120.g1843 1.7 MB/s | 148 kB 00:00 2024-09-21T00:23:25.071 INFO:teuthology.orchestra.run.smithi112.stdout:(21/157): librgw2-19.3.0-5120.g18432218.el9.x86 17 MB/s | 5.3 MB 00:00 2024-09-21T00:23:25.096 INFO:teuthology.orchestra.run.smithi112.stdout:(22/157): python3-cephfs-19.3.0-5120.g18432218. 922 kB/s | 161 kB 00:00 2024-09-21T00:23:25.105 INFO:teuthology.orchestra.run.smithi187.stdout:(2/157): ceph-fuse-19.3.0-5120.g18432218.el9.x8 2.1 MB/s | 873 kB 00:00 2024-09-21T00:23:25.121 INFO:teuthology.orchestra.run.smithi112.stdout:(23/157): python3-rados-19.3.0-5120.g18432218.e 6.3 MB/s | 321 kB 00:00 2024-09-21T00:23:25.146 INFO:teuthology.orchestra.run.smithi112.stdout:(24/157): python3-rbd-19.3.0-5120.g18432218.el9 6.0 MB/s | 307 kB 00:00 2024-09-21T00:23:25.171 INFO:teuthology.orchestra.run.smithi112.stdout:(25/157): python3-rgw-19.3.0-5120.g18432218.el9 1.9 MB/s | 99 kB 00:00 2024-09-21T00:23:25.172 INFO:teuthology.orchestra.run.smithi187.stdout:(3/157): ceph-immutable-object-cache-19.3.0-512 2.2 MB/s | 148 kB 00:00 2024-09-21T00:23:25.197 INFO:teuthology.orchestra.run.smithi112.stdout:(26/157): rbd-fuse-19.3.0-5120.g18432218.el9.x8 1.7 MB/s | 86 kB 00:00 2024-09-21T00:23:25.247 INFO:teuthology.orchestra.run.smithi112.stdout:(27/157): rbd-nbd-19.3.0-5120.g18432218.el9.x86 3.3 MB/s | 170 kB 00:00 2024-09-21T00:23:25.339 INFO:teuthology.orchestra.run.smithi112.stdout:(28/157): rbd-mirror-19.3.0-5120.g18432218.el9. 19 MB/s | 3.1 MB 00:00 2024-09-21T00:23:25.364 INFO:teuthology.orchestra.run.smithi112.stdout:(29/157): ceph-grafana-dashboards-19.3.0-5120.g 293 kB/s | 34 kB 00:00 2024-09-21T00:23:25.398 INFO:teuthology.orchestra.run.smithi112.stdout:(30/157): ceph-mgr-cephadm-19.3.0-5120.g1843221 2.6 MB/s | 154 kB 00:00 2024-09-21T00:23:25.997 INFO:teuthology.orchestra.run.smithi187.stdout:(4/157): ceph-mds-19.3.0-5120.g18432218.el9.x86 2.8 MB/s | 2.3 MB 00:00 2024-09-21T00:23:26.081 INFO:teuthology.orchestra.run.smithi112.stdout:(31/157): ceph-mgr-dashboard-19.3.0-5120.g18432 20 MB/s | 14 MB 00:00 2024-09-21T00:23:26.174 INFO:teuthology.orchestra.run.smithi112.stdout:(32/157): ceph-mgr-diskprediction-local-19.3.0- 9.5 MB/s | 7.4 MB 00:00 2024-09-21T00:23:26.199 INFO:teuthology.orchestra.run.smithi112.stdout:(33/157): ceph-mgr-modules-core-19.3.0-5120.g18 2.4 MB/s | 292 kB 00:00 2024-09-21T00:23:26.207 INFO:teuthology.orchestra.run.smithi187.stdout:(5/157): ceph-mgr-19.3.0-5120.g18432218.el9.x86 7.2 MB/s | 1.5 MB 00:00 2024-09-21T00:23:26.224 INFO:teuthology.orchestra.run.smithi112.stdout:(34/157): ceph-mgr-rook-19.3.0-5120.g18432218.e 987 kB/s | 49 kB 00:00 2024-09-21T00:23:26.241 INFO:teuthology.orchestra.run.smithi112.stdout:(35/157): ceph-prometheus-alerts-19.3.0-5120.g1 389 kB/s | 16 kB 00:00 2024-09-21T00:23:26.275 INFO:teuthology.orchestra.run.smithi112.stdout:(36/157): ceph-volume-19.3.0-5120.g18432218.el9 5.5 MB/s | 285 kB 00:00 2024-09-21T00:23:26.306 INFO:teuthology.orchestra.run.smithi187.stdout:(6/157): ceph-base-19.3.0-5120.g18432218.el9.x8 3.2 MB/s | 5.5 MB 00:01 2024-09-21T00:23:26.308 INFO:teuthology.orchestra.run.smithi112.stdout:(37/157): cephadm-19.3.0-5120.g18432218.el9.noa 11 MB/s | 784 kB 00:00 2024-09-21T00:23:26.325 INFO:teuthology.orchestra.run.smithi112.stdout:(38/157): cephfs-top-19.3.0-5120.g18432218.el9. 366 kB/s | 18 kB 00:00 2024-09-21T00:23:26.573 INFO:teuthology.orchestra.run.smithi187.stdout:(7/157): ceph-mon-19.3.0-5120.g18432218.el9.x86 14 MB/s | 5.0 MB 00:00 2024-09-21T00:23:26.676 INFO:teuthology.orchestra.run.smithi112.stdout:(39/157): ceph-test-19.3.0-5120.g18432218.el9.x 17 MB/s | 40 MB 00:02 2024-09-21T00:23:26.767 INFO:teuthology.orchestra.run.smithi112.stdout:(40/157): fuse-2.9.9-16.el9.x86_64.rpm 174 kB/s | 80 kB 00:00 2024-09-21T00:23:26.826 INFO:teuthology.orchestra.run.smithi112.stdout:(41/157): libquadmath-11.5.0-2.el9.x86_64.rpm 1.2 MB/s | 191 kB 00:00 2024-09-21T00:23:26.851 INFO:teuthology.orchestra.run.smithi112.stdout:(42/157): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.0 MB/s | 253 kB 00:00 2024-09-21T00:23:26.877 INFO:teuthology.orchestra.run.smithi112.stdout:(43/157): libgfortran-11.5.0-2.el9.x86_64.rpm 1.4 MB/s | 801 kB 00:00 2024-09-21T00:23:26.902 INFO:teuthology.orchestra.run.smithi112.stdout:(44/157): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-09-21T00:23:26.927 INFO:teuthology.orchestra.run.smithi112.stdout:(45/157): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-09-21T00:23:26.952 INFO:teuthology.orchestra.run.smithi112.stdout:(46/157): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-09-21T00:23:26.978 INFO:teuthology.orchestra.run.smithi112.stdout:(47/157): python3-urllib3-1.26.5-6.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-09-21T00:23:26.998 INFO:teuthology.orchestra.run.smithi187.stdout:(8/157): ceph-radosgw-19.3.0-5120.g18432218.el9 25 MB/s | 10 MB 00:00 2024-09-21T00:23:27.011 INFO:teuthology.orchestra.run.smithi112.stdout:(48/157): python3-cryptography-36.0.1-4.el9.x86 6.6 MB/s | 1.2 MB 00:00 2024-09-21T00:23:27.049 INFO:teuthology.orchestra.run.smithi187.stdout:(9/157): ceph-selinux-19.3.0-5120.g18432218.el9 504 kB/s | 25 kB 00:00 2024-09-21T00:23:27.211 INFO:teuthology.orchestra.run.smithi112.stdout:(49/157): boost-program-options-1.75.0-8.el9.x8 535 kB/s | 107 kB 00:00 2024-09-21T00:23:27.270 INFO:teuthology.orchestra.run.smithi112.stdout:(50/157): automake-1.16.2-8.el9.noarch.rpm 2.1 MB/s | 689 kB 00:00 2024-09-21T00:23:27.295 INFO:teuthology.orchestra.run.smithi112.stdout:(51/157): elfutils-libelf-devel-0.191-4.el9.x86 259 kB/s | 22 kB 00:00 2024-09-21T00:23:27.320 INFO:teuthology.orchestra.run.smithi112.stdout:(52/157): bison-3.7.4-5.el9.x86_64.rpm 2.7 MB/s | 943 kB 00:00 2024-09-21T00:23:27.354 INFO:teuthology.orchestra.run.smithi112.stdout:(53/157): flex-2.6.4-9.el9.x86_64.rpm 3.7 MB/s | 313 kB 00:00 2024-09-21T00:23:27.379 INFO:teuthology.orchestra.run.smithi112.stdout:(54/157): flexiblas-3.0.4-8.el9.x86_64.rpm 390 kB/s | 33 kB 00:00 2024-09-21T00:23:27.413 INFO:teuthology.orchestra.run.smithi112.stdout:(55/157): flexiblas-openblas-openmp-3.0.4-8.el9 311 kB/s | 18 kB 00:00 2024-09-21T00:23:27.446 INFO:teuthology.orchestra.run.smithi112.stdout:(56/157): libaio-devel-0.3.111-13.el9.x86_64.rp 167 kB/s | 11 kB 00:00 2024-09-21T00:23:27.522 INFO:teuthology.orchestra.run.smithi112.stdout:(57/157): flexiblas-netlib-3.0.4-8.el9.x86_64.r 15 MB/s | 3.0 MB 00:00 2024-09-21T00:23:27.547 INFO:teuthology.orchestra.run.smithi112.stdout:(58/157): libnbd-1.20.2-2.el9.x86_64.rpm 1.2 MB/s | 171 kB 00:00 2024-09-21T00:23:27.572 INFO:teuthology.orchestra.run.smithi112.stdout:(59/157): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.2 MB/s | 160 kB 00:00 2024-09-21T00:23:27.597 INFO:teuthology.orchestra.run.smithi112.stdout:(60/157): librabbitmq-0.11.0-7.el9.x86_64.rpm 600 kB/s | 45 kB 00:00 2024-09-21T00:23:27.666 INFO:teuthology.orchestra.run.smithi112.stdout:(61/157): librdkafka-1.6.1-102.el9.x86_64.rpm 5.4 MB/s | 662 kB 00:00 2024-09-21T00:23:27.690 INFO:teuthology.orchestra.run.smithi112.stdout:(62/157): libuuid-devel-2.37.4-20.el9.x86_64.rp 266 kB/s | 24 kB 00:00 2024-09-21T00:23:27.715 INFO:teuthology.orchestra.run.smithi112.stdout:(63/157): libtool-2.4.6-46.el9.x86_64.rpm 4.0 MB/s | 581 kB 00:00 2024-09-21T00:23:27.740 INFO:teuthology.orchestra.run.smithi112.stdout:(64/157): libzstd-devel-1.5.1-2.el9.x86_64.rpm 644 kB/s | 47 kB 00:00 2024-09-21T00:23:27.765 INFO:teuthology.orchestra.run.smithi112.stdout:(65/157): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.8 MB/s | 292 kB 00:00 2024-09-21T00:23:27.791 INFO:teuthology.orchestra.run.smithi112.stdout:(66/157): lua-5.4.4-4.el9.x86_64.rpm 2.4 MB/s | 188 kB 00:00 2024-09-21T00:23:27.816 INFO:teuthology.orchestra.run.smithi112.stdout:(67/157): openblas-0.3.26-2.el9.x86_64.rpm 509 kB/s | 38 kB 00:00 2024-09-21T00:23:27.958 INFO:teuthology.orchestra.run.smithi112.stdout:(68/157): openblas-openmp-0.3.26-2.el9.x86_64.r 26 MB/s | 4.9 MB 00:00 2024-09-21T00:23:28.000 INFO:teuthology.orchestra.run.smithi112.stdout:(69/157): postgresql-13.16-1.el9.x86_64.rpm 8.6 MB/s | 1.6 MB 00:00 2024-09-21T00:23:28.075 INFO:teuthology.orchestra.run.smithi112.stdout:(70/157): openssl-devel-3.2.2-6.el9.x86_64.rpm 16 MB/s | 4.4 MB 00:00 2024-09-21T00:23:28.100 INFO:teuthology.orchestra.run.smithi112.stdout:(71/157): postgresql-private-libs-13.16-1.el9.x 1.3 MB/s | 137 kB 00:00 2024-09-21T00:23:28.134 INFO:teuthology.orchestra.run.smithi112.stdout:(72/157): postgresql-contrib-13.16-1.el9.x86_64 5.1 MB/s | 910 kB 00:00 2024-09-21T00:23:28.176 INFO:teuthology.orchestra.run.smithi112.stdout:(73/157): protobuf-3.14.0-13.el9.x86_64.rpm 13 MB/s | 1.0 MB 00:00 2024-09-21T00:23:28.232 INFO:teuthology.orchestra.run.smithi187.stdout:(10/157): ceph-test-19.3.0-5120.g18432218.el9.x 34 MB/s | 40 MB 00:01 2024-09-21T00:23:28.300 INFO:teuthology.orchestra.run.smithi187.stdout:(11/157): cephfs-mirror-19.3.0-5120.g18432218.e 3.3 MB/s | 226 kB 00:00 2024-09-21T00:23:28.349 INFO:teuthology.orchestra.run.smithi187.stdout:(12/157): cephfs-shell-19.3.0-5120.g18432218.el 549 kB/s | 27 kB 00:00 2024-09-21T00:23:28.351 INFO:teuthology.orchestra.run.smithi112.stdout:(74/157): python3-babel-2.9.1-2.el9.noarch.rpm 34 MB/s | 6.0 MB 00:00 2024-09-21T00:23:28.376 INFO:teuthology.orchestra.run.smithi112.stdout:(75/157): python3-attrs-20.3.0-7.el9.noarch.rpm 361 kB/s | 87 kB 00:00 2024-09-21T00:23:28.399 INFO:teuthology.orchestra.run.smithi187.stdout:(13/157): libcephfs-devel-19.3.0-5120.g18432218 636 kB/s | 32 kB 00:00 2024-09-21T00:23:28.452 INFO:teuthology.orchestra.run.smithi112.stdout:(76/157): postgresql-server-13.16-1.el9.x86_64. 15 MB/s | 5.8 MB 00:00 2024-09-21T00:23:28.466 INFO:teuthology.orchestra.run.smithi187.stdout:(14/157): libcephfs2-19.3.0-5120.g18432218.el9. 11 MB/s | 766 kB 00:00 2024-09-21T00:23:28.477 INFO:teuthology.orchestra.run.smithi112.stdout:(77/157): python3-colorama-0.4.6-3.el9.noarch.r 489 kB/s | 61 kB 00:00 2024-09-21T00:23:28.502 INFO:teuthology.orchestra.run.smithi112.stdout:(78/157): python3-devel-3.9.19-8.el9.x86_64.rpm 1.9 MB/s | 245 kB 00:00 2024-09-21T00:23:28.517 INFO:teuthology.orchestra.run.smithi187.stdout:(15/157): libcephsqlite-19.3.0-5120.g18432218.e 3.1 MB/s | 158 kB 00:00 2024-09-21T00:23:28.536 INFO:teuthology.orchestra.run.smithi112.stdout:(79/157): python3-jinja2-2.11.3-6.el9.noarch.rp 2.9 MB/s | 248 kB 00:00 2024-09-21T00:23:28.561 INFO:teuthology.orchestra.run.smithi112.stdout:(80/157): python3-jsonpatch-1.21-16.el9.noarch. 306 kB/s | 26 kB 00:00 2024-09-21T00:23:28.567 INFO:teuthology.orchestra.run.smithi187.stdout:(16/157): librados-devel-19.3.0-5120.g18432218. 2.5 MB/s | 127 kB 00:00 2024-09-21T00:23:28.586 INFO:teuthology.orchestra.run.smithi112.stdout:(81/157): python3-jsonpointer-2.0-4.el9.noarch. 228 kB/s | 19 kB 00:00 2024-09-21T00:23:28.620 INFO:teuthology.orchestra.run.smithi187.stdout:(17/157): libradosstriper1-19.3.0-5120.g1843221 9.5 MB/s | 507 kB 00:00 2024-09-21T00:23:28.620 INFO:teuthology.orchestra.run.smithi112.stdout:(82/157): python3-mako-1.1.4-6.el9.noarch.rpm 2.0 MB/s | 172 kB 00:00 2024-09-21T00:23:28.645 INFO:teuthology.orchestra.run.smithi112.stdout:(83/157): python3-markupsafe-1.1.1-12.el9.x86_6 415 kB/s | 35 kB 00:00 2024-09-21T00:23:28.704 INFO:teuthology.orchestra.run.smithi112.stdout:(84/157): python3-oauthlib-3.1.1-5.el9.noarch.r 3.7 MB/s | 222 kB 00:00 2024-09-21T00:23:28.779 INFO:teuthology.orchestra.run.smithi112.stdout:(85/157): python3-numpy-1.20.1-5.el9.x86_64.rpm 27 MB/s | 5.1 MB 00:00 2024-09-21T00:23:28.796 INFO:teuthology.orchestra.run.smithi112.stdout:(86/157): python3-packaging-20.9-5.el9.noarch.r 841 kB/s | 77 kB 00:00 2024-09-21T00:23:28.812 INFO:teuthology.orchestra.run.smithi187.stdout:(18/157): librgw2-19.3.0-5120.g18432218.el9.x86 28 MB/s | 5.3 MB 00:00 2024-09-21T00:23:28.821 INFO:teuthology.orchestra.run.smithi112.stdout:(87/157): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.2 MB/s | 250 kB 00:00 2024-09-21T00:23:28.862 INFO:teuthology.orchestra.run.smithi187.stdout:(19/157): python3-ceph-argparse-19.3.0-5120.g18 909 kB/s | 45 kB 00:00 2024-09-21T00:23:28.863 INFO:teuthology.orchestra.run.smithi112.stdout:(88/157): python3-prettytable-0.7.2-27.el9.noar 505 kB/s | 42 kB 00:00 2024-09-21T00:23:28.888 INFO:teuthology.orchestra.run.smithi112.stdout:(89/157): python3-protobuf-3.14.0-13.el9.noarch 2.9 MB/s | 269 kB 00:00 2024-09-21T00:23:28.914 INFO:teuthology.orchestra.run.smithi112.stdout:(90/157): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 159 kB 00:00 2024-09-21T00:23:28.920 INFO:teuthology.orchestra.run.smithi187.stdout:(20/157): python3-ceph-common-19.3.0-5120.g1843 2.5 MB/s | 148 kB 00:00 2024-09-21T00:23:28.947 INFO:teuthology.orchestra.run.smithi112.stdout:(91/157): python3-pytz-2021.1-5.el9.noarch.rpm 869 kB/s | 51 kB 00:00 2024-09-21T00:23:28.971 INFO:teuthology.orchestra.run.smithi187.stdout:(21/157): python3-cephfs-19.3.0-5120.g18432218. 3.2 MB/s | 161 kB 00:00 2024-09-21T00:23:28.972 INFO:teuthology.orchestra.run.smithi112.stdout:(92/157): python3-pyasn1-modules-0.4.8-6.el9.no 2.5 MB/s | 279 kB 00:00 2024-09-21T00:23:29.014 INFO:teuthology.orchestra.run.smithi112.stdout:(93/157): python3-requests-oauthlib-1.3.0-12.el 535 kB/s | 54 kB 00:00 2024-09-21T00:23:29.021 INFO:teuthology.orchestra.run.smithi187.stdout:(22/157): python3-rados-19.3.0-5120.g18432218.e 6.3 MB/s | 321 kB 00:00 2024-09-21T00:23:29.040 INFO:teuthology.orchestra.run.smithi112.stdout:(94/157): python3-toml-0.10.2-6.el9.noarch.rpm 622 kB/s | 42 kB 00:00 2024-09-21T00:23:29.090 INFO:teuthology.orchestra.run.smithi112.stdout:(95/157): python3-wcwidth-0.2.5-8.el9.noarch.rp 587 kB/s | 44 kB 00:00 2024-09-21T00:23:29.115 INFO:teuthology.orchestra.run.smithi112.stdout:(96/157): qatlib-24.02.0-1.el9.x86_64.rpm 2.9 MB/s | 221 kB 00:00 2024-09-21T00:23:29.157 INFO:teuthology.orchestra.run.smithi112.stdout:(97/157): qatlib-service-24.02.0-1.el9.x86_64.r 539 kB/s | 36 kB 00:00 2024-09-21T00:23:29.182 INFO:teuthology.orchestra.run.smithi112.stdout:(98/157): qatzip-libs-1.2.0-1.el9.x86_64.rpm 709 kB/s | 47 kB 00:00 2024-09-21T00:23:29.230 INFO:teuthology.orchestra.run.smithi187.stdout:(23/157): ceph-osd-19.3.0-5120.g18432218.el9.x8 6.1 MB/s | 18 MB 00:02 2024-09-21T00:23:29.241 INFO:teuthology.orchestra.run.smithi112.stdout:(99/157): socat-1.7.4.1-6.el9.x86_64.rpm 3.6 MB/s | 304 kB 00:00 2024-09-21T00:23:29.255 INFO:teuthology.orchestra.run.smithi187.stdout:(24/157): python3-rbd-19.3.0-5120.g18432218.el9 1.3 MB/s | 307 kB 00:00 2024-09-21T00:23:29.266 INFO:teuthology.orchestra.run.smithi112.stdout:(100/157): uuid-1.6.2-55.el9.x86_64.rpm 690 kB/s | 58 kB 00:00 2024-09-21T00:23:29.280 INFO:teuthology.orchestra.run.smithi187.stdout:(25/157): python3-rgw-19.3.0-5120.g18432218.el9 1.9 MB/s | 99 kB 00:00 2024-09-21T00:23:29.305 INFO:teuthology.orchestra.run.smithi187.stdout:(26/157): rbd-fuse-19.3.0-5120.g18432218.el9.x8 1.7 MB/s | 86 kB 00:00 2024-09-21T00:23:29.356 INFO:teuthology.orchestra.run.smithi187.stdout:(27/157): rbd-nbd-19.3.0-5120.g18432218.el9.x86 3.3 MB/s | 170 kB 00:00 2024-09-21T00:23:29.398 INFO:teuthology.orchestra.run.smithi187.stdout:(28/157): ceph-grafana-dashboards-19.3.0-5120.g 825 kB/s | 34 kB 00:00 2024-09-21T00:23:29.434 INFO:teuthology.orchestra.run.smithi112.stdout:(101/157): python3-scipy-1.6.2-8.el9.x86_64.rpm 33 MB/s | 16 MB 00:00 2024-09-21T00:23:29.448 INFO:teuthology.orchestra.run.smithi187.stdout:(29/157): ceph-mgr-cephadm-19.3.0-5120.g1843221 3.0 MB/s | 154 kB 00:00 2024-09-21T00:23:29.459 INFO:teuthology.orchestra.run.smithi112.stdout:(102/157): xmlstarlet-1.6.1-20.el9.x86_64.rpm 332 kB/s | 64 kB 00:00 2024-09-21T00:23:29.484 INFO:teuthology.orchestra.run.smithi112.stdout:(103/157): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 395 kB/s | 96 kB 00:00 2024-09-21T00:23:29.509 INFO:teuthology.orchestra.run.smithi112.stdout:(104/157): zlib-devel-1.2.11-41.el9.x86_64.rpm 597 kB/s | 45 kB 00:00 2024-09-21T00:23:29.568 INFO:teuthology.orchestra.run.smithi112.stdout:(105/157): abseil-cpp-20211102.0-3.el9.x86_64.r 9.2 MB/s | 548 kB 00:00 2024-09-21T00:23:29.593 INFO:teuthology.orchestra.run.smithi112.stdout:(106/157): gperftools-libs-2.9.1-3.el9.x86_64.r 12 MB/s | 308 kB 00:00 2024-09-21T00:23:29.618 INFO:teuthology.orchestra.run.smithi112.stdout:(107/157): grpc-data-1.46.7-10.el9.noarch.rpm 786 kB/s | 19 kB 00:00 2024-09-21T00:23:29.727 INFO:teuthology.orchestra.run.smithi112.stdout:(108/157): libarrow-9.0.0-11.el9.x86_64.rpm 41 MB/s | 4.4 MB 00:00 2024-09-21T00:23:29.732 INFO:teuthology.orchestra.run.smithi187.stdout:(30/157): rbd-mirror-19.3.0-5120.g18432218.el9. 6.9 MB/s | 3.1 MB 00:00 2024-09-21T00:23:29.768 INFO:teuthology.orchestra.run.smithi112.stdout:(109/157): libarrow-doc-9.0.0-11.el9.noarch.rpm 956 kB/s | 26 kB 00:00 2024-09-21T00:23:29.794 INFO:teuthology.orchestra.run.smithi112.stdout:(110/157): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-21T00:23:29.819 INFO:teuthology.orchestra.run.smithi112.stdout:(111/157): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-09-21T00:23:29.836 INFO:teuthology.orchestra.run.smithi112.stdout:(112/157): luarocks-3.9.2-5.el9.noarch.rpm 9.6 MB/s | 151 kB 00:00 2024-09-21T00:23:29.878 INFO:teuthology.orchestra.run.smithi112.stdout:(113/157): parquet-libs-9.0.0-11.el9.x86_64.rpm 20 MB/s | 839 kB 00:00 2024-09-21T00:23:29.907 INFO:teuthology.orchestra.run.smithi187.stdout:(31/157): ceph-mgr-dashboard-19.3.0-5120.g18432 31 MB/s | 14 MB 00:00 2024-09-21T00:23:29.911 INFO:teuthology.orchestra.run.smithi112.stdout:(114/157): python3-asyncssh-2.13.2-5.el9.noarch 16 MB/s | 548 kB 00:00 2024-09-21T00:23:29.937 INFO:teuthology.orchestra.run.smithi112.stdout:(115/157): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-21T00:23:29.957 INFO:teuthology.orchestra.run.smithi187.stdout:(32/157): ceph-mgr-modules-core-19.3.0-5120.g18 5.7 MB/s | 292 kB 00:00 2024-09-21T00:23:29.962 INFO:teuthology.orchestra.run.smithi112.stdout:(116/157): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-21T00:23:29.979 INFO:teuthology.orchestra.run.smithi112.stdout:(117/157): python3-certifi-2023.05.07-4.el9.noa 854 kB/s | 14 kB 00:00 2024-09-21T00:23:30.004 INFO:teuthology.orchestra.run.smithi112.stdout:(118/157): python3-cheroot-8.6.0-4.el9.noarch.r 6.7 MB/s | 172 kB 00:00 2024-09-21T00:23:30.007 INFO:teuthology.orchestra.run.smithi187.stdout:(33/157): ceph-mgr-rook-19.3.0-5120.g18432218.e 989 kB/s | 49 kB 00:00 2024-09-21T00:23:30.038 INFO:teuthology.orchestra.run.smithi112.stdout:(119/157): python3-cherrypy-18.6.1-2.el9.noarch 11 MB/s | 358 kB 00:00 2024-09-21T00:23:30.058 INFO:teuthology.orchestra.run.smithi187.stdout:(34/157): ceph-prometheus-alerts-19.3.0-5120.g1 330 kB/s | 16 kB 00:00 2024-09-21T00:23:30.063 INFO:teuthology.orchestra.run.smithi112.stdout:(120/157): python3-cmd2-2.4.2-2.el9.noarch.rpm 12 MB/s | 303 kB 00:00 2024-09-21T00:23:30.080 INFO:teuthology.orchestra.run.smithi112.stdout:(121/157): lua-devel-5.4.4-4.el9.x86_64.rpm 36 kB/s | 22 kB 00:00 2024-09-21T00:23:30.113 INFO:teuthology.orchestra.run.smithi112.stdout:(122/157): python3-google-auth-2.34.0-1.el9.noa 4.4 MB/s | 220 kB 00:00 2024-09-21T00:23:30.139 INFO:teuthology.orchestra.run.smithi112.stdout:(123/157): python3-grpcio-tools-1.46.7-10.el9.x 6.1 MB/s | 144 kB 00:00 2024-09-21T00:23:30.164 INFO:teuthology.orchestra.run.smithi112.stdout:(124/157): python3-jaraco-8.2.1-3.el9.noarch.rp 428 kB/s | 11 kB 00:00 2024-09-21T00:23:30.189 INFO:teuthology.orchestra.run.smithi112.stdout:(125/157): python3-jaraco-classes-3.2.1-5.el9.n 712 kB/s | 18 kB 00:00 2024-09-21T00:23:30.215 INFO:teuthology.orchestra.run.smithi112.stdout:(126/157): python3-jaraco-collections-3.0.0-8.e 929 kB/s | 23 kB 00:00 2024-09-21T00:23:30.240 INFO:teuthology.orchestra.run.smithi112.stdout:(127/157): python3-jaraco-functools-3.5.0-2.el9 779 kB/s | 19 kB 00:00 2024-09-21T00:23:30.265 INFO:teuthology.orchestra.run.smithi112.stdout:(128/157): python3-jaraco-text-3.2.0-6.el9.noar 793 kB/s | 20 kB 00:00 2024-09-21T00:23:30.275 INFO:teuthology.orchestra.run.smithi187.stdout:(35/157): ceph-common-19.3.0-5120.g18432218.el9 3.5 MB/s | 20 MB 00:05 2024-09-21T00:23:30.300 INFO:teuthology.orchestra.run.smithi187.stdout:(36/157): ceph-volume-19.3.0-5120.g18432218.el9 1.2 MB/s | 285 kB 00:00 2024-09-21T00:23:30.307 INFO:teuthology.orchestra.run.smithi112.stdout:(129/157): python3-kubernetes-26.1.0-2.el9.noar 25 MB/s | 1.0 MB 00:00 2024-09-21T00:23:30.332 INFO:teuthology.orchestra.run.smithi112.stdout:(130/157): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-21T00:23:30.350 INFO:teuthology.orchestra.run.smithi187.stdout:(37/157): cephfs-top-19.3.0-5120.g18432218.el9. 370 kB/s | 18 kB 00:00 2024-09-21T00:23:30.358 INFO:teuthology.orchestra.run.smithi112.stdout:(131/157): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-21T00:23:30.383 INFO:teuthology.orchestra.run.smithi112.stdout:(132/157): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-21T00:23:30.417 INFO:teuthology.orchestra.run.smithi187.stdout:(38/157): cephadm-19.3.0-5120.g18432218.el9.noa 5.4 MB/s | 784 kB 00:00 2024-09-21T00:23:30.425 INFO:teuthology.orchestra.run.smithi112.stdout:(133/157): python3-grpcio-1.46.7-10.el9.x86_64. 5.9 MB/s | 2.0 MB 00:00 2024-09-21T00:23:30.450 INFO:teuthology.orchestra.run.smithi112.stdout:(134/157): python3-pecan-1.4.2-3.el9.noarch.rpm 4.0 MB/s | 272 kB 00:00 2024-09-21T00:23:30.476 INFO:teuthology.orchestra.run.smithi112.stdout:(135/157): python3-portend-3.1.0-2.el9.noarch.r 328 kB/s | 16 kB 00:00 2024-09-21T00:23:30.501 INFO:teuthology.orchestra.run.smithi112.stdout:(136/157): python3-pyperclip-1.8.2-2.el9.noarch 1.1 MB/s | 28 kB 00:00 2024-09-21T00:23:30.535 INFO:teuthology.orchestra.run.smithi112.stdout:(137/157): protobuf-compiler-3.14.0-13.el9.x86_ 822 kB/s | 863 kB 00:01 2024-09-21T00:23:30.560 INFO:teuthology.orchestra.run.smithi112.stdout:(138/157): python3-pyOpenSSL-21.0.0-1.el9.noarc 825 kB/s | 90 kB 00:00 2024-09-21T00:23:30.602 INFO:teuthology.orchestra.run.smithi112.stdout:(139/157): python3-repoze-lru-0.7-16.el9.noarch 305 kB/s | 31 kB 00:00 2024-09-21T00:23:30.635 INFO:teuthology.orchestra.run.smithi112.stdout:(140/157): python3-tempora-5.0.0-2.el9.noarch.r 1.4 MB/s | 36 kB 00:00 2024-09-21T00:23:30.661 INFO:teuthology.orchestra.run.smithi112.stdout:(141/157): python3-routes-2.5.1-5.el9.noarch.rp 1.8 MB/s | 188 kB 00:00 2024-09-21T00:23:30.686 INFO:teuthology.orchestra.run.smithi112.stdout:(142/157): python3-rsa-4.9-2.el9.noarch.rpm 471 kB/s | 59 kB 00:00 2024-09-21T00:23:30.692 INFO:teuthology.orchestra.run.smithi187.stdout:(39/157): ceph-mgr-diskprediction-local-19.3.0- 7.7 MB/s | 7.4 MB 00:00 2024-09-21T00:23:30.703 INFO:teuthology.orchestra.run.smithi112.stdout:(143/157): python3-typing-extensions-4.12.2-2.e 1.1 MB/s | 74 kB 00:00 2024-09-21T00:23:30.728 INFO:teuthology.orchestra.run.smithi112.stdout:(144/157): python3-websocket-client-1.2.3-2.el9 2.1 MB/s | 90 kB 00:00 2024-09-21T00:23:30.753 INFO:teuthology.orchestra.run.smithi112.stdout:(145/157): python3-webob-1.8.8-2.el9.noarch.rpm 2.4 MB/s | 230 kB 00:00 2024-09-21T00:23:30.778 INFO:teuthology.orchestra.run.smithi112.stdout:(146/157): python3-zc-lockfile-2.0-10.el9.noarc 398 kB/s | 20 kB 00:00 2024-09-21T00:23:30.804 INFO:teuthology.orchestra.run.smithi112.stdout:(147/157): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-21T00:23:30.846 INFO:teuthology.orchestra.run.smithi112.stdout:(148/157): python3-werkzeug-2.0.3-3.el9.1.noarc 2.9 MB/s | 427 kB 00:00 2024-09-21T00:23:30.896 INFO:teuthology.orchestra.run.smithi112.stdout:(149/157): thrift-0.15.0-2.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2024-09-21T00:23:31.034 INFO:teuthology.orchestra.run.smithi187.stdout:(40/157): fuse-2.9.9-16.el9.x86_64.rpm 117 kB/s | 80 kB 00:00 2024-09-21T00:23:31.079 INFO:teuthology.orchestra.run.smithi112.stdout:(150/157): NetworkManager-1.51.0-1.el9.x86_64.r 13 MB/s | 2.3 MB 00:00 2024-09-21T00:23:31.138 INFO:teuthology.orchestra.run.smithi112.stdout:(151/157): librbd1-19.3.0-5120.g18432218.el9.x8 11 MB/s | 3.2 MB 00:00 2024-09-21T00:23:31.196 INFO:teuthology.orchestra.run.smithi112.stdout:(152/157): librados2-19.3.0-5120.g18432218.el9. 8.6 MB/s | 3.4 MB 00:00 2024-09-21T00:23:31.222 INFO:teuthology.orchestra.run.smithi112.stdout:(153/157): NetworkManager-team-1.51.0-1.el9.x86 491 kB/s | 41 kB 00:00 2024-09-21T00:23:31.272 INFO:teuthology.orchestra.run.smithi112.stdout:(154/157): NetworkManager-libnm-1.51.0-1.el9.x8 9.4 MB/s | 1.8 MB 00:00 2024-09-21T00:23:31.306 INFO:teuthology.orchestra.run.smithi112.stdout:(155/157): NetworkManager-tui-1.51.0-1.el9.x86_ 2.2 MB/s | 249 kB 00:00 2024-09-21T00:23:31.356 INFO:teuthology.orchestra.run.smithi112.stdout:(156/157): openssl-libs-3.2.2-6.el9.x86_64.rpm 29 MB/s | 2.4 MB 00:00 2024-09-21T00:23:31.368 INFO:teuthology.orchestra.run.smithi187.stdout:(41/157): libgfortran-11.5.0-2.el9.x86_64.rpm 843 kB/s | 801 kB 00:00 2024-09-21T00:23:31.389 INFO:teuthology.orchestra.run.smithi112.stdout:(157/157): openssl-3.2.2-6.el9.x86_64.rpm 8.1 MB/s | 1.4 MB 00:00 2024-09-21T00:23:31.396 INFO:teuthology.orchestra.run.smithi112.stdout:-------------------------------------------------------------------------------- 2024-09-21T00:23:31.396 INFO:teuthology.orchestra.run.smithi112.stdout:Total 25 MB/s | 228 MB 00:09 2024-09-21T00:23:31.426 INFO:teuthology.orchestra.run.smithi187.stdout:(42/157): libquadmath-11.5.0-2.el9.x86_64.rpm 263 kB/s | 191 kB 00:00 2024-09-21T00:23:31.927 INFO:teuthology.orchestra.run.smithi187.stdout:(43/157): python3-cffi-1.14.5-5.el9.x86_64.rpm 284 kB/s | 253 kB 00:00 2024-09-21T00:23:32.111 INFO:teuthology.orchestra.run.smithi187.stdout:(44/157): python3-ply-3.11-14.el9.noarch.rpm 156 kB/s | 106 kB 00:00 2024-09-21T00:23:32.623 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction check 2024-09-21T00:23:32.627 INFO:teuthology.orchestra.run.smithi187.stdout:(45/157): python3-pycparser-2.20-6.el9.noarch.r 193 kB/s | 135 kB 00:00 2024-09-21T00:23:32.776 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction check succeeded. 2024-09-21T00:23:32.776 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction test 2024-09-21T00:23:32.877 INFO:teuthology.orchestra.run.smithi187.stdout:(46/157): python3-requests-2.25.1-8.el9.noarch. 163 kB/s | 125 kB 00:00 2024-09-21T00:23:33.003 INFO:teuthology.orchestra.run.smithi187.stdout:(47/157): python3-cryptography-36.0.1-4.el9.x86 768 kB/s | 1.2 MB 00:01 2024-09-21T00:23:33.070 INFO:teuthology.orchestra.run.smithi187.stdout:(48/157): automake-1.16.2-8.el9.noarch.rpm 3.7 MB/s | 689 kB 00:00 2024-09-21T00:23:33.112 INFO:teuthology.orchestra.run.smithi187.stdout:(49/157): boost-program-options-1.75.0-8.el9.x8 2.5 MB/s | 107 kB 00:00 2024-09-21T00:23:33.154 INFO:teuthology.orchestra.run.smithi187.stdout:(50/157): elfutils-libelf-devel-0.191-4.el9.x86 522 kB/s | 22 kB 00:00 2024-09-21T00:23:33.204 INFO:teuthology.orchestra.run.smithi187.stdout:(51/157): bison-3.7.4-5.el9.x86_64.rpm 4.6 MB/s | 943 kB 00:00 2024-09-21T00:23:33.229 INFO:teuthology.orchestra.run.smithi187.stdout:(52/157): flex-2.6.4-9.el9.x86_64.rpm 4.1 MB/s | 313 kB 00:00 2024-09-21T00:23:33.254 INFO:teuthology.orchestra.run.smithi187.stdout:(53/157): flexiblas-3.0.4-8.el9.x86_64.rpm 651 kB/s | 33 kB 00:00 2024-09-21T00:23:33.298 INFO:teuthology.orchestra.run.smithi187.stdout:(54/157): flexiblas-openblas-openmp-3.0.4-8.el9 439 kB/s | 18 kB 00:00 2024-09-21T00:23:33.347 INFO:teuthology.orchestra.run.smithi187.stdout:(55/157): flexiblas-netlib-3.0.4-8.el9.x86_64.r 26 MB/s | 3.0 MB 00:00 2024-09-21T00:23:33.372 INFO:teuthology.orchestra.run.smithi187.stdout:(56/157): libaio-devel-0.3.111-13.el9.x86_64.rp 148 kB/s | 11 kB 00:00 2024-09-21T00:23:33.397 INFO:teuthology.orchestra.run.smithi187.stdout:(57/157): libnbd-1.20.2-2.el9.x86_64.rpm 3.3 MB/s | 171 kB 00:00 2024-09-21T00:23:33.422 INFO:teuthology.orchestra.run.smithi187.stdout:(58/157): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-09-21T00:23:33.448 INFO:teuthology.orchestra.run.smithi187.stdout:(59/157): librabbitmq-0.11.0-7.el9.x86_64.rpm 901 kB/s | 45 kB 00:00 2024-09-21T00:23:33.490 INFO:teuthology.orchestra.run.smithi187.stdout:(60/157): librdkafka-1.6.1-102.el9.x86_64.rpm 9.7 MB/s | 662 kB 00:00 2024-09-21T00:23:33.523 INFO:teuthology.orchestra.run.smithi187.stdout:(61/157): libtool-2.4.6-46.el9.x86_64.rpm 7.6 MB/s | 581 kB 00:00 2024-09-21T00:23:33.548 INFO:teuthology.orchestra.run.smithi187.stdout:(62/157): libuuid-devel-2.37.4-20.el9.x86_64.rp 418 kB/s | 24 kB 00:00 2024-09-21T00:23:33.574 INFO:teuthology.orchestra.run.smithi187.stdout:(63/157): python3-urllib3-1.26.5-6.el9.noarch.r 227 kB/s | 215 kB 00:00 2024-09-21T00:23:33.599 INFO:teuthology.orchestra.run.smithi187.stdout:(64/157): libzstd-devel-1.5.1-2.el9.x86_64.rpm 629 kB/s | 47 kB 00:00 2024-09-21T00:23:33.624 INFO:teuthology.orchestra.run.smithi187.stdout:(65/157): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.8 MB/s | 292 kB 00:00 2024-09-21T00:23:33.649 INFO:teuthology.orchestra.run.smithi187.stdout:(66/157): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-09-21T00:23:33.692 INFO:teuthology.orchestra.run.smithi187.stdout:(67/157): openblas-0.3.26-2.el9.x86_64.rpm 419 kB/s | 38 kB 00:00 2024-09-21T00:23:33.775 INFO:teuthology.orchestra.run.smithi187.stdout:(68/157): openblas-openmp-0.3.26-2.el9.x86_64.r 33 MB/s | 4.9 MB 00:00 2024-09-21T00:23:33.834 INFO:teuthology.orchestra.run.smithi187.stdout:(69/157): openssl-devel-3.2.2-6.el9.x86_64.rpm 24 MB/s | 4.4 MB 00:00 2024-09-21T00:23:33.875 INFO:teuthology.orchestra.run.smithi187.stdout:(70/157): postgresql-contrib-13.16-1.el9.x86_64 8.9 MB/s | 910 kB 00:00 2024-09-21T00:23:33.901 INFO:teuthology.orchestra.run.smithi187.stdout:(71/157): postgresql-private-libs-13.16-1.el9.x 2.0 MB/s | 137 kB 00:00 2024-09-21T00:23:33.943 INFO:teuthology.orchestra.run.smithi187.stdout:(72/157): postgresql-13.16-1.el9.x86_64.rpm 6.3 MB/s | 1.6 MB 00:00 2024-09-21T00:23:33.976 INFO:teuthology.orchestra.run.smithi187.stdout:(73/157): protobuf-3.14.0-13.el9.x86_64.rpm 13 MB/s | 1.0 MB 00:00 2024-09-21T00:23:34.002 INFO:teuthology.orchestra.run.smithi187.stdout:(74/157): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-09-21T00:23:34.094 INFO:teuthology.orchestra.run.smithi187.stdout:(75/157): postgresql-server-13.16-1.el9.x86_64. 26 MB/s | 5.8 MB 00:00 2024-09-21T00:23:34.119 INFO:teuthology.orchestra.run.smithi187.stdout:(76/157): python3-colorama-0.4.6-3.el9.noarch.r 523 kB/s | 61 kB 00:00 2024-09-21T00:23:34.144 INFO:teuthology.orchestra.run.smithi187.stdout:(77/157): python3-devel-3.9.19-8.el9.x86_64.rpm 4.8 MB/s | 245 kB 00:00 2024-09-21T00:23:34.170 INFO:teuthology.orchestra.run.smithi187.stdout:(78/157): python3-jinja2-2.11.3-6.el9.noarch.rp 4.8 MB/s | 248 kB 00:00 2024-09-21T00:23:34.187 INFO:teuthology.orchestra.run.smithi187.stdout:(79/157): python3-jsonpatch-1.21-16.el9.noarch. 611 kB/s | 26 kB 00:00 2024-09-21T00:23:34.204 INFO:teuthology.orchestra.run.smithi187.stdout:(80/157): python3-jsonpointer-2.0-4.el9.noarch. 572 kB/s | 19 kB 00:00 2024-09-21T00:23:34.237 INFO:teuthology.orchestra.run.smithi187.stdout:(81/157): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2024-09-21T00:23:34.254 INFO:teuthology.orchestra.run.smithi187.stdout:(82/157): python3-markupsafe-1.1.1-12.el9.x86_6 693 kB/s | 35 kB 00:00 2024-09-21T00:23:34.338 INFO:teuthology.orchestra.run.smithi187.stdout:(83/157): python3-babel-2.9.1-2.el9.noarch.rpm 17 MB/s | 6.0 MB 00:00 2024-09-21T00:23:34.363 INFO:teuthology.orchestra.run.smithi187.stdout:(84/157): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.3 MB/s | 250 kB 00:00 2024-09-21T00:23:34.388 INFO:teuthology.orchestra.run.smithi187.stdout:(85/157): python3-oauthlib-3.1.1-5.el9.noarch.r 4.4 MB/s | 222 kB 00:00 2024-09-21T00:23:34.414 INFO:teuthology.orchestra.run.smithi187.stdout:(86/157): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-09-21T00:23:34.447 INFO:teuthology.orchestra.run.smithi187.stdout:(87/157): python3-prettytable-0.7.2-27.el9.noar 721 kB/s | 42 kB 00:00 2024-09-21T00:23:34.481 INFO:teuthology.orchestra.run.smithi187.stdout:(88/157): python3-protobuf-3.14.0-13.el9.noarch 3.9 MB/s | 269 kB 00:00 2024-09-21T00:23:34.506 INFO:teuthology.orchestra.run.smithi187.stdout:(89/157): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.6 MB/s | 159 kB 00:00 2024-09-21T00:23:34.531 INFO:teuthology.orchestra.run.smithi187.stdout:(90/157): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-09-21T00:23:34.557 INFO:teuthology.orchestra.run.smithi187.stdout:(91/157): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-09-21T00:23:34.582 INFO:teuthology.orchestra.run.smithi187.stdout:(92/157): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-09-21T00:23:34.657 INFO:teuthology.orchestra.run.smithi187.stdout:(93/157): python3-numpy-1.20.1-5.el9.x86_64.rpm 12 MB/s | 5.1 MB 00:00 2024-09-21T00:23:34.682 INFO:teuthology.orchestra.run.smithi187.stdout:(94/157): python3-toml-0.10.2-6.el9.noarch.rpm 416 kB/s | 42 kB 00:00 2024-09-21T00:23:34.708 INFO:teuthology.orchestra.run.smithi187.stdout:(95/157): python3-wcwidth-0.2.5-8.el9.noarch.rp 882 kB/s | 44 kB 00:00 2024-09-21T00:23:34.733 INFO:teuthology.orchestra.run.smithi187.stdout:(96/157): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-09-21T00:23:34.758 INFO:teuthology.orchestra.run.smithi187.stdout:(97/157): qatlib-service-24.02.0-1.el9.x86_64.r 718 kB/s | 36 kB 00:00 2024-09-21T00:23:34.783 INFO:teuthology.orchestra.run.smithi187.stdout:(98/157): qatzip-libs-1.2.0-1.el9.x86_64.rpm 947 kB/s | 47 kB 00:00 2024-09-21T00:23:34.817 INFO:teuthology.orchestra.run.smithi187.stdout:(99/157): socat-1.7.4.1-6.el9.x86_64.rpm 5.1 MB/s | 304 kB 00:00 2024-09-21T00:23:34.842 INFO:teuthology.orchestra.run.smithi187.stdout:(100/157): uuid-1.6.2-55.el9.x86_64.rpm 988 kB/s | 58 kB 00:00 2024-09-21T00:23:34.868 INFO:teuthology.orchestra.run.smithi187.stdout:(101/157): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.9 MB/s | 96 kB 00:00 2024-09-21T00:23:34.893 INFO:teuthology.orchestra.run.smithi187.stdout:(102/157): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-21T00:23:34.918 INFO:teuthology.orchestra.run.smithi187.stdout:(103/157): zlib-devel-1.2.11-41.el9.x86_64.rpm 895 kB/s | 45 kB 00:00 2024-09-21T00:23:34.986 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction test succeeded. 2024-09-21T00:23:34.987 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction 2024-09-21T00:23:35.144 INFO:teuthology.orchestra.run.smithi187.stdout:(104/157): python3-scipy-1.6.2-8.el9.x86_64.rpm 28 MB/s | 16 MB 00:00 2024-09-21T00:23:35.202 INFO:teuthology.orchestra.run.smithi187.stdout:(105/157): abseil-cpp-20211102.0-3.el9.x86_64.r 9.3 MB/s | 548 kB 00:00 2024-09-21T00:23:35.236 INFO:teuthology.orchestra.run.smithi187.stdout:(106/157): gperftools-libs-2.9.1-3.el9.x86_64.r 9.0 MB/s | 308 kB 00:00 2024-09-21T00:23:35.261 INFO:teuthology.orchestra.run.smithi187.stdout:(107/157): grpc-data-1.46.7-10.el9.noarch.rpm 780 kB/s | 19 kB 00:00 2024-09-21T00:23:35.378 INFO:teuthology.orchestra.run.smithi187.stdout:(108/157): libarrow-9.0.0-11.el9.x86_64.rpm 38 MB/s | 4.4 MB 00:00 2024-09-21T00:23:35.403 INFO:teuthology.orchestra.run.smithi187.stdout:(109/157): lua-devel-5.4.4-4.el9.x86_64.rpm 44 kB/s | 22 kB 00:00 2024-09-21T00:23:35.429 INFO:teuthology.orchestra.run.smithi187.stdout:(110/157): libarrow-doc-9.0.0-11.el9.noarch.rpm 521 kB/s | 26 kB 00:00 2024-09-21T00:23:35.462 INFO:teuthology.orchestra.run.smithi187.stdout:(111/157): libunwind-1.6.2-1.el9.x86_64.rpm 2.5 MB/s | 67 kB 00:00 2024-09-21T00:23:35.487 INFO:teuthology.orchestra.run.smithi187.stdout:(112/157): luarocks-3.9.2-5.el9.noarch.rpm 6.4 MB/s | 151 kB 00:00 2024-09-21T00:23:35.513 INFO:teuthology.orchestra.run.smithi187.stdout:(113/157): liboath-2.6.7-2.el9.x86_64.rpm 446 kB/s | 49 kB 00:00 2024-09-21T00:23:35.555 INFO:teuthology.orchestra.run.smithi187.stdout:(114/157): parquet-libs-9.0.0-11.el9.x86_64.rpm 12 MB/s | 839 kB 00:00 2024-09-21T00:23:35.580 INFO:teuthology.orchestra.run.smithi187.stdout:(115/157): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-21T00:23:35.605 INFO:teuthology.orchestra.run.smithi187.stdout:(116/157): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-21T00:23:35.630 INFO:teuthology.orchestra.run.smithi187.stdout:(117/157): python3-asyncssh-2.13.2-5.el9.noarch 4.6 MB/s | 548 kB 00:00 2024-09-21T00:23:35.656 INFO:teuthology.orchestra.run.smithi187.stdout:(118/157): python3-certifi-2023.05.07-4.el9.noa 281 kB/s | 14 kB 00:00 2024-09-21T00:23:35.681 INFO:teuthology.orchestra.run.smithi187.stdout:(119/157): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-09-21T00:23:35.731 INFO:teuthology.orchestra.run.smithi187.stdout:(120/157): python3-cherrypy-18.6.1-2.el9.noarch 4.7 MB/s | 358 kB 00:00 2024-09-21T00:23:35.765 INFO:teuthology.orchestra.run.smithi187.stdout:(121/157): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.5 MB/s | 303 kB 00:00 2024-09-21T00:23:35.799 INFO:teuthology.orchestra.run.smithi187.stdout:(122/157): protobuf-compiler-3.14.0-13.el9.x86_ 981 kB/s | 863 kB 00:00 2024-09-21T00:23:35.824 INFO:teuthology.orchestra.run.smithi187.stdout:(123/157): python3-google-auth-2.34.0-1.el9.noa 2.3 MB/s | 220 kB 00:00 2024-09-21T00:23:35.849 INFO:teuthology.orchestra.run.smithi187.stdout:(124/157): python3-jaraco-8.2.1-3.el9.noarch.rp 607 kB/s | 11 kB 00:00 2024-09-21T00:23:35.874 INFO:teuthology.orchestra.run.smithi187.stdout:(125/157): python3-jaraco-classes-3.2.1-5.el9.n 764 kB/s | 18 kB 00:00 2024-09-21T00:23:35.891 INFO:teuthology.orchestra.run.smithi187.stdout:(126/157): python3-jaraco-collections-3.0.0-8.e 1.4 MB/s | 23 kB 00:00 2024-09-21T00:23:35.917 INFO:teuthology.orchestra.run.smithi187.stdout:(127/157): python3-grpcio-tools-1.46.7-10.el9.x 1.2 MB/s | 144 kB 00:00 2024-09-21T00:23:35.934 INFO:teuthology.orchestra.run.smithi187.stdout:(128/157): python3-jaraco-functools-3.5.0-2.el9 463 kB/s | 19 kB 00:00 2024-09-21T00:23:35.959 INFO:teuthology.orchestra.run.smithi187.stdout:(129/157): python3-jaraco-text-3.2.0-6.el9.noar 473 kB/s | 20 kB 00:00 2024-09-21T00:23:35.993 INFO:teuthology.orchestra.run.smithi187.stdout:(130/157): python3-logutils-0.3.5-21.el9.noarch 1.4 MB/s | 46 kB 00:00 2024-09-21T00:23:36.018 INFO:teuthology.orchestra.run.smithi187.stdout:(131/157): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-21T00:23:36.043 INFO:teuthology.orchestra.run.smithi187.stdout:(132/157): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-21T00:23:36.085 INFO:teuthology.orchestra.run.smithi187.stdout:(133/157): python3-kubernetes-26.1.0-2.el9.noar 6.8 MB/s | 1.0 MB 00:00 2024-09-21T00:23:36.110 INFO:teuthology.orchestra.run.smithi187.stdout:(134/157): python3-portend-3.1.0-2.el9.noarch.r 663 kB/s | 16 kB 00:00 2024-09-21T00:23:36.136 INFO:teuthology.orchestra.run.smithi187.stdout:(135/157): python3-pecan-1.4.2-3.el9.noarch.rpm 2.9 MB/s | 272 kB 00:00 2024-09-21T00:23:36.170 INFO:teuthology.orchestra.run.smithi187.stdout:(136/157): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.5 MB/s | 90 kB 00:00 2024-09-21T00:23:36.195 INFO:teuthology.orchestra.run.smithi187.stdout:(137/157): python3-pyperclip-1.8.2-2.el9.noarch 480 kB/s | 28 kB 00:00 2024-09-21T00:23:36.220 INFO:teuthology.orchestra.run.smithi187.stdout:(138/157): python3-repoze-lru-0.7-16.el9.noarch 612 kB/s | 31 kB 00:00 2024-09-21T00:23:36.245 INFO:teuthology.orchestra.run.smithi187.stdout:(139/157): python3-routes-2.5.1-5.el9.noarch.rp 3.7 MB/s | 188 kB 00:00 2024-09-21T00:23:36.270 INFO:teuthology.orchestra.run.smithi187.stdout:(140/157): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-09-21T00:23:36.304 INFO:teuthology.orchestra.run.smithi187.stdout:(141/157): python3-tempora-5.0.0-2.el9.noarch.r 1.0 MB/s | 36 kB 00:00 2024-09-21T00:23:36.330 INFO:teuthology.orchestra.run.smithi187.stdout:(142/157): python3-typing-extensions-4.12.2-2.e 1.2 MB/s | 74 kB 00:00 2024-09-21T00:23:36.380 INFO:teuthology.orchestra.run.smithi187.stdout:(143/157): python3-grpcio-1.46.7-10.el9.x86_64. 3.3 MB/s | 2.0 MB 00:00 2024-09-21T00:23:36.405 INFO:teuthology.orchestra.run.smithi187.stdout:(144/157): python3-websocket-client-1.2.3-2.el9 1.2 MB/s | 90 kB 00:00 2024-09-21T00:23:36.431 INFO:teuthology.orchestra.run.smithi187.stdout:(145/157): python3-webob-1.8.8-2.el9.noarch.rpm 1.8 MB/s | 230 kB 00:00 2024-09-21T00:23:36.456 INFO:teuthology.orchestra.run.smithi187.stdout:(146/157): python3-zc-lockfile-2.0-10.el9.noarc 397 kB/s | 20 kB 00:00 2024-09-21T00:23:36.481 INFO:teuthology.orchestra.run.smithi187.stdout:(147/157): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-21T00:23:36.515 INFO:teuthology.orchestra.run.smithi187.stdout:(148/157): python3-werkzeug-2.0.3-3.el9.1.noarc 3.1 MB/s | 427 kB 00:00 2024-09-21T00:23:36.565 INFO:teuthology.orchestra.run.smithi187.stdout:(149/157): thrift-0.15.0-2.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-09-21T00:23:36.732 INFO:teuthology.orchestra.run.smithi187.stdout:(150/157): librados2-19.3.0-5120.g18432218.el9. 14 MB/s | 3.4 MB 00:00 2024-09-21T00:23:36.940 INFO:teuthology.orchestra.run.smithi187.stdout:(151/157): librbd1-19.3.0-5120.g18432218.el9.x8 7.4 MB/s | 3.2 MB 00:00 2024-09-21T00:23:37.507 INFO:teuthology.orchestra.run.smithi187.stdout:(152/157): NetworkManager-team-1.51.0-1.el9.x86 72 kB/s | 41 kB 00:00 2024-09-21T00:23:37.591 INFO:teuthology.orchestra.run.smithi187.stdout:(153/157): NetworkManager-1.51.0-1.el9.x86_64.r 2.3 MB/s | 2.3 MB 00:01 2024-09-21T00:23:37.799 INFO:teuthology.orchestra.run.smithi187.stdout:(154/157): NetworkManager-libnm-1.51.0-1.el9.x8 1.7 MB/s | 1.8 MB 00:01 2024-09-21T00:23:38.283 INFO:teuthology.orchestra.run.smithi187.stdout:(155/157): NetworkManager-tui-1.51.0-1.el9.x86_ 321 kB/s | 249 kB 00:00 2024-09-21T00:23:38.566 INFO:teuthology.orchestra.run.smithi187.stdout:(156/157): openssl-3.2.2-6.el9.x86_64.rpm 1.4 MB/s | 1.4 MB 00:00 2024-09-21T00:23:38.808 INFO:teuthology.orchestra.run.smithi187.stdout:(157/157): openssl-libs-3.2.2-6.el9.x86_64.rpm 2.4 MB/s | 2.4 MB 00:01 2024-09-21T00:23:38.816 INFO:teuthology.orchestra.run.smithi187.stdout:-------------------------------------------------------------------------------- 2024-09-21T00:23:38.816 INFO:teuthology.orchestra.run.smithi187.stdout:Total 16 MB/s | 228 MB 00:14 2024-09-21T00:23:40.072 INFO:teuthology.orchestra.run.smithi187.stdout:Running transaction check 2024-09-21T00:23:40.226 INFO:teuthology.orchestra.run.smithi187.stdout:Transaction check succeeded. 2024-09-21T00:23:40.226 INFO:teuthology.orchestra.run.smithi187.stdout:Running transaction test 2024-09-21T00:23:41.245 INFO:teuthology.orchestra.run.smithi112.stdout: Preparing : 1/1 2024-09-21T00:23:41.334 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/165 2024-09-21T00:23:41.375 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/165 2024-09-21T00:23:41.425 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/165 2024-09-21T00:23:41.459 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/165 2024-09-21T00:23:41.485 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/165 2024-09-21T00:23:41.542 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/165 2024-09-21T00:23:42.075 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/165 2024-09-21T00:23:42.194 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T00:23:42.322 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T00:23:42.345 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T00:23:42.438 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T00:23:42.463 INFO:teuthology.orchestra.run.smithi187.stdout:Transaction test succeeded. 2024-09-21T00:23:42.463 INFO:teuthology.orchestra.run.smithi187.stdout:Running transaction 2024-09-21T00:23:42.466 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T00:23:42.488 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/165 2024-09-21T00:23:42.657 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 12/165 2024-09-21T00:23:42.714 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : postgresql-13.16-1.el9.x86_64 13/165 2024-09-21T00:23:42.750 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T00:23:42.864 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T00:23:42.893 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T00:23:42.956 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T00:23:42.985 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T00:23:43.006 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T00:23:43.220 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T00:23:43.414 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T00:23:43.533 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 18/165 2024-09-21T00:23:43.654 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T00:23:43.727 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T00:23:43.953 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T00:23:44.069 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : re2-1:20211101-20.el9.x86_64 20/165 2024-09-21T00:23:44.098 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/165 2024-09-21T00:23:44.132 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/165 2024-09-21T00:23:44.205 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/165 2024-09-21T00:23:44.249 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/165 2024-09-21T00:23:44.281 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/165 2024-09-21T00:23:44.333 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/165 2024-09-21T00:23:44.366 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/165 2024-09-21T00:23:44.394 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/165 2024-09-21T00:23:44.462 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/165 2024-09-21T00:23:44.516 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/165 2024-09-21T00:23:44.547 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/165 2024-09-21T00:23:44.591 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/165 2024-09-21T00:23:44.671 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/165 2024-09-21T00:23:44.701 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 34/165 2024-09-21T00:23:44.736 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 35/165 2024-09-21T00:23:44.765 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 36/165 2024-09-21T00:23:44.833 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/165 2024-09-21T00:23:44.879 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/165 2024-09-21T00:23:44.960 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/165 2024-09-21T00:23:45.095 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/165 2024-09-21T00:23:45.147 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/165 2024-09-21T00:23:45.674 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/165 2024-09-21T00:23:45.752 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/165 2024-09-21T00:23:45.781 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/165 2024-09-21T00:23:45.824 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/165 2024-09-21T00:23:45.853 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 46/165 2024-09-21T00:23:45.895 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/165 2024-09-21T00:23:46.044 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/165 2024-09-21T00:23:46.098 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/165 2024-09-21T00:23:46.190 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/165 2024-09-21T00:23:46.245 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/165 2024-09-21T00:23:46.317 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/165 2024-09-21T00:23:46.358 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 53/165 2024-09-21T00:23:46.394 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 54/165 2024-09-21T00:23:46.440 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/165 2024-09-21T00:23:46.476 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/165 2024-09-21T00:23:46.520 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/165 2024-09-21T00:23:46.562 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/165 2024-09-21T00:23:46.698 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/165 2024-09-21T00:23:46.746 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/165 2024-09-21T00:23:46.796 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 61/165 2024-09-21T00:23:46.899 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/165 2024-09-21T00:23:46.930 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 63/165 2024-09-21T00:23:46.971 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/165 2024-09-21T00:23:47.003 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 65/165 2024-09-21T00:23:47.070 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/165 2024-09-21T00:23:47.229 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/165 2024-09-21T00:23:47.353 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/165 2024-09-21T00:23:47.385 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/165 2024-09-21T00:23:47.424 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/165 2024-09-21T00:23:47.459 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/165 2024-09-21T00:23:47.498 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/165 2024-09-21T00:23:47.530 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/165 2024-09-21T00:23:47.559 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/165 2024-09-21T00:23:47.594 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/165 2024-09-21T00:23:47.627 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/165 2024-09-21T00:23:47.669 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/165 2024-09-21T00:23:47.703 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/165 2024-09-21T00:23:47.789 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/165 2024-09-21T00:23:47.833 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/165 2024-09-21T00:23:47.864 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/165 2024-09-21T00:23:47.966 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/165 2024-09-21T00:23:48.015 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/165 2024-09-21T00:23:48.737 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/165 2024-09-21T00:23:49.021 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/165 2024-09-21T00:23:49.054 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/165 2024-09-21T00:23:49.153 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/165 2024-09-21T00:23:49.180 INFO:teuthology.orchestra.run.smithi187.stdout: Preparing : 1/1 2024-09-21T00:23:49.209 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 88/165 2024-09-21T00:23:49.254 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 89/165 2024-09-21T00:23:49.312 INFO:teuthology.orchestra.run.smithi187.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/165 2024-09-21T00:23:49.342 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 90/165 2024-09-21T00:23:49.358 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/165 2024-09-21T00:23:49.374 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 91/165 2024-09-21T00:23:49.412 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/165 2024-09-21T00:23:49.454 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/165 2024-09-21T00:23:49.480 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/165 2024-09-21T00:23:49.544 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/165 2024-09-21T00:23:49.557 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : openblas-0.3.26-2.el9.x86_64 92/165 2024-09-21T00:23:49.604 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 93/165 2024-09-21T00:23:49.686 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 94/165 2024-09-21T00:23:49.881 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/165 2024-09-21T00:23:49.997 INFO:teuthology.orchestra.run.smithi187.stdout: Upgrading : librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T00:23:50.124 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T00:23:50.157 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T00:23:50.248 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T00:23:50.286 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T00:23:50.325 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/165 2024-09-21T00:23:50.345 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 95/165 2024-09-21T00:23:50.459 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 96/165 2024-09-21T00:23:50.499 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 12/165 2024-09-21T00:23:50.559 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : postgresql-13.16-1.el9.x86_64 13/165 2024-09-21T00:23:50.587 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T00:23:50.680 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T00:23:50.704 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T00:23:50.772 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T00:23:50.797 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T00:23:50.823 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T00:23:51.024 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T00:23:51.192 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T00:23:51.344 INFO:teuthology.orchestra.run.smithi187.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 18/165 2024-09-21T00:23:51.456 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T00:23:51.538 INFO:teuthology.orchestra.run.smithi187.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T00:23:51.717 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 97/165 2024-09-21T00:23:51.764 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T00:23:51.806 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 98/165 2024-09-21T00:23:51.826 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 99/165 2024-09-21T00:23:51.851 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 100/165 2024-09-21T00:23:51.872 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : re2-1:20211101-20.el9.x86_64 20/165 2024-09-21T00:23:51.903 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/165 2024-09-21T00:23:51.935 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/165 2024-09-21T00:23:52.016 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/165 2024-09-21T00:23:52.061 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/165 2024-09-21T00:23:52.106 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/165 2024-09-21T00:23:52.154 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/165 2024-09-21T00:23:52.186 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/165 2024-09-21T00:23:52.192 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 101/165 2024-09-21T00:23:52.214 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/165 2024-09-21T00:23:52.282 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/165 2024-09-21T00:23:52.344 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/165 2024-09-21T00:23:52.375 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/165 2024-09-21T00:23:52.413 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/165 2024-09-21T00:23:52.422 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T00:23:52.490 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/165 2024-09-21T00:23:52.507 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T00:23:52.520 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 34/165 2024-09-21T00:23:52.535 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 103/165 2024-09-21T00:23:52.564 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 35/165 2024-09-21T00:23:52.571 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 104/165 2024-09-21T00:23:52.593 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 36/165 2024-09-21T00:23:52.664 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/165 2024-09-21T00:23:52.707 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/165 2024-09-21T00:23:52.790 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/165 2024-09-21T00:23:52.930 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/165 2024-09-21T00:23:52.985 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/165 2024-09-21T00:23:53.084 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 105/165 2024-09-21T00:23:53.116 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T00:23:53.188 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T00:23:53.218 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 107/165 2024-09-21T00:23:53.510 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/165 2024-09-21T00:23:53.597 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/165 2024-09-21T00:23:53.626 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/165 2024-09-21T00:23:53.667 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/165 2024-09-21T00:23:53.707 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 46/165 2024-09-21T00:23:53.749 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/165 2024-09-21T00:23:53.910 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/165 2024-09-21T00:23:53.968 INFO:teuthology.orchestra.run.smithi187.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/165 2024-09-21T00:23:54.059 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/165 2024-09-21T00:23:54.124 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/165 2024-09-21T00:23:54.196 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/165 2024-09-21T00:23:54.228 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 53/165 2024-09-21T00:23:54.264 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 54/165 2024-09-21T00:23:54.319 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/165 2024-09-21T00:23:54.363 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/165 2024-09-21T00:23:54.406 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/165 2024-09-21T00:23:54.448 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/165 2024-09-21T00:23:54.584 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/165 2024-09-21T00:23:54.633 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/165 2024-09-21T00:23:54.675 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 61/165 2024-09-21T00:23:54.777 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/165 2024-09-21T00:23:54.817 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 63/165 2024-09-21T00:23:54.861 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/165 2024-09-21T00:23:54.894 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 65/165 2024-09-21T00:23:54.919 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/165 2024-09-21T00:23:55.014 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/165 2024-09-21T00:23:55.144 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/165 2024-09-21T00:23:55.184 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/165 2024-09-21T00:23:55.212 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/165 2024-09-21T00:23:55.246 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/165 2024-09-21T00:23:55.285 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/165 2024-09-21T00:23:55.317 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/165 2024-09-21T00:23:55.347 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/165 2024-09-21T00:23:55.570 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T00:23:55.572 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/165 2024-09-21T00:23:55.614 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T00:23:55.682 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/165 2024-09-21T00:23:55.725 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/165 2024-09-21T00:23:55.758 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/165 2024-09-21T00:23:55.849 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/165 2024-09-21T00:23:55.897 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/165 2024-09-21T00:23:55.936 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/165 2024-09-21T00:23:56.048 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/165 2024-09-21T00:23:56.103 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/165 2024-09-21T00:23:56.208 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T00:23:56.250 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T00:23:56.341 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T00:23:56.341 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-21T00:23:56.341 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-21T00:23:56.341 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:23:56.385 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T00:23:56.827 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/165 2024-09-21T00:23:56.875 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/165 2024-09-21T00:23:56.909 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/165 2024-09-21T00:23:57.008 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/165 2024-09-21T00:23:57.064 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 88/165 2024-09-21T00:23:57.109 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 89/165 2024-09-21T00:23:57.197 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 90/165 2024-09-21T00:23:57.237 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 91/165 2024-09-21T00:23:57.411 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : openblas-0.3.26-2.el9.x86_64 92/165 2024-09-21T00:23:57.443 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 93/165 2024-09-21T00:23:57.520 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 94/165 2024-09-21T00:23:58.169 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 95/165 2024-09-21T00:23:58.291 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 96/165 2024-09-21T00:23:59.559 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 97/165 2024-09-21T00:23:59.655 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 98/165 2024-09-21T00:23:59.682 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 99/165 2024-09-21T00:23:59.715 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 100/165 2024-09-21T00:24:00.040 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 101/165 2024-09-21T00:24:00.087 INFO:teuthology.orchestra.run.smithi187.stdout: Upgrading : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T00:24:00.179 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T00:24:00.208 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 103/165 2024-09-21T00:24:00.253 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 104/165 2024-09-21T00:24:00.758 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 105/165 2024-09-21T00:24:00.790 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T00:24:00.852 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T00:24:00.875 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 107/165 2024-09-21T00:24:03.184 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T00:24:03.221 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T00:24:03.809 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T00:24:03.849 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T00:24:03.945 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T00:24:03.945 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-21T00:24:03.945 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-21T00:24:03.945 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:24:03.984 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T00:24:08.749 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T00:24:08.749 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /sys 2024-09-21T00:24:08.749 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /proc 2024-09-21T00:24:08.749 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /mnt 2024-09-21T00:24:08.749 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /var/tmp 2024-09-21T00:24:08.749 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /home 2024-09-21T00:24:08.750 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /root 2024-09-21T00:24:08.750 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /tmp 2024-09-21T00:24:08.750 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:24:08.931 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T00:24:08.946 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T00:24:08.946 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T00:24:08.947 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-21T00:24:08.947 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:24:10.575 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T00:24:10.602 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T00:24:10.602 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T00:24:10.602 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-21T00:24:10.602 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T00:24:10.602 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T00:24:10.602 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:24:10.834 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T00:24:10.858 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T00:24:10.858 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T00:24:10.858 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-21T00:24:10.858 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T00:24:10.858 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T00:24:10.858 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:24:11.334 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T00:24:11.358 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T00:24:11.358 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T00:24:11.358 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-21T00:24:11.358 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T00:24:11.358 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T00:24:11.359 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:24:11.481 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : automake-1.16.2-8.el9.noarch 115/165 2024-09-21T00:24:11.529 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-ply-3.11-14.el9.noarch 116/165 2024-09-21T00:24:11.585 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 117/165 2024-09-21T00:24:11.769 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 118/165 2024-09-21T00:24:11.841 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 119/165 2024-09-21T00:24:11.922 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 120/165 2024-09-21T00:24:12.009 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 121/165 2024-09-21T00:24:12.149 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 122/165 2024-09-21T00:24:12.193 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 123/165 2024-09-21T00:24:12.225 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 124/165 2024-09-21T00:24:12.253 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : fuse-2.9.9-16.el9.x86_64 125/165 2024-09-21T00:24:12.681 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T00:24:12.710 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T00:24:12.742 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 127/165 2024-09-21T00:24:12.808 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 128/165 2024-09-21T00:24:12.833 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T00:24:15.988 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T00:24:15.988 INFO:teuthology.orchestra.run.smithi187.stdout:skipping the directory /sys 2024-09-21T00:24:15.988 INFO:teuthology.orchestra.run.smithi187.stdout:skipping the directory /proc 2024-09-21T00:24:15.988 INFO:teuthology.orchestra.run.smithi187.stdout:skipping the directory /mnt 2024-09-21T00:24:15.988 INFO:teuthology.orchestra.run.smithi187.stdout:skipping the directory /var/tmp 2024-09-21T00:24:15.988 INFO:teuthology.orchestra.run.smithi187.stdout:skipping the directory /home 2024-09-21T00:24:15.989 INFO:teuthology.orchestra.run.smithi187.stdout:skipping the directory /root 2024-09-21T00:24:15.989 INFO:teuthology.orchestra.run.smithi187.stdout:skipping the directory /tmp 2024-09-21T00:24:15.989 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:24:16.083 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T00:24:16.130 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T00:24:16.164 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T00:24:16.180 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T00:24:16.181 INFO:teuthology.orchestra.run.smithi187.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T00:24:16.181 INFO:teuthology.orchestra.run.smithi187.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-21T00:24:16.181 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:24:17.109 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T00:24:17.131 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T00:24:17.245 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T00:24:17.418 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 132/165 2024-09-21T00:24:17.457 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T00:24:17.482 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T00:24:17.482 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T00:24:17.482 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-21T00:24:17.482 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T00:24:17.482 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T00:24:17.482 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:24:17.533 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T00:24:17.545 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T00:24:17.646 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-2:19.3.0-5120.g18432218.el9.x86_64 135/165 2024-09-21T00:24:17.707 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 136/165 2024-09-21T00:24:17.766 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T00:24:17.793 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T00:24:17.793 INFO:teuthology.orchestra.run.smithi187.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T00:24:17.793 INFO:teuthology.orchestra.run.smithi187.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-21T00:24:17.793 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T00:24:17.794 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T00:24:17.794 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:24:18.017 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T00:24:18.040 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T00:24:18.040 INFO:teuthology.orchestra.run.smithi187.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T00:24:18.041 INFO:teuthology.orchestra.run.smithi187.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-21T00:24:18.041 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T00:24:18.041 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T00:24:18.041 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:24:18.517 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T00:24:18.541 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T00:24:18.541 INFO:teuthology.orchestra.run.smithi187.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T00:24:18.541 INFO:teuthology.orchestra.run.smithi187.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-21T00:24:18.541 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T00:24:18.542 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T00:24:18.542 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:24:18.648 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : automake-1.16.2-8.el9.noarch 115/165 2024-09-21T00:24:18.705 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-ply-3.11-14.el9.noarch 116/165 2024-09-21T00:24:18.713 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libtool-2.4.6-46.el9.x86_64 137/165 2024-09-21T00:24:18.744 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T00:24:18.760 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 117/165 2024-09-21T00:24:18.768 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T00:24:18.768 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T00:24:18.768 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-21T00:24:18.768 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T00:24:18.768 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T00:24:18.768 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:24:18.811 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T00:24:18.834 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T00:24:18.835 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T00:24:18.835 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-21T00:24:18.835 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:24:18.887 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T00:24:18.910 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T00:24:18.911 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T00:24:18.911 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-21T00:24:18.911 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T00:24:18.911 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T00:24:18.911 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:24:18.948 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 118/165 2024-09-21T00:24:19.002 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 119/165 2024-09-21T00:24:19.081 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 120/165 2024-09-21T00:24:19.170 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 121/165 2024-09-21T00:24:19.238 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T00:24:19.261 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T00:24:19.262 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T00:24:19.262 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-21T00:24:19.262 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T00:24:19.262 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T00:24:19.262 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:24:19.318 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 122/165 2024-09-21T00:24:19.360 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 123/165 2024-09-21T00:24:19.392 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 124/165 2024-09-21T00:24:19.420 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : fuse-2.9.9-16.el9.x86_64 125/165 2024-09-21T00:24:19.804 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T00:24:19.843 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T00:24:19.875 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 127/165 2024-09-21T00:24:19.942 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 128/165 2024-09-21T00:24:19.966 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T00:24:23.180 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T00:24:23.247 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T00:24:23.384 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 142/165 2024-09-21T00:24:23.425 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 143/165 2024-09-21T00:24:23.464 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 144/165 2024-09-21T00:24:23.522 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 145/165 2024-09-21T00:24:23.549 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 146/165 2024-09-21T00:24:23.777 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 147/165 2024-09-21T00:24:23.821 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 148/165 2024-09-21T00:24:23.851 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 149/165 2024-09-21T00:24:23.880 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 150/165 2024-09-21T00:24:23.927 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 151/165 2024-09-21T00:24:24.207 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T00:24:24.239 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T00:24:24.353 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T00:24:24.514 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 132/165 2024-09-21T00:24:24.522 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T00:24:24.549 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T00:24:24.557 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T00:24:24.573 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T00:24:24.573 INFO:teuthology.orchestra.run.smithi187.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T00:24:24.573 INFO:teuthology.orchestra.run.smithi187.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-21T00:24:24.573 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T00:24:24.574 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T00:24:24.574 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:24:24.578 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T00:24:24.634 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T00:24:24.645 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T00:24:24.744 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-2:19.3.0-5120.g18432218.el9.x86_64 135/165 2024-09-21T00:24:24.805 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 136/165 2024-09-21T00:24:25.413 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 153/165 2024-09-21T00:24:25.503 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 154/165 2024-09-21T00:24:25.552 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 155/165 2024-09-21T00:24:25.644 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : flex-2.6.4-9.el9.x86_64 156/165 2024-09-21T00:24:25.673 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : bison-3.7.4-5.el9.x86_64 157/165 2024-09-21T00:24:25.698 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 158/165 2024-09-21T00:24:25.724 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 159/165 2024-09-21T00:24:25.724 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T00:24:25.768 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libtool-2.4.6-46.el9.x86_64 137/165 2024-09-21T00:24:25.794 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T00:24:25.816 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T00:24:25.817 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T00:24:25.817 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T00:24:25.817 INFO:teuthology.orchestra.run.smithi187.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T00:24:25.817 INFO:teuthology.orchestra.run.smithi187.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-21T00:24:25.817 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T00:24:25.817 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T00:24:25.817 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:24:25.861 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T00:24:25.884 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T00:24:25.884 INFO:teuthology.orchestra.run.smithi187.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T00:24:25.884 INFO:teuthology.orchestra.run.smithi187.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-21T00:24:25.884 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:24:25.892 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T00:24:25.917 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 162/165 2024-09-21T00:24:25.922 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T00:24:25.923 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T00:24:25.937 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T00:24:25.960 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T00:24:25.960 INFO:teuthology.orchestra.run.smithi187.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T00:24:25.960 INFO:teuthology.orchestra.run.smithi187.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-21T00:24:25.961 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T00:24:25.961 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T00:24:25.961 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:24:26.102 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T00:24:26.102 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T00:24:26.178 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T00:24:26.242 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T00:24:26.280 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T00:24:26.303 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T00:24:26.303 INFO:teuthology.orchestra.run.smithi187.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T00:24:26.303 INFO:teuthology.orchestra.run.smithi187.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-21T00:24:26.303 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T00:24:26.303 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T00:24:26.304 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:24:30.337 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 142/165 2024-09-21T00:24:30.391 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 143/165 2024-09-21T00:24:30.432 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 144/165 2024-09-21T00:24:30.464 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 145/165 2024-09-21T00:24:30.491 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 146/165 2024-09-21T00:24:30.708 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 147/165 2024-09-21T00:24:30.746 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 148/165 2024-09-21T00:24:30.776 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 149/165 2024-09-21T00:24:30.806 INFO:teuthology.orchestra.run.smithi187.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 150/165 2024-09-21T00:24:30.844 INFO:teuthology.orchestra.run.smithi187.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 151/165 2024-09-21T00:24:31.468 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T00:24:31.508 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T00:24:31.529 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T00:24:32.359 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 153/165 2024-09-21T00:24:32.368 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T00:24:32.369 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-2:19.3.0-5120.g18432218.el9.x86_64 1/165 2024-09-21T00:24:32.369 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2/165 2024-09-21T00:24:32.369 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 3/165 2024-09-21T00:24:32.369 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 4/165 2024-09-21T00:24:32.369 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-5120.g18432 5/165 2024-09-21T00:24:32.369 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 6/165 2024-09-21T00:24:32.369 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 7/165 2024-09-21T00:24:32.369 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T00:24:32.369 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T00:24:32.369 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T00:24:32.369 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 11/165 2024-09-21T00:24:32.369 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 12/165 2024-09-21T00:24:32.370 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 13/165 2024-09-21T00:24:32.370 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T00:24:32.370 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 15/165 2024-09-21T00:24:32.370 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 16/165 2024-09-21T00:24:32.370 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 17/165 2024-09-21T00:24:32.370 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 18/165 2024-09-21T00:24:32.370 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 19/165 2024-09-21T00:24:32.370 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 20/165 2024-09-21T00:24:32.370 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 21/165 2024-09-21T00:24:32.370 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 22/165 2024-09-21T00:24:32.370 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 23/165 2024-09-21T00:24:32.370 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 24/165 2024-09-21T00:24:32.370 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 25/165 2024-09-21T00:24:32.371 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 26/165 2024-09-21T00:24:32.371 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 27/165 2024-09-21T00:24:32.371 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 28/165 2024-09-21T00:24:32.371 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 29/165 2024-09-21T00:24:32.371 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 30/165 2024-09-21T00:24:32.371 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 31/165 2024-09-21T00:24:32.382 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 32/165 2024-09-21T00:24:32.382 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 33/165 2024-09-21T00:24:32.382 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 34/165 2024-09-21T00:24:32.383 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 35/165 2024-09-21T00:24:32.383 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 36/165 2024-09-21T00:24:32.383 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 37/165 2024-09-21T00:24:32.383 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : cephadm-2:19.3.0-5120.g18432218.el9.noarch 38/165 2024-09-21T00:24:32.383 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 39/165 2024-09-21T00:24:32.383 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/165 2024-09-21T00:24:32.383 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/165 2024-09-21T00:24:32.383 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/165 2024-09-21T00:24:32.383 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/165 2024-09-21T00:24:32.383 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/165 2024-09-21T00:24:32.383 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/165 2024-09-21T00:24:32.383 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/165 2024-09-21T00:24:32.383 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/165 2024-09-21T00:24:32.384 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/165 2024-09-21T00:24:32.384 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/165 2024-09-21T00:24:32.384 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/165 2024-09-21T00:24:32.384 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/165 2024-09-21T00:24:32.384 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/165 2024-09-21T00:24:32.384 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/165 2024-09-21T00:24:32.384 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/165 2024-09-21T00:24:32.384 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/165 2024-09-21T00:24:32.384 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/165 2024-09-21T00:24:32.384 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/165 2024-09-21T00:24:32.384 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 58/165 2024-09-21T00:24:32.384 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 59/165 2024-09-21T00:24:32.385 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 60/165 2024-09-21T00:24:32.385 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 61/165 2024-09-21T00:24:32.385 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 62/165 2024-09-21T00:24:32.385 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 63/165 2024-09-21T00:24:32.385 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 64/165 2024-09-21T00:24:32.385 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 65/165 2024-09-21T00:24:32.385 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : lua-5.4.4-4.el9.x86_64 66/165 2024-09-21T00:24:32.385 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 67/165 2024-09-21T00:24:32.385 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 68/165 2024-09-21T00:24:32.385 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 69/165 2024-09-21T00:24:32.385 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : postgresql-13.16-1.el9.x86_64 70/165 2024-09-21T00:24:32.385 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 71/165 2024-09-21T00:24:32.385 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 72/165 2024-09-21T00:24:32.386 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 73/165 2024-09-21T00:24:32.386 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 74/165 2024-09-21T00:24:32.386 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 75/165 2024-09-21T00:24:32.386 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 76/165 2024-09-21T00:24:32.386 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 77/165 2024-09-21T00:24:32.386 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 78/165 2024-09-21T00:24:32.386 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 79/165 2024-09-21T00:24:32.386 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/165 2024-09-21T00:24:32.386 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/165 2024-09-21T00:24:32.386 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/165 2024-09-21T00:24:32.386 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/165 2024-09-21T00:24:32.386 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/165 2024-09-21T00:24:32.386 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/165 2024-09-21T00:24:32.387 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/165 2024-09-21T00:24:32.387 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/165 2024-09-21T00:24:32.387 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/165 2024-09-21T00:24:32.387 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/165 2024-09-21T00:24:32.387 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/165 2024-09-21T00:24:32.387 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/165 2024-09-21T00:24:32.387 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/165 2024-09-21T00:24:32.387 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/165 2024-09-21T00:24:32.387 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/165 2024-09-21T00:24:32.387 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/165 2024-09-21T00:24:32.387 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/165 2024-09-21T00:24:32.387 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 97/165 2024-09-21T00:24:32.387 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 98/165 2024-09-21T00:24:32.387 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 99/165 2024-09-21T00:24:32.388 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 100/165 2024-09-21T00:24:32.388 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/165 2024-09-21T00:24:32.388 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 102/165 2024-09-21T00:24:32.388 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/165 2024-09-21T00:24:32.388 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 104/165 2024-09-21T00:24:32.388 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/165 2024-09-21T00:24:32.388 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/165 2024-09-21T00:24:32.388 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/165 2024-09-21T00:24:32.388 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 108/165 2024-09-21T00:24:32.388 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/165 2024-09-21T00:24:32.388 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 110/165 2024-09-21T00:24:32.388 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 111/165 2024-09-21T00:24:32.388 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/165 2024-09-21T00:24:32.388 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/165 2024-09-21T00:24:32.388 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 114/165 2024-09-21T00:24:32.389 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 115/165 2024-09-21T00:24:32.389 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/165 2024-09-21T00:24:32.389 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/165 2024-09-21T00:24:32.389 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/165 2024-09-21T00:24:32.389 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 119/165 2024-09-21T00:24:32.389 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/165 2024-09-21T00:24:32.389 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/165 2024-09-21T00:24:32.389 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/165 2024-09-21T00:24:32.389 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 123/165 2024-09-21T00:24:32.389 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 124/165 2024-09-21T00:24:32.389 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 125/165 2024-09-21T00:24:32.389 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 126/165 2024-09-21T00:24:32.389 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 127/165 2024-09-21T00:24:32.389 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 128/165 2024-09-21T00:24:32.389 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 129/165 2024-09-21T00:24:32.390 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 130/165 2024-09-21T00:24:32.390 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 131/165 2024-09-21T00:24:32.390 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 132/165 2024-09-21T00:24:32.390 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 133/165 2024-09-21T00:24:32.390 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 134/165 2024-09-21T00:24:32.390 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 135/165 2024-09-21T00:24:32.390 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 136/165 2024-09-21T00:24:32.390 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 137/165 2024-09-21T00:24:32.390 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 138/165 2024-09-21T00:24:32.390 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 139/165 2024-09-21T00:24:32.390 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 140/165 2024-09-21T00:24:32.390 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 141/165 2024-09-21T00:24:32.390 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 142/165 2024-09-21T00:24:32.390 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 143/165 2024-09-21T00:24:32.391 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 144/165 2024-09-21T00:24:32.391 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 145/165 2024-09-21T00:24:32.391 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 146/165 2024-09-21T00:24:32.391 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 147/165 2024-09-21T00:24:32.391 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : re2-1:20211101-20.el9.x86_64 148/165 2024-09-21T00:24:32.391 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 149/165 2024-09-21T00:24:32.391 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librados2-2:19.3.0-5120.g18432218.el9.x86_64 150/165 2024-09-21T00:24:32.391 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 151/165 2024-09-21T00:24:32.391 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 152/165 2024-09-21T00:24:32.391 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 153/165 2024-09-21T00:24:32.391 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 154/165 2024-09-21T00:24:32.391 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 155/165 2024-09-21T00:24:32.391 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 156/165 2024-09-21T00:24:32.392 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 157/165 2024-09-21T00:24:32.392 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 158/165 2024-09-21T00:24:32.392 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 159/165 2024-09-21T00:24:32.392 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 160/165 2024-09-21T00:24:32.392 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 161/165 2024-09-21T00:24:32.392 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 162/165 2024-09-21T00:24:32.392 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 163/165 2024-09-21T00:24:32.392 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 164/165 2024-09-21T00:24:32.438 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 154/165 2024-09-21T00:24:32.492 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 155/165 2024-09-21T00:24:32.581 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : flex-2.6.4-9.el9.x86_64 156/165 2024-09-21T00:24:32.616 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : bison-3.7.4-5.el9.x86_64 157/165 2024-09-21T00:24:32.641 INFO:teuthology.orchestra.run.smithi187.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 158/165 2024-09-21T00:24:32.675 INFO:teuthology.orchestra.run.smithi187.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 159/165 2024-09-21T00:24:32.676 INFO:teuthology.orchestra.run.smithi187.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T00:24:32.751 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T00:24:32.751 INFO:teuthology.orchestra.run.smithi187.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T00:24:32.835 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T00:24:32.860 INFO:teuthology.orchestra.run.smithi187.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 162/165 2024-09-21T00:24:32.865 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T00:24:32.866 INFO:teuthology.orchestra.run.smithi187.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T00:24:33.054 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T00:24:33.054 INFO:teuthology.orchestra.run.smithi187.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T00:24:33.121 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T00:24:33.186 INFO:teuthology.orchestra.run.smithi187.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T00:24:34.949 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T00:24:34.949 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:24:34.950 INFO:teuthology.orchestra.run.smithi112.stdout:Upgraded: 2024-09-21T00:24:34.950 INFO:teuthology.orchestra.run.smithi112.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-21T00:24:34.950 INFO:teuthology.orchestra.run.smithi112.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-21T00:24:34.950 INFO:teuthology.orchestra.run.smithi112.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-21T00:24:34.950 INFO:teuthology.orchestra.run.smithi112.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-21T00:24:34.950 INFO:teuthology.orchestra.run.smithi112.stdout: librados2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:24:34.950 INFO:teuthology.orchestra.run.smithi112.stdout: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:24:34.950 INFO:teuthology.orchestra.run.smithi112.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-21T00:24:34.950 INFO:teuthology.orchestra.run.smithi112.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-21T00:24:34.950 INFO:teuthology.orchestra.run.smithi112.stdout:Installed: 2024-09-21T00:24:34.950 INFO:teuthology.orchestra.run.smithi112.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-21T00:24:34.950 INFO:teuthology.orchestra.run.smithi112.stdout: automake-1.16.2-8.el9.noarch 2024-09-21T00:24:34.950 INFO:teuthology.orchestra.run.smithi112.stdout: bison-3.7.4-5.el9.x86_64 2024-09-21T00:24:34.951 INFO:teuthology.orchestra.run.smithi112.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-21T00:24:34.951 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:24:34.951 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:24:34.951 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:24:34.951 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:24:34.951 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-grafana-dashboards-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T00:24:34.951 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-immutable-object-cache-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:24:34.951 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:24:34.951 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:24:34.951 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T00:24:34.951 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T00:24:34.951 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-diskprediction-local-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T00:24:34.951 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T00:24:34.951 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T00:24:34.951 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:24:34.952 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:24:34.952 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-prometheus-alerts-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T00:24:34.952 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:24:34.952 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:24:34.952 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:24:34.952 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T00:24:34.952 INFO:teuthology.orchestra.run.smithi112.stdout: cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T00:24:34.952 INFO:teuthology.orchestra.run.smithi112.stdout: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:24:34.952 INFO:teuthology.orchestra.run.smithi112.stdout: cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:24:34.952 INFO:teuthology.orchestra.run.smithi112.stdout: cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T00:24:34.952 INFO:teuthology.orchestra.run.smithi112.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-21T00:24:34.952 INFO:teuthology.orchestra.run.smithi112.stdout: flex-2.6.4-9.el9.x86_64 2024-09-21T00:24:34.952 INFO:teuthology.orchestra.run.smithi112.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-21T00:24:34.953 INFO:teuthology.orchestra.run.smithi112.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-21T00:24:34.953 INFO:teuthology.orchestra.run.smithi112.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-21T00:24:34.953 INFO:teuthology.orchestra.run.smithi112.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-21T00:24:34.953 INFO:teuthology.orchestra.run.smithi112.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-21T00:24:34.953 INFO:teuthology.orchestra.run.smithi112.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-21T00:24:34.953 INFO:teuthology.orchestra.run.smithi112.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-21T00:24:34.953 INFO:teuthology.orchestra.run.smithi112.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-21T00:24:34.953 INFO:teuthology.orchestra.run.smithi112.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-21T00:24:34.953 INFO:teuthology.orchestra.run.smithi112.stdout: libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:24:34.953 INFO:teuthology.orchestra.run.smithi112.stdout: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:24:34.953 INFO:teuthology.orchestra.run.smithi112.stdout: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:24:34.953 INFO:teuthology.orchestra.run.smithi112.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-21T00:24:34.953 INFO:teuthology.orchestra.run.smithi112.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-21T00:24:34.953 INFO:teuthology.orchestra.run.smithi112.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-21T00:24:34.954 INFO:teuthology.orchestra.run.smithi112.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-21T00:24:34.954 INFO:teuthology.orchestra.run.smithi112.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-21T00:24:34.954 INFO:teuthology.orchestra.run.smithi112.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-21T00:24:34.954 INFO:teuthology.orchestra.run.smithi112.stdout: librados-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:24:34.954 INFO:teuthology.orchestra.run.smithi112.stdout: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:24:34.954 INFO:teuthology.orchestra.run.smithi112.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-21T00:24:34.954 INFO:teuthology.orchestra.run.smithi112.stdout: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:24:34.954 INFO:teuthology.orchestra.run.smithi112.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-21T00:24:34.954 INFO:teuthology.orchestra.run.smithi112.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-21T00:24:34.954 INFO:teuthology.orchestra.run.smithi112.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-21T00:24:34.954 INFO:teuthology.orchestra.run.smithi112.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-21T00:24:34.954 INFO:teuthology.orchestra.run.smithi112.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-21T00:24:34.954 INFO:teuthology.orchestra.run.smithi112.stdout: lua-5.4.4-4.el9.x86_64 2024-09-21T00:24:34.955 INFO:teuthology.orchestra.run.smithi112.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-21T00:24:34.955 INFO:teuthology.orchestra.run.smithi112.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-21T00:24:34.955 INFO:teuthology.orchestra.run.smithi112.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-21T00:24:34.955 INFO:teuthology.orchestra.run.smithi112.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-21T00:24:34.955 INFO:teuthology.orchestra.run.smithi112.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-21T00:24:34.955 INFO:teuthology.orchestra.run.smithi112.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-21T00:24:34.955 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-13.16-1.el9.x86_64 2024-09-21T00:24:34.955 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-21T00:24:34.955 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-21T00:24:34.955 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-21T00:24:34.955 INFO:teuthology.orchestra.run.smithi112.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-21T00:24:34.955 INFO:teuthology.orchestra.run.smithi112.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-21T00:24:34.955 INFO:teuthology.orchestra.run.smithi112.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-21T00:24:34.955 INFO:teuthology.orchestra.run.smithi112.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-21T00:24:34.955 INFO:teuthology.orchestra.run.smithi112.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-21T00:24:34.955 INFO:teuthology.orchestra.run.smithi112.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-21T00:24:34.956 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-21T00:24:34.956 INFO:teuthology.orchestra.run.smithi112.stdout: python3-ceph-argparse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:24:34.956 INFO:teuthology.orchestra.run.smithi112.stdout: python3-ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:24:34.956 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:24:34.956 INFO:teuthology.orchestra.run.smithi112.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-21T00:24:34.956 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-21T00:24:34.956 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-21T00:24:34.956 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-21T00:24:34.956 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-21T00:24:34.956 INFO:teuthology.orchestra.run.smithi112.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-21T00:24:34.956 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-21T00:24:34.956 INFO:teuthology.orchestra.run.smithi112.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-21T00:24:34.956 INFO:teuthology.orchestra.run.smithi112.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-21T00:24:34.956 INFO:teuthology.orchestra.run.smithi112.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-21T00:24:34.956 INFO:teuthology.orchestra.run.smithi112.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-21T00:24:34.957 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-21T00:24:34.957 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-21T00:24:34.957 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-21T00:24:34.957 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-21T00:24:34.957 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-21T00:24:34.957 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-21T00:24:34.957 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-21T00:24:34.957 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-21T00:24:34.957 INFO:teuthology.orchestra.run.smithi112.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-21T00:24:34.957 INFO:teuthology.orchestra.run.smithi112.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-21T00:24:34.957 INFO:teuthology.orchestra.run.smithi112.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-21T00:24:34.957 INFO:teuthology.orchestra.run.smithi112.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-21T00:24:34.957 INFO:teuthology.orchestra.run.smithi112.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-21T00:24:34.957 INFO:teuthology.orchestra.run.smithi112.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-21T00:24:34.957 INFO:teuthology.orchestra.run.smithi112.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-21T00:24:34.958 INFO:teuthology.orchestra.run.smithi112.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-21T00:24:34.958 INFO:teuthology.orchestra.run.smithi112.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-21T00:24:34.958 INFO:teuthology.orchestra.run.smithi112.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-21T00:24:34.958 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-21T00:24:34.958 INFO:teuthology.orchestra.run.smithi112.stdout: python3-ply-3.11-14.el9.noarch 2024-09-21T00:24:34.958 INFO:teuthology.orchestra.run.smithi112.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-21T00:24:34.958 INFO:teuthology.orchestra.run.smithi112.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-21T00:24:34.958 INFO:teuthology.orchestra.run.smithi112.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-21T00:24:34.958 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-21T00:24:34.958 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-21T00:24:34.958 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-21T00:24:34.958 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-21T00:24:34.958 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-21T00:24:34.958 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-21T00:24:34.959 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:24:34.959 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:24:34.959 INFO:teuthology.orchestra.run.smithi112.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-21T00:24:34.959 INFO:teuthology.orchestra.run.smithi112.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-21T00:24:34.959 INFO:teuthology.orchestra.run.smithi112.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-21T00:24:34.959 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:24:34.959 INFO:teuthology.orchestra.run.smithi112.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-21T00:24:34.959 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-21T00:24:34.959 INFO:teuthology.orchestra.run.smithi112.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-21T00:24:34.959 INFO:teuthology.orchestra.run.smithi112.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-21T00:24:34.959 INFO:teuthology.orchestra.run.smithi112.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-21T00:24:34.959 INFO:teuthology.orchestra.run.smithi112.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-21T00:24:34.959 INFO:teuthology.orchestra.run.smithi112.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-21T00:24:34.959 INFO:teuthology.orchestra.run.smithi112.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-21T00:24:34.959 INFO:teuthology.orchestra.run.smithi112.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-21T00:24:34.959 INFO:teuthology.orchestra.run.smithi112.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-21T00:24:34.960 INFO:teuthology.orchestra.run.smithi112.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-21T00:24:34.960 INFO:teuthology.orchestra.run.smithi112.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-21T00:24:34.960 INFO:teuthology.orchestra.run.smithi112.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-21T00:24:34.960 INFO:teuthology.orchestra.run.smithi112.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-21T00:24:34.960 INFO:teuthology.orchestra.run.smithi112.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-21T00:24:34.960 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:24:34.960 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:24:34.960 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:24:34.960 INFO:teuthology.orchestra.run.smithi112.stdout: re2-1:20211101-20.el9.x86_64 2024-09-21T00:24:34.960 INFO:teuthology.orchestra.run.smithi112.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-21T00:24:34.960 INFO:teuthology.orchestra.run.smithi112.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-21T00:24:34.960 INFO:teuthology.orchestra.run.smithi112.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-21T00:24:34.960 INFO:teuthology.orchestra.run.smithi112.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-21T00:24:34.960 INFO:teuthology.orchestra.run.smithi112.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-21T00:24:34.961 INFO:teuthology.orchestra.run.smithi112.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-21T00:24:34.961 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:24:34.961 INFO:teuthology.orchestra.run.smithi112.stdout:Complete! 2024-09-21T00:24:35.365 DEBUG:teuthology.parallel:result is None 2024-09-21T00:24:39.731 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T00:24:39.731 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-2:19.3.0-5120.g18432218.el9.x86_64 1/165 2024-09-21T00:24:39.731 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2/165 2024-09-21T00:24:39.732 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 3/165 2024-09-21T00:24:39.732 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 4/165 2024-09-21T00:24:39.732 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-5120.g18432 5/165 2024-09-21T00:24:39.732 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 6/165 2024-09-21T00:24:39.732 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 7/165 2024-09-21T00:24:39.732 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T00:24:39.732 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T00:24:39.732 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T00:24:39.732 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 11/165 2024-09-21T00:24:39.732 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 12/165 2024-09-21T00:24:39.732 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 13/165 2024-09-21T00:24:39.732 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T00:24:39.732 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 15/165 2024-09-21T00:24:39.732 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 16/165 2024-09-21T00:24:39.733 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 17/165 2024-09-21T00:24:39.733 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 18/165 2024-09-21T00:24:39.733 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 19/165 2024-09-21T00:24:39.733 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 20/165 2024-09-21T00:24:39.733 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 21/165 2024-09-21T00:24:39.733 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 22/165 2024-09-21T00:24:39.733 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 23/165 2024-09-21T00:24:39.733 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 24/165 2024-09-21T00:24:39.733 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 25/165 2024-09-21T00:24:39.733 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 26/165 2024-09-21T00:24:39.733 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 27/165 2024-09-21T00:24:39.733 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 28/165 2024-09-21T00:24:39.733 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 29/165 2024-09-21T00:24:39.733 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 30/165 2024-09-21T00:24:39.734 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 31/165 2024-09-21T00:24:39.734 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 32/165 2024-09-21T00:24:39.734 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 33/165 2024-09-21T00:24:39.734 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 34/165 2024-09-21T00:24:39.734 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 35/165 2024-09-21T00:24:39.734 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 36/165 2024-09-21T00:24:39.734 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 37/165 2024-09-21T00:24:39.734 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : cephadm-2:19.3.0-5120.g18432218.el9.noarch 38/165 2024-09-21T00:24:39.734 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 39/165 2024-09-21T00:24:39.734 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/165 2024-09-21T00:24:39.734 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/165 2024-09-21T00:24:39.734 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/165 2024-09-21T00:24:39.734 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/165 2024-09-21T00:24:39.735 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/165 2024-09-21T00:24:39.735 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/165 2024-09-21T00:24:39.735 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/165 2024-09-21T00:24:39.735 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/165 2024-09-21T00:24:39.735 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/165 2024-09-21T00:24:39.735 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/165 2024-09-21T00:24:39.735 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/165 2024-09-21T00:24:39.735 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/165 2024-09-21T00:24:39.735 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/165 2024-09-21T00:24:39.735 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/165 2024-09-21T00:24:39.735 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/165 2024-09-21T00:24:39.735 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/165 2024-09-21T00:24:39.735 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/165 2024-09-21T00:24:39.735 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/165 2024-09-21T00:24:39.735 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 58/165 2024-09-21T00:24:39.735 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 59/165 2024-09-21T00:24:39.736 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 60/165 2024-09-21T00:24:39.736 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 61/165 2024-09-21T00:24:39.736 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 62/165 2024-09-21T00:24:39.736 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 63/165 2024-09-21T00:24:39.736 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 64/165 2024-09-21T00:24:39.736 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 65/165 2024-09-21T00:24:39.736 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : lua-5.4.4-4.el9.x86_64 66/165 2024-09-21T00:24:39.736 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 67/165 2024-09-21T00:24:39.736 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 68/165 2024-09-21T00:24:39.736 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 69/165 2024-09-21T00:24:39.736 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : postgresql-13.16-1.el9.x86_64 70/165 2024-09-21T00:24:39.736 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 71/165 2024-09-21T00:24:39.736 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 72/165 2024-09-21T00:24:39.736 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 73/165 2024-09-21T00:24:39.736 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 74/165 2024-09-21T00:24:39.737 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 75/165 2024-09-21T00:24:39.737 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 76/165 2024-09-21T00:24:39.737 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 77/165 2024-09-21T00:24:39.737 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 78/165 2024-09-21T00:24:39.737 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 79/165 2024-09-21T00:24:39.737 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/165 2024-09-21T00:24:39.737 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/165 2024-09-21T00:24:39.737 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/165 2024-09-21T00:24:39.737 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/165 2024-09-21T00:24:39.737 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/165 2024-09-21T00:24:39.737 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/165 2024-09-21T00:24:39.737 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/165 2024-09-21T00:24:39.737 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/165 2024-09-21T00:24:39.738 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/165 2024-09-21T00:24:39.738 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/165 2024-09-21T00:24:39.738 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/165 2024-09-21T00:24:39.738 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/165 2024-09-21T00:24:39.738 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/165 2024-09-21T00:24:39.738 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/165 2024-09-21T00:24:39.738 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/165 2024-09-21T00:24:39.738 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/165 2024-09-21T00:24:39.738 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/165 2024-09-21T00:24:39.738 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 97/165 2024-09-21T00:24:39.738 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 98/165 2024-09-21T00:24:39.738 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 99/165 2024-09-21T00:24:39.738 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 100/165 2024-09-21T00:24:39.738 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/165 2024-09-21T00:24:39.739 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 102/165 2024-09-21T00:24:39.739 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/165 2024-09-21T00:24:39.739 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 104/165 2024-09-21T00:24:39.739 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/165 2024-09-21T00:24:39.739 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/165 2024-09-21T00:24:39.739 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/165 2024-09-21T00:24:39.739 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 108/165 2024-09-21T00:24:39.739 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/165 2024-09-21T00:24:39.739 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 110/165 2024-09-21T00:24:39.739 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 111/165 2024-09-21T00:24:39.739 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/165 2024-09-21T00:24:39.739 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/165 2024-09-21T00:24:39.739 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 114/165 2024-09-21T00:24:39.739 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 115/165 2024-09-21T00:24:39.740 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/165 2024-09-21T00:24:39.740 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/165 2024-09-21T00:24:39.740 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/165 2024-09-21T00:24:39.740 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 119/165 2024-09-21T00:24:39.740 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/165 2024-09-21T00:24:39.740 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/165 2024-09-21T00:24:39.740 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/165 2024-09-21T00:24:39.740 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 123/165 2024-09-21T00:24:39.740 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 124/165 2024-09-21T00:24:39.740 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 125/165 2024-09-21T00:24:39.740 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 126/165 2024-09-21T00:24:39.740 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 127/165 2024-09-21T00:24:39.740 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 128/165 2024-09-21T00:24:39.740 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 129/165 2024-09-21T00:24:39.741 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 130/165 2024-09-21T00:24:39.741 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 131/165 2024-09-21T00:24:39.741 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 132/165 2024-09-21T00:24:39.741 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 133/165 2024-09-21T00:24:39.741 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 134/165 2024-09-21T00:24:39.741 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 135/165 2024-09-21T00:24:39.741 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 136/165 2024-09-21T00:24:39.741 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 137/165 2024-09-21T00:24:39.741 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 138/165 2024-09-21T00:24:39.741 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 139/165 2024-09-21T00:24:39.741 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 140/165 2024-09-21T00:24:39.741 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 141/165 2024-09-21T00:24:39.741 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 142/165 2024-09-21T00:24:39.741 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 143/165 2024-09-21T00:24:39.742 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 144/165 2024-09-21T00:24:39.742 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 145/165 2024-09-21T00:24:39.742 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 146/165 2024-09-21T00:24:39.742 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 147/165 2024-09-21T00:24:39.742 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : re2-1:20211101-20.el9.x86_64 148/165 2024-09-21T00:24:39.742 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 149/165 2024-09-21T00:24:39.742 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : librados2-2:19.3.0-5120.g18432218.el9.x86_64 150/165 2024-09-21T00:24:39.742 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 151/165 2024-09-21T00:24:39.742 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 152/165 2024-09-21T00:24:39.742 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 153/165 2024-09-21T00:24:39.742 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 154/165 2024-09-21T00:24:39.742 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 155/165 2024-09-21T00:24:39.742 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 156/165 2024-09-21T00:24:39.742 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 157/165 2024-09-21T00:24:39.743 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 158/165 2024-09-21T00:24:39.743 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 159/165 2024-09-21T00:24:39.743 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 160/165 2024-09-21T00:24:39.743 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 161/165 2024-09-21T00:24:39.743 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 162/165 2024-09-21T00:24:39.743 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 163/165 2024-09-21T00:24:39.743 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 164/165 2024-09-21T00:24:42.451 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T00:24:42.451 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:24:42.451 INFO:teuthology.orchestra.run.smithi187.stdout:Upgraded: 2024-09-21T00:24:42.451 INFO:teuthology.orchestra.run.smithi187.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-21T00:24:42.451 INFO:teuthology.orchestra.run.smithi187.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-21T00:24:42.451 INFO:teuthology.orchestra.run.smithi187.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-21T00:24:42.452 INFO:teuthology.orchestra.run.smithi187.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-21T00:24:42.452 INFO:teuthology.orchestra.run.smithi187.stdout: librados2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:24:42.452 INFO:teuthology.orchestra.run.smithi187.stdout: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:24:42.452 INFO:teuthology.orchestra.run.smithi187.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-21T00:24:42.452 INFO:teuthology.orchestra.run.smithi187.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-21T00:24:42.452 INFO:teuthology.orchestra.run.smithi187.stdout:Installed: 2024-09-21T00:24:42.452 INFO:teuthology.orchestra.run.smithi187.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-21T00:24:42.452 INFO:teuthology.orchestra.run.smithi187.stdout: automake-1.16.2-8.el9.noarch 2024-09-21T00:24:42.452 INFO:teuthology.orchestra.run.smithi187.stdout: bison-3.7.4-5.el9.x86_64 2024-09-21T00:24:42.452 INFO:teuthology.orchestra.run.smithi187.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-21T00:24:42.452 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:24:42.452 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:24:42.453 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:24:42.453 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:24:42.453 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-grafana-dashboards-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T00:24:42.453 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-immutable-object-cache-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:24:42.453 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:24:42.453 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:24:42.453 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T00:24:42.453 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T00:24:42.453 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-diskprediction-local-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T00:24:42.453 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T00:24:42.453 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T00:24:42.454 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:24:42.454 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:24:42.454 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-prometheus-alerts-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T00:24:42.454 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:24:42.454 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:24:42.454 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:24:42.454 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T00:24:42.454 INFO:teuthology.orchestra.run.smithi187.stdout: cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T00:24:42.454 INFO:teuthology.orchestra.run.smithi187.stdout: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:24:42.454 INFO:teuthology.orchestra.run.smithi187.stdout: cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:24:42.454 INFO:teuthology.orchestra.run.smithi187.stdout: cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T00:24:42.455 INFO:teuthology.orchestra.run.smithi187.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-21T00:24:42.455 INFO:teuthology.orchestra.run.smithi187.stdout: flex-2.6.4-9.el9.x86_64 2024-09-21T00:24:42.455 INFO:teuthology.orchestra.run.smithi187.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-21T00:24:42.455 INFO:teuthology.orchestra.run.smithi187.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-21T00:24:42.455 INFO:teuthology.orchestra.run.smithi187.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-21T00:24:42.455 INFO:teuthology.orchestra.run.smithi187.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-21T00:24:42.455 INFO:teuthology.orchestra.run.smithi187.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-21T00:24:42.455 INFO:teuthology.orchestra.run.smithi187.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-21T00:24:42.455 INFO:teuthology.orchestra.run.smithi187.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-21T00:24:42.455 INFO:teuthology.orchestra.run.smithi187.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-21T00:24:42.455 INFO:teuthology.orchestra.run.smithi187.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-21T00:24:42.455 INFO:teuthology.orchestra.run.smithi187.stdout: libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:24:42.456 INFO:teuthology.orchestra.run.smithi187.stdout: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:24:42.456 INFO:teuthology.orchestra.run.smithi187.stdout: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:24:42.456 INFO:teuthology.orchestra.run.smithi187.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-21T00:24:42.456 INFO:teuthology.orchestra.run.smithi187.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-21T00:24:42.456 INFO:teuthology.orchestra.run.smithi187.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-21T00:24:42.456 INFO:teuthology.orchestra.run.smithi187.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-21T00:24:42.456 INFO:teuthology.orchestra.run.smithi187.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-21T00:24:42.456 INFO:teuthology.orchestra.run.smithi187.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-21T00:24:42.456 INFO:teuthology.orchestra.run.smithi187.stdout: librados-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:24:42.456 INFO:teuthology.orchestra.run.smithi187.stdout: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:24:42.456 INFO:teuthology.orchestra.run.smithi187.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-21T00:24:42.456 INFO:teuthology.orchestra.run.smithi187.stdout: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:24:42.456 INFO:teuthology.orchestra.run.smithi187.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-21T00:24:42.457 INFO:teuthology.orchestra.run.smithi187.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-21T00:24:42.457 INFO:teuthology.orchestra.run.smithi187.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-21T00:24:42.457 INFO:teuthology.orchestra.run.smithi187.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-21T00:24:42.457 INFO:teuthology.orchestra.run.smithi187.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-21T00:24:42.457 INFO:teuthology.orchestra.run.smithi187.stdout: lua-5.4.4-4.el9.x86_64 2024-09-21T00:24:42.457 INFO:teuthology.orchestra.run.smithi187.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-21T00:24:42.457 INFO:teuthology.orchestra.run.smithi187.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-21T00:24:42.457 INFO:teuthology.orchestra.run.smithi187.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-21T00:24:42.457 INFO:teuthology.orchestra.run.smithi187.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-21T00:24:42.457 INFO:teuthology.orchestra.run.smithi187.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-21T00:24:42.457 INFO:teuthology.orchestra.run.smithi187.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-21T00:24:42.457 INFO:teuthology.orchestra.run.smithi187.stdout: postgresql-13.16-1.el9.x86_64 2024-09-21T00:24:42.457 INFO:teuthology.orchestra.run.smithi187.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-21T00:24:42.458 INFO:teuthology.orchestra.run.smithi187.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-21T00:24:42.458 INFO:teuthology.orchestra.run.smithi187.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-21T00:24:42.458 INFO:teuthology.orchestra.run.smithi187.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-21T00:24:42.458 INFO:teuthology.orchestra.run.smithi187.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-21T00:24:42.458 INFO:teuthology.orchestra.run.smithi187.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-21T00:24:42.458 INFO:teuthology.orchestra.run.smithi187.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-21T00:24:42.458 INFO:teuthology.orchestra.run.smithi187.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-21T00:24:42.458 INFO:teuthology.orchestra.run.smithi187.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-21T00:24:42.458 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-21T00:24:42.458 INFO:teuthology.orchestra.run.smithi187.stdout: python3-ceph-argparse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:24:42.458 INFO:teuthology.orchestra.run.smithi187.stdout: python3-ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:24:42.458 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:24:42.459 INFO:teuthology.orchestra.run.smithi187.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-21T00:24:42.459 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-21T00:24:42.459 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-21T00:24:42.459 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-21T00:24:42.459 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-21T00:24:42.459 INFO:teuthology.orchestra.run.smithi187.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-21T00:24:42.459 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-21T00:24:42.459 INFO:teuthology.orchestra.run.smithi187.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-21T00:24:42.459 INFO:teuthology.orchestra.run.smithi187.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-21T00:24:42.459 INFO:teuthology.orchestra.run.smithi187.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-21T00:24:42.459 INFO:teuthology.orchestra.run.smithi187.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-21T00:24:42.459 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-21T00:24:42.460 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-21T00:24:42.460 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-21T00:24:42.460 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-21T00:24:42.460 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-21T00:24:42.460 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-21T00:24:42.460 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-21T00:24:42.460 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-21T00:24:42.460 INFO:teuthology.orchestra.run.smithi187.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-21T00:24:42.460 INFO:teuthology.orchestra.run.smithi187.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-21T00:24:42.460 INFO:teuthology.orchestra.run.smithi187.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-21T00:24:42.460 INFO:teuthology.orchestra.run.smithi187.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-21T00:24:42.460 INFO:teuthology.orchestra.run.smithi187.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-21T00:24:42.460 INFO:teuthology.orchestra.run.smithi187.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-21T00:24:42.461 INFO:teuthology.orchestra.run.smithi187.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-21T00:24:42.461 INFO:teuthology.orchestra.run.smithi187.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-21T00:24:42.461 INFO:teuthology.orchestra.run.smithi187.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-21T00:24:42.461 INFO:teuthology.orchestra.run.smithi187.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-21T00:24:42.461 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-21T00:24:42.461 INFO:teuthology.orchestra.run.smithi187.stdout: python3-ply-3.11-14.el9.noarch 2024-09-21T00:24:42.461 INFO:teuthology.orchestra.run.smithi187.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-21T00:24:42.461 INFO:teuthology.orchestra.run.smithi187.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-21T00:24:42.461 INFO:teuthology.orchestra.run.smithi187.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-21T00:24:42.461 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-21T00:24:42.461 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-21T00:24:42.461 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-21T00:24:42.462 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-21T00:24:42.462 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-21T00:24:42.462 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-21T00:24:42.462 INFO:teuthology.orchestra.run.smithi187.stdout: python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:24:42.462 INFO:teuthology.orchestra.run.smithi187.stdout: python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:24:42.462 INFO:teuthology.orchestra.run.smithi187.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-21T00:24:42.462 INFO:teuthology.orchestra.run.smithi187.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-21T00:24:42.462 INFO:teuthology.orchestra.run.smithi187.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-21T00:24:42.462 INFO:teuthology.orchestra.run.smithi187.stdout: python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:24:42.462 INFO:teuthology.orchestra.run.smithi187.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-21T00:24:42.462 INFO:teuthology.orchestra.run.smithi187.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-21T00:24:42.463 INFO:teuthology.orchestra.run.smithi187.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-21T00:24:42.463 INFO:teuthology.orchestra.run.smithi187.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-21T00:24:42.463 INFO:teuthology.orchestra.run.smithi187.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-21T00:24:42.463 INFO:teuthology.orchestra.run.smithi187.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-21T00:24:42.463 INFO:teuthology.orchestra.run.smithi187.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-21T00:24:42.463 INFO:teuthology.orchestra.run.smithi187.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-21T00:24:42.463 INFO:teuthology.orchestra.run.smithi187.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-21T00:24:42.463 INFO:teuthology.orchestra.run.smithi187.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-21T00:24:42.463 INFO:teuthology.orchestra.run.smithi187.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-21T00:24:42.463 INFO:teuthology.orchestra.run.smithi187.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-21T00:24:42.463 INFO:teuthology.orchestra.run.smithi187.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-21T00:24:42.464 INFO:teuthology.orchestra.run.smithi187.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-21T00:24:42.464 INFO:teuthology.orchestra.run.smithi187.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-21T00:24:42.464 INFO:teuthology.orchestra.run.smithi187.stdout: rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:24:42.464 INFO:teuthology.orchestra.run.smithi187.stdout: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:24:42.464 INFO:teuthology.orchestra.run.smithi187.stdout: rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T00:24:42.464 INFO:teuthology.orchestra.run.smithi187.stdout: re2-1:20211101-20.el9.x86_64 2024-09-21T00:24:42.464 INFO:teuthology.orchestra.run.smithi187.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-21T00:24:42.464 INFO:teuthology.orchestra.run.smithi187.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-21T00:24:42.464 INFO:teuthology.orchestra.run.smithi187.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-21T00:24:42.464 INFO:teuthology.orchestra.run.smithi187.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-21T00:24:42.464 INFO:teuthology.orchestra.run.smithi187.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-21T00:24:42.465 INFO:teuthology.orchestra.run.smithi187.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-21T00:24:42.465 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:24:42.465 INFO:teuthology.orchestra.run.smithi187.stdout:Complete! 2024-09-21T00:24:42.875 DEBUG:teuthology.parallel:result is None 2024-09-21T00:24:42.876 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T00:24:43.026 DEBUG:teuthology.orchestra.run.smithi112:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-21T00:24:43.053 INFO:teuthology.orchestra.run.smithi112.stdout:19.3.0-5120.g18432218.el9 2024-09-21T00:24:43.053 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-5120.g18432218.el9 2024-09-21T00:24:43.054 INFO:teuthology.task.install:The correct ceph version 19.3.0-5120.g18432218 is installed. 2024-09-21T00:24:43.055 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T00:24:43.203 DEBUG:teuthology.orchestra.run.smithi187:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-21T00:24:43.225 INFO:teuthology.orchestra.run.smithi187.stdout:19.3.0-5120.g18432218.el9 2024-09-21T00:24:43.225 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-5120.g18432218.el9 2024-09-21T00:24:43.225 INFO:teuthology.task.install:The correct ceph version 19.3.0-5120.g18432218 is installed. 2024-09-21T00:24:43.227 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-21T00:24:43.227 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T00:24:43.227 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-21T00:24:43.260 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-09-21T00:24:43.260 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-21T00:24:43.292 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-21T00:24:43.293 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T00:24:43.293 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/usr/bin/daemon-helper 2024-09-21T00:24:43.329 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-21T00:24:43.396 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-09-21T00:24:43.396 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd of=/usr/bin/daemon-helper 2024-09-21T00:24:43.421 DEBUG:teuthology.orchestra.run.smithi187:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-21T00:24:43.487 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-21T00:24:43.487 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T00:24:43.487 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-21T00:24:43.513 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-21T00:24:43.579 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-09-21T00:24:43.579 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-21T00:24:43.604 DEBUG:teuthology.orchestra.run.smithi187:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-21T00:24:43.668 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-21T00:24:43.668 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T00:24:43.669 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/usr/bin/stdin-killer 2024-09-21T00:24:43.694 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-21T00:24:43.760 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-09-21T00:24:43.760 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd of=/usr/bin/stdin-killer 2024-09-21T00:24:43.784 DEBUG:teuthology.orchestra.run.smithi187:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-21T00:24:43.847 INFO:teuthology.run_tasks:Running task ceph... 2024-09-21T00:24:43.924 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-21T00:24:43.924 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod 777 /var/log/ceph 2024-09-21T00:24:43.927 DEBUG:teuthology.orchestra.run.smithi187:> sudo chmod 777 /var/log/ceph 2024-09-21T00:24:43.951 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-21T00:24:43.951 DEBUG:teuthology.orchestra.run.smithi112:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-21T00:24:43.992 DEBUG:teuthology.orchestra.run.smithi187:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-21T00:24:44.016 INFO:tasks.ceph:Creating extra log directories... 2024-09-21T00:24:44.016 DEBUG:teuthology.orchestra.run.smithi112:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-21T00:24:44.057 DEBUG:teuthology.orchestra.run.smithi187:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-21T00:24:44.087 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-09-21T00:24:44.087 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-09-21T00:24:44.087 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-09-21T00:24:44.087 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T00:24:44.087 DEBUG:teuthology.orchestra.run.smithi112:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-09-21T00:24:44.135 DEBUG:teuthology.orchestra.run.smithi112:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-09-21T00:24:44.245 DEBUG:teuthology.orchestra.run.smithi112:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-09-21T00:24:44.271 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-09-21T00:24:44.271 DEBUG:teuthology.orchestra.run.smithi187:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-09-21T00:24:44.287 DEBUG:teuthology.orchestra.run.smithi187:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-09-21T00:24:44.393 DEBUG:teuthology.orchestra.run.smithi187:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-09-21T00:24:44.423 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-21T00:24:44.423 INFO:tasks.ceph:config {'conf': {'client': {'pid file': ''}, 'global': {'ms die on skipped message': False}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'fs.*is degraded', 'filesystem is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'filesystem is offline', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'client session with non-allowable root'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-21T00:24:44.423 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913327', 'branch': 'wip-pdonnell-testing-20240920.212106-debug', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/sessionmap}', 'email': 'pdonnell@redhat.com', 'first_in_suite': False, 'job_id': '7913327', 'kernel': {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-pdonnell-testing-20240920.212106-debug'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 2, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'client mount timeout': 30, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1, 'mon warn on pool no app': False}, 'mon': {'mon op complaint time': 120}, 'osd': {'osd op complaint time': 180}}, 'conf': {'client': {'pid file': ''}, 'global': {'ms die on skipped message': False}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'fs.*is degraded', 'filesystem is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'filesystem is offline', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'client session with non-allowable root'], 'sha1': '1843221895adccfb3b692060251dd6a0183bc410'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410'}}, 'kclient': {'syntax': 'v1'}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-pdonnell-testing-20240920.212106-debug', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410'}}, 'owner': 'scheduled_pdonnell@teuthology', 'priority': 90, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 7864, 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'sleep_before_teardown': 0, 'subset': '67/512', 'suite': 'fs', 'suite_branch': 'wip-pdonnell-testing-20240920.212106-debug', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi112.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNzz9m/KbL0HE2cYqytRkGlqSbiyfkuxmJ+Op4aCHHqroao9pIx9+IirS4Nprb5VnEG6c8xr5OZ321BMAQYVIkU=', 'smithi187.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKkHhPgTrEljf4AIYwRQx8iIF/4DMFO1KxevBC8kgwgIugNhsMVxP7c3BiWnPt6XvmvCuzkxgC3HLph9+odyhGs='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 2, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'client mount timeout': 30, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1, 'mon warn on pool no app': False}, 'mon': {'mon op complaint time': 120}, 'osd': {'osd op complaint time': 180}}, 'conf': {'client': {'pid file': ''}, 'global': {'ms die on skipped message': False}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'fs.*is degraded', 'filesystem is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'filesystem is offline', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'client session with non-allowable root'], 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'cluster': 'ceph'}}, {'kclient': None}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.test_sessionmap']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-20_23:48:22', 'tube': 'smithi', 'user': 'pdonnell', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2902'} 2024-09-21T00:24:44.424 DEBUG:teuthology.orchestra.run.smithi112:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-21T00:24:44.426 DEBUG:teuthology.orchestra.run.smithi187:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-21T00:24:44.473 DEBUG:teuthology.orchestra.run.smithi112:> sudo install -d -m0777 -- /var/run/ceph 2024-09-21T00:24:44.481 DEBUG:teuthology.orchestra.run.smithi187:> sudo install -d -m0777 -- /var/run/ceph 2024-09-21T00:24:44.539 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T00:24:44.539 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/scratch_devs of=/dev/stdout 2024-09-21T00:24:44.560 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-21T00:24:44.560 DEBUG:teuthology.orchestra.run.smithi112:> stat /dev/vg_nvme/lv_1 2024-09-21T00:24:44.615 INFO:teuthology.orchestra.run.smithi112.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-21T00:24:44.615 INFO:teuthology.orchestra.run.smithi112.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T00:24:44.616 INFO:teuthology.orchestra.run.smithi112.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-09-21T00:24:44.616 INFO:teuthology.orchestra.run.smithi112.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T00:24:44.616 INFO:teuthology.orchestra.run.smithi112.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T00:24:44.616 INFO:teuthology.orchestra.run.smithi112.stdout:Access: 2024-09-21 00:24:26.627736162 +0000 2024-09-21T00:24:44.616 INFO:teuthology.orchestra.run.smithi112.stdout:Modify: 2024-09-21 00:22:43.051339483 +0000 2024-09-21T00:24:44.616 INFO:teuthology.orchestra.run.smithi112.stdout:Change: 2024-09-21 00:22:43.051339483 +0000 2024-09-21T00:24:44.616 INFO:teuthology.orchestra.run.smithi112.stdout: Birth: 2024-09-21 00:22:43.051339483 +0000 2024-09-21T00:24:44.616 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-21T00:24:44.678 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records in 2024-09-21T00:24:44.679 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records out 2024-09-21T00:24:44.679 INFO:teuthology.orchestra.run.smithi112.stderr:512 bytes copied, 0.00021456 s, 2.4 MB/s 2024-09-21T00:24:44.680 DEBUG:teuthology.orchestra.run.smithi112:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-21T00:24:44.735 DEBUG:teuthology.orchestra.run.smithi112:> stat /dev/vg_nvme/lv_2 2024-09-21T00:24:44.790 INFO:teuthology.orchestra.run.smithi112.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-21T00:24:44.790 INFO:teuthology.orchestra.run.smithi112.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T00:24:44.790 INFO:teuthology.orchestra.run.smithi112.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-09-21T00:24:44.791 INFO:teuthology.orchestra.run.smithi112.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T00:24:44.791 INFO:teuthology.orchestra.run.smithi112.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T00:24:44.791 INFO:teuthology.orchestra.run.smithi112.stdout:Access: 2024-09-21 00:24:26.628736138 +0000 2024-09-21T00:24:44.791 INFO:teuthology.orchestra.run.smithi112.stdout:Modify: 2024-09-21 00:22:43.381331529 +0000 2024-09-21T00:24:44.791 INFO:teuthology.orchestra.run.smithi112.stdout:Change: 2024-09-21 00:22:43.381331529 +0000 2024-09-21T00:24:44.791 INFO:teuthology.orchestra.run.smithi112.stdout: Birth: 2024-09-21 00:22:43.381331529 +0000 2024-09-21T00:24:44.791 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-21T00:24:44.853 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records in 2024-09-21T00:24:44.853 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records out 2024-09-21T00:24:44.853 INFO:teuthology.orchestra.run.smithi112.stderr:512 bytes copied, 0.000234619 s, 2.2 MB/s 2024-09-21T00:24:44.854 DEBUG:teuthology.orchestra.run.smithi112:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-21T00:24:44.910 DEBUG:teuthology.orchestra.run.smithi112:> stat /dev/vg_nvme/lv_3 2024-09-21T00:24:44.965 INFO:teuthology.orchestra.run.smithi112.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-21T00:24:44.966 INFO:teuthology.orchestra.run.smithi112.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T00:24:44.966 INFO:teuthology.orchestra.run.smithi112.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-09-21T00:24:44.966 INFO:teuthology.orchestra.run.smithi112.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T00:24:44.966 INFO:teuthology.orchestra.run.smithi112.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T00:24:44.966 INFO:teuthology.orchestra.run.smithi112.stdout:Access: 2024-09-21 00:24:26.628736138 +0000 2024-09-21T00:24:44.966 INFO:teuthology.orchestra.run.smithi112.stdout:Modify: 2024-09-21 00:22:43.702323791 +0000 2024-09-21T00:24:44.966 INFO:teuthology.orchestra.run.smithi112.stdout:Change: 2024-09-21 00:22:43.702323791 +0000 2024-09-21T00:24:44.966 INFO:teuthology.orchestra.run.smithi112.stdout: Birth: 2024-09-21 00:22:43.702323791 +0000 2024-09-21T00:24:44.966 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-21T00:24:45.028 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records in 2024-09-21T00:24:45.029 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records out 2024-09-21T00:24:45.029 INFO:teuthology.orchestra.run.smithi112.stderr:512 bytes copied, 0.000187422 s, 2.7 MB/s 2024-09-21T00:24:45.030 DEBUG:teuthology.orchestra.run.smithi112:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-21T00:24:45.086 DEBUG:teuthology.orchestra.run.smithi112:> stat /dev/vg_nvme/lv_4 2024-09-21T00:24:45.141 INFO:teuthology.orchestra.run.smithi112.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-21T00:24:45.141 INFO:teuthology.orchestra.run.smithi112.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T00:24:45.141 INFO:teuthology.orchestra.run.smithi112.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-09-21T00:24:45.141 INFO:teuthology.orchestra.run.smithi112.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T00:24:45.141 INFO:teuthology.orchestra.run.smithi112.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T00:24:45.141 INFO:teuthology.orchestra.run.smithi112.stdout:Access: 2024-09-21 00:24:26.629736113 +0000 2024-09-21T00:24:45.142 INFO:teuthology.orchestra.run.smithi112.stdout:Modify: 2024-09-21 00:22:44.033315813 +0000 2024-09-21T00:24:45.142 INFO:teuthology.orchestra.run.smithi112.stdout:Change: 2024-09-21 00:22:44.033315813 +0000 2024-09-21T00:24:45.142 INFO:teuthology.orchestra.run.smithi112.stdout: Birth: 2024-09-21 00:22:44.033315813 +0000 2024-09-21T00:24:45.142 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-21T00:24:45.204 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records in 2024-09-21T00:24:45.204 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records out 2024-09-21T00:24:45.204 INFO:teuthology.orchestra.run.smithi112.stderr:512 bytes copied, 0.000175939 s, 2.9 MB/s 2024-09-21T00:24:45.206 DEBUG:teuthology.orchestra.run.smithi112:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-21T00:24:45.261 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-21T00:24:45.262 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-09-21T00:24:45.262 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/scratch_devs of=/dev/stdout 2024-09-21T00:24:45.281 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-21T00:24:45.281 DEBUG:teuthology.orchestra.run.smithi187:> stat /dev/vg_nvme/lv_1 2024-09-21T00:24:45.339 INFO:teuthology.orchestra.run.smithi187.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-21T00:24:45.339 INFO:teuthology.orchestra.run.smithi187.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T00:24:45.339 INFO:teuthology.orchestra.run.smithi187.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-09-21T00:24:45.339 INFO:teuthology.orchestra.run.smithi187.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T00:24:45.339 INFO:teuthology.orchestra.run.smithi187.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T00:24:45.339 INFO:teuthology.orchestra.run.smithi187.stdout:Access: 2024-09-21 00:24:33.562670650 +0000 2024-09-21T00:24:45.339 INFO:teuthology.orchestra.run.smithi187.stdout:Modify: 2024-09-21 00:22:43.056522693 +0000 2024-09-21T00:24:45.339 INFO:teuthology.orchestra.run.smithi187.stdout:Change: 2024-09-21 00:22:43.056522693 +0000 2024-09-21T00:24:45.339 INFO:teuthology.orchestra.run.smithi187.stdout: Birth: 2024-09-21 00:22:43.056522693 +0000 2024-09-21T00:24:45.339 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-21T00:24:45.406 INFO:teuthology.orchestra.run.smithi187.stderr:1+0 records in 2024-09-21T00:24:45.406 INFO:teuthology.orchestra.run.smithi187.stderr:1+0 records out 2024-09-21T00:24:45.406 INFO:teuthology.orchestra.run.smithi187.stderr:512 bytes copied, 0.000227223 s, 2.3 MB/s 2024-09-21T00:24:45.407 DEBUG:teuthology.orchestra.run.smithi187:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-21T00:24:45.467 DEBUG:teuthology.orchestra.run.smithi187:> stat /dev/vg_nvme/lv_2 2024-09-21T00:24:45.524 INFO:teuthology.orchestra.run.smithi187.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-21T00:24:45.524 INFO:teuthology.orchestra.run.smithi187.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T00:24:45.524 INFO:teuthology.orchestra.run.smithi187.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-09-21T00:24:45.524 INFO:teuthology.orchestra.run.smithi187.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T00:24:45.524 INFO:teuthology.orchestra.run.smithi187.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T00:24:45.525 INFO:teuthology.orchestra.run.smithi187.stdout:Access: 2024-09-21 00:24:33.563670635 +0000 2024-09-21T00:24:45.525 INFO:teuthology.orchestra.run.smithi187.stdout:Modify: 2024-09-21 00:22:43.381517886 +0000 2024-09-21T00:24:45.525 INFO:teuthology.orchestra.run.smithi187.stdout:Change: 2024-09-21 00:22:43.381517886 +0000 2024-09-21T00:24:45.525 INFO:teuthology.orchestra.run.smithi187.stdout: Birth: 2024-09-21 00:22:43.381517886 +0000 2024-09-21T00:24:45.525 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-21T00:24:45.591 INFO:teuthology.orchestra.run.smithi187.stderr:1+0 records in 2024-09-21T00:24:45.591 INFO:teuthology.orchestra.run.smithi187.stderr:1+0 records out 2024-09-21T00:24:45.591 INFO:teuthology.orchestra.run.smithi187.stderr:512 bytes copied, 0.000275159 s, 1.9 MB/s 2024-09-21T00:24:45.592 DEBUG:teuthology.orchestra.run.smithi187:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-21T00:24:45.650 DEBUG:teuthology.orchestra.run.smithi187:> stat /dev/vg_nvme/lv_3 2024-09-21T00:24:45.706 INFO:teuthology.orchestra.run.smithi187.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-21T00:24:45.707 INFO:teuthology.orchestra.run.smithi187.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T00:24:45.707 INFO:teuthology.orchestra.run.smithi187.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-09-21T00:24:45.707 INFO:teuthology.orchestra.run.smithi187.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T00:24:45.707 INFO:teuthology.orchestra.run.smithi187.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T00:24:45.707 INFO:teuthology.orchestra.run.smithi187.stdout:Access: 2024-09-21 00:24:33.563670635 +0000 2024-09-21T00:24:45.707 INFO:teuthology.orchestra.run.smithi187.stdout:Modify: 2024-09-21 00:22:43.706513080 +0000 2024-09-21T00:24:45.707 INFO:teuthology.orchestra.run.smithi187.stdout:Change: 2024-09-21 00:22:43.706513080 +0000 2024-09-21T00:24:45.707 INFO:teuthology.orchestra.run.smithi187.stdout: Birth: 2024-09-21 00:22:43.706513080 +0000 2024-09-21T00:24:45.707 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-21T00:24:45.772 INFO:teuthology.orchestra.run.smithi187.stderr:1+0 records in 2024-09-21T00:24:45.772 INFO:teuthology.orchestra.run.smithi187.stderr:1+0 records out 2024-09-21T00:24:45.773 INFO:teuthology.orchestra.run.smithi187.stderr:512 bytes copied, 0.000203804 s, 2.5 MB/s 2024-09-21T00:24:45.774 DEBUG:teuthology.orchestra.run.smithi187:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-21T00:24:45.832 DEBUG:teuthology.orchestra.run.smithi187:> stat /dev/vg_nvme/lv_4 2024-09-21T00:24:45.889 INFO:teuthology.orchestra.run.smithi187.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-21T00:24:45.889 INFO:teuthology.orchestra.run.smithi187.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T00:24:45.889 INFO:teuthology.orchestra.run.smithi187.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-09-21T00:24:45.889 INFO:teuthology.orchestra.run.smithi187.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T00:24:45.889 INFO:teuthology.orchestra.run.smithi187.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T00:24:45.889 INFO:teuthology.orchestra.run.smithi187.stdout:Access: 2024-09-21 00:24:33.563670635 +0000 2024-09-21T00:24:45.889 INFO:teuthology.orchestra.run.smithi187.stdout:Modify: 2024-09-21 00:22:44.030508288 +0000 2024-09-21T00:24:45.889 INFO:teuthology.orchestra.run.smithi187.stdout:Change: 2024-09-21 00:22:44.030508288 +0000 2024-09-21T00:24:45.889 INFO:teuthology.orchestra.run.smithi187.stdout: Birth: 2024-09-21 00:22:44.030508288 +0000 2024-09-21T00:24:45.890 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-21T00:24:45.954 INFO:teuthology.orchestra.run.smithi187.stderr:1+0 records in 2024-09-21T00:24:45.954 INFO:teuthology.orchestra.run.smithi187.stderr:1+0 records out 2024-09-21T00:24:45.954 INFO:teuthology.orchestra.run.smithi187.stderr:512 bytes copied, 0.000198886 s, 2.6 MB/s 2024-09-21T00:24:45.955 DEBUG:teuthology.orchestra.run.smithi187:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-21T00:24:46.013 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-21T00:24:46.013 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi112.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi187.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-21T00:24:46.013 INFO:tasks.ceph:Generating config... 2024-09-21T00:24:46.014 INFO:tasks.ceph:[client] pid file = 2024-09-21T00:24:46.015 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-09-21T00:24:46.015 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-21T00:24:46.015 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-21T00:24:46.015 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-21T00:24:46.015 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-21T00:24:46.015 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-21T00:24:46.015 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-09-21T00:24:46.015 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-21T00:24:46.015 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-21T00:24:46.015 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-09-21T00:24:46.015 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-09-21T00:24:46.015 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-21T00:24:46.015 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-21T00:24:46.015 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-09-21T00:24:46.016 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-21T00:24:46.016 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-21T00:24:46.016 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-21T00:24:46.016 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-21T00:24:46.016 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-21T00:24:46.016 INFO:tasks.ceph:Setting up mon.a... 2024-09-21T00:24:46.016 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-21T00:24:46.084 INFO:teuthology.orchestra.run.smithi112.stdout:creating /etc/ceph/ceph.keyring 2024-09-21T00:24:46.087 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-21T00:24:46.142 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-21T00:24:46.166 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.112'), ('mon.b', '172.21.15.187'), ('mon.c', '[v2:172.21.15.187:3301,v1:172.21.15.187:6790]')] 2024-09-21T00:24:46.166 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.112,172.21.15.187,[v2:172.21.15.187:3301,v1:172.21.15.187:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'pid file': ''}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}} 2024-09-21T00:24:46.167 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T00:24:46.167 DEBUG:teuthology.orchestra.run.smithi112:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-21T00:24:46.221 DEBUG:teuthology.orchestra.run.smithi112:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.112 --add b 172.21.15.187 --addv c '[v2:172.21.15.187:3301,v1:172.21.15.187:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-09-21T00:24:46.317 INFO:teuthology.orchestra.run.smithi112.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-21T00:24:46.318 INFO:teuthology.orchestra.run.smithi112.stdout:monmaptool: generated fsid 8b34692b-dc65-49b2-a08f-26f707b96ac5 2024-09-21T00:24:46.318 INFO:teuthology.orchestra.run.smithi112.stdout:setting min_mon_release = quincy 2024-09-21T00:24:46.318 INFO:teuthology.orchestra.run.smithi112.stdout:epoch 0 2024-09-21T00:24:46.318 INFO:teuthology.orchestra.run.smithi112.stdout:fsid 8b34692b-dc65-49b2-a08f-26f707b96ac5 2024-09-21T00:24:46.318 INFO:teuthology.orchestra.run.smithi112.stdout:last_changed 2024-09-21T00:24:46.317653+0000 2024-09-21T00:24:46.318 INFO:teuthology.orchestra.run.smithi112.stdout:created 2024-09-21T00:24:46.317653+0000 2024-09-21T00:24:46.318 INFO:teuthology.orchestra.run.smithi112.stdout:min_mon_release 17 (quincy) 2024-09-21T00:24:46.318 INFO:teuthology.orchestra.run.smithi112.stdout:election_strategy: 1 2024-09-21T00:24:46.318 INFO:teuthology.orchestra.run.smithi112.stdout:0: [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] mon.a 2024-09-21T00:24:46.318 INFO:teuthology.orchestra.run.smithi112.stdout:1: [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] mon.b 2024-09-21T00:24:46.318 INFO:teuthology.orchestra.run.smithi112.stdout:2: [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] mon.c 2024-09-21T00:24:46.318 INFO:teuthology.orchestra.run.smithi112.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-21T00:24:46.319 DEBUG:teuthology.orchestra.run.smithi112:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-21T00:24:46.335 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 8b34692b-dc65-49b2-a08f-26f707b96ac5... 2024-09-21T00:24:46.336 DEBUG:teuthology.orchestra.run.smithi112:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-21T00:24:46.378 DEBUG:teuthology.orchestra.run.smithi187:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-21T00:24:46.418 INFO:teuthology.orchestra.run.smithi112.stdout:[global] 2024-09-21T00:24:46.418 INFO:teuthology.orchestra.run.smithi112.stdout: chdir = "" 2024-09-21T00:24:46.419 INFO:teuthology.orchestra.run.smithi112.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-21T00:24:46.419 INFO:teuthology.orchestra.run.smithi112.stdout: auth supported = cephx 2024-09-21T00:24:46.419 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:24:46.419 INFO:teuthology.orchestra.run.smithi112.stdout: filestore xattr use omap = true 2024-09-21T00:24:46.419 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:24:46.419 INFO:teuthology.orchestra.run.smithi112.stdout: mon clock drift allowed = 1.000 2024-09-21T00:24:46.419 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:24:46.419 INFO:teuthology.orchestra.run.smithi112.stdout: osd crush chooseleaf type = 0 2024-09-21T00:24:46.419 INFO:teuthology.orchestra.run.smithi112.stdout: auth debug = true 2024-09-21T00:24:46.419 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:24:46.419 INFO:teuthology.orchestra.run.smithi112.stdout: ms die on old message = true 2024-09-21T00:24:46.419 INFO:teuthology.orchestra.run.smithi112.stdout: ms die on bug = true 2024-09-21T00:24:46.419 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:24:46.420 INFO:teuthology.orchestra.run.smithi112.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-21T00:24:46.420 INFO:teuthology.orchestra.run.smithi112.stdout: mon pg warn max object skew = 0 2024-09-21T00:24:46.420 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:24:46.420 INFO:teuthology.orchestra.run.smithi112.stdout: # disable pg_autoscaler by default for new pools 2024-09-21T00:24:46.420 INFO:teuthology.orchestra.run.smithi112.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-21T00:24:46.420 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:24:46.420 INFO:teuthology.orchestra.run.smithi112.stdout: osd pool default size = 2 2024-09-21T00:24:46.420 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:24:46.420 INFO:teuthology.orchestra.run.smithi112.stdout: mon osd allow primary affinity = true 2024-09-21T00:24:46.420 INFO:teuthology.orchestra.run.smithi112.stdout: mon osd allow pg remap = true 2024-09-21T00:24:46.420 INFO:teuthology.orchestra.run.smithi112.stdout: mon warn on legacy crush tunables = false 2024-09-21T00:24:46.420 INFO:teuthology.orchestra.run.smithi112.stdout: mon warn on crush straw calc version zero = false 2024-09-21T00:24:46.420 INFO:teuthology.orchestra.run.smithi112.stdout: mon warn on no sortbitwise = false 2024-09-21T00:24:46.421 INFO:teuthology.orchestra.run.smithi112.stdout: mon warn on osd down out interval zero = false 2024-09-21T00:24:46.421 INFO:teuthology.orchestra.run.smithi112.stdout: mon warn on too few osds = false 2024-09-21T00:24:46.421 INFO:teuthology.orchestra.run.smithi112.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-21T00:24:46.421 INFO:teuthology.orchestra.run.smithi112.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-21T00:24:46.421 INFO:teuthology.orchestra.run.smithi112.stdout: mon_allow_pool_size_one = true 2024-09-21T00:24:46.421 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:24:46.421 INFO:teuthology.orchestra.run.smithi112.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-21T00:24:46.421 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:24:46.421 INFO:teuthology.orchestra.run.smithi112.stdout: osd default data pool replay window = 5 2024-09-21T00:24:46.421 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:24:46.421 INFO:teuthology.orchestra.run.smithi112.stdout: mon allow pool delete = true 2024-09-21T00:24:46.421 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:24:46.421 INFO:teuthology.orchestra.run.smithi112.stdout: mon cluster log file level = debug 2024-09-21T00:24:46.421 INFO:teuthology.orchestra.run.smithi112.stdout: debug asserts on shutdown = true 2024-09-21T00:24:46.421 INFO:teuthology.orchestra.run.smithi112.stdout: mon health detail to clog = false 2024-09-21T00:24:46.422 INFO:teuthology.orchestra.run.smithi112.stdout: mon host = "172.21.15.112,172.21.15.187,[v2:172.21.15.187:3301,v1:172.21.15.187:6790]" 2024-09-21T00:24:46.422 INFO:teuthology.orchestra.run.smithi112.stdout: ms die on skipped message = False 2024-09-21T00:24:46.422 INFO:teuthology.orchestra.run.smithi112.stdout: fsid = 8b34692b-dc65-49b2-a08f-26f707b96ac5 2024-09-21T00:24:46.422 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:24:46.422 INFO:teuthology.orchestra.run.smithi112.stdout:[osd] 2024-09-21T00:24:46.422 INFO:teuthology.orchestra.run.smithi112.stdout: osd journal size = 100 2024-09-21T00:24:46.422 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:24:46.422 INFO:teuthology.orchestra.run.smithi112.stdout: osd scrub load threshold = 5.0 2024-09-21T00:24:46.422 INFO:teuthology.orchestra.run.smithi112.stdout: osd scrub max interval = 600 2024-09-21T00:24:46.422 INFO:teuthology.orchestra.run.smithi112.stdout: osd mclock profile = high_recovery_ops 2024-09-21T00:24:46.422 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:24:46.422 INFO:teuthology.orchestra.run.smithi112.stdout: osd recover clone overlap = true 2024-09-21T00:24:46.422 INFO:teuthology.orchestra.run.smithi112.stdout: osd recovery max chunk = 1048576 2024-09-21T00:24:46.423 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:24:46.423 INFO:teuthology.orchestra.run.smithi112.stdout: osd debug shutdown = true 2024-09-21T00:24:46.423 INFO:teuthology.orchestra.run.smithi112.stdout: osd debug op order = true 2024-09-21T00:24:46.423 INFO:teuthology.orchestra.run.smithi112.stdout: osd debug verify stray on activate = true 2024-09-21T00:24:46.423 INFO:teuthology.orchestra.run.smithi112.stdout: osd debug trim objects = true 2024-09-21T00:24:46.423 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:24:46.423 INFO:teuthology.orchestra.run.smithi112.stdout: osd open classes on start = true 2024-09-21T00:24:46.423 INFO:teuthology.orchestra.run.smithi112.stdout: osd debug pg log writeout = true 2024-09-21T00:24:46.423 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:24:46.423 INFO:teuthology.orchestra.run.smithi112.stdout: osd deep scrub update digest min age = 30 2024-09-21T00:24:46.423 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:24:46.423 INFO:teuthology.orchestra.run.smithi112.stdout: osd map max advance = 10 2024-09-21T00:24:46.424 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:24:46.424 INFO:teuthology.orchestra.run.smithi112.stdout: journal zero on create = true 2024-09-21T00:24:46.424 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:24:46.424 INFO:teuthology.orchestra.run.smithi112.stdout: filestore ondisk finisher threads = 3 2024-09-21T00:24:46.424 INFO:teuthology.orchestra.run.smithi112.stdout: filestore apply finisher threads = 3 2024-09-21T00:24:46.424 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:24:46.424 INFO:teuthology.orchestra.run.smithi112.stdout: bdev debug aio = true 2024-09-21T00:24:46.424 INFO:teuthology.orchestra.run.smithi112.stdout: osd debug misdirected ops = true 2024-09-21T00:24:46.424 INFO:teuthology.orchestra.run.smithi112.stdout: bluestore block size = 96636764160 2024-09-21T00:24:46.424 INFO:teuthology.orchestra.run.smithi112.stdout: bluestore fsck on mount = True 2024-09-21T00:24:46.424 INFO:teuthology.orchestra.run.smithi112.stdout: debug bluefs = 20 2024-09-21T00:24:46.424 INFO:teuthology.orchestra.run.smithi112.stdout: debug bluestore = 20 2024-09-21T00:24:46.424 INFO:teuthology.orchestra.run.smithi112.stdout: debug ms = 1 2024-09-21T00:24:46.424 INFO:teuthology.orchestra.run.smithi112.stdout: debug osd = 20 2024-09-21T00:24:46.424 INFO:teuthology.orchestra.run.smithi112.stdout: debug rocksdb = 10 2024-09-21T00:24:46.424 INFO:teuthology.orchestra.run.smithi112.stdout: mon osd backfillfull_ratio = 0.85 2024-09-21T00:24:46.425 INFO:teuthology.orchestra.run.smithi112.stdout: mon osd full ratio = 0.9 2024-09-21T00:24:46.425 INFO:teuthology.orchestra.run.smithi112.stdout: mon osd nearfull ratio = 0.8 2024-09-21T00:24:46.425 INFO:teuthology.orchestra.run.smithi112.stdout: osd failsafe full ratio = 0.95 2024-09-21T00:24:46.425 INFO:teuthology.orchestra.run.smithi112.stdout: osd objectstore = bluestore 2024-09-21T00:24:46.425 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:24:46.425 INFO:teuthology.orchestra.run.smithi112.stdout:[mgr] 2024-09-21T00:24:46.425 INFO:teuthology.orchestra.run.smithi112.stdout: debug ms = 1 2024-09-21T00:24:46.425 INFO:teuthology.orchestra.run.smithi112.stdout: debug mgr = 20 2024-09-21T00:24:46.425 INFO:teuthology.orchestra.run.smithi112.stdout: debug mon = 20 2024-09-21T00:24:46.425 INFO:teuthology.orchestra.run.smithi112.stdout: debug auth = 20 2024-09-21T00:24:46.425 INFO:teuthology.orchestra.run.smithi112.stdout: mon reweight min pgs per osd = 4 2024-09-21T00:24:46.425 INFO:teuthology.orchestra.run.smithi112.stdout: mon reweight min bytes per osd = 10 2024-09-21T00:24:46.425 INFO:teuthology.orchestra.run.smithi112.stdout: mgr/telemetry/nag = false 2024-09-21T00:24:46.425 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:24:46.425 INFO:teuthology.orchestra.run.smithi112.stdout:[mon] 2024-09-21T00:24:46.425 INFO:teuthology.orchestra.run.smithi112.stdout: debug ms = 1 2024-09-21T00:24:46.425 INFO:teuthology.orchestra.run.smithi112.stdout: debug mon = 20 2024-09-21T00:24:46.426 INFO:teuthology.orchestra.run.smithi112.stdout: debug paxos = 20 2024-09-21T00:24:46.426 INFO:teuthology.orchestra.run.smithi112.stdout: debug auth = 20 2024-09-21T00:24:46.426 INFO:teuthology.orchestra.run.smithi112.stdout: mon data avail warn = 5 2024-09-21T00:24:46.426 INFO:teuthology.orchestra.run.smithi112.stdout: mon mgr mkfs grace = 240 2024-09-21T00:24:46.426 INFO:teuthology.orchestra.run.smithi112.stdout: mon reweight min pgs per osd = 4 2024-09-21T00:24:46.426 INFO:teuthology.orchestra.run.smithi112.stdout: mon osd reporter subtree level = osd 2024-09-21T00:24:46.426 INFO:teuthology.orchestra.run.smithi112.stdout: mon osd prime pg temp = true 2024-09-21T00:24:46.426 INFO:teuthology.orchestra.run.smithi112.stdout: mon reweight min bytes per osd = 10 2024-09-21T00:24:46.426 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:24:46.426 INFO:teuthology.orchestra.run.smithi112.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-21T00:24:46.426 INFO:teuthology.orchestra.run.smithi112.stdout: auth mon ticket ttl = 660 # 11m 2024-09-21T00:24:46.426 INFO:teuthology.orchestra.run.smithi112.stdout: auth service ticket ttl = 240 # 4m 2024-09-21T00:24:46.426 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:24:46.426 INFO:teuthology.orchestra.run.smithi112.stdout: # don't complain about insecure global_id in the test suite 2024-09-21T00:24:46.426 INFO:teuthology.orchestra.run.smithi112.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-21T00:24:46.427 INFO:teuthology.orchestra.run.smithi112.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-21T00:24:46.427 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:24:46.427 INFO:teuthology.orchestra.run.smithi112.stdout: # 1m isn't quite enough 2024-09-21T00:24:46.427 INFO:teuthology.orchestra.run.smithi112.stdout: mon_down_mkfs_grace = 2m 2024-09-21T00:24:46.427 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:24:46.427 INFO:teuthology.orchestra.run.smithi112.stdout: mon_warn_on_filestore_osds = false 2024-09-21T00:24:46.427 INFO:teuthology.orchestra.run.smithi112.stdout: mon down mkfs grace = 300 2024-09-21T00:24:46.427 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:24:46.427 INFO:teuthology.orchestra.run.smithi112.stdout:[client] 2024-09-21T00:24:46.427 INFO:teuthology.orchestra.run.smithi112.stdout: rgw cache enabled = true 2024-09-21T00:24:46.427 INFO:teuthology.orchestra.run.smithi112.stdout: rgw enable ops log = true 2024-09-21T00:24:46.427 INFO:teuthology.orchestra.run.smithi112.stdout: rgw enable usage log = true 2024-09-21T00:24:46.427 INFO:teuthology.orchestra.run.smithi112.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-21T00:24:46.427 INFO:teuthology.orchestra.run.smithi112.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-21T00:24:46.428 INFO:teuthology.orchestra.run.smithi112.stdout: pid file = "" 2024-09-21T00:24:46.428 INFO:teuthology.orchestra.run.smithi112.stdout:[mon.a] 2024-09-21T00:24:46.428 INFO:teuthology.orchestra.run.smithi112.stdout:[mon.b] 2024-09-21T00:24:46.428 INFO:teuthology.orchestra.run.smithi112.stdout:[mon.c] 2024-09-21T00:24:46.428 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.a] 2024-09-21T00:24:46.428 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.c] 2024-09-21T00:24:46.428 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.b] 2024-09-21T00:24:46.428 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.d] 2024-09-21T00:24:46.428 INFO:teuthology.orchestra.run.smithi187.stdout:[global] 2024-09-21T00:24:46.428 INFO:teuthology.orchestra.run.smithi187.stdout: chdir = "" 2024-09-21T00:24:46.429 INFO:teuthology.orchestra.run.smithi187.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-21T00:24:46.429 INFO:teuthology.orchestra.run.smithi187.stdout: auth supported = cephx 2024-09-21T00:24:46.429 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:24:46.429 INFO:teuthology.orchestra.run.smithi187.stdout: filestore xattr use omap = true 2024-09-21T00:24:46.429 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:24:46.429 INFO:teuthology.orchestra.run.smithi187.stdout: mon clock drift allowed = 1.000 2024-09-21T00:24:46.429 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:24:46.429 INFO:teuthology.orchestra.run.smithi187.stdout: osd crush chooseleaf type = 0 2024-09-21T00:24:46.429 INFO:teuthology.orchestra.run.smithi187.stdout: auth debug = true 2024-09-21T00:24:46.429 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:24:46.429 INFO:teuthology.orchestra.run.smithi187.stdout: ms die on old message = true 2024-09-21T00:24:46.429 INFO:teuthology.orchestra.run.smithi187.stdout: ms die on bug = true 2024-09-21T00:24:46.429 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:24:46.429 INFO:teuthology.orchestra.run.smithi187.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-21T00:24:46.429 INFO:teuthology.orchestra.run.smithi187.stdout: mon pg warn max object skew = 0 2024-09-21T00:24:46.429 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:24:46.430 INFO:teuthology.orchestra.run.smithi187.stdout: # disable pg_autoscaler by default for new pools 2024-09-21T00:24:46.430 INFO:teuthology.orchestra.run.smithi187.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-21T00:24:46.430 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:24:46.430 INFO:teuthology.orchestra.run.smithi187.stdout: osd pool default size = 2 2024-09-21T00:24:46.430 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:24:46.430 INFO:teuthology.orchestra.run.smithi187.stdout: mon osd allow primary affinity = true 2024-09-21T00:24:46.430 INFO:teuthology.orchestra.run.smithi187.stdout: mon osd allow pg remap = true 2024-09-21T00:24:46.430 INFO:teuthology.orchestra.run.smithi187.stdout: mon warn on legacy crush tunables = false 2024-09-21T00:24:46.430 INFO:teuthology.orchestra.run.smithi187.stdout: mon warn on crush straw calc version zero = false 2024-09-21T00:24:46.430 INFO:teuthology.orchestra.run.smithi187.stdout: mon warn on no sortbitwise = false 2024-09-21T00:24:46.430 INFO:teuthology.orchestra.run.smithi187.stdout: mon warn on osd down out interval zero = false 2024-09-21T00:24:46.430 INFO:teuthology.orchestra.run.smithi187.stdout: mon warn on too few osds = false 2024-09-21T00:24:46.430 INFO:teuthology.orchestra.run.smithi187.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-21T00:24:46.430 INFO:teuthology.orchestra.run.smithi187.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-21T00:24:46.431 INFO:teuthology.orchestra.run.smithi187.stdout: mon_allow_pool_size_one = true 2024-09-21T00:24:46.431 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:24:46.431 INFO:teuthology.orchestra.run.smithi187.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-21T00:24:46.431 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:24:46.431 INFO:teuthology.orchestra.run.smithi187.stdout: osd default data pool replay window = 5 2024-09-21T00:24:46.431 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:24:46.431 INFO:teuthology.orchestra.run.smithi187.stdout: mon allow pool delete = true 2024-09-21T00:24:46.431 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:24:46.431 INFO:teuthology.orchestra.run.smithi187.stdout: mon cluster log file level = debug 2024-09-21T00:24:46.431 INFO:teuthology.orchestra.run.smithi187.stdout: debug asserts on shutdown = true 2024-09-21T00:24:46.431 INFO:teuthology.orchestra.run.smithi187.stdout: mon health detail to clog = false 2024-09-21T00:24:46.431 INFO:teuthology.orchestra.run.smithi187.stdout: mon host = "172.21.15.112,172.21.15.187,[v2:172.21.15.187:3301,v1:172.21.15.187:6790]" 2024-09-21T00:24:46.431 INFO:teuthology.orchestra.run.smithi187.stdout: ms die on skipped message = False 2024-09-21T00:24:46.431 INFO:teuthology.orchestra.run.smithi187.stdout: fsid = 8b34692b-dc65-49b2-a08f-26f707b96ac5 2024-09-21T00:24:46.431 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:24:46.432 INFO:teuthology.orchestra.run.smithi187.stdout:[osd] 2024-09-21T00:24:46.432 INFO:teuthology.orchestra.run.smithi187.stdout: osd journal size = 100 2024-09-21T00:24:46.432 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:24:46.432 INFO:teuthology.orchestra.run.smithi187.stdout: osd scrub load threshold = 5.0 2024-09-21T00:24:46.432 INFO:teuthology.orchestra.run.smithi187.stdout: osd scrub max interval = 600 2024-09-21T00:24:46.432 INFO:teuthology.orchestra.run.smithi187.stdout: osd mclock profile = high_recovery_ops 2024-09-21T00:24:46.432 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:24:46.432 INFO:teuthology.orchestra.run.smithi187.stdout: osd recover clone overlap = true 2024-09-21T00:24:46.432 INFO:teuthology.orchestra.run.smithi187.stdout: osd recovery max chunk = 1048576 2024-09-21T00:24:46.432 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:24:46.432 INFO:teuthology.orchestra.run.smithi187.stdout: osd debug shutdown = true 2024-09-21T00:24:46.432 INFO:teuthology.orchestra.run.smithi187.stdout: osd debug op order = true 2024-09-21T00:24:46.432 INFO:teuthology.orchestra.run.smithi187.stdout: osd debug verify stray on activate = true 2024-09-21T00:24:46.432 INFO:teuthology.orchestra.run.smithi187.stdout: osd debug trim objects = true 2024-09-21T00:24:46.433 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:24:46.433 INFO:teuthology.orchestra.run.smithi187.stdout: osd open classes on start = true 2024-09-21T00:24:46.433 INFO:teuthology.orchestra.run.smithi187.stdout: osd debug pg log writeout = true 2024-09-21T00:24:46.433 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:24:46.433 INFO:teuthology.orchestra.run.smithi187.stdout: osd deep scrub update digest min age = 30 2024-09-21T00:24:46.433 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:24:46.433 INFO:teuthology.orchestra.run.smithi187.stdout: osd map max advance = 10 2024-09-21T00:24:46.433 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:24:46.433 INFO:teuthology.orchestra.run.smithi187.stdout: journal zero on create = true 2024-09-21T00:24:46.433 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:24:46.433 INFO:teuthology.orchestra.run.smithi187.stdout: filestore ondisk finisher threads = 3 2024-09-21T00:24:46.433 INFO:teuthology.orchestra.run.smithi187.stdout: filestore apply finisher threads = 3 2024-09-21T00:24:46.433 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:24:46.433 INFO:teuthology.orchestra.run.smithi187.stdout: bdev debug aio = true 2024-09-21T00:24:46.433 INFO:teuthology.orchestra.run.smithi187.stdout: osd debug misdirected ops = true 2024-09-21T00:24:46.434 INFO:teuthology.orchestra.run.smithi187.stdout: bluestore block size = 96636764160 2024-09-21T00:24:46.434 INFO:teuthology.orchestra.run.smithi187.stdout: bluestore fsck on mount = True 2024-09-21T00:24:46.434 INFO:teuthology.orchestra.run.smithi187.stdout: debug bluefs = 20 2024-09-21T00:24:46.434 INFO:teuthology.orchestra.run.smithi187.stdout: debug bluestore = 20 2024-09-21T00:24:46.434 INFO:teuthology.orchestra.run.smithi187.stdout: debug ms = 1 2024-09-21T00:24:46.434 INFO:teuthology.orchestra.run.smithi187.stdout: debug osd = 20 2024-09-21T00:24:46.434 INFO:teuthology.orchestra.run.smithi187.stdout: debug rocksdb = 10 2024-09-21T00:24:46.434 INFO:teuthology.orchestra.run.smithi187.stdout: mon osd backfillfull_ratio = 0.85 2024-09-21T00:24:46.434 INFO:teuthology.orchestra.run.smithi187.stdout: mon osd full ratio = 0.9 2024-09-21T00:24:46.434 INFO:teuthology.orchestra.run.smithi187.stdout: mon osd nearfull ratio = 0.8 2024-09-21T00:24:46.434 INFO:teuthology.orchestra.run.smithi187.stdout: osd failsafe full ratio = 0.95 2024-09-21T00:24:46.434 INFO:teuthology.orchestra.run.smithi187.stdout: osd objectstore = bluestore 2024-09-21T00:24:46.434 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:24:46.434 INFO:teuthology.orchestra.run.smithi187.stdout:[mgr] 2024-09-21T00:24:46.434 INFO:teuthology.orchestra.run.smithi187.stdout: debug ms = 1 2024-09-21T00:24:46.435 INFO:teuthology.orchestra.run.smithi187.stdout: debug mgr = 20 2024-09-21T00:24:46.435 INFO:teuthology.orchestra.run.smithi187.stdout: debug mon = 20 2024-09-21T00:24:46.435 INFO:teuthology.orchestra.run.smithi187.stdout: debug auth = 20 2024-09-21T00:24:46.435 INFO:teuthology.orchestra.run.smithi187.stdout: mon reweight min pgs per osd = 4 2024-09-21T00:24:46.435 INFO:teuthology.orchestra.run.smithi187.stdout: mon reweight min bytes per osd = 10 2024-09-21T00:24:46.435 INFO:teuthology.orchestra.run.smithi187.stdout: mgr/telemetry/nag = false 2024-09-21T00:24:46.435 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:24:46.435 INFO:teuthology.orchestra.run.smithi187.stdout:[mon] 2024-09-21T00:24:46.435 INFO:teuthology.orchestra.run.smithi187.stdout: debug ms = 1 2024-09-21T00:24:46.435 INFO:teuthology.orchestra.run.smithi187.stdout: debug mon = 20 2024-09-21T00:24:46.435 INFO:teuthology.orchestra.run.smithi187.stdout: debug paxos = 20 2024-09-21T00:24:46.435 INFO:teuthology.orchestra.run.smithi187.stdout: debug auth = 20 2024-09-21T00:24:46.435 INFO:teuthology.orchestra.run.smithi187.stdout: mon data avail warn = 5 2024-09-21T00:24:46.435 INFO:teuthology.orchestra.run.smithi187.stdout: mon mgr mkfs grace = 240 2024-09-21T00:24:46.436 INFO:teuthology.orchestra.run.smithi187.stdout: mon reweight min pgs per osd = 4 2024-09-21T00:24:46.436 INFO:teuthology.orchestra.run.smithi187.stdout: mon osd reporter subtree level = osd 2024-09-21T00:24:46.436 INFO:teuthology.orchestra.run.smithi187.stdout: mon osd prime pg temp = true 2024-09-21T00:24:46.436 INFO:teuthology.orchestra.run.smithi187.stdout: mon reweight min bytes per osd = 10 2024-09-21T00:24:46.436 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:24:46.436 INFO:teuthology.orchestra.run.smithi187.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-21T00:24:46.436 INFO:teuthology.orchestra.run.smithi187.stdout: auth mon ticket ttl = 660 # 11m 2024-09-21T00:24:46.436 INFO:teuthology.orchestra.run.smithi187.stdout: auth service ticket ttl = 240 # 4m 2024-09-21T00:24:46.436 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:24:46.436 INFO:teuthology.orchestra.run.smithi187.stdout: # don't complain about insecure global_id in the test suite 2024-09-21T00:24:46.436 INFO:teuthology.orchestra.run.smithi187.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-21T00:24:46.436 INFO:teuthology.orchestra.run.smithi187.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-21T00:24:46.436 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:24:46.437 INFO:teuthology.orchestra.run.smithi187.stdout: # 1m isn't quite enough 2024-09-21T00:24:46.437 INFO:teuthology.orchestra.run.smithi187.stdout: mon_down_mkfs_grace = 2m 2024-09-21T00:24:46.437 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:24:46.437 INFO:teuthology.orchestra.run.smithi187.stdout: mon_warn_on_filestore_osds = false 2024-09-21T00:24:46.437 INFO:teuthology.orchestra.run.smithi187.stdout: mon down mkfs grace = 300 2024-09-21T00:24:46.437 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:24:46.437 INFO:teuthology.orchestra.run.smithi187.stdout:[client] 2024-09-21T00:24:46.437 INFO:teuthology.orchestra.run.smithi187.stdout: rgw cache enabled = true 2024-09-21T00:24:46.437 INFO:teuthology.orchestra.run.smithi187.stdout: rgw enable ops log = true 2024-09-21T00:24:46.437 INFO:teuthology.orchestra.run.smithi187.stdout: rgw enable usage log = true 2024-09-21T00:24:46.437 INFO:teuthology.orchestra.run.smithi187.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-21T00:24:46.437 INFO:teuthology.orchestra.run.smithi187.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-21T00:24:46.437 INFO:teuthology.orchestra.run.smithi187.stdout: pid file = "" 2024-09-21T00:24:46.437 INFO:teuthology.orchestra.run.smithi187.stdout:[mon.a] 2024-09-21T00:24:46.437 INFO:teuthology.orchestra.run.smithi187.stdout:[mon.b] 2024-09-21T00:24:46.437 INFO:teuthology.orchestra.run.smithi187.stdout:[mon.c] 2024-09-21T00:24:46.438 INFO:teuthology.orchestra.run.smithi187.stdout:[mds.a] 2024-09-21T00:24:46.438 INFO:teuthology.orchestra.run.smithi187.stdout:[mds.c] 2024-09-21T00:24:46.438 INFO:teuthology.orchestra.run.smithi187.stdout:[mds.b] 2024-09-21T00:24:46.438 INFO:teuthology.orchestra.run.smithi187.stdout:[mds.d] 2024-09-21T00:24:46.439 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-21T00:24:46.439 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-21T00:24:46.545 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-21T00:24:46.545 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T00:24:46.545 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-21T00:24:46.560 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T00:24:46.560 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-21T00:24:46.615 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi112.front.sepia.ceph.com 2024-09-21T00:24:46.616 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T00:24:46.616 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-21T00:24:46.616 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-21T00:24:46.690 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T00:24:46.690 DEBUG:teuthology.orchestra.run.smithi112:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-21T00:24:46.745 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi187.front.sepia.ceph.com 2024-09-21T00:24:46.745 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-09-21T00:24:46.745 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-21T00:24:46.745 DEBUG:teuthology.orchestra.run.smithi187:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-21T00:24:46.785 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-09-21T00:24:46.785 DEBUG:teuthology.orchestra.run.smithi187:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-21T00:24:46.842 INFO:tasks.ceph:Setting up mon nodes... 2024-09-21T00:24:46.842 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-21T00:24:46.842 DEBUG:teuthology.orchestra.run.smithi112:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-09-21T00:24:46.902 INFO:teuthology.orchestra.run.smithi112.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-09-21T00:24:46.905 DEBUG:teuthology.orchestra.run.smithi187:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-21T00:24:46.977 INFO:teuthology.orchestra.run.smithi187.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-21T00:24:46.980 INFO:tasks.ceph:Setting up mds nodes... 2024-09-21T00:24:46.981 DEBUG:teuthology.orchestra.run.smithi112:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-09-21T00:24:47.044 INFO:teuthology.orchestra.run.smithi112.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-09-21T00:24:47.048 DEBUG:teuthology.orchestra.run.smithi112:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-09-21T00:24:47.072 DEBUG:teuthology.orchestra.run.smithi112:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-09-21T00:24:47.173 INFO:teuthology.orchestra.run.smithi112.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-09-21T00:24:47.176 DEBUG:teuthology.orchestra.run.smithi112:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-09-21T00:24:47.201 DEBUG:teuthology.orchestra.run.smithi187:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-09-21T00:24:47.266 INFO:teuthology.orchestra.run.smithi187.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-09-21T00:24:47.270 DEBUG:teuthology.orchestra.run.smithi187:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-09-21T00:24:47.295 DEBUG:teuthology.orchestra.run.smithi187:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-09-21T00:24:47.395 INFO:teuthology.orchestra.run.smithi187.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-09-21T00:24:47.398 DEBUG:teuthology.orchestra.run.smithi187:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-09-21T00:24:47.424 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-21T00:24:47.424 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-21T00:24:47.475 INFO:teuthology.orchestra.run.smithi112.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-21T00:24:47.488 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-09-21T00:24:47.538 INFO:teuthology.orchestra.run.smithi112.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-09-21T00:24:47.551 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-21T00:24:47.601 INFO:teuthology.orchestra.run.smithi187.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-09-21T00:24:47.615 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-09-21T00:24:47.665 INFO:teuthology.orchestra.run.smithi187.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-09-21T00:24:47.678 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-21T00:24:47.678 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi112.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi187.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-21T00:24:47.678 DEBUG:teuthology.orchestra.run.smithi112:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-21T00:24:47.703 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-21T00:24:47.703 INFO:tasks.ceph:role: osd.0 2024-09-21T00:24:47.703 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi112.front.sepia.ceph.com 2024-09-21T00:24:47.704 DEBUG:teuthology.orchestra.run.smithi112:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-21T00:24:47.775 INFO:teuthology.orchestra.run.smithi112.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T00:24:47.775 INFO:teuthology.orchestra.run.smithi112.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T00:24:47.775 INFO:teuthology.orchestra.run.smithi112.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T00:24:47.775 INFO:teuthology.orchestra.run.smithi112.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-21T00:24:47.775 INFO:teuthology.orchestra.run.smithi112.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T00:24:47.775 INFO:teuthology.orchestra.run.smithi112.stdout: = sunit=0 swidth=0 blks 2024-09-21T00:24:47.775 INFO:teuthology.orchestra.run.smithi112.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T00:24:47.775 INFO:teuthology.orchestra.run.smithi112.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-21T00:24:47.775 INFO:teuthology.orchestra.run.smithi112.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T00:24:47.776 INFO:teuthology.orchestra.run.smithi112.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T00:24:48.285 INFO:teuthology.orchestra.run.smithi112.stdout:Discarding blocks...Done. 2024-09-21T00:24:48.287 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi112.front.sepia.ceph.com -o noatime 2024-09-21T00:24:48.287 DEBUG:teuthology.orchestra.run.smithi112:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-21T00:24:48.406 DEBUG:teuthology.orchestra.run.smithi112:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-21T00:24:48.437 DEBUG:teuthology.orchestra.run.smithi112:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-21T00:24:48.502 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-21T00:24:48.503 INFO:tasks.ceph:role: osd.1 2024-09-21T00:24:48.503 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi112.front.sepia.ceph.com 2024-09-21T00:24:48.503 DEBUG:teuthology.orchestra.run.smithi112:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-21T00:24:48.574 INFO:teuthology.orchestra.run.smithi112.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T00:24:48.575 INFO:teuthology.orchestra.run.smithi112.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T00:24:48.575 INFO:teuthology.orchestra.run.smithi112.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T00:24:48.575 INFO:teuthology.orchestra.run.smithi112.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-21T00:24:48.575 INFO:teuthology.orchestra.run.smithi112.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T00:24:48.575 INFO:teuthology.orchestra.run.smithi112.stdout: = sunit=0 swidth=0 blks 2024-09-21T00:24:48.575 INFO:teuthology.orchestra.run.smithi112.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T00:24:48.575 INFO:teuthology.orchestra.run.smithi112.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-21T00:24:48.575 INFO:teuthology.orchestra.run.smithi112.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T00:24:48.575 INFO:teuthology.orchestra.run.smithi112.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T00:24:49.048 INFO:teuthology.orchestra.run.smithi112.stdout:Discarding blocks...Done. 2024-09-21T00:24:49.050 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi112.front.sepia.ceph.com -o noatime 2024-09-21T00:24:49.050 DEBUG:teuthology.orchestra.run.smithi112:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-21T00:24:49.143 DEBUG:teuthology.orchestra.run.smithi112:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-21T00:24:49.172 DEBUG:teuthology.orchestra.run.smithi112:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-21T00:24:49.237 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-21T00:24:49.237 INFO:tasks.ceph:role: osd.2 2024-09-21T00:24:49.237 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi112.front.sepia.ceph.com 2024-09-21T00:24:49.237 DEBUG:teuthology.orchestra.run.smithi112:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-21T00:24:49.309 INFO:teuthology.orchestra.run.smithi112.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T00:24:49.310 INFO:teuthology.orchestra.run.smithi112.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T00:24:49.310 INFO:teuthology.orchestra.run.smithi112.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T00:24:49.310 INFO:teuthology.orchestra.run.smithi112.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-21T00:24:49.310 INFO:teuthology.orchestra.run.smithi112.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T00:24:49.310 INFO:teuthology.orchestra.run.smithi112.stdout: = sunit=0 swidth=0 blks 2024-09-21T00:24:49.310 INFO:teuthology.orchestra.run.smithi112.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T00:24:49.310 INFO:teuthology.orchestra.run.smithi112.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-21T00:24:49.310 INFO:teuthology.orchestra.run.smithi112.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T00:24:49.310 INFO:teuthology.orchestra.run.smithi112.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T00:24:49.785 INFO:teuthology.orchestra.run.smithi112.stdout:Discarding blocks...Done. 2024-09-21T00:24:49.787 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi112.front.sepia.ceph.com -o noatime 2024-09-21T00:24:49.787 DEBUG:teuthology.orchestra.run.smithi112:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-21T00:24:49.937 DEBUG:teuthology.orchestra.run.smithi112:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-21T00:24:49.967 DEBUG:teuthology.orchestra.run.smithi112:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-21T00:24:50.034 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-21T00:24:50.034 INFO:tasks.ceph:role: osd.3 2024-09-21T00:24:50.034 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi112.front.sepia.ceph.com 2024-09-21T00:24:50.034 DEBUG:teuthology.orchestra.run.smithi112:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-21T00:24:50.107 INFO:teuthology.orchestra.run.smithi112.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T00:24:50.107 INFO:teuthology.orchestra.run.smithi112.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T00:24:50.107 INFO:teuthology.orchestra.run.smithi112.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T00:24:50.107 INFO:teuthology.orchestra.run.smithi112.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-21T00:24:50.107 INFO:teuthology.orchestra.run.smithi112.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T00:24:50.108 INFO:teuthology.orchestra.run.smithi112.stdout: = sunit=0 swidth=0 blks 2024-09-21T00:24:50.108 INFO:teuthology.orchestra.run.smithi112.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T00:24:50.108 INFO:teuthology.orchestra.run.smithi112.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-21T00:24:50.108 INFO:teuthology.orchestra.run.smithi112.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T00:24:50.108 INFO:teuthology.orchestra.run.smithi112.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T00:24:50.634 INFO:teuthology.orchestra.run.smithi112.stdout:Discarding blocks...Done. 2024-09-21T00:24:50.636 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi112.front.sepia.ceph.com -o noatime 2024-09-21T00:24:50.636 DEBUG:teuthology.orchestra.run.smithi112:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-09-21T00:24:50.782 DEBUG:teuthology.orchestra.run.smithi112:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-21T00:24:50.813 DEBUG:teuthology.orchestra.run.smithi112:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T00:24:50.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:24:50.942+0000 7f45c0231640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-21T00:24:50.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:24:50.943+0000 7f45c0231640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-21T00:24:50.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:24:50.943+0000 7f45c0231640 -1 bdev(0x564f42cbf900 /var/lib/ceph/osd/ceph-0/block) open stat got: (1) Operation not permitted 2024-09-21T00:24:50.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:24:50.943+0000 7f45c0231640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-21T00:24:54.105 DEBUG:teuthology.orchestra.run.smithi112:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-21T00:24:54.134 DEBUG:teuthology.orchestra.run.smithi112:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T00:24:54.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:24:54.236+0000 7fb66f01a640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-21T00:24:54.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:24:54.236+0000 7fb66f01a640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-21T00:24:54.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:24:54.236+0000 7fb66f01a640 -1 bdev(0x5572dc455900 /var/lib/ceph/osd/ceph-1/block) open stat got: (1) Operation not permitted 2024-09-21T00:24:54.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:24:54.236+0000 7fb66f01a640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-21T00:24:57.188 DEBUG:teuthology.orchestra.run.smithi112:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-21T00:24:57.215 DEBUG:teuthology.orchestra.run.smithi112:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T00:24:57.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:24:57.313+0000 7f90ce815640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-21T00:24:57.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:24:57.313+0000 7f90ce815640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-21T00:24:57.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:24:57.313+0000 7f90ce815640 -1 bdev(0x557cf1be3900 /var/lib/ceph/osd/ceph-2/block) open stat got: (1) Operation not permitted 2024-09-21T00:24:57.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:24:57.313+0000 7f90ce815640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-21T00:25:00.487 DEBUG:teuthology.orchestra.run.smithi112:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-21T00:25:00.513 DEBUG:teuthology.orchestra.run.smithi112:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T00:25:00.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:00.611+0000 7f8923da4640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-21T00:25:00.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:00.611+0000 7f8923da4640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-21T00:25:00.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:00.611+0000 7f8923da4640 -1 bdev(0x559e7f22f900 /var/lib/ceph/osd/ceph-3/block) open stat got: (1) Operation not permitted 2024-09-21T00:25:00.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:00.611+0000 7f8923da4640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-21T00:25:03.779 DEBUG:teuthology.orchestra.run.smithi112:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-21T00:25:03.808 DEBUG:teuthology.orchestra.run.smithi187:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-09-21T00:25:03.839 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-21T00:25:03.839 INFO:tasks.ceph:role: osd.4 2024-09-21T00:25:03.839 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi187.front.sepia.ceph.com 2024-09-21T00:25:03.839 DEBUG:teuthology.orchestra.run.smithi187:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-21T00:25:03.915 INFO:teuthology.orchestra.run.smithi187.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T00:25:03.915 INFO:teuthology.orchestra.run.smithi187.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T00:25:03.915 INFO:teuthology.orchestra.run.smithi187.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T00:25:03.915 INFO:teuthology.orchestra.run.smithi187.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-21T00:25:03.916 INFO:teuthology.orchestra.run.smithi187.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T00:25:03.916 INFO:teuthology.orchestra.run.smithi187.stdout: = sunit=0 swidth=0 blks 2024-09-21T00:25:03.916 INFO:teuthology.orchestra.run.smithi187.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T00:25:03.916 INFO:teuthology.orchestra.run.smithi187.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-21T00:25:03.916 INFO:teuthology.orchestra.run.smithi187.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T00:25:03.916 INFO:teuthology.orchestra.run.smithi187.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T00:25:04.427 INFO:teuthology.orchestra.run.smithi187.stdout:Discarding blocks...Done. 2024-09-21T00:25:04.429 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi187.front.sepia.ceph.com -o noatime 2024-09-21T00:25:04.429 DEBUG:teuthology.orchestra.run.smithi187:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-09-21T00:25:04.573 DEBUG:teuthology.orchestra.run.smithi187:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-09-21T00:25:04.603 DEBUG:teuthology.orchestra.run.smithi187:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-09-21T00:25:04.670 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-21T00:25:04.670 INFO:tasks.ceph:role: osd.5 2024-09-21T00:25:04.670 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi187.front.sepia.ceph.com 2024-09-21T00:25:04.670 DEBUG:teuthology.orchestra.run.smithi187:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-21T00:25:04.743 INFO:teuthology.orchestra.run.smithi187.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T00:25:04.743 INFO:teuthology.orchestra.run.smithi187.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T00:25:04.743 INFO:teuthology.orchestra.run.smithi187.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T00:25:04.744 INFO:teuthology.orchestra.run.smithi187.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-21T00:25:04.744 INFO:teuthology.orchestra.run.smithi187.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T00:25:04.744 INFO:teuthology.orchestra.run.smithi187.stdout: = sunit=0 swidth=0 blks 2024-09-21T00:25:04.744 INFO:teuthology.orchestra.run.smithi187.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T00:25:04.744 INFO:teuthology.orchestra.run.smithi187.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-21T00:25:04.744 INFO:teuthology.orchestra.run.smithi187.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T00:25:04.744 INFO:teuthology.orchestra.run.smithi187.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T00:25:05.187 INFO:teuthology.orchestra.run.smithi187.stdout:Discarding blocks...Done. 2024-09-21T00:25:05.189 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi187.front.sepia.ceph.com -o noatime 2024-09-21T00:25:05.189 DEBUG:teuthology.orchestra.run.smithi187:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-09-21T00:25:05.280 DEBUG:teuthology.orchestra.run.smithi187:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-09-21T00:25:05.308 DEBUG:teuthology.orchestra.run.smithi187:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-09-21T00:25:05.373 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-21T00:25:05.373 INFO:tasks.ceph:role: osd.6 2024-09-21T00:25:05.373 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi187.front.sepia.ceph.com 2024-09-21T00:25:05.373 DEBUG:teuthology.orchestra.run.smithi187:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-21T00:25:05.445 INFO:teuthology.orchestra.run.smithi187.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T00:25:05.446 INFO:teuthology.orchestra.run.smithi187.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T00:25:05.446 INFO:teuthology.orchestra.run.smithi187.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T00:25:05.446 INFO:teuthology.orchestra.run.smithi187.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-21T00:25:05.446 INFO:teuthology.orchestra.run.smithi187.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T00:25:05.446 INFO:teuthology.orchestra.run.smithi187.stdout: = sunit=0 swidth=0 blks 2024-09-21T00:25:05.446 INFO:teuthology.orchestra.run.smithi187.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T00:25:05.446 INFO:teuthology.orchestra.run.smithi187.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-21T00:25:05.446 INFO:teuthology.orchestra.run.smithi187.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T00:25:05.446 INFO:teuthology.orchestra.run.smithi187.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T00:25:05.933 INFO:teuthology.orchestra.run.smithi187.stdout:Discarding blocks...Done. 2024-09-21T00:25:05.935 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi187.front.sepia.ceph.com -o noatime 2024-09-21T00:25:05.935 DEBUG:teuthology.orchestra.run.smithi187:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-09-21T00:25:06.074 DEBUG:teuthology.orchestra.run.smithi187:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-09-21T00:25:06.104 DEBUG:teuthology.orchestra.run.smithi187:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-09-21T00:25:06.169 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-21T00:25:06.169 INFO:tasks.ceph:role: osd.7 2024-09-21T00:25:06.169 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi187.front.sepia.ceph.com 2024-09-21T00:25:06.170 DEBUG:teuthology.orchestra.run.smithi187:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-21T00:25:06.243 INFO:teuthology.orchestra.run.smithi187.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T00:25:06.243 INFO:teuthology.orchestra.run.smithi187.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T00:25:06.243 INFO:teuthology.orchestra.run.smithi187.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T00:25:06.243 INFO:teuthology.orchestra.run.smithi187.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-21T00:25:06.243 INFO:teuthology.orchestra.run.smithi187.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T00:25:06.243 INFO:teuthology.orchestra.run.smithi187.stdout: = sunit=0 swidth=0 blks 2024-09-21T00:25:06.243 INFO:teuthology.orchestra.run.smithi187.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T00:25:06.243 INFO:teuthology.orchestra.run.smithi187.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-21T00:25:06.243 INFO:teuthology.orchestra.run.smithi187.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T00:25:06.243 INFO:teuthology.orchestra.run.smithi187.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T00:25:06.767 INFO:teuthology.orchestra.run.smithi187.stdout:Discarding blocks...Done. 2024-09-21T00:25:06.769 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi187.front.sepia.ceph.com -o noatime 2024-09-21T00:25:06.769 DEBUG:teuthology.orchestra.run.smithi187:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-09-21T00:25:06.887 DEBUG:teuthology.orchestra.run.smithi187:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-09-21T00:25:06.917 DEBUG:teuthology.orchestra.run.smithi187:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T00:25:07.025 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:07.023+0000 7f5eea424640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-09-21T00:25:07.025 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:07.023+0000 7f5eea424640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-09-21T00:25:07.025 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:07.023+0000 7f5eea424640 -1 bdev(0x5597ffbd1900 /var/lib/ceph/osd/ceph-4/block) open stat got: (1) Operation not permitted 2024-09-21T00:25:07.025 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:07.024+0000 7f5eea424640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-09-21T00:25:09.935 DEBUG:teuthology.orchestra.run.smithi187:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-21T00:25:09.965 DEBUG:teuthology.orchestra.run.smithi187:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T00:25:10.069 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:10.067+0000 7f830e0a5640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-09-21T00:25:10.069 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:10.067+0000 7f830e0a5640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-09-21T00:25:10.069 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:10.067+0000 7f830e0a5640 -1 bdev(0x559da0a1f900 /var/lib/ceph/osd/ceph-5/block) open stat got: (1) Operation not permitted 2024-09-21T00:25:10.069 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:10.068+0000 7f830e0a5640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-09-21T00:25:13.231 DEBUG:teuthology.orchestra.run.smithi187:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-21T00:25:13.260 DEBUG:teuthology.orchestra.run.smithi187:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T00:25:13.364 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:13.363+0000 7f9e0ce16640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-09-21T00:25:13.364 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:13.363+0000 7f9e0ce16640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-09-21T00:25:13.364 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:13.363+0000 7f9e0ce16640 -1 bdev(0x558cf0fa5900 /var/lib/ceph/osd/ceph-6/block) open stat got: (1) Operation not permitted 2024-09-21T00:25:13.364 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:13.363+0000 7f9e0ce16640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-09-21T00:25:14.425 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T00:25:14.428 DEBUG:teuthology.orchestra.run.smithi187:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T00:25:16.549 DEBUG:teuthology.orchestra.run.smithi187:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-09-21T00:25:16.577 DEBUG:teuthology.orchestra.run.smithi187:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T00:25:16.676 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:16.674+0000 7f55effdd640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-09-21T00:25:16.676 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:16.674+0000 7f55effdd640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-09-21T00:25:16.677 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:16.674+0000 7f55effdd640 -1 bdev(0x558548e3d900 /var/lib/ceph/osd/ceph-7/block) open stat got: (1) Operation not permitted 2024-09-21T00:25:16.677 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:16.674+0000 7f55effdd640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-09-21T00:25:19.598 DEBUG:teuthology.orchestra.run.smithi187:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-09-21T00:25:19.627 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-21T00:25:19.628 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T00:25:19.628 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-09-21T00:25:19.657 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T00:25:19.657 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-09-21T00:25:19.725 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T00:25:19.725 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-09-21T00:25:19.792 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T00:25:19.792 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-21T00:25:19.859 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T00:25:19.859 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-21T00:25:19.924 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T00:25:19.924 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-21T00:25:19.990 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T00:25:19.990 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-21T00:25:20.055 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-09-21T00:25:20.055 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-21T00:25:20.085 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-09-21T00:25:20.085 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-09-21T00:25:20.155 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-09-21T00:25:20.155 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-09-21T00:25:20.225 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-09-21T00:25:20.225 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-09-21T00:25:20.293 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-09-21T00:25:20.293 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-09-21T00:25:20.360 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-09-21T00:25:20.360 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-09-21T00:25:20.426 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-09-21T00:25:20.427 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-09-21T00:25:20.493 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T00:25:20.493 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-21T00:25:20.509 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T00:25:20.509 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-09-21T00:25:20.564 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-09-21T00:25:20.564 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-09-21T00:25:20.581 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-09-21T00:25:20.581 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-09-21T00:25:20.637 INFO:tasks.ceph:Adding keys to all mons... 2024-09-21T00:25:20.637 DEBUG:teuthology.orchestra.run.smithi112:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-21T00:25:20.639 DEBUG:teuthology.orchestra.run.smithi187:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-21T00:25:20.680 INFO:teuthology.orchestra.run.smithi112.stdout:[mgr.y] 2024-09-21T00:25:20.680 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQBOEu5myGjKNRAA9cMDObZYYOXMGdv2mOXkJA== 2024-09-21T00:25:20.680 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.a] 2024-09-21T00:25:20.680 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQBPEu5m3capAhAAczRM4L8ldX985NW4ep3w5Q== 2024-09-21T00:25:20.680 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.c] 2024-09-21T00:25:20.680 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQBPEu5mXNVRChAA1qsPbEybmY16jw5Jwk/v1A== 2024-09-21T00:25:20.681 INFO:teuthology.orchestra.run.smithi112.stdout:[osd.0] 2024-09-21T00:25:20.681 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQBSEu5moQg4OBAAi8mzLx7tWnORlrL64/MWlw== 2024-09-21T00:25:20.681 INFO:teuthology.orchestra.run.smithi112.stdout:[osd.1] 2024-09-21T00:25:20.681 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQBWEu5mxfobDhAAi7I6LYq9udB0W8bnYaBY9Q== 2024-09-21T00:25:20.681 INFO:teuthology.orchestra.run.smithi112.stdout:[osd.2] 2024-09-21T00:25:20.681 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQBZEu5mDqC/EhAAwzTbj22ocR4xJ8DMJvN7Lg== 2024-09-21T00:25:20.681 INFO:teuthology.orchestra.run.smithi112.stdout:[osd.3] 2024-09-21T00:25:20.681 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQBcEu5mwweGJBAAT3hZn5pto0aWt6BwGuaqcw== 2024-09-21T00:25:20.681 INFO:teuthology.orchestra.run.smithi112.stdout:[mgr.x] 2024-09-21T00:25:20.681 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQBOEu5mjGpDOhAAJFudb2wZlOBjo8Ss65Q+eg== 2024-09-21T00:25:20.681 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.b] 2024-09-21T00:25:20.681 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQBPEu5mxlDlDxAAtqngbTOhPqXDjFhJpSCLHg== 2024-09-21T00:25:20.681 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.d] 2024-09-21T00:25:20.681 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQBPEu5mnzySFxAAX3VfTgGYGcDweI7rPf3xvw== 2024-09-21T00:25:20.682 INFO:teuthology.orchestra.run.smithi112.stdout:[osd.4] 2024-09-21T00:25:20.682 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQBjEu5mIPd3ARAAkxTYphn38uKh48UUVEB+Fg== 2024-09-21T00:25:20.682 INFO:teuthology.orchestra.run.smithi112.stdout:[osd.5] 2024-09-21T00:25:20.682 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQBmEu5mP9IVBBAAftfK5V1K4vYRyFyJS2UzKQ== 2024-09-21T00:25:20.682 INFO:teuthology.orchestra.run.smithi112.stdout:[osd.6] 2024-09-21T00:25:20.682 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQBpEu5mmjiyFRAAbiW2M8JE0jJOaNg1u+1kgw== 2024-09-21T00:25:20.682 INFO:teuthology.orchestra.run.smithi112.stdout:[osd.7] 2024-09-21T00:25:20.682 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQBsEu5moklPKBAA6hzVvgvpmZMkPaFFce4mqw== 2024-09-21T00:25:20.682 INFO:teuthology.orchestra.run.smithi112.stdout:[client.0] 2024-09-21T00:25:20.682 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQBPEu5m49NPHBAAl/rcjiE8xYI+XdWwvZ6FOw== 2024-09-21T00:25:20.682 INFO:teuthology.orchestra.run.smithi112.stdout:[client.2] 2024-09-21T00:25:20.682 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQBPEu5mreUUIBAAELeN1N58srw6GqBrj8UOoQ== 2024-09-21T00:25:20.682 INFO:teuthology.orchestra.run.smithi112.stdout:[client.1] 2024-09-21T00:25:20.682 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQBPEu5mG33aIxAAirQhTlVyF/pRJx93T/0bIQ== 2024-09-21T00:25:20.683 INFO:teuthology.orchestra.run.smithi112.stdout:[client.3] 2024-09-21T00:25:20.683 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQBPEu5mcTmiJxAAY6bmTtow0R0heJHjx0ahDQ== 2024-09-21T00:25:20.702 INFO:teuthology.orchestra.run.smithi187.stdout:[mgr.y] 2024-09-21T00:25:20.702 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQBOEu5myGjKNRAA9cMDObZYYOXMGdv2mOXkJA== 2024-09-21T00:25:20.702 INFO:teuthology.orchestra.run.smithi187.stdout:[mds.a] 2024-09-21T00:25:20.702 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQBPEu5m3capAhAAczRM4L8ldX985NW4ep3w5Q== 2024-09-21T00:25:20.702 INFO:teuthology.orchestra.run.smithi187.stdout:[mds.c] 2024-09-21T00:25:20.702 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQBPEu5mXNVRChAA1qsPbEybmY16jw5Jwk/v1A== 2024-09-21T00:25:20.702 INFO:teuthology.orchestra.run.smithi187.stdout:[osd.0] 2024-09-21T00:25:20.702 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQBSEu5moQg4OBAAi8mzLx7tWnORlrL64/MWlw== 2024-09-21T00:25:20.702 INFO:teuthology.orchestra.run.smithi187.stdout:[osd.1] 2024-09-21T00:25:20.702 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQBWEu5mxfobDhAAi7I6LYq9udB0W8bnYaBY9Q== 2024-09-21T00:25:20.703 INFO:teuthology.orchestra.run.smithi187.stdout:[osd.2] 2024-09-21T00:25:20.703 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQBZEu5mDqC/EhAAwzTbj22ocR4xJ8DMJvN7Lg== 2024-09-21T00:25:20.703 INFO:teuthology.orchestra.run.smithi187.stdout:[osd.3] 2024-09-21T00:25:20.703 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQBcEu5mwweGJBAAT3hZn5pto0aWt6BwGuaqcw== 2024-09-21T00:25:20.703 INFO:teuthology.orchestra.run.smithi187.stdout:[mgr.x] 2024-09-21T00:25:20.703 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQBOEu5mjGpDOhAAJFudb2wZlOBjo8Ss65Q+eg== 2024-09-21T00:25:20.703 INFO:teuthology.orchestra.run.smithi187.stdout:[mds.b] 2024-09-21T00:25:20.703 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQBPEu5mxlDlDxAAtqngbTOhPqXDjFhJpSCLHg== 2024-09-21T00:25:20.704 INFO:teuthology.orchestra.run.smithi187.stdout:[mds.d] 2024-09-21T00:25:20.704 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQBPEu5mnzySFxAAX3VfTgGYGcDweI7rPf3xvw== 2024-09-21T00:25:20.704 INFO:teuthology.orchestra.run.smithi187.stdout:[osd.4] 2024-09-21T00:25:20.704 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQBjEu5mIPd3ARAAkxTYphn38uKh48UUVEB+Fg== 2024-09-21T00:25:20.704 INFO:teuthology.orchestra.run.smithi187.stdout:[osd.5] 2024-09-21T00:25:20.704 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQBmEu5mP9IVBBAAftfK5V1K4vYRyFyJS2UzKQ== 2024-09-21T00:25:20.704 INFO:teuthology.orchestra.run.smithi187.stdout:[osd.6] 2024-09-21T00:25:20.704 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQBpEu5mmjiyFRAAbiW2M8JE0jJOaNg1u+1kgw== 2024-09-21T00:25:20.704 INFO:teuthology.orchestra.run.smithi187.stdout:[osd.7] 2024-09-21T00:25:20.704 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQBsEu5moklPKBAA6hzVvgvpmZMkPaFFce4mqw== 2024-09-21T00:25:20.704 INFO:teuthology.orchestra.run.smithi187.stdout:[client.0] 2024-09-21T00:25:20.704 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQBPEu5m49NPHBAAl/rcjiE8xYI+XdWwvZ6FOw== 2024-09-21T00:25:20.704 INFO:teuthology.orchestra.run.smithi187.stdout:[client.2] 2024-09-21T00:25:20.704 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQBPEu5mreUUIBAAELeN1N58srw6GqBrj8UOoQ== 2024-09-21T00:25:20.705 INFO:teuthology.orchestra.run.smithi187.stdout:[client.1] 2024-09-21T00:25:20.705 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQBPEu5mG33aIxAAirQhTlVyF/pRJx93T/0bIQ== 2024-09-21T00:25:20.705 INFO:teuthology.orchestra.run.smithi187.stdout:[client.3] 2024-09-21T00:25:20.705 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQBPEu5mcTmiJxAAY6bmTtow0R0heJHjx0ahDQ== 2024-09-21T00:25:20.706 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-21T00:25:20.726 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-21T00:25:20.808 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-21T00:25:20.810 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-21T00:25:20.875 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-21T00:25:20.877 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-21T00:25:20.941 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T00:25:20.943 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T00:25:21.004 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T00:25:21.007 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T00:25:21.067 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T00:25:21.069 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T00:25:21.130 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T00:25:21.133 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T00:25:21.195 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-21T00:25:21.197 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-21T00:25:21.259 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-21T00:25:21.261 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-21T00:25:21.325 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-21T00:25:21.327 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-21T00:25:21.391 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T00:25:21.393 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T00:25:21.454 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T00:25:21.456 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T00:25:21.517 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T00:25:21.520 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T00:25:21.581 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T00:25:21.583 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T00:25:21.644 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-21T00:25:21.646 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-21T00:25:21.710 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-21T00:25:21.712 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-21T00:25:21.777 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-21T00:25:21.779 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-21T00:25:21.843 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-21T00:25:21.846 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-21T00:25:21.910 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-21T00:25:21.910 DEBUG:teuthology.orchestra.run.smithi112:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-21T00:25:21.934 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-21T00:25:22.040 DEBUG:teuthology.orchestra.run.smithi112:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-21T00:25:22.066 DEBUG:teuthology.orchestra.run.smithi187:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-21T00:25:22.091 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-21T00:25:22.198 DEBUG:teuthology.orchestra.run.smithi187:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-21T00:25:22.222 DEBUG:teuthology.orchestra.run.smithi187:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-21T00:25:22.287 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-21T00:25:22.395 DEBUG:teuthology.orchestra.run.smithi187:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-21T00:25:22.420 DEBUG:teuthology.orchestra.run.smithi112:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-21T00:25:22.422 DEBUG:teuthology.orchestra.run.smithi187:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-21T00:25:22.475 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-21T00:25:22.475 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-21T00:25:22.475 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-21T00:25:22.478 INFO:tasks.ceph.mon.a:Started 2024-09-21T00:25:22.478 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-21T00:25:22.478 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-21T00:25:22.518 INFO:tasks.ceph.mon.b:Started 2024-09-21T00:25:22.518 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-21T00:25:22.518 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-21T00:25:22.520 INFO:tasks.ceph.mon.c:Started 2024-09-21T00:25:22.520 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-21T00:25:22.520 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-09-21T00:25:22.520 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-09-21T00:25:22.522 INFO:tasks.ceph.mgr.y:Started 2024-09-21T00:25:22.522 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-21T00:25:22.523 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-21T00:25:22.525 INFO:tasks.ceph.mgr.x:Started 2024-09-21T00:25:22.525 INFO:tasks.ceph:setting config [client] client_mount_timeout = 600 2024-09-21T00:25:22.525 DEBUG:teuthology.orchestra.run.smithi112:> sudo ceph --cluster ceph config set client client_mount_timeout 600 2024-09-21T00:25:22.527 INFO:tasks.ceph:setting config [client] debug_client = 20 2024-09-21T00:25:22.527 DEBUG:teuthology.orchestra.run.smithi112:> sudo ceph --cluster ceph config set client debug_client 20 2024-09-21T00:25:22.571 INFO:tasks.ceph:setting config [client] debug_ms = 1 2024-09-21T00:25:22.571 DEBUG:teuthology.orchestra.run.smithi112:> sudo ceph --cluster ceph config set client debug_ms 1 2024-09-21T00:25:22.614 INFO:tasks.ceph:setting config [client] rados_mon_op_timeout = 900 2024-09-21T00:25:22.615 DEBUG:teuthology.orchestra.run.smithi112:> sudo ceph --cluster ceph config set client rados_mon_op_timeout 900 2024-09-21T00:25:22.658 INFO:tasks.ceph:setting config [client] rados_osd_op_timeout = 900 2024-09-21T00:25:22.658 DEBUG:teuthology.orchestra.run.smithi112:> sudo ceph --cluster ceph config set client rados_osd_op_timeout 900 2024-09-21T00:25:22.701 INFO:tasks.ceph:setting config [mds] debug_mds = 20 2024-09-21T00:25:22.702 DEBUG:teuthology.orchestra.run.smithi112:> sudo ceph --cluster ceph config set mds debug_mds 20 2024-09-21T00:25:22.744 INFO:tasks.ceph:setting config [mds] debug_mds_balancer = 20 2024-09-21T00:25:22.745 DEBUG:teuthology.orchestra.run.smithi112:> sudo ceph --cluster ceph config set mds debug_mds_balancer 20 2024-09-21T00:25:22.787 INFO:tasks.ceph:setting config [mds] debug_ms = 2 2024-09-21T00:25:22.788 DEBUG:teuthology.orchestra.run.smithi112:> sudo ceph --cluster ceph config set mds debug_ms 2 2024-09-21T00:25:22.830 INFO:tasks.ceph:setting config [mds] mds_debug_frag = True 2024-09-21T00:25:22.831 DEBUG:teuthology.orchestra.run.smithi112:> sudo ceph --cluster ceph config set mds mds_debug_frag True 2024-09-21T00:25:22.876 INFO:tasks.ceph:setting config [mds] mds_debug_scatterstat = True 2024-09-21T00:25:22.877 DEBUG:teuthology.orchestra.run.smithi112:> sudo ceph --cluster ceph config set mds mds_debug_scatterstat True 2024-09-21T00:25:22.920 INFO:tasks.ceph:setting config [mds] mds_op_complaint_time = 180 2024-09-21T00:25:22.921 DEBUG:teuthology.orchestra.run.smithi112:> sudo ceph --cluster ceph config set mds mds_op_complaint_time 180 2024-09-21T00:25:22.963 INFO:tasks.ceph:setting config [mds] mds_verify_scatter = True 2024-09-21T00:25:22.964 DEBUG:teuthology.orchestra.run.smithi112:> sudo ceph --cluster ceph config set mds mds_verify_scatter True 2024-09-21T00:25:23.007 INFO:tasks.ceph:setting config [mds] osd_op_complaint_time = 180 2024-09-21T00:25:23.007 DEBUG:teuthology.orchestra.run.smithi112:> sudo ceph --cluster ceph config set mds osd_op_complaint_time 180 2024-09-21T00:25:23.020 INFO:tasks.ceph:setting config [mds] rados_mon_op_timeout = 900 2024-09-21T00:25:23.020 DEBUG:teuthology.orchestra.run.smithi112:> sudo ceph --cluster ceph config set mds rados_mon_op_timeout 900 2024-09-21T00:25:23.043 INFO:tasks.ceph:setting config [mds] rados_osd_op_timeout = 900 2024-09-21T00:25:23.043 DEBUG:teuthology.orchestra.run.smithi112:> sudo ceph --cluster ceph config set mds rados_osd_op_timeout 900 2024-09-21T00:25:23.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.037+0000 7f3eb956e640 1 -- 172.21.15.112:0/2464809518 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3eb4155410 msgr2=0x7f3eb40ca1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:23.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.037+0000 7f3eb956e640 1 --2- 172.21.15.112:0/2464809518 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3eb4155410 0x7f3eb40ca1a0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3ea8004000 tx=0x7f3ea80012f0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.037+0000 7f3eb956e640 1 -- 172.21.15.112:0/2464809518 shutdown_connections 2024-09-21T00:25:23.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.037+0000 7f3eb956e640 1 --2- 172.21.15.112:0/2464809518 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3eb415ed20 0x7f3eb415f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.037+0000 7f3eb956e640 1 --2- 172.21.15.112:0/2464809518 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3eb4155410 0x7f3eb40ca1a0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.037+0000 7f3eb956e640 1 --2- 172.21.15.112:0/2464809518 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3e9c004800 0x7f3eb41f26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.037+0000 7f3eb956e640 1 -- 172.21.15.112:0/2464809518 >> 172.21.15.112:0/2464809518 conn(0x7f3eb40735d0 msgr2=0x7f3eb406de20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:23.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.038+0000 7f3eb956e640 1 -- 172.21.15.112:0/2464809518 shutdown_connections 2024-09-21T00:25:23.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.038+0000 7f3eb956e640 1 -- 172.21.15.112:0/2464809518 wait complete. 2024-09-21T00:25:23.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.041+0000 7f276ffff640 1 -- 172.21.15.112:0/1277610544 <== mon.2 v2:172.21.15.187:3301/0 6 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f2778043b90 con 0x7f2788162450 2024-09-21T00:25:23.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.041+0000 7f278e07f640 1 -- 172.21.15.112:0/1277610544 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2768005290 con 0x7f2788162450 2024-09-21T00:25:23.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.044+0000 7f7d75ffb640 1 -- 172.21.15.112:0/3130920667 <== mon.0 v2:172.21.15.112:3300/0 11 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v6)=0 v6) ==== 123+0+0 (secure 0 0 0) 0x7f7d70021430 con 0x7f7d8006def0 2024-09-21T00:25:23.051 INFO:tasks.ceph:setting config [mgr] client_mount_timeout = 30 2024-09-21T00:25:23.051 DEBUG:teuthology.orchestra.run.smithi112:> sudo ceph --cluster ceph config set mgr client_mount_timeout 30 2024-09-21T00:25:23.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.046+0000 7fa64cff9640 1 -- 172.21.15.112:0/2731584306 <== mon.0 v2:172.21.15.112:3300/0 11 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa64000f3e0 con 0x7fa6500cddc0 2024-09-21T00:25:23.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.048+0000 7f276ffff640 1 -- 172.21.15.112:0/1277610544 <== mon.2 v2:172.21.15.187:3301/0 7 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f2778043de0 con 0x7f2788162450 2024-09-21T00:25:23.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.048+0000 7f276ffff640 1 -- 172.21.15.112:0/1277610544 <== mon.2 v2:172.21.15.187:3301/0 8 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2778004e40 con 0x7f2788162450 2024-09-21T00:25:23.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.049+0000 7f7d75ffb640 1 -- 172.21.15.112:0/3130920667 <== mon.0 v2:172.21.15.112:3300/0 12 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7d70021690 con 0x7f7d8006def0 2024-09-21T00:25:23.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.053+0000 7ffb8847f640 1 -- 172.21.15.112:0/4091616542 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb801516d0 msgr2=0x7ffb8015b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:23.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.053+0000 7ffb8847f640 1 --2- 172.21.15.112:0/4091616542 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb801516d0 0x7ffb8015b540 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ffb800b5250 tx=0x7ffb7c02b1c0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.057 INFO:tasks.ceph:setting config [mgr] debug_client = 20 2024-09-21T00:25:23.057 DEBUG:teuthology.orchestra.run.smithi112:> sudo ceph --cluster ceph config set mgr debug_client 20 2024-09-21T00:25:23.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.053+0000 7f7d878de640 1 -- 172.21.15.112:0/3130920667 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d8006def0 msgr2=0x7f7d80145120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:23.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.053+0000 7f7d878de640 1 --2- 172.21.15.112:0/3130920667 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d8006def0 0x7f7d80145120 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f7d7000e9c0 tx=0x7f7d7000ee80 comp rx=0 tx=0).stop 2024-09-21T00:25:23.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.054+0000 7f7d878de640 1 -- 172.21.15.112:0/3130920667 shutdown_connections 2024-09-21T00:25:23.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.054+0000 7f7d878de640 1 --2- 172.21.15.112:0/3130920667 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7d8015bfc0 0x7f7d8015e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.054+0000 7f7d878de640 1 --2- 172.21.15.112:0/3130920667 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7d800b5260 0x7f7d8015dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.054+0000 7f7d878de640 1 --2- 172.21.15.112:0/3130920667 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d8006def0 0x7f7d80145120 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.054+0000 7f7d878de640 1 -- 172.21.15.112:0/3130920667 >> 172.21.15.112:0/3130920667 conn(0x7f7d800cdca0 msgr2=0x7f7d800712c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:23.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.055+0000 7f7d878de640 1 -- 172.21.15.112:0/3130920667 shutdown_connections 2024-09-21T00:25:23.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.055+0000 7f7d878de640 1 -- 172.21.15.112:0/3130920667 wait complete. 2024-09-21T00:25:23.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.058+0000 7ffb8847f640 1 -- 172.21.15.112:0/4091616542 shutdown_connections 2024-09-21T00:25:23.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.058+0000 7ffb8847f640 1 --2- 172.21.15.112:0/4091616542 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb801516d0 0x7ffb8015b540 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.058+0000 7ffb8847f640 1 -- 172.21.15.112:0/4091616542 >> 172.21.15.112:0/4091616542 conn(0x7ffb8006cde0 msgr2=0x7ffb8006d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:23.064 INFO:tasks.ceph:setting config [mgr] debug_mgr = 20 2024-09-21T00:25:23.064 DEBUG:teuthology.orchestra.run.smithi112:> sudo ceph --cluster ceph config set mgr debug_mgr 20 2024-09-21T00:25:23.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.068+0000 7ffb8847f640 1 -- 172.21.15.112:0/4091616542 shutdown_connections 2024-09-21T00:25:23.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.068+0000 7ffb8847f640 1 -- 172.21.15.112:0/4091616542 wait complete. 2024-09-21T00:25:23.071 INFO:tasks.ceph:setting config [mgr] debug_ms = 1 2024-09-21T00:25:23.071 DEBUG:teuthology.orchestra.run.smithi112:> sudo ceph --cluster ceph config set mgr debug_ms 1 2024-09-21T00:25:23.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.071+0000 7ffb8847f640 1 Processor -- start 2024-09-21T00:25:23.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.071+0000 7ffb8847f640 1 -- start start 2024-09-21T00:25:23.074 INFO:tasks.ceph:setting config [mgr] mon_warn_on_pool_no_app = False 2024-09-21T00:25:23.074 DEBUG:teuthology.orchestra.run.smithi112:> sudo ceph --cluster ceph config set mgr mon_warn_on_pool_no_app False 2024-09-21T00:25:23.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.074+0000 7ffb8847f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb8006f180 0x7ffb80154700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:23.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.077+0000 7ffb8621b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb8006f180 0x7ffb80154700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:23.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.077+0000 7ffb8621b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb8006f180 0x7ffb80154700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55484/0 (socket says 172.21.15.112:55484) 2024-09-21T00:25:23.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.077+0000 7ffb8621b640 1 -- 172.21.15.112:0/2683542254 learned_addr learned my addr 172.21.15.112:0/2683542254 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:25:23.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.077+0000 7ffb8847f640 1 --2- 172.21.15.112:0/2683542254 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffb801516d0 0x7ffb80152ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:23.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.077+0000 7ffb8847f640 1 --2- 172.21.15.112:0/2683542254 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffb8015bcf0 0x7ffb80138860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:23.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.077+0000 7ffb8847f640 1 -- 172.21.15.112:0/2683542254 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ffb80070540 con 0x7ffb8006f180 2024-09-21T00:25:23.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.077+0000 7ffb8847f640 1 -- 172.21.15.112:0/2683542254 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ffb800703c0 con 0x7ffb801516d0 2024-09-21T00:25:23.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.077+0000 7ffb8847f640 1 -- 172.21.15.112:0/2683542254 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ffb80070d30 con 0x7ffb8015bcf0 2024-09-21T00:25:23.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.077+0000 7ffb86a1c640 1 --2- 172.21.15.112:0/2683542254 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffb8015bcf0 0x7ffb80138860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:23.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.077+0000 7ffb85a1a640 1 --2- 172.21.15.112:0/2683542254 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffb801516d0 0x7ffb80152ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:23.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.082+0000 7ffb86a1c640 1 -- 172.21.15.112:0/2683542254 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffb801516d0 msgr2=0x7ffb80152ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:23.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.082+0000 7ffb86a1c640 1 --2- 172.21.15.112:0/2683542254 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffb801516d0 0x7ffb80152ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.082+0000 7ffb86a1c640 1 -- 172.21.15.112:0/2683542254 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb8006f180 msgr2=0x7ffb80154700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:23.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.082+0000 7ffb86a1c640 1 --2- 172.21.15.112:0/2683542254 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb8006f180 0x7ffb80154700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.082+0000 7ffb86a1c640 1 -- 172.21.15.112:0/2683542254 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb801531e0 con 0x7ffb8015bcf0 2024-09-21T00:25:23.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.082+0000 7ffb8621b640 1 --2- 172.21.15.112:0/2683542254 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb8006f180 0x7ffb80154700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:25:23.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.083+0000 7ffb85a1a640 1 --2- 172.21.15.112:0/2683542254 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffb801516d0 0x7ffb80152ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:25:23.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.083+0000 7ffb86a1c640 1 --2- 172.21.15.112:0/2683542254 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffb8015bcf0 0x7ffb80138860 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ffb7800ca70 tx=0x7ffb7800cf30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:25:23.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.085+0000 7ffb76ffd640 1 -- 172.21.15.112:0/2683542254 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb78016020 con 0x7ffb8015bcf0 2024-09-21T00:25:23.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.085+0000 7ffb76ffd640 1 -- 172.21.15.112:0/2683542254 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffb7800ac00 con 0x7ffb8015bcf0 2024-09-21T00:25:23.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.085+0000 7ffb8847f640 1 -- 172.21.15.112:0/2683542254 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb80153660 con 0x7ffb8015bcf0 2024-09-21T00:25:23.089 INFO:tasks.ceph:setting config [mon] mon_op_complaint_time = 120 2024-09-21T00:25:23.089 DEBUG:teuthology.orchestra.run.smithi112:> sudo ceph --cluster ceph config set mon mon_op_complaint_time 120 2024-09-21T00:25:23.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.085+0000 7ffb8847f640 1 -- 172.21.15.112:0/2683542254 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb80153c20 con 0x7ffb8015bcf0 2024-09-21T00:25:23.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.087+0000 7ffb76ffd640 1 -- 172.21.15.112:0/2683542254 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb78012420 con 0x7ffb8015bcf0 2024-09-21T00:25:23.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.088+0000 7ffb76ffd640 1 -- 172.21.15.112:0/2683542254 <== mon.2 v2:172.21.15.187:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ffb78019020 con 0x7ffb8015bcf0 2024-09-21T00:25:23.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.088+0000 7ffb76ffd640 1 -- 172.21.15.112:0/2683542254 <== mon.2 v2:172.21.15.187:3301/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ffb780052d0 con 0x7ffb8015bcf0 2024-09-21T00:25:23.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.088+0000 7ffb74ff9640 1 -- 172.21.15.112:0/2683542254 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb801533b0 con 0x7ffb8015bcf0 2024-09-21T00:25:23.104 INFO:tasks.ceph:setting config [osd] osd_op_complaint_time = 180 2024-09-21T00:25:23.104 DEBUG:teuthology.orchestra.run.smithi112:> sudo ceph --cluster ceph config set osd osd_op_complaint_time 180 2024-09-21T00:25:23.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.093+0000 7ffb76ffd640 1 -- 172.21.15.112:0/2683542254 <== mon.2 v2:172.21.15.187:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7ffb78020030 con 0x7ffb8015bcf0 2024-09-21T00:25:23.151 DEBUG:tasks.ceph:set 22 configs 2024-09-21T00:25:23.151 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'client_mount_timeout', '600'], check_status=True, hostname='smithi112') 2024-09-21T00:25:23.151 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'debug_client', '20'], check_status=True, hostname='smithi112') 2024-09-21T00:25:23.151 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'debug_ms', '1'], check_status=True, hostname='smithi112') 2024-09-21T00:25:23.151 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi112') 2024-09-21T00:25:23.151 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi112') 2024-09-21T00:25:23.151 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'debug_mds', '20'], check_status=True, hostname='smithi112') 2024-09-21T00:25:23.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.161+0000 7f3665aa5640 1 -- 172.21.15.112:0/619066912 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3658055280 msgr2=0x7f3658055d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:23.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.161+0000 7f3665aa5640 1 --2- 172.21.15.112:0/619066912 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3658055280 0x7f3658055d10 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f3660052860 tx=0x7f36600a63e0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.164+0000 7f3665aa5640 1 -- 172.21.15.112:0/619066912 shutdown_connections 2024-09-21T00:25:23.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.164+0000 7f3665aa5640 1 --2- 172.21.15.112:0/619066912 >> v2:172.21.15.187:3300/0 conn(0x7f3658036770 0x7f3658034d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.164+0000 7f3665aa5640 1 --2- 172.21.15.112:0/619066912 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3658055280 0x7f3658055d10 secure :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f3660052860 tx=0x7f36600a63e0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.164+0000 7f3665aa5640 1 --2- 172.21.15.112:0/619066912 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f36580091c0 0x7f36580557d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.164+0000 7f3665aa5640 1 -- 172.21.15.112:0/619066912 >> 172.21.15.112:0/619066912 conn(0x7f365801eae0 msgr2=0x7f365803a710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:23.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.171+0000 7f3665aa5640 1 -- 172.21.15.112:0/619066912 shutdown_connections 2024-09-21T00:25:23.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.171+0000 7f3665aa5640 1 -- 172.21.15.112:0/619066912 wait complete. 2024-09-21T00:25:23.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.174+0000 7f3665aa5640 1 Processor -- start 2024-09-21T00:25:23.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.174+0000 7f3665aa5640 1 -- start start 2024-09-21T00:25:23.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.177+0000 7f3665aa5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f36580091c0 0x7f36580d8d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:23.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.177+0000 7f3665aa5640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3658036770 0x7f3658033960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:23.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.177+0000 7f3665aa5640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3658033ea0 0x7f36580e1960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:23.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.177+0000 7f3665aa5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3658037050 con 0x7f36580091c0 2024-09-21T00:25:23.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.177+0000 7f3665aa5640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3658036ed0 con 0x7f3658036770 2024-09-21T00:25:23.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.177+0000 7f3665aa5640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3658035290 con 0x7f3658033ea0 2024-09-21T00:25:23.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.177+0000 7f3664aa3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f36580091c0 0x7f36580d8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:23.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.178+0000 7f3664aa3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f36580091c0 0x7f36580d8d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55508/0 (socket says 172.21.15.112:55508) 2024-09-21T00:25:23.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.178+0000 7f3664aa3640 1 -- 172.21.15.112:0/1458517412 learned_addr learned my addr 172.21.15.112:0/1458517412 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:25:23.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.179+0000 7f3664aa3640 1 -- 172.21.15.112:0/1458517412 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3658033ea0 msgr2=0x7f36580e1960 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T00:25:23.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.179+0000 7f3664aa3640 1 --2- 172.21.15.112:0/1458517412 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3658033ea0 0x7f36580e1960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.179+0000 7f3664aa3640 1 -- 172.21.15.112:0/1458517412 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3658036770 msgr2=0x7f3658033960 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T00:25:23.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.179+0000 7f3664aa3640 1 --2- 172.21.15.112:0/1458517412 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3658036770 0x7f3658033960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.179+0000 7f3664aa3640 1 -- 172.21.15.112:0/1458517412 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3658181eb0 con 0x7f36580091c0 2024-09-21T00:25:23.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.179+0000 7f3664aa3640 1 --2- 172.21.15.112:0/1458517412 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f36580091c0 0x7f36580d8d10 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f364c00b7b0 tx=0x7f364c00bc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:25:23.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.180+0000 7f365d7fa640 1 -- 172.21.15.112:0/1458517412 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f364c019070 con 0x7f36580091c0 2024-09-21T00:25:23.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.180+0000 7f3665aa5640 1 -- 172.21.15.112:0/1458517412 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3658182190 con 0x7f36580091c0 2024-09-21T00:25:23.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.180+0000 7f3665aa5640 1 -- 172.21.15.112:0/1458517412 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3658182750 con 0x7f36580091c0 2024-09-21T00:25:23.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.182+0000 7f364affd640 1 -- 172.21.15.112:0/1458517412 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3658035860 con 0x7f36580091c0 2024-09-21T00:25:23.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.188+0000 7f365d7fa640 1 -- 172.21.15.112:0/1458517412 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f364c00c830 con 0x7f36580091c0 2024-09-21T00:25:23.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.188+0000 7f365d7fa640 1 -- 172.21.15.112:0/1458517412 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f364c00cc80 con 0x7f36580091c0 2024-09-21T00:25:23.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.188+0000 7f365d7fa640 1 -- 172.21.15.112:0/1458517412 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f364c01c070 con 0x7f36580091c0 2024-09-21T00:25:23.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.188+0000 7f365d7fa640 1 -- 172.21.15.112:0/1458517412 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f364c01e8a0 con 0x7f36580091c0 2024-09-21T00:25:23.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.188+0000 7f365d7fa640 1 -- 172.21.15.112:0/1458517412 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f364c023020 con 0x7f36580091c0 2024-09-21T00:25:23.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.262+0000 7fda33fff640 1 -- 172.21.15.112:0/151340953 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fda1c003a80 msgr2=0x7fda1c003ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:23.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.262+0000 7fda33fff640 1 --2- 172.21.15.112:0/151340953 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fda1c003a80 0x7fda1c003ed0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fda2c00c640 tx=0x7fda1c005830 comp rx=0 tx=0).stop 2024-09-21T00:25:23.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.264+0000 7fda33fff640 1 -- 172.21.15.112:0/151340953 shutdown_connections 2024-09-21T00:25:23.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.264+0000 7fda33fff640 1 --2- 172.21.15.112:0/151340953 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fda1c003a80 0x7fda1c003ed0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.264+0000 7fda33fff640 1 -- 172.21.15.112:0/151340953 >> 172.21.15.112:0/151340953 conn(0x7fda34073460 msgr2=0x7fda34073870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:23.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.266+0000 7fda33fff640 1 -- 172.21.15.112:0/151340953 shutdown_connections 2024-09-21T00:25:23.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.266+0000 7fda33fff640 1 -- 172.21.15.112:0/151340953 wait complete. 2024-09-21T00:25:23.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.269+0000 7fda33fff640 1 Processor -- start 2024-09-21T00:25:23.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.269+0000 7fda33fff640 1 -- start start 2024-09-21T00:25:23.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.271+0000 7fda33fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda1c003a80 0x7fda340c9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:23.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.272+0000 7fda32ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda1c003a80 0x7fda340c9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:23.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.272+0000 7fda32ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda1c003a80 0x7fda340c9eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55520/0 (socket says 172.21.15.112:55520) 2024-09-21T00:25:23.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.272+0000 7fda32ffd640 1 -- 172.21.15.112:0/2664021158 learned_addr learned my addr 172.21.15.112:0/2664021158 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:25:23.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.272+0000 7fda33fff640 1 --2- 172.21.15.112:0/2664021158 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fda341f2520 0x7fda340c84e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:23.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.272+0000 7fda33fff640 1 --2- 172.21.15.112:0/2664021158 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fda341f4620 0x7fda341f4a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:23.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.272+0000 7fda33fff640 1 -- 172.21.15.112:0/2664021158 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fda3406f830 con 0x7fda1c003a80 2024-09-21T00:25:23.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.272+0000 7fda33fff640 1 -- 172.21.15.112:0/2664021158 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fda3406f6b0 con 0x7fda341f2520 2024-09-21T00:25:23.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.272+0000 7fda33fff640 1 -- 172.21.15.112:0/2664021158 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fda340696c0 con 0x7fda341f4620 2024-09-21T00:25:23.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.272+0000 7fda32ffd640 1 -- 172.21.15.112:0/2664021158 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fda341f4620 msgr2=0x7fda341f4a30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T00:25:23.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.272+0000 7fda32ffd640 1 --2- 172.21.15.112:0/2664021158 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fda341f4620 0x7fda341f4a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.272+0000 7fda32ffd640 1 -- 172.21.15.112:0/2664021158 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fda341f2520 msgr2=0x7fda340c84e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T00:25:23.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.272+0000 7fda32ffd640 1 --2- 172.21.15.112:0/2664021158 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fda341f2520 0x7fda340c84e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.272+0000 7fda32ffd640 1 -- 172.21.15.112:0/2664021158 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda341f4f70 con 0x7fda1c003a80 2024-09-21T00:25:23.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.273+0000 7fda32ffd640 1 --2- 172.21.15.112:0/2664021158 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda1c003a80 0x7fda340c9eb0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fda2c02b4a0 tx=0x7fda2c02bc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:25:23.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.273+0000 7fda1b7fe640 1 -- 172.21.15.112:0/2664021158 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda2c042070 con 0x7fda1c003a80 2024-09-21T00:25:23.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.273+0000 7fda1b7fe640 1 -- 172.21.15.112:0/2664021158 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fda2c02f040 con 0x7fda1c003a80 2024-09-21T00:25:23.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.273+0000 7fda1b7fe640 1 -- 172.21.15.112:0/2664021158 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda2c0064d0 con 0x7fda1c003a80 2024-09-21T00:25:23.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.273+0000 7fda33fff640 1 -- 172.21.15.112:0/2664021158 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda340c7440 con 0x7fda1c003a80 2024-09-21T00:25:23.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.274+0000 7fda33fff640 1 -- 172.21.15.112:0/2664021158 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda340c7a00 con 0x7fda1c003a80 2024-09-21T00:25:23.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.275+0000 7fda1b7fe640 1 -- 172.21.15.112:0/2664021158 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fda2c006670 con 0x7fda1c003a80 2024-09-21T00:25:23.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.275+0000 7fda1b7fe640 1 -- 172.21.15.112:0/2664021158 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fda2c034d30 con 0x7fda1c003a80 2024-09-21T00:25:23.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.276+0000 7fda33fff640 1 -- 172.21.15.112:0/2664021158 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda14005290 con 0x7fda1c003a80 2024-09-21T00:25:23.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.281+0000 7fda1b7fe640 1 -- 172.21.15.112:0/2664021158 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fda2c04b020 con 0x7fda1c003a80 2024-09-21T00:25:23.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.299+0000 7fa655622640 1 -- 172.21.15.112:0/2731584306 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command([{prefix=config set, name=debug_mds}] v 0) -- 0x7fa6280052f0 con 0x7fa6500cddc0 2024-09-21T00:25:23.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.308+0000 7fa64cff9640 1 -- 172.21.15.112:0/2731584306 <== mon.0 v2:172.21.15.112:3300/0 12 ==== mon_command_ack([{prefix=config set, name=debug_mds}]=0 v7)=0 v7) ==== 108+0+0 (secure 0 0 0) 0x7fa64000f670 con 0x7fa6500cddc0 2024-09-21T00:25:23.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.328+0000 7fa655622640 1 -- 172.21.15.112:0/2731584306 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6500cddc0 msgr2=0x7fa650158b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:23.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.328+0000 7fa655622640 1 --2- 172.21.15.112:0/2731584306 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6500cddc0 0x7fa650158b40 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fa64000c680 tx=0x7fa64000cb40 comp rx=0 tx=0).stop 2024-09-21T00:25:23.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.328+0000 7fa655622640 1 -- 172.21.15.112:0/2731584306 shutdown_connections 2024-09-21T00:25:23.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.328+0000 7fa655622640 1 --2- 172.21.15.112:0/2731584306 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa650155d20 0x7fa650159080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.328+0000 7fa655622640 1 --2- 172.21.15.112:0/2731584306 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6500cddc0 0x7fa650158b40 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.328+0000 7fa655622640 1 --2- 172.21.15.112:0/2731584306 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa6500743b0 0x7fa6501557e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.328+0000 7fa655622640 1 -- 172.21.15.112:0/2731584306 >> 172.21.15.112:0/2731584306 conn(0x7fa65006f920 msgr2=0x7fa6500cfcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:23.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.329+0000 7fa655622640 1 -- 172.21.15.112:0/2731584306 shutdown_connections 2024-09-21T00:25:23.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.329+0000 7fa655622640 1 -- 172.21.15.112:0/2731584306 wait complete. 2024-09-21T00:25:23.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.327+0000 7f278e07f640 1 -- 172.21.15.112:0/1277610544 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command([{prefix=config set, name=debug_mds_balancer}] v 0) -- 0x7f2768005580 con 0x7f2788162450 2024-09-21T00:25:23.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.345+0000 7f276ffff640 1 -- 172.21.15.112:0/1277610544 <== mon.2 v2:172.21.15.187:3301/0 9 ==== mon_command_ack([{prefix=config set, name=debug_mds_balancer}]=0 v8)=0 v8) ==== 117+0+0 (secure 0 0 0) 0x7f2778004ff0 con 0x7f2788162450 2024-09-21T00:25:23.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.349+0000 7f278e07f640 1 -- 172.21.15.112:0/1277610544 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2788162450 msgr2=0x7f27880b0cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:23.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.349+0000 7f278e07f640 1 --2- 172.21.15.112:0/1277610544 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2788162450 0x7f27880b0cf0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f2778031000 tx=0x7f2778005780 comp rx=0 tx=0).stop 2024-09-21T00:25:23.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.354+0000 7f278e07f640 1 -- 172.21.15.112:0/1277610544 shutdown_connections 2024-09-21T00:25:23.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.354+0000 7f278e07f640 1 --2- 172.21.15.112:0/1277610544 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2788162450 0x7f27880b0cf0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.354+0000 7f278e07f640 1 --2- 172.21.15.112:0/1277610544 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f27880cc160 0x7f27881facc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.354+0000 7f278e07f640 1 --2- 172.21.15.112:0/1277610544 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f27880aee70 0x7f27881fa650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.354+0000 7f278e07f640 1 -- 172.21.15.112:0/1277610544 >> 172.21.15.112:0/1277610544 conn(0x7f27880b1b90 msgr2=0x7f2788070c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:23.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.358+0000 7f278e07f640 1 -- 172.21.15.112:0/1277610544 shutdown_connections 2024-09-21T00:25:23.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.359+0000 7f278e07f640 1 -- 172.21.15.112:0/1277610544 wait complete. 2024-09-21T00:25:23.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.364+0000 7ff8bf369640 1 -- 172.21.15.112:0/1892202052 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff8b00550e0 msgr2=0x7ff8b0033ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:23.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.364+0000 7ff8bf369640 1 --2- 172.21.15.112:0/1892202052 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff8b00550e0 0x7ff8b0033ce0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff8b40098e0 tx=0x7ff8b402a610 comp rx=0 tx=0).stop 2024-09-21T00:25:23.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.364+0000 7ff8bf369640 1 -- 172.21.15.112:0/1892202052 shutdown_connections 2024-09-21T00:25:23.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.364+0000 7ff8bf369640 1 --2- 172.21.15.112:0/1892202052 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff8b0054130 0x7ff8b0034220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.364+0000 7ff8bf369640 1 --2- 172.21.15.112:0/1892202052 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff8b00550e0 0x7ff8b0033ce0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.364+0000 7ff8bf369640 1 -- 172.21.15.112:0/1892202052 >> 172.21.15.112:0/1892202052 conn(0x7ff8b001d9a0 msgr2=0x7ff8b0039660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:23.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.364+0000 7ff8bf369640 1 -- 172.21.15.112:0/1892202052 shutdown_connections 2024-09-21T00:25:23.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.364+0000 7ff8bf369640 1 -- 172.21.15.112:0/1892202052 wait complete. 2024-09-21T00:25:23.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.367+0000 7ff8bf369640 1 Processor -- start 2024-09-21T00:25:23.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.367+0000 7ff8bf369640 1 -- start start 2024-09-21T00:25:23.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.369+0000 7ff8bf369640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff8b00070b0 0x7ff8b0033700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:23.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.369+0000 7ff8bf369640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff8b0054130 0x7ff8b019ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:23.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.369+0000 7ff8bf369640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8b0103940 0x7ff8b0103d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:23.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.369+0000 7ff8bd105640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff8b00070b0 0x7ff8b0033700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:23.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.369+0000 7ff8bd105640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff8b00070b0 0x7ff8b0033700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:41104/0 (socket says 172.21.15.112:41104) 2024-09-21T00:25:23.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.369+0000 7ff8bc904640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff8b0054130 0x7ff8b019ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:23.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.369+0000 7ff8bc904640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff8b0054130 0x7ff8b019ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:36052/0 (socket says 172.21.15.112:36052) 2024-09-21T00:25:23.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.370+0000 7ff8bd906640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8b0103940 0x7ff8b0103d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:23.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.370+0000 7ff8bd906640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8b0103940 0x7ff8b0103d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55528/0 (socket says 172.21.15.112:55528) 2024-09-21T00:25:23.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.370+0000 7ff8bd105640 1 -- 172.21.15.112:0/1869010354 learned_addr learned my addr 172.21.15.112:0/1869010354 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:25:23.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.370+0000 7ff8bf369640 1 -- 172.21.15.112:0/1869010354 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b0006490 con 0x7ff8b0103940 2024-09-21T00:25:23.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.370+0000 7ff8bf369640 1 -- 172.21.15.112:0/1869010354 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b0006310 con 0x7ff8b0054130 2024-09-21T00:25:23.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.370+0000 7ff8bf369640 1 -- 172.21.15.112:0/1869010354 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff8b0006610 con 0x7ff8b00070b0 2024-09-21T00:25:23.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.379+0000 7ff8bc904640 1 -- 172.21.15.112:0/1869010354 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff8b00070b0 msgr2=0x7ff8b0033700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:23.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.379+0000 7ff8bc904640 1 --2- 172.21.15.112:0/1869010354 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff8b00070b0 0x7ff8b0033700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.379+0000 7ff8bc904640 1 -- 172.21.15.112:0/1869010354 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8b0103940 msgr2=0x7ff8b0103d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:23.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.379+0000 7ff8bc904640 1 --2- 172.21.15.112:0/1869010354 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8b0103940 0x7ff8b0103d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.379+0000 7ff8bc904640 1 -- 172.21.15.112:0/1869010354 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8b019d4c0 con 0x7ff8b0054130 2024-09-21T00:25:23.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.379+0000 7ff8bd906640 1 --2- 172.21.15.112:0/1869010354 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8b0103940 0x7ff8b0103d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:25:23.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.380+0000 7ff8bc904640 1 --2- 172.21.15.112:0/1869010354 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff8b0054130 0x7ff8b019ccb0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ff8b8076070 tx=0x7ff8b806a0e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:25:23.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.380+0000 7ff8adffb640 1 -- 172.21.15.112:0/1869010354 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8b80753c0 con 0x7ff8b0054130 2024-09-21T00:25:23.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.380+0000 7ff8adffb640 1 -- 172.21.15.112:0/1869010354 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff8b80762b0 con 0x7ff8b0054130 2024-09-21T00:25:23.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.380+0000 7ff8adffb640 1 -- 172.21.15.112:0/1869010354 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8b8076740 con 0x7ff8b0054130 2024-09-21T00:25:23.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.380+0000 7ff8bf369640 1 -- 172.21.15.112:0/1869010354 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8b019d7a0 con 0x7ff8b0054130 2024-09-21T00:25:23.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.380+0000 7ff8bf369640 1 -- 172.21.15.112:0/1869010354 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8b004eea0 con 0x7ff8b0054130 2024-09-21T00:25:23.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.380+0000 7ff8adffb640 1 -- 172.21.15.112:0/1869010354 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ff8b8082400 con 0x7ff8b0054130 2024-09-21T00:25:23.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.381+0000 7ff8adffb640 1 -- 172.21.15.112:0/1869010354 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff8b8081890 con 0x7ff8b0054130 2024-09-21T00:25:23.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.383+0000 7ff8bf369640 1 -- 172.21.15.112:0/1869010354 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8b00550e0 con 0x7ff8b0054130 2024-09-21T00:25:23.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.385+0000 7ff8adffb640 1 -- 172.21.15.112:0/1869010354 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7ff8b8089030 con 0x7ff8b0054130 2024-09-21T00:25:23.389 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'debug_mds_balancer', '20'], check_status=True, hostname='smithi112') 2024-09-21T00:25:23.405 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'debug_ms', '2'], check_status=True, hostname='smithi112') 2024-09-21T00:25:23.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.409+0000 7fb3dec20640 1 -- 172.21.15.112:0/501279623 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb3d80ca6f0 msgr2=0x7fb3d80cf6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:23.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.409+0000 7fb3dec20640 1 --2- 172.21.15.112:0/501279623 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb3d80ca6f0 0x7fb3d80cf6e0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fb3c800a7c0 tx=0x7fb3c802ca60 comp rx=0 tx=0).stop 2024-09-21T00:25:23.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.409+0000 7fb3dec20640 1 -- 172.21.15.112:0/501279623 shutdown_connections 2024-09-21T00:25:23.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.409+0000 7fb3dec20640 1 --2- 172.21.15.112:0/501279623 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb3d80cc0b0 0x7fb3d80cc4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.409+0000 7fb3dec20640 1 --2- 172.21.15.112:0/501279623 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb3d80ca6f0 0x7fb3d80cf6e0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.409+0000 7fb3dec20640 1 -- 172.21.15.112:0/501279623 >> 172.21.15.112:0/501279623 conn(0x7fb3d80b1b90 msgr2=0x7fb3d80b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:23.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.414+0000 7fb3dec20640 1 -- 172.21.15.112:0/501279623 shutdown_connections 2024-09-21T00:25:23.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.414+0000 7fb3dec20640 1 -- 172.21.15.112:0/501279623 wait complete. 2024-09-21T00:25:23.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.417+0000 7fb3dec20640 1 Processor -- start 2024-09-21T00:25:23.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.418+0000 7fb3dec20640 1 -- start start 2024-09-21T00:25:23.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.421+0000 7fb3dec20640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb3d806ddf0 0x7fb3d80b6450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:23.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.421+0000 7fb3dec20640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb3d80ca6f0 0x7fb3d80b6990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:23.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.421+0000 7fb3dec20640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb3d80cc0b0 0x7fb3d80b4a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:23.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.421+0000 7fb3dec20640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb3d8071000 con 0x7fb3d806ddf0 2024-09-21T00:25:23.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.421+0000 7fb3dec20640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb3d8070e80 con 0x7fb3d80cc0b0 2024-09-21T00:25:23.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.421+0000 7fb3dec20640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fb3d8071180 con 0x7fb3d80ca6f0 2024-09-21T00:25:23.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.418+0000 7fc0ffda1640 1 -- 172.21.15.112:0/1626399163 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0f8073070 msgr2=0x7fc0f808b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:23.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.418+0000 7fc0ffda1640 1 --2- 172.21.15.112:0/1626399163 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0f8073070 0x7fc0f808b9e0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fc0f4009800 tx=0x7fc0f402a3f0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.422+0000 7fc0ffda1640 1 -- 172.21.15.112:0/1626399163 shutdown_connections 2024-09-21T00:25:23.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.422+0000 7fc0ffda1640 1 --2- 172.21.15.112:0/1626399163 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0f8073070 0x7fc0f808b9e0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.422+0000 7fc0ffda1640 1 -- 172.21.15.112:0/1626399163 >> 172.21.15.112:0/1626399163 conn(0x7fc0f806cde0 msgr2=0x7fc0f806d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:23.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.422+0000 7fb3dd1bd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb3d80cc0b0 0x7fb3d80b4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:23.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.422+0000 7fb3dd1bd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb3d80cc0b0 0x7fb3d80b4a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:36062/0 (socket says 172.21.15.112:36062) 2024-09-21T00:25:23.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.422+0000 7fb3dd1bd640 1 -- 172.21.15.112:0/2772647217 learned_addr learned my addr 172.21.15.112:0/2772647217 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:25:23.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.422+0000 7fb3d7fff640 1 --2- 172.21.15.112:0/2772647217 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb3d80ca6f0 0x7fb3d80b6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:23.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.422+0000 7fb3dc9bc640 1 --2- 172.21.15.112:0/2772647217 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb3d806ddf0 0x7fb3d80b6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:23.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.423+0000 7fb3d7fff640 1 -- 172.21.15.112:0/2772647217 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb3d80cc0b0 msgr2=0x7fb3d80b4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:23.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.423+0000 7fb3d7fff640 1 --2- 172.21.15.112:0/2772647217 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb3d80cc0b0 0x7fb3d80b4a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.423+0000 7fb3d7fff640 1 -- 172.21.15.112:0/2772647217 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb3d806ddf0 msgr2=0x7fb3d80b6450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:23.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.423+0000 7fb3d7fff640 1 --2- 172.21.15.112:0/2772647217 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb3d806ddf0 0x7fb3d80b6450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.423+0000 7fb3d7fff640 1 -- 172.21.15.112:0/2772647217 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3d80b34b0 con 0x7fb3d80ca6f0 2024-09-21T00:25:23.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.423+0000 7fb3dc9bc640 1 --2- 172.21.15.112:0/2772647217 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb3d806ddf0 0x7fb3d80b6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:25:23.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.423+0000 7fb3d7fff640 1 --2- 172.21.15.112:0/2772647217 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb3d80ca6f0 0x7fb3d80b6990 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fb3cc00e9c0 tx=0x7fb3cc00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:25:23.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.423+0000 7fb3d57fa640 1 -- 172.21.15.112:0/2772647217 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3cc00ccc0 con 0x7fb3d80ca6f0 2024-09-21T00:25:23.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.424+0000 7fb3d57fa640 1 -- 172.21.15.112:0/2772647217 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb3cc010430 con 0x7fb3d80ca6f0 2024-09-21T00:25:23.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.424+0000 7fb3d57fa640 1 -- 172.21.15.112:0/2772647217 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3cc010860 con 0x7fb3d80ca6f0 2024-09-21T00:25:23.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.422+0000 7fc0ffda1640 1 -- 172.21.15.112:0/1626399163 shutdown_connections 2024-09-21T00:25:23.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.425+0000 7fb3dec20640 1 -- 172.21.15.112:0/2772647217 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3d8203fa0 con 0x7fb3d80ca6f0 2024-09-21T00:25:23.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.425+0000 7fb3dec20640 1 -- 172.21.15.112:0/2772647217 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3d80a0a30 con 0x7fb3d80ca6f0 2024-09-21T00:25:23.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.427+0000 7fb3beffd640 1 -- 172.21.15.112:0/2772647217 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3d80b3220 con 0x7fb3d80ca6f0 2024-09-21T00:25:23.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.429+0000 7fb3d57fa640 1 -- 172.21.15.112:0/2772647217 <== mon.2 v2:172.21.15.187:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb3cc01f020 con 0x7fb3d80ca6f0 2024-09-21T00:25:23.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.433+0000 7fb3d57fa640 1 -- 172.21.15.112:0/2772647217 <== mon.2 v2:172.21.15.187:3301/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb3cc005220 con 0x7fb3d80ca6f0 2024-09-21T00:25:23.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.433+0000 7fb3d57fa640 1 -- 172.21.15.112:0/2772647217 <== mon.2 v2:172.21.15.187:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fb3cc026070 con 0x7fb3d80ca6f0 2024-09-21T00:25:23.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.437+0000 7f2bd0372640 1 -- 172.21.15.112:0/3677375785 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2bc806edf0 msgr2=0x7f2bc816ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:23.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.437+0000 7f2bd0372640 1 --2- 172.21.15.112:0/3677375785 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2bc806edf0 0x7f2bc816ec10 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f2bc80cfdc0 tx=0x7f2bc002ace0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.422+0000 7fc0ffda1640 1 -- 172.21.15.112:0/1626399163 wait complete. 2024-09-21T00:25:23.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.425+0000 7fc0ffda1640 1 Processor -- start 2024-09-21T00:25:23.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.438+0000 7f2bd0372640 1 -- 172.21.15.112:0/3677375785 shutdown_connections 2024-09-21T00:25:23.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.438+0000 7f2bd0372640 1 --2- 172.21.15.112:0/3677375785 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2bc806edf0 0x7f2bc816ec10 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.438+0000 7f2bd0372640 1 --2- 172.21.15.112:0/3677375785 >> v2:172.21.15.187:3300/0 conn(0x7f2bc8073070 0x7f2bc808b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.438+0000 7f2bd0372640 1 -- 172.21.15.112:0/3677375785 >> 172.21.15.112:0/3677375785 conn(0x7f2bc806cde0 msgr2=0x7f2bc806d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:23.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.438+0000 7fc0ffda1640 1 -- start start 2024-09-21T00:25:23.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.440+0000 7fc0ffda1640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc0f806edf0 0x7fc0f808acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:23.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.440+0000 7fc0ffda1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0f8087700 0x7fc0f80cd570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:23.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.440+0000 7fc0fdb3d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc0f806edf0 0x7fc0f808acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:23.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.440+0000 7fc0fdb3d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc0f806edf0 0x7fc0f808acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:36070/0 (socket says 172.21.15.112:36070) 2024-09-21T00:25:23.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.440+0000 7fc0ffda1640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc0f80cdab0 0x7fc0f81626b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:23.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.440+0000 7fc0fdb3d640 1 -- 172.21.15.112:0/4290399292 learned_addr learned my addr 172.21.15.112:0/4290399292 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:25:23.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.441+0000 7fc0ffda1640 1 -- 172.21.15.112:0/4290399292 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc0f80b3b20 con 0x7fc0f8087700 2024-09-21T00:25:23.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.441+0000 7fc0fe33e640 1 --2- 172.21.15.112:0/4290399292 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc0f80cdab0 0x7fc0f81626b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:23.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.441+0000 7fc0ffda1640 1 -- 172.21.15.112:0/4290399292 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc0f80b39a0 con 0x7fc0f806edf0 2024-09-21T00:25:23.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.441+0000 7fc0ffda1640 1 -- 172.21.15.112:0/4290399292 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc0f80b3ca0 con 0x7fc0f80cdab0 2024-09-21T00:25:23.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.440+0000 7fc0fd33c640 1 --2- 172.21.15.112:0/4290399292 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0f8087700 0x7fc0f80cd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:23.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.447+0000 7fc0fdb3d640 1 -- 172.21.15.112:0/4290399292 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc0f80cdab0 msgr2=0x7fc0f81626b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:23.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.447+0000 7fc0fdb3d640 1 --2- 172.21.15.112:0/4290399292 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc0f80cdab0 0x7fc0f81626b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.447+0000 7fc0fdb3d640 1 -- 172.21.15.112:0/4290399292 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0f8087700 msgr2=0x7fc0f80cd570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:23.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.447+0000 7fc0fdb3d640 1 --2- 172.21.15.112:0/4290399292 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0f8087700 0x7fc0f80cd570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.447+0000 7fc0fdb3d640 1 -- 172.21.15.112:0/4290399292 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0f81658b0 con 0x7fc0f806edf0 2024-09-21T00:25:23.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.447+0000 7fc0fdb3d640 1 --2- 172.21.15.112:0/4290399292 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc0f806edf0 0x7fc0f808acc0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc0f4009920 tx=0x7fc0f4030870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:25:23.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.438+0000 7f2bd0372640 1 -- 172.21.15.112:0/3677375785 shutdown_connections 2024-09-21T00:25:23.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.438+0000 7f2bd0372640 1 -- 172.21.15.112:0/3677375785 wait complete. 2024-09-21T00:25:23.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.441+0000 7f2bd0372640 1 Processor -- start 2024-09-21T00:25:23.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.442+0000 7f2bd0372640 1 -- start start 2024-09-21T00:25:23.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.444+0000 7f2bd0372640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2bc806f200 0x7f2bc8165200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:23.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.444+0000 7f2bd0372640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2bc8073070 0x7f2bc815d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:23.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.444+0000 7f2bd0372640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2bc815d6d0 0x7f2bc8204ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:23.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.444+0000 7f2bd0372640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2bc816bd40 con 0x7f2bc815d6d0 2024-09-21T00:25:23.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.444+0000 7f2bd0372640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f2bc816f540 con 0x7f2bc8073070 2024-09-21T00:25:23.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.444+0000 7f2bd0372640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f2bc816bec0 con 0x7f2bc806f200 2024-09-21T00:25:23.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.447+0000 7fc0ee7fc640 1 -- 172.21.15.112:0/4290399292 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0f4040030 con 0x7fc0f806edf0 2024-09-21T00:25:23.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.447+0000 7fc0ee7fc640 1 -- 172.21.15.112:0/4290399292 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc0f4004110 con 0x7fc0f806edf0 2024-09-21T00:25:23.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.447+0000 7fc0ee7fc640 1 -- 172.21.15.112:0/4290399292 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0f4037040 con 0x7fc0f806edf0 2024-09-21T00:25:23.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.449+0000 7f2bce90f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2bc815d6d0 0x7f2bc8204ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:23.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.449+0000 7f2bce90f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2bc815d6d0 0x7f2bc8204ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55586/0 (socket says 172.21.15.112:55586) 2024-09-21T00:25:23.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.449+0000 7f2bce90f640 1 -- 172.21.15.112:0/694724504 learned_addr learned my addr 172.21.15.112:0/694724504 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:25:23.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.449+0000 7f2bce10e640 1 --2- 172.21.15.112:0/694724504 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2bc806f200 0x7f2bc8165200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:23.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.449+0000 7f2bcd90d640 1 --2- 172.21.15.112:0/694724504 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2bc8073070 0x7f2bc815d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:23.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.450+0000 7f2bce10e640 1 -- 172.21.15.112:0/694724504 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2bc8073070 msgr2=0x7f2bc815d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:23.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.450+0000 7f2bce10e640 1 --2- 172.21.15.112:0/694724504 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2bc8073070 0x7f2bc815d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.450+0000 7f2bce10e640 1 -- 172.21.15.112:0/694724504 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2bc815d6d0 msgr2=0x7f2bc8204ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:23.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.450+0000 7f2bce10e640 1 --2- 172.21.15.112:0/694724504 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2bc815d6d0 0x7f2bc8204ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.450+0000 7f2bce10e640 1 -- 172.21.15.112:0/694724504 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bc8165740 con 0x7f2bc806f200 2024-09-21T00:25:23.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.450+0000 7f2bce10e640 1 --2- 172.21.15.112:0/694724504 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2bc806f200 0x7f2bc8165200 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f2bc400b440 tx=0x7f2bc400b900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:25:23.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.451+0000 7fc0ffda1640 1 -- 172.21.15.112:0/4290399292 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0f81f5ff0 con 0x7fc0f806edf0 2024-09-21T00:25:23.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.451+0000 7fc0ffda1640 1 -- 172.21.15.112:0/4290399292 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0f81f5d10 con 0x7fc0f806edf0 2024-09-21T00:25:23.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.452+0000 7ffb74ff9640 1 -- 172.21.15.112:0/2683542254 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command([{prefix=config set, name=debug_ms}] v 0) -- 0x7ffb8015b540 con 0x7ffb8015bcf0 2024-09-21T00:25:23.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.452+0000 7fc0ee7fc640 1 -- 172.21.15.112:0/4290399292 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc0f40309e0 con 0x7fc0f806edf0 2024-09-21T00:25:23.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.452+0000 7fc0ee7fc640 1 -- 172.21.15.112:0/4290399292 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc0f40049d0 con 0x7fc0f806edf0 2024-09-21T00:25:23.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.453+0000 7fc0dbfff640 1 -- 172.21.15.112:0/4290399292 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0f8073070 con 0x7fc0f806edf0 2024-09-21T00:25:23.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.454+0000 7f2bbeffd640 1 -- 172.21.15.112:0/694724504 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc4015020 con 0x7f2bc806f200 2024-09-21T00:25:23.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.454+0000 7f2bbeffd640 1 -- 172.21.15.112:0/694724504 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2bc4011920 con 0x7f2bc806f200 2024-09-21T00:25:23.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.456+0000 7f2bbeffd640 1 -- 172.21.15.112:0/694724504 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc4011d70 con 0x7f2bc806f200 2024-09-21T00:25:23.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.457+0000 7fc0ee7fc640 1 -- 172.21.15.112:0/4290399292 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fc0f4030dc0 con 0x7fc0f806edf0 2024-09-21T00:25:23.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.458+0000 7ffb76ffd640 1 -- 172.21.15.112:0/2683542254 <== mon.2 v2:172.21.15.187:3301/0 7 ==== mon_command_ack([{prefix=config set, name=debug_ms}]=0 v9)=0 v9) ==== 106+0+0 (secure 0 0 0) 0x7ffb78005650 con 0x7ffb8015bcf0 2024-09-21T00:25:23.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.459+0000 7f2bd0372640 1 -- 172.21.15.112:0/694724504 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bc815ecb0 con 0x7f2bc806f200 2024-09-21T00:25:23.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.460+0000 7f2bd0372640 1 -- 172.21.15.112:0/694724504 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bc815def0 con 0x7f2bc806f200 2024-09-21T00:25:23.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.460+0000 7f2bbeffd640 1 -- 172.21.15.112:0/694724504 <== mon.2 v2:172.21.15.187:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f2bc4018020 con 0x7f2bc806f200 2024-09-21T00:25:23.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.460+0000 7f2bbeffd640 1 -- 172.21.15.112:0/694724504 <== mon.2 v2:172.21.15.187:3301/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f2bc40197f0 con 0x7f2bc806f200 2024-09-21T00:25:23.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.461+0000 7f2bd0372640 1 -- 172.21.15.112:0/694724504 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bc816c3f0 con 0x7f2bc806f200 2024-09-21T00:25:23.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.462+0000 7ffb74ff9640 1 -- 172.21.15.112:0/2683542254 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffb8015bcf0 msgr2=0x7ffb80138860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:23.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.462+0000 7ffb74ff9640 1 --2- 172.21.15.112:0/2683542254 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffb8015bcf0 0x7ffb80138860 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ffb7800ca70 tx=0x7ffb7800cf30 comp rx=0 tx=0).stop 2024-09-21T00:25:23.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.462+0000 7ffb74ff9640 1 -- 172.21.15.112:0/2683542254 shutdown_connections 2024-09-21T00:25:23.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.462+0000 7ffb74ff9640 1 --2- 172.21.15.112:0/2683542254 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffb8015bcf0 0x7ffb80138860 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.462+0000 7ffb74ff9640 1 --2- 172.21.15.112:0/2683542254 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffb801516d0 0x7ffb80152ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.462+0000 7ffb74ff9640 1 --2- 172.21.15.112:0/2683542254 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb8006f180 0x7ffb80154700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.462+0000 7ffb74ff9640 1 -- 172.21.15.112:0/2683542254 >> 172.21.15.112:0/2683542254 conn(0x7ffb8006cde0 msgr2=0x7ffb8006df90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:23.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.462+0000 7ffb74ff9640 1 -- 172.21.15.112:0/2683542254 shutdown_connections 2024-09-21T00:25:23.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.462+0000 7ffb74ff9640 1 -- 172.21.15.112:0/2683542254 wait complete. 2024-09-21T00:25:23.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.464+0000 7f2bbeffd640 1 -- 172.21.15.112:0/694724504 <== mon.2 v2:172.21.15.187:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f2bc401f020 con 0x7f2bc806f200 2024-09-21T00:25:23.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.475+0000 7fb0efb20640 1 -- 172.21.15.112:0/859530133 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0e0034410 msgr2=0x7fb0e0053140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:23.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.475+0000 7fb0efb20640 1 --2- 172.21.15.112:0/859530133 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0e0034410 0x7fb0e0053140 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb0e4005fd0 tx=0x7fb0e402a530 comp rx=0 tx=0).stop 2024-09-21T00:25:23.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.475+0000 7fb0efb20640 1 -- 172.21.15.112:0/859530133 shutdown_connections 2024-09-21T00:25:23.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.475+0000 7fb0efb20640 1 --2- 172.21.15.112:0/859530133 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0e0034410 0x7fb0e0053140 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.475+0000 7fb0efb20640 1 -- 172.21.15.112:0/859530133 >> 172.21.15.112:0/859530133 conn(0x7fb0e001d9c0 msgr2=0x7fb0e0039680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:23.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.478+0000 7fb0efb20640 1 -- 172.21.15.112:0/859530133 shutdown_connections 2024-09-21T00:25:23.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.478+0000 7fb0efb20640 1 -- 172.21.15.112:0/859530133 wait complete. 2024-09-21T00:25:23.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.480+0000 7fb0efb20640 1 Processor -- start 2024-09-21T00:25:23.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.483+0000 7fb15cd5e640 1 -- 172.21.15.112:0/2195570293 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb15816ec10 msgr2=0x7fb15816f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:23.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.483+0000 7fb15cd5e640 1 --2- 172.21.15.112:0/2195570293 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb15816ec10 0x7fb15816f020 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb14c009830 tx=0x7fb14c02a560 comp rx=0 tx=0).stop 2024-09-21T00:25:23.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.483+0000 7fb156d9d640 1 --2- 172.21.15.112:0/2195570293 >> v2:172.21.15.112:3300/0 conn(0x7fb158073070 0x7fb1580b3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:25:23.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.484+0000 7fb15cd5e640 1 -- 172.21.15.112:0/2195570293 shutdown_connections 2024-09-21T00:25:23.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.484+0000 7fb15cd5e640 1 --2- 172.21.15.112:0/2195570293 >> v2:172.21.15.112:3300/0 conn(0x7fb158073070 0x7fb1580b3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.484+0000 7fb15cd5e640 1 --2- 172.21.15.112:0/2195570293 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb15816ec10 0x7fb15816f020 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.484+0000 7fb15cd5e640 1 -- 172.21.15.112:0/2195570293 >> 172.21.15.112:0/2195570293 conn(0x7fb15806cde0 msgr2=0x7fb15806d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:23.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.481+0000 7fb0efb20640 1 -- start start 2024-09-21T00:25:23.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.487+0000 7fb15cd5e640 1 -- 172.21.15.112:0/2195570293 shutdown_connections 2024-09-21T00:25:23.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.488+0000 7fb0efb20640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0e00070d0 0x7fb0e01145c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:23.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.488+0000 7fb0efb20640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb0e0055100 0x7fb0e01a9380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:23.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.490+0000 7fb0efb20640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb0e00f7560 0x7fb0e004b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:23.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.490+0000 7fb0efb20640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb0e0054120 con 0x7fb0e00070d0 2024-09-21T00:25:23.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.490+0000 7fb0efb20640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb0e0053fa0 con 0x7fb0e00f7560 2024-09-21T00:25:23.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.490+0000 7fb0efb20640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fb0e00542a0 con 0x7fb0e0055100 2024-09-21T00:25:23.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.490+0000 7fb0ed8bc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0e00070d0 0x7fb0e01145c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:23.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.490+0000 7fb0ed8bc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0e00070d0 0x7fb0e01145c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55620/0 (socket says 172.21.15.112:55620) 2024-09-21T00:25:23.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.490+0000 7fb0ed8bc640 1 -- 172.21.15.112:0/1080893493 learned_addr learned my addr 172.21.15.112:0/1080893493 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:25:23.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.491+0000 7fb15cd5e640 1 -- 172.21.15.112:0/2195570293 wait complete. 2024-09-21T00:25:23.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.494+0000 7fee0b25c640 1 -- 172.21.15.112:0/2140790815 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fee040cc0b0 msgr2=0x7fee040cc4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:23.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.494+0000 7fee0b25c640 1 --2- 172.21.15.112:0/2140790815 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fee040cc0b0 0x7fee040cc4c0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fedf400afc0 tx=0x7fedf402e4b0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.494+0000 7fee0b25c640 1 -- 172.21.15.112:0/2140790815 shutdown_connections 2024-09-21T00:25:23.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.494+0000 7fee0b25c640 1 --2- 172.21.15.112:0/2140790815 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fee040cc0b0 0x7fee040cc4c0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.494+0000 7fee0b25c640 1 -- 172.21.15.112:0/2140790815 >> 172.21.15.112:0/2140790815 conn(0x7fee040b1b90 msgr2=0x7fee040b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:23.496 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_debug_frag', 'True'], check_status=True, hostname='smithi112') 2024-09-21T00:25:23.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.496+0000 7fb15cd5e640 1 Processor -- start 2024-09-21T00:25:23.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.496+0000 7fb15cd5e640 1 -- start start 2024-09-21T00:25:23.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.494+0000 7fee0b25c640 1 -- 172.21.15.112:0/2140790815 shutdown_connections 2024-09-21T00:25:23.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.494+0000 7fee0b25c640 1 -- 172.21.15.112:0/2140790815 wait complete. 2024-09-21T00:25:23.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.498+0000 7fee0b25c640 1 Processor -- start 2024-09-21T00:25:23.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.497+0000 7fb0ee0bd640 1 --2- 172.21.15.112:0/1080893493 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb0e00f7560 0x7fb0e004b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:23.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.497+0000 7fb0ed0bb640 1 --2- 172.21.15.112:0/1080893493 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb0e0055100 0x7fb0e01a9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:23.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.497+0000 7fb0ed8bc640 1 -- 172.21.15.112:0/1080893493 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb0e0055100 msgr2=0x7fb0e01a9380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:23.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.497+0000 7fb0ed8bc640 1 --2- 172.21.15.112:0/1080893493 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb0e0055100 0x7fb0e01a9380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.497+0000 7fb0ed8bc640 1 -- 172.21.15.112:0/1080893493 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb0e00f7560 msgr2=0x7fb0e004b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:23.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.497+0000 7fb0ed8bc640 1 --2- 172.21.15.112:0/1080893493 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb0e00f7560 0x7fb0e004b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.497+0000 7fb0ed8bc640 1 -- 172.21.15.112:0/1080893493 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0e01a98c0 con 0x7fb0e00070d0 2024-09-21T00:25:23.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.497+0000 7fb0ed0bb640 1 --2- 172.21.15.112:0/1080893493 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb0e0055100 0x7fb0e01a9380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:25:23.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.497+0000 7fb0ed8bc640 1 --2- 172.21.15.112:0/1080893493 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0e00070d0 0x7fb0e01145c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fb0e806b0e0 tx=0x7fb0e806a0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:25:23.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.499+0000 7f1af759e640 1 -- 172.21.15.112:0/2559126153 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1af80b5190 msgr2=0x7f1af816b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:23.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.499+0000 7f1af759e640 1 --2- 172.21.15.112:0/2559126153 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1af80b5190 0x7f1af816b340 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f1af0005fd0 tx=0x7f1af002a590 comp rx=0 tx=0).stop 2024-09-21T00:25:23.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.499+0000 7f1af759e640 1 -- 172.21.15.112:0/2559126153 shutdown_connections 2024-09-21T00:25:23.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.499+0000 7f1af759e640 1 --2- 172.21.15.112:0/2559126153 >> v2:172.21.15.187:3300/0 conn(0x7f1af806e530 0x7f1af806cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.499+0000 7f1af759e640 1 --2- 172.21.15.112:0/2559126153 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1af80b5190 0x7f1af816b340 secure :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f1af0005fd0 tx=0x7f1af002a590 comp rx=0 tx=0).stop 2024-09-21T00:25:23.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.499+0000 7f1af759e640 1 -- 172.21.15.112:0/2559126153 >> 172.21.15.112:0/2559126153 conn(0x7f1af80cfec0 msgr2=0x7f1af80d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:23.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.499+0000 7f1af759e640 1 -- 172.21.15.112:0/2559126153 shutdown_connections 2024-09-21T00:25:23.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.499+0000 7f1af759e640 1 -- 172.21.15.112:0/2559126153 wait complete. 2024-09-21T00:25:23.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.502+0000 7f1af759e640 1 Processor -- start 2024-09-21T00:25:23.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.501+0000 7fb0de7fc640 1 -- 172.21.15.112:0/1080893493 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0e80753c0 con 0x7fb0e00070d0 2024-09-21T00:25:23.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.501+0000 7fb0de7fc640 1 -- 172.21.15.112:0/1080893493 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb0e8076180 con 0x7fb0e00070d0 2024-09-21T00:25:23.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.501+0000 7fb0de7fc640 1 -- 172.21.15.112:0/1080893493 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0e80765d0 con 0x7fb0e00070d0 2024-09-21T00:25:23.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.502+0000 7f1af759e640 1 -- start start 2024-09-21T00:25:23.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.506+0000 7fb15cd5e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb158073070 0x7fb15820a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:23.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.506+0000 7fb15cd5e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb15808b800 0x7fb158179980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:23.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.506+0000 7fb15cd5e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb15816ec10 0x7fb158205f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:23.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.506+0000 7fb15cd5e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb1580b3080 con 0x7fb158073070 2024-09-21T00:25:23.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.506+0000 7fb15cd5e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb1580b4060 con 0x7fb15816ec10 2024-09-21T00:25:23.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.507+0000 7fb15cd5e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fb1580b3200 con 0x7fb15808b800 2024-09-21T00:25:23.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.505+0000 7fb0efb20640 1 -- 172.21.15.112:0/1080893493 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0e0114e00 con 0x7fb0e00070d0 2024-09-21T00:25:23.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.505+0000 7fb0efb20640 1 -- 172.21.15.112:0/1080893493 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0e00f5ba0 con 0x7fb0e00070d0 2024-09-21T00:25:23.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.507+0000 7fb15659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb158073070 0x7fb15820a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:23.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.507+0000 7fb15659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb158073070 0x7fb15820a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55650/0 (socket says 172.21.15.112:55650) 2024-09-21T00:25:23.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.507+0000 7fb15659c640 1 -- 172.21.15.112:0/810229568 learned_addr learned my addr 172.21.15.112:0/810229568 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:25:23.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.507+0000 7fb0de7fc640 1 -- 172.21.15.112:0/1080893493 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb0e8082400 con 0x7fb0e00070d0 2024-09-21T00:25:23.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.507+0000 7fb0de7fc640 1 -- 172.21.15.112:0/1080893493 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb0e808b640 con 0x7fb0e00070d0 2024-09-21T00:25:23.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.505+0000 7f1af759e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1af806e530 0x7f1af8162150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:23.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.505+0000 7f1af759e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1af808bb20 0x7f1af808b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:23.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.505+0000 7f1af759e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1af81606d0 0x7f1af8163bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:23.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.505+0000 7f1af759e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1af806d930 con 0x7f1af806e530 2024-09-21T00:25:23.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.505+0000 7f1af759e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f1af806d7b0 con 0x7f1af808bb20 2024-09-21T00:25:23.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.505+0000 7f1af759e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f1af806dab0 con 0x7f1af81606d0 2024-09-21T00:25:23.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.507+0000 7fb15659c640 1 -- 172.21.15.112:0/810229568 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb15808b800 msgr2=0x7fb158179980 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T00:25:23.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.507+0000 7fb15659c640 1 --2- 172.21.15.112:0/810229568 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb15808b800 0x7fb158179980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.507+0000 7fb15659c640 1 -- 172.21.15.112:0/810229568 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb15816ec10 msgr2=0x7fb158205f90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T00:25:23.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.507+0000 7fb15659c640 1 --2- 172.21.15.112:0/810229568 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb15816ec10 0x7fb158205f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.507+0000 7fb15659c640 1 -- 172.21.15.112:0/810229568 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb158179ec0 con 0x7fb158073070 2024-09-21T00:25:23.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.508+0000 7f1af659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1af806e530 0x7f1af8162150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:23.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.508+0000 7f1af659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1af806e530 0x7f1af8162150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55652/0 (socket says 172.21.15.112:55652) 2024-09-21T00:25:23.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.508+0000 7f1af659c640 1 -- 172.21.15.112:0/1711171651 learned_addr learned my addr 172.21.15.112:0/1711171651 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:25:23.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.508+0000 7fb15659c640 1 --2- 172.21.15.112:0/810229568 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb158073070 0x7fb15820a040 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fb14800c370 tx=0x7fb14800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:25:23.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.508+0000 7f1af659c640 1 -- 172.21.15.112:0/1711171651 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1af81606d0 msgr2=0x7f1af8163bc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T00:25:23.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.508+0000 7f1af659c640 1 --2- 172.21.15.112:0/1711171651 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1af81606d0 0x7f1af8163bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.508+0000 7f1af659c640 1 -- 172.21.15.112:0/1711171651 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1af808bb20 msgr2=0x7f1af808b1e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T00:25:23.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.508+0000 7f1af659c640 1 --2- 172.21.15.112:0/1711171651 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1af808bb20 0x7f1af808b1e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.509+0000 7fb146ffd640 1 -- 172.21.15.112:0/810229568 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb148017070 con 0x7fb158073070 2024-09-21T00:25:23.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.508+0000 7f1af659c640 1 -- 172.21.15.112:0/1711171651 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1af81610b0 con 0x7f1af806e530 2024-09-21T00:25:23.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.509+0000 7fb146ffd640 1 -- 172.21.15.112:0/810229568 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb1480035f0 con 0x7fb158073070 2024-09-21T00:25:23.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.509+0000 7fb146ffd640 1 -- 172.21.15.112:0/810229568 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb148003a40 con 0x7fb158073070 2024-09-21T00:25:23.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.509+0000 7fb0cbfff640 1 -- 172.21.15.112:0/1080893493 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0e0009ca0 con 0x7fb0e00070d0 2024-09-21T00:25:23.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.510+0000 7f75310a9640 1 -- 172.21.15.112:0/3772010204 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f75240e3f60 msgr2=0x7f7524008bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:23.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.510+0000 7f75310a9640 1 --2- 172.21.15.112:0/3772010204 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f75240e3f60 0x7f7524008bd0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f752400db80 tx=0x7f751802a440 comp rx=0 tx=0).stop 2024-09-21T00:25:23.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.495+0000 7f3f43084640 1 -- 172.21.15.112:0/2113395270 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f3c06edf0 msgr2=0x7f3f3c0b3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:23.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.495+0000 7f3f43084640 1 --2- 172.21.15.112:0/2113395270 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f3c06edf0 0x7f3f3c0b3a50 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f3f34009ca0 tx=0x7f3f3403ba60 comp rx=0 tx=0).stop 2024-09-21T00:25:23.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.495+0000 7f3f43084640 1 -- 172.21.15.112:0/2113395270 shutdown_connections 2024-09-21T00:25:23.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.495+0000 7f3f43084640 1 --2- 172.21.15.112:0/2113395270 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f3c06edf0 0x7f3f3c0b3a50 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.495+0000 7f3f43084640 1 -- 172.21.15.112:0/2113395270 >> 172.21.15.112:0/2113395270 conn(0x7f3f3c06cde0 msgr2=0x7f3f3c06d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:23.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.496+0000 7f3f43084640 1 -- 172.21.15.112:0/2113395270 shutdown_connections 2024-09-21T00:25:23.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.502+0000 7f3f43084640 1 -- 172.21.15.112:0/2113395270 wait complete. 2024-09-21T00:25:23.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.505+0000 7f3f43084640 1 Processor -- start 2024-09-21T00:25:23.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.505+0000 7f3f43084640 1 -- start start 2024-09-21T00:25:23.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.507+0000 7f3f43084640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f3c06edf0 0x7f3f3c15c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:23.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.507+0000 7f3f43084640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3f3c0aec20 0x7f3f3c15acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:23.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.507+0000 7f3f43084640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f3c0b3f90 0x7f3f3c1f41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:23.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.507+0000 7f3f43084640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3f3c15d0c0 con 0x7f3f3c0b3f90 2024-09-21T00:25:23.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.510+0000 7fb15cd5e640 1 -- 172.21.15.112:0/810229568 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb158206ac0 con 0x7fb158073070 2024-09-21T00:25:23.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.511+0000 7fb15cd5e640 1 -- 172.21.15.112:0/810229568 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb158209560 con 0x7fb158073070 2024-09-21T00:25:23.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.512+0000 7fb144ff9640 1 -- 172.21.15.112:0/810229568 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb15816f020 con 0x7fb158073070 2024-09-21T00:25:23.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.516+0000 7f3f41621640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f3c0b3f90 0x7f3f3c1f41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:23.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.516+0000 7f3f41621640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f3c0b3f90 0x7f3f3c1f41c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55676/0 (socket says 172.21.15.112:55676) 2024-09-21T00:25:23.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.516+0000 7f3f41621640 1 -- 172.21.15.112:0/768275211 learned_addr learned my addr 172.21.15.112:0/768275211 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:25:23.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.507+0000 7f3f43084640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3f3c074450 con 0x7f3f3c0aec20 2024-09-21T00:25:23.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.516+0000 7f3f43084640 1 -- 172.21.15.112:0/768275211 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3f3c15d240 con 0x7f3f3c06edf0 2024-09-21T00:25:23.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.516+0000 7f3f40e20640 1 --2- 172.21.15.112:0/768275211 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f3c06edf0 0x7f3f3c15c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:23.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.516+0000 7f3f41621640 1 -- 172.21.15.112:0/768275211 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f3c06edf0 msgr2=0x7f3f3c15c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:23.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.516+0000 7f3f41621640 1 --2- 172.21.15.112:0/768275211 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f3c06edf0 0x7f3f3c15c3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.516+0000 7f3f41621640 1 -- 172.21.15.112:0/768275211 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3f3c0aec20 msgr2=0x7f3f3c15acc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T00:25:23.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.516+0000 7f3f41621640 1 --2- 172.21.15.112:0/768275211 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3f3c0aec20 0x7f3f3c15acc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.516+0000 7f3f41621640 1 -- 172.21.15.112:0/768275211 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f3c15b330 con 0x7f3f3c0b3f90 2024-09-21T00:25:23.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.517+0000 7fb0de7fc640 1 -- 172.21.15.112:0/1080893493 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fb0e8089030 con 0x7fb0e00070d0 2024-09-21T00:25:23.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.517+0000 7f3f41621640 1 --2- 172.21.15.112:0/768275211 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f3c0b3f90 0x7f3f3c1f41c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f3f2c00c730 tx=0x7f3f2c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:25:23.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.517+0000 7fb146ffd640 1 -- 172.21.15.112:0/810229568 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb14801a070 con 0x7fb158073070 2024-09-21T00:25:23.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.517+0000 7fb146ffd640 1 -- 172.21.15.112:0/810229568 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb148012400 con 0x7fb158073070 2024-09-21T00:25:23.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.517+0000 7fb146ffd640 1 -- 172.21.15.112:0/810229568 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fb148021020 con 0x7fb158073070 2024-09-21T00:25:23.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.509+0000 7f1af659c640 1 --2- 172.21.15.112:0/1711171651 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1af806e530 0x7f1af8162150 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f1ae800c800 tx=0x7f1ae800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:25:23.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.517+0000 7f1ae6ffd640 1 -- 172.21.15.112:0/1711171651 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ae8017070 con 0x7f1af806e530 2024-09-21T00:25:23.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.517+0000 7f1ae6ffd640 1 -- 172.21.15.112:0/1711171651 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1ae8005270 con 0x7f1af806e530 2024-09-21T00:25:23.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.518+0000 7f1af759e640 1 -- 172.21.15.112:0/1711171651 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1af8161390 con 0x7f1af806e530 2024-09-21T00:25:23.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.518+0000 7f1af759e640 1 -- 172.21.15.112:0/1711171651 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1af8161950 con 0x7f1af806e530 2024-09-21T00:25:23.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.518+0000 7f1ae6ffd640 1 -- 172.21.15.112:0/1711171651 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ae800a640 con 0x7f1af806e530 2024-09-21T00:25:23.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.498+0000 7fee0b25c640 1 -- start start 2024-09-21T00:25:23.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.520+0000 7fee0b25c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fee0406ddf0 0x7fee04213190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:23.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.520+0000 7fee0b25c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fee040ca6f0 0x7fee04214cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:23.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.520+0000 7fee08ff8640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fee0406ddf0 0x7fee04213190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:23.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.520+0000 7fee08ff8640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fee0406ddf0 0x7fee04213190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:41174/0 (socket says 172.21.15.112:41174) 2024-09-21T00:25:23.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.520+0000 7fee03fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fee040ca6f0 0x7fee04214cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:23.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.520+0000 7fee03fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fee040ca6f0 0x7fee04214cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:36136/0 (socket says 172.21.15.112:36136) 2024-09-21T00:25:23.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.520+0000 7fee08ff8640 1 -- 172.21.15.112:0/2547265527 learned_addr learned my addr 172.21.15.112:0/2547265527 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:25:23.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.521+0000 7fee0b25c640 1 --2- 172.21.15.112:0/2547265527 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee040cc0b0 0x7fee04215230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:23.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.521+0000 7f1ae6ffd640 1 -- 172.21.15.112:0/1711171651 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f1ae801a070 con 0x7f1af806e530 2024-09-21T00:25:23.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.524+0000 7f75310a9640 1 -- 172.21.15.112:0/3772010204 shutdown_connections 2024-09-21T00:25:23.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.524+0000 7f75310a9640 1 --2- 172.21.15.112:0/3772010204 >> v2:172.21.15.112:3300/0 conn(0x7f75240e4370 0x7f7524055830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.524+0000 7f75310a9640 1 --2- 172.21.15.112:0/3772010204 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f75240e3f60 0x7f7524008bd0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.524+0000 7f75310a9640 1 -- 172.21.15.112:0/3772010204 >> 172.21.15.112:0/3772010204 conn(0x7f752401eab0 msgr2=0x7f752403a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:23.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.521+0000 7f3f397fa640 1 -- 172.21.15.112:0/768275211 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f2c0050d0 con 0x7f3f3c0b3f90 2024-09-21T00:25:23.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.521+0000 7f3f43084640 1 -- 172.21.15.112:0/768275211 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f3c15b610 con 0x7f3f3c0b3f90 2024-09-21T00:25:23.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.521+0000 7f3f43084640 1 -- 172.21.15.112:0/768275211 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f3c15bbd0 con 0x7f3f3c0b3f90 2024-09-21T00:25:23.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.521+0000 7f3f397fa640 1 -- 172.21.15.112:0/768275211 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3f2c005270 con 0x7f3f3c0b3f90 2024-09-21T00:25:23.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.521+0000 7f3f397fa640 1 -- 172.21.15.112:0/768275211 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f2c00a640 con 0x7f3f3c0b3f90 2024-09-21T00:25:23.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.522+0000 7f3f397fa640 1 -- 172.21.15.112:0/768275211 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f3f2c00a8c0 con 0x7f3f3c0b3f90 2024-09-21T00:25:23.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.522+0000 7f3f397fa640 1 -- 172.21.15.112:0/768275211 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f3f2c012760 con 0x7f3f3c0b3f90 2024-09-21T00:25:23.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.524+0000 7f3f43084640 1 -- 172.21.15.112:0/768275211 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f1c005290 con 0x7f3f3c0b3f90 2024-09-21T00:25:23.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.521+0000 7fee0b25c640 1 -- 172.21.15.112:0/2547265527 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fee04071170 con 0x7fee040cc0b0 2024-09-21T00:25:23.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.521+0000 7fee0b25c640 1 -- 172.21.15.112:0/2547265527 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fee04070ff0 con 0x7fee040ca6f0 2024-09-21T00:25:23.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.521+0000 7fee0b25c640 1 -- 172.21.15.112:0/2547265527 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fee040712f0 con 0x7fee0406ddf0 2024-09-21T00:25:23.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.522+0000 7fee097f9640 1 --2- 172.21.15.112:0/2547265527 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee040cc0b0 0x7fee04215230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:23.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.522+0000 7fee08ff8640 1 -- 172.21.15.112:0/2547265527 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fee040ca6f0 msgr2=0x7fee04214cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:23.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.522+0000 7fee08ff8640 1 --2- 172.21.15.112:0/2547265527 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fee040ca6f0 0x7fee04214cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.522+0000 7fee08ff8640 1 -- 172.21.15.112:0/2547265527 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee040cc0b0 msgr2=0x7fee04215230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:23.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.522+0000 7fee08ff8640 1 --2- 172.21.15.112:0/2547265527 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee040cc0b0 0x7fee04215230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.522+0000 7fee08ff8640 1 -- 172.21.15.112:0/2547265527 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee040c3420 con 0x7fee0406ddf0 2024-09-21T00:25:23.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.522+0000 7fee08ff8640 1 --2- 172.21.15.112:0/2547265527 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fee0406ddf0 0x7fee04213190 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fedfc00c810 tx=0x7fedfc00ccd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:25:23.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.523+0000 7fee017fa640 1 -- 172.21.15.112:0/2547265527 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedfc003da0 con 0x7fee0406ddf0 2024-09-21T00:25:23.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.523+0000 7fee017fa640 1 -- 172.21.15.112:0/2547265527 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fedfc005070 con 0x7fee0406ddf0 2024-09-21T00:25:23.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.523+0000 7fee017fa640 1 -- 172.21.15.112:0/2547265527 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedfc0054c0 con 0x7fee0406ddf0 2024-09-21T00:25:23.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.524+0000 7fee0b25c640 1 -- 172.21.15.112:0/2547265527 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee040a0f60 con 0x7fee0406ddf0 2024-09-21T00:25:23.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.524+0000 7fee0b25c640 1 -- 172.21.15.112:0/2547265527 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fee04215950 con 0x7fee0406ddf0 2024-09-21T00:25:23.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.524+0000 7fee017fa640 1 -- 172.21.15.112:0/2547265527 <== mon.2 v2:172.21.15.187:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fedfc005660 con 0x7fee0406ddf0 2024-09-21T00:25:23.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.524+0000 7fee017fa640 1 -- 172.21.15.112:0/2547265527 <== mon.2 v2:172.21.15.187:3301/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fedfc01bae0 con 0x7fee0406ddf0 2024-09-21T00:25:23.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.521+0000 7f1ae6ffd640 1 -- 172.21.15.112:0/1711171651 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f1ae801c780 con 0x7f1af806e530 2024-09-21T00:25:23.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.524+0000 7f1af759e640 1 -- 172.21.15.112:0/1711171651 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1adc005290 con 0x7f1af806e530 2024-09-21T00:25:23.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.527+0000 7fee0b25c640 1 -- 172.21.15.112:0/2547265527 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee04073e70 con 0x7fee0406ddf0 2024-09-21T00:25:23.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.529+0000 7fee017fa640 1 -- 172.21.15.112:0/2547265527 <== mon.2 v2:172.21.15.187:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fee04073e70 con 0x7fee0406ddf0 2024-09-21T00:25:23.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.531+0000 7f75310a9640 1 -- 172.21.15.112:0/3772010204 shutdown_connections 2024-09-21T00:25:23.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.531+0000 7f1ae41a1640 1 -- 172.21.15.112:0/4074265818 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1adc15bcf0 msgr2=0x7f1adc15c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:23.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.531+0000 7f1ae41a1640 1 --2- 172.21.15.112:0/4074265818 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1adc15bcf0 0x7f1adc15c100 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f1ad8009830 tx=0x7f1ad802a530 comp rx=0 tx=0).stop 2024-09-21T00:25:23.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.531+0000 7f1ae41a1640 1 -- 172.21.15.112:0/4074265818 shutdown_connections 2024-09-21T00:25:23.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.531+0000 7f1ae41a1640 1 --2- 172.21.15.112:0/4074265818 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1adc15bcf0 0x7f1adc15c100 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.531+0000 7f1ae41a1640 1 --2- 172.21.15.112:0/4074265818 >> v2:172.21.15.187:3300/0 conn(0x7f1adc1516d0 0x7f1adc15b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.531+0000 7f1ae41a1640 1 -- 172.21.15.112:0/4074265818 >> 172.21.15.112:0/4074265818 conn(0x7f1adc06cde0 msgr2=0x7f1adc06d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:23.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.531+0000 7f1ae6ffd640 1 -- 172.21.15.112:0/1711171651 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f1ae8021020 con 0x7f1af806e530 2024-09-21T00:25:23.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.531+0000 7f1ae41a1640 1 -- 172.21.15.112:0/4074265818 shutdown_connections 2024-09-21T00:25:23.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.532+0000 7f75310a9640 1 -- 172.21.15.112:0/3772010204 wait complete. 2024-09-21T00:25:23.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.534+0000 7f75310a9640 1 Processor -- start 2024-09-21T00:25:23.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.535+0000 7f75310a9640 1 -- start start 2024-09-21T00:25:23.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.537+0000 7f75310a9640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7524055d70 0x7f752404ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:23.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.537+0000 7f75310a9640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f75240e3f60 0x7f75240e26e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:23.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.534+0000 7f3f397fa640 1 -- 172.21.15.112:0/768275211 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f3f2c00ad90 con 0x7f3f3c0b3f90 2024-09-21T00:25:23.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.536+0000 7f23d7d20640 1 -- 172.21.15.112:0/3077072870 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f23bc004800 msgr2=0x7f23bc004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:23.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.536+0000 7f23d7d20640 1 --2- 172.21.15.112:0/3077072870 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f23bc004800 0x7f23bc004c10 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f23cc006b90 tx=0x7f23bc005830 comp rx=0 tx=0).stop 2024-09-21T00:25:23.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.536+0000 7f23d7d20640 1 -- 172.21.15.112:0/3077072870 shutdown_connections 2024-09-21T00:25:23.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.536+0000 7f23d7d20640 1 --2- 172.21.15.112:0/3077072870 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f23bc004800 0x7f23bc004c10 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.536+0000 7f23d7d20640 1 -- 172.21.15.112:0/3077072870 >> 172.21.15.112:0/3077072870 conn(0x7f23d006cde0 msgr2=0x7f23d006d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:23.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.532+0000 7f1ae41a1640 1 -- 172.21.15.112:0/4074265818 wait complete. 2024-09-21T00:25:23.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.534+0000 7f1ae41a1640 1 Processor -- start 2024-09-21T00:25:23.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.538+0000 7f75310a9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75240e4370 0x7f75240e2c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:23.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.538+0000 7f75308a8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75240e4370 0x7f75240e2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:23.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.538+0000 7f75308a8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75240e4370 0x7f75240e2c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55704/0 (socket says 172.21.15.112:55704) 2024-09-21T00:25:23.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.538+0000 7f75308a8640 1 -- 172.21.15.112:0/391471625 learned_addr learned my addr 172.21.15.112:0/391471625 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:25:23.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.538+0000 7f75310a9640 1 -- 172.21.15.112:0/391471625 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7524009320 con 0x7f75240e4370 2024-09-21T00:25:23.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.538+0000 7f75310a9640 1 -- 172.21.15.112:0/391471625 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7524033720 con 0x7f7524055d70 2024-09-21T00:25:23.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.538+0000 7f75310a9640 1 -- 172.21.15.112:0/391471625 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f752404a9b0 con 0x7f75240e3f60 2024-09-21T00:25:23.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.539+0000 7f75308a8640 1 -- 172.21.15.112:0/391471625 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f75240e3f60 msgr2=0x7f75240e26e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T00:25:23.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.539+0000 7f75308a8640 1 --2- 172.21.15.112:0/391471625 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f75240e3f60 0x7f75240e26e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.539+0000 7f75308a8640 1 -- 172.21.15.112:0/391471625 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7524055d70 msgr2=0x7f752404ecd0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T00:25:23.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.539+0000 7f75308a8640 1 --2- 172.21.15.112:0/391471625 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7524055d70 0x7f752404ecd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.539+0000 7f75308a8640 1 -- 172.21.15.112:0/391471625 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7524049470 con 0x7f75240e4370 2024-09-21T00:25:23.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.539+0000 7f75308a8640 1 --2- 172.21.15.112:0/391471625 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75240e4370 0x7f75240e2c20 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f752000c730 tx=0x7f752000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:25:23.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.538+0000 7f1ae41a1640 1 -- start start 2024-09-21T00:25:23.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.540+0000 7f1ae41a1640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1adc06f180 0x7f1adc150930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:23.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.540+0000 7f1ae41a1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1adc1516d0 0x7f1adc1418d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:23.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.540+0000 7f1ae41a1640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1adc15bcf0 0x7f1adc141e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:23.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.540+0000 7f1ae41a1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1adc070540 con 0x7f1adc1516d0 2024-09-21T00:25:23.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.540+0000 7f1ae41a1640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f1adc0703c0 con 0x7f1adc06f180 2024-09-21T00:25:23.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.540+0000 7f1ae41a1640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f1adc070d30 con 0x7f1adc15bcf0 2024-09-21T00:25:23.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.540+0000 7f1ae273e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1adc15bcf0 0x7f1adc141e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:23.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.540+0000 7f1ae273e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1adc15bcf0 0x7f1adc141e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:41188/0 (socket says 172.21.15.112:41188) 2024-09-21T00:25:23.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.540+0000 7f1ae273e640 1 -- 172.21.15.112:0/1683671805 learned_addr learned my addr 172.21.15.112:0/1683671805 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:25:23.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.541+0000 7f1ae173c640 1 --2- 172.21.15.112:0/1683671805 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1adc1516d0 0x7f1adc1418d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:23.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.541+0000 7f1ae273e640 1 -- 172.21.15.112:0/1683671805 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1adc06f180 msgr2=0x7f1adc150930 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T00:25:23.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.541+0000 7f1ae273e640 1 --2- 172.21.15.112:0/1683671805 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1adc06f180 0x7f1adc150930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.541+0000 7f1ae273e640 1 -- 172.21.15.112:0/1683671805 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1adc1516d0 msgr2=0x7f1adc1418d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:23.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.541+0000 7f1ae273e640 1 --2- 172.21.15.112:0/1683671805 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1adc1516d0 0x7f1adc1418d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.541+0000 7f1ae273e640 1 -- 172.21.15.112:0/1683671805 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1adc1527d0 con 0x7f1adc15bcf0 2024-09-21T00:25:23.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.541+0000 7f1ae173c640 1 --2- 172.21.15.112:0/1683671805 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1adc1516d0 0x7f1adc1418d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:25:23.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.541+0000 7f1ae273e640 1 --2- 172.21.15.112:0/1683671805 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1adc15bcf0 0x7f1adc141e10 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f1ad8009950 tx=0x7f1ad80035f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:25:23.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.543+0000 7f7528ff9640 1 -- 172.21.15.112:0/391471625 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7520016020 con 0x7f75240e4370 2024-09-21T00:25:23.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.543+0000 7f7528ff9640 1 -- 172.21.15.112:0/391471625 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f752000a640 con 0x7f75240e4370 2024-09-21T00:25:23.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.543+0000 7f7528ff9640 1 -- 172.21.15.112:0/391471625 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f752000aa30 con 0x7f75240e4370 2024-09-21T00:25:23.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.536+0000 7f23d7d20640 1 -- 172.21.15.112:0/3077072870 shutdown_connections 2024-09-21T00:25:23.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.543+0000 7f75310a9640 1 -- 172.21.15.112:0/391471625 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f752404f950 con 0x7f75240e4370 2024-09-21T00:25:23.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.536+0000 7f23d7d20640 1 -- 172.21.15.112:0/3077072870 wait complete. 2024-09-21T00:25:23.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.543+0000 7f75310a9640 1 -- 172.21.15.112:0/391471625 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7524035960 con 0x7f75240e4370 2024-09-21T00:25:23.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.539+0000 7f23d7d20640 1 Processor -- start 2024-09-21T00:25:23.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.539+0000 7f23d7d20640 1 -- start start 2024-09-21T00:25:23.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.541+0000 7f23d7d20640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23bc004800 0x7f23d02058a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:23.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.541+0000 7f23d7d20640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f23d008a720 0x7f23d01f6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:23.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.542+0000 7f23d7d20640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f23d01f67d0 0x7f23d01646c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:23.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.542+0000 7f23d7d20640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f23d008bd70 con 0x7f23bc004800 2024-09-21T00:25:23.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.542+0000 7f23d7d20640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f23d008bbf0 con 0x7f23d01f67d0 2024-09-21T00:25:23.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.542+0000 7f23d7d20640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f23d016ec10 con 0x7f23d008a720 2024-09-21T00:25:23.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.542+0000 7f23d52bb640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f23d008a720 0x7f23d01f6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:23.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.542+0000 7f23d52bb640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f23d008a720 0x7f23d01f6290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:41198/0 (socket says 172.21.15.112:41198) 2024-09-21T00:25:23.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.542+0000 7f23d52bb640 1 -- 172.21.15.112:0/2190145055 learned_addr learned my addr 172.21.15.112:0/2190145055 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:25:23.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.542+0000 7f23d5abc640 1 --2- 172.21.15.112:0/2190145055 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23bc004800 0x7f23d02058a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:23.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.543+0000 7f23d52bb640 1 -- 172.21.15.112:0/2190145055 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f23d01f67d0 msgr2=0x7f23d01646c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T00:25:23.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.543+0000 7f23d52bb640 1 --2- 172.21.15.112:0/2190145055 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f23d01f67d0 0x7f23d01646c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.543+0000 7f23d52bb640 1 -- 172.21.15.112:0/2190145055 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23bc004800 msgr2=0x7f23d02058a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:23.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.543+0000 7f23d52bb640 1 --2- 172.21.15.112:0/2190145055 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23bc004800 0x7f23d02058a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.543+0000 7f23d52bb640 1 -- 172.21.15.112:0/2190145055 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23d01f6be0 con 0x7f23d008a720 2024-09-21T00:25:23.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.543+0000 7f23d5abc640 1 --2- 172.21.15.112:0/2190145055 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23bc004800 0x7f23d02058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:25:23.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.543+0000 7f23d52bb640 1 --2- 172.21.15.112:0/2190145055 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f23d008a720 0x7f23d01f6290 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f23cc00c350 tx=0x7f23cc003bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:25:23.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.544+0000 7f23c67fc640 1 -- 172.21.15.112:0/2190145055 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23cc043030 con 0x7f23d008a720 2024-09-21T00:25:23.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.545+0000 7f23d7d20640 1 -- 172.21.15.112:0/2190145055 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23d0201b90 con 0x7f23d008a720 2024-09-21T00:25:23.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.547+0000 7f7528ff9640 1 -- 172.21.15.112:0/391471625 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7520019020 con 0x7f75240e4370 2024-09-21T00:25:23.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.547+0000 7f7528ff9640 1 -- 172.21.15.112:0/391471625 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f752001b4d0 con 0x7f75240e4370 2024-09-21T00:25:23.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.545+0000 7f23d7d20640 1 -- 172.21.15.112:0/2190145055 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f23d0202150 con 0x7f23d008a720 2024-09-21T00:25:23.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.547+0000 7f23c67fc640 1 -- 172.21.15.112:0/2190145055 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f23cc053480 con 0x7f23d008a720 2024-09-21T00:25:23.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.547+0000 7f23c67fc640 1 -- 172.21.15.112:0/2190145055 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23cc0538b0 con 0x7f23d008a720 2024-09-21T00:25:23.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.547+0000 7f23c67fc640 1 -- 172.21.15.112:0/2190145055 <== mon.2 v2:172.21.15.187:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f23cc046020 con 0x7f23d008a720 2024-09-21T00:25:23.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.547+0000 7f23c67fc640 1 -- 172.21.15.112:0/2190145055 <== mon.2 v2:172.21.15.187:3301/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f23cc03f890 con 0x7f23d008a720 2024-09-21T00:25:23.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.549+0000 7f75167fc640 1 -- 172.21.15.112:0/391471625 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7524008bd0 con 0x7f75240e4370 2024-09-21T00:25:23.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.549+0000 7f23bbfff640 1 -- 172.21.15.112:0/2190145055 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23b0005290 con 0x7f23d008a720 2024-09-21T00:25:23.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.551+0000 7f23c67fc640 1 -- 172.21.15.112:0/2190145055 <== mon.2 v2:172.21.15.187:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f23cc061080 con 0x7f23d008a720 2024-09-21T00:25:23.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.556+0000 7fda33fff640 1 -- 172.21.15.112:0/2664021158 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command([{prefix=config set, name=mds_debug_scatterstat}] v 0) -- 0x7fda14005580 con 0x7fda1c003a80 2024-09-21T00:25:23.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.557+0000 7f1ad27fc640 1 -- 172.21.15.112:0/1683671805 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ad8041070 con 0x7f1adc15bcf0 2024-09-21T00:25:23.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.556+0000 7f7528ff9640 1 -- 172.21.15.112:0/391471625 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f7520020020 con 0x7f75240e4370 2024-09-21T00:25:23.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.557+0000 7f1ae41a1640 1 -- 172.21.15.112:0/1683671805 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1adc1541b0 con 0x7f1adc15bcf0 2024-09-21T00:25:23.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.557+0000 7f1ae41a1640 1 -- 172.21.15.112:0/1683671805 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f1adc152ab0 con 0x7f1adc15bcf0 2024-09-21T00:25:23.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.557+0000 7f1ad27fc640 1 -- 172.21.15.112:0/1683671805 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1ad8003d20 con 0x7f1adc15bcf0 2024-09-21T00:25:23.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.557+0000 7f1ad27fc640 1 -- 172.21.15.112:0/1683671805 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ad803b680 con 0x7f1adc15bcf0 2024-09-21T00:25:23.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.562+0000 7f1ad27fc640 1 -- 172.21.15.112:0/1683671805 <== mon.2 v2:172.21.15.187:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f1ad8044070 con 0x7f1adc15bcf0 2024-09-21T00:25:23.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.562+0000 7f1ad27fc640 1 -- 172.21.15.112:0/1683671805 <== mon.2 v2:172.21.15.187:3301/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f1ad8005360 con 0x7f1adc15bcf0 2024-09-21T00:25:23.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.562+0000 7f1ae41a1640 1 -- 172.21.15.112:0/1683671805 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ac4005290 con 0x7f1adc15bcf0 2024-09-21T00:25:23.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.563+0000 7fda1b7fe640 1 -- 172.21.15.112:0/2664021158 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_scatterstat}]=0 v10)=0 v10) ==== 122+0+0 (secure 0 0 0) 0x7fda341f4f70 con 0x7fda1c003a80 2024-09-21T00:25:23.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.564+0000 7f364affd640 1 -- 172.21.15.112:0/1458517412 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command([{prefix=config set, name=mds_debug_frag}] v 0) -- 0x7f3658056590 con 0x7f36580091c0 2024-09-21T00:25:23.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.565+0000 7f1ad27fc640 1 -- 172.21.15.112:0/1683671805 <== mon.2 v2:172.21.15.187:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f1ad8032e20 con 0x7f1adc15bcf0 2024-09-21T00:25:23.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.581+0000 7fda33fff640 1 -- 172.21.15.112:0/2664021158 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda1c003a80 msgr2=0x7fda340c9eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:23.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.581+0000 7fda33fff640 1 --2- 172.21.15.112:0/2664021158 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda1c003a80 0x7fda340c9eb0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fda2c02b4a0 tx=0x7fda2c02bc40 comp rx=0 tx=0).stop 2024-09-21T00:25:23.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.581+0000 7fda33fff640 1 -- 172.21.15.112:0/2664021158 shutdown_connections 2024-09-21T00:25:23.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.581+0000 7fda33fff640 1 --2- 172.21.15.112:0/2664021158 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fda341f4620 0x7fda341f4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.581+0000 7fda33fff640 1 --2- 172.21.15.112:0/2664021158 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fda341f2520 0x7fda340c84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.581+0000 7fda33fff640 1 --2- 172.21.15.112:0/2664021158 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda1c003a80 0x7fda340c9eb0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.581+0000 7fda33fff640 1 -- 172.21.15.112:0/2664021158 >> 172.21.15.112:0/2664021158 conn(0x7fda34073460 msgr2=0x7fda340ca410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:23.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.581+0000 7fda33fff640 1 -- 172.21.15.112:0/2664021158 shutdown_connections 2024-09-21T00:25:23.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.581+0000 7fda33fff640 1 -- 172.21.15.112:0/2664021158 wait complete. 2024-09-21T00:25:23.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.583+0000 7f365d7fa640 1 -- 172.21.15.112:0/1458517412 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_frag}]=0 v11)=0 v11) ==== 115+0+0 (secure 0 0 0) 0x7f364c01eaf0 con 0x7f36580091c0 2024-09-21T00:25:23.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.585+0000 7f3665aa5640 1 -- 172.21.15.112:0/1458517412 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f36580091c0 msgr2=0x7f36580d8d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:23.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.585+0000 7f3665aa5640 1 --2- 172.21.15.112:0/1458517412 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f36580091c0 0x7f36580d8d10 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f364c00b7b0 tx=0x7f364c00bc70 comp rx=0 tx=0).stop 2024-09-21T00:25:23.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.585+0000 7f3665aa5640 1 -- 172.21.15.112:0/1458517412 shutdown_connections 2024-09-21T00:25:23.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.585+0000 7f3665aa5640 1 --2- 172.21.15.112:0/1458517412 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3658033ea0 0x7f36580e1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.585+0000 7f3665aa5640 1 --2- 172.21.15.112:0/1458517412 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3658036770 0x7f3658033960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.585+0000 7f3665aa5640 1 --2- 172.21.15.112:0/1458517412 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f36580091c0 0x7f36580d8d10 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.585+0000 7f3665aa5640 1 -- 172.21.15.112:0/1458517412 >> 172.21.15.112:0/1458517412 conn(0x7f365801eae0 msgr2=0x7f3658035a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:23.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.585+0000 7f3665aa5640 1 -- 172.21.15.112:0/1458517412 shutdown_connections 2024-09-21T00:25:23.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.585+0000 7f3665aa5640 1 -- 172.21.15.112:0/1458517412 wait complete. 2024-09-21T00:25:23.609 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_debug_scatterstat', 'True'], check_status=True, hostname='smithi112') 2024-09-21T00:25:23.609 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_op_complaint_time', '180'], check_status=True, hostname='smithi112') 2024-09-21T00:25:23.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.602+0000 7ff8bf369640 1 -- 172.21.15.112:0/1869010354 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command([{prefix=config set, name=mds_verify_scatter}] v 0) -- 0x7ff8b004ec90 con 0x7ff8b0054130 2024-09-21T00:25:23.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.634+0000 7ff8adffb640 1 -- 172.21.15.112:0/1869010354 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_verify_scatter}]=0 v12)=0 v12) ==== 119+0+0 (secure 0 0 0) 0x7ff8b807e030 con 0x7ff8b0054130 2024-09-21T00:25:23.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.635+0000 7ff8bf369640 1 -- 172.21.15.112:0/1869010354 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff8b0054130 msgr2=0x7ff8b019ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:23.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.635+0000 7ff8bf369640 1 --2- 172.21.15.112:0/1869010354 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff8b0054130 0x7ff8b019ccb0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ff8b8076070 tx=0x7ff8b806a0e0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.635+0000 7ff8bf369640 1 -- 172.21.15.112:0/1869010354 shutdown_connections 2024-09-21T00:25:23.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.635+0000 7ff8bf369640 1 --2- 172.21.15.112:0/1869010354 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8b0103940 0x7ff8b0103d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.635+0000 7ff8bf369640 1 --2- 172.21.15.112:0/1869010354 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff8b0054130 0x7ff8b019ccb0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.635+0000 7ff8bf369640 1 --2- 172.21.15.112:0/1869010354 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff8b00070b0 0x7ff8b0033700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.635+0000 7ff8bf369640 1 -- 172.21.15.112:0/1869010354 >> 172.21.15.112:0/1869010354 conn(0x7ff8b001d9a0 msgr2=0x7ff8b0005ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:23.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.638+0000 7fb3beffd640 1 -- 172.21.15.112:0/2772647217 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command([{prefix=config set, name=mds_op_complaint_time}] v 0) -- 0x7fb3d8204460 con 0x7fb3d80ca6f0 2024-09-21T00:25:23.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.638+0000 7ff8bf369640 1 -- 172.21.15.112:0/1869010354 shutdown_connections 2024-09-21T00:25:23.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.638+0000 7ff8bf369640 1 -- 172.21.15.112:0/1869010354 wait complete. 2024-09-21T00:25:23.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.647+0000 7fb3d57fa640 1 -- 172.21.15.112:0/2772647217 <== mon.2 v2:172.21.15.187:3301/0 7 ==== mon_command_ack([{prefix=config set, name=mds_op_complaint_time}]=0 v13)=0 v13) ==== 121+0+0 (secure 0 0 0) 0x7fb3cc005420 con 0x7fb3d80ca6f0 2024-09-21T00:25:23.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.656+0000 7fb3beffd640 1 -- 172.21.15.112:0/2772647217 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb3d80ca6f0 msgr2=0x7fb3d80b6990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:23.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.656+0000 7fb3beffd640 1 --2- 172.21.15.112:0/2772647217 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb3d80ca6f0 0x7fb3d80b6990 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fb3cc00e9c0 tx=0x7fb3cc00ee80 comp rx=0 tx=0).stop 2024-09-21T00:25:23.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.659+0000 7fb3beffd640 1 -- 172.21.15.112:0/2772647217 shutdown_connections 2024-09-21T00:25:23.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.659+0000 7fb3beffd640 1 --2- 172.21.15.112:0/2772647217 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb3d80cc0b0 0x7fb3d80b4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.659+0000 7fb3beffd640 1 --2- 172.21.15.112:0/2772647217 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb3d80ca6f0 0x7fb3d80b6990 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.659+0000 7fb3beffd640 1 --2- 172.21.15.112:0/2772647217 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb3d806ddf0 0x7fb3d80b6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.659+0000 7fb3beffd640 1 -- 172.21.15.112:0/2772647217 >> 172.21.15.112:0/2772647217 conn(0x7fb3d80b1b90 msgr2=0x7fb3d806d530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:23.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.665+0000 7fb3beffd640 1 -- 172.21.15.112:0/2772647217 shutdown_connections 2024-09-21T00:25:23.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.665+0000 7fb3beffd640 1 -- 172.21.15.112:0/2772647217 wait complete. 2024-09-21T00:25:23.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.689+0000 7f2bd0372640 1 -- 172.21.15.112:0/694724504 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) -- 0x7f2bc808bb10 con 0x7f2bc806f200 2024-09-21T00:25:23.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.693+0000 7f23bbfff640 1 -- 172.21.15.112:0/2190145055 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command([{prefix=config set, name=debug_mgr}] v 0) -- 0x7f23b0005580 con 0x7f23d008a720 2024-09-21T00:25:23.694 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_verify_scatter', 'True'], check_status=True, hostname='smithi112') 2024-09-21T00:25:23.695 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi112') 2024-09-21T00:25:23.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.696+0000 7f2bbeffd640 1 -- 172.21.15.112:0/694724504 <== mon.2 v2:172.21.15.187:3301/0 7 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v14)=0 v14) ==== 120+0+0 (secure 0 0 0) 0x7f2bc40199f0 con 0x7f2bc806f200 2024-09-21T00:25:23.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.707+0000 7f2bd0372640 1 -- 172.21.15.112:0/694724504 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2bc806f200 msgr2=0x7f2bc8165200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:23.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.707+0000 7f2bd0372640 1 --2- 172.21.15.112:0/694724504 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2bc806f200 0x7f2bc8165200 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f2bc400b440 tx=0x7f2bc400b900 comp rx=0 tx=0).stop 2024-09-21T00:25:23.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.707+0000 7f23c67fc640 1 -- 172.21.15.112:0/2190145055 <== mon.2 v2:172.21.15.187:3301/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mgr}]=0 v15)=0 v15) ==== 108+0+0 (secure 0 0 0) 0x7f23d01f6be0 con 0x7f23d008a720 2024-09-21T00:25:23.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.708+0000 7fc0dbfff640 1 -- 172.21.15.112:0/4290399292 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command([{prefix=config set, name=client_mount_timeout}] v 0) -- 0x7fc0f816e200 con 0x7fc0f806edf0 2024-09-21T00:25:23.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.709+0000 7f23d7d20640 1 -- 172.21.15.112:0/2190145055 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f23d008a720 msgr2=0x7f23d01f6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:23.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.709+0000 7f23d7d20640 1 --2- 172.21.15.112:0/2190145055 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f23d008a720 0x7f23d01f6290 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f23cc00c350 tx=0x7f23cc003bb0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.709+0000 7f2bd0372640 1 -- 172.21.15.112:0/694724504 shutdown_connections 2024-09-21T00:25:23.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.709+0000 7f2bd0372640 1 --2- 172.21.15.112:0/694724504 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2bc815d6d0 0x7f2bc8204ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.709+0000 7f2bd0372640 1 --2- 172.21.15.112:0/694724504 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2bc8073070 0x7f2bc815d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.709+0000 7f2bd0372640 1 --2- 172.21.15.112:0/694724504 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2bc806f200 0x7f2bc8165200 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.709+0000 7f2bd0372640 1 -- 172.21.15.112:0/694724504 >> 172.21.15.112:0/694724504 conn(0x7f2bc806cde0 msgr2=0x7f2bc816d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:23.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.709+0000 7f23d7d20640 1 -- 172.21.15.112:0/2190145055 shutdown_connections 2024-09-21T00:25:23.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.709+0000 7f23d7d20640 1 --2- 172.21.15.112:0/2190145055 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f23d01f67d0 0x7f23d01646c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.709+0000 7f23d7d20640 1 --2- 172.21.15.112:0/2190145055 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f23d008a720 0x7f23d01f6290 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.709+0000 7f23d7d20640 1 --2- 172.21.15.112:0/2190145055 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23bc004800 0x7f23d02058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.709+0000 7f23d7d20640 1 -- 172.21.15.112:0/2190145055 >> 172.21.15.112:0/2190145055 conn(0x7f23d006cde0 msgr2=0x7f23d016d6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:23.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.710+0000 7f23d7d20640 1 -- 172.21.15.112:0/2190145055 shutdown_connections 2024-09-21T00:25:23.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.710+0000 7f23d7d20640 1 -- 172.21.15.112:0/2190145055 wait complete. 2024-09-21T00:25:23.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.710+0000 7f2bd0372640 1 -- 172.21.15.112:0/694724504 shutdown_connections 2024-09-21T00:25:23.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.710+0000 7f3f43084640 1 -- 172.21.15.112:0/768275211 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command([{prefix=config set, name=mon_op_complaint_time}] v 0) -- 0x7f3f1c005580 con 0x7f3f3c0b3f90 2024-09-21T00:25:23.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.710+0000 7f2bd0372640 1 -- 172.21.15.112:0/694724504 wait complete. 2024-09-21T00:25:23.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.704+0000 7f1ae41a1640 1 -- 172.21.15.112:0/1683671805 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7f1ac4005580 con 0x7f1adc15bcf0 2024-09-21T00:25:23.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.715+0000 7fc0ee7fc640 1 -- 172.21.15.112:0/4290399292 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{prefix=config set, name=client_mount_timeout}]=0 v16)=0 v16) ==== 119+0+0 (secure 0 0 0) 0x7fc0f40325e0 con 0x7fc0f806edf0 2024-09-21T00:25:23.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.719+0000 7f3f397fa640 1 -- 172.21.15.112:0/768275211 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_op_complaint_time}]=0 v17)=0 v17) ==== 121+0+0 (secure 0 0 0) 0x7f3f2c0129b0 con 0x7f3f3c0b3f90 2024-09-21T00:25:23.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.722+0000 7fc0dbfff640 1 -- 172.21.15.112:0/4290399292 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc0f806edf0 msgr2=0x7fc0f808acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:23.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.722+0000 7fc0dbfff640 1 --2- 172.21.15.112:0/4290399292 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc0f806edf0 0x7fc0f808acc0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc0f4009920 tx=0x7fc0f4030870 comp rx=0 tx=0).stop 2024-09-21T00:25:23.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.722+0000 7f3f43084640 1 -- 172.21.15.112:0/768275211 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f3c0b3f90 msgr2=0x7f3f3c1f41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:23.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.722+0000 7f3f43084640 1 --2- 172.21.15.112:0/768275211 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f3c0b3f90 0x7f3f3c1f41c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f3f2c00c730 tx=0x7f3f2c00cbf0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.723+0000 7fc0dbfff640 1 -- 172.21.15.112:0/4290399292 shutdown_connections 2024-09-21T00:25:23.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.723+0000 7fc0dbfff640 1 --2- 172.21.15.112:0/4290399292 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc0f80cdab0 0x7fc0f81626b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.723+0000 7fc0dbfff640 1 --2- 172.21.15.112:0/4290399292 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0f8087700 0x7fc0f80cd570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.723+0000 7fc0dbfff640 1 --2- 172.21.15.112:0/4290399292 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc0f806edf0 0x7fc0f808acc0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.723+0000 7fc0dbfff640 1 -- 172.21.15.112:0/4290399292 >> 172.21.15.112:0/4290399292 conn(0x7fc0f806cde0 msgr2=0x7fc0f80cf3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:23.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.723+0000 7f1af759e640 1 -- 172.21.15.112:0/1711171651 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command([{prefix=config set, name=debug_ms}] v 0) -- 0x7f1adc005580 con 0x7f1af806e530 2024-09-21T00:25:23.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.723+0000 7f3f43084640 1 -- 172.21.15.112:0/768275211 shutdown_connections 2024-09-21T00:25:23.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.723+0000 7f3f43084640 1 --2- 172.21.15.112:0/768275211 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f3c0b3f90 0x7f3f3c1f41c0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.723+0000 7f3f43084640 1 --2- 172.21.15.112:0/768275211 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3f3c0aec20 0x7f3f3c15acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.723+0000 7f3f43084640 1 --2- 172.21.15.112:0/768275211 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f3c06edf0 0x7f3f3c15c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.723+0000 7f3f43084640 1 -- 172.21.15.112:0/768275211 >> 172.21.15.112:0/768275211 conn(0x7f3f3c06cde0 msgr2=0x7f3f3c06d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:23.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.724+0000 7fc0dbfff640 1 -- 172.21.15.112:0/4290399292 shutdown_connections 2024-09-21T00:25:23.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.724+0000 7fc0dbfff640 1 -- 172.21.15.112:0/4290399292 wait complete. 2024-09-21T00:25:23.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.725+0000 7f3f43084640 1 -- 172.21.15.112:0/768275211 shutdown_connections 2024-09-21T00:25:23.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.725+0000 7f3f43084640 1 -- 172.21.15.112:0/768275211 wait complete. 2024-09-21T00:25:23.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.734+0000 7f1ad27fc640 1 -- 172.21.15.112:0/1683671805 <== mon.2 v2:172.21.15.187:3301/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v18)=0 v18) ==== 121+0+0 (secure 0 0 0) 0x7f1ad8031990 con 0x7f1adc15bcf0 2024-09-21T00:25:23.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.736+0000 7f1ae41a1640 1 -- 172.21.15.112:0/1683671805 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1adc15bcf0 msgr2=0x7f1adc141e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:23.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.736+0000 7f1ae41a1640 1 --2- 172.21.15.112:0/1683671805 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1adc15bcf0 0x7f1adc141e10 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f1ad8009950 tx=0x7f1ad80035f0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.736+0000 7f1ae41a1640 1 -- 172.21.15.112:0/1683671805 shutdown_connections 2024-09-21T00:25:23.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.736+0000 7f1ae41a1640 1 --2- 172.21.15.112:0/1683671805 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1adc15bcf0 0x7f1adc141e10 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.736+0000 7f1ae41a1640 1 --2- 172.21.15.112:0/1683671805 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1adc1516d0 0x7f1adc1418d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.736+0000 7f1ae41a1640 1 --2- 172.21.15.112:0/1683671805 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1adc06f180 0x7f1adc150930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.736+0000 7f1ae41a1640 1 -- 172.21.15.112:0/1683671805 >> 172.21.15.112:0/1683671805 conn(0x7f1adc06cde0 msgr2=0x7f1adc06e080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:23.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.736+0000 7f1ae41a1640 1 -- 172.21.15.112:0/1683671805 shutdown_connections 2024-09-21T00:25:23.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.737+0000 7f1ae41a1640 1 -- 172.21.15.112:0/1683671805 wait complete. 2024-09-21T00:25:23.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.739+0000 7f1ae6ffd640 1 -- 172.21.15.112:0/1711171651 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_ms}]=0 v19)=0 v19) ==== 106+0+0 (secure 0 0 0) 0x7f1ae8003da0 con 0x7f1af806e530 2024-09-21T00:25:23.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.743+0000 7f1af759e640 1 -- 172.21.15.112:0/1711171651 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1af806e530 msgr2=0x7f1af8162150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:23.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.743+0000 7f1af759e640 1 --2- 172.21.15.112:0/1711171651 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1af806e530 0x7f1af8162150 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f1ae800c800 tx=0x7f1ae800ccc0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.743+0000 7f1af759e640 1 -- 172.21.15.112:0/1711171651 shutdown_connections 2024-09-21T00:25:23.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.743+0000 7f1af759e640 1 --2- 172.21.15.112:0/1711171651 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1af81606d0 0x7f1af8163bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.743+0000 7f1af759e640 1 --2- 172.21.15.112:0/1711171651 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1af808bb20 0x7f1af808b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.743+0000 7f1af759e640 1 --2- 172.21.15.112:0/1711171651 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1af806e530 0x7f1af8162150 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.743+0000 7f1af759e640 1 -- 172.21.15.112:0/1711171651 >> 172.21.15.112:0/1711171651 conn(0x7f1af80cfec0 msgr2=0x7f1af816e030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:23.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.743+0000 7f1af759e640 1 -- 172.21.15.112:0/1711171651 shutdown_connections 2024-09-21T00:25:23.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.743+0000 7f1af759e640 1 -- 172.21.15.112:0/1711171651 wait complete. 2024-09-21T00:25:23.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.766+0000 7fb0cbfff640 1 -- 172.21.15.112:0/1080893493 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_pool_no_app}] v 0) -- 0x7fb0e0006ba0 con 0x7fb0e00070d0 2024-09-21T00:25:23.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.769+0000 7fb0de7fc640 1 -- 172.21.15.112:0/1080893493 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_pool_no_app}]=0 v20)=0 v20) ==== 125+0+0 (secure 0 0 0) 0x7fb0e01a98c0 con 0x7fb0e00070d0 2024-09-21T00:25:23.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.770+0000 7fb0cbfff640 1 -- 172.21.15.112:0/1080893493 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0e00070d0 msgr2=0x7fb0e01145c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:23.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.770+0000 7fb0cbfff640 1 --2- 172.21.15.112:0/1080893493 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0e00070d0 0x7fb0e01145c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fb0e806b0e0 tx=0x7fb0e806a0e0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.770+0000 7fb0cbfff640 1 -- 172.21.15.112:0/1080893493 shutdown_connections 2024-09-21T00:25:23.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.770+0000 7fb0cbfff640 1 --2- 172.21.15.112:0/1080893493 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb0e00f7560 0x7fb0e004b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.770+0000 7fb0cbfff640 1 --2- 172.21.15.112:0/1080893493 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb0e0055100 0x7fb0e01a9380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.770+0000 7fb0cbfff640 1 --2- 172.21.15.112:0/1080893493 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0e00070d0 0x7fb0e01145c0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.770+0000 7fb0cbfff640 1 -- 172.21.15.112:0/1080893493 >> 172.21.15.112:0/1080893493 conn(0x7fb0e001d9c0 msgr2=0x7fb0e00519a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:23.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.770+0000 7fb0cbfff640 1 -- 172.21.15.112:0/1080893493 shutdown_connections 2024-09-21T00:25:23.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.770+0000 7fb0cbfff640 1 -- 172.21.15.112:0/1080893493 wait complete. 2024-09-21T00:25:23.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.772+0000 7fee0b25c640 1 -- 172.21.15.112:0/2547265527 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7fee04215770 con 0x7fee0406ddf0 2024-09-21T00:25:23.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.775+0000 7fb144ff9640 1 -- 172.21.15.112:0/810229568 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command([{prefix=config set, name=debug_client}] v 0) -- 0x7fb15808ad80 con 0x7fb158073070 2024-09-21T00:25:23.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.775+0000 7fee017fa640 1 -- 172.21.15.112:0/2547265527 <== mon.2 v2:172.21.15.187:3301/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v21)=0 v21) ==== 121+0+0 (secure 0 0 0) 0x7fee04215770 con 0x7fee0406ddf0 2024-09-21T00:25:23.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.776+0000 7fee0b25c640 1 -- 172.21.15.112:0/2547265527 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fee0406ddf0 msgr2=0x7fee04213190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:23.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.776+0000 7fee0b25c640 1 --2- 172.21.15.112:0/2547265527 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fee0406ddf0 0x7fee04213190 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fedfc00c810 tx=0x7fedfc00ccd0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.776+0000 7fee0b25c640 1 -- 172.21.15.112:0/2547265527 shutdown_connections 2024-09-21T00:25:23.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.776+0000 7fee0b25c640 1 --2- 172.21.15.112:0/2547265527 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee040cc0b0 0x7fee04215230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.776+0000 7fee0b25c640 1 --2- 172.21.15.112:0/2547265527 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fee040ca6f0 0x7fee04214cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.776+0000 7fee0b25c640 1 --2- 172.21.15.112:0/2547265527 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fee0406ddf0 0x7fee04213190 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.776+0000 7fee0b25c640 1 -- 172.21.15.112:0/2547265527 >> 172.21.15.112:0/2547265527 conn(0x7fee040b1b90 msgr2=0x7fee040cdc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:23.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.777+0000 7fee0b25c640 1 -- 172.21.15.112:0/2547265527 shutdown_connections 2024-09-21T00:25:23.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.777+0000 7fee0b25c640 1 -- 172.21.15.112:0/2547265527 wait complete. 2024-09-21T00:25:23.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.785+0000 7fb146ffd640 1 -- 172.21.15.112:0/810229568 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_client}]=0 v22)=0 v22) ==== 111+0+0 (secure 0 0 0) 0x7fb158179ec0 con 0x7fb158073070 2024-09-21T00:25:23.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.787+0000 7f75167fc640 1 -- 172.21.15.112:0/391471625 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) -- 0x7f75240565f0 con 0x7f75240e4370 2024-09-21T00:25:23.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.787+0000 7fb144ff9640 1 -- 172.21.15.112:0/810229568 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb158073070 msgr2=0x7fb15820a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:23.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.787+0000 7fb144ff9640 1 --2- 172.21.15.112:0/810229568 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb158073070 0x7fb15820a040 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fb14800c370 tx=0x7fb14800c830 comp rx=0 tx=0).stop 2024-09-21T00:25:23.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.787+0000 7fb144ff9640 1 -- 172.21.15.112:0/810229568 shutdown_connections 2024-09-21T00:25:23.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.787+0000 7fb144ff9640 1 --2- 172.21.15.112:0/810229568 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb15816ec10 0x7fb158205f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.787+0000 7fb144ff9640 1 --2- 172.21.15.112:0/810229568 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb15808b800 0x7fb158179980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.787+0000 7fb144ff9640 1 --2- 172.21.15.112:0/810229568 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb158073070 0x7fb15820a040 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.787+0000 7fb144ff9640 1 -- 172.21.15.112:0/810229568 >> 172.21.15.112:0/810229568 conn(0x7fb15806cde0 msgr2=0x7fb15806edf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:23.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.787+0000 7fb144ff9640 1 -- 172.21.15.112:0/810229568 shutdown_connections 2024-09-21T00:25:23.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.787+0000 7fb144ff9640 1 -- 172.21.15.112:0/810229568 wait complete. 2024-09-21T00:25:23.790 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi112') 2024-09-21T00:25:23.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.794+0000 7f7528ff9640 1 -- 172.21.15.112:0/391471625 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v23)=0 v23) ==== 120+0+0 (secure 0 0 0) 0x7f7524056180 con 0x7f75240e4370 2024-09-21T00:25:23.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.796+0000 7f75167fc640 1 -- 172.21.15.112:0/391471625 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75240e4370 msgr2=0x7f75240e2c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:23.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.796+0000 7f75167fc640 1 --2- 172.21.15.112:0/391471625 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75240e4370 0x7f75240e2c20 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f752000c730 tx=0x7f752000cbf0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.796+0000 7f75167fc640 1 -- 172.21.15.112:0/391471625 shutdown_connections 2024-09-21T00:25:23.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.796+0000 7f75167fc640 1 --2- 172.21.15.112:0/391471625 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75240e4370 0x7f75240e2c20 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.796+0000 7f75167fc640 1 --2- 172.21.15.112:0/391471625 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f75240e3f60 0x7f75240e26e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.796+0000 7f75167fc640 1 --2- 172.21.15.112:0/391471625 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7524055d70 0x7f752404ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.796+0000 7f75167fc640 1 -- 172.21.15.112:0/391471625 >> 172.21.15.112:0/391471625 conn(0x7f752401eab0 msgr2=0x7f7524006840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:23.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.796+0000 7f75167fc640 1 -- 172.21.15.112:0/391471625 shutdown_connections 2024-09-21T00:25:23.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.796+0000 7f75167fc640 1 -- 172.21.15.112:0/391471625 wait complete. 2024-09-21T00:25:23.809 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi112') 2024-09-21T00:25:23.809 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mgr', 'client_mount_timeout', '30'], check_status=True, hostname='smithi112') 2024-09-21T00:25:23.809 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mgr', 'debug_client', '20'], check_status=True, hostname='smithi112') 2024-09-21T00:25:23.809 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mgr', 'debug_mgr', '20'], check_status=True, hostname='smithi112') 2024-09-21T00:25:23.809 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mgr', 'debug_ms', '1'], check_status=True, hostname='smithi112') 2024-09-21T00:25:23.809 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mgr', 'mon_warn_on_pool_no_app', 'False'], check_status=True, hostname='smithi112') 2024-09-21T00:25:23.809 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mon', 'mon_op_complaint_time', '120'], check_status=True, hostname='smithi112') 2024-09-21T00:25:23.810 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'osd', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi112') 2024-09-21T00:25:23.810 DEBUG:teuthology.orchestra.run.smithi112:> sudo ceph --cluster ceph config dump 2024-09-21T00:25:23.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.959+0000 7f75df270640 1 -- 172.21.15.112:0/2338726653 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75d815f150 msgr2=0x7f75d815f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:23.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.959+0000 7f75df270640 1 --2- 172.21.15.112:0/2338726653 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75d815f150 0x7f75d815f560 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f75d8073dc0 tx=0x7f75c802a4d0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.959+0000 7f75df270640 1 -- 172.21.15.112:0/2338726653 shutdown_connections 2024-09-21T00:25:23.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.959+0000 7f75df270640 1 --2- 172.21.15.112:0/2338726653 >> v2:172.21.15.187:3300/0 conn(0x7f75d815faa0 0x7f75d806e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.959+0000 7f75df270640 1 --2- 172.21.15.112:0/2338726653 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75d815f150 0x7f75d815f560 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.959+0000 7f75df270640 1 --2- 172.21.15.112:0/2338726653 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f75d80cee70 0x7f75d815b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.959+0000 7f75df270640 1 -- 172.21.15.112:0/2338726653 >> 172.21.15.112:0/2338726653 conn(0x7f75d8073e30 msgr2=0x7f75d8074240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:23.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.959+0000 7f75df270640 1 -- 172.21.15.112:0/2338726653 shutdown_connections 2024-09-21T00:25:23.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.959+0000 7f75df270640 1 -- 172.21.15.112:0/2338726653 wait complete. 2024-09-21T00:25:23.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.961+0000 7f75df270640 1 Processor -- start 2024-09-21T00:25:23.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.961+0000 7f75df270640 1 -- start start 2024-09-21T00:25:23.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.962+0000 7f75df270640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f75d80cee70 0x7f75d8151140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:23.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.962+0000 7f75df270640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f75d815f150 0x7f75d81588f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:23.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.962+0000 7f75df270640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75d815faa0 0x7f75d8158e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:23.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.962+0000 7f75df270640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f75d806f210 con 0x7f75d815faa0 2024-09-21T00:25:23.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.962+0000 7f75df270640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f75d806f090 con 0x7f75d80cee70 2024-09-21T00:25:23.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.962+0000 7f75df270640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f75d806f390 con 0x7f75d815f150 2024-09-21T00:25:23.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.962+0000 7f75dd80d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75d815faa0 0x7f75d8158e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:23.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.962+0000 7f75dd80d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75d815faa0 0x7f75d8158e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55722/0 (socket says 172.21.15.112:55722) 2024-09-21T00:25:23.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.962+0000 7f75dd80d640 1 -- 172.21.15.112:0/3505863134 learned_addr learned my addr 172.21.15.112:0/3505863134 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:25:23.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.962+0000 7f75dc80b640 1 --2- 172.21.15.112:0/3505863134 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f75d815f150 0x7f75d81588f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:23.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.962+0000 7f75dd00c640 1 --2- 172.21.15.112:0/3505863134 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f75d80cee70 0x7f75d8151140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:23.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.962+0000 7f75dd80d640 1 -- 172.21.15.112:0/3505863134 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f75d815f150 msgr2=0x7f75d81588f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:23.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.962+0000 7f75dd80d640 1 --2- 172.21.15.112:0/3505863134 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f75d815f150 0x7f75d81588f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.962+0000 7f75dd80d640 1 -- 172.21.15.112:0/3505863134 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f75d80cee70 msgr2=0x7f75d8151140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:23.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.962+0000 7f75dd80d640 1 --2- 172.21.15.112:0/3505863134 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f75d80cee70 0x7f75d8151140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:23.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.962+0000 7f75dd80d640 1 -- 172.21.15.112:0/3505863134 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75d8160520 con 0x7f75d815faa0 2024-09-21T00:25:23.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.962+0000 7f75dd80d640 1 --2- 172.21.15.112:0/3505863134 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75d815faa0 0x7f75d8158e30 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f75d000ea80 tx=0x7f75d000ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:25:23.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.963+0000 7f75cdffb640 1 -- 172.21.15.112:0/3505863134 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75d0004b20 con 0x7f75d815faa0 2024-09-21T00:25:23.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.963+0000 7f75cdffb640 1 -- 172.21.15.112:0/3505863134 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f75d0004ce0 con 0x7f75d815faa0 2024-09-21T00:25:23.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.963+0000 7f75df270640 1 -- 172.21.15.112:0/3505863134 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75d8159db0 con 0x7f75d815faa0 2024-09-21T00:25:23.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.963+0000 7f75cdffb640 1 -- 172.21.15.112:0/3505863134 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75d0005280 con 0x7f75d815faa0 2024-09-21T00:25:23.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.963+0000 7f75df270640 1 -- 172.21.15.112:0/3505863134 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75d815e870 con 0x7f75d815faa0 2024-09-21T00:25:23.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.963+0000 7f75cdffb640 1 -- 172.21.15.112:0/3505863134 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f75d00057b0 con 0x7f75d815faa0 2024-09-21T00:25:23.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.963+0000 7f75cdffb640 1 -- 172.21.15.112:0/3505863134 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f75d0018770 con 0x7f75d815faa0 2024-09-21T00:25:23.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.964+0000 7f75df270640 1 -- 172.21.15.112:0/3505863134 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75d80b5850 con 0x7f75d815faa0 2024-09-21T00:25:23.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:23.966+0000 7f75cdffb640 1 -- 172.21.15.112:0/3505863134 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f75d0026070 con 0x7f75d815faa0 2024-09-21T00:25:24.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:24.027+0000 7f75df270640 1 -- 172.21.15.112:0/3505863134 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f75d80b5550 con 0x7f75d815faa0 2024-09-21T00:25:24.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:24.028+0000 7f75cdffb640 1 -- 172.21.15.112:0/3505863134 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v23) ==== 59+0+1472 (secure 0 0 0) 0x7f75d0018a80 con 0x7f75d815faa0 2024-09-21T00:25:24.029 INFO:teuthology.orchestra.run.smithi112.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-21T00:25:24.029 INFO:teuthology.orchestra.run.smithi112.stdout:mon advanced mon_op_complaint_time 120 2024-09-21T00:25:24.029 INFO:teuthology.orchestra.run.smithi112.stdout:mgr advanced client_mount_timeout 30 2024-09-21T00:25:24.029 INFO:teuthology.orchestra.run.smithi112.stdout:mgr advanced debug_client 20/20 2024-09-21T00:25:24.029 INFO:teuthology.orchestra.run.smithi112.stdout:mgr advanced debug_mgr 20/20 2024-09-21T00:25:24.029 INFO:teuthology.orchestra.run.smithi112.stdout:mgr advanced debug_ms 1/1 2024-09-21T00:25:24.029 INFO:teuthology.orchestra.run.smithi112.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-21T00:25:24.029 INFO:teuthology.orchestra.run.smithi112.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-21T00:25:24.029 INFO:teuthology.orchestra.run.smithi112.stdout:mds advanced debug_mds 20/20 2024-09-21T00:25:24.029 INFO:teuthology.orchestra.run.smithi112.stdout:mds advanced debug_mds_balancer 20/20 2024-09-21T00:25:24.029 INFO:teuthology.orchestra.run.smithi112.stdout:mds advanced debug_ms 2/2 2024-09-21T00:25:24.029 INFO:teuthology.orchestra.run.smithi112.stdout:mds dev mds_debug_frag true 2024-09-21T00:25:24.029 INFO:teuthology.orchestra.run.smithi112.stdout:mds dev mds_debug_scatterstat true 2024-09-21T00:25:24.030 INFO:teuthology.orchestra.run.smithi112.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-21T00:25:24.030 INFO:teuthology.orchestra.run.smithi112.stdout:mds dev mds_verify_scatter true 2024-09-21T00:25:24.030 INFO:teuthology.orchestra.run.smithi112.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-21T00:25:24.030 INFO:teuthology.orchestra.run.smithi112.stdout:mds advanced rados_mon_op_timeout 900 2024-09-21T00:25:24.030 INFO:teuthology.orchestra.run.smithi112.stdout:mds advanced rados_osd_op_timeout 900 2024-09-21T00:25:24.030 INFO:teuthology.orchestra.run.smithi112.stdout:client advanced client_mount_timeout 600 2024-09-21T00:25:24.030 INFO:teuthology.orchestra.run.smithi112.stdout:client advanced debug_client 20/20 2024-09-21T00:25:24.030 INFO:teuthology.orchestra.run.smithi112.stdout:client advanced debug_ms 1/1 2024-09-21T00:25:24.030 INFO:teuthology.orchestra.run.smithi112.stdout:client advanced rados_mon_op_timeout 900 2024-09-21T00:25:24.030 INFO:teuthology.orchestra.run.smithi112.stdout:client advanced rados_osd_op_timeout 900 2024-09-21T00:25:24.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:24.029+0000 7f75df270640 1 -- 172.21.15.112:0/3505863134 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75d815faa0 msgr2=0x7f75d8158e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:24.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:24.029+0000 7f75df270640 1 --2- 172.21.15.112:0/3505863134 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75d815faa0 0x7f75d8158e30 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f75d000ea80 tx=0x7f75d000ef40 comp rx=0 tx=0).stop 2024-09-21T00:25:24.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:24.029+0000 7f75df270640 1 -- 172.21.15.112:0/3505863134 shutdown_connections 2024-09-21T00:25:24.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:24.029+0000 7f75df270640 1 --2- 172.21.15.112:0/3505863134 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75d815faa0 0x7f75d8158e30 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:24.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:24.029+0000 7f75df270640 1 --2- 172.21.15.112:0/3505863134 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f75d815f150 0x7f75d81588f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:24.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:24.029+0000 7f75df270640 1 --2- 172.21.15.112:0/3505863134 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f75d80cee70 0x7f75d8151140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:24.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:24.029+0000 7f75df270640 1 -- 172.21.15.112:0/3505863134 >> 172.21.15.112:0/3505863134 conn(0x7f75d8073e30 msgr2=0x7f75d80cf350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:24.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:24.029+0000 7f75df270640 1 -- 172.21.15.112:0/3505863134 shutdown_connections 2024-09-21T00:25:24.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:24.029+0000 7f75df270640 1 -- 172.21.15.112:0/3505863134 wait complete. 2024-09-21T00:25:24.041 INFO:tasks.ceph:Setting crush tunables to default 2024-09-21T00:25:24.041 DEBUG:teuthology.orchestra.run.smithi112:> sudo ceph --cluster ceph osd crush tunables default 2024-09-21T00:25:24.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:24.152+0000 7f235f8a4640 1 -- 172.21.15.112:0/903461101 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23580d0bb0 msgr2=0x7f23580b5be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:24.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:24.152+0000 7f235f8a4640 1 --2- 172.21.15.112:0/903461101 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23580d0bb0 0x7f23580b5be0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f23580cc9c0 tx=0x7f235002a820 comp rx=0 tx=0).stop 2024-09-21T00:25:24.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:24.152+0000 7f235f8a4640 1 -- 172.21.15.112:0/903461101 shutdown_connections 2024-09-21T00:25:24.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:24.152+0000 7f235f8a4640 1 --2- 172.21.15.112:0/903461101 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23580d0bb0 0x7f23580b5be0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:24.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:24.152+0000 7f235f8a4640 1 --2- 172.21.15.112:0/903461101 >> v2:172.21.15.187:3300/0 conn(0x7f235815ca90 0x7f23580d0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:24.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:24.152+0000 7f235f8a4640 1 -- 172.21.15.112:0/903461101 >> 172.21.15.112:0/903461101 conn(0x7f23580b46e0 msgr2=0x7f23580d23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:24.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:24.152+0000 7f235f8a4640 1 -- 172.21.15.112:0/903461101 shutdown_connections 2024-09-21T00:25:24.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:24.152+0000 7f235f8a4640 1 -- 172.21.15.112:0/903461101 wait complete. 2024-09-21T00:25:24.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:24.153+0000 7f235f8a4640 1 Processor -- start 2024-09-21T00:25:24.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:24.153+0000 7f235f8a4640 1 -- start start 2024-09-21T00:25:24.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:24.155+0000 7f235f8a4640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f23580ccc40 0x7f23581f3950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:24.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:24.155+0000 7f235f8a4640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f23580d0bb0 0x7f235814f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:24.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:24.155+0000 7f235f8a4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f235815ca90 0x7f23580cadb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:24.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:24.155+0000 7f235f8a4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f23580b6670 con 0x7f235815ca90 2024-09-21T00:25:24.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:24.155+0000 7f235f8a4640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f23580b64f0 con 0x7f23580d0bb0 2024-09-21T00:25:24.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:24.155+0000 7f235f8a4640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f23580b67f0 con 0x7f23580ccc40 2024-09-21T00:25:24.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:24.155+0000 7f235de41640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f235815ca90 0x7f23580cadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:24.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:24.155+0000 7f235de41640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f235815ca90 0x7f23580cadb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55738/0 (socket says 172.21.15.112:55738) 2024-09-21T00:25:24.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:24.155+0000 7f235de41640 1 -- 172.21.15.112:0/3818553468 learned_addr learned my addr 172.21.15.112:0/3818553468 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:25:24.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:24.155+0000 7f235ce3f640 1 --2- 172.21.15.112:0/3818553468 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f23580d0bb0 0x7f235814f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:24.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:24.155+0000 7f235d640640 1 --2- 172.21.15.112:0/3818553468 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f23580ccc40 0x7f23581f3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:24.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:24.155+0000 7f235de41640 1 -- 172.21.15.112:0/3818553468 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f23580ccc40 msgr2=0x7f23581f3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:24.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:24.155+0000 7f235de41640 1 --2- 172.21.15.112:0/3818553468 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f23580ccc40 0x7f23581f3950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:24.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:24.155+0000 7f235de41640 1 -- 172.21.15.112:0/3818553468 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f23580d0bb0 msgr2=0x7f235814f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:24.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:24.155+0000 7f235de41640 1 --2- 172.21.15.112:0/3818553468 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f23580d0bb0 0x7f235814f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:24.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:24.155+0000 7f235de41640 1 -- 172.21.15.112:0/3818553468 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23581f4250 con 0x7f235815ca90 2024-09-21T00:25:24.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:24.155+0000 7f235de41640 1 --2- 172.21.15.112:0/3818553468 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f235815ca90 0x7f23580cadb0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f2350009ca0 tx=0x7f2350032c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:25:24.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:24.156+0000 7f234dffb640 1 -- 172.21.15.112:0/3818553468 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f235003f020 con 0x7f235815ca90 2024-09-21T00:25:24.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:24.156+0000 7f234dffb640 1 -- 172.21.15.112:0/3818553468 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2350004040 con 0x7f235815ca90 2024-09-21T00:25:24.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:24.156+0000 7f235f8a4640 1 -- 172.21.15.112:0/3818553468 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23581f4530 con 0x7f235815ca90 2024-09-21T00:25:24.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:24.156+0000 7f234dffb640 1 -- 172.21.15.112:0/3818553468 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2350037040 con 0x7f235815ca90 2024-09-21T00:25:24.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:24.156+0000 7f235f8a4640 1 -- 172.21.15.112:0/3818553468 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23580c9490 con 0x7f235815ca90 2024-09-21T00:25:24.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:24.156+0000 7f234dffb640 1 -- 172.21.15.112:0/3818553468 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f2350042020 con 0x7f235815ca90 2024-09-21T00:25:24.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:24.156+0000 7f234dffb640 1 -- 172.21.15.112:0/3818553468 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f2350044c70 con 0x7f235815ca90 2024-09-21T00:25:24.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:24.157+0000 7f235f8a4640 1 -- 172.21.15.112:0/3818553468 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23580b6d80 con 0x7f235815ca90 2024-09-21T00:25:24.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:24.160+0000 7f234dffb640 1 -- 172.21.15.112:0/3818553468 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f2350049030 con 0x7f235815ca90 2024-09-21T00:25:24.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:24.224+0000 7f235f8a4640 1 -- 172.21.15.112:0/3818553468 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f23580d0790 con 0x7f235815ca90 2024-09-21T00:25:24.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:24.226+0000 7f234dffb640 1 -- 172.21.15.112:0/3818553468 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7f2350044400 con 0x7f235815ca90 2024-09-21T00:25:24.227 INFO:teuthology.orchestra.run.smithi112.stderr:adjusted tunables profile to default 2024-09-21T00:25:24.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:24.227+0000 7f235f8a4640 1 -- 172.21.15.112:0/3818553468 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f235815ca90 msgr2=0x7f23580cadb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:24.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:24.227+0000 7f235f8a4640 1 --2- 172.21.15.112:0/3818553468 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f235815ca90 0x7f23580cadb0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f2350009ca0 tx=0x7f2350032c90 comp rx=0 tx=0).stop 2024-09-21T00:25:24.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:24.227+0000 7f235f8a4640 1 -- 172.21.15.112:0/3818553468 shutdown_connections 2024-09-21T00:25:24.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:24.227+0000 7f235f8a4640 1 --2- 172.21.15.112:0/3818553468 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f235815ca90 0x7f23580cadb0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:24.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:24.227+0000 7f235f8a4640 1 --2- 172.21.15.112:0/3818553468 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f23580d0bb0 0x7f235814f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:24.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:24.227+0000 7f235f8a4640 1 --2- 172.21.15.112:0/3818553468 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f23580ccc40 0x7f23581f3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:24.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:24.227+0000 7f235f8a4640 1 -- 172.21.15.112:0/3818553468 >> 172.21.15.112:0/3818553468 conn(0x7f23580b46e0 msgr2=0x7f23580cd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:24.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:24.227+0000 7f235f8a4640 1 -- 172.21.15.112:0/3818553468 shutdown_connections 2024-09-21T00:25:24.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:24.227+0000 7f235f8a4640 1 -- 172.21.15.112:0/3818553468 wait complete. 2024-09-21T00:25:24.239 INFO:tasks.ceph:check_enable_crimson: False 2024-09-21T00:25:24.239 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-21T00:25:24.239 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T00:25:24.239 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-21T00:25:24.263 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T00:25:24.263 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-21T00:25:24.329 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T00:25:24.330 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-21T00:25:24.394 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T00:25:24.394 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-21T00:25:24.459 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-09-21T00:25:24.459 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-09-21T00:25:24.483 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-09-21T00:25:24.483 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-09-21T00:25:24.549 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-09-21T00:25:24.549 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-09-21T00:25:24.613 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-09-21T00:25:24.614 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-09-21T00:25:24.679 DEBUG:teuthology.orchestra.run.smithi187:> sudo ceph --cluster ceph osd new 9c575086-67d2-4de2-b01b-d44dcc21d604 0 2024-09-21T00:25:24.833 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:24.831+0000 7f12cd3f1640 1 -- 172.21.15.187:0/561534959 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12c806def0 msgr2=0x7f12c806f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:24.833 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:24.831+0000 7f12cd3f1640 1 --2- 172.21.15.187:0/561534959 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12c806def0 0x7f12c806f410 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f12b8009830 tx=0x7f12b802a5c0 comp rx=0 tx=0).stop 2024-09-21T00:25:24.833 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:24.831+0000 7f12cd3f1640 1 -- 172.21.15.187:0/561534959 shutdown_connections 2024-09-21T00:25:24.833 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:24.831+0000 7f12cd3f1640 1 --2- 172.21.15.187:0/561534959 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12c806def0 0x7f12c806f410 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:24.833 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:24.831+0000 7f12cd3f1640 1 -- 172.21.15.187:0/561534959 >> 172.21.15.187:0/561534959 conn(0x7f12c80cdca0 msgr2=0x7f12c80ce0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:24.833 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:24.831+0000 7f12cd3f1640 1 -- 172.21.15.187:0/561534959 shutdown_connections 2024-09-21T00:25:24.833 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:24.831+0000 7f12cd3f1640 1 -- 172.21.15.187:0/561534959 wait complete. 2024-09-21T00:25:24.834 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:24.833+0000 7f12cd3f1640 1 Processor -- start 2024-09-21T00:25:24.835 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:24.833+0000 7f12cd3f1640 1 -- start start 2024-09-21T00:25:24.838 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:24.835+0000 7f12cd3f1640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f12c806def0 0x7f12c8155130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:24.838 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:24.835+0000 7f12cd3f1640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f12c80b5260 0x7f12c81f5890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:24.838 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:24.835+0000 7f12cd3f1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12c80b5670 0x7f12c81f5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:24.838 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:24.835+0000 7f12cd3f1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f12c8071480 con 0x7f12c80b5670 2024-09-21T00:25:24.838 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:24.835+0000 7f12cd3f1640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f12c8071300 con 0x7f12c80b5260 2024-09-21T00:25:24.838 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:24.835+0000 7f12cd3f1640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f12c8071600 con 0x7f12c806def0 2024-09-21T00:25:24.838 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:24.835+0000 7f12c67fc640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f12c80b5260 0x7f12c81f5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:24.838 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:24.835+0000 7f12c6ffd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f12c806def0 0x7f12c8155130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:24.838 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:24.835+0000 7f12c67fc640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f12c80b5260 0x7f12c81f5890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.187:36196/0 (socket says 172.21.15.187:36196) 2024-09-21T00:25:24.838 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:24.835+0000 7f12c6ffd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f12c806def0 0x7f12c8155130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.187:33566/0 (socket says 172.21.15.187:33566) 2024-09-21T00:25:24.838 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:24.835+0000 7f12c67fc640 1 -- 172.21.15.187:0/3905245991 learned_addr learned my addr 172.21.15.187:0/3905245991 (peer_addr_for_me v2:172.21.15.187:0/0) 2024-09-21T00:25:24.839 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:24.835+0000 7f12c77fe640 1 --2- 172.21.15.187:0/3905245991 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12c80b5670 0x7f12c81f5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:24.839 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:24.835+0000 7f12c6ffd640 1 -- 172.21.15.187:0/3905245991 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f12c80b5260 msgr2=0x7f12c81f5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:24.839 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:24.835+0000 7f12c6ffd640 1 --2- 172.21.15.187:0/3905245991 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f12c80b5260 0x7f12c81f5890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:24.839 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:24.835+0000 7f12c6ffd640 1 -- 172.21.15.187:0/3905245991 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12c80b5670 msgr2=0x7f12c81f5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:24.839 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:24.835+0000 7f12c6ffd640 1 --2- 172.21.15.187:0/3905245991 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12c80b5670 0x7f12c81f5dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:24.839 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:24.835+0000 7f12c6ffd640 1 -- 172.21.15.187:0/3905245991 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12c8146410 con 0x7f12c806def0 2024-09-21T00:25:24.839 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:24.835+0000 7f12c67fc640 1 --2- 172.21.15.187:0/3905245991 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f12c80b5260 0x7f12c81f5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:25:24.839 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:24.835+0000 7f12c77fe640 1 --2- 172.21.15.187:0/3905245991 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12c80b5670 0x7f12c81f5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:25:24.839 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:24.835+0000 7f12c6ffd640 1 --2- 172.21.15.187:0/3905245991 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f12c806def0 0x7f12c8155130 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f12b8031000 tx=0x7f12b8033ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:25:24.839 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:24.836+0000 7f12af7fe640 1 -- 172.21.15.187:0/3905245991 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12b8031b50 con 0x7f12c806def0 2024-09-21T00:25:24.839 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:24.836+0000 7f12cd3f1640 1 -- 172.21.15.187:0/3905245991 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12c8148f40 con 0x7f12c806def0 2024-09-21T00:25:24.839 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:24.836+0000 7f12af7fe640 1 -- 172.21.15.187:0/3905245991 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f12b8031de0 con 0x7f12c806def0 2024-09-21T00:25:24.839 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:24.836+0000 7f12af7fe640 1 -- 172.21.15.187:0/3905245991 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12b8005280 con 0x7f12c806def0 2024-09-21T00:25:24.839 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:24.836+0000 7f12cd3f1640 1 -- 172.21.15.187:0/3905245991 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f12c80b0e70 con 0x7f12c806def0 2024-09-21T00:25:24.840 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:24.836+0000 7f12af7fe640 1 -- 172.21.15.187:0/3905245991 <== mon.2 v2:172.21.15.187:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f12b80047a0 con 0x7f12c806def0 2024-09-21T00:25:24.840 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:24.836+0000 7f12af7fe640 1 -- 172.21.15.187:0/3905245991 <== mon.2 v2:172.21.15.187:3301/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7f12b80333a0 con 0x7f12c806def0 2024-09-21T00:25:24.840 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:24.837+0000 7f12cd3f1640 1 -- 172.21.15.187:0/3905245991 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12c81465e0 con 0x7f12c806def0 2024-09-21T00:25:24.840 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:24.839+0000 7f12af7fe640 1 -- 172.21.15.187:0/3905245991 <== mon.2 v2:172.21.15.187:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f12b803f030 con 0x7f12c806def0 2024-09-21T00:25:24.910 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:24.909+0000 7f12cd3f1640 1 -- 172.21.15.187:0/3905245991 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "osd new", "uuid": "9c575086-67d2-4de2-b01b-d44dcc21d604", "id": 0} v 0) -- 0x7f12c80b0c60 con 0x7f12c806def0 2024-09-21T00:25:24.915 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:24.914+0000 7f12af7fe640 1 -- 172.21.15.187:0/3905245991 <== mon.2 v2:172.21.15.187:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "9c575086-67d2-4de2-b01b-d44dcc21d604", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7f12b8031770 con 0x7f12c806def0 2024-09-21T00:25:24.915 INFO:teuthology.orchestra.run.smithi187.stdout:0 2024-09-21T00:25:24.916 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:24.915+0000 7f12cd3f1640 1 -- 172.21.15.187:0/3905245991 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f12c806def0 msgr2=0x7f12c8155130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:24.916 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:24.915+0000 7f12cd3f1640 1 --2- 172.21.15.187:0/3905245991 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f12c806def0 0x7f12c8155130 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f12b8031000 tx=0x7f12b8033ba0 comp rx=0 tx=0).stop 2024-09-21T00:25:24.916 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:24.915+0000 7f12cd3f1640 1 -- 172.21.15.187:0/3905245991 shutdown_connections 2024-09-21T00:25:24.916 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:24.915+0000 7f12cd3f1640 1 --2- 172.21.15.187:0/3905245991 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12c80b5670 0x7f12c81f5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:24.916 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:24.915+0000 7f12cd3f1640 1 --2- 172.21.15.187:0/3905245991 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f12c80b5260 0x7f12c81f5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:24.917 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:24.915+0000 7f12cd3f1640 1 --2- 172.21.15.187:0/3905245991 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f12c806def0 0x7f12c8155130 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:24.917 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:24.915+0000 7f12cd3f1640 1 -- 172.21.15.187:0/3905245991 >> 172.21.15.187:0/3905245991 conn(0x7f12c80cdca0 msgr2=0x7f12c815dca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:24.917 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:24.915+0000 7f12cd3f1640 1 -- 172.21.15.187:0/3905245991 shutdown_connections 2024-09-21T00:25:24.917 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:24.915+0000 7f12cd3f1640 1 -- 172.21.15.187:0/3905245991 wait complete. 2024-09-21T00:25:24.927 DEBUG:teuthology.orchestra.run.smithi187:> sudo ceph --cluster ceph osd new a983c227-a3ce-4bdd-82f1-438ba7e3ecd2 1 2024-09-21T00:25:25.041 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.039+0000 7fed339ad640 1 -- 172.21.15.187:0/3067823677 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fed2c0cd440 msgr2=0x7fed2c0d0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:25.041 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.039+0000 7fed339ad640 1 --2- 172.21.15.187:0/3067823677 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fed2c0cd440 0x7fed2c0d0670 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fed2c0d1940 tx=0x7fed2802a4a0 comp rx=0 tx=0).stop 2024-09-21T00:25:25.041 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.039+0000 7fed339ad640 1 -- 172.21.15.187:0/3067823677 shutdown_connections 2024-09-21T00:25:25.041 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.039+0000 7fed339ad640 1 --2- 172.21.15.187:0/3067823677 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fed2c0cd440 0x7fed2c0d0670 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:25.041 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.039+0000 7fed339ad640 1 --2- 172.21.15.187:0/3067823677 >> v2:172.21.15.112:3300/0 conn(0x7fed2c0ccc40 0x7fed2c15ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:25.041 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.039+0000 7fed339ad640 1 -- 172.21.15.187:0/3067823677 >> 172.21.15.187:0/3067823677 conn(0x7fed2c0b46e0 msgr2=0x7fed2c0d23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:25.041 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.039+0000 7fed339ad640 1 -- 172.21.15.187:0/3067823677 shutdown_connections 2024-09-21T00:25:25.041 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.039+0000 7fed339ad640 1 -- 172.21.15.187:0/3067823677 wait complete. 2024-09-21T00:25:25.042 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.041+0000 7fed339ad640 1 Processor -- start 2024-09-21T00:25:25.042 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.041+0000 7fed339ad640 1 -- start start 2024-09-21T00:25:25.045 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.042+0000 7fed339ad640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fed2c0ccc40 0x7fed2c1f7c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:25.045 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.042+0000 7fed339ad640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fed2c0cd440 0x7fed2c1f7270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:25.045 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.042+0000 7fed339ad640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fed2c0d0bb0 0x7fed2c0c5860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:25.045 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.042+0000 7fed339ad640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fed2c070810 con 0x7fed2c0cd440 2024-09-21T00:25:25.045 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.042+0000 7fed339ad640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fed2c0b6e00 con 0x7fed2c0d0bb0 2024-09-21T00:25:25.045 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.042+0000 7fed339ad640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fed2c070990 con 0x7fed2c0ccc40 2024-09-21T00:25:25.046 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.042+0000 7fed31749640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fed2c0ccc40 0x7fed2c1f7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:25.046 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.042+0000 7fed31749640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fed2c0ccc40 0x7fed2c1f7c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.187:33568/0 (socket says 172.21.15.187:33568) 2024-09-21T00:25:25.046 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.043+0000 7fed31749640 1 -- 172.21.15.187:0/2928103866 learned_addr learned my addr 172.21.15.187:0/2928103866 (peer_addr_for_me v2:172.21.15.187:0/0) 2024-09-21T00:25:25.046 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.043+0000 7fed31f4a640 1 --2- 172.21.15.187:0/2928103866 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fed2c0d0bb0 0x7fed2c0c5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:25.046 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.043+0000 7fed30f48640 1 --2- 172.21.15.187:0/2928103866 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fed2c0cd440 0x7fed2c1f7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:25.046 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.043+0000 7fed30f48640 1 -- 172.21.15.187:0/2928103866 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fed2c0ccc40 msgr2=0x7fed2c1f7c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:25.046 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.043+0000 7fed30f48640 1 --2- 172.21.15.187:0/2928103866 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fed2c0ccc40 0x7fed2c1f7c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:25.046 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.043+0000 7fed30f48640 1 -- 172.21.15.187:0/2928103866 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fed2c0d0bb0 msgr2=0x7fed2c0c5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:25.046 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.043+0000 7fed30f48640 1 --2- 172.21.15.187:0/2928103866 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fed2c0d0bb0 0x7fed2c0c5860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:25.046 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.043+0000 7fed30f48640 1 -- 172.21.15.187:0/2928103866 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed2c0aac40 con 0x7fed2c0cd440 2024-09-21T00:25:25.046 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.043+0000 7fed31f4a640 1 --2- 172.21.15.187:0/2928103866 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fed2c0d0bb0 0x7fed2c0c5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:25:25.046 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.043+0000 7fed31749640 1 --2- 172.21.15.187:0/2928103866 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fed2c0ccc40 0x7fed2c1f7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:25:25.047 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.043+0000 7fed30f48640 1 --2- 172.21.15.187:0/2928103866 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fed2c0cd440 0x7fed2c1f7270 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fed2c0d34e0 tx=0x7fed28033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:25:25.047 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.043+0000 7fed1dffb640 1 -- 172.21.15.187:0/2928103866 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed28031d60 con 0x7fed2c0cd440 2024-09-21T00:25:25.047 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.043+0000 7fed1dffb640 1 -- 172.21.15.187:0/2928103866 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fed28033e10 con 0x7fed2c0cd440 2024-09-21T00:25:25.047 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.043+0000 7fed339ad640 1 -- 172.21.15.187:0/2928103866 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed2c0ab8f0 con 0x7fed2c0cd440 2024-09-21T00:25:25.047 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.043+0000 7fed1dffb640 1 -- 172.21.15.187:0/2928103866 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed28005280 con 0x7fed2c0cd440 2024-09-21T00:25:25.047 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.044+0000 7fed339ad640 1 -- 172.21.15.187:0/2928103866 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed2c0ad450 con 0x7fed2c0cd440 2024-09-21T00:25:25.047 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.044+0000 7fed1dffb640 1 -- 172.21.15.187:0/2928103866 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fed280047a0 con 0x7fed2c0cd440 2024-09-21T00:25:25.047 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.044+0000 7fed1dffb640 1 -- 172.21.15.187:0/2928103866 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7fed280035f0 con 0x7fed2c0cd440 2024-09-21T00:25:25.047 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.044+0000 7fed339ad640 1 -- 172.21.15.187:0/2928103866 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed2c0aa9b0 con 0x7fed2c0cd440 2024-09-21T00:25:25.048 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.047+0000 7fed1dffb640 1 -- 172.21.15.187:0/2928103866 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fed2803f030 con 0x7fed2c0cd440 2024-09-21T00:25:25.118 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.117+0000 7fed339ad640 1 -- 172.21.15.187:0/2928103866 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd new", "uuid": "a983c227-a3ce-4bdd-82f1-438ba7e3ecd2", "id": 1} v 0) -- 0x7fed2c070f00 con 0x7fed2c0cd440 2024-09-21T00:25:25.119 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.118+0000 7fed1dffb640 1 -- 172.21.15.187:0/2928103866 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a983c227-a3ce-4bdd-82f1-438ba7e3ecd2", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7fed28031770 con 0x7fed2c0cd440 2024-09-21T00:25:25.120 INFO:teuthology.orchestra.run.smithi187.stdout:1 2024-09-21T00:25:25.121 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.119+0000 7fed339ad640 1 -- 172.21.15.187:0/2928103866 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fed2c0cd440 msgr2=0x7fed2c1f7270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:25.121 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.119+0000 7fed339ad640 1 --2- 172.21.15.187:0/2928103866 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fed2c0cd440 0x7fed2c1f7270 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fed2c0d34e0 tx=0x7fed28033c20 comp rx=0 tx=0).stop 2024-09-21T00:25:25.121 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.119+0000 7fed339ad640 1 -- 172.21.15.187:0/2928103866 shutdown_connections 2024-09-21T00:25:25.121 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.119+0000 7fed339ad640 1 --2- 172.21.15.187:0/2928103866 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fed2c0d0bb0 0x7fed2c0c5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:25.121 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.119+0000 7fed339ad640 1 --2- 172.21.15.187:0/2928103866 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fed2c0cd440 0x7fed2c1f7270 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:25.121 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.119+0000 7fed339ad640 1 --2- 172.21.15.187:0/2928103866 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fed2c0ccc40 0x7fed2c1f7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:25.121 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.119+0000 7fed339ad640 1 -- 172.21.15.187:0/2928103866 >> 172.21.15.187:0/2928103866 conn(0x7fed2c0b46e0 msgr2=0x7fed2c071110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:25.121 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.120+0000 7fed339ad640 1 -- 172.21.15.187:0/2928103866 shutdown_connections 2024-09-21T00:25:25.122 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.120+0000 7fed339ad640 1 -- 172.21.15.187:0/2928103866 wait complete. 2024-09-21T00:25:25.132 DEBUG:teuthology.orchestra.run.smithi187:> sudo ceph --cluster ceph osd new 1193c1d8-bfff-4a57-8200-5f69bf1ea3af 2 2024-09-21T00:25:25.246 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.244+0000 7f41fcb77640 1 -- 172.21.15.187:0/481766963 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f41f80aeeb0 msgr2=0x7f41f80b2ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:25.246 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.244+0000 7f41fcb77640 1 --2- 172.21.15.187:0/481766963 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f41f80aeeb0 0x7f41f80b2ca0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f41f8073950 tx=0x7f41ec02a4d0 comp rx=0 tx=0).stop 2024-09-21T00:25:25.246 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.245+0000 7f41fcb77640 1 -- 172.21.15.187:0/481766963 shutdown_connections 2024-09-21T00:25:25.246 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.245+0000 7f41fcb77640 1 --2- 172.21.15.187:0/481766963 >> v2:172.21.15.112:3300/0 conn(0x7f41f80b31e0 0x7f41f8073ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:25.246 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.245+0000 7f41fcb77640 1 --2- 172.21.15.187:0/481766963 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f41f80aeeb0 0x7f41f80b2ca0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:25.247 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.245+0000 7f41fcb77640 1 --2- 172.21.15.187:0/481766963 >> v2:172.21.15.187:3300/0 conn(0x7f41f815c750 0x7f41f80ae970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:25.247 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.245+0000 7f41fcb77640 1 -- 172.21.15.187:0/481766963 >> 172.21.15.187:0/481766963 conn(0x7f41f80cdca0 msgr2=0x7f41f80ce0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:25.247 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.245+0000 7f41fcb77640 1 -- 172.21.15.187:0/481766963 shutdown_connections 2024-09-21T00:25:25.247 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.245+0000 7f41fcb77640 1 -- 172.21.15.187:0/481766963 wait complete. 2024-09-21T00:25:25.248 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.247+0000 7f41fcb77640 1 Processor -- start 2024-09-21T00:25:25.248 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.247+0000 7f41fcb77640 1 -- start start 2024-09-21T00:25:25.251 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.248+0000 7f41fcb77640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f41f80aeeb0 0x7f41f809ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:25.251 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.248+0000 7f41fcb77640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41f80b31e0 0x7f41f81f3900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:25.251 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.248+0000 7f41fcb77640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f41f815c750 0x7f41f815b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:25.251 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.248+0000 7f41fcb77640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f41f80743e0 con 0x7f41f80b31e0 2024-09-21T00:25:25.251 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.248+0000 7f41fcb77640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f41f815ccc0 con 0x7f41f80aeeb0 2024-09-21T00:25:25.251 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.248+0000 7f41fcb77640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f41f8074560 con 0x7f41f815c750 2024-09-21T00:25:25.251 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.248+0000 7f41f6d9d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f41f815c750 0x7f41f815b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:25.251 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.248+0000 7f41f659c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f41f80aeeb0 0x7f41f809ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:25.251 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.248+0000 7f41f6d9d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f41f815c750 0x7f41f815b5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.187:33574/0 (socket says 172.21.15.187:33574) 2024-09-21T00:25:25.251 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.248+0000 7f41f6d9d640 1 -- 172.21.15.187:0/1646917503 learned_addr learned my addr 172.21.15.187:0/1646917503 (peer_addr_for_me v2:172.21.15.187:0/0) 2024-09-21T00:25:25.252 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.248+0000 7f41f659c640 1 --2- 172.21.15.187:0/1646917503 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f41f80aeeb0 0x7f41f809ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.187:36234/0 (socket says 172.21.15.187:36234) 2024-09-21T00:25:25.252 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.248+0000 7f41f5d9b640 1 --2- 172.21.15.187:0/1646917503 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41f80b31e0 0x7f41f81f3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:25.252 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.248+0000 7f41f659c640 1 -- 172.21.15.187:0/1646917503 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f41f815c750 msgr2=0x7f41f815b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:25.252 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.248+0000 7f41f659c640 1 --2- 172.21.15.187:0/1646917503 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f41f815c750 0x7f41f815b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:25.252 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.248+0000 7f41f659c640 1 -- 172.21.15.187:0/1646917503 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41f80b31e0 msgr2=0x7f41f81f3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:25.252 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.248+0000 7f41f659c640 1 --2- 172.21.15.187:0/1646917503 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41f80b31e0 0x7f41f81f3900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:25.252 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.248+0000 7f41f659c640 1 -- 172.21.15.187:0/1646917503 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41f81f3e40 con 0x7f41f80aeeb0 2024-09-21T00:25:25.252 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.248+0000 7f41f5d9b640 1 --2- 172.21.15.187:0/1646917503 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41f80b31e0 0x7f41f81f3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:25:25.252 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.248+0000 7f41f6d9d640 1 --2- 172.21.15.187:0/1646917503 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f41f815c750 0x7f41f815b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:25:25.252 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.249+0000 7f41f659c640 1 --2- 172.21.15.187:0/1646917503 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f41f80aeeb0 0x7f41f809ba30 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f41e800e9f0 tx=0x7f41e800edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:25:25.252 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.249+0000 7f41e6ffd640 1 -- 172.21.15.187:0/1646917503 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41e800ccc0 con 0x7f41f80aeeb0 2024-09-21T00:25:25.252 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.249+0000 7f41e6ffd640 1 -- 172.21.15.187:0/1646917503 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f41e8004020 con 0x7f41f80aeeb0 2024-09-21T00:25:25.252 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.249+0000 7f41fcb77640 1 -- 172.21.15.187:0/1646917503 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41f81f4330 con 0x7f41f80aeeb0 2024-09-21T00:25:25.252 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.249+0000 7f41e6ffd640 1 -- 172.21.15.187:0/1646917503 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41e800ce60 con 0x7f41f80aeeb0 2024-09-21T00:25:25.253 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.249+0000 7f41fcb77640 1 -- 172.21.15.187:0/1646917503 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41f81e4ee0 con 0x7f41f80aeeb0 2024-09-21T00:25:25.253 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.249+0000 7f41e6ffd640 1 -- 172.21.15.187:0/1646917503 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f41e8003980 con 0x7f41f80aeeb0 2024-09-21T00:25:25.253 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.249+0000 7f41e6ffd640 1 -- 172.21.15.187:0/1646917503 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7f41e8018850 con 0x7f41f80aeeb0 2024-09-21T00:25:25.253 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.250+0000 7f41fcb77640 1 -- 172.21.15.187:0/1646917503 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41f8074af0 con 0x7f41f80aeeb0 2024-09-21T00:25:25.253 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.252+0000 7f41e6ffd640 1 -- 172.21.15.187:0/1646917503 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f41e8038860 con 0x7f41f80aeeb0 2024-09-21T00:25:25.324 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.323+0000 7f41fcb77640 1 -- 172.21.15.187:0/1646917503 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd new", "uuid": "1193c1d8-bfff-4a57-8200-5f69bf1ea3af", "id": 2} v 0) -- 0x7f41f80b38d0 con 0x7f41f80aeeb0 2024-09-21T00:25:25.328 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.327+0000 7f41e6ffd640 1 -- 172.21.15.187:0/1646917503 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "1193c1d8-bfff-4a57-8200-5f69bf1ea3af", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7f41e8026030 con 0x7f41f80aeeb0 2024-09-21T00:25:25.328 INFO:teuthology.orchestra.run.smithi187.stdout:2 2024-09-21T00:25:25.329 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.328+0000 7f41fcb77640 1 -- 172.21.15.187:0/1646917503 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f41f80aeeb0 msgr2=0x7f41f809ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:25.329 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.328+0000 7f41fcb77640 1 --2- 172.21.15.187:0/1646917503 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f41f80aeeb0 0x7f41f809ba30 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f41e800e9f0 tx=0x7f41e800edf0 comp rx=0 tx=0).stop 2024-09-21T00:25:25.329 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.328+0000 7f41fcb77640 1 -- 172.21.15.187:0/1646917503 shutdown_connections 2024-09-21T00:25:25.329 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.328+0000 7f41fcb77640 1 --2- 172.21.15.187:0/1646917503 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f41f815c750 0x7f41f815b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:25.329 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.328+0000 7f41fcb77640 1 --2- 172.21.15.187:0/1646917503 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41f80b31e0 0x7f41f81f3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:25.329 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.328+0000 7f41fcb77640 1 --2- 172.21.15.187:0/1646917503 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f41f80aeeb0 0x7f41f809ba30 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:25.329 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.328+0000 7f41fcb77640 1 -- 172.21.15.187:0/1646917503 >> 172.21.15.187:0/1646917503 conn(0x7f41f80cdca0 msgr2=0x7f41f815ee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:25.330 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.328+0000 7f41fcb77640 1 -- 172.21.15.187:0/1646917503 shutdown_connections 2024-09-21T00:25:25.330 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.328+0000 7f41fcb77640 1 -- 172.21.15.187:0/1646917503 wait complete. 2024-09-21T00:25:25.339 DEBUG:teuthology.orchestra.run.smithi187:> sudo ceph --cluster ceph osd new af04330a-36ad-40cc-a8dd-fa259b221de5 3 2024-09-21T00:25:25.452 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.451+0000 7f46a0820640 1 -- 172.21.15.187:0/3575052207 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f469c0cd440 msgr2=0x7f469c0d0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:25.453 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.451+0000 7f46a0820640 1 --2- 172.21.15.187:0/3575052207 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f469c0cd440 0x7f469c0d0670 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f4690009830 tx=0x7f469002a480 comp rx=0 tx=0).stop 2024-09-21T00:25:25.453 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.451+0000 7f46a0820640 1 -- 172.21.15.187:0/3575052207 shutdown_connections 2024-09-21T00:25:25.453 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.451+0000 7f46a0820640 1 --2- 172.21.15.187:0/3575052207 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f469c0cd440 0x7f469c0d0670 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:25.453 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.451+0000 7f46a0820640 1 --2- 172.21.15.187:0/3575052207 >> v2:172.21.15.187:3300/0 conn(0x7f469c0ccc40 0x7f469c15ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:25.453 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.451+0000 7f46a0820640 1 -- 172.21.15.187:0/3575052207 >> 172.21.15.187:0/3575052207 conn(0x7f469c0b46e0 msgr2=0x7f469c0d23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:25.453 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.451+0000 7f46a0820640 1 -- 172.21.15.187:0/3575052207 shutdown_connections 2024-09-21T00:25:25.453 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.451+0000 7f46a0820640 1 -- 172.21.15.187:0/3575052207 wait complete. 2024-09-21T00:25:25.454 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.453+0000 7f46a0820640 1 Processor -- start 2024-09-21T00:25:25.454 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.453+0000 7f46a0820640 1 -- start start 2024-09-21T00:25:25.457 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.454+0000 7f46a0820640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f469c0ccc40 0x7f469c15ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:25.457 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.454+0000 7f46a0820640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f469c0cd440 0x7f469c1f3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:25.457 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.454+0000 7f46a0820640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f469c0d0bb0 0x7f469c1e5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:25.457 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.454+0000 7f46a0820640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f469c070810 con 0x7f469c0cd440 2024-09-21T00:25:25.457 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.454+0000 7f46a0820640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f469c0b6e00 con 0x7f469c0d0bb0 2024-09-21T00:25:25.457 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.454+0000 7f46a0820640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f469c070990 con 0x7f469c0ccc40 2024-09-21T00:25:25.457 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.454+0000 7f469a59c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f469c0ccc40 0x7f469c15ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:25.457 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.454+0000 7f469ad9d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f469c0d0bb0 0x7f469c1e5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:25.457 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.454+0000 7f469a59c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f469c0ccc40 0x7f469c15ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.187:33606/0 (socket says 172.21.15.187:33606) 2024-09-21T00:25:25.457 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.454+0000 7f469ad9d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f469c0d0bb0 0x7f469c1e5c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.187:36260/0 (socket says 172.21.15.187:36260) 2024-09-21T00:25:25.457 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.454+0000 7f469a59c640 1 -- 172.21.15.187:0/859465461 learned_addr learned my addr 172.21.15.187:0/859465461 (peer_addr_for_me v2:172.21.15.187:0/0) 2024-09-21T00:25:25.457 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.454+0000 7f4699d9b640 1 --2- 172.21.15.187:0/859465461 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f469c0cd440 0x7f469c1f3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:25.458 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.455+0000 7f469a59c640 1 -- 172.21.15.187:0/859465461 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f469c0d0bb0 msgr2=0x7f469c1e5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:25.458 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.455+0000 7f469a59c640 1 --2- 172.21.15.187:0/859465461 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f469c0d0bb0 0x7f469c1e5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:25.458 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.455+0000 7f469a59c640 1 -- 172.21.15.187:0/859465461 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f469c0cd440 msgr2=0x7f469c1f3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:25.458 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.455+0000 7f469a59c640 1 --2- 172.21.15.187:0/859465461 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f469c0cd440 0x7f469c1f3a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:25.458 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.455+0000 7f469a59c640 1 -- 172.21.15.187:0/859465461 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f469c1f4510 con 0x7f469c0ccc40 2024-09-21T00:25:25.458 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.455+0000 7f469ad9d640 1 --2- 172.21.15.187:0/859465461 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f469c0d0bb0 0x7f469c1e5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:25:25.458 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.455+0000 7f469a59c640 1 --2- 172.21.15.187:0/859465461 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f469c0ccc40 0x7f469c15ae30 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f468c00e9c0 tx=0x7f468c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:25:25.458 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.455+0000 7f468affd640 1 -- 172.21.15.187:0/859465461 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f468c00cd20 con 0x7f469c0ccc40 2024-09-21T00:25:25.458 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.455+0000 7f468affd640 1 -- 172.21.15.187:0/859465461 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f468c004020 con 0x7f469c0ccc40 2024-09-21T00:25:25.458 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.455+0000 7f468affd640 1 -- 172.21.15.187:0/859465461 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f468c004450 con 0x7f469c0ccc40 2024-09-21T00:25:25.458 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.455+0000 7f46a0820640 1 -- 172.21.15.187:0/859465461 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f469c1f47f0 con 0x7f469c0ccc40 2024-09-21T00:25:25.458 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.455+0000 7f46a0820640 1 -- 172.21.15.187:0/859465461 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f469c1e5470 con 0x7f469c0ccc40 2024-09-21T00:25:25.459 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.455+0000 7f468affd640 1 -- 172.21.15.187:0/859465461 <== mon.2 v2:172.21.15.187:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f468c003980 con 0x7f469c0ccc40 2024-09-21T00:25:25.459 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.455+0000 7f468affd640 1 -- 172.21.15.187:0/859465461 <== mon.2 v2:172.21.15.187:3301/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7f468c0188c0 con 0x7f469c0ccc40 2024-09-21T00:25:25.459 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.456+0000 7f46a0820640 1 -- 172.21.15.187:0/859465461 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f469c070ed0 con 0x7f469c0ccc40 2024-09-21T00:25:25.459 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.458+0000 7f468affd640 1 -- 172.21.15.187:0/859465461 <== mon.2 v2:172.21.15.187:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f468c038860 con 0x7f469c0ccc40 2024-09-21T00:25:25.530 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.528+0000 7f46a0820640 1 -- 172.21.15.187:0/859465461 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "osd new", "uuid": "af04330a-36ad-40cc-a8dd-fa259b221de5", "id": 3} v 0) -- 0x7f469c070430 con 0x7f469c0ccc40 2024-09-21T00:25:25.534 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.532+0000 7f468affd640 1 -- 172.21.15.187:0/859465461 <== mon.2 v2:172.21.15.187:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "af04330a-36ad-40cc-a8dd-fa259b221de5", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7f468c026030 con 0x7f469c0ccc40 2024-09-21T00:25:25.534 INFO:teuthology.orchestra.run.smithi187.stdout:3 2024-09-21T00:25:25.535 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.533+0000 7f46a0820640 1 -- 172.21.15.187:0/859465461 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f469c0ccc40 msgr2=0x7f469c15ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:25.535 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.533+0000 7f46a0820640 1 --2- 172.21.15.187:0/859465461 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f469c0ccc40 0x7f469c15ae30 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f468c00e9c0 tx=0x7f468c00ee80 comp rx=0 tx=0).stop 2024-09-21T00:25:25.535 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.533+0000 7f46a0820640 1 -- 172.21.15.187:0/859465461 shutdown_connections 2024-09-21T00:25:25.535 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.533+0000 7f46a0820640 1 --2- 172.21.15.187:0/859465461 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f469c0d0bb0 0x7f469c1e5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:25.535 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.533+0000 7f46a0820640 1 --2- 172.21.15.187:0/859465461 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f469c0cd440 0x7f469c1f3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:25.535 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.533+0000 7f46a0820640 1 --2- 172.21.15.187:0/859465461 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f469c0ccc40 0x7f469c15ae30 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:25.535 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.533+0000 7f46a0820640 1 -- 172.21.15.187:0/859465461 >> 172.21.15.187:0/859465461 conn(0x7f469c0b46e0 msgr2=0x7f469c0d23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:25.535 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.533+0000 7f46a0820640 1 -- 172.21.15.187:0/859465461 shutdown_connections 2024-09-21T00:25:25.535 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.534+0000 7f46a0820640 1 -- 172.21.15.187:0/859465461 wait complete. 2024-09-21T00:25:25.546 DEBUG:teuthology.orchestra.run.smithi187:> sudo ceph --cluster ceph osd new 5b893111-292c-4c5a-b184-53ad38c7199e 4 2024-09-21T00:25:25.659 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.657+0000 7f627029b640 1 -- 172.21.15.187:0/3863645128 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f62680b5670 msgr2=0x7f626815f630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:25.659 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.657+0000 7f627029b640 1 --2- 172.21.15.187:0/3863645128 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f62680b5670 0x7f626815f630 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6260005fd0 tx=0x7f626002a940 comp rx=0 tx=0).stop 2024-09-21T00:25:25.659 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.657+0000 7f627029b640 1 -- 172.21.15.187:0/3863645128 shutdown_connections 2024-09-21T00:25:25.659 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.657+0000 7f627029b640 1 --2- 172.21.15.187:0/3863645128 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f62680b5670 0x7f626815f630 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:25.659 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.657+0000 7f627029b640 1 -- 172.21.15.187:0/3863645128 >> 172.21.15.187:0/3863645128 conn(0x7f62680cdca0 msgr2=0x7f62680ce0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:25.659 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.657+0000 7f627029b640 1 -- 172.21.15.187:0/3863645128 shutdown_connections 2024-09-21T00:25:25.660 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.657+0000 7f627029b640 1 -- 172.21.15.187:0/3863645128 wait complete. 2024-09-21T00:25:25.660 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.659+0000 7f627029b640 1 Processor -- start 2024-09-21T00:25:25.660 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.659+0000 7f627029b640 1 -- start start 2024-09-21T00:25:25.663 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.660+0000 7f627029b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f626806def0 0x7f62680b1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:25.664 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.660+0000 7f627029b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f62680b5260 0x7f62680b1d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:25.664 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.660+0000 7f627029b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f62680b5670 0x7f62681f6150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:25.664 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.660+0000 7f627029b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6268071300 con 0x7f626806def0 2024-09-21T00:25:25.664 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.660+0000 7f627029b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f626815fff0 con 0x7f62680b5260 2024-09-21T00:25:25.664 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.660+0000 7f627029b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f6268071480 con 0x7f62680b5670 2024-09-21T00:25:25.664 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.660+0000 7f626d836640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f62680b5260 0x7f62680b1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:25.664 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.660+0000 7f626e838640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f62680b5670 0x7f62681f6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:25.664 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.660+0000 7f626d836640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f62680b5260 0x7f62680b1d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.187:36270/0 (socket says 172.21.15.187:36270) 2024-09-21T00:25:25.664 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.660+0000 7f626d836640 1 -- 172.21.15.187:0/102114264 learned_addr learned my addr 172.21.15.187:0/102114264 (peer_addr_for_me v2:172.21.15.187:0/0) 2024-09-21T00:25:25.664 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.661+0000 7f626e037640 1 --2- 172.21.15.187:0/102114264 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f626806def0 0x7f62680b1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:25.664 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.661+0000 7f626d836640 1 -- 172.21.15.187:0/102114264 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f62680b5670 msgr2=0x7f62681f6150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:25.664 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.661+0000 7f626d836640 1 --2- 172.21.15.187:0/102114264 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f62680b5670 0x7f62681f6150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:25.664 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.661+0000 7f626d836640 1 -- 172.21.15.187:0/102114264 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f626806def0 msgr2=0x7f62680b1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:25.664 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.661+0000 7f626d836640 1 --2- 172.21.15.187:0/102114264 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f626806def0 0x7f62680b1840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:25.664 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.661+0000 7f626d836640 1 -- 172.21.15.187:0/102114264 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6268156620 con 0x7f62680b5260 2024-09-21T00:25:25.665 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.661+0000 7f626e838640 1 --2- 172.21.15.187:0/102114264 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f62680b5670 0x7f62681f6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:25:25.665 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.661+0000 7f626e037640 1 --2- 172.21.15.187:0/102114264 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f626806def0 0x7f62680b1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:25:25.665 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.661+0000 7f626d836640 1 --2- 172.21.15.187:0/102114264 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f62680b5260 0x7f62680b1d80 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f625c00c400 tx=0x7f625c00c8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:25:25.665 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.661+0000 7f625affd640 1 -- 172.21.15.187:0/102114264 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f625c017070 con 0x7f62680b5260 2024-09-21T00:25:25.665 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.661+0000 7f625affd640 1 -- 172.21.15.187:0/102114264 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f625c0053c0 con 0x7f62680b5260 2024-09-21T00:25:25.665 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.661+0000 7f627029b640 1 -- 172.21.15.187:0/102114264 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6268156900 con 0x7f62680b5260 2024-09-21T00:25:25.665 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.661+0000 7f625affd640 1 -- 172.21.15.187:0/102114264 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f625c003850 con 0x7f62680b5260 2024-09-21T00:25:25.665 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.661+0000 7f627029b640 1 -- 172.21.15.187:0/102114264 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6268156ec0 con 0x7f62680b5260 2024-09-21T00:25:25.665 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.661+0000 7f625affd640 1 -- 172.21.15.187:0/102114264 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f625c01a070 con 0x7f62680b5260 2024-09-21T00:25:25.665 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.662+0000 7f625affd640 1 -- 172.21.15.187:0/102114264 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7f625c011bd0 con 0x7f62680b5260 2024-09-21T00:25:25.665 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.662+0000 7f627029b640 1 -- 172.21.15.187:0/102114264 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f626806acf0 con 0x7f62680b5260 2024-09-21T00:25:25.666 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.664+0000 7f625affd640 1 -- 172.21.15.187:0/102114264 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f626806acf0 con 0x7f62680b5260 2024-09-21T00:25:25.734 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.733+0000 7f627029b640 1 -- 172.21.15.187:0/102114264 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd new", "uuid": "5b893111-292c-4c5a-b184-53ad38c7199e", "id": 4} v 0) -- 0x7f626815f630 con 0x7f62680b5260 2024-09-21T00:25:25.737 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.736+0000 7f625affd640 1 -- 172.21.15.187:0/102114264 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "5b893111-292c-4c5a-b184-53ad38c7199e", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7f625c021070 con 0x7f62680b5260 2024-09-21T00:25:25.738 INFO:teuthology.orchestra.run.smithi187.stdout:4 2024-09-21T00:25:25.739 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.737+0000 7f627029b640 1 -- 172.21.15.187:0/102114264 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f62680b5260 msgr2=0x7f62680b1d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:25.739 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.737+0000 7f627029b640 1 --2- 172.21.15.187:0/102114264 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f62680b5260 0x7f62680b1d80 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f625c00c400 tx=0x7f625c00c8c0 comp rx=0 tx=0).stop 2024-09-21T00:25:25.739 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.737+0000 7f627029b640 1 -- 172.21.15.187:0/102114264 shutdown_connections 2024-09-21T00:25:25.739 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.737+0000 7f627029b640 1 --2- 172.21.15.187:0/102114264 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f62680b5670 0x7f62681f6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:25.739 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.737+0000 7f627029b640 1 --2- 172.21.15.187:0/102114264 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f62680b5260 0x7f62680b1d80 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:25.739 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.737+0000 7f627029b640 1 --2- 172.21.15.187:0/102114264 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f626806def0 0x7f62680b1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:25.739 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.737+0000 7f627029b640 1 -- 172.21.15.187:0/102114264 >> 172.21.15.187:0/102114264 conn(0x7f62680cdca0 msgr2=0x7f62680cab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:25.739 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.737+0000 7f627029b640 1 -- 172.21.15.187:0/102114264 shutdown_connections 2024-09-21T00:25:25.739 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.737+0000 7f627029b640 1 -- 172.21.15.187:0/102114264 wait complete. 2024-09-21T00:25:25.749 DEBUG:teuthology.orchestra.run.smithi187:> sudo ceph --cluster ceph osd new c0b8e011-45fe-4c5e-925e-6be2a7f1868a 5 2024-09-21T00:25:25.862 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.860+0000 7f5fb7e1d640 1 -- 172.21.15.187:0/2261779762 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5fb006def0 msgr2=0x7f5fb006f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:25.862 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.860+0000 7f5fb7e1d640 1 --2- 172.21.15.187:0/2261779762 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5fb006def0 0x7f5fb006f410 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5fac005fd0 tx=0x7f5fac02a5d0 comp rx=0 tx=0).stop 2024-09-21T00:25:25.862 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.860+0000 7f5fb7e1d640 1 -- 172.21.15.187:0/2261779762 shutdown_connections 2024-09-21T00:25:25.862 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.860+0000 7f5fb7e1d640 1 --2- 172.21.15.187:0/2261779762 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5fb00b5670 0x7f5fb015f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:25.862 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.860+0000 7f5fb7e1d640 1 --2- 172.21.15.187:0/2261779762 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5fb006def0 0x7f5fb006f410 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:25.862 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.860+0000 7f5fb7e1d640 1 -- 172.21.15.187:0/2261779762 >> 172.21.15.187:0/2261779762 conn(0x7f5fb00cdca0 msgr2=0x7f5fb00ce0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:25.862 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.860+0000 7f5fb7e1d640 1 -- 172.21.15.187:0/2261779762 shutdown_connections 2024-09-21T00:25:25.862 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.860+0000 7f5fb7e1d640 1 -- 172.21.15.187:0/2261779762 wait complete. 2024-09-21T00:25:25.863 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.862+0000 7f5fb7e1d640 1 Processor -- start 2024-09-21T00:25:25.863 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.862+0000 7f5fb7e1d640 1 -- start start 2024-09-21T00:25:25.866 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.863+0000 7f5fb7e1d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5fb006def0 0x7f5fb0154a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:25.866 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.863+0000 7f5fb7e1d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5fb00b5260 0x7f5fb015b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:25.866 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.863+0000 7f5fb7e1d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5fb00b5670 0x7f5fb0074120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:25.866 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.863+0000 7f5fb7e1d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb015ffb0 con 0x7f5fb00b5260 2024-09-21T00:25:25.866 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.863+0000 7f5fb7e1d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb015fe30 con 0x7f5fb006def0 2024-09-21T00:25:25.866 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.863+0000 7f5fb7e1d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f5fb0071300 con 0x7f5fb00b5670 2024-09-21T00:25:25.866 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.863+0000 7f5fb63ba640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5fb00b5670 0x7f5fb0074120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:25.866 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.863+0000 7f5fb5bb9640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5fb006def0 0x7f5fb0154a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:25.866 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.863+0000 7f5fb63ba640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5fb00b5670 0x7f5fb0074120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.187:33626/0 (socket says 172.21.15.187:33626) 2024-09-21T00:25:25.867 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.863+0000 7f5fb63ba640 1 -- 172.21.15.187:0/3244801238 learned_addr learned my addr 172.21.15.187:0/3244801238 (peer_addr_for_me v2:172.21.15.187:0/0) 2024-09-21T00:25:25.867 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.863+0000 7f5fb5bb9640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5fb006def0 0x7f5fb0154a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.187:36294/0 (socket says 172.21.15.187:36294) 2024-09-21T00:25:25.867 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.863+0000 7f5fb53b8640 1 --2- 172.21.15.187:0/3244801238 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5fb00b5260 0x7f5fb015b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:25.867 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.863+0000 7f5fb63ba640 1 -- 172.21.15.187:0/3244801238 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5fb006def0 msgr2=0x7f5fb0154a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:25.867 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.863+0000 7f5fb63ba640 1 --2- 172.21.15.187:0/3244801238 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5fb006def0 0x7f5fb0154a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:25.867 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.863+0000 7f5fb63ba640 1 -- 172.21.15.187:0/3244801238 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5fb00b5260 msgr2=0x7f5fb015b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:25.867 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.863+0000 7f5fb63ba640 1 --2- 172.21.15.187:0/3244801238 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5fb00b5260 0x7f5fb015b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:25.867 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.863+0000 7f5fb63ba640 1 -- 172.21.15.187:0/3244801238 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fb009ba10 con 0x7f5fb00b5670 2024-09-21T00:25:25.867 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.863+0000 7f5fb5bb9640 1 --2- 172.21.15.187:0/3244801238 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5fb006def0 0x7f5fb0154a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:25:25.867 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.864+0000 7f5fb63ba640 1 --2- 172.21.15.187:0/3244801238 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5fb00b5670 0x7f5fb0074120 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f5fa800ebf0 tx=0x7f5fa800c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:25:25.867 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.864+0000 7f5fa27fc640 1 -- 172.21.15.187:0/3244801238 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa8003b70 con 0x7f5fb00b5670 2024-09-21T00:25:25.867 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.864+0000 7f5fa27fc640 1 -- 172.21.15.187:0/3244801238 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5fa8003d10 con 0x7f5fb00b5670 2024-09-21T00:25:25.867 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.864+0000 7f5fb7e1d640 1 -- 172.21.15.187:0/3244801238 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fb009bcf0 con 0x7f5fb00b5670 2024-09-21T00:25:25.867 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.864+0000 7f5fa27fc640 1 -- 172.21.15.187:0/3244801238 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa8004280 con 0x7f5fb00b5670 2024-09-21T00:25:25.868 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.864+0000 7f5fb7e1d640 1 -- 172.21.15.187:0/3244801238 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fb009c2b0 con 0x7f5fb00b5670 2024-09-21T00:25:25.868 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.864+0000 7f5fa27fc640 1 -- 172.21.15.187:0/3244801238 <== mon.2 v2:172.21.15.187:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f5fa8021440 con 0x7f5fb00b5670 2024-09-21T00:25:25.868 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.864+0000 7f5fa27fc640 1 -- 172.21.15.187:0/3244801238 <== mon.2 v2:172.21.15.187:3301/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7f5fa80189b0 con 0x7f5fb00b5670 2024-09-21T00:25:25.868 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.865+0000 7f5fb7e1d640 1 -- 172.21.15.187:0/3244801238 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fb006f410 con 0x7f5fb00b5670 2024-09-21T00:25:25.868 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.867+0000 7f5fa27fc640 1 -- 172.21.15.187:0/3244801238 <== mon.2 v2:172.21.15.187:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f5fa8038860 con 0x7f5fb00b5670 2024-09-21T00:25:25.937 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.936+0000 7f5fb7e1d640 1 -- 172.21.15.187:0/3244801238 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "osd new", "uuid": "c0b8e011-45fe-4c5e-925e-6be2a7f1868a", "id": 5} v 0) -- 0x7f5fb00ae330 con 0x7f5fb00b5670 2024-09-21T00:25:25.941 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.940+0000 7f5fa27fc640 1 -- 172.21.15.187:0/3244801238 <== mon.2 v2:172.21.15.187:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c0b8e011-45fe-4c5e-925e-6be2a7f1868a", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7f5fa8026030 con 0x7f5fb00b5670 2024-09-21T00:25:25.941 INFO:teuthology.orchestra.run.smithi187.stdout:5 2024-09-21T00:25:25.942 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.941+0000 7f5fb7e1d640 1 -- 172.21.15.187:0/3244801238 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5fb00b5670 msgr2=0x7f5fb0074120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:25.943 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.941+0000 7f5fb7e1d640 1 --2- 172.21.15.187:0/3244801238 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5fb00b5670 0x7f5fb0074120 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f5fa800ebf0 tx=0x7f5fa800c6a0 comp rx=0 tx=0).stop 2024-09-21T00:25:25.943 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.941+0000 7f5fb7e1d640 1 -- 172.21.15.187:0/3244801238 shutdown_connections 2024-09-21T00:25:25.943 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.941+0000 7f5fb7e1d640 1 --2- 172.21.15.187:0/3244801238 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5fb00b5670 0x7f5fb0074120 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:25.943 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.941+0000 7f5fb7e1d640 1 --2- 172.21.15.187:0/3244801238 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5fb00b5260 0x7f5fb015b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:25.943 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.941+0000 7f5fb7e1d640 1 --2- 172.21.15.187:0/3244801238 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5fb006def0 0x7f5fb0154a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:25.943 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.941+0000 7f5fb7e1d640 1 -- 172.21.15.187:0/3244801238 >> 172.21.15.187:0/3244801238 conn(0x7f5fb00cdca0 msgr2=0x7f5fb00b30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:25.943 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.941+0000 7f5fb7e1d640 1 -- 172.21.15.187:0/3244801238 shutdown_connections 2024-09-21T00:25:25.943 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:25.941+0000 7f5fb7e1d640 1 -- 172.21.15.187:0/3244801238 wait complete. 2024-09-21T00:25:25.953 DEBUG:teuthology.orchestra.run.smithi187:> sudo ceph --cluster ceph osd new 87d2d650-7489-4593-98a3-a1f433ce8dd7 6 2024-09-21T00:25:26.064 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:26.062+0000 7f897bac5640 1 -- 172.21.15.187:0/655683194 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f89740b6290 msgr2=0x7f89740caf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:26.064 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:26.062+0000 7f897bac5640 1 --2- 172.21.15.187:0/655683194 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f89740b6290 0x7f89740caf40 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f8964009830 tx=0x7f896402a480 comp rx=0 tx=0).stop 2024-09-21T00:25:26.064 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:26.062+0000 7f897bac5640 1 -- 172.21.15.187:0/655683194 shutdown_connections 2024-09-21T00:25:26.065 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:26.062+0000 7f897bac5640 1 --2- 172.21.15.187:0/655683194 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f89740b6290 0x7f89740caf40 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:26.065 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:26.062+0000 7f897bac5640 1 --2- 172.21.15.187:0/655683194 >> v2:172.21.15.112:3300/0 conn(0x7f897415b730 0x7f89740b5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:26.065 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:26.062+0000 7f897bac5640 1 -- 172.21.15.187:0/655683194 >> 172.21.15.187:0/655683194 conn(0x7f89740735d0 msgr2=0x7f89740739e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:26.065 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:26.063+0000 7f897bac5640 1 -- 172.21.15.187:0/655683194 shutdown_connections 2024-09-21T00:25:26.065 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:26.063+0000 7f897bac5640 1 -- 172.21.15.187:0/655683194 wait complete. 2024-09-21T00:25:26.066 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:26.064+0000 7f897bac5640 1 Processor -- start 2024-09-21T00:25:26.066 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:26.064+0000 7f897bac5640 1 -- start start 2024-09-21T00:25:26.068 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:26.066+0000 7f897bac5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f897406ddf0 0x7f89741f62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:26.069 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:26.066+0000 7f897bac5640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f89740b6290 0x7f8974158520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:26.069 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:26.066+0000 7f897bac5640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f897415b730 0x7f89740cf650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:26.069 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:26.066+0000 7f897bac5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8974071820 con 0x7f897406ddf0 2024-09-21T00:25:26.069 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:26.066+0000 7f897bac5640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f89740716a0 con 0x7f897415b730 2024-09-21T00:25:26.069 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:26.066+0000 7f897bac5640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f89740719a0 con 0x7f89740b6290 2024-09-21T00:25:26.069 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:26.066+0000 7f8979060640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f89740b6290 0x7f8974158520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:26.069 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:26.066+0000 7f897a062640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f897415b730 0x7f89740cf650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:26.069 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:26.066+0000 7f8979060640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f89740b6290 0x7f8974158520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.187:38722/0 (socket says 172.21.15.187:38722) 2024-09-21T00:25:26.069 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:26.066+0000 7f8979060640 1 -- 172.21.15.187:0/1034580254 learned_addr learned my addr 172.21.15.187:0/1034580254 (peer_addr_for_me v2:172.21.15.187:0/0) 2024-09-21T00:25:26.069 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:26.066+0000 7f8979861640 1 --2- 172.21.15.187:0/1034580254 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f897406ddf0 0x7f89741f62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:26.069 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:26.066+0000 7f897a062640 1 -- 172.21.15.187:0/1034580254 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f89740b6290 msgr2=0x7f8974158520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:26.069 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:26.066+0000 7f897a062640 1 --2- 172.21.15.187:0/1034580254 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f89740b6290 0x7f8974158520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:26.069 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:26.066+0000 7f897a062640 1 -- 172.21.15.187:0/1034580254 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f897406ddf0 msgr2=0x7f89741f62e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:26.069 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:26.066+0000 7f897a062640 1 --2- 172.21.15.187:0/1034580254 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f897406ddf0 0x7f89741f62e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:26.070 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:26.066+0000 7f897a062640 1 -- 172.21.15.187:0/1034580254 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89740c88d0 con 0x7f897415b730 2024-09-21T00:25:26.070 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:26.066+0000 7f8979060640 1 --2- 172.21.15.187:0/1034580254 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f89740b6290 0x7f8974158520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:25:26.070 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:26.066+0000 7f8979861640 1 --2- 172.21.15.187:0/1034580254 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f897406ddf0 0x7f89741f62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:25:26.070 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:26.066+0000 7f897a062640 1 --2- 172.21.15.187:0/1034580254 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f897415b730 0x7f89740cf650 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f896c00cad0 tx=0x7f896c00cf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:25:26.070 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:26.066+0000 7f896a7fc640 1 -- 172.21.15.187:0/1034580254 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f896c016020 con 0x7f897415b730 2024-09-21T00:25:26.070 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:26.066+0000 7f896a7fc640 1 -- 172.21.15.187:0/1034580254 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f896c00ace0 con 0x7f897415b730 2024-09-21T00:25:26.070 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:26.066+0000 7f897bac5640 1 -- 172.21.15.187:0/1034580254 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89740c8bb0 con 0x7f897415b730 2024-09-21T00:25:26.070 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:26.066+0000 7f896a7fc640 1 -- 172.21.15.187:0/1034580254 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f896c012680 con 0x7f897415b730 2024-09-21T00:25:26.070 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:26.067+0000 7f897bac5640 1 -- 172.21.15.187:0/1034580254 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89740c9170 con 0x7f897415b730 2024-09-21T00:25:26.070 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:26.067+0000 7f896a7fc640 1 -- 172.21.15.187:0/1034580254 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f896c019020 con 0x7f897415b730 2024-09-21T00:25:26.070 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:26.067+0000 7f896a7fc640 1 -- 172.21.15.187:0/1034580254 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7f896c01bc20 con 0x7f897415b730 2024-09-21T00:25:26.071 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:26.068+0000 7f897bac5640 1 -- 172.21.15.187:0/1034580254 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8974071e30 con 0x7f897415b730 2024-09-21T00:25:26.071 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:26.070+0000 7f896a7fc640 1 -- 172.21.15.187:0/1034580254 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f896c020020 con 0x7f897415b730 2024-09-21T00:25:26.140 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:26.139+0000 7f897bac5640 1 -- 172.21.15.187:0/1034580254 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd new", "uuid": "87d2d650-7489-4593-98a3-a1f433ce8dd7", "id": 6} v 0) -- 0x7f89740afc30 con 0x7f897415b730 2024-09-21T00:25:26.143 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:26.142+0000 7f896a7fc640 1 -- 172.21.15.187:0/1034580254 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "87d2d650-7489-4593-98a3-a1f433ce8dd7", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7f896c012de0 con 0x7f897415b730 2024-09-21T00:25:26.144 INFO:teuthology.orchestra.run.smithi187.stdout:6 2024-09-21T00:25:26.145 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:26.143+0000 7f897bac5640 1 -- 172.21.15.187:0/1034580254 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f897415b730 msgr2=0x7f89740cf650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:26.145 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:26.143+0000 7f897bac5640 1 --2- 172.21.15.187:0/1034580254 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f897415b730 0x7f89740cf650 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f896c00cad0 tx=0x7f896c00cf90 comp rx=0 tx=0).stop 2024-09-21T00:25:26.145 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:26.143+0000 7f897bac5640 1 -- 172.21.15.187:0/1034580254 shutdown_connections 2024-09-21T00:25:26.145 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:26.143+0000 7f897bac5640 1 --2- 172.21.15.187:0/1034580254 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f897415b730 0x7f89740cf650 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:26.145 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:26.143+0000 7f897bac5640 1 --2- 172.21.15.187:0/1034580254 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f89740b6290 0x7f8974158520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:26.145 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:26.143+0000 7f897bac5640 1 --2- 172.21.15.187:0/1034580254 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f897406ddf0 0x7f89741f62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:26.145 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:26.143+0000 7f897bac5640 1 -- 172.21.15.187:0/1034580254 >> 172.21.15.187:0/1034580254 conn(0x7f89740735d0 msgr2=0x7f89740b5930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:26.145 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:26.143+0000 7f897bac5640 1 -- 172.21.15.187:0/1034580254 shutdown_connections 2024-09-21T00:25:26.146 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:26.143+0000 7f897bac5640 1 -- 172.21.15.187:0/1034580254 wait complete. 2024-09-21T00:25:26.155 DEBUG:teuthology.orchestra.run.smithi187:> sudo ceph --cluster ceph osd new c6b4f8e2-a1ee-4438-aa37-67ff4b6ef1f1 7 2024-09-21T00:25:26.268 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:26.266+0000 7fd639ed3640 1 -- 172.21.15.187:0/563977504 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd6340b5260 msgr2=0x7fd63415ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:26.268 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:26.266+0000 7fd639ed3640 1 --2- 172.21.15.187:0/563977504 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd6340b5260 0x7fd63415ba80 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fd628005fd0 tx=0x7fd62802a5d0 comp rx=0 tx=0).stop 2024-09-21T00:25:26.268 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:26.266+0000 7fd639ed3640 1 -- 172.21.15.187:0/563977504 shutdown_connections 2024-09-21T00:25:26.268 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:26.266+0000 7fd639ed3640 1 --2- 172.21.15.187:0/563977504 >> v2:172.21.15.112:3300/0 conn(0x7fd63415bfc0 0x7fd63415f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:26.268 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:26.266+0000 7fd639ed3640 1 --2- 172.21.15.187:0/563977504 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd6340b5260 0x7fd63415ba80 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:26.268 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:26.266+0000 7fd639ed3640 1 -- 172.21.15.187:0/563977504 >> 172.21.15.187:0/563977504 conn(0x7fd6340cdca0 msgr2=0x7fd6340ce0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:26.268 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:26.266+0000 7fd639ed3640 1 -- 172.21.15.187:0/563977504 shutdown_connections 2024-09-21T00:25:26.269 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:26.266+0000 7fd639ed3640 1 -- 172.21.15.187:0/563977504 wait complete. 2024-09-21T00:25:26.269 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:26.268+0000 7fd639ed3640 1 Processor -- start 2024-09-21T00:25:26.269 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:26.268+0000 7fd639ed3640 1 -- start start 2024-09-21T00:25:26.272 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:26.270+0000 7fd639ed3640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd63406def0 0x7fd63409bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:26.272 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:26.270+0000 7fd639ed3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6340b5260 0x7fd6341553d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:26.272 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:26.270+0000 7fd639ed3640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd63415bfc0 0x7fd63415a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:26.272 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:26.270+0000 7fd639ed3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd63415ff40 con 0x7fd6340b5260 2024-09-21T00:25:26.272 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:26.270+0000 7fd639ed3640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd63415fdc0 con 0x7fd63406def0 2024-09-21T00:25:26.272 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:26.270+0000 7fd639ed3640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd634071300 con 0x7fd63415bfc0 2024-09-21T00:25:26.273 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:26.270+0000 7fd633fff640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd63415bfc0 0x7fd63415a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:26.273 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:26.270+0000 7fd6337fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd63406def0 0x7fd63409bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:26.273 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:26.270+0000 7fd633fff640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd63415bfc0 0x7fd63415a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.187:38748/0 (socket says 172.21.15.187:38748) 2024-09-21T00:25:26.273 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:26.270+0000 7fd6337fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd63406def0 0x7fd63409bef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.187:60386/0 (socket says 172.21.15.187:60386) 2024-09-21T00:25:26.273 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:26.270+0000 7fd633fff640 1 -- 172.21.15.187:0/1857713640 learned_addr learned my addr 172.21.15.187:0/1857713640 (peer_addr_for_me v2:172.21.15.187:0/0) 2024-09-21T00:25:26.273 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:26.270+0000 7fd632ffd640 1 --2- 172.21.15.187:0/1857713640 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6340b5260 0x7fd6341553d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:26.273 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:26.270+0000 7fd6337fe640 1 -- 172.21.15.187:0/1857713640 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd63415bfc0 msgr2=0x7fd63415a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:26.273 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:26.270+0000 7fd6337fe640 1 --2- 172.21.15.187:0/1857713640 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd63415bfc0 0x7fd63415a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:26.273 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:26.270+0000 7fd6337fe640 1 -- 172.21.15.187:0/1857713640 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6340b5260 msgr2=0x7fd6341553d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:26.273 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:26.270+0000 7fd6337fe640 1 --2- 172.21.15.187:0/1857713640 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6340b5260 0x7fd6341553d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:26.273 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:26.270+0000 7fd6337fe640 1 -- 172.21.15.187:0/1857713640 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6341f8e20 con 0x7fd63406def0 2024-09-21T00:25:26.273 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:26.270+0000 7fd632ffd640 1 --2- 172.21.15.187:0/1857713640 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6340b5260 0x7fd6341553d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:25:26.274 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:26.270+0000 7fd633fff640 1 --2- 172.21.15.187:0/1857713640 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd63415bfc0 0x7fd63415a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:25:26.274 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:26.270+0000 7fd6337fe640 1 --2- 172.21.15.187:0/1857713640 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd63406def0 0x7fd63409bef0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd63415c430 tx=0x7fd62400c820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:25:26.274 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:26.270+0000 7fd61bfff640 1 -- 172.21.15.187:0/1857713640 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6240052d0 con 0x7fd63406def0 2024-09-21T00:25:26.274 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:26.270+0000 7fd61bfff640 1 -- 172.21.15.187:0/1857713640 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd624007020 con 0x7fd63406def0 2024-09-21T00:25:26.274 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:26.270+0000 7fd61bfff640 1 -- 172.21.15.187:0/1857713640 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd624007470 con 0x7fd63406def0 2024-09-21T00:25:26.274 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:26.270+0000 7fd639ed3640 1 -- 172.21.15.187:0/1857713640 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6341f9100 con 0x7fd63406def0 2024-09-21T00:25:26.274 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:26.271+0000 7fd639ed3640 1 -- 172.21.15.187:0/1857713640 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6341f96c0 con 0x7fd63406def0 2024-09-21T00:25:26.274 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:26.271+0000 7fd61bfff640 1 -- 172.21.15.187:0/1857713640 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd6240067d0 con 0x7fd63406def0 2024-09-21T00:25:26.274 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:26.271+0000 7fd61bfff640 1 -- 172.21.15.187:0/1857713640 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7fd624011bf0 con 0x7fd63406def0 2024-09-21T00:25:26.275 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:26.271+0000 7fd639ed3640 1 -- 172.21.15.187:0/1857713640 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd63415ba80 con 0x7fd63406def0 2024-09-21T00:25:26.275 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:26.274+0000 7fd61bfff640 1 -- 172.21.15.187:0/1857713640 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fd624011e90 con 0x7fd63406def0 2024-09-21T00:25:26.345 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:26.344+0000 7fd639ed3640 1 -- 172.21.15.187:0/1857713640 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd new", "uuid": "c6b4f8e2-a1ee-4438-aa37-67ff4b6ef1f1", "id": 7} v 0) -- 0x7fd63415f690 con 0x7fd63406def0 2024-09-21T00:25:26.349 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:26.348+0000 7fd61bfff640 1 -- 172.21.15.187:0/1857713640 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c6b4f8e2-a1ee-4438-aa37-67ff4b6ef1f1", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7fd624007610 con 0x7fd63406def0 2024-09-21T00:25:26.349 INFO:teuthology.orchestra.run.smithi187.stdout:7 2024-09-21T00:25:26.350 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:26.349+0000 7fd639ed3640 1 -- 172.21.15.187:0/1857713640 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd63406def0 msgr2=0x7fd63409bef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:26.350 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:26.349+0000 7fd639ed3640 1 --2- 172.21.15.187:0/1857713640 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd63406def0 0x7fd63409bef0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd63415c430 tx=0x7fd62400c820 comp rx=0 tx=0).stop 2024-09-21T00:25:26.350 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:26.349+0000 7fd639ed3640 1 -- 172.21.15.187:0/1857713640 shutdown_connections 2024-09-21T00:25:26.350 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:26.349+0000 7fd639ed3640 1 --2- 172.21.15.187:0/1857713640 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd63415bfc0 0x7fd63415a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:26.351 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:26.349+0000 7fd639ed3640 1 --2- 172.21.15.187:0/1857713640 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6340b5260 0x7fd6341553d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:26.351 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:26.349+0000 7fd639ed3640 1 --2- 172.21.15.187:0/1857713640 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd63406def0 0x7fd63409bef0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:26.351 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:26.349+0000 7fd639ed3640 1 -- 172.21.15.187:0/1857713640 >> 172.21.15.187:0/1857713640 conn(0x7fd6340cdca0 msgr2=0x7fd6340ae370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:26.351 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:26.349+0000 7fd639ed3640 1 -- 172.21.15.187:0/1857713640 shutdown_connections 2024-09-21T00:25:26.351 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:25:26.349+0000 7fd639ed3640 1 -- 172.21.15.187:0/1857713640 wait complete. 2024-09-21T00:25:26.361 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-21T00:25:26.361 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-21T00:25:26.363 INFO:tasks.ceph.osd.0:Started 2024-09-21T00:25:26.363 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-21T00:25:26.364 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-21T00:25:26.366 INFO:tasks.ceph.osd.1:Started 2024-09-21T00:25:26.366 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-21T00:25:26.366 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-21T00:25:26.368 INFO:tasks.ceph.osd.2:Started 2024-09-21T00:25:26.368 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-21T00:25:26.368 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-21T00:25:26.370 INFO:tasks.ceph.osd.3:Started 2024-09-21T00:25:26.370 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-21T00:25:26.370 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-21T00:25:26.372 INFO:tasks.ceph.osd.4:Started 2024-09-21T00:25:26.373 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-21T00:25:26.373 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-21T00:25:26.375 INFO:tasks.ceph.osd.5:Started 2024-09-21T00:25:26.375 INFO:tasks.ceph.osd.6:Restarting daemon 2024-09-21T00:25:26.375 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-09-21T00:25:26.377 INFO:tasks.ceph.osd.6:Started 2024-09-21T00:25:26.377 INFO:tasks.ceph.osd.7:Restarting daemon 2024-09-21T00:25:26.377 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-09-21T00:25:26.379 INFO:tasks.ceph.osd.7:Started 2024-09-21T00:25:26.380 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T00:25:26.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:26.531+0000 7f19e85c3640 1 -- 172.21.15.112:0/1466200355 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f19e0074210 msgr2=0x7f19e0158f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:26.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:26.531+0000 7f19e85c3640 1 --2- 172.21.15.112:0/1466200355 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f19e0074210 0x7f19e0158f30 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f19dc005fd0 tx=0x7f19dc02a8b0 comp rx=0 tx=0).stop 2024-09-21T00:25:26.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:26.531+0000 7f19e85c3640 1 -- 172.21.15.112:0/1466200355 shutdown_connections 2024-09-21T00:25:26.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:26.531+0000 7f19e85c3640 1 --2- 172.21.15.112:0/1466200355 >> v2:172.21.15.187:3300/0 conn(0x7f19e015bfa0 0x7f19e015c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:26.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:26.531+0000 7f19e85c3640 1 --2- 172.21.15.112:0/1466200355 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f19e0074210 0x7f19e0158f30 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:26.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:26.531+0000 7f19e85c3640 1 -- 172.21.15.112:0/1466200355 >> 172.21.15.112:0/1466200355 conn(0x7f19e0073560 msgr2=0x7f19e0073970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:26.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:26.531+0000 7f19e85c3640 1 -- 172.21.15.112:0/1466200355 shutdown_connections 2024-09-21T00:25:26.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:26.531+0000 7f19e85c3640 1 -- 172.21.15.112:0/1466200355 wait complete. 2024-09-21T00:25:26.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:26.533+0000 7f19e85c3640 1 Processor -- start 2024-09-21T00:25:26.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:26.533+0000 7f19e85c3640 1 -- start start 2024-09-21T00:25:26.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:26.534+0000 7f19e85c3640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f19e0074210 0x7f19e00c7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:26.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:26.534+0000 7f19e85c3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19e015bfa0 0x7f19e00b5f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:26.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:26.534+0000 7f19e85c3640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f19e015f970 0x7f19e0070330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:26.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:26.534+0000 7f19e85c3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f19e006df70 con 0x7f19e015bfa0 2024-09-21T00:25:26.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:26.534+0000 7f19e85c3640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f19e006ddf0 con 0x7f19e015f970 2024-09-21T00:25:26.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:26.534+0000 7f19e85c3640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f19e006e0f0 con 0x7f19e0074210 2024-09-21T00:25:26.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:26.534+0000 7f19e5b5e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19e015bfa0 0x7f19e00b5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:26.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:26.534+0000 7f19e635f640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f19e0074210 0x7f19e00c7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:26.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:26.534+0000 7f19e5b5e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19e015bfa0 0x7f19e00b5f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42292/0 (socket says 172.21.15.112:42292) 2024-09-21T00:25:26.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:26.534+0000 7f19e5b5e640 1 -- 172.21.15.112:0/1784959766 learned_addr learned my addr 172.21.15.112:0/1784959766 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:25:26.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:26.534+0000 7f19e6b60640 1 --2- 172.21.15.112:0/1784959766 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f19e015f970 0x7f19e0070330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:26.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:26.534+0000 7f19e5b5e640 1 -- 172.21.15.112:0/1784959766 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f19e0074210 msgr2=0x7f19e00c7910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:26.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:26.534+0000 7f19e5b5e640 1 --2- 172.21.15.112:0/1784959766 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f19e0074210 0x7f19e00c7910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:26.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:26.534+0000 7f19e5b5e640 1 -- 172.21.15.112:0/1784959766 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f19e015f970 msgr2=0x7f19e0070330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:26.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:26.534+0000 7f19e5b5e640 1 --2- 172.21.15.112:0/1784959766 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f19e015f970 0x7f19e0070330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:26.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:26.534+0000 7f19e5b5e640 1 -- 172.21.15.112:0/1784959766 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19e00ca6c0 con 0x7f19e015bfa0 2024-09-21T00:25:26.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:26.534+0000 7f19e635f640 1 --2- 172.21.15.112:0/1784959766 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f19e0074210 0x7f19e00c7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:25:26.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:26.535+0000 7f19e5b5e640 1 --2- 172.21.15.112:0/1784959766 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19e015bfa0 0x7f19e00b5f00 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f19d000c3c0 tx=0x7f19d000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:25:26.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:26.535+0000 7f19d6ffd640 1 -- 172.21.15.112:0/1784959766 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d0017070 con 0x7f19e015bfa0 2024-09-21T00:25:26.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:26.535+0000 7f19d6ffd640 1 -- 172.21.15.112:0/1784959766 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f19d00035f0 con 0x7f19e015bfa0 2024-09-21T00:25:26.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:26.535+0000 7f19e85c3640 1 -- 172.21.15.112:0/1784959766 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19e01f6c70 con 0x7f19e015bfa0 2024-09-21T00:25:26.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:26.535+0000 7f19d6ffd640 1 -- 172.21.15.112:0/1784959766 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d0003a80 con 0x7f19e015bfa0 2024-09-21T00:25:26.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:26.535+0000 7f19e85c3640 1 -- 172.21.15.112:0/1784959766 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19e00b56a0 con 0x7f19e015bfa0 2024-09-21T00:25:26.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:26.535+0000 7f19d6ffd640 1 -- 172.21.15.112:0/1784959766 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f19d001a070 con 0x7f19e015bfa0 2024-09-21T00:25:26.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:26.535+0000 7f19d6ffd640 1 -- 172.21.15.112:0/1784959766 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f19d0012ad0 con 0x7f19e015bfa0 2024-09-21T00:25:26.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:26.536+0000 7f19e85c3640 1 -- 172.21.15.112:0/1784959766 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19e00ca430 con 0x7f19e015bfa0 2024-09-21T00:25:26.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:26.538+0000 7f19d6ffd640 1 -- 172.21.15.112:0/1784959766 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f19d0021020 con 0x7f19e015bfa0 2024-09-21T00:25:26.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:26.598+0000 7f19e85c3640 1 -- 172.21.15.112:0/1784959766 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f19e006e680 con 0x7f19e015bfa0 2024-09-21T00:25:26.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:26.598+0000 7f19d6ffd640 1 -- 172.21.15.112:0/1784959766 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f19d0003cd0 con 0x7f19e015bfa0 2024-09-21T00:25:26.599 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:25:26.599 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":10,"fsid":"8b34692b-dc65-49b2-a08f-26f707b96ac5","created":"2024-09-21T00:25:22.788490+0000","modified":"2024-09-21T00:25:26.346500+0000","last_up_change":"0.000000","last_in_change":"2024-09-21T00:25:26.346500+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"9c575086-67d2-4de2-b01b-d44dcc21d604","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"a983c227-a3ce-4bdd-82f1-438ba7e3ecd2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"1193c1d8-bfff-4a57-8200-5f69bf1ea3af","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"af04330a-36ad-40cc-a8dd-fa259b221de5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"5b893111-292c-4c5a-b184-53ad38c7199e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"c0b8e011-45fe-4c5e-925e-6be2a7f1868a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"87d2d650-7489-4593-98a3-a1f433ce8dd7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"c6b4f8e2-a1ee-4438-aa37-67ff4b6ef1f1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T00:25:26.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:26.599+0000 7f19e85c3640 1 -- 172.21.15.112:0/1784959766 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19e015bfa0 msgr2=0x7f19e00b5f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:26.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:26.599+0000 7f19e85c3640 1 --2- 172.21.15.112:0/1784959766 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19e015bfa0 0x7f19e00b5f00 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f19d000c3c0 tx=0x7f19d000c880 comp rx=0 tx=0).stop 2024-09-21T00:25:26.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:26.599+0000 7f19e85c3640 1 -- 172.21.15.112:0/1784959766 shutdown_connections 2024-09-21T00:25:26.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:26.599+0000 7f19e85c3640 1 --2- 172.21.15.112:0/1784959766 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f19e015f970 0x7f19e0070330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:26.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:26.599+0000 7f19e85c3640 1 --2- 172.21.15.112:0/1784959766 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19e015bfa0 0x7f19e00b5f00 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:26.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:26.599+0000 7f19e85c3640 1 --2- 172.21.15.112:0/1784959766 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f19e0074210 0x7f19e00c7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:26.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:26.599+0000 7f19e85c3640 1 -- 172.21.15.112:0/1784959766 >> 172.21.15.112:0/1784959766 conn(0x7f19e0073560 msgr2=0x7f19e00c6f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:26.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:26.599+0000 7f19e85c3640 1 -- 172.21.15.112:0/1784959766 shutdown_connections 2024-09-21T00:25:26.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:26.599+0000 7f19e85c3640 1 -- 172.21.15.112:0/1784959766 wait complete. 2024-09-21T00:25:26.610 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-21T00:25:26.610 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-21T00:25:27.416 DEBUG:teuthology.orchestra.run.smithi112:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-21T00:25:27.522 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:27.521+0000 7f40d2006640 1 -- 172.21.15.112:0/3735997939 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40cc0b2500 msgr2=0x7f40cc0ccac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:27.522 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:27.521+0000 7f40d2006640 1 --2- 172.21.15.112:0/3735997939 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40cc0b2500 0x7f40cc0ccac0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f40b4005fd0 tx=0x7f40b402a5d0 comp rx=0 tx=0).stop 2024-09-21T00:25:27.522 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:27.521+0000 7f40d2006640 1 -- 172.21.15.112:0/3735997939 shutdown_connections 2024-09-21T00:25:27.522 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:27.521+0000 7f40d2006640 1 --2- 172.21.15.112:0/3735997939 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40cc0b2500 0x7f40cc0ccac0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:27.523 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:27.521+0000 7f40d2006640 1 --2- 172.21.15.112:0/3735997939 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f40cc0d19d0 0x7f40cc15c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:27.523 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:27.521+0000 7f40d2006640 1 -- 172.21.15.112:0/3735997939 >> 172.21.15.112:0/3735997939 conn(0x7f40cc0b41d0 msgr2=0x7f40cc0b45e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:27.523 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:27.521+0000 7f40d2006640 1 -- 172.21.15.112:0/3735997939 shutdown_connections 2024-09-21T00:25:27.523 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:27.521+0000 7f40d2006640 1 -- 172.21.15.112:0/3735997939 wait complete. 2024-09-21T00:25:27.523 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:27.523+0000 7f40d2006640 1 Processor -- start 2024-09-21T00:25:27.524 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:27.523+0000 7f40d2006640 1 -- start start 2024-09-21T00:25:27.526 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:27.524+0000 7f40d2006640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40cc0b2500 0x7f40cc1548b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:27.526 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:27.524+0000 7f40d2006640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f40cc0cd000 0x7f40cc06def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:27.526 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:27.524+0000 7f40d2006640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f40cc0d19d0 0x7f40cc1f9320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:27.526 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:27.524+0000 7f40d2006640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f40cc0d08e0 con 0x7f40cc0b2500 2024-09-21T00:25:27.527 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:27.524+0000 7f40d2006640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f40cc0b67b0 con 0x7f40cc0d19d0 2024-09-21T00:25:27.527 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:27.524+0000 7f40d2006640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f40cc0d0a60 con 0x7f40cc0cd000 2024-09-21T00:25:27.527 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:27.524+0000 7f40cb7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40cc0b2500 0x7f40cc1548b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:27.527 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:27.524+0000 7f40cb7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40cc0b2500 0x7f40cc1548b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42300/0 (socket says 172.21.15.112:42300) 2024-09-21T00:25:27.527 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:27.524+0000 7f40cb7fe640 1 -- 172.21.15.112:0/2424160811 learned_addr learned my addr 172.21.15.112:0/2424160811 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:25:27.527 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:27.525+0000 7f40cbfff640 1 --2- 172.21.15.112:0/2424160811 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f40cc0d19d0 0x7f40cc1f9320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:27.527 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:27.525+0000 7f40c3fff640 1 --2- 172.21.15.112:0/2424160811 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f40cc0cd000 0x7f40cc06def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:27.527 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:27.525+0000 7f40cb7fe640 1 -- 172.21.15.112:0/2424160811 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f40cc0cd000 msgr2=0x7f40cc06def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:27.527 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:27.525+0000 7f40cb7fe640 1 --2- 172.21.15.112:0/2424160811 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f40cc0cd000 0x7f40cc06def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:27.527 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:27.525+0000 7f40cb7fe640 1 -- 172.21.15.112:0/2424160811 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f40cc0d19d0 msgr2=0x7f40cc1f9320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:27.527 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:27.525+0000 7f40cb7fe640 1 --2- 172.21.15.112:0/2424160811 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f40cc0d19d0 0x7f40cc1f9320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:27.527 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:27.525+0000 7f40cb7fe640 1 -- 172.21.15.112:0/2424160811 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40cc06e430 con 0x7f40cc0b2500 2024-09-21T00:25:27.527 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:27.525+0000 7f40cb7fe640 1 --2- 172.21.15.112:0/2424160811 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40cc0b2500 0x7f40cc1548b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f40bc00e7c0 tx=0x7f40bc00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:25:27.527 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:27.525+0000 7f40c8ff9640 1 -- 172.21.15.112:0/2424160811 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40bc00ccc0 con 0x7f40cc0b2500 2024-09-21T00:25:27.528 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:27.525+0000 7f40c8ff9640 1 -- 172.21.15.112:0/2424160811 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f40bc010430 con 0x7f40cc0b2500 2024-09-21T00:25:27.528 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:27.525+0000 7f40d2006640 1 -- 172.21.15.112:0/2424160811 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40cc0af2f0 con 0x7f40cc0b2500 2024-09-21T00:25:27.528 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:27.525+0000 7f40c8ff9640 1 -- 172.21.15.112:0/2424160811 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40bc010860 con 0x7f40cc0b2500 2024-09-21T00:25:27.528 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:27.525+0000 7f40d2006640 1 -- 172.21.15.112:0/2424160811 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40cc0af8b0 con 0x7f40cc0b2500 2024-09-21T00:25:27.528 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:27.526+0000 7f40d2006640 1 -- 172.21.15.112:0/2424160811 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40cc071f40 con 0x7f40cc0b2500 2024-09-21T00:25:27.528 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:27.526+0000 7f40c8ff9640 1 -- 172.21.15.112:0/2424160811 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 2) ==== 50742+0+0 (secure 0 0 0) 0x7f40bc010a00 con 0x7f40cc0b2500 2024-09-21T00:25:27.528 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:27.527+0000 7f40c8ff9640 1 -- 172.21.15.112:0/2424160811 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f40bc052d00 con 0x7f40cc0b2500 2024-09-21T00:25:27.529 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:27.529+0000 7f40c8ff9640 1 -- 172.21.15.112:0/2424160811 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f40cc071f40 con 0x7f40cc0b2500 2024-09-21T00:25:27.590 INFO:tasks.ceph.osd.1.smithi112.stderr:2024-09-21T00:25:27.590+0000 7f69352ef640 -1 Falling back to public interface 2024-09-21T00:25:27.634 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:27.633+0000 7f40d2006640 1 -- 172.21.15.112:0/2424160811 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f40cc0d0f10 con 0x7f40cc0b2500 2024-09-21T00:25:27.634 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:27.634+0000 7f40c8ff9640 1 -- 172.21.15.112:0/2424160811 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f40bc016da0 con 0x7f40cc0b2500 2024-09-21T00:25:27.634 INFO:teuthology.misc.health.smithi112.stdout: 2024-09-21T00:25:27.634 INFO:teuthology.misc.health.smithi112.stdout:{"epoch":10,"fsid":"8b34692b-dc65-49b2-a08f-26f707b96ac5","created":"2024-09-21T00:25:22.788490+0000","modified":"2024-09-21T00:25:26.346500+0000","last_up_change":"0.000000","last_in_change":"2024-09-21T00:25:26.346500+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"9c575086-67d2-4de2-b01b-d44dcc21d604","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"a983c227-a3ce-4bdd-82f1-438ba7e3ecd2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"1193c1d8-bfff-4a57-8200-5f69bf1ea3af","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"af04330a-36ad-40cc-a8dd-fa259b221de5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"5b893111-292c-4c5a-b184-53ad38c7199e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"c0b8e011-45fe-4c5e-925e-6be2a7f1868a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"87d2d650-7489-4593-98a3-a1f433ce8dd7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"c6b4f8e2-a1ee-4438-aa37-67ff4b6ef1f1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T00:25:27.636 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:27.635+0000 7f40d2006640 1 -- 172.21.15.112:0/2424160811 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40cc0b2500 msgr2=0x7f40cc1548b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:27.636 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:27.635+0000 7f40d2006640 1 --2- 172.21.15.112:0/2424160811 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40cc0b2500 0x7f40cc1548b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f40bc00e7c0 tx=0x7f40bc00ec80 comp rx=0 tx=0).stop 2024-09-21T00:25:27.636 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:27.635+0000 7f40d2006640 1 -- 172.21.15.112:0/2424160811 shutdown_connections 2024-09-21T00:25:27.636 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:27.635+0000 7f40d2006640 1 --2- 172.21.15.112:0/2424160811 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f40cc0d19d0 0x7f40cc1f9320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:27.636 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:27.635+0000 7f40d2006640 1 --2- 172.21.15.112:0/2424160811 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f40cc0cd000 0x7f40cc06def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:27.636 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:27.635+0000 7f40d2006640 1 --2- 172.21.15.112:0/2424160811 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40cc0b2500 0x7f40cc1548b0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:27.636 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:27.635+0000 7f40d2006640 1 -- 172.21.15.112:0/2424160811 >> 172.21.15.112:0/2424160811 conn(0x7f40cc0b41d0 msgr2=0x7f40cc15b920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:27.636 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:27.635+0000 7f40d2006640 1 -- 172.21.15.112:0/2424160811 shutdown_connections 2024-09-21T00:25:27.636 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:27.635+0000 7f40d2006640 1 -- 172.21.15.112:0/2424160811 wait complete. 2024-09-21T00:25:27.645 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-21T00:25:27.841 INFO:tasks.ceph.osd.0.smithi112.stderr:2024-09-21T00:25:27.841+0000 7fb34e41e640 -1 Falling back to public interface 2024-09-21T00:25:27.855 INFO:tasks.ceph.osd.4.smithi187.stderr:2024-09-21T00:25:27.854+0000 7fb896d34640 -1 Falling back to public interface 2024-09-21T00:25:27.857 INFO:tasks.ceph.osd.3.smithi112.stderr:2024-09-21T00:25:27.856+0000 7f0b3841c640 -1 Falling back to public interface 2024-09-21T00:25:27.859 INFO:tasks.ceph.osd.6.smithi187.stderr:2024-09-21T00:25:27.858+0000 7f7db4dfd640 -1 Falling back to public interface 2024-09-21T00:25:28.099 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-09-21T00:25:28.098+0000 7f30297ce640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-21T00:25:28.099 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-09-21T00:25:28.098+0000 7f30297ce640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-21T00:25:28.106 INFO:tasks.ceph.osd.2.smithi112.stderr:2024-09-21T00:25:28.105+0000 7fec2adcc640 -1 Falling back to public interface 2024-09-21T00:25:28.121 INFO:tasks.ceph.osd.7.smithi187.stderr:2024-09-21T00:25:28.120+0000 7f098820b640 -1 Falling back to public interface 2024-09-21T00:25:28.358 INFO:tasks.ceph.osd.5.smithi187.stderr:2024-09-21T00:25:28.357+0000 7f72eb40f640 -1 Falling back to public interface 2024-09-21T00:25:31.419 INFO:tasks.ceph.osd.1.smithi112.stderr:2024-09-21T00:25:31.418+0000 7f69352ef640 -1 osd.1 0 log_to_monitors true 2024-09-21T00:25:31.423 INFO:tasks.ceph.osd.6.smithi187.stderr:2024-09-21T00:25:31.422+0000 7f7db4dfd640 -1 osd.6 0 log_to_monitors true 2024-09-21T00:25:31.636 INFO:tasks.ceph.osd.2.smithi112.stderr:2024-09-21T00:25:31.635+0000 7fec2adcc640 -1 osd.2 0 log_to_monitors true 2024-09-21T00:25:31.903 INFO:tasks.ceph.osd.0.smithi112.stderr:2024-09-21T00:25:31.903+0000 7fb34e41e640 -1 osd.0 0 log_to_monitors true 2024-09-21T00:25:32.119 INFO:tasks.ceph.osd.5.smithi187.stderr:2024-09-21T00:25:32.118+0000 7f72eb40f640 -1 osd.5 0 log_to_monitors true 2024-09-21T00:25:32.186 INFO:tasks.ceph.osd.4.smithi187.stderr:2024-09-21T00:25:32.185+0000 7fb896d34640 -1 osd.4 0 log_to_monitors true 2024-09-21T00:25:32.188 INFO:tasks.ceph.osd.3.smithi112.stderr:2024-09-21T00:25:32.187+0000 7f0b3841c640 -1 osd.3 0 log_to_monitors true 2024-09-21T00:25:32.413 INFO:tasks.ceph.osd.7.smithi187.stderr:2024-09-21T00:25:32.411+0000 7f098820b640 -1 osd.7 0 log_to_monitors true 2024-09-21T00:25:33.317 INFO:tasks.ceph.osd.1.smithi112.stderr:2024-09-21T00:25:33.316+0000 7f6931a92640 -1 osd.1 0 waiting for initial osdmap 2024-09-21T00:25:33.334 INFO:tasks.ceph.osd.1.smithi112.stderr:2024-09-21T00:25:33.334+0000 7f6929073640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T00:25:34.451 DEBUG:teuthology.orchestra.run.smithi112:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-21T00:25:34.584 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:34.583+0000 7f9090ddf640 1 -- 172.21.15.112:0/1158001546 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f908c0cb140 msgr2=0x7f908c06ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:34.584 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:34.583+0000 7f9090ddf640 1 --2- 172.21.15.112:0/1158001546 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f908c0cb140 0x7f908c06ddf0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f9080009830 tx=0x7f908002a510 comp rx=0 tx=0).stop 2024-09-21T00:25:34.584 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:34.583+0000 7f9090ddf640 1 -- 172.21.15.112:0/1158001546 shutdown_connections 2024-09-21T00:25:34.585 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:34.583+0000 7f9090ddf640 1 --2- 172.21.15.112:0/1158001546 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f908c0cb140 0x7f908c06ddf0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:34.585 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:34.583+0000 7f9090ddf640 1 --2- 172.21.15.112:0/1158001546 >> v2:172.21.15.187:3300/0 conn(0x7f908c0cb700 0x7f908c15c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:34.585 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:34.583+0000 7f9090ddf640 1 -- 172.21.15.112:0/1158001546 >> 172.21.15.112:0/1158001546 conn(0x7f908c073560 msgr2=0x7f908c073970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:34.585 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:34.583+0000 7f9090ddf640 1 -- 172.21.15.112:0/1158001546 shutdown_connections 2024-09-21T00:25:34.585 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:34.583+0000 7f9090ddf640 1 -- 172.21.15.112:0/1158001546 wait complete. 2024-09-21T00:25:34.587 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:34.586+0000 7f9090ddf640 1 Processor -- start 2024-09-21T00:25:34.587 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:34.586+0000 7f9090ddf640 1 -- start start 2024-09-21T00:25:34.590 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:34.587+0000 7f9090ddf640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f908c06e330 0x7f908c0ca0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:34.590 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:34.587+0000 7f9090ddf640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f908c0cb140 0x7f908c0ca5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:34.590 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:34.587+0000 7f9090ddf640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f908c0cb700 0x7f908c15ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:34.590 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:34.587+0000 7f9090ddf640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f908c0c5e10 con 0x7f908c06e330 2024-09-21T00:25:34.590 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:34.587+0000 7f9090ddf640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f908c070e50 con 0x7f908c0cb140 2024-09-21T00:25:34.591 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:34.587+0000 7f9090ddf640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f908c0c5f90 con 0x7f908c0cb700 2024-09-21T00:25:34.591 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:34.588+0000 7f908a59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f908c06e330 0x7f908c0ca0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:34.591 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:34.588+0000 7f908a59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f908c06e330 0x7f908c0ca0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42348/0 (socket says 172.21.15.112:42348) 2024-09-21T00:25:34.591 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:34.588+0000 7f908a59c640 1 -- 172.21.15.112:0/3723832548 learned_addr learned my addr 172.21.15.112:0/3723832548 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:25:34.591 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:34.588+0000 7f9089d9b640 1 --2- 172.21.15.112:0/3723832548 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f908c0cb140 0x7f908c0ca5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:34.591 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:34.588+0000 7f908ad9d640 1 --2- 172.21.15.112:0/3723832548 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f908c0cb700 0x7f908c15ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:34.591 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:34.588+0000 7f908a59c640 1 -- 172.21.15.112:0/3723832548 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f908c0cb700 msgr2=0x7f908c15ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:34.591 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:34.588+0000 7f908a59c640 1 --2- 172.21.15.112:0/3723832548 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f908c0cb700 0x7f908c15ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:34.591 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:34.588+0000 7f908a59c640 1 -- 172.21.15.112:0/3723832548 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f908c0cb140 msgr2=0x7f908c0ca5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:34.592 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:34.588+0000 7f908a59c640 1 --2- 172.21.15.112:0/3723832548 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f908c0cb140 0x7f908c0ca5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:34.592 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:34.588+0000 7f908a59c640 1 -- 172.21.15.112:0/3723832548 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f908c1f6c70 con 0x7f908c06e330 2024-09-21T00:25:34.592 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:34.588+0000 7f908a59c640 1 --2- 172.21.15.112:0/3723832548 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f908c06e330 0x7f908c0ca0b0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f907c00ea20 tx=0x7f907c00eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:25:34.592 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:34.588+0000 7f907affd640 1 -- 172.21.15.112:0/3723832548 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f907c00ce40 con 0x7f908c06e330 2024-09-21T00:25:34.592 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:34.588+0000 7f907affd640 1 -- 172.21.15.112:0/3723832548 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f907c004020 con 0x7f908c06e330 2024-09-21T00:25:34.592 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:34.588+0000 7f907affd640 1 -- 172.21.15.112:0/3723832548 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f907c004470 con 0x7f908c06e330 2024-09-21T00:25:34.592 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:34.588+0000 7f9090ddf640 1 -- 172.21.15.112:0/3723832548 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f908c1f6f50 con 0x7f908c06e330 2024-09-21T00:25:34.592 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:34.589+0000 7f9090ddf640 1 -- 172.21.15.112:0/3723832548 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f908c1f7510 con 0x7f908c06e330 2024-09-21T00:25:34.593 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:34.590+0000 7f9090ddf640 1 -- 172.21.15.112:0/3723832548 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f908c1588c0 con 0x7f908c06e330 2024-09-21T00:25:34.593 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:34.590+0000 7f907affd640 1 -- 172.21.15.112:0/3723832548 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 4) ==== 101385+0+0 (secure 0 0 0) 0x7f907c0035f0 con 0x7f908c06e330 2024-09-21T00:25:34.593 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:34.591+0000 7f907affd640 1 --2- 172.21.15.112:0/3723832548 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f906c078b00 0x7f906c07b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:34.594 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:34.591+0000 7f907affd640 1 -- 172.21.15.112:0/3723832548 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 2511+0+0 (secure 0 0 0) 0x7f907c09a930 con 0x7f908c06e330 2024-09-21T00:25:34.594 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:34.591+0000 7f9089d9b640 1 --2- 172.21.15.112:0/3723832548 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f906c078b00 0x7f906c07b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:34.594 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:34.592+0000 7f9089d9b640 1 --2- 172.21.15.112:0/3723832548 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f906c078b00 0x7f906c07b010 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f9080034000 tx=0x7f908000b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:25:34.600 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:34.600+0000 7f907affd640 1 -- 172.21.15.112:0/3723832548 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f907c0634f0 con 0x7f908c06e330 2024-09-21T00:25:34.908 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:34.907+0000 7f9090ddf640 1 -- 172.21.15.112:0/3723832548 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f908c1585e0 con 0x7f908c06e330 2024-09-21T00:25:34.909 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:34.908+0000 7f907affd640 1 -- 172.21.15.112:0/3723832548 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) ==== 74+0+7555 (secure 0 0 0) 0x7f907c067f20 con 0x7f908c06e330 2024-09-21T00:25:34.909 INFO:teuthology.misc.health.smithi112.stdout: 2024-09-21T00:25:34.909 INFO:teuthology.misc.health.smithi112.stdout:{"epoch":13,"fsid":"8b34692b-dc65-49b2-a08f-26f707b96ac5","created":"2024-09-21T00:25:22.788490+0000","modified":"2024-09-21T00:25:33.474688+0000","last_up_change":"2024-09-21T00:25:33.474688+0000","last_in_change":"2024-09-21T00:25:26.346500+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"9c575086-67d2-4de2-b01b-d44dcc21d604","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"a983c227-a3ce-4bdd-82f1-438ba7e3ecd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6801","nonce":344932592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6803","nonce":344932592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6807","nonce":344932592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6805","nonce":344932592}]},"public_addr":"172.21.15.112:6801/344932592","cluster_addr":"172.21.15.112:6803/344932592","heartbeat_back_addr":"172.21.15.112:6807/344932592","heartbeat_front_addr":"172.21.15.112:6805/344932592","state":["exists","up"]},{"osd":2,"uuid":"1193c1d8-bfff-4a57-8200-5f69bf1ea3af","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"af04330a-36ad-40cc-a8dd-fa259b221de5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"5b893111-292c-4c5a-b184-53ad38c7199e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"c0b8e011-45fe-4c5e-925e-6be2a7f1868a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"87d2d650-7489-4593-98a3-a1f433ce8dd7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"c6b4f8e2-a1ee-4438-aa37-67ff4b6ef1f1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T00:25:34.912 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:34.911+0000 7f9090ddf640 1 -- 172.21.15.112:0/3723832548 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f906c078b00 msgr2=0x7f906c07b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:34.912 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:34.911+0000 7f9090ddf640 1 --2- 172.21.15.112:0/3723832548 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f906c078b00 0x7f906c07b010 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f9080034000 tx=0x7f908000b040 comp rx=0 tx=0).stop 2024-09-21T00:25:34.912 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:34.911+0000 7f9090ddf640 1 -- 172.21.15.112:0/3723832548 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f908c06e330 msgr2=0x7f908c0ca0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:34.912 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:34.911+0000 7f9090ddf640 1 --2- 172.21.15.112:0/3723832548 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f908c06e330 0x7f908c0ca0b0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f907c00ea20 tx=0x7f907c00eee0 comp rx=0 tx=0).stop 2024-09-21T00:25:34.913 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:34.911+0000 7f9090ddf640 1 -- 172.21.15.112:0/3723832548 shutdown_connections 2024-09-21T00:25:34.913 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:34.911+0000 7f9090ddf640 1 --2- 172.21.15.112:0/3723832548 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f908c0cb700 0x7f908c15ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:34.913 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:34.911+0000 7f9090ddf640 1 --2- 172.21.15.112:0/3723832548 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f908c0cb140 0x7f908c0ca5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:34.913 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:34.911+0000 7f9090ddf640 1 --2- 172.21.15.112:0/3723832548 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f906c078b00 0x7f906c07b010 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:34.913 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:34.911+0000 7f9090ddf640 1 --2- 172.21.15.112:0/3723832548 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f908c06e330 0x7f908c0ca0b0 secure :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f907c00ea20 tx=0x7f907c00eee0 comp rx=0 tx=0).stop 2024-09-21T00:25:34.913 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:34.911+0000 7f9090ddf640 1 -- 172.21.15.112:0/3723832548 >> 172.21.15.112:0/3723832548 conn(0x7f908c073560 msgr2=0x7f908c068d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:34.913 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:34.912+0000 7f9090ddf640 1 -- 172.21.15.112:0/3723832548 shutdown_connections 2024-09-21T00:25:34.913 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:34.912+0000 7f9090ddf640 1 -- 172.21.15.112:0/3723832548 wait complete. 2024-09-21T00:25:34.928 DEBUG:teuthology.misc:1 of 8 OSDs are up 2024-09-21T00:25:35.160 INFO:tasks.ceph.osd.2.smithi112.stderr:2024-09-21T00:25:35.159+0000 7fec2756f640 -1 osd.2 0 waiting for initial osdmap 2024-09-21T00:25:35.170 INFO:tasks.ceph.osd.2.smithi112.stderr:2024-09-21T00:25:35.170+0000 7fec1eb50640 -1 osd.2 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T00:25:35.210 INFO:tasks.ceph.osd.0.smithi112.stderr:2024-09-21T00:25:35.209+0000 7fb349e0b640 -1 osd.0 0 waiting for initial osdmap 2024-09-21T00:25:35.219 INFO:tasks.ceph.osd.0.smithi112.stderr:2024-09-21T00:25:35.219+0000 7fb341c01640 -1 osd.0 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T00:25:35.234 INFO:tasks.ceph.osd.3.smithi112.stderr:2024-09-21T00:25:35.233+0000 7f0b345b5640 -1 osd.3 0 waiting for initial osdmap 2024-09-21T00:25:35.243 INFO:tasks.ceph.osd.3.smithi112.stderr:2024-09-21T00:25:35.243+0000 7f0b2bb96640 -1 osd.3 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T00:25:35.674 INFO:tasks.ceph.osd.7.smithi187.stderr:2024-09-21T00:25:35.673+0000 7f0983bf8640 -1 osd.7 0 waiting for initial osdmap 2024-09-21T00:25:35.691 INFO:tasks.ceph.osd.5.smithi187.stderr:2024-09-21T00:25:35.690+0000 7f72e6df8640 -1 osd.5 0 waiting for initial osdmap 2024-09-21T00:25:35.693 INFO:tasks.ceph.osd.7.smithi187.stderr:2024-09-21T00:25:35.692+0000 7f097b9ee640 -1 osd.7 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T00:25:35.695 INFO:tasks.ceph.osd.4.smithi187.stderr:2024-09-21T00:25:35.694+0000 7fb892cc2640 -1 osd.4 0 waiting for initial osdmap 2024-09-21T00:25:35.705 INFO:tasks.ceph.osd.6.smithi187.stderr:2024-09-21T00:25:35.704+0000 7f7db15a0640 -1 osd.6 0 waiting for initial osdmap 2024-09-21T00:25:35.706 INFO:tasks.ceph.osd.5.smithi187.stderr:2024-09-21T00:25:35.705+0000 7f72debee640 -1 osd.5 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T00:25:35.709 INFO:tasks.ceph.osd.4.smithi187.stderr:2024-09-21T00:25:35.708+0000 7fb88aab8640 -1 osd.4 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T00:25:35.718 INFO:tasks.ceph.osd.6.smithi187.stderr:2024-09-21T00:25:35.717+0000 7f7da8b81640 -1 osd.6 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T00:25:41.734 DEBUG:teuthology.orchestra.run.smithi112:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-21T00:25:41.851 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:41.849+0000 7f0217369640 1 -- 172.21.15.112:0/1619429656 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02100d1090 msgr2=0x7f021015a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:41.851 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:41.849+0000 7f0217369640 1 --2- 172.21.15.112:0/1619429656 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02100d1090 0x7f021015a6f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0208005fd0 tx=0x7f020802a940 comp rx=0 tx=0).stop 2024-09-21T00:25:41.851 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:41.849+0000 7f0217369640 1 -- 172.21.15.112:0/1619429656 shutdown_connections 2024-09-21T00:25:41.851 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:41.849+0000 7f0217369640 1 --2- 172.21.15.112:0/1619429656 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02100d1090 0x7f021015a6f0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:41.851 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:41.849+0000 7f0217369640 1 -- 172.21.15.112:0/1619429656 >> 172.21.15.112:0/1619429656 conn(0x7f0210072990 msgr2=0x7f0210072da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:41.851 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:41.849+0000 7f0217369640 1 -- 172.21.15.112:0/1619429656 shutdown_connections 2024-09-21T00:25:41.852 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:41.849+0000 7f0217369640 1 -- 172.21.15.112:0/1619429656 wait complete. 2024-09-21T00:25:41.852 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:41.851+0000 7f0217369640 1 Processor -- start 2024-09-21T00:25:41.853 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:41.851+0000 7f0217369640 1 -- start start 2024-09-21T00:25:41.855 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:41.852+0000 7f0217369640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f02100d01f0 0x7f02100caef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:41.856 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:41.852+0000 7f0217369640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f021015daf0 0x7f021015f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:41.856 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:41.852+0000 7f0217369640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f021015f820 0x7f02100b2880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:41.856 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:41.852+0000 7f0217369640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f02100cd3d0 con 0x7f021015daf0 2024-09-21T00:25:41.856 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:41.852+0000 7f0217369640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f02100cd250 con 0x7f02100d01f0 2024-09-21T00:25:41.856 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:41.852+0000 7f0217369640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f02100cd550 con 0x7f021015f820 2024-09-21T00:25:41.856 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:41.852+0000 7f0214904640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f021015daf0 0x7f021015f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:41.856 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:41.852+0000 7f0214904640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f021015daf0 0x7f021015f2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60526/0 (socket says 172.21.15.112:60526) 2024-09-21T00:25:41.856 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:41.852+0000 7f0214904640 1 -- 172.21.15.112:0/1036936501 learned_addr learned my addr 172.21.15.112:0/1036936501 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:25:41.856 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:41.852+0000 7f0215105640 1 --2- 172.21.15.112:0/1036936501 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f02100d01f0 0x7f02100caef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:41.856 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:41.853+0000 7f0215906640 1 --2- 172.21.15.112:0/1036936501 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f021015f820 0x7f02100b2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:41.856 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:41.853+0000 7f0214904640 1 -- 172.21.15.112:0/1036936501 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f021015f820 msgr2=0x7f02100b2880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:41.856 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:41.853+0000 7f0214904640 1 --2- 172.21.15.112:0/1036936501 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f021015f820 0x7f02100b2880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:41.856 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:41.853+0000 7f0214904640 1 -- 172.21.15.112:0/1036936501 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f02100d01f0 msgr2=0x7f02100caef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:41.856 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:41.853+0000 7f0214904640 1 --2- 172.21.15.112:0/1036936501 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f02100d01f0 0x7f02100caef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:41.857 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:41.853+0000 7f0214904640 1 -- 172.21.15.112:0/1036936501 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02100cb430 con 0x7f021015daf0 2024-09-21T00:25:41.857 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:41.853+0000 7f0215906640 1 --2- 172.21.15.112:0/1036936501 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f021015f820 0x7f02100b2880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:25:41.857 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:41.853+0000 7f0214904640 1 --2- 172.21.15.112:0/1036936501 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f021015daf0 0x7f021015f2e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f020000c370 tx=0x7f020000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:25:41.857 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:41.853+0000 7f0205ffb640 1 -- 172.21.15.112:0/1036936501 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0200017070 con 0x7f021015daf0 2024-09-21T00:25:41.857 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:41.853+0000 7f0205ffb640 1 -- 172.21.15.112:0/1036936501 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f02000035f0 con 0x7f021015daf0 2024-09-21T00:25:41.857 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:41.853+0000 7f0217369640 1 -- 172.21.15.112:0/1036936501 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02100cb700 con 0x7f021015daf0 2024-09-21T00:25:41.857 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:41.853+0000 7f0205ffb640 1 -- 172.21.15.112:0/1036936501 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0200003a40 con 0x7f021015daf0 2024-09-21T00:25:41.857 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:41.853+0000 7f0217369640 1 -- 172.21.15.112:0/1036936501 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02100a98d0 con 0x7f021015daf0 2024-09-21T00:25:41.858 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:41.854+0000 7f0217369640 1 -- 172.21.15.112:0/1036936501 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02100cfe90 con 0x7f021015daf0 2024-09-21T00:25:41.858 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:41.855+0000 7f0205ffb640 1 -- 172.21.15.112:0/1036936501 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f020001a070 con 0x7f021015daf0 2024-09-21T00:25:41.858 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:41.855+0000 7f0205ffb640 1 --2- 172.21.15.112:0/1036936501 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f01f4078bc0 0x7f01f407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:41.858 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:41.855+0000 7f0205ffb640 1 -- 172.21.15.112:0/1036936501 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7f0200099b50 con 0x7f021015daf0 2024-09-21T00:25:41.858 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:41.856+0000 7f0215105640 1 --2- 172.21.15.112:0/1036936501 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f01f4078bc0 0x7f01f407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:41.858 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:41.856+0000 7f0215105640 1 --2- 172.21.15.112:0/1036936501 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f01f4078bc0 0x7f01f407b0d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f020c000f60 tx=0x7f020c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:25:41.862 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:41.860+0000 7f0205ffb640 1 -- 172.21.15.112:0/1036936501 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f02100cfe90 con 0x7f021015daf0 2024-09-21T00:25:42.040 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:42.039+0000 7f0217369640 1 -- 172.21.15.112:0/1036936501 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f021015a6f0 con 0x7f021015daf0 2024-09-21T00:25:42.041 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:42.039+0000 7f0205ffb640 1 -- 172.21.15.112:0/1036936501 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) ==== 74+0+12920 (secure 0 0 0) 0x7f0200061d20 con 0x7f021015daf0 2024-09-21T00:25:42.041 INFO:teuthology.misc.health.smithi112.stdout: 2024-09-21T00:25:42.041 INFO:teuthology.misc.health.smithi112.stdout:{"epoch":18,"fsid":"8b34692b-dc65-49b2-a08f-26f707b96ac5","created":"2024-09-21T00:25:22.788490+0000","modified":"2024-09-21T00:25:39.170497+0000","last_up_change":"2024-09-21T00:25:36.161835+0000","last_in_change":"2024-09-21T00:25:26.346500+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T00:25:37.105006+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9c575086-67d2-4de2-b01b-d44dcc21d604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6809","nonce":755760621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6811","nonce":755760621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6815","nonce":755760621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6813","nonce":755760621}]},"public_addr":"172.21.15.112:6809/755760621","cluster_addr":"172.21.15.112:6811/755760621","heartbeat_back_addr":"172.21.15.112:6815/755760621","heartbeat_front_addr":"172.21.15.112:6813/755760621","state":["exists","up"]},{"osd":1,"uuid":"a983c227-a3ce-4bdd-82f1-438ba7e3ecd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6801","nonce":344932592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6803","nonce":344932592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6807","nonce":344932592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6805","nonce":344932592}]},"public_addr":"172.21.15.112:6801/344932592","cluster_addr":"172.21.15.112:6803/344932592","heartbeat_back_addr":"172.21.15.112:6807/344932592","heartbeat_front_addr":"172.21.15.112:6805/344932592","state":["exists","up"]},{"osd":2,"uuid":"1193c1d8-bfff-4a57-8200-5f69bf1ea3af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6825","nonce":407933616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6827","nonce":407933616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6831","nonce":407933616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6829","nonce":407933616}]},"public_addr":"172.21.15.112:6825/407933616","cluster_addr":"172.21.15.112:6827/407933616","heartbeat_back_addr":"172.21.15.112:6831/407933616","heartbeat_front_addr":"172.21.15.112:6829/407933616","state":["exists","up"]},{"osd":3,"uuid":"af04330a-36ad-40cc-a8dd-fa259b221de5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6817","nonce":4120164806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6819","nonce":4120164806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6823","nonce":4120164806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6821","nonce":4120164806}]},"public_addr":"172.21.15.112:6817/4120164806","cluster_addr":"172.21.15.112:6819/4120164806","heartbeat_back_addr":"172.21.15.112:6823/4120164806","heartbeat_front_addr":"172.21.15.112:6821/4120164806","state":["exists","up"]},{"osd":4,"uuid":"5b893111-292c-4c5a-b184-53ad38c7199e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6803","nonce":522077560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6805","nonce":522077560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6809","nonce":522077560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6807","nonce":522077560}]},"public_addr":"172.21.15.187:6803/522077560","cluster_addr":"172.21.15.187:6805/522077560","heartbeat_back_addr":"172.21.15.187:6809/522077560","heartbeat_front_addr":"172.21.15.187:6807/522077560","state":["exists","up"]},{"osd":5,"uuid":"c0b8e011-45fe-4c5e-925e-6be2a7f1868a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6827","nonce":3593144721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6829","nonce":3593144721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6833","nonce":3593144721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6831","nonce":3593144721}]},"public_addr":"172.21.15.187:6827/3593144721","cluster_addr":"172.21.15.187:6829/3593144721","heartbeat_back_addr":"172.21.15.187:6833/3593144721","heartbeat_front_addr":"172.21.15.187:6831/3593144721","state":["exists","up"]},{"osd":6,"uuid":"87d2d650-7489-4593-98a3-a1f433ce8dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6811","nonce":1494387657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6813","nonce":1494387657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6817","nonce":1494387657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6815","nonce":1494387657}]},"public_addr":"172.21.15.187:6811/1494387657","cluster_addr":"172.21.15.187:6813/1494387657","heartbeat_back_addr":"172.21.15.187:6817/1494387657","heartbeat_front_addr":"172.21.15.187:6815/1494387657","state":["exists","up"]},{"osd":7,"uuid":"c6b4f8e2-a1ee-4438-aa37-67ff4b6ef1f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6819","nonce":1624699842}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6821","nonce":1624699842}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6825","nonce":1624699842}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6823","nonce":1624699842}]},"public_addr":"172.21.15.187:6819/1624699842","cluster_addr":"172.21.15.187:6821/1624699842","heartbeat_back_addr":"172.21.15.187:6825/1624699842","heartbeat_front_addr":"172.21.15.187:6823/1624699842","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.873162+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.400506+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.664317+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.159791+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.202630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.113212+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.384786+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.448441+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T00:25:42.044 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:42.042+0000 7f0217369640 1 -- 172.21.15.112:0/1036936501 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f01f4078bc0 msgr2=0x7f01f407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:42.044 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:42.042+0000 7f0217369640 1 --2- 172.21.15.112:0/1036936501 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f01f4078bc0 0x7f01f407b0d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f020c000f60 tx=0x7f020c009040 comp rx=0 tx=0).stop 2024-09-21T00:25:42.044 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:42.042+0000 7f0217369640 1 -- 172.21.15.112:0/1036936501 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f021015daf0 msgr2=0x7f021015f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:42.044 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:42.042+0000 7f0217369640 1 --2- 172.21.15.112:0/1036936501 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f021015daf0 0x7f021015f2e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f020000c370 tx=0x7f020000c830 comp rx=0 tx=0).stop 2024-09-21T00:25:42.044 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:42.042+0000 7f0217369640 1 -- 172.21.15.112:0/1036936501 shutdown_connections 2024-09-21T00:25:42.044 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:42.042+0000 7f0217369640 1 --2- 172.21.15.112:0/1036936501 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f021015f820 0x7f02100b2880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:42.044 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:42.042+0000 7f0217369640 1 --2- 172.21.15.112:0/1036936501 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f01f4078bc0 0x7f01f407b0d0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:42.044 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:42.042+0000 7f0217369640 1 --2- 172.21.15.112:0/1036936501 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f021015daf0 0x7f021015f2e0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:42.044 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:42.042+0000 7f0217369640 1 --2- 172.21.15.112:0/1036936501 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f02100d01f0 0x7f02100caef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:42.044 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:42.042+0000 7f0217369640 1 -- 172.21.15.112:0/1036936501 >> 172.21.15.112:0/1036936501 conn(0x7f0210072990 msgr2=0x7f0210068d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:42.045 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:42.042+0000 7f0217369640 1 -- 172.21.15.112:0/1036936501 shutdown_connections 2024-09-21T00:25:42.045 INFO:teuthology.misc.health.smithi112.stderr:2024-09-21T00:25:42.042+0000 7f0217369640 1 -- 172.21.15.112:0/1036936501 wait complete. 2024-09-21T00:25:42.053 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-09-21T00:25:42.053 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-21T00:25:42.053 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-21T00:25:42.054 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-21T00:25:42.056 INFO:tasks.ceph.mds.a:Started 2024-09-21T00:25:42.056 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-21T00:25:42.056 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-21T00:25:42.058 INFO:tasks.ceph.mds.c:Started 2024-09-21T00:25:42.058 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-21T00:25:42.058 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-21T00:25:42.060 INFO:tasks.ceph.mds.b:Started 2024-09-21T00:25:42.061 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-21T00:25:42.061 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-21T00:25:42.063 INFO:tasks.ceph.mds.d:Started 2024-09-21T00:25:42.063 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-09-21T00:25:42.063 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T00:25:42.140 INFO:tasks.ceph.mds.c.smithi112.stdout:starting mds.c at 2024-09-21T00:25:42.141 INFO:tasks.ceph.mds.a.smithi112.stdout:starting mds.a at 2024-09-21T00:25:42.159 INFO:tasks.ceph.mds.b.smithi187.stdout:starting mds.b at 2024-09-21T00:25:42.159 INFO:tasks.ceph.mds.d.smithi187.stdout:starting mds.d at 2024-09-21T00:25:42.159 INFO:tasks.ceph.mds.d.smithi187.stderr:2024-09-21T00:25:42.158+0000 7fed132d9640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2024-09-21T00:25:42.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:42.183+0000 7fb29534a640 1 -- 172.21.15.112:0/3741332866 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb2900c5700 msgr2=0x7fb2900afa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:42.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:42.183+0000 7fb29534a640 1 --2- 172.21.15.112:0/3741332866 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb2900c5700 0x7fb2900afa50 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb278009830 tx=0x7fb27802a480 comp rx=0 tx=0).stop 2024-09-21T00:25:42.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:42.183+0000 7fb29534a640 1 -- 172.21.15.112:0/3741332866 shutdown_connections 2024-09-21T00:25:42.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:42.183+0000 7fb29534a640 1 --2- 172.21.15.112:0/3741332866 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb2900c5700 0x7fb2900afa50 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:42.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:42.183+0000 7fb29534a640 1 --2- 172.21.15.112:0/3741332866 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb29015ad20 0x7fb2900cc1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:42.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:42.184+0000 7fb29534a640 1 -- 172.21.15.112:0/3741332866 >> 172.21.15.112:0/3741332866 conn(0x7fb290073560 msgr2=0x7fb290073970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:42.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:42.184+0000 7fb29534a640 1 -- 172.21.15.112:0/3741332866 shutdown_connections 2024-09-21T00:25:42.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:42.184+0000 7fb29534a640 1 -- 172.21.15.112:0/3741332866 wait complete. 2024-09-21T00:25:42.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:42.185+0000 7fb29534a640 1 Processor -- start 2024-09-21T00:25:42.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:42.186+0000 7fb29534a640 1 -- start start 2024-09-21T00:25:42.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:42.187+0000 7fb29534a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb29006ddf0 0x7fb290149310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:42.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:42.187+0000 7fb29534a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb2900c5700 0x7fb2900c9b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:42.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:42.187+0000 7fb29534a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb29015ad20 0x7fb2900ca0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:42.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:42.187+0000 7fb29534a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb2900cedf0 con 0x7fb29006ddf0 2024-09-21T00:25:42.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:42.187+0000 7fb29534a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb2900cec70 con 0x7fb2900c5700 2024-09-21T00:25:42.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:42.187+0000 7fb29534a640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fb2900cef70 con 0x7fb29015ad20 2024-09-21T00:25:42.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:42.187+0000 7fb28effd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb29006ddf0 0x7fb290149310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:42.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:42.187+0000 7fb28effd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb29006ddf0 0x7fb290149310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60582/0 (socket says 172.21.15.112:60582) 2024-09-21T00:25:42.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:42.187+0000 7fb28effd640 1 -- 172.21.15.112:0/2880073073 learned_addr learned my addr 172.21.15.112:0/2880073073 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:25:42.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:42.187+0000 7fb287fff640 1 --2- 172.21.15.112:0/2880073073 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb2900c5700 0x7fb2900c9b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:42.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:42.187+0000 7fb28f7fe640 1 --2- 172.21.15.112:0/2880073073 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb29015ad20 0x7fb2900ca0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:42.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:42.187+0000 7fb28effd640 1 -- 172.21.15.112:0/2880073073 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb29015ad20 msgr2=0x7fb2900ca0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:42.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:42.187+0000 7fb28effd640 1 --2- 172.21.15.112:0/2880073073 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb29015ad20 0x7fb2900ca0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:42.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:42.187+0000 7fb28effd640 1 -- 172.21.15.112:0/2880073073 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb2900c5700 msgr2=0x7fb2900c9b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:42.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:42.187+0000 7fb28effd640 1 --2- 172.21.15.112:0/2880073073 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb2900c5700 0x7fb2900c9b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:42.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:42.187+0000 7fb28effd640 1 -- 172.21.15.112:0/2880073073 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb290149850 con 0x7fb29006ddf0 2024-09-21T00:25:42.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:42.187+0000 7fb28effd640 1 --2- 172.21.15.112:0/2880073073 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb29006ddf0 0x7fb290149310 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb28000e9f0 tx=0x7fb28000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:25:42.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:42.188+0000 7fb2877fe640 1 -- 172.21.15.112:0/2880073073 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb28000cd80 con 0x7fb29006ddf0 2024-09-21T00:25:42.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:42.188+0000 7fb2877fe640 1 -- 172.21.15.112:0/2880073073 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb280004020 con 0x7fb29006ddf0 2024-09-21T00:25:42.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:42.188+0000 7fb2877fe640 1 -- 172.21.15.112:0/2880073073 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb280004450 con 0x7fb29006ddf0 2024-09-21T00:25:42.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:42.188+0000 7fb29534a640 1 -- 172.21.15.112:0/2880073073 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2900c8150 con 0x7fb29006ddf0 2024-09-21T00:25:42.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:42.188+0000 7fb29534a640 1 -- 172.21.15.112:0/2880073073 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2901e6470 con 0x7fb29006ddf0 2024-09-21T00:25:42.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:42.189+0000 7fb29534a640 1 -- 172.21.15.112:0/2880073073 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2900c6c20 con 0x7fb29006ddf0 2024-09-21T00:25:42.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:42.189+0000 7fb2877fe640 1 -- 172.21.15.112:0/2880073073 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb2800035f0 con 0x7fb29006ddf0 2024-09-21T00:25:42.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:42.190+0000 7fb2877fe640 1 --2- 172.21.15.112:0/2880073073 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fb270078bc0 0x7fb27007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:42.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:42.190+0000 7fb2877fe640 1 -- 172.21.15.112:0/2880073073 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7fb28009b2b0 con 0x7fb29006ddf0 2024-09-21T00:25:42.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:42.190+0000 7fb287fff640 1 --2- 172.21.15.112:0/2880073073 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fb270078bc0 0x7fb27007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:42.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:42.190+0000 7fb287fff640 1 --2- 172.21.15.112:0/2880073073 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fb270078bc0 0x7fb27007b0d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fb278034000 tx=0x7fb2780046c0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:25:42.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:42.194+0000 7fb2877fe640 1 -- 172.21.15.112:0/2880073073 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb290149850 con 0x7fb29006ddf0 2024-09-21T00:25:42.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:42.407+0000 7fb29534a640 1 -- 172.21.15.112:0/2880073073 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb290076580 con 0x7fb29006ddf0 2024-09-21T00:25:42.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:42.407+0000 7fb2877fe640 1 -- 172.21.15.112:0/2880073073 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1434 (secure 0 0 0) 0x7fb280063ad0 con 0x7fb29006ddf0 2024-09-21T00:25:42.409 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:25:42.409 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":2,"btime":"2024-09-21T00:25:42:145061+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4413,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/200348130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":200348130},{"type":"v1","addr":"172.21.15.112:6833","nonce":200348130}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-21T00:25:42.410 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 2 2024-09-21T00:25:42.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:42.409+0000 7fb29534a640 1 -- 172.21.15.112:0/2880073073 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fb270078bc0 msgr2=0x7fb27007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:42.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:42.409+0000 7fb29534a640 1 --2- 172.21.15.112:0/2880073073 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fb270078bc0 0x7fb27007b0d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fb278034000 tx=0x7fb2780046c0 comp rx=0 tx=0).stop 2024-09-21T00:25:42.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:42.409+0000 7fb29534a640 1 -- 172.21.15.112:0/2880073073 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb29006ddf0 msgr2=0x7fb290149310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:42.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:42.409+0000 7fb29534a640 1 --2- 172.21.15.112:0/2880073073 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb29006ddf0 0x7fb290149310 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb28000e9f0 tx=0x7fb28000eeb0 comp rx=0 tx=0).stop 2024-09-21T00:25:42.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:42.409+0000 7fb29534a640 1 -- 172.21.15.112:0/2880073073 shutdown_connections 2024-09-21T00:25:42.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:42.409+0000 7fb29534a640 1 --2- 172.21.15.112:0/2880073073 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb29015ad20 0x7fb2900ca0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:42.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:42.409+0000 7fb29534a640 1 --2- 172.21.15.112:0/2880073073 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb2900c5700 0x7fb2900c9b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:42.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:42.409+0000 7fb29534a640 1 --2- 172.21.15.112:0/2880073073 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fb270078bc0 0x7fb27007b0d0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:42.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:42.409+0000 7fb29534a640 1 --2- 172.21.15.112:0/2880073073 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb29006ddf0 0x7fb290149310 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:42.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:42.409+0000 7fb29534a640 1 -- 172.21.15.112:0/2880073073 >> 172.21.15.112:0/2880073073 conn(0x7fb290073560 msgr2=0x7fb2900b1780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:42.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:42.410+0000 7fb29534a640 1 -- 172.21.15.112:0/2880073073 shutdown_connections 2024-09-21T00:25:42.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:42.410+0000 7fb29534a640 1 -- 172.21.15.112:0/2880073073 wait complete. 2024-09-21T00:25:44.423 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T00:25:44.510 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T00:25:44.513 DEBUG:teuthology.orchestra.run.smithi187:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T00:25:44.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:44.548+0000 7fac4ac2e640 1 -- 172.21.15.112:0/1222389168 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fac44162590 msgr2=0x7fac441629a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:44.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:44.548+0000 7fac4ac2e640 1 --2- 172.21.15.112:0/1222389168 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fac44162590 0x7fac441629a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fac38009830 tx=0x7fac3802a510 comp rx=0 tx=0).stop 2024-09-21T00:25:44.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:44.548+0000 7fac4ac2e640 1 -- 172.21.15.112:0/1222389168 shutdown_connections 2024-09-21T00:25:44.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:44.548+0000 7fac4ac2e640 1 --2- 172.21.15.112:0/1222389168 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fac44162590 0x7fac441629a0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:44.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:44.548+0000 7fac4ac2e640 1 --2- 172.21.15.112:0/1222389168 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fac440cfe00 0x7fac440d2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:44.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:44.548+0000 7fac4ac2e640 1 -- 172.21.15.112:0/1222389168 >> 172.21.15.112:0/1222389168 conn(0x7fac440d4440 msgr2=0x7fac440d4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:44.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:44.548+0000 7fac4ac2e640 1 -- 172.21.15.112:0/1222389168 shutdown_connections 2024-09-21T00:25:44.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:44.548+0000 7fac4ac2e640 1 -- 172.21.15.112:0/1222389168 wait complete. 2024-09-21T00:25:44.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:44.550+0000 7fac4ac2e640 1 Processor -- start 2024-09-21T00:25:44.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:44.550+0000 7fac4ac2e640 1 -- start start 2024-09-21T00:25:44.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:44.551+0000 7fac4ac2e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac440b2e60 0x7fac441565b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:44.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:44.551+0000 7fac4ac2e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fac440cfe00 0x7fac4415a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:44.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:44.551+0000 7fac4ac2e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fac44162590 0x7fac4415ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:44.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:44.551+0000 7fac4ac2e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fac440b3b00 con 0x7fac440b2e60 2024-09-21T00:25:44.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:44.551+0000 7fac4ac2e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fac440b3980 con 0x7fac440cfe00 2024-09-21T00:25:44.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:44.551+0000 7fac4ac2e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fac440b3f60 con 0x7fac44162590 2024-09-21T00:25:44.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:44.551+0000 7fac489ca640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac440b2e60 0x7fac441565b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:44.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:44.552+0000 7fac489ca640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac440b2e60 0x7fac441565b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60588/0 (socket says 172.21.15.112:60588) 2024-09-21T00:25:44.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:44.552+0000 7fac489ca640 1 -- 172.21.15.112:0/1814884278 learned_addr learned my addr 172.21.15.112:0/1814884278 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:25:44.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:44.552+0000 7fac491cb640 1 --2- 172.21.15.112:0/1814884278 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fac44162590 0x7fac4415ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:44.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:44.552+0000 7fac43fff640 1 --2- 172.21.15.112:0/1814884278 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fac440cfe00 0x7fac4415a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:44.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:44.552+0000 7fac489ca640 1 -- 172.21.15.112:0/1814884278 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fac44162590 msgr2=0x7fac4415ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:44.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:44.552+0000 7fac489ca640 1 --2- 172.21.15.112:0/1814884278 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fac44162590 0x7fac4415ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:44.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:44.552+0000 7fac489ca640 1 -- 172.21.15.112:0/1814884278 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fac440cfe00 msgr2=0x7fac4415a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:44.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:44.552+0000 7fac489ca640 1 --2- 172.21.15.112:0/1814884278 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fac440cfe00 0x7fac4415a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:44.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:44.552+0000 7fac489ca640 1 -- 172.21.15.112:0/1814884278 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac440ce480 con 0x7fac440b2e60 2024-09-21T00:25:44.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:44.552+0000 7fac43fff640 1 --2- 172.21.15.112:0/1814884278 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fac440cfe00 0x7fac4415a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:25:44.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:44.552+0000 7fac489ca640 1 --2- 172.21.15.112:0/1814884278 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac440b2e60 0x7fac441565b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fac3400e9f0 tx=0x7fac3400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:25:44.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:44.552+0000 7fac417fa640 1 -- 172.21.15.112:0/1814884278 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac3400cd80 con 0x7fac440b2e60 2024-09-21T00:25:44.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:44.552+0000 7fac4ac2e640 1 -- 172.21.15.112:0/1814884278 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac440ce760 con 0x7fac440b2e60 2024-09-21T00:25:44.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:44.552+0000 7fac417fa640 1 -- 172.21.15.112:0/1814884278 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fac34004020 con 0x7fac440b2e60 2024-09-21T00:25:44.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:44.552+0000 7fac417fa640 1 -- 172.21.15.112:0/1814884278 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac34004470 con 0x7fac440b2e60 2024-09-21T00:25:44.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:44.552+0000 7fac4ac2e640 1 -- 172.21.15.112:0/1814884278 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac440c8c00 con 0x7fac440b2e60 2024-09-21T00:25:44.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:44.553+0000 7fac4ac2e640 1 -- 172.21.15.112:0/1814884278 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac440c7b80 con 0x7fac440b2e60 2024-09-21T00:25:44.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:44.554+0000 7fac417fa640 1 -- 172.21.15.112:0/1814884278 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fac340035f0 con 0x7fac440b2e60 2024-09-21T00:25:44.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:44.555+0000 7fac417fa640 1 --2- 172.21.15.112:0/1814884278 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fac30078b90 0x7fac3007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:44.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:44.555+0000 7fac417fa640 1 -- 172.21.15.112:0/1814884278 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7fac3409b490 con 0x7fac440b2e60 2024-09-21T00:25:44.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:44.555+0000 7fac43fff640 1 --2- 172.21.15.112:0/1814884278 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fac30078b90 0x7fac3007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:44.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:44.555+0000 7fac43fff640 1 --2- 172.21.15.112:0/1814884278 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fac30078b90 0x7fac3007b0a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fac38034000 tx=0x7fac38004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:25:44.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:44.559+0000 7fac417fa640 1 -- 172.21.15.112:0/1814884278 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fac440c7b80 con 0x7fac440b2e60 2024-09-21T00:25:44.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:44.774+0000 7fac4ac2e640 1 -- 172.21.15.112:0/1814884278 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fac440d2a20 con 0x7fac440b2e60 2024-09-21T00:25:44.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:44.775+0000 7fac417fa640 1 -- 172.21.15.112:0/1814884278 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+3915 (secure 0 0 0) 0x7fac34063c30 con 0x7fac440b2e60 2024-09-21T00:25:44.777 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:25:44.777 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":3,"btime":"2024-09-21T00:25:43:146484+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4316,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6837/2271985854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6835","nonce":2271985854},{"type":"v1","addr":"172.21.15.187:6837","nonce":2271985854}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4413,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/200348130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":200348130},{"type":"v1","addr":"172.21.15.112:6833","nonce":200348130}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4416,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/3821201561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3821201561},{"type":"v1","addr":"172.21.15.112:6835","nonce":3821201561}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4425,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6836/2921304789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":2921304789},{"type":"v1","addr":"172.21.15.187:6836","nonce":2921304789}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-21T00:25:44.777 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 3 2024-09-21T00:25:44.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:44.777+0000 7fac4ac2e640 1 -- 172.21.15.112:0/1814884278 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fac30078b90 msgr2=0x7fac3007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:44.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:44.777+0000 7fac4ac2e640 1 --2- 172.21.15.112:0/1814884278 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fac30078b90 0x7fac3007b0a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fac38034000 tx=0x7fac38004730 comp rx=0 tx=0).stop 2024-09-21T00:25:44.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:44.777+0000 7fac4ac2e640 1 -- 172.21.15.112:0/1814884278 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac440b2e60 msgr2=0x7fac441565b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:44.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:44.777+0000 7fac4ac2e640 1 --2- 172.21.15.112:0/1814884278 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac440b2e60 0x7fac441565b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fac3400e9f0 tx=0x7fac3400eeb0 comp rx=0 tx=0).stop 2024-09-21T00:25:44.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:44.777+0000 7fac4ac2e640 1 -- 172.21.15.112:0/1814884278 shutdown_connections 2024-09-21T00:25:44.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:44.777+0000 7fac4ac2e640 1 --2- 172.21.15.112:0/1814884278 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fac44162590 0x7fac4415ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:44.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:44.777+0000 7fac4ac2e640 1 --2- 172.21.15.112:0/1814884278 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fac440cfe00 0x7fac4415a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:44.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:44.777+0000 7fac4ac2e640 1 --2- 172.21.15.112:0/1814884278 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fac30078b90 0x7fac3007b0a0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:44.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:44.777+0000 7fac4ac2e640 1 --2- 172.21.15.112:0/1814884278 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac440b2e60 0x7fac441565b0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:44.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:44.777+0000 7fac4ac2e640 1 -- 172.21.15.112:0/1814884278 >> 172.21.15.112:0/1814884278 conn(0x7fac440d4440 msgr2=0x7fac44161a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:44.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:44.777+0000 7fac4ac2e640 1 -- 172.21.15.112:0/1814884278 shutdown_connections 2024-09-21T00:25:44.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:44.777+0000 7fac4ac2e640 1 -- 172.21.15.112:0/1814884278 wait complete. 2024-09-21T00:25:44.790 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-21T00:25:44.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:44.900+0000 7fb57d350640 1 -- 172.21.15.112:0/1868596463 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb578162590 msgr2=0x7fb5781629a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:44.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:44.900+0000 7fb57d350640 1 --2- 172.21.15.112:0/1868596463 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb578162590 0x7fb5781629a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb57808d3b0 tx=0x7fb56002a5f0 comp rx=0 tx=0).stop 2024-09-21T00:25:44.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:44.900+0000 7fb57d350640 1 -- 172.21.15.112:0/1868596463 shutdown_connections 2024-09-21T00:25:44.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:44.900+0000 7fb57d350640 1 --2- 172.21.15.112:0/1868596463 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb578162590 0x7fb5781629a0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:44.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:44.900+0000 7fb57d350640 1 --2- 172.21.15.112:0/1868596463 >> v2:172.21.15.187:3300/0 conn(0x7fb5780cfe00 0x7fb5780d2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:44.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:44.900+0000 7fb57d350640 1 -- 172.21.15.112:0/1868596463 >> 172.21.15.112:0/1868596463 conn(0x7fb5780d4440 msgr2=0x7fb5780d4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:44.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:44.900+0000 7fb57d350640 1 -- 172.21.15.112:0/1868596463 shutdown_connections 2024-09-21T00:25:44.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:44.900+0000 7fb57d350640 1 -- 172.21.15.112:0/1868596463 wait complete. 2024-09-21T00:25:44.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:44.902+0000 7fb57d350640 1 Processor -- start 2024-09-21T00:25:44.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:44.902+0000 7fb57d350640 1 -- start start 2024-09-21T00:25:44.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:44.904+0000 7fb57d350640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb5780b2e60 0x7fb5781e9f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:44.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:44.904+0000 7fb57d350640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb5780cfe00 0x7fb57815dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:44.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:44.904+0000 7fb57d350640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb578162590 0x7fb57814aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:44.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:44.904+0000 7fb57d350640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb5780b3980 con 0x7fb5780b2e60 2024-09-21T00:25:44.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:44.904+0000 7fb57d350640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb5780cf5a0 con 0x7fb5780cfe00 2024-09-21T00:25:44.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:44.904+0000 7fb57d350640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fb5780b3b00 con 0x7fb578162590 2024-09-21T00:25:44.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:44.904+0000 7fb576ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb5780b2e60 0x7fb5781e9f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:44.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:44.904+0000 7fb576ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb5780b2e60 0x7fb5781e9f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60610/0 (socket says 172.21.15.112:60610) 2024-09-21T00:25:44.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:44.904+0000 7fb576ffd640 1 -- 172.21.15.112:0/356387677 learned_addr learned my addr 172.21.15.112:0/356387677 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:25:44.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:44.904+0000 7fb5777fe640 1 --2- 172.21.15.112:0/356387677 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb578162590 0x7fb57814aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:44.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:44.904+0000 7fb56e7fc640 1 --2- 172.21.15.112:0/356387677 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb5780cfe00 0x7fb57815dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:44.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:44.904+0000 7fb576ffd640 1 -- 172.21.15.112:0/356387677 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb578162590 msgr2=0x7fb57814aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:44.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:44.904+0000 7fb576ffd640 1 --2- 172.21.15.112:0/356387677 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb578162590 0x7fb57814aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:44.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:44.904+0000 7fb576ffd640 1 -- 172.21.15.112:0/356387677 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb5780cfe00 msgr2=0x7fb57815dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:44.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:44.904+0000 7fb576ffd640 1 --2- 172.21.15.112:0/356387677 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb5780cfe00 0x7fb57815dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:44.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:44.904+0000 7fb576ffd640 1 -- 172.21.15.112:0/356387677 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5781ea4c0 con 0x7fb5780b2e60 2024-09-21T00:25:44.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:44.904+0000 7fb56e7fc640 1 --2- 172.21.15.112:0/356387677 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb5780cfe00 0x7fb57815dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:25:44.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:44.904+0000 7fb576ffd640 1 --2- 172.21.15.112:0/356387677 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb5780b2e60 0x7fb5781e9f80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb56800e7c0 tx=0x7fb56800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:25:44.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:44.904+0000 7fb56ffff640 1 -- 172.21.15.112:0/356387677 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb56800cc60 con 0x7fb5780b2e60 2024-09-21T00:25:44.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:44.904+0000 7fb56ffff640 1 -- 172.21.15.112:0/356387677 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb56800ce20 con 0x7fb5780b2e60 2024-09-21T00:25:44.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:44.904+0000 7fb57d350640 1 -- 172.21.15.112:0/356387677 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5781ea9d0 con 0x7fb5780b2e60 2024-09-21T00:25:44.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:44.904+0000 7fb56ffff640 1 -- 172.21.15.112:0/356387677 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb568005280 con 0x7fb5780b2e60 2024-09-21T00:25:44.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:44.904+0000 7fb57d350640 1 -- 172.21.15.112:0/356387677 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5781eaf90 con 0x7fb5780b2e60 2024-09-21T00:25:44.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:44.905+0000 7fb57d350640 1 -- 172.21.15.112:0/356387677 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5780d2a20 con 0x7fb5780b2e60 2024-09-21T00:25:44.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:44.906+0000 7fb56ffff640 1 -- 172.21.15.112:0/356387677 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb568005420 con 0x7fb5780b2e60 2024-09-21T00:25:44.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:44.906+0000 7fb56ffff640 1 --2- 172.21.15.112:0/356387677 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fb558078b90 0x7fb55807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:44.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:44.907+0000 7fb56ffff640 1 -- 172.21.15.112:0/356387677 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7fb56809b2a0 con 0x7fb5780b2e60 2024-09-21T00:25:44.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:44.907+0000 7fb56e7fc640 1 --2- 172.21.15.112:0/356387677 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fb558078b90 0x7fb55807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:44.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:44.907+0000 7fb56e7fc640 1 --2- 172.21.15.112:0/356387677 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fb558078b90 0x7fb55807b0a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb560033000 tx=0x7fb560004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:25:44.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:44.911+0000 7fb56ffff640 1 -- 172.21.15.112:0/356387677 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb568063a30 con 0x7fb5780b2e60 2024-09-21T00:25:45.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:45.088+0000 7fb57d350640 1 -- 172.21.15.112:0/356387677 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fb5780d4aa0 con 0x7fb5780b2e60 2024-09-21T00:25:45.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:45.089+0000 7fb56ffff640 1 -- 172.21.15.112:0/356387677 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v18) ==== 84+0+65 (secure 0 0 0) 0x7fb568063330 con 0x7fb5780b2e60 2024-09-21T00:25:45.091 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:25:45.091 INFO:teuthology.orchestra.run.smithi112.stdout:[ 2024-09-21T00:25:45.091 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:25:45.091 INFO:teuthology.orchestra.run.smithi112.stdout: "poolnum": 1, 2024-09-21T00:25:45.091 INFO:teuthology.orchestra.run.smithi112.stdout: "poolname": ".mgr" 2024-09-21T00:25:45.091 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:25:45.091 INFO:teuthology.orchestra.run.smithi112.stdout:] 2024-09-21T00:25:45.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:45.091+0000 7fb57d350640 1 -- 172.21.15.112:0/356387677 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fb558078b90 msgr2=0x7fb55807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:45.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:45.091+0000 7fb57d350640 1 --2- 172.21.15.112:0/356387677 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fb558078b90 0x7fb55807b0a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb560033000 tx=0x7fb560004730 comp rx=0 tx=0).stop 2024-09-21T00:25:45.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:45.091+0000 7fb57d350640 1 -- 172.21.15.112:0/356387677 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb5780b2e60 msgr2=0x7fb5781e9f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:45.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:45.091+0000 7fb57d350640 1 --2- 172.21.15.112:0/356387677 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb5780b2e60 0x7fb5781e9f80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb56800e7c0 tx=0x7fb56800ec80 comp rx=0 tx=0).stop 2024-09-21T00:25:45.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:45.091+0000 7fb57d350640 1 -- 172.21.15.112:0/356387677 shutdown_connections 2024-09-21T00:25:45.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:45.091+0000 7fb57d350640 1 --2- 172.21.15.112:0/356387677 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb578162590 0x7fb57814aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:45.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:45.091+0000 7fb57d350640 1 --2- 172.21.15.112:0/356387677 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb5780cfe00 0x7fb57815dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:45.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:45.091+0000 7fb57d350640 1 --2- 172.21.15.112:0/356387677 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fb558078b90 0x7fb55807b0a0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:45.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:45.091+0000 7fb57d350640 1 --2- 172.21.15.112:0/356387677 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb5780b2e60 0x7fb5781e9f80 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:45.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:45.091+0000 7fb57d350640 1 -- 172.21.15.112:0/356387677 >> 172.21.15.112:0/356387677 conn(0x7fb5780d4440 msgr2=0x7fb5780ce7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:45.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:45.091+0000 7fb57d350640 1 -- 172.21.15.112:0/356387677 shutdown_connections 2024-09-21T00:25:45.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:45.091+0000 7fb57d350640 1 -- 172.21.15.112:0/356387677 wait complete. 2024-09-21T00:25:45.103 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-21T00:25:45.103 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-21T00:25:45.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:45.219+0000 7f7d1ef50640 1 -- 172.21.15.112:0/3632229379 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d1815c5e0 msgr2=0x7f7d18160870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:45.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:45.219+0000 7f7d1ef50640 1 --2- 172.21.15.112:0/3632229379 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d1815c5e0 0x7f7d18160870 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c005fd0 tx=0x7f7d0c02a540 comp rx=0 tx=0).stop 2024-09-21T00:25:45.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:45.219+0000 7f7d1ef50640 1 -- 172.21.15.112:0/3632229379 shutdown_connections 2024-09-21T00:25:45.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:45.219+0000 7f7d1ef50640 1 --2- 172.21.15.112:0/3632229379 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d1815c5e0 0x7f7d18160870 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:45.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:45.219+0000 7f7d1ef50640 1 -- 172.21.15.112:0/3632229379 >> 172.21.15.112:0/3632229379 conn(0x7f7d180d2280 msgr2=0x7f7d180d2690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:45.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:45.219+0000 7f7d1ef50640 1 -- 172.21.15.112:0/3632229379 shutdown_connections 2024-09-21T00:25:45.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:45.219+0000 7f7d1ef50640 1 -- 172.21.15.112:0/3632229379 wait complete. 2024-09-21T00:25:45.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:45.221+0000 7f7d1ef50640 1 Processor -- start 2024-09-21T00:25:45.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:45.221+0000 7f7d1ef50640 1 -- start start 2024-09-21T00:25:45.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:45.222+0000 7f7d1ef50640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7d180b2480 0x7f7d18150e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:45.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:45.222+0000 7f7d1ef50640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7d180cfe50 0x7f7d1815f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:45.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:45.222+0000 7f7d1ef50640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d1815c5e0 0x7f7d180b69d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:45.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:45.222+0000 7f7d1ef50640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7d180b3120 con 0x7f7d1815c5e0 2024-09-21T00:25:45.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:45.222+0000 7f7d1ef50640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7d180b2fa0 con 0x7f7d180b2480 2024-09-21T00:25:45.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:45.222+0000 7f7d1ef50640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f7d180b32a0 con 0x7f7d180cfe50 2024-09-21T00:25:45.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:45.222+0000 7f7d1d4ed640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d1815c5e0 0x7f7d180b69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:45.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:45.222+0000 7f7d1d4ed640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d1815c5e0 0x7f7d180b69d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60624/0 (socket says 172.21.15.112:60624) 2024-09-21T00:25:45.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:45.222+0000 7f7d1d4ed640 1 -- 172.21.15.112:0/393288380 learned_addr learned my addr 172.21.15.112:0/393288380 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:25:45.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:45.222+0000 7f7d17fff640 1 --2- 172.21.15.112:0/393288380 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7d180cfe50 0x7f7d1815f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:45.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:45.223+0000 7f7d1ccec640 1 --2- 172.21.15.112:0/393288380 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7d180b2480 0x7f7d18150e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:45.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:45.223+0000 7f7d1d4ed640 1 -- 172.21.15.112:0/393288380 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7d180cfe50 msgr2=0x7f7d1815f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:45.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:45.223+0000 7f7d1d4ed640 1 --2- 172.21.15.112:0/393288380 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7d180cfe50 0x7f7d1815f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:45.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:45.223+0000 7f7d1d4ed640 1 -- 172.21.15.112:0/393288380 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7d180b2480 msgr2=0x7f7d18150e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:45.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:45.223+0000 7f7d1d4ed640 1 --2- 172.21.15.112:0/393288380 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7d180b2480 0x7f7d18150e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:45.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:45.223+0000 7f7d1d4ed640 1 -- 172.21.15.112:0/393288380 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d181f3760 con 0x7f7d1815c5e0 2024-09-21T00:25:45.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:45.223+0000 7f7d1d4ed640 1 --2- 172.21.15.112:0/393288380 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d1815c5e0 0x7f7d180b69d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7d1000c760 tx=0x7f7d1000cb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:25:45.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:45.223+0000 7f7d157fa640 1 -- 172.21.15.112:0/393288380 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d10003b70 con 0x7f7d1815c5e0 2024-09-21T00:25:45.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:45.223+0000 7f7d157fa640 1 -- 172.21.15.112:0/393288380 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7d10003d10 con 0x7f7d1815c5e0 2024-09-21T00:25:45.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:45.223+0000 7f7d157fa640 1 -- 172.21.15.112:0/393288380 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d100052d0 con 0x7f7d1815c5e0 2024-09-21T00:25:45.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:45.223+0000 7f7d1ef50640 1 -- 172.21.15.112:0/393288380 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d181f3a40 con 0x7f7d1815c5e0 2024-09-21T00:25:45.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:45.223+0000 7f7d1ef50640 1 -- 172.21.15.112:0/393288380 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d181f4000 con 0x7f7d1815c5e0 2024-09-21T00:25:45.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:45.224+0000 7f7d1ef50640 1 -- 172.21.15.112:0/393288380 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d180b2890 con 0x7f7d1815c5e0 2024-09-21T00:25:45.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:45.225+0000 7f7d157fa640 1 -- 172.21.15.112:0/393288380 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f7d10005470 con 0x7f7d1815c5e0 2024-09-21T00:25:45.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:45.226+0000 7f7d157fa640 1 --2- 172.21.15.112:0/393288380 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f7d04078b90 0x7f7d0407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:45.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:45.226+0000 7f7d157fa640 1 -- 172.21.15.112:0/393288380 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7f7d1009a5f0 con 0x7f7d1815c5e0 2024-09-21T00:25:45.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:45.226+0000 7f7d1ccec640 1 --2- 172.21.15.112:0/393288380 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f7d04078b90 0x7f7d0407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:45.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:45.226+0000 7f7d1ccec640 1 --2- 172.21.15.112:0/393288380 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f7d04078b90 0x7f7d0407b0a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f7d08006fd0 tx=0x7f7d08009060 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:25:45.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:45.230+0000 7f7d157fa640 1 -- 172.21.15.112:0/393288380 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f7d100627c0 con 0x7f7d1815c5e0 2024-09-21T00:25:45.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:45.409+0000 7f7d1ef50640 1 -- 172.21.15.112:0/393288380 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f7d180d1610 con 0x7f7d1815c5e0 2024-09-21T00:25:46.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:46.199+0000 7f7d157fa640 1 -- 172.21.15.112:0/393288380 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v19) ==== 138+0+0 (secure 0 0 0) 0x7f7d100671f0 con 0x7f7d1815c5e0 2024-09-21T00:25:46.201 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'cephfs_metadata' created 2024-09-21T00:25:46.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:46.201+0000 7f7d1ef50640 1 -- 172.21.15.112:0/393288380 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f7d04078b90 msgr2=0x7f7d0407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:46.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:46.201+0000 7f7d1ef50640 1 --2- 172.21.15.112:0/393288380 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f7d04078b90 0x7f7d0407b0a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f7d08006fd0 tx=0x7f7d08009060 comp rx=0 tx=0).stop 2024-09-21T00:25:46.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:46.201+0000 7f7d1ef50640 1 -- 172.21.15.112:0/393288380 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d1815c5e0 msgr2=0x7f7d180b69d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:46.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:46.201+0000 7f7d1ef50640 1 --2- 172.21.15.112:0/393288380 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d1815c5e0 0x7f7d180b69d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7d1000c760 tx=0x7f7d1000cb60 comp rx=0 tx=0).stop 2024-09-21T00:25:46.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:46.202+0000 7f7d1ef50640 1 -- 172.21.15.112:0/393288380 shutdown_connections 2024-09-21T00:25:46.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:46.202+0000 7f7d1ef50640 1 --2- 172.21.15.112:0/393288380 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f7d04078b90 0x7f7d0407b0a0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:46.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:46.202+0000 7f7d1ef50640 1 --2- 172.21.15.112:0/393288380 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d1815c5e0 0x7f7d180b69d0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:46.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:46.202+0000 7f7d1ef50640 1 --2- 172.21.15.112:0/393288380 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7d180cfe50 0x7f7d1815f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:46.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:46.202+0000 7f7d1ef50640 1 --2- 172.21.15.112:0/393288380 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7d180b2480 0x7f7d18150e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:46.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:46.202+0000 7f7d1ef50640 1 -- 172.21.15.112:0/393288380 >> 172.21.15.112:0/393288380 conn(0x7f7d180d2280 msgr2=0x7f7d18070970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:46.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:46.202+0000 7f7d1ef50640 1 -- 172.21.15.112:0/393288380 shutdown_connections 2024-09-21T00:25:46.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:46.202+0000 7f7d1ef50640 1 -- 172.21.15.112:0/393288380 wait complete. 2024-09-21T00:25:46.218 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-09-21T00:25:46.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:46.337+0000 7f1c410bd640 1 -- 172.21.15.112:0/1618700669 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c3c0d19f0 msgr2=0x7f1c3c15cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:46.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:46.337+0000 7f1c410bd640 1 --2- 172.21.15.112:0/1618700669 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c3c0d19f0 0x7f1c3c15cf20 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1c24005fd0 tx=0x7f1c2402a5d0 comp rx=0 tx=0).stop 2024-09-21T00:25:46.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:46.337+0000 7f1c410bd640 1 -- 172.21.15.112:0/1618700669 shutdown_connections 2024-09-21T00:25:46.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:46.337+0000 7f1c410bd640 1 --2- 172.21.15.112:0/1618700669 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c3c0d19f0 0x7f1c3c15cf20 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:46.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:46.337+0000 7f1c410bd640 1 -- 172.21.15.112:0/1618700669 >> 172.21.15.112:0/1618700669 conn(0x7f1c3c0d2350 msgr2=0x7f1c3c0d2760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:46.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:46.337+0000 7f1c410bd640 1 -- 172.21.15.112:0/1618700669 shutdown_connections 2024-09-21T00:25:46.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:46.337+0000 7f1c410bd640 1 -- 172.21.15.112:0/1618700669 wait complete. 2024-09-21T00:25:46.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:46.339+0000 7f1c410bd640 1 Processor -- start 2024-09-21T00:25:46.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:46.339+0000 7f1c410bd640 1 -- start start 2024-09-21T00:25:46.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:46.341+0000 7f1c410bd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1c3c0b6250 0x7f1c3c0c74f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:46.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:46.341+0000 7f1c410bd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c3c0d19f0 0x7f1c3c151de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:46.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:46.341+0000 7f1c410bd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1c3c15cb10 0x7f1c3c0cda50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:46.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:46.341+0000 7f1c410bd640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1c3c0b7100 con 0x7f1c3c0d19f0 2024-09-21T00:25:46.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:46.341+0000 7f1c410bd640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f1c3c0b6f80 con 0x7f1c3c0b6250 2024-09-21T00:25:46.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:46.341+0000 7f1c410bd640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f1c3c0b7280 con 0x7f1c3c15cb10 2024-09-21T00:25:46.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:46.341+0000 7f1c3259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c3c0d19f0 0x7f1c3c151de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:46.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:46.341+0000 7f1c3259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c3c0d19f0 0x7f1c3c151de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59034/0 (socket says 172.21.15.112:59034) 2024-09-21T00:25:46.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:46.341+0000 7f1c3259c640 1 -- 172.21.15.112:0/2583382029 learned_addr learned my addr 172.21.15.112:0/2583382029 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:25:46.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:46.341+0000 7f1c3ad9d640 1 --2- 172.21.15.112:0/2583382029 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1c3c0b6250 0x7f1c3c0c74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:46.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:46.341+0000 7f1c3b59e640 1 --2- 172.21.15.112:0/2583382029 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1c3c15cb10 0x7f1c3c0cda50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:46.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:46.341+0000 7f1c3259c640 1 -- 172.21.15.112:0/2583382029 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1c3c15cb10 msgr2=0x7f1c3c0cda50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:46.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:46.341+0000 7f1c3259c640 1 --2- 172.21.15.112:0/2583382029 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1c3c15cb10 0x7f1c3c0cda50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:46.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:46.341+0000 7f1c3259c640 1 -- 172.21.15.112:0/2583382029 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1c3c0b6250 msgr2=0x7f1c3c0c74f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:46.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:46.341+0000 7f1c3259c640 1 --2- 172.21.15.112:0/2583382029 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1c3c0b6250 0x7f1c3c0c74f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:46.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:46.341+0000 7f1c3259c640 1 -- 172.21.15.112:0/2583382029 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c3c0c7bb0 con 0x7f1c3c0d19f0 2024-09-21T00:25:46.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:46.341+0000 7f1c3b59e640 1 --2- 172.21.15.112:0/2583382029 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1c3c15cb10 0x7f1c3c0cda50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:25:46.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:46.341+0000 7f1c3ad9d640 1 --2- 172.21.15.112:0/2583382029 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1c3c0b6250 0x7f1c3c0c74f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:25:46.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:46.341+0000 7f1c3259c640 1 --2- 172.21.15.112:0/2583382029 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c3c0d19f0 0x7f1c3c151de0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1c24006010 tx=0x7f1c240038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:25:46.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:46.341+0000 7f1c33fff640 1 -- 172.21.15.112:0/2583382029 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c2403e030 con 0x7f1c3c0d19f0 2024-09-21T00:25:46.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:46.341+0000 7f1c33fff640 1 -- 172.21.15.112:0/2583382029 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1c24003ca0 con 0x7f1c3c0d19f0 2024-09-21T00:25:46.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:46.341+0000 7f1c410bd640 1 -- 172.21.15.112:0/2583382029 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c3c0c7e90 con 0x7f1c3c0d19f0 2024-09-21T00:25:46.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:46.341+0000 7f1c33fff640 1 -- 172.21.15.112:0/2583382029 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c24035040 con 0x7f1c3c0d19f0 2024-09-21T00:25:46.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:46.341+0000 7f1c410bd640 1 -- 172.21.15.112:0/2583382029 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c3c0c8450 con 0x7f1c3c0d19f0 2024-09-21T00:25:46.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:46.342+0000 7f1c410bd640 1 -- 172.21.15.112:0/2583382029 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c3c0c9d60 con 0x7f1c3c0d19f0 2024-09-21T00:25:46.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:46.343+0000 7f1c33fff640 1 -- 172.21.15.112:0/2583382029 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1c24041030 con 0x7f1c3c0d19f0 2024-09-21T00:25:46.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:46.343+0000 7f1c33fff640 1 --2- 172.21.15.112:0/2583382029 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1c1c078bc0 0x7f1c1c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:46.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:46.344+0000 7f1c33fff640 1 -- 172.21.15.112:0/2583382029 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5227+0+0 (secure 0 0 0) 0x7f1c240bb120 con 0x7f1c3c0d19f0 2024-09-21T00:25:46.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:46.344+0000 7f1c3ad9d640 1 --2- 172.21.15.112:0/2583382029 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1c1c078bc0 0x7f1c1c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:46.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:46.344+0000 7f1c3ad9d640 1 --2- 172.21.15.112:0/2583382029 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1c1c078bc0 0x7f1c1c07b0d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1c2c000f60 tx=0x7f1c2c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:25:46.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:46.348+0000 7f1c33fff640 1 -- 172.21.15.112:0/2583382029 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1c3c0c9d60 con 0x7f1c3c0d19f0 2024-09-21T00:25:46.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:46.527+0000 7f1c410bd640 1 -- 172.21.15.112:0/2583382029 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7f1c3c15cf20 con 0x7f1c3c0d19f0 2024-09-21T00:25:47.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:47.207+0000 7f1c33fff640 1 -- 172.21.15.112:0/2583382029 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v20) ==== 171+0+0 (secure 0 0 0) 0x7f1c240837d0 con 0x7f1c3c0d19f0 2024-09-21T00:25:47.209 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'cephfs_data' created 2024-09-21T00:25:47.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:47.209+0000 7f1c410bd640 1 -- 172.21.15.112:0/2583382029 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1c1c078bc0 msgr2=0x7f1c1c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:47.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:47.209+0000 7f1c410bd640 1 --2- 172.21.15.112:0/2583382029 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1c1c078bc0 0x7f1c1c07b0d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1c2c000f60 tx=0x7f1c2c009040 comp rx=0 tx=0).stop 2024-09-21T00:25:47.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:47.209+0000 7f1c410bd640 1 -- 172.21.15.112:0/2583382029 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c3c0d19f0 msgr2=0x7f1c3c151de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:47.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:47.209+0000 7f1c410bd640 1 --2- 172.21.15.112:0/2583382029 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c3c0d19f0 0x7f1c3c151de0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1c24006010 tx=0x7f1c240038a0 comp rx=0 tx=0).stop 2024-09-21T00:25:47.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:47.209+0000 7f1c410bd640 1 -- 172.21.15.112:0/2583382029 shutdown_connections 2024-09-21T00:25:47.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:47.209+0000 7f1c410bd640 1 --2- 172.21.15.112:0/2583382029 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1c3c15cb10 0x7f1c3c0cda50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:47.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:47.209+0000 7f1c410bd640 1 --2- 172.21.15.112:0/2583382029 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1c1c078bc0 0x7f1c1c07b0d0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:47.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:47.209+0000 7f1c410bd640 1 --2- 172.21.15.112:0/2583382029 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c3c0d19f0 0x7f1c3c151de0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:47.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:47.209+0000 7f1c410bd640 1 --2- 172.21.15.112:0/2583382029 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1c3c0b6250 0x7f1c3c0c74f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:47.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:47.209+0000 7f1c410bd640 1 -- 172.21.15.112:0/2583382029 >> 172.21.15.112:0/2583382029 conn(0x7f1c3c0d2350 msgr2=0x7f1c3c160090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:47.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:47.209+0000 7f1c410bd640 1 -- 172.21.15.112:0/2583382029 shutdown_connections 2024-09-21T00:25:47.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:47.209+0000 7f1c410bd640 1 -- 172.21.15.112:0/2583382029 wait complete. 2024-09-21T00:25:47.234 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-21T00:25:47.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:47.350+0000 7f1b88dc6640 1 -- 172.21.15.112:0/1265315842 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b6c0039e0 msgr2=0x7f1b6c003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:47.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:47.350+0000 7f1b88dc6640 1 --2- 172.21.15.112:0/1265315842 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b6c0039e0 0x7f1b6c003e10 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1b74009b50 tx=0x7f1b6c005830 comp rx=0 tx=0).stop 2024-09-21T00:25:47.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:47.350+0000 7f1b82d9d640 1 -- 172.21.15.112:0/1265315842 reap_dead start 2024-09-21T00:25:47.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:47.350+0000 7f1b88dc6640 1 -- 172.21.15.112:0/1265315842 shutdown_connections 2024-09-21T00:25:47.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:47.350+0000 7f1b88dc6640 1 -- 172.21.15.112:0/1265315842 >> 172.21.15.112:0/1265315842 conn(0x7f1b8409d830 msgr2=0x7f1b840b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:47.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:47.350+0000 7f1b88dc6640 1 -- 172.21.15.112:0/1265315842 shutdown_connections 2024-09-21T00:25:47.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:47.350+0000 7f1b88dc6640 1 -- 172.21.15.112:0/1265315842 wait complete. 2024-09-21T00:25:47.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:47.352+0000 7f1b88dc6640 1 Processor -- start 2024-09-21T00:25:47.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:47.352+0000 7f1b88dc6640 1 -- start start 2024-09-21T00:25:47.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:47.354+0000 7f1b88dc6640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1b6c0039e0 0x7f1b841f64a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:47.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:47.354+0000 7f1b88dc6640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1b84157350 0x7f1b84157760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:47.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:47.354+0000 7f1b88dc6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b840b2620 0x7f1b840b2a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:47.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:47.354+0000 7f1b88dc6640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1b840d0500 con 0x7f1b840b2620 2024-09-21T00:25:47.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:47.354+0000 7f1b88dc6640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f1b840d0380 con 0x7f1b84157350 2024-09-21T00:25:47.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:47.354+0000 7f1b88dc6640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f1b840b6580 con 0x7f1b6c0039e0 2024-09-21T00:25:47.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:47.354+0000 7f1b82d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b840b2620 0x7f1b840b2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:47.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:47.354+0000 7f1b82d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b840b2620 0x7f1b840b2a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59056/0 (socket says 172.21.15.112:59056) 2024-09-21T00:25:47.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:47.354+0000 7f1b82d9d640 1 -- 172.21.15.112:0/3543392200 learned_addr learned my addr 172.21.15.112:0/3543392200 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:25:47.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:47.354+0000 7f1b8259c640 1 --2- 172.21.15.112:0/3543392200 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1b6c0039e0 0x7f1b841f64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:47.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:47.354+0000 7f1b81d9b640 1 --2- 172.21.15.112:0/3543392200 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1b84157350 0x7f1b84157760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:47.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:47.354+0000 7f1b82d9d640 1 -- 172.21.15.112:0/3543392200 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1b6c0039e0 msgr2=0x7f1b841f64a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:47.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:47.354+0000 7f1b82d9d640 1 --2- 172.21.15.112:0/3543392200 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1b6c0039e0 0x7f1b841f64a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:47.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:47.354+0000 7f1b82d9d640 1 -- 172.21.15.112:0/3543392200 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1b84157350 msgr2=0x7f1b84157760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:47.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:47.354+0000 7f1b82d9d640 1 --2- 172.21.15.112:0/3543392200 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1b84157350 0x7f1b84157760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:47.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:47.354+0000 7f1b82d9d640 1 -- 172.21.15.112:0/3543392200 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b840b2f70 con 0x7f1b840b2620 2024-09-21T00:25:47.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:47.354+0000 7f1b82d9d640 1 --2- 172.21.15.112:0/3543392200 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b840b2620 0x7f1b840b2a30 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c004780 tx=0x7f1b7c001a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:25:47.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:47.354+0000 7f1b72ffd640 1 -- 172.21.15.112:0/3543392200 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b7c011870 con 0x7f1b840b2620 2024-09-21T00:25:47.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:47.354+0000 7f1b72ffd640 1 -- 172.21.15.112:0/3543392200 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1b7c017070 con 0x7f1b840b2620 2024-09-21T00:25:47.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:47.354+0000 7f1b88dc6640 1 -- 172.21.15.112:0/3543392200 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b8406eff0 con 0x7f1b840b2620 2024-09-21T00:25:47.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:47.354+0000 7f1b72ffd640 1 -- 172.21.15.112:0/3543392200 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b7c011da0 con 0x7f1b840b2620 2024-09-21T00:25:47.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:47.354+0000 7f1b88dc6640 1 -- 172.21.15.112:0/3543392200 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b840cb130 con 0x7f1b840b2620 2024-09-21T00:25:47.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:47.356+0000 7f1b88dc6640 1 -- 172.21.15.112:0/3543392200 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b8406acf0 con 0x7f1b840b2620 2024-09-21T00:25:47.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:47.356+0000 7f1b72ffd640 1 -- 172.21.15.112:0/3543392200 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1b7c01c460 con 0x7f1b840b2620 2024-09-21T00:25:47.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:47.357+0000 7f1b72ffd640 1 --2- 172.21.15.112:0/3543392200 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1b64078b90 0x7f1b6407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:47.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:47.357+0000 7f1b72ffd640 1 -- 172.21.15.112:0/3543392200 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5609+0+0 (secure 0 0 0) 0x7f1b7c09b460 con 0x7f1b840b2620 2024-09-21T00:25:47.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:47.357+0000 7f1b8259c640 1 --2- 172.21.15.112:0/3543392200 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1b64078b90 0x7f1b6407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:47.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:47.358+0000 7f1b8259c640 1 --2- 172.21.15.112:0/3543392200 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1b64078b90 0x7f1b6407b0a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1b74035000 tx=0x7f1b7402d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:25:47.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:47.362+0000 7f1b72ffd640 1 -- 172.21.15.112:0/3543392200 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1b8406acf0 con 0x7f1b840b2620 2024-09-21T00:25:47.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:47.577+0000 7f1b88dc6640 1 -- 172.21.15.112:0/3543392200 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f1b8406ed80 con 0x7f1b840b2620 2024-09-21T00:25:48.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:48.208+0000 7f1b72ffd640 1 -- 172.21.15.112:0/3543392200 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7f1b840b2f70 con 0x7f1b840b2620 2024-09-21T00:25:48.210 INFO:teuthology.orchestra.run.smithi112.stderr:new fs with metadata pool 2 and data pool 3 2024-09-21T00:25:48.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:48.211+0000 7f1b88dc6640 1 -- 172.21.15.112:0/3543392200 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1b64078b90 msgr2=0x7f1b6407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:48.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:48.211+0000 7f1b88dc6640 1 --2- 172.21.15.112:0/3543392200 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1b64078b90 0x7f1b6407b0a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1b74035000 tx=0x7f1b7402d040 comp rx=0 tx=0).stop 2024-09-21T00:25:48.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:48.211+0000 7f1b88dc6640 1 -- 172.21.15.112:0/3543392200 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b840b2620 msgr2=0x7f1b840b2a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:48.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:48.211+0000 7f1b88dc6640 1 --2- 172.21.15.112:0/3543392200 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b840b2620 0x7f1b840b2a30 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c004780 tx=0x7f1b7c001a90 comp rx=0 tx=0).stop 2024-09-21T00:25:48.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:48.211+0000 7f1b88dc6640 1 -- 172.21.15.112:0/3543392200 shutdown_connections 2024-09-21T00:25:48.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:48.211+0000 7f1b88dc6640 1 --2- 172.21.15.112:0/3543392200 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1b64078b90 0x7f1b6407b0a0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:48.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:48.211+0000 7f1b88dc6640 1 --2- 172.21.15.112:0/3543392200 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b840b2620 0x7f1b840b2a30 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:48.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:48.211+0000 7f1b88dc6640 1 --2- 172.21.15.112:0/3543392200 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1b84157350 0x7f1b84157760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:48.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:48.211+0000 7f1b88dc6640 1 --2- 172.21.15.112:0/3543392200 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1b6c0039e0 0x7f1b841f64a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:48.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:48.211+0000 7f1b88dc6640 1 -- 172.21.15.112:0/3543392200 >> 172.21.15.112:0/3543392200 conn(0x7f1b8409d830 msgr2=0x7f1b84160aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:48.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:48.211+0000 7f1b88dc6640 1 -- 172.21.15.112:0/3543392200 shutdown_connections 2024-09-21T00:25:48.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:48.211+0000 7f1b88dc6640 1 -- 172.21.15.112:0/3543392200 wait complete. 2024-09-21T00:25:48.235 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-09-21T00:25:48.235 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-09-21T00:25:48.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:48.360+0000 7f0e5b2b6640 1 -- 172.21.15.112:0/1446297947 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e540cf740 msgr2=0x7f0e54162600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:48.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:48.360+0000 7f0e5b2b6640 1 --2- 172.21.15.112:0/1446297947 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e540cf740 0x7f0e54162600 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f0e4c005fd0 tx=0x7f0e4c02aa90 comp rx=0 tx=0).stop 2024-09-21T00:25:48.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:48.360+0000 7f0e5b2b6640 1 -- 172.21.15.112:0/1446297947 shutdown_connections 2024-09-21T00:25:48.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:48.360+0000 7f0e5b2b6640 1 --2- 172.21.15.112:0/1446297947 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e540cf740 0x7f0e54162600 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:48.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:48.360+0000 7f0e5b2b6640 1 --2- 172.21.15.112:0/1446297947 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0e54161ed0 0x7f0e540cf200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:48.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:48.360+0000 7f0e5b2b6640 1 -- 172.21.15.112:0/1446297947 >> 172.21.15.112:0/1446297947 conn(0x7f0e540d4510 msgr2=0x7f0e540d4920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:48.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:48.360+0000 7f0e5b2b6640 1 -- 172.21.15.112:0/1446297947 shutdown_connections 2024-09-21T00:25:48.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:48.360+0000 7f0e5b2b6640 1 -- 172.21.15.112:0/1446297947 wait complete. 2024-09-21T00:25:48.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:48.362+0000 7f0e5b2b6640 1 Processor -- start 2024-09-21T00:25:48.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:48.362+0000 7f0e5b2b6640 1 -- start start 2024-09-21T00:25:48.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:48.363+0000 7f0e5b2b6640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0e540cf740 0x7f0e54158220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:48.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:48.363+0000 7f0e5b2b6640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0e540d0460 0x7f0e5415b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:48.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:48.363+0000 7f0e5b2b6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e54161ed0 0x7f0e540cc240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:48.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:48.363+0000 7f0e5b2b6640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0e54162b40 con 0x7f0e54161ed0 2024-09-21T00:25:48.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:48.363+0000 7f0e5b2b6640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0e540b8270 con 0x7f0e540cf740 2024-09-21T00:25:48.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:48.363+0000 7f0e5b2b6640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f0e54162cc0 con 0x7f0e540d0460 2024-09-21T00:25:48.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:48.364+0000 7f0e59853640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e54161ed0 0x7f0e540cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:48.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:48.364+0000 7f0e59853640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e54161ed0 0x7f0e540cc240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59064/0 (socket says 172.21.15.112:59064) 2024-09-21T00:25:48.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:48.364+0000 7f0e59853640 1 -- 172.21.15.112:0/3649257963 learned_addr learned my addr 172.21.15.112:0/3649257963 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:25:48.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:48.364+0000 7f0e58851640 1 --2- 172.21.15.112:0/3649257963 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0e540d0460 0x7f0e5415b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:48.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:48.364+0000 7f0e59052640 1 --2- 172.21.15.112:0/3649257963 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0e540cf740 0x7f0e54158220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:48.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:48.364+0000 7f0e59853640 1 -- 172.21.15.112:0/3649257963 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0e540d0460 msgr2=0x7f0e5415b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:48.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:48.364+0000 7f0e59853640 1 --2- 172.21.15.112:0/3649257963 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0e540d0460 0x7f0e5415b130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:48.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:48.364+0000 7f0e59853640 1 -- 172.21.15.112:0/3649257963 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0e540cf740 msgr2=0x7f0e54158220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:48.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:48.364+0000 7f0e59853640 1 --2- 172.21.15.112:0/3649257963 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0e540cf740 0x7f0e54158220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:48.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:48.364+0000 7f0e59853640 1 -- 172.21.15.112:0/3649257963 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e54158a70 con 0x7f0e54161ed0 2024-09-21T00:25:48.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:48.364+0000 7f0e59853640 1 --2- 172.21.15.112:0/3649257963 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e54161ed0 0x7f0e540cc240 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0e4c009d90 tx=0x7f0e4c003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:25:48.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:48.364+0000 7f0e49ffb640 1 -- 172.21.15.112:0/3649257963 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e4c044070 con 0x7f0e54161ed0 2024-09-21T00:25:48.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:48.364+0000 7f0e49ffb640 1 -- 172.21.15.112:0/3649257963 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0e4c004250 con 0x7f0e54161ed0 2024-09-21T00:25:48.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:48.364+0000 7f0e5b2b6640 1 -- 172.21.15.112:0/3649257963 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e540cb760 con 0x7f0e54161ed0 2024-09-21T00:25:48.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:48.364+0000 7f0e49ffb640 1 -- 172.21.15.112:0/3649257963 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e4c004660 con 0x7f0e54161ed0 2024-09-21T00:25:48.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:48.364+0000 7f0e5b2b6640 1 -- 172.21.15.112:0/3649257963 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e540cbd20 con 0x7f0e54161ed0 2024-09-21T00:25:48.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:48.365+0000 7f0e5b2b6640 1 -- 172.21.15.112:0/3649257963 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e541587e0 con 0x7f0e54161ed0 2024-09-21T00:25:48.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:48.366+0000 7f0e49ffb640 1 -- 172.21.15.112:0/3649257963 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0e4c047070 con 0x7f0e54161ed0 2024-09-21T00:25:48.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:48.366+0000 7f0e49ffb640 1 --2- 172.21.15.112:0/3649257963 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f0e38078b90 0x7f0e3807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:48.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:48.367+0000 7f0e49ffb640 1 -- 172.21.15.112:0/3649257963 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f0e4c0bbd30 con 0x7f0e54161ed0 2024-09-21T00:25:48.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:48.367+0000 7f0e59052640 1 --2- 172.21.15.112:0/3649257963 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f0e38078b90 0x7f0e3807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:48.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:48.367+0000 7f0e59052640 1 --2- 172.21.15.112:0/3649257963 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f0e38078b90 0x7f0e3807b0a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0e500015a0 tx=0x7f0e50009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:25:48.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:48.371+0000 7f0e49ffb640 1 -- 172.21.15.112:0/3649257963 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f0e4c032030 con 0x7f0e54161ed0 2024-09-21T00:25:48.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:48.556+0000 7f0e5b2b6640 1 -- 172.21.15.112:0/3649257963 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7f0e54162600 con 0x7f0e54161ed0 2024-09-21T00:25:49.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:49.210+0000 7f0e49ffb640 1 -- 172.21.15.112:0/3649257963 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v22) ==== 157+0+0 (secure 0 0 0) 0x7f0e4c084170 con 0x7f0e54161ed0 2024-09-21T00:25:49.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:49.214+0000 7f0e5b2b6640 1 -- 172.21.15.112:0/3649257963 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f0e38078b90 msgr2=0x7f0e3807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:49.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:49.214+0000 7f0e5b2b6640 1 --2- 172.21.15.112:0/3649257963 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f0e38078b90 0x7f0e3807b0a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0e500015a0 tx=0x7f0e50009040 comp rx=0 tx=0).stop 2024-09-21T00:25:49.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:49.214+0000 7f0e5b2b6640 1 -- 172.21.15.112:0/3649257963 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e54161ed0 msgr2=0x7f0e540cc240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:49.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:49.214+0000 7f0e5b2b6640 1 --2- 172.21.15.112:0/3649257963 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e54161ed0 0x7f0e540cc240 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0e4c009d90 tx=0x7f0e4c003ce0 comp rx=0 tx=0).stop 2024-09-21T00:25:49.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:49.214+0000 7f0e5b2b6640 1 -- 172.21.15.112:0/3649257963 shutdown_connections 2024-09-21T00:25:49.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:49.214+0000 7f0e5b2b6640 1 --2- 172.21.15.112:0/3649257963 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f0e38078b90 0x7f0e3807b0a0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:49.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:49.214+0000 7f0e5b2b6640 1 --2- 172.21.15.112:0/3649257963 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e54161ed0 0x7f0e540cc240 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:49.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:49.214+0000 7f0e5b2b6640 1 --2- 172.21.15.112:0/3649257963 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0e540d0460 0x7f0e5415b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:49.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:49.214+0000 7f0e5b2b6640 1 --2- 172.21.15.112:0/3649257963 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0e540cf740 0x7f0e54158220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:49.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:49.214+0000 7f0e5b2b6640 1 -- 172.21.15.112:0/3649257963 >> 172.21.15.112:0/3649257963 conn(0x7f0e540d4510 msgr2=0x7f0e5415f550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:49.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:49.214+0000 7f0e5b2b6640 1 -- 172.21.15.112:0/3649257963 shutdown_connections 2024-09-21T00:25:49.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:49.214+0000 7f0e5b2b6640 1 -- 172.21.15.112:0/3649257963 wait complete. 2024-09-21T00:25:49.230 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-09-21T00:25:49.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:49.354+0000 7f75d3415640 1 -- 172.21.15.112:0/2348303592 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75cc0b33e0 msgr2=0x7f75cc162d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:49.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:49.354+0000 7f75d3415640 1 --2- 172.21.15.112:0/2348303592 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75cc0b33e0 0x7f75cc162d90 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f75cc0cf5f0 tx=0x7f75c802a560 comp rx=0 tx=0).stop 2024-09-21T00:25:49.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:49.354+0000 7f75d3415640 1 -- 172.21.15.112:0/2348303592 shutdown_connections 2024-09-21T00:25:49.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:49.354+0000 7f75d3415640 1 --2- 172.21.15.112:0/2348303592 >> v2:172.21.15.187:3300/0 conn(0x7f75cc15e690 0x7f75cc0d0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:49.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:49.354+0000 7f75d3415640 1 --2- 172.21.15.112:0/2348303592 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f75cc0d1460 0x7f75cc0b5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:49.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:49.354+0000 7f75d3415640 1 --2- 172.21.15.112:0/2348303592 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75cc0b33e0 0x7f75cc162d90 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:49.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:49.354+0000 7f75d3415640 1 -- 172.21.15.112:0/2348303592 >> 172.21.15.112:0/2348303592 conn(0x7f75cc06cde0 msgr2=0x7f75cc06d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:49.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:49.355+0000 7f75d3415640 1 -- 172.21.15.112:0/2348303592 shutdown_connections 2024-09-21T00:25:49.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:49.355+0000 7f75d3415640 1 -- 172.21.15.112:0/2348303592 wait complete. 2024-09-21T00:25:49.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:49.356+0000 7f75d3415640 1 Processor -- start 2024-09-21T00:25:49.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:49.356+0000 7f75d3415640 1 -- start start 2024-09-21T00:25:49.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:49.358+0000 7f75d3415640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f75cc0b33e0 0x7f75cc1f9960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:49.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:49.358+0000 7f75d3415640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f75cc0d1460 0x7f75cc0ca430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:49.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:49.358+0000 7f75d3415640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75cc15e690 0x7f75cc0b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:49.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:49.358+0000 7f75d3415640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f75cc0b66b0 con 0x7f75cc15e690 2024-09-21T00:25:49.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:49.358+0000 7f75d3415640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f75cc0b6530 con 0x7f75cc0d1460 2024-09-21T00:25:49.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:49.358+0000 7f75d3415640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f75cc0b6830 con 0x7f75cc0b33e0 2024-09-21T00:25:49.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:49.358+0000 7f75d19b2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75cc15e690 0x7f75cc0b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:49.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:49.358+0000 7f75d19b2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75cc15e690 0x7f75cc0b8b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59072/0 (socket says 172.21.15.112:59072) 2024-09-21T00:25:49.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:49.358+0000 7f75d19b2640 1 -- 172.21.15.112:0/535537962 learned_addr learned my addr 172.21.15.112:0/535537962 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:25:49.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:49.358+0000 7f75d11b1640 1 --2- 172.21.15.112:0/535537962 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f75cc0b33e0 0x7f75cc1f9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:49.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:49.358+0000 7f75d09b0640 1 --2- 172.21.15.112:0/535537962 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f75cc0d1460 0x7f75cc0ca430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:49.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:49.358+0000 7f75d19b2640 1 -- 172.21.15.112:0/535537962 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f75cc0b33e0 msgr2=0x7f75cc1f9960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:49.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:49.358+0000 7f75d19b2640 1 --2- 172.21.15.112:0/535537962 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f75cc0b33e0 0x7f75cc1f9960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:49.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:49.358+0000 7f75d19b2640 1 -- 172.21.15.112:0/535537962 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f75cc0d1460 msgr2=0x7f75cc0ca430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:49.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:49.358+0000 7f75d19b2640 1 --2- 172.21.15.112:0/535537962 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f75cc0d1460 0x7f75cc0ca430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:49.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:49.358+0000 7f75d19b2640 1 -- 172.21.15.112:0/535537962 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75cc0ca970 con 0x7f75cc15e690 2024-09-21T00:25:49.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:49.358+0000 7f75d19b2640 1 --2- 172.21.15.112:0/535537962 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75cc15e690 0x7f75cc0b8b80 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f75c400e9c0 tx=0x7f75c400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:25:49.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:49.358+0000 7f75c1ffb640 1 -- 172.21.15.112:0/535537962 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c4004d20 con 0x7f75cc15e690 2024-09-21T00:25:49.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:49.358+0000 7f75c1ffb640 1 -- 172.21.15.112:0/535537962 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f75c400cb20 con 0x7f75cc15e690 2024-09-21T00:25:49.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:49.358+0000 7f75d3415640 1 -- 172.21.15.112:0/535537962 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75cc15a890 con 0x7f75cc15e690 2024-09-21T00:25:49.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:49.358+0000 7f75c1ffb640 1 -- 172.21.15.112:0/535537962 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c4005020 con 0x7f75cc15e690 2024-09-21T00:25:49.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:49.359+0000 7f75d3415640 1 -- 172.21.15.112:0/535537962 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75cc0cb710 con 0x7f75cc15e690 2024-09-21T00:25:49.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:49.360+0000 7f75d3415640 1 -- 172.21.15.112:0/535537962 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75cc0ceb60 con 0x7f75cc15e690 2024-09-21T00:25:49.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:49.360+0000 7f75c1ffb640 1 -- 172.21.15.112:0/535537962 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f75c40051c0 con 0x7f75cc15e690 2024-09-21T00:25:49.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:49.360+0000 7f75c1ffb640 1 --2- 172.21.15.112:0/535537962 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f75b0078bc0 0x7f75b007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:49.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:49.361+0000 7f75c1ffb640 1 -- 172.21.15.112:0/535537962 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 6007+0+0 (secure 0 0 0) 0x7f75c409b4f0 con 0x7f75cc15e690 2024-09-21T00:25:49.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:49.361+0000 7f75d11b1640 1 --2- 172.21.15.112:0/535537962 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f75b0078bc0 0x7f75b007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:49.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:49.361+0000 7f75d11b1640 1 --2- 172.21.15.112:0/535537962 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f75b0078bc0 0x7f75b007b0d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f75c8033000 tx=0x7f75c8004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:25:49.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:49.365+0000 7f75c1ffb640 1 -- 172.21.15.112:0/535537962 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f75cc0ceb60 con 0x7f75cc15e690 2024-09-21T00:25:49.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:49.544+0000 7f75d3415640 1 -- 172.21.15.112:0/535537962 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f75cc0b6cc0 con 0x7f75cc15e690 2024-09-21T00:25:51.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:51.242+0000 7f75c1ffb640 1 -- 172.21.15.112:0/535537962 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v24) ==== 228+0+0 (secure 0 0 0) 0x7f75cc0ca970 con 0x7f75cc15e690 2024-09-21T00:25:51.244 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'cephfs_data_ec' created 2024-09-21T00:25:51.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:51.246+0000 7f75d3415640 1 -- 172.21.15.112:0/535537962 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f75b0078bc0 msgr2=0x7f75b007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:51.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:51.246+0000 7f75d3415640 1 --2- 172.21.15.112:0/535537962 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f75b0078bc0 0x7f75b007b0d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f75c8033000 tx=0x7f75c8004730 comp rx=0 tx=0).stop 2024-09-21T00:25:51.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:51.246+0000 7f75d3415640 1 -- 172.21.15.112:0/535537962 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75cc15e690 msgr2=0x7f75cc0b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:51.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:51.246+0000 7f75d3415640 1 --2- 172.21.15.112:0/535537962 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75cc15e690 0x7f75cc0b8b80 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f75c400e9c0 tx=0x7f75c400ee80 comp rx=0 tx=0).stop 2024-09-21T00:25:51.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:51.246+0000 7f75d3415640 1 -- 172.21.15.112:0/535537962 shutdown_connections 2024-09-21T00:25:51.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:51.246+0000 7f75d3415640 1 --2- 172.21.15.112:0/535537962 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f75b0078bc0 0x7f75b007b0d0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:51.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:51.246+0000 7f75d3415640 1 --2- 172.21.15.112:0/535537962 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75cc15e690 0x7f75cc0b8b80 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:51.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:51.246+0000 7f75d3415640 1 --2- 172.21.15.112:0/535537962 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f75cc0d1460 0x7f75cc0ca430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:51.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:51.246+0000 7f75d3415640 1 --2- 172.21.15.112:0/535537962 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f75cc0b33e0 0x7f75cc1f9960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:51.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:51.246+0000 7f75d3415640 1 -- 172.21.15.112:0/535537962 >> 172.21.15.112:0/535537962 conn(0x7f75cc06cde0 msgr2=0x7f75cc0b3970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:51.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:51.247+0000 7f75d3415640 1 -- 172.21.15.112:0/535537962 shutdown_connections 2024-09-21T00:25:51.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:51.247+0000 7f75d3415640 1 -- 172.21.15.112:0/535537962 wait complete. 2024-09-21T00:25:51.269 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-09-21T00:25:51.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:51.388+0000 7f12df015640 1 -- 172.21.15.112:0/3659859562 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12c0004d40 msgr2=0x7f12c0005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:51.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:51.388+0000 7f12df015640 1 --2- 172.21.15.112:0/3659859562 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12c0004d40 0x7f12c0005150 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f12d000c260 tx=0x7f12c0005830 comp rx=0 tx=0).stop 2024-09-21T00:25:51.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:51.388+0000 7f12dd5b2640 1 -- 172.21.15.112:0/3659859562 reap_dead start 2024-09-21T00:25:51.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:51.388+0000 7f12df015640 1 -- 172.21.15.112:0/3659859562 shutdown_connections 2024-09-21T00:25:51.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:51.388+0000 7f12df015640 1 -- 172.21.15.112:0/3659859562 >> 172.21.15.112:0/3659859562 conn(0x7f12d80d22e0 msgr2=0x7f12d80d26f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:51.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:51.389+0000 7f12df015640 1 -- 172.21.15.112:0/3659859562 shutdown_connections 2024-09-21T00:25:51.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:51.389+0000 7f12df015640 1 -- 172.21.15.112:0/3659859562 wait complete. 2024-09-21T00:25:51.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:51.390+0000 7f12df015640 1 Processor -- start 2024-09-21T00:25:51.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:51.390+0000 7f12df015640 1 -- start start 2024-09-21T00:25:51.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:51.392+0000 7f12df015640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f12d8140200 0x7f12d81f3e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:51.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:51.392+0000 7f12df015640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f12d81f2bd0 0x7f12d81f2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:51.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:51.392+0000 7f12df015640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12d8158b20 0x7f12d8158f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:51.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:51.392+0000 7f12df015640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f12d815cb00 con 0x7f12d8158b20 2024-09-21T00:25:51.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:51.392+0000 7f12df015640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f12d80b7370 con 0x7f12d81f2bd0 2024-09-21T00:25:51.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:51.392+0000 7f12df015640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f12d815cc80 con 0x7f12d8140200 2024-09-21T00:25:51.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:51.392+0000 7f12dd5b2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12d8158b20 0x7f12d8158f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:51.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:51.392+0000 7f12dd5b2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12d8158b20 0x7f12d8158f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59080/0 (socket says 172.21.15.112:59080) 2024-09-21T00:25:51.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:51.392+0000 7f12dd5b2640 1 -- 172.21.15.112:0/2720361889 learned_addr learned my addr 172.21.15.112:0/2720361889 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:25:51.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:51.392+0000 7f12d7fff640 1 --2- 172.21.15.112:0/2720361889 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f12d81f2bd0 0x7f12d81f2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:51.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:51.392+0000 7f12dcdb1640 1 --2- 172.21.15.112:0/2720361889 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f12d8140200 0x7f12d81f3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:51.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:51.392+0000 7f12dd5b2640 1 -- 172.21.15.112:0/2720361889 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f12d8140200 msgr2=0x7f12d81f3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:51.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:51.392+0000 7f12dd5b2640 1 --2- 172.21.15.112:0/2720361889 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f12d8140200 0x7f12d81f3e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:51.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:51.392+0000 7f12dd5b2640 1 -- 172.21.15.112:0/2720361889 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f12d81f2bd0 msgr2=0x7f12d81f2fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:51.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:51.392+0000 7f12dd5b2640 1 --2- 172.21.15.112:0/2720361889 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f12d81f2bd0 0x7f12d81f2fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:51.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:51.392+0000 7f12dd5b2640 1 -- 172.21.15.112:0/2720361889 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12d81f3680 con 0x7f12d8158b20 2024-09-21T00:25:51.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:51.392+0000 7f12dd5b2640 1 --2- 172.21.15.112:0/2720361889 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12d8158b20 0x7f12d8158f30 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f12d000c350 tx=0x7f12d0000f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:25:51.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:51.393+0000 7f12d57fa640 1 -- 172.21.15.112:0/2720361889 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12d0031050 con 0x7f12d8158b20 2024-09-21T00:25:51.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:51.393+0000 7f12d57fa640 1 -- 172.21.15.112:0/2720361889 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f12d00039c0 con 0x7f12d8158b20 2024-09-21T00:25:51.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:51.393+0000 7f12df015640 1 -- 172.21.15.112:0/2720361889 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12d81f7730 con 0x7f12d8158b20 2024-09-21T00:25:51.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:51.393+0000 7f12d57fa640 1 -- 172.21.15.112:0/2720361889 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12d0001cf0 con 0x7f12d8158b20 2024-09-21T00:25:51.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:51.393+0000 7f12df015640 1 -- 172.21.15.112:0/2720361889 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12d815a240 con 0x7f12d8158b20 2024-09-21T00:25:51.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:51.394+0000 7f12df015640 1 -- 172.21.15.112:0/2720361889 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12d81f7970 con 0x7f12d8158b20 2024-09-21T00:25:51.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:51.394+0000 7f12d57fa640 1 -- 172.21.15.112:0/2720361889 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f12d0034070 con 0x7f12d8158b20 2024-09-21T00:25:51.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:51.395+0000 7f12d57fa640 1 --2- 172.21.15.112:0/2720361889 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f12c4078b90 0x7f12c407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:51.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:51.395+0000 7f12d57fa640 1 -- 172.21.15.112:0/2720361889 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6500+0+0 (secure 0 0 0) 0x7f12d00ba990 con 0x7f12d8158b20 2024-09-21T00:25:51.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:51.395+0000 7f12dcdb1640 1 --2- 172.21.15.112:0/2720361889 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f12c4078b90 0x7f12c407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:51.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:51.396+0000 7f12dcdb1640 1 --2- 172.21.15.112:0/2720361889 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f12c4078b90 0x7f12c407b0a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f12c8009c50 tx=0x7f12c8005f90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:25:51.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:51.400+0000 7f12d57fa640 1 -- 172.21.15.112:0/2720361889 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f12d0082c00 con 0x7f12d8158b20 2024-09-21T00:25:51.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:51.576+0000 7f12df015640 1 -- 172.21.15.112:0/2720361889 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7f12d80b55d0 con 0x7f12d8158b20 2024-09-21T00:25:52.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:52.244+0000 7f12d57fa640 1 -- 172.21.15.112:0/2720361889 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v25) ==== 169+0+0 (secure 0 0 0) 0x7f12d0006630 con 0x7f12d8158b20 2024-09-21T00:25:52.246 INFO:teuthology.orchestra.run.smithi112.stderr:set pool 4 allow_ec_overwrites to true 2024-09-21T00:25:52.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:52.246+0000 7f12df015640 1 -- 172.21.15.112:0/2720361889 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f12c4078b90 msgr2=0x7f12c407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:52.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:52.246+0000 7f12df015640 1 --2- 172.21.15.112:0/2720361889 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f12c4078b90 0x7f12c407b0a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f12c8009c50 tx=0x7f12c8005f90 comp rx=0 tx=0).stop 2024-09-21T00:25:52.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:52.246+0000 7f12df015640 1 -- 172.21.15.112:0/2720361889 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12d8158b20 msgr2=0x7f12d8158f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:52.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:52.246+0000 7f12df015640 1 --2- 172.21.15.112:0/2720361889 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12d8158b20 0x7f12d8158f30 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f12d000c350 tx=0x7f12d0000f10 comp rx=0 tx=0).stop 2024-09-21T00:25:52.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:52.246+0000 7f12df015640 1 -- 172.21.15.112:0/2720361889 shutdown_connections 2024-09-21T00:25:52.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:52.246+0000 7f12df015640 1 --2- 172.21.15.112:0/2720361889 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f12c4078b90 0x7f12c407b0a0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:52.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:52.246+0000 7f12df015640 1 --2- 172.21.15.112:0/2720361889 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12d8158b20 0x7f12d8158f30 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:52.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:52.246+0000 7f12df015640 1 --2- 172.21.15.112:0/2720361889 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f12d81f2bd0 0x7f12d81f2fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:52.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:52.246+0000 7f12df015640 1 --2- 172.21.15.112:0/2720361889 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f12d8140200 0x7f12d81f3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:52.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:52.246+0000 7f12df015640 1 -- 172.21.15.112:0/2720361889 >> 172.21.15.112:0/2720361889 conn(0x7f12d80d22e0 msgr2=0x7f12d80b61b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:52.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:52.246+0000 7f12df015640 1 -- 172.21.15.112:0/2720361889 shutdown_connections 2024-09-21T00:25:52.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:52.246+0000 7f12df015640 1 -- 172.21.15.112:0/2720361889 wait complete. 2024-09-21T00:25:52.271 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-09-21T00:25:52.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:52.389+0000 7fcab9d57640 1 -- 172.21.15.112:0/2524556590 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fca9c0053a0 msgr2=0x7fca9c00ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:52.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:52.389+0000 7fcab9d57640 1 --2- 172.21.15.112:0/2524556590 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fca9c0053a0 0x7fca9c00ba50 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fcaac00c750 tx=0x7fca9c005830 comp rx=0 tx=0).stop 2024-09-21T00:25:52.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:52.389+0000 7fcab9d57640 1 -- 172.21.15.112:0/2524556590 shutdown_connections 2024-09-21T00:25:52.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:52.389+0000 7fcab9d57640 1 --2- 172.21.15.112:0/2524556590 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fca9c0053a0 0x7fca9c00ba50 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:52.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:52.389+0000 7fcab9d57640 1 -- 172.21.15.112:0/2524556590 >> 172.21.15.112:0/2524556590 conn(0x7fcab409d830 msgr2=0x7fcab40b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:52.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:52.389+0000 7fcab9d57640 1 -- 172.21.15.112:0/2524556590 shutdown_connections 2024-09-21T00:25:52.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:52.389+0000 7fcab9d57640 1 -- 172.21.15.112:0/2524556590 wait complete. 2024-09-21T00:25:52.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:52.391+0000 7fcab9d57640 1 Processor -- start 2024-09-21T00:25:52.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:52.391+0000 7fcab9d57640 1 -- start start 2024-09-21T00:25:52.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:52.392+0000 7fcab9d57640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fca9c0053a0 0x7fcab41f2a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:52.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:52.392+0000 7fcab9d57640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcab406e650 0x7fcab41f5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:52.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:52.392+0000 7fcab9d57640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcab406d9f0 0x7fcab41594b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:52.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:52.392+0000 7fcab9d57640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcab40cf120 con 0x7fcab406e650 2024-09-21T00:25:52.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:52.392+0000 7fcab9d57640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fcab40cefa0 con 0x7fcab406d9f0 2024-09-21T00:25:52.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:52.392+0000 7fcab9d57640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fcab40cf2a0 con 0x7fca9c0053a0 2024-09-21T00:25:52.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:52.392+0000 7fcab2ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcab406e650 0x7fcab41f5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:52.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:52.392+0000 7fcab2ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcab406e650 0x7fcab41f5f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59098/0 (socket says 172.21.15.112:59098) 2024-09-21T00:25:52.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:52.392+0000 7fcab2ffd640 1 -- 172.21.15.112:0/1066997107 learned_addr learned my addr 172.21.15.112:0/1066997107 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:25:52.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:52.392+0000 7fcab37fe640 1 --2- 172.21.15.112:0/1066997107 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fca9c0053a0 0x7fcab41f2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:52.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:52.392+0000 7fcab3fff640 1 --2- 172.21.15.112:0/1066997107 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcab406d9f0 0x7fcab41594b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:52.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:52.393+0000 7fcab2ffd640 1 -- 172.21.15.112:0/1066997107 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fca9c0053a0 msgr2=0x7fcab41f2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:52.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:52.393+0000 7fcab2ffd640 1 --2- 172.21.15.112:0/1066997107 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fca9c0053a0 0x7fcab41f2a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:52.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:52.393+0000 7fcab2ffd640 1 -- 172.21.15.112:0/1066997107 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcab406d9f0 msgr2=0x7fcab41594b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:52.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:52.393+0000 7fcab2ffd640 1 --2- 172.21.15.112:0/1066997107 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcab406d9f0 0x7fcab41594b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:52.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:52.393+0000 7fcab2ffd640 1 -- 172.21.15.112:0/1066997107 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcab406ed60 con 0x7fcab406e650 2024-09-21T00:25:52.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:52.393+0000 7fcab2ffd640 1 --2- 172.21.15.112:0/1066997107 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcab406e650 0x7fcab41f5f20 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fcaa80068a0 tx=0x7fcaa8001620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:25:52.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:52.393+0000 7fca9bfff640 1 -- 172.21.15.112:0/1066997107 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaa8011660 con 0x7fcab406e650 2024-09-21T00:25:52.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:52.393+0000 7fca9bfff640 1 -- 172.21.15.112:0/1066997107 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcaa8016070 con 0x7fcab406e650 2024-09-21T00:25:52.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:52.393+0000 7fcab9d57640 1 -- 172.21.15.112:0/1066997107 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcab406cf10 con 0x7fcab406e650 2024-09-21T00:25:52.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:52.393+0000 7fca9bfff640 1 -- 172.21.15.112:0/1066997107 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaa8011b30 con 0x7fcab406e650 2024-09-21T00:25:52.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:52.393+0000 7fcab9d57640 1 -- 172.21.15.112:0/1066997107 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcab4159ff0 con 0x7fcab406e650 2024-09-21T00:25:52.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:52.394+0000 7fcab9d57640 1 -- 172.21.15.112:0/1066997107 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcab40d0be0 con 0x7fcab406e650 2024-09-21T00:25:52.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:52.395+0000 7fca9bfff640 1 -- 172.21.15.112:0/1066997107 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fcaa8011cd0 con 0x7fcab406e650 2024-09-21T00:25:52.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:52.395+0000 7fca9bfff640 1 --2- 172.21.15.112:0/1066997107 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fcaa0078bc0 0x7fcaa007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:52.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:52.395+0000 7fca9bfff640 1 -- 172.21.15.112:0/1066997107 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6500+0+0 (secure 0 0 0) 0x7fcaa809b590 con 0x7fcab406e650 2024-09-21T00:25:52.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:52.395+0000 7fcab37fe640 1 --2- 172.21.15.112:0/1066997107 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fcaa0078bc0 0x7fcaa007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:52.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:52.396+0000 7fcab37fe640 1 --2- 172.21.15.112:0/1066997107 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fcaa0078bc0 0x7fcaa007b0d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fcaa4009bd0 tx=0x7fcaa4005f90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:25:52.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:52.400+0000 7fca9bfff640 1 -- 172.21.15.112:0/1066997107 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fcaa80637d0 con 0x7fcab406e650 2024-09-21T00:25:52.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:52.621+0000 7fcab9d57640 1 -- 172.21.15.112:0/1066997107 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7fcab406eaf0 con 0x7fcab406e650 2024-09-21T00:25:53.159 INFO:tasks.ceph.mds.b.smithi187.stderr:2024-09-21T00:25:53.157+0000 7ff382263640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T00:25:53.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:53.257+0000 7fca9bfff640 1 -- 172.21.15.112:0/1066997107 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v7) ==== 137+0+0 (secure 0 0 0) 0x7fcaa80630d0 con 0x7fcab406e650 2024-09-21T00:25:53.259 INFO:teuthology.orchestra.run.smithi112.stderr:added data pool 4 to fsmap 2024-09-21T00:25:53.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:53.261+0000 7fcab9d57640 1 -- 172.21.15.112:0/1066997107 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fcaa0078bc0 msgr2=0x7fcaa007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:53.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:53.261+0000 7fcab9d57640 1 --2- 172.21.15.112:0/1066997107 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fcaa0078bc0 0x7fcaa007b0d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fcaa4009bd0 tx=0x7fcaa4005f90 comp rx=0 tx=0).stop 2024-09-21T00:25:53.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:53.261+0000 7fcab9d57640 1 -- 172.21.15.112:0/1066997107 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcab406e650 msgr2=0x7fcab41f5f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:53.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:53.261+0000 7fcab9d57640 1 --2- 172.21.15.112:0/1066997107 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcab406e650 0x7fcab41f5f20 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fcaa80068a0 tx=0x7fcaa8001620 comp rx=0 tx=0).stop 2024-09-21T00:25:53.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:53.261+0000 7fcab9d57640 1 -- 172.21.15.112:0/1066997107 shutdown_connections 2024-09-21T00:25:53.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:53.261+0000 7fcab9d57640 1 --2- 172.21.15.112:0/1066997107 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcab406d9f0 0x7fcab41594b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:53.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:53.261+0000 7fcab9d57640 1 --2- 172.21.15.112:0/1066997107 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fcaa0078bc0 0x7fcaa007b0d0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:53.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:53.261+0000 7fcab9d57640 1 --2- 172.21.15.112:0/1066997107 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcab406e650 0x7fcab41f5f20 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:53.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:53.261+0000 7fcab9d57640 1 --2- 172.21.15.112:0/1066997107 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fca9c0053a0 0x7fcab41f2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:53.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:53.261+0000 7fcab9d57640 1 -- 172.21.15.112:0/1066997107 >> 172.21.15.112:0/1066997107 conn(0x7fcab409d830 msgr2=0x7fcab415ff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:53.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:53.261+0000 7fcab9d57640 1 -- 172.21.15.112:0/1066997107 shutdown_connections 2024-09-21T00:25:53.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:53.261+0000 7fcab9d57640 1 -- 172.21.15.112:0/1066997107 wait complete. 2024-09-21T00:25:53.278 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T00:25:53.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:53.393+0000 7f497bef9640 1 -- 172.21.15.112:0/4093654609 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49740d0930 msgr2=0x7f497415f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:53.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:53.393+0000 7f497bef9640 1 --2- 172.21.15.112:0/4093654609 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49740d0930 0x7f497415f5e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f4970005fd0 tx=0x7f497002a5d0 comp rx=0 tx=0).stop 2024-09-21T00:25:53.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:53.394+0000 7f497bef9640 1 -- 172.21.15.112:0/4093654609 shutdown_connections 2024-09-21T00:25:53.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:53.394+0000 7f497bef9640 1 --2- 172.21.15.112:0/4093654609 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4974073ea0 0x7f49740742b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:53.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:53.394+0000 7f497bef9640 1 --2- 172.21.15.112:0/4093654609 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49740d0930 0x7f497415f5e0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:53.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:53.394+0000 7f497bef9640 1 -- 172.21.15.112:0/4093654609 >> 172.21.15.112:0/4093654609 conn(0x7f49740d1140 msgr2=0x7f49740d1550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:53.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:53.394+0000 7f497bef9640 1 -- 172.21.15.112:0/4093654609 shutdown_connections 2024-09-21T00:25:53.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:53.394+0000 7f497bef9640 1 -- 172.21.15.112:0/4093654609 wait complete. 2024-09-21T00:25:53.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:53.395+0000 7f497bef9640 1 Processor -- start 2024-09-21T00:25:53.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:53.395+0000 7f497bef9640 1 -- start start 2024-09-21T00:25:53.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:53.397+0000 7f497bef9640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4974073ea0 0x7f49740b5320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:53.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:53.397+0000 7f497bef9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49740adee0 0x7f497415d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:53.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:53.397+0000 7f497bef9640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f49740d0930 0x7f49741f8f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:53.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:53.397+0000 7f497bef9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4974071730 con 0x7f49740adee0 2024-09-21T00:25:53.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:53.397+0000 7f497bef9640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f49740715b0 con 0x7f4974073ea0 2024-09-21T00:25:53.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:53.397+0000 7f497bef9640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f49740718b0 con 0x7f49740d0930 2024-09-21T00:25:53.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:53.397+0000 7f4979494640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49740adee0 0x7f497415d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:53.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:53.397+0000 7f4979494640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49740adee0 0x7f497415d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59120/0 (socket says 172.21.15.112:59120) 2024-09-21T00:25:53.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:53.397+0000 7f4979494640 1 -- 172.21.15.112:0/3792899533 learned_addr learned my addr 172.21.15.112:0/3792899533 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:25:53.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:53.397+0000 7f4979c95640 1 --2- 172.21.15.112:0/3792899533 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4974073ea0 0x7f49740b5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:53.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:53.397+0000 7f497a496640 1 --2- 172.21.15.112:0/3792899533 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f49740d0930 0x7f49741f8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:53.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:53.397+0000 7f4979494640 1 -- 172.21.15.112:0/3792899533 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f49740d0930 msgr2=0x7f49741f8f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:53.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:53.397+0000 7f4979494640 1 --2- 172.21.15.112:0/3792899533 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f49740d0930 0x7f49741f8f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:53.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:53.397+0000 7f4979494640 1 -- 172.21.15.112:0/3792899533 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4974073ea0 msgr2=0x7f49740b5320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:53.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:53.397+0000 7f4979494640 1 --2- 172.21.15.112:0/3792899533 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4974073ea0 0x7f49740b5320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:53.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:53.397+0000 7f4979494640 1 -- 172.21.15.112:0/3792899533 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49740b5860 con 0x7f49740adee0 2024-09-21T00:25:53.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:53.397+0000 7f4979494640 1 --2- 172.21.15.112:0/3792899533 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49740adee0 0x7f497415d910 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f496400d750 tx=0x7f496400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:25:53.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:53.397+0000 7f496a7fc640 1 -- 172.21.15.112:0/3792899533 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4964017070 con 0x7f49740adee0 2024-09-21T00:25:53.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:53.397+0000 7f496a7fc640 1 -- 172.21.15.112:0/3792899533 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4964004020 con 0x7f49740adee0 2024-09-21T00:25:53.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:53.397+0000 7f497bef9640 1 -- 172.21.15.112:0/3792899533 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49740b5d70 con 0x7f49740adee0 2024-09-21T00:25:53.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:53.397+0000 7f496a7fc640 1 -- 172.21.15.112:0/3792899533 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4964004450 con 0x7f49740adee0 2024-09-21T00:25:53.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:53.397+0000 7f497bef9640 1 -- 172.21.15.112:0/3792899533 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49740b6330 con 0x7f49740adee0 2024-09-21T00:25:53.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:53.398+0000 7f497bef9640 1 -- 172.21.15.112:0/3792899533 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49740b5ae0 con 0x7f49740adee0 2024-09-21T00:25:53.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:53.399+0000 7f496a7fc640 1 -- 172.21.15.112:0/3792899533 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f49640099d0 con 0x7f49740adee0 2024-09-21T00:25:53.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:53.400+0000 7f496a7fc640 1 --2- 172.21.15.112:0/3792899533 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4958078bc0 0x7f495807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:53.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:53.400+0000 7f496a7fc640 1 -- 172.21.15.112:0/3792899533 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f496409b1f0 con 0x7f49740adee0 2024-09-21T00:25:53.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:53.400+0000 7f4979c95640 1 --2- 172.21.15.112:0/3792899533 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4958078bc0 0x7f495807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:53.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:53.401+0000 7f4979c95640 1 --2- 172.21.15.112:0/3792899533 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4958078bc0 0x7f495807b0d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4970033000 tx=0x7f4970004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:25:53.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:53.404+0000 7f496a7fc640 1 -- 172.21.15.112:0/3792899533 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f4964020020 con 0x7f49740adee0 2024-09-21T00:25:53.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:53.616+0000 7f497bef9640 1 -- 172.21.15.112:0/3792899533 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f49740742b0 con 0x7f49740adee0 2024-09-21T00:25:53.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:53.617+0000 7f496a7fc640 1 -- 172.21.15.112:0/3792899533 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5228 (secure 0 0 0) 0x7f4964062d30 con 0x7f49740adee0 2024-09-21T00:25:53.619 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:25:53.619 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":7,"btime":"2024-09-21T00:25:53:257714+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4316,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6837/2271985854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6835","nonce":2271985854},{"type":"v1","addr":"172.21.15.187:6837","nonce":2271985854}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4413,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/200348130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":200348130},{"type":"v1","addr":"172.21.15.112:6833","nonce":200348130}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4416,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/3821201561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3821201561},{"type":"v1","addr":"172.21.15.112:6835","nonce":3821201561}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T00:25:47.578960+0000","modified":"2024-09-21T00:25:52.623390+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4425},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4425":{"gid":4425,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6836/2921304789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":2921304789},{"type":"v1","addr":"172.21.15.187:6836","nonce":2921304789}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":4425,"qdb_cluster":[4425]},"id":1}]} 2024-09-21T00:25:53.619 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 7 2024-09-21T00:25:53.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:53.619+0000 7f497bef9640 1 -- 172.21.15.112:0/3792899533 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4958078bc0 msgr2=0x7f495807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:53.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:53.619+0000 7f497bef9640 1 --2- 172.21.15.112:0/3792899533 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4958078bc0 0x7f495807b0d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4970033000 tx=0x7f4970004730 comp rx=0 tx=0).stop 2024-09-21T00:25:53.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:53.619+0000 7f497bef9640 1 -- 172.21.15.112:0/3792899533 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49740adee0 msgr2=0x7f497415d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:53.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:53.619+0000 7f497bef9640 1 --2- 172.21.15.112:0/3792899533 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49740adee0 0x7f497415d910 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f496400d750 tx=0x7f496400dc10 comp rx=0 tx=0).stop 2024-09-21T00:25:53.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:53.619+0000 7f497bef9640 1 -- 172.21.15.112:0/3792899533 shutdown_connections 2024-09-21T00:25:53.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:53.619+0000 7f497bef9640 1 --2- 172.21.15.112:0/3792899533 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f49740d0930 0x7f49741f8f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:53.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:53.619+0000 7f497bef9640 1 --2- 172.21.15.112:0/3792899533 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4958078bc0 0x7f495807b0d0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:53.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:53.619+0000 7f497bef9640 1 --2- 172.21.15.112:0/3792899533 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49740adee0 0x7f497415d910 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:53.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:53.619+0000 7f497bef9640 1 --2- 172.21.15.112:0/3792899533 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4974073ea0 0x7f49740b5320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:53.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:53.619+0000 7f497bef9640 1 -- 172.21.15.112:0/3792899533 >> 172.21.15.112:0/3792899533 conn(0x7f49740d1140 msgr2=0x7f49740b14f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:53.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:53.619+0000 7f497bef9640 1 -- 172.21.15.112:0/3792899533 shutdown_connections 2024-09-21T00:25:53.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:53.619+0000 7f497bef9640 1 -- 172.21.15.112:0/3792899533 wait complete. 2024-09-21T00:25:53.633 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T00:25:53.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:53.743+0000 7f2779c25640 1 -- 172.21.15.112:0/1155895190 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2774074460 msgr2=0x7f27740b43d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:53.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:53.743+0000 7f2779c25640 1 --2- 172.21.15.112:0/1155895190 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2774074460 0x7f27740b43d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f277406e3e0 tx=0x7f276c02a8b0 comp rx=0 tx=0).stop 2024-09-21T00:25:53.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:53.743+0000 7f2779c25640 1 -- 172.21.15.112:0/1155895190 shutdown_connections 2024-09-21T00:25:53.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:53.743+0000 7f2779c25640 1 --2- 172.21.15.112:0/1155895190 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2774074460 0x7f27740b43d0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:53.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:53.743+0000 7f2779c25640 1 -- 172.21.15.112:0/1155895190 >> 172.21.15.112:0/1155895190 conn(0x7f27740b2cc0 msgr2=0x7f27740b30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:53.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:53.743+0000 7f2779c25640 1 -- 172.21.15.112:0/1155895190 shutdown_connections 2024-09-21T00:25:53.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:53.743+0000 7f2779c25640 1 -- 172.21.15.112:0/1155895190 wait complete. 2024-09-21T00:25:53.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:53.745+0000 7f2779c25640 1 Processor -- start 2024-09-21T00:25:53.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:53.745+0000 7f2779c25640 1 -- start start 2024-09-21T00:25:53.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:53.747+0000 7f2779c25640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f27740b3fc0 0x7f277414e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:53.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:53.747+0000 7f2779c25640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f27740d0b40 0x7f27740aad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:53.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:53.747+0000 7f2779c25640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f27740cad20 0x7f27741f2b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:53.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:53.747+0000 7f2779c25640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f277415b960 con 0x7f27740cad20 2024-09-21T00:25:53.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:53.747+0000 7f2779c25640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f277415b7e0 con 0x7f27740b3fc0 2024-09-21T00:25:53.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:53.747+0000 7f2779c25640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f277415bae0 con 0x7f27740d0b40 2024-09-21T00:25:53.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:53.747+0000 7f2773fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f27740cad20 0x7f27741f2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:53.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:53.747+0000 7f2773fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f27740cad20 0x7f27741f2b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59136/0 (socket says 172.21.15.112:59136) 2024-09-21T00:25:53.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:53.747+0000 7f2773fff640 1 -- 172.21.15.112:0/3619825368 learned_addr learned my addr 172.21.15.112:0/3619825368 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:25:53.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:53.747+0000 7f27737fe640 1 --2- 172.21.15.112:0/3619825368 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f27740b3fc0 0x7f277414e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:53.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:53.747+0000 7f2772ffd640 1 --2- 172.21.15.112:0/3619825368 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f27740d0b40 0x7f27740aad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:53.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:53.747+0000 7f2773fff640 1 -- 172.21.15.112:0/3619825368 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f27740d0b40 msgr2=0x7f27740aad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:53.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:53.747+0000 7f2773fff640 1 --2- 172.21.15.112:0/3619825368 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f27740d0b40 0x7f27740aad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:53.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:53.747+0000 7f2773fff640 1 -- 172.21.15.112:0/3619825368 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f27740b3fc0 msgr2=0x7f277414e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:53.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:53.747+0000 7f2773fff640 1 --2- 172.21.15.112:0/3619825368 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f27740b3fc0 0x7f277414e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:53.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:53.747+0000 7f2773fff640 1 -- 172.21.15.112:0/3619825368 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27740ab260 con 0x7f27740cad20 2024-09-21T00:25:53.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:53.747+0000 7f2772ffd640 1 --2- 172.21.15.112:0/3619825368 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f27740d0b40 0x7f27740aad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:25:53.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:53.747+0000 7f2773fff640 1 --2- 172.21.15.112:0/3619825368 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f27740cad20 0x7f27741f2b40 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f276c035040 tx=0x7f276c003f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:25:53.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:53.747+0000 7f275bfff640 1 -- 172.21.15.112:0/3619825368 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f276c042020 con 0x7f27740cad20 2024-09-21T00:25:53.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:53.747+0000 7f275bfff640 1 -- 172.21.15.112:0/3619825368 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f276c033c00 con 0x7f27740cad20 2024-09-21T00:25:53.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:53.747+0000 7f2779c25640 1 -- 172.21.15.112:0/3619825368 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27740cb450 con 0x7f27740cad20 2024-09-21T00:25:53.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:53.747+0000 7f275bfff640 1 -- 172.21.15.112:0/3619825368 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f276c037040 con 0x7f27740cad20 2024-09-21T00:25:53.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:53.747+0000 7f2779c25640 1 -- 172.21.15.112:0/3619825368 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27740cba10 con 0x7f27740cad20 2024-09-21T00:25:53.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:53.748+0000 7f2779c25640 1 -- 172.21.15.112:0/3619825368 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27740cb1c0 con 0x7f27740cad20 2024-09-21T00:25:53.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:53.749+0000 7f275bfff640 1 -- 172.21.15.112:0/3619825368 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f276c045020 con 0x7f27740cad20 2024-09-21T00:25:53.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:53.750+0000 7f275bfff640 1 --2- 172.21.15.112:0/3619825368 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2760078bc0 0x7f276007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:53.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:53.750+0000 7f275bfff640 1 -- 172.21.15.112:0/3619825368 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f276c0bbb90 con 0x7f27740cad20 2024-09-21T00:25:53.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:53.750+0000 7f27737fe640 1 --2- 172.21.15.112:0/3619825368 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2760078bc0 0x7f276007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:53.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:53.750+0000 7f27737fe640 1 --2- 172.21.15.112:0/3619825368 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2760078bc0 0x7f276007b0d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2764001860 tx=0x7f2764001690 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:25:53.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:53.753+0000 7f275bfff640 1 -- 172.21.15.112:0/3619825368 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f276c04c070 con 0x7f27740cad20 2024-09-21T00:25:53.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:53.931+0000 7f2779c25640 1 -- 172.21.15.112:0/3619825368 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f277414b4f0 con 0x7f27740cad20 2024-09-21T00:25:53.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:53.932+0000 7f275bfff640 1 -- 172.21.15.112:0/3619825368 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18622 (secure 0 0 0) 0x7f277414b4f0 con 0x7f27740cad20 2024-09-21T00:25:53.935 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:25:53.935 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":26,"fsid":"8b34692b-dc65-49b2-a08f-26f707b96ac5","created":"2024-09-21T00:25:22.788490+0000","modified":"2024-09-21T00:25:52.623213+0000","last_up_change":"2024-09-21T00:25:36.161835+0000","last_in_change":"2024-09-21T00:25:26.346500+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T00:25:37.105006+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T00:25:45.413895+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T00:25:46.532124+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T00:25:50.244145+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9c575086-67d2-4de2-b01b-d44dcc21d604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6809","nonce":755760621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6811","nonce":755760621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6815","nonce":755760621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6813","nonce":755760621}]},"public_addr":"172.21.15.112:6809/755760621","cluster_addr":"172.21.15.112:6811/755760621","heartbeat_back_addr":"172.21.15.112:6815/755760621","heartbeat_front_addr":"172.21.15.112:6813/755760621","state":["exists","up"]},{"osd":1,"uuid":"a983c227-a3ce-4bdd-82f1-438ba7e3ecd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6801","nonce":344932592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6803","nonce":344932592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6807","nonce":344932592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6805","nonce":344932592}]},"public_addr":"172.21.15.112:6801/344932592","cluster_addr":"172.21.15.112:6803/344932592","heartbeat_back_addr":"172.21.15.112:6807/344932592","heartbeat_front_addr":"172.21.15.112:6805/344932592","state":["exists","up"]},{"osd":2,"uuid":"1193c1d8-bfff-4a57-8200-5f69bf1ea3af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6825","nonce":407933616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6827","nonce":407933616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6831","nonce":407933616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6829","nonce":407933616}]},"public_addr":"172.21.15.112:6825/407933616","cluster_addr":"172.21.15.112:6827/407933616","heartbeat_back_addr":"172.21.15.112:6831/407933616","heartbeat_front_addr":"172.21.15.112:6829/407933616","state":["exists","up"]},{"osd":3,"uuid":"af04330a-36ad-40cc-a8dd-fa259b221de5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6817","nonce":4120164806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6819","nonce":4120164806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6823","nonce":4120164806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6821","nonce":4120164806}]},"public_addr":"172.21.15.112:6817/4120164806","cluster_addr":"172.21.15.112:6819/4120164806","heartbeat_back_addr":"172.21.15.112:6823/4120164806","heartbeat_front_addr":"172.21.15.112:6821/4120164806","state":["exists","up"]},{"osd":4,"uuid":"5b893111-292c-4c5a-b184-53ad38c7199e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6803","nonce":522077560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6805","nonce":522077560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6809","nonce":522077560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6807","nonce":522077560}]},"public_addr":"172.21.15.187:6803/522077560","cluster_addr":"172.21.15.187:6805/522077560","heartbeat_back_addr":"172.21.15.187:6809/522077560","heartbeat_front_addr":"172.21.15.187:6807/522077560","state":["exists","up"]},{"osd":5,"uuid":"c0b8e011-45fe-4c5e-925e-6be2a7f1868a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6827","nonce":3593144721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6829","nonce":3593144721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6833","nonce":3593144721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6831","nonce":3593144721}]},"public_addr":"172.21.15.187:6827/3593144721","cluster_addr":"172.21.15.187:6829/3593144721","heartbeat_back_addr":"172.21.15.187:6833/3593144721","heartbeat_front_addr":"172.21.15.187:6831/3593144721","state":["exists","up"]},{"osd":6,"uuid":"87d2d650-7489-4593-98a3-a1f433ce8dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6811","nonce":1494387657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6813","nonce":1494387657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6817","nonce":1494387657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6815","nonce":1494387657}]},"public_addr":"172.21.15.187:6811/1494387657","cluster_addr":"172.21.15.187:6813/1494387657","heartbeat_back_addr":"172.21.15.187:6817/1494387657","heartbeat_front_addr":"172.21.15.187:6815/1494387657","state":["exists","up"]},{"osd":7,"uuid":"c6b4f8e2-a1ee-4438-aa37-67ff4b6ef1f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6819","nonce":1624699842}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6821","nonce":1624699842}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6825","nonce":1624699842}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6823","nonce":1624699842}]},"public_addr":"172.21.15.187:6819/1624699842","cluster_addr":"172.21.15.187:6821/1624699842","heartbeat_back_addr":"172.21.15.187:6825/1624699842","heartbeat_front_addr":"172.21.15.187:6823/1624699842","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.873162+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.400506+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.664317+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.159791+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.202630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.113212+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.384786+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.448441+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T00:25:53.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:53.935+0000 7f2779c25640 1 -- 172.21.15.112:0/3619825368 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2760078bc0 msgr2=0x7f276007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:53.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:53.935+0000 7f2779c25640 1 --2- 172.21.15.112:0/3619825368 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2760078bc0 0x7f276007b0d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2764001860 tx=0x7f2764001690 comp rx=0 tx=0).stop 2024-09-21T00:25:53.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:53.935+0000 7f2779c25640 1 -- 172.21.15.112:0/3619825368 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f27740cad20 msgr2=0x7f27741f2b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:53.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:53.935+0000 7f2779c25640 1 --2- 172.21.15.112:0/3619825368 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f27740cad20 0x7f27741f2b40 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f276c035040 tx=0x7f276c003f50 comp rx=0 tx=0).stop 2024-09-21T00:25:53.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:53.935+0000 7f2779c25640 1 -- 172.21.15.112:0/3619825368 shutdown_connections 2024-09-21T00:25:53.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:53.935+0000 7f2779c25640 1 --2- 172.21.15.112:0/3619825368 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2760078bc0 0x7f276007b0d0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:53.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:53.935+0000 7f2779c25640 1 --2- 172.21.15.112:0/3619825368 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f27740cad20 0x7f27741f2b40 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:53.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:53.935+0000 7f2779c25640 1 --2- 172.21.15.112:0/3619825368 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f27740d0b40 0x7f27740aad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:53.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:53.935+0000 7f2779c25640 1 --2- 172.21.15.112:0/3619825368 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f27740b3fc0 0x7f277414e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:53.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:53.935+0000 7f2779c25640 1 -- 172.21.15.112:0/3619825368 >> 172.21.15.112:0/3619825368 conn(0x7f27740b2cc0 msgr2=0x7f27740cefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:53.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:53.935+0000 7f2779c25640 1 -- 172.21.15.112:0/3619825368 shutdown_connections 2024-09-21T00:25:53.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:53.935+0000 7f2779c25640 1 -- 172.21.15.112:0/3619825368 wait complete. 2024-09-21T00:25:53.948 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T00:25:54.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.057+0000 7fc2547e4640 1 -- 172.21.15.112:0/2397719794 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc238004800 msgr2=0x7fc238004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:54.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.057+0000 7fc2547e4640 1 --2- 172.21.15.112:0/2397719794 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc238004800 0x7fc238004c10 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc23c0016c0 tx=0x7fc238005830 comp rx=0 tx=0).stop 2024-09-21T00:25:54.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.057+0000 7fc252d81640 1 -- 172.21.15.112:0/2397719794 reap_dead start 2024-09-21T00:25:54.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.057+0000 7fc2547e4640 1 -- 172.21.15.112:0/2397719794 shutdown_connections 2024-09-21T00:25:54.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.057+0000 7fc2547e4640 1 -- 172.21.15.112:0/2397719794 >> 172.21.15.112:0/2397719794 conn(0x7fc24c0b6b20 msgr2=0x7fc24c0d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:54.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.057+0000 7fc2547e4640 1 -- 172.21.15.112:0/2397719794 shutdown_connections 2024-09-21T00:25:54.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.057+0000 7fc2547e4640 1 -- 172.21.15.112:0/2397719794 wait complete. 2024-09-21T00:25:54.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.059+0000 7fc2547e4640 1 Processor -- start 2024-09-21T00:25:54.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.059+0000 7fc2547e4640 1 -- start start 2024-09-21T00:25:54.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.060+0000 7fc2547e4640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc238004800 0x7fc24c0b2750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:54.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.060+0000 7fc2547e4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc24c0cadc0 0x7fc24c15c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:54.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.060+0000 7fc2547e4640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc24c0cb1d0 0x7fc24c15c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:54.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.060+0000 7fc2547e4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc24c15f640 con 0x7fc24c0cadc0 2024-09-21T00:25:54.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.060+0000 7fc2547e4640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc24c15f4c0 con 0x7fc24c0cb1d0 2024-09-21T00:25:54.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.060+0000 7fc2547e4640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc24c15f7c0 con 0x7fc238004800 2024-09-21T00:25:54.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.060+0000 7fc251d7f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc24c0cadc0 0x7fc24c15c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:54.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.060+0000 7fc251d7f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc24c0cadc0 0x7fc24c15c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59152/0 (socket says 172.21.15.112:59152) 2024-09-21T00:25:54.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.061+0000 7fc251d7f640 1 -- 172.21.15.112:0/358887809 learned_addr learned my addr 172.21.15.112:0/358887809 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:25:54.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.061+0000 7fc252580640 1 --2- 172.21.15.112:0/358887809 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc238004800 0x7fc24c0b2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:54.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.061+0000 7fc252d81640 1 --2- 172.21.15.112:0/358887809 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc24c0cb1d0 0x7fc24c15c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:54.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.061+0000 7fc251d7f640 1 -- 172.21.15.112:0/358887809 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc238004800 msgr2=0x7fc24c0b2750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:54.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.061+0000 7fc251d7f640 1 --2- 172.21.15.112:0/358887809 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc238004800 0x7fc24c0b2750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:54.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.061+0000 7fc251d7f640 1 -- 172.21.15.112:0/358887809 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc24c0cb1d0 msgr2=0x7fc24c15c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:54.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.061+0000 7fc251d7f640 1 --2- 172.21.15.112:0/358887809 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc24c0cb1d0 0x7fc24c15c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:54.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.061+0000 7fc251d7f640 1 -- 172.21.15.112:0/358887809 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc24c15bf20 con 0x7fc24c0cadc0 2024-09-21T00:25:54.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.061+0000 7fc252d81640 1 --2- 172.21.15.112:0/358887809 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc24c0cb1d0 0x7fc24c15c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:25:54.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.061+0000 7fc251d7f640 1 --2- 172.21.15.112:0/358887809 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc24c0cadc0 0x7fc24c15c460 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc23c0017b0 tx=0x7fc23c006310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:25:54.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.061+0000 7fc242ffd640 1 -- 172.21.15.112:0/358887809 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc23c041070 con 0x7fc24c0cadc0 2024-09-21T00:25:54.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.061+0000 7fc242ffd640 1 -- 172.21.15.112:0/358887809 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc23c006530 con 0x7fc24c0cadc0 2024-09-21T00:25:54.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.061+0000 7fc242ffd640 1 -- 172.21.15.112:0/358887809 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc23c038070 con 0x7fc24c0cadc0 2024-09-21T00:25:54.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.061+0000 7fc2547e4640 1 -- 172.21.15.112:0/358887809 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc24c0cb9a0 con 0x7fc24c0cadc0 2024-09-21T00:25:54.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.061+0000 7fc2547e4640 1 -- 172.21.15.112:0/358887809 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc24c14a670 con 0x7fc24c0cadc0 2024-09-21T00:25:54.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.062+0000 7fc2547e4640 1 -- 172.21.15.112:0/358887809 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc24c0b3250 con 0x7fc24c0cadc0 2024-09-21T00:25:54.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.063+0000 7fc242ffd640 1 -- 172.21.15.112:0/358887809 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc23c044070 con 0x7fc24c0cadc0 2024-09-21T00:25:54.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.063+0000 7fc242ffd640 1 --2- 172.21.15.112:0/358887809 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fc230078bc0 0x7fc23007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:54.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.063+0000 7fc242ffd640 1 -- 172.21.15.112:0/358887809 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fc23c0ba790 con 0x7fc24c0cadc0 2024-09-21T00:25:54.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.063+0000 7fc252580640 1 --2- 172.21.15.112:0/358887809 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fc230078bc0 0x7fc23007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:54.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.064+0000 7fc252580640 1 --2- 172.21.15.112:0/358887809 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fc230078bc0 0x7fc23007b0d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc248003700 tx=0x7fc24800b260 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:25:54.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.068+0000 7fc242ffd640 1 -- 172.21.15.112:0/358887809 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc23c082240 con 0x7fc24c0cadc0 2024-09-21T00:25:54.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.246+0000 7fc2547e4640 1 -- 172.21.15.112:0/358887809 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc24c0b3e80 con 0x7fc24c0cadc0 2024-09-21T00:25:54.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.247+0000 7fc242ffd640 1 -- 172.21.15.112:0/358887809 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18622 (secure 0 0 0) 0x7fc23c086c70 con 0x7fc24c0cadc0 2024-09-21T00:25:54.249 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:25:54.249 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":26,"fsid":"8b34692b-dc65-49b2-a08f-26f707b96ac5","created":"2024-09-21T00:25:22.788490+0000","modified":"2024-09-21T00:25:52.623213+0000","last_up_change":"2024-09-21T00:25:36.161835+0000","last_in_change":"2024-09-21T00:25:26.346500+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T00:25:37.105006+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T00:25:45.413895+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T00:25:46.532124+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T00:25:50.244145+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9c575086-67d2-4de2-b01b-d44dcc21d604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6809","nonce":755760621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6811","nonce":755760621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6815","nonce":755760621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6813","nonce":755760621}]},"public_addr":"172.21.15.112:6809/755760621","cluster_addr":"172.21.15.112:6811/755760621","heartbeat_back_addr":"172.21.15.112:6815/755760621","heartbeat_front_addr":"172.21.15.112:6813/755760621","state":["exists","up"]},{"osd":1,"uuid":"a983c227-a3ce-4bdd-82f1-438ba7e3ecd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6801","nonce":344932592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6803","nonce":344932592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6807","nonce":344932592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6805","nonce":344932592}]},"public_addr":"172.21.15.112:6801/344932592","cluster_addr":"172.21.15.112:6803/344932592","heartbeat_back_addr":"172.21.15.112:6807/344932592","heartbeat_front_addr":"172.21.15.112:6805/344932592","state":["exists","up"]},{"osd":2,"uuid":"1193c1d8-bfff-4a57-8200-5f69bf1ea3af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6825","nonce":407933616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6827","nonce":407933616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6831","nonce":407933616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6829","nonce":407933616}]},"public_addr":"172.21.15.112:6825/407933616","cluster_addr":"172.21.15.112:6827/407933616","heartbeat_back_addr":"172.21.15.112:6831/407933616","heartbeat_front_addr":"172.21.15.112:6829/407933616","state":["exists","up"]},{"osd":3,"uuid":"af04330a-36ad-40cc-a8dd-fa259b221de5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6817","nonce":4120164806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6819","nonce":4120164806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6823","nonce":4120164806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6821","nonce":4120164806}]},"public_addr":"172.21.15.112:6817/4120164806","cluster_addr":"172.21.15.112:6819/4120164806","heartbeat_back_addr":"172.21.15.112:6823/4120164806","heartbeat_front_addr":"172.21.15.112:6821/4120164806","state":["exists","up"]},{"osd":4,"uuid":"5b893111-292c-4c5a-b184-53ad38c7199e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6803","nonce":522077560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6805","nonce":522077560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6809","nonce":522077560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6807","nonce":522077560}]},"public_addr":"172.21.15.187:6803/522077560","cluster_addr":"172.21.15.187:6805/522077560","heartbeat_back_addr":"172.21.15.187:6809/522077560","heartbeat_front_addr":"172.21.15.187:6807/522077560","state":["exists","up"]},{"osd":5,"uuid":"c0b8e011-45fe-4c5e-925e-6be2a7f1868a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6827","nonce":3593144721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6829","nonce":3593144721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6833","nonce":3593144721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6831","nonce":3593144721}]},"public_addr":"172.21.15.187:6827/3593144721","cluster_addr":"172.21.15.187:6829/3593144721","heartbeat_back_addr":"172.21.15.187:6833/3593144721","heartbeat_front_addr":"172.21.15.187:6831/3593144721","state":["exists","up"]},{"osd":6,"uuid":"87d2d650-7489-4593-98a3-a1f433ce8dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6811","nonce":1494387657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6813","nonce":1494387657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6817","nonce":1494387657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6815","nonce":1494387657}]},"public_addr":"172.21.15.187:6811/1494387657","cluster_addr":"172.21.15.187:6813/1494387657","heartbeat_back_addr":"172.21.15.187:6817/1494387657","heartbeat_front_addr":"172.21.15.187:6815/1494387657","state":["exists","up"]},{"osd":7,"uuid":"c6b4f8e2-a1ee-4438-aa37-67ff4b6ef1f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6819","nonce":1624699842}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6821","nonce":1624699842}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6825","nonce":1624699842}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6823","nonce":1624699842}]},"public_addr":"172.21.15.187:6819/1624699842","cluster_addr":"172.21.15.187:6821/1624699842","heartbeat_back_addr":"172.21.15.187:6825/1624699842","heartbeat_front_addr":"172.21.15.187:6823/1624699842","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.873162+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.400506+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.664317+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.159791+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.202630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.113212+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.384786+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.448441+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T00:25:54.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.249+0000 7fc2547e4640 1 -- 172.21.15.112:0/358887809 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fc230078bc0 msgr2=0x7fc23007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:54.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.249+0000 7fc2547e4640 1 --2- 172.21.15.112:0/358887809 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fc230078bc0 0x7fc23007b0d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc248003700 tx=0x7fc24800b260 comp rx=0 tx=0).stop 2024-09-21T00:25:54.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.249+0000 7fc2547e4640 1 -- 172.21.15.112:0/358887809 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc24c0cadc0 msgr2=0x7fc24c15c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:54.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.249+0000 7fc2547e4640 1 --2- 172.21.15.112:0/358887809 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc24c0cadc0 0x7fc24c15c460 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc23c0017b0 tx=0x7fc23c006310 comp rx=0 tx=0).stop 2024-09-21T00:25:54.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.249+0000 7fc2547e4640 1 -- 172.21.15.112:0/358887809 shutdown_connections 2024-09-21T00:25:54.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.249+0000 7fc2547e4640 1 --2- 172.21.15.112:0/358887809 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc24c0cb1d0 0x7fc24c15c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:54.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.249+0000 7fc2547e4640 1 --2- 172.21.15.112:0/358887809 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fc230078bc0 0x7fc23007b0d0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:54.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.249+0000 7fc2547e4640 1 --2- 172.21.15.112:0/358887809 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc24c0cadc0 0x7fc24c15c460 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:54.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.249+0000 7fc2547e4640 1 --2- 172.21.15.112:0/358887809 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc238004800 0x7fc24c0b2750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:54.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.249+0000 7fc2547e4640 1 -- 172.21.15.112:0/358887809 >> 172.21.15.112:0/358887809 conn(0x7fc24c0b6b20 msgr2=0x7fc24c06e610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:54.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.249+0000 7fc2547e4640 1 -- 172.21.15.112:0/358887809 shutdown_connections 2024-09-21T00:25:54.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.249+0000 7fc2547e4640 1 -- 172.21.15.112:0/358887809 wait complete. 2024-09-21T00:25:54.277 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T00:25:54.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.388+0000 7fc023913640 1 -- 172.21.15.112:0/521653174 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc01c162f00 msgr2=0x7fc01c0d24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:54.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.388+0000 7fc023913640 1 --2- 172.21.15.112:0/521653174 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc01c162f00 0x7fc01c0d24d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc00c005fd0 tx=0x7fc00c02a8b0 comp rx=0 tx=0).stop 2024-09-21T00:25:54.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.389+0000 7fc023913640 1 -- 172.21.15.112:0/521653174 shutdown_connections 2024-09-21T00:25:54.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.389+0000 7fc023913640 1 --2- 172.21.15.112:0/521653174 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc01c162280 0x7fc01c162690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:54.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.389+0000 7fc023913640 1 --2- 172.21.15.112:0/521653174 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc01c162f00 0x7fc01c0d24d0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:54.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.389+0000 7fc023913640 1 -- 172.21.15.112:0/521653174 >> 172.21.15.112:0/521653174 conn(0x7fc01c0b6b20 msgr2=0x7fc01c0d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:54.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.389+0000 7fc023913640 1 -- 172.21.15.112:0/521653174 shutdown_connections 2024-09-21T00:25:54.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.389+0000 7fc023913640 1 -- 172.21.15.112:0/521653174 wait complete. 2024-09-21T00:25:54.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.390+0000 7fc023913640 1 Processor -- start 2024-09-21T00:25:54.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.390+0000 7fc023913640 1 -- start start 2024-09-21T00:25:54.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.392+0000 7fc023913640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc01c0b2e40 0x7fc01c157210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:54.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.392+0000 7fc023913640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc01c162280 0x7fc01c1f8890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:54.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.392+0000 7fc023913640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc01c157fb0 0x7fc01c1f8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:54.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.392+0000 7fc023913640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc01c15e630 con 0x7fc01c162280 2024-09-21T00:25:54.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.392+0000 7fc023913640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc01c15e4b0 con 0x7fc01c157fb0 2024-09-21T00:25:54.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.392+0000 7fc023913640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc01c15e7b0 con 0x7fc01c0b2e40 2024-09-21T00:25:54.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.392+0000 7fc020eae640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc01c162280 0x7fc01c1f8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:54.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.392+0000 7fc020eae640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc01c162280 0x7fc01c1f8890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59160/0 (socket says 172.21.15.112:59160) 2024-09-21T00:25:54.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.392+0000 7fc020eae640 1 -- 172.21.15.112:0/3163878073 learned_addr learned my addr 172.21.15.112:0/3163878073 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:25:54.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.392+0000 7fc0216af640 1 --2- 172.21.15.112:0/3163878073 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc01c0b2e40 0x7fc01c157210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:54.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.392+0000 7fc021eb0640 1 --2- 172.21.15.112:0/3163878073 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc01c157fb0 0x7fc01c1f8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:54.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.392+0000 7fc020eae640 1 -- 172.21.15.112:0/3163878073 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc01c0b2e40 msgr2=0x7fc01c157210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:54.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.392+0000 7fc020eae640 1 --2- 172.21.15.112:0/3163878073 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc01c0b2e40 0x7fc01c157210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:54.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.392+0000 7fc020eae640 1 -- 172.21.15.112:0/3163878073 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc01c157fb0 msgr2=0x7fc01c1f8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:54.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.392+0000 7fc020eae640 1 --2- 172.21.15.112:0/3163878073 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc01c157fb0 0x7fc01c1f8dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:54.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.392+0000 7fc020eae640 1 -- 172.21.15.112:0/3163878073 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc01c157a60 con 0x7fc01c162280 2024-09-21T00:25:54.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.392+0000 7fc0216af640 1 --2- 172.21.15.112:0/3163878073 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc01c0b2e40 0x7fc01c157210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:25:54.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.392+0000 7fc020eae640 1 --2- 172.21.15.112:0/3163878073 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc01c162280 0x7fc01c1f8890 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc01000d6d0 tx=0x7fc01000db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:25:54.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.392+0000 7fc019ffb640 1 -- 172.21.15.112:0/3163878073 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc010017070 con 0x7fc01c162280 2024-09-21T00:25:54.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.392+0000 7fc019ffb640 1 -- 172.21.15.112:0/3163878073 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc010004310 con 0x7fc01c162280 2024-09-21T00:25:54.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.392+0000 7fc023913640 1 -- 172.21.15.112:0/3163878073 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc01c1f9610 con 0x7fc01c162280 2024-09-21T00:25:54.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.392+0000 7fc019ffb640 1 -- 172.21.15.112:0/3163878073 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc010009a80 con 0x7fc01c162280 2024-09-21T00:25:54.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.393+0000 7fc023913640 1 -- 172.21.15.112:0/3163878073 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc01c1f9310 con 0x7fc01c162280 2024-09-21T00:25:54.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.394+0000 7fc023913640 1 -- 172.21.15.112:0/3163878073 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc01c1e9f60 con 0x7fc01c162280 2024-09-21T00:25:54.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.394+0000 7fc019ffb640 1 -- 172.21.15.112:0/3163878073 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc0100035f0 con 0x7fc01c162280 2024-09-21T00:25:54.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.394+0000 7fc019ffb640 1 --2- 172.21.15.112:0/3163878073 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fc008078b90 0x7fc00807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:54.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.395+0000 7fc019ffb640 1 -- 172.21.15.112:0/3163878073 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fc01009b2c0 con 0x7fc01c162280 2024-09-21T00:25:54.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.395+0000 7fc0216af640 1 --2- 172.21.15.112:0/3163878073 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fc008078b90 0x7fc00807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:54.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.395+0000 7fc0216af640 1 --2- 172.21.15.112:0/3163878073 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fc008078b90 0x7fc00807b0a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc00c034000 tx=0x7fc00c004ab0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:25:54.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.399+0000 7fc019ffb640 1 -- 172.21.15.112:0/3163878073 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc010063450 con 0x7fc01c162280 2024-09-21T00:25:54.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.615+0000 7fc023913640 1 -- 172.21.15.112:0/3163878073 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc01c157c30 con 0x7fc01c162280 2024-09-21T00:25:54.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.615+0000 7fc019ffb640 1 -- 172.21.15.112:0/3163878073 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5228 (secure 0 0 0) 0x7fc010062d50 con 0x7fc01c162280 2024-09-21T00:25:54.617 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:25:54.617 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":7,"btime":"2024-09-21T00:25:53:257714+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4316,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6837/2271985854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6835","nonce":2271985854},{"type":"v1","addr":"172.21.15.187:6837","nonce":2271985854}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4413,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/200348130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":200348130},{"type":"v1","addr":"172.21.15.112:6833","nonce":200348130}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4416,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/3821201561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3821201561},{"type":"v1","addr":"172.21.15.112:6835","nonce":3821201561}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T00:25:47.578960+0000","modified":"2024-09-21T00:25:52.623390+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4425},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4425":{"gid":4425,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6836/2921304789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":2921304789},{"type":"v1","addr":"172.21.15.187:6836","nonce":2921304789}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":4425,"qdb_cluster":[4425]},"id":1}]} 2024-09-21T00:25:54.617 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 7 2024-09-21T00:25:54.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.617+0000 7fc023913640 1 -- 172.21.15.112:0/3163878073 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fc008078b90 msgr2=0x7fc00807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:54.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.617+0000 7fc023913640 1 --2- 172.21.15.112:0/3163878073 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fc008078b90 0x7fc00807b0a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc00c034000 tx=0x7fc00c004ab0 comp rx=0 tx=0).stop 2024-09-21T00:25:54.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.617+0000 7fc023913640 1 -- 172.21.15.112:0/3163878073 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc01c162280 msgr2=0x7fc01c1f8890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:54.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.617+0000 7fc023913640 1 --2- 172.21.15.112:0/3163878073 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc01c162280 0x7fc01c1f8890 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc01000d6d0 tx=0x7fc01000db90 comp rx=0 tx=0).stop 2024-09-21T00:25:54.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.617+0000 7fc023913640 1 -- 172.21.15.112:0/3163878073 shutdown_connections 2024-09-21T00:25:54.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.617+0000 7fc023913640 1 --2- 172.21.15.112:0/3163878073 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc01c157fb0 0x7fc01c1f8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:54.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.617+0000 7fc023913640 1 --2- 172.21.15.112:0/3163878073 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fc008078b90 0x7fc00807b0a0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:54.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.617+0000 7fc023913640 1 --2- 172.21.15.112:0/3163878073 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc01c162280 0x7fc01c1f8890 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:54.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.617+0000 7fc023913640 1 --2- 172.21.15.112:0/3163878073 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc01c0b2e40 0x7fc01c157210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:54.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.617+0000 7fc023913640 1 -- 172.21.15.112:0/3163878073 >> 172.21.15.112:0/3163878073 conn(0x7fc01c0b6b20 msgr2=0x7fc01c06d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:54.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.617+0000 7fc023913640 1 -- 172.21.15.112:0/3163878073 shutdown_connections 2024-09-21T00:25:54.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.617+0000 7fc023913640 1 -- 172.21.15.112:0/3163878073 wait complete. 2024-09-21T00:25:54.630 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T00:25:54.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.740+0000 7f70f92b6640 1 -- 172.21.15.112:0/3949643279 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f70f4162590 msgr2=0x7f70f41629a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:54.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.740+0000 7f70f92b6640 1 --2- 172.21.15.112:0/3949643279 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f70f4162590 0x7f70f41629a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f70e8005fd0 tx=0x7f70e802a530 comp rx=0 tx=0).stop 2024-09-21T00:25:54.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.740+0000 7f70f92b6640 1 -- 172.21.15.112:0/3949643279 shutdown_connections 2024-09-21T00:25:54.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.740+0000 7f70f92b6640 1 --2- 172.21.15.112:0/3949643279 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f70f4162590 0x7f70f41629a0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:54.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.740+0000 7f70f92b6640 1 --2- 172.21.15.112:0/3949643279 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f70f40cfe00 0x7f70f40d2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:54.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.740+0000 7f70f92b6640 1 -- 172.21.15.112:0/3949643279 >> 172.21.15.112:0/3949643279 conn(0x7f70f40d4440 msgr2=0x7f70f40d4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:54.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.740+0000 7f70f92b6640 1 -- 172.21.15.112:0/3949643279 shutdown_connections 2024-09-21T00:25:54.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.740+0000 7f70f92b6640 1 -- 172.21.15.112:0/3949643279 wait complete. 2024-09-21T00:25:54.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.742+0000 7f70f92b6640 1 Processor -- start 2024-09-21T00:25:54.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.742+0000 7f70f92b6640 1 -- start start 2024-09-21T00:25:54.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.743+0000 7f70f92b6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f70f40b2e60 0x7f70f41f5610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:54.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.743+0000 7f70f92b6640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f70f40cfe00 0x7f70f41f9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:54.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.743+0000 7f70f92b6640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f70f4162590 0x7f70f41607e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:54.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.743+0000 7f70f92b6640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f70f40b3b00 con 0x7f70f40b2e60 2024-09-21T00:25:54.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.743+0000 7f70f92b6640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f70f40b3980 con 0x7f70f4162590 2024-09-21T00:25:54.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.743+0000 7f70f92b6640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f70f40b3f60 con 0x7f70f40cfe00 2024-09-21T00:25:54.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.743+0000 7f70f2ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f70f40b2e60 0x7f70f41f5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:54.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.743+0000 7f70f2ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f70f40b2e60 0x7f70f41f5610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59174/0 (socket says 172.21.15.112:59174) 2024-09-21T00:25:54.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.743+0000 7f70f2ffd640 1 -- 172.21.15.112:0/2828190122 learned_addr learned my addr 172.21.15.112:0/2828190122 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:25:54.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.743+0000 7f70f27fc640 1 --2- 172.21.15.112:0/2828190122 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f70f40cfe00 0x7f70f41f9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:54.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.743+0000 7f70f37fe640 1 --2- 172.21.15.112:0/2828190122 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f70f4162590 0x7f70f41607e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:54.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.743+0000 7f70f2ffd640 1 -- 172.21.15.112:0/2828190122 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f70f40cfe00 msgr2=0x7f70f41f9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:54.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.743+0000 7f70f2ffd640 1 --2- 172.21.15.112:0/2828190122 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f70f40cfe00 0x7f70f41f9aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:54.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.743+0000 7f70f2ffd640 1 -- 172.21.15.112:0/2828190122 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f70f4162590 msgr2=0x7f70f41607e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:54.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.743+0000 7f70f2ffd640 1 --2- 172.21.15.112:0/2828190122 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f70f4162590 0x7f70f41607e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:54.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.743+0000 7f70f2ffd640 1 -- 172.21.15.112:0/2828190122 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70f40c82f0 con 0x7f70f40b2e60 2024-09-21T00:25:54.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.743+0000 7f70f2ffd640 1 --2- 172.21.15.112:0/2828190122 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f70f40b2e60 0x7f70f41f5610 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f70e400cce0 tx=0x7f70e40075f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:25:54.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.744+0000 7f70db7fe640 1 -- 172.21.15.112:0/2828190122 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70e4004b20 con 0x7f70f40b2e60 2024-09-21T00:25:54.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.744+0000 7f70db7fe640 1 -- 172.21.15.112:0/2828190122 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f70e4004db0 con 0x7f70f40b2e60 2024-09-21T00:25:54.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.744+0000 7f70f92b6640 1 -- 172.21.15.112:0/2828190122 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70f415a800 con 0x7f70f40b2e60 2024-09-21T00:25:54.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.744+0000 7f70db7fe640 1 -- 172.21.15.112:0/2828190122 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70e4005280 con 0x7f70f40b2e60 2024-09-21T00:25:54.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.744+0000 7f70f92b6640 1 -- 172.21.15.112:0/2828190122 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70f4148a90 con 0x7f70f40b2e60 2024-09-21T00:25:54.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.745+0000 7f70f92b6640 1 -- 172.21.15.112:0/2828190122 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70f40c7ef0 con 0x7f70f40b2e60 2024-09-21T00:25:54.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.745+0000 7f70db7fe640 1 -- 172.21.15.112:0/2828190122 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f70e4005420 con 0x7f70f40b2e60 2024-09-21T00:25:54.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.746+0000 7f70db7fe640 1 --2- 172.21.15.112:0/2828190122 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f70e0078b90 0x7f70e007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:54.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.746+0000 7f70db7fe640 1 -- 172.21.15.112:0/2828190122 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f70e409b8e0 con 0x7f70f40b2e60 2024-09-21T00:25:54.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.746+0000 7f70f27fc640 1 --2- 172.21.15.112:0/2828190122 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f70e0078b90 0x7f70e007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:54.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.746+0000 7f70f27fc640 1 --2- 172.21.15.112:0/2828190122 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f70e0078b90 0x7f70e007b0a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f70e8033000 tx=0x7f70e8004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:25:54.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.750+0000 7f70db7fe640 1 -- 172.21.15.112:0/2828190122 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f70e4014030 con 0x7f70f40b2e60 2024-09-21T00:25:54.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.930+0000 7f70f92b6640 1 -- 172.21.15.112:0/2828190122 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f70f415a620 con 0x7f70f40b2e60 2024-09-21T00:25:54.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.931+0000 7f70db7fe640 1 -- 172.21.15.112:0/2828190122 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18622 (secure 0 0 0) 0x7f70f415a620 con 0x7f70f40b2e60 2024-09-21T00:25:54.933 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:25:54.934 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":26,"fsid":"8b34692b-dc65-49b2-a08f-26f707b96ac5","created":"2024-09-21T00:25:22.788490+0000","modified":"2024-09-21T00:25:52.623213+0000","last_up_change":"2024-09-21T00:25:36.161835+0000","last_in_change":"2024-09-21T00:25:26.346500+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T00:25:37.105006+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T00:25:45.413895+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T00:25:46.532124+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T00:25:50.244145+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9c575086-67d2-4de2-b01b-d44dcc21d604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6809","nonce":755760621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6811","nonce":755760621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6815","nonce":755760621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6813","nonce":755760621}]},"public_addr":"172.21.15.112:6809/755760621","cluster_addr":"172.21.15.112:6811/755760621","heartbeat_back_addr":"172.21.15.112:6815/755760621","heartbeat_front_addr":"172.21.15.112:6813/755760621","state":["exists","up"]},{"osd":1,"uuid":"a983c227-a3ce-4bdd-82f1-438ba7e3ecd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6801","nonce":344932592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6803","nonce":344932592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6807","nonce":344932592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6805","nonce":344932592}]},"public_addr":"172.21.15.112:6801/344932592","cluster_addr":"172.21.15.112:6803/344932592","heartbeat_back_addr":"172.21.15.112:6807/344932592","heartbeat_front_addr":"172.21.15.112:6805/344932592","state":["exists","up"]},{"osd":2,"uuid":"1193c1d8-bfff-4a57-8200-5f69bf1ea3af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6825","nonce":407933616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6827","nonce":407933616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6831","nonce":407933616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6829","nonce":407933616}]},"public_addr":"172.21.15.112:6825/407933616","cluster_addr":"172.21.15.112:6827/407933616","heartbeat_back_addr":"172.21.15.112:6831/407933616","heartbeat_front_addr":"172.21.15.112:6829/407933616","state":["exists","up"]},{"osd":3,"uuid":"af04330a-36ad-40cc-a8dd-fa259b221de5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6817","nonce":4120164806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6819","nonce":4120164806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6823","nonce":4120164806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6821","nonce":4120164806}]},"public_addr":"172.21.15.112:6817/4120164806","cluster_addr":"172.21.15.112:6819/4120164806","heartbeat_back_addr":"172.21.15.112:6823/4120164806","heartbeat_front_addr":"172.21.15.112:6821/4120164806","state":["exists","up"]},{"osd":4,"uuid":"5b893111-292c-4c5a-b184-53ad38c7199e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6803","nonce":522077560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6805","nonce":522077560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6809","nonce":522077560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6807","nonce":522077560}]},"public_addr":"172.21.15.187:6803/522077560","cluster_addr":"172.21.15.187:6805/522077560","heartbeat_back_addr":"172.21.15.187:6809/522077560","heartbeat_front_addr":"172.21.15.187:6807/522077560","state":["exists","up"]},{"osd":5,"uuid":"c0b8e011-45fe-4c5e-925e-6be2a7f1868a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6827","nonce":3593144721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6829","nonce":3593144721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6833","nonce":3593144721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6831","nonce":3593144721}]},"public_addr":"172.21.15.187:6827/3593144721","cluster_addr":"172.21.15.187:6829/3593144721","heartbeat_back_addr":"172.21.15.187:6833/3593144721","heartbeat_front_addr":"172.21.15.187:6831/3593144721","state":["exists","up"]},{"osd":6,"uuid":"87d2d650-7489-4593-98a3-a1f433ce8dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6811","nonce":1494387657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6813","nonce":1494387657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6817","nonce":1494387657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6815","nonce":1494387657}]},"public_addr":"172.21.15.187:6811/1494387657","cluster_addr":"172.21.15.187:6813/1494387657","heartbeat_back_addr":"172.21.15.187:6817/1494387657","heartbeat_front_addr":"172.21.15.187:6815/1494387657","state":["exists","up"]},{"osd":7,"uuid":"c6b4f8e2-a1ee-4438-aa37-67ff4b6ef1f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6819","nonce":1624699842}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6821","nonce":1624699842}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6825","nonce":1624699842}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6823","nonce":1624699842}]},"public_addr":"172.21.15.187:6819/1624699842","cluster_addr":"172.21.15.187:6821/1624699842","heartbeat_back_addr":"172.21.15.187:6825/1624699842","heartbeat_front_addr":"172.21.15.187:6823/1624699842","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.873162+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.400506+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.664317+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.159791+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.202630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.113212+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.384786+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.448441+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T00:25:54.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.933+0000 7f70f92b6640 1 -- 172.21.15.112:0/2828190122 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f70e0078b90 msgr2=0x7f70e007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:54.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.934+0000 7f70f92b6640 1 --2- 172.21.15.112:0/2828190122 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f70e0078b90 0x7f70e007b0a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f70e8033000 tx=0x7f70e8004730 comp rx=0 tx=0).stop 2024-09-21T00:25:54.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.934+0000 7f70f92b6640 1 -- 172.21.15.112:0/2828190122 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f70f40b2e60 msgr2=0x7f70f41f5610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:54.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.934+0000 7f70f92b6640 1 --2- 172.21.15.112:0/2828190122 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f70f40b2e60 0x7f70f41f5610 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f70e400cce0 tx=0x7f70e40075f0 comp rx=0 tx=0).stop 2024-09-21T00:25:54.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.934+0000 7f70f92b6640 1 -- 172.21.15.112:0/2828190122 shutdown_connections 2024-09-21T00:25:54.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.934+0000 7f70f92b6640 1 --2- 172.21.15.112:0/2828190122 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f70f4162590 0x7f70f41607e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:54.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.934+0000 7f70f92b6640 1 --2- 172.21.15.112:0/2828190122 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f70f40cfe00 0x7f70f41f9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:54.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.934+0000 7f70f92b6640 1 --2- 172.21.15.112:0/2828190122 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f70e0078b90 0x7f70e007b0a0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:54.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.934+0000 7f70f92b6640 1 --2- 172.21.15.112:0/2828190122 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f70f40b2e60 0x7f70f41f5610 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:54.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.934+0000 7f70f92b6640 1 -- 172.21.15.112:0/2828190122 >> 172.21.15.112:0/2828190122 conn(0x7f70f40d4440 msgr2=0x7f70f40d4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:54.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.934+0000 7f70f92b6640 1 -- 172.21.15.112:0/2828190122 shutdown_connections 2024-09-21T00:25:54.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:54.934+0000 7f70f92b6640 1 -- 172.21.15.112:0/2828190122 wait complete. 2024-09-21T00:25:55.349 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T00:25:47.578960+0000', 'modified': '2024-09-21T00:25:52.623390+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4425}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4425': {'gid': 4425, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.187:6836/2921304789', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.187:6834', 'nonce': 2921304789}, {'type': 'v1', 'addr': '172.21.15.187:6836', 'nonce': 2921304789}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 4425, 'qdb_cluster': [4425]} 2024-09-21T00:25:55.349 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-21T00:25:55.349 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-09-21T00:25:55.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.463+0000 7f48630f9640 1 -- 172.21.15.112:0/2701560953 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f485c0b9ad0 msgr2=0x7f485c0d0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:55.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.463+0000 7f48630f9640 1 --2- 172.21.15.112:0/2701560953 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f485c0b9ad0 0x7f485c0d0be0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f485c1623b0 tx=0x7f485002a4b0 comp rx=0 tx=0).stop 2024-09-21T00:25:55.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.463+0000 7f48630f9640 1 -- 172.21.15.112:0/2701560953 shutdown_connections 2024-09-21T00:25:55.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.463+0000 7f48630f9640 1 --2- 172.21.15.112:0/2701560953 >> v2:172.21.15.187:3300/0 conn(0x7f485c0d1120 0x7f485c0ceb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:55.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.463+0000 7f48630f9640 1 --2- 172.21.15.112:0/2701560953 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f485c0b9ad0 0x7f485c0d0be0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:55.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.463+0000 7f48630f9640 1 --2- 172.21.15.112:0/2701560953 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f485c15ee80 0x7f485c0d0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:55.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.463+0000 7f48630f9640 1 -- 172.21.15.112:0/2701560953 >> 172.21.15.112:0/2701560953 conn(0x7f485c09d830 msgr2=0x7f485c0b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:55.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.463+0000 7f48630f9640 1 -- 172.21.15.112:0/2701560953 shutdown_connections 2024-09-21T00:25:55.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.463+0000 7f48630f9640 1 -- 172.21.15.112:0/2701560953 wait complete. 2024-09-21T00:25:55.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.465+0000 7f48630f9640 1 Processor -- start 2024-09-21T00:25:55.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.465+0000 7f48630f9640 1 -- start start 2024-09-21T00:25:55.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.466+0000 7f48630f9640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f485c0b9ad0 0x7f485c1589e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:55.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.466+0000 7f48630f9640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f485c0d1120 0x7f485c0cd030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:55.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.466+0000 7f48630f9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f485c15ee80 0x7f485c0d33e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:55.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.466+0000 7f48630f9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f485c0cf460 con 0x7f485c15ee80 2024-09-21T00:25:55.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.466+0000 7f48630f9640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f485c0cf2e0 con 0x7f485c0d1120 2024-09-21T00:25:55.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.466+0000 7f48630f9640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f485c0b6580 con 0x7f485c0b9ad0 2024-09-21T00:25:55.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.467+0000 7f4861696640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f485c15ee80 0x7f485c0d33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:55.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.467+0000 7f4861696640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f485c15ee80 0x7f485c0d33e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59204/0 (socket says 172.21.15.112:59204) 2024-09-21T00:25:55.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.467+0000 7f4861696640 1 -- 172.21.15.112:0/2264027695 learned_addr learned my addr 172.21.15.112:0/2264027695 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:25:55.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.467+0000 7f4860e95640 1 --2- 172.21.15.112:0/2264027695 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f485c0b9ad0 0x7f485c1589e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:55.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.467+0000 7f485bfff640 1 --2- 172.21.15.112:0/2264027695 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f485c0d1120 0x7f485c0cd030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:55.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.467+0000 7f4861696640 1 -- 172.21.15.112:0/2264027695 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f485c0b9ad0 msgr2=0x7f485c1589e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:55.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.467+0000 7f4861696640 1 --2- 172.21.15.112:0/2264027695 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f485c0b9ad0 0x7f485c1589e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:55.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.467+0000 7f4861696640 1 -- 172.21.15.112:0/2264027695 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f485c0d1120 msgr2=0x7f485c0cd030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:55.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.467+0000 7f4861696640 1 --2- 172.21.15.112:0/2264027695 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f485c0d1120 0x7f485c0cd030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:55.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.467+0000 7f4861696640 1 -- 172.21.15.112:0/2264027695 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f485c09e1a0 con 0x7f485c15ee80 2024-09-21T00:25:55.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.467+0000 7f4861696640 1 --2- 172.21.15.112:0/2264027695 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f485c15ee80 0x7f485c0d33e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f485400c770 tx=0x7f485400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:25:55.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.467+0000 7f48597fa640 1 -- 172.21.15.112:0/2264027695 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4854004230 con 0x7f485c15ee80 2024-09-21T00:25:55.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.467+0000 7f48597fa640 1 -- 172.21.15.112:0/2264027695 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f48540079b0 con 0x7f485c15ee80 2024-09-21T00:25:55.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.467+0000 7f48630f9640 1 -- 172.21.15.112:0/2264027695 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f485c0cc500 con 0x7f485c15ee80 2024-09-21T00:25:55.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.467+0000 7f48597fa640 1 -- 172.21.15.112:0/2264027695 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4854007e00 con 0x7f485c15ee80 2024-09-21T00:25:55.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.467+0000 7f48630f9640 1 -- 172.21.15.112:0/2264027695 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f485c0cbc60 con 0x7f485c15ee80 2024-09-21T00:25:55.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.469+0000 7f48597fa640 1 -- 172.21.15.112:0/2264027695 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f48540035f0 con 0x7f485c15ee80 2024-09-21T00:25:55.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.469+0000 7f48597fa640 1 --2- 172.21.15.112:0/2264027695 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4848078b90 0x7f484807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:55.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.470+0000 7f48597fa640 1 -- 172.21.15.112:0/2264027695 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f485409b4b0 con 0x7f485c15ee80 2024-09-21T00:25:55.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.470+0000 7f4860e95640 1 --2- 172.21.15.112:0/2264027695 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4848078b90 0x7f484807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:55.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.470+0000 7f486a06c740 1 Processor -- start 2024-09-21T00:25:55.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.470+0000 7f486a06c740 1 -- start start 2024-09-21T00:25:55.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.470+0000 7f4860e95640 1 --2- 172.21.15.112:0/2264027695 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4848078b90 0x7f484807b0a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f484c005c40 tx=0x7f484c00a550 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:25:55.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.471+0000 7f486a06c740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a4c0865270 0x55a4c0c36fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:55.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.471+0000 7f486a06c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a4c083c3b0 0x55a4c0c37500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:55.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.471+0000 7f486a06c740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a4c0844590 0x55a4c0c54400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:55.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.471+0000 7f486a06c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a4c0b7f820 con 0x55a4c083c3b0 2024-09-21T00:25:55.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.471+0000 7f486a06c740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55a4c0b6ec50 con 0x55a4c0844590 2024-09-21T00:25:55.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.471+0000 7f486a06c740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x55a4c0b72a60 con 0x55a4c0865270 2024-09-21T00:25:55.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.472+0000 7f485bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a4c083c3b0 0x55a4c0c37500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:55.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.472+0000 7f485bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a4c083c3b0 0x55a4c0c37500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59220/0 (socket says 172.21.15.112:59220) 2024-09-21T00:25:55.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.472+0000 7f485bfff640 1 -- 172.21.15.112:0/3495819963 learned_addr learned my addr 172.21.15.112:0/3495819963 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:25:55.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.472+0000 7f4860e95640 1 --2- 172.21.15.112:0/3495819963 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a4c0865270 0x55a4c0c36fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:55.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.472+0000 7f4861696640 1 --2- 172.21.15.112:0/3495819963 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a4c0844590 0x55a4c0c54400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:55.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.472+0000 7f485bfff640 1 -- 172.21.15.112:0/3495819963 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a4c0865270 msgr2=0x55a4c0c36fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:25:55.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.472+0000 7f485bfff640 1 --2- 172.21.15.112:0/3495819963 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a4c0865270 0x55a4c0c36fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:55.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.472+0000 7f485bfff640 1 -- 172.21.15.112:0/3495819963 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a4c0844590 msgr2=0x55a4c0c54400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:25:55.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.472+0000 7f485bfff640 1 --2- 172.21.15.112:0/3495819963 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a4c0844590 0x55a4c0c54400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:55.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.472+0000 7f485bfff640 1 -- 172.21.15.112:0/3495819963 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4c08d0850 con 0x55a4c083c3b0 2024-09-21T00:25:55.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.472+0000 7f485bfff640 1 --2- 172.21.15.112:0/3495819963 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a4c083c3b0 0x55a4c0c37500 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f48500097b0 tx=0x7f48500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c002b79d64c1bfd3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:25:55.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.472+0000 7f48427fc640 1 -- 172.21.15.112:0/3495819963 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4850004030 con 0x55a4c083c3b0 2024-09-21T00:25:55.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.472+0000 7f48427fc640 1 -- 172.21.15.112:0/3495819963 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f48500042c0 con 0x55a4c083c3b0 2024-09-21T00:25:55.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.472+0000 7f48427fc640 1 -- 172.21.15.112:0/3495819963 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4850004a00 con 0x55a4c083c3b0 2024-09-21T00:25:55.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.473+0000 7f486a06c740 1 -- 172.21.15.112:0/3495819963 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a4c083c3b0 msgr2=0x55a4c0c37500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:55.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.473+0000 7f486a06c740 1 --2- 172.21.15.112:0/3495819963 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a4c083c3b0 0x55a4c0c37500 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f48500097b0 tx=0x7f48500046c0 comp rx=0 tx=0).stop 2024-09-21T00:25:55.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.473+0000 7f486a06c740 1 -- 172.21.15.112:0/3495819963 shutdown_connections 2024-09-21T00:25:55.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.473+0000 7f486a06c740 1 --2- 172.21.15.112:0/3495819963 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a4c0844590 0x55a4c0c54400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:55.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.473+0000 7f486a06c740 1 --2- 172.21.15.112:0/3495819963 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a4c083c3b0 0x55a4c0c37500 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:55.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.473+0000 7f486a06c740 1 --2- 172.21.15.112:0/3495819963 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a4c0865270 0x55a4c0c36fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:55.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.473+0000 7f486a06c740 1 -- 172.21.15.112:0/3495819963 >> 172.21.15.112:0/3495819963 conn(0x55a4c08593c0 msgr2=0x55a4c0c0d280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:55.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.473+0000 7f486a06c740 1 -- 172.21.15.112:0/3495819963 shutdown_connections 2024-09-21T00:25:55.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.473+0000 7f486a06c740 1 -- 172.21.15.112:0/3495819963 wait complete. 2024-09-21T00:25:55.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.474+0000 7f486a06c740 1 -- start start 2024-09-21T00:25:55.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.474+0000 7f486a06c740 1 Processor -- start 2024-09-21T00:25:55.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.475+0000 7f486a06c740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a4c083c3b0 0x55a4c0c48d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:55.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.475+0000 7f486a06c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a4c0844590 0x55a4c0c229b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:55.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.475+0000 7f486a06c740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a4c0865270 0x55a4c0c556f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:55.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.475+0000 7f486a06c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a4c08df620 con 0x55a4c0844590 2024-09-21T00:25:55.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.476+0000 7f486a06c740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55a4c0b145f0 con 0x55a4c083c3b0 2024-09-21T00:25:55.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.476+0000 7f486a06c740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x55a4c0c37a40 con 0x55a4c0865270 2024-09-21T00:25:55.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.476+0000 7f486a06c740 10 client.0 _ll_register_callbacks cb 0x55a4c0b180a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:25:55.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.476+0000 7f485bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a4c0844590 0x55a4c0c229b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:55.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.476+0000 7f485bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a4c0844590 0x55a4c0c229b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59228/0 (socket says 172.21.15.112:59228) 2024-09-21T00:25:55.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.476+0000 7f485bfff640 1 -- 172.21.15.112:0/2553802334 learned_addr learned my addr 172.21.15.112:0/2553802334 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:25:55.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.476+0000 7f4861696640 1 --2- 172.21.15.112:0/2553802334 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a4c0865270 0x55a4c0c556f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:55.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.476+0000 7f4860e95640 1 --2- 172.21.15.112:0/2553802334 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a4c083c3b0 0x55a4c0c48d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:55.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.476+0000 7f485bfff640 1 -- 172.21.15.112:0/2553802334 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a4c0865270 msgr2=0x55a4c0c556f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:25:55.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.476+0000 7f485bfff640 1 --2- 172.21.15.112:0/2553802334 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a4c0865270 0x55a4c0c556f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:55.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.476+0000 7f485bfff640 1 -- 172.21.15.112:0/2553802334 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a4c083c3b0 msgr2=0x55a4c0c48d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:25:55.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.476+0000 7f485bfff640 1 --2- 172.21.15.112:0/2553802334 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a4c083c3b0 0x55a4c0c48d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:55.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.476+0000 7f485bfff640 1 -- 172.21.15.112:0/2553802334 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4c09476b0 con 0x55a4c0844590 2024-09-21T00:25:55.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.476+0000 7f485bfff640 1 --2- 172.21.15.112:0/2553802334 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a4c0844590 0x55a4c0c229b0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f485002aaa0 tx=0x7f4850005080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2564b0cd5595b29 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:25:55.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.476+0000 7f483b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T00:25:55.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.476+0000 7f483b7fe640 1 -- 172.21.15.112:0/2553802334 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48500049b0 con 0x55a4c0844590 2024-09-21T00:25:55.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.476+0000 7f486a06c740 1 -- 172.21.15.112:0/2553802334 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f48500042c0 con 0x55a4c0844590 2024-09-21T00:25:55.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.476+0000 7f483b7fe640 1 -- 172.21.15.112:0/2553802334 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f4850004b50 con 0x55a4c0844590 2024-09-21T00:25:55.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.476+0000 7f483b7fe640 1 -- 172.21.15.112:0/2553802334 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7f4850057050 con 0x55a4c0844590 2024-09-21T00:25:55.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.476+0000 7f486a06c740 10 client.4533 fetch_fsmap learned FSMap version 7 2024-09-21T00:25:55.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.476+0000 7f486a06c740 1 -- 172.21.15.112:0/2553802334 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x55a4c0c49e70 con 0x55a4c0844590 2024-09-21T00:25:55.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.477+0000 7f483b7fe640 1 -- 172.21.15.112:0/2553802334 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f48500042c0 con 0x55a4c0844590 2024-09-21T00:25:55.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.477+0000 7f483b7fe640 1 -- 172.21.15.112:0/2553802334 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 7) ==== 3510+0+0 (secure 0 0 0) 0x7f485005a020 con 0x55a4c0844590 2024-09-21T00:25:55.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.477+0000 7f486a06c740 10 client.4533 fetch_fsmap finished waiting for FSMap version 7 2024-09-21T00:25:55.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.477+0000 7f486a06c740 10 client.4533 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T00:25:55.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.477+0000 7f486a06c740 20 client.4533 populate_metadata read hostname 'smithi112' 2024-09-21T00:25:55.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.477+0000 7f486a06c740 1 --2- 172.21.15.112:0/2553802334 >> [v2:172.21.15.187:6834/2921304789,v1:172.21.15.187:6836/2921304789] conn(0x55a4c0823e40 0x55a4c0c55c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:55.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.477+0000 7f486a06c740 4 client.4533 mds_command: new command op to 4425 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T00:25:55.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.477+0000 7f486a06c740 1 -- 172.21.15.112:0/2553802334 --> [v2:172.21.15.187:6834/2921304789,v1:172.21.15.187:6836/2921304789] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a4c0b81770 con 0x55a4c0823e40 2024-09-21T00:25:55.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.477+0000 7f4860e95640 1 --2- 172.21.15.112:0/2553802334 >> [v2:172.21.15.187:6834/2921304789,v1:172.21.15.187:6836/2921304789] conn(0x55a4c0823e40 0x55a4c0c55c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:55.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.477+0000 7f483b7fe640 10 client.4533 ms_handle_connect on v2:172.21.15.187:6834/2921304789 2024-09-21T00:25:55.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.477+0000 7f4860e95640 1 --2- 172.21.15.112:0/2553802334 >> [v2:172.21.15.187:6834/2921304789,v1:172.21.15.187:6836/2921304789] conn(0x55a4c0823e40 0x55a4c0c55c30 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a9778b2728e0117 server_cookie=256011b6f6d66d89 in_seq=0 out_seq=0 2024-09-21T00:25:55.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.478+0000 7f483b7fe640 1 -- 172.21.15.112:0/2553802334 <== mds.0 v2:172.21.15.187:6834/2921304789 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a4c0b81770 con 0x55a4c0823e40 2024-09-21T00:25:55.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.478+0000 7f483b7fe640 10 client.4533 handle_command_reply: tid=0 2024-09-21T00:25:55.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.478+0000 7f486a06c740 1 client.4533 shutdown 2024-09-21T00:25:55.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.478+0000 7f486a06c740 1 -- 172.21.15.112:0/2553802334 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a4c0844590 msgr2=0x55a4c0c229b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:55.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.478+0000 7f486a06c740 1 --2- 172.21.15.112:0/2553802334 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a4c0844590 0x55a4c0c229b0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f485002aaa0 tx=0x7f4850005080 comp rx=0 tx=0).stop 2024-09-21T00:25:55.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.478+0000 7f486a06c740 1 -- 172.21.15.112:0/2553802334 shutdown_connections 2024-09-21T00:25:55.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.478+0000 7f486a06c740 1 --2- 172.21.15.112:0/2553802334 >> [v2:172.21.15.187:6834/2921304789,v1:172.21.15.187:6836/2921304789] conn(0x55a4c0823e40 0x55a4c0c55c30 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:55.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.478+0000 7f486a06c740 1 --2- 172.21.15.112:0/2553802334 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a4c0865270 0x55a4c0c556f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:55.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.478+0000 7f486a06c740 1 --2- 172.21.15.112:0/2553802334 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a4c0844590 0x55a4c0c229b0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:55.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.478+0000 7f486a06c740 1 --2- 172.21.15.112:0/2553802334 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a4c083c3b0 0x55a4c0c48d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:55.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.478+0000 7f483b7fe640 0 client.4533 ms_handle_reset on v2:172.21.15.187:6834/2921304789 2024-09-21T00:25:55.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.478+0000 7f486a06c740 1 -- 172.21.15.112:0/2553802334 >> 172.21.15.112:0/2553802334 conn(0x55a4c08593c0 msgr2=0x55a4c0c0d130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:55.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.479+0000 7f486a06c740 1 -- 172.21.15.112:0/2553802334 shutdown_connections 2024-09-21T00:25:55.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.479+0000 7f486a06c740 1 -- 172.21.15.112:0/2553802334 wait complete. 2024-09-21T00:25:55.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.479+0000 7f486a06c740 20 client.4533 trim_cache size 0 max 16384 2024-09-21T00:25:55.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.497+0000 7f486a06c740 1 Processor -- start 2024-09-21T00:25:55.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.497+0000 7f486a06c740 1 -- start start 2024-09-21T00:25:55.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.498+0000 7f486a06c740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a4c083c3b0 0x55a4c0c61d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:55.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.498+0000 7f486a06c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a4c0844590 0x55a4c0c74f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:55.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.498+0000 7f486a06c740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a4c0865270 0x55a4c0ccb740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:55.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.498+0000 7f486a06c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f485c202460 con 0x55a4c0844590 2024-09-21T00:25:55.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.498+0000 7f486a06c740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f485c201840 con 0x55a4c0865270 2024-09-21T00:25:55.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.498+0000 7f486a06c740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f485c201e90 con 0x55a4c083c3b0 2024-09-21T00:25:55.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.498+0000 7f485bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a4c0844590 0x55a4c0c74f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:55.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.498+0000 7f485bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a4c0844590 0x55a4c0c74f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59230/0 (socket says 172.21.15.112:59230) 2024-09-21T00:25:55.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.498+0000 7f485bfff640 1 -- 172.21.15.112:0/247520270 learned_addr learned my addr 172.21.15.112:0/247520270 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:25:55.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.498+0000 7f4860e95640 1 --2- 172.21.15.112:0/247520270 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a4c083c3b0 0x55a4c0c61d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:55.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.498+0000 7f485bfff640 1 -- 172.21.15.112:0/247520270 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a4c083c3b0 msgr2=0x55a4c0c61d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:25:55.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.498+0000 7f485bfff640 1 --2- 172.21.15.112:0/247520270 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a4c083c3b0 0x55a4c0c61d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:55.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.498+0000 7f4861696640 1 --2- 172.21.15.112:0/247520270 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a4c0865270 0x55a4c0ccb740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:55.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.498+0000 7f485bfff640 1 -- 172.21.15.112:0/247520270 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a4c0865270 msgr2=0x55a4c0ccb740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:25:55.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.498+0000 7f485bfff640 1 --2- 172.21.15.112:0/247520270 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a4c0865270 0x55a4c0ccb740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:55.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.498+0000 7f485bfff640 1 -- 172.21.15.112:0/247520270 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4c0c4c190 con 0x55a4c0844590 2024-09-21T00:25:55.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.498+0000 7f4861696640 1 --2- 172.21.15.112:0/247520270 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a4c0865270 0x55a4c0ccb740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:25:55.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.499+0000 7f485bfff640 1 --2- 172.21.15.112:0/247520270 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a4c0844590 0x55a4c0c74f40 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f4850005f10 tx=0x7f4850005820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe24b4953f088a8a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:25:55.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.499+0000 7f4841ffb640 1 -- 172.21.15.112:0/247520270 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48500049b0 con 0x55a4c0844590 2024-09-21T00:25:55.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.499+0000 7f4841ffb640 1 -- 172.21.15.112:0/247520270 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f48500035f0 con 0x55a4c0844590 2024-09-21T00:25:55.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.499+0000 7f4841ffb640 1 -- 172.21.15.112:0/247520270 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4850003a20 con 0x55a4c0844590 2024-09-21T00:25:55.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.499+0000 7f486a06c740 1 -- 172.21.15.112:0/247520270 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a4c0844590 msgr2=0x55a4c0c74f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:55.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.499+0000 7f486a06c740 1 --2- 172.21.15.112:0/247520270 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a4c0844590 0x55a4c0c74f40 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f4850005f10 tx=0x7f4850005820 comp rx=0 tx=0).stop 2024-09-21T00:25:55.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.499+0000 7f486a06c740 1 -- 172.21.15.112:0/247520270 shutdown_connections 2024-09-21T00:25:55.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.499+0000 7f486a06c740 1 --2- 172.21.15.112:0/247520270 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a4c0865270 0x55a4c0ccb740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:55.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.499+0000 7f486a06c740 1 --2- 172.21.15.112:0/247520270 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a4c0844590 0x55a4c0c74f40 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:55.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.499+0000 7f486a06c740 1 --2- 172.21.15.112:0/247520270 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a4c083c3b0 0x55a4c0c61d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:55.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.499+0000 7f486a06c740 1 -- 172.21.15.112:0/247520270 >> 172.21.15.112:0/247520270 conn(0x55a4c08593c0 msgr2=0x55a4c0c47db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:55.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.499+0000 7f486a06c740 1 -- 172.21.15.112:0/247520270 shutdown_connections 2024-09-21T00:25:55.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.499+0000 7f486a06c740 1 -- 172.21.15.112:0/247520270 wait complete. 2024-09-21T00:25:55.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.500+0000 7f486a06c740 1 -- start start 2024-09-21T00:25:55.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.500+0000 7f486a06c740 1 Processor -- start 2024-09-21T00:25:55.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.502+0000 7f486a06c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a4c083c3b0 0x55a4c0c611d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:55.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.502+0000 7f486a06c740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a4c0844590 0x55a4c0cc97b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:55.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.502+0000 7f486a06c740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a4c0865270 0x55a4c0cb3290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:55.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.502+0000 7f486a06c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a4c08df620 con 0x55a4c083c3b0 2024-09-21T00:25:55.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.502+0000 7f486a06c740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55a4c0b145f0 con 0x55a4c0865270 2024-09-21T00:25:55.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.502+0000 7f486a06c740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x55a4c0c644b0 con 0x55a4c0844590 2024-09-21T00:25:55.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.502+0000 7f486a06c740 10 client.0 _ll_register_callbacks cb 0x7f484c00a3f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:25:55.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.502+0000 7f4860e95640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a4c083c3b0 0x55a4c0c611d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:55.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.502+0000 7f4860e95640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a4c083c3b0 0x55a4c0c611d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59234/0 (socket says 172.21.15.112:59234) 2024-09-21T00:25:55.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.502+0000 7f4860e95640 1 -- 172.21.15.112:0/1208397033 learned_addr learned my addr 172.21.15.112:0/1208397033 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:25:55.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.502+0000 7f485bfff640 1 --2- 172.21.15.112:0/1208397033 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a4c0844590 0x55a4c0cc97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:55.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.502+0000 7f4861696640 1 --2- 172.21.15.112:0/1208397033 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a4c0865270 0x55a4c0cb3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:55.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.502+0000 7f4860e95640 1 -- 172.21.15.112:0/1208397033 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a4c0844590 msgr2=0x55a4c0cc97b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:25:55.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.502+0000 7f4860e95640 1 --2- 172.21.15.112:0/1208397033 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a4c0844590 0x55a4c0cc97b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:55.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.502+0000 7f4860e95640 1 -- 172.21.15.112:0/1208397033 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a4c0865270 msgr2=0x55a4c0cb3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:25:55.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.502+0000 7f4860e95640 1 --2- 172.21.15.112:0/1208397033 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a4c0865270 0x55a4c0cb3290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:55.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.502+0000 7f4860e95640 1 -- 172.21.15.112:0/1208397033 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4c0cc9cf0 con 0x55a4c083c3b0 2024-09-21T00:25:55.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.502+0000 7f4860e95640 1 --2- 172.21.15.112:0/1208397033 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a4c083c3b0 0x55a4c0c611d0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f484c00fc20 tx=0x7f484c012070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50aa423d711df79e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:25:55.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.502+0000 7f483b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T00:25:55.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.502+0000 7f483b7fe640 1 -- 172.21.15.112:0/1208397033 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f484c02b030 con 0x55a4c083c3b0 2024-09-21T00:25:55.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.502+0000 7f483b7fe640 1 -- 172.21.15.112:0/1208397033 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f484c004b20 con 0x55a4c083c3b0 2024-09-21T00:25:55.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.502+0000 7f486a06c740 1 -- 172.21.15.112:0/1208397033 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f48500035f0 con 0x55a4c083c3b0 2024-09-21T00:25:55.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.503+0000 7f483b7fe640 1 -- 172.21.15.112:0/1208397033 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7f484c036050 con 0x55a4c083c3b0 2024-09-21T00:25:55.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.503+0000 7f486a06c740 10 client.4539 fetch_fsmap learned FSMap version 7 2024-09-21T00:25:55.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.503+0000 7f486a06c740 1 -- 172.21.15.112:0/1208397033 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x55a4c0c4a3d0 con 0x55a4c083c3b0 2024-09-21T00:25:55.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.503+0000 7f483b7fe640 1 -- 172.21.15.112:0/1208397033 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f48500035f0 con 0x55a4c083c3b0 2024-09-21T00:25:55.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.503+0000 7f483b7fe640 1 -- 172.21.15.112:0/1208397033 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 7) ==== 3510+0+0 (secure 0 0 0) 0x7f484c039020 con 0x55a4c083c3b0 2024-09-21T00:25:55.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.503+0000 7f486a06c740 10 client.4539 fetch_fsmap finished waiting for FSMap version 7 2024-09-21T00:25:55.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.503+0000 7f486a06c740 10 client.4539 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T00:25:55.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.503+0000 7f486a06c740 20 client.4539 populate_metadata read hostname 'smithi112' 2024-09-21T00:25:55.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.503+0000 7f486a06c740 1 --2- 172.21.15.112:0/1208397033 >> [v2:172.21.15.187:6834/2921304789,v1:172.21.15.187:6836/2921304789] conn(0x55a4c0c229c0 0x55a4c0c73960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:55.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.503+0000 7f486a06c740 4 client.4539 mds_command: new command op to 4425 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T00:25:55.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.503+0000 7f486a06c740 1 -- 172.21.15.112:0/1208397033 --> [v2:172.21.15.187:6834/2921304789,v1:172.21.15.187:6836/2921304789] -- command(tid 0: {"prefix": "status"}) -- 0x55a4c0b7d220 con 0x55a4c0c229c0 2024-09-21T00:25:55.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.503+0000 7f485bfff640 1 --2- 172.21.15.112:0/1208397033 >> [v2:172.21.15.187:6834/2921304789,v1:172.21.15.187:6836/2921304789] conn(0x55a4c0c229c0 0x55a4c0c73960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:55.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.503+0000 7f483b7fe640 10 client.4539 ms_handle_connect on v2:172.21.15.187:6834/2921304789 2024-09-21T00:25:55.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.503+0000 7f485bfff640 1 --2- 172.21.15.112:0/1208397033 >> [v2:172.21.15.187:6834/2921304789,v1:172.21.15.187:6836/2921304789] conn(0x55a4c0c229c0 0x55a4c0c73960 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=914a727e981e34a4 server_cookie=11c5413d1368aed in_seq=0 out_seq=0 2024-09-21T00:25:55.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.504+0000 7f483b7fe640 1 -- 172.21.15.112:0/1208397033 <== mds.0 v2:172.21.15.187:6834/2921304789 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x55a4c0b7d220 con 0x55a4c0c229c0 2024-09-21T00:25:55.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.504+0000 7f483b7fe640 10 client.4539 handle_command_reply: tid=0 2024-09-21T00:25:55.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.504+0000 7f486a06c740 1 client.4539 shutdown 2024-09-21T00:25:55.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.504+0000 7f486a06c740 1 -- 172.21.15.112:0/1208397033 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a4c083c3b0 msgr2=0x55a4c0c611d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:55.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.504+0000 7f486a06c740 1 --2- 172.21.15.112:0/1208397033 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a4c083c3b0 0x55a4c0c611d0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f484c00fc20 tx=0x7f484c012070 comp rx=0 tx=0).stop 2024-09-21T00:25:55.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.504+0000 7f486a06c740 1 -- 172.21.15.112:0/1208397033 shutdown_connections 2024-09-21T00:25:55.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.504+0000 7f486a06c740 1 --2- 172.21.15.112:0/1208397033 >> [v2:172.21.15.187:6834/2921304789,v1:172.21.15.187:6836/2921304789] conn(0x55a4c0c229c0 0x55a4c0c73960 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:55.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.504+0000 7f486a06c740 1 --2- 172.21.15.112:0/1208397033 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a4c0865270 0x55a4c0cb3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:55.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.504+0000 7f483b7fe640 0 client.4539 ms_handle_reset on v2:172.21.15.187:6834/2921304789 2024-09-21T00:25:55.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.504+0000 7f486a06c740 1 --2- 172.21.15.112:0/1208397033 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a4c0844590 0x55a4c0cc97b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:55.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.504+0000 7f486a06c740 1 --2- 172.21.15.112:0/1208397033 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a4c083c3b0 0x55a4c0c611d0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:55.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.504+0000 7f486a06c740 1 -- 172.21.15.112:0/1208397033 >> 172.21.15.112:0/1208397033 conn(0x55a4c08593c0 msgr2=0x55a4c0c74f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:55.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.504+0000 7f486a06c740 1 -- 172.21.15.112:0/1208397033 shutdown_connections 2024-09-21T00:25:55.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.504+0000 7f486a06c740 1 -- 172.21.15.112:0/1208397033 wait complete. 2024-09-21T00:25:55.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.504+0000 7f486a06c740 20 client.4539 trim_cache size 0 max 16384 2024-09-21T00:25:55.510 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-21T00:25:55.510 INFO:teuthology.orchestra.run.smithi112.stdout: "cluster_fsid": "8b34692b-dc65-49b2-a08f-26f707b96ac5", 2024-09-21T00:25:55.510 INFO:teuthology.orchestra.run.smithi112.stdout: "whoami": 0, 2024-09-21T00:25:55.510 INFO:teuthology.orchestra.run.smithi112.stdout: "id": 4425, 2024-09-21T00:25:55.511 INFO:teuthology.orchestra.run.smithi112.stdout: "want_state": "up:active", 2024-09-21T00:25:55.511 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "up:active", 2024-09-21T00:25:55.511 INFO:teuthology.orchestra.run.smithi112.stdout: "fs_name": "cephfs", 2024-09-21T00:25:55.511 INFO:teuthology.orchestra.run.smithi112.stdout: "rank_uptime": 7.2948923409999997, 2024-09-21T00:25:55.511 INFO:teuthology.orchestra.run.smithi112.stdout: "mdsmap_epoch": 7, 2024-09-21T00:25:55.511 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch": 26, 2024-09-21T00:25:55.511 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch_barrier": 25, 2024-09-21T00:25:55.511 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 13.345284783 2024-09-21T00:25:55.511 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-21T00:25:55.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.504+0000 7f48630f9640 1 -- 172.21.15.112:0/2264027695 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4848078b90 msgr2=0x7f484807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:55.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.504+0000 7f48630f9640 1 --2- 172.21.15.112:0/2264027695 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4848078b90 0x7f484807b0a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f484c005c40 tx=0x7f484c00a550 comp rx=0 tx=0).stop 2024-09-21T00:25:55.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.505+0000 7f48630f9640 1 -- 172.21.15.112:0/2264027695 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f485c15ee80 msgr2=0x7f485c0d33e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:55.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.505+0000 7f48630f9640 1 --2- 172.21.15.112:0/2264027695 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f485c15ee80 0x7f485c0d33e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f485400c770 tx=0x7f485400cc30 comp rx=0 tx=0).stop 2024-09-21T00:25:55.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.505+0000 7f48630f9640 1 -- 172.21.15.112:0/2264027695 shutdown_connections 2024-09-21T00:25:55.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.505+0000 7f48630f9640 1 --2- 172.21.15.112:0/2264027695 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4848078b90 0x7f484807b0a0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:55.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.505+0000 7f48630f9640 1 --2- 172.21.15.112:0/2264027695 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f485c15ee80 0x7f485c0d33e0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:55.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.505+0000 7f48630f9640 1 --2- 172.21.15.112:0/2264027695 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f485c0d1120 0x7f485c0cd030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:55.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.505+0000 7f48630f9640 1 --2- 172.21.15.112:0/2264027695 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f485c0b9ad0 0x7f485c1589e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:55.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.505+0000 7f48630f9640 1 -- 172.21.15.112:0/2264027695 >> 172.21.15.112:0/2264027695 conn(0x7f485c09d830 msgr2=0x7f485c0d09e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:55.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.505+0000 7f48630f9640 1 -- 172.21.15.112:0/2264027695 shutdown_connections 2024-09-21T00:25:55.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.505+0000 7f48630f9640 1 -- 172.21.15.112:0/2264027695 wait complete. 2024-09-21T00:25:55.517 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-21T00:25:55.517 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-09-21T00:25:55.518 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T00:25:55.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.628+0000 7f5a201f9640 1 -- 172.21.15.112:0/3561412847 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a180cfe00 msgr2=0x7f5a180d2a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:55.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.628+0000 7f5a201f9640 1 --2- 172.21.15.112:0/3561412847 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a180cfe00 0x7f5a180d2a20 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f5a14005fd0 tx=0x7f5a1402aa90 comp rx=0 tx=0).stop 2024-09-21T00:25:55.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.628+0000 7f5a201f9640 1 -- 172.21.15.112:0/3561412847 shutdown_connections 2024-09-21T00:25:55.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.628+0000 7f5a201f9640 1 --2- 172.21.15.112:0/3561412847 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5a180b2e60 0x7f5a180b3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:55.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.628+0000 7f5a201f9640 1 --2- 172.21.15.112:0/3561412847 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a180cfe00 0x7f5a180d2a20 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:55.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.628+0000 7f5a201f9640 1 -- 172.21.15.112:0/3561412847 >> 172.21.15.112:0/3561412847 conn(0x7f5a180d4440 msgr2=0x7f5a180d4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:55.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.628+0000 7f5a201f9640 1 -- 172.21.15.112:0/3561412847 shutdown_connections 2024-09-21T00:25:55.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.628+0000 7f5a201f9640 1 -- 172.21.15.112:0/3561412847 wait complete. 2024-09-21T00:25:55.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.629+0000 7f5a201f9640 1 Processor -- start 2024-09-21T00:25:55.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.629+0000 7f5a201f9640 1 -- start start 2024-09-21T00:25:55.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.631+0000 7f5a201f9640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5a180b2e60 0x7f5a181595d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:55.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.631+0000 7f5a201f9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a180cfe00 0x7f5a1815e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:55.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.631+0000 7f5a201f9640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5a18162590 0x7f5a18157440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:55.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.631+0000 7f5a201f9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5a180b3930 con 0x7f5a180cfe00 2024-09-21T00:25:55.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.631+0000 7f5a201f9640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5a180b37b0 con 0x7f5a180b2e60 2024-09-21T00:25:55.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.631+0000 7f5a201f9640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f5a180b3ab0 con 0x7f5a18162590 2024-09-21T00:25:55.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.631+0000 7f5a1d794640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a180cfe00 0x7f5a1815e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:55.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.631+0000 7f5a1d794640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a180cfe00 0x7f5a1815e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59244/0 (socket says 172.21.15.112:59244) 2024-09-21T00:25:55.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.631+0000 7f5a1d794640 1 -- 172.21.15.112:0/1852819157 learned_addr learned my addr 172.21.15.112:0/1852819157 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:25:55.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.631+0000 7f5a1e796640 1 --2- 172.21.15.112:0/1852819157 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5a18162590 0x7f5a18157440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:55.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.631+0000 7f5a1df95640 1 --2- 172.21.15.112:0/1852819157 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5a180b2e60 0x7f5a181595d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:55.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.631+0000 7f5a1d794640 1 -- 172.21.15.112:0/1852819157 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5a18162590 msgr2=0x7f5a18157440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:55.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.631+0000 7f5a1d794640 1 --2- 172.21.15.112:0/1852819157 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5a18162590 0x7f5a18157440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:55.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.631+0000 7f5a1d794640 1 -- 172.21.15.112:0/1852819157 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5a180b2e60 msgr2=0x7f5a181595d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:55.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.631+0000 7f5a1d794640 1 --2- 172.21.15.112:0/1852819157 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5a180b2e60 0x7f5a181595d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:55.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.631+0000 7f5a1d794640 1 -- 172.21.15.112:0/1852819157 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a181e9500 con 0x7f5a180cfe00 2024-09-21T00:25:55.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.631+0000 7f5a1d794640 1 --2- 172.21.15.112:0/1852819157 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a180cfe00 0x7f5a1815e5d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f5a0800c3d0 tx=0x7f5a0800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:25:55.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.631+0000 7f5a0e7fc640 1 -- 172.21.15.112:0/1852819157 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a08017070 con 0x7f5a180cfe00 2024-09-21T00:25:55.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.631+0000 7f5a0e7fc640 1 -- 172.21.15.112:0/1852819157 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5a080053c0 con 0x7f5a180cfe00 2024-09-21T00:25:55.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.631+0000 7f5a201f9640 1 -- 172.21.15.112:0/1852819157 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a181453f0 con 0x7f5a180cfe00 2024-09-21T00:25:55.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.631+0000 7f5a0e7fc640 1 -- 172.21.15.112:0/1852819157 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a08003850 con 0x7f5a180cfe00 2024-09-21T00:25:55.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.632+0000 7f5a201f9640 1 -- 172.21.15.112:0/1852819157 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a181427f0 con 0x7f5a180cfe00 2024-09-21T00:25:55.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.632+0000 7f5a201f9640 1 -- 172.21.15.112:0/1852819157 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a180b3270 con 0x7f5a180cfe00 2024-09-21T00:25:55.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.633+0000 7f5a0e7fc640 1 -- 172.21.15.112:0/1852819157 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5a0801a070 con 0x7f5a180cfe00 2024-09-21T00:25:55.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.633+0000 7f5a0e7fc640 1 --2- 172.21.15.112:0/1852819157 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f59fc078b90 0x7f59fc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:55.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.634+0000 7f5a0e7fc640 1 -- 172.21.15.112:0/1852819157 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f5a0809a3f0 con 0x7f5a180cfe00 2024-09-21T00:25:55.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.634+0000 7f5a1df95640 1 --2- 172.21.15.112:0/1852819157 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f59fc078b90 0x7f59fc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:55.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.634+0000 7f5a1df95640 1 --2- 172.21.15.112:0/1852819157 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f59fc078b90 0x7f59fc07b0a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5a14035000 tx=0x7f5a14038040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:25:55.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.638+0000 7f5a0e7fc640 1 -- 172.21.15.112:0/1852819157 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5a08062580 con 0x7f5a180cfe00 2024-09-21T00:25:55.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.850+0000 7f5a201f9640 1 -- 172.21.15.112:0/1852819157 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f5a18159230 con 0x7f5a180cfe00 2024-09-21T00:25:55.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.850+0000 7f5a0e7fc640 1 -- 172.21.15.112:0/1852819157 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+88 (secure 0 0 0) 0x7f5a08061e80 con 0x7f5a180cfe00 2024-09-21T00:25:55.852 INFO:teuthology.orchestra.run.smithi112.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-21T00:25:55.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.853+0000 7f5a201f9640 1 -- 172.21.15.112:0/1852819157 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f59fc078b90 msgr2=0x7f59fc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:55.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.853+0000 7f5a201f9640 1 --2- 172.21.15.112:0/1852819157 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f59fc078b90 0x7f59fc07b0a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5a14035000 tx=0x7f5a14038040 comp rx=0 tx=0).stop 2024-09-21T00:25:55.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.853+0000 7f5a201f9640 1 -- 172.21.15.112:0/1852819157 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a180cfe00 msgr2=0x7f5a1815e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:55.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.853+0000 7f5a201f9640 1 --2- 172.21.15.112:0/1852819157 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a180cfe00 0x7f5a1815e5d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f5a0800c3d0 tx=0x7f5a0800c890 comp rx=0 tx=0).stop 2024-09-21T00:25:55.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.853+0000 7f5a201f9640 1 -- 172.21.15.112:0/1852819157 shutdown_connections 2024-09-21T00:25:55.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.853+0000 7f5a201f9640 1 --2- 172.21.15.112:0/1852819157 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5a18162590 0x7f5a18157440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:55.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.853+0000 7f5a201f9640 1 --2- 172.21.15.112:0/1852819157 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f59fc078b90 0x7f59fc07b0a0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:55.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.853+0000 7f5a201f9640 1 --2- 172.21.15.112:0/1852819157 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a180cfe00 0x7f5a1815e5d0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:55.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.853+0000 7f5a201f9640 1 --2- 172.21.15.112:0/1852819157 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5a180b2e60 0x7f5a181595d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:55.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.853+0000 7f5a201f9640 1 -- 172.21.15.112:0/1852819157 >> 172.21.15.112:0/1852819157 conn(0x7f5a180d4440 msgr2=0x7f5a180b4b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:55.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.853+0000 7f5a201f9640 1 -- 172.21.15.112:0/1852819157 shutdown_connections 2024-09-21T00:25:55.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55.853+0000 7f5a201f9640 1 -- 172.21.15.112:0/1852819157 wait complete. 2024-09-21T00:25:55.865 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T00:25:55.865 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T00:25:55.865 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi112.front.sepia.ceph.com 2024-09-21T00:25:55.865 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-09-21T00:25:55.865 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-09-21T00:25:55.865 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T00:25:55.866 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-21T00:25:55.866 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T00:25:55.866 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-09-21T00:25:55.866 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:25:55.866 DEBUG:teuthology.orchestra.run.smithi112:> ip addr 2024-09-21T00:25:55.881 INFO:teuthology.orchestra.run.smithi112.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T00:25:55.881 INFO:teuthology.orchestra.run.smithi112.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T00:25:55.881 INFO:teuthology.orchestra.run.smithi112.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T00:25:55.881 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:25:55.882 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 ::1/128 scope host 2024-09-21T00:25:55.882 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:25:55.882 INFO:teuthology.orchestra.run.smithi112.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T00:25:55.882 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:d8:a5:2c brd ff:ff:ff:ff:ff:ff 2024-09-21T00:25:55.882 INFO:teuthology.orchestra.run.smithi112.stdout: altname enp7s0f0 2024-09-21T00:25:55.882 INFO:teuthology.orchestra.run.smithi112.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T00:25:55.882 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:d8:a5:2d brd ff:ff:ff:ff:ff:ff 2024-09-21T00:25:55.882 INFO:teuthology.orchestra.run.smithi112.stdout: altname enp7s0f1 2024-09-21T00:25:55.882 INFO:teuthology.orchestra.run.smithi112.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T00:25:55.882 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:8f:cb:68 brd ff:ff:ff:ff:ff:ff 2024-09-21T00:25:55.882 INFO:teuthology.orchestra.run.smithi112.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T00:25:55.882 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:8f:cb:69 brd ff:ff:ff:ff:ff:ff 2024-09-21T00:25:55.882 INFO:teuthology.orchestra.run.smithi112.stdout: inet 172.21.15.112/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-21T00:25:55.882 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft 42766sec preferred_lft 42766sec 2024-09-21T00:25:55.883 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::ec4:7aff:fe8f:cb69/64 scope link noprefixroute 2024-09-21T00:25:55.883 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:25:55.883 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-21T00:25:55.883 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T00:25:55.883 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2024-09-21T00:25:55.883 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link add name ceph-brx type bridge 2024-09-21T00:25:55.883 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip addr flush dev ceph-brx 2024-09-21T00:25:55.883 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link set ceph-brx up 2024-09-21T00:25:55.883 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-21T00:25:55.884 DEBUG:teuthology.orchestra.run.smithi112:> ') 2024-09-21T00:25:55.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T00:25:56.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T00:25:56.080 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:25:56.080 DEBUG:teuthology.orchestra.run.smithi112:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T00:25:56.104 INFO:teuthology.orchestra.run.smithi112.stdout:1 2024-09-21T00:25:56.106 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:25:56.106 DEBUG:teuthology.orchestra.run.smithi112:> route 2024-09-21T00:25:56.176 INFO:teuthology.orchestra.run.smithi112.stdout:Kernel IP routing table 2024-09-21T00:25:56.176 INFO:teuthology.orchestra.run.smithi112.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T00:25:56.176 INFO:teuthology.orchestra.run.smithi112.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-21T00:25:56.176 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-21T00:25:56.177 INFO:teuthology.orchestra.run.smithi112.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T00:25:56.177 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T00:25:56.177 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2024-09-21T00:25:56.177 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-21T00:25:56.177 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-21T00:25:56.177 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-21T00:25:56.177 DEBUG:teuthology.orchestra.run.smithi112:> ') 2024-09-21T00:25:56.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:56 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T00:25:56.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T00:25:56.458 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:25:56.458 DEBUG:teuthology.orchestra.run.smithi112:> ip netns list 2024-09-21T00:25:56.474 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:25:56.474 DEBUG:teuthology.orchestra.run.smithi112:> ip netns list-id 2024-09-21T00:25:56.529 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T00:25:56.529 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2024-09-21T00:25:56.529 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-21T00:25:56.529 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-09-21T00:25:56.529 DEBUG:teuthology.orchestra.run.smithi112:> ') 2024-09-21T00:25:56.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:56 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T00:25:56.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T00:25:56.645 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-09-21T00:25:56.645 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T00:25:56.645 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2024-09-21T00:25:56.645 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-09-21T00:25:56.645 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-21T00:25:56.646 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-09-21T00:25:56.646 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-09-21T00:25:56.646 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-09-21T00:25:56.646 DEBUG:teuthology.orchestra.run.smithi112:> ') 2024-09-21T00:25:56.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:56 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T00:25:56.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T00:25:56.770 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T00:25:56.770 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2024-09-21T00:25:56.770 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link set brx.0 up 2024-09-21T00:25:56.770 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link set dev brx.0 master ceph-brx 2024-09-21T00:25:56.770 DEBUG:teuthology.orchestra.run.smithi112:> ') 2024-09-21T00:25:56.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:56 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T00:25:56.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T00:25:56.846 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-09-21T00:25:56.846 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T00:25:56.846 DEBUG:teuthology.orchestra.run.smithi112:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-09-21T00:25:56.861 INFO:teuthology.orchestra.run.smithi112.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-09-21T00:25:56.861 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T00:25:56.861 DEBUG:teuthology.orchestra.run.smithi112:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-09-21T00:25:56.916 DEBUG:teuthology.orchestra.run.smithi112:> sudo modprobe fuse 2024-09-21T00:25:56.981 DEBUG:teuthology.orchestra.run.smithi112:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T00:25:57.037 INFO:teuthology.orchestra.run.smithi112.stdout:/proc 2024-09-21T00:25:57.037 INFO:teuthology.orchestra.run.smithi112.stdout:/sys 2024-09-21T00:25:57.038 INFO:teuthology.orchestra.run.smithi112.stdout:/dev 2024-09-21T00:25:57.038 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/kernel/security 2024-09-21T00:25:57.038 INFO:teuthology.orchestra.run.smithi112.stdout:/dev/shm 2024-09-21T00:25:57.038 INFO:teuthology.orchestra.run.smithi112.stdout:/dev/pts 2024-09-21T00:25:57.038 INFO:teuthology.orchestra.run.smithi112.stdout:/run 2024-09-21T00:25:57.038 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/fs/cgroup 2024-09-21T00:25:57.038 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/fs/pstore 2024-09-21T00:25:57.038 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/fs/bpf 2024-09-21T00:25:57.038 INFO:teuthology.orchestra.run.smithi112.stdout:/ 2024-09-21T00:25:57.038 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/fs/selinux 2024-09-21T00:25:57.038 INFO:teuthology.orchestra.run.smithi112.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T00:25:57.038 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/kernel/debug 2024-09-21T00:25:57.038 INFO:teuthology.orchestra.run.smithi112.stdout:/dev/mqueue 2024-09-21T00:25:57.038 INFO:teuthology.orchestra.run.smithi112.stdout:/dev/hugepages 2024-09-21T00:25:57.039 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/kernel/tracing 2024-09-21T00:25:57.039 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/fs/fuse/connections 2024-09-21T00:25:57.039 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/kernel/config 2024-09-21T00:25:57.039 INFO:teuthology.orchestra.run.smithi112.stdout:/run/credentials/systemd-sysctl.service 2024-09-21T00:25:57.039 INFO:teuthology.orchestra.run.smithi112.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-21T00:25:57.039 INFO:teuthology.orchestra.run.smithi112.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-21T00:25:57.039 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/nfs/rpc_pipefs 2024-09-21T00:25:57.039 INFO:teuthology.orchestra.run.smithi112.stdout:/run/user/1000 2024-09-21T00:25:57.039 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/ceph 2024-09-21T00:25:57.039 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/ceph/osd/ceph-0 2024-09-21T00:25:57.039 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/ceph/osd/ceph-1 2024-09-21T00:25:57.039 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/ceph/osd/ceph-2 2024-09-21T00:25:57.039 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/ceph/osd/ceph-3 2024-09-21T00:25:57.039 INFO:teuthology.orchestra.run.smithi112.stdout:/run/netns 2024-09-21T00:25:57.039 INFO:teuthology.orchestra.run.smithi112.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-21T00:25:57.040 INFO:teuthology.orchestra.run.smithi112.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-21T00:25:57.040 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:25:57.040 DEBUG:teuthology.orchestra.run.smithi112:> ls /sys/fs/fuse/connections 2024-09-21T00:25:57.093 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-21T00:25:57.093 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-09-21T00:25:57.136 DEBUG:teuthology.orchestra.run.smithi112:> sudo modprobe fuse 2024-09-21T00:25:57.159 DEBUG:teuthology.orchestra.run.smithi112:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T00:25:57.216 INFO:teuthology.orchestra.run.smithi112.stdout:/proc 2024-09-21T00:25:57.217 INFO:teuthology.orchestra.run.smithi112.stdout:/sys 2024-09-21T00:25:57.217 INFO:teuthology.orchestra.run.smithi112.stdout:/dev 2024-09-21T00:25:57.217 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/kernel/security 2024-09-21T00:25:57.217 INFO:teuthology.orchestra.run.smithi112.stdout:/dev/shm 2024-09-21T00:25:57.217 INFO:teuthology.orchestra.run.smithi112.stdout:/dev/pts 2024-09-21T00:25:57.217 INFO:teuthology.orchestra.run.smithi112.stdout:/run 2024-09-21T00:25:57.217 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/fs/cgroup 2024-09-21T00:25:57.217 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/fs/pstore 2024-09-21T00:25:57.217 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/fs/bpf 2024-09-21T00:25:57.217 INFO:teuthology.orchestra.run.smithi112.stdout:/ 2024-09-21T00:25:57.217 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/fs/selinux 2024-09-21T00:25:57.217 INFO:teuthology.orchestra.run.smithi112.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T00:25:57.217 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/kernel/debug 2024-09-21T00:25:57.217 INFO:teuthology.orchestra.run.smithi112.stdout:/dev/mqueue 2024-09-21T00:25:57.218 INFO:teuthology.orchestra.run.smithi112.stdout:/dev/hugepages 2024-09-21T00:25:57.218 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/kernel/tracing 2024-09-21T00:25:57.218 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/fs/fuse/connections 2024-09-21T00:25:57.218 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/kernel/config 2024-09-21T00:25:57.218 INFO:teuthology.orchestra.run.smithi112.stdout:/run/credentials/systemd-sysctl.service 2024-09-21T00:25:57.218 INFO:teuthology.orchestra.run.smithi112.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-21T00:25:57.218 INFO:teuthology.orchestra.run.smithi112.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-21T00:25:57.218 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/nfs/rpc_pipefs 2024-09-21T00:25:57.218 INFO:teuthology.orchestra.run.smithi112.stdout:/run/user/1000 2024-09-21T00:25:57.218 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/ceph 2024-09-21T00:25:57.218 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/ceph/osd/ceph-0 2024-09-21T00:25:57.218 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/ceph/osd/ceph-1 2024-09-21T00:25:57.218 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/ceph/osd/ceph-2 2024-09-21T00:25:57.219 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/ceph/osd/ceph-3 2024-09-21T00:25:57.219 INFO:teuthology.orchestra.run.smithi112.stdout:/run/netns 2024-09-21T00:25:57.219 INFO:teuthology.orchestra.run.smithi112.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-21T00:25:57.219 INFO:teuthology.orchestra.run.smithi112.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-21T00:25:57.219 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:25:57.219 DEBUG:teuthology.orchestra.run.smithi112:> ls /sys/fs/fuse/connections 2024-09-21T00:25:57.233 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi112.stderr:2024-09-21T00:25:57.231+0000 7f5a4a51c480 -1 init, newargv = 0x557a454585d0 newargc=15 2024-09-21T00:25:57.233 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi112.stderr:ceph-fuse[44027]: starting ceph client 2024-09-21T00:25:57.241 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi112.stderr:ceph-fuse[44027]: starting fuse 2024-09-21T00:25:57.248 INFO:teuthology.orchestra.run.smithi112.stdout:43 2024-09-21T00:25:57.248 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-09-21T00:25:57.248 DEBUG:teuthology.orchestra.run.smithi112:> sudo stdin-killer -- python3 -c ' 2024-09-21T00:25:57.248 DEBUG:teuthology.orchestra.run.smithi112:> import glob 2024-09-21T00:25:57.249 DEBUG:teuthology.orchestra.run.smithi112:> import re 2024-09-21T00:25:57.249 DEBUG:teuthology.orchestra.run.smithi112:> import os 2024-09-21T00:25:57.249 DEBUG:teuthology.orchestra.run.smithi112:> import subprocess 2024-09-21T00:25:57.249 DEBUG:teuthology.orchestra.run.smithi112:> 2024-09-21T00:25:57.249 DEBUG:teuthology.orchestra.run.smithi112:> def _find_admin_socket(client_name): 2024-09-21T00:25:57.249 DEBUG:teuthology.orchestra.run.smithi112:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-09-21T00:25:57.249 DEBUG:teuthology.orchestra.run.smithi112:> files = glob.glob(asok_path) 2024-09-21T00:25:57.249 DEBUG:teuthology.orchestra.run.smithi112:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-09-21T00:25:57.249 DEBUG:teuthology.orchestra.run.smithi112:> 2024-09-21T00:25:57.249 DEBUG:teuthology.orchestra.run.smithi112:> # Given a non-glob path, it better be there 2024-09-21T00:25:57.249 DEBUG:teuthology.orchestra.run.smithi112:> if "*" not in asok_path: 2024-09-21T00:25:57.249 DEBUG:teuthology.orchestra.run.smithi112:> assert(len(files) == 1) 2024-09-21T00:25:57.249 DEBUG:teuthology.orchestra.run.smithi112:> return files[0] 2024-09-21T00:25:57.249 DEBUG:teuthology.orchestra.run.smithi112:> 2024-09-21T00:25:57.249 DEBUG:teuthology.orchestra.run.smithi112:> for f in files: 2024-09-21T00:25:57.249 DEBUG:teuthology.orchestra.run.smithi112:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-21T00:25:57.250 DEBUG:teuthology.orchestra.run.smithi112:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-21T00:25:57.250 DEBUG:teuthology.orchestra.run.smithi112:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-21T00:25:57.250 DEBUG:teuthology.orchestra.run.smithi112:> contents = proc_f.read() 2024-09-21T00:25:57.250 DEBUG:teuthology.orchestra.run.smithi112:> if mountpoint in contents: 2024-09-21T00:25:57.250 DEBUG:teuthology.orchestra.run.smithi112:> return f 2024-09-21T00:25:57.250 DEBUG:teuthology.orchestra.run.smithi112:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-21T00:25:57.250 DEBUG:teuthology.orchestra.run.smithi112:> 2024-09-21T00:25:57.250 DEBUG:teuthology.orchestra.run.smithi112:> print(_find_admin_socket("client.admin")) 2024-09-21T00:25:57.250 DEBUG:teuthology.orchestra.run.smithi112:> ' 2024-09-21T00:25:57.365 INFO:teuthology.orchestra.run.smithi112.stdout:/var/run/ceph/ceph-client.admin.44027.asok 2024-09-21T00:25:57.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T00:25:57.375 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.44027.asok 2024-09-21T00:25:57.375 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:25:57.375 DEBUG:teuthology.orchestra.run.smithi112:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.44027.asok status 2024-09-21T00:25:57.516 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-21T00:25:57.516 INFO:teuthology.orchestra.run.smithi112.stdout: "metadata": { 2024-09-21T00:25:57.516 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_sha1": "1843221895adccfb3b692060251dd6a0183bc410", 2024-09-21T00:25:57.517 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", 2024-09-21T00:25:57.517 INFO:teuthology.orchestra.run.smithi112.stdout: "entity_id": "admin", 2024-09-21T00:25:57.517 INFO:teuthology.orchestra.run.smithi112.stdout: "hostname": "smithi112", 2024-09-21T00:25:57.517 INFO:teuthology.orchestra.run.smithi112.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-09-21T00:25:57.517 INFO:teuthology.orchestra.run.smithi112.stdout: "pid": "44027", 2024-09-21T00:25:57.517 INFO:teuthology.orchestra.run.smithi112.stdout: "root": "/" 2024-09-21T00:25:57.517 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:25:57.517 INFO:teuthology.orchestra.run.smithi112.stdout: "dentry_count": 0, 2024-09-21T00:25:57.517 INFO:teuthology.orchestra.run.smithi112.stdout: "dentry_pinned_count": 0, 2024-09-21T00:25:57.517 INFO:teuthology.orchestra.run.smithi112.stdout: "id": 4551, 2024-09-21T00:25:57.517 INFO:teuthology.orchestra.run.smithi112.stdout: "inst": { 2024-09-21T00:25:57.517 INFO:teuthology.orchestra.run.smithi112.stdout: "name": { 2024-09-21T00:25:57.517 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "client", 2024-09-21T00:25:57.517 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 4551 2024-09-21T00:25:57.517 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:25:57.518 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": { 2024-09-21T00:25:57.518 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-09-21T00:25:57.518 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "192.168.0.1:0", 2024-09-21T00:25:57.518 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 1345968874 2024-09-21T00:25:57.518 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:25:57.518 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:25:57.518 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": { 2024-09-21T00:25:57.518 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-09-21T00:25:57.518 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "192.168.0.1:0", 2024-09-21T00:25:57.518 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 1345968874 2024-09-21T00:25:57.518 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:25:57.518 INFO:teuthology.orchestra.run.smithi112.stdout: "inst_str": "client.4551 192.168.0.1:0/1345968874", 2024-09-21T00:25:57.518 INFO:teuthology.orchestra.run.smithi112.stdout: "addr_str": "192.168.0.1:0/1345968874", 2024-09-21T00:25:57.518 INFO:teuthology.orchestra.run.smithi112.stdout: "inode_count": 1, 2024-09-21T00:25:57.518 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_epoch": 7, 2024-09-21T00:25:57.519 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_epoch": 26, 2024-09-21T00:25:57.519 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_epoch_barrier": 0, 2024-09-21T00:25:57.519 INFO:teuthology.orchestra.run.smithi112.stdout: "blocklisted": false, 2024-09-21T00:25:57.519 INFO:teuthology.orchestra.run.smithi112.stdout: "fs_name": "cephfs" 2024-09-21T00:25:57.519 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-21T00:25:57.525 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:25:57.525 DEBUG:teuthology.orchestra.run.smithi112:> stat --file-system '--printf=%T 2024-09-21T00:25:57.525 DEBUG:teuthology.orchestra.run.smithi112:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-09-21T00:25:57.540 INFO:teuthology.orchestra.run.smithi112.stdout:fuseblk 2024-09-21T00:25:57.540 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-09-21T00:25:57.540 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:25:57.540 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-09-21T00:25:57.607 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-09-21T00:25:57.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:57 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T00:25:57.717 INFO:teuthology.orchestra.run.smithi112.stdout:# file: . 2024-09-21T00:25:57.717 INFO:teuthology.orchestra.run.smithi112.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-09-21T00:25:57.717 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:25:57.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T00:25:57.723 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T00:25:57.723 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:25:57.739 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T00:25:57.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:57.889+0000 7fda1460e640 1 -- 172.21.15.112:0/3179909159 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda0c06def0 msgr2=0x7fda0c06f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:57.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:57.889+0000 7fda1460e640 1 --2- 172.21.15.112:0/3179909159 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda0c06def0 0x7fda0c06f410 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fda0c073190 tx=0x7fd9fc02a9e0 comp rx=0 tx=0).stop 2024-09-21T00:25:57.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:57.889+0000 7fda1460e640 1 -- 172.21.15.112:0/3179909159 shutdown_connections 2024-09-21T00:25:57.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:57.889+0000 7fda1460e640 1 --2- 172.21.15.112:0/3179909159 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fda0c0b5260 0x7fda0c15ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:57.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:57.889+0000 7fda1460e640 1 --2- 172.21.15.112:0/3179909159 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda0c06def0 0x7fda0c06f410 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:57.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:57.889+0000 7fda1460e640 1 -- 172.21.15.112:0/3179909159 >> 172.21.15.112:0/3179909159 conn(0x7fda0c0cdca0 msgr2=0x7fda0c0ce0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:57.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:57.889+0000 7fda1460e640 1 -- 172.21.15.112:0/3179909159 shutdown_connections 2024-09-21T00:25:57.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:57.889+0000 7fda1460e640 1 -- 172.21.15.112:0/3179909159 wait complete. 2024-09-21T00:25:57.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:57.891+0000 7fda1460e640 1 Processor -- start 2024-09-21T00:25:57.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:57.891+0000 7fda1460e640 1 -- start start 2024-09-21T00:25:57.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:57.892+0000 7fda1460e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fda0c06def0 0x7fda0c155600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:57.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:57.892+0000 7fda1460e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda0c0b5260 0x7fda0c1e6610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:57.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:57.892+0000 7fda1460e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fda0c15bfc0 0x7fda0c1e6b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:57.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:57.892+0000 7fda1460e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fda0c0b5a00 con 0x7fda0c0b5260 2024-09-21T00:25:57.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:57.892+0000 7fda1460e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fda0c0b5880 con 0x7fda0c15bfc0 2024-09-21T00:25:57.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:57.892+0000 7fda1460e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fda0c071300 con 0x7fda0c06def0 2024-09-21T00:25:57.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:57.892+0000 7fda11ba9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda0c0b5260 0x7fda0c1e6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:57.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:57.892+0000 7fda11ba9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda0c0b5260 0x7fda0c1e6610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45456/0 (socket says 172.21.15.112:45456) 2024-09-21T00:25:57.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:57.892+0000 7fda11ba9640 1 -- 172.21.15.112:0/2321599316 learned_addr learned my addr 172.21.15.112:0/2321599316 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:25:57.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:57.892+0000 7fda123aa640 1 --2- 172.21.15.112:0/2321599316 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fda0c06def0 0x7fda0c155600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:57.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:57.892+0000 7fda12bab640 1 --2- 172.21.15.112:0/2321599316 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fda0c15bfc0 0x7fda0c1e6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:57.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:57.893+0000 7fda11ba9640 1 -- 172.21.15.112:0/2321599316 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fda0c06def0 msgr2=0x7fda0c155600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:57.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:57.893+0000 7fda11ba9640 1 --2- 172.21.15.112:0/2321599316 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fda0c06def0 0x7fda0c155600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:57.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:57.893+0000 7fda11ba9640 1 -- 172.21.15.112:0/2321599316 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fda0c15bfc0 msgr2=0x7fda0c1e6b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:57.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:57.893+0000 7fda11ba9640 1 --2- 172.21.15.112:0/2321599316 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fda0c15bfc0 0x7fda0c1e6b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:57.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:57.893+0000 7fda11ba9640 1 -- 172.21.15.112:0/2321599316 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda0c1e4df0 con 0x7fda0c0b5260 2024-09-21T00:25:57.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:57.893+0000 7fda11ba9640 1 --2- 172.21.15.112:0/2321599316 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda0c0b5260 0x7fda0c1e6610 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fda0000c970 tx=0x7fda0000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:25:57.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:57.893+0000 7fda0affd640 1 -- 172.21.15.112:0/2321599316 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda00007bb0 con 0x7fda0c0b5260 2024-09-21T00:25:57.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:57.893+0000 7fda0affd640 1 -- 172.21.15.112:0/2321599316 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fda00007e40 con 0x7fda0c0b5260 2024-09-21T00:25:57.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:57.893+0000 7fda0affd640 1 -- 172.21.15.112:0/2321599316 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda00005280 con 0x7fda0c0b5260 2024-09-21T00:25:57.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:57.893+0000 7fda1460e640 1 -- 172.21.15.112:0/2321599316 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda0c1f53b0 con 0x7fda0c0b5260 2024-09-21T00:25:57.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:57.893+0000 7fda1460e640 1 -- 172.21.15.112:0/2321599316 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda0c1f5970 con 0x7fda0c0b5260 2024-09-21T00:25:57.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:57.894+0000 7fda1460e640 1 -- 172.21.15.112:0/2321599316 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda0c06f410 con 0x7fda0c0b5260 2024-09-21T00:25:57.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:57.895+0000 7fda0affd640 1 -- 172.21.15.112:0/2321599316 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fda00005420 con 0x7fda0c0b5260 2024-09-21T00:25:57.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:57.895+0000 7fda0affd640 1 --2- 172.21.15.112:0/2321599316 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fd9f8078b90 0x7fd9f807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:57.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:57.895+0000 7fda0affd640 1 -- 172.21.15.112:0/2321599316 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fda0009b370 con 0x7fda0c0b5260 2024-09-21T00:25:57.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:57.895+0000 7fda123aa640 1 --2- 172.21.15.112:0/2321599316 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fd9f8078b90 0x7fd9f807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:57.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:57.896+0000 7fda123aa640 1 --2- 172.21.15.112:0/2321599316 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fd9f8078b90 0x7fd9f807b0a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc035000 tx=0x7fd9fc004ab0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:25:57.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:57.900+0000 7fda0affd640 1 -- 172.21.15.112:0/2321599316 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fda00014030 con 0x7fda0c0b5260 2024-09-21T00:25:58.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:58.087+0000 7fda1460e640 1 -- 172.21.15.112:0/2321599316 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fda0c1f6030 con 0x7fda0c0b5260 2024-09-21T00:25:58.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:58.088+0000 7fda0affd640 1 -- 172.21.15.112:0/2321599316 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7fda00063000 con 0x7fda0c0b5260 2024-09-21T00:25:58.090 INFO:teuthology.orchestra.run.smithi112.stderr:listed 0 entries 2024-09-21T00:25:58.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:58.090+0000 7fda1460e640 1 -- 172.21.15.112:0/2321599316 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fd9f8078b90 msgr2=0x7fd9f807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:58.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:58.090+0000 7fda1460e640 1 --2- 172.21.15.112:0/2321599316 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fd9f8078b90 0x7fd9f807b0a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc035000 tx=0x7fd9fc004ab0 comp rx=0 tx=0).stop 2024-09-21T00:25:58.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:58.090+0000 7fda1460e640 1 -- 172.21.15.112:0/2321599316 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda0c0b5260 msgr2=0x7fda0c1e6610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:58.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:58.090+0000 7fda1460e640 1 --2- 172.21.15.112:0/2321599316 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda0c0b5260 0x7fda0c1e6610 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fda0000c970 tx=0x7fda0000ce30 comp rx=0 tx=0).stop 2024-09-21T00:25:58.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:58.090+0000 7fda1460e640 1 -- 172.21.15.112:0/2321599316 shutdown_connections 2024-09-21T00:25:58.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:58.090+0000 7fda1460e640 1 --2- 172.21.15.112:0/2321599316 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fda0c15bfc0 0x7fda0c1e6b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:58.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:58.090+0000 7fda1460e640 1 --2- 172.21.15.112:0/2321599316 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fd9f8078b90 0x7fd9f807b0a0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:58.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:58.090+0000 7fda1460e640 1 --2- 172.21.15.112:0/2321599316 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda0c0b5260 0x7fda0c1e6610 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:58.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:58.090+0000 7fda1460e640 1 --2- 172.21.15.112:0/2321599316 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fda0c06def0 0x7fda0c155600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:58.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:58.090+0000 7fda1460e640 1 -- 172.21.15.112:0/2321599316 >> 172.21.15.112:0/2321599316 conn(0x7fda0c0cdca0 msgr2=0x7fda0c0b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:58.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:58.090+0000 7fda1460e640 1 -- 172.21.15.112:0/2321599316 shutdown_connections 2024-09-21T00:25:58.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:58.090+0000 7fda1460e640 1 -- 172.21.15.112:0/2321599316 wait complete. 2024-09-21T00:25:58.102 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T00:25:58.103 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:25:58.118 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T00:25:58.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:58.270+0000 7f41e996e640 1 -- 172.21.15.112:0/538083304 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41d0004800 msgr2=0x7f41d0004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:58.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:58.270+0000 7f41e996e640 1 --2- 172.21.15.112:0/538083304 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41d0004800 0x7f41d0004c10 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f41cc006fc0 tx=0x7f41d0005830 comp rx=0 tx=0).stop 2024-09-21T00:25:58.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:58.270+0000 7f41e37fe640 1 -- 172.21.15.112:0/538083304 reap_dead start 2024-09-21T00:25:58.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:58.270+0000 7f41e996e640 1 -- 172.21.15.112:0/538083304 shutdown_connections 2024-09-21T00:25:58.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:58.270+0000 7f41e996e640 1 -- 172.21.15.112:0/538083304 >> 172.21.15.112:0/538083304 conn(0x7f41e40cdca0 msgr2=0x7f41e40ce0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:58.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:58.270+0000 7f41e996e640 1 -- 172.21.15.112:0/538083304 shutdown_connections 2024-09-21T00:25:58.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:58.270+0000 7f41e996e640 1 -- 172.21.15.112:0/538083304 wait complete. 2024-09-21T00:25:58.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:58.272+0000 7f41e996e640 1 Processor -- start 2024-09-21T00:25:58.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:58.272+0000 7f41e996e640 1 -- start start 2024-09-21T00:25:58.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:58.273+0000 7f41e996e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f41d0004800 0x7f41e40cd1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:58.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:58.273+0000 7f41e996e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f41e414ba00 0x7f41e415d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:58.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:58.273+0000 7f41e996e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41e415d820 0x7f41e406cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:58.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:58.273+0000 7f41e996e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f41e406f410 con 0x7f41e415d820 2024-09-21T00:25:58.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:58.273+0000 7f41e996e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f41e4160000 con 0x7f41d0004800 2024-09-21T00:25:58.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:58.273+0000 7f41e996e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f41e406f590 con 0x7f41e414ba00 2024-09-21T00:25:58.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:58.273+0000 7f41e37fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41e415d820 0x7f41e406cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:58.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:58.273+0000 7f41e37fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41e415d820 0x7f41e406cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45484/0 (socket says 172.21.15.112:45484) 2024-09-21T00:25:58.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:58.273+0000 7f41e37fe640 1 -- 172.21.15.112:0/2800657872 learned_addr learned my addr 172.21.15.112:0/2800657872 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:25:58.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:58.274+0000 7f41da7fc640 1 --2- 172.21.15.112:0/2800657872 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f41e414ba00 0x7f41e415d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:58.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:58.274+0000 7f41e2ffd640 1 --2- 172.21.15.112:0/2800657872 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f41d0004800 0x7f41e40cd1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:58.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:58.274+0000 7f41e37fe640 1 -- 172.21.15.112:0/2800657872 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f41e414ba00 msgr2=0x7f41e415d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:58.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:58.274+0000 7f41e37fe640 1 --2- 172.21.15.112:0/2800657872 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f41e414ba00 0x7f41e415d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:58.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:58.274+0000 7f41e37fe640 1 -- 172.21.15.112:0/2800657872 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f41d0004800 msgr2=0x7f41e40cd1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:58.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:58.274+0000 7f41e37fe640 1 --2- 172.21.15.112:0/2800657872 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f41d0004800 0x7f41e40cd1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:58.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:58.274+0000 7f41e37fe640 1 -- 172.21.15.112:0/2800657872 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41e409ad00 con 0x7f41e415d820 2024-09-21T00:25:58.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:58.274+0000 7f41e37fe640 1 --2- 172.21.15.112:0/2800657872 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41e415d820 0x7f41e406cde0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f41e40ae0f0 tx=0x7f41dc006430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:25:58.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:58.274+0000 7f41e0ff9640 1 -- 172.21.15.112:0/2800657872 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41dc015070 con 0x7f41e415d820 2024-09-21T00:25:58.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:58.274+0000 7f41e0ff9640 1 -- 172.21.15.112:0/2800657872 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f41dc011840 con 0x7f41e415d820 2024-09-21T00:25:58.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:58.274+0000 7f41e996e640 1 -- 172.21.15.112:0/2800657872 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41e415dc30 con 0x7f41e415d820 2024-09-21T00:25:58.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:58.274+0000 7f41e0ff9640 1 -- 172.21.15.112:0/2800657872 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41dc011c90 con 0x7f41e415d820 2024-09-21T00:25:58.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:58.274+0000 7f41e996e640 1 -- 172.21.15.112:0/2800657872 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41e409d880 con 0x7f41e415d820 2024-09-21T00:25:58.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:58.275+0000 7f41e996e640 1 -- 172.21.15.112:0/2800657872 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41e406f910 con 0x7f41e415d820 2024-09-21T00:25:58.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:58.276+0000 7f41e0ff9640 1 -- 172.21.15.112:0/2800657872 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f41dc018070 con 0x7f41e415d820 2024-09-21T00:25:58.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:58.276+0000 7f41e0ff9640 1 --2- 172.21.15.112:0/2800657872 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f41c4078bc0 0x7f41c407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:58.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:58.277+0000 7f41e0ff9640 1 -- 172.21.15.112:0/2800657872 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f41dc09bb90 con 0x7f41e415d820 2024-09-21T00:25:58.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:58.277+0000 7f41e2ffd640 1 --2- 172.21.15.112:0/2800657872 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f41c4078bc0 0x7f41c407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:58.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:58.277+0000 7f41e2ffd640 1 --2- 172.21.15.112:0/2800657872 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f41c4078bc0 0x7f41c407b0d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f41e415efd0 tx=0x7f41d4005f90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:25:58.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:58.281+0000 7f41e0ff9640 1 -- 172.21.15.112:0/2800657872 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f41e415dc30 con 0x7f41e415d820 2024-09-21T00:25:58.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:58.468+0000 7f41e996e640 1 -- 172.21.15.112:0/2800657872 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f41e4072360 con 0x7f41e415d820 2024-09-21T00:25:58.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:58.468+0000 7f41e0ff9640 1 -- 172.21.15.112:0/2800657872 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7f41e409ad00 con 0x7f41e415d820 2024-09-21T00:25:58.470 INFO:teuthology.orchestra.run.smithi112.stderr:listed 0 entries 2024-09-21T00:25:58.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:58.470+0000 7f41e996e640 1 -- 172.21.15.112:0/2800657872 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f41c4078bc0 msgr2=0x7f41c407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:58.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:58.470+0000 7f41e996e640 1 --2- 172.21.15.112:0/2800657872 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f41c4078bc0 0x7f41c407b0d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f41e415efd0 tx=0x7f41d4005f90 comp rx=0 tx=0).stop 2024-09-21T00:25:58.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:58.470+0000 7f41e996e640 1 -- 172.21.15.112:0/2800657872 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41e415d820 msgr2=0x7f41e406cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:58.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:58.470+0000 7f41e996e640 1 --2- 172.21.15.112:0/2800657872 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41e415d820 0x7f41e406cde0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f41e40ae0f0 tx=0x7f41dc006430 comp rx=0 tx=0).stop 2024-09-21T00:25:58.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:58.470+0000 7f41e996e640 1 -- 172.21.15.112:0/2800657872 shutdown_connections 2024-09-21T00:25:58.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:58.470+0000 7f41e996e640 1 --2- 172.21.15.112:0/2800657872 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f41c4078bc0 0x7f41c407b0d0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:58.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:58.470+0000 7f41e996e640 1 --2- 172.21.15.112:0/2800657872 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41e415d820 0x7f41e406cde0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:58.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:58.470+0000 7f41e996e640 1 --2- 172.21.15.112:0/2800657872 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f41e414ba00 0x7f41e415d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:58.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:58.470+0000 7f41e996e640 1 --2- 172.21.15.112:0/2800657872 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f41d0004800 0x7f41e40cd1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:58.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:58.470+0000 7f41e996e640 1 -- 172.21.15.112:0/2800657872 >> 172.21.15.112:0/2800657872 conn(0x7f41e40cdca0 msgr2=0x7f41e40d11b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:58.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:58.471+0000 7f41e996e640 1 -- 172.21.15.112:0/2800657872 shutdown_connections 2024-09-21T00:25:58.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:58.471+0000 7f41e996e640 1 -- 172.21.15.112:0/2800657872 wait complete. 2024-09-21T00:25:58.483 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi112.front.sepia.ceph.com... 2024-09-21T00:25:58.483 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:25:58.483 DEBUG:teuthology.orchestra.run.smithi112:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-09-21T00:25:58.507 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi112.stderr:ceph-fuse[44027]: fuse finished with error 0 and tester_r 0 2024-09-21T00:25:58.509 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T00:25:58.609 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi112.front.sepia.ceph.com 2024-09-21T00:25:58.609 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:25:58.609 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-09-21T00:25:58.631 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T00:25:58.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:58.781+0000 7fe9c6abd640 1 -- 172.21.15.112:0/3655006502 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9ac003a30 msgr2=0x7fe9ac003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:58.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:58.781+0000 7fe9c6abd640 1 --2- 172.21.15.112:0/3655006502 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9ac003a30 0x7fe9ac003ea0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fe9b0009ed0 tx=0x7fe9ac005830 comp rx=0 tx=0).stop 2024-09-21T00:25:58.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:58.781+0000 7fe9c505a640 1 -- 172.21.15.112:0/3655006502 reap_dead start 2024-09-21T00:25:58.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:58.781+0000 7fe9c6abd640 1 -- 172.21.15.112:0/3655006502 shutdown_connections 2024-09-21T00:25:58.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:58.781+0000 7fe9c6abd640 1 -- 172.21.15.112:0/3655006502 >> 172.21.15.112:0/3655006502 conn(0x7fe9c00d1140 msgr2=0x7fe9c00d1550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:58.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:58.781+0000 7fe9c6abd640 1 -- 172.21.15.112:0/3655006502 shutdown_connections 2024-09-21T00:25:58.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:58.781+0000 7fe9c6abd640 1 -- 172.21.15.112:0/3655006502 wait complete. 2024-09-21T00:25:58.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:58.783+0000 7fe9c6abd640 1 Processor -- start 2024-09-21T00:25:58.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:58.783+0000 7fe9c6abd640 1 -- start start 2024-09-21T00:25:58.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:58.784+0000 7fe9c6abd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe9ac003a30 0x7fe9c00c6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:58.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:58.784+0000 7fe9c6abd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe9c00b4d30 0x7fe9c0156400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:58.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:58.785+0000 7fe9c6abd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9c006d380 0x7fe9c006d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:58.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:58.785+0000 7fe9c6abd640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe9c0070370 con 0x7fe9c006d380 2024-09-21T00:25:58.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:58.785+0000 7fe9c6abd640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe9c00701f0 con 0x7fe9c00b4d30 2024-09-21T00:25:58.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:58.785+0000 7fe9c6abd640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe9c006ec50 con 0x7fe9ac003a30 2024-09-21T00:25:58.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:58.785+0000 7fe9c505a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9c006d380 0x7fe9c006d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:58.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:58.785+0000 7fe9c505a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9c006d380 0x7fe9c006d790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45492/0 (socket says 172.21.15.112:45492) 2024-09-21T00:25:58.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:58.785+0000 7fe9c505a640 1 -- 172.21.15.112:0/2795670219 learned_addr learned my addr 172.21.15.112:0/2795670219 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:25:58.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:58.785+0000 7fe9c4859640 1 --2- 172.21.15.112:0/2795670219 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe9ac003a30 0x7fe9c00c6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:58.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:58.785+0000 7fe9bffff640 1 --2- 172.21.15.112:0/2795670219 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe9c00b4d30 0x7fe9c0156400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:58.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:58.785+0000 7fe9c505a640 1 -- 172.21.15.112:0/2795670219 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe9ac003a30 msgr2=0x7fe9c00c6ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:58.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:58.785+0000 7fe9c505a640 1 --2- 172.21.15.112:0/2795670219 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe9ac003a30 0x7fe9c00c6ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:58.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:58.785+0000 7fe9c505a640 1 -- 172.21.15.112:0/2795670219 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe9c00b4d30 msgr2=0x7fe9c0156400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:58.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:58.785+0000 7fe9c505a640 1 --2- 172.21.15.112:0/2795670219 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe9c00b4d30 0x7fe9c0156400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:58.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:58.785+0000 7fe9c505a640 1 -- 172.21.15.112:0/2795670219 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9c006dcd0 con 0x7fe9c006d380 2024-09-21T00:25:58.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:58.785+0000 7fe9c4859640 1 --2- 172.21.15.112:0/2795670219 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe9ac003a30 0x7fe9c00c6ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:25:58.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:58.785+0000 7fe9c505a640 1 --2- 172.21.15.112:0/2795670219 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9c006d380 0x7fe9c006d790 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fe9b8000fb0 tx=0x7fe9b800e9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:25:58.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:58.785+0000 7fe9bd7fa640 1 -- 172.21.15.112:0/2795670219 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9b8011870 con 0x7fe9c006d380 2024-09-21T00:25:58.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:58.785+0000 7fe9bd7fa640 1 -- 172.21.15.112:0/2795670219 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe9b8017070 con 0x7fe9c006d380 2024-09-21T00:25:58.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:58.785+0000 7fe9bd7fa640 1 -- 172.21.15.112:0/2795670219 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9b8011da0 con 0x7fe9c006d380 2024-09-21T00:25:58.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:58.785+0000 7fe9c6abd640 1 -- 172.21.15.112:0/2795670219 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9c00c79a0 con 0x7fe9c006d380 2024-09-21T00:25:58.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:58.785+0000 7fe9c6abd640 1 -- 172.21.15.112:0/2795670219 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9c00b4250 con 0x7fe9c006d380 2024-09-21T00:25:58.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:58.786+0000 7fe9c6abd640 1 -- 172.21.15.112:0/2795670219 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9c00c7710 con 0x7fe9c006d380 2024-09-21T00:25:58.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:58.787+0000 7fe9bd7fa640 1 -- 172.21.15.112:0/2795670219 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe9b801c460 con 0x7fe9c006d380 2024-09-21T00:25:58.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:58.787+0000 7fe9bd7fa640 1 --2- 172.21.15.112:0/2795670219 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fe9a0078bc0 0x7fe9a007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:58.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:58.788+0000 7fe9bd7fa640 1 -- 172.21.15.112:0/2795670219 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fe9b809b290 con 0x7fe9c006d380 2024-09-21T00:25:58.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:58.788+0000 7fe9c4859640 1 --2- 172.21.15.112:0/2795670219 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fe9a0078bc0 0x7fe9a007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:58.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:58.788+0000 7fe9c4859640 1 --2- 172.21.15.112:0/2795670219 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fe9a0078bc0 0x7fe9a007b0d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe9b0035000 tx=0x7fe9b0004a40 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:25:58.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:58.792+0000 7fe9bd7fa640 1 -- 172.21.15.112:0/2795670219 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe9c006dcd0 con 0x7fe9c006d380 2024-09-21T00:25:58.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:58.971+0000 7fe9c6abd640 1 -- 172.21.15.112:0/2795670219 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe9c0074f20 con 0x7fe9c006d380 2024-09-21T00:25:58.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:58.972+0000 7fe9bd7fa640 1 -- 172.21.15.112:0/2795670219 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18622 (secure 0 0 0) 0x7fe9b8063420 con 0x7fe9c006d380 2024-09-21T00:25:58.974 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:25:58.974 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":26,"fsid":"8b34692b-dc65-49b2-a08f-26f707b96ac5","created":"2024-09-21T00:25:22.788490+0000","modified":"2024-09-21T00:25:52.623213+0000","last_up_change":"2024-09-21T00:25:36.161835+0000","last_in_change":"2024-09-21T00:25:26.346500+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T00:25:37.105006+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T00:25:45.413895+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T00:25:46.532124+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T00:25:50.244145+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9c575086-67d2-4de2-b01b-d44dcc21d604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6809","nonce":755760621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6811","nonce":755760621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6815","nonce":755760621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6813","nonce":755760621}]},"public_addr":"172.21.15.112:6809/755760621","cluster_addr":"172.21.15.112:6811/755760621","heartbeat_back_addr":"172.21.15.112:6815/755760621","heartbeat_front_addr":"172.21.15.112:6813/755760621","state":["exists","up"]},{"osd":1,"uuid":"a983c227-a3ce-4bdd-82f1-438ba7e3ecd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6801","nonce":344932592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6803","nonce":344932592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6807","nonce":344932592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6805","nonce":344932592}]},"public_addr":"172.21.15.112:6801/344932592","cluster_addr":"172.21.15.112:6803/344932592","heartbeat_back_addr":"172.21.15.112:6807/344932592","heartbeat_front_addr":"172.21.15.112:6805/344932592","state":["exists","up"]},{"osd":2,"uuid":"1193c1d8-bfff-4a57-8200-5f69bf1ea3af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6825","nonce":407933616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6827","nonce":407933616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6831","nonce":407933616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6829","nonce":407933616}]},"public_addr":"172.21.15.112:6825/407933616","cluster_addr":"172.21.15.112:6827/407933616","heartbeat_back_addr":"172.21.15.112:6831/407933616","heartbeat_front_addr":"172.21.15.112:6829/407933616","state":["exists","up"]},{"osd":3,"uuid":"af04330a-36ad-40cc-a8dd-fa259b221de5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6817","nonce":4120164806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6819","nonce":4120164806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6823","nonce":4120164806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6821","nonce":4120164806}]},"public_addr":"172.21.15.112:6817/4120164806","cluster_addr":"172.21.15.112:6819/4120164806","heartbeat_back_addr":"172.21.15.112:6823/4120164806","heartbeat_front_addr":"172.21.15.112:6821/4120164806","state":["exists","up"]},{"osd":4,"uuid":"5b893111-292c-4c5a-b184-53ad38c7199e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6803","nonce":522077560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6805","nonce":522077560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6809","nonce":522077560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6807","nonce":522077560}]},"public_addr":"172.21.15.187:6803/522077560","cluster_addr":"172.21.15.187:6805/522077560","heartbeat_back_addr":"172.21.15.187:6809/522077560","heartbeat_front_addr":"172.21.15.187:6807/522077560","state":["exists","up"]},{"osd":5,"uuid":"c0b8e011-45fe-4c5e-925e-6be2a7f1868a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6827","nonce":3593144721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6829","nonce":3593144721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6833","nonce":3593144721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6831","nonce":3593144721}]},"public_addr":"172.21.15.187:6827/3593144721","cluster_addr":"172.21.15.187:6829/3593144721","heartbeat_back_addr":"172.21.15.187:6833/3593144721","heartbeat_front_addr":"172.21.15.187:6831/3593144721","state":["exists","up"]},{"osd":6,"uuid":"87d2d650-7489-4593-98a3-a1f433ce8dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6811","nonce":1494387657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6813","nonce":1494387657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6817","nonce":1494387657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6815","nonce":1494387657}]},"public_addr":"172.21.15.187:6811/1494387657","cluster_addr":"172.21.15.187:6813/1494387657","heartbeat_back_addr":"172.21.15.187:6817/1494387657","heartbeat_front_addr":"172.21.15.187:6815/1494387657","state":["exists","up"]},{"osd":7,"uuid":"c6b4f8e2-a1ee-4438-aa37-67ff4b6ef1f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6819","nonce":1624699842}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6821","nonce":1624699842}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6825","nonce":1624699842}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6823","nonce":1624699842}]},"public_addr":"172.21.15.187:6819/1624699842","cluster_addr":"172.21.15.187:6821/1624699842","heartbeat_back_addr":"172.21.15.187:6825/1624699842","heartbeat_front_addr":"172.21.15.187:6823/1624699842","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.873162+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.400506+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.664317+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.159791+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.202630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.113212+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.384786+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.448441+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T00:25:58.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:58.974+0000 7fe9c6abd640 1 -- 172.21.15.112:0/2795670219 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fe9a0078bc0 msgr2=0x7fe9a007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:58.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:58.974+0000 7fe9c6abd640 1 --2- 172.21.15.112:0/2795670219 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fe9a0078bc0 0x7fe9a007b0d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe9b0035000 tx=0x7fe9b0004a40 comp rx=0 tx=0).stop 2024-09-21T00:25:58.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:58.974+0000 7fe9c6abd640 1 -- 172.21.15.112:0/2795670219 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9c006d380 msgr2=0x7fe9c006d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:58.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:58.974+0000 7fe9c6abd640 1 --2- 172.21.15.112:0/2795670219 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9c006d380 0x7fe9c006d790 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fe9b8000fb0 tx=0x7fe9b800e9a0 comp rx=0 tx=0).stop 2024-09-21T00:25:58.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:58.974+0000 7fe9c6abd640 1 -- 172.21.15.112:0/2795670219 shutdown_connections 2024-09-21T00:25:58.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:58.974+0000 7fe9c6abd640 1 --2- 172.21.15.112:0/2795670219 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fe9a0078bc0 0x7fe9a007b0d0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:58.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:58.974+0000 7fe9c6abd640 1 --2- 172.21.15.112:0/2795670219 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9c006d380 0x7fe9c006d790 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:58.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:58.974+0000 7fe9c6abd640 1 --2- 172.21.15.112:0/2795670219 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe9c00b4d30 0x7fe9c0156400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:58.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:58.974+0000 7fe9c6abd640 1 --2- 172.21.15.112:0/2795670219 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe9ac003a30 0x7fe9c00c6ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:58.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:58.974+0000 7fe9c6abd640 1 -- 172.21.15.112:0/2795670219 >> 172.21.15.112:0/2795670219 conn(0x7fe9c00d1140 msgr2=0x7fe9c0156fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:58.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:58.974+0000 7fe9c6abd640 1 -- 172.21.15.112:0/2795670219 shutdown_connections 2024-09-21T00:25:58.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:58.974+0000 7fe9c6abd640 1 -- 172.21.15.112:0/2795670219 wait complete. 2024-09-21T00:25:58.987 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T00:25:59.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:59.097+0000 7f914ce40640 1 -- 172.21.15.112:0/1215652372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9134005330 msgr2=0x7f913400b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:59.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:59.097+0000 7f914ce40640 1 --2- 172.21.15.112:0/1215652372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9134005330 0x7f913400b9e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f91400043e0 tx=0x7f9134005830 comp rx=0 tx=0).stop 2024-09-21T00:25:59.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:59.097+0000 7f914ce40640 1 -- 172.21.15.112:0/1215652372 shutdown_connections 2024-09-21T00:25:59.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:59.097+0000 7f914ce40640 1 --2- 172.21.15.112:0/1215652372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9134005330 0x7f913400b9e0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:59.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:59.097+0000 7f914ce40640 1 -- 172.21.15.112:0/1215652372 >> 172.21.15.112:0/1215652372 conn(0x7f91480b2cc0 msgr2=0x7f91480b30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:59.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:59.097+0000 7f914ce40640 1 -- 172.21.15.112:0/1215652372 shutdown_connections 2024-09-21T00:25:59.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:59.097+0000 7f914ce40640 1 -- 172.21.15.112:0/1215652372 wait complete. 2024-09-21T00:25:59.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:59.099+0000 7f914ce40640 1 Processor -- start 2024-09-21T00:25:59.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:59.100+0000 7f914ce40640 1 -- start start 2024-09-21T00:25:59.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:59.101+0000 7f914ce40640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9134005330 0x7f91480cc6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:59.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:59.101+0000 7f914ce40640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f91481f5e10 0x7f91481f6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:59.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:59.101+0000 7f914ce40640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f91480c6170 0x7f91480c6580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:59.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:59.101+0000 7f914ce40640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f914815d140 con 0x7f9134005330 2024-09-21T00:25:59.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:59.101+0000 7f914ce40640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f914815cfc0 con 0x7f91480c6170 2024-09-21T00:25:59.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:59.101+0000 7f914ce40640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f9148073d50 con 0x7f91481f5e10 2024-09-21T00:25:59.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:59.101+0000 7f914659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9134005330 0x7f91480cc6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:59.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:59.101+0000 7f914659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9134005330 0x7f91480cc6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45508/0 (socket says 172.21.15.112:45508) 2024-09-21T00:25:59.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:59.101+0000 7f914659c640 1 -- 172.21.15.112:0/1002039637 learned_addr learned my addr 172.21.15.112:0/1002039637 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:25:59.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:59.101+0000 7f914659c640 1 -- 172.21.15.112:0/1002039637 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f91481f5e10 msgr2=0x7f91481f6220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:59.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:59.101+0000 7f913dd9b640 1 --2- 172.21.15.112:0/1002039637 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f91481f5e10 0x7f91481f6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:59.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:59.101+0000 7f914659c640 1 --2- 172.21.15.112:0/1002039637 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f91481f5e10 0x7f91481f6220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:59.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:59.101+0000 7f914659c640 1 -- 172.21.15.112:0/1002039637 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f91480c6170 msgr2=0x7f91480c6580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:59.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:59.101+0000 7f914659c640 1 --2- 172.21.15.112:0/1002039637 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f91480c6170 0x7f91480c6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:59.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:59.101+0000 7f914659c640 1 -- 172.21.15.112:0/1002039637 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91481f6760 con 0x7f9134005330 2024-09-21T00:25:59.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:59.101+0000 7f914659c640 1 --2- 172.21.15.112:0/1002039637 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9134005330 0x7f91480cc6f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f9138006630 tx=0x7f91380039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:25:59.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:59.101+0000 7f913f7fe640 1 -- 172.21.15.112:0/1002039637 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9138015070 con 0x7f9134005330 2024-09-21T00:25:59.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:59.102+0000 7f913f7fe640 1 -- 172.21.15.112:0/1002039637 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9138011900 con 0x7f9134005330 2024-09-21T00:25:59.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:59.102+0000 7f914ce40640 1 -- 172.21.15.112:0/1002039637 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9148159690 con 0x7f9134005330 2024-09-21T00:25:59.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:59.102+0000 7f913f7fe640 1 -- 172.21.15.112:0/1002039637 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9138011d50 con 0x7f9134005330 2024-09-21T00:25:59.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:59.102+0000 7f914ce40640 1 -- 172.21.15.112:0/1002039637 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9148159c50 con 0x7f9134005330 2024-09-21T00:25:59.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:59.103+0000 7f914ce40640 1 -- 172.21.15.112:0/1002039637 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91480742b0 con 0x7f9134005330 2024-09-21T00:25:59.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:59.103+0000 7f913f7fe640 1 -- 172.21.15.112:0/1002039637 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9138018070 con 0x7f9134005330 2024-09-21T00:25:59.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:59.104+0000 7f913f7fe640 1 --2- 172.21.15.112:0/1002039637 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f9128078bc0 0x7f912807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:59.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:59.104+0000 7f913f7fe640 1 -- 172.21.15.112:0/1002039637 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f913809bdb0 con 0x7f9134005330 2024-09-21T00:25:59.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:59.104+0000 7f913dd9b640 1 --2- 172.21.15.112:0/1002039637 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f9128078bc0 0x7f912807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:59.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:59.104+0000 7f913dd9b640 1 --2- 172.21.15.112:0/1002039637 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f9128078bc0 0x7f912807b0d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f91480d0610 tx=0x7f9130004120 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:25:59.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:59.108+0000 7f913f7fe640 1 -- 172.21.15.112:0/1002039637 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f91481f6760 con 0x7f9134005330 2024-09-21T00:25:59.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:59.290+0000 7f914ce40640 1 -- 172.21.15.112:0/1002039637 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f91480d0360 con 0x7f9134005330 2024-09-21T00:25:59.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:59.291+0000 7f913f7fe640 1 -- 172.21.15.112:0/1002039637 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18622 (secure 0 0 0) 0x7f9138063f50 con 0x7f9134005330 2024-09-21T00:25:59.293 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:25:59.293 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":26,"fsid":"8b34692b-dc65-49b2-a08f-26f707b96ac5","created":"2024-09-21T00:25:22.788490+0000","modified":"2024-09-21T00:25:52.623213+0000","last_up_change":"2024-09-21T00:25:36.161835+0000","last_in_change":"2024-09-21T00:25:26.346500+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T00:25:37.105006+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T00:25:45.413895+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T00:25:46.532124+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T00:25:50.244145+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9c575086-67d2-4de2-b01b-d44dcc21d604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6809","nonce":755760621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6811","nonce":755760621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6815","nonce":755760621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6813","nonce":755760621}]},"public_addr":"172.21.15.112:6809/755760621","cluster_addr":"172.21.15.112:6811/755760621","heartbeat_back_addr":"172.21.15.112:6815/755760621","heartbeat_front_addr":"172.21.15.112:6813/755760621","state":["exists","up"]},{"osd":1,"uuid":"a983c227-a3ce-4bdd-82f1-438ba7e3ecd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6801","nonce":344932592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6803","nonce":344932592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6807","nonce":344932592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6805","nonce":344932592}]},"public_addr":"172.21.15.112:6801/344932592","cluster_addr":"172.21.15.112:6803/344932592","heartbeat_back_addr":"172.21.15.112:6807/344932592","heartbeat_front_addr":"172.21.15.112:6805/344932592","state":["exists","up"]},{"osd":2,"uuid":"1193c1d8-bfff-4a57-8200-5f69bf1ea3af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6825","nonce":407933616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6827","nonce":407933616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6831","nonce":407933616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6829","nonce":407933616}]},"public_addr":"172.21.15.112:6825/407933616","cluster_addr":"172.21.15.112:6827/407933616","heartbeat_back_addr":"172.21.15.112:6831/407933616","heartbeat_front_addr":"172.21.15.112:6829/407933616","state":["exists","up"]},{"osd":3,"uuid":"af04330a-36ad-40cc-a8dd-fa259b221de5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6817","nonce":4120164806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6819","nonce":4120164806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6823","nonce":4120164806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6821","nonce":4120164806}]},"public_addr":"172.21.15.112:6817/4120164806","cluster_addr":"172.21.15.112:6819/4120164806","heartbeat_back_addr":"172.21.15.112:6823/4120164806","heartbeat_front_addr":"172.21.15.112:6821/4120164806","state":["exists","up"]},{"osd":4,"uuid":"5b893111-292c-4c5a-b184-53ad38c7199e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6803","nonce":522077560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6805","nonce":522077560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6809","nonce":522077560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6807","nonce":522077560}]},"public_addr":"172.21.15.187:6803/522077560","cluster_addr":"172.21.15.187:6805/522077560","heartbeat_back_addr":"172.21.15.187:6809/522077560","heartbeat_front_addr":"172.21.15.187:6807/522077560","state":["exists","up"]},{"osd":5,"uuid":"c0b8e011-45fe-4c5e-925e-6be2a7f1868a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6827","nonce":3593144721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6829","nonce":3593144721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6833","nonce":3593144721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6831","nonce":3593144721}]},"public_addr":"172.21.15.187:6827/3593144721","cluster_addr":"172.21.15.187:6829/3593144721","heartbeat_back_addr":"172.21.15.187:6833/3593144721","heartbeat_front_addr":"172.21.15.187:6831/3593144721","state":["exists","up"]},{"osd":6,"uuid":"87d2d650-7489-4593-98a3-a1f433ce8dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6811","nonce":1494387657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6813","nonce":1494387657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6817","nonce":1494387657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6815","nonce":1494387657}]},"public_addr":"172.21.15.187:6811/1494387657","cluster_addr":"172.21.15.187:6813/1494387657","heartbeat_back_addr":"172.21.15.187:6817/1494387657","heartbeat_front_addr":"172.21.15.187:6815/1494387657","state":["exists","up"]},{"osd":7,"uuid":"c6b4f8e2-a1ee-4438-aa37-67ff4b6ef1f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6819","nonce":1624699842}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6821","nonce":1624699842}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6825","nonce":1624699842}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6823","nonce":1624699842}]},"public_addr":"172.21.15.187:6819/1624699842","cluster_addr":"172.21.15.187:6821/1624699842","heartbeat_back_addr":"172.21.15.187:6825/1624699842","heartbeat_front_addr":"172.21.15.187:6823/1624699842","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.873162+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.400506+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.664317+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.159791+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.202630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.113212+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.384786+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.448441+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T00:25:59.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:59.293+0000 7f914ce40640 1 -- 172.21.15.112:0/1002039637 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f9128078bc0 msgr2=0x7f912807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:59.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:59.293+0000 7f914ce40640 1 --2- 172.21.15.112:0/1002039637 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f9128078bc0 0x7f912807b0d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f91480d0610 tx=0x7f9130004120 comp rx=0 tx=0).stop 2024-09-21T00:25:59.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:59.293+0000 7f914ce40640 1 -- 172.21.15.112:0/1002039637 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9134005330 msgr2=0x7f91480cc6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:59.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:59.293+0000 7f914ce40640 1 --2- 172.21.15.112:0/1002039637 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9134005330 0x7f91480cc6f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f9138006630 tx=0x7f91380039c0 comp rx=0 tx=0).stop 2024-09-21T00:25:59.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:59.293+0000 7f914ce40640 1 -- 172.21.15.112:0/1002039637 shutdown_connections 2024-09-21T00:25:59.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:59.293+0000 7f914ce40640 1 --2- 172.21.15.112:0/1002039637 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f91480c6170 0x7f91480c6580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:59.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:59.293+0000 7f914ce40640 1 --2- 172.21.15.112:0/1002039637 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f91481f5e10 0x7f91481f6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:59.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:59.293+0000 7f914ce40640 1 --2- 172.21.15.112:0/1002039637 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f9128078bc0 0x7f912807b0d0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:59.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:59.293+0000 7f914ce40640 1 --2- 172.21.15.112:0/1002039637 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9134005330 0x7f91480cc6f0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:59.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:59.293+0000 7f914ce40640 1 -- 172.21.15.112:0/1002039637 >> 172.21.15.112:0/1002039637 conn(0x7f91480b2cc0 msgr2=0x7f91480b30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:59.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:59.293+0000 7f914ce40640 1 -- 172.21.15.112:0/1002039637 shutdown_connections 2024-09-21T00:25:59.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:59.293+0000 7f914ce40640 1 -- 172.21.15.112:0/1002039637 wait complete. 2024-09-21T00:25:59.306 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-21T00:25:59.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:59.419+0000 7fcf58bcc640 1 -- 172.21.15.112:0/511271994 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf3c004d90 msgr2=0x7fcf3c005220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:59.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:59.419+0000 7fcf58bcc640 1 --2- 172.21.15.112:0/511271994 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf3c004d90 0x7fcf3c005220 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c009b50 tx=0x7fcf3c005830 comp rx=0 tx=0).stop 2024-09-21T00:25:59.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:59.419+0000 7fcf52d9d640 1 -- 172.21.15.112:0/511271994 reap_dead start 2024-09-21T00:25:59.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:59.419+0000 7fcf58bcc640 1 -- 172.21.15.112:0/511271994 shutdown_connections 2024-09-21T00:25:59.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:59.419+0000 7fcf58bcc640 1 -- 172.21.15.112:0/511271994 >> 172.21.15.112:0/511271994 conn(0x7fcf5409d830 msgr2=0x7fcf540b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:25:59.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:59.419+0000 7fcf58bcc640 1 -- 172.21.15.112:0/511271994 shutdown_connections 2024-09-21T00:25:59.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:59.419+0000 7fcf58bcc640 1 -- 172.21.15.112:0/511271994 wait complete. 2024-09-21T00:25:59.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:59.421+0000 7fcf58bcc640 1 Processor -- start 2024-09-21T00:25:59.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:59.421+0000 7fcf58bcc640 1 -- start start 2024-09-21T00:25:59.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:59.422+0000 7fcf58bcc640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcf3c004d90 0x7fcf540cb940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:59.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:59.423+0000 7fcf58bcc640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcf54156b30 0x7fcf540d4250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:59.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:59.423+0000 7fcf58bcc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf541489d0 0x7fcf54148de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:59.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:59.423+0000 7fcf58bcc640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcf540cf330 con 0x7fcf541489d0 2024-09-21T00:25:59.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:59.423+0000 7fcf58bcc640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fcf540cf1b0 con 0x7fcf3c004d90 2024-09-21T00:25:59.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:59.423+0000 7fcf58bcc640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fcf540b6580 con 0x7fcf54156b30 2024-09-21T00:25:59.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:59.423+0000 7fcf52d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf541489d0 0x7fcf54148de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:59.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:59.423+0000 7fcf52d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf541489d0 0x7fcf54148de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45528/0 (socket says 172.21.15.112:45528) 2024-09-21T00:25:59.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:59.423+0000 7fcf52d9d640 1 -- 172.21.15.112:0/2051317615 learned_addr learned my addr 172.21.15.112:0/2051317615 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:25:59.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:59.423+0000 7fcf5259c640 1 --2- 172.21.15.112:0/2051317615 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcf3c004d90 0x7fcf540cb940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:59.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:59.423+0000 7fcf51d9b640 1 --2- 172.21.15.112:0/2051317615 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcf54156b30 0x7fcf540d4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:59.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:59.423+0000 7fcf52d9d640 1 -- 172.21.15.112:0/2051317615 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcf54156b30 msgr2=0x7fcf540d4250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:59.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:59.423+0000 7fcf52d9d640 1 --2- 172.21.15.112:0/2051317615 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcf54156b30 0x7fcf540d4250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:59.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:59.423+0000 7fcf52d9d640 1 -- 172.21.15.112:0/2051317615 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcf3c004d90 msgr2=0x7fcf540cb940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:25:59.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:59.423+0000 7fcf52d9d640 1 --2- 172.21.15.112:0/2051317615 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcf3c004d90 0x7fcf540cb940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:25:59.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:59.423+0000 7fcf52d9d640 1 -- 172.21.15.112:0/2051317615 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf540d31b0 con 0x7fcf541489d0 2024-09-21T00:25:59.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:59.423+0000 7fcf51d9b640 1 --2- 172.21.15.112:0/2051317615 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcf54156b30 0x7fcf540d4250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:25:59.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:59.423+0000 7fcf52d9d640 1 --2- 172.21.15.112:0/2051317615 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf541489d0 0x7fcf54148de0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c036040 tx=0x7fcf4c033670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:25:59.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:59.423+0000 7fcf42ffd640 1 -- 172.21.15.112:0/2051317615 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf4c038040 con 0x7fcf541489d0 2024-09-21T00:25:59.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:59.423+0000 7fcf42ffd640 1 -- 172.21.15.112:0/2051317615 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcf4c004020 con 0x7fcf541489d0 2024-09-21T00:25:59.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:59.423+0000 7fcf58bcc640 1 -- 172.21.15.112:0/2051317615 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf540d3490 con 0x7fcf541489d0 2024-09-21T00:25:59.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:59.423+0000 7fcf42ffd640 1 -- 172.21.15.112:0/2051317615 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf4c004470 con 0x7fcf541489d0 2024-09-21T00:25:59.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:59.423+0000 7fcf58bcc640 1 -- 172.21.15.112:0/2051317615 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf540d3a50 con 0x7fcf541489d0 2024-09-21T00:25:59.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:59.424+0000 7fcf58bcc640 1 -- 172.21.15.112:0/2051317615 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf540d2f20 con 0x7fcf541489d0 2024-09-21T00:25:59.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:59.425+0000 7fcf42ffd640 1 -- 172.21.15.112:0/2051317615 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fcf4c044030 con 0x7fcf541489d0 2024-09-21T00:25:59.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:59.426+0000 7fcf42ffd640 1 --2- 172.21.15.112:0/2051317615 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fcf34078bc0 0x7fcf3407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:25:59.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:59.426+0000 7fcf42ffd640 1 -- 172.21.15.112:0/2051317615 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fcf4c0bb060 con 0x7fcf541489d0 2024-09-21T00:25:59.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:59.426+0000 7fcf5259c640 1 --2- 172.21.15.112:0/2051317615 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fcf34078bc0 0x7fcf3407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:25:59.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:59.426+0000 7fcf5259c640 1 --2- 172.21.15.112:0/2051317615 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fcf34078bc0 0x7fcf3407b0d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fcf44003e50 tx=0x7fcf44006e90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:25:59.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:59.430+0000 7fcf42ffd640 1 -- 172.21.15.112:0/2051317615 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fcf4c04b020 con 0x7fcf541489d0 2024-09-21T00:25:59.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:25:59.645+0000 7fcf58bcc640 1 -- 172.21.15.112:0/2051317615 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fcf540b5bb0 con 0x7fcf541489d0 2024-09-21T00:26:00.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:00.293+0000 7fcf42ffd640 1 -- 172.21.15.112:0/2051317615 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v8) ==== 118+0+0 (secure 0 0 0) 0x7fcf4c082ba0 con 0x7fcf541489d0 2024-09-21T00:26:00.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:00.295+0000 7fcf58bcc640 1 -- 172.21.15.112:0/2051317615 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fcf34078bc0 msgr2=0x7fcf3407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:00.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:00.295+0000 7fcf58bcc640 1 --2- 172.21.15.112:0/2051317615 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fcf34078bc0 0x7fcf3407b0d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fcf44003e50 tx=0x7fcf44006e90 comp rx=0 tx=0).stop 2024-09-21T00:26:00.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:00.295+0000 7fcf58bcc640 1 -- 172.21.15.112:0/2051317615 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf541489d0 msgr2=0x7fcf54148de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:00.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:00.295+0000 7fcf58bcc640 1 --2- 172.21.15.112:0/2051317615 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf541489d0 0x7fcf54148de0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c036040 tx=0x7fcf4c033670 comp rx=0 tx=0).stop 2024-09-21T00:26:00.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:00.295+0000 7fcf58bcc640 1 -- 172.21.15.112:0/2051317615 shutdown_connections 2024-09-21T00:26:00.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:00.295+0000 7fcf58bcc640 1 --2- 172.21.15.112:0/2051317615 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fcf34078bc0 0x7fcf3407b0d0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:00.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:00.295+0000 7fcf58bcc640 1 --2- 172.21.15.112:0/2051317615 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf541489d0 0x7fcf54148de0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:00.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:00.295+0000 7fcf58bcc640 1 --2- 172.21.15.112:0/2051317615 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcf54156b30 0x7fcf540d4250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:00.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:00.295+0000 7fcf58bcc640 1 --2- 172.21.15.112:0/2051317615 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcf3c004d90 0x7fcf540cb940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:00.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:00.295+0000 7fcf58bcc640 1 -- 172.21.15.112:0/2051317615 >> 172.21.15.112:0/2051317615 conn(0x7fcf5409d830 msgr2=0x7fcf540b4c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:00.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:00.295+0000 7fcf58bcc640 1 -- 172.21.15.112:0/2051317615 shutdown_connections 2024-09-21T00:26:00.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:00.295+0000 7fcf58bcc640 1 -- 172.21.15.112:0/2051317615 wait complete. 2024-09-21T00:26:00.309 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']} 2024-09-21T00:26:00.309 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-21T00:26:00.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:00.423+0000 7f2223f39640 1 -- 172.21.15.112:0/3307591128 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f221c06def0 msgr2=0x7f221c06f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:00.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:00.423+0000 7f2223f39640 1 --2- 172.21.15.112:0/3307591128 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f221c06def0 0x7f221c06f410 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2210009830 tx=0x7f221002a4a0 comp rx=0 tx=0).stop 2024-09-21T00:26:00.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:00.423+0000 7f2223f39640 1 -- 172.21.15.112:0/3307591128 shutdown_connections 2024-09-21T00:26:00.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:00.423+0000 7f2223f39640 1 --2- 172.21.15.112:0/3307591128 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f221c0b5670 0x7f221c15f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:00.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:00.423+0000 7f2223f39640 1 --2- 172.21.15.112:0/3307591128 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f221c06def0 0x7f221c06f410 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:00.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:00.423+0000 7f2223f39640 1 -- 172.21.15.112:0/3307591128 >> 172.21.15.112:0/3307591128 conn(0x7f221c0cdca0 msgr2=0x7f221c0ce0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:00.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:00.423+0000 7f2223f39640 1 -- 172.21.15.112:0/3307591128 shutdown_connections 2024-09-21T00:26:00.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:00.423+0000 7f2223f39640 1 -- 172.21.15.112:0/3307591128 wait complete. 2024-09-21T00:26:00.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:00.425+0000 7f2223f39640 1 Processor -- start 2024-09-21T00:26:00.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:00.425+0000 7f2223f39640 1 -- start start 2024-09-21T00:26:00.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:00.426+0000 7f2223f39640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f221c06def0 0x7f221c1f9370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:00.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:00.426+0000 7f2223f39640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f221c0b5260 0x7f221c14f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:00.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:00.426+0000 7f2223f39640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f221c0b5670 0x7f221c15a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:00.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:00.426+0000 7f2223f39640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f221c15ffb0 con 0x7f221c06def0 2024-09-21T00:26:00.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:00.426+0000 7f2223f39640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f221c15fe30 con 0x7f221c0b5670 2024-09-21T00:26:00.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:00.426+0000 7f2223f39640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f221c071300 con 0x7f221c0b5260 2024-09-21T00:26:00.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:00.426+0000 7f2221cd5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f221c06def0 0x7f221c1f9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:00.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:00.426+0000 7f2221cd5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f221c06def0 0x7f221c1f9370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45532/0 (socket says 172.21.15.112:45532) 2024-09-21T00:26:00.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:00.426+0000 7f2221cd5640 1 -- 172.21.15.112:0/919641530 learned_addr learned my addr 172.21.15.112:0/919641530 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:26:00.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:00.426+0000 7f22214d4640 1 --2- 172.21.15.112:0/919641530 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f221c0b5260 0x7f221c14f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:00.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:00.426+0000 7f22224d6640 1 --2- 172.21.15.112:0/919641530 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f221c0b5670 0x7f221c15a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:00.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:00.427+0000 7f2221cd5640 1 -- 172.21.15.112:0/919641530 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f221c0b5260 msgr2=0x7f221c14f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:00.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:00.427+0000 7f2221cd5640 1 --2- 172.21.15.112:0/919641530 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f221c0b5260 0x7f221c14f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:00.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:00.427+0000 7f2221cd5640 1 -- 172.21.15.112:0/919641530 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f221c0b5670 msgr2=0x7f221c15a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:00.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:00.427+0000 7f2221cd5640 1 --2- 172.21.15.112:0/919641530 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f221c0b5670 0x7f221c15a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:00.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:00.427+0000 7f2221cd5640 1 -- 172.21.15.112:0/919641530 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f221c14f990 con 0x7f221c06def0 2024-09-21T00:26:00.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:00.427+0000 7f2221cd5640 1 --2- 172.21.15.112:0/919641530 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f221c06def0 0x7f221c1f9370 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f221002aa90 tx=0x7f2210003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:00.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:00.427+0000 7f220e7fc640 1 -- 172.21.15.112:0/919641530 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2210005c90 con 0x7f221c06def0 2024-09-21T00:26:00.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:00.427+0000 7f2223f39640 1 -- 172.21.15.112:0/919641530 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f221c0cb7a0 con 0x7f221c06def0 2024-09-21T00:26:00.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:00.427+0000 7f220e7fc640 1 -- 172.21.15.112:0/919641530 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2210005020 con 0x7f221c06def0 2024-09-21T00:26:00.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:00.427+0000 7f220e7fc640 1 -- 172.21.15.112:0/919641530 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2210005450 con 0x7f221c06def0 2024-09-21T00:26:00.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:00.427+0000 7f2223f39640 1 -- 172.21.15.112:0/919641530 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f221c09b700 con 0x7f221c06def0 2024-09-21T00:26:00.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:00.428+0000 7f2223f39640 1 -- 172.21.15.112:0/919641530 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f221c0cb970 con 0x7f221c06def0 2024-09-21T00:26:00.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:00.429+0000 7f220e7fc640 1 -- 172.21.15.112:0/919641530 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f22100317c0 con 0x7f221c06def0 2024-09-21T00:26:00.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:00.429+0000 7f220e7fc640 1 --2- 172.21.15.112:0/919641530 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2200078b90 0x7f220007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:00.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:00.429+0000 7f220e7fc640 1 -- 172.21.15.112:0/919641530 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f22100bc070 con 0x7f221c06def0 2024-09-21T00:26:00.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:00.429+0000 7f22214d4640 1 --2- 172.21.15.112:0/919641530 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2200078b90 0x7f220007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:00.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:00.430+0000 7f22214d4640 1 --2- 172.21.15.112:0/919641530 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2200078b90 0x7f220007b0a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2218001eb0 tx=0x7f2218009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:00.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:00.434+0000 7f220e7fc640 1 -- 172.21.15.112:0/919641530 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f221c14f990 con 0x7f221c06def0 2024-09-21T00:26:00.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:00.649+0000 7f2223f39640 1 -- 172.21.15.112:0/919641530 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f221c15f630 con 0x7f221c06def0 2024-09-21T00:26:01.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:01.298+0000 7f220e7fc640 1 -- 172.21.15.112:0/919641530 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v9) ==== 122+0+0 (secure 0 0 0) 0x7f221003f030 con 0x7f221c06def0 2024-09-21T00:26:01.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:01.301+0000 7f2223f39640 1 -- 172.21.15.112:0/919641530 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2200078b90 msgr2=0x7f220007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:01.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:01.301+0000 7f2223f39640 1 --2- 172.21.15.112:0/919641530 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2200078b90 0x7f220007b0a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2218001eb0 tx=0x7f2218009040 comp rx=0 tx=0).stop 2024-09-21T00:26:01.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:01.301+0000 7f2223f39640 1 -- 172.21.15.112:0/919641530 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f221c06def0 msgr2=0x7f221c1f9370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:01.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:01.301+0000 7f2223f39640 1 --2- 172.21.15.112:0/919641530 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f221c06def0 0x7f221c1f9370 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f221002aa90 tx=0x7f2210003c10 comp rx=0 tx=0).stop 2024-09-21T00:26:01.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:01.301+0000 7f2223f39640 1 -- 172.21.15.112:0/919641530 shutdown_connections 2024-09-21T00:26:01.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:01.301+0000 7f2223f39640 1 --2- 172.21.15.112:0/919641530 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f221c0b5670 0x7f221c15a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:01.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:01.301+0000 7f2223f39640 1 --2- 172.21.15.112:0/919641530 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f221c0b5260 0x7f221c14f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:01.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:01.301+0000 7f2223f39640 1 --2- 172.21.15.112:0/919641530 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2200078b90 0x7f220007b0a0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:01.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:01.301+0000 7f2223f39640 1 --2- 172.21.15.112:0/919641530 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f221c06def0 0x7f221c1f9370 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:01.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:01.301+0000 7f2223f39640 1 -- 172.21.15.112:0/919641530 >> 172.21.15.112:0/919641530 conn(0x7f221c0cdca0 msgr2=0x7f221c0ce0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:01.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:01.301+0000 7f2223f39640 1 -- 172.21.15.112:0/919641530 shutdown_connections 2024-09-21T00:26:01.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:01.301+0000 7f2223f39640 1 -- 172.21.15.112:0/919641530 wait complete. 2024-09-21T00:26:01.315 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-21T00:26:01.315 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T00:26:01.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:01.427+0000 7ff5dd494640 1 -- 172.21.15.112:0/1645112342 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5d80cfe00 msgr2=0x7ff5d80d2a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:01.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:01.427+0000 7ff5dd494640 1 --2- 172.21.15.112:0/1645112342 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5d80cfe00 0x7ff5d80d2a20 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff5d80b5a40 tx=0x7ff5c802a560 comp rx=0 tx=0).stop 2024-09-21T00:26:01.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:01.427+0000 7ff5dd494640 1 -- 172.21.15.112:0/1645112342 shutdown_connections 2024-09-21T00:26:01.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:01.427+0000 7ff5dd494640 1 --2- 172.21.15.112:0/1645112342 >> v2:172.21.15.187:3300/0 conn(0x7ff5d80b2e60 0x7ff5d80b3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:01.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:01.427+0000 7ff5dd494640 1 --2- 172.21.15.112:0/1645112342 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5d80cfe00 0x7ff5d80d2a20 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:01.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:01.427+0000 7ff5dd494640 1 -- 172.21.15.112:0/1645112342 >> 172.21.15.112:0/1645112342 conn(0x7ff5d80d4440 msgr2=0x7ff5d80d4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:01.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:01.427+0000 7ff5dd494640 1 -- 172.21.15.112:0/1645112342 shutdown_connections 2024-09-21T00:26:01.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:01.427+0000 7ff5dd494640 1 -- 172.21.15.112:0/1645112342 wait complete. 2024-09-21T00:26:01.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:01.429+0000 7ff5dd494640 1 Processor -- start 2024-09-21T00:26:01.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:01.429+0000 7ff5dd494640 1 -- start start 2024-09-21T00:26:01.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:01.430+0000 7ff5dd494640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5d80b2e60 0x7ff5d815b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:01.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:01.431+0000 7ff5dd494640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff5d80cfe00 0x7ff5d81595d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:01.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:01.431+0000 7ff5dd494640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff5d8162590 0x7ff5d8159b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:01.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:01.431+0000 7ff5dd494640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff5d80b3930 con 0x7ff5d80b2e60 2024-09-21T00:26:01.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:01.431+0000 7ff5dd494640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff5d80b37b0 con 0x7ff5d8162590 2024-09-21T00:26:01.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:01.431+0000 7ff5dd494640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff5d80b3ab0 con 0x7ff5d80cfe00 2024-09-21T00:26:01.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:01.431+0000 7ff5d6ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5d80b2e60 0x7ff5d815b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:01.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:01.431+0000 7ff5d6ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5d80b2e60 0x7ff5d815b930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45558/0 (socket says 172.21.15.112:45558) 2024-09-21T00:26:01.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:01.431+0000 7ff5d6ffd640 1 -- 172.21.15.112:0/2007249621 learned_addr learned my addr 172.21.15.112:0/2007249621 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:26:01.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:01.431+0000 7ff5ce7fc640 1 --2- 172.21.15.112:0/2007249621 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff5d80cfe00 0x7ff5d81595d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:01.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:01.431+0000 7ff5d77fe640 1 --2- 172.21.15.112:0/2007249621 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff5d8162590 0x7ff5d8159b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:01.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:01.431+0000 7ff5d6ffd640 1 -- 172.21.15.112:0/2007249621 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff5d80cfe00 msgr2=0x7ff5d81595d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:01.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:01.431+0000 7ff5d6ffd640 1 --2- 172.21.15.112:0/2007249621 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff5d80cfe00 0x7ff5d81595d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:01.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:01.431+0000 7ff5d6ffd640 1 -- 172.21.15.112:0/2007249621 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff5d8162590 msgr2=0x7ff5d8159b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:01.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:01.431+0000 7ff5d6ffd640 1 --2- 172.21.15.112:0/2007249621 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff5d8162590 0x7ff5d8159b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:01.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:01.431+0000 7ff5d6ffd640 1 -- 172.21.15.112:0/2007249621 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5d8156de0 con 0x7ff5d80b2e60 2024-09-21T00:26:01.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:01.431+0000 7ff5d6ffd640 1 --2- 172.21.15.112:0/2007249621 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5d80b2e60 0x7ff5d815b930 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff5c8000c00 tx=0x7ff5c80039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:01.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:01.431+0000 7ff5cffff640 1 -- 172.21.15.112:0/2007249621 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c803e030 con 0x7ff5d80b2e60 2024-09-21T00:26:01.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:01.431+0000 7ff5cffff640 1 -- 172.21.15.112:0/2007249621 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff5c80047a0 con 0x7ff5d80b2e60 2024-09-21T00:26:01.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:01.431+0000 7ff5dd494640 1 -- 172.21.15.112:0/2007249621 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5d81570c0 con 0x7ff5d80b2e60 2024-09-21T00:26:01.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:01.431+0000 7ff5cffff640 1 -- 172.21.15.112:0/2007249621 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c8035040 con 0x7ff5d80b2e60 2024-09-21T00:26:01.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:01.431+0000 7ff5dd494640 1 -- 172.21.15.112:0/2007249621 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5d8157680 con 0x7ff5d80b2e60 2024-09-21T00:26:01.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:01.432+0000 7ff5dd494640 1 -- 172.21.15.112:0/2007249621 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5d80b3270 con 0x7ff5d80b2e60 2024-09-21T00:26:01.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:01.433+0000 7ff5cffff640 1 -- 172.21.15.112:0/2007249621 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff5c8041030 con 0x7ff5d80b2e60 2024-09-21T00:26:01.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:01.434+0000 7ff5cffff640 1 --2- 172.21.15.112:0/2007249621 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff5b8078b90 0x7ff5b807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:01.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:01.434+0000 7ff5cffff640 1 -- 172.21.15.112:0/2007249621 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7ff5c80bb8b0 con 0x7ff5d80b2e60 2024-09-21T00:26:01.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:01.434+0000 7ff5ce7fc640 1 --2- 172.21.15.112:0/2007249621 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff5b8078b90 0x7ff5b807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:01.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:01.434+0000 7ff5ce7fc640 1 --2- 172.21.15.112:0/2007249621 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff5b8078b90 0x7ff5b807b0a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff5d8154360 tx=0x7ff5c0009060 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:01.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:01.438+0000 7ff5cffff640 1 -- 172.21.15.112:0/2007249621 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ff5c8083360 con 0x7ff5d80b2e60 2024-09-21T00:26:01.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:01.653+0000 7ff5dd494640 1 -- 172.21.15.112:0/2007249621 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff5d8159120 con 0x7ff5d80b2e60 2024-09-21T00:26:01.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:01.654+0000 7ff5cffff640 1 -- 172.21.15.112:0/2007249621 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5228 (secure 0 0 0) 0x7ff5c8087d90 con 0x7ff5d80b2e60 2024-09-21T00:26:01.656 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:26:01.656 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":9,"btime":"2024-09-21T00:26:01:294823+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4316,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6837/2271985854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6835","nonce":2271985854},{"type":"v1","addr":"172.21.15.187:6837","nonce":2271985854}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4413,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/200348130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":200348130},{"type":"v1","addr":"172.21.15.112:6833","nonce":200348130}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4416,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/3821201561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3821201561},{"type":"v1","addr":"172.21.15.112:6835","nonce":3821201561}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T00:25:47.578960+0000","modified":"2024-09-21T00:26:00.650942+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4425},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4425":{"gid":4425,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6836/2921304789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":2921304789},{"type":"v1","addr":"172.21.15.187:6836","nonce":2921304789}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4425,"qdb_cluster":[4425]},"id":1}]} 2024-09-21T00:26:01.656 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 9 2024-09-21T00:26:01.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:01.656+0000 7ff5dd494640 1 -- 172.21.15.112:0/2007249621 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff5b8078b90 msgr2=0x7ff5b807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:01.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:01.656+0000 7ff5dd494640 1 --2- 172.21.15.112:0/2007249621 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff5b8078b90 0x7ff5b807b0a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff5d8154360 tx=0x7ff5c0009060 comp rx=0 tx=0).stop 2024-09-21T00:26:01.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:01.656+0000 7ff5dd494640 1 -- 172.21.15.112:0/2007249621 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5d80b2e60 msgr2=0x7ff5d815b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:01.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:01.656+0000 7ff5dd494640 1 --2- 172.21.15.112:0/2007249621 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5d80b2e60 0x7ff5d815b930 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff5c8000c00 tx=0x7ff5c80039e0 comp rx=0 tx=0).stop 2024-09-21T00:26:01.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:01.656+0000 7ff5dd494640 1 -- 172.21.15.112:0/2007249621 shutdown_connections 2024-09-21T00:26:01.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:01.656+0000 7ff5dd494640 1 --2- 172.21.15.112:0/2007249621 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff5d8162590 0x7ff5d8159b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:01.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:01.656+0000 7ff5dd494640 1 --2- 172.21.15.112:0/2007249621 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff5d80cfe00 0x7ff5d81595d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:01.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:01.656+0000 7ff5dd494640 1 --2- 172.21.15.112:0/2007249621 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff5b8078b90 0x7ff5b807b0a0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:01.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:01.656+0000 7ff5dd494640 1 --2- 172.21.15.112:0/2007249621 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5d80b2e60 0x7ff5d815b930 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:01.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:01.656+0000 7ff5dd494640 1 -- 172.21.15.112:0/2007249621 >> 172.21.15.112:0/2007249621 conn(0x7ff5d80d4440 msgr2=0x7ff5d806d9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:01.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:01.656+0000 7ff5dd494640 1 -- 172.21.15.112:0/2007249621 shutdown_connections 2024-09-21T00:26:01.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:01.656+0000 7ff5dd494640 1 -- 172.21.15.112:0/2007249621 wait complete. 2024-09-21T00:26:01.669 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T00:26:01.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:01.780+0000 7f8400ef2640 1 -- 172.21.15.112:0/3120888647 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83fc162f00 msgr2=0x7f83fc0d24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:01.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:01.780+0000 7f8400ef2640 1 --2- 172.21.15.112:0/3120888647 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83fc162f00 0x7f83fc0d24d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f83ec005fd0 tx=0x7f83ec02a530 comp rx=0 tx=0).stop 2024-09-21T00:26:01.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:01.780+0000 7f8400ef2640 1 -- 172.21.15.112:0/3120888647 shutdown_connections 2024-09-21T00:26:01.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:01.780+0000 7f8400ef2640 1 --2- 172.21.15.112:0/3120888647 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f83fc162280 0x7f83fc162690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:01.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:01.780+0000 7f8400ef2640 1 --2- 172.21.15.112:0/3120888647 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83fc162f00 0x7f83fc0d24d0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:01.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:01.780+0000 7f8400ef2640 1 -- 172.21.15.112:0/3120888647 >> 172.21.15.112:0/3120888647 conn(0x7f83fc0b6b20 msgr2=0x7f83fc0d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:01.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:01.780+0000 7f8400ef2640 1 -- 172.21.15.112:0/3120888647 shutdown_connections 2024-09-21T00:26:01.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:01.780+0000 7f8400ef2640 1 -- 172.21.15.112:0/3120888647 wait complete. 2024-09-21T00:26:01.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:01.782+0000 7f8400ef2640 1 Processor -- start 2024-09-21T00:26:01.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:01.782+0000 7f8400ef2640 1 -- start start 2024-09-21T00:26:01.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:01.783+0000 7f8400ef2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83fc0b2e40 0x7f83fc0acbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:01.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:01.783+0000 7f8400ef2640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f83fc162280 0x7f83fc1f58d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:01.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:01.783+0000 7f8400ef2640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f83fc1f5e10 0x7f83fc15ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:01.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:01.783+0000 7f8400ef2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f83fc15e630 con 0x7f83fc0b2e40 2024-09-21T00:26:01.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:01.783+0000 7f8400ef2640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f83fc15e4b0 con 0x7f83fc1f5e10 2024-09-21T00:26:01.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:01.783+0000 7f8400ef2640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f83fc15e7b0 con 0x7f83fc162280 2024-09-21T00:26:01.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:01.783+0000 7f83fa59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83fc0b2e40 0x7f83fc0acbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:01.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:01.783+0000 7f83fa59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83fc0b2e40 0x7f83fc0acbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45576/0 (socket says 172.21.15.112:45576) 2024-09-21T00:26:01.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:01.783+0000 7f83fa59c640 1 -- 172.21.15.112:0/3449747245 learned_addr learned my addr 172.21.15.112:0/3449747245 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:26:01.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:01.783+0000 7f83f9d9b640 1 --2- 172.21.15.112:0/3449747245 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f83fc162280 0x7f83fc1f58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:01.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:01.783+0000 7f83fad9d640 1 --2- 172.21.15.112:0/3449747245 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f83fc1f5e10 0x7f83fc15ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:01.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:01.784+0000 7f83fa59c640 1 -- 172.21.15.112:0/3449747245 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f83fc162280 msgr2=0x7f83fc1f58d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:01.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:01.784+0000 7f83fa59c640 1 --2- 172.21.15.112:0/3449747245 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f83fc162280 0x7f83fc1f58d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:01.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:01.784+0000 7f83fa59c640 1 -- 172.21.15.112:0/3449747245 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f83fc1f5e10 msgr2=0x7f83fc15ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:01.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:01.784+0000 7f83fa59c640 1 --2- 172.21.15.112:0/3449747245 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f83fc1f5e10 0x7f83fc15ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:01.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:01.784+0000 7f83fa59c640 1 -- 172.21.15.112:0/3449747245 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83fc0ad4f0 con 0x7f83fc0b2e40 2024-09-21T00:26:01.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:01.784+0000 7f83fa59c640 1 --2- 172.21.15.112:0/3449747245 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83fc0b2e40 0x7f83fc0acbd0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f83ec034040 tx=0x7f83ec0039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:01.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:01.784+0000 7f83eaffd640 1 -- 172.21.15.112:0/3449747245 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83ec03e030 con 0x7f83fc0b2e40 2024-09-21T00:26:01.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:01.784+0000 7f83eaffd640 1 -- 172.21.15.112:0/3449747245 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f83ec0047a0 con 0x7f83fc0b2e40 2024-09-21T00:26:01.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:01.784+0000 7f8400ef2640 1 -- 172.21.15.112:0/3449747245 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83fc0abf90 con 0x7f83fc0b2e40 2024-09-21T00:26:01.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:01.784+0000 7f83eaffd640 1 -- 172.21.15.112:0/3449747245 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83ec035040 con 0x7f83fc0b2e40 2024-09-21T00:26:01.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:01.784+0000 7f8400ef2640 1 -- 172.21.15.112:0/3449747245 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83fc0ab1d0 con 0x7f83fc0b2e40 2024-09-21T00:26:01.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:01.785+0000 7f8400ef2640 1 -- 172.21.15.112:0/3449747245 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83fc162f00 con 0x7f83fc0b2e40 2024-09-21T00:26:01.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:01.786+0000 7f83eaffd640 1 -- 172.21.15.112:0/3449747245 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f83ec041030 con 0x7f83fc0b2e40 2024-09-21T00:26:01.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:01.786+0000 7f83eaffd640 1 --2- 172.21.15.112:0/3449747245 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f83dc078b90 0x7f83dc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:01.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:01.786+0000 7f83eaffd640 1 -- 172.21.15.112:0/3449747245 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f83ec0bb820 con 0x7f83fc0b2e40 2024-09-21T00:26:01.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:01.786+0000 7f83f9d9b640 1 --2- 172.21.15.112:0/3449747245 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f83dc078b90 0x7f83dc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:01.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:01.787+0000 7f83f9d9b640 1 --2- 172.21.15.112:0/3449747245 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f83dc078b90 0x7f83dc07b0a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f83fc15b720 tx=0x7f83f000a2d0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:01.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:01.791+0000 7f83eaffd640 1 -- 172.21.15.112:0/3449747245 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f83fc162bd0 con 0x7f83fc0b2e40 2024-09-21T00:26:01.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:01.971+0000 7f8400ef2640 1 -- 172.21.15.112:0/3449747245 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f83fc0ac9b0 con 0x7f83fc0b2e40 2024-09-21T00:26:01.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:01.972+0000 7f83eaffd640 1 -- 172.21.15.112:0/3449747245 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18622 (secure 0 0 0) 0x7f83ec083360 con 0x7f83fc0b2e40 2024-09-21T00:26:01.974 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:26:01.974 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":26,"fsid":"8b34692b-dc65-49b2-a08f-26f707b96ac5","created":"2024-09-21T00:25:22.788490+0000","modified":"2024-09-21T00:25:52.623213+0000","last_up_change":"2024-09-21T00:25:36.161835+0000","last_in_change":"2024-09-21T00:25:26.346500+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T00:25:37.105006+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T00:25:45.413895+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T00:25:46.532124+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T00:25:50.244145+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9c575086-67d2-4de2-b01b-d44dcc21d604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6809","nonce":755760621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6811","nonce":755760621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6815","nonce":755760621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6813","nonce":755760621}]},"public_addr":"172.21.15.112:6809/755760621","cluster_addr":"172.21.15.112:6811/755760621","heartbeat_back_addr":"172.21.15.112:6815/755760621","heartbeat_front_addr":"172.21.15.112:6813/755760621","state":["exists","up"]},{"osd":1,"uuid":"a983c227-a3ce-4bdd-82f1-438ba7e3ecd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6801","nonce":344932592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6803","nonce":344932592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6807","nonce":344932592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6805","nonce":344932592}]},"public_addr":"172.21.15.112:6801/344932592","cluster_addr":"172.21.15.112:6803/344932592","heartbeat_back_addr":"172.21.15.112:6807/344932592","heartbeat_front_addr":"172.21.15.112:6805/344932592","state":["exists","up"]},{"osd":2,"uuid":"1193c1d8-bfff-4a57-8200-5f69bf1ea3af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6825","nonce":407933616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6827","nonce":407933616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6831","nonce":407933616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6829","nonce":407933616}]},"public_addr":"172.21.15.112:6825/407933616","cluster_addr":"172.21.15.112:6827/407933616","heartbeat_back_addr":"172.21.15.112:6831/407933616","heartbeat_front_addr":"172.21.15.112:6829/407933616","state":["exists","up"]},{"osd":3,"uuid":"af04330a-36ad-40cc-a8dd-fa259b221de5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6817","nonce":4120164806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6819","nonce":4120164806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6823","nonce":4120164806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6821","nonce":4120164806}]},"public_addr":"172.21.15.112:6817/4120164806","cluster_addr":"172.21.15.112:6819/4120164806","heartbeat_back_addr":"172.21.15.112:6823/4120164806","heartbeat_front_addr":"172.21.15.112:6821/4120164806","state":["exists","up"]},{"osd":4,"uuid":"5b893111-292c-4c5a-b184-53ad38c7199e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6803","nonce":522077560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6805","nonce":522077560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6809","nonce":522077560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6807","nonce":522077560}]},"public_addr":"172.21.15.187:6803/522077560","cluster_addr":"172.21.15.187:6805/522077560","heartbeat_back_addr":"172.21.15.187:6809/522077560","heartbeat_front_addr":"172.21.15.187:6807/522077560","state":["exists","up"]},{"osd":5,"uuid":"c0b8e011-45fe-4c5e-925e-6be2a7f1868a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6827","nonce":3593144721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6829","nonce":3593144721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6833","nonce":3593144721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6831","nonce":3593144721}]},"public_addr":"172.21.15.187:6827/3593144721","cluster_addr":"172.21.15.187:6829/3593144721","heartbeat_back_addr":"172.21.15.187:6833/3593144721","heartbeat_front_addr":"172.21.15.187:6831/3593144721","state":["exists","up"]},{"osd":6,"uuid":"87d2d650-7489-4593-98a3-a1f433ce8dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6811","nonce":1494387657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6813","nonce":1494387657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6817","nonce":1494387657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6815","nonce":1494387657}]},"public_addr":"172.21.15.187:6811/1494387657","cluster_addr":"172.21.15.187:6813/1494387657","heartbeat_back_addr":"172.21.15.187:6817/1494387657","heartbeat_front_addr":"172.21.15.187:6815/1494387657","state":["exists","up"]},{"osd":7,"uuid":"c6b4f8e2-a1ee-4438-aa37-67ff4b6ef1f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6819","nonce":1624699842}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6821","nonce":1624699842}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6825","nonce":1624699842}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6823","nonce":1624699842}]},"public_addr":"172.21.15.187:6819/1624699842","cluster_addr":"172.21.15.187:6821/1624699842","heartbeat_back_addr":"172.21.15.187:6825/1624699842","heartbeat_front_addr":"172.21.15.187:6823/1624699842","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.873162+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.400506+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.664317+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.159791+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.202630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.113212+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.384786+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.448441+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T00:26:01.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:01.974+0000 7f8400ef2640 1 -- 172.21.15.112:0/3449747245 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f83dc078b90 msgr2=0x7f83dc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:01.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:01.974+0000 7f8400ef2640 1 --2- 172.21.15.112:0/3449747245 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f83dc078b90 0x7f83dc07b0a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f83fc15b720 tx=0x7f83f000a2d0 comp rx=0 tx=0).stop 2024-09-21T00:26:01.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:01.974+0000 7f8400ef2640 1 -- 172.21.15.112:0/3449747245 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83fc0b2e40 msgr2=0x7f83fc0acbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:01.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:01.974+0000 7f8400ef2640 1 --2- 172.21.15.112:0/3449747245 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83fc0b2e40 0x7f83fc0acbd0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f83ec034040 tx=0x7f83ec0039e0 comp rx=0 tx=0).stop 2024-09-21T00:26:01.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:01.974+0000 7f8400ef2640 1 -- 172.21.15.112:0/3449747245 shutdown_connections 2024-09-21T00:26:01.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:01.974+0000 7f8400ef2640 1 --2- 172.21.15.112:0/3449747245 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f83fc1f5e10 0x7f83fc15ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:01.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:01.974+0000 7f8400ef2640 1 --2- 172.21.15.112:0/3449747245 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f83fc162280 0x7f83fc1f58d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:01.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:01.974+0000 7f8400ef2640 1 --2- 172.21.15.112:0/3449747245 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f83dc078b90 0x7f83dc07b0a0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:01.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:01.974+0000 7f8400ef2640 1 --2- 172.21.15.112:0/3449747245 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83fc0b2e40 0x7f83fc0acbd0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:01.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:01.974+0000 7f8400ef2640 1 -- 172.21.15.112:0/3449747245 >> 172.21.15.112:0/3449747245 conn(0x7f83fc0b6b20 msgr2=0x7f83fc15eca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:01.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:01.974+0000 7f8400ef2640 1 -- 172.21.15.112:0/3449747245 shutdown_connections 2024-09-21T00:26:01.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:01.974+0000 7f8400ef2640 1 -- 172.21.15.112:0/3449747245 wait complete. 2024-09-21T00:26:01.987 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T00:26:01.988 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T00:26:02.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.099+0000 7f574c07e640 1 -- 172.21.15.112:0/4167880735 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5730004800 msgr2=0x7f5730004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:02.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.099+0000 7f574c07e640 1 --2- 172.21.15.112:0/4167880735 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5730004800 0x7f5730004c10 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5734006f90 tx=0x7f5730005830 comp rx=0 tx=0).stop 2024-09-21T00:26:02.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.099+0000 7f574a61b640 1 -- 172.21.15.112:0/4167880735 reap_dead start 2024-09-21T00:26:02.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.099+0000 7f574c07e640 1 -- 172.21.15.112:0/4167880735 shutdown_connections 2024-09-21T00:26:02.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.099+0000 7f574c07e640 1 -- 172.21.15.112:0/4167880735 >> 172.21.15.112:0/4167880735 conn(0x7f57440d0a30 msgr2=0x7f57440d0e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:02.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.099+0000 7f574c07e640 1 -- 172.21.15.112:0/4167880735 shutdown_connections 2024-09-21T00:26:02.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.099+0000 7f574c07e640 1 -- 172.21.15.112:0/4167880735 wait complete. 2024-09-21T00:26:02.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.101+0000 7f574c07e640 1 Processor -- start 2024-09-21T00:26:02.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.101+0000 7f574c07e640 1 -- start start 2024-09-21T00:26:02.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.102+0000 7f574c07e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5730004800 0x7f5744142e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:02.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.102+0000 7f574c07e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f57441f0580 0x7f5744158910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:02.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.102+0000 7f574c07e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f57441f6610 0x7f57441f6a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:02.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.102+0000 7f574c07e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f57440b0100 con 0x7f57441f6610 2024-09-21T00:26:02.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.102+0000 7f574c07e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f57440aff80 con 0x7f57441f0580 2024-09-21T00:26:02.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.102+0000 7f574c07e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f57440b0280 con 0x7f5730004800 2024-09-21T00:26:02.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.102+0000 7f574a61b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f57441f6610 0x7f57441f6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:02.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.102+0000 7f574a61b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f57441f6610 0x7f57441f6a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45590/0 (socket says 172.21.15.112:45590) 2024-09-21T00:26:02.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.102+0000 7f574a61b640 1 -- 172.21.15.112:0/3329591245 learned_addr learned my addr 172.21.15.112:0/3329591245 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:26:02.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.102+0000 7f5749e1a640 1 --2- 172.21.15.112:0/3329591245 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5730004800 0x7f5744142e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:02.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.102+0000 7f5749619640 1 --2- 172.21.15.112:0/3329591245 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f57441f0580 0x7f5744158910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:02.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.102+0000 7f574a61b640 1 -- 172.21.15.112:0/3329591245 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5730004800 msgr2=0x7f5744142e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:02.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.102+0000 7f574a61b640 1 --2- 172.21.15.112:0/3329591245 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5730004800 0x7f5744142e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:02.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.102+0000 7f574a61b640 1 -- 172.21.15.112:0/3329591245 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f57441f0580 msgr2=0x7f5744158910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:02.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.102+0000 7f574a61b640 1 --2- 172.21.15.112:0/3329591245 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f57441f0580 0x7f5744158910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:02.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.102+0000 7f574a61b640 1 -- 172.21.15.112:0/3329591245 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57441f6f60 con 0x7f57441f6610 2024-09-21T00:26:02.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.102+0000 7f574a61b640 1 --2- 172.21.15.112:0/3329591245 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f57441f6610 0x7f57441f6a20 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f57440cfda0 tx=0x7f573c006430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:02.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.103+0000 7f573a7fc640 1 -- 172.21.15.112:0/3329591245 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f573c011900 con 0x7f57441f6610 2024-09-21T00:26:02.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.103+0000 7f573a7fc640 1 -- 172.21.15.112:0/3329591245 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f573c017070 con 0x7f57441f6610 2024-09-21T00:26:02.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.103+0000 7f573a7fc640 1 -- 172.21.15.112:0/3329591245 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f573c011d20 con 0x7f57441f6610 2024-09-21T00:26:02.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.103+0000 7f574c07e640 1 -- 172.21.15.112:0/3329591245 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57441f3e10 con 0x7f57441f6610 2024-09-21T00:26:02.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.103+0000 7f574c07e640 1 -- 172.21.15.112:0/3329591245 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5744153bb0 con 0x7f57441f6610 2024-09-21T00:26:02.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.104+0000 7f574c07e640 1 -- 172.21.15.112:0/3329591245 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5744158e50 con 0x7f57441f6610 2024-09-21T00:26:02.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.104+0000 7f573a7fc640 1 -- 172.21.15.112:0/3329591245 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f573c01c460 con 0x7f57441f6610 2024-09-21T00:26:02.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.105+0000 7f573a7fc640 1 --2- 172.21.15.112:0/3329591245 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f5728078bc0 0x7f572807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:02.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.105+0000 7f573a7fc640 1 -- 172.21.15.112:0/3329591245 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f573c09b670 con 0x7f57441f6610 2024-09-21T00:26:02.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.105+0000 7f5749e1a640 1 --2- 172.21.15.112:0/3329591245 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f5728078bc0 0x7f572807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:02.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.106+0000 7f5749e1a640 1 --2- 172.21.15.112:0/3329591245 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f5728078bc0 0x7f572807b0d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5740009af0 tx=0x7f5740005f90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:02.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.110+0000 7f573a7fc640 1 -- 172.21.15.112:0/3329591245 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f57441f6f60 con 0x7f57441f6610 2024-09-21T00:26:02.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.286+0000 7f574c07e640 1 -- 172.21.15.112:0/3329591245 --> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f57441f3ba0 con 0x7f5728078bc0 2024-09-21T00:26:02.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.294+0000 7f573a7fc640 1 -- 172.21.15.112:0/3329591245 <== mgr.4103 v2:172.21.15.187:6800/2921522906 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332619 (secure 0 0 0) 0x7f57441f3ba0 con 0x7f5728078bc0 2024-09-21T00:26:02.298 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:26:02.300 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2024-09-21T00:26:02.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.297+0000 7f574c07e640 1 -- 172.21.15.112:0/3329591245 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f5728078bc0 msgr2=0x7f572807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:02.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.297+0000 7f574c07e640 1 --2- 172.21.15.112:0/3329591245 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f5728078bc0 0x7f572807b0d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5740009af0 tx=0x7f5740005f90 comp rx=0 tx=0).stop 2024-09-21T00:26:02.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.297+0000 7f574c07e640 1 -- 172.21.15.112:0/3329591245 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f57441f6610 msgr2=0x7f57441f6a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:02.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.297+0000 7f574c07e640 1 --2- 172.21.15.112:0/3329591245 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f57441f6610 0x7f57441f6a20 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f57440cfda0 tx=0x7f573c006430 comp rx=0 tx=0).stop 2024-09-21T00:26:02.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.297+0000 7f574c07e640 1 -- 172.21.15.112:0/3329591245 shutdown_connections 2024-09-21T00:26:02.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.297+0000 7f574c07e640 1 --2- 172.21.15.112:0/3329591245 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f5728078bc0 0x7f572807b0d0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:02.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.297+0000 7f574c07e640 1 --2- 172.21.15.112:0/3329591245 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f57441f6610 0x7f57441f6a20 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:02.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.297+0000 7f574c07e640 1 --2- 172.21.15.112:0/3329591245 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f57441f0580 0x7f5744158910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:02.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.297+0000 7f574c07e640 1 --2- 172.21.15.112:0/3329591245 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5730004800 0x7f5744142e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:02.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.297+0000 7f574c07e640 1 -- 172.21.15.112:0/3329591245 >> 172.21.15.112:0/3329591245 conn(0x7f57440d0a30 msgr2=0x7f574415f6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:02.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.297+0000 7f574c07e640 1 -- 172.21.15.112:0/3329591245 shutdown_connections 2024-09-21T00:26:02.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.297+0000 7f574c07e640 1 -- 172.21.15.112:0/3329591245 wait complete. 2024-09-21T00:26:02.310 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":33,"stamp":"2024-09-21T00:26:01.090766+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":308,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219584,"kb_used_data":3336,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755136,"statfs":{"total":773094113280,"available":772869259264,"internally_reserved":0,"allocated":3416064,"data_stored":2034996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.844092"},"pg_stats":[{"pgid":"2.1f","version":"21'4","reported_seq":27,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.826824+0000","last_change":"2024-09-21T00:25:47.232163+0000","last_active":"2024-09-21T00:25:54.826824+0000","last_peered":"2024-09-21T00:25:54.826824+0000","last_clean":"2024-09-21T00:25:54.826824+0000","last_became_active":"2024-09-21T00:25:47.231843+0000","last_became_peered":"2024-09-21T00:25:47.231843+0000","last_unstale":"2024-09-21T00:25:54.826824+0000","last_undegraded":"2024-09-21T00:25:54.826824+0000","last_fullsized":"2024-09-21T00:25:54.826824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_clean_scrub_stamp":"2024-09-21T00:25:46.193690+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:28:45.453931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.264854+0000","last_change":"2024-09-21T00:25:48.231612+0000","last_active":"2024-09-21T00:25:53.264854+0000","last_peered":"2024-09-21T00:25:53.264854+0000","last_clean":"2024-09-21T00:25:53.264854+0000","last_became_active":"2024-09-21T00:25:48.230971+0000","last_became_peered":"2024-09-21T00:25:48.230971+0000","last_unstale":"2024-09-21T00:25:53.264854+0000","last_undegraded":"2024-09-21T00:25:53.264854+0000","last_fullsized":"2024-09-21T00:25:53.264854+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:07:41.736005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.263784+0000","last_change":"2024-09-21T00:25:52.552378+0000","last_active":"2024-09-21T00:25:53.263784+0000","last_peered":"2024-09-21T00:25:53.263784+0000","last_clean":"2024-09-21T00:25:53.263784+0000","last_became_active":"2024-09-21T00:25:52.551577+0000","last_became_peered":"2024-09-21T00:25:52.551577+0000","last_unstale":"2024-09-21T00:25:53.263784+0000","last_undegraded":"2024-09-21T00:25:53.263784+0000","last_fullsized":"2024-09-21T00:25:53.263784+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_clean_scrub_stamp":"2024-09-21T00:25:51.238933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:54:49.138927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.265044+0000","last_change":"2024-09-21T00:25:47.232065+0000","last_active":"2024-09-21T00:25:53.265044+0000","last_peered":"2024-09-21T00:25:53.265044+0000","last_clean":"2024-09-21T00:25:53.265044+0000","last_became_active":"2024-09-21T00:25:47.231772+0000","last_became_peered":"2024-09-21T00:25:47.231772+0000","last_unstale":"2024-09-21T00:25:53.265044+0000","last_undegraded":"2024-09-21T00:25:53.265044+0000","last_fullsized":"2024-09-21T00:25:53.265044+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_clean_scrub_stamp":"2024-09-21T00:25:46.193690+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:28:45.453931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.826715+0000","last_change":"2024-09-21T00:25:48.233905+0000","last_active":"2024-09-21T00:25:54.826715+0000","last_peered":"2024-09-21T00:25:54.826715+0000","last_clean":"2024-09-21T00:25:54.826715+0000","last_became_active":"2024-09-21T00:25:48.233606+0000","last_became_peered":"2024-09-21T00:25:48.233606+0000","last_unstale":"2024-09-21T00:25:54.826715+0000","last_undegraded":"2024-09-21T00:25:54.826715+0000","last_fullsized":"2024-09-21T00:25:54.826715+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:28:33.742798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.262279+0000","last_change":"2024-09-21T00:25:52.281768+0000","last_active":"2024-09-21T00:25:53.262279+0000","last_peered":"2024-09-21T00:25:53.262279+0000","last_clean":"2024-09-21T00:25:53.262279+0000","last_became_active":"2024-09-21T00:25:52.281352+0000","last_became_peered":"2024-09-21T00:25:52.281352+0000","last_unstale":"2024-09-21T00:25:53.262279+0000","last_undegraded":"2024-09-21T00:25:53.262279+0000","last_fullsized":"2024-09-21T00:25:53.262279+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_clean_scrub_stamp":"2024-09-21T00:25:51.238933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:51:39.910640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.827174+0000","last_change":"2024-09-21T00:25:47.238564+0000","last_active":"2024-09-21T00:25:54.827174+0000","last_peered":"2024-09-21T00:25:54.827174+0000","last_clean":"2024-09-21T00:25:54.827174+0000","last_became_active":"2024-09-21T00:25:47.238114+0000","last_became_peered":"2024-09-21T00:25:47.238114+0000","last_unstale":"2024-09-21T00:25:54.827174+0000","last_undegraded":"2024-09-21T00:25:54.827174+0000","last_fullsized":"2024-09-21T00:25:54.827174+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_clean_scrub_stamp":"2024-09-21T00:25:46.193690+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:37:44.588178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.263783+0000","last_change":"2024-09-21T00:25:48.223307+0000","last_active":"2024-09-21T00:25:53.263783+0000","last_peered":"2024-09-21T00:25:53.263783+0000","last_clean":"2024-09-21T00:25:53.263783+0000","last_became_active":"2024-09-21T00:25:48.222984+0000","last_became_peered":"2024-09-21T00:25:48.222984+0000","last_unstale":"2024-09-21T00:25:53.263783+0000","last_undegraded":"2024-09-21T00:25:53.263783+0000","last_fullsized":"2024-09-21T00:25:53.263783+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:47:38.377031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.263168+0000","last_change":"2024-09-21T00:25:52.282049+0000","last_active":"2024-09-21T00:25:53.263168+0000","last_peered":"2024-09-21T00:25:53.263168+0000","last_clean":"2024-09-21T00:25:53.263168+0000","last_became_active":"2024-09-21T00:25:52.281616+0000","last_became_peered":"2024-09-21T00:25:52.281616+0000","last_unstale":"2024-09-21T00:25:53.263168+0000","last_undegraded":"2024-09-21T00:25:53.263168+0000","last_fullsized":"2024-09-21T00:25:53.263168+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_clean_scrub_stamp":"2024-09-21T00:25:51.238933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:32:36.647007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.826681+0000","last_change":"2024-09-21T00:25:47.238389+0000","last_active":"2024-09-21T00:25:54.826681+0000","last_peered":"2024-09-21T00:25:54.826681+0000","last_clean":"2024-09-21T00:25:54.826681+0000","last_became_active":"2024-09-21T00:25:47.238069+0000","last_became_peered":"2024-09-21T00:25:47.238069+0000","last_unstale":"2024-09-21T00:25:54.826681+0000","last_undegraded":"2024-09-21T00:25:54.826681+0000","last_fullsized":"2024-09-21T00:25:54.826681+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_clean_scrub_stamp":"2024-09-21T00:25:46.193690+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T00:47:54.121411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.264628+0000","last_change":"2024-09-21T00:25:48.220825+0000","last_active":"2024-09-21T00:25:53.264628+0000","last_peered":"2024-09-21T00:25:53.264628+0000","last_clean":"2024-09-21T00:25:53.264628+0000","last_became_active":"2024-09-21T00:25:48.220517+0000","last_became_peered":"2024-09-21T00:25:48.220517+0000","last_unstale":"2024-09-21T00:25:53.264628+0000","last_undegraded":"2024-09-21T00:25:53.264628+0000","last_fullsized":"2024-09-21T00:25:53.264628+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:42:57.617812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.262582+0000","last_change":"2024-09-21T00:25:52.266314+0000","last_active":"2024-09-21T00:25:53.262582+0000","last_peered":"2024-09-21T00:25:53.262582+0000","last_clean":"2024-09-21T00:25:53.262582+0000","last_became_active":"2024-09-21T00:25:52.265684+0000","last_became_peered":"2024-09-21T00:25:52.265684+0000","last_unstale":"2024-09-21T00:25:53.262582+0000","last_undegraded":"2024-09-21T00:25:53.262582+0000","last_fullsized":"2024-09-21T00:25:53.262582+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_clean_scrub_stamp":"2024-09-21T00:25:51.238933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:56:45.075934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.826737+0000","last_change":"2024-09-21T00:25:48.242274+0000","last_active":"2024-09-21T00:25:54.826737+0000","last_peered":"2024-09-21T00:25:54.826737+0000","last_clean":"2024-09-21T00:25:54.826737+0000","last_became_active":"2024-09-21T00:25:48.241953+0000","last_became_peered":"2024-09-21T00:25:48.241953+0000","last_unstale":"2024-09-21T00:25:54.826737+0000","last_undegraded":"2024-09-21T00:25:54.826737+0000","last_fullsized":"2024-09-21T00:25:54.826737+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:23:42.611364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.826740+0000","last_change":"2024-09-21T00:25:47.233509+0000","last_active":"2024-09-21T00:25:54.826740+0000","last_peered":"2024-09-21T00:25:54.826740+0000","last_clean":"2024-09-21T00:25:54.826740+0000","last_became_active":"2024-09-21T00:25:47.233118+0000","last_became_peered":"2024-09-21T00:25:47.233118+0000","last_unstale":"2024-09-21T00:25:54.826740+0000","last_undegraded":"2024-09-21T00:25:54.826740+0000","last_fullsized":"2024-09-21T00:25:54.826740+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_clean_scrub_stamp":"2024-09-21T00:25:46.193690+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:24:04.907044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.264178+0000","last_change":"2024-09-21T00:25:52.276976+0000","last_active":"2024-09-21T00:25:53.264178+0000","last_peered":"2024-09-21T00:25:53.264178+0000","last_clean":"2024-09-21T00:25:53.264178+0000","last_became_active":"2024-09-21T00:25:52.276454+0000","last_became_peered":"2024-09-21T00:25:52.276454+0000","last_unstale":"2024-09-21T00:25:53.264178+0000","last_undegraded":"2024-09-21T00:25:53.264178+0000","last_fullsized":"2024-09-21T00:25:53.264178+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_clean_scrub_stamp":"2024-09-21T00:25:51.238933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T00:45:52.375548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.264258+0000","last_change":"2024-09-21T00:25:48.237366+0000","last_active":"2024-09-21T00:25:53.264258+0000","last_peered":"2024-09-21T00:25:53.264258+0000","last_clean":"2024-09-21T00:25:53.264258+0000","last_became_active":"2024-09-21T00:25:48.237067+0000","last_became_peered":"2024-09-21T00:25:48.237067+0000","last_unstale":"2024-09-21T00:25:53.264258+0000","last_undegraded":"2024-09-21T00:25:53.264258+0000","last_fullsized":"2024-09-21T00:25:53.264258+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:48:25.219726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.324479+0000","last_change":"2024-09-21T00:25:47.227138+0000","last_active":"2024-09-21T00:25:54.324479+0000","last_peered":"2024-09-21T00:25:54.324479+0000","last_clean":"2024-09-21T00:25:54.324479+0000","last_became_active":"2024-09-21T00:25:47.226840+0000","last_became_peered":"2024-09-21T00:25:47.226840+0000","last_unstale":"2024-09-21T00:25:54.324479+0000","last_undegraded":"2024-09-21T00:25:54.324479+0000","last_fullsized":"2024-09-21T00:25:54.324479+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_clean_scrub_stamp":"2024-09-21T00:25:46.193690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:35:33.351025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.262544+0000","last_change":"2024-09-21T00:25:52.281424+0000","last_active":"2024-09-21T00:25:53.262544+0000","last_peered":"2024-09-21T00:25:53.262544+0000","last_clean":"2024-09-21T00:25:53.262544+0000","last_became_active":"2024-09-21T00:25:52.280787+0000","last_became_peered":"2024-09-21T00:25:52.280787+0000","last_unstale":"2024-09-21T00:25:53.262544+0000","last_undegraded":"2024-09-21T00:25:53.262544+0000","last_fullsized":"2024-09-21T00:25:53.262544+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_clean_scrub_stamp":"2024-09-21T00:25:51.238933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:12:43.948562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.264302+0000","last_change":"2024-09-21T00:25:48.239140+0000","last_active":"2024-09-21T00:25:53.264302+0000","last_peered":"2024-09-21T00:25:53.264302+0000","last_clean":"2024-09-21T00:25:53.264302+0000","last_became_active":"2024-09-21T00:25:48.238778+0000","last_became_peered":"2024-09-21T00:25:48.238778+0000","last_unstale":"2024-09-21T00:25:53.264302+0000","last_undegraded":"2024-09-21T00:25:53.264302+0000","last_fullsized":"2024-09-21T00:25:53.264302+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:10:22.176582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.324123+0000","last_change":"2024-09-21T00:25:47.228464+0000","last_active":"2024-09-21T00:25:54.324123+0000","last_peered":"2024-09-21T00:25:54.324123+0000","last_clean":"2024-09-21T00:25:54.324123+0000","last_became_active":"2024-09-21T00:25:47.228152+0000","last_became_peered":"2024-09-21T00:25:47.228152+0000","last_unstale":"2024-09-21T00:25:54.324123+0000","last_undegraded":"2024-09-21T00:25:54.324123+0000","last_fullsized":"2024-09-21T00:25:54.324123+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_clean_scrub_stamp":"2024-09-21T00:25:46.193690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:06:39.901371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.324932+0000","last_change":"2024-09-21T00:25:52.550315+0000","last_active":"2024-09-21T00:25:54.324932+0000","last_peered":"2024-09-21T00:25:54.324932+0000","last_clean":"2024-09-21T00:25:54.324932+0000","last_became_active":"2024-09-21T00:25:52.549843+0000","last_became_peered":"2024-09-21T00:25:52.549843+0000","last_unstale":"2024-09-21T00:25:54.324932+0000","last_undegraded":"2024-09-21T00:25:54.324932+0000","last_fullsized":"2024-09-21T00:25:54.324932+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_clean_scrub_stamp":"2024-09-21T00:25:51.238933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:20:29.694351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.263895+0000","last_change":"2024-09-21T00:25:48.235177+0000","last_active":"2024-09-21T00:25:53.263895+0000","last_peered":"2024-09-21T00:25:53.263895+0000","last_clean":"2024-09-21T00:25:53.263895+0000","last_became_active":"2024-09-21T00:25:48.234870+0000","last_became_peered":"2024-09-21T00:25:48.234870+0000","last_unstale":"2024-09-21T00:25:53.263895+0000","last_undegraded":"2024-09-21T00:25:53.263895+0000","last_fullsized":"2024-09-21T00:25:53.263895+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:55:29.494036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.826534+0000","last_change":"2024-09-21T00:25:47.237638+0000","last_active":"2024-09-21T00:25:54.826534+0000","last_peered":"2024-09-21T00:25:54.826534+0000","last_clean":"2024-09-21T00:25:54.826534+0000","last_became_active":"2024-09-21T00:25:47.237293+0000","last_became_peered":"2024-09-21T00:25:47.237293+0000","last_unstale":"2024-09-21T00:25:54.826534+0000","last_undegraded":"2024-09-21T00:25:54.826534+0000","last_fullsized":"2024-09-21T00:25:54.826534+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_clean_scrub_stamp":"2024-09-21T00:25:46.193690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:57:58.912899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.264463+0000","last_change":"2024-09-21T00:25:52.277124+0000","last_active":"2024-09-21T00:25:53.264463+0000","last_peered":"2024-09-21T00:25:53.264463+0000","last_clean":"2024-09-21T00:25:53.264463+0000","last_became_active":"2024-09-21T00:25:52.276353+0000","last_became_peered":"2024-09-21T00:25:52.276353+0000","last_unstale":"2024-09-21T00:25:53.264463+0000","last_undegraded":"2024-09-21T00:25:53.264463+0000","last_fullsized":"2024-09-21T00:25:53.264463+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_clean_scrub_stamp":"2024-09-21T00:25:51.238933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:13:43.796680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.826871+0000","last_change":"2024-09-21T00:25:48.229327+0000","last_active":"2024-09-21T00:25:54.826871+0000","last_peered":"2024-09-21T00:25:54.826871+0000","last_clean":"2024-09-21T00:25:54.826871+0000","last_became_active":"2024-09-21T00:25:48.228524+0000","last_became_peered":"2024-09-21T00:25:48.228524+0000","last_unstale":"2024-09-21T00:25:54.826871+0000","last_undegraded":"2024-09-21T00:25:54.826871+0000","last_fullsized":"2024-09-21T00:25:54.826871+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:28:02.233472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.827937+0000","last_change":"2024-09-21T00:25:47.232666+0000","last_active":"2024-09-21T00:25:54.827937+0000","last_peered":"2024-09-21T00:25:54.827937+0000","last_clean":"2024-09-21T00:25:54.827937+0000","last_became_active":"2024-09-21T00:25:47.232345+0000","last_became_peered":"2024-09-21T00:25:47.232345+0000","last_unstale":"2024-09-21T00:25:54.827937+0000","last_undegraded":"2024-09-21T00:25:54.827937+0000","last_fullsized":"2024-09-21T00:25:54.827937+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_clean_scrub_stamp":"2024-09-21T00:25:46.193690+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:43:08.714258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.263529+0000","last_change":"2024-09-21T00:25:52.550535+0000","last_active":"2024-09-21T00:25:53.263529+0000","last_peered":"2024-09-21T00:25:53.263529+0000","last_clean":"2024-09-21T00:25:53.263529+0000","last_became_active":"2024-09-21T00:25:52.550055+0000","last_became_peered":"2024-09-21T00:25:52.550055+0000","last_unstale":"2024-09-21T00:25:53.263529+0000","last_undegraded":"2024-09-21T00:25:53.263529+0000","last_fullsized":"2024-09-21T00:25:53.263529+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_clean_scrub_stamp":"2024-09-21T00:25:51.238933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:02:53.617106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.265087+0000","last_change":"2024-09-21T00:25:48.238219+0000","last_active":"2024-09-21T00:25:53.265087+0000","last_peered":"2024-09-21T00:25:53.265087+0000","last_clean":"2024-09-21T00:25:53.265087+0000","last_became_active":"2024-09-21T00:25:48.237770+0000","last_became_peered":"2024-09-21T00:25:48.237770+0000","last_unstale":"2024-09-21T00:25:53.265087+0000","last_undegraded":"2024-09-21T00:25:53.265087+0000","last_fullsized":"2024-09-21T00:25:53.265087+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:34:07.237786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.324720+0000","last_change":"2024-09-21T00:25:47.229361+0000","last_active":"2024-09-21T00:25:54.324720+0000","last_peered":"2024-09-21T00:25:54.324720+0000","last_clean":"2024-09-21T00:25:54.324720+0000","last_became_active":"2024-09-21T00:25:47.228906+0000","last_became_peered":"2024-09-21T00:25:47.228906+0000","last_unstale":"2024-09-21T00:25:54.324720+0000","last_undegraded":"2024-09-21T00:25:54.324720+0000","last_fullsized":"2024-09-21T00:25:54.324720+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_clean_scrub_stamp":"2024-09-21T00:25:46.193690+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:35:43.503231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.263731+0000","last_change":"2024-09-21T00:25:52.550909+0000","last_active":"2024-09-21T00:25:53.263731+0000","last_peered":"2024-09-21T00:25:53.263731+0000","last_clean":"2024-09-21T00:25:53.263731+0000","last_became_active":"2024-09-21T00:25:52.550506+0000","last_became_peered":"2024-09-21T00:25:52.550506+0000","last_unstale":"2024-09-21T00:25:53.263731+0000","last_undegraded":"2024-09-21T00:25:53.263731+0000","last_fullsized":"2024-09-21T00:25:53.263731+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_clean_scrub_stamp":"2024-09-21T00:25:51.238933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:52:16.289687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.324538+0000","last_change":"2024-09-21T00:25:48.232779+0000","last_active":"2024-09-21T00:25:54.324538+0000","last_peered":"2024-09-21T00:25:54.324538+0000","last_clean":"2024-09-21T00:25:54.324538+0000","last_became_active":"2024-09-21T00:25:48.232461+0000","last_became_peered":"2024-09-21T00:25:48.232461+0000","last_unstale":"2024-09-21T00:25:54.324538+0000","last_undegraded":"2024-09-21T00:25:54.324538+0000","last_fullsized":"2024-09-21T00:25:54.324538+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:52:33.265365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.827491+0000","last_change":"2024-09-21T00:25:47.237661+0000","last_active":"2024-09-21T00:25:54.827491+0000","last_peered":"2024-09-21T00:25:54.827491+0000","last_clean":"2024-09-21T00:25:54.827491+0000","last_became_active":"2024-09-21T00:25:47.237273+0000","last_became_peered":"2024-09-21T00:25:47.237273+0000","last_unstale":"2024-09-21T00:25:54.827491+0000","last_undegraded":"2024-09-21T00:25:54.827491+0000","last_fullsized":"2024-09-21T00:25:54.827491+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_clean_scrub_stamp":"2024-09-21T00:25:46.193690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:03:52.795421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.826240+0000","last_change":"2024-09-21T00:25:52.549849+0000","last_active":"2024-09-21T00:25:54.826240+0000","last_peered":"2024-09-21T00:25:54.826240+0000","last_clean":"2024-09-21T00:25:54.826240+0000","last_became_active":"2024-09-21T00:25:52.549433+0000","last_became_peered":"2024-09-21T00:25:52.549433+0000","last_unstale":"2024-09-21T00:25:54.826240+0000","last_undegraded":"2024-09-21T00:25:54.826240+0000","last_fullsized":"2024-09-21T00:25:54.826240+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_clean_scrub_stamp":"2024-09-21T00:25:51.238933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:51:49.334552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.264813+0000","last_change":"2024-09-21T00:25:48.232652+0000","last_active":"2024-09-21T00:25:53.264813+0000","last_peered":"2024-09-21T00:25:53.264813+0000","last_clean":"2024-09-21T00:25:53.264813+0000","last_became_active":"2024-09-21T00:25:48.232099+0000","last_became_peered":"2024-09-21T00:25:48.232099+0000","last_unstale":"2024-09-21T00:25:53.264813+0000","last_undegraded":"2024-09-21T00:25:53.264813+0000","last_fullsized":"2024-09-21T00:25:53.264813+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:48:25.219726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.324657+0000","last_change":"2024-09-21T00:25:47.223473+0000","last_active":"2024-09-21T00:25:54.324657+0000","last_peered":"2024-09-21T00:25:54.324657+0000","last_clean":"2024-09-21T00:25:54.324657+0000","last_became_active":"2024-09-21T00:25:47.223142+0000","last_became_peered":"2024-09-21T00:25:47.223142+0000","last_unstale":"2024-09-21T00:25:54.324657+0000","last_undegraded":"2024-09-21T00:25:54.324657+0000","last_fullsized":"2024-09-21T00:25:54.324657+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_clean_scrub_stamp":"2024-09-21T00:25:46.193690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:54:03.173672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.324126+0000","last_change":"2024-09-21T00:25:52.276922+0000","last_active":"2024-09-21T00:25:54.324126+0000","last_peered":"2024-09-21T00:25:54.324126+0000","last_clean":"2024-09-21T00:25:54.324126+0000","last_became_active":"2024-09-21T00:25:52.276329+0000","last_became_peered":"2024-09-21T00:25:52.276329+0000","last_unstale":"2024-09-21T00:25:54.324126+0000","last_undegraded":"2024-09-21T00:25:54.324126+0000","last_fullsized":"2024-09-21T00:25:54.324126+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_clean_scrub_stamp":"2024-09-21T00:25:51.238933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:29:17.920252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.264377+0000","last_change":"2024-09-21T00:25:48.230144+0000","last_active":"2024-09-21T00:25:53.264377+0000","last_peered":"2024-09-21T00:25:53.264377+0000","last_clean":"2024-09-21T00:25:53.264377+0000","last_became_active":"2024-09-21T00:25:48.229848+0000","last_became_peered":"2024-09-21T00:25:48.229848+0000","last_unstale":"2024-09-21T00:25:53.264377+0000","last_undegraded":"2024-09-21T00:25:53.264377+0000","last_fullsized":"2024-09-21T00:25:53.264377+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:57:59.920308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.263649+0000","last_change":"2024-09-21T00:25:47.228030+0000","last_active":"2024-09-21T00:25:53.263649+0000","last_peered":"2024-09-21T00:25:53.263649+0000","last_clean":"2024-09-21T00:25:53.263649+0000","last_became_active":"2024-09-21T00:25:47.227688+0000","last_became_peered":"2024-09-21T00:25:47.227688+0000","last_unstale":"2024-09-21T00:25:53.263649+0000","last_undegraded":"2024-09-21T00:25:53.263649+0000","last_fullsized":"2024-09-21T00:25:53.263649+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_clean_scrub_stamp":"2024-09-21T00:25:46.193690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:34:30.210849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.324624+0000","last_change":"2024-09-21T00:25:52.276547+0000","last_active":"2024-09-21T00:25:54.324624+0000","last_peered":"2024-09-21T00:25:54.324624+0000","last_clean":"2024-09-21T00:25:54.324624+0000","last_became_active":"2024-09-21T00:25:52.276100+0000","last_became_peered":"2024-09-21T00:25:52.276100+0000","last_unstale":"2024-09-21T00:25:54.324624+0000","last_undegraded":"2024-09-21T00:25:54.324624+0000","last_fullsized":"2024-09-21T00:25:54.324624+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_clean_scrub_stamp":"2024-09-21T00:25:51.238933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:08:54.038371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.324189+0000","last_change":"2024-09-21T00:25:48.233554+0000","last_active":"2024-09-21T00:25:54.324189+0000","last_peered":"2024-09-21T00:25:54.324189+0000","last_clean":"2024-09-21T00:25:54.324189+0000","last_became_active":"2024-09-21T00:25:48.233249+0000","last_became_peered":"2024-09-21T00:25:48.233249+0000","last_unstale":"2024-09-21T00:25:54.324189+0000","last_undegraded":"2024-09-21T00:25:54.324189+0000","last_fullsized":"2024-09-21T00:25:54.324189+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:18:14.964429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.242121+0000","last_change":"2024-09-21T00:25:47.228079+0000","last_active":"2024-09-21T00:25:54.242121+0000","last_peered":"2024-09-21T00:25:54.242121+0000","last_clean":"2024-09-21T00:25:54.242121+0000","last_became_active":"2024-09-21T00:25:47.227731+0000","last_became_peered":"2024-09-21T00:25:47.227731+0000","last_unstale":"2024-09-21T00:25:54.242121+0000","last_undegraded":"2024-09-21T00:25:54.242121+0000","last_fullsized":"2024-09-21T00:25:54.242121+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_clean_scrub_stamp":"2024-09-21T00:25:46.193690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:53:37.081118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.324987+0000","last_change":"2024-09-21T00:25:52.278680+0000","last_active":"2024-09-21T00:25:54.324987+0000","last_peered":"2024-09-21T00:25:54.324987+0000","last_clean":"2024-09-21T00:25:54.324987+0000","last_became_active":"2024-09-21T00:25:52.278128+0000","last_became_peered":"2024-09-21T00:25:52.278128+0000","last_unstale":"2024-09-21T00:25:54.324987+0000","last_undegraded":"2024-09-21T00:25:54.324987+0000","last_fullsized":"2024-09-21T00:25:54.324987+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_clean_scrub_stamp":"2024-09-21T00:25:51.238933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:39:02.970919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.826121+0000","last_change":"2024-09-21T00:25:48.245138+0000","last_active":"2024-09-21T00:25:54.826121+0000","last_peered":"2024-09-21T00:25:54.826121+0000","last_clean":"2024-09-21T00:25:54.826121+0000","last_became_active":"2024-09-21T00:25:48.244796+0000","last_became_peered":"2024-09-21T00:25:48.244796+0000","last_unstale":"2024-09-21T00:25:54.826121+0000","last_undegraded":"2024-09-21T00:25:54.826121+0000","last_fullsized":"2024-09-21T00:25:54.826121+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:53:56.396789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.826939+0000","last_change":"2024-09-21T00:25:47.235076+0000","last_active":"2024-09-21T00:25:54.826939+0000","last_peered":"2024-09-21T00:25:54.826939+0000","last_clean":"2024-09-21T00:25:54.826939+0000","last_became_active":"2024-09-21T00:25:47.234751+0000","last_became_peered":"2024-09-21T00:25:47.234751+0000","last_unstale":"2024-09-21T00:25:54.826939+0000","last_undegraded":"2024-09-21T00:25:54.826939+0000","last_fullsized":"2024-09-21T00:25:54.826939+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_clean_scrub_stamp":"2024-09-21T00:25:46.193690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:40:10.553216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.826789+0000","last_change":"2024-09-21T00:25:52.546980+0000","last_active":"2024-09-21T00:25:54.826789+0000","last_peered":"2024-09-21T00:25:54.826789+0000","last_clean":"2024-09-21T00:25:54.826789+0000","last_became_active":"2024-09-21T00:25:52.546572+0000","last_became_peered":"2024-09-21T00:25:52.546572+0000","last_unstale":"2024-09-21T00:25:54.826789+0000","last_undegraded":"2024-09-21T00:25:54.826789+0000","last_fullsized":"2024-09-21T00:25:54.826789+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_clean_scrub_stamp":"2024-09-21T00:25:51.238933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:34:14.702564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.263892+0000","last_change":"2024-09-21T00:25:48.227417+0000","last_active":"2024-09-21T00:25:53.263892+0000","last_peered":"2024-09-21T00:25:53.263892+0000","last_clean":"2024-09-21T00:25:53.263892+0000","last_became_active":"2024-09-21T00:25:48.227056+0000","last_became_peered":"2024-09-21T00:25:48.227056+0000","last_unstale":"2024-09-21T00:25:53.263892+0000","last_undegraded":"2024-09-21T00:25:53.263892+0000","last_fullsized":"2024-09-21T00:25:53.263892+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:16:33.163623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.264390+0000","last_change":"2024-09-21T00:25:47.228317+0000","last_active":"2024-09-21T00:25:53.264390+0000","last_peered":"2024-09-21T00:25:53.264390+0000","last_clean":"2024-09-21T00:25:53.264390+0000","last_became_active":"2024-09-21T00:25:47.227983+0000","last_became_peered":"2024-09-21T00:25:47.227983+0000","last_unstale":"2024-09-21T00:25:53.264390+0000","last_undegraded":"2024-09-21T00:25:53.264390+0000","last_fullsized":"2024-09-21T00:25:53.264390+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_clean_scrub_stamp":"2024-09-21T00:25:46.193690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:17:12.345201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":119,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.827077+0000","last_change":"2024-09-21T00:25:38.725792+0000","last_active":"2024-09-21T00:25:54.827077+0000","last_peered":"2024-09-21T00:25:54.827077+0000","last_clean":"2024-09-21T00:25:54.827077+0000","last_became_active":"2024-09-21T00:25:38.724668+0000","last_became_peered":"2024-09-21T00:25:38.724668+0000","last_unstale":"2024-09-21T00:25:54.827077+0000","last_undegraded":"2024-09-21T00:25:54.827077+0000","last_fullsized":"2024-09-21T00:25:54.827077+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:37.163216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:37.163216+0000","last_clean_scrub_stamp":"2024-09-21T00:25:37.163216+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:14:36.588660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.827086+0000","last_change":"2024-09-21T00:25:52.551370+0000","last_active":"2024-09-21T00:25:54.827086+0000","last_peered":"2024-09-21T00:25:54.827086+0000","last_clean":"2024-09-21T00:25:54.827086+0000","last_became_active":"2024-09-21T00:25:52.550883+0000","last_became_peered":"2024-09-21T00:25:52.550883+0000","last_unstale":"2024-09-21T00:25:54.827086+0000","last_undegraded":"2024-09-21T00:25:54.827086+0000","last_fullsized":"2024-09-21T00:25:54.827086+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_clean_scrub_stamp":"2024-09-21T00:25:51.238933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T00:31:31.841290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.826927+0000","last_change":"2024-09-21T00:25:48.247427+0000","last_active":"2024-09-21T00:25:54.826927+0000","last_peered":"2024-09-21T00:25:54.826927+0000","last_clean":"2024-09-21T00:25:54.826927+0000","last_became_active":"2024-09-21T00:25:48.246876+0000","last_became_peered":"2024-09-21T00:25:48.246876+0000","last_unstale":"2024-09-21T00:25:54.826927+0000","last_undegraded":"2024-09-21T00:25:54.826927+0000","last_fullsized":"2024-09-21T00:25:54.826927+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:10:01.543628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.242401+0000","last_change":"2024-09-21T00:25:47.233336+0000","last_active":"2024-09-21T00:25:54.242401+0000","last_peered":"2024-09-21T00:25:54.242401+0000","last_clean":"2024-09-21T00:25:54.242401+0000","last_became_active":"2024-09-21T00:25:47.233083+0000","last_became_peered":"2024-09-21T00:25:47.233083+0000","last_unstale":"2024-09-21T00:25:54.242401+0000","last_undegraded":"2024-09-21T00:25:54.242401+0000","last_fullsized":"2024-09-21T00:25:54.242401+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_clean_scrub_stamp":"2024-09-21T00:25:46.193690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:43:39.508965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.826781+0000","last_change":"2024-09-21T00:25:52.550144+0000","last_active":"2024-09-21T00:25:54.826781+0000","last_peered":"2024-09-21T00:25:54.826781+0000","last_clean":"2024-09-21T00:25:54.826781+0000","last_became_active":"2024-09-21T00:25:52.549636+0000","last_became_peered":"2024-09-21T00:25:52.549636+0000","last_unstale":"2024-09-21T00:25:54.826781+0000","last_undegraded":"2024-09-21T00:25:54.826781+0000","last_fullsized":"2024-09-21T00:25:54.826781+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_clean_scrub_stamp":"2024-09-21T00:25:51.238933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:09:27.474968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.263877+0000","last_change":"2024-09-21T00:25:48.229999+0000","last_active":"2024-09-21T00:25:53.263877+0000","last_peered":"2024-09-21T00:25:53.263877+0000","last_clean":"2024-09-21T00:25:53.263877+0000","last_became_active":"2024-09-21T00:25:48.229678+0000","last_became_peered":"2024-09-21T00:25:48.229678+0000","last_unstale":"2024-09-21T00:25:53.263877+0000","last_undegraded":"2024-09-21T00:25:53.263877+0000","last_fullsized":"2024-09-21T00:25:53.263877+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:15:47.572701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.324827+0000","last_change":"2024-09-21T00:25:47.219135+0000","last_active":"2024-09-21T00:25:54.324827+0000","last_peered":"2024-09-21T00:25:54.324827+0000","last_clean":"2024-09-21T00:25:54.324827+0000","last_became_active":"2024-09-21T00:25:47.218722+0000","last_became_peered":"2024-09-21T00:25:47.218722+0000","last_unstale":"2024-09-21T00:25:54.324827+0000","last_undegraded":"2024-09-21T00:25:54.324827+0000","last_fullsized":"2024-09-21T00:25:54.324827+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_clean_scrub_stamp":"2024-09-21T00:25:46.193690+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:52:45.409090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.826420+0000","last_change":"2024-09-21T00:25:52.279946+0000","last_active":"2024-09-21T00:25:54.826420+0000","last_peered":"2024-09-21T00:25:54.826420+0000","last_clean":"2024-09-21T00:25:54.826420+0000","last_became_active":"2024-09-21T00:25:52.279489+0000","last_became_peered":"2024-09-21T00:25:52.279489+0000","last_unstale":"2024-09-21T00:25:54.826420+0000","last_undegraded":"2024-09-21T00:25:54.826420+0000","last_fullsized":"2024-09-21T00:25:54.826420+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_clean_scrub_stamp":"2024-09-21T00:25:51.238933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:47:42.414865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.827648+0000","last_change":"2024-09-21T00:25:48.244146+0000","last_active":"2024-09-21T00:25:54.827648+0000","last_peered":"2024-09-21T00:25:54.827648+0000","last_clean":"2024-09-21T00:25:54.827648+0000","last_became_active":"2024-09-21T00:25:48.243782+0000","last_became_peered":"2024-09-21T00:25:48.243782+0000","last_unstale":"2024-09-21T00:25:54.827648+0000","last_undegraded":"2024-09-21T00:25:54.827648+0000","last_fullsized":"2024-09-21T00:25:54.827648+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:43:59.908291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.242461+0000","last_change":"2024-09-21T00:25:47.230785+0000","last_active":"2024-09-21T00:25:54.242461+0000","last_peered":"2024-09-21T00:25:54.242461+0000","last_clean":"2024-09-21T00:25:54.242461+0000","last_became_active":"2024-09-21T00:25:47.230492+0000","last_became_peered":"2024-09-21T00:25:47.230492+0000","last_unstale":"2024-09-21T00:25:54.242461+0000","last_undegraded":"2024-09-21T00:25:54.242461+0000","last_fullsized":"2024-09-21T00:25:54.242461+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_clean_scrub_stamp":"2024-09-21T00:25:46.193690+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:33:07.366842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.262402+0000","last_change":"2024-09-21T00:25:52.547298+0000","last_active":"2024-09-21T00:25:53.262402+0000","last_peered":"2024-09-21T00:25:53.262402+0000","last_clean":"2024-09-21T00:25:53.262402+0000","last_became_active":"2024-09-21T00:25:52.546737+0000","last_became_peered":"2024-09-21T00:25:52.546737+0000","last_unstale":"2024-09-21T00:25:53.262402+0000","last_undegraded":"2024-09-21T00:25:53.262402+0000","last_fullsized":"2024-09-21T00:25:53.262402+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_clean_scrub_stamp":"2024-09-21T00:25:51.238933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:54:49.138927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.827233+0000","last_change":"2024-09-21T00:25:48.244124+0000","last_active":"2024-09-21T00:25:54.827233+0000","last_peered":"2024-09-21T00:25:54.827233+0000","last_clean":"2024-09-21T00:25:54.827233+0000","last_became_active":"2024-09-21T00:25:48.243767+0000","last_became_peered":"2024-09-21T00:25:48.243767+0000","last_unstale":"2024-09-21T00:25:54.827233+0000","last_undegraded":"2024-09-21T00:25:54.827233+0000","last_fullsized":"2024-09-21T00:25:54.827233+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:15:47.572701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.264246+0000","last_change":"2024-09-21T00:25:47.220781+0000","last_active":"2024-09-21T00:25:53.264246+0000","last_peered":"2024-09-21T00:25:53.264246+0000","last_clean":"2024-09-21T00:25:53.264246+0000","last_became_active":"2024-09-21T00:25:47.220429+0000","last_became_peered":"2024-09-21T00:25:47.220429+0000","last_unstale":"2024-09-21T00:25:53.264246+0000","last_undegraded":"2024-09-21T00:25:53.264246+0000","last_fullsized":"2024-09-21T00:25:53.264246+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_clean_scrub_stamp":"2024-09-21T00:25:46.193690+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:08:24.394801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.262197+0000","last_change":"2024-09-21T00:25:52.276941+0000","last_active":"2024-09-21T00:25:53.262197+0000","last_peered":"2024-09-21T00:25:53.262197+0000","last_clean":"2024-09-21T00:25:53.262197+0000","last_became_active":"2024-09-21T00:25:52.276374+0000","last_became_peered":"2024-09-21T00:25:52.276374+0000","last_unstale":"2024-09-21T00:25:53.262197+0000","last_undegraded":"2024-09-21T00:25:53.262197+0000","last_fullsized":"2024-09-21T00:25:53.262197+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_clean_scrub_stamp":"2024-09-21T00:25:51.238933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:02:24.501560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.826441+0000","last_change":"2024-09-21T00:25:48.244881+0000","last_active":"2024-09-21T00:25:54.826441+0000","last_peered":"2024-09-21T00:25:54.826441+0000","last_clean":"2024-09-21T00:25:54.826441+0000","last_became_active":"2024-09-21T00:25:48.244530+0000","last_became_peered":"2024-09-21T00:25:48.244530+0000","last_unstale":"2024-09-21T00:25:54.826441+0000","last_undegraded":"2024-09-21T00:25:54.826441+0000","last_fullsized":"2024-09-21T00:25:54.826441+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:51:31.831106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":31,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.242013+0000","last_change":"2024-09-21T00:25:47.230620+0000","last_active":"2024-09-21T00:25:54.242013+0000","last_peered":"2024-09-21T00:25:54.242013+0000","last_clean":"2024-09-21T00:25:54.242013+0000","last_became_active":"2024-09-21T00:25:47.230252+0000","last_became_peered":"2024-09-21T00:25:47.230252+0000","last_unstale":"2024-09-21T00:25:54.242013+0000","last_undegraded":"2024-09-21T00:25:54.242013+0000","last_fullsized":"2024-09-21T00:25:54.242013+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_clean_scrub_stamp":"2024-09-21T00:25:46.193690+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:16:00.090070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.826590+0000","last_change":"2024-09-21T00:25:52.266721+0000","last_active":"2024-09-21T00:25:54.826590+0000","last_peered":"2024-09-21T00:25:54.826590+0000","last_clean":"2024-09-21T00:25:54.826590+0000","last_became_active":"2024-09-21T00:25:52.266271+0000","last_became_peered":"2024-09-21T00:25:52.266271+0000","last_unstale":"2024-09-21T00:25:54.826590+0000","last_undegraded":"2024-09-21T00:25:54.826590+0000","last_fullsized":"2024-09-21T00:25:54.826590+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_clean_scrub_stamp":"2024-09-21T00:25:51.238933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:15:37.112368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.826580+0000","last_change":"2024-09-21T00:25:48.239188+0000","last_active":"2024-09-21T00:25:54.826580+0000","last_peered":"2024-09-21T00:25:54.826580+0000","last_clean":"2024-09-21T00:25:54.826580+0000","last_became_active":"2024-09-21T00:25:48.238774+0000","last_became_peered":"2024-09-21T00:25:48.238774+0000","last_unstale":"2024-09-21T00:25:54.826580+0000","last_undegraded":"2024-09-21T00:25:54.826580+0000","last_fullsized":"2024-09-21T00:25:54.826580+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T00:56:56.658210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.826897+0000","last_change":"2024-09-21T00:25:47.230533+0000","last_active":"2024-09-21T00:25:54.826897+0000","last_peered":"2024-09-21T00:25:54.826897+0000","last_clean":"2024-09-21T00:25:54.826897+0000","last_became_active":"2024-09-21T00:25:47.230211+0000","last_became_peered":"2024-09-21T00:25:47.230211+0000","last_unstale":"2024-09-21T00:25:54.826897+0000","last_undegraded":"2024-09-21T00:25:54.826897+0000","last_fullsized":"2024-09-21T00:25:54.826897+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_clean_scrub_stamp":"2024-09-21T00:25:46.193690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:54:03.173672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.265006+0000","last_change":"2024-09-21T00:25:52.550405+0000","last_active":"2024-09-21T00:25:53.265006+0000","last_peered":"2024-09-21T00:25:53.265006+0000","last_clean":"2024-09-21T00:25:53.265006+0000","last_became_active":"2024-09-21T00:25:52.549743+0000","last_became_peered":"2024-09-21T00:25:52.549743+0000","last_unstale":"2024-09-21T00:25:53.265006+0000","last_undegraded":"2024-09-21T00:25:53.265006+0000","last_fullsized":"2024-09-21T00:25:53.265006+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_clean_scrub_stamp":"2024-09-21T00:25:51.238933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T00:40:38.549590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.826686+0000","last_change":"2024-09-21T00:25:48.244073+0000","last_active":"2024-09-21T00:25:54.826686+0000","last_peered":"2024-09-21T00:25:54.826686+0000","last_clean":"2024-09-21T00:25:54.826686+0000","last_became_active":"2024-09-21T00:25:48.243749+0000","last_became_peered":"2024-09-21T00:25:48.243749+0000","last_unstale":"2024-09-21T00:25:54.826686+0000","last_undegraded":"2024-09-21T00:25:54.826686+0000","last_fullsized":"2024-09-21T00:25:54.826686+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:24:47.112012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.263546+0000","last_change":"2024-09-21T00:25:47.229294+0000","last_active":"2024-09-21T00:25:53.263546+0000","last_peered":"2024-09-21T00:25:53.263546+0000","last_clean":"2024-09-21T00:25:53.263546+0000","last_became_active":"2024-09-21T00:25:47.228901+0000","last_became_peered":"2024-09-21T00:25:47.228901+0000","last_unstale":"2024-09-21T00:25:53.263546+0000","last_undegraded":"2024-09-21T00:25:53.263546+0000","last_fullsized":"2024-09-21T00:25:53.263546+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_clean_scrub_stamp":"2024-09-21T00:25:46.193690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:55:28.486627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.264577+0000","last_change":"2024-09-21T00:25:52.547298+0000","last_active":"2024-09-21T00:25:53.264577+0000","last_peered":"2024-09-21T00:25:53.264577+0000","last_clean":"2024-09-21T00:25:53.264577+0000","last_became_active":"2024-09-21T00:25:52.546647+0000","last_became_peered":"2024-09-21T00:25:52.546647+0000","last_unstale":"2024-09-21T00:25:53.264577+0000","last_undegraded":"2024-09-21T00:25:53.264577+0000","last_fullsized":"2024-09-21T00:25:53.264577+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_clean_scrub_stamp":"2024-09-21T00:25:51.238933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:10:45.091073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.826321+0000","last_change":"2024-09-21T00:25:48.237722+0000","last_active":"2024-09-21T00:25:54.826321+0000","last_peered":"2024-09-21T00:25:54.826321+0000","last_clean":"2024-09-21T00:25:54.826321+0000","last_became_active":"2024-09-21T00:25:48.237400+0000","last_became_peered":"2024-09-21T00:25:48.237400+0000","last_unstale":"2024-09-21T00:25:54.826321+0000","last_undegraded":"2024-09-21T00:25:54.826321+0000","last_fullsized":"2024-09-21T00:25:54.826321+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:00:11.346065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.826654+0000","last_change":"2024-09-21T00:25:47.225035+0000","last_active":"2024-09-21T00:25:54.826654+0000","last_peered":"2024-09-21T00:25:54.826654+0000","last_clean":"2024-09-21T00:25:54.826654+0000","last_became_active":"2024-09-21T00:25:47.224732+0000","last_became_peered":"2024-09-21T00:25:47.224732+0000","last_unstale":"2024-09-21T00:25:54.826654+0000","last_undegraded":"2024-09-21T00:25:54.826654+0000","last_fullsized":"2024-09-21T00:25:54.826654+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_clean_scrub_stamp":"2024-09-21T00:25:46.193690+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:16:00.090070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.264690+0000","last_change":"2024-09-21T00:25:52.550751+0000","last_active":"2024-09-21T00:25:53.264690+0000","last_peered":"2024-09-21T00:25:53.264690+0000","last_clean":"2024-09-21T00:25:53.264690+0000","last_became_active":"2024-09-21T00:25:52.550166+0000","last_became_peered":"2024-09-21T00:25:52.550166+0000","last_unstale":"2024-09-21T00:25:53.264690+0000","last_undegraded":"2024-09-21T00:25:53.264690+0000","last_fullsized":"2024-09-21T00:25:53.264690+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_clean_scrub_stamp":"2024-09-21T00:25:51.238933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:01:14.362816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.826761+0000","last_change":"2024-09-21T00:25:48.240297+0000","last_active":"2024-09-21T00:25:54.826761+0000","last_peered":"2024-09-21T00:25:54.826761+0000","last_clean":"2024-09-21T00:25:54.826761+0000","last_became_active":"2024-09-21T00:25:48.239975+0000","last_became_peered":"2024-09-21T00:25:48.239975+0000","last_unstale":"2024-09-21T00:25:54.826761+0000","last_undegraded":"2024-09-21T00:25:54.826761+0000","last_fullsized":"2024-09-21T00:25:54.826761+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:07:41.736005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":29,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.827734+0000","last_change":"2024-09-21T00:25:47.228804+0000","last_active":"2024-09-21T00:25:54.827734+0000","last_peered":"2024-09-21T00:25:54.827734+0000","last_clean":"2024-09-21T00:25:54.827734+0000","last_became_active":"2024-09-21T00:25:47.228490+0000","last_became_peered":"2024-09-21T00:25:47.228490+0000","last_unstale":"2024-09-21T00:25:54.827734+0000","last_undegraded":"2024-09-21T00:25:54.827734+0000","last_fullsized":"2024-09-21T00:25:54.827734+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_clean_scrub_stamp":"2024-09-21T00:25:46.193690+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:35:43.503231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.324519+0000","last_change":"2024-09-21T00:25:52.273360+0000","last_active":"2024-09-21T00:25:54.324519+0000","last_peered":"2024-09-21T00:25:54.324519+0000","last_clean":"2024-09-21T00:25:54.324519+0000","last_became_active":"2024-09-21T00:25:52.272888+0000","last_became_peered":"2024-09-21T00:25:52.272888+0000","last_unstale":"2024-09-21T00:25:54.324519+0000","last_undegraded":"2024-09-21T00:25:54.324519+0000","last_fullsized":"2024-09-21T00:25:54.324519+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_clean_scrub_stamp":"2024-09-21T00:25:51.238933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:31:19.736643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.264518+0000","last_change":"2024-09-21T00:25:48.238506+0000","last_active":"2024-09-21T00:25:53.264518+0000","last_peered":"2024-09-21T00:25:53.264518+0000","last_clean":"2024-09-21T00:25:53.264518+0000","last_became_active":"2024-09-21T00:25:48.237355+0000","last_became_peered":"2024-09-21T00:25:48.237355+0000","last_unstale":"2024-09-21T00:25:53.264518+0000","last_undegraded":"2024-09-21T00:25:53.264518+0000","last_fullsized":"2024-09-21T00:25:53.264518+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:40:11.560626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.324726+0000","last_change":"2024-09-21T00:25:47.224421+0000","last_active":"2024-09-21T00:25:54.324726+0000","last_peered":"2024-09-21T00:25:54.324726+0000","last_clean":"2024-09-21T00:25:54.324726+0000","last_became_active":"2024-09-21T00:25:47.224096+0000","last_became_peered":"2024-09-21T00:25:47.224096+0000","last_unstale":"2024-09-21T00:25:54.324726+0000","last_undegraded":"2024-09-21T00:25:54.324726+0000","last_fullsized":"2024-09-21T00:25:54.324726+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_clean_scrub_stamp":"2024-09-21T00:25:46.193690+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:43:08.714258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.263548+0000","last_change":"2024-09-21T00:25:52.277635+0000","last_active":"2024-09-21T00:25:53.263548+0000","last_peered":"2024-09-21T00:25:53.263548+0000","last_clean":"2024-09-21T00:25:53.263548+0000","last_became_active":"2024-09-21T00:25:52.276739+0000","last_became_peered":"2024-09-21T00:25:52.276739+0000","last_unstale":"2024-09-21T00:25:53.263548+0000","last_undegraded":"2024-09-21T00:25:53.263548+0000","last_fullsized":"2024-09-21T00:25:53.263548+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_clean_scrub_stamp":"2024-09-21T00:25:51.238933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:46:09.670107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.826816+0000","last_change":"2024-09-21T00:25:48.233598+0000","last_active":"2024-09-21T00:25:54.826816+0000","last_peered":"2024-09-21T00:25:54.826816+0000","last_clean":"2024-09-21T00:25:54.826816+0000","last_became_active":"2024-09-21T00:25:48.233203+0000","last_became_peered":"2024-09-21T00:25:48.233203+0000","last_unstale":"2024-09-21T00:25:54.826816+0000","last_undegraded":"2024-09-21T00:25:54.826816+0000","last_fullsized":"2024-09-21T00:25:54.826816+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:36:54.554284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.263639+0000","last_change":"2024-09-21T00:25:47.225271+0000","last_active":"2024-09-21T00:25:53.263639+0000","last_peered":"2024-09-21T00:25:53.263639+0000","last_clean":"2024-09-21T00:25:53.263639+0000","last_became_active":"2024-09-21T00:25:47.224935+0000","last_became_peered":"2024-09-21T00:25:47.224935+0000","last_unstale":"2024-09-21T00:25:53.263639+0000","last_undegraded":"2024-09-21T00:25:53.263639+0000","last_fullsized":"2024-09-21T00:25:53.263639+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_clean_scrub_stamp":"2024-09-21T00:25:46.193690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:06:59.889895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.264083+0000","last_change":"2024-09-21T00:25:52.278653+0000","last_active":"2024-09-21T00:25:53.264083+0000","last_peered":"2024-09-21T00:25:53.264083+0000","last_clean":"2024-09-21T00:25:53.264083+0000","last_became_active":"2024-09-21T00:25:52.278111+0000","last_became_peered":"2024-09-21T00:25:52.278111+0000","last_unstale":"2024-09-21T00:25:53.264083+0000","last_undegraded":"2024-09-21T00:25:53.264083+0000","last_fullsized":"2024-09-21T00:25:53.264083+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_clean_scrub_stamp":"2024-09-21T00:25:51.238933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:01:24.675942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.263506+0000","last_change":"2024-09-21T00:25:48.239666+0000","last_active":"2024-09-21T00:25:53.263506+0000","last_peered":"2024-09-21T00:25:53.263506+0000","last_clean":"2024-09-21T00:25:53.263506+0000","last_became_active":"2024-09-21T00:25:48.239196+0000","last_became_peered":"2024-09-21T00:25:48.239196+0000","last_unstale":"2024-09-21T00:25:53.263506+0000","last_undegraded":"2024-09-21T00:25:53.263506+0000","last_fullsized":"2024-09-21T00:25:53.263506+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:00:19.889965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.827810+0000","last_change":"2024-09-21T00:25:47.225244+0000","last_active":"2024-09-21T00:25:54.827810+0000","last_peered":"2024-09-21T00:25:54.827810+0000","last_clean":"2024-09-21T00:25:54.827810+0000","last_became_active":"2024-09-21T00:25:47.224812+0000","last_became_peered":"2024-09-21T00:25:47.224812+0000","last_unstale":"2024-09-21T00:25:54.827810+0000","last_undegraded":"2024-09-21T00:25:54.827810+0000","last_fullsized":"2024-09-21T00:25:54.827810+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_clean_scrub_stamp":"2024-09-21T00:25:46.193690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:06:39.901371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.264361+0000","last_change":"2024-09-21T00:25:52.550130+0000","last_active":"2024-09-21T00:25:53.264361+0000","last_peered":"2024-09-21T00:25:53.264361+0000","last_clean":"2024-09-21T00:25:53.264361+0000","last_became_active":"2024-09-21T00:25:52.549675+0000","last_became_peered":"2024-09-21T00:25:52.549675+0000","last_unstale":"2024-09-21T00:25:53.264361+0000","last_undegraded":"2024-09-21T00:25:53.264361+0000","last_fullsized":"2024-09-21T00:25:53.264361+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_clean_scrub_stamp":"2024-09-21T00:25:51.238933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:55:10.499479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.826619+0000","last_change":"2024-09-21T00:25:52.550630+0000","last_active":"2024-09-21T00:25:54.826619+0000","last_peered":"2024-09-21T00:25:54.826619+0000","last_clean":"2024-09-21T00:25:54.826619+0000","last_became_active":"2024-09-21T00:25:52.550098+0000","last_became_peered":"2024-09-21T00:25:52.550098+0000","last_unstale":"2024-09-21T00:25:54.826619+0000","last_undegraded":"2024-09-21T00:25:54.826619+0000","last_fullsized":"2024-09-21T00:25:54.826619+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_clean_scrub_stamp":"2024-09-21T00:25:51.238933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:43:32.503216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.324003+0000","last_change":"2024-09-21T00:25:48.227526+0000","last_active":"2024-09-21T00:25:54.324003+0000","last_peered":"2024-09-21T00:25:54.324003+0000","last_clean":"2024-09-21T00:25:54.324003+0000","last_became_active":"2024-09-21T00:25:48.227194+0000","last_became_peered":"2024-09-21T00:25:48.227194+0000","last_unstale":"2024-09-21T00:25:54.324003+0000","last_undegraded":"2024-09-21T00:25:54.324003+0000","last_fullsized":"2024-09-21T00:25:54.324003+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:54:09.449880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.262751+0000","last_change":"2024-09-21T00:25:47.220721+0000","last_active":"2024-09-21T00:25:53.262751+0000","last_peered":"2024-09-21T00:25:53.262751+0000","last_clean":"2024-09-21T00:25:53.262751+0000","last_became_active":"2024-09-21T00:25:47.220393+0000","last_became_peered":"2024-09-21T00:25:47.220393+0000","last_unstale":"2024-09-21T00:25:53.262751+0000","last_undegraded":"2024-09-21T00:25:53.262751+0000","last_fullsized":"2024-09-21T00:25:53.262751+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_clean_scrub_stamp":"2024-09-21T00:25:46.193690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:00:04.423924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.827438+0000","last_change":"2024-09-21T00:25:52.280179+0000","last_active":"2024-09-21T00:25:54.827438+0000","last_peered":"2024-09-21T00:25:54.827438+0000","last_clean":"2024-09-21T00:25:54.827438+0000","last_became_active":"2024-09-21T00:25:52.279617+0000","last_became_peered":"2024-09-21T00:25:52.279617+0000","last_unstale":"2024-09-21T00:25:54.827438+0000","last_undegraded":"2024-09-21T00:25:54.827438+0000","last_fullsized":"2024-09-21T00:25:54.827438+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_clean_scrub_stamp":"2024-09-21T00:25:51.238933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:30:11.179850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.264234+0000","last_change":"2024-09-21T00:25:48.237852+0000","last_active":"2024-09-21T00:25:53.264234+0000","last_peered":"2024-09-21T00:25:53.264234+0000","last_clean":"2024-09-21T00:25:53.264234+0000","last_became_active":"2024-09-21T00:25:48.237559+0000","last_became_peered":"2024-09-21T00:25:48.237559+0000","last_unstale":"2024-09-21T00:25:53.264234+0000","last_undegraded":"2024-09-21T00:25:53.264234+0000","last_fullsized":"2024-09-21T00:25:53.264234+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:28:02.233472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.264305+0000","last_change":"2024-09-21T00:25:47.227879+0000","last_active":"2024-09-21T00:25:53.264305+0000","last_peered":"2024-09-21T00:25:53.264305+0000","last_clean":"2024-09-21T00:25:53.264305+0000","last_became_active":"2024-09-21T00:25:47.227569+0000","last_became_peered":"2024-09-21T00:25:47.227569+0000","last_unstale":"2024-09-21T00:25:53.264305+0000","last_undegraded":"2024-09-21T00:25:53.264305+0000","last_fullsized":"2024-09-21T00:25:53.264305+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_clean_scrub_stamp":"2024-09-21T00:25:46.193690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:16:00.090070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.827281+0000","last_change":"2024-09-21T00:25:47.231506+0000","last_active":"2024-09-21T00:25:54.827281+0000","last_peered":"2024-09-21T00:25:54.827281+0000","last_clean":"2024-09-21T00:25:54.827281+0000","last_became_active":"2024-09-21T00:25:47.231135+0000","last_became_peered":"2024-09-21T00:25:47.231135+0000","last_unstale":"2024-09-21T00:25:54.827281+0000","last_undegraded":"2024-09-21T00:25:54.827281+0000","last_fullsized":"2024-09-21T00:25:54.827281+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_clean_scrub_stamp":"2024-09-21T00:25:46.193690+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:52:45.409090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.826519+0000","last_change":"2024-09-21T00:25:48.245503+0000","last_active":"2024-09-21T00:25:54.826519+0000","last_peered":"2024-09-21T00:25:54.826519+0000","last_clean":"2024-09-21T00:25:54.826519+0000","last_became_active":"2024-09-21T00:25:48.245168+0000","last_became_peered":"2024-09-21T00:25:48.245168+0000","last_unstale":"2024-09-21T00:25:54.826519+0000","last_undegraded":"2024-09-21T00:25:54.826519+0000","last_fullsized":"2024-09-21T00:25:54.826519+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:51:35.872806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.242222+0000","last_change":"2024-09-21T00:25:52.278289+0000","last_active":"2024-09-21T00:25:54.242222+0000","last_peered":"2024-09-21T00:25:54.242222+0000","last_clean":"2024-09-21T00:25:54.242222+0000","last_became_active":"2024-09-21T00:25:52.277874+0000","last_became_peered":"2024-09-21T00:25:52.277874+0000","last_unstale":"2024-09-21T00:25:54.242222+0000","last_undegraded":"2024-09-21T00:25:54.242222+0000","last_fullsized":"2024-09-21T00:25:54.242222+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_clean_scrub_stamp":"2024-09-21T00:25:51.238933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T00:47:59.166653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.264404+0000","last_change":"2024-09-21T00:25:47.232974+0000","last_active":"2024-09-21T00:25:53.264404+0000","last_peered":"2024-09-21T00:25:53.264404+0000","last_clean":"2024-09-21T00:25:53.264404+0000","last_became_active":"2024-09-21T00:25:47.232605+0000","last_became_peered":"2024-09-21T00:25:47.232605+0000","last_unstale":"2024-09-21T00:25:53.264404+0000","last_undegraded":"2024-09-21T00:25:53.264404+0000","last_fullsized":"2024-09-21T00:25:53.264404+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_clean_scrub_stamp":"2024-09-21T00:25:46.193690+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:53:37.081118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.263168+0000","last_change":"2024-09-21T00:25:48.223460+0000","last_active":"2024-09-21T00:25:53.263168+0000","last_peered":"2024-09-21T00:25:53.263168+0000","last_clean":"2024-09-21T00:25:53.263168+0000","last_became_active":"2024-09-21T00:25:48.222174+0000","last_became_peered":"2024-09-21T00:25:48.222174+0000","last_unstale":"2024-09-21T00:25:53.263168+0000","last_undegraded":"2024-09-21T00:25:53.263168+0000","last_fullsized":"2024-09-21T00:25:53.263168+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:07:00.897304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.264073+0000","last_change":"2024-09-21T00:25:52.277698+0000","last_active":"2024-09-21T00:25:53.264073+0000","last_peered":"2024-09-21T00:25:53.264073+0000","last_clean":"2024-09-21T00:25:53.264073+0000","last_became_active":"2024-09-21T00:25:52.277232+0000","last_became_peered":"2024-09-21T00:25:52.277232+0000","last_unstale":"2024-09-21T00:25:53.264073+0000","last_undegraded":"2024-09-21T00:25:53.264073+0000","last_fullsized":"2024-09-21T00:25:53.264073+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_clean_scrub_stamp":"2024-09-21T00:25:51.238933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:01:14.362816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.826302+0000","last_change":"2024-09-21T00:25:48.240360+0000","last_active":"2024-09-21T00:25:54.826302+0000","last_peered":"2024-09-21T00:25:54.826302+0000","last_clean":"2024-09-21T00:25:54.826302+0000","last_became_active":"2024-09-21T00:25:48.240038+0000","last_became_peered":"2024-09-21T00:25:48.240038+0000","last_unstale":"2024-09-21T00:25:54.826302+0000","last_undegraded":"2024-09-21T00:25:54.826302+0000","last_fullsized":"2024-09-21T00:25:54.826302+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:18:14.964429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.242242+0000","last_change":"2024-09-21T00:25:48.243928+0000","last_active":"2024-09-21T00:25:54.242242+0000","last_peered":"2024-09-21T00:25:54.242242+0000","last_clean":"2024-09-21T00:25:54.242242+0000","last_became_active":"2024-09-21T00:25:48.243641+0000","last_became_peered":"2024-09-21T00:25:48.243641+0000","last_unstale":"2024-09-21T00:25:54.242242+0000","last_undegraded":"2024-09-21T00:25:54.242242+0000","last_fullsized":"2024-09-21T00:25:54.242242+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:01:32.510097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.263436+0000","last_change":"2024-09-21T00:25:48.229253+0000","last_active":"2024-09-21T00:25:53.263436+0000","last_peered":"2024-09-21T00:25:53.263436+0000","last_clean":"2024-09-21T00:25:53.263436+0000","last_became_active":"2024-09-21T00:25:48.228880+0000","last_became_peered":"2024-09-21T00:25:48.228880+0000","last_unstale":"2024-09-21T00:25:53.263436+0000","last_undegraded":"2024-09-21T00:25:53.263436+0000","last_fullsized":"2024-09-21T00:25:53.263436+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:24:47.112012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.826434+0000","last_change":"2024-09-21T00:25:48.232024+0000","last_active":"2024-09-21T00:25:54.826434+0000","last_peered":"2024-09-21T00:25:54.826434+0000","last_clean":"2024-09-21T00:25:54.826434+0000","last_became_active":"2024-09-21T00:25:48.231506+0000","last_became_peered":"2024-09-21T00:25:48.231506+0000","last_unstale":"2024-09-21T00:25:54.826434+0000","last_undegraded":"2024-09-21T00:25:54.826434+0000","last_fullsized":"2024-09-21T00:25:54.826434+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:02:20.463726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.264854+0000","last_change":"2024-09-21T00:25:48.242030+0000","last_active":"2024-09-21T00:25:53.264854+0000","last_peered":"2024-09-21T00:25:53.264854+0000","last_clean":"2024-09-21T00:25:53.264854+0000","last_became_active":"2024-09-21T00:25:48.241710+0000","last_became_peered":"2024-09-21T00:25:48.241710+0000","last_unstale":"2024-09-21T00:25:53.264854+0000","last_undegraded":"2024-09-21T00:25:53.264854+0000","last_fullsized":"2024-09-21T00:25:53.264854+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:00:19.889965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.324542+0000","last_change":"2024-09-21T00:25:48.230301+0000","last_active":"2024-09-21T00:25:54.324542+0000","last_peered":"2024-09-21T00:25:54.324542+0000","last_clean":"2024-09-21T00:25:54.324542+0000","last_became_active":"2024-09-21T00:25:48.229759+0000","last_became_peered":"2024-09-21T00:25:48.229759+0000","last_unstale":"2024-09-21T00:25:54.324542+0000","last_undegraded":"2024-09-21T00:25:54.324542+0000","last_fullsized":"2024-09-21T00:25:54.324542+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:23:28.920719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.827515+0000","last_change":"2024-09-21T00:25:48.236977+0000","last_active":"2024-09-21T00:25:54.827515+0000","last_peered":"2024-09-21T00:25:54.827515+0000","last_clean":"2024-09-21T00:25:54.827515+0000","last_became_active":"2024-09-21T00:25:48.236636+0000","last_became_peered":"2024-09-21T00:25:48.236636+0000","last_unstale":"2024-09-21T00:25:54.827515+0000","last_undegraded":"2024-09-21T00:25:54.827515+0000","last_fullsized":"2024-09-21T00:25:54.827515+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:34:36.110489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.264907+0000","last_change":"2024-09-21T00:25:48.242792+0000","last_active":"2024-09-21T00:25:53.264907+0000","last_peered":"2024-09-21T00:25:53.264907+0000","last_clean":"2024-09-21T00:25:53.264907+0000","last_became_active":"2024-09-21T00:25:48.242512+0000","last_became_peered":"2024-09-21T00:25:48.242512+0000","last_unstale":"2024-09-21T00:25:53.264907+0000","last_undegraded":"2024-09-21T00:25:53.264907+0000","last_fullsized":"2024-09-21T00:25:53.264907+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:58:22.324727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.262616+0000","last_change":"2024-09-21T00:25:48.229627+0000","last_active":"2024-09-21T00:25:53.262616+0000","last_peered":"2024-09-21T00:25:53.262616+0000","last_clean":"2024-09-21T00:25:53.262616+0000","last_became_active":"2024-09-21T00:25:48.229297+0000","last_became_peered":"2024-09-21T00:25:48.229297+0000","last_unstale":"2024-09-21T00:25:53.262616+0000","last_undegraded":"2024-09-21T00:25:53.262616+0000","last_fullsized":"2024-09-21T00:25:53.262616+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:58:22.324727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.264187+0000","last_change":"2024-09-21T00:25:48.243911+0000","last_active":"2024-09-21T00:25:53.264187+0000","last_peered":"2024-09-21T00:25:53.264187+0000","last_clean":"2024-09-21T00:25:53.264187+0000","last_became_active":"2024-09-21T00:25:48.243581+0000","last_became_peered":"2024-09-21T00:25:48.243581+0000","last_unstale":"2024-09-21T00:25:53.264187+0000","last_undegraded":"2024-09-21T00:25:53.264187+0000","last_fullsized":"2024-09-21T00:25:53.264187+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:42:57.617812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.263316+0000","last_change":"2024-09-21T00:25:48.238597+0000","last_active":"2024-09-21T00:25:53.263316+0000","last_peered":"2024-09-21T00:25:53.263316+0000","last_clean":"2024-09-21T00:25:53.263316+0000","last_became_active":"2024-09-21T00:25:48.238265+0000","last_became_peered":"2024-09-21T00:25:48.238265+0000","last_unstale":"2024-09-21T00:25:53.263316+0000","last_undegraded":"2024-09-21T00:25:53.263316+0000","last_fullsized":"2024-09-21T00:25:53.263316+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:10:01.543628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.826223+0000","last_change":"2024-09-21T00:25:48.237837+0000","last_active":"2024-09-21T00:25:54.826223+0000","last_peered":"2024-09-21T00:25:54.826223+0000","last_clean":"2024-09-21T00:25:54.826223+0000","last_became_active":"2024-09-21T00:25:48.237502+0000","last_became_peered":"2024-09-21T00:25:48.237502+0000","last_unstale":"2024-09-21T00:25:54.826223+0000","last_undegraded":"2024-09-21T00:25:54.826223+0000","last_fullsized":"2024-09-21T00:25:54.826223+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:52:28.861327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.827511+0000","last_change":"2024-09-21T00:25:48.241417+0000","last_active":"2024-09-21T00:25:54.827511+0000","last_peered":"2024-09-21T00:25:54.827511+0000","last_clean":"2024-09-21T00:25:54.827511+0000","last_became_active":"2024-09-21T00:25:48.241107+0000","last_became_peered":"2024-09-21T00:25:48.241107+0000","last_unstale":"2024-09-21T00:25:54.827511+0000","last_undegraded":"2024-09-21T00:25:54.827511+0000","last_fullsized":"2024-09-21T00:25:54.827511+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:52:33.265365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.827391+0000","last_change":"2024-09-21T00:25:48.246529+0000","last_active":"2024-09-21T00:25:54.827391+0000","last_peered":"2024-09-21T00:25:54.827391+0000","last_clean":"2024-09-21T00:25:54.827391+0000","last_became_active":"2024-09-21T00:25:48.245952+0000","last_became_peered":"2024-09-21T00:25:48.245952+0000","last_unstale":"2024-09-21T00:25:54.827391+0000","last_undegraded":"2024-09-21T00:25:54.827391+0000","last_fullsized":"2024-09-21T00:25:54.827391+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:16:07.250921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.242392+0000","last_change":"2024-09-21T00:25:48.235870+0000","last_active":"2024-09-21T00:25:54.242392+0000","last_peered":"2024-09-21T00:25:54.242392+0000","last_clean":"2024-09-21T00:25:54.242392+0000","last_became_active":"2024-09-21T00:25:48.235552+0000","last_became_peered":"2024-09-21T00:25:48.235552+0000","last_unstale":"2024-09-21T00:25:54.242392+0000","last_undegraded":"2024-09-21T00:25:54.242392+0000","last_fullsized":"2024-09-21T00:25:54.242392+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:03:13.400804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.264193+0000","last_change":"2024-09-21T00:25:48.238458+0000","last_active":"2024-09-21T00:25:53.264193+0000","last_peered":"2024-09-21T00:25:53.264193+0000","last_clean":"2024-09-21T00:25:53.264193+0000","last_became_active":"2024-09-21T00:25:48.238154+0000","last_became_peered":"2024-09-21T00:25:48.238154+0000","last_unstale":"2024-09-21T00:25:53.264193+0000","last_undegraded":"2024-09-21T00:25:53.264193+0000","last_fullsized":"2024-09-21T00:25:53.264193+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:51:31.831106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.262441+0000","last_change":"2024-09-21T00:25:48.232187+0000","last_active":"2024-09-21T00:25:53.262441+0000","last_peered":"2024-09-21T00:25:53.262441+0000","last_clean":"2024-09-21T00:25:53.262441+0000","last_became_active":"2024-09-21T00:25:48.230817+0000","last_became_peered":"2024-09-21T00:25:48.230817+0000","last_unstale":"2024-09-21T00:25:53.262441+0000","last_undegraded":"2024-09-21T00:25:53.262441+0000","last_fullsized":"2024-09-21T00:25:53.262441+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:32:32.609174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.324087+0000","last_change":"2024-09-21T00:25:48.220646+0000","last_active":"2024-09-21T00:25:54.324087+0000","last_peered":"2024-09-21T00:25:54.324087+0000","last_clean":"2024-09-21T00:25:54.324087+0000","last_became_active":"2024-09-21T00:25:48.220325+0000","last_became_peered":"2024-09-21T00:25:48.220325+0000","last_unstale":"2024-09-21T00:25:54.324087+0000","last_undegraded":"2024-09-21T00:25:54.324087+0000","last_fullsized":"2024-09-21T00:25:54.324087+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:52:34.711918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.826294+0000","last_change":"2024-09-21T00:25:48.228802+0000","last_active":"2024-09-21T00:25:54.826294+0000","last_peered":"2024-09-21T00:25:54.826294+0000","last_clean":"2024-09-21T00:25:54.826294+0000","last_became_active":"2024-09-21T00:25:48.228464+0000","last_became_peered":"2024-09-21T00:25:48.228464+0000","last_unstale":"2024-09-21T00:25:54.826294+0000","last_undegraded":"2024-09-21T00:25:54.826294+0000","last_fullsized":"2024-09-21T00:25:54.826294+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:50:09.155321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.264138+0000","last_change":"2024-09-21T00:25:48.243998+0000","last_active":"2024-09-21T00:25:53.264138+0000","last_peered":"2024-09-21T00:25:53.264138+0000","last_clean":"2024-09-21T00:25:53.264138+0000","last_became_active":"2024-09-21T00:25:48.242668+0000","last_became_peered":"2024-09-21T00:25:48.242668+0000","last_unstale":"2024-09-21T00:25:53.264138+0000","last_undegraded":"2024-09-21T00:25:53.264138+0000","last_fullsized":"2024-09-21T00:25:53.264138+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:44:10.293149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.264479+0000","last_change":"2024-09-21T00:25:48.230464+0000","last_active":"2024-09-21T00:25:53.264479+0000","last_peered":"2024-09-21T00:25:53.264479+0000","last_clean":"2024-09-21T00:25:53.264479+0000","last_became_active":"2024-09-21T00:25:48.229463+0000","last_became_peered":"2024-09-21T00:25:48.229463+0000","last_unstale":"2024-09-21T00:25:53.264479+0000","last_undegraded":"2024-09-21T00:25:53.264479+0000","last_fullsized":"2024-09-21T00:25:53.264479+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:24:05.914453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.827241+0000","last_change":"2024-09-21T00:25:48.233368+0000","last_active":"2024-09-21T00:25:54.827241+0000","last_peered":"2024-09-21T00:25:54.827241+0000","last_clean":"2024-09-21T00:25:54.827241+0000","last_became_active":"2024-09-21T00:25:48.233049+0000","last_became_peered":"2024-09-21T00:25:48.233049+0000","last_unstale":"2024-09-21T00:25:54.827241+0000","last_undegraded":"2024-09-21T00:25:54.827241+0000","last_fullsized":"2024-09-21T00:25:54.827241+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:52:34.711918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.827398+0000","last_change":"2024-09-21T00:25:48.240634+0000","last_active":"2024-09-21T00:25:54.827398+0000","last_peered":"2024-09-21T00:25:54.827398+0000","last_clean":"2024-09-21T00:25:54.827398+0000","last_became_active":"2024-09-21T00:25:48.240302+0000","last_became_peered":"2024-09-21T00:25:48.240302+0000","last_unstale":"2024-09-21T00:25:54.827398+0000","last_undegraded":"2024-09-21T00:25:54.827398+0000","last_fullsized":"2024-09-21T00:25:54.827398+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:30:32.308490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.827582+0000","last_change":"2024-09-21T00:25:48.243347+0000","last_active":"2024-09-21T00:25:54.827582+0000","last_peered":"2024-09-21T00:25:54.827582+0000","last_clean":"2024-09-21T00:25:54.827582+0000","last_became_active":"2024-09-21T00:25:48.243035+0000","last_became_peered":"2024-09-21T00:25:48.243035+0000","last_unstale":"2024-09-21T00:25:54.827582+0000","last_undegraded":"2024-09-21T00:25:54.827582+0000","last_fullsized":"2024-09-21T00:25:54.827582+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:10:10.125591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.263815+0000","last_change":"2024-09-21T00:25:48.235929+0000","last_active":"2024-09-21T00:25:53.263815+0000","last_peered":"2024-09-21T00:25:53.263815+0000","last_clean":"2024-09-21T00:25:53.263815+0000","last_became_active":"2024-09-21T00:25:48.235620+0000","last_became_peered":"2024-09-21T00:25:48.235620+0000","last_unstale":"2024-09-21T00:25:53.263815+0000","last_undegraded":"2024-09-21T00:25:53.263815+0000","last_fullsized":"2024-09-21T00:25:53.263815+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:53:08.415542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.826376+0000","last_change":"2024-09-21T00:25:48.247624+0000","last_active":"2024-09-21T00:25:54.826376+0000","last_peered":"2024-09-21T00:25:54.826376+0000","last_clean":"2024-09-21T00:25:54.826376+0000","last_became_active":"2024-09-21T00:25:48.247318+0000","last_became_peered":"2024-09-21T00:25:48.247318+0000","last_unstale":"2024-09-21T00:25:54.826376+0000","last_undegraded":"2024-09-21T00:25:54.826376+0000","last_fullsized":"2024-09-21T00:25:54.826376+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:34:07.237786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.263196+0000","last_change":"2024-09-21T00:25:48.232468+0000","last_active":"2024-09-21T00:25:53.263196+0000","last_peered":"2024-09-21T00:25:53.263196+0000","last_clean":"2024-09-21T00:25:53.263196+0000","last_became_active":"2024-09-21T00:25:48.231259+0000","last_became_peered":"2024-09-21T00:25:48.231259+0000","last_unstale":"2024-09-21T00:25:53.263196+0000","last_undegraded":"2024-09-21T00:25:53.263196+0000","last_fullsized":"2024-09-21T00:25:53.263196+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:07:41.736005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.826608+0000","last_change":"2024-09-21T00:25:48.246495+0000","last_active":"2024-09-21T00:25:54.826608+0000","last_peered":"2024-09-21T00:25:54.826608+0000","last_clean":"2024-09-21T00:25:54.826608+0000","last_became_active":"2024-09-21T00:25:48.245978+0000","last_became_peered":"2024-09-21T00:25:48.245978+0000","last_unstale":"2024-09-21T00:25:54.826608+0000","last_undegraded":"2024-09-21T00:25:54.826608+0000","last_fullsized":"2024-09-21T00:25:54.826608+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:48:34.879434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.263683+0000","last_change":"2024-09-21T00:25:48.237710+0000","last_active":"2024-09-21T00:25:53.263683+0000","last_peered":"2024-09-21T00:25:53.263683+0000","last_clean":"2024-09-21T00:25:53.263683+0000","last_became_active":"2024-09-21T00:25:48.232344+0000","last_became_peered":"2024-09-21T00:25:48.232344+0000","last_unstale":"2024-09-21T00:25:53.263683+0000","last_undegraded":"2024-09-21T00:25:53.263683+0000","last_fullsized":"2024-09-21T00:25:53.263683+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:48:34.879434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.826561+0000","last_change":"2024-09-21T00:25:48.240393+0000","last_active":"2024-09-21T00:25:54.826561+0000","last_peered":"2024-09-21T00:25:54.826561+0000","last_clean":"2024-09-21T00:25:54.826561+0000","last_became_active":"2024-09-21T00:25:48.240086+0000","last_became_peered":"2024-09-21T00:25:48.240086+0000","last_unstale":"2024-09-21T00:25:54.826561+0000","last_undegraded":"2024-09-21T00:25:54.826561+0000","last_fullsized":"2024-09-21T00:25:54.826561+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:12:39.910729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.324667+0000","last_change":"2024-09-21T00:25:48.230210+0000","last_active":"2024-09-21T00:25:54.324667+0000","last_peered":"2024-09-21T00:25:54.324667+0000","last_clean":"2024-09-21T00:25:54.324667+0000","last_became_active":"2024-09-21T00:25:48.229837+0000","last_became_peered":"2024-09-21T00:25:48.229837+0000","last_unstale":"2024-09-21T00:25:54.324667+0000","last_undegraded":"2024-09-21T00:25:54.324667+0000","last_fullsized":"2024-09-21T00:25:54.324667+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:23:42.611364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.827462+0000","last_change":"2024-09-21T00:25:48.242566+0000","last_active":"2024-09-21T00:25:54.827462+0000","last_peered":"2024-09-21T00:25:54.827462+0000","last_clean":"2024-09-21T00:25:54.827462+0000","last_became_active":"2024-09-21T00:25:48.242265+0000","last_became_peered":"2024-09-21T00:25:48.242265+0000","last_unstale":"2024-09-21T00:25:54.827462+0000","last_undegraded":"2024-09-21T00:25:54.827462+0000","last_fullsized":"2024-09-21T00:25:54.827462+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:49:57.219481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509446,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27880,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343960,"statfs":{"total":96636764160,"available":96608215040,"internally_reserved":0,"allocated":868352,"data_stored":697130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509447,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":282624,"data_stored":107792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509446,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":274432,"data_stored":106270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509445,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":282624,"data_stored":106306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509447,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":282624,"data_stored":107792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509447,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":270336,"data_stored":106180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509447,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27888,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343952,"statfs":{"total":96636764160,"available":96608206848,"internally_reserved":0,"allocated":876544,"data_stored":697186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":278528,"data_stored":106340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T00:26:02.315 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T00:26:02.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.461+0000 7fe20cf58640 1 -- 172.21.15.112:0/3936742129 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe1f4005350 msgr2=0x7fe1f400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:02.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.461+0000 7fe20cf58640 1 --2- 172.21.15.112:0/3936742129 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe1f4005350 0x7fe1f400b9c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe200001d90 tx=0x7fe1f4005830 comp rx=0 tx=0).stop 2024-09-21T00:26:02.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.461+0000 7fe20cf58640 1 -- 172.21.15.112:0/3936742129 shutdown_connections 2024-09-21T00:26:02.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.461+0000 7fe20cf58640 1 --2- 172.21.15.112:0/3936742129 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe1f4005350 0x7fe1f400b9c0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:02.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.461+0000 7fe20cf58640 1 -- 172.21.15.112:0/3936742129 >> 172.21.15.112:0/3936742129 conn(0x7fe2080b6b20 msgr2=0x7fe2080d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:02.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.461+0000 7fe20cf58640 1 -- 172.21.15.112:0/3936742129 shutdown_connections 2024-09-21T00:26:02.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.461+0000 7fe20cf58640 1 -- 172.21.15.112:0/3936742129 wait complete. 2024-09-21T00:26:02.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.463+0000 7fe20cf58640 1 Processor -- start 2024-09-21T00:26:02.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.463+0000 7fe20cf58640 1 -- start start 2024-09-21T00:26:02.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.464+0000 7fe20cf58640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe1f4005350 0x7fe2081f8530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:02.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.464+0000 7fe20cf58640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2081f5970 0x7fe20814d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:02.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.464+0000 7fe20cf58640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe2080af680 0x7fe2080afa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:02.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.464+0000 7fe20cf58640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe208162810 con 0x7fe1f4005350 2024-09-21T00:26:02.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.464+0000 7fe20cf58640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe208162690 con 0x7fe2080af680 2024-09-21T00:26:02.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.464+0000 7fe20cf58640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe208162990 con 0x7fe2081f5970 2024-09-21T00:26:02.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.464+0000 7fe20659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe1f4005350 0x7fe2081f8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:02.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.464+0000 7fe20659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe1f4005350 0x7fe2081f8530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45610/0 (socket says 172.21.15.112:45610) 2024-09-21T00:26:02.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.464+0000 7fe20659c640 1 -- 172.21.15.112:0/3253416457 learned_addr learned my addr 172.21.15.112:0/3253416457 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:26:02.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.464+0000 7fe1fdd9b640 1 --2- 172.21.15.112:0/3253416457 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2081f5970 0x7fe20814d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:02.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.464+0000 7fe206d9d640 1 --2- 172.21.15.112:0/3253416457 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe2080af680 0x7fe2080afa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:02.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.465+0000 7fe20659c640 1 -- 172.21.15.112:0/3253416457 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2081f5970 msgr2=0x7fe20814d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:02.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.465+0000 7fe20659c640 1 --2- 172.21.15.112:0/3253416457 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2081f5970 0x7fe20814d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:02.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.465+0000 7fe20659c640 1 -- 172.21.15.112:0/3253416457 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe2080af680 msgr2=0x7fe2080afa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:02.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.465+0000 7fe20659c640 1 --2- 172.21.15.112:0/3253416457 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe2080af680 0x7fe2080afa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:02.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.465+0000 7fe20659c640 1 -- 172.21.15.112:0/3253416457 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2081f8a70 con 0x7fe1f4005350 2024-09-21T00:26:02.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.465+0000 7fe206d9d640 1 --2- 172.21.15.112:0/3253416457 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe2080af680 0x7fe2080afa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:26:02.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.465+0000 7fe1fdd9b640 1 --2- 172.21.15.112:0/3253416457 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2081f5970 0x7fe20814d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:26:02.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.465+0000 7fe20659c640 1 --2- 172.21.15.112:0/3253416457 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe1f4005350 0x7fe2081f8530 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fe1f8006680 tx=0x7fe1f8006f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:02.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.465+0000 7fe1ff7fe640 1 -- 172.21.15.112:0/3253416457 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1f8015070 con 0x7fe1f4005350 2024-09-21T00:26:02.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.465+0000 7fe1ff7fe640 1 -- 172.21.15.112:0/3253416457 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe1f8011900 con 0x7fe1f4005350 2024-09-21T00:26:02.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.465+0000 7fe20cf58640 1 -- 172.21.15.112:0/3253416457 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2081f8f80 con 0x7fe1f4005350 2024-09-21T00:26:02.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.465+0000 7fe1ff7fe640 1 -- 172.21.15.112:0/3253416457 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1f8011d10 con 0x7fe1f4005350 2024-09-21T00:26:02.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.465+0000 7fe20cf58640 1 -- 172.21.15.112:0/3253416457 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2081f5170 con 0x7fe1f4005350 2024-09-21T00:26:02.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.466+0000 7fe20cf58640 1 -- 172.21.15.112:0/3253416457 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2080d2850 con 0x7fe1f4005350 2024-09-21T00:26:02.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.467+0000 7fe1ff7fe640 1 -- 172.21.15.112:0/3253416457 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe1f8018070 con 0x7fe1f4005350 2024-09-21T00:26:02.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.467+0000 7fe1ff7fe640 1 --2- 172.21.15.112:0/3253416457 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fe1e8078bc0 0x7fe1e807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:02.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.467+0000 7fe1ff7fe640 1 -- 172.21.15.112:0/3253416457 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fe1f809bc90 con 0x7fe1f4005350 2024-09-21T00:26:02.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.468+0000 7fe1fdd9b640 1 --2- 172.21.15.112:0/3253416457 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fe1e8078bc0 0x7fe1e807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:02.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.468+0000 7fe1fdd9b640 1 --2- 172.21.15.112:0/3253416457 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fe1e8078bc0 0x7fe1e807b0d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe2080b2e10 tx=0x7fe1f0005f90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:02.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.472+0000 7fe1ff7fe640 1 -- 172.21.15.112:0/3253416457 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe208162bd0 con 0x7fe1f4005350 2024-09-21T00:26:02.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.646+0000 7fe20cf58640 1 -- 172.21.15.112:0/3253416457 --> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe208161dd0 con 0x7fe1e8078bc0 2024-09-21T00:26:02.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.653+0000 7fe1ff7fe640 1 -- 172.21.15.112:0/3253416457 <== mgr.4103 v2:172.21.15.187:6800/2921522906 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332619 (secure 0 0 0) 0x7fe208161dd0 con 0x7fe1e8078bc0 2024-09-21T00:26:02.657 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:26:02.659 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2024-09-21T00:26:02.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.656+0000 7fe20cf58640 1 -- 172.21.15.112:0/3253416457 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fe1e8078bc0 msgr2=0x7fe1e807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:02.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.656+0000 7fe20cf58640 1 --2- 172.21.15.112:0/3253416457 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fe1e8078bc0 0x7fe1e807b0d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe2080b2e10 tx=0x7fe1f0005f90 comp rx=0 tx=0).stop 2024-09-21T00:26:02.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.656+0000 7fe20cf58640 1 -- 172.21.15.112:0/3253416457 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe1f4005350 msgr2=0x7fe2081f8530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:02.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.656+0000 7fe20cf58640 1 --2- 172.21.15.112:0/3253416457 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe1f4005350 0x7fe2081f8530 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fe1f8006680 tx=0x7fe1f8006f90 comp rx=0 tx=0).stop 2024-09-21T00:26:02.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.656+0000 7fe20cf58640 1 -- 172.21.15.112:0/3253416457 shutdown_connections 2024-09-21T00:26:02.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.656+0000 7fe20cf58640 1 --2- 172.21.15.112:0/3253416457 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe2080af680 0x7fe2080afa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:02.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.656+0000 7fe20cf58640 1 --2- 172.21.15.112:0/3253416457 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2081f5970 0x7fe20814d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:02.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.656+0000 7fe20cf58640 1 --2- 172.21.15.112:0/3253416457 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fe1e8078bc0 0x7fe1e807b0d0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:02.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.656+0000 7fe20cf58640 1 --2- 172.21.15.112:0/3253416457 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe1f4005350 0x7fe2081f8530 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:02.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.656+0000 7fe20cf58640 1 -- 172.21.15.112:0/3253416457 >> 172.21.15.112:0/3253416457 conn(0x7fe2080b6b20 msgr2=0x7fe2080b8890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:02.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.656+0000 7fe20cf58640 1 -- 172.21.15.112:0/3253416457 shutdown_connections 2024-09-21T00:26:02.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.656+0000 7fe20cf58640 1 -- 172.21.15.112:0/3253416457 wait complete. 2024-09-21T00:26:02.669 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":33,"stamp":"2024-09-21T00:26:01.090766+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":308,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219584,"kb_used_data":3336,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755136,"statfs":{"total":773094113280,"available":772869259264,"internally_reserved":0,"allocated":3416064,"data_stored":2034996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.844092"},"pg_stats":[{"pgid":"2.1f","version":"21'4","reported_seq":27,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.826824+0000","last_change":"2024-09-21T00:25:47.232163+0000","last_active":"2024-09-21T00:25:54.826824+0000","last_peered":"2024-09-21T00:25:54.826824+0000","last_clean":"2024-09-21T00:25:54.826824+0000","last_became_active":"2024-09-21T00:25:47.231843+0000","last_became_peered":"2024-09-21T00:25:47.231843+0000","last_unstale":"2024-09-21T00:25:54.826824+0000","last_undegraded":"2024-09-21T00:25:54.826824+0000","last_fullsized":"2024-09-21T00:25:54.826824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_clean_scrub_stamp":"2024-09-21T00:25:46.193690+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:28:45.453931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.264854+0000","last_change":"2024-09-21T00:25:48.231612+0000","last_active":"2024-09-21T00:25:53.264854+0000","last_peered":"2024-09-21T00:25:53.264854+0000","last_clean":"2024-09-21T00:25:53.264854+0000","last_became_active":"2024-09-21T00:25:48.230971+0000","last_became_peered":"2024-09-21T00:25:48.230971+0000","last_unstale":"2024-09-21T00:25:53.264854+0000","last_undegraded":"2024-09-21T00:25:53.264854+0000","last_fullsized":"2024-09-21T00:25:53.264854+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:07:41.736005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.263784+0000","last_change":"2024-09-21T00:25:52.552378+0000","last_active":"2024-09-21T00:25:53.263784+0000","last_peered":"2024-09-21T00:25:53.263784+0000","last_clean":"2024-09-21T00:25:53.263784+0000","last_became_active":"2024-09-21T00:25:52.551577+0000","last_became_peered":"2024-09-21T00:25:52.551577+0000","last_unstale":"2024-09-21T00:25:53.263784+0000","last_undegraded":"2024-09-21T00:25:53.263784+0000","last_fullsized":"2024-09-21T00:25:53.263784+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_clean_scrub_stamp":"2024-09-21T00:25:51.238933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:54:49.138927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.265044+0000","last_change":"2024-09-21T00:25:47.232065+0000","last_active":"2024-09-21T00:25:53.265044+0000","last_peered":"2024-09-21T00:25:53.265044+0000","last_clean":"2024-09-21T00:25:53.265044+0000","last_became_active":"2024-09-21T00:25:47.231772+0000","last_became_peered":"2024-09-21T00:25:47.231772+0000","last_unstale":"2024-09-21T00:25:53.265044+0000","last_undegraded":"2024-09-21T00:25:53.265044+0000","last_fullsized":"2024-09-21T00:25:53.265044+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_clean_scrub_stamp":"2024-09-21T00:25:46.193690+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:28:45.453931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.826715+0000","last_change":"2024-09-21T00:25:48.233905+0000","last_active":"2024-09-21T00:25:54.826715+0000","last_peered":"2024-09-21T00:25:54.826715+0000","last_clean":"2024-09-21T00:25:54.826715+0000","last_became_active":"2024-09-21T00:25:48.233606+0000","last_became_peered":"2024-09-21T00:25:48.233606+0000","last_unstale":"2024-09-21T00:25:54.826715+0000","last_undegraded":"2024-09-21T00:25:54.826715+0000","last_fullsized":"2024-09-21T00:25:54.826715+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:28:33.742798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.262279+0000","last_change":"2024-09-21T00:25:52.281768+0000","last_active":"2024-09-21T00:25:53.262279+0000","last_peered":"2024-09-21T00:25:53.262279+0000","last_clean":"2024-09-21T00:25:53.262279+0000","last_became_active":"2024-09-21T00:25:52.281352+0000","last_became_peered":"2024-09-21T00:25:52.281352+0000","last_unstale":"2024-09-21T00:25:53.262279+0000","last_undegraded":"2024-09-21T00:25:53.262279+0000","last_fullsized":"2024-09-21T00:25:53.262279+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_clean_scrub_stamp":"2024-09-21T00:25:51.238933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:51:39.910640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.827174+0000","last_change":"2024-09-21T00:25:47.238564+0000","last_active":"2024-09-21T00:25:54.827174+0000","last_peered":"2024-09-21T00:25:54.827174+0000","last_clean":"2024-09-21T00:25:54.827174+0000","last_became_active":"2024-09-21T00:25:47.238114+0000","last_became_peered":"2024-09-21T00:25:47.238114+0000","last_unstale":"2024-09-21T00:25:54.827174+0000","last_undegraded":"2024-09-21T00:25:54.827174+0000","last_fullsized":"2024-09-21T00:25:54.827174+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_clean_scrub_stamp":"2024-09-21T00:25:46.193690+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:37:44.588178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.263783+0000","last_change":"2024-09-21T00:25:48.223307+0000","last_active":"2024-09-21T00:25:53.263783+0000","last_peered":"2024-09-21T00:25:53.263783+0000","last_clean":"2024-09-21T00:25:53.263783+0000","last_became_active":"2024-09-21T00:25:48.222984+0000","last_became_peered":"2024-09-21T00:25:48.222984+0000","last_unstale":"2024-09-21T00:25:53.263783+0000","last_undegraded":"2024-09-21T00:25:53.263783+0000","last_fullsized":"2024-09-21T00:25:53.263783+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:47:38.377031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.263168+0000","last_change":"2024-09-21T00:25:52.282049+0000","last_active":"2024-09-21T00:25:53.263168+0000","last_peered":"2024-09-21T00:25:53.263168+0000","last_clean":"2024-09-21T00:25:53.263168+0000","last_became_active":"2024-09-21T00:25:52.281616+0000","last_became_peered":"2024-09-21T00:25:52.281616+0000","last_unstale":"2024-09-21T00:25:53.263168+0000","last_undegraded":"2024-09-21T00:25:53.263168+0000","last_fullsized":"2024-09-21T00:25:53.263168+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_clean_scrub_stamp":"2024-09-21T00:25:51.238933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:32:36.647007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.826681+0000","last_change":"2024-09-21T00:25:47.238389+0000","last_active":"2024-09-21T00:25:54.826681+0000","last_peered":"2024-09-21T00:25:54.826681+0000","last_clean":"2024-09-21T00:25:54.826681+0000","last_became_active":"2024-09-21T00:25:47.238069+0000","last_became_peered":"2024-09-21T00:25:47.238069+0000","last_unstale":"2024-09-21T00:25:54.826681+0000","last_undegraded":"2024-09-21T00:25:54.826681+0000","last_fullsized":"2024-09-21T00:25:54.826681+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_clean_scrub_stamp":"2024-09-21T00:25:46.193690+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T00:47:54.121411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.264628+0000","last_change":"2024-09-21T00:25:48.220825+0000","last_active":"2024-09-21T00:25:53.264628+0000","last_peered":"2024-09-21T00:25:53.264628+0000","last_clean":"2024-09-21T00:25:53.264628+0000","last_became_active":"2024-09-21T00:25:48.220517+0000","last_became_peered":"2024-09-21T00:25:48.220517+0000","last_unstale":"2024-09-21T00:25:53.264628+0000","last_undegraded":"2024-09-21T00:25:53.264628+0000","last_fullsized":"2024-09-21T00:25:53.264628+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:42:57.617812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.262582+0000","last_change":"2024-09-21T00:25:52.266314+0000","last_active":"2024-09-21T00:25:53.262582+0000","last_peered":"2024-09-21T00:25:53.262582+0000","last_clean":"2024-09-21T00:25:53.262582+0000","last_became_active":"2024-09-21T00:25:52.265684+0000","last_became_peered":"2024-09-21T00:25:52.265684+0000","last_unstale":"2024-09-21T00:25:53.262582+0000","last_undegraded":"2024-09-21T00:25:53.262582+0000","last_fullsized":"2024-09-21T00:25:53.262582+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_clean_scrub_stamp":"2024-09-21T00:25:51.238933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:56:45.075934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.826737+0000","last_change":"2024-09-21T00:25:48.242274+0000","last_active":"2024-09-21T00:25:54.826737+0000","last_peered":"2024-09-21T00:25:54.826737+0000","last_clean":"2024-09-21T00:25:54.826737+0000","last_became_active":"2024-09-21T00:25:48.241953+0000","last_became_peered":"2024-09-21T00:25:48.241953+0000","last_unstale":"2024-09-21T00:25:54.826737+0000","last_undegraded":"2024-09-21T00:25:54.826737+0000","last_fullsized":"2024-09-21T00:25:54.826737+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:23:42.611364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.826740+0000","last_change":"2024-09-21T00:25:47.233509+0000","last_active":"2024-09-21T00:25:54.826740+0000","last_peered":"2024-09-21T00:25:54.826740+0000","last_clean":"2024-09-21T00:25:54.826740+0000","last_became_active":"2024-09-21T00:25:47.233118+0000","last_became_peered":"2024-09-21T00:25:47.233118+0000","last_unstale":"2024-09-21T00:25:54.826740+0000","last_undegraded":"2024-09-21T00:25:54.826740+0000","last_fullsized":"2024-09-21T00:25:54.826740+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_clean_scrub_stamp":"2024-09-21T00:25:46.193690+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:24:04.907044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.264178+0000","last_change":"2024-09-21T00:25:52.276976+0000","last_active":"2024-09-21T00:25:53.264178+0000","last_peered":"2024-09-21T00:25:53.264178+0000","last_clean":"2024-09-21T00:25:53.264178+0000","last_became_active":"2024-09-21T00:25:52.276454+0000","last_became_peered":"2024-09-21T00:25:52.276454+0000","last_unstale":"2024-09-21T00:25:53.264178+0000","last_undegraded":"2024-09-21T00:25:53.264178+0000","last_fullsized":"2024-09-21T00:25:53.264178+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_clean_scrub_stamp":"2024-09-21T00:25:51.238933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T00:45:52.375548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.264258+0000","last_change":"2024-09-21T00:25:48.237366+0000","last_active":"2024-09-21T00:25:53.264258+0000","last_peered":"2024-09-21T00:25:53.264258+0000","last_clean":"2024-09-21T00:25:53.264258+0000","last_became_active":"2024-09-21T00:25:48.237067+0000","last_became_peered":"2024-09-21T00:25:48.237067+0000","last_unstale":"2024-09-21T00:25:53.264258+0000","last_undegraded":"2024-09-21T00:25:53.264258+0000","last_fullsized":"2024-09-21T00:25:53.264258+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:48:25.219726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.324479+0000","last_change":"2024-09-21T00:25:47.227138+0000","last_active":"2024-09-21T00:25:54.324479+0000","last_peered":"2024-09-21T00:25:54.324479+0000","last_clean":"2024-09-21T00:25:54.324479+0000","last_became_active":"2024-09-21T00:25:47.226840+0000","last_became_peered":"2024-09-21T00:25:47.226840+0000","last_unstale":"2024-09-21T00:25:54.324479+0000","last_undegraded":"2024-09-21T00:25:54.324479+0000","last_fullsized":"2024-09-21T00:25:54.324479+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_clean_scrub_stamp":"2024-09-21T00:25:46.193690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:35:33.351025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.262544+0000","last_change":"2024-09-21T00:25:52.281424+0000","last_active":"2024-09-21T00:25:53.262544+0000","last_peered":"2024-09-21T00:25:53.262544+0000","last_clean":"2024-09-21T00:25:53.262544+0000","last_became_active":"2024-09-21T00:25:52.280787+0000","last_became_peered":"2024-09-21T00:25:52.280787+0000","last_unstale":"2024-09-21T00:25:53.262544+0000","last_undegraded":"2024-09-21T00:25:53.262544+0000","last_fullsized":"2024-09-21T00:25:53.262544+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_clean_scrub_stamp":"2024-09-21T00:25:51.238933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:12:43.948562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.264302+0000","last_change":"2024-09-21T00:25:48.239140+0000","last_active":"2024-09-21T00:25:53.264302+0000","last_peered":"2024-09-21T00:25:53.264302+0000","last_clean":"2024-09-21T00:25:53.264302+0000","last_became_active":"2024-09-21T00:25:48.238778+0000","last_became_peered":"2024-09-21T00:25:48.238778+0000","last_unstale":"2024-09-21T00:25:53.264302+0000","last_undegraded":"2024-09-21T00:25:53.264302+0000","last_fullsized":"2024-09-21T00:25:53.264302+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:10:22.176582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.324123+0000","last_change":"2024-09-21T00:25:47.228464+0000","last_active":"2024-09-21T00:25:54.324123+0000","last_peered":"2024-09-21T00:25:54.324123+0000","last_clean":"2024-09-21T00:25:54.324123+0000","last_became_active":"2024-09-21T00:25:47.228152+0000","last_became_peered":"2024-09-21T00:25:47.228152+0000","last_unstale":"2024-09-21T00:25:54.324123+0000","last_undegraded":"2024-09-21T00:25:54.324123+0000","last_fullsized":"2024-09-21T00:25:54.324123+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_clean_scrub_stamp":"2024-09-21T00:25:46.193690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:06:39.901371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.324932+0000","last_change":"2024-09-21T00:25:52.550315+0000","last_active":"2024-09-21T00:25:54.324932+0000","last_peered":"2024-09-21T00:25:54.324932+0000","last_clean":"2024-09-21T00:25:54.324932+0000","last_became_active":"2024-09-21T00:25:52.549843+0000","last_became_peered":"2024-09-21T00:25:52.549843+0000","last_unstale":"2024-09-21T00:25:54.324932+0000","last_undegraded":"2024-09-21T00:25:54.324932+0000","last_fullsized":"2024-09-21T00:25:54.324932+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_clean_scrub_stamp":"2024-09-21T00:25:51.238933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:20:29.694351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.263895+0000","last_change":"2024-09-21T00:25:48.235177+0000","last_active":"2024-09-21T00:25:53.263895+0000","last_peered":"2024-09-21T00:25:53.263895+0000","last_clean":"2024-09-21T00:25:53.263895+0000","last_became_active":"2024-09-21T00:25:48.234870+0000","last_became_peered":"2024-09-21T00:25:48.234870+0000","last_unstale":"2024-09-21T00:25:53.263895+0000","last_undegraded":"2024-09-21T00:25:53.263895+0000","last_fullsized":"2024-09-21T00:25:53.263895+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:55:29.494036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.826534+0000","last_change":"2024-09-21T00:25:47.237638+0000","last_active":"2024-09-21T00:25:54.826534+0000","last_peered":"2024-09-21T00:25:54.826534+0000","last_clean":"2024-09-21T00:25:54.826534+0000","last_became_active":"2024-09-21T00:25:47.237293+0000","last_became_peered":"2024-09-21T00:25:47.237293+0000","last_unstale":"2024-09-21T00:25:54.826534+0000","last_undegraded":"2024-09-21T00:25:54.826534+0000","last_fullsized":"2024-09-21T00:25:54.826534+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_clean_scrub_stamp":"2024-09-21T00:25:46.193690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:57:58.912899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.264463+0000","last_change":"2024-09-21T00:25:52.277124+0000","last_active":"2024-09-21T00:25:53.264463+0000","last_peered":"2024-09-21T00:25:53.264463+0000","last_clean":"2024-09-21T00:25:53.264463+0000","last_became_active":"2024-09-21T00:25:52.276353+0000","last_became_peered":"2024-09-21T00:25:52.276353+0000","last_unstale":"2024-09-21T00:25:53.264463+0000","last_undegraded":"2024-09-21T00:25:53.264463+0000","last_fullsized":"2024-09-21T00:25:53.264463+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_clean_scrub_stamp":"2024-09-21T00:25:51.238933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:13:43.796680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.826871+0000","last_change":"2024-09-21T00:25:48.229327+0000","last_active":"2024-09-21T00:25:54.826871+0000","last_peered":"2024-09-21T00:25:54.826871+0000","last_clean":"2024-09-21T00:25:54.826871+0000","last_became_active":"2024-09-21T00:25:48.228524+0000","last_became_peered":"2024-09-21T00:25:48.228524+0000","last_unstale":"2024-09-21T00:25:54.826871+0000","last_undegraded":"2024-09-21T00:25:54.826871+0000","last_fullsized":"2024-09-21T00:25:54.826871+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:28:02.233472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.827937+0000","last_change":"2024-09-21T00:25:47.232666+0000","last_active":"2024-09-21T00:25:54.827937+0000","last_peered":"2024-09-21T00:25:54.827937+0000","last_clean":"2024-09-21T00:25:54.827937+0000","last_became_active":"2024-09-21T00:25:47.232345+0000","last_became_peered":"2024-09-21T00:25:47.232345+0000","last_unstale":"2024-09-21T00:25:54.827937+0000","last_undegraded":"2024-09-21T00:25:54.827937+0000","last_fullsized":"2024-09-21T00:25:54.827937+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_clean_scrub_stamp":"2024-09-21T00:25:46.193690+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:43:08.714258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.263529+0000","last_change":"2024-09-21T00:25:52.550535+0000","last_active":"2024-09-21T00:25:53.263529+0000","last_peered":"2024-09-21T00:25:53.263529+0000","last_clean":"2024-09-21T00:25:53.263529+0000","last_became_active":"2024-09-21T00:25:52.550055+0000","last_became_peered":"2024-09-21T00:25:52.550055+0000","last_unstale":"2024-09-21T00:25:53.263529+0000","last_undegraded":"2024-09-21T00:25:53.263529+0000","last_fullsized":"2024-09-21T00:25:53.263529+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_clean_scrub_stamp":"2024-09-21T00:25:51.238933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:02:53.617106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.265087+0000","last_change":"2024-09-21T00:25:48.238219+0000","last_active":"2024-09-21T00:25:53.265087+0000","last_peered":"2024-09-21T00:25:53.265087+0000","last_clean":"2024-09-21T00:25:53.265087+0000","last_became_active":"2024-09-21T00:25:48.237770+0000","last_became_peered":"2024-09-21T00:25:48.237770+0000","last_unstale":"2024-09-21T00:25:53.265087+0000","last_undegraded":"2024-09-21T00:25:53.265087+0000","last_fullsized":"2024-09-21T00:25:53.265087+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:34:07.237786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.324720+0000","last_change":"2024-09-21T00:25:47.229361+0000","last_active":"2024-09-21T00:25:54.324720+0000","last_peered":"2024-09-21T00:25:54.324720+0000","last_clean":"2024-09-21T00:25:54.324720+0000","last_became_active":"2024-09-21T00:25:47.228906+0000","last_became_peered":"2024-09-21T00:25:47.228906+0000","last_unstale":"2024-09-21T00:25:54.324720+0000","last_undegraded":"2024-09-21T00:25:54.324720+0000","last_fullsized":"2024-09-21T00:25:54.324720+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_clean_scrub_stamp":"2024-09-21T00:25:46.193690+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:35:43.503231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.263731+0000","last_change":"2024-09-21T00:25:52.550909+0000","last_active":"2024-09-21T00:25:53.263731+0000","last_peered":"2024-09-21T00:25:53.263731+0000","last_clean":"2024-09-21T00:25:53.263731+0000","last_became_active":"2024-09-21T00:25:52.550506+0000","last_became_peered":"2024-09-21T00:25:52.550506+0000","last_unstale":"2024-09-21T00:25:53.263731+0000","last_undegraded":"2024-09-21T00:25:53.263731+0000","last_fullsized":"2024-09-21T00:25:53.263731+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_clean_scrub_stamp":"2024-09-21T00:25:51.238933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:52:16.289687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.324538+0000","last_change":"2024-09-21T00:25:48.232779+0000","last_active":"2024-09-21T00:25:54.324538+0000","last_peered":"2024-09-21T00:25:54.324538+0000","last_clean":"2024-09-21T00:25:54.324538+0000","last_became_active":"2024-09-21T00:25:48.232461+0000","last_became_peered":"2024-09-21T00:25:48.232461+0000","last_unstale":"2024-09-21T00:25:54.324538+0000","last_undegraded":"2024-09-21T00:25:54.324538+0000","last_fullsized":"2024-09-21T00:25:54.324538+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:52:33.265365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.827491+0000","last_change":"2024-09-21T00:25:47.237661+0000","last_active":"2024-09-21T00:25:54.827491+0000","last_peered":"2024-09-21T00:25:54.827491+0000","last_clean":"2024-09-21T00:25:54.827491+0000","last_became_active":"2024-09-21T00:25:47.237273+0000","last_became_peered":"2024-09-21T00:25:47.237273+0000","last_unstale":"2024-09-21T00:25:54.827491+0000","last_undegraded":"2024-09-21T00:25:54.827491+0000","last_fullsized":"2024-09-21T00:25:54.827491+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_clean_scrub_stamp":"2024-09-21T00:25:46.193690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:03:52.795421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.826240+0000","last_change":"2024-09-21T00:25:52.549849+0000","last_active":"2024-09-21T00:25:54.826240+0000","last_peered":"2024-09-21T00:25:54.826240+0000","last_clean":"2024-09-21T00:25:54.826240+0000","last_became_active":"2024-09-21T00:25:52.549433+0000","last_became_peered":"2024-09-21T00:25:52.549433+0000","last_unstale":"2024-09-21T00:25:54.826240+0000","last_undegraded":"2024-09-21T00:25:54.826240+0000","last_fullsized":"2024-09-21T00:25:54.826240+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_clean_scrub_stamp":"2024-09-21T00:25:51.238933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:51:49.334552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.264813+0000","last_change":"2024-09-21T00:25:48.232652+0000","last_active":"2024-09-21T00:25:53.264813+0000","last_peered":"2024-09-21T00:25:53.264813+0000","last_clean":"2024-09-21T00:25:53.264813+0000","last_became_active":"2024-09-21T00:25:48.232099+0000","last_became_peered":"2024-09-21T00:25:48.232099+0000","last_unstale":"2024-09-21T00:25:53.264813+0000","last_undegraded":"2024-09-21T00:25:53.264813+0000","last_fullsized":"2024-09-21T00:25:53.264813+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:48:25.219726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.324657+0000","last_change":"2024-09-21T00:25:47.223473+0000","last_active":"2024-09-21T00:25:54.324657+0000","last_peered":"2024-09-21T00:25:54.324657+0000","last_clean":"2024-09-21T00:25:54.324657+0000","last_became_active":"2024-09-21T00:25:47.223142+0000","last_became_peered":"2024-09-21T00:25:47.223142+0000","last_unstale":"2024-09-21T00:25:54.324657+0000","last_undegraded":"2024-09-21T00:25:54.324657+0000","last_fullsized":"2024-09-21T00:25:54.324657+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_clean_scrub_stamp":"2024-09-21T00:25:46.193690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:54:03.173672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.324126+0000","last_change":"2024-09-21T00:25:52.276922+0000","last_active":"2024-09-21T00:25:54.324126+0000","last_peered":"2024-09-21T00:25:54.324126+0000","last_clean":"2024-09-21T00:25:54.324126+0000","last_became_active":"2024-09-21T00:25:52.276329+0000","last_became_peered":"2024-09-21T00:25:52.276329+0000","last_unstale":"2024-09-21T00:25:54.324126+0000","last_undegraded":"2024-09-21T00:25:54.324126+0000","last_fullsized":"2024-09-21T00:25:54.324126+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_clean_scrub_stamp":"2024-09-21T00:25:51.238933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:29:17.920252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.264377+0000","last_change":"2024-09-21T00:25:48.230144+0000","last_active":"2024-09-21T00:25:53.264377+0000","last_peered":"2024-09-21T00:25:53.264377+0000","last_clean":"2024-09-21T00:25:53.264377+0000","last_became_active":"2024-09-21T00:25:48.229848+0000","last_became_peered":"2024-09-21T00:25:48.229848+0000","last_unstale":"2024-09-21T00:25:53.264377+0000","last_undegraded":"2024-09-21T00:25:53.264377+0000","last_fullsized":"2024-09-21T00:25:53.264377+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:57:59.920308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.263649+0000","last_change":"2024-09-21T00:25:47.228030+0000","last_active":"2024-09-21T00:25:53.263649+0000","last_peered":"2024-09-21T00:25:53.263649+0000","last_clean":"2024-09-21T00:25:53.263649+0000","last_became_active":"2024-09-21T00:25:47.227688+0000","last_became_peered":"2024-09-21T00:25:47.227688+0000","last_unstale":"2024-09-21T00:25:53.263649+0000","last_undegraded":"2024-09-21T00:25:53.263649+0000","last_fullsized":"2024-09-21T00:25:53.263649+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_clean_scrub_stamp":"2024-09-21T00:25:46.193690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:34:30.210849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.324624+0000","last_change":"2024-09-21T00:25:52.276547+0000","last_active":"2024-09-21T00:25:54.324624+0000","last_peered":"2024-09-21T00:25:54.324624+0000","last_clean":"2024-09-21T00:25:54.324624+0000","last_became_active":"2024-09-21T00:25:52.276100+0000","last_became_peered":"2024-09-21T00:25:52.276100+0000","last_unstale":"2024-09-21T00:25:54.324624+0000","last_undegraded":"2024-09-21T00:25:54.324624+0000","last_fullsized":"2024-09-21T00:25:54.324624+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_clean_scrub_stamp":"2024-09-21T00:25:51.238933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:08:54.038371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.324189+0000","last_change":"2024-09-21T00:25:48.233554+0000","last_active":"2024-09-21T00:25:54.324189+0000","last_peered":"2024-09-21T00:25:54.324189+0000","last_clean":"2024-09-21T00:25:54.324189+0000","last_became_active":"2024-09-21T00:25:48.233249+0000","last_became_peered":"2024-09-21T00:25:48.233249+0000","last_unstale":"2024-09-21T00:25:54.324189+0000","last_undegraded":"2024-09-21T00:25:54.324189+0000","last_fullsized":"2024-09-21T00:25:54.324189+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:18:14.964429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.242121+0000","last_change":"2024-09-21T00:25:47.228079+0000","last_active":"2024-09-21T00:25:54.242121+0000","last_peered":"2024-09-21T00:25:54.242121+0000","last_clean":"2024-09-21T00:25:54.242121+0000","last_became_active":"2024-09-21T00:25:47.227731+0000","last_became_peered":"2024-09-21T00:25:47.227731+0000","last_unstale":"2024-09-21T00:25:54.242121+0000","last_undegraded":"2024-09-21T00:25:54.242121+0000","last_fullsized":"2024-09-21T00:25:54.242121+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_clean_scrub_stamp":"2024-09-21T00:25:46.193690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:53:37.081118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.324987+0000","last_change":"2024-09-21T00:25:52.278680+0000","last_active":"2024-09-21T00:25:54.324987+0000","last_peered":"2024-09-21T00:25:54.324987+0000","last_clean":"2024-09-21T00:25:54.324987+0000","last_became_active":"2024-09-21T00:25:52.278128+0000","last_became_peered":"2024-09-21T00:25:52.278128+0000","last_unstale":"2024-09-21T00:25:54.324987+0000","last_undegraded":"2024-09-21T00:25:54.324987+0000","last_fullsized":"2024-09-21T00:25:54.324987+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_clean_scrub_stamp":"2024-09-21T00:25:51.238933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:39:02.970919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.826121+0000","last_change":"2024-09-21T00:25:48.245138+0000","last_active":"2024-09-21T00:25:54.826121+0000","last_peered":"2024-09-21T00:25:54.826121+0000","last_clean":"2024-09-21T00:25:54.826121+0000","last_became_active":"2024-09-21T00:25:48.244796+0000","last_became_peered":"2024-09-21T00:25:48.244796+0000","last_unstale":"2024-09-21T00:25:54.826121+0000","last_undegraded":"2024-09-21T00:25:54.826121+0000","last_fullsized":"2024-09-21T00:25:54.826121+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:53:56.396789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.826939+0000","last_change":"2024-09-21T00:25:47.235076+0000","last_active":"2024-09-21T00:25:54.826939+0000","last_peered":"2024-09-21T00:25:54.826939+0000","last_clean":"2024-09-21T00:25:54.826939+0000","last_became_active":"2024-09-21T00:25:47.234751+0000","last_became_peered":"2024-09-21T00:25:47.234751+0000","last_unstale":"2024-09-21T00:25:54.826939+0000","last_undegraded":"2024-09-21T00:25:54.826939+0000","last_fullsized":"2024-09-21T00:25:54.826939+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_clean_scrub_stamp":"2024-09-21T00:25:46.193690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:40:10.553216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.826789+0000","last_change":"2024-09-21T00:25:52.546980+0000","last_active":"2024-09-21T00:25:54.826789+0000","last_peered":"2024-09-21T00:25:54.826789+0000","last_clean":"2024-09-21T00:25:54.826789+0000","last_became_active":"2024-09-21T00:25:52.546572+0000","last_became_peered":"2024-09-21T00:25:52.546572+0000","last_unstale":"2024-09-21T00:25:54.826789+0000","last_undegraded":"2024-09-21T00:25:54.826789+0000","last_fullsized":"2024-09-21T00:25:54.826789+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_clean_scrub_stamp":"2024-09-21T00:25:51.238933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:34:14.702564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.263892+0000","last_change":"2024-09-21T00:25:48.227417+0000","last_active":"2024-09-21T00:25:53.263892+0000","last_peered":"2024-09-21T00:25:53.263892+0000","last_clean":"2024-09-21T00:25:53.263892+0000","last_became_active":"2024-09-21T00:25:48.227056+0000","last_became_peered":"2024-09-21T00:25:48.227056+0000","last_unstale":"2024-09-21T00:25:53.263892+0000","last_undegraded":"2024-09-21T00:25:53.263892+0000","last_fullsized":"2024-09-21T00:25:53.263892+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:16:33.163623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.264390+0000","last_change":"2024-09-21T00:25:47.228317+0000","last_active":"2024-09-21T00:25:53.264390+0000","last_peered":"2024-09-21T00:25:53.264390+0000","last_clean":"2024-09-21T00:25:53.264390+0000","last_became_active":"2024-09-21T00:25:47.227983+0000","last_became_peered":"2024-09-21T00:25:47.227983+0000","last_unstale":"2024-09-21T00:25:53.264390+0000","last_undegraded":"2024-09-21T00:25:53.264390+0000","last_fullsized":"2024-09-21T00:25:53.264390+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_clean_scrub_stamp":"2024-09-21T00:25:46.193690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:17:12.345201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":119,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.827077+0000","last_change":"2024-09-21T00:25:38.725792+0000","last_active":"2024-09-21T00:25:54.827077+0000","last_peered":"2024-09-21T00:25:54.827077+0000","last_clean":"2024-09-21T00:25:54.827077+0000","last_became_active":"2024-09-21T00:25:38.724668+0000","last_became_peered":"2024-09-21T00:25:38.724668+0000","last_unstale":"2024-09-21T00:25:54.827077+0000","last_undegraded":"2024-09-21T00:25:54.827077+0000","last_fullsized":"2024-09-21T00:25:54.827077+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:37.163216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:37.163216+0000","last_clean_scrub_stamp":"2024-09-21T00:25:37.163216+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:14:36.588660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.827086+0000","last_change":"2024-09-21T00:25:52.551370+0000","last_active":"2024-09-21T00:25:54.827086+0000","last_peered":"2024-09-21T00:25:54.827086+0000","last_clean":"2024-09-21T00:25:54.827086+0000","last_became_active":"2024-09-21T00:25:52.550883+0000","last_became_peered":"2024-09-21T00:25:52.550883+0000","last_unstale":"2024-09-21T00:25:54.827086+0000","last_undegraded":"2024-09-21T00:25:54.827086+0000","last_fullsized":"2024-09-21T00:25:54.827086+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_clean_scrub_stamp":"2024-09-21T00:25:51.238933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T00:31:31.841290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.826927+0000","last_change":"2024-09-21T00:25:48.247427+0000","last_active":"2024-09-21T00:25:54.826927+0000","last_peered":"2024-09-21T00:25:54.826927+0000","last_clean":"2024-09-21T00:25:54.826927+0000","last_became_active":"2024-09-21T00:25:48.246876+0000","last_became_peered":"2024-09-21T00:25:48.246876+0000","last_unstale":"2024-09-21T00:25:54.826927+0000","last_undegraded":"2024-09-21T00:25:54.826927+0000","last_fullsized":"2024-09-21T00:25:54.826927+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:10:01.543628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.242401+0000","last_change":"2024-09-21T00:25:47.233336+0000","last_active":"2024-09-21T00:25:54.242401+0000","last_peered":"2024-09-21T00:25:54.242401+0000","last_clean":"2024-09-21T00:25:54.242401+0000","last_became_active":"2024-09-21T00:25:47.233083+0000","last_became_peered":"2024-09-21T00:25:47.233083+0000","last_unstale":"2024-09-21T00:25:54.242401+0000","last_undegraded":"2024-09-21T00:25:54.242401+0000","last_fullsized":"2024-09-21T00:25:54.242401+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_clean_scrub_stamp":"2024-09-21T00:25:46.193690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:43:39.508965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.826781+0000","last_change":"2024-09-21T00:25:52.550144+0000","last_active":"2024-09-21T00:25:54.826781+0000","last_peered":"2024-09-21T00:25:54.826781+0000","last_clean":"2024-09-21T00:25:54.826781+0000","last_became_active":"2024-09-21T00:25:52.549636+0000","last_became_peered":"2024-09-21T00:25:52.549636+0000","last_unstale":"2024-09-21T00:25:54.826781+0000","last_undegraded":"2024-09-21T00:25:54.826781+0000","last_fullsized":"2024-09-21T00:25:54.826781+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_clean_scrub_stamp":"2024-09-21T00:25:51.238933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:09:27.474968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.263877+0000","last_change":"2024-09-21T00:25:48.229999+0000","last_active":"2024-09-21T00:25:53.263877+0000","last_peered":"2024-09-21T00:25:53.263877+0000","last_clean":"2024-09-21T00:25:53.263877+0000","last_became_active":"2024-09-21T00:25:48.229678+0000","last_became_peered":"2024-09-21T00:25:48.229678+0000","last_unstale":"2024-09-21T00:25:53.263877+0000","last_undegraded":"2024-09-21T00:25:53.263877+0000","last_fullsized":"2024-09-21T00:25:53.263877+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:15:47.572701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.324827+0000","last_change":"2024-09-21T00:25:47.219135+0000","last_active":"2024-09-21T00:25:54.324827+0000","last_peered":"2024-09-21T00:25:54.324827+0000","last_clean":"2024-09-21T00:25:54.324827+0000","last_became_active":"2024-09-21T00:25:47.218722+0000","last_became_peered":"2024-09-21T00:25:47.218722+0000","last_unstale":"2024-09-21T00:25:54.324827+0000","last_undegraded":"2024-09-21T00:25:54.324827+0000","last_fullsized":"2024-09-21T00:25:54.324827+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_clean_scrub_stamp":"2024-09-21T00:25:46.193690+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:52:45.409090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.826420+0000","last_change":"2024-09-21T00:25:52.279946+0000","last_active":"2024-09-21T00:25:54.826420+0000","last_peered":"2024-09-21T00:25:54.826420+0000","last_clean":"2024-09-21T00:25:54.826420+0000","last_became_active":"2024-09-21T00:25:52.279489+0000","last_became_peered":"2024-09-21T00:25:52.279489+0000","last_unstale":"2024-09-21T00:25:54.826420+0000","last_undegraded":"2024-09-21T00:25:54.826420+0000","last_fullsized":"2024-09-21T00:25:54.826420+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_clean_scrub_stamp":"2024-09-21T00:25:51.238933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:47:42.414865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.827648+0000","last_change":"2024-09-21T00:25:48.244146+0000","last_active":"2024-09-21T00:25:54.827648+0000","last_peered":"2024-09-21T00:25:54.827648+0000","last_clean":"2024-09-21T00:25:54.827648+0000","last_became_active":"2024-09-21T00:25:48.243782+0000","last_became_peered":"2024-09-21T00:25:48.243782+0000","last_unstale":"2024-09-21T00:25:54.827648+0000","last_undegraded":"2024-09-21T00:25:54.827648+0000","last_fullsized":"2024-09-21T00:25:54.827648+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:43:59.908291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.242461+0000","last_change":"2024-09-21T00:25:47.230785+0000","last_active":"2024-09-21T00:25:54.242461+0000","last_peered":"2024-09-21T00:25:54.242461+0000","last_clean":"2024-09-21T00:25:54.242461+0000","last_became_active":"2024-09-21T00:25:47.230492+0000","last_became_peered":"2024-09-21T00:25:47.230492+0000","last_unstale":"2024-09-21T00:25:54.242461+0000","last_undegraded":"2024-09-21T00:25:54.242461+0000","last_fullsized":"2024-09-21T00:25:54.242461+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_clean_scrub_stamp":"2024-09-21T00:25:46.193690+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:33:07.366842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.262402+0000","last_change":"2024-09-21T00:25:52.547298+0000","last_active":"2024-09-21T00:25:53.262402+0000","last_peered":"2024-09-21T00:25:53.262402+0000","last_clean":"2024-09-21T00:25:53.262402+0000","last_became_active":"2024-09-21T00:25:52.546737+0000","last_became_peered":"2024-09-21T00:25:52.546737+0000","last_unstale":"2024-09-21T00:25:53.262402+0000","last_undegraded":"2024-09-21T00:25:53.262402+0000","last_fullsized":"2024-09-21T00:25:53.262402+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_clean_scrub_stamp":"2024-09-21T00:25:51.238933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:54:49.138927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.827233+0000","last_change":"2024-09-21T00:25:48.244124+0000","last_active":"2024-09-21T00:25:54.827233+0000","last_peered":"2024-09-21T00:25:54.827233+0000","last_clean":"2024-09-21T00:25:54.827233+0000","last_became_active":"2024-09-21T00:25:48.243767+0000","last_became_peered":"2024-09-21T00:25:48.243767+0000","last_unstale":"2024-09-21T00:25:54.827233+0000","last_undegraded":"2024-09-21T00:25:54.827233+0000","last_fullsized":"2024-09-21T00:25:54.827233+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:15:47.572701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.264246+0000","last_change":"2024-09-21T00:25:47.220781+0000","last_active":"2024-09-21T00:25:53.264246+0000","last_peered":"2024-09-21T00:25:53.264246+0000","last_clean":"2024-09-21T00:25:53.264246+0000","last_became_active":"2024-09-21T00:25:47.220429+0000","last_became_peered":"2024-09-21T00:25:47.220429+0000","last_unstale":"2024-09-21T00:25:53.264246+0000","last_undegraded":"2024-09-21T00:25:53.264246+0000","last_fullsized":"2024-09-21T00:25:53.264246+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_clean_scrub_stamp":"2024-09-21T00:25:46.193690+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:08:24.394801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.262197+0000","last_change":"2024-09-21T00:25:52.276941+0000","last_active":"2024-09-21T00:25:53.262197+0000","last_peered":"2024-09-21T00:25:53.262197+0000","last_clean":"2024-09-21T00:25:53.262197+0000","last_became_active":"2024-09-21T00:25:52.276374+0000","last_became_peered":"2024-09-21T00:25:52.276374+0000","last_unstale":"2024-09-21T00:25:53.262197+0000","last_undegraded":"2024-09-21T00:25:53.262197+0000","last_fullsized":"2024-09-21T00:25:53.262197+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_clean_scrub_stamp":"2024-09-21T00:25:51.238933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:02:24.501560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.826441+0000","last_change":"2024-09-21T00:25:48.244881+0000","last_active":"2024-09-21T00:25:54.826441+0000","last_peered":"2024-09-21T00:25:54.826441+0000","last_clean":"2024-09-21T00:25:54.826441+0000","last_became_active":"2024-09-21T00:25:48.244530+0000","last_became_peered":"2024-09-21T00:25:48.244530+0000","last_unstale":"2024-09-21T00:25:54.826441+0000","last_undegraded":"2024-09-21T00:25:54.826441+0000","last_fullsized":"2024-09-21T00:25:54.826441+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:51:31.831106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":31,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.242013+0000","last_change":"2024-09-21T00:25:47.230620+0000","last_active":"2024-09-21T00:25:54.242013+0000","last_peered":"2024-09-21T00:25:54.242013+0000","last_clean":"2024-09-21T00:25:54.242013+0000","last_became_active":"2024-09-21T00:25:47.230252+0000","last_became_peered":"2024-09-21T00:25:47.230252+0000","last_unstale":"2024-09-21T00:25:54.242013+0000","last_undegraded":"2024-09-21T00:25:54.242013+0000","last_fullsized":"2024-09-21T00:25:54.242013+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_clean_scrub_stamp":"2024-09-21T00:25:46.193690+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:16:00.090070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.826590+0000","last_change":"2024-09-21T00:25:52.266721+0000","last_active":"2024-09-21T00:25:54.826590+0000","last_peered":"2024-09-21T00:25:54.826590+0000","last_clean":"2024-09-21T00:25:54.826590+0000","last_became_active":"2024-09-21T00:25:52.266271+0000","last_became_peered":"2024-09-21T00:25:52.266271+0000","last_unstale":"2024-09-21T00:25:54.826590+0000","last_undegraded":"2024-09-21T00:25:54.826590+0000","last_fullsized":"2024-09-21T00:25:54.826590+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_clean_scrub_stamp":"2024-09-21T00:25:51.238933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:15:37.112368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.826580+0000","last_change":"2024-09-21T00:25:48.239188+0000","last_active":"2024-09-21T00:25:54.826580+0000","last_peered":"2024-09-21T00:25:54.826580+0000","last_clean":"2024-09-21T00:25:54.826580+0000","last_became_active":"2024-09-21T00:25:48.238774+0000","last_became_peered":"2024-09-21T00:25:48.238774+0000","last_unstale":"2024-09-21T00:25:54.826580+0000","last_undegraded":"2024-09-21T00:25:54.826580+0000","last_fullsized":"2024-09-21T00:25:54.826580+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T00:56:56.658210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.826897+0000","last_change":"2024-09-21T00:25:47.230533+0000","last_active":"2024-09-21T00:25:54.826897+0000","last_peered":"2024-09-21T00:25:54.826897+0000","last_clean":"2024-09-21T00:25:54.826897+0000","last_became_active":"2024-09-21T00:25:47.230211+0000","last_became_peered":"2024-09-21T00:25:47.230211+0000","last_unstale":"2024-09-21T00:25:54.826897+0000","last_undegraded":"2024-09-21T00:25:54.826897+0000","last_fullsized":"2024-09-21T00:25:54.826897+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_clean_scrub_stamp":"2024-09-21T00:25:46.193690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:54:03.173672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.265006+0000","last_change":"2024-09-21T00:25:52.550405+0000","last_active":"2024-09-21T00:25:53.265006+0000","last_peered":"2024-09-21T00:25:53.265006+0000","last_clean":"2024-09-21T00:25:53.265006+0000","last_became_active":"2024-09-21T00:25:52.549743+0000","last_became_peered":"2024-09-21T00:25:52.549743+0000","last_unstale":"2024-09-21T00:25:53.265006+0000","last_undegraded":"2024-09-21T00:25:53.265006+0000","last_fullsized":"2024-09-21T00:25:53.265006+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_clean_scrub_stamp":"2024-09-21T00:25:51.238933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T00:40:38.549590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.826686+0000","last_change":"2024-09-21T00:25:48.244073+0000","last_active":"2024-09-21T00:25:54.826686+0000","last_peered":"2024-09-21T00:25:54.826686+0000","last_clean":"2024-09-21T00:25:54.826686+0000","last_became_active":"2024-09-21T00:25:48.243749+0000","last_became_peered":"2024-09-21T00:25:48.243749+0000","last_unstale":"2024-09-21T00:25:54.826686+0000","last_undegraded":"2024-09-21T00:25:54.826686+0000","last_fullsized":"2024-09-21T00:25:54.826686+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:24:47.112012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.263546+0000","last_change":"2024-09-21T00:25:47.229294+0000","last_active":"2024-09-21T00:25:53.263546+0000","last_peered":"2024-09-21T00:25:53.263546+0000","last_clean":"2024-09-21T00:25:53.263546+0000","last_became_active":"2024-09-21T00:25:47.228901+0000","last_became_peered":"2024-09-21T00:25:47.228901+0000","last_unstale":"2024-09-21T00:25:53.263546+0000","last_undegraded":"2024-09-21T00:25:53.263546+0000","last_fullsized":"2024-09-21T00:25:53.263546+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_clean_scrub_stamp":"2024-09-21T00:25:46.193690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:55:28.486627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.264577+0000","last_change":"2024-09-21T00:25:52.547298+0000","last_active":"2024-09-21T00:25:53.264577+0000","last_peered":"2024-09-21T00:25:53.264577+0000","last_clean":"2024-09-21T00:25:53.264577+0000","last_became_active":"2024-09-21T00:25:52.546647+0000","last_became_peered":"2024-09-21T00:25:52.546647+0000","last_unstale":"2024-09-21T00:25:53.264577+0000","last_undegraded":"2024-09-21T00:25:53.264577+0000","last_fullsized":"2024-09-21T00:25:53.264577+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_clean_scrub_stamp":"2024-09-21T00:25:51.238933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:10:45.091073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.826321+0000","last_change":"2024-09-21T00:25:48.237722+0000","last_active":"2024-09-21T00:25:54.826321+0000","last_peered":"2024-09-21T00:25:54.826321+0000","last_clean":"2024-09-21T00:25:54.826321+0000","last_became_active":"2024-09-21T00:25:48.237400+0000","last_became_peered":"2024-09-21T00:25:48.237400+0000","last_unstale":"2024-09-21T00:25:54.826321+0000","last_undegraded":"2024-09-21T00:25:54.826321+0000","last_fullsized":"2024-09-21T00:25:54.826321+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:00:11.346065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.826654+0000","last_change":"2024-09-21T00:25:47.225035+0000","last_active":"2024-09-21T00:25:54.826654+0000","last_peered":"2024-09-21T00:25:54.826654+0000","last_clean":"2024-09-21T00:25:54.826654+0000","last_became_active":"2024-09-21T00:25:47.224732+0000","last_became_peered":"2024-09-21T00:25:47.224732+0000","last_unstale":"2024-09-21T00:25:54.826654+0000","last_undegraded":"2024-09-21T00:25:54.826654+0000","last_fullsized":"2024-09-21T00:25:54.826654+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_clean_scrub_stamp":"2024-09-21T00:25:46.193690+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:16:00.090070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.264690+0000","last_change":"2024-09-21T00:25:52.550751+0000","last_active":"2024-09-21T00:25:53.264690+0000","last_peered":"2024-09-21T00:25:53.264690+0000","last_clean":"2024-09-21T00:25:53.264690+0000","last_became_active":"2024-09-21T00:25:52.550166+0000","last_became_peered":"2024-09-21T00:25:52.550166+0000","last_unstale":"2024-09-21T00:25:53.264690+0000","last_undegraded":"2024-09-21T00:25:53.264690+0000","last_fullsized":"2024-09-21T00:25:53.264690+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_clean_scrub_stamp":"2024-09-21T00:25:51.238933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:01:14.362816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.826761+0000","last_change":"2024-09-21T00:25:48.240297+0000","last_active":"2024-09-21T00:25:54.826761+0000","last_peered":"2024-09-21T00:25:54.826761+0000","last_clean":"2024-09-21T00:25:54.826761+0000","last_became_active":"2024-09-21T00:25:48.239975+0000","last_became_peered":"2024-09-21T00:25:48.239975+0000","last_unstale":"2024-09-21T00:25:54.826761+0000","last_undegraded":"2024-09-21T00:25:54.826761+0000","last_fullsized":"2024-09-21T00:25:54.826761+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:07:41.736005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":29,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.827734+0000","last_change":"2024-09-21T00:25:47.228804+0000","last_active":"2024-09-21T00:25:54.827734+0000","last_peered":"2024-09-21T00:25:54.827734+0000","last_clean":"2024-09-21T00:25:54.827734+0000","last_became_active":"2024-09-21T00:25:47.228490+0000","last_became_peered":"2024-09-21T00:25:47.228490+0000","last_unstale":"2024-09-21T00:25:54.827734+0000","last_undegraded":"2024-09-21T00:25:54.827734+0000","last_fullsized":"2024-09-21T00:25:54.827734+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_clean_scrub_stamp":"2024-09-21T00:25:46.193690+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:35:43.503231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.324519+0000","last_change":"2024-09-21T00:25:52.273360+0000","last_active":"2024-09-21T00:25:54.324519+0000","last_peered":"2024-09-21T00:25:54.324519+0000","last_clean":"2024-09-21T00:25:54.324519+0000","last_became_active":"2024-09-21T00:25:52.272888+0000","last_became_peered":"2024-09-21T00:25:52.272888+0000","last_unstale":"2024-09-21T00:25:54.324519+0000","last_undegraded":"2024-09-21T00:25:54.324519+0000","last_fullsized":"2024-09-21T00:25:54.324519+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_clean_scrub_stamp":"2024-09-21T00:25:51.238933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:31:19.736643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.264518+0000","last_change":"2024-09-21T00:25:48.238506+0000","last_active":"2024-09-21T00:25:53.264518+0000","last_peered":"2024-09-21T00:25:53.264518+0000","last_clean":"2024-09-21T00:25:53.264518+0000","last_became_active":"2024-09-21T00:25:48.237355+0000","last_became_peered":"2024-09-21T00:25:48.237355+0000","last_unstale":"2024-09-21T00:25:53.264518+0000","last_undegraded":"2024-09-21T00:25:53.264518+0000","last_fullsized":"2024-09-21T00:25:53.264518+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:40:11.560626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.324726+0000","last_change":"2024-09-21T00:25:47.224421+0000","last_active":"2024-09-21T00:25:54.324726+0000","last_peered":"2024-09-21T00:25:54.324726+0000","last_clean":"2024-09-21T00:25:54.324726+0000","last_became_active":"2024-09-21T00:25:47.224096+0000","last_became_peered":"2024-09-21T00:25:47.224096+0000","last_unstale":"2024-09-21T00:25:54.324726+0000","last_undegraded":"2024-09-21T00:25:54.324726+0000","last_fullsized":"2024-09-21T00:25:54.324726+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_clean_scrub_stamp":"2024-09-21T00:25:46.193690+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:43:08.714258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.263548+0000","last_change":"2024-09-21T00:25:52.277635+0000","last_active":"2024-09-21T00:25:53.263548+0000","last_peered":"2024-09-21T00:25:53.263548+0000","last_clean":"2024-09-21T00:25:53.263548+0000","last_became_active":"2024-09-21T00:25:52.276739+0000","last_became_peered":"2024-09-21T00:25:52.276739+0000","last_unstale":"2024-09-21T00:25:53.263548+0000","last_undegraded":"2024-09-21T00:25:53.263548+0000","last_fullsized":"2024-09-21T00:25:53.263548+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_clean_scrub_stamp":"2024-09-21T00:25:51.238933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:46:09.670107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.826816+0000","last_change":"2024-09-21T00:25:48.233598+0000","last_active":"2024-09-21T00:25:54.826816+0000","last_peered":"2024-09-21T00:25:54.826816+0000","last_clean":"2024-09-21T00:25:54.826816+0000","last_became_active":"2024-09-21T00:25:48.233203+0000","last_became_peered":"2024-09-21T00:25:48.233203+0000","last_unstale":"2024-09-21T00:25:54.826816+0000","last_undegraded":"2024-09-21T00:25:54.826816+0000","last_fullsized":"2024-09-21T00:25:54.826816+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:36:54.554284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.263639+0000","last_change":"2024-09-21T00:25:47.225271+0000","last_active":"2024-09-21T00:25:53.263639+0000","last_peered":"2024-09-21T00:25:53.263639+0000","last_clean":"2024-09-21T00:25:53.263639+0000","last_became_active":"2024-09-21T00:25:47.224935+0000","last_became_peered":"2024-09-21T00:25:47.224935+0000","last_unstale":"2024-09-21T00:25:53.263639+0000","last_undegraded":"2024-09-21T00:25:53.263639+0000","last_fullsized":"2024-09-21T00:25:53.263639+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_clean_scrub_stamp":"2024-09-21T00:25:46.193690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:06:59.889895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.264083+0000","last_change":"2024-09-21T00:25:52.278653+0000","last_active":"2024-09-21T00:25:53.264083+0000","last_peered":"2024-09-21T00:25:53.264083+0000","last_clean":"2024-09-21T00:25:53.264083+0000","last_became_active":"2024-09-21T00:25:52.278111+0000","last_became_peered":"2024-09-21T00:25:52.278111+0000","last_unstale":"2024-09-21T00:25:53.264083+0000","last_undegraded":"2024-09-21T00:25:53.264083+0000","last_fullsized":"2024-09-21T00:25:53.264083+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_clean_scrub_stamp":"2024-09-21T00:25:51.238933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:01:24.675942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.263506+0000","last_change":"2024-09-21T00:25:48.239666+0000","last_active":"2024-09-21T00:25:53.263506+0000","last_peered":"2024-09-21T00:25:53.263506+0000","last_clean":"2024-09-21T00:25:53.263506+0000","last_became_active":"2024-09-21T00:25:48.239196+0000","last_became_peered":"2024-09-21T00:25:48.239196+0000","last_unstale":"2024-09-21T00:25:53.263506+0000","last_undegraded":"2024-09-21T00:25:53.263506+0000","last_fullsized":"2024-09-21T00:25:53.263506+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:00:19.889965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.827810+0000","last_change":"2024-09-21T00:25:47.225244+0000","last_active":"2024-09-21T00:25:54.827810+0000","last_peered":"2024-09-21T00:25:54.827810+0000","last_clean":"2024-09-21T00:25:54.827810+0000","last_became_active":"2024-09-21T00:25:47.224812+0000","last_became_peered":"2024-09-21T00:25:47.224812+0000","last_unstale":"2024-09-21T00:25:54.827810+0000","last_undegraded":"2024-09-21T00:25:54.827810+0000","last_fullsized":"2024-09-21T00:25:54.827810+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_clean_scrub_stamp":"2024-09-21T00:25:46.193690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:06:39.901371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.264361+0000","last_change":"2024-09-21T00:25:52.550130+0000","last_active":"2024-09-21T00:25:53.264361+0000","last_peered":"2024-09-21T00:25:53.264361+0000","last_clean":"2024-09-21T00:25:53.264361+0000","last_became_active":"2024-09-21T00:25:52.549675+0000","last_became_peered":"2024-09-21T00:25:52.549675+0000","last_unstale":"2024-09-21T00:25:53.264361+0000","last_undegraded":"2024-09-21T00:25:53.264361+0000","last_fullsized":"2024-09-21T00:25:53.264361+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_clean_scrub_stamp":"2024-09-21T00:25:51.238933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:55:10.499479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.826619+0000","last_change":"2024-09-21T00:25:52.550630+0000","last_active":"2024-09-21T00:25:54.826619+0000","last_peered":"2024-09-21T00:25:54.826619+0000","last_clean":"2024-09-21T00:25:54.826619+0000","last_became_active":"2024-09-21T00:25:52.550098+0000","last_became_peered":"2024-09-21T00:25:52.550098+0000","last_unstale":"2024-09-21T00:25:54.826619+0000","last_undegraded":"2024-09-21T00:25:54.826619+0000","last_fullsized":"2024-09-21T00:25:54.826619+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_clean_scrub_stamp":"2024-09-21T00:25:51.238933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:43:32.503216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.324003+0000","last_change":"2024-09-21T00:25:48.227526+0000","last_active":"2024-09-21T00:25:54.324003+0000","last_peered":"2024-09-21T00:25:54.324003+0000","last_clean":"2024-09-21T00:25:54.324003+0000","last_became_active":"2024-09-21T00:25:48.227194+0000","last_became_peered":"2024-09-21T00:25:48.227194+0000","last_unstale":"2024-09-21T00:25:54.324003+0000","last_undegraded":"2024-09-21T00:25:54.324003+0000","last_fullsized":"2024-09-21T00:25:54.324003+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:54:09.449880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.262751+0000","last_change":"2024-09-21T00:25:47.220721+0000","last_active":"2024-09-21T00:25:53.262751+0000","last_peered":"2024-09-21T00:25:53.262751+0000","last_clean":"2024-09-21T00:25:53.262751+0000","last_became_active":"2024-09-21T00:25:47.220393+0000","last_became_peered":"2024-09-21T00:25:47.220393+0000","last_unstale":"2024-09-21T00:25:53.262751+0000","last_undegraded":"2024-09-21T00:25:53.262751+0000","last_fullsized":"2024-09-21T00:25:53.262751+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_clean_scrub_stamp":"2024-09-21T00:25:46.193690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:00:04.423924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.827438+0000","last_change":"2024-09-21T00:25:52.280179+0000","last_active":"2024-09-21T00:25:54.827438+0000","last_peered":"2024-09-21T00:25:54.827438+0000","last_clean":"2024-09-21T00:25:54.827438+0000","last_became_active":"2024-09-21T00:25:52.279617+0000","last_became_peered":"2024-09-21T00:25:52.279617+0000","last_unstale":"2024-09-21T00:25:54.827438+0000","last_undegraded":"2024-09-21T00:25:54.827438+0000","last_fullsized":"2024-09-21T00:25:54.827438+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_clean_scrub_stamp":"2024-09-21T00:25:51.238933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:30:11.179850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.264234+0000","last_change":"2024-09-21T00:25:48.237852+0000","last_active":"2024-09-21T00:25:53.264234+0000","last_peered":"2024-09-21T00:25:53.264234+0000","last_clean":"2024-09-21T00:25:53.264234+0000","last_became_active":"2024-09-21T00:25:48.237559+0000","last_became_peered":"2024-09-21T00:25:48.237559+0000","last_unstale":"2024-09-21T00:25:53.264234+0000","last_undegraded":"2024-09-21T00:25:53.264234+0000","last_fullsized":"2024-09-21T00:25:53.264234+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:28:02.233472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.264305+0000","last_change":"2024-09-21T00:25:47.227879+0000","last_active":"2024-09-21T00:25:53.264305+0000","last_peered":"2024-09-21T00:25:53.264305+0000","last_clean":"2024-09-21T00:25:53.264305+0000","last_became_active":"2024-09-21T00:25:47.227569+0000","last_became_peered":"2024-09-21T00:25:47.227569+0000","last_unstale":"2024-09-21T00:25:53.264305+0000","last_undegraded":"2024-09-21T00:25:53.264305+0000","last_fullsized":"2024-09-21T00:25:53.264305+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_clean_scrub_stamp":"2024-09-21T00:25:46.193690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:16:00.090070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.827281+0000","last_change":"2024-09-21T00:25:47.231506+0000","last_active":"2024-09-21T00:25:54.827281+0000","last_peered":"2024-09-21T00:25:54.827281+0000","last_clean":"2024-09-21T00:25:54.827281+0000","last_became_active":"2024-09-21T00:25:47.231135+0000","last_became_peered":"2024-09-21T00:25:47.231135+0000","last_unstale":"2024-09-21T00:25:54.827281+0000","last_undegraded":"2024-09-21T00:25:54.827281+0000","last_fullsized":"2024-09-21T00:25:54.827281+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_clean_scrub_stamp":"2024-09-21T00:25:46.193690+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:52:45.409090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.826519+0000","last_change":"2024-09-21T00:25:48.245503+0000","last_active":"2024-09-21T00:25:54.826519+0000","last_peered":"2024-09-21T00:25:54.826519+0000","last_clean":"2024-09-21T00:25:54.826519+0000","last_became_active":"2024-09-21T00:25:48.245168+0000","last_became_peered":"2024-09-21T00:25:48.245168+0000","last_unstale":"2024-09-21T00:25:54.826519+0000","last_undegraded":"2024-09-21T00:25:54.826519+0000","last_fullsized":"2024-09-21T00:25:54.826519+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:51:35.872806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.242222+0000","last_change":"2024-09-21T00:25:52.278289+0000","last_active":"2024-09-21T00:25:54.242222+0000","last_peered":"2024-09-21T00:25:54.242222+0000","last_clean":"2024-09-21T00:25:54.242222+0000","last_became_active":"2024-09-21T00:25:52.277874+0000","last_became_peered":"2024-09-21T00:25:52.277874+0000","last_unstale":"2024-09-21T00:25:54.242222+0000","last_undegraded":"2024-09-21T00:25:54.242222+0000","last_fullsized":"2024-09-21T00:25:54.242222+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_clean_scrub_stamp":"2024-09-21T00:25:51.238933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T00:47:59.166653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.264404+0000","last_change":"2024-09-21T00:25:47.232974+0000","last_active":"2024-09-21T00:25:53.264404+0000","last_peered":"2024-09-21T00:25:53.264404+0000","last_clean":"2024-09-21T00:25:53.264404+0000","last_became_active":"2024-09-21T00:25:47.232605+0000","last_became_peered":"2024-09-21T00:25:47.232605+0000","last_unstale":"2024-09-21T00:25:53.264404+0000","last_undegraded":"2024-09-21T00:25:53.264404+0000","last_fullsized":"2024-09-21T00:25:53.264404+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_clean_scrub_stamp":"2024-09-21T00:25:46.193690+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:53:37.081118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.263168+0000","last_change":"2024-09-21T00:25:48.223460+0000","last_active":"2024-09-21T00:25:53.263168+0000","last_peered":"2024-09-21T00:25:53.263168+0000","last_clean":"2024-09-21T00:25:53.263168+0000","last_became_active":"2024-09-21T00:25:48.222174+0000","last_became_peered":"2024-09-21T00:25:48.222174+0000","last_unstale":"2024-09-21T00:25:53.263168+0000","last_undegraded":"2024-09-21T00:25:53.263168+0000","last_fullsized":"2024-09-21T00:25:53.263168+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:07:00.897304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.264073+0000","last_change":"2024-09-21T00:25:52.277698+0000","last_active":"2024-09-21T00:25:53.264073+0000","last_peered":"2024-09-21T00:25:53.264073+0000","last_clean":"2024-09-21T00:25:53.264073+0000","last_became_active":"2024-09-21T00:25:52.277232+0000","last_became_peered":"2024-09-21T00:25:52.277232+0000","last_unstale":"2024-09-21T00:25:53.264073+0000","last_undegraded":"2024-09-21T00:25:53.264073+0000","last_fullsized":"2024-09-21T00:25:53.264073+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_clean_scrub_stamp":"2024-09-21T00:25:51.238933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:01:14.362816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.826302+0000","last_change":"2024-09-21T00:25:48.240360+0000","last_active":"2024-09-21T00:25:54.826302+0000","last_peered":"2024-09-21T00:25:54.826302+0000","last_clean":"2024-09-21T00:25:54.826302+0000","last_became_active":"2024-09-21T00:25:48.240038+0000","last_became_peered":"2024-09-21T00:25:48.240038+0000","last_unstale":"2024-09-21T00:25:54.826302+0000","last_undegraded":"2024-09-21T00:25:54.826302+0000","last_fullsized":"2024-09-21T00:25:54.826302+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:18:14.964429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.242242+0000","last_change":"2024-09-21T00:25:48.243928+0000","last_active":"2024-09-21T00:25:54.242242+0000","last_peered":"2024-09-21T00:25:54.242242+0000","last_clean":"2024-09-21T00:25:54.242242+0000","last_became_active":"2024-09-21T00:25:48.243641+0000","last_became_peered":"2024-09-21T00:25:48.243641+0000","last_unstale":"2024-09-21T00:25:54.242242+0000","last_undegraded":"2024-09-21T00:25:54.242242+0000","last_fullsized":"2024-09-21T00:25:54.242242+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:01:32.510097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.263436+0000","last_change":"2024-09-21T00:25:48.229253+0000","last_active":"2024-09-21T00:25:53.263436+0000","last_peered":"2024-09-21T00:25:53.263436+0000","last_clean":"2024-09-21T00:25:53.263436+0000","last_became_active":"2024-09-21T00:25:48.228880+0000","last_became_peered":"2024-09-21T00:25:48.228880+0000","last_unstale":"2024-09-21T00:25:53.263436+0000","last_undegraded":"2024-09-21T00:25:53.263436+0000","last_fullsized":"2024-09-21T00:25:53.263436+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:24:47.112012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.826434+0000","last_change":"2024-09-21T00:25:48.232024+0000","last_active":"2024-09-21T00:25:54.826434+0000","last_peered":"2024-09-21T00:25:54.826434+0000","last_clean":"2024-09-21T00:25:54.826434+0000","last_became_active":"2024-09-21T00:25:48.231506+0000","last_became_peered":"2024-09-21T00:25:48.231506+0000","last_unstale":"2024-09-21T00:25:54.826434+0000","last_undegraded":"2024-09-21T00:25:54.826434+0000","last_fullsized":"2024-09-21T00:25:54.826434+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:02:20.463726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.264854+0000","last_change":"2024-09-21T00:25:48.242030+0000","last_active":"2024-09-21T00:25:53.264854+0000","last_peered":"2024-09-21T00:25:53.264854+0000","last_clean":"2024-09-21T00:25:53.264854+0000","last_became_active":"2024-09-21T00:25:48.241710+0000","last_became_peered":"2024-09-21T00:25:48.241710+0000","last_unstale":"2024-09-21T00:25:53.264854+0000","last_undegraded":"2024-09-21T00:25:53.264854+0000","last_fullsized":"2024-09-21T00:25:53.264854+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:00:19.889965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.324542+0000","last_change":"2024-09-21T00:25:48.230301+0000","last_active":"2024-09-21T00:25:54.324542+0000","last_peered":"2024-09-21T00:25:54.324542+0000","last_clean":"2024-09-21T00:25:54.324542+0000","last_became_active":"2024-09-21T00:25:48.229759+0000","last_became_peered":"2024-09-21T00:25:48.229759+0000","last_unstale":"2024-09-21T00:25:54.324542+0000","last_undegraded":"2024-09-21T00:25:54.324542+0000","last_fullsized":"2024-09-21T00:25:54.324542+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:23:28.920719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.827515+0000","last_change":"2024-09-21T00:25:48.236977+0000","last_active":"2024-09-21T00:25:54.827515+0000","last_peered":"2024-09-21T00:25:54.827515+0000","last_clean":"2024-09-21T00:25:54.827515+0000","last_became_active":"2024-09-21T00:25:48.236636+0000","last_became_peered":"2024-09-21T00:25:48.236636+0000","last_unstale":"2024-09-21T00:25:54.827515+0000","last_undegraded":"2024-09-21T00:25:54.827515+0000","last_fullsized":"2024-09-21T00:25:54.827515+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:34:36.110489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.264907+0000","last_change":"2024-09-21T00:25:48.242792+0000","last_active":"2024-09-21T00:25:53.264907+0000","last_peered":"2024-09-21T00:25:53.264907+0000","last_clean":"2024-09-21T00:25:53.264907+0000","last_became_active":"2024-09-21T00:25:48.242512+0000","last_became_peered":"2024-09-21T00:25:48.242512+0000","last_unstale":"2024-09-21T00:25:53.264907+0000","last_undegraded":"2024-09-21T00:25:53.264907+0000","last_fullsized":"2024-09-21T00:25:53.264907+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:58:22.324727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.262616+0000","last_change":"2024-09-21T00:25:48.229627+0000","last_active":"2024-09-21T00:25:53.262616+0000","last_peered":"2024-09-21T00:25:53.262616+0000","last_clean":"2024-09-21T00:25:53.262616+0000","last_became_active":"2024-09-21T00:25:48.229297+0000","last_became_peered":"2024-09-21T00:25:48.229297+0000","last_unstale":"2024-09-21T00:25:53.262616+0000","last_undegraded":"2024-09-21T00:25:53.262616+0000","last_fullsized":"2024-09-21T00:25:53.262616+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:58:22.324727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.264187+0000","last_change":"2024-09-21T00:25:48.243911+0000","last_active":"2024-09-21T00:25:53.264187+0000","last_peered":"2024-09-21T00:25:53.264187+0000","last_clean":"2024-09-21T00:25:53.264187+0000","last_became_active":"2024-09-21T00:25:48.243581+0000","last_became_peered":"2024-09-21T00:25:48.243581+0000","last_unstale":"2024-09-21T00:25:53.264187+0000","last_undegraded":"2024-09-21T00:25:53.264187+0000","last_fullsized":"2024-09-21T00:25:53.264187+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:42:57.617812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.263316+0000","last_change":"2024-09-21T00:25:48.238597+0000","last_active":"2024-09-21T00:25:53.263316+0000","last_peered":"2024-09-21T00:25:53.263316+0000","last_clean":"2024-09-21T00:25:53.263316+0000","last_became_active":"2024-09-21T00:25:48.238265+0000","last_became_peered":"2024-09-21T00:25:48.238265+0000","last_unstale":"2024-09-21T00:25:53.263316+0000","last_undegraded":"2024-09-21T00:25:53.263316+0000","last_fullsized":"2024-09-21T00:25:53.263316+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:10:01.543628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.826223+0000","last_change":"2024-09-21T00:25:48.237837+0000","last_active":"2024-09-21T00:25:54.826223+0000","last_peered":"2024-09-21T00:25:54.826223+0000","last_clean":"2024-09-21T00:25:54.826223+0000","last_became_active":"2024-09-21T00:25:48.237502+0000","last_became_peered":"2024-09-21T00:25:48.237502+0000","last_unstale":"2024-09-21T00:25:54.826223+0000","last_undegraded":"2024-09-21T00:25:54.826223+0000","last_fullsized":"2024-09-21T00:25:54.826223+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:52:28.861327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.827511+0000","last_change":"2024-09-21T00:25:48.241417+0000","last_active":"2024-09-21T00:25:54.827511+0000","last_peered":"2024-09-21T00:25:54.827511+0000","last_clean":"2024-09-21T00:25:54.827511+0000","last_became_active":"2024-09-21T00:25:48.241107+0000","last_became_peered":"2024-09-21T00:25:48.241107+0000","last_unstale":"2024-09-21T00:25:54.827511+0000","last_undegraded":"2024-09-21T00:25:54.827511+0000","last_fullsized":"2024-09-21T00:25:54.827511+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:52:33.265365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.827391+0000","last_change":"2024-09-21T00:25:48.246529+0000","last_active":"2024-09-21T00:25:54.827391+0000","last_peered":"2024-09-21T00:25:54.827391+0000","last_clean":"2024-09-21T00:25:54.827391+0000","last_became_active":"2024-09-21T00:25:48.245952+0000","last_became_peered":"2024-09-21T00:25:48.245952+0000","last_unstale":"2024-09-21T00:25:54.827391+0000","last_undegraded":"2024-09-21T00:25:54.827391+0000","last_fullsized":"2024-09-21T00:25:54.827391+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:16:07.250921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.242392+0000","last_change":"2024-09-21T00:25:48.235870+0000","last_active":"2024-09-21T00:25:54.242392+0000","last_peered":"2024-09-21T00:25:54.242392+0000","last_clean":"2024-09-21T00:25:54.242392+0000","last_became_active":"2024-09-21T00:25:48.235552+0000","last_became_peered":"2024-09-21T00:25:48.235552+0000","last_unstale":"2024-09-21T00:25:54.242392+0000","last_undegraded":"2024-09-21T00:25:54.242392+0000","last_fullsized":"2024-09-21T00:25:54.242392+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:03:13.400804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.264193+0000","last_change":"2024-09-21T00:25:48.238458+0000","last_active":"2024-09-21T00:25:53.264193+0000","last_peered":"2024-09-21T00:25:53.264193+0000","last_clean":"2024-09-21T00:25:53.264193+0000","last_became_active":"2024-09-21T00:25:48.238154+0000","last_became_peered":"2024-09-21T00:25:48.238154+0000","last_unstale":"2024-09-21T00:25:53.264193+0000","last_undegraded":"2024-09-21T00:25:53.264193+0000","last_fullsized":"2024-09-21T00:25:53.264193+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:51:31.831106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.262441+0000","last_change":"2024-09-21T00:25:48.232187+0000","last_active":"2024-09-21T00:25:53.262441+0000","last_peered":"2024-09-21T00:25:53.262441+0000","last_clean":"2024-09-21T00:25:53.262441+0000","last_became_active":"2024-09-21T00:25:48.230817+0000","last_became_peered":"2024-09-21T00:25:48.230817+0000","last_unstale":"2024-09-21T00:25:53.262441+0000","last_undegraded":"2024-09-21T00:25:53.262441+0000","last_fullsized":"2024-09-21T00:25:53.262441+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:32:32.609174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.324087+0000","last_change":"2024-09-21T00:25:48.220646+0000","last_active":"2024-09-21T00:25:54.324087+0000","last_peered":"2024-09-21T00:25:54.324087+0000","last_clean":"2024-09-21T00:25:54.324087+0000","last_became_active":"2024-09-21T00:25:48.220325+0000","last_became_peered":"2024-09-21T00:25:48.220325+0000","last_unstale":"2024-09-21T00:25:54.324087+0000","last_undegraded":"2024-09-21T00:25:54.324087+0000","last_fullsized":"2024-09-21T00:25:54.324087+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:52:34.711918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.826294+0000","last_change":"2024-09-21T00:25:48.228802+0000","last_active":"2024-09-21T00:25:54.826294+0000","last_peered":"2024-09-21T00:25:54.826294+0000","last_clean":"2024-09-21T00:25:54.826294+0000","last_became_active":"2024-09-21T00:25:48.228464+0000","last_became_peered":"2024-09-21T00:25:48.228464+0000","last_unstale":"2024-09-21T00:25:54.826294+0000","last_undegraded":"2024-09-21T00:25:54.826294+0000","last_fullsized":"2024-09-21T00:25:54.826294+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:50:09.155321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.264138+0000","last_change":"2024-09-21T00:25:48.243998+0000","last_active":"2024-09-21T00:25:53.264138+0000","last_peered":"2024-09-21T00:25:53.264138+0000","last_clean":"2024-09-21T00:25:53.264138+0000","last_became_active":"2024-09-21T00:25:48.242668+0000","last_became_peered":"2024-09-21T00:25:48.242668+0000","last_unstale":"2024-09-21T00:25:53.264138+0000","last_undegraded":"2024-09-21T00:25:53.264138+0000","last_fullsized":"2024-09-21T00:25:53.264138+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:44:10.293149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.264479+0000","last_change":"2024-09-21T00:25:48.230464+0000","last_active":"2024-09-21T00:25:53.264479+0000","last_peered":"2024-09-21T00:25:53.264479+0000","last_clean":"2024-09-21T00:25:53.264479+0000","last_became_active":"2024-09-21T00:25:48.229463+0000","last_became_peered":"2024-09-21T00:25:48.229463+0000","last_unstale":"2024-09-21T00:25:53.264479+0000","last_undegraded":"2024-09-21T00:25:53.264479+0000","last_fullsized":"2024-09-21T00:25:53.264479+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:24:05.914453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.827241+0000","last_change":"2024-09-21T00:25:48.233368+0000","last_active":"2024-09-21T00:25:54.827241+0000","last_peered":"2024-09-21T00:25:54.827241+0000","last_clean":"2024-09-21T00:25:54.827241+0000","last_became_active":"2024-09-21T00:25:48.233049+0000","last_became_peered":"2024-09-21T00:25:48.233049+0000","last_unstale":"2024-09-21T00:25:54.827241+0000","last_undegraded":"2024-09-21T00:25:54.827241+0000","last_fullsized":"2024-09-21T00:25:54.827241+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:52:34.711918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.827398+0000","last_change":"2024-09-21T00:25:48.240634+0000","last_active":"2024-09-21T00:25:54.827398+0000","last_peered":"2024-09-21T00:25:54.827398+0000","last_clean":"2024-09-21T00:25:54.827398+0000","last_became_active":"2024-09-21T00:25:48.240302+0000","last_became_peered":"2024-09-21T00:25:48.240302+0000","last_unstale":"2024-09-21T00:25:54.827398+0000","last_undegraded":"2024-09-21T00:25:54.827398+0000","last_fullsized":"2024-09-21T00:25:54.827398+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:30:32.308490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.827582+0000","last_change":"2024-09-21T00:25:48.243347+0000","last_active":"2024-09-21T00:25:54.827582+0000","last_peered":"2024-09-21T00:25:54.827582+0000","last_clean":"2024-09-21T00:25:54.827582+0000","last_became_active":"2024-09-21T00:25:48.243035+0000","last_became_peered":"2024-09-21T00:25:48.243035+0000","last_unstale":"2024-09-21T00:25:54.827582+0000","last_undegraded":"2024-09-21T00:25:54.827582+0000","last_fullsized":"2024-09-21T00:25:54.827582+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:10:10.125591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.263815+0000","last_change":"2024-09-21T00:25:48.235929+0000","last_active":"2024-09-21T00:25:53.263815+0000","last_peered":"2024-09-21T00:25:53.263815+0000","last_clean":"2024-09-21T00:25:53.263815+0000","last_became_active":"2024-09-21T00:25:48.235620+0000","last_became_peered":"2024-09-21T00:25:48.235620+0000","last_unstale":"2024-09-21T00:25:53.263815+0000","last_undegraded":"2024-09-21T00:25:53.263815+0000","last_fullsized":"2024-09-21T00:25:53.263815+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:53:08.415542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.826376+0000","last_change":"2024-09-21T00:25:48.247624+0000","last_active":"2024-09-21T00:25:54.826376+0000","last_peered":"2024-09-21T00:25:54.826376+0000","last_clean":"2024-09-21T00:25:54.826376+0000","last_became_active":"2024-09-21T00:25:48.247318+0000","last_became_peered":"2024-09-21T00:25:48.247318+0000","last_unstale":"2024-09-21T00:25:54.826376+0000","last_undegraded":"2024-09-21T00:25:54.826376+0000","last_fullsized":"2024-09-21T00:25:54.826376+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:34:07.237786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.263196+0000","last_change":"2024-09-21T00:25:48.232468+0000","last_active":"2024-09-21T00:25:53.263196+0000","last_peered":"2024-09-21T00:25:53.263196+0000","last_clean":"2024-09-21T00:25:53.263196+0000","last_became_active":"2024-09-21T00:25:48.231259+0000","last_became_peered":"2024-09-21T00:25:48.231259+0000","last_unstale":"2024-09-21T00:25:53.263196+0000","last_undegraded":"2024-09-21T00:25:53.263196+0000","last_fullsized":"2024-09-21T00:25:53.263196+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:07:41.736005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.826608+0000","last_change":"2024-09-21T00:25:48.246495+0000","last_active":"2024-09-21T00:25:54.826608+0000","last_peered":"2024-09-21T00:25:54.826608+0000","last_clean":"2024-09-21T00:25:54.826608+0000","last_became_active":"2024-09-21T00:25:48.245978+0000","last_became_peered":"2024-09-21T00:25:48.245978+0000","last_unstale":"2024-09-21T00:25:54.826608+0000","last_undegraded":"2024-09-21T00:25:54.826608+0000","last_fullsized":"2024-09-21T00:25:54.826608+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:48:34.879434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.263683+0000","last_change":"2024-09-21T00:25:48.237710+0000","last_active":"2024-09-21T00:25:53.263683+0000","last_peered":"2024-09-21T00:25:53.263683+0000","last_clean":"2024-09-21T00:25:53.263683+0000","last_became_active":"2024-09-21T00:25:48.232344+0000","last_became_peered":"2024-09-21T00:25:48.232344+0000","last_unstale":"2024-09-21T00:25:53.263683+0000","last_undegraded":"2024-09-21T00:25:53.263683+0000","last_fullsized":"2024-09-21T00:25:53.263683+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:48:34.879434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.826561+0000","last_change":"2024-09-21T00:25:48.240393+0000","last_active":"2024-09-21T00:25:54.826561+0000","last_peered":"2024-09-21T00:25:54.826561+0000","last_clean":"2024-09-21T00:25:54.826561+0000","last_became_active":"2024-09-21T00:25:48.240086+0000","last_became_peered":"2024-09-21T00:25:48.240086+0000","last_unstale":"2024-09-21T00:25:54.826561+0000","last_undegraded":"2024-09-21T00:25:54.826561+0000","last_fullsized":"2024-09-21T00:25:54.826561+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:12:39.910729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.324667+0000","last_change":"2024-09-21T00:25:48.230210+0000","last_active":"2024-09-21T00:25:54.324667+0000","last_peered":"2024-09-21T00:25:54.324667+0000","last_clean":"2024-09-21T00:25:54.324667+0000","last_became_active":"2024-09-21T00:25:48.229837+0000","last_became_peered":"2024-09-21T00:25:48.229837+0000","last_unstale":"2024-09-21T00:25:54.324667+0000","last_undegraded":"2024-09-21T00:25:54.324667+0000","last_fullsized":"2024-09-21T00:25:54.324667+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:23:42.611364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.827462+0000","last_change":"2024-09-21T00:25:48.242566+0000","last_active":"2024-09-21T00:25:54.827462+0000","last_peered":"2024-09-21T00:25:54.827462+0000","last_clean":"2024-09-21T00:25:54.827462+0000","last_became_active":"2024-09-21T00:25:48.242265+0000","last_became_peered":"2024-09-21T00:25:48.242265+0000","last_unstale":"2024-09-21T00:25:54.827462+0000","last_undegraded":"2024-09-21T00:25:54.827462+0000","last_fullsized":"2024-09-21T00:25:54.827462+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:49:57.219481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509446,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27880,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343960,"statfs":{"total":96636764160,"available":96608215040,"internally_reserved":0,"allocated":868352,"data_stored":697130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509447,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":282624,"data_stored":107792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509446,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":274432,"data_stored":106270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509445,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":282624,"data_stored":106306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509447,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":282624,"data_stored":107792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509447,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":270336,"data_stored":106180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509447,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27888,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343952,"statfs":{"total":96636764160,"available":96608206848,"internally_reserved":0,"allocated":876544,"data_stored":697186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":278528,"data_stored":106340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T00:26:02.673 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T00:26:02.674 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config log 1 --format=json 2024-09-21T00:26:02.674 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-21T00:26:02.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.820+0000 7fc2be430640 1 -- 172.21.15.112:0/2364918772 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2b806def0 msgr2=0x7fc2b806f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:02.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.820+0000 7fc2be430640 1 --2- 172.21.15.112:0/2364918772 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2b806def0 0x7fc2b806f410 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc2b8075150 tx=0x7fc2a802a5f0 comp rx=0 tx=0).stop 2024-09-21T00:26:02.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.820+0000 7fc2be430640 1 -- 172.21.15.112:0/2364918772 shutdown_connections 2024-09-21T00:26:02.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.820+0000 7fc2be430640 1 --2- 172.21.15.112:0/2364918772 >> v2:172.21.15.187:3300/0 conn(0x7fc2b80b5260 0x7fc2b815ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:02.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.820+0000 7fc2be430640 1 --2- 172.21.15.112:0/2364918772 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2b806def0 0x7fc2b806f410 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:02.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.820+0000 7fc2be430640 1 -- 172.21.15.112:0/2364918772 >> 172.21.15.112:0/2364918772 conn(0x7fc2b80cdca0 msgr2=0x7fc2b80ce0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:02.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.820+0000 7fc2be430640 1 -- 172.21.15.112:0/2364918772 shutdown_connections 2024-09-21T00:26:02.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.820+0000 7fc2be430640 1 -- 172.21.15.112:0/2364918772 wait complete. 2024-09-21T00:26:02.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.822+0000 7fc2be430640 1 Processor -- start 2024-09-21T00:26:02.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.822+0000 7fc2be430640 1 -- start start 2024-09-21T00:26:02.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.823+0000 7fc2be430640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc2b806def0 0x7fc2b806cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:02.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.823+0000 7fc2be430640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2b80b5260 0x7fc2b8154e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:02.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.823+0000 7fc2be430640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc2b815bfc0 0x7fc2b8155350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:02.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.823+0000 7fc2be430640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc2b80b5a00 con 0x7fc2b80b5260 2024-09-21T00:26:02.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.823+0000 7fc2be430640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc2b80b5880 con 0x7fc2b806def0 2024-09-21T00:26:02.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.823+0000 7fc2be430640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc2b8071300 con 0x7fc2b815bfc0 2024-09-21T00:26:02.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.823+0000 7fc2b77fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2b80b5260 0x7fc2b8154e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:02.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.823+0000 7fc2b77fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2b80b5260 0x7fc2b8154e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45636/0 (socket says 172.21.15.112:45636) 2024-09-21T00:26:02.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.824+0000 7fc2b77fe640 1 -- 172.21.15.112:0/891906562 learned_addr learned my addr 172.21.15.112:0/891906562 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:26:02.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.824+0000 7fc2b7fff640 1 --2- 172.21.15.112:0/891906562 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc2b806def0 0x7fc2b806cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:02.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.824+0000 7fc2bc9cd640 1 --2- 172.21.15.112:0/891906562 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc2b815bfc0 0x7fc2b8155350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:02.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.824+0000 7fc2b77fe640 1 -- 172.21.15.112:0/891906562 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc2b815bfc0 msgr2=0x7fc2b8155350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:02.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.824+0000 7fc2b77fe640 1 --2- 172.21.15.112:0/891906562 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc2b815bfc0 0x7fc2b8155350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:02.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.824+0000 7fc2b77fe640 1 -- 172.21.15.112:0/891906562 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc2b806def0 msgr2=0x7fc2b806cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:02.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.824+0000 7fc2b77fe640 1 --2- 172.21.15.112:0/891906562 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc2b806def0 0x7fc2b806cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:02.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.824+0000 7fc2b77fe640 1 -- 172.21.15.112:0/891906562 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2b809dd60 con 0x7fc2b80b5260 2024-09-21T00:26:02.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.824+0000 7fc2b77fe640 1 --2- 172.21.15.112:0/891906562 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2b80b5260 0x7fc2b8154e10 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac00d6d0 tx=0x7fc2ac00db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:02.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.824+0000 7fc2b4ff9640 1 -- 172.21.15.112:0/891906562 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2ac017070 con 0x7fc2b80b5260 2024-09-21T00:26:02.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.824+0000 7fc2b4ff9640 1 -- 172.21.15.112:0/891906562 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc2ac009b10 con 0x7fc2b80b5260 2024-09-21T00:26:02.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.824+0000 7fc2b4ff9640 1 -- 172.21.15.112:0/891906562 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2ac010460 con 0x7fc2b80b5260 2024-09-21T00:26:02.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.824+0000 7fc2be430640 1 -- 172.21.15.112:0/891906562 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2b80d0010 con 0x7fc2b80b5260 2024-09-21T00:26:02.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.824+0000 7fc2be430640 1 -- 172.21.15.112:0/891906562 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2b80d0590 con 0x7fc2b80b5260 2024-09-21T00:26:02.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.825+0000 7fc2be430640 1 -- 172.21.15.112:0/891906562 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2b806f410 con 0x7fc2b80b5260 2024-09-21T00:26:02.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.826+0000 7fc2b4ff9640 1 -- 172.21.15.112:0/891906562 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc2ac01a070 con 0x7fc2b80b5260 2024-09-21T00:26:02.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.826+0000 7fc2b4ff9640 1 --2- 172.21.15.112:0/891906562 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fc2a4078b90 0x7fc2a407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:02.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.826+0000 7fc2b4ff9640 1 -- 172.21.15.112:0/891906562 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fc2ac09b1b0 con 0x7fc2b80b5260 2024-09-21T00:26:02.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.826+0000 7fc2b7fff640 1 --2- 172.21.15.112:0/891906562 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fc2a4078b90 0x7fc2a407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:02.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.827+0000 7fc2b7fff640 1 --2- 172.21.15.112:0/891906562 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fc2a4078b90 0x7fc2a407b0a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc2a8033000 tx=0x7fc2a8004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:02.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:02.831+0000 7fc2b4ff9640 1 -- 172.21.15.112:0/891906562 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc2ac062c60 con 0x7fc2b80b5260 2024-09-21T00:26:03.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.007+0000 7fc2be430640 1 -- 172.21.15.112:0/891906562 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "config log", "num": 1, "format": "json"} v 0) -- 0x7fc2b80d1d10 con 0x7fc2b80b5260 2024-09-21T00:26:03.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.008+0000 7fc2b4ff9640 1 -- 172.21.15.112:0/891906562 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "config log", "num": 1, "format": "json"}]=0 v31) ==== 86+0+160 (secure 0 0 0) 0x7fc2ac067690 con 0x7fc2b80b5260 2024-09-21T00:26:03.010 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:26:03.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.010+0000 7fc2be430640 1 -- 172.21.15.112:0/891906562 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fc2a4078b90 msgr2=0x7fc2a407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:03.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.010+0000 7fc2be430640 1 --2- 172.21.15.112:0/891906562 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fc2a4078b90 0x7fc2a407b0a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc2a8033000 tx=0x7fc2a8004730 comp rx=0 tx=0).stop 2024-09-21T00:26:03.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.010+0000 7fc2be430640 1 -- 172.21.15.112:0/891906562 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2b80b5260 msgr2=0x7fc2b8154e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:03.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.010+0000 7fc2be430640 1 --2- 172.21.15.112:0/891906562 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2b80b5260 0x7fc2b8154e10 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac00d6d0 tx=0x7fc2ac00db90 comp rx=0 tx=0).stop 2024-09-21T00:26:03.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.010+0000 7fc2be430640 1 -- 172.21.15.112:0/891906562 shutdown_connections 2024-09-21T00:26:03.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.010+0000 7fc2be430640 1 --2- 172.21.15.112:0/891906562 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc2b815bfc0 0x7fc2b8155350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:03.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.010+0000 7fc2be430640 1 --2- 172.21.15.112:0/891906562 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fc2a4078b90 0x7fc2a407b0a0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:03.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.010+0000 7fc2be430640 1 --2- 172.21.15.112:0/891906562 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2b80b5260 0x7fc2b8154e10 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:03.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.010+0000 7fc2be430640 1 --2- 172.21.15.112:0/891906562 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc2b806def0 0x7fc2b806cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:03.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.010+0000 7fc2be430640 1 -- 172.21.15.112:0/891906562 >> 172.21.15.112:0/891906562 conn(0x7fc2b80cdca0 msgr2=0x7fc2b8074690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:03.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.010+0000 7fc2be430640 1 -- 172.21.15.112:0/891906562 shutdown_connections 2024-09-21T00:26:03.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.010+0000 7fc2be430640 1 -- 172.21.15.112:0/891906562 wait complete. 2024-09-21T00:26:03.023 INFO:teuthology.orchestra.run.smithi112.stdout:[{"version":31,"timestamp":"2024-09-21T00:25:35.705509+0000","name":"","changes":[{"name":"osd.6/osd_mclock_max_capacity_iops_ssd","new_value":"4244.674337"}]}] 2024-09-21T00:26:03.023 INFO:tasks.ceph_manager:config epoch is 31 2024-09-21T00:26:03.023 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-21T00:26:03.023 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-21T00:26:03.024 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-21T00:26:03.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.134+0000 7f9591bb6640 1 -- 172.21.15.112:0/2990430592 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f958c0b2e40 msgr2=0x7f958c0b3250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:03.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.134+0000 7f9591bb6640 1 --2- 172.21.15.112:0/2990430592 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f958c0b2e40 0x7f958c0b3250 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f958c0b7460 tx=0x7f958402aa30 comp rx=0 tx=0).stop 2024-09-21T00:26:03.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.134+0000 7f9591bb6640 1 -- 172.21.15.112:0/2990430592 shutdown_connections 2024-09-21T00:26:03.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.134+0000 7f9591bb6640 1 --2- 172.21.15.112:0/2990430592 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f958c0b2e40 0x7f958c0b3250 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:03.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.134+0000 7f9591bb6640 1 --2- 172.21.15.112:0/2990430592 >> v2:172.21.15.187:3300/0 conn(0x7f958c162280 0x7f958c162690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:03.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.134+0000 7f9591bb6640 1 -- 172.21.15.112:0/2990430592 >> 172.21.15.112:0/2990430592 conn(0x7f958c0b6b20 msgr2=0x7f958c0d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:03.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.134+0000 7f9591bb6640 1 -- 172.21.15.112:0/2990430592 shutdown_connections 2024-09-21T00:26:03.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.134+0000 7f9591bb6640 1 -- 172.21.15.112:0/2990430592 wait complete. 2024-09-21T00:26:03.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.136+0000 7f9591bb6640 1 Processor -- start 2024-09-21T00:26:03.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.136+0000 7f9591bb6640 1 -- start start 2024-09-21T00:26:03.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.137+0000 7f9591bb6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f958c0b2e40 0x7f958c150f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:03.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.137+0000 7f9591bb6640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f958c162280 0x7f958c15fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:03.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.137+0000 7f9591bb6640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f958c160370 0x7f958c159740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:03.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.137+0000 7f9591bb6640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f958c15e4b0 con 0x7f958c0b2e40 2024-09-21T00:26:03.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.137+0000 7f9591bb6640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f958c0b39a0 con 0x7f958c162280 2024-09-21T00:26:03.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.137+0000 7f9591bb6640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f958c15e630 con 0x7f958c160370 2024-09-21T00:26:03.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.137+0000 7f958b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f958c0b2e40 0x7f958c150f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:03.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.137+0000 7f958b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f958c0b2e40 0x7f958c150f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45662/0 (socket says 172.21.15.112:45662) 2024-09-21T00:26:03.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.137+0000 7f958b7fe640 1 -- 172.21.15.112:0/2035004182 learned_addr learned my addr 172.21.15.112:0/2035004182 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:26:03.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.137+0000 7f958affd640 1 --2- 172.21.15.112:0/2035004182 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f958c162280 0x7f958c15fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:03.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.137+0000 7f958bfff640 1 --2- 172.21.15.112:0/2035004182 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f958c160370 0x7f958c159740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:03.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.137+0000 7f958b7fe640 1 -- 172.21.15.112:0/2035004182 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f958c160370 msgr2=0x7f958c159740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:03.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.137+0000 7f958b7fe640 1 --2- 172.21.15.112:0/2035004182 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f958c160370 0x7f958c159740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:03.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.137+0000 7f958b7fe640 1 -- 172.21.15.112:0/2035004182 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f958c162280 msgr2=0x7f958c15fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:03.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.137+0000 7f958b7fe640 1 --2- 172.21.15.112:0/2035004182 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f958c162280 0x7f958c15fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:03.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.137+0000 7f958b7fe640 1 -- 172.21.15.112:0/2035004182 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f958c15aff0 con 0x7f958c0b2e40 2024-09-21T00:26:03.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.137+0000 7f958bfff640 1 --2- 172.21.15.112:0/2035004182 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f958c160370 0x7f958c159740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:26:03.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.138+0000 7f958b7fe640 1 --2- 172.21.15.112:0/2035004182 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f958c0b2e40 0x7f958c150f00 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f957c00c3a0 tx=0x7f957c00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:03.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.138+0000 7f9573fff640 1 -- 172.21.15.112:0/2035004182 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f957c017070 con 0x7f958c0b2e40 2024-09-21T00:26:03.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.138+0000 7f9573fff640 1 -- 172.21.15.112:0/2035004182 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f957c0053c0 con 0x7f958c0b2e40 2024-09-21T00:26:03.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.138+0000 7f9591bb6640 1 -- 172.21.15.112:0/2035004182 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f958c1588b0 con 0x7f958c0b2e40 2024-09-21T00:26:03.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.138+0000 7f9573fff640 1 -- 172.21.15.112:0/2035004182 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f957c003850 con 0x7f958c0b2e40 2024-09-21T00:26:03.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.138+0000 7f9591bb6640 1 -- 172.21.15.112:0/2035004182 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f958c15aad0 con 0x7f958c0b2e40 2024-09-21T00:26:03.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.139+0000 7f9591bb6640 1 -- 172.21.15.112:0/2035004182 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f958c162f00 con 0x7f958c0b2e40 2024-09-21T00:26:03.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.139+0000 7f9573fff640 1 -- 172.21.15.112:0/2035004182 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f957c01a070 con 0x7f958c0b2e40 2024-09-21T00:26:03.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.140+0000 7f9573fff640 1 --2- 172.21.15.112:0/2035004182 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f9578078b90 0x7f957807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:03.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.140+0000 7f9573fff640 1 -- 172.21.15.112:0/2035004182 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f957c09a2a0 con 0x7f958c0b2e40 2024-09-21T00:26:03.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.140+0000 7f958affd640 1 --2- 172.21.15.112:0/2035004182 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f9578078b90 0x7f957807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:03.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.141+0000 7f958affd640 1 --2- 172.21.15.112:0/2035004182 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f9578078b90 0x7f957807b0a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f958c161a30 tx=0x7f9580005f90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:03.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.144+0000 7f9573fff640 1 -- 172.21.15.112:0/2035004182 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f958c162bd0 con 0x7f958c0b2e40 2024-09-21T00:26:03.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.370+0000 7f9591bb6640 1 -- 172.21.15.112:0/2035004182 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f958c151440 con 0x7f958c0b2e40 2024-09-21T00:26:03.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.373+0000 7f9573fff640 1 -- 172.21.15.112:0/2035004182 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+194720 (secure 0 0 0) 0x7f957c061e70 con 0x7f958c0b2e40 2024-09-21T00:26:03.377 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:26:03.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.376+0000 7f9591bb6640 1 -- 172.21.15.112:0/2035004182 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f9578078b90 msgr2=0x7f957807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:03.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.376+0000 7f9591bb6640 1 --2- 172.21.15.112:0/2035004182 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f9578078b90 0x7f957807b0a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f958c161a30 tx=0x7f9580005f90 comp rx=0 tx=0).stop 2024-09-21T00:26:03.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.376+0000 7f9591bb6640 1 -- 172.21.15.112:0/2035004182 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f958c0b2e40 msgr2=0x7f958c150f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:03.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.376+0000 7f9591bb6640 1 --2- 172.21.15.112:0/2035004182 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f958c0b2e40 0x7f958c150f00 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f957c00c3a0 tx=0x7f957c00c860 comp rx=0 tx=0).stop 2024-09-21T00:26:03.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.376+0000 7f9591bb6640 1 -- 172.21.15.112:0/2035004182 shutdown_connections 2024-09-21T00:26:03.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.376+0000 7f9591bb6640 1 --2- 172.21.15.112:0/2035004182 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f958c160370 0x7f958c159740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:03.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.376+0000 7f9591bb6640 1 --2- 172.21.15.112:0/2035004182 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f958c162280 0x7f958c15fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:03.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.376+0000 7f9591bb6640 1 --2- 172.21.15.112:0/2035004182 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f9578078b90 0x7f957807b0a0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:03.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.376+0000 7f9591bb6640 1 --2- 172.21.15.112:0/2035004182 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f958c0b2e40 0x7f958c150f00 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:03.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.376+0000 7f9591bb6640 1 -- 172.21.15.112:0/2035004182 >> 172.21.15.112:0/2035004182 conn(0x7f958c0b6b20 msgr2=0x7f958c06e2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:03.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.376+0000 7f9591bb6640 1 -- 172.21.15.112:0/2035004182 shutdown_connections 2024-09-21T00:26:03.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.376+0000 7f9591bb6640 1 -- 172.21.15.112:0/2035004182 wait complete. 2024-09-21T00:26:03.389 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":5,"flags":0,"active_gid":4103,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":2921522906},{"type":"v1","addr":"172.21.15.187:6801","nonce":2921522906}]},"active_addr":"172.21.15.187:6801/2921522906","active_change":"2024-09-21T00:25:27.079546+0000","active_mgr_features":4540719139924082687,"available":true,"standbys":[{"gid":4116,"name":"y","mgr_features":4540719139924082687,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.8","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.8","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.187:0","nonce":3036884177}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.187:0","nonce":2567967546}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.187:0","nonce":161808188}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.187:0","nonce":461859304}]}]} 2024-09-21T00:26:03.393 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-21T00:26:03.393 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-21T00:26:03.393 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T00:26:03.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.504+0000 7fe79e151640 1 -- 172.21.15.112:0/185275335 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe798073d50 msgr2=0x7fe798074160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:03.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.504+0000 7fe79e151640 1 --2- 172.21.15.112:0/185275335 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe798073d50 0x7fe798074160 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe7980d2120 tx=0x7fe78c02a560 comp rx=0 tx=0).stop 2024-09-21T00:26:03.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.504+0000 7fe79e151640 1 -- 172.21.15.112:0/185275335 shutdown_connections 2024-09-21T00:26:03.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.504+0000 7fe79e151640 1 --2- 172.21.15.112:0/185275335 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe798073d50 0x7fe798074160 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:03.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.504+0000 7fe79e151640 1 --2- 172.21.15.112:0/185275335 >> v2:172.21.15.187:3300/0 conn(0x7fe798160b50 0x7fe798145030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:03.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.504+0000 7fe79e151640 1 -- 172.21.15.112:0/185275335 >> 172.21.15.112:0/185275335 conn(0x7fe7980b2cc0 msgr2=0x7fe7980b30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:03.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.504+0000 7fe79e151640 1 -- 172.21.15.112:0/185275335 shutdown_connections 2024-09-21T00:26:03.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.504+0000 7fe79e151640 1 -- 172.21.15.112:0/185275335 wait complete. 2024-09-21T00:26:03.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.506+0000 7fe79e151640 1 Processor -- start 2024-09-21T00:26:03.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.506+0000 7fe79e151640 1 -- start start 2024-09-21T00:26:03.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.507+0000 7fe79e151640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe798073d50 0x7fe798152ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:03.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.507+0000 7fe79e151640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe7980cffc0 0x7fe79813d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:03.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.507+0000 7fe79e151640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe798160b50 0x7fe79813dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:03.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.507+0000 7fe79e151640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe79806e010 con 0x7fe798160b50 2024-09-21T00:26:03.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.507+0000 7fe79e151640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe7980cdbe0 con 0x7fe7980cffc0 2024-09-21T00:26:03.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.507+0000 7fe79e151640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe79806e190 con 0x7fe798073d50 2024-09-21T00:26:03.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.508+0000 7fe797fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe798160b50 0x7fe79813dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:03.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.508+0000 7fe797fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe798160b50 0x7fe79813dce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45668/0 (socket says 172.21.15.112:45668) 2024-09-21T00:26:03.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.508+0000 7fe797fff640 1 -- 172.21.15.112:0/3476761438 learned_addr learned my addr 172.21.15.112:0/3476761438 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:26:03.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.508+0000 7fe7977fe640 1 --2- 172.21.15.112:0/3476761438 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe798073d50 0x7fe798152ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:03.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.508+0000 7fe796ffd640 1 --2- 172.21.15.112:0/3476761438 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe7980cffc0 0x7fe79813d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:03.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.508+0000 7fe797fff640 1 -- 172.21.15.112:0/3476761438 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe798073d50 msgr2=0x7fe798152ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:03.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.508+0000 7fe797fff640 1 --2- 172.21.15.112:0/3476761438 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe798073d50 0x7fe798152ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:03.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.508+0000 7fe797fff640 1 -- 172.21.15.112:0/3476761438 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe7980cffc0 msgr2=0x7fe79813d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:03.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.508+0000 7fe797fff640 1 --2- 172.21.15.112:0/3476761438 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe7980cffc0 0x7fe79813d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:03.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.508+0000 7fe797fff640 1 -- 172.21.15.112:0/3476761438 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7981f3410 con 0x7fe798160b50 2024-09-21T00:26:03.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.508+0000 7fe797fff640 1 --2- 172.21.15.112:0/3476761438 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe798160b50 0x7fe79813dce0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe79000c730 tx=0x7fe79000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:03.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.508+0000 7fe77ffff640 1 -- 172.21.15.112:0/3476761438 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe790017070 con 0x7fe798160b50 2024-09-21T00:26:03.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.508+0000 7fe77ffff640 1 -- 172.21.15.112:0/3476761438 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe790003c60 con 0x7fe798160b50 2024-09-21T00:26:03.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.508+0000 7fe77ffff640 1 -- 172.21.15.112:0/3476761438 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7900052d0 con 0x7fe798160b50 2024-09-21T00:26:03.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.508+0000 7fe79e151640 1 -- 172.21.15.112:0/3476761438 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe79813c700 con 0x7fe798160b50 2024-09-21T00:26:03.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.508+0000 7fe79e151640 1 -- 172.21.15.112:0/3476761438 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe79813ccc0 con 0x7fe798160b50 2024-09-21T00:26:03.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.509+0000 7fe79e151640 1 -- 172.21.15.112:0/3476761438 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe79813c470 con 0x7fe798160b50 2024-09-21T00:26:03.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.510+0000 7fe77ffff640 1 -- 172.21.15.112:0/3476761438 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe79001a070 con 0x7fe798160b50 2024-09-21T00:26:03.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.511+0000 7fe77ffff640 1 --2- 172.21.15.112:0/3476761438 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fe784078b90 0x7fe78407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:03.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.511+0000 7fe77ffff640 1 -- 172.21.15.112:0/3476761438 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fe79009a7b0 con 0x7fe798160b50 2024-09-21T00:26:03.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.511+0000 7fe7977fe640 1 --2- 172.21.15.112:0/3476761438 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fe784078b90 0x7fe78407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:03.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.511+0000 7fe7977fe640 1 --2- 172.21.15.112:0/3476761438 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fe784078b90 0x7fe78407b0a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe7880058c0 tx=0x7fe78800c440 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:03.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.515+0000 7fe77ffff640 1 -- 172.21.15.112:0/3476761438 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe798145fb0 con 0x7fe798160b50 2024-09-21T00:26:03.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.693+0000 7fe79e151640 1 -- 172.21.15.112:0/3476761438 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe7980d0cf0 con 0x7fe798160b50 2024-09-21T00:26:03.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.694+0000 7fe77ffff640 1 -- 172.21.15.112:0/3476761438 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18622 (secure 0 0 0) 0x7fe7900629d0 con 0x7fe798160b50 2024-09-21T00:26:03.696 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:26:03.697 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":26,"fsid":"8b34692b-dc65-49b2-a08f-26f707b96ac5","created":"2024-09-21T00:25:22.788490+0000","modified":"2024-09-21T00:25:52.623213+0000","last_up_change":"2024-09-21T00:25:36.161835+0000","last_in_change":"2024-09-21T00:25:26.346500+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T00:25:37.105006+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T00:25:45.413895+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T00:25:46.532124+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T00:25:50.244145+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9c575086-67d2-4de2-b01b-d44dcc21d604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6809","nonce":755760621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6811","nonce":755760621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6815","nonce":755760621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6813","nonce":755760621}]},"public_addr":"172.21.15.112:6809/755760621","cluster_addr":"172.21.15.112:6811/755760621","heartbeat_back_addr":"172.21.15.112:6815/755760621","heartbeat_front_addr":"172.21.15.112:6813/755760621","state":["exists","up"]},{"osd":1,"uuid":"a983c227-a3ce-4bdd-82f1-438ba7e3ecd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6801","nonce":344932592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6803","nonce":344932592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6807","nonce":344932592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6805","nonce":344932592}]},"public_addr":"172.21.15.112:6801/344932592","cluster_addr":"172.21.15.112:6803/344932592","heartbeat_back_addr":"172.21.15.112:6807/344932592","heartbeat_front_addr":"172.21.15.112:6805/344932592","state":["exists","up"]},{"osd":2,"uuid":"1193c1d8-bfff-4a57-8200-5f69bf1ea3af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6825","nonce":407933616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6827","nonce":407933616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6831","nonce":407933616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6829","nonce":407933616}]},"public_addr":"172.21.15.112:6825/407933616","cluster_addr":"172.21.15.112:6827/407933616","heartbeat_back_addr":"172.21.15.112:6831/407933616","heartbeat_front_addr":"172.21.15.112:6829/407933616","state":["exists","up"]},{"osd":3,"uuid":"af04330a-36ad-40cc-a8dd-fa259b221de5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6817","nonce":4120164806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6819","nonce":4120164806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6823","nonce":4120164806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6821","nonce":4120164806}]},"public_addr":"172.21.15.112:6817/4120164806","cluster_addr":"172.21.15.112:6819/4120164806","heartbeat_back_addr":"172.21.15.112:6823/4120164806","heartbeat_front_addr":"172.21.15.112:6821/4120164806","state":["exists","up"]},{"osd":4,"uuid":"5b893111-292c-4c5a-b184-53ad38c7199e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6803","nonce":522077560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6805","nonce":522077560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6809","nonce":522077560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6807","nonce":522077560}]},"public_addr":"172.21.15.187:6803/522077560","cluster_addr":"172.21.15.187:6805/522077560","heartbeat_back_addr":"172.21.15.187:6809/522077560","heartbeat_front_addr":"172.21.15.187:6807/522077560","state":["exists","up"]},{"osd":5,"uuid":"c0b8e011-45fe-4c5e-925e-6be2a7f1868a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6827","nonce":3593144721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6829","nonce":3593144721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6833","nonce":3593144721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6831","nonce":3593144721}]},"public_addr":"172.21.15.187:6827/3593144721","cluster_addr":"172.21.15.187:6829/3593144721","heartbeat_back_addr":"172.21.15.187:6833/3593144721","heartbeat_front_addr":"172.21.15.187:6831/3593144721","state":["exists","up"]},{"osd":6,"uuid":"87d2d650-7489-4593-98a3-a1f433ce8dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6811","nonce":1494387657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6813","nonce":1494387657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6817","nonce":1494387657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6815","nonce":1494387657}]},"public_addr":"172.21.15.187:6811/1494387657","cluster_addr":"172.21.15.187:6813/1494387657","heartbeat_back_addr":"172.21.15.187:6817/1494387657","heartbeat_front_addr":"172.21.15.187:6815/1494387657","state":["exists","up"]},{"osd":7,"uuid":"c6b4f8e2-a1ee-4438-aa37-67ff4b6ef1f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6819","nonce":1624699842}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6821","nonce":1624699842}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6825","nonce":1624699842}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6823","nonce":1624699842}]},"public_addr":"172.21.15.187:6819/1624699842","cluster_addr":"172.21.15.187:6821/1624699842","heartbeat_back_addr":"172.21.15.187:6825/1624699842","heartbeat_front_addr":"172.21.15.187:6823/1624699842","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.873162+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.400506+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.664317+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.159791+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.202630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.113212+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.384786+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.448441+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T00:26:03.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.696+0000 7fe79e151640 1 -- 172.21.15.112:0/3476761438 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fe784078b90 msgr2=0x7fe78407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:03.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.696+0000 7fe79e151640 1 --2- 172.21.15.112:0/3476761438 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fe784078b90 0x7fe78407b0a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe7880058c0 tx=0x7fe78800c440 comp rx=0 tx=0).stop 2024-09-21T00:26:03.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.697+0000 7fe79e151640 1 -- 172.21.15.112:0/3476761438 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe798160b50 msgr2=0x7fe79813dce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:03.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.697+0000 7fe79e151640 1 --2- 172.21.15.112:0/3476761438 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe798160b50 0x7fe79813dce0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe79000c730 tx=0x7fe79000cbf0 comp rx=0 tx=0).stop 2024-09-21T00:26:03.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.697+0000 7fe79e151640 1 -- 172.21.15.112:0/3476761438 shutdown_connections 2024-09-21T00:26:03.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.697+0000 7fe79e151640 1 --2- 172.21.15.112:0/3476761438 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fe784078b90 0x7fe78407b0a0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:03.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.697+0000 7fe79e151640 1 --2- 172.21.15.112:0/3476761438 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe798160b50 0x7fe79813dce0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:03.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.697+0000 7fe79e151640 1 --2- 172.21.15.112:0/3476761438 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe7980cffc0 0x7fe79813d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:03.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.697+0000 7fe79e151640 1 --2- 172.21.15.112:0/3476761438 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe798073d50 0x7fe798152ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:03.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.697+0000 7fe79e151640 1 -- 172.21.15.112:0/3476761438 >> 172.21.15.112:0/3476761438 conn(0x7fe7980b2cc0 msgr2=0x7fe798140da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:03.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.697+0000 7fe79e151640 1 -- 172.21.15.112:0/3476761438 shutdown_connections 2024-09-21T00:26:03.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.697+0000 7fe79e151640 1 -- 172.21.15.112:0/3476761438 wait complete. 2024-09-21T00:26:03.710 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-21T00:26:03.710 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T00:26:03.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.821+0000 7f494c00e640 1 -- 172.21.15.112:0/2175091512 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f494406e1d0 msgr2=0x7f49440afd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:03.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.821+0000 7f494c00e640 1 --2- 172.21.15.112:0/2175091512 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f494406e1d0 0x7f49440afd70 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4940009830 tx=0x7f494002a510 comp rx=0 tx=0).stop 2024-09-21T00:26:03.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.821+0000 7f494c00e640 1 -- 172.21.15.112:0/2175091512 shutdown_connections 2024-09-21T00:26:03.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.821+0000 7f494c00e640 1 --2- 172.21.15.112:0/2175091512 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f49440b4740 0x7f494406f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:03.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.821+0000 7f494c00e640 1 --2- 172.21.15.112:0/2175091512 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f494406e1d0 0x7f49440afd70 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:03.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.821+0000 7f494c00e640 1 -- 172.21.15.112:0/2175091512 >> 172.21.15.112:0/2175091512 conn(0x7f49440d0a30 msgr2=0x7f49440d0e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:03.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.821+0000 7f494c00e640 1 -- 172.21.15.112:0/2175091512 shutdown_connections 2024-09-21T00:26:03.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.821+0000 7f494c00e640 1 -- 172.21.15.112:0/2175091512 wait complete. 2024-09-21T00:26:03.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.823+0000 7f494c00e640 1 Processor -- start 2024-09-21T00:26:03.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.823+0000 7f494c00e640 1 -- start start 2024-09-21T00:26:03.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.824+0000 7f494c00e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f494406e1d0 0x7f49440aef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:03.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.824+0000 7f494c00e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f49440b4740 0x7f49441ef2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:03.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.824+0000 7f494c00e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49440b4b50 0x7f49441f5570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:03.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.824+0000 7f494c00e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f494415b3c0 con 0x7f49440b4b50 2024-09-21T00:26:03.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.824+0000 7f494c00e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f494415b240 con 0x7f494406e1d0 2024-09-21T00:26:03.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.824+0000 7f494c00e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f494415b540 con 0x7f49440b4740 2024-09-21T00:26:03.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.824+0000 7f494a5ab640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49440b4b50 0x7f49441f5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:03.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.824+0000 7f494a5ab640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49440b4b50 0x7f49441f5570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45670/0 (socket says 172.21.15.112:45670) 2024-09-21T00:26:03.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.824+0000 7f494a5ab640 1 -- 172.21.15.112:0/2094633136 learned_addr learned my addr 172.21.15.112:0/2094633136 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:26:03.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.824+0000 7f49495a9640 1 --2- 172.21.15.112:0/2094633136 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f49440b4740 0x7f49441ef2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:03.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.824+0000 7f4949daa640 1 --2- 172.21.15.112:0/2094633136 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f494406e1d0 0x7f49440aef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:03.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.824+0000 7f494a5ab640 1 -- 172.21.15.112:0/2094633136 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f49440b4740 msgr2=0x7f49441ef2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:03.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.824+0000 7f494a5ab640 1 --2- 172.21.15.112:0/2094633136 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f49440b4740 0x7f49441ef2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:03.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.824+0000 7f494a5ab640 1 -- 172.21.15.112:0/2094633136 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f494406e1d0 msgr2=0x7f49440aef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:03.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.824+0000 7f494a5ab640 1 --2- 172.21.15.112:0/2094633136 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f494406e1d0 0x7f49440aef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:03.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.824+0000 7f494a5ab640 1 -- 172.21.15.112:0/2094633136 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49440af440 con 0x7f49440b4b50 2024-09-21T00:26:03.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.824+0000 7f494a5ab640 1 --2- 172.21.15.112:0/2094633136 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49440b4b50 0x7f49441f5570 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f493c00c730 tx=0x7f493c00cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:03.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.825+0000 7f493a7fc640 1 -- 172.21.15.112:0/2094633136 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f493c003b70 con 0x7f49440b4b50 2024-09-21T00:26:03.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.825+0000 7f493a7fc640 1 -- 172.21.15.112:0/2094633136 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f493c003d10 con 0x7f49440b4b50 2024-09-21T00:26:03.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.825+0000 7f494c00e640 1 -- 172.21.15.112:0/2094633136 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49440ab510 con 0x7f49440b4b50 2024-09-21T00:26:03.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.825+0000 7f493a7fc640 1 -- 172.21.15.112:0/2094633136 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f493c005330 con 0x7f49440b4b50 2024-09-21T00:26:03.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.825+0000 7f494c00e640 1 -- 172.21.15.112:0/2094633136 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49440abad0 con 0x7f49440b4b50 2024-09-21T00:26:03.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.826+0000 7f494c00e640 1 -- 172.21.15.112:0/2094633136 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49440afd70 con 0x7f49440b4b50 2024-09-21T00:26:03.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.826+0000 7f493a7fc640 1 -- 172.21.15.112:0/2094633136 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f493c0054d0 con 0x7f49440b4b50 2024-09-21T00:26:03.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.827+0000 7f493a7fc640 1 --2- 172.21.15.112:0/2094633136 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4928078b90 0x7f492807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:03.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.827+0000 7f493a7fc640 1 -- 172.21.15.112:0/2094633136 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f493c09ac10 con 0x7f49440b4b50 2024-09-21T00:26:03.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.827+0000 7f4949daa640 1 --2- 172.21.15.112:0/2094633136 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4928078b90 0x7f492807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:03.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.828+0000 7f4949daa640 1 --2- 172.21.15.112:0/2094633136 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4928078b90 0x7f492807b0a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4940034000 tx=0x7f4940004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:03.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:03.832+0000 7f493a7fc640 1 -- 172.21.15.112:0/2094633136 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f493c062750 con 0x7f49440b4b50 2024-09-21T00:26:04.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.008+0000 7f494c00e640 1 -- 172.21.15.112:0/2094633136 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f49440cf1a0 con 0x7f49440b4b50 2024-09-21T00:26:04.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.009+0000 7f493a7fc640 1 -- 172.21.15.112:0/2094633136 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18622 (secure 0 0 0) 0x7f493c067180 con 0x7f49440b4b50 2024-09-21T00:26:04.011 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:26:04.011 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":26,"fsid":"8b34692b-dc65-49b2-a08f-26f707b96ac5","created":"2024-09-21T00:25:22.788490+0000","modified":"2024-09-21T00:25:52.623213+0000","last_up_change":"2024-09-21T00:25:36.161835+0000","last_in_change":"2024-09-21T00:25:26.346500+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T00:25:37.105006+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T00:25:45.413895+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T00:25:46.532124+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T00:25:50.244145+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9c575086-67d2-4de2-b01b-d44dcc21d604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6809","nonce":755760621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6811","nonce":755760621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6815","nonce":755760621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6813","nonce":755760621}]},"public_addr":"172.21.15.112:6809/755760621","cluster_addr":"172.21.15.112:6811/755760621","heartbeat_back_addr":"172.21.15.112:6815/755760621","heartbeat_front_addr":"172.21.15.112:6813/755760621","state":["exists","up"]},{"osd":1,"uuid":"a983c227-a3ce-4bdd-82f1-438ba7e3ecd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6801","nonce":344932592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6803","nonce":344932592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6807","nonce":344932592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6805","nonce":344932592}]},"public_addr":"172.21.15.112:6801/344932592","cluster_addr":"172.21.15.112:6803/344932592","heartbeat_back_addr":"172.21.15.112:6807/344932592","heartbeat_front_addr":"172.21.15.112:6805/344932592","state":["exists","up"]},{"osd":2,"uuid":"1193c1d8-bfff-4a57-8200-5f69bf1ea3af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6825","nonce":407933616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6827","nonce":407933616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6831","nonce":407933616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6829","nonce":407933616}]},"public_addr":"172.21.15.112:6825/407933616","cluster_addr":"172.21.15.112:6827/407933616","heartbeat_back_addr":"172.21.15.112:6831/407933616","heartbeat_front_addr":"172.21.15.112:6829/407933616","state":["exists","up"]},{"osd":3,"uuid":"af04330a-36ad-40cc-a8dd-fa259b221de5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6817","nonce":4120164806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6819","nonce":4120164806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6823","nonce":4120164806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6821","nonce":4120164806}]},"public_addr":"172.21.15.112:6817/4120164806","cluster_addr":"172.21.15.112:6819/4120164806","heartbeat_back_addr":"172.21.15.112:6823/4120164806","heartbeat_front_addr":"172.21.15.112:6821/4120164806","state":["exists","up"]},{"osd":4,"uuid":"5b893111-292c-4c5a-b184-53ad38c7199e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6803","nonce":522077560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6805","nonce":522077560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6809","nonce":522077560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6807","nonce":522077560}]},"public_addr":"172.21.15.187:6803/522077560","cluster_addr":"172.21.15.187:6805/522077560","heartbeat_back_addr":"172.21.15.187:6809/522077560","heartbeat_front_addr":"172.21.15.187:6807/522077560","state":["exists","up"]},{"osd":5,"uuid":"c0b8e011-45fe-4c5e-925e-6be2a7f1868a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6827","nonce":3593144721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6829","nonce":3593144721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6833","nonce":3593144721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6831","nonce":3593144721}]},"public_addr":"172.21.15.187:6827/3593144721","cluster_addr":"172.21.15.187:6829/3593144721","heartbeat_back_addr":"172.21.15.187:6833/3593144721","heartbeat_front_addr":"172.21.15.187:6831/3593144721","state":["exists","up"]},{"osd":6,"uuid":"87d2d650-7489-4593-98a3-a1f433ce8dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6811","nonce":1494387657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6813","nonce":1494387657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6817","nonce":1494387657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6815","nonce":1494387657}]},"public_addr":"172.21.15.187:6811/1494387657","cluster_addr":"172.21.15.187:6813/1494387657","heartbeat_back_addr":"172.21.15.187:6817/1494387657","heartbeat_front_addr":"172.21.15.187:6815/1494387657","state":["exists","up"]},{"osd":7,"uuid":"c6b4f8e2-a1ee-4438-aa37-67ff4b6ef1f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6819","nonce":1624699842}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6821","nonce":1624699842}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6825","nonce":1624699842}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6823","nonce":1624699842}]},"public_addr":"172.21.15.187:6819/1624699842","cluster_addr":"172.21.15.187:6821/1624699842","heartbeat_back_addr":"172.21.15.187:6825/1624699842","heartbeat_front_addr":"172.21.15.187:6823/1624699842","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.873162+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.400506+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.664317+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.159791+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.202630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.113212+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.384786+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.448441+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T00:26:04.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.011+0000 7f494c00e640 1 -- 172.21.15.112:0/2094633136 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4928078b90 msgr2=0x7f492807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:04.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.011+0000 7f494c00e640 1 --2- 172.21.15.112:0/2094633136 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4928078b90 0x7f492807b0a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4940034000 tx=0x7f4940004730 comp rx=0 tx=0).stop 2024-09-21T00:26:04.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.011+0000 7f494c00e640 1 -- 172.21.15.112:0/2094633136 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49440b4b50 msgr2=0x7f49441f5570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:04.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.011+0000 7f494c00e640 1 --2- 172.21.15.112:0/2094633136 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49440b4b50 0x7f49441f5570 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f493c00c730 tx=0x7f493c00cb30 comp rx=0 tx=0).stop 2024-09-21T00:26:04.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.011+0000 7f494c00e640 1 -- 172.21.15.112:0/2094633136 shutdown_connections 2024-09-21T00:26:04.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.011+0000 7f494c00e640 1 --2- 172.21.15.112:0/2094633136 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4928078b90 0x7f492807b0a0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:04.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.011+0000 7f494c00e640 1 --2- 172.21.15.112:0/2094633136 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49440b4b50 0x7f49441f5570 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:04.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.011+0000 7f494c00e640 1 --2- 172.21.15.112:0/2094633136 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f49440b4740 0x7f49441ef2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:04.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.011+0000 7f494c00e640 1 --2- 172.21.15.112:0/2094633136 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f494406e1d0 0x7f49440aef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:04.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.012+0000 7f494c00e640 1 -- 172.21.15.112:0/2094633136 >> 172.21.15.112:0/2094633136 conn(0x7f49440d0a30 msgr2=0x7f49440b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:04.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.012+0000 7f494c00e640 1 -- 172.21.15.112:0/2094633136 shutdown_connections 2024-09-21T00:26:04.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.012+0000 7f494c00e640 1 -- 172.21.15.112:0/2094633136 wait complete. 2024-09-21T00:26:04.025 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-21T00:26:04.025 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-21T00:26:04.026 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-21T00:26:04.026 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-21T00:26:04.027 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-21T00:26:04.027 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-21T00:26:04.028 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-21T00:26:04.028 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-21T00:26:04.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.248+0000 7f4eaef2e640 1 -- 172.21.15.112:0/250346265 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4e940079a0 msgr2=0x7f4e94007db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:04.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.248+0000 7f4eaef2e640 1 --2- 172.21.15.112:0/250346265 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4e940079a0 0x7f4e94007db0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f4ea000c680 tx=0x7f4e94005830 comp rx=0 tx=0).stop 2024-09-21T00:26:04.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.248+0000 7f4eaef2e640 1 -- 172.21.15.112:0/250346265 shutdown_connections 2024-09-21T00:26:04.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.248+0000 7f4eaef2e640 1 --2- 172.21.15.112:0/250346265 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4e940079a0 0x7f4e94007db0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:04.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.248+0000 7f4eaef2e640 1 -- 172.21.15.112:0/250346265 >> 172.21.15.112:0/250346265 conn(0x7f4ea80d05c0 msgr2=0x7f4ea806cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:04.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.248+0000 7f4eaef2e640 1 -- 172.21.15.112:0/250346265 shutdown_connections 2024-09-21T00:26:04.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.249+0000 7f7ff37fe640 1 -- 172.21.15.112:0/66926313 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fe8031600 con 0x7f7ff40cf4f0 2024-09-21T00:26:04.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.249+0000 7f4eaef2e640 1 -- 172.21.15.112:0/250346265 wait complete. 2024-09-21T00:26:04.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.249+0000 7f772bfff640 1 -- 172.21.15.112:0/4024636302 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f772c0cf7e0 msgr2=0x7f772c162230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:04.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.249+0000 7f772bfff640 1 --2- 172.21.15.112:0/4024636302 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f772c0cf7e0 0x7f772c162230 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f772c06ef30 tx=0x7f772002a560 comp rx=0 tx=0).stop 2024-09-21T00:26:04.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.249+0000 7f7ffaa86640 1 -- 172.21.15.112:0/66926313 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ff40cf4f0 msgr2=0x7f7ff40b2ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:04.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.249+0000 7f7ffaa86640 1 --2- 172.21.15.112:0/66926313 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ff40cf4f0 0x7f7ff40b2ed0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f7fe8009830 tx=0x7f7fe802a520 comp rx=0 tx=0).stop 2024-09-21T00:26:04.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.249+0000 7faa0495d640 1 -- 172.21.15.112:0/1955064799 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9f8035510 msgr2=0x7fa9f8054240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:04.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.249+0000 7f7ffaa86640 1 -- 172.21.15.112:0/66926313 shutdown_connections 2024-09-21T00:26:04.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.249+0000 7f7ffaa86640 1 --2- 172.21.15.112:0/66926313 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ff40cf4f0 0x7f7ff40b2ed0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:04.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.249+0000 7f012b301640 1 -- 172.21.15.112:0/2405600186 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f011c0520b0 msgr2=0x7f011c007c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:04.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.249+0000 7f012b301640 1 --2- 172.21.15.112:0/2405600186 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f011c0520b0 0x7f011c007c60 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f011400a940 tx=0x7f011402b7d0 comp rx=0 tx=0).stop 2024-09-21T00:26:04.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.249+0000 7f7ffaa86640 1 -- 172.21.15.112:0/66926313 >> 172.21.15.112:0/66926313 conn(0x7f7ff40cfec0 msgr2=0x7f7ff40d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:04.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.249+0000 7faa0495d640 1 --2- 172.21.15.112:0/1955064799 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9f8035510 0x7fa9f8054240 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa9ec009880 tx=0x7fa9ec02a560 comp rx=0 tx=0).stop 2024-09-21T00:26:04.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.250+0000 7f012b301640 1 -- 172.21.15.112:0/2405600186 shutdown_connections 2024-09-21T00:26:04.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.250+0000 7f012b301640 1 --2- 172.21.15.112:0/2405600186 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f011c0520b0 0x7f011c007c60 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:04.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.250+0000 7f012b301640 1 -- 172.21.15.112:0/2405600186 >> 172.21.15.112:0/2405600186 conn(0x7f011c01e260 msgr2=0x7f011c039e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:04.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.249+0000 7f4b0159a640 1 -- 172.21.15.112:0/2234573650 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4af40047a0 con 0x7f4b04070e80 2024-09-21T00:26:04.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.250+0000 7f772bfff640 1 -- 172.21.15.112:0/4024636302 shutdown_connections 2024-09-21T00:26:04.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.250+0000 7f772bfff640 1 --2- 172.21.15.112:0/4024636302 >> v2:172.21.15.187:3300/0 conn(0x7f772c0cfbf0 0x7f772c162770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:04.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.250+0000 7f772bfff640 1 --2- 172.21.15.112:0/4024636302 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f772c0cf7e0 0x7f772c162230 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:04.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.250+0000 7f772bfff640 1 --2- 172.21.15.112:0/4024636302 >> v2:172.21.15.112:3300/0 conn(0x7f772c0d2ba0 0x7f772c0b9dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:04.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.250+0000 7f772bfff640 1 -- 172.21.15.112:0/4024636302 >> 172.21.15.112:0/4024636302 conn(0x7f772c0d42e0 msgr2=0x7f772c0d46f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:04.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.250+0000 7ff08475e640 1 -- 172.21.15.112:0/3750875486 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff074056dd0 msgr2=0x7ff074057cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:04.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.250+0000 7f012b301640 1 -- 172.21.15.112:0/2405600186 shutdown_connections 2024-09-21T00:26:04.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.250+0000 7ff08475e640 1 --2- 172.21.15.112:0/3750875486 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff074056dd0 0x7ff074057cb0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff078009860 tx=0x7ff07802a5b0 comp rx=0 tx=0).stop 2024-09-21T00:26:04.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.250+0000 7f4b08a38640 1 -- 172.21.15.112:0/2234573650 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b04070e80 msgr2=0x7f4b04070070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:04.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.250+0000 7f4b08a38640 1 --2- 172.21.15.112:0/2234573650 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b04070e80 0x7f4b04070070 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f4b0415b030 tx=0x7f4af402a560 comp rx=0 tx=0).stop 2024-09-21T00:26:04.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.250+0000 7f7ffaa86640 1 -- 172.21.15.112:0/66926313 shutdown_connections 2024-09-21T00:26:04.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.250+0000 7f7ffaa86640 1 -- 172.21.15.112:0/66926313 wait complete. 2024-09-21T00:26:04.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.250+0000 7faa0495d640 1 -- 172.21.15.112:0/1955064799 shutdown_connections 2024-09-21T00:26:04.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.250+0000 7faa0495d640 1 --2- 172.21.15.112:0/1955064799 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa9f80081d0 0x7fa9f8054780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:04.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.250+0000 7faa0495d640 1 --2- 172.21.15.112:0/1955064799 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9f8035510 0x7fa9f8054240 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:04.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.250+0000 7faa0495d640 1 -- 172.21.15.112:0/1955064799 >> 172.21.15.112:0/1955064799 conn(0x7fa9f801eac0 msgr2=0x7fa9f803a780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:04.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.250+0000 7f772bfff640 1 -- 172.21.15.112:0/4024636302 shutdown_connections 2024-09-21T00:26:04.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.250+0000 7faa0495d640 1 -- 172.21.15.112:0/1955064799 shutdown_connections 2024-09-21T00:26:04.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.250+0000 7f772bfff640 1 -- 172.21.15.112:0/4024636302 wait complete. 2024-09-21T00:26:04.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.250+0000 7f4b08a38640 1 -- 172.21.15.112:0/2234573650 shutdown_connections 2024-09-21T00:26:04.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.250+0000 7f4b08a38640 1 --2- 172.21.15.112:0/2234573650 >> v2:172.21.15.187:3300/0 conn(0x7f4b0406e500 0x7f4b04165560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:04.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.250+0000 7f4b08a38640 1 --2- 172.21.15.112:0/2234573650 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b04070e80 0x7f4b04070070 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:04.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.250+0000 7f4b08a38640 1 -- 172.21.15.112:0/2234573650 >> 172.21.15.112:0/2234573650 conn(0x7f4b040b1b90 msgr2=0x7f4b040b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:04.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.250+0000 7f012b301640 1 -- 172.21.15.112:0/2405600186 wait complete. 2024-09-21T00:26:04.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.250+0000 7faa0495d640 1 -- 172.21.15.112:0/1955064799 wait complete. 2024-09-21T00:26:04.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.250+0000 7f2b6a49e640 1 -- 172.21.15.112:0/2517979229 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b6406d2b0 msgr2=0x7f2b641621d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:04.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.250+0000 7f2b6a49e640 1 --2- 172.21.15.112:0/2517979229 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b6406d2b0 0x7f2b641621d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2b640d0070 tx=0x7f2b6002a4d0 comp rx=0 tx=0).stop 2024-09-21T00:26:04.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.251+0000 7f2b6a49e640 1 -- 172.21.15.112:0/2517979229 shutdown_connections 2024-09-21T00:26:04.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.251+0000 7f2b6a49e640 1 --2- 172.21.15.112:0/2517979229 >> v2:172.21.15.187:3300/0 conn(0x7f2b64162710 0x7f2b640cf300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:04.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.251+0000 7f2b6a49e640 1 --2- 172.21.15.112:0/2517979229 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2b640cf840 0x7f2b64154ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:04.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.251+0000 7f2b6a49e640 1 --2- 172.21.15.112:0/2517979229 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b6406d2b0 0x7f2b641621d0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:04.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.251+0000 7f2b6a49e640 1 -- 172.21.15.112:0/2517979229 >> 172.21.15.112:0/2517979229 conn(0x7f2b640d1960 msgr2=0x7f2b640d42e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:04.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.251+0000 7f2b6a49e640 1 -- 172.21.15.112:0/2517979229 shutdown_connections 2024-09-21T00:26:04.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.251+0000 7f2b6a49e640 1 -- 172.21.15.112:0/2517979229 wait complete. 2024-09-21T00:26:04.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.251+0000 7f4b08a38640 1 -- 172.21.15.112:0/2234573650 shutdown_connections 2024-09-21T00:26:04.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.251+0000 7f4b08a38640 1 -- 172.21.15.112:0/2234573650 wait complete. 2024-09-21T00:26:04.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.251+0000 7ff08475e640 1 -- 172.21.15.112:0/3750875486 shutdown_connections 2024-09-21T00:26:04.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.251+0000 7ff08475e640 1 --2- 172.21.15.112:0/3750875486 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff074056dd0 0x7ff074057cb0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:04.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.251+0000 7ff08475e640 1 -- 172.21.15.112:0/3750875486 >> 172.21.15.112:0/3750875486 conn(0x7ff07403a620 msgr2=0x7ff07403aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:04.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.252+0000 7f4eaef2e640 1 Processor -- start 2024-09-21T00:26:04.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.252+0000 7f4eaef2e640 1 -- start start 2024-09-21T00:26:04.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.252+0000 7ff08475e640 1 -- 172.21.15.112:0/3750875486 shutdown_connections 2024-09-21T00:26:04.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.252+0000 7ff08475e640 1 -- 172.21.15.112:0/3750875486 wait complete. 2024-09-21T00:26:04.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.253+0000 7f012b301640 1 Processor -- start 2024-09-21T00:26:04.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.253+0000 7f7ffaa86640 1 Processor -- start 2024-09-21T00:26:04.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.253+0000 7f7ffaa86640 1 -- start start 2024-09-21T00:26:04.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.253+0000 7f772bfff640 1 Processor -- start 2024-09-21T00:26:04.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.254+0000 7f772bfff640 1 -- start start 2024-09-21T00:26:04.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.254+0000 7faa0495d640 1 Processor -- start 2024-09-21T00:26:04.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.254+0000 7f2b6a49e640 1 Processor -- start 2024-09-21T00:26:04.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.254+0000 7f2b6a49e640 1 -- start start 2024-09-21T00:26:04.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.254+0000 7faa0495d640 1 -- start start 2024-09-21T00:26:04.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.254+0000 7f4b08a38640 1 Processor -- start 2024-09-21T00:26:04.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.254+0000 7f4eaef2e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4e940079a0 0x7f4ea80b3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:04.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.255+0000 7f4b08a38640 1 -- start start 2024-09-21T00:26:04.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.255+0000 7f4eaef2e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ea80b3d40 0x7f4ea8144300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:04.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.255+0000 7f4eaef2e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4ea814df20 0x7f4ea8144840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:04.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.253+0000 7f012b301640 1 -- start start 2024-09-21T00:26:04.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.255+0000 7f4eaef2e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4ea815e960 con 0x7f4ea80b3d40 2024-09-21T00:26:04.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.255+0000 7f012b301640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f011c053710 0x7f011c0db5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:04.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.255+0000 7f4eaef2e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f4ea815e7e0 con 0x7f4e940079a0 2024-09-21T00:26:04.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.255+0000 7f4eaef2e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f4ea815eae0 con 0x7f4ea814df20 2024-09-21T00:26:04.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.255+0000 7f4eae72d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4ea814df20 0x7f4ea8144840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:04.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.255+0000 7ff08475e640 1 Processor -- start 2024-09-21T00:26:04.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.255+0000 7f4eae72d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4ea814df20 0x7f4ea8144840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:59510/0 (socket says 172.21.15.112:59510) 2024-09-21T00:26:04.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.255+0000 7f4eae72d640 1 -- 172.21.15.112:0/2010927137 learned_addr learned my addr 172.21.15.112:0/2010927137 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:26:04.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.255+0000 7ff08475e640 1 -- start start 2024-09-21T00:26:04.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.256+0000 7f012909d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f011c053710 0x7f011c0db5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:04.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.256+0000 7f7ffaa86640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ff406dfc0 0x7f7ff4153ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:04.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.256+0000 7f012909d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f011c053710 0x7f011c0db5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45778/0 (socket says 172.21.15.112:45778) 2024-09-21T00:26:04.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.256+0000 7f012909d640 1 -- 172.21.15.112:0/78355995 learned_addr learned my addr 172.21.15.112:0/78355995 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:26:04.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.256+0000 7f012b301640 1 --2- 172.21.15.112:0/78355995 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f011c0df800 0x7f011c04aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:04.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.256+0000 7f772bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f772c0cf7e0 0x7f772c0cb390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:04.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.256+0000 7f7ff8822640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ff406dfc0 0x7f7ff4153ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:04.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.256+0000 7f7ff8822640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ff406dfc0 0x7f7ff4153ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45788/0 (socket says 172.21.15.112:45788) 2024-09-21T00:26:04.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.256+0000 7f4eae72d640 1 -- 172.21.15.112:0/2010927137 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4e940079a0 msgr2=0x7f4ea80b3800 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T00:26:04.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.256+0000 7f4eae72d640 1 --2- 172.21.15.112:0/2010927137 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4e940079a0 0x7f4ea80b3800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:04.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.256+0000 7f7ff8822640 1 -- 172.21.15.112:0/3788287230 learned_addr learned my addr 172.21.15.112:0/3788287230 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:26:04.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.256+0000 7f4eae72d640 1 -- 172.21.15.112:0/2010927137 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ea80b3d40 msgr2=0x7f4ea8144300 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T00:26:04.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.256+0000 7f4eae72d640 1 --2- 172.21.15.112:0/2010927137 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ea80b3d40 0x7f4ea8144300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:04.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.256+0000 7f4eae72d640 1 -- 172.21.15.112:0/2010927137 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ea81f4890 con 0x7f4ea814df20 2024-09-21T00:26:04.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.256+0000 7f7ffaa86640 1 --2- 172.21.15.112:0/3788287230 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7ff40cf4f0 0x7f7ff413bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:04.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.256+0000 7f772bfff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f772c0cfbf0 0x7f772c1f97c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:04.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.256+0000 7f772bfff640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f772c0d2ba0 0x7f772c158090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:04.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.256+0000 7f772bfff640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f772c155540 con 0x7f772c0cf7e0 2024-09-21T00:26:04.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.256+0000 7f772bfff640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f772c1553c0 con 0x7f772c0cfbf0 2024-09-21T00:26:04.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.256+0000 7f772bfff640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f772c1556c0 con 0x7f772c0d2ba0 2024-09-21T00:26:04.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.256+0000 7f7ffaa86640 1 --2- 172.21.15.112:0/3788287230 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7ff414de00 0x7f7ff415ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:04.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.256+0000 7f012b301640 1 --2- 172.21.15.112:0/78355995 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f011c0dc1c0 0x7f011c04af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:04.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.256+0000 7f7ffaa86640 1 -- 172.21.15.112:0/3788287230 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7ff40b3590 con 0x7f7ff406dfc0 2024-09-21T00:26:04.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.256+0000 7f7ffaa86640 1 -- 172.21.15.112:0/3788287230 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7ff40b3410 con 0x7f7ff414de00 2024-09-21T00:26:04.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.256+0000 7f7ff3fff640 1 --2- 172.21.15.112:0/3788287230 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7ff40cf4f0 0x7f7ff413bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:04.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.257+0000 7f7ff9023640 1 --2- 172.21.15.112:0/3788287230 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7ff414de00 0x7f7ff415ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:04.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.257+0000 7f7ffaa86640 1 -- 172.21.15.112:0/3788287230 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f7ff40b3710 con 0x7f7ff40cf4f0 2024-09-21T00:26:04.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.257+0000 7faa0495d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa9f80081d0 0x7fa9f80d0720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:04.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.257+0000 7f2b6a49e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2b6406d2b0 0x7f2b641f5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:04.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.257+0000 7f4eae72d640 1 --2- 172.21.15.112:0/2010927137 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4ea814df20 0x7f4ea8144840 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f4ea000c770 tx=0x7f4ea0003b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:04.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.257+0000 7faa0495d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa9f8056200 0x7fa9f804d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:04.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.257+0000 7f4b08a38640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b04070e80 0x7f4b04154090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:04.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.257+0000 7f2b6a49e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2b640cf840 0x7f2b641601e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:04.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.257+0000 7faa0495d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9f804f090 0x7fa9f804e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:04.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.257+0000 7f772b7fe640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f772c0d2ba0 0x7f772c158090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:04.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.257+0000 7faa0495d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa9f8055050 con 0x7fa9f804f090 2024-09-21T00:26:04.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.257+0000 7faa0495d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa9f8054ed0 con 0x7fa9f8056200 2024-09-21T00:26:04.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.257+0000 7f2b6949c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2b6406d2b0 0x7f2b641f5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:04.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.257+0000 7f2b6949c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2b6406d2b0 0x7f2b641f5bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:43356/0 (socket says 172.21.15.112:43356) 2024-09-21T00:26:04.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.257+0000 7faa0495d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa9f80551d0 con 0x7fa9f80081d0 2024-09-21T00:26:04.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.257+0000 7f2b6a49e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b64162710 0x7f2b641592a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:04.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.257+0000 7fa9fed9d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa9f80081d0 0x7fa9f80d0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:04.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.257+0000 7fa9fed9d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa9f80081d0 0x7fa9f80d0720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:59540/0 (socket says 172.21.15.112:59540) 2024-09-21T00:26:04.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.257+0000 7fa9fed9d640 1 -- 172.21.15.112:0/3971453688 learned_addr learned my addr 172.21.15.112:0/3971453688 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:26:04.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.257+0000 7f772b7fe640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f772c0d2ba0 0x7f772c158090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:59516/0 (socket says 172.21.15.112:59516) 2024-09-21T00:26:04.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.257+0000 7f772b7fe640 1 -- 172.21.15.112:0/2912969700 learned_addr learned my addr 172.21.15.112:0/2912969700 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:26:04.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.257+0000 7f2b68c9b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2b640cf840 0x7f2b641601e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:04.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.257+0000 7fa9fe59c640 1 --2- 172.21.15.112:0/3971453688 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa9f8056200 0x7fa9f804d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:04.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.257+0000 7f2b68c9b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2b640cf840 0x7f2b641601e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:59548/0 (socket says 172.21.15.112:59548) 2024-09-21T00:26:04.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.257+0000 7f4b0259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b04070e80 0x7f4b04154090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:04.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.256+0000 7f012b301640 1 -- 172.21.15.112:0/78355995 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f011c006110 con 0x7f011c053710 2024-09-21T00:26:04.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.257+0000 7f2b6949c640 1 -- 172.21.15.112:0/2740563253 learned_addr learned my addr 172.21.15.112:0/2740563253 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:26:04.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.257+0000 7f4b08a38640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4b0415ac20 0x7f4b040b55b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:04.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.256+0000 7f012b301640 1 -- 172.21.15.112:0/78355995 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f011c005f90 con 0x7f011c0df800 2024-09-21T00:26:04.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.256+0000 7f012b301640 1 -- 172.21.15.112:0/78355995 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f011c006290 con 0x7f011c0dc1c0 2024-09-21T00:26:04.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.256+0000 7f012889c640 1 --2- 172.21.15.112:0/78355995 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f011c0df800 0x7f011c04aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:04.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.257+0000 7f012989e640 1 --2- 172.21.15.112:0/78355995 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f011c0dc1c0 0x7f011c04af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:04.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.257+0000 7f2b6a49e640 1 -- 172.21.15.112:0/2740563253 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2b64155770 con 0x7f2b64162710 2024-09-21T00:26:04.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.257+0000 7f2b6a49e640 1 -- 172.21.15.112:0/2740563253 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f2b641555f0 con 0x7f2b6406d2b0 2024-09-21T00:26:04.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.257+0000 7f4b08a38640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4b040b5af0 0x7f4b041f4490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:04.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.257+0000 7f2b6a49e640 1 -- 172.21.15.112:0/2740563253 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f2b641558f0 con 0x7f2b640cf840 2024-09-21T00:26:04.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.257+0000 7f4b08a38640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4b04073120 con 0x7f4b04070e80 2024-09-21T00:26:04.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.257+0000 7f4b08a38640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f4b04072fa0 con 0x7f4b0415ac20 2024-09-21T00:26:04.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.257+0000 7f4b08a38640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f4b040732a0 con 0x7f4b040b5af0 2024-09-21T00:26:04.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.257+0000 7f4e9e7fc640 1 -- 172.21.15.112:0/2010927137 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea00020a0 con 0x7f4ea814df20 2024-09-21T00:26:04.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.257+0000 7f4e9e7fc640 1 -- 172.21.15.112:0/2010927137 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4ea002b890 con 0x7f4ea814df20 2024-09-21T00:26:04.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.257+0000 7f4e9e7fc640 1 -- 172.21.15.112:0/2010927137 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea002bca0 con 0x7f4ea814df20 2024-09-21T00:26:04.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.257+0000 7f4eaef2e640 1 -- 172.21.15.112:0/2010927137 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ea81f4b70 con 0x7f4ea814df20 2024-09-21T00:26:04.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.257+0000 7f4eaef2e640 1 -- 172.21.15.112:0/2010927137 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ea81f5130 con 0x7f4ea814df20 2024-09-21T00:26:04.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.257+0000 7f012909d640 1 -- 172.21.15.112:0/78355995 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f011c0dc1c0 msgr2=0x7f011c04af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:04.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.257+0000 7f012909d640 1 --2- 172.21.15.112:0/78355995 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f011c0dc1c0 0x7f011c04af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:04.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.257+0000 7f012909d640 1 -- 172.21.15.112:0/78355995 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f011c0df800 msgr2=0x7f011c04aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:04.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.257+0000 7f012909d640 1 --2- 172.21.15.112:0/78355995 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f011c0df800 0x7f011c04aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:04.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.257+0000 7f012909d640 1 -- 172.21.15.112:0/78355995 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f011c0f0f80 con 0x7f011c053710 2024-09-21T00:26:04.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.258+0000 7f772a7fc640 1 --2- 172.21.15.112:0/2912969700 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f772c0cfbf0 0x7f772c1f97c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:04.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.257+0000 7f7ff3fff640 1 -- 172.21.15.112:0/3788287230 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7ff414de00 msgr2=0x7f7ff415ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:04.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.257+0000 7f7ff3fff640 1 --2- 172.21.15.112:0/3788287230 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7ff414de00 0x7f7ff415ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:04.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.257+0000 7f7ff3fff640 1 -- 172.21.15.112:0/3788287230 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ff406dfc0 msgr2=0x7f7ff4153ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:04.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.257+0000 7f7ff3fff640 1 --2- 172.21.15.112:0/3788287230 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ff406dfc0 0x7f7ff4153ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:04.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.257+0000 7f7ff3fff640 1 -- 172.21.15.112:0/3788287230 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ff415bdb0 con 0x7f7ff40cf4f0 2024-09-21T00:26:04.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.258+0000 7f2b69c9d640 1 --2- 172.21.15.112:0/2740563253 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b64162710 0x7f2b641592a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:04.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.258+0000 7f772affd640 1 --2- 172.21.15.112:0/2912969700 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f772c0cf7e0 0x7f772c0cb390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:04.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.258+0000 7f4b0259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b04070e80 0x7f4b04154090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45802/0 (socket says 172.21.15.112:45802) 2024-09-21T00:26:04.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.258+0000 7f4b0259c640 1 -- 172.21.15.112:0/950694403 learned_addr learned my addr 172.21.15.112:0/950694403 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:26:04.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.258+0000 7ff08475e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff074054390 0x7ff0740d1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:04.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.258+0000 7ff08475e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff074056dd0 0x7ff07417d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:04.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.258+0000 7ff08475e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0740e2f70 0x7ff07417dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:04.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.258+0000 7ff08475e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff074007450 con 0x7ff0740e2f70 2024-09-21T00:26:04.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.258+0000 7ff08475e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff0740072d0 con 0x7ff074056dd0 2024-09-21T00:26:04.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.258+0000 7ff08475e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff0740075d0 con 0x7ff074054390 2024-09-21T00:26:04.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.258+0000 7f2b6949c640 1 -- 172.21.15.112:0/2740563253 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2b640cf840 msgr2=0x7f2b641601e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:04.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.258+0000 7f2b6949c640 1 --2- 172.21.15.112:0/2740563253 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2b640cf840 0x7f2b641601e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:04.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.258+0000 7f2b6949c640 1 -- 172.21.15.112:0/2740563253 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b64162710 msgr2=0x7f2b641592a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:04.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.258+0000 7f2b6949c640 1 --2- 172.21.15.112:0/2740563253 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b64162710 0x7f2b641592a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:04.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.258+0000 7f2b6949c640 1 -- 172.21.15.112:0/2740563253 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b641f6110 con 0x7f2b6406d2b0 2024-09-21T00:26:04.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.258+0000 7f4b02d9d640 1 --2- 172.21.15.112:0/950694403 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4b040b5af0 0x7f4b041f4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:04.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.258+0000 7ff082cfb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0740e2f70 0x7ff07417dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:04.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.258+0000 7ff082cfb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0740e2f70 0x7ff07417dde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45848/0 (socket says 172.21.15.112:45848) 2024-09-21T00:26:04.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.258+0000 7ff082cfb640 1 -- 172.21.15.112:0/1194921374 learned_addr learned my addr 172.21.15.112:0/1194921374 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:26:04.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.258+0000 7f2b69c9d640 1 --2- 172.21.15.112:0/2740563253 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b64162710 0x7f2b641592a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:26:04.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.258+0000 7fa9ff59e640 1 --2- 172.21.15.112:0/3971453688 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9f804f090 0x7fa9f804e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:04.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.258+0000 7fa9fe59c640 1 -- 172.21.15.112:0/3971453688 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa9f80081d0 msgr2=0x7fa9f80d0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:04.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.258+0000 7ff081cf9640 1 --2- 172.21.15.112:0/1194921374 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff074056dd0 0x7ff07417d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:04.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.258+0000 7fa9fe59c640 1 --2- 172.21.15.112:0/3971453688 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa9f80081d0 0x7fa9f80d0720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:04.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.258+0000 7fa9fe59c640 1 -- 172.21.15.112:0/3971453688 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9f804f090 msgr2=0x7fa9f804e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:04.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.258+0000 7fa9fe59c640 1 --2- 172.21.15.112:0/3971453688 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9f804f090 0x7fa9f804e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:04.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.258+0000 7fa9fe59c640 1 -- 172.21.15.112:0/3971453688 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9f804f610 con 0x7fa9f8056200 2024-09-21T00:26:04.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.258+0000 7fa9fed9d640 1 --2- 172.21.15.112:0/3971453688 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa9f80081d0 0x7fa9f80d0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:26:04.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.258+0000 7fa9ff59e640 1 --2- 172.21.15.112:0/3971453688 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9f804f090 0x7fa9f804e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:26:04.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.258+0000 7f7ff3fff640 1 --2- 172.21.15.112:0/3788287230 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7ff40cf4f0 0x7f7ff413bac0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7fe8031000 tx=0x7f7fe8033f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:04.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.258+0000 7ff0824fa640 1 --2- 172.21.15.112:0/1194921374 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff074054390 0x7ff0740d1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:04.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.258+0000 7f2b6949c640 1 --2- 172.21.15.112:0/2740563253 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2b6406d2b0 0x7f2b641f5bd0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f2b60034040 tx=0x7f2b60003ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:04.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.258+0000 7f772a7fc640 1 -- 172.21.15.112:0/2912969700 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f772c0d2ba0 msgr2=0x7f772c158090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:04.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.258+0000 7f772a7fc640 1 --2- 172.21.15.112:0/2912969700 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f772c0d2ba0 0x7f772c158090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:04.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.258+0000 7f772a7fc640 1 -- 172.21.15.112:0/2912969700 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f772c0cf7e0 msgr2=0x7f772c0cb390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:04.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.258+0000 7f772a7fc640 1 --2- 172.21.15.112:0/2912969700 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f772c0cf7e0 0x7f772c0cb390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:04.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.258+0000 7f772a7fc640 1 -- 172.21.15.112:0/2912969700 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f772c0cd050 con 0x7f772c0cfbf0 2024-09-21T00:26:04.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.258+0000 7f012909d640 1 --2- 172.21.15.112:0/78355995 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f011c053710 0x7f011c0db5a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f012000c760 tx=0x7f012000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:04.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.259+0000 7fa9fe59c640 1 --2- 172.21.15.112:0/3971453688 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa9f8056200 0x7fa9f804d5e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa9ec031000 tx=0x7fa9ec033ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:04.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.258+0000 7f4b01d9b640 1 --2- 172.21.15.112:0/950694403 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4b0415ac20 0x7f4b040b55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:04.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.258+0000 7f4b01d9b640 1 -- 172.21.15.112:0/950694403 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4b040b5af0 msgr2=0x7f4b041f4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:04.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.258+0000 7f4b01d9b640 1 --2- 172.21.15.112:0/950694403 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4b040b5af0 0x7f4b041f4490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:04.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.258+0000 7f4b01d9b640 1 -- 172.21.15.112:0/950694403 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b04070e80 msgr2=0x7f4b04154090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:04.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.258+0000 7f4b01d9b640 1 --2- 172.21.15.112:0/950694403 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b04070e80 0x7f4b04154090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:04.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.259+0000 7f4b01d9b640 1 -- 172.21.15.112:0/950694403 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b041f2340 con 0x7f4b0415ac20 2024-09-21T00:26:04.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.258+0000 7f7ff17fa640 1 -- 172.21.15.112:0/3788287230 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fe8031600 con 0x7f7ff40cf4f0 2024-09-21T00:26:04.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.259+0000 7f7ff17fa640 1 -- 172.21.15.112:0/3788287230 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7fe8031d50 con 0x7f7ff40cf4f0 2024-09-21T00:26:04.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.259+0000 7f4b01d9b640 1 --2- 172.21.15.112:0/950694403 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4b0415ac20 0x7f4b040b55b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f4af80015d0 tx=0x7f4af8001a90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:04.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.259+0000 7f2b59ffb640 1 -- 172.21.15.112:0/2740563253 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b6003e070 con 0x7f2b6406d2b0 2024-09-21T00:26:04.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.259+0000 7f2b59ffb640 1 -- 172.21.15.112:0/2740563253 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2b60004a90 con 0x7f2b6406d2b0 2024-09-21T00:26:04.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.259+0000 7f2b6a49e640 1 -- 172.21.15.112:0/2740563253 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b64160d50 con 0x7f2b6406d2b0 2024-09-21T00:26:04.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.259+0000 7ff081cf9640 1 -- 172.21.15.112:0/1194921374 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff074054390 msgr2=0x7ff0740d1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:04.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.259+0000 7ff081cf9640 1 --2- 172.21.15.112:0/1194921374 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff074054390 0x7ff0740d1eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:04.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.259+0000 7ff081cf9640 1 -- 172.21.15.112:0/1194921374 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0740e2f70 msgr2=0x7ff07417dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:04.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.259+0000 7ff081cf9640 1 --2- 172.21.15.112:0/1194921374 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0740e2f70 0x7ff07417dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:04.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.259+0000 7ff081cf9640 1 -- 172.21.15.112:0/1194921374 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0740d5d10 con 0x7ff074056dd0 2024-09-21T00:26:04.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.259+0000 7f4af2ffd640 1 -- 172.21.15.112:0/950694403 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4af800e280 con 0x7f4b0415ac20 2024-09-21T00:26:04.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.259+0000 7f2b6a49e640 1 -- 172.21.15.112:0/2740563253 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b64161310 con 0x7f2b6406d2b0 2024-09-21T00:26:04.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.259+0000 7f7ff17fa640 1 -- 172.21.15.112:0/3788287230 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fe8005260 con 0x7f7ff40cf4f0 2024-09-21T00:26:04.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.259+0000 7fa9f77fe640 1 -- 172.21.15.112:0/3971453688 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9ec031b50 con 0x7fa9f8056200 2024-09-21T00:26:04.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.259+0000 7f7ffaa86640 1 -- 172.21.15.112:0/3788287230 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ff415c090 con 0x7f7ff40cf4f0 2024-09-21T00:26:04.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.259+0000 7f7ffaa86640 1 -- 172.21.15.112:0/3788287230 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ff415c650 con 0x7f7ff40cf4f0 2024-09-21T00:26:04.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.259+0000 7fa9f77fe640 1 -- 172.21.15.112:0/3971453688 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa9ec031d10 con 0x7fa9f8056200 2024-09-21T00:26:04.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.259+0000 7fa9f77fe640 1 -- 172.21.15.112:0/3971453688 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9ec0332d0 con 0x7fa9f8056200 2024-09-21T00:26:04.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.259+0000 7faa0495d640 1 -- 172.21.15.112:0/3971453688 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9f80cf960 con 0x7fa9f8056200 2024-09-21T00:26:04.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.259+0000 7faa0495d640 1 -- 172.21.15.112:0/3971453688 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9f80cff20 con 0x7fa9f8056200 2024-09-21T00:26:04.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.259+0000 7f772a7fc640 1 --2- 172.21.15.112:0/2912969700 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f772c0cfbf0 0x7f772c1f97c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f7720031000 tx=0x7f77200038a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:04.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.259+0000 7f4af2ffd640 1 -- 172.21.15.112:0/950694403 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4af80050d0 con 0x7f4b0415ac20 2024-09-21T00:26:04.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.260+0000 7ff081cf9640 1 --2- 172.21.15.112:0/1194921374 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff074056dd0 0x7ff07417d8a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff0740e3550 tx=0x7ff06c00c820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:04.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.260+0000 7f4eaef2e640 1 -- 172.21.15.112:0/2010927137 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4ea815b9c0 con 0x7f4ea814df20 2024-09-21T00:26:04.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.260+0000 7f77137fe640 1 -- 172.21.15.112:0/2912969700 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f772003e030 con 0x7f772c0cfbf0 2024-09-21T00:26:04.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.260+0000 7f77137fe640 1 -- 172.21.15.112:0/2912969700 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7720003d90 con 0x7f772c0cfbf0 2024-09-21T00:26:04.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.260+0000 7f77137fe640 1 -- 172.21.15.112:0/2912969700 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7720035040 con 0x7f772c0cfbf0 2024-09-21T00:26:04.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.260+0000 7f0119ffb640 1 -- 172.21.15.112:0/78355995 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0120016020 con 0x7f011c053710 2024-09-21T00:26:04.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.260+0000 7f0119ffb640 1 -- 172.21.15.112:0/78355995 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f012000a860 con 0x7f011c053710 2024-09-21T00:26:04.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.260+0000 7f0119ffb640 1 -- 172.21.15.112:0/78355995 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f012000acb0 con 0x7f011c053710 2024-09-21T00:26:04.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.260+0000 7f012b301640 1 -- 172.21.15.112:0/78355995 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f011c0f1260 con 0x7f011c053710 2024-09-21T00:26:04.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.261+0000 7f012b301640 1 -- 172.21.15.112:0/78355995 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f011c031bf0 con 0x7f011c053710 2024-09-21T00:26:04.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.260+0000 7ff072ffd640 1 -- 172.21.15.112:0/1194921374 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff06c011790 con 0x7ff074056dd0 2024-09-21T00:26:04.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.260+0000 7ff072ffd640 1 -- 172.21.15.112:0/1194921374 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff06c017070 con 0x7ff074056dd0 2024-09-21T00:26:04.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.260+0000 7ff072ffd640 1 -- 172.21.15.112:0/1194921374 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff06c011c30 con 0x7ff074056dd0 2024-09-21T00:26:04.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.260+0000 7ff08475e640 1 -- 172.21.15.112:0/1194921374 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0740d5ff0 con 0x7ff074056dd0 2024-09-21T00:26:04.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.260+0000 7ff08475e640 1 -- 172.21.15.112:0/1194921374 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0740d65b0 con 0x7ff074056dd0 2024-09-21T00:26:04.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.260+0000 7f2b59ffb640 1 -- 172.21.15.112:0/2740563253 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b60035040 con 0x7f2b6406d2b0 2024-09-21T00:26:04.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.260+0000 7f2b6a49e640 1 -- 172.21.15.112:0/2740563253 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2b640ccb00 con 0x7f2b6406d2b0 2024-09-21T00:26:04.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.261+0000 7f7ffaa86640 1 -- 172.21.15.112:0/3788287230 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7ff413c000 con 0x7f7ff40cf4f0 2024-09-21T00:26:04.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.261+0000 7f4b08a38640 1 -- 172.21.15.112:0/950694403 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b041f2620 con 0x7f4b0415ac20 2024-09-21T00:26:04.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.261+0000 7f4b08a38640 1 -- 172.21.15.112:0/950694403 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b041f2be0 con 0x7f4b0415ac20 2024-09-21T00:26:04.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.261+0000 7f4af2ffd640 1 -- 172.21.15.112:0/950694403 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4af80198d0 con 0x7f4b0415ac20 2024-09-21T00:26:04.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.261+0000 7faa0495d640 1 -- 172.21.15.112:0/3971453688 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa9f8054240 con 0x7fa9f8056200 2024-09-21T00:26:04.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.261+0000 7f772bfff640 1 -- 172.21.15.112:0/2912969700 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f772c0cd330 con 0x7f772c0cfbf0 2024-09-21T00:26:04.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.261+0000 7f772bfff640 1 -- 172.21.15.112:0/2912969700 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f772c0cd8f0 con 0x7f772c0cfbf0 2024-09-21T00:26:04.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.262+0000 7f012b301640 1 -- 172.21.15.112:0/78355995 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f011c007c60 con 0x7f011c053710 2024-09-21T00:26:04.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.261+0000 7ff08475e640 1 -- 172.21.15.112:0/1194921374 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff064001030 con 0x7ff074056dd0 2024-09-21T00:26:04.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.262+0000 7f4b08a38640 1 -- 172.21.15.112:0/950694403 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4ae8001030 con 0x7f4b0415ac20 2024-09-21T00:26:04.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.260+0000 7f4e9e7fc640 1 -- 172.21.15.112:0/2010927137 <== mon.2 v2:172.21.15.187:3301/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f4ea003c030 con 0x7f4ea814df20 2024-09-21T00:26:04.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.261+0000 7f4e9e7fc640 1 --2- 172.21.15.112:0/2010927137 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4e8c078c60 0x7f4e8c07b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:04.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.262+0000 7f4e9e7fc640 1 -- 172.21.15.112:0/2010927137 <== mon.2 v2:172.21.15.187:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f4ea00ba7a0 con 0x7f4ea814df20 2024-09-21T00:26:04.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.262+0000 7f4e9e7fc640 1 --2- 172.21.15.112:0/2010927137 >> [v2:172.21.15.112:6800/344932592,v1:172.21.15.112:6801/344932592] conn(0x7f4e8c0823b0 0x7f4e8c084840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:04.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.263+0000 7fa9f77fe640 1 -- 172.21.15.112:0/3971453688 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa9ec066d10 con 0x7fa9f8056200 2024-09-21T00:26:04.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.262+0000 7f4e9e7fc640 1 -- 172.21.15.112:0/2010927137 --> [v2:172.21.15.112:6800/344932592,v1:172.21.15.112:6801/344932592] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4ea003be50 con 0x7f4e8c0823b0 2024-09-21T00:26:04.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.262+0000 7f4ead72b640 1 --2- 172.21.15.112:0/2010927137 >> [v2:172.21.15.112:6800/344932592,v1:172.21.15.112:6801/344932592] conn(0x7f4e8c0823b0 0x7f4e8c084840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:04.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.262+0000 7f4eadf2c640 1 --2- 172.21.15.112:0/2010927137 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4e8c078c60 0x7f4e8c07b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:04.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.263+0000 7f4ead72b640 1 --2- 172.21.15.112:0/2010927137 >> [v2:172.21.15.112:6800/344932592,v1:172.21.15.112:6801/344932592] conn(0x7f4e8c0823b0 0x7f4e8c084840 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:04.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.263+0000 7f4e9e7fc640 1 -- 172.21.15.112:0/2010927137 <== mon.2 v2:172.21.15.187:3301/0 6 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4ea00c1050 con 0x7f4ea814df20 2024-09-21T00:26:04.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.264+0000 7f7ff17fa640 1 -- 172.21.15.112:0/3788287230 <== mon.2 v2:172.21.15.187:3301/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f7fe8005400 con 0x7f7ff40cf4f0 2024-09-21T00:26:04.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.264+0000 7fa9f77fe640 1 --2- 172.21.15.112:0/3971453688 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fa9e8078c60 0x7fa9e807b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:04.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.264+0000 7fa9f77fe640 1 -- 172.21.15.112:0/3971453688 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fa9ec0bc5c0 con 0x7fa9f8056200 2024-09-21T00:26:04.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.264+0000 7f772bfff640 1 -- 172.21.15.112:0/2912969700 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f772c0b9dd0 con 0x7f772c0cfbf0 2024-09-21T00:26:04.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.264+0000 7fa9f77fe640 1 --2- 172.21.15.112:0/3971453688 >> [v2:172.21.15.187:6818/1624699842,v1:172.21.15.187:6819/1624699842] conn(0x7fa9e8082420 0x7fa9e80848b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:04.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.264+0000 7f2b59ffb640 1 -- 172.21.15.112:0/2740563253 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2b60041070 con 0x7f2b6406d2b0 2024-09-21T00:26:04.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.264+0000 7fa9f77fe640 1 -- 172.21.15.112:0/3971453688 --> [v2:172.21.15.187:6818/1624699842,v1:172.21.15.187:6819/1624699842] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa9ec069e50 con 0x7fa9e8082420 2024-09-21T00:26:04.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.264+0000 7ff072ffd640 1 -- 172.21.15.112:0/1194921374 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff06c005020 con 0x7ff074056dd0 2024-09-21T00:26:04.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.264+0000 7fa9f77fe640 1 -- 172.21.15.112:0/3971453688 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa9ec0bc9b0 con 0x7fa9f8056200 2024-09-21T00:26:04.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.265+0000 7f4e9e7fc640 1 -- 172.21.15.112:0/2010927137 <== osd.1 v2:172.21.15.112:6800/344932592 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4ea003be50 con 0x7f4e8c0823b0 2024-09-21T00:26:04.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.265+0000 7fa9fed9d640 1 --2- 172.21.15.112:0/3971453688 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fa9e8078c60 0x7fa9e807b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:04.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.265+0000 7ff072ffd640 1 --2- 172.21.15.112:0/1194921374 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff060078be0 0x7ff06007b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:04.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.265+0000 7f7ff17fa640 1 --2- 172.21.15.112:0/3788287230 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f7fd8078c60 0x7f7fd807b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:04.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.265+0000 7fa9ff59e640 1 --2- 172.21.15.112:0/3971453688 >> [v2:172.21.15.187:6818/1624699842,v1:172.21.15.187:6819/1624699842] conn(0x7fa9e8082420 0x7fa9e80848b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:04.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.265+0000 7f4af2ffd640 1 -- 172.21.15.112:0/950694403 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f4af8019a70 con 0x7f4b0415ac20 2024-09-21T00:26:04.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.264+0000 7f0119ffb640 1 -- 172.21.15.112:0/78355995 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0120019020 con 0x7f011c053710 2024-09-21T00:26:04.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.265+0000 7f0119ffb640 1 --2- 172.21.15.112:0/78355995 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f0108078c60 0x7f010807b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:04.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.265+0000 7ff072ffd640 1 -- 172.21.15.112:0/1194921374 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7ff06c09abf0 con 0x7ff074056dd0 2024-09-21T00:26:04.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.265+0000 7f7ff17fa640 1 -- 172.21.15.112:0/3788287230 <== mon.2 v2:172.21.15.187:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f7fe80bc0b0 con 0x7f7ff40cf4f0 2024-09-21T00:26:04.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.265+0000 7f2b59ffb640 1 --2- 172.21.15.112:0/2740563253 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2b48078be0 0x7f2b4807b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:04.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.265+0000 7f0119ffb640 1 -- 172.21.15.112:0/78355995 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f012009afe0 con 0x7f011c053710 2024-09-21T00:26:04.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.265+0000 7ff072ffd640 1 --2- 172.21.15.112:0/1194921374 >> [v2:172.21.15.112:6816/4120164806,v1:172.21.15.112:6817/4120164806] conn(0x7ff0600823a0 0x7ff060084830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:04.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.265+0000 7f2b59ffb640 1 -- 172.21.15.112:0/2740563253 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f2b600bc1d0 con 0x7f2b6406d2b0 2024-09-21T00:26:04.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.265+0000 7f0119ffb640 1 --2- 172.21.15.112:0/78355995 >> [v2:172.21.15.112:6824/407933616,v1:172.21.15.112:6825/407933616] conn(0x7f0108082430 0x7f01080848c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:04.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.265+0000 7f2b68c9b640 1 --2- 172.21.15.112:0/2740563253 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2b48078be0 0x7f2b4807b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:04.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.265+0000 7f7ff17fa640 1 --2- 172.21.15.112:0/3788287230 >> [v2:172.21.15.187:6810/1494387657,v1:172.21.15.187:6811/1494387657] conn(0x7f7fd8082420 0x7f7fd80848b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:04.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.265+0000 7f7ff17fa640 1 -- 172.21.15.112:0/3788287230 --> [v2:172.21.15.187:6810/1494387657,v1:172.21.15.187:6811/1494387657] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7fd8084f80 con 0x7f7fd8082420 2024-09-21T00:26:04.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.265+0000 7f4af2ffd640 1 --2- 172.21.15.112:0/950694403 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4ae4078c60 0x7f4ae407b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:04.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.266+0000 7f7ff17fa640 1 -- 172.21.15.112:0/3788287230 <== mon.2 v2:172.21.15.187:3301/0 6 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7fe80c3050 con 0x7f7ff40cf4f0 2024-09-21T00:26:04.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.266+0000 7f4eadf2c640 1 --2- 172.21.15.112:0/2010927137 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4e8c078c60 0x7f4e8c07b170 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4e98003bd0 tx=0x7f4e980079b0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:04.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.265+0000 7ff072ffd640 1 -- 172.21.15.112:0/1194921374 --> [v2:172.21.15.112:6816/4120164806,v1:172.21.15.112:6817/4120164806] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff06c004e70 con 0x7ff0600823a0 2024-09-21T00:26:04.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.266+0000 7f2b59ffb640 1 --2- 172.21.15.112:0/2740563253 >> [v2:172.21.15.187:6826/3593144721,v1:172.21.15.187:6827/3593144721] conn(0x7f2b480823b0 0x7f2b48084840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:04.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.266+0000 7f2b59ffb640 1 -- 172.21.15.112:0/2740563253 --> [v2:172.21.15.187:6826/3593144721,v1:172.21.15.187:6827/3593144721] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2b60005310 con 0x7f2b480823b0 2024-09-21T00:26:04.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.266+0000 7f2b59ffb640 1 -- 172.21.15.112:0/2740563253 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2b600bc6b0 con 0x7f2b6406d2b0 2024-09-21T00:26:04.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.265+0000 7ff072ffd640 1 -- 172.21.15.112:0/1194921374 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff06c09b0d0 con 0x7ff074056dd0 2024-09-21T00:26:04.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.266+0000 7f4af2ffd640 1 -- 172.21.15.112:0/950694403 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f4af80a33e0 con 0x7f4b0415ac20 2024-09-21T00:26:04.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.265+0000 7ff0824fa640 1 --2- 172.21.15.112:0/1194921374 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff060078be0 0x7ff06007b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:04.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.266+0000 7f7ff9023640 1 --2- 172.21.15.112:0/3788287230 >> [v2:172.21.15.187:6810/1494387657,v1:172.21.15.187:6811/1494387657] conn(0x7f7fd8082420 0x7f7fd80848b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:04.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.266+0000 7f012989e640 1 --2- 172.21.15.112:0/78355995 >> [v2:172.21.15.112:6824/407933616,v1:172.21.15.112:6825/407933616] conn(0x7f0108082430 0x7f01080848c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:04.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.266+0000 7f4b0259c640 1 --2- 172.21.15.112:0/950694403 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4ae4078c60 0x7f4ae407b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:04.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.266+0000 7f7ff8822640 1 --2- 172.21.15.112:0/3788287230 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f7fd8078c60 0x7f7fd807b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:04.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.266+0000 7f0119ffb640 1 -- 172.21.15.112:0/78355995 --> [v2:172.21.15.112:6824/407933616,v1:172.21.15.112:6825/407933616] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0120019ec0 con 0x7f0108082430 2024-09-21T00:26:04.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.266+0000 7f012889c640 1 --2- 172.21.15.112:0/78355995 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f0108078c60 0x7f010807b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:04.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.265+0000 7f77137fe640 1 -- 172.21.15.112:0/2912969700 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f7720041030 con 0x7f772c0cfbf0 2024-09-21T00:26:04.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.266+0000 7f77137fe640 1 --2- 172.21.15.112:0/2912969700 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f7718078c60 0x7f771807b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:04.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.266+0000 7f4af2ffd640 1 --2- 172.21.15.112:0/950694403 >> [v2:172.21.15.112:6808/755760621,v1:172.21.15.112:6809/755760621] conn(0x7f4ae4082430 0x7f4ae40848c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:04.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.266+0000 7f2b68c9b640 1 --2- 172.21.15.112:0/2740563253 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2b48078be0 0x7f2b4807b0f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2b641f97f0 tx=0x7f2b5c0095e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:04.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.266+0000 7f4b02d9d640 1 --2- 172.21.15.112:0/950694403 >> [v2:172.21.15.112:6808/755760621,v1:172.21.15.112:6809/755760621] conn(0x7f4ae4082430 0x7f4ae40848c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:04.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.266+0000 7f77137fe640 1 -- 172.21.15.112:0/2912969700 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f77200bbaf0 con 0x7f772c0cfbf0 2024-09-21T00:26:04.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.266+0000 7f772affd640 1 --2- 172.21.15.112:0/2912969700 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f7718078c60 0x7f771807b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:04.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.266+0000 7f77137fe640 1 --2- 172.21.15.112:0/2912969700 >> [v2:172.21.15.187:6802/522077560,v1:172.21.15.187:6803/522077560] conn(0x7f77180823b0 0x7f7718084840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:04.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.266+0000 7f77137fe640 1 -- 172.21.15.112:0/2912969700 --> [v2:172.21.15.187:6802/522077560,v1:172.21.15.187:6803/522077560] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7720005310 con 0x7f77180823b0 2024-09-21T00:26:04.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.266+0000 7f77137fe640 1 -- 172.21.15.112:0/2912969700 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f772007fbd0 con 0x7f772c0cfbf0 2024-09-21T00:26:04.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.266+0000 7f2b69c9d640 1 --2- 172.21.15.112:0/2740563253 >> [v2:172.21.15.187:6826/3593144721,v1:172.21.15.187:6827/3593144721] conn(0x7f2b480823b0 0x7f2b48084840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:04.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.266+0000 7f7ff9023640 1 --2- 172.21.15.112:0/3788287230 >> [v2:172.21.15.187:6810/1494387657,v1:172.21.15.187:6811/1494387657] conn(0x7f7fd8082420 0x7f7fd80848b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:04.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.267+0000 7f012989e640 1 --2- 172.21.15.112:0/78355995 >> [v2:172.21.15.112:6824/407933616,v1:172.21.15.112:6825/407933616] conn(0x7f0108082430 0x7f01080848c0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:04.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.267+0000 7ff0824fa640 1 --2- 172.21.15.112:0/1194921374 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff060078be0 0x7ff06007b0f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff078034000 tx=0x7ff0780046c0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:04.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.267+0000 7fa9fed9d640 1 --2- 172.21.15.112:0/3971453688 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fa9e8078c60 0x7fa9e807b170 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7faa0007acc0 tx=0x7faa00070040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:04.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.267+0000 7f4b0259c640 1 --2- 172.21.15.112:0/950694403 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4ae4078c60 0x7f4ae407b170 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4af4009230 tx=0x7f4af4004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:04.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.267+0000 7f4af2ffd640 1 -- 172.21.15.112:0/950694403 --> [v2:172.21.15.112:6808/755760621,v1:172.21.15.112:6809/755760621] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4ae4084f90 con 0x7f4ae4082430 2024-09-21T00:26:04.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.267+0000 7f4af2ffd640 1 -- 172.21.15.112:0/950694403 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4af80a37d0 con 0x7f4b0415ac20 2024-09-21T00:26:04.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.267+0000 7fa9ff59e640 1 --2- 172.21.15.112:0/3971453688 >> [v2:172.21.15.187:6818/1624699842,v1:172.21.15.187:6819/1624699842] conn(0x7fa9e8082420 0x7fa9e80848b0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:04.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.267+0000 7ff082cfb640 1 --2- 172.21.15.112:0/1194921374 >> [v2:172.21.15.112:6816/4120164806,v1:172.21.15.112:6817/4120164806] conn(0x7ff0600823a0 0x7ff060084830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:04.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.267+0000 7f772b7fe640 1 --2- 172.21.15.112:0/2912969700 >> [v2:172.21.15.187:6802/522077560,v1:172.21.15.187:6803/522077560] conn(0x7f77180823b0 0x7f7718084840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:04.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.267+0000 7f0119ffb640 1 -- 172.21.15.112:0/78355995 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f012009b3d0 con 0x7f011c053710 2024-09-21T00:26:04.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.267+0000 7f4b02d9d640 1 --2- 172.21.15.112:0/950694403 >> [v2:172.21.15.112:6808/755760621,v1:172.21.15.112:6809/755760621] conn(0x7f4ae4082430 0x7f4ae40848c0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:04.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.267+0000 7f012889c640 1 --2- 172.21.15.112:0/78355995 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f0108078c60 0x7f010807b170 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0114036000 tx=0x7f0114004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:04.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.267+0000 7f7ff8822640 1 --2- 172.21.15.112:0/3788287230 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f7fd8078c60 0x7f7fd807b170 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7fec002260 tx=0x7f7fec001f10 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:04.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.267+0000 7ff082cfb640 1 --2- 172.21.15.112:0/1194921374 >> [v2:172.21.15.112:6816/4120164806,v1:172.21.15.112:6817/4120164806] conn(0x7ff0600823a0 0x7ff060084830 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:04.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.267+0000 7f772affd640 1 --2- 172.21.15.112:0/2912969700 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f7718078c60 0x7f771807b170 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f771c00bb50 tx=0x7f771c00b2e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:04.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.267+0000 7f2b69c9d640 1 --2- 172.21.15.112:0/2740563253 >> [v2:172.21.15.187:6826/3593144721,v1:172.21.15.187:6827/3593144721] conn(0x7f2b480823b0 0x7f2b48084840 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:04.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.267+0000 7f772b7fe640 1 --2- 172.21.15.112:0/2912969700 >> [v2:172.21.15.187:6802/522077560,v1:172.21.15.187:6803/522077560] conn(0x7f77180823b0 0x7f7718084840 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:04.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.268+0000 7f7ff17fa640 1 -- 172.21.15.112:0/3788287230 <== osd.6 v2:172.21.15.187:6810/1494387657 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7fd8084f80 con 0x7f7fd8082420 2024-09-21T00:26:04.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.268+0000 7fa9f77fe640 1 -- 172.21.15.112:0/3971453688 <== osd.7 v2:172.21.15.187:6818/1624699842 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa9ec069e50 con 0x7fa9e8082420 2024-09-21T00:26:04.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.268+0000 7f4af2ffd640 1 -- 172.21.15.112:0/950694403 <== osd.0 v2:172.21.15.112:6808/755760621 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4ae4084f90 con 0x7f4ae4082430 2024-09-21T00:26:04.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.268+0000 7f0119ffb640 1 -- 172.21.15.112:0/78355995 <== osd.2 v2:172.21.15.112:6824/407933616 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0120019ec0 con 0x7f0108082430 2024-09-21T00:26:04.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.268+0000 7f2b59ffb640 1 -- 172.21.15.112:0/2740563253 <== osd.5 v2:172.21.15.187:6826/3593144721 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2b60005310 con 0x7f2b480823b0 2024-09-21T00:26:04.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.268+0000 7f77137fe640 1 -- 172.21.15.112:0/2912969700 <== osd.4 v2:172.21.15.187:6802/522077560 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7720005310 con 0x7f77180823b0 2024-09-21T00:26:04.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.269+0000 7ff072ffd640 1 -- 172.21.15.112:0/1194921374 <== osd.3 v2:172.21.15.112:6816/4120164806 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff06c004e70 con 0x7ff0600823a0 2024-09-21T00:26:04.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.296+0000 7f4eaef2e640 1 -- 172.21.15.112:0/2010927137 --> [v2:172.21.15.112:6800/344932592,v1:172.21.15.112:6801/344932592] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4ea80cc950 con 0x7f4e8c0823b0 2024-09-21T00:26:04.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.296+0000 7f4e9e7fc640 1 -- 172.21.15.112:0/2010927137 <== osd.1 v2:172.21.15.112:6800/344932592 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f4ea80cc950 con 0x7f4e8c0823b0 2024-09-21T00:26:04.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.297+0000 7f4eaef2e640 1 -- 172.21.15.112:0/2010927137 >> [v2:172.21.15.112:6800/344932592,v1:172.21.15.112:6801/344932592] conn(0x7f4e8c0823b0 msgr2=0x7f4e8c084840 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:04.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.297+0000 7f4eaef2e640 1 --2- 172.21.15.112:0/2010927137 >> [v2:172.21.15.112:6800/344932592,v1:172.21.15.112:6801/344932592] conn(0x7f4e8c0823b0 0x7f4e8c084840 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:04.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.297+0000 7f4eaef2e640 1 -- 172.21.15.112:0/2010927137 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4e8c078c60 msgr2=0x7f4e8c07b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:04.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.297+0000 7f4eaef2e640 1 --2- 172.21.15.112:0/2010927137 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4e8c078c60 0x7f4e8c07b170 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4e98003bd0 tx=0x7f4e980079b0 comp rx=0 tx=0).stop 2024-09-21T00:26:04.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.298+0000 7f4eaef2e640 1 -- 172.21.15.112:0/2010927137 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4ea814df20 msgr2=0x7f4ea8144840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:04.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.298+0000 7f4eaef2e640 1 --2- 172.21.15.112:0/2010927137 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4ea814df20 0x7f4ea8144840 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f4ea000c770 tx=0x7f4ea0003b80 comp rx=0 tx=0).stop 2024-09-21T00:26:04.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.298+0000 7f4eae72d640 1 -- 172.21.15.112:0/2010927137 reap_dead start 2024-09-21T00:26:04.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.298+0000 7f4eaef2e640 1 -- 172.21.15.112:0/2010927137 shutdown_connections 2024-09-21T00:26:04.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.298+0000 7f4eaef2e640 1 -- 172.21.15.112:0/2010927137 >> 172.21.15.112:0/2010927137 conn(0x7f4ea80d05c0 msgr2=0x7f4ea815b2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:04.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.298+0000 7f4eaef2e640 1 -- 172.21.15.112:0/2010927137 shutdown_connections 2024-09-21T00:26:04.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.298+0000 7f4eaef2e640 1 -- 172.21.15.112:0/2010927137 wait complete. 2024-09-21T00:26:04.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.298+0000 7f7ffaa86640 1 -- 172.21.15.112:0/3788287230 --> [v2:172.21.15.187:6810/1494387657,v1:172.21.15.187:6811/1494387657] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7ff415bb60 con 0x7f7fd8082420 2024-09-21T00:26:04.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.298+0000 7f4b08a38640 1 -- 172.21.15.112:0/950694403 --> [v2:172.21.15.112:6808/755760621,v1:172.21.15.112:6809/755760621] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4ae8002d50 con 0x7f4ae4082430 2024-09-21T00:26:04.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.298+0000 7faa0495d640 1 -- 172.21.15.112:0/3971453688 --> [v2:172.21.15.187:6818/1624699842,v1:172.21.15.187:6819/1624699842] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa9f8006b50 con 0x7fa9e8082420 2024-09-21T00:26:04.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.299+0000 7f4af2ffd640 1 -- 172.21.15.112:0/950694403 <== osd.0 v2:172.21.15.112:6808/755760621 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f4ae8002d50 con 0x7f4ae4082430 2024-09-21T00:26:04.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.299+0000 7f7ff17fa640 1 -- 172.21.15.112:0/3788287230 <== osd.6 v2:172.21.15.187:6810/1494387657 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f7ff415bb60 con 0x7f7fd8082420 2024-09-21T00:26:04.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.299+0000 7fa9f77fe640 1 -- 172.21.15.112:0/3971453688 <== osd.7 v2:172.21.15.187:6818/1624699842 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fa9f8006b50 con 0x7fa9e8082420 2024-09-21T00:26:04.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.299+0000 7f7fdeffd640 1 -- 172.21.15.112:0/3788287230 >> [v2:172.21.15.187:6810/1494387657,v1:172.21.15.187:6811/1494387657] conn(0x7f7fd8082420 msgr2=0x7f7fd80848b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:04.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.299+0000 7f7fdeffd640 1 --2- 172.21.15.112:0/3788287230 >> [v2:172.21.15.187:6810/1494387657,v1:172.21.15.187:6811/1494387657] conn(0x7f7fd8082420 0x7f7fd80848b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:04.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.299+0000 7f772bfff640 1 -- 172.21.15.112:0/2912969700 --> [v2:172.21.15.187:6802/522077560,v1:172.21.15.187:6803/522077560] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f772c1f9f50 con 0x7f77180823b0 2024-09-21T00:26:04.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.300+0000 7f7fdeffd640 1 -- 172.21.15.112:0/3788287230 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f7fd8078c60 msgr2=0x7f7fd807b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:04.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.300+0000 7f7fdeffd640 1 --2- 172.21.15.112:0/3788287230 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f7fd8078c60 0x7f7fd807b170 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7fec002260 tx=0x7f7fec001f10 comp rx=0 tx=0).stop 2024-09-21T00:26:04.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.300+0000 7faa0495d640 1 -- 172.21.15.112:0/3971453688 >> [v2:172.21.15.187:6818/1624699842,v1:172.21.15.187:6819/1624699842] conn(0x7fa9e8082420 msgr2=0x7fa9e80848b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:04.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.300+0000 7faa0495d640 1 --2- 172.21.15.112:0/3971453688 >> [v2:172.21.15.187:6818/1624699842,v1:172.21.15.187:6819/1624699842] conn(0x7fa9e8082420 0x7fa9e80848b0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:04.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.300+0000 7f2b6a49e640 1 -- 172.21.15.112:0/2740563253 --> [v2:172.21.15.187:6826/3593144721,v1:172.21.15.187:6827/3593144721] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2b640b9b30 con 0x7f2b480823b0 2024-09-21T00:26:04.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.300+0000 7f4b08a38640 1 -- 172.21.15.112:0/950694403 >> [v2:172.21.15.112:6808/755760621,v1:172.21.15.112:6809/755760621] conn(0x7f4ae4082430 msgr2=0x7f4ae40848c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:04.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.300+0000 7f4b08a38640 1 --2- 172.21.15.112:0/950694403 >> [v2:172.21.15.112:6808/755760621,v1:172.21.15.112:6809/755760621] conn(0x7f4ae4082430 0x7f4ae40848c0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:04.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.300+0000 7f7fdeffd640 1 -- 172.21.15.112:0/3788287230 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7ff40cf4f0 msgr2=0x7f7ff413bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:04.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.300+0000 7f7fdeffd640 1 --2- 172.21.15.112:0/3788287230 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7ff40cf4f0 0x7f7ff413bac0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7fe8031000 tx=0x7f7fe8033f50 comp rx=0 tx=0).stop 2024-09-21T00:26:04.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.300+0000 7f7ff9023640 1 -- 172.21.15.112:0/3788287230 reap_dead start 2024-09-21T00:26:04.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.300+0000 7faa0495d640 1 -- 172.21.15.112:0/3971453688 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fa9e8078c60 msgr2=0x7fa9e807b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:04.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.300+0000 7faa0495d640 1 --2- 172.21.15.112:0/3971453688 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fa9e8078c60 0x7fa9e807b170 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7faa0007acc0 tx=0x7faa00070040 comp rx=0 tx=0).stop 2024-09-21T00:26:04.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.300+0000 7faa0495d640 1 -- 172.21.15.112:0/3971453688 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa9f8056200 msgr2=0x7fa9f804d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:04.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.300+0000 7f4b08a38640 1 -- 172.21.15.112:0/950694403 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4ae4078c60 msgr2=0x7f4ae407b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:04.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.300+0000 7f012b301640 1 -- 172.21.15.112:0/78355995 --> [v2:172.21.15.112:6824/407933616,v1:172.21.15.112:6825/407933616] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f011c039e90 con 0x7f0108082430 2024-09-21T00:26:04.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.300+0000 7ff08475e640 1 -- 172.21.15.112:0/1194921374 --> [v2:172.21.15.112:6816/4120164806,v1:172.21.15.112:6817/4120164806] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff064002dd0 con 0x7ff0600823a0 2024-09-21T00:26:04.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.300+0000 7f4b08a38640 1 --2- 172.21.15.112:0/950694403 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4ae4078c60 0x7f4ae407b170 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4af4009230 tx=0x7f4af4004730 comp rx=0 tx=0).stop 2024-09-21T00:26:04.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.300+0000 7f77137fe640 1 -- 172.21.15.112:0/2912969700 <== osd.4 v2:172.21.15.187:6802/522077560 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f7724012450 con 0x7f77180823b0 2024-09-21T00:26:04.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.300+0000 7faa0495d640 1 --2- 172.21.15.112:0/3971453688 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa9f8056200 0x7fa9f804d5e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa9ec031000 tx=0x7fa9ec033ba0 comp rx=0 tx=0).stop 2024-09-21T00:26:04.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.300+0000 7fa9ff59e640 1 -- 172.21.15.112:0/3971453688 reap_dead start 2024-09-21T00:26:04.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.300+0000 7faa0495d640 1 -- 172.21.15.112:0/3971453688 shutdown_connections 2024-09-21T00:26:04.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.300+0000 7faa0495d640 1 -- 172.21.15.112:0/3971453688 >> 172.21.15.112:0/3971453688 conn(0x7fa9f801eac0 msgr2=0x7fa9f80d5420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:04.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.300+0000 7f4b08a38640 1 -- 172.21.15.112:0/950694403 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4b0415ac20 msgr2=0x7f4b040b55b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:04.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.300+0000 7f4b08a38640 1 --2- 172.21.15.112:0/950694403 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4b0415ac20 0x7f4b040b55b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f4af80015d0 tx=0x7f4af8001a90 comp rx=0 tx=0).stop 2024-09-21T00:26:04.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.300+0000 7f4b02d9d640 1 -- 172.21.15.112:0/950694403 reap_dead start 2024-09-21T00:26:04.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.300+0000 7f2b59ffb640 1 -- 172.21.15.112:0/2740563253 <== osd.5 v2:172.21.15.187:6826/3593144721 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f2b640b9b30 con 0x7f2b480823b0 2024-09-21T00:26:04.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.300+0000 7f7fdeffd640 1 -- 172.21.15.112:0/3788287230 shutdown_connections 2024-09-21T00:26:04.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.300+0000 7f7fdeffd640 1 -- 172.21.15.112:0/3788287230 >> 172.21.15.112:0/3788287230 conn(0x7f7ff40cfec0 msgr2=0x7f7ff40d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:04.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.300+0000 7f0119ffb640 1 -- 172.21.15.112:0/78355995 <== osd.2 v2:172.21.15.112:6824/407933616 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f011c039e90 con 0x7f0108082430 2024-09-21T00:26:04.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.300+0000 7f4b08a38640 1 -- 172.21.15.112:0/950694403 shutdown_connections 2024-09-21T00:26:04.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.300+0000 7f4b08a38640 1 -- 172.21.15.112:0/950694403 >> 172.21.15.112:0/950694403 conn(0x7f4b040b1b90 msgr2=0x7f4b040b0cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:04.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.300+0000 7faa0495d640 1 -- 172.21.15.112:0/3971453688 shutdown_connections 2024-09-21T00:26:04.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.301+0000 7f772bfff640 1 -- 172.21.15.112:0/2912969700 >> [v2:172.21.15.187:6802/522077560,v1:172.21.15.187:6803/522077560] conn(0x7f77180823b0 msgr2=0x7f7718084840 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:04.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.301+0000 7f772bfff640 1 --2- 172.21.15.112:0/2912969700 >> [v2:172.21.15.187:6802/522077560,v1:172.21.15.187:6803/522077560] conn(0x7f77180823b0 0x7f7718084840 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:04.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.301+0000 7faa0495d640 1 -- 172.21.15.112:0/3971453688 wait complete. 2024-09-21T00:26:04.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.301+0000 7f4b08a38640 1 -- 172.21.15.112:0/950694403 shutdown_connections 2024-09-21T00:26:04.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.301+0000 7ff072ffd640 1 -- 172.21.15.112:0/1194921374 <== osd.3 v2:172.21.15.112:6816/4120164806 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7ff064002dd0 con 0x7ff0600823a0 2024-09-21T00:26:04.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.301+0000 7f4b08a38640 1 -- 172.21.15.112:0/950694403 wait complete. 2024-09-21T00:26:04.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.301+0000 7f772bfff640 1 -- 172.21.15.112:0/2912969700 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f7718078c60 msgr2=0x7f771807b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:04.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.301+0000 7f772bfff640 1 --2- 172.21.15.112:0/2912969700 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f7718078c60 0x7f771807b170 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f771c00bb50 tx=0x7f771c00b2e0 comp rx=0 tx=0).stop 2024-09-21T00:26:04.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.301+0000 7f772bfff640 1 -- 172.21.15.112:0/2912969700 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f772c0cfbf0 msgr2=0x7f772c1f97c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:04.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.301+0000 7f772bfff640 1 --2- 172.21.15.112:0/2912969700 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f772c0cfbf0 0x7f772c1f97c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f7720031000 tx=0x7f77200038a0 comp rx=0 tx=0).stop 2024-09-21T00:26:04.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.301+0000 7f772b7fe640 1 -- 172.21.15.112:0/2912969700 reap_dead start 2024-09-21T00:26:04.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.301+0000 7f772bfff640 1 -- 172.21.15.112:0/2912969700 shutdown_connections 2024-09-21T00:26:04.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.301+0000 7f7fdeffd640 1 -- 172.21.15.112:0/3788287230 shutdown_connections 2024-09-21T00:26:04.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.301+0000 7f772bfff640 1 -- 172.21.15.112:0/2912969700 >> 172.21.15.112:0/2912969700 conn(0x7f772c0d42e0 msgr2=0x7f772c0b79a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:04.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.301+0000 7f2b6a49e640 1 -- 172.21.15.112:0/2740563253 >> [v2:172.21.15.187:6826/3593144721,v1:172.21.15.187:6827/3593144721] conn(0x7f2b480823b0 msgr2=0x7f2b48084840 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:04.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.301+0000 7f2b6a49e640 1 --2- 172.21.15.112:0/2740563253 >> [v2:172.21.15.187:6826/3593144721,v1:172.21.15.187:6827/3593144721] conn(0x7f2b480823b0 0x7f2b48084840 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:04.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.301+0000 7ff08475e640 1 -- 172.21.15.112:0/1194921374 >> [v2:172.21.15.112:6816/4120164806,v1:172.21.15.112:6817/4120164806] conn(0x7ff0600823a0 msgr2=0x7ff060084830 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:04.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.301+0000 7ff08475e640 1 --2- 172.21.15.112:0/1194921374 >> [v2:172.21.15.112:6816/4120164806,v1:172.21.15.112:6817/4120164806] conn(0x7ff0600823a0 0x7ff060084830 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:04.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.301+0000 7f772bfff640 1 -- 172.21.15.112:0/2912969700 shutdown_connections 2024-09-21T00:26:04.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.301+0000 7f7fdeffd640 1 -- 172.21.15.112:0/3788287230 wait complete. 2024-09-21T00:26:04.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.301+0000 7f772bfff640 1 -- 172.21.15.112:0/2912969700 wait complete. 2024-09-21T00:26:04.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.301+0000 7ff08475e640 1 -- 172.21.15.112:0/1194921374 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff060078be0 msgr2=0x7ff06007b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:04.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.301+0000 7f2b6a49e640 1 -- 172.21.15.112:0/2740563253 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2b48078be0 msgr2=0x7f2b4807b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:04.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.301+0000 7f2b6a49e640 1 --2- 172.21.15.112:0/2740563253 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2b48078be0 0x7f2b4807b0f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2b641f97f0 tx=0x7f2b5c0095e0 comp rx=0 tx=0).stop 2024-09-21T00:26:04.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.301+0000 7f012b301640 1 -- 172.21.15.112:0/78355995 >> [v2:172.21.15.112:6824/407933616,v1:172.21.15.112:6825/407933616] conn(0x7f0108082430 msgr2=0x7f01080848c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:04.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.301+0000 7f012b301640 1 --2- 172.21.15.112:0/78355995 >> [v2:172.21.15.112:6824/407933616,v1:172.21.15.112:6825/407933616] conn(0x7f0108082430 0x7f01080848c0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:04.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.301+0000 7ff08475e640 1 --2- 172.21.15.112:0/1194921374 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff060078be0 0x7ff06007b0f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff078034000 tx=0x7ff0780046c0 comp rx=0 tx=0).stop 2024-09-21T00:26:04.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.301+0000 7ff08475e640 1 -- 172.21.15.112:0/1194921374 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff074056dd0 msgr2=0x7ff07417d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:04.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.301+0000 7ff08475e640 1 --2- 172.21.15.112:0/1194921374 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff074056dd0 0x7ff07417d8a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff0740e3550 tx=0x7ff06c00c820 comp rx=0 tx=0).stop 2024-09-21T00:26:04.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.301+0000 7f2b6a49e640 1 -- 172.21.15.112:0/2740563253 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2b6406d2b0 msgr2=0x7f2b641f5bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:04.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.301+0000 7ff082cfb640 1 -- 172.21.15.112:0/1194921374 reap_dead start 2024-09-21T00:26:04.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.301+0000 7f2b6a49e640 1 --2- 172.21.15.112:0/2740563253 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2b6406d2b0 0x7f2b641f5bd0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f2b60034040 tx=0x7f2b60003ba0 comp rx=0 tx=0).stop 2024-09-21T00:26:04.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.302+0000 7f2b69c9d640 1 -- 172.21.15.112:0/2740563253 reap_dead start 2024-09-21T00:26:04.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.302+0000 7f2b6a49e640 1 -- 172.21.15.112:0/2740563253 shutdown_connections 2024-09-21T00:26:04.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.302+0000 7f2b6a49e640 1 -- 172.21.15.112:0/2740563253 >> 172.21.15.112:0/2740563253 conn(0x7f2b640d1960 msgr2=0x7f2b640b7a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:04.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.302+0000 7ff08475e640 1 -- 172.21.15.112:0/1194921374 shutdown_connections 2024-09-21T00:26:04.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.302+0000 7ff08475e640 1 -- 172.21.15.112:0/1194921374 >> 172.21.15.112:0/1194921374 conn(0x7ff07403a620 msgr2=0x7ff07403c8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:04.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.302+0000 7ff08475e640 1 -- 172.21.15.112:0/1194921374 shutdown_connections 2024-09-21T00:26:04.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.302+0000 7f2b6a49e640 1 -- 172.21.15.112:0/2740563253 shutdown_connections 2024-09-21T00:26:04.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.302+0000 7f012b301640 1 -- 172.21.15.112:0/78355995 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f0108078c60 msgr2=0x7f010807b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:04.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.302+0000 7f012b301640 1 --2- 172.21.15.112:0/78355995 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f0108078c60 0x7f010807b170 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0114036000 tx=0x7f0114004730 comp rx=0 tx=0).stop 2024-09-21T00:26:04.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.302+0000 7f012b301640 1 -- 172.21.15.112:0/78355995 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f011c053710 msgr2=0x7f011c0db5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:04.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.302+0000 7f012b301640 1 --2- 172.21.15.112:0/78355995 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f011c053710 0x7f011c0db5a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f012000c760 tx=0x7f012000cc20 comp rx=0 tx=0).stop 2024-09-21T00:26:04.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.302+0000 7f012989e640 1 -- 172.21.15.112:0/78355995 reap_dead start 2024-09-21T00:26:04.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.302+0000 7f012b301640 1 -- 172.21.15.112:0/78355995 shutdown_connections 2024-09-21T00:26:04.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.302+0000 7f012b301640 1 -- 172.21.15.112:0/78355995 >> 172.21.15.112:0/78355995 conn(0x7f011c01e260 msgr2=0x7f011c053b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:04.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.302+0000 7ff08475e640 1 -- 172.21.15.112:0/1194921374 wait complete. 2024-09-21T00:26:04.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.302+0000 7f2b6a49e640 1 -- 172.21.15.112:0/2740563253 wait complete. 2024-09-21T00:26:04.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.302+0000 7f012b301640 1 -- 172.21.15.112:0/78355995 shutdown_connections 2024-09-21T00:26:04.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.302+0000 7f012b301640 1 -- 172.21.15.112:0/78355995 wait complete. 2024-09-21T00:26:04.341 INFO:teuthology.orchestra.run.smithi112.stdout:55834574857 2024-09-21T00:26:04.341 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-21T00:26:04.342 INFO:teuthology.orchestra.run.smithi112.stdout:64424509449 2024-09-21T00:26:04.342 INFO:teuthology.orchestra.run.smithi112.stdout:64424509449 2024-09-21T00:26:04.343 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-21T00:26:04.343 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-21T00:26:04.344 INFO:teuthology.orchestra.run.smithi112.stdout:64424509447 2024-09-21T00:26:04.344 INFO:teuthology.orchestra.run.smithi112.stdout:64424509448 2024-09-21T00:26:04.345 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-21T00:26:04.345 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-21T00:26:04.346 INFO:teuthology.orchestra.run.smithi112.stdout:64424509449 2024-09-21T00:26:04.346 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-21T00:26:04.347 INFO:teuthology.orchestra.run.smithi112.stdout:64424509449 2024-09-21T00:26:04.347 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-21T00:26:04.348 INFO:teuthology.orchestra.run.smithi112.stdout:64424509448 2024-09-21T00:26:04.348 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-21T00:26:04.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.599+0000 7fed867fc640 1 -- 172.21.15.112:0/299264798 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed7c031bb0 con 0x7fed88155780 2024-09-21T00:26:04.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.600+0000 7fa683fff640 1 -- 172.21.15.112:0/2689605933 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa68c06a5a0 con 0x7fa678004d40 2024-09-21T00:26:04.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.600+0000 7fed8ca5b640 1 -- 172.21.15.112:0/299264798 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fed88155780 msgr2=0x7fed8806da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:04.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.600+0000 7fed8ca5b640 1 --2- 172.21.15.112:0/299264798 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fed88155780 0x7fed8806da60 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fed7c009860 tx=0x7fed7c02a510 comp rx=0 tx=0).stop 2024-09-21T00:26:04.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.600+0000 7fed8ca5b640 1 -- 172.21.15.112:0/299264798 shutdown_connections 2024-09-21T00:26:04.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.600+0000 7fed8ca5b640 1 --2- 172.21.15.112:0/299264798 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fed88155780 0x7fed8806da60 secure :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fed7c009860 tx=0x7fed7c02a510 comp rx=0 tx=0).stop 2024-09-21T00:26:04.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.600+0000 7fed8ca5b640 1 -- 172.21.15.112:0/299264798 >> 172.21.15.112:0/299264798 conn(0x7fed880cfec0 msgr2=0x7fed880d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:04.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.600+0000 7fed8ca5b640 1 -- 172.21.15.112:0/299264798 shutdown_connections 2024-09-21T00:26:04.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.600+0000 7fed8ca5b640 1 -- 172.21.15.112:0/299264798 wait complete. 2024-09-21T00:26:04.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.601+0000 7fb0ae5ac640 1 -- 172.21.15.112:0/1062210165 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb0a0054360 msgr2=0x7fb0a0057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:04.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.601+0000 7fb0ae5ac640 1 --2- 172.21.15.112:0/1062210165 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb0a0054360 0x7fb0a0057c80 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fb0a0006880 tx=0x7fb0a80a62e0 comp rx=0 tx=0).stop 2024-09-21T00:26:04.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.601+0000 7fa6923a4640 1 -- 172.21.15.112:0/2689605933 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa678004d40 msgr2=0x7fa678005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:04.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.601+0000 7fa6923a4640 1 --2- 172.21.15.112:0/2689605933 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa678004d40 0x7fa678005150 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa68c06bbc0 tx=0x7fa678005830 comp rx=0 tx=0).stop 2024-09-21T00:26:04.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.601+0000 7fb0ae5ac640 1 -- 172.21.15.112:0/1062210165 shutdown_connections 2024-09-21T00:26:04.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.601+0000 7fb0ae5ac640 1 --2- 172.21.15.112:0/1062210165 >> v2:172.21.15.187:3300/0 conn(0x7fb0a00e2f40 0x7fb0a003c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:04.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.601+0000 7fb0ae5ac640 1 --2- 172.21.15.112:0/1062210165 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb0a0054360 0x7fb0a0057c80 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:04.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.601+0000 7fb0ae5ac640 1 -- 172.21.15.112:0/1062210165 >> 172.21.15.112:0/1062210165 conn(0x7fb0a003a5f0 msgr2=0x7fb0a003aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:04.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.601+0000 7fa691ba3640 1 -- 172.21.15.112:0/2689605933 reap_dead start 2024-09-21T00:26:04.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.601+0000 7fa6923a4640 1 -- 172.21.15.112:0/2689605933 shutdown_connections 2024-09-21T00:26:04.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.601+0000 7fa6923a4640 1 -- 172.21.15.112:0/2689605933 >> 172.21.15.112:0/2689605933 conn(0x7fa68401eab0 msgr2=0x7fa68403a6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:04.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.601+0000 7fa6923a4640 1 -- 172.21.15.112:0/2689605933 shutdown_connections 2024-09-21T00:26:04.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.602+0000 7fb0ae5ac640 1 -- 172.21.15.112:0/1062210165 shutdown_connections 2024-09-21T00:26:04.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.602+0000 7fb0ae5ac640 1 -- 172.21.15.112:0/1062210165 wait complete. 2024-09-21T00:26:04.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.601+0000 7fa6923a4640 1 -- 172.21.15.112:0/2689605933 wait complete. 2024-09-21T00:26:04.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.603+0000 7f750e92b640 1 -- 172.21.15.112:0/1889546681 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75080706e0 msgr2=0x7f750815e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:04.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.603+0000 7fed8ca5b640 1 Processor -- start 2024-09-21T00:26:04.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.603+0000 7f750e92b640 1 --2- 172.21.15.112:0/1889546681 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75080706e0 0x7f750815e920 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f7500005fd0 tx=0x7f750002a5d0 comp rx=0 tx=0).stop 2024-09-21T00:26:04.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.603+0000 7fac68627640 1 -- 172.21.15.112:0/1428450991 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac580df1b0 msgr2=0x7fac58005c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:04.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.603+0000 7fac68627640 1 --2- 172.21.15.112:0/1428450991 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac580df1b0 0x7fac58005c90 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fac5c007350 tx=0x7fac5c02b150 comp rx=0 tx=0).stop 2024-09-21T00:26:04.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.604+0000 7fed8ca5b640 1 -- start start 2024-09-21T00:26:04.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.604+0000 7fac68627640 1 -- 172.21.15.112:0/1428450991 shutdown_connections 2024-09-21T00:26:04.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.604+0000 7fac68627640 1 --2- 172.21.15.112:0/1428450991 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac580df1b0 0x7fac58005c90 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:04.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.604+0000 7fac68627640 1 -- 172.21.15.112:0/1428450991 >> 172.21.15.112:0/1428450991 conn(0x7fac58051450 msgr2=0x7fac58051890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:04.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.604+0000 7fac68627640 1 -- 172.21.15.112:0/1428450991 shutdown_connections 2024-09-21T00:26:04.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.604+0000 7f750e92b640 1 -- 172.21.15.112:0/1889546681 shutdown_connections 2024-09-21T00:26:04.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.604+0000 7f750e92b640 1 --2- 172.21.15.112:0/1889546681 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7508071260 0x7f75080b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:04.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.604+0000 7f750e92b640 1 --2- 172.21.15.112:0/1889546681 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75080706e0 0x7f750815e920 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:04.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.604+0000 7f750e92b640 1 -- 172.21.15.112:0/1889546681 >> 172.21.15.112:0/1889546681 conn(0x7f75080cdca0 msgr2=0x7f75080ce0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:04.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.604+0000 7fac68627640 1 -- 172.21.15.112:0/1428450991 wait complete. 2024-09-21T00:26:04.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.604+0000 7f750e92b640 1 -- 172.21.15.112:0/1889546681 shutdown_connections 2024-09-21T00:26:04.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.604+0000 7f750e92b640 1 -- 172.21.15.112:0/1889546681 wait complete. 2024-09-21T00:26:04.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.604+0000 7f117ee15640 1 -- 172.21.15.112:0/59421863 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f11780b5260 msgr2=0x7f117815ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:04.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.604+0000 7f00bc8eb640 1 -- 172.21.15.112:0/525973908 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f00b0036b40 msgr2=0x7f00b00df180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:04.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.604+0000 7f117ee15640 1 --2- 172.21.15.112:0/59421863 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f11780b5260 0x7f117815ba80 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1178075150 tx=0x7f116c02c4f0 comp rx=0 tx=0).stop 2024-09-21T00:26:04.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.604+0000 7f00bc8eb640 1 --2- 172.21.15.112:0/525973908 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f00b0036b40 0x7f00b00df180 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f00a00083e0 tx=0x7f00a002c5c0 comp rx=0 tx=0).stop 2024-09-21T00:26:04.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.604+0000 7f117ee15640 1 -- 172.21.15.112:0/59421863 shutdown_connections 2024-09-21T00:26:04.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.605+0000 7f00bc8eb640 1 -- 172.21.15.112:0/525973908 shutdown_connections 2024-09-21T00:26:04.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.605+0000 7f00bc8eb640 1 --2- 172.21.15.112:0/525973908 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f00b0036b40 0x7f00b00df180 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:04.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.605+0000 7f00bc8eb640 1 -- 172.21.15.112:0/525973908 >> 172.21.15.112:0/525973908 conn(0x7f00b0051420 msgr2=0x7f00b0051860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:04.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.605+0000 7fdbad9e6640 1 -- 172.21.15.112:0/520703194 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb90003a00 msgr2=0x7fdb90003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:04.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.605+0000 7fdbad9e6640 1 --2- 172.21.15.112:0/520703194 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb90003a00 0x7fdb90003e50 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c006fd0 tx=0x7fdb90005830 comp rx=0 tx=0).stop 2024-09-21T00:26:04.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.605+0000 7fa6923a4640 1 Processor -- start 2024-09-21T00:26:04.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.605+0000 7f00bc8eb640 1 -- 172.21.15.112:0/525973908 shutdown_connections 2024-09-21T00:26:04.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.605+0000 7fb0ae5ac640 1 Processor -- start 2024-09-21T00:26:04.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.605+0000 7f117ee15640 1 --2- 172.21.15.112:0/59421863 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f11780b5260 0x7f117815ba80 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:04.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.605+0000 7f117ee15640 1 --2- 172.21.15.112:0/59421863 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f117815bfc0 0x7f117815f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:04.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.605+0000 7f117ee15640 1 --2- 172.21.15.112:0/59421863 >> v2:172.21.15.112:3300/0 conn(0x7f117806def0 0x7f117806f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:04.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.605+0000 7f117ee15640 1 -- 172.21.15.112:0/59421863 >> 172.21.15.112:0/59421863 conn(0x7f11780cdca0 msgr2=0x7f11780ce0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:04.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.605+0000 7fb0ae5ac640 1 -- start start 2024-09-21T00:26:04.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.605+0000 7fdbad9e6640 1 -- 172.21.15.112:0/520703194 shutdown_connections 2024-09-21T00:26:04.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.605+0000 7fdbad9e6640 1 --2- 172.21.15.112:0/520703194 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb90003a00 0x7fdb90003e50 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:04.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.605+0000 7fdbad9e6640 1 -- 172.21.15.112:0/520703194 >> 172.21.15.112:0/520703194 conn(0x7fdba003a620 msgr2=0x7fdba003aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:04.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.605+0000 7f00bc8eb640 1 -- 172.21.15.112:0/525973908 wait complete. 2024-09-21T00:26:04.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.605+0000 7f117ee15640 1 -- 172.21.15.112:0/59421863 shutdown_connections 2024-09-21T00:26:04.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.605+0000 7f117ee15640 1 -- 172.21.15.112:0/59421863 wait complete. 2024-09-21T00:26:04.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.606+0000 7fed8ca5b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fed88070050 0x7fed881fc500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:04.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.606+0000 7fed8ca5b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fed880aef90 0x7fed880ace10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:04.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.606+0000 7fed8ca5b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fed88162570 0x7fed880ad350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:04.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.606+0000 7fed877fe640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fed88070050 0x7fed881fc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:04.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.606+0000 7fed877fe640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fed88070050 0x7fed881fc500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:59614/0 (socket says 172.21.15.112:59614) 2024-09-21T00:26:04.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.607+0000 7fed877fe640 1 -- 172.21.15.112:0/3442869174 learned_addr learned my addr 172.21.15.112:0/3442869174 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:26:04.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.607+0000 7fed87fff640 1 --2- 172.21.15.112:0/3442869174 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fed88162570 0x7fed880ad350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:04.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.605+0000 7fdbad9e6640 1 -- 172.21.15.112:0/520703194 shutdown_connections 2024-09-21T00:26:04.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.605+0000 7fdbad9e6640 1 -- 172.21.15.112:0/520703194 wait complete. 2024-09-21T00:26:04.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.607+0000 7fed8ca5b640 1 -- 172.21.15.112:0/3442869174 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fed880ce3c0 con 0x7fed880aef90 2024-09-21T00:26:04.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.607+0000 7fed8ca5b640 1 -- 172.21.15.112:0/3442869174 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fed880ce240 con 0x7fed88162570 2024-09-21T00:26:04.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.607+0000 7fed8ca5b640 1 -- 172.21.15.112:0/3442869174 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fed880ce540 con 0x7fed88070050 2024-09-21T00:26:04.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.607+0000 7fac68627640 1 Processor -- start 2024-09-21T00:26:04.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.605+0000 7fa6923a4640 1 -- start start 2024-09-21T00:26:04.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.607+0000 7fb0ae5ac640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb0a0054360 0x7fb0a00d6f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:04.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.607+0000 7fed877fe640 1 -- 172.21.15.112:0/3442869174 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fed88162570 msgr2=0x7fed880ad350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:04.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.607+0000 7fed877fe640 1 --2- 172.21.15.112:0/3442869174 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fed88162570 0x7fed880ad350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:04.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.607+0000 7fed877fe640 1 -- 172.21.15.112:0/3442869174 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fed880aef90 msgr2=0x7fed880ace10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T00:26:04.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.607+0000 7fb0ae5ac640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0a0056da0 0x7fb0a0050570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:04.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.607+0000 7fac68627640 1 -- start start 2024-09-21T00:26:04.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.607+0000 7fed86ffd640 1 --2- 172.21.15.112:0/3442869174 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fed880aef90 0x7fed880ace10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:04.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.607+0000 7fa6923a4640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa6840cec50 0x7fa6840cf060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:04.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.607+0000 7fed877fe640 1 --2- 172.21.15.112:0/3442869174 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fed880aef90 0x7fed880ace10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:04.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.607+0000 7fed877fe640 1 -- 172.21.15.112:0/3442869174 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed88203570 con 0x7fed88070050 2024-09-21T00:26:04.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.607+0000 7fa6923a4640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa6840da2d0 0x7fa6840da6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:04.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.607+0000 7fa6923a4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa684049020 0x7fa684049430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:04.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.608+0000 7fed86ffd640 1 --2- 172.21.15.112:0/3442869174 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fed880aef90 0x7fed880ace10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:26:04.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.607+0000 7fa6923a4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa6840358c0 con 0x7fa684049020 2024-09-21T00:26:04.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.607+0000 7fa6923a4640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa684035740 con 0x7fa6840da2d0 2024-09-21T00:26:04.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.607+0000 7fa6923a4640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa684032410 con 0x7fa6840cec50 2024-09-21T00:26:04.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.608+0000 7fa691ba3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa684049020 0x7fa684049430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:04.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.608+0000 7fa691ba3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa684049020 0x7fa684049430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45900/0 (socket says 172.21.15.112:45900) 2024-09-21T00:26:04.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.608+0000 7fa691ba3640 1 -- 172.21.15.112:0/2824632459 learned_addr learned my addr 172.21.15.112:0/2824632459 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:26:04.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.608+0000 7fed877fe640 1 --2- 172.21.15.112:0/3442869174 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fed88070050 0x7fed881fc500 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fed7800c3a0 tx=0x7fed7800c860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:04.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.608+0000 7fed6ffff640 1 -- 172.21.15.112:0/3442869174 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed78011790 con 0x7fed88070050 2024-09-21T00:26:04.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.608+0000 7fb0ae5ac640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb0a00e2f40 0x7fb0a00e5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:04.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.608+0000 7fb0acda9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0a0056da0 0x7fb0a0050570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:04.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.608+0000 7fb0ae5ac640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb0a0007140 con 0x7fb0a0056da0 2024-09-21T00:26:04.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.608+0000 7fb0ae5ac640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb0a0006fc0 con 0x7fb0a00e2f40 2024-09-21T00:26:04.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.608+0000 7fb0ae5ac640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fb0a00072c0 con 0x7fb0a0054360 2024-09-21T00:26:04.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.608+0000 7f117ee15640 1 Processor -- start 2024-09-21T00:26:04.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.608+0000 7f00bc8eb640 1 Processor -- start 2024-09-21T00:26:04.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.608+0000 7fed8ca5b640 1 -- 172.21.15.112:0/3442869174 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed881fca40 con 0x7fed88070050 2024-09-21T00:26:04.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.608+0000 7fb0addab640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb0a00e2f40 0x7fb0a00e5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:04.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.608+0000 7fb0addab640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb0a00e2f40 0x7fb0a00e5140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:43454/0 (socket says 172.21.15.112:43454) 2024-09-21T00:26:04.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.608+0000 7fb0addab640 1 -- 172.21.15.112:0/1448321289 learned_addr learned my addr 172.21.15.112:0/1448321289 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:26:04.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.608+0000 7f117ee15640 1 -- start start 2024-09-21T00:26:04.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.608+0000 7fed6ffff640 1 -- 172.21.15.112:0/3442869174 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fed78017070 con 0x7fed88070050 2024-09-21T00:26:04.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.609+0000 7fdbad9e6640 1 Processor -- start 2024-09-21T00:26:04.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.609+0000 7fa691ba3640 1 -- 172.21.15.112:0/2824632459 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa6840cec50 msgr2=0x7fa6840cf060 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T00:26:04.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.609+0000 7fa691ba3640 1 --2- 172.21.15.112:0/2824632459 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa6840cec50 0x7fa6840cf060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:04.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.609+0000 7fa691ba3640 1 -- 172.21.15.112:0/2824632459 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa6840da2d0 msgr2=0x7fa6840da6e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T00:26:04.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.609+0000 7fa691ba3640 1 --2- 172.21.15.112:0/2824632459 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa6840da2d0 0x7fa6840da6e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:04.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.609+0000 7fa691ba3640 1 -- 172.21.15.112:0/2824632459 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6840daf40 con 0x7fa684049020 2024-09-21T00:26:04.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.609+0000 7fed8ca5b640 1 -- 172.21.15.112:0/3442869174 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fed88160780 con 0x7fed88070050 2024-09-21T00:26:04.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.609+0000 7fed6ffff640 1 -- 172.21.15.112:0/3442869174 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed78011cc0 con 0x7fed88070050 2024-09-21T00:26:04.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.609+0000 7fb0addab640 1 -- 172.21.15.112:0/1448321289 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb0a0054360 msgr2=0x7fb0a00d6f60 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T00:26:04.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.609+0000 7fb0addab640 1 --2- 172.21.15.112:0/1448321289 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb0a0054360 0x7fb0a00d6f60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:04.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.609+0000 7fb0addab640 1 -- 172.21.15.112:0/1448321289 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0a0056da0 msgr2=0x7fb0a0050570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:04.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.609+0000 7fb0addab640 1 --2- 172.21.15.112:0/1448321289 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0a0056da0 0x7fb0a0050570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:04.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.609+0000 7fb0addab640 1 -- 172.21.15.112:0/1448321289 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0a0050ab0 con 0x7fb0a00e2f40 2024-09-21T00:26:04.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.609+0000 7fdbad9e6640 1 -- start start 2024-09-21T00:26:04.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.609+0000 7fb0addab640 1 --2- 172.21.15.112:0/1448321289 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb0a00e2f40 0x7fb0a00e5140 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb0a400e6d0 tx=0x7fb0a400eb90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:04.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.609+0000 7fa691ba3640 1 --2- 172.21.15.112:0/2824632459 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa684049020 0x7fa684049430 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa68c051d10 tx=0x7fa68c0acea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:04.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.608+0000 7f00bc8eb640 1 -- start start 2024-09-21T00:26:04.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.610+0000 7fb09dffb640 1 -- 172.21.15.112:0/1448321289 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a400cab0 con 0x7fb0a00e2f40 2024-09-21T00:26:04.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.610+0000 7fb09dffb640 1 -- 172.21.15.112:0/1448321289 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb0a400cc50 con 0x7fb0a00e2f40 2024-09-21T00:26:04.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.610+0000 7fb09dffb640 1 -- 172.21.15.112:0/1448321289 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a4005280 con 0x7fb0a00e2f40 2024-09-21T00:26:04.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.610+0000 7fa681ffb640 1 -- 172.21.15.112:0/2824632459 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa68c0af040 con 0x7fa684049020 2024-09-21T00:26:04.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.610+0000 7fac68627640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fac58009bd0 0x7fac581789f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:04.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.610+0000 7fac68627640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac580df1b0 0x7fac58178f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:04.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.610+0000 7fac68627640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fac580de460 0x7fac580d0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:04.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.610+0000 7fac68627640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fac58006da0 con 0x7fac580df1b0 2024-09-21T00:26:04.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.610+0000 7fb0ae5ac640 1 -- 172.21.15.112:0/1448321289 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0a00511a0 con 0x7fb0a00e2f40 2024-09-21T00:26:04.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.610+0000 7fac68627640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fac5800be70 con 0x7fac58009bd0 2024-09-21T00:26:04.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.610+0000 7fac68627640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fac58006f20 con 0x7fac580de460 2024-09-21T00:26:04.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.608+0000 7f750e92b640 1 Processor -- start 2024-09-21T00:26:04.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.608+0000 7f750e92b640 1 -- start start 2024-09-21T00:26:04.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.610+0000 7f750e92b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f750806ff00 0x7f750815c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:04.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.610+0000 7f750e92b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f75080706e0 0x7f75081f8d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:04.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.610+0000 7f750e92b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75081f92d0 0x7f75081f96e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:04.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.610+0000 7f750e92b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f75080b5920 con 0x7f75081f92d0 2024-09-21T00:26:04.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.610+0000 7f750e92b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f75080b57a0 con 0x7f750806ff00 2024-09-21T00:26:04.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.610+0000 7f750e92b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f75080b5aa0 con 0x7f75080706e0 2024-09-21T00:26:04.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.610+0000 7f75077fe640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f75080706e0 0x7f75081f8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:04.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.610+0000 7f75077fe640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f75080706e0 0x7f75081f8d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:59644/0 (socket says 172.21.15.112:59644) 2024-09-21T00:26:04.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.610+0000 7f75077fe640 1 -- 172.21.15.112:0/442800014 learned_addr learned my addr 172.21.15.112:0/442800014 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:26:04.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.611+0000 7fb0ae5ac640 1 -- 172.21.15.112:0/1448321289 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0a0051760 con 0x7fb0a00e2f40 2024-09-21T00:26:04.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.611+0000 7f117ee15640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f117806def0 0x7f11780ca700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:04.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.611+0000 7f117ee15640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f11780b5260 0x7f11780cc690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:04.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.611+0000 7f117cbb1640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f117806def0 0x7f11780ca700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:04.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.611+0000 7f117cbb1640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f117806def0 0x7f11780ca700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:59664/0 (socket says 172.21.15.112:59664) 2024-09-21T00:26:04.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.611+0000 7f117cbb1640 1 -- 172.21.15.112:0/867394825 learned_addr learned my addr 172.21.15.112:0/867394825 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:26:04.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.611+0000 7f1177fff640 1 --2- 172.21.15.112:0/867394825 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f11780b5260 0x7f11780cc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:04.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.611+0000 7f117ee15640 1 --2- 172.21.15.112:0/867394825 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f117815bfc0 0x7f11780ccbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:04.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.611+0000 7f117ee15640 1 -- 172.21.15.112:0/867394825 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f117815ff00 con 0x7f117815bfc0 2024-09-21T00:26:04.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.611+0000 7f117ee15640 1 -- 172.21.15.112:0/867394825 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f117815fd80 con 0x7f11780b5260 2024-09-21T00:26:04.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.611+0000 7f117ee15640 1 -- 172.21.15.112:0/867394825 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f11780b0b80 con 0x7f117806def0 2024-09-21T00:26:04.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.612+0000 7f1177fff640 1 -- 172.21.15.112:0/867394825 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f117806def0 msgr2=0x7f11780ca700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:04.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.612+0000 7f1177fff640 1 --2- 172.21.15.112:0/867394825 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f117806def0 0x7f11780ca700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:04.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.612+0000 7f1177fff640 1 -- 172.21.15.112:0/867394825 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f117815bfc0 msgr2=0x7f11780ccbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:04.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.612+0000 7f1177fff640 1 --2- 172.21.15.112:0/867394825 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f117815bfc0 0x7f11780ccbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:04.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.612+0000 7f1177fff640 1 -- 172.21.15.112:0/867394825 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11780cf040 con 0x7f11780b5260 2024-09-21T00:26:04.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.610+0000 7fa6923a4640 1 -- 172.21.15.112:0/2824632459 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6840db280 con 0x7fa684049020 2024-09-21T00:26:04.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.610+0000 7fa6923a4640 1 -- 172.21.15.112:0/2824632459 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6840d87c0 con 0x7fa684049020 2024-09-21T00:26:04.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.611+0000 7fa681ffb640 1 -- 172.21.15.112:0/2824632459 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa68c06a580 con 0x7fa684049020 2024-09-21T00:26:04.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.611+0000 7fa681ffb640 1 -- 172.21.15.112:0/2824632459 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa68800b720 con 0x7fa684049020 2024-09-21T00:26:04.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.611+0000 7fed6ffff640 1 -- 172.21.15.112:0/3442869174 <== mon.2 v2:172.21.15.187:3301/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fed7801c460 con 0x7fed88070050 2024-09-21T00:26:04.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.611+0000 7f75077fe640 1 -- 172.21.15.112:0/442800014 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f750806ff00 msgr2=0x7f750815c7e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T00:26:04.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.611+0000 7f75077fe640 1 --2- 172.21.15.112:0/442800014 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f750806ff00 0x7f750815c7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:04.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.611+0000 7f75077fe640 1 -- 172.21.15.112:0/442800014 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75081f92d0 msgr2=0x7f75081f96e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:04.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.611+0000 7f75077fe640 1 --2- 172.21.15.112:0/442800014 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75081f92d0 0x7f75081f96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:04.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.611+0000 7f75077fe640 1 -- 172.21.15.112:0/442800014 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f750815cd20 con 0x7f75080706e0 2024-09-21T00:26:04.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.611+0000 7f75077fe640 1 --2- 172.21.15.112:0/442800014 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f75080706e0 0x7f75081f8d90 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f74f800c7b0 tx=0x7f74f800cc70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:04.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.612+0000 7f7504ff9640 1 -- 172.21.15.112:0/442800014 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74f8003b70 con 0x7f75080706e0 2024-09-21T00:26:04.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.612+0000 7f7504ff9640 1 -- 172.21.15.112:0/442800014 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f74f8003d10 con 0x7f75080706e0 2024-09-21T00:26:04.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.612+0000 7f7504ff9640 1 -- 172.21.15.112:0/442800014 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74f800aab0 con 0x7f75080706e0 2024-09-21T00:26:04.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.612+0000 7f750e92b640 1 -- 172.21.15.112:0/442800014 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75081600b0 con 0x7f75080706e0 2024-09-21T00:26:04.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.612+0000 7f750e92b640 1 -- 172.21.15.112:0/442800014 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f7508160670 con 0x7f75080706e0 2024-09-21T00:26:04.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.611+0000 7fac65bc2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac580df1b0 0x7fac58178f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:04.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.611+0000 7fac663c3640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fac58009bd0 0x7fac581789f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:04.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.611+0000 7fac65bc2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac580df1b0 0x7fac58178f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45902/0 (socket says 172.21.15.112:45902) 2024-09-21T00:26:04.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.611+0000 7fac65bc2640 1 -- 172.21.15.112:0/1633671512 learned_addr learned my addr 172.21.15.112:0/1633671512 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:26:04.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.611+0000 7fac66bc4640 1 --2- 172.21.15.112:0/1633671512 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fac580de460 0x7fac580d0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:04.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.611+0000 7fac65bc2640 1 -- 172.21.15.112:0/1633671512 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fac580de460 msgr2=0x7fac580d0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:04.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.611+0000 7fac65bc2640 1 --2- 172.21.15.112:0/1633671512 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fac580de460 0x7fac580d0ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:04.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.611+0000 7fac65bc2640 1 -- 172.21.15.112:0/1633671512 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fac58009bd0 msgr2=0x7fac581789f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:04.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.611+0000 7fac65bc2640 1 --2- 172.21.15.112:0/1633671512 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fac58009bd0 0x7fac581789f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:04.632 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.611+0000 7fac65bc2640 1 -- 172.21.15.112:0/1633671512 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac5801e690 con 0x7fac580df1b0 2024-09-21T00:26:04.632 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.611+0000 7fac65bc2640 1 --2- 172.21.15.112:0/1633671512 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac580df1b0 0x7fac58178f30 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fac5000c400 tx=0x7fac5000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:04.632 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.612+0000 7fac56ffd640 1 -- 172.21.15.112:0/1633671512 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac50017070 con 0x7fac580df1b0 2024-09-21T00:26:04.632 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.612+0000 7fac56ffd640 1 -- 172.21.15.112:0/1633671512 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fac500052d0 con 0x7fac580df1b0 2024-09-21T00:26:04.632 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.612+0000 7fac68627640 1 -- 172.21.15.112:0/1633671512 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac5801e970 con 0x7fac580df1b0 2024-09-21T00:26:04.632 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.612+0000 7fac56ffd640 1 -- 172.21.15.112:0/1633671512 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac50003830 con 0x7fac580df1b0 2024-09-21T00:26:04.632 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.612+0000 7fac68627640 1 -- 172.21.15.112:0/1633671512 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac580204d0 con 0x7fac580df1b0 2024-09-21T00:26:04.632 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.611+0000 7fdbad9e6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb90003a00 0x7fdba0009af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:04.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.611+0000 7fdbad9e6640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdba004edb0 0x7fdba000d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:04.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.611+0000 7fdbad9e6640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdba00ded80 0x7fdba00ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:04.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.612+0000 7fdbad9e6640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fdba003c3a0 con 0x7fdb90003a00 2024-09-21T00:26:04.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.612+0000 7fdbad9e6640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fdba003c220 con 0x7fdba00ded80 2024-09-21T00:26:04.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.612+0000 7fdbad9e6640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fdba003c520 con 0x7fdba004edb0 2024-09-21T00:26:04.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.612+0000 7fdba77fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdba00ded80 0x7fdba00ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:04.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.612+0000 7fdba77fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdba00ded80 0x7fdba00ccc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:43480/0 (socket says 172.21.15.112:43480) 2024-09-21T00:26:04.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.612+0000 7fdba77fe640 1 -- 172.21.15.112:0/4063739748 learned_addr learned my addr 172.21.15.112:0/4063739748 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:26:04.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.613+0000 7f1177fff640 1 --2- 172.21.15.112:0/867394825 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f11780b5260 0x7f11780cc690 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f116c032000 tx=0x7f116c004820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:04.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.613+0000 7fdba67fc640 1 --2- 172.21.15.112:0/4063739748 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdba004edb0 0x7fdba000d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:04.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.612+0000 7fed6ffff640 1 --2- 172.21.15.112:0/3442869174 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fed74078c60 0x7fed7407b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:04.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.613+0000 7fed6ffff640 1 -- 172.21.15.112:0/3442869174 <== mon.2 v2:172.21.15.187:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fed7809ae60 con 0x7fed88070050 2024-09-21T00:26:04.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.613+0000 7f11757fa640 1 -- 172.21.15.112:0/867394825 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f116c042030 con 0x7f11780b5260 2024-09-21T00:26:04.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.613+0000 7fed8ca5b640 1 -- 172.21.15.112:0/3442869174 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed68005290 con 0x7fed88070050 2024-09-21T00:26:04.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.611+0000 7f00bc8eb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f00b0005c60 0x7f00b002bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:04.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.611+0000 7f00bc8eb640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f00b0009ba0 0x7f00b0178e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:04.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.611+0000 7f00bc8eb640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f00b01793a0 0x7f00b0176ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:04.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.611+0000 7f00bc8eb640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f00b0006d70 con 0x7f00b0005c60 2024-09-21T00:26:04.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.611+0000 7f00bc8eb640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f00b000be40 con 0x7f00b0009ba0 2024-09-21T00:26:04.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.611+0000 7f00bc8eb640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f00b0006ef0 con 0x7f00b01793a0 2024-09-21T00:26:04.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.613+0000 7fdba77fe640 1 -- 172.21.15.112:0/4063739748 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdba004edb0 msgr2=0x7fdba000d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:04.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.613+0000 7fdba77fe640 1 --2- 172.21.15.112:0/4063739748 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdba004edb0 0x7fdba000d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:04.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.613+0000 7fdba77fe640 1 -- 172.21.15.112:0/4063739748 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb90003a00 msgr2=0x7fdba0009af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:04.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.613+0000 7fdba77fe640 1 --2- 172.21.15.112:0/4063739748 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb90003a00 0x7fdba0009af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:04.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.613+0000 7fdba77fe640 1 -- 172.21.15.112:0/4063739748 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdba00cfaf0 con 0x7fdba00ded80 2024-09-21T00:26:04.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.613+0000 7f00b659c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f00b0009ba0 0x7f00b0178e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:04.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.613+0000 7f00b659c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f00b0009ba0 0x7f00b0178e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:43492/0 (socket says 172.21.15.112:43492) 2024-09-21T00:26:04.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.613+0000 7f00b659c640 1 -- 172.21.15.112:0/3526267485 learned_addr learned my addr 172.21.15.112:0/3526267485 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:26:04.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.614+0000 7fdba77fe640 1 --2- 172.21.15.112:0/4063739748 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdba00ded80 0x7fdba00ccc90 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fdba8052020 tx=0x7fdba8051980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:04.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.614+0000 7fed86ffd640 1 --2- 172.21.15.112:0/3442869174 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fed74078c60 0x7fed7407b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:04.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.614+0000 7f750e92b640 1 -- 172.21.15.112:0/442800014 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74e8005290 con 0x7f75080706e0 2024-09-21T00:26:04.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.615+0000 7fb0ae5ac640 1 -- 172.21.15.112:0/1448321289 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb088005290 con 0x7fb0a00e2f40 2024-09-21T00:26:04.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.615+0000 7fac54ff9640 1 -- 172.21.15.112:0/1633671512 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac580073d0 con 0x7fac580df1b0 2024-09-21T00:26:04.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.615+0000 7fb09dffb640 1 -- 172.21.15.112:0/1448321289 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb0a4005420 con 0x7fb0a00e2f40 2024-09-21T00:26:04.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.613+0000 7f117ee15640 1 -- 172.21.15.112:0/867394825 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11780cf320 con 0x7f11780b5260 2024-09-21T00:26:04.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.613+0000 7f117ee15640 1 -- 172.21.15.112:0/867394825 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11780cf8e0 con 0x7f11780b5260 2024-09-21T00:26:04.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.614+0000 7f11757fa640 1 -- 172.21.15.112:0/867394825 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f116c004d10 con 0x7f11780b5260 2024-09-21T00:26:04.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.615+0000 7f11757fa640 1 -- 172.21.15.112:0/867394825 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f116c00d310 con 0x7f11780b5260 2024-09-21T00:26:04.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.614+0000 7f00b6d9d640 1 --2- 172.21.15.112:0/3526267485 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f00b0005c60 0x7f00b002bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:04.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.614+0000 7f00b759e640 1 --2- 172.21.15.112:0/3526267485 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f00b01793a0 0x7f00b0176ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:04.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.615+0000 7f00b659c640 1 -- 172.21.15.112:0/3526267485 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f00b01793a0 msgr2=0x7f00b0176ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:04.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.615+0000 7f00b659c640 1 --2- 172.21.15.112:0/3526267485 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f00b01793a0 0x7f00b0176ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:04.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.615+0000 7f00b659c640 1 -- 172.21.15.112:0/3526267485 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f00b0005c60 msgr2=0x7f00b002bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:04.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.615+0000 7f00b659c640 1 --2- 172.21.15.112:0/3526267485 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f00b0005c60 0x7f00b002bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:04.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.615+0000 7f00b659c640 1 -- 172.21.15.112:0/3526267485 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00b01797b0 con 0x7f00b0009ba0 2024-09-21T00:26:04.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.615+0000 7f117ee15640 1 -- 172.21.15.112:0/867394825 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1158005290 con 0x7f11780b5260 2024-09-21T00:26:04.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.614+0000 7fa6923a4640 1 -- 172.21.15.112:0/2824632459 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa66c005290 con 0x7fa684049020 2024-09-21T00:26:04.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.616+0000 7fa681ffb640 1 -- 172.21.15.112:0/2824632459 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa68c0b2070 con 0x7fa684049020 2024-09-21T00:26:04.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.615+0000 7f7504ff9640 1 -- 172.21.15.112:0/442800014 <== mon.2 v2:172.21.15.187:3301/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f74f800ac50 con 0x7f75080706e0 2024-09-21T00:26:04.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.615+0000 7f7504ff9640 1 --2- 172.21.15.112:0/442800014 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f74f4078b90 0x7f74f407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:04.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.616+0000 7f7504ff9640 1 -- 172.21.15.112:0/442800014 <== mon.2 v2:172.21.15.187:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f74f809ae10 con 0x7f75080706e0 2024-09-21T00:26:04.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.616+0000 7f7507fff640 1 --2- 172.21.15.112:0/442800014 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f74f4078b90 0x7f74f407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:04.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.616+0000 7fb09dffb640 1 --2- 172.21.15.112:0/1448321289 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fb08c078c10 0x7fb08c07b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:04.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.616+0000 7fb09dffb640 1 -- 172.21.15.112:0/1448321289 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fb0a409b830 con 0x7fb0a00e2f40 2024-09-21T00:26:04.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.616+0000 7fb0ad5aa640 1 --2- 172.21.15.112:0/1448321289 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fb08c078c10 0x7fb08c07b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:04.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.615+0000 7fdb8ffff640 1 -- 172.21.15.112:0/4063739748 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba8099070 con 0x7fdba00ded80 2024-09-21T00:26:04.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.616+0000 7fdbad9e6640 1 -- 172.21.15.112:0/4063739748 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdba004e2d0 con 0x7fdba00ded80 2024-09-21T00:26:04.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.616+0000 7fdbad9e6640 1 -- 172.21.15.112:0/4063739748 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdba004e890 con 0x7fdba00ded80 2024-09-21T00:26:04.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.616+0000 7f00b659c640 1 --2- 172.21.15.112:0/3526267485 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f00b0009ba0 0x7f00b0178e60 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f00a0008330 tx=0x7f00a000a910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:04.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.616+0000 7fed86ffd640 1 --2- 172.21.15.112:0/3442869174 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fed74078c60 0x7fed7407b170 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fed7c034000 tx=0x7fed7c004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:04.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.616+0000 7fac56ffd640 1 -- 172.21.15.112:0/1633671512 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fac5001a070 con 0x7fac580df1b0 2024-09-21T00:26:04.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.616+0000 7fdb8ffff640 1 -- 172.21.15.112:0/4063739748 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdba8075350 con 0x7fdba00ded80 2024-09-21T00:26:04.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.616+0000 7fdb8ffff640 1 -- 172.21.15.112:0/4063739748 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba80763e0 con 0x7fdba00ded80 2024-09-21T00:26:04.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.617+0000 7fa681ffb640 1 --2- 172.21.15.112:0/2824632459 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fa670078c60 0x7fa67007b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:04.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.617+0000 7f7507fff640 1 --2- 172.21.15.112:0/442800014 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f74f4078b90 0x7f74f407b0a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7500033000 tx=0x7f7500004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:04.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.617+0000 7fa6913a2640 1 --2- 172.21.15.112:0/2824632459 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fa670078c60 0x7fa67007b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:04.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.617+0000 7fa681ffb640 1 -- 172.21.15.112:0/2824632459 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fa68c136850 con 0x7fa684049020 2024-09-21T00:26:04.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.617+0000 7fb0ad5aa640 1 --2- 172.21.15.112:0/1448321289 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fb08c078c10 0x7fb08c07b120 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb0a0038530 tx=0x7fb098006b50 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:04.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.617+0000 7fac56ffd640 1 --2- 172.21.15.112:0/1633671512 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fac44078be0 0x7fac4407b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:04.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.617+0000 7fac56ffd640 1 -- 172.21.15.112:0/1633671512 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fac5009a6c0 con 0x7fac580df1b0 2024-09-21T00:26:04.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.617+0000 7fac663c3640 1 --2- 172.21.15.112:0/1633671512 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fac44078be0 0x7fac4407b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:04.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.617+0000 7fa6913a2640 1 --2- 172.21.15.112:0/2824632459 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fa670078c60 0x7fa67007b170 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa684054db0 tx=0x7fa67c00f040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:04.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.617+0000 7fdbad9e6640 1 -- 172.21.15.112:0/4063739748 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdba0053df0 con 0x7fdba00ded80 2024-09-21T00:26:04.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.618+0000 7fac663c3640 1 --2- 172.21.15.112:0/1633671512 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fac44078be0 0x7fac4407b0f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fac6006bcf0 tx=0x7fac60070040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:04.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.619+0000 7fed6ffff640 1 -- 172.21.15.112:0/3442869174 <== mon.2 v2:172.21.15.187:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fed8806e8b0 con 0x7fed88070050 2024-09-21T00:26:04.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.621+0000 7fac56ffd640 1 -- 172.21.15.112:0/1633671512 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fac50022020 con 0x7fac580df1b0 2024-09-21T00:26:04.646 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.621+0000 7fb09dffb640 1 -- 172.21.15.112:0/1448321289 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb0a4063b00 con 0x7fb0a00e2f40 2024-09-21T00:26:04.646 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.622+0000 7f00af7fe640 1 -- 172.21.15.112:0/3526267485 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00a0041070 con 0x7f00b0009ba0 2024-09-21T00:26:04.646 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.623+0000 7f11757fa640 1 -- 172.21.15.112:0/867394825 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f116c045030 con 0x7f11780b5260 2024-09-21T00:26:04.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.622+0000 7f00bc8eb640 1 -- 172.21.15.112:0/3526267485 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00b002c880 con 0x7f00b0009ba0 2024-09-21T00:26:04.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.622+0000 7f00bc8eb640 1 -- 172.21.15.112:0/3526267485 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00b002ce40 con 0x7f00b0009ba0 2024-09-21T00:26:04.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.622+0000 7f00af7fe640 1 -- 172.21.15.112:0/3526267485 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f00a0008e50 con 0x7f00b0009ba0 2024-09-21T00:26:04.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.622+0000 7f00af7fe640 1 -- 172.21.15.112:0/3526267485 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00a0038040 con 0x7f00b0009ba0 2024-09-21T00:26:04.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.623+0000 7f00bc8eb640 1 -- 172.21.15.112:0/3526267485 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f009c005290 con 0x7f00b0009ba0 2024-09-21T00:26:04.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.623+0000 7fdb8ffff640 1 -- 172.21.15.112:0/4063739748 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fdba809c070 con 0x7fdba00ded80 2024-09-21T00:26:04.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.624+0000 7f7504ff9640 1 -- 172.21.15.112:0/442800014 <== mon.2 v2:172.21.15.187:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f750815cd20 con 0x7f75080706e0 2024-09-21T00:26:04.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.624+0000 7f11757fa640 1 --2- 172.21.15.112:0/867394825 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f115c078c60 0x7f115c07b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:04.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.624+0000 7f117cbb1640 1 --2- 172.21.15.112:0/867394825 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f115c078c60 0x7f115c07b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:04.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.624+0000 7fdb8ffff640 1 --2- 172.21.15.112:0/4063739748 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fdb94078c60 0x7fdb9407b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:04.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.624+0000 7fdb8ffff640 1 -- 172.21.15.112:0/4063739748 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fdba8117580 con 0x7fdba00ded80 2024-09-21T00:26:04.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.626+0000 7f11757fa640 1 -- 172.21.15.112:0/867394825 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f116c0bde00 con 0x7f11780b5260 2024-09-21T00:26:04.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.626+0000 7f117cbb1640 1 --2- 172.21.15.112:0/867394825 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f115c078c60 0x7f115c07b170 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1164009970 tx=0x7f1164009110 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:04.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.626+0000 7fdba6ffd640 1 --2- 172.21.15.112:0/4063739748 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fdb94078c60 0x7fdb9407b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:04.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.626+0000 7fdba6ffd640 1 --2- 172.21.15.112:0/4063739748 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fdb94078c60 0x7fdb9407b170 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fdba0038560 tx=0x7fdb9c001890 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:04.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.627+0000 7f11757fa640 1 -- 172.21.15.112:0/867394825 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f116c085920 con 0x7f11780b5260 2024-09-21T00:26:04.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.627+0000 7fa681ffb640 1 -- 172.21.15.112:0/2824632459 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa6840db280 con 0x7fa684049020 2024-09-21T00:26:04.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.632+0000 7fdb8ffff640 1 -- 172.21.15.112:0/4063739748 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fdba80df0a0 con 0x7fdba00ded80 2024-09-21T00:26:04.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.634+0000 7f00af7fe640 1 -- 172.21.15.112:0/3526267485 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f00a0044070 con 0x7f00b0009ba0 2024-09-21T00:26:04.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.635+0000 7f00af7fe640 1 --2- 172.21.15.112:0/3526267485 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f0098078bc0 0x7f009807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:04.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.635+0000 7f00af7fe640 1 -- 172.21.15.112:0/3526267485 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f00a00bac90 con 0x7f00b0009ba0 2024-09-21T00:26:04.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.635+0000 7f00b6d9d640 1 --2- 172.21.15.112:0/3526267485 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f0098078bc0 0x7f009807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:04.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.636+0000 7f00b6d9d640 1 --2- 172.21.15.112:0/3526267485 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f0098078bc0 0x7f009807b0d0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f00a8004e80 tx=0x7f00a8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:04.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:04.637+0000 7f00af7fe640 1 -- 172.21.15.112:0/3526267485 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f00a004b080 con 0x7f00b0009ba0 2024-09-21T00:26:05.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:05.045+0000 7fed8ca5b640 1 -- 172.21.15.112:0/3442869174 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fed68005580 con 0x7fed88070050 2024-09-21T00:26:05.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:05.046+0000 7fed6ffff640 1 -- 172.21.15.112:0/3442869174 <== mon.2 v2:172.21.15.187:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fed78062fd0 con 0x7fed88070050 2024-09-21T00:26:05.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:05.046+0000 7f750e92b640 1 -- 172.21.15.112:0/442800014 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f74e8005580 con 0x7f75080706e0 2024-09-21T00:26:05.047 INFO:teuthology.orchestra.run.smithi112.stdout:64424509448 2024-09-21T00:26:05.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:05.047+0000 7fac54ff9640 1 -- 172.21.15.112:0/1633671512 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fac580068e0 con 0x7fac580df1b0 2024-09-21T00:26:05.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:05.046+0000 7f7504ff9640 1 -- 172.21.15.112:0/442800014 <== mon.2 v2:172.21.15.187:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f74f8062f90 con 0x7f75080706e0 2024-09-21T00:26:05.048 INFO:teuthology.orchestra.run.smithi112.stdout:64424509447 2024-09-21T00:26:05.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:05.047+0000 7fac56ffd640 1 -- 172.21.15.112:0/1633671512 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fac500628c0 con 0x7fac580df1b0 2024-09-21T00:26:05.048 INFO:teuthology.orchestra.run.smithi112.stdout:64424509448 2024-09-21T00:26:05.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:05.050+0000 7fed8ca5b640 1 -- 172.21.15.112:0/3442869174 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fed74078c60 msgr2=0x7fed7407b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:05.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:05.050+0000 7fed8ca5b640 1 --2- 172.21.15.112:0/3442869174 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fed74078c60 0x7fed7407b170 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fed7c034000 tx=0x7fed7c004730 comp rx=0 tx=0).stop 2024-09-21T00:26:05.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:05.050+0000 7fed8ca5b640 1 -- 172.21.15.112:0/3442869174 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fed88070050 msgr2=0x7fed881fc500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:05.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:05.050+0000 7f750e92b640 1 -- 172.21.15.112:0/442800014 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f74f4078b90 msgr2=0x7f74f407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:05.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:05.050+0000 7fed8ca5b640 1 --2- 172.21.15.112:0/3442869174 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fed88070050 0x7fed881fc500 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fed7800c3a0 tx=0x7fed7800c860 comp rx=0 tx=0).stop 2024-09-21T00:26:05.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:05.050+0000 7f750e92b640 1 --2- 172.21.15.112:0/442800014 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f74f4078b90 0x7f74f407b0a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7500033000 tx=0x7f7500004730 comp rx=0 tx=0).stop 2024-09-21T00:26:05.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:05.050+0000 7f750e92b640 1 -- 172.21.15.112:0/442800014 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f75080706e0 msgr2=0x7f75081f8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:05.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:05.051+0000 7fed8ca5b640 1 -- 172.21.15.112:0/3442869174 shutdown_connections 2024-09-21T00:26:05.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:05.051+0000 7fed8ca5b640 1 --2- 172.21.15.112:0/3442869174 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fed88162570 0x7fed880ad350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:05.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:05.051+0000 7fed8ca5b640 1 --2- 172.21.15.112:0/3442869174 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fed74078c60 0x7fed7407b170 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:05.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:05.051+0000 7fed8ca5b640 1 --2- 172.21.15.112:0/3442869174 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fed880aef90 0x7fed880ace10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:05.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:05.051+0000 7fed8ca5b640 1 --2- 172.21.15.112:0/3442869174 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fed88070050 0x7fed881fc500 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:05.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:05.051+0000 7fed8ca5b640 1 -- 172.21.15.112:0/3442869174 >> 172.21.15.112:0/3442869174 conn(0x7fed880cfec0 msgr2=0x7fed880b0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:05.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:05.051+0000 7f750e92b640 1 --2- 172.21.15.112:0/442800014 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f75080706e0 0x7f75081f8d90 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f74f800c7b0 tx=0x7f74f800cc70 comp rx=0 tx=0).stop 2024-09-21T00:26:05.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:05.051+0000 7fac54ff9640 1 -- 172.21.15.112:0/1633671512 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fac44078be0 msgr2=0x7fac4407b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:05.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:05.051+0000 7fac54ff9640 1 --2- 172.21.15.112:0/1633671512 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fac44078be0 0x7fac4407b0f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fac6006bcf0 tx=0x7fac60070040 comp rx=0 tx=0).stop 2024-09-21T00:26:05.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:05.051+0000 7fed8ca5b640 1 -- 172.21.15.112:0/3442869174 shutdown_connections 2024-09-21T00:26:05.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:05.051+0000 7fac54ff9640 1 -- 172.21.15.112:0/1633671512 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac580df1b0 msgr2=0x7fac58178f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:05.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:05.051+0000 7fac54ff9640 1 --2- 172.21.15.112:0/1633671512 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac580df1b0 0x7fac58178f30 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fac5000c400 tx=0x7fac5000c8c0 comp rx=0 tx=0).stop 2024-09-21T00:26:05.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:05.051+0000 7f750e92b640 1 -- 172.21.15.112:0/442800014 shutdown_connections 2024-09-21T00:26:05.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:05.051+0000 7f750e92b640 1 --2- 172.21.15.112:0/442800014 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f74f4078b90 0x7f74f407b0a0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:05.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:05.051+0000 7f750e92b640 1 --2- 172.21.15.112:0/442800014 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75081f92d0 0x7f75081f96e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:05.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:05.051+0000 7f750e92b640 1 --2- 172.21.15.112:0/442800014 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f75080706e0 0x7f75081f8d90 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:05.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:05.051+0000 7f750e92b640 1 --2- 172.21.15.112:0/442800014 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f750806ff00 0x7f750815c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:05.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:05.051+0000 7f750e92b640 1 -- 172.21.15.112:0/442800014 >> 172.21.15.112:0/442800014 conn(0x7f75080cdca0 msgr2=0x7f7508162280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:05.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:05.051+0000 7fed8ca5b640 1 -- 172.21.15.112:0/3442869174 wait complete. 2024-09-21T00:26:05.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:05.051+0000 7fac54ff9640 1 -- 172.21.15.112:0/1633671512 shutdown_connections 2024-09-21T00:26:05.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:05.051+0000 7f750e92b640 1 -- 172.21.15.112:0/442800014 shutdown_connections 2024-09-21T00:26:05.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:05.051+0000 7fac54ff9640 1 --2- 172.21.15.112:0/1633671512 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fac580de460 0x7fac580d0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:05.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:05.051+0000 7fac54ff9640 1 --2- 172.21.15.112:0/1633671512 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fac44078be0 0x7fac4407b0f0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:05.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:05.051+0000 7fac54ff9640 1 --2- 172.21.15.112:0/1633671512 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac580df1b0 0x7fac58178f30 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:05.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:05.051+0000 7fac54ff9640 1 --2- 172.21.15.112:0/1633671512 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fac58009bd0 0x7fac581789f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:05.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:05.051+0000 7fac54ff9640 1 -- 172.21.15.112:0/1633671512 >> 172.21.15.112:0/1633671512 conn(0x7fac58051450 msgr2=0x7fac58035b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:05.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:05.051+0000 7f750e92b640 1 -- 172.21.15.112:0/442800014 wait complete. 2024-09-21T00:26:05.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:05.052+0000 7fac54ff9640 1 -- 172.21.15.112:0/1633671512 shutdown_connections 2024-09-21T00:26:05.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:05.052+0000 7fac54ff9640 1 -- 172.21.15.112:0/1633671512 wait complete. 2024-09-21T00:26:05.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:05.054+0000 7fb0ae5ac640 1 -- 172.21.15.112:0/1448321289 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fb088005580 con 0x7fb0a00e2f40 2024-09-21T00:26:05.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:05.055+0000 7fb09dffb640 1 -- 172.21.15.112:0/1448321289 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fb0a4063400 con 0x7fb0a00e2f40 2024-09-21T00:26:05.059 INFO:teuthology.orchestra.run.smithi112.stdout:64424509448 2024-09-21T00:26:05.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:05.055+0000 7fa6923a4640 1 -- 172.21.15.112:0/2824632459 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fa66c005580 con 0x7fa684049020 2024-09-21T00:26:05.060 INFO:teuthology.orchestra.run.smithi112.stdout:55834574855 2024-09-21T00:26:05.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:05.056+0000 7fa681ffb640 1 -- 172.21.15.112:0/2824632459 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fa68c0c2060 con 0x7fa684049020 2024-09-21T00:26:05.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:05.058+0000 7f117ee15640 1 -- 172.21.15.112:0/867394825 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f1158005850 con 0x7f11780b5260 2024-09-21T00:26:05.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:05.059+0000 7fb0ae5ac640 1 -- 172.21.15.112:0/1448321289 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fb08c078c10 msgr2=0x7fb08c07b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:05.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:05.059+0000 7fb0ae5ac640 1 --2- 172.21.15.112:0/1448321289 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fb08c078c10 0x7fb08c07b120 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb0a0038530 tx=0x7fb098006b50 comp rx=0 tx=0).stop 2024-09-21T00:26:05.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:05.059+0000 7fb0ae5ac640 1 -- 172.21.15.112:0/1448321289 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb0a00e2f40 msgr2=0x7fb0a00e5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:05.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:05.059+0000 7fb0ae5ac640 1 --2- 172.21.15.112:0/1448321289 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb0a00e2f40 0x7fb0a00e5140 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb0a400e6d0 tx=0x7fb0a400eb90 comp rx=0 tx=0).stop 2024-09-21T00:26:05.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:05.059+0000 7fb0ae5ac640 1 -- 172.21.15.112:0/1448321289 shutdown_connections 2024-09-21T00:26:05.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:05.059+0000 7fb0ae5ac640 1 --2- 172.21.15.112:0/1448321289 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb0a00e2f40 0x7fb0a00e5140 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:05.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:05.059+0000 7fb0ae5ac640 1 --2- 172.21.15.112:0/1448321289 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fb08c078c10 0x7fb08c07b120 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:05.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:05.059+0000 7fb0ae5ac640 1 --2- 172.21.15.112:0/1448321289 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0a0056da0 0x7fb0a0050570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:05.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:05.059+0000 7fb0ae5ac640 1 --2- 172.21.15.112:0/1448321289 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb0a0054360 0x7fb0a00d6f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:05.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:05.059+0000 7fb0ae5ac640 1 -- 172.21.15.112:0/1448321289 >> 172.21.15.112:0/1448321289 conn(0x7fb0a003a5f0 msgr2=0x7fb0a0038680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:05.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:05.059+0000 7f11757fa640 1 -- 172.21.15.112:0/867394825 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f116c08a350 con 0x7f11780b5260 2024-09-21T00:26:05.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:05.059+0000 7fa6923a4640 1 -- 172.21.15.112:0/2824632459 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fa670078c60 msgr2=0x7fa67007b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:05.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:05.059+0000 7fa6923a4640 1 --2- 172.21.15.112:0/2824632459 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fa670078c60 0x7fa67007b170 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa684054db0 tx=0x7fa67c00f040 comp rx=0 tx=0).stop 2024-09-21T00:26:05.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:05.059+0000 7fa6923a4640 1 -- 172.21.15.112:0/2824632459 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa684049020 msgr2=0x7fa684049430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:05.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:05.059+0000 7fa6923a4640 1 --2- 172.21.15.112:0/2824632459 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa684049020 0x7fa684049430 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa68c051d10 tx=0x7fa68c0acea0 comp rx=0 tx=0).stop 2024-09-21T00:26:05.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:05.059+0000 7fa6923a4640 1 -- 172.21.15.112:0/2824632459 shutdown_connections 2024-09-21T00:26:05.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:05.059+0000 7fa6923a4640 1 --2- 172.21.15.112:0/2824632459 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fa670078c60 0x7fa67007b170 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:05.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:05.059+0000 7fa6923a4640 1 --2- 172.21.15.112:0/2824632459 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa684049020 0x7fa684049430 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:05.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:05.060+0000 7fa6923a4640 1 --2- 172.21.15.112:0/2824632459 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa6840da2d0 0x7fa6840da6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:05.064 INFO:teuthology.orchestra.run.smithi112.stdout:64424509448 2024-09-21T00:26:05.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:05.060+0000 7fa6923a4640 1 --2- 172.21.15.112:0/2824632459 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa6840cec50 0x7fa6840cf060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:05.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:05.060+0000 7fa6923a4640 1 -- 172.21.15.112:0/2824632459 >> 172.21.15.112:0/2824632459 conn(0x7fa68401eab0 msgr2=0x7fa684037180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:05.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:05.060+0000 7fb0ae5ac640 1 -- 172.21.15.112:0/1448321289 shutdown_connections 2024-09-21T00:26:05.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:05.060+0000 7fa6923a4640 1 -- 172.21.15.112:0/2824632459 shutdown_connections 2024-09-21T00:26:05.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:05.060+0000 7fa6923a4640 1 -- 172.21.15.112:0/2824632459 wait complete. 2024-09-21T00:26:05.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:05.061+0000 7fb0ae5ac640 1 -- 172.21.15.112:0/1448321289 wait complete. 2024-09-21T00:26:05.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:05.061+0000 7fdbad9e6640 1 -- 172.21.15.112:0/4063739748 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fdba0053270 con 0x7fdba00ded80 2024-09-21T00:26:05.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:05.061+0000 7f00bc8eb640 1 -- 172.21.15.112:0/3526267485 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f009c005850 con 0x7f00b0009ba0 2024-09-21T00:26:05.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:05.062+0000 7f00af7fe640 1 -- 172.21.15.112:0/3526267485 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f00b01797b0 con 0x7f00b0009ba0 2024-09-21T00:26:05.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:05.062+0000 7fdb8ffff640 1 -- 172.21.15.112:0/4063739748 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fdba80e3ad0 con 0x7fdba00ded80 2024-09-21T00:26:05.067 INFO:teuthology.orchestra.run.smithi112.stdout:64424509446 2024-09-21T00:26:05.067 INFO:teuthology.orchestra.run.smithi112.stdout:64424509447 2024-09-21T00:26:05.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:05.062+0000 7f117ee15640 1 -- 172.21.15.112:0/867394825 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f115c078c60 msgr2=0x7f115c07b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:05.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:05.062+0000 7f117ee15640 1 --2- 172.21.15.112:0/867394825 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f115c078c60 0x7f115c07b170 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1164009970 tx=0x7f1164009110 comp rx=0 tx=0).stop 2024-09-21T00:26:05.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:05.063+0000 7f117ee15640 1 -- 172.21.15.112:0/867394825 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f11780b5260 msgr2=0x7f11780cc690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:05.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:05.063+0000 7f117ee15640 1 --2- 172.21.15.112:0/867394825 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f11780b5260 0x7f11780cc690 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f116c032000 tx=0x7f116c004820 comp rx=0 tx=0).stop 2024-09-21T00:26:05.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:05.063+0000 7f117ee15640 1 -- 172.21.15.112:0/867394825 shutdown_connections 2024-09-21T00:26:05.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:05.063+0000 7f117ee15640 1 --2- 172.21.15.112:0/867394825 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f115c078c60 0x7f115c07b170 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:05.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:05.063+0000 7f117ee15640 1 --2- 172.21.15.112:0/867394825 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f117815bfc0 0x7f11780ccbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:05.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:05.063+0000 7f117ee15640 1 --2- 172.21.15.112:0/867394825 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f11780b5260 0x7f11780cc690 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:05.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:05.063+0000 7f117ee15640 1 --2- 172.21.15.112:0/867394825 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f117806def0 0x7f11780ca700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:05.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:05.063+0000 7f117ee15640 1 -- 172.21.15.112:0/867394825 >> 172.21.15.112:0/867394825 conn(0x7f11780cdca0 msgr2=0x7f11780aeb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:05.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:05.063+0000 7f117ee15640 1 -- 172.21.15.112:0/867394825 shutdown_connections 2024-09-21T00:26:05.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:05.063+0000 7f117ee15640 1 -- 172.21.15.112:0/867394825 wait complete. 2024-09-21T00:26:05.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:05.066+0000 7f00bc8eb640 1 -- 172.21.15.112:0/3526267485 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f0098078bc0 msgr2=0x7f009807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:05.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:05.066+0000 7fdbad9e6640 1 -- 172.21.15.112:0/4063739748 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fdb94078c60 msgr2=0x7fdb9407b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:05.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:05.066+0000 7fdbad9e6640 1 --2- 172.21.15.112:0/4063739748 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fdb94078c60 0x7fdb9407b170 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fdba0038560 tx=0x7fdb9c001890 comp rx=0 tx=0).stop 2024-09-21T00:26:05.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:05.066+0000 7f00bc8eb640 1 --2- 172.21.15.112:0/3526267485 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f0098078bc0 0x7f009807b0d0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f00a8004e80 tx=0x7f00a8009040 comp rx=0 tx=0).stop 2024-09-21T00:26:05.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:05.066+0000 7fdbad9e6640 1 -- 172.21.15.112:0/4063739748 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdba00ded80 msgr2=0x7fdba00ccc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:05.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:05.066+0000 7fdbad9e6640 1 --2- 172.21.15.112:0/4063739748 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdba00ded80 0x7fdba00ccc90 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fdba8052020 tx=0x7fdba8051980 comp rx=0 tx=0).stop 2024-09-21T00:26:05.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:05.067+0000 7fdbad9e6640 1 -- 172.21.15.112:0/4063739748 shutdown_connections 2024-09-21T00:26:05.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:05.067+0000 7fdbad9e6640 1 --2- 172.21.15.112:0/4063739748 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdba00ded80 0x7fdba00ccc90 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:05.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:05.067+0000 7fdbad9e6640 1 --2- 172.21.15.112:0/4063739748 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdba004edb0 0x7fdba000d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:05.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:05.067+0000 7fdbad9e6640 1 --2- 172.21.15.112:0/4063739748 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fdb94078c60 0x7fdb9407b170 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:05.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:05.067+0000 7fdbad9e6640 1 --2- 172.21.15.112:0/4063739748 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb90003a00 0x7fdba0009af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:05.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:05.067+0000 7fdbad9e6640 1 -- 172.21.15.112:0/4063739748 >> 172.21.15.112:0/4063739748 conn(0x7fdba003a620 msgr2=0x7fdba00e5160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:05.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:05.067+0000 7f00bc8eb640 1 -- 172.21.15.112:0/3526267485 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f00b0009ba0 msgr2=0x7f00b0178e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:05.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:05.067+0000 7f00bc8eb640 1 --2- 172.21.15.112:0/3526267485 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f00b0009ba0 0x7f00b0178e60 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f00a0008330 tx=0x7f00a000a910 comp rx=0 tx=0).stop 2024-09-21T00:26:05.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:05.067+0000 7fdbad9e6640 1 -- 172.21.15.112:0/4063739748 shutdown_connections 2024-09-21T00:26:05.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:05.067+0000 7fdbad9e6640 1 -- 172.21.15.112:0/4063739748 wait complete. 2024-09-21T00:26:05.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:05.067+0000 7f00bc8eb640 1 -- 172.21.15.112:0/3526267485 shutdown_connections 2024-09-21T00:26:05.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:05.067+0000 7f00bc8eb640 1 --2- 172.21.15.112:0/3526267485 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f00b01793a0 0x7f00b0176ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:05.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:05.067+0000 7f00bc8eb640 1 --2- 172.21.15.112:0/3526267485 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f00b0009ba0 0x7f00b0178e60 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:05.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:05.067+0000 7f00bc8eb640 1 --2- 172.21.15.112:0/3526267485 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f0098078bc0 0x7f009807b0d0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:05.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:05.067+0000 7f00bc8eb640 1 --2- 172.21.15.112:0/3526267485 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f00b0005c60 0x7f00b002bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:05.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:05.067+0000 7f00bc8eb640 1 -- 172.21.15.112:0/3526267485 >> 172.21.15.112:0/3526267485 conn(0x7f00b0051420 msgr2=0x7f00b001cd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:05.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:05.073+0000 7f00bc8eb640 1 -- 172.21.15.112:0/3526267485 shutdown_connections 2024-09-21T00:26:05.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:05.073+0000 7f00bc8eb640 1 -- 172.21.15.112:0/3526267485 wait complete. 2024-09-21T00:26:05.075 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509447 for osd.7 2024-09-21T00:26:05.075 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509449 got 64424509448 for osd.3 2024-09-21T00:26:05.076 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509449 got 64424509448 for osd.0 2024-09-21T00:26:05.083 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509449 got 64424509448 for osd.2 2024-09-21T00:26:05.084 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574855 for osd.1 2024-09-21T00:26:05.085 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509449 got 64424509448 for osd.6 2024-09-21T00:26:05.089 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509447 for osd.5 2024-09-21T00:26:05.089 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509446 for osd.4 2024-09-21T00:26:06.075 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-21T00:26:06.076 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-21T00:26:06.076 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-21T00:26:06.084 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-21T00:26:06.085 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-21T00:26:06.086 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-21T00:26:06.090 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-21T00:26:06.090 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-21T00:26:06.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.264+0000 7fe6bd064640 1 -- 172.21.15.112:0/1385265617 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6b806d830 msgr2=0x7fe6b815eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:06.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.264+0000 7fe6bd064640 1 --2- 172.21.15.112:0/1385265617 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6b806d830 0x7fe6b815eeb0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe6a8005fd0 tx=0x7fe6a802a5d0 comp rx=0 tx=0).stop 2024-09-21T00:26:06.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.265+0000 7fe6bd064640 1 -- 172.21.15.112:0/1385265617 shutdown_connections 2024-09-21T00:26:06.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.265+0000 7fe6bd064640 1 --2- 172.21.15.112:0/1385265617 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe6b806dc40 0x7fe6b80735f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:06.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.265+0000 7fe6bd064640 1 --2- 172.21.15.112:0/1385265617 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6b806d830 0x7fe6b815eeb0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:06.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.265+0000 7fe6bd064640 1 -- 172.21.15.112:0/1385265617 >> 172.21.15.112:0/1385265617 conn(0x7fe6b80cd460 msgr2=0x7fe6b80cd870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:06.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.265+0000 7fe6bd064640 1 -- 172.21.15.112:0/1385265617 shutdown_connections 2024-09-21T00:26:06.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.265+0000 7fe6bd064640 1 -- 172.21.15.112:0/1385265617 wait complete. 2024-09-21T00:26:06.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.268+0000 7ff09028b640 1 -- 172.21.15.112:0/4041776930 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff074003a30 msgr2=0x7ff074003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:06.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.268+0000 7ff09028b640 1 --2- 172.21.15.112:0/4041776930 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff074003a30 0x7ff074003e80 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff084009880 tx=0x7ff074005830 comp rx=0 tx=0).stop 2024-09-21T00:26:06.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.269+0000 7fe6bd064640 1 Processor -- start 2024-09-21T00:26:06.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.268+0000 7f213f572640 1 -- 172.21.15.112:0/757250453 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f213806e330 msgr2=0x7f21381622d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:06.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.268+0000 7f213f572640 1 --2- 172.21.15.112:0/757250453 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f213806e330 0x7f21381622d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f2138074da0 tx=0x7f2130033190 comp rx=0 tx=0).stop 2024-09-21T00:26:06.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.269+0000 7ff08e828640 1 -- 172.21.15.112:0/4041776930 reap_dead start 2024-09-21T00:26:06.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.269+0000 7ff09028b640 1 -- 172.21.15.112:0/4041776930 shutdown_connections 2024-09-21T00:26:06.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.269+0000 7ff09028b640 1 -- 172.21.15.112:0/4041776930 >> 172.21.15.112:0/4041776930 conn(0x7ff0880735d0 msgr2=0x7ff0880739e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:06.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.269+0000 7fe6bd064640 1 -- start start 2024-09-21T00:26:06.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.269+0000 7ff09028b640 1 -- 172.21.15.112:0/4041776930 shutdown_connections 2024-09-21T00:26:06.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.269+0000 7ff09028b640 1 -- 172.21.15.112:0/4041776930 wait complete. 2024-09-21T00:26:06.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.269+0000 7f213f572640 1 -- 172.21.15.112:0/757250453 shutdown_connections 2024-09-21T00:26:06.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.269+0000 7f213f572640 1 --2- 172.21.15.112:0/757250453 >> v2:172.21.15.187:3300/0 conn(0x7f21380cf3b0 0x7f213806ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:06.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.269+0000 7f213f572640 1 --2- 172.21.15.112:0/757250453 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f213806e330 0x7f21381622d0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:06.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.269+0000 7f213f572640 1 --2- 172.21.15.112:0/757250453 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f213815e990 0x7f21380cee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:06.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.269+0000 7f213f572640 1 -- 172.21.15.112:0/757250453 >> 172.21.15.112:0/757250453 conn(0x7f21380735d0 msgr2=0x7f21380739e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:06.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.270+0000 7f213f572640 1 -- 172.21.15.112:0/757250453 shutdown_connections 2024-09-21T00:26:06.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.271+0000 7fe6bd064640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe6b806d830 0x7fe6b81fa620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:06.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.271+0000 7fe6bd064640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6b806dc40 0x7fe6b814a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:06.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.272+0000 7ff09028b640 1 Processor -- start 2024-09-21T00:26:06.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.272+0000 7fe6b659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6b806dc40 0x7fe6b814a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:06.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.272+0000 7fe6b659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6b806dc40 0x7fe6b814a0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59944/0 (socket says 172.21.15.112:59944) 2024-09-21T00:26:06.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.272+0000 7fe6b6d9d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe6b806d830 0x7fe6b81fa620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:06.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.272+0000 7fe6b6d9d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe6b806d830 0x7fe6b81fa620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:53666/0 (socket says 172.21.15.112:53666) 2024-09-21T00:26:06.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.272+0000 7fe6b659c640 1 -- 172.21.15.112:0/650544257 learned_addr learned my addr 172.21.15.112:0/650544257 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:26:06.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.272+0000 7fe6bd064640 1 --2- 172.21.15.112:0/650544257 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe6b815f3f0 0x7fe6b8162350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:06.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.272+0000 7ff09028b640 1 -- start start 2024-09-21T00:26:06.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.272+0000 7fe6bd064640 1 -- 172.21.15.112:0/650544257 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe6b8073cb0 con 0x7fe6b806dc40 2024-09-21T00:26:06.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.272+0000 7fe6bd064640 1 -- 172.21.15.112:0/650544257 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe6b8073b30 con 0x7fe6b806d830 2024-09-21T00:26:06.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.272+0000 7fe6b759e640 1 --2- 172.21.15.112:0/650544257 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe6b815f3f0 0x7fe6b8162350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:06.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.273+0000 7fe6bd064640 1 -- 172.21.15.112:0/650544257 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe6b8073e30 con 0x7fe6b815f3f0 2024-09-21T00:26:06.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.270+0000 7f213f572640 1 -- 172.21.15.112:0/757250453 wait complete. 2024-09-21T00:26:06.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.273+0000 7f213f572640 1 Processor -- start 2024-09-21T00:26:06.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.273+0000 7f213f572640 1 -- start start 2024-09-21T00:26:06.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.274+0000 7fe6b659c640 1 -- 172.21.15.112:0/650544257 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe6b815f3f0 msgr2=0x7fe6b8162350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:06.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.274+0000 7fe6b659c640 1 --2- 172.21.15.112:0/650544257 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe6b815f3f0 0x7fe6b8162350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:06.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.274+0000 7fe6b659c640 1 -- 172.21.15.112:0/650544257 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe6b806d830 msgr2=0x7fe6b81fa620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:06.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.274+0000 7fe6b659c640 1 --2- 172.21.15.112:0/650544257 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe6b806d830 0x7fe6b81fa620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:06.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.274+0000 7fe6b659c640 1 -- 172.21.15.112:0/650544257 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6b80b20f0 con 0x7fe6b806dc40 2024-09-21T00:26:06.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.274+0000 7fe6b659c640 1 --2- 172.21.15.112:0/650544257 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6b806dc40 0x7fe6b814a0d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe6b000ed30 tx=0x7fe6b000c640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:06.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.274+0000 7fe69f7fe640 1 -- 172.21.15.112:0/650544257 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6b0011470 con 0x7fe6b806dc40 2024-09-21T00:26:06.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.274+0000 7fe6b6d9d640 1 --2- 172.21.15.112:0/650544257 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe6b806d830 0x7fe6b81fa620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:26:06.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.275+0000 7fe6b759e640 1 --2- 172.21.15.112:0/650544257 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe6b815f3f0 0x7fe6b8162350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:26:06.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.275+0000 7fe6bd064640 1 -- 172.21.15.112:0/650544257 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6b80b1e10 con 0x7fe6b806dc40 2024-09-21T00:26:06.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.275+0000 7fe6bd064640 1 -- 172.21.15.112:0/650544257 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6b8151640 con 0x7fe6b806dc40 2024-09-21T00:26:06.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.275+0000 7fe69f7fe640 1 -- 172.21.15.112:0/650544257 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe6b0011700 con 0x7fe6b806dc40 2024-09-21T00:26:06.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.276+0000 7fe69f7fe640 1 -- 172.21.15.112:0/650544257 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6b0011b50 con 0x7fe6b806dc40 2024-09-21T00:26:06.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.276+0000 7f213f572640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f213806e330 0x7f2138157d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:06.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.276+0000 7f213f572640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f21380cf3b0 0x7f213815f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:06.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.276+0000 7f213f572640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f213815e990 0x7f213815f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:06.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.276+0000 7f213f572640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2138162990 con 0x7f213815e990 2024-09-21T00:26:06.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.276+0000 7f213f572640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f2138162810 con 0x7f21380cf3b0 2024-09-21T00:26:06.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.276+0000 7f213f572640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f2138162b10 con 0x7f213806e330 2024-09-21T00:26:06.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.277+0000 7f213cb0d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f21380cf3b0 0x7f213815f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:06.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.277+0000 7f213d30e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f213806e330 0x7f2138157d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:06.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.277+0000 7f213d30e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f213806e330 0x7f2138157d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:60686/0 (socket says 172.21.15.112:60686) 2024-09-21T00:26:06.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.277+0000 7f213d30e640 1 -- 172.21.15.112:0/169691057 learned_addr learned my addr 172.21.15.112:0/169691057 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:26:06.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.277+0000 7ff09028b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff088160c00 0x7ff0880b4b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:06.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.277+0000 7ff09028b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff0880b50d0 0x7ff08806cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:06.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.277+0000 7ff09028b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff08806d320 0x7ff08815e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:06.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.277+0000 7ff09028b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff08806df70 con 0x7ff088160c00 2024-09-21T00:26:06.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.277+0000 7ff09028b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff08806ddf0 con 0x7ff08806d320 2024-09-21T00:26:06.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.277+0000 7ff09028b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff08806e0f0 con 0x7ff0880b50d0 2024-09-21T00:26:06.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.277+0000 7ff08e027640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff088160c00 0x7ff0880b4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:06.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.277+0000 7ff08e027640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff088160c00 0x7ff0880b4b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59972/0 (socket says 172.21.15.112:59972) 2024-09-21T00:26:06.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.277+0000 7ff08e027640 1 -- 172.21.15.112:0/357097098 learned_addr learned my addr 172.21.15.112:0/357097098 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:26:06.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.278+0000 7ff08e027640 1 -- 172.21.15.112:0/357097098 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff0880b50d0 msgr2=0x7ff08806cde0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T00:26:06.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.278+0000 7ff08e027640 1 --2- 172.21.15.112:0/357097098 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff0880b50d0 0x7ff08806cde0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:06.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.278+0000 7ff08e027640 1 -- 172.21.15.112:0/357097098 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff08806d320 msgr2=0x7ff08815e6e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T00:26:06.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.278+0000 7ff08e027640 1 --2- 172.21.15.112:0/357097098 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff08806d320 0x7ff08815e6e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:06.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.278+0000 7ff08e027640 1 -- 172.21.15.112:0/357097098 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff08806d730 con 0x7ff088160c00 2024-09-21T00:26:06.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.278+0000 7ff08e027640 1 --2- 172.21.15.112:0/357097098 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff088160c00 0x7ff0880b4b90 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff084000c00 tx=0x7ff084032ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:06.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.277+0000 7f213db0f640 1 --2- 172.21.15.112:0/169691057 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f213815e990 0x7f213815f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:06.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.279+0000 7fe6bd064640 1 -- 172.21.15.112:0/650544257 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe698005290 con 0x7fe6b806dc40 2024-09-21T00:26:06.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.279+0000 7f213d30e640 1 -- 172.21.15.112:0/169691057 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f21380cf3b0 msgr2=0x7f213815f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:06.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.279+0000 7f213d30e640 1 --2- 172.21.15.112:0/169691057 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f21380cf3b0 0x7f213815f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:06.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.279+0000 7f213d30e640 1 -- 172.21.15.112:0/169691057 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f213815e990 msgr2=0x7f213815f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:06.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.279+0000 7f213d30e640 1 --2- 172.21.15.112:0/169691057 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f213815e990 0x7f213815f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:06.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.279+0000 7f213d30e640 1 -- 172.21.15.112:0/169691057 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21381619c0 con 0x7f213806e330 2024-09-21T00:26:06.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.279+0000 7fe69f7fe640 1 -- 172.21.15.112:0/650544257 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe6b0011cf0 con 0x7fe6b806dc40 2024-09-21T00:26:06.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.280+0000 7fe69f7fe640 1 --2- 172.21.15.112:0/650544257 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fe6a4078b90 0x7fe6a407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:06.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.280+0000 7fe69f7fe640 1 -- 172.21.15.112:0/650544257 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fe6b009bd60 con 0x7fe6b806dc40 2024-09-21T00:26:06.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.280+0000 7fe6b6d9d640 1 --2- 172.21.15.112:0/650544257 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fe6a4078b90 0x7fe6a407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:06.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.281+0000 7ff07effd640 1 -- 172.21.15.112:0/357097098 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff084037050 con 0x7ff088160c00 2024-09-21T00:26:06.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.281+0000 7ff09028b640 1 -- 172.21.15.112:0/357097098 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0880b3720 con 0x7ff088160c00 2024-09-21T00:26:06.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.281+0000 7ff09028b640 1 -- 172.21.15.112:0/357097098 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0880b3ce0 con 0x7ff088160c00 2024-09-21T00:26:06.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.281+0000 7f213d30e640 1 --2- 172.21.15.112:0/169691057 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f213806e330 0x7f2138157d50 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f212800b410 tx=0x7f212800b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:06.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.282+0000 7fe6b6d9d640 1 --2- 172.21.15.112:0/650544257 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fe6a4078b90 0x7fe6a407b0a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe6a8033000 tx=0x7fe6a8004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:06.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.282+0000 7f212dffb640 1 -- 172.21.15.112:0/169691057 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2128015020 con 0x7f213806e330 2024-09-21T00:26:06.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.282+0000 7f212dffb640 1 -- 172.21.15.112:0/169691057 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2128011920 con 0x7f213806e330 2024-09-21T00:26:06.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.282+0000 7f213f572640 1 -- 172.21.15.112:0/169691057 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2138161ca0 con 0x7f213806e330 2024-09-21T00:26:06.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.282+0000 7f212dffb640 1 -- 172.21.15.112:0/169691057 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2128015020 con 0x7f213806e330 2024-09-21T00:26:06.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.282+0000 7f213f572640 1 -- 172.21.15.112:0/169691057 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f21380c8df0 con 0x7f213806e330 2024-09-21T00:26:06.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.283+0000 7ff09028b640 1 -- 172.21.15.112:0/357097098 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff08806e650 con 0x7ff088160c00 2024-09-21T00:26:06.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.284+0000 7f213f572640 1 -- 172.21.15.112:0/169691057 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2118005290 con 0x7f213806e330 2024-09-21T00:26:06.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.285+0000 7f212dffb640 1 -- 172.21.15.112:0/169691057 <== mon.2 v2:172.21.15.187:3301/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2128018020 con 0x7f213806e330 2024-09-21T00:26:06.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.286+0000 7f212dffb640 1 --2- 172.21.15.112:0/169691057 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f211c078f90 0x7f211c07b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:06.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.286+0000 7f212dffb640 1 -- 172.21.15.112:0/169691057 <== mon.2 v2:172.21.15.187:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f2128011630 con 0x7f213806e330 2024-09-21T00:26:06.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.287+0000 7f213cb0d640 1 --2- 172.21.15.112:0/169691057 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f211c078f90 0x7f211c07b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:06.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.287+0000 7f213cb0d640 1 --2- 172.21.15.112:0/169691057 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f211c078f90 0x7f211c07b4a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f21380caa90 tx=0x7f2134009060 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:06.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.288+0000 7fe69f7fe640 1 -- 172.21.15.112:0/650544257 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe6b0016030 con 0x7fe6b806dc40 2024-09-21T00:26:06.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.288+0000 7ff07effd640 1 -- 172.21.15.112:0/357097098 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff084054070 con 0x7ff088160c00 2024-09-21T00:26:06.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.288+0000 7ff07effd640 1 -- 172.21.15.112:0/357097098 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff084037050 con 0x7ff088160c00 2024-09-21T00:26:06.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.292+0000 7ff07effd640 1 -- 172.21.15.112:0/357097098 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff084030850 con 0x7ff088160c00 2024-09-21T00:26:06.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.294+0000 7f212dffb640 1 -- 172.21.15.112:0/169691057 <== mon.2 v2:172.21.15.187:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f212801f020 con 0x7f213806e330 2024-09-21T00:26:06.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.293+0000 7ff07effd640 1 --2- 172.21.15.112:0/357097098 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff06c078dc0 0x7ff06c07b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:06.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.293+0000 7ff07effd640 1 -- 172.21.15.112:0/357097098 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7ff0840cd710 con 0x7ff088160c00 2024-09-21T00:26:06.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.294+0000 7ff08d826640 1 --2- 172.21.15.112:0/357097098 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff06c078dc0 0x7ff06c07b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:06.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.298+0000 7ff08d826640 1 --2- 172.21.15.112:0/357097098 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff06c078dc0 0x7ff06c07b2d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff088070b30 tx=0x7ff080009260 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:06.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.298+0000 7ff07effd640 1 -- 172.21.15.112:0/357097098 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ff08806d730 con 0x7ff088160c00 2024-09-21T00:26:06.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.326+0000 7fb107fff640 1 -- 172.21.15.112:0/1980026771 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb100031bf0 con 0x7fb10800bc30 2024-09-21T00:26:06.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.326+0000 7fb1165b3640 1 -- 172.21.15.112:0/1980026771 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb10800bc30 msgr2=0x7fb1080df180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:06.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.326+0000 7fb1165b3640 1 --2- 172.21.15.112:0/1980026771 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb10800bc30 0x7fb1080df180 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb1000098e0 tx=0x7fb10002a520 comp rx=0 tx=0).stop 2024-09-21T00:26:06.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.326+0000 7fb1165b3640 1 -- 172.21.15.112:0/1980026771 shutdown_connections 2024-09-21T00:26:06.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.326+0000 7fb1165b3640 1 --2- 172.21.15.112:0/1980026771 >> v2:172.21.15.112:3300/0 conn(0x7fb108009ba0 0x7fb108005c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:06.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.326+0000 7fb1165b3640 1 --2- 172.21.15.112:0/1980026771 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb10800bc30 0x7fb1080df180 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:06.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.326+0000 7fb1165b3640 1 -- 172.21.15.112:0/1980026771 >> 172.21.15.112:0/1980026771 conn(0x7fb108051420 msgr2=0x7fb108051860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:06.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.327+0000 7fb1165b3640 1 -- 172.21.15.112:0/1980026771 shutdown_connections 2024-09-21T00:26:06.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.327+0000 7f7ea2bc6640 1 -- 172.21.15.112:0/3905113982 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e9c0afff0 msgr2=0x7f7e9c0b0400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:06.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.327+0000 7f7ea2bc6640 1 --2- 172.21.15.112:0/3905113982 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e9c0afff0 0x7f7e9c0b0400 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c072780 tx=0x7f7e9002b270 comp rx=0 tx=0).stop 2024-09-21T00:26:06.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.327+0000 7fefdffff640 1 -- 172.21.15.112:0/2313724593 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefe40047a0 con 0x7fefe8162240 2024-09-21T00:26:06.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.327+0000 7fb1165b3640 1 -- 172.21.15.112:0/1980026771 wait complete. 2024-09-21T00:26:06.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.327+0000 7f7ea2bc6640 1 -- 172.21.15.112:0/3905113982 shutdown_connections 2024-09-21T00:26:06.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.327+0000 7f7ea2bc6640 1 --2- 172.21.15.112:0/3905113982 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e9c0afff0 0x7f7e9c0b0400 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:06.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.327+0000 7f7ea2bc6640 1 --2- 172.21.15.112:0/3905113982 >> v2:172.21.15.187:3300/0 conn(0x7f7e9c15dc10 0x7f7e9c074db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:06.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.327+0000 7f7ea2bc6640 1 -- 172.21.15.112:0/3905113982 >> 172.21.15.112:0/3905113982 conn(0x7f7e9c0731e0 msgr2=0x7f7e9c0b5150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:06.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.328+0000 7fefee229640 1 -- 172.21.15.112:0/2313724593 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fefe8162240 msgr2=0x7fefe8162650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:06.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.328+0000 7f7ea2bc6640 1 -- 172.21.15.112:0/3905113982 shutdown_connections 2024-09-21T00:26:06.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.328+0000 7fefee229640 1 --2- 172.21.15.112:0/2313724593 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fefe8162240 0x7fefe8162650 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fefe80b6dd0 tx=0x7fefe402a550 comp rx=0 tx=0).stop 2024-09-21T00:26:06.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.328+0000 7fefee229640 1 -- 172.21.15.112:0/2313724593 shutdown_connections 2024-09-21T00:26:06.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.328+0000 7fefee229640 1 --2- 172.21.15.112:0/2313724593 >> v2:172.21.15.187:3300/0 conn(0x7fefe80cef00 0x7fefe80cf310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:06.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.328+0000 7fefee229640 1 --2- 172.21.15.112:0/2313724593 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fefe8162240 0x7fefe8162650 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:06.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.328+0000 7fefee229640 1 --2- 172.21.15.112:0/2313724593 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fefe815f6d0 0x7fefe815e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:06.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.328+0000 7fefee229640 1 -- 172.21.15.112:0/2313724593 >> 172.21.15.112:0/2313724593 conn(0x7fefe80d41e0 msgr2=0x7fefe80d45f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:06.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.328+0000 7fefee229640 1 -- 172.21.15.112:0/2313724593 shutdown_connections 2024-09-21T00:26:06.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.328+0000 7f7ea2bc6640 1 -- 172.21.15.112:0/3905113982 wait complete. 2024-09-21T00:26:06.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.328+0000 7fefee229640 1 -- 172.21.15.112:0/2313724593 wait complete. 2024-09-21T00:26:06.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.330+0000 7fb1165b3640 1 Processor -- start 2024-09-21T00:26:06.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.331+0000 7f7ea2bc6640 1 Processor -- start 2024-09-21T00:26:06.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.331+0000 7fb1165b3640 1 -- start start 2024-09-21T00:26:06.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.332+0000 7fefee229640 1 Processor -- start 2024-09-21T00:26:06.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.331+0000 7f7ea2bc6640 1 -- start start 2024-09-21T00:26:06.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.332+0000 7fefee229640 1 -- start start 2024-09-21T00:26:06.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.333+0000 7fb1165b3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb108009ba0 0x7fb108020360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:06.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.333+0000 7fb1165b3640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb10800bc30 0x7fb1080ee710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:06.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.333+0000 7fb1165b3640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb108036400 0x7fb1080eec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:06.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.333+0000 7fb1165b3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb108006aa0 con 0x7fb108009ba0 2024-09-21T00:26:06.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.333+0000 7fb1165b3640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb108006920 con 0x7fb108036400 2024-09-21T00:26:06.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.333+0000 7fb1165b3640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fb108006c20 con 0x7fb10800bc30 2024-09-21T00:26:06.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.333+0000 7f3eceffd640 1 -- 172.21.15.112:0/1491727359 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec8030e20 con 0x7f3ed0070070 2024-09-21T00:26:06.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.333+0000 7fb1155b1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb108009ba0 0x7fb108020360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:06.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.333+0000 7f7ea2bc6640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7e9c0afff0 0x7f7e9c14acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:06.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.333+0000 7f7ea2bc6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e9c15dc10 0x7f7e9c151e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:06.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.333+0000 7f7ea2bc6640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7e9c15e170 0x7f7e9c152390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:06.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.333+0000 7f7ea2bc6640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7e9c06ad40 con 0x7f7e9c15dc10 2024-09-21T00:26:06.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.333+0000 7f7ea2bc6640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7e9c15b080 con 0x7f7e9c0afff0 2024-09-21T00:26:06.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.333+0000 7f7ea2bc6640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f7e9c1501c0 con 0x7f7e9c15e170 2024-09-21T00:26:06.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.333+0000 7f3ed678b640 1 -- 172.21.15.112:0/1491727359 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ed0070070 msgr2=0x7f3ed006ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:06.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.333+0000 7f3ed678b640 1 --2- 172.21.15.112:0/1491727359 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ed0070070 0x7f3ed006ddf0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f3ed006a1b0 tx=0x7f3ec802a980 comp rx=0 tx=0).stop 2024-09-21T00:26:06.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.333+0000 7fb1155b1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb108009ba0 0x7fb108020360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60020/0 (socket says 172.21.15.112:60020) 2024-09-21T00:26:06.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.333+0000 7fb1155b1640 1 -- 172.21.15.112:0/1599951844 learned_addr learned my addr 172.21.15.112:0/1599951844 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:26:06.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.334+0000 7f3ed678b640 1 -- 172.21.15.112:0/1491727359 shutdown_connections 2024-09-21T00:26:06.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.334+0000 7f3ed678b640 1 --2- 172.21.15.112:0/1491727359 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3ed0070480 0x7f3ed006e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:06.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.334+0000 7f3ed678b640 1 --2- 172.21.15.112:0/1491727359 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ed0070070 0x7f3ed006ddf0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:06.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.334+0000 7f3ed678b640 1 --2- 172.21.15.112:0/1491727359 >> v2:172.21.15.187:3300/0 conn(0x7f3ed0070e80 0x7f3ed015aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:06.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.334+0000 7f3ed678b640 1 -- 172.21.15.112:0/1491727359 >> 172.21.15.112:0/1491727359 conn(0x7f3ed00b1b90 msgr2=0x7f3ed00b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:06.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.334+0000 7f7e9bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e9c15dc10 0x7f7e9c151e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:06.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.334+0000 7f7e9bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e9c15dc10 0x7f7e9c151e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60030/0 (socket says 172.21.15.112:60030) 2024-09-21T00:26:06.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.334+0000 7f7e9bfff640 1 -- 172.21.15.112:0/3339375509 learned_addr learned my addr 172.21.15.112:0/3339375509 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:26:06.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.334+0000 7fb115db2640 1 --2- 172.21.15.112:0/1599951844 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb108036400 0x7fb1080eec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:06.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.334+0000 7fb114db0640 1 --2- 172.21.15.112:0/1599951844 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb10800bc30 0x7fb1080ee710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:06.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.334+0000 7f7ea1163640 1 --2- 172.21.15.112:0/3339375509 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7e9c15e170 0x7f7e9c152390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:06.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.334+0000 7f7e9bfff640 1 -- 172.21.15.112:0/3339375509 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7e9c15e170 msgr2=0x7f7e9c152390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:06.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.334+0000 7f7e9bfff640 1 --2- 172.21.15.112:0/3339375509 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7e9c15e170 0x7f7e9c152390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:06.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.334+0000 7f7e9bfff640 1 -- 172.21.15.112:0/3339375509 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7e9c0afff0 msgr2=0x7f7e9c14acf0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T00:26:06.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.334+0000 7f7e9bfff640 1 --2- 172.21.15.112:0/3339375509 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7e9c0afff0 0x7f7e9c14acf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:06.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.334+0000 7f7e9bfff640 1 -- 172.21.15.112:0/3339375509 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e9c1f5df0 con 0x7f7e9c15dc10 2024-09-21T00:26:06.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.334+0000 7f3ed678b640 1 -- 172.21.15.112:0/1491727359 shutdown_connections 2024-09-21T00:26:06.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.334+0000 7f7e9bfff640 1 --2- 172.21.15.112:0/3339375509 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e9c15dc10 0x7f7e9c151e50 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c00c9a0 tx=0x7f7e8c00ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:06.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.334+0000 7f3ed678b640 1 -- 172.21.15.112:0/1491727359 wait complete. 2024-09-21T00:26:06.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.335+0000 7fb1155b1640 1 -- 172.21.15.112:0/1599951844 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb10800bc30 msgr2=0x7fb1080ee710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:06.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.335+0000 7fb1155b1640 1 --2- 172.21.15.112:0/1599951844 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb10800bc30 0x7fb1080ee710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:06.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.335+0000 7fb1155b1640 1 -- 172.21.15.112:0/1599951844 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb108036400 msgr2=0x7fb1080eec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:06.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.335+0000 7fb1155b1640 1 --2- 172.21.15.112:0/1599951844 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb108036400 0x7fb1080eec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:06.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.335+0000 7fb1155b1640 1 -- 172.21.15.112:0/1599951844 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1080208a0 con 0x7fb108009ba0 2024-09-21T00:26:06.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.335+0000 7fefee229640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fefe80cef00 0x7fefe8152d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:06.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.335+0000 7fb1155b1640 1 --2- 172.21.15.112:0/1599951844 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb108009ba0 0x7fb108020360 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fb10002ab10 tx=0x7fb100005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:06.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.335+0000 7fb105ffb640 1 -- 172.21.15.112:0/1599951844 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb100004220 con 0x7fb108009ba0 2024-09-21T00:26:06.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.335+0000 7fb105ffb640 1 -- 172.21.15.112:0/1599951844 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb1000043c0 con 0x7fb108009ba0 2024-09-21T00:26:06.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.335+0000 7f7e997fa640 1 -- 172.21.15.112:0/3339375509 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e8c007bb0 con 0x7f7e9c15dc10 2024-09-21T00:26:06.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.335+0000 7fb1165b3640 1 -- 172.21.15.112:0/1599951844 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1080ed670 con 0x7fb108009ba0 2024-09-21T00:26:06.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.335+0000 7fb105ffb640 1 -- 172.21.15.112:0/1599951844 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb100031bb0 con 0x7fb108009ba0 2024-09-21T00:26:06.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.335+0000 7fb1165b3640 1 -- 172.21.15.112:0/1599951844 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1080edc30 con 0x7fb108009ba0 2024-09-21T00:26:06.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.336+0000 7f11fc8dd640 1 -- 172.21.15.112:0/3660691049 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f11f0057c80 msgr2=0x7f11f00e2f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:06.342 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.336+0000 7f11fc8dd640 1 --2- 172.21.15.112:0/3660691049 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f11f0057c80 0x7f11f00e2f40 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f11ec009860 tx=0x7f11ec02a5a0 comp rx=0 tx=0).stop 2024-09-21T00:26:06.342 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.335+0000 7f7ea2bc6640 1 -- 172.21.15.112:0/3339375509 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e9c1f60d0 con 0x7f7e9c15dc10 2024-09-21T00:26:06.342 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.335+0000 7f7ea2bc6640 1 -- 172.21.15.112:0/3339375509 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e9c1f6690 con 0x7f7e9c15dc10 2024-09-21T00:26:06.342 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.337+0000 7f7e997fa640 1 -- 172.21.15.112:0/3339375509 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7e8c007e40 con 0x7f7e9c15dc10 2024-09-21T00:26:06.342 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.337+0000 7f7e997fa640 1 -- 172.21.15.112:0/3339375509 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e8c005280 con 0x7f7e9c15dc10 2024-09-21T00:26:06.343 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.337+0000 7f3ed678b640 1 Processor -- start 2024-09-21T00:26:06.343 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.337+0000 7f11fc8dd640 1 -- 172.21.15.112:0/3660691049 shutdown_connections 2024-09-21T00:26:06.343 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.337+0000 7f11fc8dd640 1 --2- 172.21.15.112:0/3660691049 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f11f0057c80 0x7f11f00e2f40 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:06.343 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.337+0000 7f11fc8dd640 1 -- 172.21.15.112:0/3660691049 >> 172.21.15.112:0/3660691049 conn(0x7f11f003a5f0 msgr2=0x7f11f003aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:06.343 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.337+0000 7fefee229640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fefe815f6d0 0x7fefe80b2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:06.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.337+0000 7fefee229640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fefe8162240 0x7fefe80b3430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:06.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.338+0000 7fefee229640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fefe815eb00 con 0x7fefe8162240 2024-09-21T00:26:06.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.338+0000 7fefee229640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fefe806dce0 con 0x7fefe815f6d0 2024-09-21T00:26:06.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.338+0000 7fefee229640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fefe8162b90 con 0x7fefe80cef00 2024-09-21T00:26:06.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.338+0000 7fefed227640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fefe80cef00 0x7fefe8152d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:06.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.338+0000 7fefed227640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fefe80cef00 0x7fefe8152d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:60736/0 (socket says 172.21.15.112:60736) 2024-09-21T00:26:06.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.338+0000 7f3ed678b640 1 -- start start 2024-09-21T00:26:06.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.338+0000 7fefed227640 1 -- 172.21.15.112:0/17789537 learned_addr learned my addr 172.21.15.112:0/17789537 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:26:06.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.338+0000 7fefeca26640 1 --2- 172.21.15.112:0/17789537 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fefe815f6d0 0x7fefe80b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:06.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.338+0000 7fefed227640 1 -- 172.21.15.112:0/17789537 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fefe815f6d0 msgr2=0x7fefe80b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:06.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.338+0000 7fefed227640 1 --2- 172.21.15.112:0/17789537 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fefe815f6d0 0x7fefe80b2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:06.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.338+0000 7fefed227640 1 -- 172.21.15.112:0/17789537 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fefe8162240 msgr2=0x7fefe80b3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:06.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.338+0000 7fefed227640 1 --2- 172.21.15.112:0/17789537 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fefe8162240 0x7fefe80b3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:06.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.338+0000 7fefed227640 1 -- 172.21.15.112:0/17789537 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefe81f8d90 con 0x7fefe80cef00 2024-09-21T00:26:06.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.338+0000 7fb105ffb640 1 -- 172.21.15.112:0/1599951844 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb1000047a0 con 0x7fb108009ba0 2024-09-21T00:26:06.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.338+0000 7f11fc8dd640 1 -- 172.21.15.112:0/3660691049 shutdown_connections 2024-09-21T00:26:06.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.338+0000 7fefed227640 1 --2- 172.21.15.112:0/17789537 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fefe80cef00 0x7fefe8152d80 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fefe000ea50 tx=0x7fefe000ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:06.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.339+0000 7f11fc8dd640 1 -- 172.21.15.112:0/3660691049 wait complete. 2024-09-21T00:26:06.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.339+0000 7fb1165b3640 1 -- 172.21.15.112:0/1599951844 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb108005c60 con 0x7fb108009ba0 2024-09-21T00:26:06.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.339+0000 7fb105ffb640 1 --2- 172.21.15.112:0/1599951844 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fb0f4078bc0 0x7fb0f407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:06.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.339+0000 7fb105ffb640 1 -- 172.21.15.112:0/1599951844 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fb1000bbee0 con 0x7fb108009ba0 2024-09-21T00:26:06.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.341+0000 7fb114db0640 1 --2- 172.21.15.112:0/1599951844 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fb0f4078bc0 0x7fb0f407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:06.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.339+0000 7fefddffb640 1 -- 172.21.15.112:0/17789537 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefe000cdc0 con 0x7fefe80cef00 2024-09-21T00:26:06.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.339+0000 7fefddffb640 1 -- 172.21.15.112:0/17789537 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fefe0011560 con 0x7fefe80cef00 2024-09-21T00:26:06.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.339+0000 7fefddffb640 1 -- 172.21.15.112:0/17789537 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefe0011990 con 0x7fefe80cef00 2024-09-21T00:26:06.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.339+0000 7fefee229640 1 -- 172.21.15.112:0/17789537 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefe81f9070 con 0x7fefe80cef00 2024-09-21T00:26:06.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.339+0000 7fefee229640 1 -- 172.21.15.112:0/17789537 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fefe81f9630 con 0x7fefe80cef00 2024-09-21T00:26:06.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.342+0000 7f7ea2bc6640 1 -- 172.21.15.112:0/3339375509 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e7c005290 con 0x7f7e9c15dc10 2024-09-21T00:26:06.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.341+0000 7f3ed678b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3ed0070070 0x7f3ed0158bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:06.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.341+0000 7f3ed678b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3ed0070480 0x7f3ed00b3730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:06.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.341+0000 7f3ed678b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ed0070e80 0x7f3ed00c9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:06.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.341+0000 7f3ed678b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed006f310 con 0x7f3ed0070e80 2024-09-21T00:26:06.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.341+0000 7f3ed678b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed006f190 con 0x7f3ed0070480 2024-09-21T00:26:06.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.341+0000 7f3ed678b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3ed006f490 con 0x7f3ed0070070 2024-09-21T00:26:06.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.342+0000 7f3ecffff640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3ed0070070 0x7f3ed0158bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:06.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.342+0000 7f3ecffff640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3ed0070070 0x7f3ed0158bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:60740/0 (socket says 172.21.15.112:60740) 2024-09-21T00:26:06.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.342+0000 7f3ecffff640 1 -- 172.21.15.112:0/2460729556 learned_addr learned my addr 172.21.15.112:0/2460729556 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:26:06.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.342+0000 7fb114db0640 1 --2- 172.21.15.112:0/1599951844 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fb0f4078bc0 0x7fb0f407b0d0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb1080de580 tx=0x7fb10c005f90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:06.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.342+0000 7f11fc8dd640 1 Processor -- start 2024-09-21T00:26:06.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.342+0000 7f3ecffff640 1 -- 172.21.15.112:0/2460729556 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3ed0070480 msgr2=0x7f3ed00b3730 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T00:26:06.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.342+0000 7f3ecffff640 1 --2- 172.21.15.112:0/2460729556 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3ed0070480 0x7f3ed00b3730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:06.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.342+0000 7f3ecffff640 1 -- 172.21.15.112:0/2460729556 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ed0070e80 msgr2=0x7f3ed00c9cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:06.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.342+0000 7f3ecffff640 1 --2- 172.21.15.112:0/2460729556 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ed0070e80 0x7f3ed00c9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:06.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.342+0000 7f3ecffff640 1 -- 172.21.15.112:0/2460729556 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ed0155ab0 con 0x7f3ed0070070 2024-09-21T00:26:06.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.342+0000 7f7e997fa640 1 -- 172.21.15.112:0/3339375509 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f7e8c005420 con 0x7f7e9c15dc10 2024-09-21T00:26:06.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.342+0000 7fefddffb640 1 -- 172.21.15.112:0/17789537 <== mon.2 v2:172.21.15.187:3301/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fefe0011b30 con 0x7fefe80cef00 2024-09-21T00:26:06.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.342+0000 7f11fc8dd640 1 -- start start 2024-09-21T00:26:06.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.342+0000 7f3ecffff640 1 --2- 172.21.15.112:0/2460729556 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3ed0070070 0x7f3ed0158bc0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3ec000d770 tx=0x7f3ec000dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:06.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.343+0000 7f3eccff9640 1 -- 172.21.15.112:0/2460729556 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec0017070 con 0x7f3ed0070070 2024-09-21T00:26:06.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.343+0000 7f3ed678b640 1 -- 172.21.15.112:0/2460729556 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ed0155d90 con 0x7f3ed0070070 2024-09-21T00:26:06.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.343+0000 7f3ed678b640 1 -- 172.21.15.112:0/2460729556 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ed0156070 con 0x7f3ed0070070 2024-09-21T00:26:06.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.343+0000 7f3eccff9640 1 -- 172.21.15.112:0/2460729556 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3ec0009d20 con 0x7f3ed0070070 2024-09-21T00:26:06.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.343+0000 7f3eccff9640 1 -- 172.21.15.112:0/2460729556 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec00106c0 con 0x7f3ed0070070 2024-09-21T00:26:06.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.343+0000 7fefddffb640 1 --2- 172.21.15.112:0/17789537 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fefcc078c10 0x7fefcc07b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:06.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.343+0000 7fefddffb640 1 -- 172.21.15.112:0/17789537 <== mon.2 v2:172.21.15.187:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fefe009bab0 con 0x7fefe80cef00 2024-09-21T00:26:06.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.343+0000 7fefeca26640 1 --2- 172.21.15.112:0/17789537 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fefcc078c10 0x7fefcc07b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:06.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.343+0000 7f7e997fa640 1 --2- 172.21.15.112:0/3339375509 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f7e88078c60 0x7f7e8807b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:06.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.343+0000 7f7e997fa640 1 -- 172.21.15.112:0/3339375509 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f7e8c09b420 con 0x7f7e9c15dc10 2024-09-21T00:26:06.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.343+0000 7f7ea0962640 1 --2- 172.21.15.112:0/3339375509 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f7e88078c60 0x7f7e8807b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:06.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.342+0000 7fefd37fe640 1 -- 172.21.15.112:0/17789537 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefc0005290 con 0x7fefe80cef00 2024-09-21T00:26:06.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.344+0000 7f7ea0962640 1 --2- 172.21.15.112:0/3339375509 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f7e88078c60 0x7f7e8807b170 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f7e94001bb0 tx=0x7f7e94001640 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:06.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.345+0000 7f3ed678b640 1 -- 172.21.15.112:0/2460729556 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3eb0005290 con 0x7f3ed0070070 2024-09-21T00:26:06.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.345+0000 7fefeca26640 1 --2- 172.21.15.112:0/17789537 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fefcc078c10 0x7fefcc07b120 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fefe80ceed0 tx=0x7fefd8005f90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:06.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.345+0000 7f3eccff9640 1 -- 172.21.15.112:0/2460729556 <== mon.2 v2:172.21.15.187:3301/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3ec001a070 con 0x7f3ed0070070 2024-09-21T00:26:06.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.346+0000 7f11fc8dd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f11f0054360 0x7f11f0187e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:06.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.346+0000 7f11fc8dd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f11f0056da0 0x7f11f00db750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:06.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.346+0000 7f11f6d9d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f11f0054360 0x7f11f0187e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:06.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.346+0000 7f11f6d9d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f11f0054360 0x7f11f0187e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:60748/0 (socket says 172.21.15.112:60748) 2024-09-21T00:26:06.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.346+0000 7f3eccff9640 1 --2- 172.21.15.112:0/2460729556 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f3ebc078be0 0x7f3ebc07b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:06.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.346+0000 7f11f659c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f11f0056da0 0x7f11f00db750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:06.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.346+0000 7f11f659c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f11f0056da0 0x7f11f00db750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:53714/0 (socket says 172.21.15.112:53714) 2024-09-21T00:26:06.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.346+0000 7f3eccff9640 1 -- 172.21.15.112:0/2460729556 <== mon.2 v2:172.21.15.187:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f3ec009b0a0 con 0x7f3ed0070070 2024-09-21T00:26:06.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.346+0000 7f3ecf7fe640 1 --2- 172.21.15.112:0/2460729556 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f3ebc078be0 0x7f3ebc07b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:06.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.346+0000 7f11f6d9d640 1 -- 172.21.15.112:0/1055008448 learned_addr learned my addr 172.21.15.112:0/1055008448 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:26:06.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.346+0000 7f11fc8dd640 1 --2- 172.21.15.112:0/1055008448 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11f0057c80 0x7f11f004e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:06.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.347+0000 7f11fc8dd640 1 -- 172.21.15.112:0/1055008448 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f11f00072a0 con 0x7f11f0057c80 2024-09-21T00:26:06.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.347+0000 7f11fc8dd640 1 -- 172.21.15.112:0/1055008448 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f11f003c760 con 0x7f11f0056da0 2024-09-21T00:26:06.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.347+0000 7f11f759e640 1 --2- 172.21.15.112:0/1055008448 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11f0057c80 0x7f11f004e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:06.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.347+0000 7f11fc8dd640 1 -- 172.21.15.112:0/1055008448 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f11f0007420 con 0x7f11f0054360 2024-09-21T00:26:06.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.347+0000 7f11f659c640 1 -- 172.21.15.112:0/1055008448 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f11f0054360 msgr2=0x7f11f0187e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:06.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.347+0000 7f11f659c640 1 --2- 172.21.15.112:0/1055008448 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f11f0054360 0x7f11f0187e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:06.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.347+0000 7f11f659c640 1 -- 172.21.15.112:0/1055008448 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11f0057c80 msgr2=0x7f11f004e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:06.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.347+0000 7f11f659c640 1 --2- 172.21.15.112:0/1055008448 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11f0057c80 0x7f11f004e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:06.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.347+0000 7f11f659c640 1 -- 172.21.15.112:0/1055008448 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11f004d7a0 con 0x7f11f0056da0 2024-09-21T00:26:06.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.347+0000 7f11f659c640 1 --2- 172.21.15.112:0/1055008448 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f11f0056da0 0x7f11f00db750 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f11f8052910 tx=0x7f11f806a0e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:06.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.348+0000 7f3ecf7fe640 1 --2- 172.21.15.112:0/2460729556 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f3ebc078be0 0x7f3ebc07b0f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3ec8035000 tx=0x7f3ec8004ab0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:06.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.348+0000 7f11df7fe640 1 -- 172.21.15.112:0/1055008448 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f806adb0 con 0x7f11f0056da0 2024-09-21T00:26:06.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.348+0000 7f11df7fe640 1 -- 172.21.15.112:0/1055008448 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f11f8076270 con 0x7f11f0056da0 2024-09-21T00:26:06.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.348+0000 7f11fc8dd640 1 -- 172.21.15.112:0/1055008448 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11f004da80 con 0x7f11f0056da0 2024-09-21T00:26:06.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.348+0000 7f11df7fe640 1 -- 172.21.15.112:0/1055008448 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f80766a0 con 0x7f11f0056da0 2024-09-21T00:26:06.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.348+0000 7f11fc8dd640 1 -- 172.21.15.112:0/1055008448 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11f004e040 con 0x7f11f0056da0 2024-09-21T00:26:06.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.349+0000 7fefddffb640 1 -- 172.21.15.112:0/17789537 <== mon.2 v2:172.21.15.187:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fefe0016030 con 0x7fefe80cef00 2024-09-21T00:26:06.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.349+0000 7f11fc8dd640 1 -- 172.21.15.112:0/1055008448 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11f00e2f40 con 0x7f11f0056da0 2024-09-21T00:26:06.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.350+0000 7f11df7fe640 1 -- 172.21.15.112:0/1055008448 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f11f8093400 con 0x7f11f0056da0 2024-09-21T00:26:06.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.350+0000 7fb105ffb640 1 -- 172.21.15.112:0/1599951844 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb10003f030 con 0x7fb108009ba0 2024-09-21T00:26:06.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.350+0000 7f11df7fe640 1 --2- 172.21.15.112:0/1055008448 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f11e4078bc0 0x7f11e407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:06.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.351+0000 7f11df7fe640 1 -- 172.21.15.112:0/1055008448 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f11f8115f40 con 0x7f11f0056da0 2024-09-21T00:26:06.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.351+0000 7f11f6d9d640 1 --2- 172.21.15.112:0/1055008448 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f11e4078bc0 0x7f11e407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:06.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.351+0000 7f11f6d9d640 1 --2- 172.21.15.112:0/1055008448 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f11e4078bc0 0x7f11e407b0d0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f11e80046b0 tx=0x7f11e8007f20 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:06.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.355+0000 7f3eccff9640 1 -- 172.21.15.112:0/2460729556 <== mon.2 v2:172.21.15.187:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3ec0063280 con 0x7f3ed0070070 2024-09-21T00:26:06.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.355+0000 7f11df7fe640 1 -- 172.21.15.112:0/1055008448 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f11f808f030 con 0x7f11f0056da0 2024-09-21T00:26:06.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.360+0000 7f7e997fa640 1 -- 172.21.15.112:0/3339375509 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f7e8c014030 con 0x7f7e9c15dc10 2024-09-21T00:26:06.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.696+0000 7fe6bd064640 1 -- 172.21.15.112:0/650544257 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fe698005580 con 0x7fe6b806dc40 2024-09-21T00:26:06.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.696+0000 7fe69f7fe640 1 -- 172.21.15.112:0/650544257 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fe6b0063dc0 con 0x7fe6b806dc40 2024-09-21T00:26:06.697 INFO:teuthology.orchestra.run.smithi112.stdout:64424509449 2024-09-21T00:26:06.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.700+0000 7fe69d7fa640 1 -- 172.21.15.112:0/650544257 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fe6a4078b90 msgr2=0x7fe6a407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:06.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.700+0000 7fe69d7fa640 1 --2- 172.21.15.112:0/650544257 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fe6a4078b90 0x7fe6a407b0a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe6a8033000 tx=0x7fe6a8004730 comp rx=0 tx=0).stop 2024-09-21T00:26:06.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.701+0000 7fe69d7fa640 1 -- 172.21.15.112:0/650544257 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6b806dc40 msgr2=0x7fe6b814a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:06.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.701+0000 7fe69d7fa640 1 --2- 172.21.15.112:0/650544257 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6b806dc40 0x7fe6b814a0d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe6b000ed30 tx=0x7fe6b000c640 comp rx=0 tx=0).stop 2024-09-21T00:26:06.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.701+0000 7fe69d7fa640 1 -- 172.21.15.112:0/650544257 shutdown_connections 2024-09-21T00:26:06.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.701+0000 7fe69d7fa640 1 --2- 172.21.15.112:0/650544257 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe6b815f3f0 0x7fe6b8162350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:06.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.701+0000 7fe69d7fa640 1 --2- 172.21.15.112:0/650544257 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fe6a4078b90 0x7fe6a407b0a0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:06.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.701+0000 7fe69d7fa640 1 --2- 172.21.15.112:0/650544257 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6b806dc40 0x7fe6b814a0d0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:06.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.701+0000 7fe69d7fa640 1 --2- 172.21.15.112:0/650544257 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe6b806d830 0x7fe6b81fa620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:06.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.701+0000 7fe69d7fa640 1 -- 172.21.15.112:0/650544257 >> 172.21.15.112:0/650544257 conn(0x7fe6b80cd460 msgr2=0x7fe6b8157020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:06.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.701+0000 7fe69d7fa640 1 -- 172.21.15.112:0/650544257 shutdown_connections 2024-09-21T00:26:06.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.702+0000 7fe69d7fa640 1 -- 172.21.15.112:0/650544257 wait complete. 2024-09-21T00:26:06.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.707+0000 7f213f572640 1 -- 172.21.15.112:0/169691057 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f2118005850 con 0x7f213806e330 2024-09-21T00:26:06.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.708+0000 7f212dffb640 1 -- 172.21.15.112:0/169691057 <== mon.2 v2:172.21.15.187:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f212801f020 con 0x7f213806e330 2024-09-21T00:26:06.709 INFO:teuthology.orchestra.run.smithi112.stdout:64424509449 2024-09-21T00:26:06.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.713+0000 7f21237fe640 1 -- 172.21.15.112:0/169691057 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f211c078f90 msgr2=0x7f211c07b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:06.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.713+0000 7f21237fe640 1 --2- 172.21.15.112:0/169691057 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f211c078f90 0x7f211c07b4a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f21380caa90 tx=0x7f2134009060 comp rx=0 tx=0).stop 2024-09-21T00:26:06.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.713+0000 7f21237fe640 1 -- 172.21.15.112:0/169691057 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f213806e330 msgr2=0x7f2138157d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:06.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.713+0000 7f21237fe640 1 --2- 172.21.15.112:0/169691057 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f213806e330 0x7f2138157d50 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f212800b410 tx=0x7f212800b8d0 comp rx=0 tx=0).stop 2024-09-21T00:26:06.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.713+0000 7f21237fe640 1 -- 172.21.15.112:0/169691057 shutdown_connections 2024-09-21T00:26:06.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.713+0000 7f21237fe640 1 --2- 172.21.15.112:0/169691057 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f211c078f90 0x7f211c07b4a0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:06.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.714+0000 7f21237fe640 1 --2- 172.21.15.112:0/169691057 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f213815e990 0x7f213815f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:06.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.714+0000 7f21237fe640 1 --2- 172.21.15.112:0/169691057 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f21380cf3b0 0x7f213815f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:06.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.714+0000 7f21237fe640 1 --2- 172.21.15.112:0/169691057 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f213806e330 0x7f2138157d50 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:06.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.714+0000 7f21237fe640 1 -- 172.21.15.112:0/169691057 >> 172.21.15.112:0/169691057 conn(0x7f21380735d0 msgr2=0x7f21380cec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:06.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.714+0000 7f21237fe640 1 -- 172.21.15.112:0/169691057 shutdown_connections 2024-09-21T00:26:06.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.714+0000 7f21237fe640 1 -- 172.21.15.112:0/169691057 wait complete. 2024-09-21T00:26:06.720 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509449 got 64424509449 for osd.0 2024-09-21T00:26:06.720 DEBUG:teuthology.parallel:result is None 2024-09-21T00:26:06.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.721+0000 7ff09028b640 1 -- 172.21.15.112:0/357097098 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7ff088070e40 con 0x7ff088160c00 2024-09-21T00:26:06.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.722+0000 7ff07effd640 1 -- 172.21.15.112:0/357097098 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ff084095910 con 0x7ff088160c00 2024-09-21T00:26:06.723 INFO:teuthology.orchestra.run.smithi112.stdout:64424509448 2024-09-21T00:26:06.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.726+0000 7ff09028b640 1 -- 172.21.15.112:0/357097098 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff06c078dc0 msgr2=0x7ff06c07b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:06.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.726+0000 7ff09028b640 1 --2- 172.21.15.112:0/357097098 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff06c078dc0 0x7ff06c07b2d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff088070b30 tx=0x7ff080009260 comp rx=0 tx=0).stop 2024-09-21T00:26:06.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.726+0000 7ff09028b640 1 -- 172.21.15.112:0/357097098 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff088160c00 msgr2=0x7ff0880b4b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:06.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.726+0000 7ff09028b640 1 --2- 172.21.15.112:0/357097098 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff088160c00 0x7ff0880b4b90 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff084000c00 tx=0x7ff084032ed0 comp rx=0 tx=0).stop 2024-09-21T00:26:06.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.726+0000 7ff09028b640 1 -- 172.21.15.112:0/357097098 shutdown_connections 2024-09-21T00:26:06.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.726+0000 7ff09028b640 1 --2- 172.21.15.112:0/357097098 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff08806d320 0x7ff08815e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:06.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.726+0000 7ff09028b640 1 --2- 172.21.15.112:0/357097098 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff0880b50d0 0x7ff08806cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:06.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.726+0000 7ff09028b640 1 --2- 172.21.15.112:0/357097098 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff06c078dc0 0x7ff06c07b2d0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:06.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.726+0000 7ff09028b640 1 --2- 172.21.15.112:0/357097098 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff088160c00 0x7ff0880b4b90 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:06.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.726+0000 7ff09028b640 1 -- 172.21.15.112:0/357097098 >> 172.21.15.112:0/357097098 conn(0x7ff0880735d0 msgr2=0x7ff088071360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:06.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.726+0000 7ff09028b640 1 -- 172.21.15.112:0/357097098 shutdown_connections 2024-09-21T00:26:06.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.726+0000 7ff09028b640 1 -- 172.21.15.112:0/357097098 wait complete. 2024-09-21T00:26:06.732 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509449 got 64424509449 for osd.3 2024-09-21T00:26:06.732 DEBUG:teuthology.parallel:result is None 2024-09-21T00:26:06.743 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509448 for osd.7 2024-09-21T00:26:06.743 DEBUG:teuthology.parallel:result is None 2024-09-21T00:26:06.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.746+0000 7f7ea2bc6640 1 -- 172.21.15.112:0/3339375509 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f7e7c005580 con 0x7f7e9c15dc10 2024-09-21T00:26:06.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.746+0000 7f7e997fa640 1 -- 172.21.15.112:0/3339375509 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f7e8c0636e0 con 0x7f7e9c15dc10 2024-09-21T00:26:06.747 INFO:teuthology.orchestra.run.smithi112.stdout:64424509448 2024-09-21T00:26:06.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.748+0000 7f7ea2bc6640 1 -- 172.21.15.112:0/3339375509 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f7e88078c60 msgr2=0x7f7e8807b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:06.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.748+0000 7f7ea2bc6640 1 --2- 172.21.15.112:0/3339375509 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f7e88078c60 0x7f7e8807b170 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f7e94001bb0 tx=0x7f7e94001640 comp rx=0 tx=0).stop 2024-09-21T00:26:06.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.748+0000 7f7ea2bc6640 1 -- 172.21.15.112:0/3339375509 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e9c15dc10 msgr2=0x7f7e9c151e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:06.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.748+0000 7f7ea2bc6640 1 --2- 172.21.15.112:0/3339375509 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e9c15dc10 0x7f7e9c151e50 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c00c9a0 tx=0x7f7e8c00ce60 comp rx=0 tx=0).stop 2024-09-21T00:26:06.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.749+0000 7f7ea2bc6640 1 -- 172.21.15.112:0/3339375509 shutdown_connections 2024-09-21T00:26:06.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.749+0000 7f7ea2bc6640 1 --2- 172.21.15.112:0/3339375509 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7e9c15e170 0x7f7e9c152390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:06.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.749+0000 7f7ea2bc6640 1 --2- 172.21.15.112:0/3339375509 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f7e88078c60 0x7f7e8807b170 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:06.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.749+0000 7f7ea2bc6640 1 --2- 172.21.15.112:0/3339375509 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e9c15dc10 0x7f7e9c151e50 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:06.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.749+0000 7f7ea2bc6640 1 --2- 172.21.15.112:0/3339375509 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7e9c0afff0 0x7f7e9c14acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:06.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.749+0000 7f7ea2bc6640 1 -- 172.21.15.112:0/3339375509 >> 172.21.15.112:0/3339375509 conn(0x7f7e9c0731e0 msgr2=0x7f7e9c15c5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:06.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.749+0000 7f7ea2bc6640 1 -- 172.21.15.112:0/3339375509 shutdown_connections 2024-09-21T00:26:06.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.749+0000 7f7ea2bc6640 1 -- 172.21.15.112:0/3339375509 wait complete. 2024-09-21T00:26:06.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.751+0000 7f11fc8dd640 1 -- 172.21.15.112:0/1055008448 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f11f00dbdd0 con 0x7f11f0056da0 2024-09-21T00:26:06.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.752+0000 7f11df7fe640 1 -- 172.21.15.112:0/1055008448 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f11f80dda70 con 0x7f11f0056da0 2024-09-21T00:26:06.752 INFO:teuthology.orchestra.run.smithi112.stdout:64424509449 2024-09-21T00:26:06.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.754+0000 7f11fc8dd640 1 -- 172.21.15.112:0/1055008448 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f11e4078bc0 msgr2=0x7f11e407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:06.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.754+0000 7f11fc8dd640 1 --2- 172.21.15.112:0/1055008448 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f11e4078bc0 0x7f11e407b0d0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f11e80046b0 tx=0x7f11e8007f20 comp rx=0 tx=0).stop 2024-09-21T00:26:06.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.754+0000 7f11fc8dd640 1 -- 172.21.15.112:0/1055008448 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f11f0056da0 msgr2=0x7f11f00db750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:06.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.754+0000 7f11fc8dd640 1 --2- 172.21.15.112:0/1055008448 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f11f0056da0 0x7f11f00db750 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f11f8052910 tx=0x7f11f806a0e0 comp rx=0 tx=0).stop 2024-09-21T00:26:06.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.754+0000 7f11fc8dd640 1 -- 172.21.15.112:0/1055008448 shutdown_connections 2024-09-21T00:26:06.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.754+0000 7f11fc8dd640 1 --2- 172.21.15.112:0/1055008448 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f11e4078bc0 0x7f11e407b0d0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:06.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.754+0000 7f11fc8dd640 1 --2- 172.21.15.112:0/1055008448 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11f0057c80 0x7f11f004e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:06.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.754+0000 7f11fc8dd640 1 --2- 172.21.15.112:0/1055008448 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f11f0056da0 0x7f11f00db750 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:06.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.754+0000 7f11fc8dd640 1 --2- 172.21.15.112:0/1055008448 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f11f0054360 0x7f11f0187e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:06.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.754+0000 7f11fc8dd640 1 -- 172.21.15.112:0/1055008448 >> 172.21.15.112:0/1055008448 conn(0x7f11f003a5f0 msgr2=0x7f11f003c8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:06.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.754+0000 7f11fc8dd640 1 -- 172.21.15.112:0/1055008448 shutdown_connections 2024-09-21T00:26:06.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.754+0000 7f11fc8dd640 1 -- 172.21.15.112:0/1055008448 wait complete. 2024-09-21T00:26:06.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.755+0000 7fefd37fe640 1 -- 172.21.15.112:0/17789537 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fefc0005850 con 0x7fefe80cef00 2024-09-21T00:26:06.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.755+0000 7fefddffb640 1 -- 172.21.15.112:0/17789537 <== mon.2 v2:172.21.15.187:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fefe0011e20 con 0x7fefe80cef00 2024-09-21T00:26:06.757 INFO:teuthology.orchestra.run.smithi112.stdout:64424509449 2024-09-21T00:26:06.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.758+0000 7fefd37fe640 1 -- 172.21.15.112:0/17789537 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fefcc078c10 msgr2=0x7fefcc07b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:06.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.758+0000 7fefd37fe640 1 --2- 172.21.15.112:0/17789537 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fefcc078c10 0x7fefcc07b120 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fefe80ceed0 tx=0x7fefd8005f90 comp rx=0 tx=0).stop 2024-09-21T00:26:06.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.758+0000 7fefd37fe640 1 -- 172.21.15.112:0/17789537 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fefe80cef00 msgr2=0x7fefe8152d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:06.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.758+0000 7fefd37fe640 1 --2- 172.21.15.112:0/17789537 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fefe80cef00 0x7fefe8152d80 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fefe000ea50 tx=0x7fefe000ef10 comp rx=0 tx=0).stop 2024-09-21T00:26:06.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.758+0000 7fefd37fe640 1 -- 172.21.15.112:0/17789537 shutdown_connections 2024-09-21T00:26:06.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.758+0000 7fefd37fe640 1 --2- 172.21.15.112:0/17789537 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fefcc078c10 0x7fefcc07b120 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:06.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.758+0000 7fefd37fe640 1 --2- 172.21.15.112:0/17789537 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fefe8162240 0x7fefe80b3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:06.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.758+0000 7fefd37fe640 1 --2- 172.21.15.112:0/17789537 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fefe815f6d0 0x7fefe80b2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:06.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.758+0000 7fefd37fe640 1 --2- 172.21.15.112:0/17789537 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fefe80cef00 0x7fefe8152d80 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:06.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.758+0000 7fefd37fe640 1 -- 172.21.15.112:0/17789537 >> 172.21.15.112:0/17789537 conn(0x7fefe80d41e0 msgr2=0x7fefe815d730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:06.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.758+0000 7fefd37fe640 1 -- 172.21.15.112:0/17789537 shutdown_connections 2024-09-21T00:26:06.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.758+0000 7fefd37fe640 1 -- 172.21.15.112:0/17789537 wait complete. 2024-09-21T00:26:06.763 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509448 for osd.5 2024-09-21T00:26:06.763 DEBUG:teuthology.parallel:result is None 2024-09-21T00:26:06.769 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509449 got 64424509449 for osd.2 2024-09-21T00:26:06.769 DEBUG:teuthology.parallel:result is None 2024-09-21T00:26:06.771 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509449 got 64424509449 for osd.6 2024-09-21T00:26:06.771 DEBUG:teuthology.parallel:result is None 2024-09-21T00:26:06.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.774+0000 7fb1165b3640 1 -- 172.21.15.112:0/1599951844 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fb108035d20 con 0x7fb108009ba0 2024-09-21T00:26:06.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.775+0000 7fb105ffb640 1 -- 172.21.15.112:0/1599951844 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fb100031770 con 0x7fb108009ba0 2024-09-21T00:26:06.775 INFO:teuthology.orchestra.run.smithi112.stdout:55834574857 2024-09-21T00:26:06.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.778+0000 7fb1165b3640 1 -- 172.21.15.112:0/1599951844 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fb0f4078bc0 msgr2=0x7fb0f407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:06.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.778+0000 7fb1165b3640 1 --2- 172.21.15.112:0/1599951844 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fb0f4078bc0 0x7fb0f407b0d0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb1080de580 tx=0x7fb10c005f90 comp rx=0 tx=0).stop 2024-09-21T00:26:06.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.778+0000 7fb1165b3640 1 -- 172.21.15.112:0/1599951844 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb108009ba0 msgr2=0x7fb108020360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:06.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.778+0000 7fb1165b3640 1 --2- 172.21.15.112:0/1599951844 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb108009ba0 0x7fb108020360 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fb10002ab10 tx=0x7fb100005270 comp rx=0 tx=0).stop 2024-09-21T00:26:06.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.778+0000 7fb1165b3640 1 -- 172.21.15.112:0/1599951844 shutdown_connections 2024-09-21T00:26:06.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.778+0000 7fb1165b3640 1 --2- 172.21.15.112:0/1599951844 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb108036400 0x7fb1080eec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:06.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.778+0000 7fb1165b3640 1 --2- 172.21.15.112:0/1599951844 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb10800bc30 0x7fb1080ee710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:06.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.778+0000 7fb1165b3640 1 --2- 172.21.15.112:0/1599951844 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fb0f4078bc0 0x7fb0f407b0d0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:06.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.778+0000 7fb1165b3640 1 --2- 172.21.15.112:0/1599951844 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb108009ba0 0x7fb108020360 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:06.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.778+0000 7fb1165b3640 1 -- 172.21.15.112:0/1599951844 >> 172.21.15.112:0/1599951844 conn(0x7fb108051420 msgr2=0x7fb108054790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:06.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.778+0000 7fb1165b3640 1 -- 172.21.15.112:0/1599951844 shutdown_connections 2024-09-21T00:26:06.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.778+0000 7fb1165b3640 1 -- 172.21.15.112:0/1599951844 wait complete. 2024-09-21T00:26:06.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.782+0000 7f3ed678b640 1 -- 172.21.15.112:0/2460729556 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f3eb0005850 con 0x7f3ed0070070 2024-09-21T00:26:06.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.783+0000 7f3eccff9640 1 -- 172.21.15.112:0/2460729556 <== mon.2 v2:172.21.15.187:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f3ec0062b80 con 0x7f3ed0070070 2024-09-21T00:26:06.783 INFO:teuthology.orchestra.run.smithi112.stdout:64424509447 2024-09-21T00:26:06.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.786+0000 7f3ed678b640 1 -- 172.21.15.112:0/2460729556 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f3ebc078be0 msgr2=0x7f3ebc07b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:06.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.786+0000 7f3ed678b640 1 --2- 172.21.15.112:0/2460729556 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f3ebc078be0 0x7f3ebc07b0f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3ec8035000 tx=0x7f3ec8004ab0 comp rx=0 tx=0).stop 2024-09-21T00:26:06.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.786+0000 7f3ed678b640 1 -- 172.21.15.112:0/2460729556 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3ed0070070 msgr2=0x7f3ed0158bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:06.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.786+0000 7f3ed678b640 1 --2- 172.21.15.112:0/2460729556 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3ed0070070 0x7f3ed0158bc0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3ec000d770 tx=0x7f3ec000dc30 comp rx=0 tx=0).stop 2024-09-21T00:26:06.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.786+0000 7f3ed678b640 1 -- 172.21.15.112:0/2460729556 shutdown_connections 2024-09-21T00:26:06.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.786+0000 7f3ed678b640 1 --2- 172.21.15.112:0/2460729556 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f3ebc078be0 0x7f3ebc07b0f0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:06.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.786+0000 7f3ed678b640 1 --2- 172.21.15.112:0/2460729556 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ed0070e80 0x7f3ed00c9cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:06.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.786+0000 7f3ed678b640 1 --2- 172.21.15.112:0/2460729556 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3ed0070480 0x7f3ed00b3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:06.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.786+0000 7f3ed678b640 1 --2- 172.21.15.112:0/2460729556 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3ed0070070 0x7f3ed0158bc0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:06.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.786+0000 7f3ed678b640 1 -- 172.21.15.112:0/2460729556 >> 172.21.15.112:0/2460729556 conn(0x7f3ed00b1b90 msgr2=0x7f3ed00cdc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:06.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.787+0000 7f3ed678b640 1 -- 172.21.15.112:0/2460729556 shutdown_connections 2024-09-21T00:26:06.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.787+0000 7f3ed678b640 1 -- 172.21.15.112:0/2460729556 wait complete. 2024-09-21T00:26:06.795 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.1 2024-09-21T00:26:06.795 DEBUG:teuthology.parallel:result is None 2024-09-21T00:26:06.799 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509447 for osd.4 2024-09-21T00:26:06.799 DEBUG:teuthology.parallel:result is None 2024-09-21T00:26:06.799 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T00:26:06.800 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T00:26:06.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.952+0000 7ff6db4a9640 1 -- 172.21.15.112:0/861762710 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6c0004d40 msgr2=0x7ff6c0005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:06.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.952+0000 7ff6db4a9640 1 --2- 172.21.15.112:0/861762710 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6c0004d40 0x7ff6c0005210 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc00c610 tx=0x7ff6c0005830 comp rx=0 tx=0).stop 2024-09-21T00:26:06.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.952+0000 7ff6d9a46640 1 -- 172.21.15.112:0/861762710 reap_dead start 2024-09-21T00:26:06.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.952+0000 7ff6db4a9640 1 -- 172.21.15.112:0/861762710 shutdown_connections 2024-09-21T00:26:06.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.952+0000 7ff6db4a9640 1 -- 172.21.15.112:0/861762710 >> 172.21.15.112:0/861762710 conn(0x7ff6d40b2cc0 msgr2=0x7ff6d40b30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:06.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.952+0000 7ff6db4a9640 1 -- 172.21.15.112:0/861762710 shutdown_connections 2024-09-21T00:26:06.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.952+0000 7ff6db4a9640 1 -- 172.21.15.112:0/861762710 wait complete. 2024-09-21T00:26:06.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.954+0000 7ff6db4a9640 1 Processor -- start 2024-09-21T00:26:06.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.954+0000 7ff6db4a9640 1 -- start start 2024-09-21T00:26:06.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.955+0000 7ff6db4a9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6c0004d40 0x7ff6d4154840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:06.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.955+0000 7ff6db4a9640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff6d41f3140 0x7ff6d4151080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:06.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.955+0000 7ff6db4a9640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff6d4159840 0x7ff6d41564e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:06.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.955+0000 7ff6db4a9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff6d40b48a0 con 0x7ff6c0004d40 2024-09-21T00:26:06.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.955+0000 7ff6db4a9640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff6d40b4720 con 0x7ff6d41f3140 2024-09-21T00:26:06.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.955+0000 7ff6db4a9640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff6d415b5d0 con 0x7ff6d4159840 2024-09-21T00:26:06.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.955+0000 7ff6d9245640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6c0004d40 0x7ff6d4154840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:06.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.955+0000 7ff6d9245640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6c0004d40 0x7ff6d4154840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60086/0 (socket says 172.21.15.112:60086) 2024-09-21T00:26:06.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.955+0000 7ff6d9245640 1 -- 172.21.15.112:0/3397649142 learned_addr learned my addr 172.21.15.112:0/3397649142 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:26:06.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.955+0000 7ff6d9a46640 1 --2- 172.21.15.112:0/3397649142 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff6d4159840 0x7ff6d41564e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:06.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.955+0000 7ff6d9245640 1 -- 172.21.15.112:0/3397649142 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff6d4159840 msgr2=0x7ff6d41564e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:06.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.955+0000 7ff6d8a44640 1 --2- 172.21.15.112:0/3397649142 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff6d41f3140 0x7ff6d4151080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:06.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.955+0000 7ff6d9245640 1 --2- 172.21.15.112:0/3397649142 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff6d4159840 0x7ff6d41564e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:06.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.955+0000 7ff6d9245640 1 -- 172.21.15.112:0/3397649142 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff6d41f3140 msgr2=0x7ff6d4151080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:06.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.955+0000 7ff6d9245640 1 --2- 172.21.15.112:0/3397649142 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff6d41f3140 0x7ff6d4151080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:06.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.955+0000 7ff6d9245640 1 -- 172.21.15.112:0/3397649142 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6d41515c0 con 0x7ff6c0004d40 2024-09-21T00:26:06.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.955+0000 7ff6d9a46640 1 --2- 172.21.15.112:0/3397649142 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff6d4159840 0x7ff6d41564e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:26:06.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.955+0000 7ff6d9245640 1 --2- 172.21.15.112:0/3397649142 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6c0004d40 0x7ff6d4154840 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ff6d0003f80 tx=0x7ff6d00016f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:06.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.955+0000 7ff6c9ffb640 1 -- 172.21.15.112:0/3397649142 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6d0016030 con 0x7ff6c0004d40 2024-09-21T00:26:06.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.956+0000 7ff6c9ffb640 1 -- 172.21.15.112:0/3397649142 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff6d0011700 con 0x7ff6c0004d40 2024-09-21T00:26:06.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.956+0000 7ff6db4a9640 1 -- 172.21.15.112:0/3397649142 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6d41f3650 con 0x7ff6c0004d40 2024-09-21T00:26:06.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.956+0000 7ff6c9ffb640 1 -- 172.21.15.112:0/3397649142 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6d0011b30 con 0x7ff6c0004d40 2024-09-21T00:26:06.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.956+0000 7ff6db4a9640 1 -- 172.21.15.112:0/3397649142 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6d41f3c10 con 0x7ff6c0004d40 2024-09-21T00:26:06.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.957+0000 7ff6db4a9640 1 -- 172.21.15.112:0/3397649142 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6d406e6f0 con 0x7ff6c0004d40 2024-09-21T00:26:06.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.957+0000 7ff6c9ffb640 1 -- 172.21.15.112:0/3397649142 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff6d0019070 con 0x7ff6c0004d40 2024-09-21T00:26:06.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.958+0000 7ff6c9ffb640 1 --2- 172.21.15.112:0/3397649142 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff6b8078bc0 0x7ff6b807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:06.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.958+0000 7ff6c9ffb640 1 -- 172.21.15.112:0/3397649142 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7ff6d009b7d0 con 0x7ff6c0004d40 2024-09-21T00:26:06.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.958+0000 7ff6d8a44640 1 --2- 172.21.15.112:0/3397649142 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff6b8078bc0 0x7ff6b807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:06.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.959+0000 7ff6d8a44640 1 --2- 172.21.15.112:0/3397649142 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff6b8078bc0 0x7ff6b807b0d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff6d40cd860 tx=0x7ff6c4005f90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:06.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:06.962+0000 7ff6c9ffb640 1 -- 172.21.15.112:0/3397649142 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ff6d0063850 con 0x7ff6c0004d40 2024-09-21T00:26:07.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:07.136+0000 7ff6db4a9640 1 -- 172.21.15.112:0/3397649142 --> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff6d4154d80 con 0x7ff6b8078bc0 2024-09-21T00:26:07.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:07.142+0000 7ff6c9ffb640 1 -- 172.21.15.112:0/3397649142 <== mgr.4103 v2:172.21.15.187:6800/2921522906 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332523 (secure 0 0 0) 0x7ff6d4154d80 con 0x7ff6b8078bc0 2024-09-21T00:26:07.146 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:26:07.148 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2024-09-21T00:26:07.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:07.146+0000 7ff6db4a9640 1 -- 172.21.15.112:0/3397649142 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff6b8078bc0 msgr2=0x7ff6b807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:07.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:07.146+0000 7ff6db4a9640 1 --2- 172.21.15.112:0/3397649142 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff6b8078bc0 0x7ff6b807b0d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff6d40cd860 tx=0x7ff6c4005f90 comp rx=0 tx=0).stop 2024-09-21T00:26:07.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:07.146+0000 7ff6db4a9640 1 -- 172.21.15.112:0/3397649142 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6c0004d40 msgr2=0x7ff6d4154840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:07.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:07.146+0000 7ff6db4a9640 1 --2- 172.21.15.112:0/3397649142 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6c0004d40 0x7ff6d4154840 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ff6d0003f80 tx=0x7ff6d00016f0 comp rx=0 tx=0).stop 2024-09-21T00:26:07.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:07.146+0000 7ff6db4a9640 1 -- 172.21.15.112:0/3397649142 shutdown_connections 2024-09-21T00:26:07.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:07.146+0000 7ff6db4a9640 1 --2- 172.21.15.112:0/3397649142 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff6d4159840 0x7ff6d41564e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:07.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:07.146+0000 7ff6db4a9640 1 --2- 172.21.15.112:0/3397649142 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff6d41f3140 0x7ff6d4151080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:07.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:07.146+0000 7ff6db4a9640 1 --2- 172.21.15.112:0/3397649142 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff6b8078bc0 0x7ff6b807b0d0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:07.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:07.146+0000 7ff6db4a9640 1 --2- 172.21.15.112:0/3397649142 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6c0004d40 0x7ff6d4154840 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:07.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:07.146+0000 7ff6db4a9640 1 -- 172.21.15.112:0/3397649142 >> 172.21.15.112:0/3397649142 conn(0x7ff6d40b2cc0 msgr2=0x7ff6d4153fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:07.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:07.146+0000 7ff6db4a9640 1 -- 172.21.15.112:0/3397649142 shutdown_connections 2024-09-21T00:26:07.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:07.146+0000 7ff6db4a9640 1 -- 172.21.15.112:0/3397649142 wait complete. 2024-09-21T00:26:07.158 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":36,"stamp":"2024-09-21T00:26:07.093110+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":149,"num_write_kb":1646,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":114,"ondisk_log_size":114,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219572,"kb_used_data":3336,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755148,"statfs":{"total":773094113280,"available":772869271552,"internally_reserved":0,"allocated":3416064,"data_stored":2039456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003931"},"pg_stats":[{"pgid":"2.1f","version":"21'4","reported_seq":27,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.826824+0000","last_change":"2024-09-21T00:25:47.232163+0000","last_active":"2024-09-21T00:25:54.826824+0000","last_peered":"2024-09-21T00:25:54.826824+0000","last_clean":"2024-09-21T00:25:54.826824+0000","last_became_active":"2024-09-21T00:25:47.231843+0000","last_became_peered":"2024-09-21T00:25:47.231843+0000","last_unstale":"2024-09-21T00:25:54.826824+0000","last_undegraded":"2024-09-21T00:25:54.826824+0000","last_fullsized":"2024-09-21T00:25:54.826824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_clean_scrub_stamp":"2024-09-21T00:25:46.193690+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:28:45.453931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.264854+0000","last_change":"2024-09-21T00:25:48.231612+0000","last_active":"2024-09-21T00:25:53.264854+0000","last_peered":"2024-09-21T00:25:53.264854+0000","last_clean":"2024-09-21T00:25:53.264854+0000","last_became_active":"2024-09-21T00:25:48.230971+0000","last_became_peered":"2024-09-21T00:25:48.230971+0000","last_unstale":"2024-09-21T00:25:53.264854+0000","last_undegraded":"2024-09-21T00:25:53.264854+0000","last_fullsized":"2024-09-21T00:25:53.264854+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:07:41.736005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.263784+0000","last_change":"2024-09-21T00:25:52.552378+0000","last_active":"2024-09-21T00:25:53.263784+0000","last_peered":"2024-09-21T00:25:53.263784+0000","last_clean":"2024-09-21T00:25:53.263784+0000","last_became_active":"2024-09-21T00:25:52.551577+0000","last_became_peered":"2024-09-21T00:25:52.551577+0000","last_unstale":"2024-09-21T00:25:53.263784+0000","last_undegraded":"2024-09-21T00:25:53.263784+0000","last_fullsized":"2024-09-21T00:25:53.263784+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_clean_scrub_stamp":"2024-09-21T00:25:51.238933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:54:49.138927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.265044+0000","last_change":"2024-09-21T00:25:47.232065+0000","last_active":"2024-09-21T00:25:53.265044+0000","last_peered":"2024-09-21T00:25:53.265044+0000","last_clean":"2024-09-21T00:25:53.265044+0000","last_became_active":"2024-09-21T00:25:47.231772+0000","last_became_peered":"2024-09-21T00:25:47.231772+0000","last_unstale":"2024-09-21T00:25:53.265044+0000","last_undegraded":"2024-09-21T00:25:53.265044+0000","last_fullsized":"2024-09-21T00:25:53.265044+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_clean_scrub_stamp":"2024-09-21T00:25:46.193690+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:28:45.453931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.826715+0000","last_change":"2024-09-21T00:25:48.233905+0000","last_active":"2024-09-21T00:25:54.826715+0000","last_peered":"2024-09-21T00:25:54.826715+0000","last_clean":"2024-09-21T00:25:54.826715+0000","last_became_active":"2024-09-21T00:25:48.233606+0000","last_became_peered":"2024-09-21T00:25:48.233606+0000","last_unstale":"2024-09-21T00:25:54.826715+0000","last_undegraded":"2024-09-21T00:25:54.826715+0000","last_fullsized":"2024-09-21T00:25:54.826715+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:28:33.742798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.262279+0000","last_change":"2024-09-21T00:25:52.281768+0000","last_active":"2024-09-21T00:25:53.262279+0000","last_peered":"2024-09-21T00:25:53.262279+0000","last_clean":"2024-09-21T00:25:53.262279+0000","last_became_active":"2024-09-21T00:25:52.281352+0000","last_became_peered":"2024-09-21T00:25:52.281352+0000","last_unstale":"2024-09-21T00:25:53.262279+0000","last_undegraded":"2024-09-21T00:25:53.262279+0000","last_fullsized":"2024-09-21T00:25:53.262279+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_clean_scrub_stamp":"2024-09-21T00:25:51.238933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:51:39.910640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.827174+0000","last_change":"2024-09-21T00:25:47.238564+0000","last_active":"2024-09-21T00:25:54.827174+0000","last_peered":"2024-09-21T00:25:54.827174+0000","last_clean":"2024-09-21T00:25:54.827174+0000","last_became_active":"2024-09-21T00:25:47.238114+0000","last_became_peered":"2024-09-21T00:25:47.238114+0000","last_unstale":"2024-09-21T00:25:54.827174+0000","last_undegraded":"2024-09-21T00:25:54.827174+0000","last_fullsized":"2024-09-21T00:25:54.827174+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_clean_scrub_stamp":"2024-09-21T00:25:46.193690+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:37:44.588178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.263783+0000","last_change":"2024-09-21T00:25:48.223307+0000","last_active":"2024-09-21T00:25:53.263783+0000","last_peered":"2024-09-21T00:25:53.263783+0000","last_clean":"2024-09-21T00:25:53.263783+0000","last_became_active":"2024-09-21T00:25:48.222984+0000","last_became_peered":"2024-09-21T00:25:48.222984+0000","last_unstale":"2024-09-21T00:25:53.263783+0000","last_undegraded":"2024-09-21T00:25:53.263783+0000","last_fullsized":"2024-09-21T00:25:53.263783+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:47:38.377031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.263168+0000","last_change":"2024-09-21T00:25:52.282049+0000","last_active":"2024-09-21T00:25:53.263168+0000","last_peered":"2024-09-21T00:25:53.263168+0000","last_clean":"2024-09-21T00:25:53.263168+0000","last_became_active":"2024-09-21T00:25:52.281616+0000","last_became_peered":"2024-09-21T00:25:52.281616+0000","last_unstale":"2024-09-21T00:25:53.263168+0000","last_undegraded":"2024-09-21T00:25:53.263168+0000","last_fullsized":"2024-09-21T00:25:53.263168+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_clean_scrub_stamp":"2024-09-21T00:25:51.238933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:32:36.647007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.826681+0000","last_change":"2024-09-21T00:25:47.238389+0000","last_active":"2024-09-21T00:25:54.826681+0000","last_peered":"2024-09-21T00:25:54.826681+0000","last_clean":"2024-09-21T00:25:54.826681+0000","last_became_active":"2024-09-21T00:25:47.238069+0000","last_became_peered":"2024-09-21T00:25:47.238069+0000","last_unstale":"2024-09-21T00:25:54.826681+0000","last_undegraded":"2024-09-21T00:25:54.826681+0000","last_fullsized":"2024-09-21T00:25:54.826681+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_clean_scrub_stamp":"2024-09-21T00:25:46.193690+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T00:47:54.121411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.264628+0000","last_change":"2024-09-21T00:25:48.220825+0000","last_active":"2024-09-21T00:25:53.264628+0000","last_peered":"2024-09-21T00:25:53.264628+0000","last_clean":"2024-09-21T00:25:53.264628+0000","last_became_active":"2024-09-21T00:25:48.220517+0000","last_became_peered":"2024-09-21T00:25:48.220517+0000","last_unstale":"2024-09-21T00:25:53.264628+0000","last_undegraded":"2024-09-21T00:25:53.264628+0000","last_fullsized":"2024-09-21T00:25:53.264628+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:42:57.617812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.262582+0000","last_change":"2024-09-21T00:25:52.266314+0000","last_active":"2024-09-21T00:25:53.262582+0000","last_peered":"2024-09-21T00:25:53.262582+0000","last_clean":"2024-09-21T00:25:53.262582+0000","last_became_active":"2024-09-21T00:25:52.265684+0000","last_became_peered":"2024-09-21T00:25:52.265684+0000","last_unstale":"2024-09-21T00:25:53.262582+0000","last_undegraded":"2024-09-21T00:25:53.262582+0000","last_fullsized":"2024-09-21T00:25:53.262582+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_clean_scrub_stamp":"2024-09-21T00:25:51.238933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:56:45.075934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.826737+0000","last_change":"2024-09-21T00:25:48.242274+0000","last_active":"2024-09-21T00:25:54.826737+0000","last_peered":"2024-09-21T00:25:54.826737+0000","last_clean":"2024-09-21T00:25:54.826737+0000","last_became_active":"2024-09-21T00:25:48.241953+0000","last_became_peered":"2024-09-21T00:25:48.241953+0000","last_unstale":"2024-09-21T00:25:54.826737+0000","last_undegraded":"2024-09-21T00:25:54.826737+0000","last_fullsized":"2024-09-21T00:25:54.826737+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:23:42.611364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.826740+0000","last_change":"2024-09-21T00:25:47.233509+0000","last_active":"2024-09-21T00:25:54.826740+0000","last_peered":"2024-09-21T00:25:54.826740+0000","last_clean":"2024-09-21T00:25:54.826740+0000","last_became_active":"2024-09-21T00:25:47.233118+0000","last_became_peered":"2024-09-21T00:25:47.233118+0000","last_unstale":"2024-09-21T00:25:54.826740+0000","last_undegraded":"2024-09-21T00:25:54.826740+0000","last_fullsized":"2024-09-21T00:25:54.826740+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_clean_scrub_stamp":"2024-09-21T00:25:46.193690+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:24:04.907044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.264178+0000","last_change":"2024-09-21T00:25:52.276976+0000","last_active":"2024-09-21T00:25:53.264178+0000","last_peered":"2024-09-21T00:25:53.264178+0000","last_clean":"2024-09-21T00:25:53.264178+0000","last_became_active":"2024-09-21T00:25:52.276454+0000","last_became_peered":"2024-09-21T00:25:52.276454+0000","last_unstale":"2024-09-21T00:25:53.264178+0000","last_undegraded":"2024-09-21T00:25:53.264178+0000","last_fullsized":"2024-09-21T00:25:53.264178+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_clean_scrub_stamp":"2024-09-21T00:25:51.238933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T00:45:52.375548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.264258+0000","last_change":"2024-09-21T00:25:48.237366+0000","last_active":"2024-09-21T00:25:53.264258+0000","last_peered":"2024-09-21T00:25:53.264258+0000","last_clean":"2024-09-21T00:25:53.264258+0000","last_became_active":"2024-09-21T00:25:48.237067+0000","last_became_peered":"2024-09-21T00:25:48.237067+0000","last_unstale":"2024-09-21T00:25:53.264258+0000","last_undegraded":"2024-09-21T00:25:53.264258+0000","last_fullsized":"2024-09-21T00:25:53.264258+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:48:25.219726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.324479+0000","last_change":"2024-09-21T00:25:47.227138+0000","last_active":"2024-09-21T00:25:54.324479+0000","last_peered":"2024-09-21T00:25:54.324479+0000","last_clean":"2024-09-21T00:25:54.324479+0000","last_became_active":"2024-09-21T00:25:47.226840+0000","last_became_peered":"2024-09-21T00:25:47.226840+0000","last_unstale":"2024-09-21T00:25:54.324479+0000","last_undegraded":"2024-09-21T00:25:54.324479+0000","last_fullsized":"2024-09-21T00:25:54.324479+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_clean_scrub_stamp":"2024-09-21T00:25:46.193690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:35:33.351025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.262544+0000","last_change":"2024-09-21T00:25:52.281424+0000","last_active":"2024-09-21T00:25:53.262544+0000","last_peered":"2024-09-21T00:25:53.262544+0000","last_clean":"2024-09-21T00:25:53.262544+0000","last_became_active":"2024-09-21T00:25:52.280787+0000","last_became_peered":"2024-09-21T00:25:52.280787+0000","last_unstale":"2024-09-21T00:25:53.262544+0000","last_undegraded":"2024-09-21T00:25:53.262544+0000","last_fullsized":"2024-09-21T00:25:53.262544+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_clean_scrub_stamp":"2024-09-21T00:25:51.238933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:12:43.948562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.264302+0000","last_change":"2024-09-21T00:25:48.239140+0000","last_active":"2024-09-21T00:25:53.264302+0000","last_peered":"2024-09-21T00:25:53.264302+0000","last_clean":"2024-09-21T00:25:53.264302+0000","last_became_active":"2024-09-21T00:25:48.238778+0000","last_became_peered":"2024-09-21T00:25:48.238778+0000","last_unstale":"2024-09-21T00:25:53.264302+0000","last_undegraded":"2024-09-21T00:25:53.264302+0000","last_fullsized":"2024-09-21T00:25:53.264302+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:10:22.176582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.324123+0000","last_change":"2024-09-21T00:25:47.228464+0000","last_active":"2024-09-21T00:25:54.324123+0000","last_peered":"2024-09-21T00:25:54.324123+0000","last_clean":"2024-09-21T00:25:54.324123+0000","last_became_active":"2024-09-21T00:25:47.228152+0000","last_became_peered":"2024-09-21T00:25:47.228152+0000","last_unstale":"2024-09-21T00:25:54.324123+0000","last_undegraded":"2024-09-21T00:25:54.324123+0000","last_fullsized":"2024-09-21T00:25:54.324123+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_clean_scrub_stamp":"2024-09-21T00:25:46.193690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:06:39.901371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.324932+0000","last_change":"2024-09-21T00:25:52.550315+0000","last_active":"2024-09-21T00:25:54.324932+0000","last_peered":"2024-09-21T00:25:54.324932+0000","last_clean":"2024-09-21T00:25:54.324932+0000","last_became_active":"2024-09-21T00:25:52.549843+0000","last_became_peered":"2024-09-21T00:25:52.549843+0000","last_unstale":"2024-09-21T00:25:54.324932+0000","last_undegraded":"2024-09-21T00:25:54.324932+0000","last_fullsized":"2024-09-21T00:25:54.324932+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_clean_scrub_stamp":"2024-09-21T00:25:51.238933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:20:29.694351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.263895+0000","last_change":"2024-09-21T00:25:48.235177+0000","last_active":"2024-09-21T00:25:53.263895+0000","last_peered":"2024-09-21T00:25:53.263895+0000","last_clean":"2024-09-21T00:25:53.263895+0000","last_became_active":"2024-09-21T00:25:48.234870+0000","last_became_peered":"2024-09-21T00:25:48.234870+0000","last_unstale":"2024-09-21T00:25:53.263895+0000","last_undegraded":"2024-09-21T00:25:53.263895+0000","last_fullsized":"2024-09-21T00:25:53.263895+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:55:29.494036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.826534+0000","last_change":"2024-09-21T00:25:47.237638+0000","last_active":"2024-09-21T00:25:54.826534+0000","last_peered":"2024-09-21T00:25:54.826534+0000","last_clean":"2024-09-21T00:25:54.826534+0000","last_became_active":"2024-09-21T00:25:47.237293+0000","last_became_peered":"2024-09-21T00:25:47.237293+0000","last_unstale":"2024-09-21T00:25:54.826534+0000","last_undegraded":"2024-09-21T00:25:54.826534+0000","last_fullsized":"2024-09-21T00:25:54.826534+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_clean_scrub_stamp":"2024-09-21T00:25:46.193690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:57:58.912899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.264463+0000","last_change":"2024-09-21T00:25:52.277124+0000","last_active":"2024-09-21T00:25:53.264463+0000","last_peered":"2024-09-21T00:25:53.264463+0000","last_clean":"2024-09-21T00:25:53.264463+0000","last_became_active":"2024-09-21T00:25:52.276353+0000","last_became_peered":"2024-09-21T00:25:52.276353+0000","last_unstale":"2024-09-21T00:25:53.264463+0000","last_undegraded":"2024-09-21T00:25:53.264463+0000","last_fullsized":"2024-09-21T00:25:53.264463+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_clean_scrub_stamp":"2024-09-21T00:25:51.238933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:13:43.796680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.826871+0000","last_change":"2024-09-21T00:25:48.229327+0000","last_active":"2024-09-21T00:25:54.826871+0000","last_peered":"2024-09-21T00:25:54.826871+0000","last_clean":"2024-09-21T00:25:54.826871+0000","last_became_active":"2024-09-21T00:25:48.228524+0000","last_became_peered":"2024-09-21T00:25:48.228524+0000","last_unstale":"2024-09-21T00:25:54.826871+0000","last_undegraded":"2024-09-21T00:25:54.826871+0000","last_fullsized":"2024-09-21T00:25:54.826871+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:28:02.233472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.827937+0000","last_change":"2024-09-21T00:25:47.232666+0000","last_active":"2024-09-21T00:25:54.827937+0000","last_peered":"2024-09-21T00:25:54.827937+0000","last_clean":"2024-09-21T00:25:54.827937+0000","last_became_active":"2024-09-21T00:25:47.232345+0000","last_became_peered":"2024-09-21T00:25:47.232345+0000","last_unstale":"2024-09-21T00:25:54.827937+0000","last_undegraded":"2024-09-21T00:25:54.827937+0000","last_fullsized":"2024-09-21T00:25:54.827937+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_clean_scrub_stamp":"2024-09-21T00:25:46.193690+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:43:08.714258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.263529+0000","last_change":"2024-09-21T00:25:52.550535+0000","last_active":"2024-09-21T00:25:53.263529+0000","last_peered":"2024-09-21T00:25:53.263529+0000","last_clean":"2024-09-21T00:25:53.263529+0000","last_became_active":"2024-09-21T00:25:52.550055+0000","last_became_peered":"2024-09-21T00:25:52.550055+0000","last_unstale":"2024-09-21T00:25:53.263529+0000","last_undegraded":"2024-09-21T00:25:53.263529+0000","last_fullsized":"2024-09-21T00:25:53.263529+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_clean_scrub_stamp":"2024-09-21T00:25:51.238933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:02:53.617106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.265087+0000","last_change":"2024-09-21T00:25:48.238219+0000","last_active":"2024-09-21T00:25:53.265087+0000","last_peered":"2024-09-21T00:25:53.265087+0000","last_clean":"2024-09-21T00:25:53.265087+0000","last_became_active":"2024-09-21T00:25:48.237770+0000","last_became_peered":"2024-09-21T00:25:48.237770+0000","last_unstale":"2024-09-21T00:25:53.265087+0000","last_undegraded":"2024-09-21T00:25:53.265087+0000","last_fullsized":"2024-09-21T00:25:53.265087+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:34:07.237786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.324720+0000","last_change":"2024-09-21T00:25:47.229361+0000","last_active":"2024-09-21T00:25:54.324720+0000","last_peered":"2024-09-21T00:25:54.324720+0000","last_clean":"2024-09-21T00:25:54.324720+0000","last_became_active":"2024-09-21T00:25:47.228906+0000","last_became_peered":"2024-09-21T00:25:47.228906+0000","last_unstale":"2024-09-21T00:25:54.324720+0000","last_undegraded":"2024-09-21T00:25:54.324720+0000","last_fullsized":"2024-09-21T00:25:54.324720+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_clean_scrub_stamp":"2024-09-21T00:25:46.193690+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:35:43.503231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.263731+0000","last_change":"2024-09-21T00:25:52.550909+0000","last_active":"2024-09-21T00:25:53.263731+0000","last_peered":"2024-09-21T00:25:53.263731+0000","last_clean":"2024-09-21T00:25:53.263731+0000","last_became_active":"2024-09-21T00:25:52.550506+0000","last_became_peered":"2024-09-21T00:25:52.550506+0000","last_unstale":"2024-09-21T00:25:53.263731+0000","last_undegraded":"2024-09-21T00:25:53.263731+0000","last_fullsized":"2024-09-21T00:25:53.263731+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_clean_scrub_stamp":"2024-09-21T00:25:51.238933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:52:16.289687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.324538+0000","last_change":"2024-09-21T00:25:48.232779+0000","last_active":"2024-09-21T00:25:54.324538+0000","last_peered":"2024-09-21T00:25:54.324538+0000","last_clean":"2024-09-21T00:25:54.324538+0000","last_became_active":"2024-09-21T00:25:48.232461+0000","last_became_peered":"2024-09-21T00:25:48.232461+0000","last_unstale":"2024-09-21T00:25:54.324538+0000","last_undegraded":"2024-09-21T00:25:54.324538+0000","last_fullsized":"2024-09-21T00:25:54.324538+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:52:33.265365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.827491+0000","last_change":"2024-09-21T00:25:47.237661+0000","last_active":"2024-09-21T00:25:54.827491+0000","last_peered":"2024-09-21T00:25:54.827491+0000","last_clean":"2024-09-21T00:25:54.827491+0000","last_became_active":"2024-09-21T00:25:47.237273+0000","last_became_peered":"2024-09-21T00:25:47.237273+0000","last_unstale":"2024-09-21T00:25:54.827491+0000","last_undegraded":"2024-09-21T00:25:54.827491+0000","last_fullsized":"2024-09-21T00:25:54.827491+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_clean_scrub_stamp":"2024-09-21T00:25:46.193690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:03:52.795421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.826240+0000","last_change":"2024-09-21T00:25:52.549849+0000","last_active":"2024-09-21T00:25:54.826240+0000","last_peered":"2024-09-21T00:25:54.826240+0000","last_clean":"2024-09-21T00:25:54.826240+0000","last_became_active":"2024-09-21T00:25:52.549433+0000","last_became_peered":"2024-09-21T00:25:52.549433+0000","last_unstale":"2024-09-21T00:25:54.826240+0000","last_undegraded":"2024-09-21T00:25:54.826240+0000","last_fullsized":"2024-09-21T00:25:54.826240+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_clean_scrub_stamp":"2024-09-21T00:25:51.238933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:51:49.334552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.264813+0000","last_change":"2024-09-21T00:25:48.232652+0000","last_active":"2024-09-21T00:25:53.264813+0000","last_peered":"2024-09-21T00:25:53.264813+0000","last_clean":"2024-09-21T00:25:53.264813+0000","last_became_active":"2024-09-21T00:25:48.232099+0000","last_became_peered":"2024-09-21T00:25:48.232099+0000","last_unstale":"2024-09-21T00:25:53.264813+0000","last_undegraded":"2024-09-21T00:25:53.264813+0000","last_fullsized":"2024-09-21T00:25:53.264813+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:48:25.219726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.324657+0000","last_change":"2024-09-21T00:25:47.223473+0000","last_active":"2024-09-21T00:25:54.324657+0000","last_peered":"2024-09-21T00:25:54.324657+0000","last_clean":"2024-09-21T00:25:54.324657+0000","last_became_active":"2024-09-21T00:25:47.223142+0000","last_became_peered":"2024-09-21T00:25:47.223142+0000","last_unstale":"2024-09-21T00:25:54.324657+0000","last_undegraded":"2024-09-21T00:25:54.324657+0000","last_fullsized":"2024-09-21T00:25:54.324657+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_clean_scrub_stamp":"2024-09-21T00:25:46.193690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:54:03.173672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.324126+0000","last_change":"2024-09-21T00:25:52.276922+0000","last_active":"2024-09-21T00:25:54.324126+0000","last_peered":"2024-09-21T00:25:54.324126+0000","last_clean":"2024-09-21T00:25:54.324126+0000","last_became_active":"2024-09-21T00:25:52.276329+0000","last_became_peered":"2024-09-21T00:25:52.276329+0000","last_unstale":"2024-09-21T00:25:54.324126+0000","last_undegraded":"2024-09-21T00:25:54.324126+0000","last_fullsized":"2024-09-21T00:25:54.324126+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_clean_scrub_stamp":"2024-09-21T00:25:51.238933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:29:17.920252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.264377+0000","last_change":"2024-09-21T00:25:48.230144+0000","last_active":"2024-09-21T00:25:53.264377+0000","last_peered":"2024-09-21T00:25:53.264377+0000","last_clean":"2024-09-21T00:25:53.264377+0000","last_became_active":"2024-09-21T00:25:48.229848+0000","last_became_peered":"2024-09-21T00:25:48.229848+0000","last_unstale":"2024-09-21T00:25:53.264377+0000","last_undegraded":"2024-09-21T00:25:53.264377+0000","last_fullsized":"2024-09-21T00:25:53.264377+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:57:59.920308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.263649+0000","last_change":"2024-09-21T00:25:47.228030+0000","last_active":"2024-09-21T00:25:53.263649+0000","last_peered":"2024-09-21T00:25:53.263649+0000","last_clean":"2024-09-21T00:25:53.263649+0000","last_became_active":"2024-09-21T00:25:47.227688+0000","last_became_peered":"2024-09-21T00:25:47.227688+0000","last_unstale":"2024-09-21T00:25:53.263649+0000","last_undegraded":"2024-09-21T00:25:53.263649+0000","last_fullsized":"2024-09-21T00:25:53.263649+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_clean_scrub_stamp":"2024-09-21T00:25:46.193690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:34:30.210849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.324624+0000","last_change":"2024-09-21T00:25:52.276547+0000","last_active":"2024-09-21T00:25:54.324624+0000","last_peered":"2024-09-21T00:25:54.324624+0000","last_clean":"2024-09-21T00:25:54.324624+0000","last_became_active":"2024-09-21T00:25:52.276100+0000","last_became_peered":"2024-09-21T00:25:52.276100+0000","last_unstale":"2024-09-21T00:25:54.324624+0000","last_undegraded":"2024-09-21T00:25:54.324624+0000","last_fullsized":"2024-09-21T00:25:54.324624+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_clean_scrub_stamp":"2024-09-21T00:25:51.238933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:08:54.038371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.324189+0000","last_change":"2024-09-21T00:25:48.233554+0000","last_active":"2024-09-21T00:25:54.324189+0000","last_peered":"2024-09-21T00:25:54.324189+0000","last_clean":"2024-09-21T00:25:54.324189+0000","last_became_active":"2024-09-21T00:25:48.233249+0000","last_became_peered":"2024-09-21T00:25:48.233249+0000","last_unstale":"2024-09-21T00:25:54.324189+0000","last_undegraded":"2024-09-21T00:25:54.324189+0000","last_fullsized":"2024-09-21T00:25:54.324189+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:18:14.964429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.242121+0000","last_change":"2024-09-21T00:25:47.228079+0000","last_active":"2024-09-21T00:25:54.242121+0000","last_peered":"2024-09-21T00:25:54.242121+0000","last_clean":"2024-09-21T00:25:54.242121+0000","last_became_active":"2024-09-21T00:25:47.227731+0000","last_became_peered":"2024-09-21T00:25:47.227731+0000","last_unstale":"2024-09-21T00:25:54.242121+0000","last_undegraded":"2024-09-21T00:25:54.242121+0000","last_fullsized":"2024-09-21T00:25:54.242121+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_clean_scrub_stamp":"2024-09-21T00:25:46.193690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:53:37.081118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.324987+0000","last_change":"2024-09-21T00:25:52.278680+0000","last_active":"2024-09-21T00:25:54.324987+0000","last_peered":"2024-09-21T00:25:54.324987+0000","last_clean":"2024-09-21T00:25:54.324987+0000","last_became_active":"2024-09-21T00:25:52.278128+0000","last_became_peered":"2024-09-21T00:25:52.278128+0000","last_unstale":"2024-09-21T00:25:54.324987+0000","last_undegraded":"2024-09-21T00:25:54.324987+0000","last_fullsized":"2024-09-21T00:25:54.324987+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_clean_scrub_stamp":"2024-09-21T00:25:51.238933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:39:02.970919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.826121+0000","last_change":"2024-09-21T00:25:48.245138+0000","last_active":"2024-09-21T00:25:54.826121+0000","last_peered":"2024-09-21T00:25:54.826121+0000","last_clean":"2024-09-21T00:25:54.826121+0000","last_became_active":"2024-09-21T00:25:48.244796+0000","last_became_peered":"2024-09-21T00:25:48.244796+0000","last_unstale":"2024-09-21T00:25:54.826121+0000","last_undegraded":"2024-09-21T00:25:54.826121+0000","last_fullsized":"2024-09-21T00:25:54.826121+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:53:56.396789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.826939+0000","last_change":"2024-09-21T00:25:47.235076+0000","last_active":"2024-09-21T00:25:54.826939+0000","last_peered":"2024-09-21T00:25:54.826939+0000","last_clean":"2024-09-21T00:25:54.826939+0000","last_became_active":"2024-09-21T00:25:47.234751+0000","last_became_peered":"2024-09-21T00:25:47.234751+0000","last_unstale":"2024-09-21T00:25:54.826939+0000","last_undegraded":"2024-09-21T00:25:54.826939+0000","last_fullsized":"2024-09-21T00:25:54.826939+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_clean_scrub_stamp":"2024-09-21T00:25:46.193690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:40:10.553216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.826789+0000","last_change":"2024-09-21T00:25:52.546980+0000","last_active":"2024-09-21T00:25:54.826789+0000","last_peered":"2024-09-21T00:25:54.826789+0000","last_clean":"2024-09-21T00:25:54.826789+0000","last_became_active":"2024-09-21T00:25:52.546572+0000","last_became_peered":"2024-09-21T00:25:52.546572+0000","last_unstale":"2024-09-21T00:25:54.826789+0000","last_undegraded":"2024-09-21T00:25:54.826789+0000","last_fullsized":"2024-09-21T00:25:54.826789+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_clean_scrub_stamp":"2024-09-21T00:25:51.238933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:34:14.702564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.263892+0000","last_change":"2024-09-21T00:25:48.227417+0000","last_active":"2024-09-21T00:25:53.263892+0000","last_peered":"2024-09-21T00:25:53.263892+0000","last_clean":"2024-09-21T00:25:53.263892+0000","last_became_active":"2024-09-21T00:25:48.227056+0000","last_became_peered":"2024-09-21T00:25:48.227056+0000","last_unstale":"2024-09-21T00:25:53.263892+0000","last_undegraded":"2024-09-21T00:25:53.263892+0000","last_fullsized":"2024-09-21T00:25:53.263892+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:16:33.163623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.264390+0000","last_change":"2024-09-21T00:25:47.228317+0000","last_active":"2024-09-21T00:25:53.264390+0000","last_peered":"2024-09-21T00:25:53.264390+0000","last_clean":"2024-09-21T00:25:53.264390+0000","last_became_active":"2024-09-21T00:25:47.227983+0000","last_became_peered":"2024-09-21T00:25:47.227983+0000","last_unstale":"2024-09-21T00:25:53.264390+0000","last_undegraded":"2024-09-21T00:25:53.264390+0000","last_fullsized":"2024-09-21T00:25:53.264390+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_clean_scrub_stamp":"2024-09-21T00:25:46.193690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:17:12.345201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":119,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.827077+0000","last_change":"2024-09-21T00:25:38.725792+0000","last_active":"2024-09-21T00:25:54.827077+0000","last_peered":"2024-09-21T00:25:54.827077+0000","last_clean":"2024-09-21T00:25:54.827077+0000","last_became_active":"2024-09-21T00:25:38.724668+0000","last_became_peered":"2024-09-21T00:25:38.724668+0000","last_unstale":"2024-09-21T00:25:54.827077+0000","last_undegraded":"2024-09-21T00:25:54.827077+0000","last_fullsized":"2024-09-21T00:25:54.827077+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:37.163216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:37.163216+0000","last_clean_scrub_stamp":"2024-09-21T00:25:37.163216+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:14:36.588660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.827086+0000","last_change":"2024-09-21T00:25:52.551370+0000","last_active":"2024-09-21T00:25:54.827086+0000","last_peered":"2024-09-21T00:25:54.827086+0000","last_clean":"2024-09-21T00:25:54.827086+0000","last_became_active":"2024-09-21T00:25:52.550883+0000","last_became_peered":"2024-09-21T00:25:52.550883+0000","last_unstale":"2024-09-21T00:25:54.827086+0000","last_undegraded":"2024-09-21T00:25:54.827086+0000","last_fullsized":"2024-09-21T00:25:54.827086+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_clean_scrub_stamp":"2024-09-21T00:25:51.238933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T00:31:31.841290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.826927+0000","last_change":"2024-09-21T00:25:48.247427+0000","last_active":"2024-09-21T00:25:54.826927+0000","last_peered":"2024-09-21T00:25:54.826927+0000","last_clean":"2024-09-21T00:25:54.826927+0000","last_became_active":"2024-09-21T00:25:48.246876+0000","last_became_peered":"2024-09-21T00:25:48.246876+0000","last_unstale":"2024-09-21T00:25:54.826927+0000","last_undegraded":"2024-09-21T00:25:54.826927+0000","last_fullsized":"2024-09-21T00:25:54.826927+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:10:01.543628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.242401+0000","last_change":"2024-09-21T00:25:47.233336+0000","last_active":"2024-09-21T00:25:54.242401+0000","last_peered":"2024-09-21T00:25:54.242401+0000","last_clean":"2024-09-21T00:25:54.242401+0000","last_became_active":"2024-09-21T00:25:47.233083+0000","last_became_peered":"2024-09-21T00:25:47.233083+0000","last_unstale":"2024-09-21T00:25:54.242401+0000","last_undegraded":"2024-09-21T00:25:54.242401+0000","last_fullsized":"2024-09-21T00:25:54.242401+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_clean_scrub_stamp":"2024-09-21T00:25:46.193690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:43:39.508965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.826781+0000","last_change":"2024-09-21T00:25:52.550144+0000","last_active":"2024-09-21T00:25:54.826781+0000","last_peered":"2024-09-21T00:25:54.826781+0000","last_clean":"2024-09-21T00:25:54.826781+0000","last_became_active":"2024-09-21T00:25:52.549636+0000","last_became_peered":"2024-09-21T00:25:52.549636+0000","last_unstale":"2024-09-21T00:25:54.826781+0000","last_undegraded":"2024-09-21T00:25:54.826781+0000","last_fullsized":"2024-09-21T00:25:54.826781+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_clean_scrub_stamp":"2024-09-21T00:25:51.238933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:09:27.474968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.263877+0000","last_change":"2024-09-21T00:25:48.229999+0000","last_active":"2024-09-21T00:25:53.263877+0000","last_peered":"2024-09-21T00:25:53.263877+0000","last_clean":"2024-09-21T00:25:53.263877+0000","last_became_active":"2024-09-21T00:25:48.229678+0000","last_became_peered":"2024-09-21T00:25:48.229678+0000","last_unstale":"2024-09-21T00:25:53.263877+0000","last_undegraded":"2024-09-21T00:25:53.263877+0000","last_fullsized":"2024-09-21T00:25:53.263877+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:15:47.572701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.324827+0000","last_change":"2024-09-21T00:25:47.219135+0000","last_active":"2024-09-21T00:25:54.324827+0000","last_peered":"2024-09-21T00:25:54.324827+0000","last_clean":"2024-09-21T00:25:54.324827+0000","last_became_active":"2024-09-21T00:25:47.218722+0000","last_became_peered":"2024-09-21T00:25:47.218722+0000","last_unstale":"2024-09-21T00:25:54.324827+0000","last_undegraded":"2024-09-21T00:25:54.324827+0000","last_fullsized":"2024-09-21T00:25:54.324827+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_clean_scrub_stamp":"2024-09-21T00:25:46.193690+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:52:45.409090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.826420+0000","last_change":"2024-09-21T00:25:52.279946+0000","last_active":"2024-09-21T00:25:54.826420+0000","last_peered":"2024-09-21T00:25:54.826420+0000","last_clean":"2024-09-21T00:25:54.826420+0000","last_became_active":"2024-09-21T00:25:52.279489+0000","last_became_peered":"2024-09-21T00:25:52.279489+0000","last_unstale":"2024-09-21T00:25:54.826420+0000","last_undegraded":"2024-09-21T00:25:54.826420+0000","last_fullsized":"2024-09-21T00:25:54.826420+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_clean_scrub_stamp":"2024-09-21T00:25:51.238933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:47:42.414865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.827648+0000","last_change":"2024-09-21T00:25:48.244146+0000","last_active":"2024-09-21T00:25:54.827648+0000","last_peered":"2024-09-21T00:25:54.827648+0000","last_clean":"2024-09-21T00:25:54.827648+0000","last_became_active":"2024-09-21T00:25:48.243782+0000","last_became_peered":"2024-09-21T00:25:48.243782+0000","last_unstale":"2024-09-21T00:25:54.827648+0000","last_undegraded":"2024-09-21T00:25:54.827648+0000","last_fullsized":"2024-09-21T00:25:54.827648+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:43:59.908291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.242461+0000","last_change":"2024-09-21T00:25:47.230785+0000","last_active":"2024-09-21T00:25:54.242461+0000","last_peered":"2024-09-21T00:25:54.242461+0000","last_clean":"2024-09-21T00:25:54.242461+0000","last_became_active":"2024-09-21T00:25:47.230492+0000","last_became_peered":"2024-09-21T00:25:47.230492+0000","last_unstale":"2024-09-21T00:25:54.242461+0000","last_undegraded":"2024-09-21T00:25:54.242461+0000","last_fullsized":"2024-09-21T00:25:54.242461+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_clean_scrub_stamp":"2024-09-21T00:25:46.193690+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:33:07.366842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.262402+0000","last_change":"2024-09-21T00:25:52.547298+0000","last_active":"2024-09-21T00:25:53.262402+0000","last_peered":"2024-09-21T00:25:53.262402+0000","last_clean":"2024-09-21T00:25:53.262402+0000","last_became_active":"2024-09-21T00:25:52.546737+0000","last_became_peered":"2024-09-21T00:25:52.546737+0000","last_unstale":"2024-09-21T00:25:53.262402+0000","last_undegraded":"2024-09-21T00:25:53.262402+0000","last_fullsized":"2024-09-21T00:25:53.262402+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_clean_scrub_stamp":"2024-09-21T00:25:51.238933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:54:49.138927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.827233+0000","last_change":"2024-09-21T00:25:48.244124+0000","last_active":"2024-09-21T00:25:54.827233+0000","last_peered":"2024-09-21T00:25:54.827233+0000","last_clean":"2024-09-21T00:25:54.827233+0000","last_became_active":"2024-09-21T00:25:48.243767+0000","last_became_peered":"2024-09-21T00:25:48.243767+0000","last_unstale":"2024-09-21T00:25:54.827233+0000","last_undegraded":"2024-09-21T00:25:54.827233+0000","last_fullsized":"2024-09-21T00:25:54.827233+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:15:47.572701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.264246+0000","last_change":"2024-09-21T00:25:47.220781+0000","last_active":"2024-09-21T00:25:53.264246+0000","last_peered":"2024-09-21T00:25:53.264246+0000","last_clean":"2024-09-21T00:25:53.264246+0000","last_became_active":"2024-09-21T00:25:47.220429+0000","last_became_peered":"2024-09-21T00:25:47.220429+0000","last_unstale":"2024-09-21T00:25:53.264246+0000","last_undegraded":"2024-09-21T00:25:53.264246+0000","last_fullsized":"2024-09-21T00:25:53.264246+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_clean_scrub_stamp":"2024-09-21T00:25:46.193690+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:08:24.394801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.262197+0000","last_change":"2024-09-21T00:25:52.276941+0000","last_active":"2024-09-21T00:25:53.262197+0000","last_peered":"2024-09-21T00:25:53.262197+0000","last_clean":"2024-09-21T00:25:53.262197+0000","last_became_active":"2024-09-21T00:25:52.276374+0000","last_became_peered":"2024-09-21T00:25:52.276374+0000","last_unstale":"2024-09-21T00:25:53.262197+0000","last_undegraded":"2024-09-21T00:25:53.262197+0000","last_fullsized":"2024-09-21T00:25:53.262197+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_clean_scrub_stamp":"2024-09-21T00:25:51.238933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:02:24.501560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.826441+0000","last_change":"2024-09-21T00:25:48.244881+0000","last_active":"2024-09-21T00:25:54.826441+0000","last_peered":"2024-09-21T00:25:54.826441+0000","last_clean":"2024-09-21T00:25:54.826441+0000","last_became_active":"2024-09-21T00:25:48.244530+0000","last_became_peered":"2024-09-21T00:25:48.244530+0000","last_unstale":"2024-09-21T00:25:54.826441+0000","last_undegraded":"2024-09-21T00:25:54.826441+0000","last_fullsized":"2024-09-21T00:25:54.826441+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:51:31.831106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":31,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.242013+0000","last_change":"2024-09-21T00:25:47.230620+0000","last_active":"2024-09-21T00:25:54.242013+0000","last_peered":"2024-09-21T00:25:54.242013+0000","last_clean":"2024-09-21T00:25:54.242013+0000","last_became_active":"2024-09-21T00:25:47.230252+0000","last_became_peered":"2024-09-21T00:25:47.230252+0000","last_unstale":"2024-09-21T00:25:54.242013+0000","last_undegraded":"2024-09-21T00:25:54.242013+0000","last_fullsized":"2024-09-21T00:25:54.242013+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_clean_scrub_stamp":"2024-09-21T00:25:46.193690+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:16:00.090070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.826590+0000","last_change":"2024-09-21T00:25:52.266721+0000","last_active":"2024-09-21T00:25:54.826590+0000","last_peered":"2024-09-21T00:25:54.826590+0000","last_clean":"2024-09-21T00:25:54.826590+0000","last_became_active":"2024-09-21T00:25:52.266271+0000","last_became_peered":"2024-09-21T00:25:52.266271+0000","last_unstale":"2024-09-21T00:25:54.826590+0000","last_undegraded":"2024-09-21T00:25:54.826590+0000","last_fullsized":"2024-09-21T00:25:54.826590+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_clean_scrub_stamp":"2024-09-21T00:25:51.238933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:15:37.112368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.826580+0000","last_change":"2024-09-21T00:25:48.239188+0000","last_active":"2024-09-21T00:25:54.826580+0000","last_peered":"2024-09-21T00:25:54.826580+0000","last_clean":"2024-09-21T00:25:54.826580+0000","last_became_active":"2024-09-21T00:25:48.238774+0000","last_became_peered":"2024-09-21T00:25:48.238774+0000","last_unstale":"2024-09-21T00:25:54.826580+0000","last_undegraded":"2024-09-21T00:25:54.826580+0000","last_fullsized":"2024-09-21T00:25:54.826580+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T00:56:56.658210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.826897+0000","last_change":"2024-09-21T00:25:47.230533+0000","last_active":"2024-09-21T00:25:54.826897+0000","last_peered":"2024-09-21T00:25:54.826897+0000","last_clean":"2024-09-21T00:25:54.826897+0000","last_became_active":"2024-09-21T00:25:47.230211+0000","last_became_peered":"2024-09-21T00:25:47.230211+0000","last_unstale":"2024-09-21T00:25:54.826897+0000","last_undegraded":"2024-09-21T00:25:54.826897+0000","last_fullsized":"2024-09-21T00:25:54.826897+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_clean_scrub_stamp":"2024-09-21T00:25:46.193690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:54:03.173672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.265006+0000","last_change":"2024-09-21T00:25:52.550405+0000","last_active":"2024-09-21T00:25:53.265006+0000","last_peered":"2024-09-21T00:25:53.265006+0000","last_clean":"2024-09-21T00:25:53.265006+0000","last_became_active":"2024-09-21T00:25:52.549743+0000","last_became_peered":"2024-09-21T00:25:52.549743+0000","last_unstale":"2024-09-21T00:25:53.265006+0000","last_undegraded":"2024-09-21T00:25:53.265006+0000","last_fullsized":"2024-09-21T00:25:53.265006+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_clean_scrub_stamp":"2024-09-21T00:25:51.238933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T00:40:38.549590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.826686+0000","last_change":"2024-09-21T00:25:48.244073+0000","last_active":"2024-09-21T00:25:54.826686+0000","last_peered":"2024-09-21T00:25:54.826686+0000","last_clean":"2024-09-21T00:25:54.826686+0000","last_became_active":"2024-09-21T00:25:48.243749+0000","last_became_peered":"2024-09-21T00:25:48.243749+0000","last_unstale":"2024-09-21T00:25:54.826686+0000","last_undegraded":"2024-09-21T00:25:54.826686+0000","last_fullsized":"2024-09-21T00:25:54.826686+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:24:47.112012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.263546+0000","last_change":"2024-09-21T00:25:47.229294+0000","last_active":"2024-09-21T00:25:53.263546+0000","last_peered":"2024-09-21T00:25:53.263546+0000","last_clean":"2024-09-21T00:25:53.263546+0000","last_became_active":"2024-09-21T00:25:47.228901+0000","last_became_peered":"2024-09-21T00:25:47.228901+0000","last_unstale":"2024-09-21T00:25:53.263546+0000","last_undegraded":"2024-09-21T00:25:53.263546+0000","last_fullsized":"2024-09-21T00:25:53.263546+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_clean_scrub_stamp":"2024-09-21T00:25:46.193690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:55:28.486627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.264577+0000","last_change":"2024-09-21T00:25:52.547298+0000","last_active":"2024-09-21T00:25:53.264577+0000","last_peered":"2024-09-21T00:25:53.264577+0000","last_clean":"2024-09-21T00:25:53.264577+0000","last_became_active":"2024-09-21T00:25:52.546647+0000","last_became_peered":"2024-09-21T00:25:52.546647+0000","last_unstale":"2024-09-21T00:25:53.264577+0000","last_undegraded":"2024-09-21T00:25:53.264577+0000","last_fullsized":"2024-09-21T00:25:53.264577+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_clean_scrub_stamp":"2024-09-21T00:25:51.238933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:10:45.091073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.826321+0000","last_change":"2024-09-21T00:25:48.237722+0000","last_active":"2024-09-21T00:25:54.826321+0000","last_peered":"2024-09-21T00:25:54.826321+0000","last_clean":"2024-09-21T00:25:54.826321+0000","last_became_active":"2024-09-21T00:25:48.237400+0000","last_became_peered":"2024-09-21T00:25:48.237400+0000","last_unstale":"2024-09-21T00:25:54.826321+0000","last_undegraded":"2024-09-21T00:25:54.826321+0000","last_fullsized":"2024-09-21T00:25:54.826321+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:00:11.346065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.826654+0000","last_change":"2024-09-21T00:25:47.225035+0000","last_active":"2024-09-21T00:25:54.826654+0000","last_peered":"2024-09-21T00:25:54.826654+0000","last_clean":"2024-09-21T00:25:54.826654+0000","last_became_active":"2024-09-21T00:25:47.224732+0000","last_became_peered":"2024-09-21T00:25:47.224732+0000","last_unstale":"2024-09-21T00:25:54.826654+0000","last_undegraded":"2024-09-21T00:25:54.826654+0000","last_fullsized":"2024-09-21T00:25:54.826654+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_clean_scrub_stamp":"2024-09-21T00:25:46.193690+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:16:00.090070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.264690+0000","last_change":"2024-09-21T00:25:52.550751+0000","last_active":"2024-09-21T00:25:53.264690+0000","last_peered":"2024-09-21T00:25:53.264690+0000","last_clean":"2024-09-21T00:25:53.264690+0000","last_became_active":"2024-09-21T00:25:52.550166+0000","last_became_peered":"2024-09-21T00:25:52.550166+0000","last_unstale":"2024-09-21T00:25:53.264690+0000","last_undegraded":"2024-09-21T00:25:53.264690+0000","last_fullsized":"2024-09-21T00:25:53.264690+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_clean_scrub_stamp":"2024-09-21T00:25:51.238933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:01:14.362816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.826761+0000","last_change":"2024-09-21T00:25:48.240297+0000","last_active":"2024-09-21T00:25:54.826761+0000","last_peered":"2024-09-21T00:25:54.826761+0000","last_clean":"2024-09-21T00:25:54.826761+0000","last_became_active":"2024-09-21T00:25:48.239975+0000","last_became_peered":"2024-09-21T00:25:48.239975+0000","last_unstale":"2024-09-21T00:25:54.826761+0000","last_undegraded":"2024-09-21T00:25:54.826761+0000","last_fullsized":"2024-09-21T00:25:54.826761+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:07:41.736005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"26'9","reported_seq":32,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:57.713994+0000","last_change":"2024-09-21T00:25:47.228804+0000","last_active":"2024-09-21T00:25:57.713994+0000","last_peered":"2024-09-21T00:25:57.713994+0000","last_clean":"2024-09-21T00:25:57.713994+0000","last_became_active":"2024-09-21T00:25:47.228490+0000","last_became_peered":"2024-09-21T00:25:47.228490+0000","last_unstale":"2024-09-21T00:25:57.713994+0000","last_undegraded":"2024-09-21T00:25:57.713994+0000","last_fullsized":"2024-09-21T00:25:57.713994+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_clean_scrub_stamp":"2024-09-21T00:25:46.193690+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:35:43.503231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.324519+0000","last_change":"2024-09-21T00:25:52.273360+0000","last_active":"2024-09-21T00:25:54.324519+0000","last_peered":"2024-09-21T00:25:54.324519+0000","last_clean":"2024-09-21T00:25:54.324519+0000","last_became_active":"2024-09-21T00:25:52.272888+0000","last_became_peered":"2024-09-21T00:25:52.272888+0000","last_unstale":"2024-09-21T00:25:54.324519+0000","last_undegraded":"2024-09-21T00:25:54.324519+0000","last_fullsized":"2024-09-21T00:25:54.324519+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_clean_scrub_stamp":"2024-09-21T00:25:51.238933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:31:19.736643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.264518+0000","last_change":"2024-09-21T00:25:48.238506+0000","last_active":"2024-09-21T00:25:53.264518+0000","last_peered":"2024-09-21T00:25:53.264518+0000","last_clean":"2024-09-21T00:25:53.264518+0000","last_became_active":"2024-09-21T00:25:48.237355+0000","last_became_peered":"2024-09-21T00:25:48.237355+0000","last_unstale":"2024-09-21T00:25:53.264518+0000","last_undegraded":"2024-09-21T00:25:53.264518+0000","last_fullsized":"2024-09-21T00:25:53.264518+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:40:11.560626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.324726+0000","last_change":"2024-09-21T00:25:47.224421+0000","last_active":"2024-09-21T00:25:54.324726+0000","last_peered":"2024-09-21T00:25:54.324726+0000","last_clean":"2024-09-21T00:25:54.324726+0000","last_became_active":"2024-09-21T00:25:47.224096+0000","last_became_peered":"2024-09-21T00:25:47.224096+0000","last_unstale":"2024-09-21T00:25:54.324726+0000","last_undegraded":"2024-09-21T00:25:54.324726+0000","last_fullsized":"2024-09-21T00:25:54.324726+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_clean_scrub_stamp":"2024-09-21T00:25:46.193690+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:43:08.714258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.263548+0000","last_change":"2024-09-21T00:25:52.277635+0000","last_active":"2024-09-21T00:25:53.263548+0000","last_peered":"2024-09-21T00:25:53.263548+0000","last_clean":"2024-09-21T00:25:53.263548+0000","last_became_active":"2024-09-21T00:25:52.276739+0000","last_became_peered":"2024-09-21T00:25:52.276739+0000","last_unstale":"2024-09-21T00:25:53.263548+0000","last_undegraded":"2024-09-21T00:25:53.263548+0000","last_fullsized":"2024-09-21T00:25:53.263548+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_clean_scrub_stamp":"2024-09-21T00:25:51.238933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:46:09.670107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.826816+0000","last_change":"2024-09-21T00:25:48.233598+0000","last_active":"2024-09-21T00:25:54.826816+0000","last_peered":"2024-09-21T00:25:54.826816+0000","last_clean":"2024-09-21T00:25:54.826816+0000","last_became_active":"2024-09-21T00:25:48.233203+0000","last_became_peered":"2024-09-21T00:25:48.233203+0000","last_unstale":"2024-09-21T00:25:54.826816+0000","last_undegraded":"2024-09-21T00:25:54.826816+0000","last_fullsized":"2024-09-21T00:25:54.826816+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:36:54.554284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.263639+0000","last_change":"2024-09-21T00:25:47.225271+0000","last_active":"2024-09-21T00:25:53.263639+0000","last_peered":"2024-09-21T00:25:53.263639+0000","last_clean":"2024-09-21T00:25:53.263639+0000","last_became_active":"2024-09-21T00:25:47.224935+0000","last_became_peered":"2024-09-21T00:25:47.224935+0000","last_unstale":"2024-09-21T00:25:53.263639+0000","last_undegraded":"2024-09-21T00:25:53.263639+0000","last_fullsized":"2024-09-21T00:25:53.263639+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_clean_scrub_stamp":"2024-09-21T00:25:46.193690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:06:59.889895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.264083+0000","last_change":"2024-09-21T00:25:52.278653+0000","last_active":"2024-09-21T00:25:53.264083+0000","last_peered":"2024-09-21T00:25:53.264083+0000","last_clean":"2024-09-21T00:25:53.264083+0000","last_became_active":"2024-09-21T00:25:52.278111+0000","last_became_peered":"2024-09-21T00:25:52.278111+0000","last_unstale":"2024-09-21T00:25:53.264083+0000","last_undegraded":"2024-09-21T00:25:53.264083+0000","last_fullsized":"2024-09-21T00:25:53.264083+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_clean_scrub_stamp":"2024-09-21T00:25:51.238933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:01:24.675942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.263506+0000","last_change":"2024-09-21T00:25:48.239666+0000","last_active":"2024-09-21T00:25:53.263506+0000","last_peered":"2024-09-21T00:25:53.263506+0000","last_clean":"2024-09-21T00:25:53.263506+0000","last_became_active":"2024-09-21T00:25:48.239196+0000","last_became_peered":"2024-09-21T00:25:48.239196+0000","last_unstale":"2024-09-21T00:25:53.263506+0000","last_undegraded":"2024-09-21T00:25:53.263506+0000","last_fullsized":"2024-09-21T00:25:53.263506+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:00:19.889965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.827810+0000","last_change":"2024-09-21T00:25:47.225244+0000","last_active":"2024-09-21T00:25:54.827810+0000","last_peered":"2024-09-21T00:25:54.827810+0000","last_clean":"2024-09-21T00:25:54.827810+0000","last_became_active":"2024-09-21T00:25:47.224812+0000","last_became_peered":"2024-09-21T00:25:47.224812+0000","last_unstale":"2024-09-21T00:25:54.827810+0000","last_undegraded":"2024-09-21T00:25:54.827810+0000","last_fullsized":"2024-09-21T00:25:54.827810+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_clean_scrub_stamp":"2024-09-21T00:25:46.193690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:06:39.901371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.264361+0000","last_change":"2024-09-21T00:25:52.550130+0000","last_active":"2024-09-21T00:25:53.264361+0000","last_peered":"2024-09-21T00:25:53.264361+0000","last_clean":"2024-09-21T00:25:53.264361+0000","last_became_active":"2024-09-21T00:25:52.549675+0000","last_became_peered":"2024-09-21T00:25:52.549675+0000","last_unstale":"2024-09-21T00:25:53.264361+0000","last_undegraded":"2024-09-21T00:25:53.264361+0000","last_fullsized":"2024-09-21T00:25:53.264361+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_clean_scrub_stamp":"2024-09-21T00:25:51.238933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:55:10.499479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.826619+0000","last_change":"2024-09-21T00:25:52.550630+0000","last_active":"2024-09-21T00:25:54.826619+0000","last_peered":"2024-09-21T00:25:54.826619+0000","last_clean":"2024-09-21T00:25:54.826619+0000","last_became_active":"2024-09-21T00:25:52.550098+0000","last_became_peered":"2024-09-21T00:25:52.550098+0000","last_unstale":"2024-09-21T00:25:54.826619+0000","last_undegraded":"2024-09-21T00:25:54.826619+0000","last_fullsized":"2024-09-21T00:25:54.826619+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_clean_scrub_stamp":"2024-09-21T00:25:51.238933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:43:32.503216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.324003+0000","last_change":"2024-09-21T00:25:48.227526+0000","last_active":"2024-09-21T00:25:54.324003+0000","last_peered":"2024-09-21T00:25:54.324003+0000","last_clean":"2024-09-21T00:25:54.324003+0000","last_became_active":"2024-09-21T00:25:48.227194+0000","last_became_peered":"2024-09-21T00:25:48.227194+0000","last_unstale":"2024-09-21T00:25:54.324003+0000","last_undegraded":"2024-09-21T00:25:54.324003+0000","last_fullsized":"2024-09-21T00:25:54.324003+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:54:09.449880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.262751+0000","last_change":"2024-09-21T00:25:47.220721+0000","last_active":"2024-09-21T00:25:53.262751+0000","last_peered":"2024-09-21T00:25:53.262751+0000","last_clean":"2024-09-21T00:25:53.262751+0000","last_became_active":"2024-09-21T00:25:47.220393+0000","last_became_peered":"2024-09-21T00:25:47.220393+0000","last_unstale":"2024-09-21T00:25:53.262751+0000","last_undegraded":"2024-09-21T00:25:53.262751+0000","last_fullsized":"2024-09-21T00:25:53.262751+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_clean_scrub_stamp":"2024-09-21T00:25:46.193690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:00:04.423924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.827438+0000","last_change":"2024-09-21T00:25:52.280179+0000","last_active":"2024-09-21T00:25:54.827438+0000","last_peered":"2024-09-21T00:25:54.827438+0000","last_clean":"2024-09-21T00:25:54.827438+0000","last_became_active":"2024-09-21T00:25:52.279617+0000","last_became_peered":"2024-09-21T00:25:52.279617+0000","last_unstale":"2024-09-21T00:25:54.827438+0000","last_undegraded":"2024-09-21T00:25:54.827438+0000","last_fullsized":"2024-09-21T00:25:54.827438+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_clean_scrub_stamp":"2024-09-21T00:25:51.238933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:30:11.179850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.264234+0000","last_change":"2024-09-21T00:25:48.237852+0000","last_active":"2024-09-21T00:25:53.264234+0000","last_peered":"2024-09-21T00:25:53.264234+0000","last_clean":"2024-09-21T00:25:53.264234+0000","last_became_active":"2024-09-21T00:25:48.237559+0000","last_became_peered":"2024-09-21T00:25:48.237559+0000","last_unstale":"2024-09-21T00:25:53.264234+0000","last_undegraded":"2024-09-21T00:25:53.264234+0000","last_fullsized":"2024-09-21T00:25:53.264234+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:28:02.233472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.264305+0000","last_change":"2024-09-21T00:25:47.227879+0000","last_active":"2024-09-21T00:25:53.264305+0000","last_peered":"2024-09-21T00:25:53.264305+0000","last_clean":"2024-09-21T00:25:53.264305+0000","last_became_active":"2024-09-21T00:25:47.227569+0000","last_became_peered":"2024-09-21T00:25:47.227569+0000","last_unstale":"2024-09-21T00:25:53.264305+0000","last_undegraded":"2024-09-21T00:25:53.264305+0000","last_fullsized":"2024-09-21T00:25:53.264305+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_clean_scrub_stamp":"2024-09-21T00:25:46.193690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:16:00.090070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.827281+0000","last_change":"2024-09-21T00:25:47.231506+0000","last_active":"2024-09-21T00:25:54.827281+0000","last_peered":"2024-09-21T00:25:54.827281+0000","last_clean":"2024-09-21T00:25:54.827281+0000","last_became_active":"2024-09-21T00:25:47.231135+0000","last_became_peered":"2024-09-21T00:25:47.231135+0000","last_unstale":"2024-09-21T00:25:54.827281+0000","last_undegraded":"2024-09-21T00:25:54.827281+0000","last_fullsized":"2024-09-21T00:25:54.827281+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_clean_scrub_stamp":"2024-09-21T00:25:46.193690+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:52:45.409090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.826519+0000","last_change":"2024-09-21T00:25:48.245503+0000","last_active":"2024-09-21T00:25:54.826519+0000","last_peered":"2024-09-21T00:25:54.826519+0000","last_clean":"2024-09-21T00:25:54.826519+0000","last_became_active":"2024-09-21T00:25:48.245168+0000","last_became_peered":"2024-09-21T00:25:48.245168+0000","last_unstale":"2024-09-21T00:25:54.826519+0000","last_undegraded":"2024-09-21T00:25:54.826519+0000","last_fullsized":"2024-09-21T00:25:54.826519+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:51:35.872806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.242222+0000","last_change":"2024-09-21T00:25:52.278289+0000","last_active":"2024-09-21T00:25:54.242222+0000","last_peered":"2024-09-21T00:25:54.242222+0000","last_clean":"2024-09-21T00:25:54.242222+0000","last_became_active":"2024-09-21T00:25:52.277874+0000","last_became_peered":"2024-09-21T00:25:52.277874+0000","last_unstale":"2024-09-21T00:25:54.242222+0000","last_undegraded":"2024-09-21T00:25:54.242222+0000","last_fullsized":"2024-09-21T00:25:54.242222+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_clean_scrub_stamp":"2024-09-21T00:25:51.238933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T00:47:59.166653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.264404+0000","last_change":"2024-09-21T00:25:47.232974+0000","last_active":"2024-09-21T00:25:53.264404+0000","last_peered":"2024-09-21T00:25:53.264404+0000","last_clean":"2024-09-21T00:25:53.264404+0000","last_became_active":"2024-09-21T00:25:47.232605+0000","last_became_peered":"2024-09-21T00:25:47.232605+0000","last_unstale":"2024-09-21T00:25:53.264404+0000","last_undegraded":"2024-09-21T00:25:53.264404+0000","last_fullsized":"2024-09-21T00:25:53.264404+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_clean_scrub_stamp":"2024-09-21T00:25:46.193690+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:53:37.081118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.263168+0000","last_change":"2024-09-21T00:25:48.223460+0000","last_active":"2024-09-21T00:25:53.263168+0000","last_peered":"2024-09-21T00:25:53.263168+0000","last_clean":"2024-09-21T00:25:53.263168+0000","last_became_active":"2024-09-21T00:25:48.222174+0000","last_became_peered":"2024-09-21T00:25:48.222174+0000","last_unstale":"2024-09-21T00:25:53.263168+0000","last_undegraded":"2024-09-21T00:25:53.263168+0000","last_fullsized":"2024-09-21T00:25:53.263168+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:07:00.897304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.264073+0000","last_change":"2024-09-21T00:25:52.277698+0000","last_active":"2024-09-21T00:25:53.264073+0000","last_peered":"2024-09-21T00:25:53.264073+0000","last_clean":"2024-09-21T00:25:53.264073+0000","last_became_active":"2024-09-21T00:25:52.277232+0000","last_became_peered":"2024-09-21T00:25:52.277232+0000","last_unstale":"2024-09-21T00:25:53.264073+0000","last_undegraded":"2024-09-21T00:25:53.264073+0000","last_fullsized":"2024-09-21T00:25:53.264073+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_clean_scrub_stamp":"2024-09-21T00:25:51.238933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:01:14.362816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.826302+0000","last_change":"2024-09-21T00:25:48.240360+0000","last_active":"2024-09-21T00:25:54.826302+0000","last_peered":"2024-09-21T00:25:54.826302+0000","last_clean":"2024-09-21T00:25:54.826302+0000","last_became_active":"2024-09-21T00:25:48.240038+0000","last_became_peered":"2024-09-21T00:25:48.240038+0000","last_unstale":"2024-09-21T00:25:54.826302+0000","last_undegraded":"2024-09-21T00:25:54.826302+0000","last_fullsized":"2024-09-21T00:25:54.826302+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:18:14.964429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.242242+0000","last_change":"2024-09-21T00:25:48.243928+0000","last_active":"2024-09-21T00:25:54.242242+0000","last_peered":"2024-09-21T00:25:54.242242+0000","last_clean":"2024-09-21T00:25:54.242242+0000","last_became_active":"2024-09-21T00:25:48.243641+0000","last_became_peered":"2024-09-21T00:25:48.243641+0000","last_unstale":"2024-09-21T00:25:54.242242+0000","last_undegraded":"2024-09-21T00:25:54.242242+0000","last_fullsized":"2024-09-21T00:25:54.242242+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:01:32.510097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.263436+0000","last_change":"2024-09-21T00:25:48.229253+0000","last_active":"2024-09-21T00:25:53.263436+0000","last_peered":"2024-09-21T00:25:53.263436+0000","last_clean":"2024-09-21T00:25:53.263436+0000","last_became_active":"2024-09-21T00:25:48.228880+0000","last_became_peered":"2024-09-21T00:25:48.228880+0000","last_unstale":"2024-09-21T00:25:53.263436+0000","last_undegraded":"2024-09-21T00:25:53.263436+0000","last_fullsized":"2024-09-21T00:25:53.263436+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:24:47.112012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.826434+0000","last_change":"2024-09-21T00:25:48.232024+0000","last_active":"2024-09-21T00:25:54.826434+0000","last_peered":"2024-09-21T00:25:54.826434+0000","last_clean":"2024-09-21T00:25:54.826434+0000","last_became_active":"2024-09-21T00:25:48.231506+0000","last_became_peered":"2024-09-21T00:25:48.231506+0000","last_unstale":"2024-09-21T00:25:54.826434+0000","last_undegraded":"2024-09-21T00:25:54.826434+0000","last_fullsized":"2024-09-21T00:25:54.826434+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:02:20.463726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.264854+0000","last_change":"2024-09-21T00:25:48.242030+0000","last_active":"2024-09-21T00:25:53.264854+0000","last_peered":"2024-09-21T00:25:53.264854+0000","last_clean":"2024-09-21T00:25:53.264854+0000","last_became_active":"2024-09-21T00:25:48.241710+0000","last_became_peered":"2024-09-21T00:25:48.241710+0000","last_unstale":"2024-09-21T00:25:53.264854+0000","last_undegraded":"2024-09-21T00:25:53.264854+0000","last_fullsized":"2024-09-21T00:25:53.264854+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:00:19.889965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.324542+0000","last_change":"2024-09-21T00:25:48.230301+0000","last_active":"2024-09-21T00:25:54.324542+0000","last_peered":"2024-09-21T00:25:54.324542+0000","last_clean":"2024-09-21T00:25:54.324542+0000","last_became_active":"2024-09-21T00:25:48.229759+0000","last_became_peered":"2024-09-21T00:25:48.229759+0000","last_unstale":"2024-09-21T00:25:54.324542+0000","last_undegraded":"2024-09-21T00:25:54.324542+0000","last_fullsized":"2024-09-21T00:25:54.324542+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:23:28.920719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.827515+0000","last_change":"2024-09-21T00:25:48.236977+0000","last_active":"2024-09-21T00:25:54.827515+0000","last_peered":"2024-09-21T00:25:54.827515+0000","last_clean":"2024-09-21T00:25:54.827515+0000","last_became_active":"2024-09-21T00:25:48.236636+0000","last_became_peered":"2024-09-21T00:25:48.236636+0000","last_unstale":"2024-09-21T00:25:54.827515+0000","last_undegraded":"2024-09-21T00:25:54.827515+0000","last_fullsized":"2024-09-21T00:25:54.827515+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:34:36.110489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.264907+0000","last_change":"2024-09-21T00:25:48.242792+0000","last_active":"2024-09-21T00:25:53.264907+0000","last_peered":"2024-09-21T00:25:53.264907+0000","last_clean":"2024-09-21T00:25:53.264907+0000","last_became_active":"2024-09-21T00:25:48.242512+0000","last_became_peered":"2024-09-21T00:25:48.242512+0000","last_unstale":"2024-09-21T00:25:53.264907+0000","last_undegraded":"2024-09-21T00:25:53.264907+0000","last_fullsized":"2024-09-21T00:25:53.264907+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:58:22.324727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.262616+0000","last_change":"2024-09-21T00:25:48.229627+0000","last_active":"2024-09-21T00:25:53.262616+0000","last_peered":"2024-09-21T00:25:53.262616+0000","last_clean":"2024-09-21T00:25:53.262616+0000","last_became_active":"2024-09-21T00:25:48.229297+0000","last_became_peered":"2024-09-21T00:25:48.229297+0000","last_unstale":"2024-09-21T00:25:53.262616+0000","last_undegraded":"2024-09-21T00:25:53.262616+0000","last_fullsized":"2024-09-21T00:25:53.262616+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:58:22.324727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.264187+0000","last_change":"2024-09-21T00:25:48.243911+0000","last_active":"2024-09-21T00:25:53.264187+0000","last_peered":"2024-09-21T00:25:53.264187+0000","last_clean":"2024-09-21T00:25:53.264187+0000","last_became_active":"2024-09-21T00:25:48.243581+0000","last_became_peered":"2024-09-21T00:25:48.243581+0000","last_unstale":"2024-09-21T00:25:53.264187+0000","last_undegraded":"2024-09-21T00:25:53.264187+0000","last_fullsized":"2024-09-21T00:25:53.264187+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:42:57.617812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.263316+0000","last_change":"2024-09-21T00:25:48.238597+0000","last_active":"2024-09-21T00:25:53.263316+0000","last_peered":"2024-09-21T00:25:53.263316+0000","last_clean":"2024-09-21T00:25:53.263316+0000","last_became_active":"2024-09-21T00:25:48.238265+0000","last_became_peered":"2024-09-21T00:25:48.238265+0000","last_unstale":"2024-09-21T00:25:53.263316+0000","last_undegraded":"2024-09-21T00:25:53.263316+0000","last_fullsized":"2024-09-21T00:25:53.263316+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:10:01.543628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.826223+0000","last_change":"2024-09-21T00:25:48.237837+0000","last_active":"2024-09-21T00:25:54.826223+0000","last_peered":"2024-09-21T00:25:54.826223+0000","last_clean":"2024-09-21T00:25:54.826223+0000","last_became_active":"2024-09-21T00:25:48.237502+0000","last_became_peered":"2024-09-21T00:25:48.237502+0000","last_unstale":"2024-09-21T00:25:54.826223+0000","last_undegraded":"2024-09-21T00:25:54.826223+0000","last_fullsized":"2024-09-21T00:25:54.826223+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:52:28.861327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.827511+0000","last_change":"2024-09-21T00:25:48.241417+0000","last_active":"2024-09-21T00:25:54.827511+0000","last_peered":"2024-09-21T00:25:54.827511+0000","last_clean":"2024-09-21T00:25:54.827511+0000","last_became_active":"2024-09-21T00:25:48.241107+0000","last_became_peered":"2024-09-21T00:25:48.241107+0000","last_unstale":"2024-09-21T00:25:54.827511+0000","last_undegraded":"2024-09-21T00:25:54.827511+0000","last_fullsized":"2024-09-21T00:25:54.827511+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:52:33.265365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.827391+0000","last_change":"2024-09-21T00:25:48.246529+0000","last_active":"2024-09-21T00:25:54.827391+0000","last_peered":"2024-09-21T00:25:54.827391+0000","last_clean":"2024-09-21T00:25:54.827391+0000","last_became_active":"2024-09-21T00:25:48.245952+0000","last_became_peered":"2024-09-21T00:25:48.245952+0000","last_unstale":"2024-09-21T00:25:54.827391+0000","last_undegraded":"2024-09-21T00:25:54.827391+0000","last_fullsized":"2024-09-21T00:25:54.827391+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:16:07.250921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.242392+0000","last_change":"2024-09-21T00:25:48.235870+0000","last_active":"2024-09-21T00:25:54.242392+0000","last_peered":"2024-09-21T00:25:54.242392+0000","last_clean":"2024-09-21T00:25:54.242392+0000","last_became_active":"2024-09-21T00:25:48.235552+0000","last_became_peered":"2024-09-21T00:25:48.235552+0000","last_unstale":"2024-09-21T00:25:54.242392+0000","last_undegraded":"2024-09-21T00:25:54.242392+0000","last_fullsized":"2024-09-21T00:25:54.242392+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:03:13.400804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.264193+0000","last_change":"2024-09-21T00:25:48.238458+0000","last_active":"2024-09-21T00:25:53.264193+0000","last_peered":"2024-09-21T00:25:53.264193+0000","last_clean":"2024-09-21T00:25:53.264193+0000","last_became_active":"2024-09-21T00:25:48.238154+0000","last_became_peered":"2024-09-21T00:25:48.238154+0000","last_unstale":"2024-09-21T00:25:53.264193+0000","last_undegraded":"2024-09-21T00:25:53.264193+0000","last_fullsized":"2024-09-21T00:25:53.264193+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:51:31.831106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.262441+0000","last_change":"2024-09-21T00:25:48.232187+0000","last_active":"2024-09-21T00:25:53.262441+0000","last_peered":"2024-09-21T00:25:53.262441+0000","last_clean":"2024-09-21T00:25:53.262441+0000","last_became_active":"2024-09-21T00:25:48.230817+0000","last_became_peered":"2024-09-21T00:25:48.230817+0000","last_unstale":"2024-09-21T00:25:53.262441+0000","last_undegraded":"2024-09-21T00:25:53.262441+0000","last_fullsized":"2024-09-21T00:25:53.262441+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:32:32.609174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.324087+0000","last_change":"2024-09-21T00:25:48.220646+0000","last_active":"2024-09-21T00:25:54.324087+0000","last_peered":"2024-09-21T00:25:54.324087+0000","last_clean":"2024-09-21T00:25:54.324087+0000","last_became_active":"2024-09-21T00:25:48.220325+0000","last_became_peered":"2024-09-21T00:25:48.220325+0000","last_unstale":"2024-09-21T00:25:54.324087+0000","last_undegraded":"2024-09-21T00:25:54.324087+0000","last_fullsized":"2024-09-21T00:25:54.324087+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:52:34.711918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.826294+0000","last_change":"2024-09-21T00:25:48.228802+0000","last_active":"2024-09-21T00:25:54.826294+0000","last_peered":"2024-09-21T00:25:54.826294+0000","last_clean":"2024-09-21T00:25:54.826294+0000","last_became_active":"2024-09-21T00:25:48.228464+0000","last_became_peered":"2024-09-21T00:25:48.228464+0000","last_unstale":"2024-09-21T00:25:54.826294+0000","last_undegraded":"2024-09-21T00:25:54.826294+0000","last_fullsized":"2024-09-21T00:25:54.826294+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:50:09.155321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.264138+0000","last_change":"2024-09-21T00:25:48.243998+0000","last_active":"2024-09-21T00:25:53.264138+0000","last_peered":"2024-09-21T00:25:53.264138+0000","last_clean":"2024-09-21T00:25:53.264138+0000","last_became_active":"2024-09-21T00:25:48.242668+0000","last_became_peered":"2024-09-21T00:25:48.242668+0000","last_unstale":"2024-09-21T00:25:53.264138+0000","last_undegraded":"2024-09-21T00:25:53.264138+0000","last_fullsized":"2024-09-21T00:25:53.264138+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:44:10.293149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.264479+0000","last_change":"2024-09-21T00:25:48.230464+0000","last_active":"2024-09-21T00:25:53.264479+0000","last_peered":"2024-09-21T00:25:53.264479+0000","last_clean":"2024-09-21T00:25:53.264479+0000","last_became_active":"2024-09-21T00:25:48.229463+0000","last_became_peered":"2024-09-21T00:25:48.229463+0000","last_unstale":"2024-09-21T00:25:53.264479+0000","last_undegraded":"2024-09-21T00:25:53.264479+0000","last_fullsized":"2024-09-21T00:25:53.264479+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:24:05.914453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.827241+0000","last_change":"2024-09-21T00:25:48.233368+0000","last_active":"2024-09-21T00:25:54.827241+0000","last_peered":"2024-09-21T00:25:54.827241+0000","last_clean":"2024-09-21T00:25:54.827241+0000","last_became_active":"2024-09-21T00:25:48.233049+0000","last_became_peered":"2024-09-21T00:25:48.233049+0000","last_unstale":"2024-09-21T00:25:54.827241+0000","last_undegraded":"2024-09-21T00:25:54.827241+0000","last_fullsized":"2024-09-21T00:25:54.827241+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:52:34.711918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.827398+0000","last_change":"2024-09-21T00:25:48.240634+0000","last_active":"2024-09-21T00:25:54.827398+0000","last_peered":"2024-09-21T00:25:54.827398+0000","last_clean":"2024-09-21T00:25:54.827398+0000","last_became_active":"2024-09-21T00:25:48.240302+0000","last_became_peered":"2024-09-21T00:25:48.240302+0000","last_unstale":"2024-09-21T00:25:54.827398+0000","last_undegraded":"2024-09-21T00:25:54.827398+0000","last_fullsized":"2024-09-21T00:25:54.827398+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:30:32.308490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.827582+0000","last_change":"2024-09-21T00:25:48.243347+0000","last_active":"2024-09-21T00:25:54.827582+0000","last_peered":"2024-09-21T00:25:54.827582+0000","last_clean":"2024-09-21T00:25:54.827582+0000","last_became_active":"2024-09-21T00:25:48.243035+0000","last_became_peered":"2024-09-21T00:25:48.243035+0000","last_unstale":"2024-09-21T00:25:54.827582+0000","last_undegraded":"2024-09-21T00:25:54.827582+0000","last_fullsized":"2024-09-21T00:25:54.827582+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:10:10.125591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.263815+0000","last_change":"2024-09-21T00:25:48.235929+0000","last_active":"2024-09-21T00:25:53.263815+0000","last_peered":"2024-09-21T00:25:53.263815+0000","last_clean":"2024-09-21T00:25:53.263815+0000","last_became_active":"2024-09-21T00:25:48.235620+0000","last_became_peered":"2024-09-21T00:25:48.235620+0000","last_unstale":"2024-09-21T00:25:53.263815+0000","last_undegraded":"2024-09-21T00:25:53.263815+0000","last_fullsized":"2024-09-21T00:25:53.263815+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:53:08.415542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.826376+0000","last_change":"2024-09-21T00:25:48.247624+0000","last_active":"2024-09-21T00:25:54.826376+0000","last_peered":"2024-09-21T00:25:54.826376+0000","last_clean":"2024-09-21T00:25:54.826376+0000","last_became_active":"2024-09-21T00:25:48.247318+0000","last_became_peered":"2024-09-21T00:25:48.247318+0000","last_unstale":"2024-09-21T00:25:54.826376+0000","last_undegraded":"2024-09-21T00:25:54.826376+0000","last_fullsized":"2024-09-21T00:25:54.826376+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:34:07.237786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.263196+0000","last_change":"2024-09-21T00:25:48.232468+0000","last_active":"2024-09-21T00:25:53.263196+0000","last_peered":"2024-09-21T00:25:53.263196+0000","last_clean":"2024-09-21T00:25:53.263196+0000","last_became_active":"2024-09-21T00:25:48.231259+0000","last_became_peered":"2024-09-21T00:25:48.231259+0000","last_unstale":"2024-09-21T00:25:53.263196+0000","last_undegraded":"2024-09-21T00:25:53.263196+0000","last_fullsized":"2024-09-21T00:25:53.263196+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:07:41.736005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.826608+0000","last_change":"2024-09-21T00:25:48.246495+0000","last_active":"2024-09-21T00:25:54.826608+0000","last_peered":"2024-09-21T00:25:54.826608+0000","last_clean":"2024-09-21T00:25:54.826608+0000","last_became_active":"2024-09-21T00:25:48.245978+0000","last_became_peered":"2024-09-21T00:25:48.245978+0000","last_unstale":"2024-09-21T00:25:54.826608+0000","last_undegraded":"2024-09-21T00:25:54.826608+0000","last_fullsized":"2024-09-21T00:25:54.826608+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:48:34.879434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.263683+0000","last_change":"2024-09-21T00:25:48.237710+0000","last_active":"2024-09-21T00:25:53.263683+0000","last_peered":"2024-09-21T00:25:53.263683+0000","last_clean":"2024-09-21T00:25:53.263683+0000","last_became_active":"2024-09-21T00:25:48.232344+0000","last_became_peered":"2024-09-21T00:25:48.232344+0000","last_unstale":"2024-09-21T00:25:53.263683+0000","last_undegraded":"2024-09-21T00:25:53.263683+0000","last_fullsized":"2024-09-21T00:25:53.263683+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:48:34.879434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.826561+0000","last_change":"2024-09-21T00:25:48.240393+0000","last_active":"2024-09-21T00:25:54.826561+0000","last_peered":"2024-09-21T00:25:54.826561+0000","last_clean":"2024-09-21T00:25:54.826561+0000","last_became_active":"2024-09-21T00:25:48.240086+0000","last_became_peered":"2024-09-21T00:25:48.240086+0000","last_unstale":"2024-09-21T00:25:54.826561+0000","last_undegraded":"2024-09-21T00:25:54.826561+0000","last_fullsized":"2024-09-21T00:25:54.826561+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:12:39.910729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.324667+0000","last_change":"2024-09-21T00:25:48.230210+0000","last_active":"2024-09-21T00:25:54.324667+0000","last_peered":"2024-09-21T00:25:54.324667+0000","last_clean":"2024-09-21T00:25:54.324667+0000","last_became_active":"2024-09-21T00:25:48.229837+0000","last_became_peered":"2024-09-21T00:25:48.229837+0000","last_unstale":"2024-09-21T00:25:54.324667+0000","last_undegraded":"2024-09-21T00:25:54.324667+0000","last_fullsized":"2024-09-21T00:25:54.324667+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:23:42.611364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.827462+0000","last_change":"2024-09-21T00:25:48.242566+0000","last_active":"2024-09-21T00:25:54.827462+0000","last_peered":"2024-09-21T00:25:54.827462+0000","last_clean":"2024-09-21T00:25:54.827462+0000","last_became_active":"2024-09-21T00:25:48.242265+0000","last_became_peered":"2024-09-21T00:25:48.242265+0000","last_unstale":"2024-09-21T00:25:54.827462+0000","last_undegraded":"2024-09-21T00:25:54.827462+0000","last_fullsized":"2024-09-21T00:25:54.827462+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:49:57.219481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509449,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27880,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343960,"statfs":{"total":96636764160,"available":96608215040,"internally_reserved":0,"allocated":868352,"data_stored":697130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509450,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":282624,"data_stored":110022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509449,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":274432,"data_stored":106270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509448,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":282624,"data_stored":106306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509450,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":282624,"data_stored":110022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509450,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":270336,"data_stored":106180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509450,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27888,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343952,"statfs":{"total":96636764160,"available":96608206848,"internally_reserved":0,"allocated":876544,"data_stored":697186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574857,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":278528,"data_stored":106340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T00:26:07.163 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T00:26:07.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:07.310+0000 7fd4153c3640 1 -- 172.21.15.112:0/123669851 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd410162280 msgr2=0x7fd410162690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:07.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:07.310+0000 7fd4153c3640 1 --2- 172.21.15.112:0/123669851 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd410162280 0x7fd410162690 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd3f8009860 tx=0x7fd3f802a530 comp rx=0 tx=0).stop 2024-09-21T00:26:07.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:07.310+0000 7fd4153c3640 1 -- 172.21.15.112:0/123669851 shutdown_connections 2024-09-21T00:26:07.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:07.310+0000 7fd4153c3640 1 --2- 172.21.15.112:0/123669851 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd410162280 0x7fd410162690 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:07.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:07.310+0000 7fd4153c3640 1 -- 172.21.15.112:0/123669851 >> 172.21.15.112:0/123669851 conn(0x7fd4100b6b20 msgr2=0x7fd4100d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:07.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:07.310+0000 7fd4153c3640 1 -- 172.21.15.112:0/123669851 shutdown_connections 2024-09-21T00:26:07.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:07.310+0000 7fd4153c3640 1 -- 172.21.15.112:0/123669851 wait complete. 2024-09-21T00:26:07.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:07.312+0000 7fd4153c3640 1 Processor -- start 2024-09-21T00:26:07.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:07.312+0000 7fd4153c3640 1 -- start start 2024-09-21T00:26:07.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:07.313+0000 7fd4153c3640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd4100b2e40 0x7fd4100cee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:07.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:07.313+0000 7fd4153c3640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd410162280 0x7fd410158260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:07.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:07.313+0000 7fd4153c3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4101f8650 0x7fd4101f8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:07.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:07.313+0000 7fd4153c3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd41015f640 con 0x7fd4101f8650 2024-09-21T00:26:07.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:07.313+0000 7fd4153c3640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd41015f4c0 con 0x7fd410162280 2024-09-21T00:26:07.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:07.313+0000 7fd4153c3640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd41015f7c0 con 0x7fd4100b2e40 2024-09-21T00:26:07.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:07.313+0000 7fd40f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4101f8650 0x7fd4101f8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:07.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:07.313+0000 7fd40f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4101f8650 0x7fd4101f8a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60094/0 (socket says 172.21.15.112:60094) 2024-09-21T00:26:07.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:07.313+0000 7fd40f7fe640 1 -- 172.21.15.112:0/1739931746 learned_addr learned my addr 172.21.15.112:0/1739931746 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:26:07.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:07.313+0000 7fd40effd640 1 --2- 172.21.15.112:0/1739931746 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd4100b2e40 0x7fd4100cee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:07.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:07.314+0000 7fd4067fc640 1 --2- 172.21.15.112:0/1739931746 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd410162280 0x7fd410158260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:07.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:07.314+0000 7fd40f7fe640 1 -- 172.21.15.112:0/1739931746 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd4100b2e40 msgr2=0x7fd4100cee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:07.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:07.314+0000 7fd40f7fe640 1 --2- 172.21.15.112:0/1739931746 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd4100b2e40 0x7fd4100cee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:07.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:07.314+0000 7fd40f7fe640 1 -- 172.21.15.112:0/1739931746 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd410162280 msgr2=0x7fd410158260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:07.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:07.314+0000 7fd40f7fe640 1 --2- 172.21.15.112:0/1739931746 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd410162280 0x7fd410158260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:07.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:07.314+0000 7fd40f7fe640 1 -- 172.21.15.112:0/1739931746 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4100cf390 con 0x7fd4101f8650 2024-09-21T00:26:07.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:07.314+0000 7fd40f7fe640 1 --2- 172.21.15.112:0/1739931746 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4101f8650 0x7fd4101f8a60 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd408001fe0 tx=0x7fd40800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:07.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:07.314+0000 7fd407fff640 1 -- 172.21.15.112:0/1739931746 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd408012420 con 0x7fd4101f8650 2024-09-21T00:26:07.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:07.314+0000 7fd407fff640 1 -- 172.21.15.112:0/1739931746 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd408017070 con 0x7fd4101f8650 2024-09-21T00:26:07.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:07.314+0000 7fd4153c3640 1 -- 172.21.15.112:0/1739931746 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4101f8fa0 con 0x7fd4101f8650 2024-09-21T00:26:07.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:07.314+0000 7fd407fff640 1 -- 172.21.15.112:0/1739931746 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd408012970 con 0x7fd4101f8650 2024-09-21T00:26:07.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:07.314+0000 7fd4153c3640 1 -- 172.21.15.112:0/1739931746 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd410158cb0 con 0x7fd4101f8650 2024-09-21T00:26:07.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:07.315+0000 7fd4153c3640 1 -- 172.21.15.112:0/1739931746 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd410162f00 con 0x7fd4101f8650 2024-09-21T00:26:07.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:07.316+0000 7fd407fff640 1 -- 172.21.15.112:0/1739931746 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd408012b10 con 0x7fd4101f8650 2024-09-21T00:26:07.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:07.316+0000 7fd407fff640 1 --2- 172.21.15.112:0/1739931746 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fd3f0078bc0 0x7fd3f007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:07.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:07.316+0000 7fd407fff640 1 -- 172.21.15.112:0/1739931746 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fd40809b5e0 con 0x7fd4101f8650 2024-09-21T00:26:07.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:07.316+0000 7fd40effd640 1 --2- 172.21.15.112:0/1739931746 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fd3f0078bc0 0x7fd3f007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:07.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:07.317+0000 7fd40effd640 1 --2- 172.21.15.112:0/1739931746 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fd3f0078bc0 0x7fd3f007b0d0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd400001120 tx=0x7fd400009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:07.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:07.321+0000 7fd407fff640 1 -- 172.21.15.112:0/1739931746 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd4101f8fa0 con 0x7fd4101f8650 2024-09-21T00:26:07.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:07.498+0000 7fd4153c3640 1 -- 172.21.15.112:0/1739931746 --> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd41015dcd0 con 0x7fd3f0078bc0 2024-09-21T00:26:07.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:07.504+0000 7fd407fff640 1 -- 172.21.15.112:0/1739931746 <== mgr.4103 v2:172.21.15.187:6800/2921522906 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332523 (secure 0 0 0) 0x7fd41015dcd0 con 0x7fd3f0078bc0 2024-09-21T00:26:07.509 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:26:07.510 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2024-09-21T00:26:07.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:07.508+0000 7fd4153c3640 1 -- 172.21.15.112:0/1739931746 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fd3f0078bc0 msgr2=0x7fd3f007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:07.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:07.508+0000 7fd4153c3640 1 --2- 172.21.15.112:0/1739931746 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fd3f0078bc0 0x7fd3f007b0d0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd400001120 tx=0x7fd400009040 comp rx=0 tx=0).stop 2024-09-21T00:26:07.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:07.508+0000 7fd4153c3640 1 -- 172.21.15.112:0/1739931746 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4101f8650 msgr2=0x7fd4101f8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:07.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:07.508+0000 7fd4153c3640 1 --2- 172.21.15.112:0/1739931746 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4101f8650 0x7fd4101f8a60 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd408001fe0 tx=0x7fd40800a430 comp rx=0 tx=0).stop 2024-09-21T00:26:07.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:07.508+0000 7fd4153c3640 1 -- 172.21.15.112:0/1739931746 shutdown_connections 2024-09-21T00:26:07.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:07.508+0000 7fd4153c3640 1 --2- 172.21.15.112:0/1739931746 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fd3f0078bc0 0x7fd3f007b0d0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:07.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:07.508+0000 7fd4153c3640 1 --2- 172.21.15.112:0/1739931746 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4101f8650 0x7fd4101f8a60 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:07.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:07.508+0000 7fd4153c3640 1 --2- 172.21.15.112:0/1739931746 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd410162280 0x7fd410158260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:07.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:07.508+0000 7fd4153c3640 1 --2- 172.21.15.112:0/1739931746 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd4100b2e40 0x7fd4100cee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:07.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:07.508+0000 7fd4153c3640 1 -- 172.21.15.112:0/1739931746 >> 172.21.15.112:0/1739931746 conn(0x7fd4100b6b20 msgr2=0x7fd41006cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:07.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:07.508+0000 7fd4153c3640 1 -- 172.21.15.112:0/1739931746 shutdown_connections 2024-09-21T00:26:07.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:07.508+0000 7fd4153c3640 1 -- 172.21.15.112:0/1739931746 wait complete. 2024-09-21T00:26:07.519 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":36,"stamp":"2024-09-21T00:26:07.093110+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":149,"num_write_kb":1646,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":114,"ondisk_log_size":114,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219572,"kb_used_data":3336,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755148,"statfs":{"total":773094113280,"available":772869271552,"internally_reserved":0,"allocated":3416064,"data_stored":2039456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003931"},"pg_stats":[{"pgid":"2.1f","version":"21'4","reported_seq":27,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.826824+0000","last_change":"2024-09-21T00:25:47.232163+0000","last_active":"2024-09-21T00:25:54.826824+0000","last_peered":"2024-09-21T00:25:54.826824+0000","last_clean":"2024-09-21T00:25:54.826824+0000","last_became_active":"2024-09-21T00:25:47.231843+0000","last_became_peered":"2024-09-21T00:25:47.231843+0000","last_unstale":"2024-09-21T00:25:54.826824+0000","last_undegraded":"2024-09-21T00:25:54.826824+0000","last_fullsized":"2024-09-21T00:25:54.826824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_clean_scrub_stamp":"2024-09-21T00:25:46.193690+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:28:45.453931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.264854+0000","last_change":"2024-09-21T00:25:48.231612+0000","last_active":"2024-09-21T00:25:53.264854+0000","last_peered":"2024-09-21T00:25:53.264854+0000","last_clean":"2024-09-21T00:25:53.264854+0000","last_became_active":"2024-09-21T00:25:48.230971+0000","last_became_peered":"2024-09-21T00:25:48.230971+0000","last_unstale":"2024-09-21T00:25:53.264854+0000","last_undegraded":"2024-09-21T00:25:53.264854+0000","last_fullsized":"2024-09-21T00:25:53.264854+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:07:41.736005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.263784+0000","last_change":"2024-09-21T00:25:52.552378+0000","last_active":"2024-09-21T00:25:53.263784+0000","last_peered":"2024-09-21T00:25:53.263784+0000","last_clean":"2024-09-21T00:25:53.263784+0000","last_became_active":"2024-09-21T00:25:52.551577+0000","last_became_peered":"2024-09-21T00:25:52.551577+0000","last_unstale":"2024-09-21T00:25:53.263784+0000","last_undegraded":"2024-09-21T00:25:53.263784+0000","last_fullsized":"2024-09-21T00:25:53.263784+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_clean_scrub_stamp":"2024-09-21T00:25:51.238933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:54:49.138927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.265044+0000","last_change":"2024-09-21T00:25:47.232065+0000","last_active":"2024-09-21T00:25:53.265044+0000","last_peered":"2024-09-21T00:25:53.265044+0000","last_clean":"2024-09-21T00:25:53.265044+0000","last_became_active":"2024-09-21T00:25:47.231772+0000","last_became_peered":"2024-09-21T00:25:47.231772+0000","last_unstale":"2024-09-21T00:25:53.265044+0000","last_undegraded":"2024-09-21T00:25:53.265044+0000","last_fullsized":"2024-09-21T00:25:53.265044+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_clean_scrub_stamp":"2024-09-21T00:25:46.193690+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:28:45.453931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.826715+0000","last_change":"2024-09-21T00:25:48.233905+0000","last_active":"2024-09-21T00:25:54.826715+0000","last_peered":"2024-09-21T00:25:54.826715+0000","last_clean":"2024-09-21T00:25:54.826715+0000","last_became_active":"2024-09-21T00:25:48.233606+0000","last_became_peered":"2024-09-21T00:25:48.233606+0000","last_unstale":"2024-09-21T00:25:54.826715+0000","last_undegraded":"2024-09-21T00:25:54.826715+0000","last_fullsized":"2024-09-21T00:25:54.826715+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:28:33.742798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.262279+0000","last_change":"2024-09-21T00:25:52.281768+0000","last_active":"2024-09-21T00:25:53.262279+0000","last_peered":"2024-09-21T00:25:53.262279+0000","last_clean":"2024-09-21T00:25:53.262279+0000","last_became_active":"2024-09-21T00:25:52.281352+0000","last_became_peered":"2024-09-21T00:25:52.281352+0000","last_unstale":"2024-09-21T00:25:53.262279+0000","last_undegraded":"2024-09-21T00:25:53.262279+0000","last_fullsized":"2024-09-21T00:25:53.262279+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_clean_scrub_stamp":"2024-09-21T00:25:51.238933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:51:39.910640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.827174+0000","last_change":"2024-09-21T00:25:47.238564+0000","last_active":"2024-09-21T00:25:54.827174+0000","last_peered":"2024-09-21T00:25:54.827174+0000","last_clean":"2024-09-21T00:25:54.827174+0000","last_became_active":"2024-09-21T00:25:47.238114+0000","last_became_peered":"2024-09-21T00:25:47.238114+0000","last_unstale":"2024-09-21T00:25:54.827174+0000","last_undegraded":"2024-09-21T00:25:54.827174+0000","last_fullsized":"2024-09-21T00:25:54.827174+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_clean_scrub_stamp":"2024-09-21T00:25:46.193690+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:37:44.588178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.263783+0000","last_change":"2024-09-21T00:25:48.223307+0000","last_active":"2024-09-21T00:25:53.263783+0000","last_peered":"2024-09-21T00:25:53.263783+0000","last_clean":"2024-09-21T00:25:53.263783+0000","last_became_active":"2024-09-21T00:25:48.222984+0000","last_became_peered":"2024-09-21T00:25:48.222984+0000","last_unstale":"2024-09-21T00:25:53.263783+0000","last_undegraded":"2024-09-21T00:25:53.263783+0000","last_fullsized":"2024-09-21T00:25:53.263783+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:47:38.377031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.263168+0000","last_change":"2024-09-21T00:25:52.282049+0000","last_active":"2024-09-21T00:25:53.263168+0000","last_peered":"2024-09-21T00:25:53.263168+0000","last_clean":"2024-09-21T00:25:53.263168+0000","last_became_active":"2024-09-21T00:25:52.281616+0000","last_became_peered":"2024-09-21T00:25:52.281616+0000","last_unstale":"2024-09-21T00:25:53.263168+0000","last_undegraded":"2024-09-21T00:25:53.263168+0000","last_fullsized":"2024-09-21T00:25:53.263168+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_clean_scrub_stamp":"2024-09-21T00:25:51.238933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:32:36.647007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.826681+0000","last_change":"2024-09-21T00:25:47.238389+0000","last_active":"2024-09-21T00:25:54.826681+0000","last_peered":"2024-09-21T00:25:54.826681+0000","last_clean":"2024-09-21T00:25:54.826681+0000","last_became_active":"2024-09-21T00:25:47.238069+0000","last_became_peered":"2024-09-21T00:25:47.238069+0000","last_unstale":"2024-09-21T00:25:54.826681+0000","last_undegraded":"2024-09-21T00:25:54.826681+0000","last_fullsized":"2024-09-21T00:25:54.826681+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_clean_scrub_stamp":"2024-09-21T00:25:46.193690+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T00:47:54.121411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.264628+0000","last_change":"2024-09-21T00:25:48.220825+0000","last_active":"2024-09-21T00:25:53.264628+0000","last_peered":"2024-09-21T00:25:53.264628+0000","last_clean":"2024-09-21T00:25:53.264628+0000","last_became_active":"2024-09-21T00:25:48.220517+0000","last_became_peered":"2024-09-21T00:25:48.220517+0000","last_unstale":"2024-09-21T00:25:53.264628+0000","last_undegraded":"2024-09-21T00:25:53.264628+0000","last_fullsized":"2024-09-21T00:25:53.264628+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:42:57.617812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.262582+0000","last_change":"2024-09-21T00:25:52.266314+0000","last_active":"2024-09-21T00:25:53.262582+0000","last_peered":"2024-09-21T00:25:53.262582+0000","last_clean":"2024-09-21T00:25:53.262582+0000","last_became_active":"2024-09-21T00:25:52.265684+0000","last_became_peered":"2024-09-21T00:25:52.265684+0000","last_unstale":"2024-09-21T00:25:53.262582+0000","last_undegraded":"2024-09-21T00:25:53.262582+0000","last_fullsized":"2024-09-21T00:25:53.262582+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_clean_scrub_stamp":"2024-09-21T00:25:51.238933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:56:45.075934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.826737+0000","last_change":"2024-09-21T00:25:48.242274+0000","last_active":"2024-09-21T00:25:54.826737+0000","last_peered":"2024-09-21T00:25:54.826737+0000","last_clean":"2024-09-21T00:25:54.826737+0000","last_became_active":"2024-09-21T00:25:48.241953+0000","last_became_peered":"2024-09-21T00:25:48.241953+0000","last_unstale":"2024-09-21T00:25:54.826737+0000","last_undegraded":"2024-09-21T00:25:54.826737+0000","last_fullsized":"2024-09-21T00:25:54.826737+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:23:42.611364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.826740+0000","last_change":"2024-09-21T00:25:47.233509+0000","last_active":"2024-09-21T00:25:54.826740+0000","last_peered":"2024-09-21T00:25:54.826740+0000","last_clean":"2024-09-21T00:25:54.826740+0000","last_became_active":"2024-09-21T00:25:47.233118+0000","last_became_peered":"2024-09-21T00:25:47.233118+0000","last_unstale":"2024-09-21T00:25:54.826740+0000","last_undegraded":"2024-09-21T00:25:54.826740+0000","last_fullsized":"2024-09-21T00:25:54.826740+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_clean_scrub_stamp":"2024-09-21T00:25:46.193690+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:24:04.907044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.264178+0000","last_change":"2024-09-21T00:25:52.276976+0000","last_active":"2024-09-21T00:25:53.264178+0000","last_peered":"2024-09-21T00:25:53.264178+0000","last_clean":"2024-09-21T00:25:53.264178+0000","last_became_active":"2024-09-21T00:25:52.276454+0000","last_became_peered":"2024-09-21T00:25:52.276454+0000","last_unstale":"2024-09-21T00:25:53.264178+0000","last_undegraded":"2024-09-21T00:25:53.264178+0000","last_fullsized":"2024-09-21T00:25:53.264178+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_clean_scrub_stamp":"2024-09-21T00:25:51.238933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T00:45:52.375548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.264258+0000","last_change":"2024-09-21T00:25:48.237366+0000","last_active":"2024-09-21T00:25:53.264258+0000","last_peered":"2024-09-21T00:25:53.264258+0000","last_clean":"2024-09-21T00:25:53.264258+0000","last_became_active":"2024-09-21T00:25:48.237067+0000","last_became_peered":"2024-09-21T00:25:48.237067+0000","last_unstale":"2024-09-21T00:25:53.264258+0000","last_undegraded":"2024-09-21T00:25:53.264258+0000","last_fullsized":"2024-09-21T00:25:53.264258+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:48:25.219726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.324479+0000","last_change":"2024-09-21T00:25:47.227138+0000","last_active":"2024-09-21T00:25:54.324479+0000","last_peered":"2024-09-21T00:25:54.324479+0000","last_clean":"2024-09-21T00:25:54.324479+0000","last_became_active":"2024-09-21T00:25:47.226840+0000","last_became_peered":"2024-09-21T00:25:47.226840+0000","last_unstale":"2024-09-21T00:25:54.324479+0000","last_undegraded":"2024-09-21T00:25:54.324479+0000","last_fullsized":"2024-09-21T00:25:54.324479+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_clean_scrub_stamp":"2024-09-21T00:25:46.193690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:35:33.351025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.262544+0000","last_change":"2024-09-21T00:25:52.281424+0000","last_active":"2024-09-21T00:25:53.262544+0000","last_peered":"2024-09-21T00:25:53.262544+0000","last_clean":"2024-09-21T00:25:53.262544+0000","last_became_active":"2024-09-21T00:25:52.280787+0000","last_became_peered":"2024-09-21T00:25:52.280787+0000","last_unstale":"2024-09-21T00:25:53.262544+0000","last_undegraded":"2024-09-21T00:25:53.262544+0000","last_fullsized":"2024-09-21T00:25:53.262544+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_clean_scrub_stamp":"2024-09-21T00:25:51.238933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:12:43.948562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.264302+0000","last_change":"2024-09-21T00:25:48.239140+0000","last_active":"2024-09-21T00:25:53.264302+0000","last_peered":"2024-09-21T00:25:53.264302+0000","last_clean":"2024-09-21T00:25:53.264302+0000","last_became_active":"2024-09-21T00:25:48.238778+0000","last_became_peered":"2024-09-21T00:25:48.238778+0000","last_unstale":"2024-09-21T00:25:53.264302+0000","last_undegraded":"2024-09-21T00:25:53.264302+0000","last_fullsized":"2024-09-21T00:25:53.264302+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:10:22.176582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.324123+0000","last_change":"2024-09-21T00:25:47.228464+0000","last_active":"2024-09-21T00:25:54.324123+0000","last_peered":"2024-09-21T00:25:54.324123+0000","last_clean":"2024-09-21T00:25:54.324123+0000","last_became_active":"2024-09-21T00:25:47.228152+0000","last_became_peered":"2024-09-21T00:25:47.228152+0000","last_unstale":"2024-09-21T00:25:54.324123+0000","last_undegraded":"2024-09-21T00:25:54.324123+0000","last_fullsized":"2024-09-21T00:25:54.324123+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_clean_scrub_stamp":"2024-09-21T00:25:46.193690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:06:39.901371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.324932+0000","last_change":"2024-09-21T00:25:52.550315+0000","last_active":"2024-09-21T00:25:54.324932+0000","last_peered":"2024-09-21T00:25:54.324932+0000","last_clean":"2024-09-21T00:25:54.324932+0000","last_became_active":"2024-09-21T00:25:52.549843+0000","last_became_peered":"2024-09-21T00:25:52.549843+0000","last_unstale":"2024-09-21T00:25:54.324932+0000","last_undegraded":"2024-09-21T00:25:54.324932+0000","last_fullsized":"2024-09-21T00:25:54.324932+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_clean_scrub_stamp":"2024-09-21T00:25:51.238933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:20:29.694351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.263895+0000","last_change":"2024-09-21T00:25:48.235177+0000","last_active":"2024-09-21T00:25:53.263895+0000","last_peered":"2024-09-21T00:25:53.263895+0000","last_clean":"2024-09-21T00:25:53.263895+0000","last_became_active":"2024-09-21T00:25:48.234870+0000","last_became_peered":"2024-09-21T00:25:48.234870+0000","last_unstale":"2024-09-21T00:25:53.263895+0000","last_undegraded":"2024-09-21T00:25:53.263895+0000","last_fullsized":"2024-09-21T00:25:53.263895+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:55:29.494036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.826534+0000","last_change":"2024-09-21T00:25:47.237638+0000","last_active":"2024-09-21T00:25:54.826534+0000","last_peered":"2024-09-21T00:25:54.826534+0000","last_clean":"2024-09-21T00:25:54.826534+0000","last_became_active":"2024-09-21T00:25:47.237293+0000","last_became_peered":"2024-09-21T00:25:47.237293+0000","last_unstale":"2024-09-21T00:25:54.826534+0000","last_undegraded":"2024-09-21T00:25:54.826534+0000","last_fullsized":"2024-09-21T00:25:54.826534+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_clean_scrub_stamp":"2024-09-21T00:25:46.193690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:57:58.912899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.264463+0000","last_change":"2024-09-21T00:25:52.277124+0000","last_active":"2024-09-21T00:25:53.264463+0000","last_peered":"2024-09-21T00:25:53.264463+0000","last_clean":"2024-09-21T00:25:53.264463+0000","last_became_active":"2024-09-21T00:25:52.276353+0000","last_became_peered":"2024-09-21T00:25:52.276353+0000","last_unstale":"2024-09-21T00:25:53.264463+0000","last_undegraded":"2024-09-21T00:25:53.264463+0000","last_fullsized":"2024-09-21T00:25:53.264463+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_clean_scrub_stamp":"2024-09-21T00:25:51.238933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:13:43.796680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.826871+0000","last_change":"2024-09-21T00:25:48.229327+0000","last_active":"2024-09-21T00:25:54.826871+0000","last_peered":"2024-09-21T00:25:54.826871+0000","last_clean":"2024-09-21T00:25:54.826871+0000","last_became_active":"2024-09-21T00:25:48.228524+0000","last_became_peered":"2024-09-21T00:25:48.228524+0000","last_unstale":"2024-09-21T00:25:54.826871+0000","last_undegraded":"2024-09-21T00:25:54.826871+0000","last_fullsized":"2024-09-21T00:25:54.826871+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:28:02.233472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.827937+0000","last_change":"2024-09-21T00:25:47.232666+0000","last_active":"2024-09-21T00:25:54.827937+0000","last_peered":"2024-09-21T00:25:54.827937+0000","last_clean":"2024-09-21T00:25:54.827937+0000","last_became_active":"2024-09-21T00:25:47.232345+0000","last_became_peered":"2024-09-21T00:25:47.232345+0000","last_unstale":"2024-09-21T00:25:54.827937+0000","last_undegraded":"2024-09-21T00:25:54.827937+0000","last_fullsized":"2024-09-21T00:25:54.827937+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_clean_scrub_stamp":"2024-09-21T00:25:46.193690+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:43:08.714258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.263529+0000","last_change":"2024-09-21T00:25:52.550535+0000","last_active":"2024-09-21T00:25:53.263529+0000","last_peered":"2024-09-21T00:25:53.263529+0000","last_clean":"2024-09-21T00:25:53.263529+0000","last_became_active":"2024-09-21T00:25:52.550055+0000","last_became_peered":"2024-09-21T00:25:52.550055+0000","last_unstale":"2024-09-21T00:25:53.263529+0000","last_undegraded":"2024-09-21T00:25:53.263529+0000","last_fullsized":"2024-09-21T00:25:53.263529+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_clean_scrub_stamp":"2024-09-21T00:25:51.238933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:02:53.617106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.265087+0000","last_change":"2024-09-21T00:25:48.238219+0000","last_active":"2024-09-21T00:25:53.265087+0000","last_peered":"2024-09-21T00:25:53.265087+0000","last_clean":"2024-09-21T00:25:53.265087+0000","last_became_active":"2024-09-21T00:25:48.237770+0000","last_became_peered":"2024-09-21T00:25:48.237770+0000","last_unstale":"2024-09-21T00:25:53.265087+0000","last_undegraded":"2024-09-21T00:25:53.265087+0000","last_fullsized":"2024-09-21T00:25:53.265087+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:34:07.237786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.324720+0000","last_change":"2024-09-21T00:25:47.229361+0000","last_active":"2024-09-21T00:25:54.324720+0000","last_peered":"2024-09-21T00:25:54.324720+0000","last_clean":"2024-09-21T00:25:54.324720+0000","last_became_active":"2024-09-21T00:25:47.228906+0000","last_became_peered":"2024-09-21T00:25:47.228906+0000","last_unstale":"2024-09-21T00:25:54.324720+0000","last_undegraded":"2024-09-21T00:25:54.324720+0000","last_fullsized":"2024-09-21T00:25:54.324720+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_clean_scrub_stamp":"2024-09-21T00:25:46.193690+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:35:43.503231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.263731+0000","last_change":"2024-09-21T00:25:52.550909+0000","last_active":"2024-09-21T00:25:53.263731+0000","last_peered":"2024-09-21T00:25:53.263731+0000","last_clean":"2024-09-21T00:25:53.263731+0000","last_became_active":"2024-09-21T00:25:52.550506+0000","last_became_peered":"2024-09-21T00:25:52.550506+0000","last_unstale":"2024-09-21T00:25:53.263731+0000","last_undegraded":"2024-09-21T00:25:53.263731+0000","last_fullsized":"2024-09-21T00:25:53.263731+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_clean_scrub_stamp":"2024-09-21T00:25:51.238933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:52:16.289687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.324538+0000","last_change":"2024-09-21T00:25:48.232779+0000","last_active":"2024-09-21T00:25:54.324538+0000","last_peered":"2024-09-21T00:25:54.324538+0000","last_clean":"2024-09-21T00:25:54.324538+0000","last_became_active":"2024-09-21T00:25:48.232461+0000","last_became_peered":"2024-09-21T00:25:48.232461+0000","last_unstale":"2024-09-21T00:25:54.324538+0000","last_undegraded":"2024-09-21T00:25:54.324538+0000","last_fullsized":"2024-09-21T00:25:54.324538+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:52:33.265365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.827491+0000","last_change":"2024-09-21T00:25:47.237661+0000","last_active":"2024-09-21T00:25:54.827491+0000","last_peered":"2024-09-21T00:25:54.827491+0000","last_clean":"2024-09-21T00:25:54.827491+0000","last_became_active":"2024-09-21T00:25:47.237273+0000","last_became_peered":"2024-09-21T00:25:47.237273+0000","last_unstale":"2024-09-21T00:25:54.827491+0000","last_undegraded":"2024-09-21T00:25:54.827491+0000","last_fullsized":"2024-09-21T00:25:54.827491+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_clean_scrub_stamp":"2024-09-21T00:25:46.193690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:03:52.795421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.826240+0000","last_change":"2024-09-21T00:25:52.549849+0000","last_active":"2024-09-21T00:25:54.826240+0000","last_peered":"2024-09-21T00:25:54.826240+0000","last_clean":"2024-09-21T00:25:54.826240+0000","last_became_active":"2024-09-21T00:25:52.549433+0000","last_became_peered":"2024-09-21T00:25:52.549433+0000","last_unstale":"2024-09-21T00:25:54.826240+0000","last_undegraded":"2024-09-21T00:25:54.826240+0000","last_fullsized":"2024-09-21T00:25:54.826240+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_clean_scrub_stamp":"2024-09-21T00:25:51.238933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:51:49.334552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.264813+0000","last_change":"2024-09-21T00:25:48.232652+0000","last_active":"2024-09-21T00:25:53.264813+0000","last_peered":"2024-09-21T00:25:53.264813+0000","last_clean":"2024-09-21T00:25:53.264813+0000","last_became_active":"2024-09-21T00:25:48.232099+0000","last_became_peered":"2024-09-21T00:25:48.232099+0000","last_unstale":"2024-09-21T00:25:53.264813+0000","last_undegraded":"2024-09-21T00:25:53.264813+0000","last_fullsized":"2024-09-21T00:25:53.264813+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:48:25.219726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.324657+0000","last_change":"2024-09-21T00:25:47.223473+0000","last_active":"2024-09-21T00:25:54.324657+0000","last_peered":"2024-09-21T00:25:54.324657+0000","last_clean":"2024-09-21T00:25:54.324657+0000","last_became_active":"2024-09-21T00:25:47.223142+0000","last_became_peered":"2024-09-21T00:25:47.223142+0000","last_unstale":"2024-09-21T00:25:54.324657+0000","last_undegraded":"2024-09-21T00:25:54.324657+0000","last_fullsized":"2024-09-21T00:25:54.324657+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_clean_scrub_stamp":"2024-09-21T00:25:46.193690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:54:03.173672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.324126+0000","last_change":"2024-09-21T00:25:52.276922+0000","last_active":"2024-09-21T00:25:54.324126+0000","last_peered":"2024-09-21T00:25:54.324126+0000","last_clean":"2024-09-21T00:25:54.324126+0000","last_became_active":"2024-09-21T00:25:52.276329+0000","last_became_peered":"2024-09-21T00:25:52.276329+0000","last_unstale":"2024-09-21T00:25:54.324126+0000","last_undegraded":"2024-09-21T00:25:54.324126+0000","last_fullsized":"2024-09-21T00:25:54.324126+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_clean_scrub_stamp":"2024-09-21T00:25:51.238933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:29:17.920252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.264377+0000","last_change":"2024-09-21T00:25:48.230144+0000","last_active":"2024-09-21T00:25:53.264377+0000","last_peered":"2024-09-21T00:25:53.264377+0000","last_clean":"2024-09-21T00:25:53.264377+0000","last_became_active":"2024-09-21T00:25:48.229848+0000","last_became_peered":"2024-09-21T00:25:48.229848+0000","last_unstale":"2024-09-21T00:25:53.264377+0000","last_undegraded":"2024-09-21T00:25:53.264377+0000","last_fullsized":"2024-09-21T00:25:53.264377+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:57:59.920308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.263649+0000","last_change":"2024-09-21T00:25:47.228030+0000","last_active":"2024-09-21T00:25:53.263649+0000","last_peered":"2024-09-21T00:25:53.263649+0000","last_clean":"2024-09-21T00:25:53.263649+0000","last_became_active":"2024-09-21T00:25:47.227688+0000","last_became_peered":"2024-09-21T00:25:47.227688+0000","last_unstale":"2024-09-21T00:25:53.263649+0000","last_undegraded":"2024-09-21T00:25:53.263649+0000","last_fullsized":"2024-09-21T00:25:53.263649+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_clean_scrub_stamp":"2024-09-21T00:25:46.193690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:34:30.210849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.324624+0000","last_change":"2024-09-21T00:25:52.276547+0000","last_active":"2024-09-21T00:25:54.324624+0000","last_peered":"2024-09-21T00:25:54.324624+0000","last_clean":"2024-09-21T00:25:54.324624+0000","last_became_active":"2024-09-21T00:25:52.276100+0000","last_became_peered":"2024-09-21T00:25:52.276100+0000","last_unstale":"2024-09-21T00:25:54.324624+0000","last_undegraded":"2024-09-21T00:25:54.324624+0000","last_fullsized":"2024-09-21T00:25:54.324624+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_clean_scrub_stamp":"2024-09-21T00:25:51.238933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:08:54.038371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.324189+0000","last_change":"2024-09-21T00:25:48.233554+0000","last_active":"2024-09-21T00:25:54.324189+0000","last_peered":"2024-09-21T00:25:54.324189+0000","last_clean":"2024-09-21T00:25:54.324189+0000","last_became_active":"2024-09-21T00:25:48.233249+0000","last_became_peered":"2024-09-21T00:25:48.233249+0000","last_unstale":"2024-09-21T00:25:54.324189+0000","last_undegraded":"2024-09-21T00:25:54.324189+0000","last_fullsized":"2024-09-21T00:25:54.324189+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:18:14.964429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.242121+0000","last_change":"2024-09-21T00:25:47.228079+0000","last_active":"2024-09-21T00:25:54.242121+0000","last_peered":"2024-09-21T00:25:54.242121+0000","last_clean":"2024-09-21T00:25:54.242121+0000","last_became_active":"2024-09-21T00:25:47.227731+0000","last_became_peered":"2024-09-21T00:25:47.227731+0000","last_unstale":"2024-09-21T00:25:54.242121+0000","last_undegraded":"2024-09-21T00:25:54.242121+0000","last_fullsized":"2024-09-21T00:25:54.242121+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_clean_scrub_stamp":"2024-09-21T00:25:46.193690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:53:37.081118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.324987+0000","last_change":"2024-09-21T00:25:52.278680+0000","last_active":"2024-09-21T00:25:54.324987+0000","last_peered":"2024-09-21T00:25:54.324987+0000","last_clean":"2024-09-21T00:25:54.324987+0000","last_became_active":"2024-09-21T00:25:52.278128+0000","last_became_peered":"2024-09-21T00:25:52.278128+0000","last_unstale":"2024-09-21T00:25:54.324987+0000","last_undegraded":"2024-09-21T00:25:54.324987+0000","last_fullsized":"2024-09-21T00:25:54.324987+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_clean_scrub_stamp":"2024-09-21T00:25:51.238933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:39:02.970919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.826121+0000","last_change":"2024-09-21T00:25:48.245138+0000","last_active":"2024-09-21T00:25:54.826121+0000","last_peered":"2024-09-21T00:25:54.826121+0000","last_clean":"2024-09-21T00:25:54.826121+0000","last_became_active":"2024-09-21T00:25:48.244796+0000","last_became_peered":"2024-09-21T00:25:48.244796+0000","last_unstale":"2024-09-21T00:25:54.826121+0000","last_undegraded":"2024-09-21T00:25:54.826121+0000","last_fullsized":"2024-09-21T00:25:54.826121+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:53:56.396789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.826939+0000","last_change":"2024-09-21T00:25:47.235076+0000","last_active":"2024-09-21T00:25:54.826939+0000","last_peered":"2024-09-21T00:25:54.826939+0000","last_clean":"2024-09-21T00:25:54.826939+0000","last_became_active":"2024-09-21T00:25:47.234751+0000","last_became_peered":"2024-09-21T00:25:47.234751+0000","last_unstale":"2024-09-21T00:25:54.826939+0000","last_undegraded":"2024-09-21T00:25:54.826939+0000","last_fullsized":"2024-09-21T00:25:54.826939+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_clean_scrub_stamp":"2024-09-21T00:25:46.193690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:40:10.553216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.826789+0000","last_change":"2024-09-21T00:25:52.546980+0000","last_active":"2024-09-21T00:25:54.826789+0000","last_peered":"2024-09-21T00:25:54.826789+0000","last_clean":"2024-09-21T00:25:54.826789+0000","last_became_active":"2024-09-21T00:25:52.546572+0000","last_became_peered":"2024-09-21T00:25:52.546572+0000","last_unstale":"2024-09-21T00:25:54.826789+0000","last_undegraded":"2024-09-21T00:25:54.826789+0000","last_fullsized":"2024-09-21T00:25:54.826789+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_clean_scrub_stamp":"2024-09-21T00:25:51.238933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:34:14.702564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.263892+0000","last_change":"2024-09-21T00:25:48.227417+0000","last_active":"2024-09-21T00:25:53.263892+0000","last_peered":"2024-09-21T00:25:53.263892+0000","last_clean":"2024-09-21T00:25:53.263892+0000","last_became_active":"2024-09-21T00:25:48.227056+0000","last_became_peered":"2024-09-21T00:25:48.227056+0000","last_unstale":"2024-09-21T00:25:53.263892+0000","last_undegraded":"2024-09-21T00:25:53.263892+0000","last_fullsized":"2024-09-21T00:25:53.263892+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:16:33.163623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.264390+0000","last_change":"2024-09-21T00:25:47.228317+0000","last_active":"2024-09-21T00:25:53.264390+0000","last_peered":"2024-09-21T00:25:53.264390+0000","last_clean":"2024-09-21T00:25:53.264390+0000","last_became_active":"2024-09-21T00:25:47.227983+0000","last_became_peered":"2024-09-21T00:25:47.227983+0000","last_unstale":"2024-09-21T00:25:53.264390+0000","last_undegraded":"2024-09-21T00:25:53.264390+0000","last_fullsized":"2024-09-21T00:25:53.264390+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_clean_scrub_stamp":"2024-09-21T00:25:46.193690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:17:12.345201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":119,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.827077+0000","last_change":"2024-09-21T00:25:38.725792+0000","last_active":"2024-09-21T00:25:54.827077+0000","last_peered":"2024-09-21T00:25:54.827077+0000","last_clean":"2024-09-21T00:25:54.827077+0000","last_became_active":"2024-09-21T00:25:38.724668+0000","last_became_peered":"2024-09-21T00:25:38.724668+0000","last_unstale":"2024-09-21T00:25:54.827077+0000","last_undegraded":"2024-09-21T00:25:54.827077+0000","last_fullsized":"2024-09-21T00:25:54.827077+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:37.163216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:37.163216+0000","last_clean_scrub_stamp":"2024-09-21T00:25:37.163216+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:14:36.588660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.827086+0000","last_change":"2024-09-21T00:25:52.551370+0000","last_active":"2024-09-21T00:25:54.827086+0000","last_peered":"2024-09-21T00:25:54.827086+0000","last_clean":"2024-09-21T00:25:54.827086+0000","last_became_active":"2024-09-21T00:25:52.550883+0000","last_became_peered":"2024-09-21T00:25:52.550883+0000","last_unstale":"2024-09-21T00:25:54.827086+0000","last_undegraded":"2024-09-21T00:25:54.827086+0000","last_fullsized":"2024-09-21T00:25:54.827086+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_clean_scrub_stamp":"2024-09-21T00:25:51.238933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T00:31:31.841290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.826927+0000","last_change":"2024-09-21T00:25:48.247427+0000","last_active":"2024-09-21T00:25:54.826927+0000","last_peered":"2024-09-21T00:25:54.826927+0000","last_clean":"2024-09-21T00:25:54.826927+0000","last_became_active":"2024-09-21T00:25:48.246876+0000","last_became_peered":"2024-09-21T00:25:48.246876+0000","last_unstale":"2024-09-21T00:25:54.826927+0000","last_undegraded":"2024-09-21T00:25:54.826927+0000","last_fullsized":"2024-09-21T00:25:54.826927+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:10:01.543628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.242401+0000","last_change":"2024-09-21T00:25:47.233336+0000","last_active":"2024-09-21T00:25:54.242401+0000","last_peered":"2024-09-21T00:25:54.242401+0000","last_clean":"2024-09-21T00:25:54.242401+0000","last_became_active":"2024-09-21T00:25:47.233083+0000","last_became_peered":"2024-09-21T00:25:47.233083+0000","last_unstale":"2024-09-21T00:25:54.242401+0000","last_undegraded":"2024-09-21T00:25:54.242401+0000","last_fullsized":"2024-09-21T00:25:54.242401+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_clean_scrub_stamp":"2024-09-21T00:25:46.193690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:43:39.508965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.826781+0000","last_change":"2024-09-21T00:25:52.550144+0000","last_active":"2024-09-21T00:25:54.826781+0000","last_peered":"2024-09-21T00:25:54.826781+0000","last_clean":"2024-09-21T00:25:54.826781+0000","last_became_active":"2024-09-21T00:25:52.549636+0000","last_became_peered":"2024-09-21T00:25:52.549636+0000","last_unstale":"2024-09-21T00:25:54.826781+0000","last_undegraded":"2024-09-21T00:25:54.826781+0000","last_fullsized":"2024-09-21T00:25:54.826781+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_clean_scrub_stamp":"2024-09-21T00:25:51.238933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:09:27.474968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.263877+0000","last_change":"2024-09-21T00:25:48.229999+0000","last_active":"2024-09-21T00:25:53.263877+0000","last_peered":"2024-09-21T00:25:53.263877+0000","last_clean":"2024-09-21T00:25:53.263877+0000","last_became_active":"2024-09-21T00:25:48.229678+0000","last_became_peered":"2024-09-21T00:25:48.229678+0000","last_unstale":"2024-09-21T00:25:53.263877+0000","last_undegraded":"2024-09-21T00:25:53.263877+0000","last_fullsized":"2024-09-21T00:25:53.263877+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:15:47.572701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.324827+0000","last_change":"2024-09-21T00:25:47.219135+0000","last_active":"2024-09-21T00:25:54.324827+0000","last_peered":"2024-09-21T00:25:54.324827+0000","last_clean":"2024-09-21T00:25:54.324827+0000","last_became_active":"2024-09-21T00:25:47.218722+0000","last_became_peered":"2024-09-21T00:25:47.218722+0000","last_unstale":"2024-09-21T00:25:54.324827+0000","last_undegraded":"2024-09-21T00:25:54.324827+0000","last_fullsized":"2024-09-21T00:25:54.324827+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_clean_scrub_stamp":"2024-09-21T00:25:46.193690+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:52:45.409090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.826420+0000","last_change":"2024-09-21T00:25:52.279946+0000","last_active":"2024-09-21T00:25:54.826420+0000","last_peered":"2024-09-21T00:25:54.826420+0000","last_clean":"2024-09-21T00:25:54.826420+0000","last_became_active":"2024-09-21T00:25:52.279489+0000","last_became_peered":"2024-09-21T00:25:52.279489+0000","last_unstale":"2024-09-21T00:25:54.826420+0000","last_undegraded":"2024-09-21T00:25:54.826420+0000","last_fullsized":"2024-09-21T00:25:54.826420+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_clean_scrub_stamp":"2024-09-21T00:25:51.238933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:47:42.414865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.827648+0000","last_change":"2024-09-21T00:25:48.244146+0000","last_active":"2024-09-21T00:25:54.827648+0000","last_peered":"2024-09-21T00:25:54.827648+0000","last_clean":"2024-09-21T00:25:54.827648+0000","last_became_active":"2024-09-21T00:25:48.243782+0000","last_became_peered":"2024-09-21T00:25:48.243782+0000","last_unstale":"2024-09-21T00:25:54.827648+0000","last_undegraded":"2024-09-21T00:25:54.827648+0000","last_fullsized":"2024-09-21T00:25:54.827648+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:43:59.908291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.242461+0000","last_change":"2024-09-21T00:25:47.230785+0000","last_active":"2024-09-21T00:25:54.242461+0000","last_peered":"2024-09-21T00:25:54.242461+0000","last_clean":"2024-09-21T00:25:54.242461+0000","last_became_active":"2024-09-21T00:25:47.230492+0000","last_became_peered":"2024-09-21T00:25:47.230492+0000","last_unstale":"2024-09-21T00:25:54.242461+0000","last_undegraded":"2024-09-21T00:25:54.242461+0000","last_fullsized":"2024-09-21T00:25:54.242461+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_clean_scrub_stamp":"2024-09-21T00:25:46.193690+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:33:07.366842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.262402+0000","last_change":"2024-09-21T00:25:52.547298+0000","last_active":"2024-09-21T00:25:53.262402+0000","last_peered":"2024-09-21T00:25:53.262402+0000","last_clean":"2024-09-21T00:25:53.262402+0000","last_became_active":"2024-09-21T00:25:52.546737+0000","last_became_peered":"2024-09-21T00:25:52.546737+0000","last_unstale":"2024-09-21T00:25:53.262402+0000","last_undegraded":"2024-09-21T00:25:53.262402+0000","last_fullsized":"2024-09-21T00:25:53.262402+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_clean_scrub_stamp":"2024-09-21T00:25:51.238933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:54:49.138927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.827233+0000","last_change":"2024-09-21T00:25:48.244124+0000","last_active":"2024-09-21T00:25:54.827233+0000","last_peered":"2024-09-21T00:25:54.827233+0000","last_clean":"2024-09-21T00:25:54.827233+0000","last_became_active":"2024-09-21T00:25:48.243767+0000","last_became_peered":"2024-09-21T00:25:48.243767+0000","last_unstale":"2024-09-21T00:25:54.827233+0000","last_undegraded":"2024-09-21T00:25:54.827233+0000","last_fullsized":"2024-09-21T00:25:54.827233+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:15:47.572701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.264246+0000","last_change":"2024-09-21T00:25:47.220781+0000","last_active":"2024-09-21T00:25:53.264246+0000","last_peered":"2024-09-21T00:25:53.264246+0000","last_clean":"2024-09-21T00:25:53.264246+0000","last_became_active":"2024-09-21T00:25:47.220429+0000","last_became_peered":"2024-09-21T00:25:47.220429+0000","last_unstale":"2024-09-21T00:25:53.264246+0000","last_undegraded":"2024-09-21T00:25:53.264246+0000","last_fullsized":"2024-09-21T00:25:53.264246+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_clean_scrub_stamp":"2024-09-21T00:25:46.193690+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:08:24.394801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.262197+0000","last_change":"2024-09-21T00:25:52.276941+0000","last_active":"2024-09-21T00:25:53.262197+0000","last_peered":"2024-09-21T00:25:53.262197+0000","last_clean":"2024-09-21T00:25:53.262197+0000","last_became_active":"2024-09-21T00:25:52.276374+0000","last_became_peered":"2024-09-21T00:25:52.276374+0000","last_unstale":"2024-09-21T00:25:53.262197+0000","last_undegraded":"2024-09-21T00:25:53.262197+0000","last_fullsized":"2024-09-21T00:25:53.262197+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_clean_scrub_stamp":"2024-09-21T00:25:51.238933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:02:24.501560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.826441+0000","last_change":"2024-09-21T00:25:48.244881+0000","last_active":"2024-09-21T00:25:54.826441+0000","last_peered":"2024-09-21T00:25:54.826441+0000","last_clean":"2024-09-21T00:25:54.826441+0000","last_became_active":"2024-09-21T00:25:48.244530+0000","last_became_peered":"2024-09-21T00:25:48.244530+0000","last_unstale":"2024-09-21T00:25:54.826441+0000","last_undegraded":"2024-09-21T00:25:54.826441+0000","last_fullsized":"2024-09-21T00:25:54.826441+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:51:31.831106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":31,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.242013+0000","last_change":"2024-09-21T00:25:47.230620+0000","last_active":"2024-09-21T00:25:54.242013+0000","last_peered":"2024-09-21T00:25:54.242013+0000","last_clean":"2024-09-21T00:25:54.242013+0000","last_became_active":"2024-09-21T00:25:47.230252+0000","last_became_peered":"2024-09-21T00:25:47.230252+0000","last_unstale":"2024-09-21T00:25:54.242013+0000","last_undegraded":"2024-09-21T00:25:54.242013+0000","last_fullsized":"2024-09-21T00:25:54.242013+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_clean_scrub_stamp":"2024-09-21T00:25:46.193690+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:16:00.090070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.826590+0000","last_change":"2024-09-21T00:25:52.266721+0000","last_active":"2024-09-21T00:25:54.826590+0000","last_peered":"2024-09-21T00:25:54.826590+0000","last_clean":"2024-09-21T00:25:54.826590+0000","last_became_active":"2024-09-21T00:25:52.266271+0000","last_became_peered":"2024-09-21T00:25:52.266271+0000","last_unstale":"2024-09-21T00:25:54.826590+0000","last_undegraded":"2024-09-21T00:25:54.826590+0000","last_fullsized":"2024-09-21T00:25:54.826590+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_clean_scrub_stamp":"2024-09-21T00:25:51.238933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:15:37.112368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.826580+0000","last_change":"2024-09-21T00:25:48.239188+0000","last_active":"2024-09-21T00:25:54.826580+0000","last_peered":"2024-09-21T00:25:54.826580+0000","last_clean":"2024-09-21T00:25:54.826580+0000","last_became_active":"2024-09-21T00:25:48.238774+0000","last_became_peered":"2024-09-21T00:25:48.238774+0000","last_unstale":"2024-09-21T00:25:54.826580+0000","last_undegraded":"2024-09-21T00:25:54.826580+0000","last_fullsized":"2024-09-21T00:25:54.826580+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T00:56:56.658210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.826897+0000","last_change":"2024-09-21T00:25:47.230533+0000","last_active":"2024-09-21T00:25:54.826897+0000","last_peered":"2024-09-21T00:25:54.826897+0000","last_clean":"2024-09-21T00:25:54.826897+0000","last_became_active":"2024-09-21T00:25:47.230211+0000","last_became_peered":"2024-09-21T00:25:47.230211+0000","last_unstale":"2024-09-21T00:25:54.826897+0000","last_undegraded":"2024-09-21T00:25:54.826897+0000","last_fullsized":"2024-09-21T00:25:54.826897+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_clean_scrub_stamp":"2024-09-21T00:25:46.193690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:54:03.173672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.265006+0000","last_change":"2024-09-21T00:25:52.550405+0000","last_active":"2024-09-21T00:25:53.265006+0000","last_peered":"2024-09-21T00:25:53.265006+0000","last_clean":"2024-09-21T00:25:53.265006+0000","last_became_active":"2024-09-21T00:25:52.549743+0000","last_became_peered":"2024-09-21T00:25:52.549743+0000","last_unstale":"2024-09-21T00:25:53.265006+0000","last_undegraded":"2024-09-21T00:25:53.265006+0000","last_fullsized":"2024-09-21T00:25:53.265006+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_clean_scrub_stamp":"2024-09-21T00:25:51.238933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T00:40:38.549590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.826686+0000","last_change":"2024-09-21T00:25:48.244073+0000","last_active":"2024-09-21T00:25:54.826686+0000","last_peered":"2024-09-21T00:25:54.826686+0000","last_clean":"2024-09-21T00:25:54.826686+0000","last_became_active":"2024-09-21T00:25:48.243749+0000","last_became_peered":"2024-09-21T00:25:48.243749+0000","last_unstale":"2024-09-21T00:25:54.826686+0000","last_undegraded":"2024-09-21T00:25:54.826686+0000","last_fullsized":"2024-09-21T00:25:54.826686+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:24:47.112012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.263546+0000","last_change":"2024-09-21T00:25:47.229294+0000","last_active":"2024-09-21T00:25:53.263546+0000","last_peered":"2024-09-21T00:25:53.263546+0000","last_clean":"2024-09-21T00:25:53.263546+0000","last_became_active":"2024-09-21T00:25:47.228901+0000","last_became_peered":"2024-09-21T00:25:47.228901+0000","last_unstale":"2024-09-21T00:25:53.263546+0000","last_undegraded":"2024-09-21T00:25:53.263546+0000","last_fullsized":"2024-09-21T00:25:53.263546+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_clean_scrub_stamp":"2024-09-21T00:25:46.193690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:55:28.486627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.264577+0000","last_change":"2024-09-21T00:25:52.547298+0000","last_active":"2024-09-21T00:25:53.264577+0000","last_peered":"2024-09-21T00:25:53.264577+0000","last_clean":"2024-09-21T00:25:53.264577+0000","last_became_active":"2024-09-21T00:25:52.546647+0000","last_became_peered":"2024-09-21T00:25:52.546647+0000","last_unstale":"2024-09-21T00:25:53.264577+0000","last_undegraded":"2024-09-21T00:25:53.264577+0000","last_fullsized":"2024-09-21T00:25:53.264577+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_clean_scrub_stamp":"2024-09-21T00:25:51.238933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:10:45.091073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.826321+0000","last_change":"2024-09-21T00:25:48.237722+0000","last_active":"2024-09-21T00:25:54.826321+0000","last_peered":"2024-09-21T00:25:54.826321+0000","last_clean":"2024-09-21T00:25:54.826321+0000","last_became_active":"2024-09-21T00:25:48.237400+0000","last_became_peered":"2024-09-21T00:25:48.237400+0000","last_unstale":"2024-09-21T00:25:54.826321+0000","last_undegraded":"2024-09-21T00:25:54.826321+0000","last_fullsized":"2024-09-21T00:25:54.826321+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:00:11.346065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.826654+0000","last_change":"2024-09-21T00:25:47.225035+0000","last_active":"2024-09-21T00:25:54.826654+0000","last_peered":"2024-09-21T00:25:54.826654+0000","last_clean":"2024-09-21T00:25:54.826654+0000","last_became_active":"2024-09-21T00:25:47.224732+0000","last_became_peered":"2024-09-21T00:25:47.224732+0000","last_unstale":"2024-09-21T00:25:54.826654+0000","last_undegraded":"2024-09-21T00:25:54.826654+0000","last_fullsized":"2024-09-21T00:25:54.826654+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_clean_scrub_stamp":"2024-09-21T00:25:46.193690+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:16:00.090070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.264690+0000","last_change":"2024-09-21T00:25:52.550751+0000","last_active":"2024-09-21T00:25:53.264690+0000","last_peered":"2024-09-21T00:25:53.264690+0000","last_clean":"2024-09-21T00:25:53.264690+0000","last_became_active":"2024-09-21T00:25:52.550166+0000","last_became_peered":"2024-09-21T00:25:52.550166+0000","last_unstale":"2024-09-21T00:25:53.264690+0000","last_undegraded":"2024-09-21T00:25:53.264690+0000","last_fullsized":"2024-09-21T00:25:53.264690+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_clean_scrub_stamp":"2024-09-21T00:25:51.238933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:01:14.362816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.826761+0000","last_change":"2024-09-21T00:25:48.240297+0000","last_active":"2024-09-21T00:25:54.826761+0000","last_peered":"2024-09-21T00:25:54.826761+0000","last_clean":"2024-09-21T00:25:54.826761+0000","last_became_active":"2024-09-21T00:25:48.239975+0000","last_became_peered":"2024-09-21T00:25:48.239975+0000","last_unstale":"2024-09-21T00:25:54.826761+0000","last_undegraded":"2024-09-21T00:25:54.826761+0000","last_fullsized":"2024-09-21T00:25:54.826761+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:07:41.736005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"26'9","reported_seq":32,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:57.713994+0000","last_change":"2024-09-21T00:25:47.228804+0000","last_active":"2024-09-21T00:25:57.713994+0000","last_peered":"2024-09-21T00:25:57.713994+0000","last_clean":"2024-09-21T00:25:57.713994+0000","last_became_active":"2024-09-21T00:25:47.228490+0000","last_became_peered":"2024-09-21T00:25:47.228490+0000","last_unstale":"2024-09-21T00:25:57.713994+0000","last_undegraded":"2024-09-21T00:25:57.713994+0000","last_fullsized":"2024-09-21T00:25:57.713994+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_clean_scrub_stamp":"2024-09-21T00:25:46.193690+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:35:43.503231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.324519+0000","last_change":"2024-09-21T00:25:52.273360+0000","last_active":"2024-09-21T00:25:54.324519+0000","last_peered":"2024-09-21T00:25:54.324519+0000","last_clean":"2024-09-21T00:25:54.324519+0000","last_became_active":"2024-09-21T00:25:52.272888+0000","last_became_peered":"2024-09-21T00:25:52.272888+0000","last_unstale":"2024-09-21T00:25:54.324519+0000","last_undegraded":"2024-09-21T00:25:54.324519+0000","last_fullsized":"2024-09-21T00:25:54.324519+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_clean_scrub_stamp":"2024-09-21T00:25:51.238933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:31:19.736643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.264518+0000","last_change":"2024-09-21T00:25:48.238506+0000","last_active":"2024-09-21T00:25:53.264518+0000","last_peered":"2024-09-21T00:25:53.264518+0000","last_clean":"2024-09-21T00:25:53.264518+0000","last_became_active":"2024-09-21T00:25:48.237355+0000","last_became_peered":"2024-09-21T00:25:48.237355+0000","last_unstale":"2024-09-21T00:25:53.264518+0000","last_undegraded":"2024-09-21T00:25:53.264518+0000","last_fullsized":"2024-09-21T00:25:53.264518+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:40:11.560626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.324726+0000","last_change":"2024-09-21T00:25:47.224421+0000","last_active":"2024-09-21T00:25:54.324726+0000","last_peered":"2024-09-21T00:25:54.324726+0000","last_clean":"2024-09-21T00:25:54.324726+0000","last_became_active":"2024-09-21T00:25:47.224096+0000","last_became_peered":"2024-09-21T00:25:47.224096+0000","last_unstale":"2024-09-21T00:25:54.324726+0000","last_undegraded":"2024-09-21T00:25:54.324726+0000","last_fullsized":"2024-09-21T00:25:54.324726+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_clean_scrub_stamp":"2024-09-21T00:25:46.193690+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:43:08.714258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.263548+0000","last_change":"2024-09-21T00:25:52.277635+0000","last_active":"2024-09-21T00:25:53.263548+0000","last_peered":"2024-09-21T00:25:53.263548+0000","last_clean":"2024-09-21T00:25:53.263548+0000","last_became_active":"2024-09-21T00:25:52.276739+0000","last_became_peered":"2024-09-21T00:25:52.276739+0000","last_unstale":"2024-09-21T00:25:53.263548+0000","last_undegraded":"2024-09-21T00:25:53.263548+0000","last_fullsized":"2024-09-21T00:25:53.263548+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_clean_scrub_stamp":"2024-09-21T00:25:51.238933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:46:09.670107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.826816+0000","last_change":"2024-09-21T00:25:48.233598+0000","last_active":"2024-09-21T00:25:54.826816+0000","last_peered":"2024-09-21T00:25:54.826816+0000","last_clean":"2024-09-21T00:25:54.826816+0000","last_became_active":"2024-09-21T00:25:48.233203+0000","last_became_peered":"2024-09-21T00:25:48.233203+0000","last_unstale":"2024-09-21T00:25:54.826816+0000","last_undegraded":"2024-09-21T00:25:54.826816+0000","last_fullsized":"2024-09-21T00:25:54.826816+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:36:54.554284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.263639+0000","last_change":"2024-09-21T00:25:47.225271+0000","last_active":"2024-09-21T00:25:53.263639+0000","last_peered":"2024-09-21T00:25:53.263639+0000","last_clean":"2024-09-21T00:25:53.263639+0000","last_became_active":"2024-09-21T00:25:47.224935+0000","last_became_peered":"2024-09-21T00:25:47.224935+0000","last_unstale":"2024-09-21T00:25:53.263639+0000","last_undegraded":"2024-09-21T00:25:53.263639+0000","last_fullsized":"2024-09-21T00:25:53.263639+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_clean_scrub_stamp":"2024-09-21T00:25:46.193690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:06:59.889895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.264083+0000","last_change":"2024-09-21T00:25:52.278653+0000","last_active":"2024-09-21T00:25:53.264083+0000","last_peered":"2024-09-21T00:25:53.264083+0000","last_clean":"2024-09-21T00:25:53.264083+0000","last_became_active":"2024-09-21T00:25:52.278111+0000","last_became_peered":"2024-09-21T00:25:52.278111+0000","last_unstale":"2024-09-21T00:25:53.264083+0000","last_undegraded":"2024-09-21T00:25:53.264083+0000","last_fullsized":"2024-09-21T00:25:53.264083+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_clean_scrub_stamp":"2024-09-21T00:25:51.238933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:01:24.675942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.263506+0000","last_change":"2024-09-21T00:25:48.239666+0000","last_active":"2024-09-21T00:25:53.263506+0000","last_peered":"2024-09-21T00:25:53.263506+0000","last_clean":"2024-09-21T00:25:53.263506+0000","last_became_active":"2024-09-21T00:25:48.239196+0000","last_became_peered":"2024-09-21T00:25:48.239196+0000","last_unstale":"2024-09-21T00:25:53.263506+0000","last_undegraded":"2024-09-21T00:25:53.263506+0000","last_fullsized":"2024-09-21T00:25:53.263506+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:00:19.889965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.827810+0000","last_change":"2024-09-21T00:25:47.225244+0000","last_active":"2024-09-21T00:25:54.827810+0000","last_peered":"2024-09-21T00:25:54.827810+0000","last_clean":"2024-09-21T00:25:54.827810+0000","last_became_active":"2024-09-21T00:25:47.224812+0000","last_became_peered":"2024-09-21T00:25:47.224812+0000","last_unstale":"2024-09-21T00:25:54.827810+0000","last_undegraded":"2024-09-21T00:25:54.827810+0000","last_fullsized":"2024-09-21T00:25:54.827810+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_clean_scrub_stamp":"2024-09-21T00:25:46.193690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:06:39.901371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.264361+0000","last_change":"2024-09-21T00:25:52.550130+0000","last_active":"2024-09-21T00:25:53.264361+0000","last_peered":"2024-09-21T00:25:53.264361+0000","last_clean":"2024-09-21T00:25:53.264361+0000","last_became_active":"2024-09-21T00:25:52.549675+0000","last_became_peered":"2024-09-21T00:25:52.549675+0000","last_unstale":"2024-09-21T00:25:53.264361+0000","last_undegraded":"2024-09-21T00:25:53.264361+0000","last_fullsized":"2024-09-21T00:25:53.264361+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_clean_scrub_stamp":"2024-09-21T00:25:51.238933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:55:10.499479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.826619+0000","last_change":"2024-09-21T00:25:52.550630+0000","last_active":"2024-09-21T00:25:54.826619+0000","last_peered":"2024-09-21T00:25:54.826619+0000","last_clean":"2024-09-21T00:25:54.826619+0000","last_became_active":"2024-09-21T00:25:52.550098+0000","last_became_peered":"2024-09-21T00:25:52.550098+0000","last_unstale":"2024-09-21T00:25:54.826619+0000","last_undegraded":"2024-09-21T00:25:54.826619+0000","last_fullsized":"2024-09-21T00:25:54.826619+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_clean_scrub_stamp":"2024-09-21T00:25:51.238933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:43:32.503216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.324003+0000","last_change":"2024-09-21T00:25:48.227526+0000","last_active":"2024-09-21T00:25:54.324003+0000","last_peered":"2024-09-21T00:25:54.324003+0000","last_clean":"2024-09-21T00:25:54.324003+0000","last_became_active":"2024-09-21T00:25:48.227194+0000","last_became_peered":"2024-09-21T00:25:48.227194+0000","last_unstale":"2024-09-21T00:25:54.324003+0000","last_undegraded":"2024-09-21T00:25:54.324003+0000","last_fullsized":"2024-09-21T00:25:54.324003+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:54:09.449880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.262751+0000","last_change":"2024-09-21T00:25:47.220721+0000","last_active":"2024-09-21T00:25:53.262751+0000","last_peered":"2024-09-21T00:25:53.262751+0000","last_clean":"2024-09-21T00:25:53.262751+0000","last_became_active":"2024-09-21T00:25:47.220393+0000","last_became_peered":"2024-09-21T00:25:47.220393+0000","last_unstale":"2024-09-21T00:25:53.262751+0000","last_undegraded":"2024-09-21T00:25:53.262751+0000","last_fullsized":"2024-09-21T00:25:53.262751+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_clean_scrub_stamp":"2024-09-21T00:25:46.193690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:00:04.423924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.827438+0000","last_change":"2024-09-21T00:25:52.280179+0000","last_active":"2024-09-21T00:25:54.827438+0000","last_peered":"2024-09-21T00:25:54.827438+0000","last_clean":"2024-09-21T00:25:54.827438+0000","last_became_active":"2024-09-21T00:25:52.279617+0000","last_became_peered":"2024-09-21T00:25:52.279617+0000","last_unstale":"2024-09-21T00:25:54.827438+0000","last_undegraded":"2024-09-21T00:25:54.827438+0000","last_fullsized":"2024-09-21T00:25:54.827438+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_clean_scrub_stamp":"2024-09-21T00:25:51.238933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:30:11.179850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.264234+0000","last_change":"2024-09-21T00:25:48.237852+0000","last_active":"2024-09-21T00:25:53.264234+0000","last_peered":"2024-09-21T00:25:53.264234+0000","last_clean":"2024-09-21T00:25:53.264234+0000","last_became_active":"2024-09-21T00:25:48.237559+0000","last_became_peered":"2024-09-21T00:25:48.237559+0000","last_unstale":"2024-09-21T00:25:53.264234+0000","last_undegraded":"2024-09-21T00:25:53.264234+0000","last_fullsized":"2024-09-21T00:25:53.264234+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:28:02.233472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.264305+0000","last_change":"2024-09-21T00:25:47.227879+0000","last_active":"2024-09-21T00:25:53.264305+0000","last_peered":"2024-09-21T00:25:53.264305+0000","last_clean":"2024-09-21T00:25:53.264305+0000","last_became_active":"2024-09-21T00:25:47.227569+0000","last_became_peered":"2024-09-21T00:25:47.227569+0000","last_unstale":"2024-09-21T00:25:53.264305+0000","last_undegraded":"2024-09-21T00:25:53.264305+0000","last_fullsized":"2024-09-21T00:25:53.264305+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_clean_scrub_stamp":"2024-09-21T00:25:46.193690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:16:00.090070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.827281+0000","last_change":"2024-09-21T00:25:47.231506+0000","last_active":"2024-09-21T00:25:54.827281+0000","last_peered":"2024-09-21T00:25:54.827281+0000","last_clean":"2024-09-21T00:25:54.827281+0000","last_became_active":"2024-09-21T00:25:47.231135+0000","last_became_peered":"2024-09-21T00:25:47.231135+0000","last_unstale":"2024-09-21T00:25:54.827281+0000","last_undegraded":"2024-09-21T00:25:54.827281+0000","last_fullsized":"2024-09-21T00:25:54.827281+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_clean_scrub_stamp":"2024-09-21T00:25:46.193690+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:52:45.409090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.826519+0000","last_change":"2024-09-21T00:25:48.245503+0000","last_active":"2024-09-21T00:25:54.826519+0000","last_peered":"2024-09-21T00:25:54.826519+0000","last_clean":"2024-09-21T00:25:54.826519+0000","last_became_active":"2024-09-21T00:25:48.245168+0000","last_became_peered":"2024-09-21T00:25:48.245168+0000","last_unstale":"2024-09-21T00:25:54.826519+0000","last_undegraded":"2024-09-21T00:25:54.826519+0000","last_fullsized":"2024-09-21T00:25:54.826519+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:51:35.872806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.242222+0000","last_change":"2024-09-21T00:25:52.278289+0000","last_active":"2024-09-21T00:25:54.242222+0000","last_peered":"2024-09-21T00:25:54.242222+0000","last_clean":"2024-09-21T00:25:54.242222+0000","last_became_active":"2024-09-21T00:25:52.277874+0000","last_became_peered":"2024-09-21T00:25:52.277874+0000","last_unstale":"2024-09-21T00:25:54.242222+0000","last_undegraded":"2024-09-21T00:25:54.242222+0000","last_fullsized":"2024-09-21T00:25:54.242222+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_clean_scrub_stamp":"2024-09-21T00:25:51.238933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T00:47:59.166653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.264404+0000","last_change":"2024-09-21T00:25:47.232974+0000","last_active":"2024-09-21T00:25:53.264404+0000","last_peered":"2024-09-21T00:25:53.264404+0000","last_clean":"2024-09-21T00:25:53.264404+0000","last_became_active":"2024-09-21T00:25:47.232605+0000","last_became_peered":"2024-09-21T00:25:47.232605+0000","last_unstale":"2024-09-21T00:25:53.264404+0000","last_undegraded":"2024-09-21T00:25:53.264404+0000","last_fullsized":"2024-09-21T00:25:53.264404+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:46.193690+0000","last_clean_scrub_stamp":"2024-09-21T00:25:46.193690+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:53:37.081118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.263168+0000","last_change":"2024-09-21T00:25:48.223460+0000","last_active":"2024-09-21T00:25:53.263168+0000","last_peered":"2024-09-21T00:25:53.263168+0000","last_clean":"2024-09-21T00:25:53.263168+0000","last_became_active":"2024-09-21T00:25:48.222174+0000","last_became_peered":"2024-09-21T00:25:48.222174+0000","last_unstale":"2024-09-21T00:25:53.263168+0000","last_undegraded":"2024-09-21T00:25:53.263168+0000","last_fullsized":"2024-09-21T00:25:53.263168+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:07:00.897304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.264073+0000","last_change":"2024-09-21T00:25:52.277698+0000","last_active":"2024-09-21T00:25:53.264073+0000","last_peered":"2024-09-21T00:25:53.264073+0000","last_clean":"2024-09-21T00:25:53.264073+0000","last_became_active":"2024-09-21T00:25:52.277232+0000","last_became_peered":"2024-09-21T00:25:52.277232+0000","last_unstale":"2024-09-21T00:25:53.264073+0000","last_undegraded":"2024-09-21T00:25:53.264073+0000","last_fullsized":"2024-09-21T00:25:53.264073+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:51.238933+0000","last_clean_scrub_stamp":"2024-09-21T00:25:51.238933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:01:14.362816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.826302+0000","last_change":"2024-09-21T00:25:48.240360+0000","last_active":"2024-09-21T00:25:54.826302+0000","last_peered":"2024-09-21T00:25:54.826302+0000","last_clean":"2024-09-21T00:25:54.826302+0000","last_became_active":"2024-09-21T00:25:48.240038+0000","last_became_peered":"2024-09-21T00:25:48.240038+0000","last_unstale":"2024-09-21T00:25:54.826302+0000","last_undegraded":"2024-09-21T00:25:54.826302+0000","last_fullsized":"2024-09-21T00:25:54.826302+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:18:14.964429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.242242+0000","last_change":"2024-09-21T00:25:48.243928+0000","last_active":"2024-09-21T00:25:54.242242+0000","last_peered":"2024-09-21T00:25:54.242242+0000","last_clean":"2024-09-21T00:25:54.242242+0000","last_became_active":"2024-09-21T00:25:48.243641+0000","last_became_peered":"2024-09-21T00:25:48.243641+0000","last_unstale":"2024-09-21T00:25:54.242242+0000","last_undegraded":"2024-09-21T00:25:54.242242+0000","last_fullsized":"2024-09-21T00:25:54.242242+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:01:32.510097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.263436+0000","last_change":"2024-09-21T00:25:48.229253+0000","last_active":"2024-09-21T00:25:53.263436+0000","last_peered":"2024-09-21T00:25:53.263436+0000","last_clean":"2024-09-21T00:25:53.263436+0000","last_became_active":"2024-09-21T00:25:48.228880+0000","last_became_peered":"2024-09-21T00:25:48.228880+0000","last_unstale":"2024-09-21T00:25:53.263436+0000","last_undegraded":"2024-09-21T00:25:53.263436+0000","last_fullsized":"2024-09-21T00:25:53.263436+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:24:47.112012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.826434+0000","last_change":"2024-09-21T00:25:48.232024+0000","last_active":"2024-09-21T00:25:54.826434+0000","last_peered":"2024-09-21T00:25:54.826434+0000","last_clean":"2024-09-21T00:25:54.826434+0000","last_became_active":"2024-09-21T00:25:48.231506+0000","last_became_peered":"2024-09-21T00:25:48.231506+0000","last_unstale":"2024-09-21T00:25:54.826434+0000","last_undegraded":"2024-09-21T00:25:54.826434+0000","last_fullsized":"2024-09-21T00:25:54.826434+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:02:20.463726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.264854+0000","last_change":"2024-09-21T00:25:48.242030+0000","last_active":"2024-09-21T00:25:53.264854+0000","last_peered":"2024-09-21T00:25:53.264854+0000","last_clean":"2024-09-21T00:25:53.264854+0000","last_became_active":"2024-09-21T00:25:48.241710+0000","last_became_peered":"2024-09-21T00:25:48.241710+0000","last_unstale":"2024-09-21T00:25:53.264854+0000","last_undegraded":"2024-09-21T00:25:53.264854+0000","last_fullsized":"2024-09-21T00:25:53.264854+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:00:19.889965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.324542+0000","last_change":"2024-09-21T00:25:48.230301+0000","last_active":"2024-09-21T00:25:54.324542+0000","last_peered":"2024-09-21T00:25:54.324542+0000","last_clean":"2024-09-21T00:25:54.324542+0000","last_became_active":"2024-09-21T00:25:48.229759+0000","last_became_peered":"2024-09-21T00:25:48.229759+0000","last_unstale":"2024-09-21T00:25:54.324542+0000","last_undegraded":"2024-09-21T00:25:54.324542+0000","last_fullsized":"2024-09-21T00:25:54.324542+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:23:28.920719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.827515+0000","last_change":"2024-09-21T00:25:48.236977+0000","last_active":"2024-09-21T00:25:54.827515+0000","last_peered":"2024-09-21T00:25:54.827515+0000","last_clean":"2024-09-21T00:25:54.827515+0000","last_became_active":"2024-09-21T00:25:48.236636+0000","last_became_peered":"2024-09-21T00:25:48.236636+0000","last_unstale":"2024-09-21T00:25:54.827515+0000","last_undegraded":"2024-09-21T00:25:54.827515+0000","last_fullsized":"2024-09-21T00:25:54.827515+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:34:36.110489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.264907+0000","last_change":"2024-09-21T00:25:48.242792+0000","last_active":"2024-09-21T00:25:53.264907+0000","last_peered":"2024-09-21T00:25:53.264907+0000","last_clean":"2024-09-21T00:25:53.264907+0000","last_became_active":"2024-09-21T00:25:48.242512+0000","last_became_peered":"2024-09-21T00:25:48.242512+0000","last_unstale":"2024-09-21T00:25:53.264907+0000","last_undegraded":"2024-09-21T00:25:53.264907+0000","last_fullsized":"2024-09-21T00:25:53.264907+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:58:22.324727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.262616+0000","last_change":"2024-09-21T00:25:48.229627+0000","last_active":"2024-09-21T00:25:53.262616+0000","last_peered":"2024-09-21T00:25:53.262616+0000","last_clean":"2024-09-21T00:25:53.262616+0000","last_became_active":"2024-09-21T00:25:48.229297+0000","last_became_peered":"2024-09-21T00:25:48.229297+0000","last_unstale":"2024-09-21T00:25:53.262616+0000","last_undegraded":"2024-09-21T00:25:53.262616+0000","last_fullsized":"2024-09-21T00:25:53.262616+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:58:22.324727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.264187+0000","last_change":"2024-09-21T00:25:48.243911+0000","last_active":"2024-09-21T00:25:53.264187+0000","last_peered":"2024-09-21T00:25:53.264187+0000","last_clean":"2024-09-21T00:25:53.264187+0000","last_became_active":"2024-09-21T00:25:48.243581+0000","last_became_peered":"2024-09-21T00:25:48.243581+0000","last_unstale":"2024-09-21T00:25:53.264187+0000","last_undegraded":"2024-09-21T00:25:53.264187+0000","last_fullsized":"2024-09-21T00:25:53.264187+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:42:57.617812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.263316+0000","last_change":"2024-09-21T00:25:48.238597+0000","last_active":"2024-09-21T00:25:53.263316+0000","last_peered":"2024-09-21T00:25:53.263316+0000","last_clean":"2024-09-21T00:25:53.263316+0000","last_became_active":"2024-09-21T00:25:48.238265+0000","last_became_peered":"2024-09-21T00:25:48.238265+0000","last_unstale":"2024-09-21T00:25:53.263316+0000","last_undegraded":"2024-09-21T00:25:53.263316+0000","last_fullsized":"2024-09-21T00:25:53.263316+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:10:01.543628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.826223+0000","last_change":"2024-09-21T00:25:48.237837+0000","last_active":"2024-09-21T00:25:54.826223+0000","last_peered":"2024-09-21T00:25:54.826223+0000","last_clean":"2024-09-21T00:25:54.826223+0000","last_became_active":"2024-09-21T00:25:48.237502+0000","last_became_peered":"2024-09-21T00:25:48.237502+0000","last_unstale":"2024-09-21T00:25:54.826223+0000","last_undegraded":"2024-09-21T00:25:54.826223+0000","last_fullsized":"2024-09-21T00:25:54.826223+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:52:28.861327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.827511+0000","last_change":"2024-09-21T00:25:48.241417+0000","last_active":"2024-09-21T00:25:54.827511+0000","last_peered":"2024-09-21T00:25:54.827511+0000","last_clean":"2024-09-21T00:25:54.827511+0000","last_became_active":"2024-09-21T00:25:48.241107+0000","last_became_peered":"2024-09-21T00:25:48.241107+0000","last_unstale":"2024-09-21T00:25:54.827511+0000","last_undegraded":"2024-09-21T00:25:54.827511+0000","last_fullsized":"2024-09-21T00:25:54.827511+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:52:33.265365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.827391+0000","last_change":"2024-09-21T00:25:48.246529+0000","last_active":"2024-09-21T00:25:54.827391+0000","last_peered":"2024-09-21T00:25:54.827391+0000","last_clean":"2024-09-21T00:25:54.827391+0000","last_became_active":"2024-09-21T00:25:48.245952+0000","last_became_peered":"2024-09-21T00:25:48.245952+0000","last_unstale":"2024-09-21T00:25:54.827391+0000","last_undegraded":"2024-09-21T00:25:54.827391+0000","last_fullsized":"2024-09-21T00:25:54.827391+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:16:07.250921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.242392+0000","last_change":"2024-09-21T00:25:48.235870+0000","last_active":"2024-09-21T00:25:54.242392+0000","last_peered":"2024-09-21T00:25:54.242392+0000","last_clean":"2024-09-21T00:25:54.242392+0000","last_became_active":"2024-09-21T00:25:48.235552+0000","last_became_peered":"2024-09-21T00:25:48.235552+0000","last_unstale":"2024-09-21T00:25:54.242392+0000","last_undegraded":"2024-09-21T00:25:54.242392+0000","last_fullsized":"2024-09-21T00:25:54.242392+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:03:13.400804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.264193+0000","last_change":"2024-09-21T00:25:48.238458+0000","last_active":"2024-09-21T00:25:53.264193+0000","last_peered":"2024-09-21T00:25:53.264193+0000","last_clean":"2024-09-21T00:25:53.264193+0000","last_became_active":"2024-09-21T00:25:48.238154+0000","last_became_peered":"2024-09-21T00:25:48.238154+0000","last_unstale":"2024-09-21T00:25:53.264193+0000","last_undegraded":"2024-09-21T00:25:53.264193+0000","last_fullsized":"2024-09-21T00:25:53.264193+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:51:31.831106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.262441+0000","last_change":"2024-09-21T00:25:48.232187+0000","last_active":"2024-09-21T00:25:53.262441+0000","last_peered":"2024-09-21T00:25:53.262441+0000","last_clean":"2024-09-21T00:25:53.262441+0000","last_became_active":"2024-09-21T00:25:48.230817+0000","last_became_peered":"2024-09-21T00:25:48.230817+0000","last_unstale":"2024-09-21T00:25:53.262441+0000","last_undegraded":"2024-09-21T00:25:53.262441+0000","last_fullsized":"2024-09-21T00:25:53.262441+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:32:32.609174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.324087+0000","last_change":"2024-09-21T00:25:48.220646+0000","last_active":"2024-09-21T00:25:54.324087+0000","last_peered":"2024-09-21T00:25:54.324087+0000","last_clean":"2024-09-21T00:25:54.324087+0000","last_became_active":"2024-09-21T00:25:48.220325+0000","last_became_peered":"2024-09-21T00:25:48.220325+0000","last_unstale":"2024-09-21T00:25:54.324087+0000","last_undegraded":"2024-09-21T00:25:54.324087+0000","last_fullsized":"2024-09-21T00:25:54.324087+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:52:34.711918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.826294+0000","last_change":"2024-09-21T00:25:48.228802+0000","last_active":"2024-09-21T00:25:54.826294+0000","last_peered":"2024-09-21T00:25:54.826294+0000","last_clean":"2024-09-21T00:25:54.826294+0000","last_became_active":"2024-09-21T00:25:48.228464+0000","last_became_peered":"2024-09-21T00:25:48.228464+0000","last_unstale":"2024-09-21T00:25:54.826294+0000","last_undegraded":"2024-09-21T00:25:54.826294+0000","last_fullsized":"2024-09-21T00:25:54.826294+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:50:09.155321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.264138+0000","last_change":"2024-09-21T00:25:48.243998+0000","last_active":"2024-09-21T00:25:53.264138+0000","last_peered":"2024-09-21T00:25:53.264138+0000","last_clean":"2024-09-21T00:25:53.264138+0000","last_became_active":"2024-09-21T00:25:48.242668+0000","last_became_peered":"2024-09-21T00:25:48.242668+0000","last_unstale":"2024-09-21T00:25:53.264138+0000","last_undegraded":"2024-09-21T00:25:53.264138+0000","last_fullsized":"2024-09-21T00:25:53.264138+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:44:10.293149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.264479+0000","last_change":"2024-09-21T00:25:48.230464+0000","last_active":"2024-09-21T00:25:53.264479+0000","last_peered":"2024-09-21T00:25:53.264479+0000","last_clean":"2024-09-21T00:25:53.264479+0000","last_became_active":"2024-09-21T00:25:48.229463+0000","last_became_peered":"2024-09-21T00:25:48.229463+0000","last_unstale":"2024-09-21T00:25:53.264479+0000","last_undegraded":"2024-09-21T00:25:53.264479+0000","last_fullsized":"2024-09-21T00:25:53.264479+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:24:05.914453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.827241+0000","last_change":"2024-09-21T00:25:48.233368+0000","last_active":"2024-09-21T00:25:54.827241+0000","last_peered":"2024-09-21T00:25:54.827241+0000","last_clean":"2024-09-21T00:25:54.827241+0000","last_became_active":"2024-09-21T00:25:48.233049+0000","last_became_peered":"2024-09-21T00:25:48.233049+0000","last_unstale":"2024-09-21T00:25:54.827241+0000","last_undegraded":"2024-09-21T00:25:54.827241+0000","last_fullsized":"2024-09-21T00:25:54.827241+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:52:34.711918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.827398+0000","last_change":"2024-09-21T00:25:48.240634+0000","last_active":"2024-09-21T00:25:54.827398+0000","last_peered":"2024-09-21T00:25:54.827398+0000","last_clean":"2024-09-21T00:25:54.827398+0000","last_became_active":"2024-09-21T00:25:48.240302+0000","last_became_peered":"2024-09-21T00:25:48.240302+0000","last_unstale":"2024-09-21T00:25:54.827398+0000","last_undegraded":"2024-09-21T00:25:54.827398+0000","last_fullsized":"2024-09-21T00:25:54.827398+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:30:32.308490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.827582+0000","last_change":"2024-09-21T00:25:48.243347+0000","last_active":"2024-09-21T00:25:54.827582+0000","last_peered":"2024-09-21T00:25:54.827582+0000","last_clean":"2024-09-21T00:25:54.827582+0000","last_became_active":"2024-09-21T00:25:48.243035+0000","last_became_peered":"2024-09-21T00:25:48.243035+0000","last_unstale":"2024-09-21T00:25:54.827582+0000","last_undegraded":"2024-09-21T00:25:54.827582+0000","last_fullsized":"2024-09-21T00:25:54.827582+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:10:10.125591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.263815+0000","last_change":"2024-09-21T00:25:48.235929+0000","last_active":"2024-09-21T00:25:53.263815+0000","last_peered":"2024-09-21T00:25:53.263815+0000","last_clean":"2024-09-21T00:25:53.263815+0000","last_became_active":"2024-09-21T00:25:48.235620+0000","last_became_peered":"2024-09-21T00:25:48.235620+0000","last_unstale":"2024-09-21T00:25:53.263815+0000","last_undegraded":"2024-09-21T00:25:53.263815+0000","last_fullsized":"2024-09-21T00:25:53.263815+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:53:08.415542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.826376+0000","last_change":"2024-09-21T00:25:48.247624+0000","last_active":"2024-09-21T00:25:54.826376+0000","last_peered":"2024-09-21T00:25:54.826376+0000","last_clean":"2024-09-21T00:25:54.826376+0000","last_became_active":"2024-09-21T00:25:48.247318+0000","last_became_peered":"2024-09-21T00:25:48.247318+0000","last_unstale":"2024-09-21T00:25:54.826376+0000","last_undegraded":"2024-09-21T00:25:54.826376+0000","last_fullsized":"2024-09-21T00:25:54.826376+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:34:07.237786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.263196+0000","last_change":"2024-09-21T00:25:48.232468+0000","last_active":"2024-09-21T00:25:53.263196+0000","last_peered":"2024-09-21T00:25:53.263196+0000","last_clean":"2024-09-21T00:25:53.263196+0000","last_became_active":"2024-09-21T00:25:48.231259+0000","last_became_peered":"2024-09-21T00:25:48.231259+0000","last_unstale":"2024-09-21T00:25:53.263196+0000","last_undegraded":"2024-09-21T00:25:53.263196+0000","last_fullsized":"2024-09-21T00:25:53.263196+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:07:41.736005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.826608+0000","last_change":"2024-09-21T00:25:48.246495+0000","last_active":"2024-09-21T00:25:54.826608+0000","last_peered":"2024-09-21T00:25:54.826608+0000","last_clean":"2024-09-21T00:25:54.826608+0000","last_became_active":"2024-09-21T00:25:48.245978+0000","last_became_peered":"2024-09-21T00:25:48.245978+0000","last_unstale":"2024-09-21T00:25:54.826608+0000","last_undegraded":"2024-09-21T00:25:54.826608+0000","last_fullsized":"2024-09-21T00:25:54.826608+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:48:34.879434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:53.263683+0000","last_change":"2024-09-21T00:25:48.237710+0000","last_active":"2024-09-21T00:25:53.263683+0000","last_peered":"2024-09-21T00:25:53.263683+0000","last_clean":"2024-09-21T00:25:53.263683+0000","last_became_active":"2024-09-21T00:25:48.232344+0000","last_became_peered":"2024-09-21T00:25:48.232344+0000","last_unstale":"2024-09-21T00:25:53.263683+0000","last_undegraded":"2024-09-21T00:25:53.263683+0000","last_fullsized":"2024-09-21T00:25:53.263683+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:48:34.879434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.826561+0000","last_change":"2024-09-21T00:25:48.240393+0000","last_active":"2024-09-21T00:25:54.826561+0000","last_peered":"2024-09-21T00:25:54.826561+0000","last_clean":"2024-09-21T00:25:54.826561+0000","last_became_active":"2024-09-21T00:25:48.240086+0000","last_became_peered":"2024-09-21T00:25:48.240086+0000","last_unstale":"2024-09-21T00:25:54.826561+0000","last_undegraded":"2024-09-21T00:25:54.826561+0000","last_fullsized":"2024-09-21T00:25:54.826561+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:12:39.910729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.324667+0000","last_change":"2024-09-21T00:25:48.230210+0000","last_active":"2024-09-21T00:25:54.324667+0000","last_peered":"2024-09-21T00:25:54.324667+0000","last_clean":"2024-09-21T00:25:54.324667+0000","last_became_active":"2024-09-21T00:25:48.229837+0000","last_became_peered":"2024-09-21T00:25:48.229837+0000","last_unstale":"2024-09-21T00:25:54.324667+0000","last_undegraded":"2024-09-21T00:25:54.324667+0000","last_fullsized":"2024-09-21T00:25:54.324667+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:23:42.611364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T00:25:54.827462+0000","last_change":"2024-09-21T00:25:48.242566+0000","last_active":"2024-09-21T00:25:54.827462+0000","last_peered":"2024-09-21T00:25:54.827462+0000","last_clean":"2024-09-21T00:25:54.827462+0000","last_became_active":"2024-09-21T00:25:48.242265+0000","last_became_peered":"2024-09-21T00:25:48.242265+0000","last_unstale":"2024-09-21T00:25:54.827462+0000","last_undegraded":"2024-09-21T00:25:54.827462+0000","last_fullsized":"2024-09-21T00:25:54.827462+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:47.201099+0000","last_clean_scrub_stamp":"2024-09-21T00:25:47.201099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:49:57.219481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509449,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27880,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343960,"statfs":{"total":96636764160,"available":96608215040,"internally_reserved":0,"allocated":868352,"data_stored":697130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509450,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":282624,"data_stored":110022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509449,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":274432,"data_stored":106270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509448,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":282624,"data_stored":106306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509450,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":282624,"data_stored":110022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509450,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":270336,"data_stored":106180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509450,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27888,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343952,"statfs":{"total":96636764160,"available":96608206848,"internally_reserved":0,"allocated":876544,"data_stored":697186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574857,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":278528,"data_stored":106340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T00:26:07.524 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T00:26:07.524 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-21T00:26:07.525 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-21T00:26:07.525 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-21T00:26:07.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:07.670+0000 7fc4e2d2b640 1 -- 172.21.15.112:0/941755737 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4dc0cbea0 msgr2=0x7fc4dc06cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:07.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:07.670+0000 7fc4e2d2b640 1 --2- 172.21.15.112:0/941755737 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4dc0cbea0 0x7fc4dc06cde0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc4dc073950 tx=0x7fc4d402a980 comp rx=0 tx=0).stop 2024-09-21T00:26:07.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:07.670+0000 7fc4e2d2b640 1 -- 172.21.15.112:0/941755737 shutdown_connections 2024-09-21T00:26:07.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:07.670+0000 7fc4e2d2b640 1 --2- 172.21.15.112:0/941755737 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4dc0cbea0 0x7fc4dc06cde0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:07.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:07.670+0000 7fc4e2d2b640 1 --2- 172.21.15.112:0/941755737 >> v2:172.21.15.187:3300/0 conn(0x7fc4dc070f10 0x7fc4dc071630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:07.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:07.670+0000 7fc4e2d2b640 1 -- 172.21.15.112:0/941755737 >> 172.21.15.112:0/941755737 conn(0x7fc4dc0d1140 msgr2=0x7fc4dc0d1550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:07.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:07.671+0000 7fc4e2d2b640 1 -- 172.21.15.112:0/941755737 shutdown_connections 2024-09-21T00:26:07.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:07.671+0000 7fc4e2d2b640 1 -- 172.21.15.112:0/941755737 wait complete. 2024-09-21T00:26:07.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:07.672+0000 7fc4e2d2b640 1 Processor -- start 2024-09-21T00:26:07.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:07.672+0000 7fc4e2d2b640 1 -- start start 2024-09-21T00:26:07.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:07.674+0000 7fc4e2d2b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4dc070f10 0x7fc4dc0b46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:07.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:07.674+0000 7fc4e2d2b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc4dc074a70 0x7fc4dc15b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:07.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:07.674+0000 7fc4e2d2b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc4dc15b860 0x7fc4dc155780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:07.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:07.674+0000 7fc4e2d2b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc4dc06d890 con 0x7fc4dc070f10 2024-09-21T00:26:07.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:07.674+0000 7fc4e2d2b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc4dc06d710 con 0x7fc4dc074a70 2024-09-21T00:26:07.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:07.674+0000 7fc4e2d2b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc4dc06da10 con 0x7fc4dc15b860 2024-09-21T00:26:07.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:07.674+0000 7fc4e0ac7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4dc070f10 0x7fc4dc0b46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:07.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:07.674+0000 7fc4e0ac7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4dc070f10 0x7fc4dc0b46d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60114/0 (socket says 172.21.15.112:60114) 2024-09-21T00:26:07.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:07.674+0000 7fc4e0ac7640 1 -- 172.21.15.112:0/1660536219 learned_addr learned my addr 172.21.15.112:0/1660536219 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:26:07.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:07.674+0000 7fc4d3fff640 1 --2- 172.21.15.112:0/1660536219 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc4dc074a70 0x7fc4dc15b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:07.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:07.674+0000 7fc4e12c8640 1 --2- 172.21.15.112:0/1660536219 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc4dc15b860 0x7fc4dc155780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:07.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:07.674+0000 7fc4e0ac7640 1 -- 172.21.15.112:0/1660536219 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc4dc15b860 msgr2=0x7fc4dc155780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:07.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:07.674+0000 7fc4e0ac7640 1 --2- 172.21.15.112:0/1660536219 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc4dc15b860 0x7fc4dc155780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:07.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:07.674+0000 7fc4e0ac7640 1 -- 172.21.15.112:0/1660536219 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc4dc074a70 msgr2=0x7fc4dc15b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:07.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:07.674+0000 7fc4e0ac7640 1 --2- 172.21.15.112:0/1660536219 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc4dc074a70 0x7fc4dc15b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:07.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:07.674+0000 7fc4e0ac7640 1 -- 172.21.15.112:0/1660536219 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4dc0b4c10 con 0x7fc4dc070f10 2024-09-21T00:26:07.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:07.674+0000 7fc4e0ac7640 1 --2- 172.21.15.112:0/1660536219 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4dc070f10 0x7fc4dc0b46d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc4d800c970 tx=0x7fc4d800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:07.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:07.674+0000 7fc4d17fa640 1 -- 172.21.15.112:0/1660536219 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d8007bb0 con 0x7fc4dc070f10 2024-09-21T00:26:07.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:07.675+0000 7fc4d17fa640 1 -- 172.21.15.112:0/1660536219 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc4d8005020 con 0x7fc4dc070f10 2024-09-21T00:26:07.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:07.675+0000 7fc4d17fa640 1 -- 172.21.15.112:0/1660536219 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d8007e60 con 0x7fc4dc070f10 2024-09-21T00:26:07.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:07.675+0000 7fc4e2d2b640 1 -- 172.21.15.112:0/1660536219 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4dc1546e0 con 0x7fc4dc070f10 2024-09-21T00:26:07.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:07.675+0000 7fc4e2d2b640 1 -- 172.21.15.112:0/1660536219 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4dc154ca0 con 0x7fc4dc070f10 2024-09-21T00:26:07.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:07.676+0000 7fc4e2d2b640 1 -- 172.21.15.112:0/1660536219 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4dc071630 con 0x7fc4dc070f10 2024-09-21T00:26:07.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:07.676+0000 7fc4d17fa640 1 -- 172.21.15.112:0/1660536219 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc4d8005470 con 0x7fc4dc070f10 2024-09-21T00:26:07.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:07.677+0000 7fc4d17fa640 1 --2- 172.21.15.112:0/1660536219 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fc4c8078b90 0x7fc4c807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:07.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:07.677+0000 7fc4d17fa640 1 -- 172.21.15.112:0/1660536219 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fc4d809b420 con 0x7fc4dc070f10 2024-09-21T00:26:07.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:07.677+0000 7fc4d3fff640 1 --2- 172.21.15.112:0/1660536219 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fc4c8078b90 0x7fc4c807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:07.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:07.678+0000 7fc4d3fff640 1 --2- 172.21.15.112:0/1660536219 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fc4c8078b90 0x7fc4c807b0a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc4dc0c7a70 tx=0x7fc4cc009060 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:07.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:07.682+0000 7fc4d17fa640 1 -- 172.21.15.112:0/1660536219 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc4d8014030 con 0x7fc4dc070f10 2024-09-21T00:26:07.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:07.897+0000 7fc4e2d2b640 1 -- 172.21.15.112:0/1660536219 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fc4dc150dd0 con 0x7fc4dc070f10 2024-09-21T00:26:07.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:07.898+0000 7fc4d17fa640 1 -- 172.21.15.112:0/1660536219 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fc4d8062f60 con 0x7fc4dc070f10 2024-09-21T00:26:07.899 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:26:07.899 INFO:teuthology.orchestra.run.smithi112.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-21T00:26:07.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:07.900+0000 7fc4e2d2b640 1 -- 172.21.15.112:0/1660536219 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fc4c8078b90 msgr2=0x7fc4c807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:07.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:07.900+0000 7fc4e2d2b640 1 --2- 172.21.15.112:0/1660536219 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fc4c8078b90 0x7fc4c807b0a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc4dc0c7a70 tx=0x7fc4cc009060 comp rx=0 tx=0).stop 2024-09-21T00:26:07.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:07.900+0000 7fc4e2d2b640 1 -- 172.21.15.112:0/1660536219 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4dc070f10 msgr2=0x7fc4dc0b46d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:07.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:07.900+0000 7fc4e2d2b640 1 --2- 172.21.15.112:0/1660536219 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4dc070f10 0x7fc4dc0b46d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc4d800c970 tx=0x7fc4d800ce30 comp rx=0 tx=0).stop 2024-09-21T00:26:07.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:07.900+0000 7fc4e2d2b640 1 -- 172.21.15.112:0/1660536219 shutdown_connections 2024-09-21T00:26:07.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:07.900+0000 7fc4e2d2b640 1 --2- 172.21.15.112:0/1660536219 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc4dc15b860 0x7fc4dc155780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:07.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:07.900+0000 7fc4e2d2b640 1 --2- 172.21.15.112:0/1660536219 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc4dc074a70 0x7fc4dc15b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:07.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:07.900+0000 7fc4e2d2b640 1 --2- 172.21.15.112:0/1660536219 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fc4c8078b90 0x7fc4c807b0a0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:07.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:07.900+0000 7fc4e2d2b640 1 --2- 172.21.15.112:0/1660536219 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4dc070f10 0x7fc4dc0b46d0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:07.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:07.900+0000 7fc4e2d2b640 1 -- 172.21.15.112:0/1660536219 >> 172.21.15.112:0/1660536219 conn(0x7fc4dc0d1140 msgr2=0x7fc4dc0d1550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:07.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:07.900+0000 7fc4e2d2b640 1 -- 172.21.15.112:0/1660536219 shutdown_connections 2024-09-21T00:26:07.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:07.900+0000 7fc4e2d2b640 1 -- 172.21.15.112:0/1660536219 wait complete. 2024-09-21T00:26:07.912 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-21T00:26:07.912 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T00:26:08.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.023+0000 7fdfd9fa1640 1 -- 172.21.15.112:0/1949632836 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdfd40d0b40 msgr2=0x7fdfd406e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:08.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.023+0000 7fdfd9fa1640 1 --2- 172.21.15.112:0/1949632836 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdfd40d0b40 0x7fdfd406e6f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fdfd40cd2c0 tx=0x7fdfc402a5f0 comp rx=0 tx=0).stop 2024-09-21T00:26:08.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.023+0000 7fdfd9fa1640 1 -- 172.21.15.112:0/1949632836 shutdown_connections 2024-09-21T00:26:08.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.023+0000 7fdfd9fa1640 1 --2- 172.21.15.112:0/1949632836 >> v2:172.21.15.187:3300/0 conn(0x7fdfd406ec30 0x7fdfd40b3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:08.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.023+0000 7fdfd9fa1640 1 --2- 172.21.15.112:0/1949632836 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdfd40d0b40 0x7fdfd406e6f0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:08.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.023+0000 7fdfd9fa1640 1 -- 172.21.15.112:0/1949632836 >> 172.21.15.112:0/1949632836 conn(0x7fdfd40b2cc0 msgr2=0x7fdfd40b30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:08.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.023+0000 7fdfd9fa1640 1 -- 172.21.15.112:0/1949632836 shutdown_connections 2024-09-21T00:26:08.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.023+0000 7fdfd9fa1640 1 -- 172.21.15.112:0/1949632836 wait complete. 2024-09-21T00:26:08.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.025+0000 7fdfd9fa1640 1 Processor -- start 2024-09-21T00:26:08.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.025+0000 7fdfd9fa1640 1 -- start start 2024-09-21T00:26:08.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.026+0000 7fdfd9fa1640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdfd406ec30 0x7fdfd414e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:08.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.026+0000 7fdfd9fa1640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdfd40b4500 0x7fdfd40c85d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:08.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.026+0000 7fdfd9fa1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdfd40d0b40 0x7fdfd40c8b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:08.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.026+0000 7fdfd9fa1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fdfd415b750 con 0x7fdfd40d0b40 2024-09-21T00:26:08.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.026+0000 7fdfd9fa1640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fdfd415b5d0 con 0x7fdfd406ec30 2024-09-21T00:26:08.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.026+0000 7fdfd9fa1640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fdfd415b8d0 con 0x7fdfd40b4500 2024-09-21T00:26:08.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.027+0000 7fdfd3fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdfd40d0b40 0x7fdfd40c8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:08.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.027+0000 7fdfd3fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdfd40d0b40 0x7fdfd40c8b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60126/0 (socket says 172.21.15.112:60126) 2024-09-21T00:26:08.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.027+0000 7fdfd3fff640 1 -- 172.21.15.112:0/1924441344 learned_addr learned my addr 172.21.15.112:0/1924441344 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:26:08.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.027+0000 7fdfd37fe640 1 --2- 172.21.15.112:0/1924441344 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdfd406ec30 0x7fdfd414e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:08.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.027+0000 7fdfd2ffd640 1 --2- 172.21.15.112:0/1924441344 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdfd40b4500 0x7fdfd40c85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:08.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.027+0000 7fdfd3fff640 1 -- 172.21.15.112:0/1924441344 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdfd40b4500 msgr2=0x7fdfd40c85d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:08.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.027+0000 7fdfd3fff640 1 --2- 172.21.15.112:0/1924441344 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdfd40b4500 0x7fdfd40c85d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:08.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.027+0000 7fdfd3fff640 1 -- 172.21.15.112:0/1924441344 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdfd406ec30 msgr2=0x7fdfd414e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:08.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.027+0000 7fdfd3fff640 1 --2- 172.21.15.112:0/1924441344 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdfd406ec30 0x7fdfd414e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:08.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.027+0000 7fdfd3fff640 1 -- 172.21.15.112:0/1924441344 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfd40cba60 con 0x7fdfd40d0b40 2024-09-21T00:26:08.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.027+0000 7fdfd3fff640 1 --2- 172.21.15.112:0/1924441344 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdfd40d0b40 0x7fdfd40c8b10 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc00c6e0 tx=0x7fdfcc00cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:08.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.027+0000 7fdfbbfff640 1 -- 172.21.15.112:0/1924441344 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfcc003b70 con 0x7fdfd40d0b40 2024-09-21T00:26:08.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.027+0000 7fdfbbfff640 1 -- 172.21.15.112:0/1924441344 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdfcc003d30 con 0x7fdfd40d0b40 2024-09-21T00:26:08.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.027+0000 7fdfd9fa1640 1 -- 172.21.15.112:0/1924441344 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfd40a9440 con 0x7fdfd40d0b40 2024-09-21T00:26:08.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.027+0000 7fdfbbfff640 1 -- 172.21.15.112:0/1924441344 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfcc005330 con 0x7fdfd40d0b40 2024-09-21T00:26:08.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.027+0000 7fdfd9fa1640 1 -- 172.21.15.112:0/1924441344 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfd40a9a00 con 0x7fdfd40d0b40 2024-09-21T00:26:08.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.028+0000 7fdfd9fa1640 1 -- 172.21.15.112:0/1924441344 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfd415bd80 con 0x7fdfd40d0b40 2024-09-21T00:26:08.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.029+0000 7fdfbbfff640 1 -- 172.21.15.112:0/1924441344 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fdfcc0054d0 con 0x7fdfd40d0b40 2024-09-21T00:26:08.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.029+0000 7fdfbbfff640 1 --2- 172.21.15.112:0/1924441344 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fdfc0078b90 0x7fdfc007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:08.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.030+0000 7fdfbbfff640 1 -- 172.21.15.112:0/1924441344 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fdfcc09ad00 con 0x7fdfd40d0b40 2024-09-21T00:26:08.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.030+0000 7fdfd37fe640 1 --2- 172.21.15.112:0/1924441344 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fdfc0078b90 0x7fdfc007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:08.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.030+0000 7fdfd37fe640 1 --2- 172.21.15.112:0/1924441344 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fdfc0078b90 0x7fdfc007b0a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fdfc4033000 tx=0x7fdfc4004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:08.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.034+0000 7fdfbbfff640 1 -- 172.21.15.112:0/1924441344 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fdfcc0627b0 con 0x7fdfd40d0b40 2024-09-21T00:26:08.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.245+0000 7fdfd9fa1640 1 -- 172.21.15.112:0/1924441344 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdfd40cb7f0 con 0x7fdfd40d0b40 2024-09-21T00:26:08.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.246+0000 7fdfbbfff640 1 -- 172.21.15.112:0/1924441344 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5228 (secure 0 0 0) 0x7fdfcc0671e0 con 0x7fdfd40d0b40 2024-09-21T00:26:08.247 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:26:08.247 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":9,"btime":"2024-09-21T00:26:01:294823+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4316,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6837/2271985854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6835","nonce":2271985854},{"type":"v1","addr":"172.21.15.187:6837","nonce":2271985854}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4413,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/200348130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":200348130},{"type":"v1","addr":"172.21.15.112:6833","nonce":200348130}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4416,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/3821201561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3821201561},{"type":"v1","addr":"172.21.15.112:6835","nonce":3821201561}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T00:25:47.578960+0000","modified":"2024-09-21T00:26:00.650942+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4425},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4425":{"gid":4425,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6836/2921304789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":2921304789},{"type":"v1","addr":"172.21.15.187:6836","nonce":2921304789}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4425,"qdb_cluster":[4425]},"id":1}]} 2024-09-21T00:26:08.247 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 9 2024-09-21T00:26:08.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.248+0000 7fdfd9fa1640 1 -- 172.21.15.112:0/1924441344 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fdfc0078b90 msgr2=0x7fdfc007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:08.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.248+0000 7fdfd9fa1640 1 --2- 172.21.15.112:0/1924441344 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fdfc0078b90 0x7fdfc007b0a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fdfc4033000 tx=0x7fdfc4004730 comp rx=0 tx=0).stop 2024-09-21T00:26:08.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.248+0000 7fdfd9fa1640 1 -- 172.21.15.112:0/1924441344 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdfd40d0b40 msgr2=0x7fdfd40c8b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:08.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.248+0000 7fdfd9fa1640 1 --2- 172.21.15.112:0/1924441344 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdfd40d0b40 0x7fdfd40c8b10 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc00c6e0 tx=0x7fdfcc00cba0 comp rx=0 tx=0).stop 2024-09-21T00:26:08.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.248+0000 7fdfd9fa1640 1 -- 172.21.15.112:0/1924441344 shutdown_connections 2024-09-21T00:26:08.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.248+0000 7fdfd9fa1640 1 --2- 172.21.15.112:0/1924441344 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fdfc0078b90 0x7fdfc007b0a0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:08.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.248+0000 7fdfd9fa1640 1 --2- 172.21.15.112:0/1924441344 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdfd40d0b40 0x7fdfd40c8b10 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:08.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.248+0000 7fdfd9fa1640 1 --2- 172.21.15.112:0/1924441344 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdfd40b4500 0x7fdfd40c85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:08.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.248+0000 7fdfd9fa1640 1 --2- 172.21.15.112:0/1924441344 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdfd406ec30 0x7fdfd414e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:08.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.248+0000 7fdfd9fa1640 1 -- 172.21.15.112:0/1924441344 >> 172.21.15.112:0/1924441344 conn(0x7fdfd40b2cc0 msgr2=0x7fdfd40cf0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:08.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.248+0000 7fdfd9fa1640 1 -- 172.21.15.112:0/1924441344 shutdown_connections 2024-09-21T00:26:08.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.248+0000 7fdfd9fa1640 1 -- 172.21.15.112:0/1924441344 wait complete. 2024-09-21T00:26:08.260 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T00:26:08.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.374+0000 7f1d12838640 1 -- 172.21.15.112:0/1758925853 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1d0c162280 msgr2=0x7f1d0c162690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:08.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.374+0000 7f1d12838640 1 --2- 172.21.15.112:0/1758925853 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1d0c162280 0x7f1d0c162690 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1d00009830 tx=0x7f1d0002a510 comp rx=0 tx=0).stop 2024-09-21T00:26:08.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.374+0000 7f1d12838640 1 -- 172.21.15.112:0/1758925853 shutdown_connections 2024-09-21T00:26:08.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.374+0000 7f1d12838640 1 --2- 172.21.15.112:0/1758925853 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1d0c162280 0x7f1d0c162690 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:08.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.374+0000 7f1d12838640 1 --2- 172.21.15.112:0/1758925853 >> v2:172.21.15.187:3300/0 conn(0x7f1d0c162f00 0x7f1d0c0d24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:08.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.374+0000 7f1d12838640 1 -- 172.21.15.112:0/1758925853 >> 172.21.15.112:0/1758925853 conn(0x7f1d0c0b6b20 msgr2=0x7f1d0c0d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:08.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.375+0000 7f1d12838640 1 -- 172.21.15.112:0/1758925853 shutdown_connections 2024-09-21T00:26:08.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.375+0000 7f1d12838640 1 -- 172.21.15.112:0/1758925853 wait complete. 2024-09-21T00:26:08.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.376+0000 7f1d12838640 1 Processor -- start 2024-09-21T00:26:08.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.376+0000 7f1d12838640 1 -- start start 2024-09-21T00:26:08.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.378+0000 7f1d12838640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1d0c0b2e40 0x7f1d0c1f4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:08.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.378+0000 7f1d12838640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1d0c162280 0x7f1d0c1f5310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:08.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.378+0000 7f1d12838640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d0c0b16e0 0x7f1d0c14da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:08.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.378+0000 7f1d12838640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1d0c15e630 con 0x7f1d0c0b16e0 2024-09-21T00:26:08.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.378+0000 7f1d12838640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f1d0c15e4b0 con 0x7f1d0c0b2e40 2024-09-21T00:26:08.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.378+0000 7f1d12838640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f1d0c15e7b0 con 0x7f1d0c162280 2024-09-21T00:26:08.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.378+0000 7f1d10dd5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d0c0b16e0 0x7f1d0c14da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:08.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.378+0000 7f1d10dd5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d0c0b16e0 0x7f1d0c14da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60140/0 (socket says 172.21.15.112:60140) 2024-09-21T00:26:08.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.378+0000 7f1d10dd5640 1 -- 172.21.15.112:0/1157834755 learned_addr learned my addr 172.21.15.112:0/1157834755 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:26:08.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.378+0000 7f1d0bfff640 1 --2- 172.21.15.112:0/1157834755 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1d0c0b2e40 0x7f1d0c1f4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:08.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.378+0000 7f1d0b7fe640 1 --2- 172.21.15.112:0/1157834755 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1d0c162280 0x7f1d0c1f5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:08.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.378+0000 7f1d10dd5640 1 -- 172.21.15.112:0/1157834755 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1d0c162280 msgr2=0x7f1d0c1f5310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:08.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.378+0000 7f1d10dd5640 1 --2- 172.21.15.112:0/1157834755 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1d0c162280 0x7f1d0c1f5310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:08.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.378+0000 7f1d10dd5640 1 -- 172.21.15.112:0/1157834755 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1d0c0b2e40 msgr2=0x7f1d0c1f4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:08.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.378+0000 7f1d10dd5640 1 --2- 172.21.15.112:0/1157834755 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1d0c0b2e40 0x7f1d0c1f4dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:08.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.378+0000 7f1d10dd5640 1 -- 172.21.15.112:0/1157834755 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d0c0b1ef0 con 0x7f1d0c0b16e0 2024-09-21T00:26:08.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.378+0000 7f1d10dd5640 1 --2- 172.21.15.112:0/1157834755 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d0c0b16e0 0x7f1d0c14da60 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f1d0400caa0 tx=0x7f1d0400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:08.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.379+0000 7f1d08ff9640 1 -- 172.21.15.112:0/1157834755 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d04016020 con 0x7f1d0c0b16e0 2024-09-21T00:26:08.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.379+0000 7f1d08ff9640 1 -- 172.21.15.112:0/1157834755 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1d0400acf0 con 0x7f1d0c0b16e0 2024-09-21T00:26:08.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.379+0000 7f1d08ff9640 1 -- 172.21.15.112:0/1157834755 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d04012680 con 0x7f1d0c0b16e0 2024-09-21T00:26:08.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.379+0000 7f1d12838640 1 -- 172.21.15.112:0/1157834755 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d0c0b21d0 con 0x7f1d0c0b16e0 2024-09-21T00:26:08.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.379+0000 7f1d12838640 1 -- 172.21.15.112:0/1157834755 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d0c0b2790 con 0x7f1d0c0b16e0 2024-09-21T00:26:08.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.380+0000 7f1d12838640 1 -- 172.21.15.112:0/1157834755 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d0c162f00 con 0x7f1d0c0b16e0 2024-09-21T00:26:08.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.380+0000 7f1d08ff9640 1 -- 172.21.15.112:0/1157834755 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1d04019020 con 0x7f1d0c0b16e0 2024-09-21T00:26:08.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.381+0000 7f1d08ff9640 1 --2- 172.21.15.112:0/1157834755 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1cf8078b90 0x7f1cf807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:08.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.381+0000 7f1d08ff9640 1 -- 172.21.15.112:0/1157834755 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f1d0409b520 con 0x7f1d0c0b16e0 2024-09-21T00:26:08.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.381+0000 7f1d0bfff640 1 --2- 172.21.15.112:0/1157834755 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1cf8078b90 0x7f1cf807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:08.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.382+0000 7f1d0bfff640 1 --2- 172.21.15.112:0/1157834755 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1cf8078b90 0x7f1cf807b0a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1cfc0095f0 tx=0x7f1cfc005f90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:08.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.385+0000 7f1d08ff9640 1 -- 172.21.15.112:0/1157834755 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1d04020020 con 0x7f1d0c0b16e0 2024-09-21T00:26:08.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.600+0000 7f1d12838640 1 -- 172.21.15.112:0/1157834755 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1d0c0b1c80 con 0x7f1d0c0b16e0 2024-09-21T00:26:08.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.600+0000 7f1d08ff9640 1 -- 172.21.15.112:0/1157834755 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5228 (secure 0 0 0) 0x7f1d0c162bd0 con 0x7f1d0c0b16e0 2024-09-21T00:26:08.601 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:26:08.601 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":9,"btime":"2024-09-21T00:26:01:294823+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4316,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6837/2271985854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6835","nonce":2271985854},{"type":"v1","addr":"172.21.15.187:6837","nonce":2271985854}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4413,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/200348130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":200348130},{"type":"v1","addr":"172.21.15.112:6833","nonce":200348130}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4416,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/3821201561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3821201561},{"type":"v1","addr":"172.21.15.112:6835","nonce":3821201561}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T00:25:47.578960+0000","modified":"2024-09-21T00:26:00.650942+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4425},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4425":{"gid":4425,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6836/2921304789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":2921304789},{"type":"v1","addr":"172.21.15.187:6836","nonce":2921304789}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4425,"qdb_cluster":[4425]},"id":1}]} 2024-09-21T00:26:08.601 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 9 2024-09-21T00:26:08.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.602+0000 7f1d12838640 1 -- 172.21.15.112:0/1157834755 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1cf8078b90 msgr2=0x7f1cf807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:08.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.602+0000 7f1d12838640 1 --2- 172.21.15.112:0/1157834755 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1cf8078b90 0x7f1cf807b0a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1cfc0095f0 tx=0x7f1cfc005f90 comp rx=0 tx=0).stop 2024-09-21T00:26:08.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.602+0000 7f1d12838640 1 -- 172.21.15.112:0/1157834755 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d0c0b16e0 msgr2=0x7f1d0c14da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:08.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.602+0000 7f1d12838640 1 --2- 172.21.15.112:0/1157834755 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d0c0b16e0 0x7f1d0c14da60 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f1d0400caa0 tx=0x7f1d0400cf60 comp rx=0 tx=0).stop 2024-09-21T00:26:08.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.602+0000 7f1d12838640 1 -- 172.21.15.112:0/1157834755 shutdown_connections 2024-09-21T00:26:08.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.602+0000 7f1d12838640 1 --2- 172.21.15.112:0/1157834755 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1cf8078b90 0x7f1cf807b0a0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:08.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.602+0000 7f1d12838640 1 --2- 172.21.15.112:0/1157834755 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d0c0b16e0 0x7f1d0c14da60 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:08.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.602+0000 7f1d12838640 1 --2- 172.21.15.112:0/1157834755 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1d0c162280 0x7f1d0c1f5310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:08.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.602+0000 7f1d12838640 1 --2- 172.21.15.112:0/1157834755 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1d0c0b2e40 0x7f1d0c1f4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:08.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.602+0000 7f1d12838640 1 -- 172.21.15.112:0/1157834755 >> 172.21.15.112:0/1157834755 conn(0x7f1d0c0b6b20 msgr2=0x7f1d0c15dac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:08.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.602+0000 7f1d12838640 1 -- 172.21.15.112:0/1157834755 shutdown_connections 2024-09-21T00:26:08.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.602+0000 7f1d12838640 1 -- 172.21.15.112:0/1157834755 wait complete. 2024-09-21T00:26:08.614 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T00:26:08.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.726+0000 7fa4f5169640 1 -- 172.21.15.112:0/2644963246 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa4f0162280 msgr2=0x7fa4f0162690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:08.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.726+0000 7fa4f5169640 1 --2- 172.21.15.112:0/2644963246 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa4f0162280 0x7fa4f0162690 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa4d8009860 tx=0x7fa4d802a530 comp rx=0 tx=0).stop 2024-09-21T00:26:08.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.726+0000 7fa4f5169640 1 -- 172.21.15.112:0/2644963246 shutdown_connections 2024-09-21T00:26:08.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.726+0000 7fa4f5169640 1 --2- 172.21.15.112:0/2644963246 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa4f0162280 0x7fa4f0162690 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:08.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.726+0000 7fa4f5169640 1 -- 172.21.15.112:0/2644963246 >> 172.21.15.112:0/2644963246 conn(0x7fa4f00b6b20 msgr2=0x7fa4f00d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:08.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.726+0000 7fa4f5169640 1 -- 172.21.15.112:0/2644963246 shutdown_connections 2024-09-21T00:26:08.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.726+0000 7fa4f5169640 1 -- 172.21.15.112:0/2644963246 wait complete. 2024-09-21T00:26:08.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.728+0000 7fa4f5169640 1 Processor -- start 2024-09-21T00:26:08.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.728+0000 7fa4f5169640 1 -- start start 2024-09-21T00:26:08.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.730+0000 7fa4f5169640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa4f00b2e40 0x7fa4f01f50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:08.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.730+0000 7fa4f5169640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa4f0162280 0x7fa4f01f55e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:08.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.730+0000 7fa4f5169640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa4f014bf80 0x7fa4f014c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:08.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.730+0000 7fa4f5169640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f015f640 con 0x7fa4f0162280 2024-09-21T00:26:08.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.730+0000 7fa4f5169640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f015f4c0 con 0x7fa4f00b2e40 2024-09-21T00:26:08.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.730+0000 7fa4f5169640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa4f015f7c0 con 0x7fa4f014bf80 2024-09-21T00:26:08.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.730+0000 7fa4ee59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa4f0162280 0x7fa4f01f55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:08.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.730+0000 7fa4ee59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa4f0162280 0x7fa4f01f55e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60144/0 (socket says 172.21.15.112:60144) 2024-09-21T00:26:08.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.730+0000 7fa4ee59c640 1 -- 172.21.15.112:0/1428999287 learned_addr learned my addr 172.21.15.112:0/1428999287 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:26:08.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.730+0000 7fa4ef59e640 1 --2- 172.21.15.112:0/1428999287 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa4f014bf80 0x7fa4f014c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:08.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.730+0000 7fa4eed9d640 1 --2- 172.21.15.112:0/1428999287 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa4f00b2e40 0x7fa4f01f50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:08.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.730+0000 7fa4ee59c640 1 -- 172.21.15.112:0/1428999287 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa4f014bf80 msgr2=0x7fa4f014c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:08.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.730+0000 7fa4ee59c640 1 --2- 172.21.15.112:0/1428999287 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa4f014bf80 0x7fa4f014c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:08.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.730+0000 7fa4ee59c640 1 -- 172.21.15.112:0/1428999287 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa4f00b2e40 msgr2=0x7fa4f01f50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:08.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.730+0000 7fa4ee59c640 1 --2- 172.21.15.112:0/1428999287 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa4f00b2e40 0x7fa4f01f50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:08.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.730+0000 7fa4ee59c640 1 -- 172.21.15.112:0/1428999287 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4f00abfb0 con 0x7fa4f0162280 2024-09-21T00:26:08.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.730+0000 7fa4eed9d640 1 --2- 172.21.15.112:0/1428999287 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa4f00b2e40 0x7fa4f01f50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:26:08.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.730+0000 7fa4ee59c640 1 --2- 172.21.15.112:0/1428999287 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa4f0162280 0x7fa4f01f55e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa4d8009980 tx=0x7fa4d8003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:08.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.730+0000 7fa4e77fe640 1 -- 172.21.15.112:0/1428999287 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d8005c90 con 0x7fa4f0162280 2024-09-21T00:26:08.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.730+0000 7fa4e77fe640 1 -- 172.21.15.112:0/1428999287 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa4d8005310 con 0x7fa4f0162280 2024-09-21T00:26:08.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.730+0000 7fa4f5169640 1 -- 172.21.15.112:0/1428999287 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4f00ac290 con 0x7fa4f0162280 2024-09-21T00:26:08.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.730+0000 7fa4e77fe640 1 -- 172.21.15.112:0/1428999287 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d80047a0 con 0x7fa4f0162280 2024-09-21T00:26:08.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.731+0000 7fa4f5169640 1 -- 172.21.15.112:0/1428999287 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4f00ac850 con 0x7fa4f0162280 2024-09-21T00:26:08.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.731+0000 7fa4f5169640 1 -- 172.21.15.112:0/1428999287 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4f0050f60 con 0x7fa4f0162280 2024-09-21T00:26:08.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.732+0000 7fa4e77fe640 1 -- 172.21.15.112:0/1428999287 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa4d8004940 con 0x7fa4f0162280 2024-09-21T00:26:08.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.733+0000 7fa4e77fe640 1 --2- 172.21.15.112:0/1428999287 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fa4d0078b90 0x7fa4d007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:08.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.733+0000 7fa4e77fe640 1 -- 172.21.15.112:0/1428999287 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fa4d80bc000 con 0x7fa4f0162280 2024-09-21T00:26:08.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.733+0000 7fa4eed9d640 1 --2- 172.21.15.112:0/1428999287 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fa4d0078b90 0x7fa4d007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:08.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.733+0000 7fa4eed9d640 1 --2- 172.21.15.112:0/1428999287 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fa4d0078b90 0x7fa4d007b0a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa4e0001460 tx=0x7fa4e0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:08.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.737+0000 7fa4e77fe640 1 -- 172.21.15.112:0/1428999287 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa4f0050f60 con 0x7fa4f0162280 2024-09-21T00:26:08.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.915+0000 7fa4f5169640 1 -- 172.21.15.112:0/1428999287 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa4f014fa00 con 0x7fa4f0162280 2024-09-21T00:26:08.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.916+0000 7fa4e77fe640 1 -- 172.21.15.112:0/1428999287 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18622 (secure 0 0 0) 0x7fa4d803f030 con 0x7fa4f0162280 2024-09-21T00:26:08.917 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:26:08.917 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":26,"fsid":"8b34692b-dc65-49b2-a08f-26f707b96ac5","created":"2024-09-21T00:25:22.788490+0000","modified":"2024-09-21T00:25:52.623213+0000","last_up_change":"2024-09-21T00:25:36.161835+0000","last_in_change":"2024-09-21T00:25:26.346500+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T00:25:37.105006+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T00:25:45.413895+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T00:25:46.532124+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T00:25:50.244145+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9c575086-67d2-4de2-b01b-d44dcc21d604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6809","nonce":755760621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6811","nonce":755760621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6815","nonce":755760621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6813","nonce":755760621}]},"public_addr":"172.21.15.112:6809/755760621","cluster_addr":"172.21.15.112:6811/755760621","heartbeat_back_addr":"172.21.15.112:6815/755760621","heartbeat_front_addr":"172.21.15.112:6813/755760621","state":["exists","up"]},{"osd":1,"uuid":"a983c227-a3ce-4bdd-82f1-438ba7e3ecd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6801","nonce":344932592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6803","nonce":344932592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6807","nonce":344932592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6805","nonce":344932592}]},"public_addr":"172.21.15.112:6801/344932592","cluster_addr":"172.21.15.112:6803/344932592","heartbeat_back_addr":"172.21.15.112:6807/344932592","heartbeat_front_addr":"172.21.15.112:6805/344932592","state":["exists","up"]},{"osd":2,"uuid":"1193c1d8-bfff-4a57-8200-5f69bf1ea3af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6825","nonce":407933616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6827","nonce":407933616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6831","nonce":407933616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6829","nonce":407933616}]},"public_addr":"172.21.15.112:6825/407933616","cluster_addr":"172.21.15.112:6827/407933616","heartbeat_back_addr":"172.21.15.112:6831/407933616","heartbeat_front_addr":"172.21.15.112:6829/407933616","state":["exists","up"]},{"osd":3,"uuid":"af04330a-36ad-40cc-a8dd-fa259b221de5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6817","nonce":4120164806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6819","nonce":4120164806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6823","nonce":4120164806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6821","nonce":4120164806}]},"public_addr":"172.21.15.112:6817/4120164806","cluster_addr":"172.21.15.112:6819/4120164806","heartbeat_back_addr":"172.21.15.112:6823/4120164806","heartbeat_front_addr":"172.21.15.112:6821/4120164806","state":["exists","up"]},{"osd":4,"uuid":"5b893111-292c-4c5a-b184-53ad38c7199e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6803","nonce":522077560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6805","nonce":522077560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6809","nonce":522077560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6807","nonce":522077560}]},"public_addr":"172.21.15.187:6803/522077560","cluster_addr":"172.21.15.187:6805/522077560","heartbeat_back_addr":"172.21.15.187:6809/522077560","heartbeat_front_addr":"172.21.15.187:6807/522077560","state":["exists","up"]},{"osd":5,"uuid":"c0b8e011-45fe-4c5e-925e-6be2a7f1868a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6827","nonce":3593144721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6829","nonce":3593144721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6833","nonce":3593144721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6831","nonce":3593144721}]},"public_addr":"172.21.15.187:6827/3593144721","cluster_addr":"172.21.15.187:6829/3593144721","heartbeat_back_addr":"172.21.15.187:6833/3593144721","heartbeat_front_addr":"172.21.15.187:6831/3593144721","state":["exists","up"]},{"osd":6,"uuid":"87d2d650-7489-4593-98a3-a1f433ce8dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6811","nonce":1494387657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6813","nonce":1494387657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6817","nonce":1494387657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6815","nonce":1494387657}]},"public_addr":"172.21.15.187:6811/1494387657","cluster_addr":"172.21.15.187:6813/1494387657","heartbeat_back_addr":"172.21.15.187:6817/1494387657","heartbeat_front_addr":"172.21.15.187:6815/1494387657","state":["exists","up"]},{"osd":7,"uuid":"c6b4f8e2-a1ee-4438-aa37-67ff4b6ef1f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6819","nonce":1624699842}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6821","nonce":1624699842}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6825","nonce":1624699842}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6823","nonce":1624699842}]},"public_addr":"172.21.15.187:6819/1624699842","cluster_addr":"172.21.15.187:6821/1624699842","heartbeat_back_addr":"172.21.15.187:6825/1624699842","heartbeat_front_addr":"172.21.15.187:6823/1624699842","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.873162+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.400506+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.664317+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.159791+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.202630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.113212+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.384786+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.448441+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T00:26:08.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.918+0000 7fa4f5169640 1 -- 172.21.15.112:0/1428999287 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fa4d0078b90 msgr2=0x7fa4d007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:08.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.918+0000 7fa4f5169640 1 --2- 172.21.15.112:0/1428999287 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fa4d0078b90 0x7fa4d007b0a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa4e0001460 tx=0x7fa4e0009040 comp rx=0 tx=0).stop 2024-09-21T00:26:08.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.918+0000 7fa4f5169640 1 -- 172.21.15.112:0/1428999287 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa4f0162280 msgr2=0x7fa4f01f55e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:08.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.918+0000 7fa4f5169640 1 --2- 172.21.15.112:0/1428999287 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa4f0162280 0x7fa4f01f55e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa4d8009980 tx=0x7fa4d8003b90 comp rx=0 tx=0).stop 2024-09-21T00:26:08.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.918+0000 7fa4f5169640 1 -- 172.21.15.112:0/1428999287 shutdown_connections 2024-09-21T00:26:08.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.918+0000 7fa4f5169640 1 --2- 172.21.15.112:0/1428999287 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa4f014bf80 0x7fa4f014c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:08.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.918+0000 7fa4f5169640 1 --2- 172.21.15.112:0/1428999287 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fa4d0078b90 0x7fa4d007b0a0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:08.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.918+0000 7fa4f5169640 1 --2- 172.21.15.112:0/1428999287 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa4f0162280 0x7fa4f01f55e0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:08.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.918+0000 7fa4f5169640 1 --2- 172.21.15.112:0/1428999287 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa4f00b2e40 0x7fa4f01f50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:08.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.918+0000 7fa4f5169640 1 -- 172.21.15.112:0/1428999287 >> 172.21.15.112:0/1428999287 conn(0x7fa4f00b6b20 msgr2=0x7fa4f00b3be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:08.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.918+0000 7fa4f5169640 1 -- 172.21.15.112:0/1428999287 shutdown_connections 2024-09-21T00:26:08.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:08.918+0000 7fa4f5169640 1 -- 172.21.15.112:0/1428999287 wait complete. 2024-09-21T00:26:08.930 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T00:26:09.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.041+0000 7f16fb913640 1 -- 172.21.15.112:0/644073207 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16f40d0b40 msgr2=0x7f16f406e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:09.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.041+0000 7f16fb913640 1 --2- 172.21.15.112:0/644073207 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16f40d0b40 0x7f16f406e6f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f16f40cd2c0 tx=0x7f16f002a560 comp rx=0 tx=0).stop 2024-09-21T00:26:09.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.041+0000 7f16fb913640 1 -- 172.21.15.112:0/644073207 shutdown_connections 2024-09-21T00:26:09.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.041+0000 7f16fb913640 1 --2- 172.21.15.112:0/644073207 >> v2:172.21.15.187:3300/0 conn(0x7f16f406ec30 0x7f16f40b3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:09.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.041+0000 7f16fb913640 1 --2- 172.21.15.112:0/644073207 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16f40d0b40 0x7f16f406e6f0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:09.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.041+0000 7f16fb913640 1 -- 172.21.15.112:0/644073207 >> 172.21.15.112:0/644073207 conn(0x7f16f40b2cc0 msgr2=0x7f16f40b30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:09.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.041+0000 7f16fb913640 1 -- 172.21.15.112:0/644073207 shutdown_connections 2024-09-21T00:26:09.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.041+0000 7f16fb913640 1 -- 172.21.15.112:0/644073207 wait complete. 2024-09-21T00:26:09.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.043+0000 7f16fb913640 1 Processor -- start 2024-09-21T00:26:09.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.043+0000 7f16fb913640 1 -- start start 2024-09-21T00:26:09.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.044+0000 7f16fb913640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f16f406ec30 0x7f16f40c7860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:09.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.044+0000 7f16fb913640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f16f40b4500 0x7f16f41597e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:09.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.044+0000 7f16fb913640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16f40d0b40 0x7f16f40b0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:09.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.044+0000 7f16fb913640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f16f415b750 con 0x7f16f40d0b40 2024-09-21T00:26:09.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.044+0000 7f16fb913640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f16f415b5d0 con 0x7f16f40b4500 2024-09-21T00:26:09.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.044+0000 7f16fb913640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f16f415b8d0 con 0x7f16f406ec30 2024-09-21T00:26:09.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.044+0000 7f16f9eb0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16f40d0b40 0x7f16f40b0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:09.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.044+0000 7f16f9eb0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16f40d0b40 0x7f16f40b0b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60166/0 (socket says 172.21.15.112:60166) 2024-09-21T00:26:09.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.044+0000 7f16f9eb0640 1 -- 172.21.15.112:0/3347479050 learned_addr learned my addr 172.21.15.112:0/3347479050 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:26:09.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.045+0000 7f16f8eae640 1 --2- 172.21.15.112:0/3347479050 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f16f40b4500 0x7f16f41597e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:09.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.045+0000 7f16f96af640 1 --2- 172.21.15.112:0/3347479050 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f16f406ec30 0x7f16f40c7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:09.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.045+0000 7f16f9eb0640 1 -- 172.21.15.112:0/3347479050 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f16f406ec30 msgr2=0x7f16f40c7860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:09.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.045+0000 7f16f9eb0640 1 --2- 172.21.15.112:0/3347479050 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f16f406ec30 0x7f16f40c7860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:09.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.045+0000 7f16f9eb0640 1 -- 172.21.15.112:0/3347479050 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f16f40b4500 msgr2=0x7f16f41597e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:09.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.045+0000 7f16f9eb0640 1 --2- 172.21.15.112:0/3347479050 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f16f40b4500 0x7f16f41597e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:09.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.045+0000 7f16f9eb0640 1 -- 172.21.15.112:0/3347479050 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16f41f8fc0 con 0x7f16f40d0b40 2024-09-21T00:26:09.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.045+0000 7f16f9eb0640 1 --2- 172.21.15.112:0/3347479050 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16f40d0b40 0x7f16f40b0b80 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f16ec00b990 tx=0x7f16ec00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:09.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.045+0000 7f16e9ffb640 1 -- 172.21.15.112:0/3347479050 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16ec019070 con 0x7f16f40d0b40 2024-09-21T00:26:09.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.045+0000 7f16e9ffb640 1 -- 172.21.15.112:0/3347479050 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f16ec00cc90 con 0x7f16f40d0b40 2024-09-21T00:26:09.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.045+0000 7f16fb913640 1 -- 172.21.15.112:0/3347479050 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16f41f62b0 con 0x7f16f40d0b40 2024-09-21T00:26:09.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.045+0000 7f16e9ffb640 1 -- 172.21.15.112:0/3347479050 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16ec0146a0 con 0x7f16f40d0b40 2024-09-21T00:26:09.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.045+0000 7f16fb913640 1 -- 172.21.15.112:0/3347479050 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16f40c6d80 con 0x7f16f40d0b40 2024-09-21T00:26:09.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.046+0000 7f16fb913640 1 -- 172.21.15.112:0/3347479050 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16f415be00 con 0x7f16f40d0b40 2024-09-21T00:26:09.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.047+0000 7f16e9ffb640 1 -- 172.21.15.112:0/3347479050 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f16ec01c070 con 0x7f16f40d0b40 2024-09-21T00:26:09.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.048+0000 7f16e9ffb640 1 --2- 172.21.15.112:0/3347479050 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f16d8078b90 0x7f16d807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:09.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.048+0000 7f16e9ffb640 1 -- 172.21.15.112:0/3347479050 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f16ec09aca0 con 0x7f16f40d0b40 2024-09-21T00:26:09.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.048+0000 7f16f96af640 1 --2- 172.21.15.112:0/3347479050 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f16d8078b90 0x7f16d807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:09.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.048+0000 7f16f96af640 1 --2- 172.21.15.112:0/3347479050 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f16d8078b90 0x7f16d807b0a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f16f0033000 tx=0x7f16f0004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:09.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.052+0000 7f16e9ffb640 1 -- 172.21.15.112:0/3347479050 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f16ec062870 con 0x7f16f40d0b40 2024-09-21T00:26:09.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.265+0000 7f16fb913640 1 -- 172.21.15.112:0/3347479050 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f16f40cc660 con 0x7f16f40d0b40 2024-09-21T00:26:09.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.266+0000 7f16e9ffb640 1 -- 172.21.15.112:0/3347479050 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5228 (secure 0 0 0) 0x7f16ec0672a0 con 0x7f16f40d0b40 2024-09-21T00:26:09.267 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:26:09.267 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":9,"btime":"2024-09-21T00:26:01:294823+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4316,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6837/2271985854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6835","nonce":2271985854},{"type":"v1","addr":"172.21.15.187:6837","nonce":2271985854}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4413,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/200348130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":200348130},{"type":"v1","addr":"172.21.15.112:6833","nonce":200348130}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4416,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/3821201561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3821201561},{"type":"v1","addr":"172.21.15.112:6835","nonce":3821201561}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T00:25:47.578960+0000","modified":"2024-09-21T00:26:00.650942+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4425},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4425":{"gid":4425,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6836/2921304789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":2921304789},{"type":"v1","addr":"172.21.15.187:6836","nonce":2921304789}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4425,"qdb_cluster":[4425]},"id":1}]} 2024-09-21T00:26:09.267 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 9 2024-09-21T00:26:09.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.268+0000 7f16fb913640 1 -- 172.21.15.112:0/3347479050 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f16d8078b90 msgr2=0x7f16d807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:09.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.268+0000 7f16fb913640 1 --2- 172.21.15.112:0/3347479050 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f16d8078b90 0x7f16d807b0a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f16f0033000 tx=0x7f16f0004730 comp rx=0 tx=0).stop 2024-09-21T00:26:09.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.268+0000 7f16fb913640 1 -- 172.21.15.112:0/3347479050 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16f40d0b40 msgr2=0x7f16f40b0b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:09.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.268+0000 7f16fb913640 1 --2- 172.21.15.112:0/3347479050 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16f40d0b40 0x7f16f40b0b80 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f16ec00b990 tx=0x7f16ec00be50 comp rx=0 tx=0).stop 2024-09-21T00:26:09.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.268+0000 7f16fb913640 1 -- 172.21.15.112:0/3347479050 shutdown_connections 2024-09-21T00:26:09.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.268+0000 7f16fb913640 1 --2- 172.21.15.112:0/3347479050 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f16d8078b90 0x7f16d807b0a0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:09.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.268+0000 7f16fb913640 1 --2- 172.21.15.112:0/3347479050 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16f40d0b40 0x7f16f40b0b80 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:09.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.268+0000 7f16fb913640 1 --2- 172.21.15.112:0/3347479050 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f16f40b4500 0x7f16f41597e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:09.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.268+0000 7f16fb913640 1 --2- 172.21.15.112:0/3347479050 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f16f406ec30 0x7f16f40c7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:09.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.268+0000 7f16fb913640 1 -- 172.21.15.112:0/3347479050 >> 172.21.15.112:0/3347479050 conn(0x7f16f40b2cc0 msgr2=0x7f16f40b30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:09.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.268+0000 7f16fb913640 1 -- 172.21.15.112:0/3347479050 shutdown_connections 2024-09-21T00:26:09.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.268+0000 7f16fb913640 1 -- 172.21.15.112:0/3347479050 wait complete. 2024-09-21T00:26:09.683 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 9, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T00:25:47.578960+0000', 'modified': '2024-09-21T00:26:00.650942+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4425}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4425': {'gid': 4425, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.187:6836/2921304789', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.187:6834', 'nonce': 2921304789}, {'type': 'v1', 'addr': '172.21.15.187:6836', 'nonce': 2921304789}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4425, 'qdb_cluster': [4425]} 2024-09-21T00:26:09.683 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-21T00:26:09.683 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-09-21T00:26:09.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.799+0000 7f49da48b640 1 -- 172.21.15.112:0/1672838595 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49d415ca90 msgr2=0x7f49d40d0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:09.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.799+0000 7f49da48b640 1 --2- 172.21.15.112:0/1672838595 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49d415ca90 0x7f49d40d0670 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f49c8005fd0 tx=0x7f49c802a540 comp rx=0 tx=0).stop 2024-09-21T00:26:09.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.799+0000 7f49da48b640 1 -- 172.21.15.112:0/1672838595 shutdown_connections 2024-09-21T00:26:09.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.799+0000 7f49da48b640 1 --2- 172.21.15.112:0/1672838595 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49d415ca90 0x7f49d40d0670 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:09.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.799+0000 7f49da48b640 1 -- 172.21.15.112:0/1672838595 >> 172.21.15.112:0/1672838595 conn(0x7f49d40b46e0 msgr2=0x7f49d40d23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:09.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.799+0000 7f49da48b640 1 -- 172.21.15.112:0/1672838595 shutdown_connections 2024-09-21T00:26:09.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.799+0000 7f49da48b640 1 -- 172.21.15.112:0/1672838595 wait complete. 2024-09-21T00:26:09.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.801+0000 7f49da48b640 1 Processor -- start 2024-09-21T00:26:09.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.801+0000 7f49da48b640 1 -- start start 2024-09-21T00:26:09.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.802+0000 7f49da48b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f49d40ccc40 0x7f49d40c9910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:09.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.802+0000 7f49da48b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49d40d0bb0 0x7f49d41e65f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:09.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.802+0000 7f49da48b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f49d415ca90 0x7f49d41e6b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:09.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.802+0000 7f49da48b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f49d40b6840 con 0x7f49d40d0bb0 2024-09-21T00:26:09.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.802+0000 7f49da48b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f49d40b66c0 con 0x7f49d40ccc40 2024-09-21T00:26:09.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.802+0000 7f49da48b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f49d40b69c0 con 0x7f49d415ca90 2024-09-21T00:26:09.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.802+0000 7f49d37fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49d40d0bb0 0x7f49d41e65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:09.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.802+0000 7f49d37fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49d40d0bb0 0x7f49d41e65f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60180/0 (socket says 172.21.15.112:60180) 2024-09-21T00:26:09.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.802+0000 7f49d37fe640 1 -- 172.21.15.112:0/2718766803 learned_addr learned my addr 172.21.15.112:0/2718766803 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:26:09.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.802+0000 7f49d3fff640 1 --2- 172.21.15.112:0/2718766803 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f49d40ccc40 0x7f49d40c9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:09.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.802+0000 7f49d8a28640 1 --2- 172.21.15.112:0/2718766803 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f49d415ca90 0x7f49d41e6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:09.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.802+0000 7f49d37fe640 1 -- 172.21.15.112:0/2718766803 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f49d415ca90 msgr2=0x7f49d41e6b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:09.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.802+0000 7f49d37fe640 1 --2- 172.21.15.112:0/2718766803 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f49d415ca90 0x7f49d41e6b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:09.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.802+0000 7f49d37fe640 1 -- 172.21.15.112:0/2718766803 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f49d40ccc40 msgr2=0x7f49d40c9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:09.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.802+0000 7f49d37fe640 1 --2- 172.21.15.112:0/2718766803 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f49d40ccc40 0x7f49d40c9910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:09.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.802+0000 7f49d37fe640 1 -- 172.21.15.112:0/2718766803 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49d40ca1f0 con 0x7f49d40d0bb0 2024-09-21T00:26:09.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.802+0000 7f49d3fff640 1 --2- 172.21.15.112:0/2718766803 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f49d40ccc40 0x7f49d40c9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:26:09.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.802+0000 7f49d8a28640 1 --2- 172.21.15.112:0/2718766803 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f49d415ca90 0x7f49d41e6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:26:09.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.803+0000 7f49d37fe640 1 --2- 172.21.15.112:0/2718766803 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49d40d0bb0 0x7f49d41e65f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f49c8034040 tx=0x7f49c80039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:09.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.803+0000 7f49d0ff9640 1 -- 172.21.15.112:0/2718766803 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49c803e030 con 0x7f49d40d0bb0 2024-09-21T00:26:09.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.803+0000 7f49d0ff9640 1 -- 172.21.15.112:0/2718766803 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f49c80047a0 con 0x7f49d40d0bb0 2024-09-21T00:26:09.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.803+0000 7f49d0ff9640 1 -- 172.21.15.112:0/2718766803 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49c8035040 con 0x7f49d40d0bb0 2024-09-21T00:26:09.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.803+0000 7f49da48b640 1 -- 172.21.15.112:0/2718766803 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49d40ca4d0 con 0x7f49d40d0bb0 2024-09-21T00:26:09.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.803+0000 7f49da48b640 1 -- 172.21.15.112:0/2718766803 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49d40caa90 con 0x7f49d40d0bb0 2024-09-21T00:26:09.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.805+0000 7f49d0ff9640 1 -- 172.21.15.112:0/2718766803 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f49c8041030 con 0x7f49d40d0bb0 2024-09-21T00:26:09.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.805+0000 7f49d0ff9640 1 --2- 172.21.15.112:0/2718766803 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f49c0078b90 0x7f49c007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:09.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.805+0000 7f49d0ff9640 1 -- 172.21.15.112:0/2718766803 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f49c80bb840 con 0x7f49d40d0bb0 2024-09-21T00:26:09.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.806+0000 7f49d3fff640 1 --2- 172.21.15.112:0/2718766803 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f49c0078b90 0x7f49c007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:09.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.806+0000 7f49e13f8740 1 Processor -- start 2024-09-21T00:26:09.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.806+0000 7f49d3fff640 1 --2- 172.21.15.112:0/2718766803 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f49c0078b90 0x7f49c007b0a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f49c4000f60 tx=0x7f49c4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:09.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.806+0000 7f49e13f8740 1 -- start start 2024-09-21T00:26:09.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.808+0000 7f49e13f8740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a5c3d90270 0x55a5c4161fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:09.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.808+0000 7f49e13f8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a5c3d673b0 0x55a5c41624e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:09.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.808+0000 7f49e13f8740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a5c3d6f590 0x55a5c417f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:09.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.808+0000 7f49e13f8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a5c40aa820 con 0x55a5c3d673b0 2024-09-21T00:26:09.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.808+0000 7f49e13f8740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55a5c4099c50 con 0x55a5c3d6f590 2024-09-21T00:26:09.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.808+0000 7f49e13f8740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x55a5c409da60 con 0x55a5c3d90270 2024-09-21T00:26:09.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.808+0000 7f49d37fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a5c3d673b0 0x55a5c41624e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:09.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.808+0000 7f49d37fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a5c3d673b0 0x55a5c41624e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60184/0 (socket says 172.21.15.112:60184) 2024-09-21T00:26:09.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.808+0000 7f49d37fe640 1 -- 172.21.15.112:0/2170330249 learned_addr learned my addr 172.21.15.112:0/2170330249 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:26:09.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.808+0000 7f49d3fff640 1 --2- 172.21.15.112:0/2170330249 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a5c3d90270 0x55a5c4161fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:09.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.808+0000 7f49d37fe640 1 -- 172.21.15.112:0/2170330249 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a5c3d90270 msgr2=0x55a5c4161fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:26:09.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.808+0000 7f49d37fe640 1 --2- 172.21.15.112:0/2170330249 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a5c3d90270 0x55a5c4161fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:09.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.808+0000 7f49d37fe640 1 -- 172.21.15.112:0/2170330249 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a5c3d6f590 msgr2=0x55a5c417f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:26:09.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.808+0000 7f49d37fe640 1 --2- 172.21.15.112:0/2170330249 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a5c3d6f590 0x55a5c417f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:09.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.808+0000 7f49d37fe640 1 -- 172.21.15.112:0/2170330249 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5c3dfb850 con 0x55a5c3d673b0 2024-09-21T00:26:09.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.808+0000 7f49d37fe640 1 --2- 172.21.15.112:0/2170330249 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a5c3d673b0 0x55a5c41624e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f49c80b9a60 tx=0x7f49c802a540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b78f0f21441b4da9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:09.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.808+0000 7f49ba7fc640 1 -- 172.21.15.112:0/2170330249 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49c8048040 con 0x55a5c3d673b0 2024-09-21T00:26:09.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.808+0000 7f49ba7fc640 1 -- 172.21.15.112:0/2170330249 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f49c80b9e50 con 0x55a5c3d673b0 2024-09-21T00:26:09.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.808+0000 7f49ba7fc640 1 -- 172.21.15.112:0/2170330249 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49c808ca80 con 0x55a5c3d673b0 2024-09-21T00:26:09.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.809+0000 7f49e13f8740 1 -- 172.21.15.112:0/2170330249 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a5c3d673b0 msgr2=0x55a5c41624e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:09.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.809+0000 7f49e13f8740 1 --2- 172.21.15.112:0/2170330249 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a5c3d673b0 0x55a5c41624e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f49c80b9a60 tx=0x7f49c802a540 comp rx=0 tx=0).stop 2024-09-21T00:26:09.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.809+0000 7f49e13f8740 1 -- 172.21.15.112:0/2170330249 shutdown_connections 2024-09-21T00:26:09.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.809+0000 7f49e13f8740 1 --2- 172.21.15.112:0/2170330249 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a5c3d6f590 0x55a5c417f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:09.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.809+0000 7f49e13f8740 1 --2- 172.21.15.112:0/2170330249 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a5c3d673b0 0x55a5c41624e0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:09.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.809+0000 7f49e13f8740 1 --2- 172.21.15.112:0/2170330249 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a5c3d90270 0x55a5c4161fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:09.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.809+0000 7f49e13f8740 1 -- 172.21.15.112:0/2170330249 >> 172.21.15.112:0/2170330249 conn(0x55a5c3d843c0 msgr2=0x55a5c4138260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:09.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.809+0000 7f49e13f8740 1 -- 172.21.15.112:0/2170330249 shutdown_connections 2024-09-21T00:26:09.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.809+0000 7f49e13f8740 1 -- 172.21.15.112:0/2170330249 wait complete. 2024-09-21T00:26:09.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.810+0000 7f49e13f8740 1 -- start start 2024-09-21T00:26:09.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.810+0000 7f49e13f8740 1 Processor -- start 2024-09-21T00:26:09.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.811+0000 7f49e13f8740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a5c3d673b0 0x55a5c415efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:09.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.811+0000 7f49e13f8740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a5c3d6f590 0x55a5c418c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:09.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.811+0000 7f49e13f8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a5c3d90270 0x55a5c41759f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:09.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.811+0000 7f49e13f8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a5c3e0a620 con 0x55a5c3d90270 2024-09-21T00:26:09.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.811+0000 7f49e13f8740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55a5c403f5f0 con 0x55a5c3d6f590 2024-09-21T00:26:09.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.811+0000 7f49e13f8740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x55a5c4162a20 con 0x55a5c3d673b0 2024-09-21T00:26:09.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.811+0000 7f49e13f8740 10 client.0 _ll_register_callbacks cb 0x55a5c40430a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:26:09.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.812+0000 7f49d8a28640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a5c3d90270 0x55a5c41759f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:09.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.812+0000 7f49d8a28640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a5c3d90270 0x55a5c41759f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60190/0 (socket says 172.21.15.112:60190) 2024-09-21T00:26:09.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.812+0000 7f49d8a28640 1 -- 172.21.15.112:0/3337320690 learned_addr learned my addr 172.21.15.112:0/3337320690 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:26:09.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.812+0000 7f49d3fff640 1 --2- 172.21.15.112:0/3337320690 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a5c3d673b0 0x55a5c415efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:09.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.812+0000 7f49d37fe640 1 --2- 172.21.15.112:0/3337320690 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a5c3d6f590 0x55a5c418c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:09.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.812+0000 7f49d8a28640 1 -- 172.21.15.112:0/3337320690 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a5c3d673b0 msgr2=0x55a5c415efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:26:09.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.812+0000 7f49d8a28640 1 --2- 172.21.15.112:0/3337320690 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a5c3d673b0 0x55a5c415efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:09.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.812+0000 7f49d8a28640 1 -- 172.21.15.112:0/3337320690 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a5c3d6f590 msgr2=0x55a5c418c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:26:09.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.812+0000 7f49d8a28640 1 --2- 172.21.15.112:0/3337320690 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a5c3d6f590 0x55a5c418c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:09.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.812+0000 7f49d8a28640 1 -- 172.21.15.112:0/3337320690 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5c3e726b0 con 0x55a5c3d90270 2024-09-21T00:26:09.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.812+0000 7f49d8a28640 1 --2- 172.21.15.112:0/3337320690 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a5c3d90270 0x55a5c41759f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f49cc00c9b0 tx=0x7f49cc00ce70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66243a3f904905df server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:09.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.812+0000 7f49b2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T00:26:09.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.812+0000 7f49b2ffd640 1 -- 172.21.15.112:0/3337320690 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49cc007b40 con 0x55a5c3d90270 2024-09-21T00:26:09.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.812+0000 7f49b2ffd640 1 -- 172.21.15.112:0/3337320690 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f49cc007ce0 con 0x55a5c3d90270 2024-09-21T00:26:09.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.812+0000 7f49e13f8740 1 -- 172.21.15.112:0/3337320690 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f49c80b9e50 con 0x55a5c3d90270 2024-09-21T00:26:09.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.812+0000 7f49b2ffd640 1 -- 172.21.15.112:0/3337320690 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) ==== 24+0+0 (secure 0 0 0) 0x7f49cc01f050 con 0x55a5c3d90270 2024-09-21T00:26:09.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.812+0000 7f49e13f8740 10 client.4815 fetch_fsmap learned FSMap version 9 2024-09-21T00:26:09.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.812+0000 7f49e13f8740 1 -- 172.21.15.112:0/3337320690 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) -- 0x55a5c415f4e0 con 0x55a5c3d90270 2024-09-21T00:26:09.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.812+0000 7f49b2ffd640 1 -- 172.21.15.112:0/3337320690 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f49c80b9e50 con 0x55a5c3d90270 2024-09-21T00:26:09.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.813+0000 7f49b2ffd640 1 -- 172.21.15.112:0/3337320690 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 9) ==== 3510+0+0 (secure 0 0 0) 0x7f49cc022020 con 0x55a5c3d90270 2024-09-21T00:26:09.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.813+0000 7f49e13f8740 10 client.4815 fetch_fsmap finished waiting for FSMap version 9 2024-09-21T00:26:09.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.813+0000 7f49e13f8740 10 client.4815 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T00:26:09.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.813+0000 7f49e13f8740 20 client.4815 populate_metadata read hostname 'smithi112' 2024-09-21T00:26:09.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.813+0000 7f49e13f8740 1 --2- 172.21.15.112:0/3337320690 >> [v2:172.21.15.187:6834/2921304789,v1:172.21.15.187:6836/2921304789] conn(0x55a5c3d4ee40 0x55a5c4199d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:09.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.813+0000 7f49e13f8740 4 client.4815 mds_command: new command op to 4425 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T00:26:09.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.813+0000 7f49e13f8740 1 -- 172.21.15.112:0/3337320690 --> [v2:172.21.15.187:6834/2921304789,v1:172.21.15.187:6836/2921304789] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a5c40ac770 con 0x55a5c3d4ee40 2024-09-21T00:26:09.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.813+0000 7f49d3fff640 1 --2- 172.21.15.112:0/3337320690 >> [v2:172.21.15.187:6834/2921304789,v1:172.21.15.187:6836/2921304789] conn(0x55a5c3d4ee40 0x55a5c4199d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:09.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.813+0000 7f49b2ffd640 10 client.4815 ms_handle_connect on v2:172.21.15.187:6834/2921304789 2024-09-21T00:26:09.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.813+0000 7f49d3fff640 1 --2- 172.21.15.112:0/3337320690 >> [v2:172.21.15.187:6834/2921304789,v1:172.21.15.187:6836/2921304789] conn(0x55a5c3d4ee40 0x55a5c4199d80 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=668724eccaf76346 server_cookie=30af5c824f083a76 in_seq=0 out_seq=0 2024-09-21T00:26:09.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.814+0000 7f49b2ffd640 1 -- 172.21.15.112:0/3337320690 <== mds.0 v2:172.21.15.187:6834/2921304789 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a5c40ac770 con 0x55a5c3d4ee40 2024-09-21T00:26:09.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.814+0000 7f49b2ffd640 10 client.4815 handle_command_reply: tid=0 2024-09-21T00:26:09.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.814+0000 7f49e13f8740 1 client.4815 shutdown 2024-09-21T00:26:09.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.814+0000 7f49e13f8740 1 -- 172.21.15.112:0/3337320690 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a5c3d90270 msgr2=0x55a5c41759f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:09.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.814+0000 7f49e13f8740 1 --2- 172.21.15.112:0/3337320690 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a5c3d90270 0x55a5c41759f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f49cc00c9b0 tx=0x7f49cc00ce70 comp rx=0 tx=0).stop 2024-09-21T00:26:09.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.814+0000 7f49e13f8740 1 -- 172.21.15.112:0/3337320690 shutdown_connections 2024-09-21T00:26:09.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.814+0000 7f49e13f8740 1 --2- 172.21.15.112:0/3337320690 >> [v2:172.21.15.187:6834/2921304789,v1:172.21.15.187:6836/2921304789] conn(0x55a5c3d4ee40 0x55a5c4199d80 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:09.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.814+0000 7f49e13f8740 1 --2- 172.21.15.112:0/3337320690 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a5c3d90270 0x55a5c41759f0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:09.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.814+0000 7f49e13f8740 1 --2- 172.21.15.112:0/3337320690 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a5c3d6f590 0x55a5c418c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:09.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.814+0000 7f49b2ffd640 0 client.4815 ms_handle_reset on v2:172.21.15.187:6834/2921304789 2024-09-21T00:26:09.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.814+0000 7f49e13f8740 1 --2- 172.21.15.112:0/3337320690 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a5c3d673b0 0x55a5c415efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:09.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.814+0000 7f49e13f8740 1 -- 172.21.15.112:0/3337320690 >> 172.21.15.112:0/3337320690 conn(0x55a5c3d843c0 msgr2=0x55a5c4138110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:09.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.814+0000 7f49e13f8740 1 -- 172.21.15.112:0/3337320690 shutdown_connections 2024-09-21T00:26:09.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.814+0000 7f49e13f8740 1 -- 172.21.15.112:0/3337320690 wait complete. 2024-09-21T00:26:09.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.814+0000 7f49e13f8740 20 client.4815 trim_cache size 0 max 16384 2024-09-21T00:26:09.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.833+0000 7f49e13f8740 1 Processor -- start 2024-09-21T00:26:09.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.833+0000 7f49e13f8740 1 -- start start 2024-09-21T00:26:09.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.834+0000 7f49e13f8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a5c3d843c0 0x55a5c415dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:09.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.834+0000 7f49e13f8740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a5c3d673b0 0x55a5c415e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:09.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.834+0000 7f49e13f8740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a5c3d6f590 0x55a5c41404e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:09.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.834+0000 7f49e13f8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f49d42013d0 con 0x55a5c3d843c0 2024-09-21T00:26:09.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.834+0000 7f49e13f8740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f49d41f4af0 con 0x55a5c3d673b0 2024-09-21T00:26:09.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.834+0000 7f49e13f8740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f49d41e5920 con 0x55a5c3d6f590 2024-09-21T00:26:09.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.834+0000 7f49d3fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a5c3d843c0 0x55a5c415dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:09.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.834+0000 7f49d3fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a5c3d843c0 0x55a5c415dd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60202/0 (socket says 172.21.15.112:60202) 2024-09-21T00:26:09.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.834+0000 7f49d3fff640 1 -- 172.21.15.112:0/2032299831 learned_addr learned my addr 172.21.15.112:0/2032299831 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:26:09.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.834+0000 7f49d8a28640 1 --2- 172.21.15.112:0/2032299831 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a5c3d6f590 0x55a5c41404e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:09.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.834+0000 7f49d37fe640 1 --2- 172.21.15.112:0/2032299831 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a5c3d673b0 0x55a5c415e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:09.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.835+0000 7f49d3fff640 1 -- 172.21.15.112:0/2032299831 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a5c3d6f590 msgr2=0x55a5c41404e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:26:09.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.835+0000 7f49d3fff640 1 --2- 172.21.15.112:0/2032299831 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a5c3d6f590 0x55a5c41404e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:09.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.835+0000 7f49d3fff640 1 -- 172.21.15.112:0/2032299831 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a5c3d673b0 msgr2=0x55a5c415e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:26:09.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.835+0000 7f49d3fff640 1 --2- 172.21.15.112:0/2032299831 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a5c3d673b0 0x55a5c415e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:09.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.835+0000 7f49d3fff640 1 -- 172.21.15.112:0/2032299831 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5c415db00 con 0x55a5c3d843c0 2024-09-21T00:26:09.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.835+0000 7f49d3fff640 1 --2- 172.21.15.112:0/2032299831 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a5c3d843c0 0x55a5c415dd10 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f49c4007fd0 tx=0x7f49c4017040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7156e5a03ad50b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:09.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.835+0000 7f49b9ffb640 1 -- 172.21.15.112:0/2032299831 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49c40035f0 con 0x55a5c3d843c0 2024-09-21T00:26:09.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.835+0000 7f49b9ffb640 1 -- 172.21.15.112:0/2032299831 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f49c4003790 con 0x55a5c3d843c0 2024-09-21T00:26:09.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.835+0000 7f49b9ffb640 1 -- 172.21.15.112:0/2032299831 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49c4003b40 con 0x55a5c3d843c0 2024-09-21T00:26:09.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.836+0000 7f49e13f8740 1 -- 172.21.15.112:0/2032299831 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a5c3d843c0 msgr2=0x55a5c415dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:09.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.836+0000 7f49e13f8740 1 --2- 172.21.15.112:0/2032299831 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a5c3d843c0 0x55a5c415dd10 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f49c4007fd0 tx=0x7f49c4017040 comp rx=0 tx=0).stop 2024-09-21T00:26:09.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.836+0000 7f49e13f8740 1 -- 172.21.15.112:0/2032299831 shutdown_connections 2024-09-21T00:26:09.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.836+0000 7f49e13f8740 1 --2- 172.21.15.112:0/2032299831 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a5c3d6f590 0x55a5c41404e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:09.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.836+0000 7f49e13f8740 1 --2- 172.21.15.112:0/2032299831 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a5c3d673b0 0x55a5c415e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:09.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.836+0000 7f49e13f8740 1 --2- 172.21.15.112:0/2032299831 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a5c3d843c0 0x55a5c415dd10 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:09.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.836+0000 7f49e13f8740 1 -- 172.21.15.112:0/2032299831 >> 172.21.15.112:0/2032299831 conn(0x55a5c4175b20 msgr2=0x55a5c41380f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:09.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.836+0000 7f49e13f8740 1 -- 172.21.15.112:0/2032299831 shutdown_connections 2024-09-21T00:26:09.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.836+0000 7f49e13f8740 1 -- 172.21.15.112:0/2032299831 wait complete. 2024-09-21T00:26:09.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.837+0000 7f49e13f8740 1 -- start start 2024-09-21T00:26:09.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.837+0000 7f49e13f8740 1 Processor -- start 2024-09-21T00:26:09.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.838+0000 7f49e13f8740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a5c3d673b0 0x55a5c413df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:09.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.838+0000 7f49e13f8740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a5c3d6f590 0x55a5c4200720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:09.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.838+0000 7f49e13f8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a5c3d843c0 0x55a5c413c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:09.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.838+0000 7f49e13f8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a5c4142d80 con 0x55a5c3d843c0 2024-09-21T00:26:09.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.838+0000 7f49e13f8740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f49d41f44d0 con 0x55a5c3d6f590 2024-09-21T00:26:09.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.838+0000 7f49e13f8740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x55a5c4175f30 con 0x55a5c3d673b0 2024-09-21T00:26:09.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.838+0000 7f49e13f8740 10 client.0 _ll_register_callbacks cb 0x7f49cc009690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:26:09.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.838+0000 7f49d8a28640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a5c3d843c0 0x55a5c413c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:09.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.838+0000 7f49d8a28640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a5c3d843c0 0x55a5c413c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60216/0 (socket says 172.21.15.112:60216) 2024-09-21T00:26:09.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.838+0000 7f49d8a28640 1 -- 172.21.15.112:0/570343204 learned_addr learned my addr 172.21.15.112:0/570343204 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:26:09.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.838+0000 7f49d3fff640 1 --2- 172.21.15.112:0/570343204 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a5c3d673b0 0x55a5c413df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:09.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.838+0000 7f49d37fe640 1 --2- 172.21.15.112:0/570343204 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a5c3d6f590 0x55a5c4200720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:09.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.838+0000 7f49d8a28640 1 -- 172.21.15.112:0/570343204 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a5c3d673b0 msgr2=0x55a5c413df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:26:09.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.838+0000 7f49d8a28640 1 --2- 172.21.15.112:0/570343204 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a5c3d673b0 0x55a5c413df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:09.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.838+0000 7f49d8a28640 1 -- 172.21.15.112:0/570343204 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a5c3d6f590 msgr2=0x55a5c4200720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:26:09.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.838+0000 7f49d8a28640 1 --2- 172.21.15.112:0/570343204 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a5c3d6f590 0x55a5c4200720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:09.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.838+0000 7f49d8a28640 1 -- 172.21.15.112:0/570343204 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5c41398a0 con 0x55a5c3d843c0 2024-09-21T00:26:09.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.839+0000 7f49d8a28640 1 --2- 172.21.15.112:0/570343204 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a5c3d843c0 0x55a5c413c280 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f49cc004ab0 tx=0x7f49cc003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ac2c572a758b269 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:09.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.839+0000 7f49b2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T00:26:09.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.839+0000 7f49b2ffd640 1 -- 172.21.15.112:0/570343204 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49cc027070 con 0x55a5c3d843c0 2024-09-21T00:26:09.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.839+0000 7f49b2ffd640 1 -- 172.21.15.112:0/570343204 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f49cc004040 con 0x55a5c3d843c0 2024-09-21T00:26:09.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.839+0000 7f49e13f8740 1 -- 172.21.15.112:0/570343204 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f49c4003790 con 0x55a5c3d843c0 2024-09-21T00:26:09.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.839+0000 7f49b2ffd640 1 -- 172.21.15.112:0/570343204 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) ==== 24+0+0 (secure 0 0 0) 0x7f49cc032050 con 0x55a5c3d843c0 2024-09-21T00:26:09.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.839+0000 7f49e13f8740 10 client.4821 fetch_fsmap learned FSMap version 9 2024-09-21T00:26:09.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.839+0000 7f49e13f8740 1 -- 172.21.15.112:0/570343204 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) -- 0x55a5c413e4d0 con 0x55a5c3d843c0 2024-09-21T00:26:09.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.839+0000 7f49b2ffd640 1 -- 172.21.15.112:0/570343204 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f49c4003790 con 0x55a5c3d843c0 2024-09-21T00:26:09.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.839+0000 7f49b2ffd640 1 -- 172.21.15.112:0/570343204 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 9) ==== 3510+0+0 (secure 0 0 0) 0x7f49cc035020 con 0x55a5c3d843c0 2024-09-21T00:26:09.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.839+0000 7f49e13f8740 10 client.4821 fetch_fsmap finished waiting for FSMap version 9 2024-09-21T00:26:09.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.839+0000 7f49e13f8740 10 client.4821 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T00:26:09.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.839+0000 7f49e13f8740 20 client.4821 populate_metadata read hostname 'smithi112' 2024-09-21T00:26:09.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.839+0000 7f49e13f8740 1 --2- 172.21.15.112:0/570343204 >> [v2:172.21.15.187:6834/2921304789,v1:172.21.15.187:6836/2921304789] conn(0x55a5c3d24280 0x55a5c4161c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:09.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.839+0000 7f49e13f8740 4 client.4821 mds_command: new command op to 4425 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T00:26:09.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.839+0000 7f49e13f8740 1 -- 172.21.15.112:0/570343204 --> [v2:172.21.15.187:6834/2921304789,v1:172.21.15.187:6836/2921304789] -- command(tid 0: {"prefix": "status"}) -- 0x55a5c40a8220 con 0x55a5c3d24280 2024-09-21T00:26:09.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.839+0000 7f49d3fff640 1 --2- 172.21.15.112:0/570343204 >> [v2:172.21.15.187:6834/2921304789,v1:172.21.15.187:6836/2921304789] conn(0x55a5c3d24280 0x55a5c4161c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:09.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.840+0000 7f49b2ffd640 10 client.4821 ms_handle_connect on v2:172.21.15.187:6834/2921304789 2024-09-21T00:26:09.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.840+0000 7f49d3fff640 1 --2- 172.21.15.112:0/570343204 >> [v2:172.21.15.187:6834/2921304789,v1:172.21.15.187:6836/2921304789] conn(0x55a5c3d24280 0x55a5c4161c60 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee8c799e56db0a25 server_cookie=ad97a749c873b6ff in_seq=0 out_seq=0 2024-09-21T00:26:09.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.840+0000 7f49b2ffd640 1 -- 172.21.15.112:0/570343204 <== mds.0 v2:172.21.15.187:6834/2921304789 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x55a5c40a8220 con 0x55a5c3d24280 2024-09-21T00:26:09.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.840+0000 7f49b2ffd640 10 client.4821 handle_command_reply: tid=0 2024-09-21T00:26:09.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.840+0000 7f49e13f8740 1 client.4821 shutdown 2024-09-21T00:26:09.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.840+0000 7f49e13f8740 1 -- 172.21.15.112:0/570343204 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a5c3d843c0 msgr2=0x55a5c413c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:09.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.840+0000 7f49e13f8740 1 --2- 172.21.15.112:0/570343204 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a5c3d843c0 0x55a5c413c280 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f49cc004ab0 tx=0x7f49cc003ba0 comp rx=0 tx=0).stop 2024-09-21T00:26:09.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.840+0000 7f49e13f8740 1 -- 172.21.15.112:0/570343204 shutdown_connections 2024-09-21T00:26:09.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.840+0000 7f49e13f8740 1 --2- 172.21.15.112:0/570343204 >> [v2:172.21.15.187:6834/2921304789,v1:172.21.15.187:6836/2921304789] conn(0x55a5c3d24280 0x55a5c4161c60 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:09.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.840+0000 7f49e13f8740 1 --2- 172.21.15.112:0/570343204 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a5c3d843c0 0x55a5c413c280 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:09.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.840+0000 7f49e13f8740 1 --2- 172.21.15.112:0/570343204 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a5c3d6f590 0x55a5c4200720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:09.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.840+0000 7f49b2ffd640 0 client.4821 ms_handle_reset on v2:172.21.15.187:6834/2921304789 2024-09-21T00:26:09.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.840+0000 7f49e13f8740 1 --2- 172.21.15.112:0/570343204 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a5c3d673b0 0x55a5c413df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:09.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.840+0000 7f49e13f8740 1 -- 172.21.15.112:0/570343204 >> 172.21.15.112:0/570343204 conn(0x55a5c4175b20 msgr2=0x55a5c415dd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:09.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.840+0000 7f49e13f8740 1 -- 172.21.15.112:0/570343204 shutdown_connections 2024-09-21T00:26:09.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.840+0000 7f49e13f8740 1 -- 172.21.15.112:0/570343204 wait complete. 2024-09-21T00:26:09.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.840+0000 7f49e13f8740 20 client.4821 trim_cache size 0 max 16384 2024-09-21T00:26:09.847 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-21T00:26:09.847 INFO:teuthology.orchestra.run.smithi112.stdout: "cluster_fsid": "8b34692b-dc65-49b2-a08f-26f707b96ac5", 2024-09-21T00:26:09.847 INFO:teuthology.orchestra.run.smithi112.stdout: "whoami": 0, 2024-09-21T00:26:09.847 INFO:teuthology.orchestra.run.smithi112.stdout: "id": 4425, 2024-09-21T00:26:09.847 INFO:teuthology.orchestra.run.smithi112.stdout: "want_state": "up:active", 2024-09-21T00:26:09.847 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "up:active", 2024-09-21T00:26:09.847 INFO:teuthology.orchestra.run.smithi112.stdout: "fs_name": "cephfs", 2024-09-21T00:26:09.847 INFO:teuthology.orchestra.run.smithi112.stdout: "rank_uptime": 21.630266105, 2024-09-21T00:26:09.848 INFO:teuthology.orchestra.run.smithi112.stdout: "mdsmap_epoch": 9, 2024-09-21T00:26:09.848 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch": 26, 2024-09-21T00:26:09.848 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch_barrier": 26, 2024-09-21T00:26:09.848 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 27.680659335000001 2024-09-21T00:26:09.848 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-21T00:26:09.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.841+0000 7f49da48b640 1 -- 172.21.15.112:0/2718766803 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f49c0078b90 msgr2=0x7f49c007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:09.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.841+0000 7f49da48b640 1 --2- 172.21.15.112:0/2718766803 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f49c0078b90 0x7f49c007b0a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f49c4000f60 tx=0x7f49c4009040 comp rx=0 tx=0).stop 2024-09-21T00:26:09.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.841+0000 7f49da48b640 1 -- 172.21.15.112:0/2718766803 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49d40d0bb0 msgr2=0x7f49d41e65f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:09.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.841+0000 7f49da48b640 1 --2- 172.21.15.112:0/2718766803 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49d40d0bb0 0x7f49d41e65f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f49c8034040 tx=0x7f49c80039e0 comp rx=0 tx=0).stop 2024-09-21T00:26:09.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.841+0000 7f49da48b640 1 -- 172.21.15.112:0/2718766803 shutdown_connections 2024-09-21T00:26:09.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.841+0000 7f49da48b640 1 --2- 172.21.15.112:0/2718766803 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f49d415ca90 0x7f49d41e6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:09.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.841+0000 7f49da48b640 1 --2- 172.21.15.112:0/2718766803 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f49c0078b90 0x7f49c007b0a0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:09.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.841+0000 7f49da48b640 1 --2- 172.21.15.112:0/2718766803 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49d40d0bb0 0x7f49d41e65f0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:09.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.841+0000 7f49da48b640 1 --2- 172.21.15.112:0/2718766803 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f49d40ccc40 0x7f49d40c9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:09.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.841+0000 7f49da48b640 1 -- 172.21.15.112:0/2718766803 >> 172.21.15.112:0/2718766803 conn(0x7f49d40b46e0 msgr2=0x7f49d40cf790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:09.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.841+0000 7f49da48b640 1 -- 172.21.15.112:0/2718766803 shutdown_connections 2024-09-21T00:26:09.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:09.841+0000 7f49da48b640 1 -- 172.21.15.112:0/2718766803 wait complete. 2024-09-21T00:26:09.853 INFO:teuthology.run_tasks:Running task kclient... 2024-09-21T00:26:09.872 INFO:tasks.kclient:Mounting kernel clients... 2024-09-21T00:26:09.873 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-09-21T00:26:09.873 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:26:09.873 DEBUG:teuthology.orchestra.run.smithi112:> ip netns list 2024-09-21T00:26:09.888 INFO:teuthology.orchestra.run.smithi112.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-09-21T00:26:09.888 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:26:09.888 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-21T00:26:09.953 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:26:09.953 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link delete ceph-brx 2024-09-21T00:26:10.058 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:26:10.058 DEBUG:teuthology.orchestra.run.smithi112:> ip netns list 2024-09-21T00:26:10.074 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:26:10.074 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link delete ceph-brx 2024-09-21T00:26:10.142 INFO:teuthology.orchestra.run.smithi112.stderr:Cannot find device "ceph-brx" 2024-09-21T00:26:10.143 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T00:26:10.143 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:26:10.143 DEBUG:teuthology.orchestra.run.smithi187:> ip netns list 2024-09-21T00:26:10.162 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:26:10.162 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip link delete ceph-brx 2024-09-21T00:26:10.235 INFO:teuthology.orchestra.run.smithi187.stderr:Cannot find device "ceph-brx" 2024-09-21T00:26:10.236 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T00:26:10.237 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:26:10.237 DEBUG:teuthology.orchestra.run.smithi187:> ip netns list 2024-09-21T00:26:10.295 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:26:10.295 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip link delete ceph-brx 2024-09-21T00:26:10.364 INFO:teuthology.orchestra.run.smithi187.stderr:Cannot find device "ceph-brx" 2024-09-21T00:26:10.365 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T00:26:10.366 INFO:tasks.kclient:client.0 config is {'syntax': 'v1'} 2024-09-21T00:26:10.366 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-21T00:26:10.366 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T00:26:10.366 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T00:26:10.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:10.480+0000 7fdb622de640 1 -- 172.21.15.112:0/1636262784 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdb5c0b2e40 msgr2=0x7fdb5c0b3250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:10.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:10.480+0000 7fdb622de640 1 --2- 172.21.15.112:0/1636262784 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdb5c0b2e40 0x7fdb5c0b3250 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fdb54005fd0 tx=0x7fdb5402aa90 comp rx=0 tx=0).stop 2024-09-21T00:26:10.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:10.480+0000 7fdb622de640 1 -- 172.21.15.112:0/1636262784 shutdown_connections 2024-09-21T00:26:10.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:10.480+0000 7fdb622de640 1 --2- 172.21.15.112:0/1636262784 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdb5c0b2e40 0x7fdb5c0b3250 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:10.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:10.480+0000 7fdb622de640 1 --2- 172.21.15.112:0/1636262784 >> v2:172.21.15.187:3300/0 conn(0x7fdb5c162f00 0x7fdb5c0d24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:10.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:10.480+0000 7fdb622de640 1 -- 172.21.15.112:0/1636262784 >> 172.21.15.112:0/1636262784 conn(0x7fdb5c0b6b20 msgr2=0x7fdb5c0d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:10.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:10.480+0000 7fdb622de640 1 -- 172.21.15.112:0/1636262784 shutdown_connections 2024-09-21T00:26:10.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:10.480+0000 7fdb622de640 1 -- 172.21.15.112:0/1636262784 wait complete. 2024-09-21T00:26:10.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:10.482+0000 7fdb622de640 1 Processor -- start 2024-09-21T00:26:10.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:10.482+0000 7fdb622de640 1 -- start start 2024-09-21T00:26:10.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:10.484+0000 7fdb622de640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdb5c0b2e40 0x7fdb5c0b2780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:10.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:10.484+0000 7fdb622de640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdb5c162280 0x7fdb5c15b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:10.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:10.484+0000 7fdb622de640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb5c0cea00 0x7fdb5c0cee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:10.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:10.484+0000 7fdb622de640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fdb5c15e630 con 0x7fdb5c0cea00 2024-09-21T00:26:10.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:10.484+0000 7fdb622de640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fdb5c15e4b0 con 0x7fdb5c0b2e40 2024-09-21T00:26:10.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:10.484+0000 7fdb622de640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fdb5c15e7b0 con 0x7fdb5c162280 2024-09-21T00:26:10.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:10.484+0000 7fdb6087b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb5c0cea00 0x7fdb5c0cee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:10.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:10.484+0000 7fdb6087b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb5c0cea00 0x7fdb5c0cee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60226/0 (socket says 172.21.15.112:60226) 2024-09-21T00:26:10.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:10.484+0000 7fdb6087b640 1 -- 172.21.15.112:0/1120100744 learned_addr learned my addr 172.21.15.112:0/1120100744 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:26:10.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:10.484+0000 7fdb5b7fe640 1 --2- 172.21.15.112:0/1120100744 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdb5c162280 0x7fdb5c15b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:10.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:10.484+0000 7fdb5bfff640 1 --2- 172.21.15.112:0/1120100744 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdb5c0b2e40 0x7fdb5c0b2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:10.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:10.484+0000 7fdb6087b640 1 -- 172.21.15.112:0/1120100744 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdb5c162280 msgr2=0x7fdb5c15b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:10.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:10.484+0000 7fdb6087b640 1 --2- 172.21.15.112:0/1120100744 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdb5c162280 0x7fdb5c15b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:10.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:10.484+0000 7fdb6087b640 1 -- 172.21.15.112:0/1120100744 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdb5c0b2e40 msgr2=0x7fdb5c0b2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:10.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:10.484+0000 7fdb6087b640 1 --2- 172.21.15.112:0/1120100744 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdb5c0b2e40 0x7fdb5c0b2780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:10.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:10.484+0000 7fdb6087b640 1 -- 172.21.15.112:0/1120100744 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb5c0cf350 con 0x7fdb5c0cea00 2024-09-21T00:26:10.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:10.484+0000 7fdb6087b640 1 --2- 172.21.15.112:0/1120100744 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb5c0cea00 0x7fdb5c0cee10 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fdb54009d90 tx=0x7fdb54003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:10.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:10.485+0000 7fdb58ff9640 1 -- 172.21.15.112:0/1120100744 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb54044070 con 0x7fdb5c0cea00 2024-09-21T00:26:10.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:10.485+0000 7fdb58ff9640 1 -- 172.21.15.112:0/1120100744 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdb54004040 con 0x7fdb5c0cea00 2024-09-21T00:26:10.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:10.485+0000 7fdb622de640 1 -- 172.21.15.112:0/1120100744 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb5c1f5520 con 0x7fdb5c0cea00 2024-09-21T00:26:10.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:10.485+0000 7fdb58ff9640 1 -- 172.21.15.112:0/1120100744 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb54004430 con 0x7fdb5c0cea00 2024-09-21T00:26:10.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:10.485+0000 7fdb622de640 1 -- 172.21.15.112:0/1120100744 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb5c1f5d00 con 0x7fdb5c0cea00 2024-09-21T00:26:10.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:10.486+0000 7fdb622de640 1 -- 172.21.15.112:0/1120100744 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb5c1f5790 con 0x7fdb5c0cea00 2024-09-21T00:26:10.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:10.486+0000 7fdb58ff9640 1 -- 172.21.15.112:0/1120100744 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fdb54047070 con 0x7fdb5c0cea00 2024-09-21T00:26:10.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:10.487+0000 7fdb58ff9640 1 --2- 172.21.15.112:0/1120100744 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fdb48078b90 0x7fdb4807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:10.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:10.487+0000 7fdb58ff9640 1 -- 172.21.15.112:0/1120100744 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fdb540bbe40 con 0x7fdb5c0cea00 2024-09-21T00:26:10.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:10.487+0000 7fdb5bfff640 1 --2- 172.21.15.112:0/1120100744 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fdb48078b90 0x7fdb4807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:10.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:10.488+0000 7fdb5bfff640 1 --2- 172.21.15.112:0/1120100744 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fdb48078b90 0x7fdb4807b0a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c009ac0 tx=0x7fdb4c005f90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:10.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:10.491+0000 7fdb58ff9640 1 -- 172.21.15.112:0/1120100744 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fdb5c0cf350 con 0x7fdb5c0cea00 2024-09-21T00:26:10.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:10.703+0000 7fdb622de640 1 -- 172.21.15.112:0/1120100744 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fdb5c162f00 con 0x7fdb5c0cea00 2024-09-21T00:26:10.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:10.704+0000 7fdb58ff9640 1 -- 172.21.15.112:0/1120100744 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7fdb54032030 con 0x7fdb5c0cea00 2024-09-21T00:26:10.705 INFO:teuthology.orchestra.run.smithi112.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-21T00:26:10.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:10.706+0000 7fdb622de640 1 -- 172.21.15.112:0/1120100744 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fdb48078b90 msgr2=0x7fdb4807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:10.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:10.706+0000 7fdb622de640 1 --2- 172.21.15.112:0/1120100744 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fdb48078b90 0x7fdb4807b0a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c009ac0 tx=0x7fdb4c005f90 comp rx=0 tx=0).stop 2024-09-21T00:26:10.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:10.706+0000 7fdb622de640 1 -- 172.21.15.112:0/1120100744 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb5c0cea00 msgr2=0x7fdb5c0cee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:10.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:10.706+0000 7fdb622de640 1 --2- 172.21.15.112:0/1120100744 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb5c0cea00 0x7fdb5c0cee10 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fdb54009d90 tx=0x7fdb54003ce0 comp rx=0 tx=0).stop 2024-09-21T00:26:10.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:10.706+0000 7fdb622de640 1 -- 172.21.15.112:0/1120100744 shutdown_connections 2024-09-21T00:26:10.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:10.706+0000 7fdb622de640 1 --2- 172.21.15.112:0/1120100744 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fdb48078b90 0x7fdb4807b0a0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:10.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:10.706+0000 7fdb622de640 1 --2- 172.21.15.112:0/1120100744 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb5c0cea00 0x7fdb5c0cee10 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:10.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:10.706+0000 7fdb622de640 1 --2- 172.21.15.112:0/1120100744 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdb5c162280 0x7fdb5c15b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:10.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:10.706+0000 7fdb622de640 1 --2- 172.21.15.112:0/1120100744 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdb5c0b2e40 0x7fdb5c0b2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:10.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:10.706+0000 7fdb622de640 1 -- 172.21.15.112:0/1120100744 >> 172.21.15.112:0/1120100744 conn(0x7fdb5c0b6b20 msgr2=0x7fdb5c15f0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:10.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:10.706+0000 7fdb622de640 1 -- 172.21.15.112:0/1120100744 shutdown_connections 2024-09-21T00:26:10.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:10.706+0000 7fdb622de640 1 -- 172.21.15.112:0/1120100744 wait complete. 2024-09-21T00:26:10.718 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-21T00:26:10.718 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T00:26:10.718 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi112.front.sepia.ceph.com 2024-09-21T00:26:10.718 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T00:26:10.718 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T00:26:10.719 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-21T00:26:10.719 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-21T00:26:10.719 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T00:26:10.719 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T00:26:10.719 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:26:10.719 DEBUG:teuthology.orchestra.run.smithi112:> ip addr 2024-09-21T00:26:10.734 INFO:teuthology.orchestra.run.smithi112.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T00:26:10.734 INFO:teuthology.orchestra.run.smithi112.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T00:26:10.735 INFO:teuthology.orchestra.run.smithi112.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T00:26:10.735 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:26:10.735 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 ::1/128 scope host 2024-09-21T00:26:10.735 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:26:10.735 INFO:teuthology.orchestra.run.smithi112.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T00:26:10.735 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:d8:a5:2c brd ff:ff:ff:ff:ff:ff 2024-09-21T00:26:10.735 INFO:teuthology.orchestra.run.smithi112.stdout: altname enp7s0f0 2024-09-21T00:26:10.735 INFO:teuthology.orchestra.run.smithi112.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T00:26:10.735 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:d8:a5:2d brd ff:ff:ff:ff:ff:ff 2024-09-21T00:26:10.735 INFO:teuthology.orchestra.run.smithi112.stdout: altname enp7s0f1 2024-09-21T00:26:10.735 INFO:teuthology.orchestra.run.smithi112.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T00:26:10.735 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:8f:cb:68 brd ff:ff:ff:ff:ff:ff 2024-09-21T00:26:10.736 INFO:teuthology.orchestra.run.smithi112.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T00:26:10.736 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:8f:cb:69 brd ff:ff:ff:ff:ff:ff 2024-09-21T00:26:10.736 INFO:teuthology.orchestra.run.smithi112.stdout: inet 172.21.15.112/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-21T00:26:10.736 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft 42751sec preferred_lft 42751sec 2024-09-21T00:26:10.736 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::ec4:7aff:fe8f:cb69/64 scope link noprefixroute 2024-09-21T00:26:10.736 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:26:10.736 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-21T00:26:10.736 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T00:26:10.736 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2024-09-21T00:26:10.736 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link add name ceph-brx type bridge 2024-09-21T00:26:10.736 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip addr flush dev ceph-brx 2024-09-21T00:26:10.736 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link set ceph-brx up 2024-09-21T00:26:10.737 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-21T00:26:10.737 DEBUG:teuthology.orchestra.run.smithi112:> ') 2024-09-21T00:26:10.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:10 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T00:26:10.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T00:26:10.873 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:26:10.874 DEBUG:teuthology.orchestra.run.smithi112:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T00:26:10.896 INFO:teuthology.orchestra.run.smithi112.stdout:1 2024-09-21T00:26:10.898 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:26:10.898 DEBUG:teuthology.orchestra.run.smithi112:> route 2024-09-21T00:26:10.954 INFO:teuthology.orchestra.run.smithi112.stdout:Kernel IP routing table 2024-09-21T00:26:10.955 INFO:teuthology.orchestra.run.smithi112.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T00:26:10.955 INFO:teuthology.orchestra.run.smithi112.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-21T00:26:10.955 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-21T00:26:10.955 INFO:teuthology.orchestra.run.smithi112.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T00:26:10.955 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T00:26:10.955 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2024-09-21T00:26:10.955 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-21T00:26:10.955 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-21T00:26:10.955 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-21T00:26:10.956 DEBUG:teuthology.orchestra.run.smithi112:> ') 2024-09-21T00:26:11.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T00:26:11.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T00:26:11.080 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:26:11.080 DEBUG:teuthology.orchestra.run.smithi112:> ip netns list 2024-09-21T00:26:11.095 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:26:11.095 DEBUG:teuthology.orchestra.run.smithi112:> ip netns list-id 2024-09-21T00:26:11.150 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T00:26:11.150 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2024-09-21T00:26:11.151 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T00:26:11.151 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-21T00:26:11.151 DEBUG:teuthology.orchestra.run.smithi112:> ') 2024-09-21T00:26:11.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T00:26:11.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T00:26:11.268 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-21T00:26:11.268 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T00:26:11.268 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2024-09-21T00:26:11.268 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-21T00:26:11.268 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-21T00:26:11.268 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-21T00:26:11.268 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-21T00:26:11.268 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-21T00:26:11.268 DEBUG:teuthology.orchestra.run.smithi112:> ') 2024-09-21T00:26:11.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T00:26:11.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T00:26:11.381 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T00:26:11.381 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2024-09-21T00:26:11.381 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link set brx.0 up 2024-09-21T00:26:11.381 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link set dev brx.0 master ceph-brx 2024-09-21T00:26:11.381 DEBUG:teuthology.orchestra.run.smithi112:> ') 2024-09-21T00:26:11.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T00:26:11.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T00:26:11.457 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T00:26:11.458 DEBUG:teuthology.orchestra.run.smithi112:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T00:26:11.472 INFO:teuthology.orchestra.run.smithi112.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T00:26:11.473 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T00:26:11.473 DEBUG:teuthology.orchestra.run.smithi112:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T00:26:11.527 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-21T00:26:11.527 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:26:11.527 DEBUG:teuthology.orchestra.run.smithi112:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T00:26:11.621 INFO:teuthology.orchestra.run.smithi112.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T00:26:11.622 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T00:26:11.622 INFO:teuthology.orchestra.run.smithi112.stdout:invalid new device string format 2024-09-21T00:26:11.762 INFO:teuthology.orchestra.run.smithi112.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T00:26:11.762 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T00:26:11.762 INFO:teuthology.orchestra.run.smithi112.stdout:invalid new device string format 2024-09-21T00:26:11.762 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: resolved to: "172.21.15.112:3300,172.21.15.187:3300,172.21.15.187:3301" 2024-09-21T00:26:11.762 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T00:26:11.768 INFO:teuthology.orchestra.run.smithi112.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T00:26:11.768 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T00:26:11.768 INFO:teuthology.orchestra.run.smithi112.stdout:invalid new device string format 2024-09-21T00:26:11.768 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: resolved to: "172.21.15.112:3300,172.21.15.187:3300,172.21.15.187:3301" 2024-09-21T00:26:11.768 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T00:26:11.768 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: resolved to: "172.21.15.112:6789,172.21.15.187:6789,172.21.15.187:6790" 2024-09-21T00:26:11.768 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.112:6789,172.21.15.187:6789,172.21.15.187:6790:/ 2024-09-21T00:26:11.769 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=8b34692b-dc65-49b2-a08f-26f707b96ac5" will pass to kernel 2024-09-21T00:26:11.769 INFO:teuthology.orchestra.run.smithi112.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-21T00:26:11.769 INFO:teuthology.orchestra.run.smithi112.stdout: You just mounted a file system that supports labels which does not 2024-09-21T00:26:11.769 INFO:teuthology.orchestra.run.smithi112.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T00:26:11.769 INFO:teuthology.orchestra.run.smithi112.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T00:26:11.769 INFO:teuthology.orchestra.run.smithi112.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T00:26:11.771 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T00:26:11.771 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:26:11.771 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T00:26:11.795 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T00:26:11.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:11 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T00:26:11.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T00:26:11.891 INFO:teuthology.orchestra.run.smithi112.stdout:client4598 2024-09-21T00:26:11.892 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T00:26:11.892 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:26:11.907 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T00:26:11.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:11 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T00:26:11.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T00:26:12.003 INFO:teuthology.orchestra.run.smithi112.stdout:client4598 2024-09-21T00:26:12.003 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T00:26:12.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:12 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T00:26:12.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T00:26:12.058 INFO:teuthology.orchestra.run.smithi112.stdout:client4598 2024-09-21T00:26:12.058 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/8b34692b-dc65-49b2-a08f-26f707b96ac5.client4598/status') 2024-09-21T00:26:12.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:12 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T00:26:12.116 INFO:teuthology.orchestra.run.smithi112.stdout:instance: client.4598 (3)172.21.15.112:0/609281481 2024-09-21T00:26:12.116 INFO:teuthology.orchestra.run.smithi112.stdout:blocklisted: false 2024-09-21T00:26:12.116 INFO:teuthology.orchestra.run.smithi112.stderr:0+1 records in 2024-09-21T00:26:12.116 INFO:teuthology.orchestra.run.smithi112.stderr:0+1 records out 2024-09-21T00:26:12.116 INFO:teuthology.orchestra.run.smithi112.stderr:70 bytes copied, 3.844e-05 s, 1.8 MB/s 2024-09-21T00:26:12.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T00:26:12.123 INFO:tasks.kclient:client.2 config is {'syntax': 'v1'} 2024-09-21T00:26:12.123 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-21T00:26:12.123 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-09-21T00:26:12.123 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T00:26:12.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:12.234+0000 7f5a877ba640 1 -- 172.21.15.112:0/2578939178 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a80162f00 msgr2=0x7f5a800d24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:12.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:12.234+0000 7f5a877ba640 1 --2- 172.21.15.112:0/2578939178 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a80162f00 0x7f5a800d24d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f5a7c009830 tx=0x7f5a7c02a510 comp rx=0 tx=0).stop 2024-09-21T00:26:12.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:12.235+0000 7f5a877ba640 1 -- 172.21.15.112:0/2578939178 shutdown_connections 2024-09-21T00:26:12.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:12.235+0000 7f5a877ba640 1 --2- 172.21.15.112:0/2578939178 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5a80162280 0x7f5a80162690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:12.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:12.235+0000 7f5a877ba640 1 --2- 172.21.15.112:0/2578939178 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a80162f00 0x7f5a800d24d0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:12.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:12.235+0000 7f5a877ba640 1 -- 172.21.15.112:0/2578939178 >> 172.21.15.112:0/2578939178 conn(0x7f5a800b6b20 msgr2=0x7f5a800d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:12.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:12.235+0000 7f5a877ba640 1 -- 172.21.15.112:0/2578939178 shutdown_connections 2024-09-21T00:26:12.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:12.235+0000 7f5a877ba640 1 -- 172.21.15.112:0/2578939178 wait complete. 2024-09-21T00:26:12.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:12.236+0000 7f5a877ba640 1 Processor -- start 2024-09-21T00:26:12.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:12.236+0000 7f5a877ba640 1 -- start start 2024-09-21T00:26:12.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:12.238+0000 7f5a877ba640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a800b2e40 0x7f5a800ced20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:12.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:12.238+0000 7f5a877ba640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5a80162280 0x7f5a800ad250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:12.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:12.238+0000 7f5a877ba640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5a8014cce0 0x7f5a8014ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:12.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:12.238+0000 7f5a877ba640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5a8015e630 con 0x7f5a800b2e40 2024-09-21T00:26:12.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:12.238+0000 7f5a877ba640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5a8015e4b0 con 0x7f5a80162280 2024-09-21T00:26:12.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:12.238+0000 7f5a877ba640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f5a8015e7b0 con 0x7f5a8014cce0 2024-09-21T00:26:12.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:12.238+0000 7f5a85556640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a800b2e40 0x7f5a800ced20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:12.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:12.238+0000 7f5a85556640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a800b2e40 0x7f5a800ced20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60246/0 (socket says 172.21.15.112:60246) 2024-09-21T00:26:12.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:12.238+0000 7f5a85556640 1 -- 172.21.15.112:0/4156690056 learned_addr learned my addr 172.21.15.112:0/4156690056 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:26:12.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:12.238+0000 7f5a84d55640 1 --2- 172.21.15.112:0/4156690056 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5a80162280 0x7f5a800ad250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:12.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:12.238+0000 7f5a85d57640 1 --2- 172.21.15.112:0/4156690056 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5a8014cce0 0x7f5a8014ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:12.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:12.238+0000 7f5a85556640 1 -- 172.21.15.112:0/4156690056 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5a8014cce0 msgr2=0x7f5a8014ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:12.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:12.238+0000 7f5a85556640 1 --2- 172.21.15.112:0/4156690056 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5a8014cce0 0x7f5a8014ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:12.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:12.238+0000 7f5a85556640 1 -- 172.21.15.112:0/4156690056 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5a80162280 msgr2=0x7f5a800ad250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:12.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:12.238+0000 7f5a85556640 1 --2- 172.21.15.112:0/4156690056 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5a80162280 0x7f5a800ad250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:12.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:12.238+0000 7f5a85556640 1 -- 172.21.15.112:0/4156690056 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a800cf260 con 0x7f5a800b2e40 2024-09-21T00:26:12.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:12.238+0000 7f5a85556640 1 --2- 172.21.15.112:0/4156690056 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a800b2e40 0x7f5a800ced20 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f5a7c000c00 tx=0x7f5a7c033ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:12.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:12.239+0000 7f5a75ffb640 1 -- 172.21.15.112:0/4156690056 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a7c031b50 con 0x7f5a800b2e40 2024-09-21T00:26:12.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:12.239+0000 7f5a75ffb640 1 -- 172.21.15.112:0/4156690056 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5a7c031de0 con 0x7f5a800b2e40 2024-09-21T00:26:12.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:12.239+0000 7f5a877ba640 1 -- 172.21.15.112:0/4156690056 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a800ac1b0 con 0x7f5a800b2e40 2024-09-21T00:26:12.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:12.239+0000 7f5a75ffb640 1 -- 172.21.15.112:0/4156690056 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a7c0332d0 con 0x7f5a800b2e40 2024-09-21T00:26:12.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:12.239+0000 7f5a877ba640 1 -- 172.21.15.112:0/4156690056 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a800ac770 con 0x7f5a800b2e40 2024-09-21T00:26:12.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:12.240+0000 7f5a877ba640 1 -- 172.21.15.112:0/4156690056 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a800d03d0 con 0x7f5a800b2e40 2024-09-21T00:26:12.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:12.241+0000 7f5a75ffb640 1 -- 172.21.15.112:0/4156690056 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5a7c033470 con 0x7f5a800b2e40 2024-09-21T00:26:12.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:12.241+0000 7f5a75ffb640 1 --2- 172.21.15.112:0/4156690056 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f5a64078b90 0x7f5a6407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:12.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:12.241+0000 7f5a75ffb640 1 -- 172.21.15.112:0/4156690056 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f5a7c0bc270 con 0x7f5a800b2e40 2024-09-21T00:26:12.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:12.241+0000 7f5a84d55640 1 --2- 172.21.15.112:0/4156690056 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f5a64078b90 0x7f5a6407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:12.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:12.242+0000 7f5a84d55640 1 --2- 172.21.15.112:0/4156690056 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f5a64078b90 0x7f5a6407b0a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5a8006a860 tx=0x7f5a70005f90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:12.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:12.246+0000 7f5a75ffb640 1 -- 172.21.15.112:0/4156690056 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5a800d03d0 con 0x7f5a800b2e40 2024-09-21T00:26:12.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:12.461+0000 7f5a877ba640 1 -- 172.21.15.112:0/4156690056 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f5a800b34a0 con 0x7f5a800b2e40 2024-09-21T00:26:12.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:12.462+0000 7f5a75ffb640 1 -- 172.21.15.112:0/4156690056 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7f5a7c03f030 con 0x7f5a800b2e40 2024-09-21T00:26:12.463 INFO:teuthology.orchestra.run.smithi112.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-21T00:26:12.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:12.464+0000 7f5a877ba640 1 -- 172.21.15.112:0/4156690056 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f5a64078b90 msgr2=0x7f5a6407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:12.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:12.464+0000 7f5a877ba640 1 --2- 172.21.15.112:0/4156690056 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f5a64078b90 0x7f5a6407b0a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5a8006a860 tx=0x7f5a70005f90 comp rx=0 tx=0).stop 2024-09-21T00:26:12.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:12.464+0000 7f5a877ba640 1 -- 172.21.15.112:0/4156690056 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a800b2e40 msgr2=0x7f5a800ced20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:12.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:12.464+0000 7f5a877ba640 1 --2- 172.21.15.112:0/4156690056 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a800b2e40 0x7f5a800ced20 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f5a7c000c00 tx=0x7f5a7c033ba0 comp rx=0 tx=0).stop 2024-09-21T00:26:12.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:12.464+0000 7f5a877ba640 1 -- 172.21.15.112:0/4156690056 shutdown_connections 2024-09-21T00:26:12.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:12.464+0000 7f5a877ba640 1 --2- 172.21.15.112:0/4156690056 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5a8014cce0 0x7f5a8014ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:12.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:12.464+0000 7f5a877ba640 1 --2- 172.21.15.112:0/4156690056 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5a80162280 0x7f5a800ad250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:12.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:12.464+0000 7f5a877ba640 1 --2- 172.21.15.112:0/4156690056 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f5a64078b90 0x7f5a6407b0a0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:12.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:12.464+0000 7f5a877ba640 1 --2- 172.21.15.112:0/4156690056 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a800b2e40 0x7f5a800ced20 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:12.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:12.464+0000 7f5a877ba640 1 -- 172.21.15.112:0/4156690056 >> 172.21.15.112:0/4156690056 conn(0x7f5a800b6b20 msgr2=0x7f5a800b3be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:12.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:12.464+0000 7f5a877ba640 1 -- 172.21.15.112:0/4156690056 shutdown_connections 2024-09-21T00:26:12.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:12.464+0000 7f5a877ba640 1 -- 172.21.15.112:0/4156690056 wait complete. 2024-09-21T00:26:12.476 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-21T00:26:12.476 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T00:26:12.476 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi112.front.sepia.ceph.com 2024-09-21T00:26:12.476 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-09-21T00:26:12.476 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-09-21T00:26:12.476 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-21T00:26:12.476 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-21T00:26:12.476 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T00:26:12.476 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-09-21T00:26:12.476 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:26:12.476 DEBUG:teuthology.orchestra.run.smithi112:> ip addr 2024-09-21T00:26:12.492 INFO:teuthology.orchestra.run.smithi112.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T00:26:12.492 INFO:teuthology.orchestra.run.smithi112.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T00:26:12.492 INFO:teuthology.orchestra.run.smithi112.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T00:26:12.492 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:26:12.492 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 ::1/128 scope host 2024-09-21T00:26:12.492 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:26:12.492 INFO:teuthology.orchestra.run.smithi112.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T00:26:12.492 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:d8:a5:2c brd ff:ff:ff:ff:ff:ff 2024-09-21T00:26:12.492 INFO:teuthology.orchestra.run.smithi112.stdout: altname enp7s0f0 2024-09-21T00:26:12.492 INFO:teuthology.orchestra.run.smithi112.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T00:26:12.493 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:d8:a5:2d brd ff:ff:ff:ff:ff:ff 2024-09-21T00:26:12.493 INFO:teuthology.orchestra.run.smithi112.stdout: altname enp7s0f1 2024-09-21T00:26:12.493 INFO:teuthology.orchestra.run.smithi112.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T00:26:12.493 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:8f:cb:68 brd ff:ff:ff:ff:ff:ff 2024-09-21T00:26:12.493 INFO:teuthology.orchestra.run.smithi112.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T00:26:12.493 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:8f:cb:69 brd ff:ff:ff:ff:ff:ff 2024-09-21T00:26:12.493 INFO:teuthology.orchestra.run.smithi112.stdout: inet 172.21.15.112/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-21T00:26:12.493 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft 42750sec preferred_lft 42750sec 2024-09-21T00:26:12.493 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::ec4:7aff:fe8f:cb69/64 scope link noprefixroute 2024-09-21T00:26:12.493 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:26:12.493 INFO:teuthology.orchestra.run.smithi112.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T00:26:12.493 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 1a:39:64:e7:fe:d7 brd ff:ff:ff:ff:ff:ff 2024-09-21T00:26:12.493 INFO:teuthology.orchestra.run.smithi112.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T00:26:12.493 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:26:12.494 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::5858:3dff:feaf:1774/64 scope link 2024-09-21T00:26:12.494 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:26:12.494 INFO:teuthology.orchestra.run.smithi112.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T00:26:12.494 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 1a:39:64:e7:fe:d7 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T00:26:12.494 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::1839:64ff:fee7:fed7/64 scope link tentative 2024-09-21T00:26:12.494 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:26:12.494 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:26:12.494 DEBUG:teuthology.orchestra.run.smithi112:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T00:26:12.556 INFO:teuthology.orchestra.run.smithi112.stdout:1 2024-09-21T00:26:12.557 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:26:12.557 DEBUG:teuthology.orchestra.run.smithi112:> route 2024-09-21T00:26:12.613 INFO:teuthology.orchestra.run.smithi112.stdout:Kernel IP routing table 2024-09-21T00:26:12.613 INFO:teuthology.orchestra.run.smithi112.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T00:26:12.613 INFO:teuthology.orchestra.run.smithi112.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-21T00:26:12.613 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-21T00:26:12.613 INFO:teuthology.orchestra.run.smithi112.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T00:26:12.614 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T00:26:12.614 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2024-09-21T00:26:12.614 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-21T00:26:12.614 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-21T00:26:12.614 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-21T00:26:12.614 DEBUG:teuthology.orchestra.run.smithi112:> ') 2024-09-21T00:26:12.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:12 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T00:26:12.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T00:26:12.739 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:26:12.739 DEBUG:teuthology.orchestra.run.smithi112:> ip netns list 2024-09-21T00:26:12.754 INFO:teuthology.orchestra.run.smithi112.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T00:26:12.754 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:26:12.754 DEBUG:teuthology.orchestra.run.smithi112:> ip netns list-id 2024-09-21T00:26:12.810 INFO:teuthology.orchestra.run.smithi112.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-09-21T00:26:12.810 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T00:26:12.810 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2024-09-21T00:26:12.810 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T00:26:12.810 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-09-21T00:26:12.810 DEBUG:teuthology.orchestra.run.smithi112:> ') 2024-09-21T00:26:12.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:12 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T00:26:12.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T00:26:12.927 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:26:12.927 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-09-21T00:26:12.951 INFO:teuthology.orchestra.run.smithi112.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T00:26:12.951 INFO:teuthology.orchestra.run.smithi112.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T00:26:12.951 INFO:teuthology.orchestra.run.smithi112.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T00:26:12.951 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:26:12.951 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 ::1/128 scope host 2024-09-21T00:26:12.951 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:26:12.952 INFO:teuthology.orchestra.run.smithi112.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T00:26:12.952 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 22:f4:60:3a:39:3d brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-21T00:26:12.952 INFO:teuthology.orchestra.run.smithi112.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-21T00:26:12.952 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:26:12.952 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::20f4:60ff:fe3a:393d/64 scope link 2024-09-21T00:26:12.952 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:26:12.953 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:26:12.953 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-09-21T00:26:13.017 INFO:teuthology.orchestra.run.smithi112.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T00:26:13.018 INFO:teuthology.orchestra.run.smithi112.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T00:26:13.018 INFO:teuthology.orchestra.run.smithi112.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T00:26:13.018 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:26:13.018 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 ::1/128 scope host 2024-09-21T00:26:13.018 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:26:13.018 INFO:teuthology.orchestra.run.smithi112.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T00:26:13.018 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 22:f4:60:3a:39:3d brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-21T00:26:13.018 INFO:teuthology.orchestra.run.smithi112.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-21T00:26:13.018 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:26:13.018 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::20f4:60ff:fe3a:393d/64 scope link 2024-09-21T00:26:13.018 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:26:13.019 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-09-21T00:26:13.019 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T00:26:13.020 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2024-09-21T00:26:13.020 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-09-21T00:26:13.020 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-09-21T00:26:13.020 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-09-21T00:26:13.020 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-09-21T00:26:13.020 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-09-21T00:26:13.020 DEBUG:teuthology.orchestra.run.smithi112:> ') 2024-09-21T00:26:13.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:13 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T00:26:13.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T00:26:13.170 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T00:26:13.170 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2024-09-21T00:26:13.170 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link set brx.1 up 2024-09-21T00:26:13.171 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link set dev brx.1 master ceph-brx 2024-09-21T00:26:13.171 DEBUG:teuthology.orchestra.run.smithi112:> ') 2024-09-21T00:26:13.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:13 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T00:26:13.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T00:26:13.247 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T00:26:13.247 DEBUG:teuthology.orchestra.run.smithi112:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-09-21T00:26:13.261 INFO:teuthology.orchestra.run.smithi112.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-09-21T00:26:13.261 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T00:26:13.262 DEBUG:teuthology.orchestra.run.smithi112:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-09-21T00:26:13.316 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-21T00:26:13.316 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:26:13.316 DEBUG:teuthology.orchestra.run.smithi112:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-09-21T00:26:13.411 INFO:teuthology.orchestra.run.smithi112.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-09-21T00:26:13.411 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-09-21T00:26:13.411 INFO:teuthology.orchestra.run.smithi112.stdout:invalid new device string format 2024-09-21T00:26:13.421 INFO:teuthology.orchestra.run.smithi112.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-09-21T00:26:13.422 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-09-21T00:26:13.422 INFO:teuthology.orchestra.run.smithi112.stdout:invalid new device string format 2024-09-21T00:26:13.422 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: resolved to: "172.21.15.112:3300,172.21.15.187:3300,172.21.15.187:3301" 2024-09-21T00:26:13.422 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T00:26:13.441 INFO:teuthology.orchestra.run.smithi112.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-09-21T00:26:13.441 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-09-21T00:26:13.441 INFO:teuthology.orchestra.run.smithi112.stdout:invalid new device string format 2024-09-21T00:26:13.441 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: resolved to: "172.21.15.112:3300,172.21.15.187:3300,172.21.15.187:3301" 2024-09-21T00:26:13.441 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T00:26:13.441 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: resolved to: "172.21.15.112:6789,172.21.15.187:6789,172.21.15.187:6790" 2024-09-21T00:26:13.441 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.112:6789,172.21.15.187:6789,172.21.15.187:6790:/ 2024-09-21T00:26:13.441 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs,key=2,fsid=8b34692b-dc65-49b2-a08f-26f707b96ac5" will pass to kernel 2024-09-21T00:26:13.442 INFO:teuthology.orchestra.run.smithi112.stdout:mount: /home/ubuntu/cephtest/mnt.2 does not contain SELinux labels. 2024-09-21T00:26:13.442 INFO:teuthology.orchestra.run.smithi112.stdout: You just mounted a file system that supports labels which does not 2024-09-21T00:26:13.442 INFO:teuthology.orchestra.run.smithi112.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T00:26:13.442 INFO:teuthology.orchestra.run.smithi112.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T00:26:13.442 INFO:teuthology.orchestra.run.smithi112.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T00:26:13.444 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T00:26:13.444 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:26:13.444 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-09-21T00:26:13.468 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T00:26:13.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:13 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T00:26:13.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T00:26:13.565 INFO:teuthology.orchestra.run.smithi112.stdout:client4607 2024-09-21T00:26:13.565 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T00:26:13.565 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:26:13.580 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T00:26:13.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:13 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T00:26:13.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T00:26:13.675 INFO:teuthology.orchestra.run.smithi112.stdout:client4607 2024-09-21T00:26:13.676 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T00:26:13.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:13 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T00:26:13.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T00:26:13.732 INFO:teuthology.orchestra.run.smithi112.stdout:client4607 2024-09-21T00:26:13.732 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/8b34692b-dc65-49b2-a08f-26f707b96ac5.client4607/status') 2024-09-21T00:26:13.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:13 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T00:26:13.790 INFO:teuthology.orchestra.run.smithi112.stdout:instance: client.4607 (3)172.21.15.112:0/1682249439 2024-09-21T00:26:13.790 INFO:teuthology.orchestra.run.smithi112.stdout:blocklisted: false 2024-09-21T00:26:13.790 INFO:teuthology.orchestra.run.smithi112.stderr:0+1 records in 2024-09-21T00:26:13.790 INFO:teuthology.orchestra.run.smithi112.stderr:0+1 records out 2024-09-21T00:26:13.790 INFO:teuthology.orchestra.run.smithi112.stderr:71 bytes copied, 2.0473e-05 s, 3.5 MB/s 2024-09-21T00:26:13.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T00:26:13.797 INFO:tasks.kclient:client.1 config is {'syntax': 'v1'} 2024-09-21T00:26:13.797 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-21T00:26:13.797 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-21T00:26:13.797 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T00:26:13.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:13.909+0000 7fda485c3640 1 -- 172.21.15.112:0/546973692 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda2c004db0 msgr2=0x7fda2c005280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:13.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:13.909+0000 7fda485c3640 1 --2- 172.21.15.112:0/546973692 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda2c004db0 0x7fda2c005280 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fda38006fd0 tx=0x7fda2c005830 comp rx=0 tx=0).stop 2024-09-21T00:26:13.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:13.909+0000 7fda46b60640 1 -- 172.21.15.112:0/546973692 reap_dead start 2024-09-21T00:26:13.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:13.910+0000 7fda485c3640 1 -- 172.21.15.112:0/546973692 shutdown_connections 2024-09-21T00:26:13.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:13.910+0000 7fda485c3640 1 -- 172.21.15.112:0/546973692 >> 172.21.15.112:0/546973692 conn(0x7fda400d2350 msgr2=0x7fda400d2760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:13.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:13.910+0000 7fda485c3640 1 -- 172.21.15.112:0/546973692 shutdown_connections 2024-09-21T00:26:13.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:13.910+0000 7fda485c3640 1 -- 172.21.15.112:0/546973692 wait complete. 2024-09-21T00:26:13.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:13.911+0000 7fda485c3640 1 Processor -- start 2024-09-21T00:26:13.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:13.911+0000 7fda485c3640 1 -- start start 2024-09-21T00:26:13.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:13.913+0000 7fda485c3640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fda2c004db0 0x7fda400c9f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:13.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:13.913+0000 7fda485c3640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fda401438f0 0x7fda40145b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:13.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:13.913+0000 7fda485c3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda40146080 0x7fda401f6710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:13.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:13.913+0000 7fda485c3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fda4015d2b0 con 0x7fda40146080 2024-09-21T00:26:13.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:13.913+0000 7fda485c3640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fda4015d130 con 0x7fda401438f0 2024-09-21T00:26:13.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:13.913+0000 7fda485c3640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fda400b6d70 con 0x7fda2c004db0 2024-09-21T00:26:13.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:13.913+0000 7fda46b60640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda40146080 0x7fda401f6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:13.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:13.913+0000 7fda46b60640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda40146080 0x7fda401f6710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60266/0 (socket says 172.21.15.112:60266) 2024-09-21T00:26:13.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:13.913+0000 7fda46b60640 1 -- 172.21.15.112:0/3776221690 learned_addr learned my addr 172.21.15.112:0/3776221690 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:26:13.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:13.913+0000 7fda4635f640 1 --2- 172.21.15.112:0/3776221690 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fda2c004db0 0x7fda400c9f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:13.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:13.913+0000 7fda45b5e640 1 --2- 172.21.15.112:0/3776221690 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fda401438f0 0x7fda40145b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:13.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:13.913+0000 7fda46b60640 1 -- 172.21.15.112:0/3776221690 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fda2c004db0 msgr2=0x7fda400c9f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:13.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:13.913+0000 7fda46b60640 1 --2- 172.21.15.112:0/3776221690 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fda2c004db0 0x7fda400c9f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:13.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:13.913+0000 7fda46b60640 1 -- 172.21.15.112:0/3776221690 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fda401438f0 msgr2=0x7fda40145b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:13.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:13.913+0000 7fda46b60640 1 --2- 172.21.15.112:0/3776221690 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fda401438f0 0x7fda40145b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:13.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:13.913+0000 7fda46b60640 1 -- 172.21.15.112:0/3776221690 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda400c9bb0 con 0x7fda40146080 2024-09-21T00:26:13.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:13.913+0000 7fda4635f640 1 --2- 172.21.15.112:0/3776221690 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fda2c004db0 0x7fda400c9f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:26:13.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:13.913+0000 7fda46b60640 1 --2- 172.21.15.112:0/3776221690 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda40146080 0x7fda401f6710 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fda3800c6f0 tx=0x7fda38001aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:13.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:13.913+0000 7fda36ffd640 1 -- 172.21.15.112:0/3776221690 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda38039070 con 0x7fda40146080 2024-09-21T00:26:13.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:13.913+0000 7fda36ffd640 1 -- 172.21.15.112:0/3776221690 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fda3802bbb0 con 0x7fda40146080 2024-09-21T00:26:13.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:13.913+0000 7fda485c3640 1 -- 172.21.15.112:0/3776221690 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda4015a360 con 0x7fda40146080 2024-09-21T00:26:13.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:13.913+0000 7fda36ffd640 1 -- 172.21.15.112:0/3776221690 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda3800c280 con 0x7fda40146080 2024-09-21T00:26:13.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:13.913+0000 7fda485c3640 1 -- 172.21.15.112:0/3776221690 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda4015a920 con 0x7fda40146080 2024-09-21T00:26:13.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:13.914+0000 7fda485c3640 1 -- 172.21.15.112:0/3776221690 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda400b72d0 con 0x7fda40146080 2024-09-21T00:26:13.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:13.915+0000 7fda36ffd640 1 -- 172.21.15.112:0/3776221690 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fda38006140 con 0x7fda40146080 2024-09-21T00:26:13.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:13.916+0000 7fda36ffd640 1 --2- 172.21.15.112:0/3776221690 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fda24078bc0 0x7fda2407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:13.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:13.916+0000 7fda36ffd640 1 -- 172.21.15.112:0/3776221690 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fda380baae0 con 0x7fda40146080 2024-09-21T00:26:13.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:13.916+0000 7fda4635f640 1 --2- 172.21.15.112:0/3776221690 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fda24078bc0 0x7fda2407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:13.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:13.916+0000 7fda4635f640 1 --2- 172.21.15.112:0/3776221690 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fda24078bc0 0x7fda2407b0d0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fda3c000c00 tx=0x7fda3c006b70 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:13.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:13.920+0000 7fda36ffd640 1 -- 172.21.15.112:0/3776221690 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fda38082d80 con 0x7fda40146080 2024-09-21T00:26:14.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:14.134+0000 7fda485c3640 1 -- 172.21.15.112:0/3776221690 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fda40146640 con 0x7fda40146080 2024-09-21T00:26:14.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:14.134+0000 7fda36ffd640 1 -- 172.21.15.112:0/3776221690 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7fda38082680 con 0x7fda40146080 2024-09-21T00:26:14.135 INFO:teuthology.orchestra.run.smithi112.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-21T00:26:14.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:14.136+0000 7fda485c3640 1 -- 172.21.15.112:0/3776221690 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fda24078bc0 msgr2=0x7fda2407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:14.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:14.136+0000 7fda485c3640 1 --2- 172.21.15.112:0/3776221690 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fda24078bc0 0x7fda2407b0d0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fda3c000c00 tx=0x7fda3c006b70 comp rx=0 tx=0).stop 2024-09-21T00:26:14.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:14.136+0000 7fda485c3640 1 -- 172.21.15.112:0/3776221690 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda40146080 msgr2=0x7fda401f6710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:14.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:14.136+0000 7fda485c3640 1 --2- 172.21.15.112:0/3776221690 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda40146080 0x7fda401f6710 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fda3800c6f0 tx=0x7fda38001aa0 comp rx=0 tx=0).stop 2024-09-21T00:26:14.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:14.137+0000 7fda485c3640 1 -- 172.21.15.112:0/3776221690 shutdown_connections 2024-09-21T00:26:14.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:14.137+0000 7fda485c3640 1 --2- 172.21.15.112:0/3776221690 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fda24078bc0 0x7fda2407b0d0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:14.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:14.137+0000 7fda485c3640 1 --2- 172.21.15.112:0/3776221690 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda40146080 0x7fda401f6710 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:14.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:14.137+0000 7fda485c3640 1 --2- 172.21.15.112:0/3776221690 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fda401438f0 0x7fda40145b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:14.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:14.137+0000 7fda485c3640 1 --2- 172.21.15.112:0/3776221690 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fda2c004db0 0x7fda400c9f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:14.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:14.137+0000 7fda485c3640 1 -- 172.21.15.112:0/3776221690 >> 172.21.15.112:0/3776221690 conn(0x7fda400d2350 msgr2=0x7fda40070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:14.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:14.137+0000 7fda485c3640 1 -- 172.21.15.112:0/3776221690 shutdown_connections 2024-09-21T00:26:14.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:14.137+0000 7fda485c3640 1 -- 172.21.15.112:0/3776221690 wait complete. 2024-09-21T00:26:14.148 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-21T00:26:14.148 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T00:26:14.149 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi187.front.sepia.ceph.com 2024-09-21T00:26:14.149 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-21T00:26:14.149 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-21T00:26:14.149 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-21T00:26:14.149 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-21T00:26:14.149 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T00:26:14.149 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-21T00:26:14.149 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:26:14.149 DEBUG:teuthology.orchestra.run.smithi187:> ip addr 2024-09-21T00:26:14.168 INFO:teuthology.orchestra.run.smithi187.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T00:26:14.168 INFO:teuthology.orchestra.run.smithi187.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T00:26:14.168 INFO:teuthology.orchestra.run.smithi187.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T00:26:14.168 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:26:14.168 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 ::1/128 scope host 2024-09-21T00:26:14.168 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:26:14.168 INFO:teuthology.orchestra.run.smithi187.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T00:26:14.169 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 0c:c4:7a:d9:36:78 brd ff:ff:ff:ff:ff:ff 2024-09-21T00:26:14.169 INFO:teuthology.orchestra.run.smithi187.stdout: altname enp7s0f0 2024-09-21T00:26:14.169 INFO:teuthology.orchestra.run.smithi187.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T00:26:14.169 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 0c:c4:7a:d9:36:79 brd ff:ff:ff:ff:ff:ff 2024-09-21T00:26:14.169 INFO:teuthology.orchestra.run.smithi187.stdout: altname enp7s0f1 2024-09-21T00:26:14.169 INFO:teuthology.orchestra.run.smithi187.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T00:26:14.169 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 0c:c4:7a:88:77:b0 brd ff:ff:ff:ff:ff:ff 2024-09-21T00:26:14.169 INFO:teuthology.orchestra.run.smithi187.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T00:26:14.169 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 0c:c4:7a:88:77:b1 brd ff:ff:ff:ff:ff:ff 2024-09-21T00:26:14.169 INFO:teuthology.orchestra.run.smithi187.stdout: inet 172.21.15.187/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-21T00:26:14.169 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft 42737sec preferred_lft 42737sec 2024-09-21T00:26:14.169 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 fe80::ec4:7aff:fe88:77b1/64 scope link noprefixroute 2024-09-21T00:26:14.169 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:26:14.170 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-21T00:26:14.170 DEBUG:teuthology.orchestra.run.smithi187:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T00:26:14.170 DEBUG:teuthology.orchestra.run.smithi187:> set -e 2024-09-21T00:26:14.170 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip link add name ceph-brx type bridge 2024-09-21T00:26:14.170 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip addr flush dev ceph-brx 2024-09-21T00:26:14.170 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip link set ceph-brx up 2024-09-21T00:26:14.170 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-21T00:26:14.170 DEBUG:teuthology.orchestra.run.smithi187:> ') 2024-09-21T00:26:14.263 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:26:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T00:26:14.373 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:26:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T00:26:14.380 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:26:14.380 DEBUG:teuthology.orchestra.run.smithi187:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T00:26:14.406 INFO:teuthology.orchestra.run.smithi187.stdout:1 2024-09-21T00:26:14.407 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:26:14.407 DEBUG:teuthology.orchestra.run.smithi187:> route 2024-09-21T00:26:14.478 INFO:teuthology.orchestra.run.smithi187.stdout:Kernel IP routing table 2024-09-21T00:26:14.478 INFO:teuthology.orchestra.run.smithi187.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T00:26:14.478 INFO:teuthology.orchestra.run.smithi187.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-21T00:26:14.479 INFO:teuthology.orchestra.run.smithi187.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-21T00:26:14.479 INFO:teuthology.orchestra.run.smithi187.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T00:26:14.479 DEBUG:teuthology.orchestra.run.smithi187:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T00:26:14.479 DEBUG:teuthology.orchestra.run.smithi187:> set -e 2024-09-21T00:26:14.479 DEBUG:teuthology.orchestra.run.smithi187:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-21T00:26:14.479 DEBUG:teuthology.orchestra.run.smithi187:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-21T00:26:14.479 DEBUG:teuthology.orchestra.run.smithi187:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-21T00:26:14.479 DEBUG:teuthology.orchestra.run.smithi187:> ') 2024-09-21T00:26:14.566 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:26:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T00:26:14.644 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T00:26:14.646 DEBUG:teuthology.orchestra.run.smithi187:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T00:26:14.790 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:26:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T00:26:14.796 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:26:14.796 DEBUG:teuthology.orchestra.run.smithi187:> ip netns list 2024-09-21T00:26:14.811 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:26:14.811 DEBUG:teuthology.orchestra.run.smithi187:> ip netns list-id 2024-09-21T00:26:14.867 DEBUG:teuthology.orchestra.run.smithi187:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T00:26:14.867 DEBUG:teuthology.orchestra.run.smithi187:> set -e 2024-09-21T00:26:14.867 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T00:26:14.867 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-09-21T00:26:14.867 DEBUG:teuthology.orchestra.run.smithi187:> ') 2024-09-21T00:26:14.955 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:26:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T00:26:14.980 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:26:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T00:26:14.986 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-09-21T00:26:14.987 DEBUG:teuthology.orchestra.run.smithi187:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T00:26:14.987 DEBUG:teuthology.orchestra.run.smithi187:> set -e 2024-09-21T00:26:14.987 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-09-21T00:26:14.987 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-21T00:26:14.987 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-09-21T00:26:14.987 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-09-21T00:26:14.987 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-09-21T00:26:14.987 DEBUG:teuthology.orchestra.run.smithi187:> ') 2024-09-21T00:26:15.036 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:26:15 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T00:26:15.121 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:26:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T00:26:15.126 DEBUG:teuthology.orchestra.run.smithi187:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T00:26:15.126 DEBUG:teuthology.orchestra.run.smithi187:> set -e 2024-09-21T00:26:15.127 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip link set brx.0 up 2024-09-21T00:26:15.127 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip link set dev brx.0 master ceph-brx 2024-09-21T00:26:15.127 DEBUG:teuthology.orchestra.run.smithi187:> ') 2024-09-21T00:26:15.175 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:26:15 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T00:26:15.197 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:26:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T00:26:15.203 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T00:26:15.203 DEBUG:teuthology.orchestra.run.smithi187:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-21T00:26:15.218 INFO:teuthology.orchestra.run.smithi187.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-21T00:26:15.218 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T00:26:15.218 DEBUG:teuthology.orchestra.run.smithi187:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-21T00:26:15.273 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-21T00:26:15.273 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:26:15.273 DEBUG:teuthology.orchestra.run.smithi187:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-21T00:26:15.373 INFO:teuthology.orchestra.run.smithi187.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-21T00:26:15.373 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-21T00:26:15.373 INFO:teuthology.orchestra.run.smithi187.stdout:invalid new device string format 2024-09-21T00:26:15.498 INFO:teuthology.orchestra.run.smithi187.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-21T00:26:15.498 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-21T00:26:15.498 INFO:teuthology.orchestra.run.smithi187.stdout:invalid new device string format 2024-09-21T00:26:15.498 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: resolved to: "172.21.15.112:3300,172.21.15.187:3300,172.21.15.187:3301" 2024-09-21T00:26:15.498 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T00:26:15.516 INFO:teuthology.orchestra.run.smithi187.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-21T00:26:15.516 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-21T00:26:15.516 INFO:teuthology.orchestra.run.smithi187.stdout:invalid new device string format 2024-09-21T00:26:15.516 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: resolved to: "172.21.15.112:3300,172.21.15.187:3300,172.21.15.187:3301" 2024-09-21T00:26:15.516 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T00:26:15.516 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: resolved to: "172.21.15.112:6789,172.21.15.187:6789,172.21.15.187:6790" 2024-09-21T00:26:15.516 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.112:6789,172.21.15.187:6789,172.21.15.187:6790:/ 2024-09-21T00:26:15.516 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=8b34692b-dc65-49b2-a08f-26f707b96ac5" will pass to kernel 2024-09-21T00:26:15.517 INFO:teuthology.orchestra.run.smithi187.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-09-21T00:26:15.517 INFO:teuthology.orchestra.run.smithi187.stdout: You just mounted a file system that supports labels which does not 2024-09-21T00:26:15.517 INFO:teuthology.orchestra.run.smithi187.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T00:26:15.517 INFO:teuthology.orchestra.run.smithi187.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T00:26:15.517 INFO:teuthology.orchestra.run.smithi187.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T00:26:15.518 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T00:26:15.518 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:26:15.519 DEBUG:teuthology.orchestra.run.smithi187:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-21T00:26:15.543 DEBUG:teuthology.orchestra.run.smithi187:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T00:26:15.631 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:26:15 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T00:26:15.633 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:26:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T00:26:15.638 INFO:teuthology.orchestra.run.smithi187.stdout:client4616 2024-09-21T00:26:15.639 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-09-21T00:26:15.639 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:26:15.654 DEBUG:teuthology.orchestra.run.smithi187:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T00:26:15.742 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:26:15 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T00:26:15.744 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:26:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T00:26:15.749 INFO:teuthology.orchestra.run.smithi187.stdout:client4616 2024-09-21T00:26:15.750 DEBUG:teuthology.orchestra.run.smithi187:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T00:26:15.797 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:26:15 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T00:26:15.799 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:26:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T00:26:15.805 INFO:teuthology.orchestra.run.smithi187.stdout:client4616 2024-09-21T00:26:15.805 DEBUG:teuthology.orchestra.run.smithi187:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/8b34692b-dc65-49b2-a08f-26f707b96ac5.client4616/status') 2024-09-21T00:26:15.853 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:26:15 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T00:26:15.867 INFO:teuthology.orchestra.run.smithi187.stdout:instance: client.4616 (3)192.168.0.1:0/1268333409 2024-09-21T00:26:15.868 INFO:teuthology.orchestra.run.smithi187.stdout:blocklisted: false 2024-09-21T00:26:15.868 INFO:teuthology.orchestra.run.smithi187.stderr:0+1 records in 2024-09-21T00:26:15.868 INFO:teuthology.orchestra.run.smithi187.stderr:0+1 records out 2024-09-21T00:26:15.868 INFO:teuthology.orchestra.run.smithi187.stderr:69 bytes copied, 3.8582e-05 s, 1.8 MB/s 2024-09-21T00:26:15.869 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:26:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T00:26:15.875 INFO:tasks.kclient:client.3 config is {'syntax': 'v1'} 2024-09-21T00:26:15.875 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-21T00:26:15.875 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-09-21T00:26:15.875 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T00:26:16.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:16.001+0000 7f2ff6840640 1 -- 172.21.15.112:0/1678035560 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ff0162280 msgr2=0x7f2ff0162690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:16.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:16.001+0000 7f2ff6840640 1 --2- 172.21.15.112:0/1678035560 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ff0162280 0x7f2ff0162690 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f2fe4009830 tx=0x7f2fe402a510 comp rx=0 tx=0).stop 2024-09-21T00:26:16.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:16.001+0000 7f2ff6840640 1 -- 172.21.15.112:0/1678035560 shutdown_connections 2024-09-21T00:26:16.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:16.001+0000 7f2ff6840640 1 --2- 172.21.15.112:0/1678035560 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2ff00b2e40 0x7f2ff00b3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:16.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:16.001+0000 7f2ff6840640 1 --2- 172.21.15.112:0/1678035560 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ff0162280 0x7f2ff0162690 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:16.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:16.001+0000 7f2ff6840640 1 -- 172.21.15.112:0/1678035560 >> 172.21.15.112:0/1678035560 conn(0x7f2ff00b6b20 msgr2=0x7f2ff00d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:16.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:16.001+0000 7f2ff6840640 1 -- 172.21.15.112:0/1678035560 shutdown_connections 2024-09-21T00:26:16.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:16.001+0000 7f2ff6840640 1 -- 172.21.15.112:0/1678035560 wait complete. 2024-09-21T00:26:16.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:16.003+0000 7f2ff6840640 1 Processor -- start 2024-09-21T00:26:16.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:16.003+0000 7f2ff6840640 1 -- start start 2024-09-21T00:26:16.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:16.004+0000 7f2ff6840640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2ff00b2e40 0x7f2ff015a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:16.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:16.004+0000 7f2ff6840640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2ff0162280 0x7f2ff014da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:16.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:16.004+0000 7f2ff6840640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ff00aadc0 0x7f2ff01f88e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:16.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:16.004+0000 7f2ff6840640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2ff015e4b0 con 0x7f2ff00aadc0 2024-09-21T00:26:16.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:16.004+0000 7f2ff6840640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f2ff00b39d0 con 0x7f2ff0162280 2024-09-21T00:26:16.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:16.004+0000 7f2ff6840640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f2ff015e630 con 0x7f2ff00b2e40 2024-09-21T00:26:16.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:16.004+0000 7f2ff4ddd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ff00aadc0 0x7f2ff01f88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:16.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:16.004+0000 7f2feffff640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2ff00b2e40 0x7f2ff015a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:16.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:16.004+0000 7f2ff4ddd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ff00aadc0 0x7f2ff01f88e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60280/0 (socket says 172.21.15.112:60280) 2024-09-21T00:26:16.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:16.004+0000 7f2ff4ddd640 1 -- 172.21.15.112:0/3606238706 learned_addr learned my addr 172.21.15.112:0/3606238706 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:26:16.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:16.004+0000 7f2fef7fe640 1 --2- 172.21.15.112:0/3606238706 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2ff0162280 0x7f2ff014da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:16.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:16.005+0000 7f2ff4ddd640 1 -- 172.21.15.112:0/3606238706 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2ff00b2e40 msgr2=0x7f2ff015a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:16.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:16.005+0000 7f2ff4ddd640 1 --2- 172.21.15.112:0/3606238706 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2ff00b2e40 0x7f2ff015a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:16.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:16.005+0000 7f2ff4ddd640 1 -- 172.21.15.112:0/3606238706 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2ff0162280 msgr2=0x7f2ff014da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:16.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:16.005+0000 7f2ff4ddd640 1 --2- 172.21.15.112:0/3606238706 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2ff0162280 0x7f2ff014da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:16.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:16.005+0000 7f2ff4ddd640 1 -- 172.21.15.112:0/3606238706 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ff014df50 con 0x7f2ff00aadc0 2024-09-21T00:26:16.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:16.005+0000 7f2ff4ddd640 1 --2- 172.21.15.112:0/3606238706 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ff00aadc0 0x7f2ff01f88e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f2fe800e990 tx=0x7f2fe800ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:16.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:16.005+0000 7f2fecff9640 1 -- 172.21.15.112:0/3606238706 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fe8004d20 con 0x7f2ff00aadc0 2024-09-21T00:26:16.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:16.005+0000 7f2fecff9640 1 -- 172.21.15.112:0/3606238706 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2fe800caa0 con 0x7f2ff00aadc0 2024-09-21T00:26:16.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:16.005+0000 7f2ff6840640 1 -- 172.21.15.112:0/3606238706 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ff00b1a50 con 0x7f2ff00aadc0 2024-09-21T00:26:16.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:16.005+0000 7f2fecff9640 1 -- 172.21.15.112:0/3606238706 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fe8005020 con 0x7f2ff00aadc0 2024-09-21T00:26:16.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:16.005+0000 7f2ff6840640 1 -- 172.21.15.112:0/3606238706 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ff014e740 con 0x7f2ff00aadc0 2024-09-21T00:26:16.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:16.006+0000 7f2ff6840640 1 -- 172.21.15.112:0/3606238706 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ff0162f00 con 0x7f2ff00aadc0 2024-09-21T00:26:16.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:16.007+0000 7f2fecff9640 1 -- 172.21.15.112:0/3606238706 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2fe80051c0 con 0x7f2ff00aadc0 2024-09-21T00:26:16.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:16.007+0000 7f2fecff9640 1 --2- 172.21.15.112:0/3606238706 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2fdc078b90 0x7f2fdc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:16.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:16.007+0000 7f2fecff9640 1 -- 172.21.15.112:0/3606238706 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f2fe809ba00 con 0x7f2ff00aadc0 2024-09-21T00:26:16.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:16.008+0000 7f2feffff640 1 --2- 172.21.15.112:0/3606238706 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2fdc078b90 0x7f2fdc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:16.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:16.008+0000 7f2feffff640 1 --2- 172.21.15.112:0/3606238706 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2fdc078b90 0x7f2fdc07b0a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2fe0001720 tx=0x7f2fe0007e90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:16.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:16.012+0000 7f2fecff9640 1 -- 172.21.15.112:0/3606238706 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2fe8016030 con 0x7f2ff00aadc0 2024-09-21T00:26:16.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:16.224+0000 7f2ff6840640 1 -- 172.21.15.112:0/3606238706 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f2ff015dca0 con 0x7f2ff00aadc0 2024-09-21T00:26:16.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:16.225+0000 7f2fecff9640 1 -- 172.21.15.112:0/3606238706 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7f2fe8063430 con 0x7f2ff00aadc0 2024-09-21T00:26:16.226 INFO:teuthology.orchestra.run.smithi112.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-21T00:26:16.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:16.227+0000 7f2ff6840640 1 -- 172.21.15.112:0/3606238706 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2fdc078b90 msgr2=0x7f2fdc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:16.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:16.227+0000 7f2ff6840640 1 --2- 172.21.15.112:0/3606238706 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2fdc078b90 0x7f2fdc07b0a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2fe0001720 tx=0x7f2fe0007e90 comp rx=0 tx=0).stop 2024-09-21T00:26:16.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:16.227+0000 7f2ff6840640 1 -- 172.21.15.112:0/3606238706 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ff00aadc0 msgr2=0x7f2ff01f88e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:16.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:16.227+0000 7f2ff6840640 1 --2- 172.21.15.112:0/3606238706 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ff00aadc0 0x7f2ff01f88e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f2fe800e990 tx=0x7f2fe800ee50 comp rx=0 tx=0).stop 2024-09-21T00:26:16.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:16.227+0000 7f2ff6840640 1 -- 172.21.15.112:0/3606238706 shutdown_connections 2024-09-21T00:26:16.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:16.227+0000 7f2ff6840640 1 --2- 172.21.15.112:0/3606238706 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2fdc078b90 0x7f2fdc07b0a0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:16.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:16.227+0000 7f2ff6840640 1 --2- 172.21.15.112:0/3606238706 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ff00aadc0 0x7f2ff01f88e0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:16.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:16.227+0000 7f2ff6840640 1 --2- 172.21.15.112:0/3606238706 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2ff0162280 0x7f2ff014da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:16.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:16.227+0000 7f2ff6840640 1 --2- 172.21.15.112:0/3606238706 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2ff00b2e40 0x7f2ff015a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:16.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:16.227+0000 7f2ff6840640 1 -- 172.21.15.112:0/3606238706 >> 172.21.15.112:0/3606238706 conn(0x7f2ff00b6b20 msgr2=0x7f2ff006cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:16.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:16.227+0000 7f2ff6840640 1 -- 172.21.15.112:0/3606238706 shutdown_connections 2024-09-21T00:26:16.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:16.227+0000 7f2ff6840640 1 -- 172.21.15.112:0/3606238706 wait complete. 2024-09-21T00:26:16.239 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-21T00:26:16.239 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T00:26:16.239 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi187.front.sepia.ceph.com 2024-09-21T00:26:16.239 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-09-21T00:26:16.239 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-09-21T00:26:16.239 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-21T00:26:16.239 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-21T00:26:16.239 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T00:26:16.239 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-09-21T00:26:16.240 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:26:16.240 DEBUG:teuthology.orchestra.run.smithi187:> ip addr 2024-09-21T00:26:16.256 INFO:teuthology.orchestra.run.smithi187.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T00:26:16.256 INFO:teuthology.orchestra.run.smithi187.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T00:26:16.256 INFO:teuthology.orchestra.run.smithi187.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T00:26:16.256 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:26:16.256 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 ::1/128 scope host 2024-09-21T00:26:16.256 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:26:16.256 INFO:teuthology.orchestra.run.smithi187.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T00:26:16.257 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 0c:c4:7a:d9:36:78 brd ff:ff:ff:ff:ff:ff 2024-09-21T00:26:16.257 INFO:teuthology.orchestra.run.smithi187.stdout: altname enp7s0f0 2024-09-21T00:26:16.257 INFO:teuthology.orchestra.run.smithi187.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T00:26:16.257 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 0c:c4:7a:d9:36:79 brd ff:ff:ff:ff:ff:ff 2024-09-21T00:26:16.257 INFO:teuthology.orchestra.run.smithi187.stdout: altname enp7s0f1 2024-09-21T00:26:16.257 INFO:teuthology.orchestra.run.smithi187.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T00:26:16.257 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 0c:c4:7a:88:77:b0 brd ff:ff:ff:ff:ff:ff 2024-09-21T00:26:16.257 INFO:teuthology.orchestra.run.smithi187.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T00:26:16.257 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 0c:c4:7a:88:77:b1 brd ff:ff:ff:ff:ff:ff 2024-09-21T00:26:16.257 INFO:teuthology.orchestra.run.smithi187.stdout: inet 172.21.15.187/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-21T00:26:16.257 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft 42735sec preferred_lft 42735sec 2024-09-21T00:26:16.257 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 fe80::ec4:7aff:fe88:77b1/64 scope link noprefixroute 2024-09-21T00:26:16.257 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:26:16.257 INFO:teuthology.orchestra.run.smithi187.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T00:26:16.258 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether ca:0b:a0:f5:2b:b5 brd ff:ff:ff:ff:ff:ff 2024-09-21T00:26:16.258 INFO:teuthology.orchestra.run.smithi187.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T00:26:16.258 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:26:16.258 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 fe80::ac6f:97ff:fe09:b82e/64 scope link 2024-09-21T00:26:16.258 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:26:16.258 INFO:teuthology.orchestra.run.smithi187.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T00:26:16.258 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether ca:0b:a0:f5:2b:b5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T00:26:16.258 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 fe80::c80b:a0ff:fef5:2bb5/64 scope link tentative 2024-09-21T00:26:16.258 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:26:16.258 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:26:16.258 DEBUG:teuthology.orchestra.run.smithi187:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T00:26:16.319 INFO:teuthology.orchestra.run.smithi187.stdout:1 2024-09-21T00:26:16.321 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:26:16.321 DEBUG:teuthology.orchestra.run.smithi187:> route 2024-09-21T00:26:16.378 INFO:teuthology.orchestra.run.smithi187.stdout:Kernel IP routing table 2024-09-21T00:26:16.379 INFO:teuthology.orchestra.run.smithi187.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T00:26:16.379 INFO:teuthology.orchestra.run.smithi187.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-21T00:26:16.379 INFO:teuthology.orchestra.run.smithi187.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-21T00:26:16.379 INFO:teuthology.orchestra.run.smithi187.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T00:26:16.379 DEBUG:teuthology.orchestra.run.smithi187:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T00:26:16.379 DEBUG:teuthology.orchestra.run.smithi187:> set -e 2024-09-21T00:26:16.379 DEBUG:teuthology.orchestra.run.smithi187:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-21T00:26:16.379 DEBUG:teuthology.orchestra.run.smithi187:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-21T00:26:16.379 DEBUG:teuthology.orchestra.run.smithi187:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-21T00:26:16.379 DEBUG:teuthology.orchestra.run.smithi187:> ') 2024-09-21T00:26:16.466 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:26:16 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T00:26:16.499 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:26:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T00:26:16.505 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:26:16.505 DEBUG:teuthology.orchestra.run.smithi187:> ip netns list 2024-09-21T00:26:16.520 INFO:teuthology.orchestra.run.smithi187.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-21T00:26:16.520 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:26:16.521 DEBUG:teuthology.orchestra.run.smithi187:> ip netns list-id 2024-09-21T00:26:16.575 INFO:teuthology.orchestra.run.smithi187.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-09-21T00:26:16.575 DEBUG:teuthology.orchestra.run.smithi187:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T00:26:16.575 DEBUG:teuthology.orchestra.run.smithi187:> set -e 2024-09-21T00:26:16.575 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-21T00:26:16.576 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-09-21T00:26:16.576 DEBUG:teuthology.orchestra.run.smithi187:> ') 2024-09-21T00:26:16.662 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:26:16 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T00:26:16.685 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:26:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T00:26:16.691 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:26:16.691 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-09-21T00:26:16.716 INFO:teuthology.orchestra.run.smithi187.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T00:26:16.716 INFO:teuthology.orchestra.run.smithi187.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T00:26:16.716 INFO:teuthology.orchestra.run.smithi187.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T00:26:16.716 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:26:16.716 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 ::1/128 scope host 2024-09-21T00:26:16.716 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:26:16.716 INFO:teuthology.orchestra.run.smithi187.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T00:26:16.716 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 76:73:83:1c:cf:37 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-21T00:26:16.716 INFO:teuthology.orchestra.run.smithi187.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-21T00:26:16.717 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:26:16.717 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 fe80::7473:83ff:fe1c:cf37/64 scope link 2024-09-21T00:26:16.717 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:26:16.718 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:26:16.718 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-09-21T00:26:16.782 INFO:teuthology.orchestra.run.smithi187.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T00:26:16.782 INFO:teuthology.orchestra.run.smithi187.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T00:26:16.782 INFO:teuthology.orchestra.run.smithi187.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T00:26:16.783 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:26:16.783 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 ::1/128 scope host 2024-09-21T00:26:16.783 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:26:16.783 INFO:teuthology.orchestra.run.smithi187.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T00:26:16.783 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 76:73:83:1c:cf:37 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-21T00:26:16.783 INFO:teuthology.orchestra.run.smithi187.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-21T00:26:16.783 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:26:16.783 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 fe80::7473:83ff:fe1c:cf37/64 scope link 2024-09-21T00:26:16.783 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:26:16.784 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-09-21T00:26:16.784 DEBUG:teuthology.orchestra.run.smithi187:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T00:26:16.784 DEBUG:teuthology.orchestra.run.smithi187:> set -e 2024-09-21T00:26:16.784 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-09-21T00:26:16.784 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-09-21T00:26:16.784 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-09-21T00:26:16.784 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-09-21T00:26:16.785 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-09-21T00:26:16.785 DEBUG:teuthology.orchestra.run.smithi187:> ') 2024-09-21T00:26:16.872 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:26:16 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T00:26:16.930 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:26:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T00:26:16.936 DEBUG:teuthology.orchestra.run.smithi187:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T00:26:16.936 DEBUG:teuthology.orchestra.run.smithi187:> set -e 2024-09-21T00:26:16.936 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip link set brx.1 up 2024-09-21T00:26:16.936 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip link set dev brx.1 master ceph-brx 2024-09-21T00:26:16.936 DEBUG:teuthology.orchestra.run.smithi187:> ') 2024-09-21T00:26:16.984 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:26:16 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T00:26:17.006 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:26:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T00:26:17.012 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T00:26:17.013 DEBUG:teuthology.orchestra.run.smithi187:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-09-21T00:26:17.040 INFO:teuthology.orchestra.run.smithi187.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-09-21T00:26:17.040 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T00:26:17.040 DEBUG:teuthology.orchestra.run.smithi187:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-09-21T00:26:17.095 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-21T00:26:17.095 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:26:17.095 DEBUG:teuthology.orchestra.run.smithi187:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-09-21T00:26:17.189 INFO:teuthology.orchestra.run.smithi187.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-09-21T00:26:17.189 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-09-21T00:26:17.189 INFO:teuthology.orchestra.run.smithi187.stdout:invalid new device string format 2024-09-21T00:26:17.200 INFO:teuthology.orchestra.run.smithi187.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-09-21T00:26:17.200 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-09-21T00:26:17.200 INFO:teuthology.orchestra.run.smithi187.stdout:invalid new device string format 2024-09-21T00:26:17.200 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: resolved to: "172.21.15.112:3300,172.21.15.187:3300,172.21.15.187:3301" 2024-09-21T00:26:17.200 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T00:26:17.210 INFO:teuthology.orchestra.run.smithi187.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-09-21T00:26:17.210 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-09-21T00:26:17.210 INFO:teuthology.orchestra.run.smithi187.stdout:invalid new device string format 2024-09-21T00:26:17.210 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: resolved to: "172.21.15.112:3300,172.21.15.187:3300,172.21.15.187:3301" 2024-09-21T00:26:17.210 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T00:26:17.210 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: resolved to: "172.21.15.112:6789,172.21.15.187:6789,172.21.15.187:6790" 2024-09-21T00:26:17.210 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.112:6789,172.21.15.187:6789,172.21.15.187:6790:/ 2024-09-21T00:26:17.210 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs,key=3,fsid=8b34692b-dc65-49b2-a08f-26f707b96ac5" will pass to kernel 2024-09-21T00:26:17.211 INFO:teuthology.orchestra.run.smithi187.stdout:mount: /home/ubuntu/cephtest/mnt.3 does not contain SELinux labels. 2024-09-21T00:26:17.211 INFO:teuthology.orchestra.run.smithi187.stdout: You just mounted a file system that supports labels which does not 2024-09-21T00:26:17.211 INFO:teuthology.orchestra.run.smithi187.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T00:26:17.211 INFO:teuthology.orchestra.run.smithi187.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T00:26:17.211 INFO:teuthology.orchestra.run.smithi187.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T00:26:17.212 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T00:26:17.213 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:26:17.213 DEBUG:teuthology.orchestra.run.smithi187:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-09-21T00:26:17.237 DEBUG:teuthology.orchestra.run.smithi187:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T00:26:17.325 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:26:17 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T00:26:17.327 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:26:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T00:26:17.332 INFO:teuthology.orchestra.run.smithi187.stdout:client4738 2024-09-21T00:26:17.333 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-09-21T00:26:17.333 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:26:17.348 DEBUG:teuthology.orchestra.run.smithi187:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T00:26:17.437 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:26:17 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T00:26:17.438 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:26:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T00:26:17.444 INFO:teuthology.orchestra.run.smithi187.stdout:client4738 2024-09-21T00:26:17.444 DEBUG:teuthology.orchestra.run.smithi187:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T00:26:17.492 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:26:17 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T00:26:17.493 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:26:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T00:26:17.499 INFO:teuthology.orchestra.run.smithi187.stdout:client4738 2024-09-21T00:26:17.499 DEBUG:teuthology.orchestra.run.smithi187:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/8b34692b-dc65-49b2-a08f-26f707b96ac5.client4738/status') 2024-09-21T00:26:17.547 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:26:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T00:26:17.557 INFO:teuthology.orchestra.run.smithi187.stdout:instance: client.4738 (3)192.168.0.2:0/4195524154 2024-09-21T00:26:17.557 INFO:teuthology.orchestra.run.smithi187.stdout:blocklisted: false 2024-09-21T00:26:17.557 INFO:teuthology.orchestra.run.smithi187.stderr:0+1 records in 2024-09-21T00:26:17.557 INFO:teuthology.orchestra.run.smithi187.stderr:0+1 records out 2024-09-21T00:26:17.558 INFO:teuthology.orchestra.run.smithi187.stderr:69 bytes copied, 3.8757e-05 s, 1.8 MB/s 2024-09-21T00:26:17.558 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:26:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T00:26:17.565 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-09-21T00:26:17.619 INFO:tasks.cephfs_test_runner:Starting test: test_mount_conn_close (tasks.cephfs.test_sessionmap.TestSessionMap) That when a client unmounts, the thread count on the MDS goes back 2024-09-21T00:26:17.619 INFO:tasks.cephfs_test_runner:test_mount_conn_close (tasks.cephfs.test_sessionmap.TestSessionMap) 2024-09-21T00:26:17.619 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T00:26:17.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:17.744+0000 7fd2d9372640 1 -- 172.21.15.112:0/718171995 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2d40b2e60 msgr2=0x7fd2d40b3270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:17.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:17.744+0000 7fd2d9372640 1 --2- 172.21.15.112:0/718171995 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2d40b2e60 0x7fd2d40b3270 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fd2cc009830 tx=0x7fd2cc02a500 comp rx=0 tx=0).stop 2024-09-21T00:26:17.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:17.744+0000 7fd2d9372640 1 -- 172.21.15.112:0/718171995 shutdown_connections 2024-09-21T00:26:17.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:17.744+0000 7fd2d9372640 1 --2- 172.21.15.112:0/718171995 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2d40b2e60 0x7fd2d40b3270 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:17.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:17.744+0000 7fd2d9372640 1 --2- 172.21.15.112:0/718171995 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd2d4162590 0x7fd2d41629a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:17.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:17.744+0000 7fd2d9372640 1 -- 172.21.15.112:0/718171995 >> 172.21.15.112:0/718171995 conn(0x7fd2d40d4440 msgr2=0x7fd2d40d4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:17.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:17.744+0000 7fd2d9372640 1 -- 172.21.15.112:0/718171995 shutdown_connections 2024-09-21T00:26:17.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:17.744+0000 7fd2d9372640 1 -- 172.21.15.112:0/718171995 wait complete. 2024-09-21T00:26:17.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:17.746+0000 7fd2d9372640 1 Processor -- start 2024-09-21T00:26:17.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:17.746+0000 7fd2d9372640 1 -- start start 2024-09-21T00:26:17.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:17.747+0000 7fd2d9372640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd2d40b2e60 0x7fd2d41f5430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:17.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:17.747+0000 7fd2d9372640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2d40cfe00 0x7fd2d415d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:17.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:17.747+0000 7fd2d9372640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd2d4162590 0x7fd2d41439c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:17.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:17.747+0000 7fd2d9372640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd2d40b3930 con 0x7fd2d40cfe00 2024-09-21T00:26:17.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:17.747+0000 7fd2d9372640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd2d40b37b0 con 0x7fd2d4162590 2024-09-21T00:26:17.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:17.747+0000 7fd2d9372640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd2d40b3ab0 con 0x7fd2d40b2e60 2024-09-21T00:26:17.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:17.747+0000 7fd2d27fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2d40cfe00 0x7fd2d415d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:17.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:17.747+0000 7fd2d27fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2d40cfe00 0x7fd2d415d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41308/0 (socket says 172.21.15.112:41308) 2024-09-21T00:26:17.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:17.747+0000 7fd2d27fc640 1 -- 172.21.15.112:0/776887107 learned_addr learned my addr 172.21.15.112:0/776887107 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:26:17.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:17.747+0000 7fd2d2ffd640 1 --2- 172.21.15.112:0/776887107 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd2d40b2e60 0x7fd2d41f5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:17.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:17.747+0000 7fd2d37fe640 1 --2- 172.21.15.112:0/776887107 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd2d4162590 0x7fd2d41439c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:17.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:17.747+0000 7fd2d27fc640 1 -- 172.21.15.112:0/776887107 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd2d40b2e60 msgr2=0x7fd2d41f5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:17.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:17.747+0000 7fd2d27fc640 1 --2- 172.21.15.112:0/776887107 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd2d40b2e60 0x7fd2d41f5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:17.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:17.748+0000 7fd2d27fc640 1 -- 172.21.15.112:0/776887107 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd2d4162590 msgr2=0x7fd2d41439c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:17.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:17.748+0000 7fd2d27fc640 1 --2- 172.21.15.112:0/776887107 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd2d4162590 0x7fd2d41439c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:17.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:17.748+0000 7fd2d27fc640 1 -- 172.21.15.112:0/776887107 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2d41f5970 con 0x7fd2d40cfe00 2024-09-21T00:26:17.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:17.748+0000 7fd2d37fe640 1 --2- 172.21.15.112:0/776887107 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd2d4162590 0x7fd2d41439c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:26:17.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:17.748+0000 7fd2d27fc640 1 --2- 172.21.15.112:0/776887107 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2d40cfe00 0x7fd2d415d310 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fd2c800e7c0 tx=0x7fd2c800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:17.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:17.748+0000 7fd2bb7fe640 1 -- 172.21.15.112:0/776887107 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2c800ccc0 con 0x7fd2d40cfe00 2024-09-21T00:26:17.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:17.748+0000 7fd2bb7fe640 1 -- 172.21.15.112:0/776887107 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd2c8010430 con 0x7fd2d40cfe00 2024-09-21T00:26:17.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:17.748+0000 7fd2d9372640 1 -- 172.21.15.112:0/776887107 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2d415d850 con 0x7fd2d40cfe00 2024-09-21T00:26:17.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:17.748+0000 7fd2bb7fe640 1 -- 172.21.15.112:0/776887107 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2c8010860 con 0x7fd2d40cfe00 2024-09-21T00:26:17.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:17.748+0000 7fd2d9372640 1 -- 172.21.15.112:0/776887107 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2d415c550 con 0x7fd2d40cfe00 2024-09-21T00:26:17.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:17.749+0000 7fd2d9372640 1 -- 172.21.15.112:0/776887107 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2d415bfe0 con 0x7fd2d40cfe00 2024-09-21T00:26:17.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:17.750+0000 7fd2bb7fe640 1 -- 172.21.15.112:0/776887107 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd2c8010a00 con 0x7fd2d40cfe00 2024-09-21T00:26:17.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:17.750+0000 7fd2bb7fe640 1 --2- 172.21.15.112:0/776887107 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fd2c0078bc0 0x7fd2c007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:17.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:17.750+0000 7fd2bb7fe640 1 -- 172.21.15.112:0/776887107 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fd2c809b8f0 con 0x7fd2d40cfe00 2024-09-21T00:26:17.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:17.750+0000 7fd2d2ffd640 1 --2- 172.21.15.112:0/776887107 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fd2c0078bc0 0x7fd2c007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:17.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:17.751+0000 7fd2d2ffd640 1 --2- 172.21.15.112:0/776887107 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fd2c0078bc0 0x7fd2c007b0d0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd2c4001790 tx=0x7fd2c4001e90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:17.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:17.755+0000 7fd2bb7fe640 1 -- 172.21.15.112:0/776887107 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd2c80634c0 con 0x7fd2d40cfe00 2024-09-21T00:26:17.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:17.933+0000 7fd2d9372640 1 -- 172.21.15.112:0/776887107 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd2d40b3270 con 0x7fd2d40cfe00 2024-09-21T00:26:17.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:17.934+0000 7fd2bb7fe640 1 -- 172.21.15.112:0/776887107 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18622 (secure 0 0 0) 0x7fd2c8067ef0 con 0x7fd2d40cfe00 2024-09-21T00:26:17.935 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:26:17.935 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":26,"fsid":"8b34692b-dc65-49b2-a08f-26f707b96ac5","created":"2024-09-21T00:25:22.788490+0000","modified":"2024-09-21T00:25:52.623213+0000","last_up_change":"2024-09-21T00:25:36.161835+0000","last_in_change":"2024-09-21T00:25:26.346500+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T00:25:37.105006+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T00:25:45.413895+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T00:25:46.532124+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T00:25:50.244145+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9c575086-67d2-4de2-b01b-d44dcc21d604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6809","nonce":755760621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6811","nonce":755760621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6815","nonce":755760621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6813","nonce":755760621}]},"public_addr":"172.21.15.112:6809/755760621","cluster_addr":"172.21.15.112:6811/755760621","heartbeat_back_addr":"172.21.15.112:6815/755760621","heartbeat_front_addr":"172.21.15.112:6813/755760621","state":["exists","up"]},{"osd":1,"uuid":"a983c227-a3ce-4bdd-82f1-438ba7e3ecd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6801","nonce":344932592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6803","nonce":344932592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6807","nonce":344932592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6805","nonce":344932592}]},"public_addr":"172.21.15.112:6801/344932592","cluster_addr":"172.21.15.112:6803/344932592","heartbeat_back_addr":"172.21.15.112:6807/344932592","heartbeat_front_addr":"172.21.15.112:6805/344932592","state":["exists","up"]},{"osd":2,"uuid":"1193c1d8-bfff-4a57-8200-5f69bf1ea3af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6825","nonce":407933616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6827","nonce":407933616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6831","nonce":407933616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6829","nonce":407933616}]},"public_addr":"172.21.15.112:6825/407933616","cluster_addr":"172.21.15.112:6827/407933616","heartbeat_back_addr":"172.21.15.112:6831/407933616","heartbeat_front_addr":"172.21.15.112:6829/407933616","state":["exists","up"]},{"osd":3,"uuid":"af04330a-36ad-40cc-a8dd-fa259b221de5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6817","nonce":4120164806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6819","nonce":4120164806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6823","nonce":4120164806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6821","nonce":4120164806}]},"public_addr":"172.21.15.112:6817/4120164806","cluster_addr":"172.21.15.112:6819/4120164806","heartbeat_back_addr":"172.21.15.112:6823/4120164806","heartbeat_front_addr":"172.21.15.112:6821/4120164806","state":["exists","up"]},{"osd":4,"uuid":"5b893111-292c-4c5a-b184-53ad38c7199e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6803","nonce":522077560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6805","nonce":522077560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6809","nonce":522077560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6807","nonce":522077560}]},"public_addr":"172.21.15.187:6803/522077560","cluster_addr":"172.21.15.187:6805/522077560","heartbeat_back_addr":"172.21.15.187:6809/522077560","heartbeat_front_addr":"172.21.15.187:6807/522077560","state":["exists","up"]},{"osd":5,"uuid":"c0b8e011-45fe-4c5e-925e-6be2a7f1868a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6827","nonce":3593144721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6829","nonce":3593144721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6833","nonce":3593144721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6831","nonce":3593144721}]},"public_addr":"172.21.15.187:6827/3593144721","cluster_addr":"172.21.15.187:6829/3593144721","heartbeat_back_addr":"172.21.15.187:6833/3593144721","heartbeat_front_addr":"172.21.15.187:6831/3593144721","state":["exists","up"]},{"osd":6,"uuid":"87d2d650-7489-4593-98a3-a1f433ce8dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6811","nonce":1494387657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6813","nonce":1494387657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6817","nonce":1494387657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6815","nonce":1494387657}]},"public_addr":"172.21.15.187:6811/1494387657","cluster_addr":"172.21.15.187:6813/1494387657","heartbeat_back_addr":"172.21.15.187:6817/1494387657","heartbeat_front_addr":"172.21.15.187:6815/1494387657","state":["exists","up"]},{"osd":7,"uuid":"c6b4f8e2-a1ee-4438-aa37-67ff4b6ef1f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6819","nonce":1624699842}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6821","nonce":1624699842}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6825","nonce":1624699842}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6823","nonce":1624699842}]},"public_addr":"172.21.15.187:6819/1624699842","cluster_addr":"172.21.15.187:6821/1624699842","heartbeat_back_addr":"172.21.15.187:6825/1624699842","heartbeat_front_addr":"172.21.15.187:6823/1624699842","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.873162+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.400506+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.664317+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.159791+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.202630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.113212+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.384786+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.448441+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T00:26:17.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:17.936+0000 7fd2d9372640 1 -- 172.21.15.112:0/776887107 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fd2c0078bc0 msgr2=0x7fd2c007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:17.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:17.936+0000 7fd2d9372640 1 --2- 172.21.15.112:0/776887107 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fd2c0078bc0 0x7fd2c007b0d0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd2c4001790 tx=0x7fd2c4001e90 comp rx=0 tx=0).stop 2024-09-21T00:26:17.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:17.936+0000 7fd2d9372640 1 -- 172.21.15.112:0/776887107 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2d40cfe00 msgr2=0x7fd2d415d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:17.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:17.936+0000 7fd2d9372640 1 --2- 172.21.15.112:0/776887107 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2d40cfe00 0x7fd2d415d310 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fd2c800e7c0 tx=0x7fd2c800ec80 comp rx=0 tx=0).stop 2024-09-21T00:26:17.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:17.936+0000 7fd2d9372640 1 -- 172.21.15.112:0/776887107 shutdown_connections 2024-09-21T00:26:17.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:17.936+0000 7fd2d9372640 1 --2- 172.21.15.112:0/776887107 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd2d4162590 0x7fd2d41439c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:17.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:17.936+0000 7fd2d9372640 1 --2- 172.21.15.112:0/776887107 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fd2c0078bc0 0x7fd2c007b0d0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:17.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:17.936+0000 7fd2d9372640 1 --2- 172.21.15.112:0/776887107 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2d40cfe00 0x7fd2d415d310 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:17.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:17.936+0000 7fd2d9372640 1 --2- 172.21.15.112:0/776887107 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd2d40b2e60 0x7fd2d41f5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:17.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:17.936+0000 7fd2d9372640 1 -- 172.21.15.112:0/776887107 >> 172.21.15.112:0/776887107 conn(0x7fd2d40d4440 msgr2=0x7fd2d406cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:17.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:17.936+0000 7fd2d9372640 1 -- 172.21.15.112:0/776887107 shutdown_connections 2024-09-21T00:26:17.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:17.936+0000 7fd2d9372640 1 -- 172.21.15.112:0/776887107 wait complete. 2024-09-21T00:26:17.949 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T00:25:37.105006+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-21T00:25:45.413895+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-21T00:25:46.532124+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-21T00:25:50.244145+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-21T00:26:17.949 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-21T00:26:18.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.064+0000 7f2fd0a8b640 1 -- 172.21.15.112:0/552818914 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2fcc0ceb90 msgr2=0x7f2fcc0cefa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:18.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.064+0000 7f2fd0a8b640 1 --2- 172.21.15.112:0/552818914 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2fcc0ceb90 0x7f2fcc0cefa0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2fc4005fd0 tx=0x7f2fc402aa10 comp rx=0 tx=0).stop 2024-09-21T00:26:18.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.064+0000 7f2fd0a8b640 1 -- 172.21.15.112:0/552818914 shutdown_connections 2024-09-21T00:26:18.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.064+0000 7f2fd0a8b640 1 --2- 172.21.15.112:0/552818914 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2fcc0ceb90 0x7f2fcc0cefa0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:18.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.064+0000 7f2fd0a8b640 1 --2- 172.21.15.112:0/552818914 >> v2:172.21.15.187:3300/0 conn(0x7f2fcc15ee80 0x7f2fcc0d0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:18.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.064+0000 7f2fd0a8b640 1 -- 172.21.15.112:0/552818914 >> 172.21.15.112:0/552818914 conn(0x7f2fcc09d830 msgr2=0x7f2fcc0b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:18.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.064+0000 7f2fd0a8b640 1 -- 172.21.15.112:0/552818914 shutdown_connections 2024-09-21T00:26:18.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.064+0000 7f2fd0a8b640 1 -- 172.21.15.112:0/552818914 wait complete. 2024-09-21T00:26:18.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.066+0000 7f2fd0a8b640 1 Processor -- start 2024-09-21T00:26:18.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.066+0000 7f2fd0a8b640 1 -- start start 2024-09-21T00:26:18.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.068+0000 7f2fd0a8b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2fcc0b9ad0 0x7f2fcc14c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:18.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.068+0000 7f2fd0a8b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2fcc0ceb90 0x7f2fcc1f9640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:18.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.068+0000 7f2fd0a8b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2fcc15ee80 0x7f2fcc0b4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:18.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.068+0000 7f2fd0a8b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2fcc0b6700 con 0x7f2fcc0ceb90 2024-09-21T00:26:18.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.068+0000 7f2fd0a8b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f2fcc0b6580 con 0x7f2fcc0b9ad0 2024-09-21T00:26:18.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.068+0000 7f2fd0a8b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f2fcc0b6880 con 0x7f2fcc15ee80 2024-09-21T00:26:18.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.068+0000 7f2fc9d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2fcc0ceb90 0x7f2fcc1f9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:18.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.068+0000 7f2fc9d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2fcc0ceb90 0x7f2fcc1f9640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41320/0 (socket says 172.21.15.112:41320) 2024-09-21T00:26:18.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.068+0000 7f2fc9d9b640 1 -- 172.21.15.112:0/2254866109 learned_addr learned my addr 172.21.15.112:0/2254866109 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:26:18.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.068+0000 7f2fcad9d640 1 --2- 172.21.15.112:0/2254866109 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2fcc15ee80 0x7f2fcc0b4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:18.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.068+0000 7f2fca59c640 1 --2- 172.21.15.112:0/2254866109 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2fcc0b9ad0 0x7f2fcc14c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:18.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.068+0000 7f2fc9d9b640 1 -- 172.21.15.112:0/2254866109 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2fcc15ee80 msgr2=0x7f2fcc0b4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:18.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.068+0000 7f2fc9d9b640 1 --2- 172.21.15.112:0/2254866109 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2fcc15ee80 0x7f2fcc0b4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:18.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.068+0000 7f2fc9d9b640 1 -- 172.21.15.112:0/2254866109 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2fcc0b9ad0 msgr2=0x7f2fcc14c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:18.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.068+0000 7f2fc9d9b640 1 --2- 172.21.15.112:0/2254866109 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2fcc0b9ad0 0x7f2fcc14c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:18.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.068+0000 7f2fc9d9b640 1 -- 172.21.15.112:0/2254866109 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fcc0cb610 con 0x7f2fcc0ceb90 2024-09-21T00:26:18.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.068+0000 7f2fc9d9b640 1 --2- 172.21.15.112:0/2254866109 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2fcc0ceb90 0x7f2fcc1f9640 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f2fc000c440 tx=0x7f2fc000c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:18.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.068+0000 7f2fbaffd640 1 -- 172.21.15.112:0/2254866109 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fc0017070 con 0x7f2fcc0ceb90 2024-09-21T00:26:18.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.068+0000 7f2fbaffd640 1 -- 172.21.15.112:0/2254866109 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2fc00053c0 con 0x7f2fcc0ceb90 2024-09-21T00:26:18.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.068+0000 7f2fd0a8b640 1 -- 172.21.15.112:0/2254866109 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fcc06e650 con 0x7f2fcc0ceb90 2024-09-21T00:26:18.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.068+0000 7f2fbaffd640 1 -- 172.21.15.112:0/2254866109 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fc00038a0 con 0x7f2fcc0ceb90 2024-09-21T00:26:18.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.069+0000 7f2fd0a8b640 1 -- 172.21.15.112:0/2254866109 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fcc14ce80 con 0x7f2fcc0ceb90 2024-09-21T00:26:18.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.070+0000 7f2fd0a8b640 1 -- 172.21.15.112:0/2254866109 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fcc0b6d30 con 0x7f2fcc0ceb90 2024-09-21T00:26:18.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.070+0000 7f2fbaffd640 1 -- 172.21.15.112:0/2254866109 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2fc001a070 con 0x7f2fcc0ceb90 2024-09-21T00:26:18.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.071+0000 7f2fbaffd640 1 --2- 172.21.15.112:0/2254866109 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2fac078b90 0x7f2fac07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:18.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.071+0000 7f2fbaffd640 1 -- 172.21.15.112:0/2254866109 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f2fc009a2b0 con 0x7f2fcc0ceb90 2024-09-21T00:26:18.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.071+0000 7f2fca59c640 1 --2- 172.21.15.112:0/2254866109 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2fac078b90 0x7f2fac07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:18.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.071+0000 7f2fca59c640 1 --2- 172.21.15.112:0/2254866109 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2fac078b90 0x7f2fac07b0a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f2fbc009b30 tx=0x7f2fbc005f90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:18.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.075+0000 7f2fbaffd640 1 -- 172.21.15.112:0/2254866109 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2fcc1592d0 con 0x7f2fcc0ceb90 2024-09-21T00:26:18.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.251+0000 7f2fd0a8b640 1 -- 172.21.15.112:0/2254866109 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f2fcc06eb50 con 0x7f2fcc0ceb90 2024-09-21T00:26:18.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.252+0000 7f2fbaffd640 1 -- 172.21.15.112:0/2254866109 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7f2fc0061e80 con 0x7f2fcc0ceb90 2024-09-21T00:26:18.253 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2024-09-21T00:26:18.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.254+0000 7f2fd0a8b640 1 -- 172.21.15.112:0/2254866109 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2fac078b90 msgr2=0x7f2fac07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:18.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.254+0000 7f2fd0a8b640 1 --2- 172.21.15.112:0/2254866109 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2fac078b90 0x7f2fac07b0a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f2fbc009b30 tx=0x7f2fbc005f90 comp rx=0 tx=0).stop 2024-09-21T00:26:18.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.254+0000 7f2fd0a8b640 1 -- 172.21.15.112:0/2254866109 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2fcc0ceb90 msgr2=0x7f2fcc1f9640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:18.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.254+0000 7f2fd0a8b640 1 --2- 172.21.15.112:0/2254866109 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2fcc0ceb90 0x7f2fcc1f9640 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f2fc000c440 tx=0x7f2fc000c900 comp rx=0 tx=0).stop 2024-09-21T00:26:18.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.254+0000 7f2fd0a8b640 1 -- 172.21.15.112:0/2254866109 shutdown_connections 2024-09-21T00:26:18.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.254+0000 7f2fd0a8b640 1 --2- 172.21.15.112:0/2254866109 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2fcc15ee80 0x7f2fcc0b4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:18.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.254+0000 7f2fd0a8b640 1 --2- 172.21.15.112:0/2254866109 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2fac078b90 0x7f2fac07b0a0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:18.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.254+0000 7f2fd0a8b640 1 --2- 172.21.15.112:0/2254866109 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2fcc0ceb90 0x7f2fcc1f9640 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:18.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.254+0000 7f2fd0a8b640 1 --2- 172.21.15.112:0/2254866109 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2fcc0b9ad0 0x7f2fcc14c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:18.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.254+0000 7f2fd0a8b640 1 -- 172.21.15.112:0/2254866109 >> 172.21.15.112:0/2254866109 conn(0x7f2fcc09d830 msgr2=0x7f2fcc160930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:18.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.254+0000 7f2fd0a8b640 1 -- 172.21.15.112:0/2254866109 shutdown_connections 2024-09-21T00:26:18.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.254+0000 7f2fd0a8b640 1 -- 172.21.15.112:0/2254866109 wait complete. 2024-09-21T00:26:18.266 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-21T00:26:18.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.381+0000 7fc7ff8a9640 1 -- 172.21.15.112:0/1361852338 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc7f815bfc0 msgr2=0x7fc7f815f630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:18.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.381+0000 7fc7ff8a9640 1 --2- 172.21.15.112:0/1361852338 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc7f815bfc0 0x7fc7f815f630 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc7f806e6e0 tx=0x7fc7f002a540 comp rx=0 tx=0).stop 2024-09-21T00:26:18.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.381+0000 7fc7ff8a9640 1 -- 172.21.15.112:0/1361852338 shutdown_connections 2024-09-21T00:26:18.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.381+0000 7fc7ff8a9640 1 --2- 172.21.15.112:0/1361852338 >> v2:172.21.15.187:3300/0 conn(0x7fc7f80b5260 0x7fc7f815ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:18.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.381+0000 7fc7ff8a9640 1 --2- 172.21.15.112:0/1361852338 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc7f815bfc0 0x7fc7f815f630 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:18.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.381+0000 7fc7ff8a9640 1 --2- 172.21.15.112:0/1361852338 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc7f806def0 0x7fc7f806f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:18.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.381+0000 7fc7ff8a9640 1 -- 172.21.15.112:0/1361852338 >> 172.21.15.112:0/1361852338 conn(0x7fc7f80cdca0 msgr2=0x7fc7f80ce0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:18.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.381+0000 7fc7ff8a9640 1 -- 172.21.15.112:0/1361852338 shutdown_connections 2024-09-21T00:26:18.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.381+0000 7fc7ff8a9640 1 -- 172.21.15.112:0/1361852338 wait complete. 2024-09-21T00:26:18.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.383+0000 7fc7ff8a9640 1 Processor -- start 2024-09-21T00:26:18.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.383+0000 7fc7ff8a9640 1 -- start start 2024-09-21T00:26:18.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.384+0000 7fc7ff8a9640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc7f806def0 0x7fc7f8168d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:18.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.384+0000 7fc7ff8a9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc7f80b5260 0x7fc7f80b2250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:18.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.384+0000 7fc7ff8a9640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc7f815bfc0 0x7fc7f80b2790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:18.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.384+0000 7fc7ff8a9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f815fcf0 con 0x7fc7f80b5260 2024-09-21T00:26:18.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.384+0000 7fc7ff8a9640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f815fb70 con 0x7fc7f806def0 2024-09-21T00:26:18.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.384+0000 7fc7ff8a9640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc7f815fe70 con 0x7fc7f815bfc0 2024-09-21T00:26:18.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.384+0000 7fc7fce44640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc7f80b5260 0x7fc7f80b2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:18.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.384+0000 7fc7fce44640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc7f80b5260 0x7fc7f80b2250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41338/0 (socket says 172.21.15.112:41338) 2024-09-21T00:26:18.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.384+0000 7fc7fce44640 1 -- 172.21.15.112:0/269162703 learned_addr learned my addr 172.21.15.112:0/269162703 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:26:18.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.385+0000 7fc7fde46640 1 --2- 172.21.15.112:0/269162703 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc7f815bfc0 0x7fc7f80b2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:18.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.385+0000 7fc7fd645640 1 --2- 172.21.15.112:0/269162703 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc7f806def0 0x7fc7f8168d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:18.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.385+0000 7fc7fce44640 1 -- 172.21.15.112:0/269162703 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc7f815bfc0 msgr2=0x7fc7f80b2790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:18.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.385+0000 7fc7fce44640 1 --2- 172.21.15.112:0/269162703 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc7f815bfc0 0x7fc7f80b2790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:18.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.385+0000 7fc7fce44640 1 -- 172.21.15.112:0/269162703 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc7f806def0 msgr2=0x7fc7f8168d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:18.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.385+0000 7fc7fce44640 1 --2- 172.21.15.112:0/269162703 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc7f806def0 0x7fc7f8168d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:18.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.385+0000 7fc7fce44640 1 -- 172.21.15.112:0/269162703 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7f81548e0 con 0x7fc7f80b5260 2024-09-21T00:26:18.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.385+0000 7fc7fde46640 1 --2- 172.21.15.112:0/269162703 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc7f815bfc0 0x7fc7f80b2790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:26:18.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.385+0000 7fc7fce44640 1 --2- 172.21.15.112:0/269162703 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc7f80b5260 0x7fc7f80b2250 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fc7ec00c970 tx=0x7fc7ec00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:18.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.385+0000 7fc7f5ffb640 1 -- 172.21.15.112:0/269162703 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7ec007bb0 con 0x7fc7f80b5260 2024-09-21T00:26:18.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.385+0000 7fc7f5ffb640 1 -- 172.21.15.112:0/269162703 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc7ec007e40 con 0x7fc7f80b5260 2024-09-21T00:26:18.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.385+0000 7fc7ff8a9640 1 -- 172.21.15.112:0/269162703 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7f8154bc0 con 0x7fc7f80b5260 2024-09-21T00:26:18.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.385+0000 7fc7f5ffb640 1 -- 172.21.15.112:0/269162703 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7ec00f690 con 0x7fc7f80b5260 2024-09-21T00:26:18.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.385+0000 7fc7ff8a9640 1 -- 172.21.15.112:0/269162703 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7f8155180 con 0x7fc7f80b5260 2024-09-21T00:26:18.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.386+0000 7fc7ff8a9640 1 -- 172.21.15.112:0/269162703 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7f8154630 con 0x7fc7f80b5260 2024-09-21T00:26:18.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.387+0000 7fc7f5ffb640 1 -- 172.21.15.112:0/269162703 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc7ec00f830 con 0x7fc7f80b5260 2024-09-21T00:26:18.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.387+0000 7fc7f5ffb640 1 --2- 172.21.15.112:0/269162703 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fc7e4078bc0 0x7fc7e407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:18.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.387+0000 7fc7f5ffb640 1 -- 172.21.15.112:0/269162703 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fc7ec09b880 con 0x7fc7f80b5260 2024-09-21T00:26:18.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.388+0000 7fc7fd645640 1 --2- 172.21.15.112:0/269162703 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fc7e4078bc0 0x7fc7e407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:18.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.388+0000 7fc7fd645640 1 --2- 172.21.15.112:0/269162703 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fc7e4078bc0 0x7fc7e407b0d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc7f815efd0 tx=0x7fc7e8008230 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:18.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.392+0000 7fc7f5ffb640 1 -- 172.21.15.112:0/269162703 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc7ec014030 con 0x7fc7f80b5260 2024-09-21T00:26:18.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.570+0000 7fc7ff8a9640 1 -- 172.21.15.112:0/269162703 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fc7f815ba80 con 0x7fc7f80b5260 2024-09-21T00:26:18.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.570+0000 7fc7f5ffb640 1 -- 172.21.15.112:0/269162703 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7fc7ec063980 con 0x7fc7f80b5260 2024-09-21T00:26:18.571 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 32 2024-09-21T00:26:18.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.572+0000 7fc7ff8a9640 1 -- 172.21.15.112:0/269162703 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fc7e4078bc0 msgr2=0x7fc7e407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:18.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.572+0000 7fc7ff8a9640 1 --2- 172.21.15.112:0/269162703 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fc7e4078bc0 0x7fc7e407b0d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc7f815efd0 tx=0x7fc7e8008230 comp rx=0 tx=0).stop 2024-09-21T00:26:18.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.572+0000 7fc7ff8a9640 1 -- 172.21.15.112:0/269162703 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc7f80b5260 msgr2=0x7fc7f80b2250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:18.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.572+0000 7fc7ff8a9640 1 --2- 172.21.15.112:0/269162703 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc7f80b5260 0x7fc7f80b2250 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fc7ec00c970 tx=0x7fc7ec00ce30 comp rx=0 tx=0).stop 2024-09-21T00:26:18.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.572+0000 7fc7ff8a9640 1 -- 172.21.15.112:0/269162703 shutdown_connections 2024-09-21T00:26:18.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.572+0000 7fc7ff8a9640 1 --2- 172.21.15.112:0/269162703 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc7f815bfc0 0x7fc7f80b2790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:18.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.573+0000 7fc7ff8a9640 1 --2- 172.21.15.112:0/269162703 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fc7e4078bc0 0x7fc7e407b0d0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:18.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.573+0000 7fc7ff8a9640 1 --2- 172.21.15.112:0/269162703 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc7f80b5260 0x7fc7f80b2250 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:18.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.573+0000 7fc7ff8a9640 1 --2- 172.21.15.112:0/269162703 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc7f806def0 0x7fc7f8168d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:18.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.573+0000 7fc7ff8a9640 1 -- 172.21.15.112:0/269162703 >> 172.21.15.112:0/269162703 conn(0x7fc7f80cdca0 msgr2=0x7fc7f80b3140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:18.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.573+0000 7fc7ff8a9640 1 -- 172.21.15.112:0/269162703 shutdown_connections 2024-09-21T00:26:18.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.573+0000 7fc7ff8a9640 1 -- 172.21.15.112:0/269162703 wait complete. 2024-09-21T00:26:18.585 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-21T00:26:18.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.699+0000 7fc00e863640 1 -- 172.21.15.112:0/4277016887 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0080b9ad0 msgr2=0x7fc0080d0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:18.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.699+0000 7fc00e863640 1 --2- 172.21.15.112:0/4277016887 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0080b9ad0 0x7fc0080d0be0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fc0081623b0 tx=0x7fbff002a4b0 comp rx=0 tx=0).stop 2024-09-21T00:26:18.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.699+0000 7fc00e863640 1 -- 172.21.15.112:0/4277016887 shutdown_connections 2024-09-21T00:26:18.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.699+0000 7fc00e863640 1 --2- 172.21.15.112:0/4277016887 >> v2:172.21.15.187:3300/0 conn(0x7fc0080d1120 0x7fc0080ceb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:18.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.699+0000 7fc00e863640 1 --2- 172.21.15.112:0/4277016887 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0080b9ad0 0x7fc0080d0be0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:18.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.699+0000 7fc00e863640 1 --2- 172.21.15.112:0/4277016887 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc00815ee80 0x7fc0080d0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:18.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.699+0000 7fc00e863640 1 -- 172.21.15.112:0/4277016887 >> 172.21.15.112:0/4277016887 conn(0x7fc00809d830 msgr2=0x7fc0080b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:18.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.699+0000 7fc00e863640 1 -- 172.21.15.112:0/4277016887 shutdown_connections 2024-09-21T00:26:18.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.699+0000 7fc00e863640 1 -- 172.21.15.112:0/4277016887 wait complete. 2024-09-21T00:26:18.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.701+0000 7fc00e863640 1 Processor -- start 2024-09-21T00:26:18.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.701+0000 7fc00e863640 1 -- start start 2024-09-21T00:26:18.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.702+0000 7fc00e863640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc0080b9ad0 0x7fc008157ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:18.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.702+0000 7fc00e863640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc0080d1120 0x7fc008155620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:18.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.703+0000 7fc00e863640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc00815ee80 0x7fc008156080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:18.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.703+0000 7fc00e863640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc0080cf460 con 0x7fc00815ee80 2024-09-21T00:26:18.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.703+0000 7fc00e863640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc0080cf2e0 con 0x7fc0080d1120 2024-09-21T00:26:18.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.703+0000 7fc00e863640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc0080b6580 con 0x7fc0080b9ad0 2024-09-21T00:26:18.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.703+0000 7fc00ce00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc00815ee80 0x7fc008156080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:18.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.703+0000 7fc00ce00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc00815ee80 0x7fc008156080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41362/0 (socket says 172.21.15.112:41362) 2024-09-21T00:26:18.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.703+0000 7fc00ce00640 1 -- 172.21.15.112:0/3438450296 learned_addr learned my addr 172.21.15.112:0/3438450296 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:26:18.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.703+0000 7fbfff7fe640 1 --2- 172.21.15.112:0/3438450296 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc0080d1120 0x7fc008155620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:18.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.703+0000 7fc007fff640 1 --2- 172.21.15.112:0/3438450296 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc0080b9ad0 0x7fc008157ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:18.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.703+0000 7fc00ce00640 1 -- 172.21.15.112:0/3438450296 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc0080b9ad0 msgr2=0x7fc008157ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:18.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.703+0000 7fc00ce00640 1 --2- 172.21.15.112:0/3438450296 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc0080b9ad0 0x7fc008157ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:18.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.703+0000 7fc00ce00640 1 -- 172.21.15.112:0/3438450296 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc0080d1120 msgr2=0x7fc008155620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:18.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.703+0000 7fc00ce00640 1 --2- 172.21.15.112:0/3438450296 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc0080d1120 0x7fc008155620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:18.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.703+0000 7fc00ce00640 1 -- 172.21.15.112:0/3438450296 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc008158330 con 0x7fc00815ee80 2024-09-21T00:26:18.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.703+0000 7fc00ce00640 1 --2- 172.21.15.112:0/3438450296 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc00815ee80 0x7fc008156080 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc00000ca90 tx=0x7fc00000cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:18.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.703+0000 7fc0057fa640 1 -- 172.21.15.112:0/3438450296 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc000007df0 con 0x7fc00815ee80 2024-09-21T00:26:18.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.703+0000 7fc0057fa640 1 -- 172.21.15.112:0/3438450296 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc00000f540 con 0x7fc00815ee80 2024-09-21T00:26:18.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.703+0000 7fc0057fa640 1 -- 172.21.15.112:0/3438450296 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc00000f990 con 0x7fc00815ee80 2024-09-21T00:26:18.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.703+0000 7fc00e863640 1 -- 172.21.15.112:0/3438450296 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc008158610 con 0x7fc00815ee80 2024-09-21T00:26:18.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.703+0000 7fc00e863640 1 -- 172.21.15.112:0/3438450296 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0081f8eb0 con 0x7fc00815ee80 2024-09-21T00:26:18.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.705+0000 7fc00e863640 1 -- 172.21.15.112:0/3438450296 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0081580a0 con 0x7fc00815ee80 2024-09-21T00:26:18.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.705+0000 7fc0057fa640 1 -- 172.21.15.112:0/3438450296 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc00000fb30 con 0x7fc00815ee80 2024-09-21T00:26:18.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.706+0000 7fc0057fa640 1 --2- 172.21.15.112:0/3438450296 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fbfe8078bc0 0x7fbfe807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:18.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.706+0000 7fc0057fa640 1 -- 172.21.15.112:0/3438450296 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fc00009ba10 con 0x7fc00815ee80 2024-09-21T00:26:18.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.706+0000 7fc007fff640 1 --2- 172.21.15.112:0/3438450296 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fbfe8078bc0 0x7fbfe807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:18.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.707+0000 7fc007fff640 1 --2- 172.21.15.112:0/3438450296 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fbfe8078bc0 0x7fbfe807b0d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fbff8004630 tx=0x7fbff800b2e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:18.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.710+0000 7fc0057fa640 1 -- 172.21.15.112:0/3438450296 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc000014070 con 0x7fc00815ee80 2024-09-21T00:26:18.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.888+0000 7fc00e863640 1 -- 172.21.15.112:0/3438450296 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc0080ceb90 con 0x7fc00815ee80 2024-09-21T00:26:18.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.888+0000 7fc0057fa640 1 -- 172.21.15.112:0/3438450296 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7fc00000fe20 con 0x7fc00815ee80 2024-09-21T00:26:18.889 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 64 2024-09-21T00:26:18.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.890+0000 7fc00e863640 1 -- 172.21.15.112:0/3438450296 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fbfe8078bc0 msgr2=0x7fbfe807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:18.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.890+0000 7fc00e863640 1 --2- 172.21.15.112:0/3438450296 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fbfe8078bc0 0x7fbfe807b0d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fbff8004630 tx=0x7fbff800b2e0 comp rx=0 tx=0).stop 2024-09-21T00:26:18.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.890+0000 7fc00e863640 1 -- 172.21.15.112:0/3438450296 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc00815ee80 msgr2=0x7fc008156080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:18.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.890+0000 7fc00e863640 1 --2- 172.21.15.112:0/3438450296 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc00815ee80 0x7fc008156080 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc00000ca90 tx=0x7fc00000cf50 comp rx=0 tx=0).stop 2024-09-21T00:26:18.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.890+0000 7fc00e863640 1 -- 172.21.15.112:0/3438450296 shutdown_connections 2024-09-21T00:26:18.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.890+0000 7fc00e863640 1 --2- 172.21.15.112:0/3438450296 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fbfe8078bc0 0x7fbfe807b0d0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:18.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.890+0000 7fc00e863640 1 --2- 172.21.15.112:0/3438450296 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc00815ee80 0x7fc008156080 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:18.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.890+0000 7fc00e863640 1 --2- 172.21.15.112:0/3438450296 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc0080d1120 0x7fc008155620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:18.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.890+0000 7fc00e863640 1 --2- 172.21.15.112:0/3438450296 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc0080b9ad0 0x7fc008157ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:18.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.890+0000 7fc00e863640 1 -- 172.21.15.112:0/3438450296 >> 172.21.15.112:0/3438450296 conn(0x7fc00809d830 msgr2=0x7fc0080d09e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:18.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.890+0000 7fc00e863640 1 -- 172.21.15.112:0/3438450296 shutdown_connections 2024-09-21T00:26:18.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:18.890+0000 7fc00e863640 1 -- 172.21.15.112:0/3438450296 wait complete. 2024-09-21T00:26:18.902 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-21T00:26:19.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:19.017+0000 7f6782dad640 1 -- 172.21.15.112:0/3988712025 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f676c0051a0 msgr2=0x7f676c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:19.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:19.017+0000 7f6782dad640 1 --2- 172.21.15.112:0/3988712025 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f676c0051a0 0x7f676c00b9c0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f6774001ed0 tx=0x7f676c005830 comp rx=0 tx=0).stop 2024-09-21T00:26:19.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:19.017+0000 7f678134a640 1 -- 172.21.15.112:0/3988712025 reap_dead start 2024-09-21T00:26:19.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:19.017+0000 7f6782dad640 1 -- 172.21.15.112:0/3988712025 shutdown_connections 2024-09-21T00:26:19.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:19.017+0000 7f6782dad640 1 -- 172.21.15.112:0/3988712025 >> 172.21.15.112:0/3988712025 conn(0x7f677c0b1b90 msgr2=0x7f677c0b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:19.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:19.017+0000 7f6782dad640 1 -- 172.21.15.112:0/3988712025 shutdown_connections 2024-09-21T00:26:19.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:19.017+0000 7f6782dad640 1 -- 172.21.15.112:0/3988712025 wait complete. 2024-09-21T00:26:19.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:19.019+0000 7f6782dad640 1 Processor -- start 2024-09-21T00:26:19.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:19.019+0000 7f6782dad640 1 -- start start 2024-09-21T00:26:19.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:19.020+0000 7f6782dad640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f676c0051a0 0x7f677c0a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:19.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:19.020+0000 7f6782dad640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f677c09f3b0 0x7f677c1f8e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:19.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:19.020+0000 7f6782dad640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f677c1f45c0 0x7f677c0b2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:19.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:19.020+0000 7f6782dad640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f677c06e500 con 0x7f677c09f3b0 2024-09-21T00:26:19.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:19.020+0000 7f6782dad640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f677c0cfcf0 con 0x7f676c0051a0 2024-09-21T00:26:19.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:19.020+0000 7f6782dad640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f677c06e680 con 0x7f677c1f45c0 2024-09-21T00:26:19.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:19.020+0000 7f677bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f677c09f3b0 0x7f677c1f8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:19.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:19.020+0000 7f677bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f677c09f3b0 0x7f677c1f8e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41376/0 (socket says 172.21.15.112:41376) 2024-09-21T00:26:19.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:19.020+0000 7f677bfff640 1 -- 172.21.15.112:0/1559211785 learned_addr learned my addr 172.21.15.112:0/1559211785 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:26:19.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:19.021+0000 7f6780b49640 1 --2- 172.21.15.112:0/1559211785 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f676c0051a0 0x7f677c0a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:19.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:19.021+0000 7f678134a640 1 --2- 172.21.15.112:0/1559211785 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f677c1f45c0 0x7f677c0b2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:19.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:19.021+0000 7f677bfff640 1 -- 172.21.15.112:0/1559211785 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f677c1f45c0 msgr2=0x7f677c0b2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:19.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:19.021+0000 7f677bfff640 1 --2- 172.21.15.112:0/1559211785 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f677c1f45c0 0x7f677c0b2ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:19.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:19.021+0000 7f677bfff640 1 -- 172.21.15.112:0/1559211785 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f676c0051a0 msgr2=0x7f677c0a0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:19.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:19.021+0000 7f677bfff640 1 --2- 172.21.15.112:0/1559211785 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f676c0051a0 0x7f677c0a0b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:19.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:19.021+0000 7f677bfff640 1 -- 172.21.15.112:0/1559211785 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f677c09fac0 con 0x7f677c09f3b0 2024-09-21T00:26:19.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:19.021+0000 7f677bfff640 1 --2- 172.21.15.112:0/1559211785 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f677c09f3b0 0x7f677c1f8e60 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f6768011b10 tx=0x7f6768009e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:19.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:19.021+0000 7f67797fa640 1 -- 172.21.15.112:0/1559211785 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f676801e070 con 0x7f677c09f3b0 2024-09-21T00:26:19.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:19.021+0000 7f67797fa640 1 -- 172.21.15.112:0/1559211785 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6768004050 con 0x7f677c09f3b0 2024-09-21T00:26:19.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:19.021+0000 7f6782dad640 1 -- 172.21.15.112:0/1559211785 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f677c09fda0 con 0x7f677c09f3b0 2024-09-21T00:26:19.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:19.021+0000 7f67797fa640 1 -- 172.21.15.112:0/1559211785 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f676800fea0 con 0x7f677c09f3b0 2024-09-21T00:26:19.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:19.021+0000 7f6782dad640 1 -- 172.21.15.112:0/1559211785 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f677c0a0360 con 0x7f677c09f3b0 2024-09-21T00:26:19.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:19.022+0000 7f6782dad640 1 -- 172.21.15.112:0/1559211785 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f677c0cf450 con 0x7f677c09f3b0 2024-09-21T00:26:19.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:19.023+0000 7f67797fa640 1 -- 172.21.15.112:0/1559211785 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6768021070 con 0x7f677c09f3b0 2024-09-21T00:26:19.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:19.023+0000 7f67797fa640 1 --2- 172.21.15.112:0/1559211785 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f6760078bc0 0x7f676007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:19.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:19.023+0000 7f67797fa640 1 -- 172.21.15.112:0/1559211785 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f676809c300 con 0x7f677c09f3b0 2024-09-21T00:26:19.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:19.024+0000 7f6780b49640 1 --2- 172.21.15.112:0/1559211785 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f6760078bc0 0x7f676007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:19.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:19.024+0000 7f6780b49640 1 --2- 172.21.15.112:0/1559211785 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f6760078bc0 0x7f676007b0d0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6770001050 tx=0x7f6770001c10 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:19.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:19.027+0000 7f67797fa640 1 -- 172.21.15.112:0/1559211785 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f6768063f60 con 0x7f677c09f3b0 2024-09-21T00:26:19.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:19.203+0000 7f6782dad640 1 -- 172.21.15.112:0/1559211785 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f677c09f850 con 0x7f677c09f3b0 2024-09-21T00:26:19.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:19.204+0000 7f67797fa640 1 -- 172.21.15.112:0/1559211785 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7f6768068990 con 0x7f677c09f3b0 2024-09-21T00:26:19.204 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 32 2024-09-21T00:26:19.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:19.206+0000 7f6782dad640 1 -- 172.21.15.112:0/1559211785 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f6760078bc0 msgr2=0x7f676007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:19.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:19.206+0000 7f6782dad640 1 --2- 172.21.15.112:0/1559211785 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f6760078bc0 0x7f676007b0d0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6770001050 tx=0x7f6770001c10 comp rx=0 tx=0).stop 2024-09-21T00:26:19.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:19.206+0000 7f6782dad640 1 -- 172.21.15.112:0/1559211785 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f677c09f3b0 msgr2=0x7f677c1f8e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:19.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:19.206+0000 7f6782dad640 1 --2- 172.21.15.112:0/1559211785 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f677c09f3b0 0x7f677c1f8e60 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f6768011b10 tx=0x7f6768009e90 comp rx=0 tx=0).stop 2024-09-21T00:26:19.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:19.206+0000 7f6782dad640 1 -- 172.21.15.112:0/1559211785 shutdown_connections 2024-09-21T00:26:19.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:19.206+0000 7f6782dad640 1 --2- 172.21.15.112:0/1559211785 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f677c1f45c0 0x7f677c0b2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:19.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:19.206+0000 7f6782dad640 1 --2- 172.21.15.112:0/1559211785 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f6760078bc0 0x7f676007b0d0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:19.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:19.206+0000 7f6782dad640 1 --2- 172.21.15.112:0/1559211785 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f677c09f3b0 0x7f677c1f8e60 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:19.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:19.206+0000 7f6782dad640 1 --2- 172.21.15.112:0/1559211785 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f676c0051a0 0x7f677c0a0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:19.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:19.206+0000 7f6782dad640 1 -- 172.21.15.112:0/1559211785 >> 172.21.15.112:0/1559211785 conn(0x7f677c0b1b90 msgr2=0x7f677c15b1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:19.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:19.206+0000 7f6782dad640 1 -- 172.21.15.112:0/1559211785 shutdown_connections 2024-09-21T00:26:19.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:19.206+0000 7f6782dad640 1 -- 172.21.15.112:0/1559211785 wait complete. 2024-09-21T00:26:19.218 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_sessionmap.TestSessionMap.test_mount_conn_close' 2024-09-21T00:26:19.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:19.332+0000 7fa3dd551640 1 -- 172.21.15.112:0/609034742 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3d815e800 msgr2=0x7fa3d815ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:19.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:19.332+0000 7fa3dd551640 1 --2- 172.21.15.112:0/609034742 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3d815e800 0x7fa3d815ec10 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fa3d80cd750 tx=0x7fa3d002aa30 comp rx=0 tx=0).stop 2024-09-21T00:26:19.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:19.332+0000 7fa3dd551640 1 -- 172.21.15.112:0/609034742 shutdown_connections 2024-09-21T00:26:19.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:19.332+0000 7fa3dd551640 1 --2- 172.21.15.112:0/609034742 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3d815e800 0x7fa3d815ec10 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:19.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:19.332+0000 7fa3dd551640 1 --2- 172.21.15.112:0/609034742 >> v2:172.21.15.187:3300/0 conn(0x7fa3d8073e20 0x7fa3d8074230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:19.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:19.332+0000 7fa3dd551640 1 -- 172.21.15.112:0/609034742 >> 172.21.15.112:0/609034742 conn(0x7fa3d80cca40 msgr2=0x7fa3d80cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:19.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:19.332+0000 7fa3dd551640 1 -- 172.21.15.112:0/609034742 shutdown_connections 2024-09-21T00:26:19.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:19.332+0000 7fa3dd551640 1 -- 172.21.15.112:0/609034742 wait complete. 2024-09-21T00:26:19.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:19.334+0000 7fa3dd551640 1 Processor -- start 2024-09-21T00:26:19.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:19.334+0000 7fa3dd551640 1 -- start start 2024-09-21T00:26:19.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:19.335+0000 7fa3dd551640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3d8073e20 0x7fa3d81f3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:19.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:19.336+0000 7fa3dd551640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa3d80ca610 0x7fa3d80b6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:19.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:19.336+0000 7fa3dd551640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa3d815e800 0x7fa3d80b4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:19.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:19.336+0000 7fa3dd551640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa3d815f2d0 con 0x7fa3d8073e20 2024-09-21T00:26:19.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:19.336+0000 7fa3dd551640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa3d815f150 con 0x7fa3d80ca610 2024-09-21T00:26:19.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:19.336+0000 7fa3dd551640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa3d815f450 con 0x7fa3d815e800 2024-09-21T00:26:19.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:19.336+0000 7fa3d6ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3d8073e20 0x7fa3d81f3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:19.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:19.336+0000 7fa3d6ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3d8073e20 0x7fa3d81f3bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41400/0 (socket says 172.21.15.112:41400) 2024-09-21T00:26:19.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:19.336+0000 7fa3d6ffd640 1 -- 172.21.15.112:0/263709073 learned_addr learned my addr 172.21.15.112:0/263709073 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:26:19.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:19.336+0000 7fa3d67fc640 1 --2- 172.21.15.112:0/263709073 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa3d80ca610 0x7fa3d80b6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:19.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:19.336+0000 7fa3d77fe640 1 --2- 172.21.15.112:0/263709073 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa3d815e800 0x7fa3d80b4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:19.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:19.336+0000 7fa3d6ffd640 1 -- 172.21.15.112:0/263709073 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa3d815e800 msgr2=0x7fa3d80b4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:19.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:19.336+0000 7fa3d6ffd640 1 --2- 172.21.15.112:0/263709073 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa3d815e800 0x7fa3d80b4100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:19.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:19.336+0000 7fa3d6ffd640 1 -- 172.21.15.112:0/263709073 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa3d80ca610 msgr2=0x7fa3d80b6370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:19.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:19.336+0000 7fa3d6ffd640 1 --2- 172.21.15.112:0/263709073 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa3d80ca610 0x7fa3d80b6370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:19.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:19.336+0000 7fa3d6ffd640 1 -- 172.21.15.112:0/263709073 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3d80b3060 con 0x7fa3d8073e20 2024-09-21T00:26:19.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:19.336+0000 7fa3d6ffd640 1 --2- 172.21.15.112:0/263709073 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3d8073e20 0x7fa3d81f3bb0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa3c800b3f0 tx=0x7fa3c800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:19.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:19.336+0000 7fa3bf7fe640 1 -- 172.21.15.112:0/263709073 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3c8015020 con 0x7fa3d8073e20 2024-09-21T00:26:19.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:19.336+0000 7fa3bf7fe640 1 -- 172.21.15.112:0/263709073 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa3c8011920 con 0x7fa3d8073e20 2024-09-21T00:26:19.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:19.336+0000 7fa3dd551640 1 -- 172.21.15.112:0/263709073 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3d80b3340 con 0x7fa3d8073e20 2024-09-21T00:26:19.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:19.336+0000 7fa3bf7fe640 1 -- 172.21.15.112:0/263709073 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3c8011d70 con 0x7fa3d8073e20 2024-09-21T00:26:19.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:19.336+0000 7fa3dd551640 1 -- 172.21.15.112:0/263709073 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3d80b3900 con 0x7fa3d8073e20 2024-09-21T00:26:19.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:19.338+0000 7fa3dd551640 1 -- 172.21.15.112:0/263709073 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3d8074230 con 0x7fa3d8073e20 2024-09-21T00:26:19.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:19.338+0000 7fa3bf7fe640 1 -- 172.21.15.112:0/263709073 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa3c8018020 con 0x7fa3d8073e20 2024-09-21T00:26:19.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:19.339+0000 7fa3bf7fe640 1 --2- 172.21.15.112:0/263709073 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fa3c4078b90 0x7fa3c407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:19.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:19.339+0000 7fa3bf7fe640 1 -- 172.21.15.112:0/263709073 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fa3c809ae60 con 0x7fa3d8073e20 2024-09-21T00:26:19.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:19.339+0000 7fa3d67fc640 1 --2- 172.21.15.112:0/263709073 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fa3c4078b90 0x7fa3c407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:19.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:19.340+0000 7fa3d67fc640 1 --2- 172.21.15.112:0/263709073 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fa3c4078b90 0x7fa3c407b0a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc0058c0 tx=0x7fa3cc005f60 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:19.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:19.343+0000 7fa3bf7fe640 1 -- 172.21.15.112:0/263709073 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa3c801f020 con 0x7fa3d8073e20 2024-09-21T00:26:19.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:19.598+0000 7fa3dd551640 1 -- 172.21.15.112:0/263709073 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_sessionmap.TestSessionMap.test_mount_conn_close"]} v 0) -- 0x7fa3d8075050 con 0x7fa3d8073e20 2024-09-21T00:26:20.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:20.365+0000 7fa3bf7fe640 1 -- 172.21.15.112:0/263709073 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_sessionmap.TestSessionMap.test_mount_conn_close"]}]=0 v54) ==== 147+0+0 (secure 0 0 0) 0x7fa3c8062a30 con 0x7fa3d8073e20 2024-09-21T00:26:20.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:20.368+0000 7fa3dd551640 1 -- 172.21.15.112:0/263709073 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fa3c4078b90 msgr2=0x7fa3c407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:20.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:20.368+0000 7fa3dd551640 1 --2- 172.21.15.112:0/263709073 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fa3c4078b90 0x7fa3c407b0a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc0058c0 tx=0x7fa3cc005f60 comp rx=0 tx=0).stop 2024-09-21T00:26:20.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:20.368+0000 7fa3dd551640 1 -- 172.21.15.112:0/263709073 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3d8073e20 msgr2=0x7fa3d81f3bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:20.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:20.368+0000 7fa3dd551640 1 --2- 172.21.15.112:0/263709073 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3d8073e20 0x7fa3d81f3bb0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa3c800b3f0 tx=0x7fa3c800b8b0 comp rx=0 tx=0).stop 2024-09-21T00:26:20.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:20.368+0000 7fa3dd551640 1 -- 172.21.15.112:0/263709073 shutdown_connections 2024-09-21T00:26:20.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:20.368+0000 7fa3dd551640 1 --2- 172.21.15.112:0/263709073 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa3d815e800 0x7fa3d80b4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:20.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:20.369+0000 7fa3dd551640 1 --2- 172.21.15.112:0/263709073 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa3d80ca610 0x7fa3d80b6370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:20.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:20.369+0000 7fa3dd551640 1 --2- 172.21.15.112:0/263709073 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fa3c4078b90 0x7fa3c407b0a0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:20.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:20.369+0000 7fa3dd551640 1 --2- 172.21.15.112:0/263709073 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3d8073e20 0x7fa3d81f3bb0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:20.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:20.369+0000 7fa3dd551640 1 -- 172.21.15.112:0/263709073 >> 172.21.15.112:0/263709073 conn(0x7fa3d80cca40 msgr2=0x7fa3d8154d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:20.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:20.369+0000 7fa3dd551640 1 -- 172.21.15.112:0/263709073 shutdown_connections 2024-09-21T00:26:20.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:20.369+0000 7fa3dd551640 1 -- 172.21.15.112:0/263709073 wait complete. 2024-09-21T00:26:20.385 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-21T00:26:20.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:20.498+0000 7ffb8378a640 1 -- 172.21.15.112:0/3700267663 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb7c0d01f0 msgr2=0x7ffb7c15daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:20.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:20.498+0000 7ffb8378a640 1 --2- 172.21.15.112:0/3700267663 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb7c0d01f0 0x7ffb7c15daf0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ffb78005fd0 tx=0x7ffb7802a530 comp rx=0 tx=0).stop 2024-09-21T00:26:20.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:20.498+0000 7ffb8378a640 1 -- 172.21.15.112:0/3700267663 shutdown_connections 2024-09-21T00:26:20.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:20.498+0000 7ffb8378a640 1 --2- 172.21.15.112:0/3700267663 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffb7c0d0980 0x7ffb7c0d0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:20.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:20.498+0000 7ffb8378a640 1 --2- 172.21.15.112:0/3700267663 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb7c0d01f0 0x7ffb7c15daf0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:20.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:20.498+0000 7ffb8378a640 1 -- 172.21.15.112:0/3700267663 >> 172.21.15.112:0/3700267663 conn(0x7ffb7c072990 msgr2=0x7ffb7c072da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:20.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:20.498+0000 7ffb8378a640 1 -- 172.21.15.112:0/3700267663 shutdown_connections 2024-09-21T00:26:20.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:20.498+0000 7ffb8378a640 1 -- 172.21.15.112:0/3700267663 wait complete. 2024-09-21T00:26:20.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:20.500+0000 7ffb8378a640 1 Processor -- start 2024-09-21T00:26:20.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:20.500+0000 7ffb8378a640 1 -- start start 2024-09-21T00:26:20.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:20.501+0000 7ffb8378a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffb7c0d01f0 0x7ffb7c0cbe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:20.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:20.501+0000 7ffb8378a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffb7c0d0980 0x7ffb7c0c6f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:20.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:20.501+0000 7ffb8378a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb7c15a6f0 0x7ffb7c148710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:20.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:20.501+0000 7ffb8378a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ffb7c0cd340 con 0x7ffb7c15a6f0 2024-09-21T00:26:20.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:20.501+0000 7ffb8378a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ffb7c0cd1c0 con 0x7ffb7c0d01f0 2024-09-21T00:26:20.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:20.501+0000 7ffb8378a640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ffb7c0cd4c0 con 0x7ffb7c0d0980 2024-09-21T00:26:20.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:20.501+0000 7ffb81d27640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb7c15a6f0 0x7ffb7c148710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:20.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:20.501+0000 7ffb81d27640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb7c15a6f0 0x7ffb7c148710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41414/0 (socket says 172.21.15.112:41414) 2024-09-21T00:26:20.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:20.501+0000 7ffb81d27640 1 -- 172.21.15.112:0/4088361892 learned_addr learned my addr 172.21.15.112:0/4088361892 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:26:20.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:20.501+0000 7ffb81526640 1 --2- 172.21.15.112:0/4088361892 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffb7c0d01f0 0x7ffb7c0cbe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:20.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:20.501+0000 7ffb80d25640 1 --2- 172.21.15.112:0/4088361892 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffb7c0d0980 0x7ffb7c0c6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:20.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:20.501+0000 7ffb81d27640 1 -- 172.21.15.112:0/4088361892 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffb7c0d0980 msgr2=0x7ffb7c0c6f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:20.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:20.501+0000 7ffb81d27640 1 --2- 172.21.15.112:0/4088361892 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffb7c0d0980 0x7ffb7c0c6f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:20.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:20.501+0000 7ffb81d27640 1 -- 172.21.15.112:0/4088361892 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffb7c0d01f0 msgr2=0x7ffb7c0cbe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:20.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:20.501+0000 7ffb81d27640 1 --2- 172.21.15.112:0/4088361892 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffb7c0d01f0 0x7ffb7c0cbe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:20.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:20.501+0000 7ffb81d27640 1 -- 172.21.15.112:0/4088361892 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb7c0c7990 con 0x7ffb7c15a6f0 2024-09-21T00:26:20.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:20.501+0000 7ffb81d27640 1 --2- 172.21.15.112:0/4088361892 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb7c15a6f0 0x7ffb7c148710 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ffb7400cab0 tx=0x7ffb7400cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:20.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:20.502+0000 7ffb71ffb640 1 -- 172.21.15.112:0/4088361892 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb74007d90 con 0x7ffb7c15a6f0 2024-09-21T00:26:20.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:20.502+0000 7ffb71ffb640 1 -- 172.21.15.112:0/4088361892 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffb7400f560 con 0x7ffb7c15a6f0 2024-09-21T00:26:20.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:20.502+0000 7ffb8378a640 1 -- 172.21.15.112:0/4088361892 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb7c0c7c70 con 0x7ffb7c15a6f0 2024-09-21T00:26:20.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:20.502+0000 7ffb71ffb640 1 -- 172.21.15.112:0/4088361892 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb7400f9b0 con 0x7ffb7c15a6f0 2024-09-21T00:26:20.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:20.502+0000 7ffb8378a640 1 -- 172.21.15.112:0/4088361892 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb7c157740 con 0x7ffb7c15a6f0 2024-09-21T00:26:20.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:20.503+0000 7ffb8378a640 1 -- 172.21.15.112:0/4088361892 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb7c1579a0 con 0x7ffb7c15a6f0 2024-09-21T00:26:20.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:20.503+0000 7ffb71ffb640 1 -- 172.21.15.112:0/4088361892 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ffb7400fb50 con 0x7ffb7c15a6f0 2024-09-21T00:26:20.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:20.504+0000 7ffb71ffb640 1 --2- 172.21.15.112:0/4088361892 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ffb60078b90 0x7ffb6007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:20.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:20.504+0000 7ffb71ffb640 1 -- 172.21.15.112:0/4088361892 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7ffb7409ba30 con 0x7ffb7c15a6f0 2024-09-21T00:26:20.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:20.504+0000 7ffb81526640 1 --2- 172.21.15.112:0/4088361892 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ffb60078b90 0x7ffb6007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:20.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:20.505+0000 7ffb81526640 1 --2- 172.21.15.112:0/4088361892 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ffb60078b90 0x7ffb6007b0a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ffb78033000 tx=0x7ffb78004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:20.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:20.508+0000 7ffb71ffb640 1 -- 172.21.15.112:0/4088361892 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ffb74014070 con 0x7ffb7c15a6f0 2024-09-21T00:26:20.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:20.685+0000 7ffb8378a640 1 -- 172.21.15.112:0/4088361892 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7ffb7c149540 con 0x7ffb7c15a6f0 2024-09-21T00:26:20.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:20.685+0000 7ffb71ffb640 1 -- 172.21.15.112:0/4088361892 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v31) ==== 59+0+2294 (secure 0 0 0) 0x7ffb74063bc0 con 0x7ffb7c15a6f0 2024-09-21T00:26:20.687 INFO:teuthology.orchestra.run.smithi112.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-21T00:26:20.687 INFO:teuthology.orchestra.run.smithi112.stdout:mon advanced mon_op_complaint_time 120 2024-09-21T00:26:20.687 INFO:teuthology.orchestra.run.smithi112.stdout:mgr advanced client_mount_timeout 30 2024-09-21T00:26:20.687 INFO:teuthology.orchestra.run.smithi112.stdout:mgr advanced debug_client 20/20 2024-09-21T00:26:20.688 INFO:teuthology.orchestra.run.smithi112.stdout:mgr advanced debug_mgr 20/20 2024-09-21T00:26:20.688 INFO:teuthology.orchestra.run.smithi112.stdout:mgr advanced debug_ms 1/1 2024-09-21T00:26:20.688 INFO:teuthology.orchestra.run.smithi112.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-21T00:26:20.688 INFO:teuthology.orchestra.run.smithi112.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-21T00:26:20.688 INFO:teuthology.orchestra.run.smithi112.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 5090.401743 2024-09-21T00:26:20.688 INFO:teuthology.orchestra.run.smithi112.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 7876.743127 2024-09-21T00:26:20.688 INFO:teuthology.orchestra.run.smithi112.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 4865.073913 2024-09-21T00:26:20.688 INFO:teuthology.orchestra.run.smithi112.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 5117.206240 2024-09-21T00:26:20.688 INFO:teuthology.orchestra.run.smithi112.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 4329.674604 2024-09-21T00:26:20.688 INFO:teuthology.orchestra.run.smithi112.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4322.652151 2024-09-21T00:26:20.688 INFO:teuthology.orchestra.run.smithi112.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 4244.674337 2024-09-21T00:26:20.688 INFO:teuthology.orchestra.run.smithi112.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 4472.316624 2024-09-21T00:26:20.688 INFO:teuthology.orchestra.run.smithi112.stdout:mds advanced debug_mds 20/20 2024-09-21T00:26:20.689 INFO:teuthology.orchestra.run.smithi112.stdout:mds advanced debug_mds_balancer 20/20 2024-09-21T00:26:20.689 INFO:teuthology.orchestra.run.smithi112.stdout:mds advanced debug_ms 2/2 2024-09-21T00:26:20.689 INFO:teuthology.orchestra.run.smithi112.stdout:mds dev mds_debug_frag true 2024-09-21T00:26:20.689 INFO:teuthology.orchestra.run.smithi112.stdout:mds dev mds_debug_scatterstat true 2024-09-21T00:26:20.689 INFO:teuthology.orchestra.run.smithi112.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-21T00:26:20.689 INFO:teuthology.orchestra.run.smithi112.stdout:mds dev mds_verify_scatter true 2024-09-21T00:26:20.689 INFO:teuthology.orchestra.run.smithi112.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-21T00:26:20.689 INFO:teuthology.orchestra.run.smithi112.stdout:mds advanced rados_mon_op_timeout 900 2024-09-21T00:26:20.689 INFO:teuthology.orchestra.run.smithi112.stdout:mds advanced rados_osd_op_timeout 900 2024-09-21T00:26:20.689 INFO:teuthology.orchestra.run.smithi112.stdout:client advanced client_mount_timeout 600 2024-09-21T00:26:20.689 INFO:teuthology.orchestra.run.smithi112.stdout:client advanced debug_client 20/20 2024-09-21T00:26:20.689 INFO:teuthology.orchestra.run.smithi112.stdout:client advanced debug_ms 1/1 2024-09-21T00:26:20.689 INFO:teuthology.orchestra.run.smithi112.stdout:client advanced rados_mon_op_timeout 900 2024-09-21T00:26:20.690 INFO:teuthology.orchestra.run.smithi112.stdout:client advanced rados_osd_op_timeout 900 2024-09-21T00:26:20.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:20.687+0000 7ffb8378a640 1 -- 172.21.15.112:0/4088361892 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ffb60078b90 msgr2=0x7ffb6007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:20.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:20.687+0000 7ffb8378a640 1 --2- 172.21.15.112:0/4088361892 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ffb60078b90 0x7ffb6007b0a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ffb78033000 tx=0x7ffb78004730 comp rx=0 tx=0).stop 2024-09-21T00:26:20.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:20.687+0000 7ffb8378a640 1 -- 172.21.15.112:0/4088361892 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb7c15a6f0 msgr2=0x7ffb7c148710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:20.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:20.687+0000 7ffb8378a640 1 --2- 172.21.15.112:0/4088361892 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb7c15a6f0 0x7ffb7c148710 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ffb7400cab0 tx=0x7ffb7400cf70 comp rx=0 tx=0).stop 2024-09-21T00:26:20.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:20.687+0000 7ffb8378a640 1 -- 172.21.15.112:0/4088361892 shutdown_connections 2024-09-21T00:26:20.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:20.687+0000 7ffb8378a640 1 --2- 172.21.15.112:0/4088361892 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ffb60078b90 0x7ffb6007b0a0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:20.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:20.688+0000 7ffb8378a640 1 --2- 172.21.15.112:0/4088361892 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb7c15a6f0 0x7ffb7c148710 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:20.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:20.688+0000 7ffb8378a640 1 --2- 172.21.15.112:0/4088361892 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffb7c0d0980 0x7ffb7c0c6f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:20.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:20.688+0000 7ffb8378a640 1 --2- 172.21.15.112:0/4088361892 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffb7c0d01f0 0x7ffb7c0cbe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:20.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:20.688+0000 7ffb8378a640 1 -- 172.21.15.112:0/4088361892 >> 172.21.15.112:0/4088361892 conn(0x7ffb7c072990 msgr2=0x7ffb7c15c740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:20.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:20.688+0000 7ffb8378a640 1 -- 172.21.15.112:0/4088361892 shutdown_connections 2024-09-21T00:26:20.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:20.688+0000 7ffb8378a640 1 -- 172.21.15.112:0/4088361892 wait complete. 2024-09-21T00:26:20.701 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-21T00:26:20.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:20.813+0000 7f04fe10b640 1 -- 172.21.15.112:0/2936103364 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04f815ee80 msgr2=0x7f04f80d0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:20.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:20.813+0000 7f04fe10b640 1 --2- 172.21.15.112:0/2936103364 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04f815ee80 0x7f04f80d0170 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f04e8009830 tx=0x7f04e802a480 comp rx=0 tx=0).stop 2024-09-21T00:26:20.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:20.813+0000 7f04fe10b640 1 -- 172.21.15.112:0/2936103364 shutdown_connections 2024-09-21T00:26:20.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:20.813+0000 7f04fe10b640 1 --2- 172.21.15.112:0/2936103364 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04f80ceb90 0x7f04f80cefa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:20.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:20.813+0000 7f04fe10b640 1 --2- 172.21.15.112:0/2936103364 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04f815ee80 0x7f04f80d0170 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:20.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:20.813+0000 7f04fe10b640 1 -- 172.21.15.112:0/2936103364 >> 172.21.15.112:0/2936103364 conn(0x7f04f809d830 msgr2=0x7f04f80b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:20.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:20.814+0000 7f04fe10b640 1 -- 172.21.15.112:0/2936103364 shutdown_connections 2024-09-21T00:26:20.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:20.814+0000 7f04fe10b640 1 -- 172.21.15.112:0/2936103364 wait complete. 2024-09-21T00:26:20.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:20.815+0000 7f04fe10b640 1 Processor -- start 2024-09-21T00:26:20.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:20.815+0000 7f04fe10b640 1 -- start start 2024-09-21T00:26:20.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:20.817+0000 7f04fe10b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04f80b9ad0 0x7f04f81507f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:20.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:20.817+0000 7f04fe10b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04f80ceb90 0x7f04f806f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:20.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:20.817+0000 7f04fe10b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04f815ee80 0x7f04f8161780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:20.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:20.817+0000 7f04fe10b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f04f80b6700 con 0x7f04f815ee80 2024-09-21T00:26:20.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:20.817+0000 7f04fe10b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f04f80b6580 con 0x7f04f80ceb90 2024-09-21T00:26:20.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:20.817+0000 7f04fe10b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f04f80b6880 con 0x7f04f80b9ad0 2024-09-21T00:26:20.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:20.817+0000 7f04f7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04f815ee80 0x7f04f8161780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:20.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:20.817+0000 7f04f7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04f815ee80 0x7f04f8161780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41428/0 (socket says 172.21.15.112:41428) 2024-09-21T00:26:20.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:20.817+0000 7f04f7fff640 1 -- 172.21.15.112:0/40933379 learned_addr learned my addr 172.21.15.112:0/40933379 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:26:20.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:20.817+0000 7f04f77fe640 1 --2- 172.21.15.112:0/40933379 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04f80b9ad0 0x7f04f81507f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:20.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:20.817+0000 7f04f6ffd640 1 --2- 172.21.15.112:0/40933379 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04f80ceb90 0x7f04f806f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:20.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:20.817+0000 7f04f7fff640 1 -- 172.21.15.112:0/40933379 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04f80b9ad0 msgr2=0x7f04f81507f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:20.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:20.817+0000 7f04f7fff640 1 --2- 172.21.15.112:0/40933379 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04f80b9ad0 0x7f04f81507f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:20.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:20.817+0000 7f04f7fff640 1 -- 172.21.15.112:0/40933379 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04f80ceb90 msgr2=0x7f04f806f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:20.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:20.817+0000 7f04f7fff640 1 --2- 172.21.15.112:0/40933379 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04f80ceb90 0x7f04f806f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:20.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:20.817+0000 7f04f7fff640 1 -- 172.21.15.112:0/40933379 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04f806e080 con 0x7f04f815ee80 2024-09-21T00:26:20.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:20.817+0000 7f04f6ffd640 1 --2- 172.21.15.112:0/40933379 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04f80ceb90 0x7f04f806f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:26:20.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:20.817+0000 7f04f7fff640 1 --2- 172.21.15.112:0/40933379 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04f815ee80 0x7f04f8161780 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f04f000b970 tx=0x7f04f000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:20.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:20.817+0000 7f04effff640 1 -- 172.21.15.112:0/40933379 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f0019070 con 0x7f04f815ee80 2024-09-21T00:26:20.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:20.818+0000 7f04effff640 1 -- 172.21.15.112:0/40933379 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f04f000cc50 con 0x7f04f815ee80 2024-09-21T00:26:20.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:20.818+0000 7f04effff640 1 -- 172.21.15.112:0/40933379 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f00146a0 con 0x7f04f815ee80 2024-09-21T00:26:20.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:20.818+0000 7f04fe10b640 1 -- 172.21.15.112:0/40933379 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04f8151180 con 0x7f04f815ee80 2024-09-21T00:26:20.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:20.818+0000 7f04fe10b640 1 -- 172.21.15.112:0/40933379 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04f806e640 con 0x7f04f815ee80 2024-09-21T00:26:20.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:20.819+0000 7f04fe10b640 1 -- 172.21.15.112:0/40933379 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04f80b6d30 con 0x7f04f815ee80 2024-09-21T00:26:20.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:20.819+0000 7f04effff640 1 -- 172.21.15.112:0/40933379 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f04f001c070 con 0x7f04f815ee80 2024-09-21T00:26:20.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:20.820+0000 7f04effff640 1 --2- 172.21.15.112:0/40933379 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f04d8078b90 0x7f04d807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:20.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:20.820+0000 7f04effff640 1 -- 172.21.15.112:0/40933379 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f04f009ac80 con 0x7f04f815ee80 2024-09-21T00:26:20.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:20.820+0000 7f04f77fe640 1 --2- 172.21.15.112:0/40933379 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f04d8078b90 0x7f04d807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:20.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:20.820+0000 7f04f77fe640 1 --2- 172.21.15.112:0/40933379 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f04d8078b90 0x7f04d807b0a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f04e8034000 tx=0x7f04e8004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:20.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:20.824+0000 7f04effff640 1 -- 172.21.15.112:0/40933379 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f04f00627c0 con 0x7f04f815ee80 2024-09-21T00:26:21.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:21.004+0000 7f04fe10b640 1 -- 172.21.15.112:0/40933379 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f04f806a580 con 0x7f04f815ee80 2024-09-21T00:26:21.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:21.007+0000 7f04effff640 1 -- 172.21.15.112:0/40933379 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v32)=0 v32) ==== 122+0+0 (secure 0 0 0) 0x7f04f00671f0 con 0x7f04f815ee80 2024-09-21T00:26:21.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:21.009+0000 7f04fe10b640 1 -- 172.21.15.112:0/40933379 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f04d8078b90 msgr2=0x7f04d807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:21.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:21.009+0000 7f04fe10b640 1 --2- 172.21.15.112:0/40933379 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f04d8078b90 0x7f04d807b0a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f04e8034000 tx=0x7f04e8004730 comp rx=0 tx=0).stop 2024-09-21T00:26:21.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:21.009+0000 7f04fe10b640 1 -- 172.21.15.112:0/40933379 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04f815ee80 msgr2=0x7f04f8161780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:21.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:21.009+0000 7f04fe10b640 1 --2- 172.21.15.112:0/40933379 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04f815ee80 0x7f04f8161780 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f04f000b970 tx=0x7f04f000be30 comp rx=0 tx=0).stop 2024-09-21T00:26:21.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:21.009+0000 7f04fe10b640 1 -- 172.21.15.112:0/40933379 shutdown_connections 2024-09-21T00:26:21.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:21.009+0000 7f04fe10b640 1 --2- 172.21.15.112:0/40933379 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f04d8078b90 0x7f04d807b0a0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:21.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:21.009+0000 7f04fe10b640 1 --2- 172.21.15.112:0/40933379 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04f815ee80 0x7f04f8161780 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:21.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:21.009+0000 7f04fe10b640 1 --2- 172.21.15.112:0/40933379 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04f80ceb90 0x7f04f806f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:21.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:21.009+0000 7f04fe10b640 1 --2- 172.21.15.112:0/40933379 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04f80b9ad0 0x7f04f81507f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:21.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:21.009+0000 7f04fe10b640 1 -- 172.21.15.112:0/40933379 >> 172.21.15.112:0/40933379 conn(0x7f04f809d830 msgr2=0x7f04f80b4f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:21.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:21.010+0000 7f04fe10b640 1 -- 172.21.15.112:0/40933379 shutdown_connections 2024-09-21T00:26:21.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:21.010+0000 7f04fe10b640 1 -- 172.21.15.112:0/40933379 wait complete. 2024-09-21T00:26:21.022 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T00:26:21.023 DEBUG:teuthology.orchestra.run.smithi112:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T00:26:21.065 DEBUG:teuthology.orchestra.run.smithi187:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T00:26:21.110 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T00:26:21.110 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T00:26:21.110 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:26:21.126 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T00:26:21.126 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:26:21.182 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T00:26:21.182 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:26:21.239 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T00:26:21.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:21.392+0000 7f9d31a38640 1 -- 172.21.15.112:0/3071130267 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9d18004d40 msgr2=0x7f9d18005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:21.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:21.392+0000 7f9d31a38640 1 --2- 172.21.15.112:0/3071130267 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9d18004d40 0x7f9d18005210 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f9d24009e20 tx=0x7f9d18005830 comp rx=0 tx=0).stop 2024-09-21T00:26:21.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:21.392+0000 7f9d2b7fe640 1 -- 172.21.15.112:0/3071130267 reap_dead start 2024-09-21T00:26:21.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:21.393+0000 7f9d31a38640 1 -- 172.21.15.112:0/3071130267 shutdown_connections 2024-09-21T00:26:21.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:21.393+0000 7f9d31a38640 1 -- 172.21.15.112:0/3071130267 >> 172.21.15.112:0/3071130267 conn(0x7f9d2c09d830 msgr2=0x7f9d2c0b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:21.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:21.393+0000 7f9d31a38640 1 -- 172.21.15.112:0/3071130267 shutdown_connections 2024-09-21T00:26:21.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:21.393+0000 7f9d31a38640 1 -- 172.21.15.112:0/3071130267 wait complete. 2024-09-21T00:26:21.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:21.394+0000 7f9d31a38640 1 Processor -- start 2024-09-21T00:26:21.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:21.395+0000 7f9d31a38640 1 -- start start 2024-09-21T00:26:21.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:21.396+0000 7f9d31a38640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d18004d40 0x7f9d2c156110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:21.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:21.396+0000 7f9d31a38640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9d2c0ccc60 0x7f9d2c0cd070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:21.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:21.396+0000 7f9d31a38640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d2c06f060 0x7f9d2c06f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:21.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:21.396+0000 7f9d31a38640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9d2c0cf190 con 0x7f9d2c0ccc60 2024-09-21T00:26:21.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:21.396+0000 7f9d31a38640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9d2c0cf010 con 0x7f9d2c06f060 2024-09-21T00:26:21.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:21.396+0000 7f9d31a38640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f9d2c0cf310 con 0x7f9d18004d40 2024-09-21T00:26:21.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:21.396+0000 7f9d227fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9d2c0ccc60 0x7f9d2c0cd070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:21.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:21.396+0000 7f9d227fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9d2c0ccc60 0x7f9d2c0cd070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41452/0 (socket says 172.21.15.112:41452) 2024-09-21T00:26:21.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:21.396+0000 7f9d227fc640 1 -- 172.21.15.112:0/3506062250 learned_addr learned my addr 172.21.15.112:0/3506062250 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:26:21.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:21.396+0000 7f9d2affd640 1 --2- 172.21.15.112:0/3506062250 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d18004d40 0x7f9d2c156110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:21.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:21.396+0000 7f9d2b7fe640 1 --2- 172.21.15.112:0/3506062250 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d2c06f060 0x7f9d2c06f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:21.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:21.396+0000 7f9d227fc640 1 -- 172.21.15.112:0/3506062250 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d18004d40 msgr2=0x7f9d2c156110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:21.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:21.396+0000 7f9d227fc640 1 --2- 172.21.15.112:0/3506062250 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d18004d40 0x7f9d2c156110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:21.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:21.396+0000 7f9d227fc640 1 -- 172.21.15.112:0/3506062250 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d2c06f060 msgr2=0x7f9d2c06f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:21.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:21.396+0000 7f9d227fc640 1 --2- 172.21.15.112:0/3506062250 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d2c06f060 0x7f9d2c06f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:21.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:21.396+0000 7f9d227fc640 1 -- 172.21.15.112:0/3506062250 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d2c06f9b0 con 0x7f9d2c0ccc60 2024-09-21T00:26:21.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:21.396+0000 7f9d227fc640 1 --2- 172.21.15.112:0/3506062250 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9d2c0ccc60 0x7f9d2c0cd070 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f9d14010a70 tx=0x7f9d14010f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:21.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:21.396+0000 7f9d28ff9640 1 -- 172.21.15.112:0/3506062250 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d1400ee00 con 0x7f9d2c0ccc60 2024-09-21T00:26:21.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:21.397+0000 7f9d28ff9640 1 -- 172.21.15.112:0/3506062250 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9d14004020 con 0x7f9d2c0ccc60 2024-09-21T00:26:21.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:21.397+0000 7f9d31a38640 1 -- 172.21.15.112:0/3506062250 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d2c0d42c0 con 0x7f9d2c0ccc60 2024-09-21T00:26:21.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:21.397+0000 7f9d28ff9640 1 -- 172.21.15.112:0/3506062250 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d14004470 con 0x7f9d2c0ccc60 2024-09-21T00:26:21.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:21.397+0000 7f9d31a38640 1 -- 172.21.15.112:0/3506062250 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d2c0d4880 con 0x7f9d2c0ccc60 2024-09-21T00:26:21.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:21.397+0000 7f9d31a38640 1 -- 172.21.15.112:0/3506062250 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d2c0d4d20 con 0x7f9d2c0ccc60 2024-09-21T00:26:21.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:21.398+0000 7f9d28ff9640 1 -- 172.21.15.112:0/3506062250 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9d140035f0 con 0x7f9d2c0ccc60 2024-09-21T00:26:21.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:21.399+0000 7f9d28ff9640 1 --2- 172.21.15.112:0/3506062250 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f9d0c078bc0 0x7f9d0c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:21.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:21.399+0000 7f9d28ff9640 1 -- 172.21.15.112:0/3506062250 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f9d1409bca0 con 0x7f9d2c0ccc60 2024-09-21T00:26:21.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:21.399+0000 7f9d2affd640 1 --2- 172.21.15.112:0/3506062250 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f9d0c078bc0 0x7f9d0c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:21.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:21.399+0000 7f9d2affd640 1 --2- 172.21.15.112:0/3506062250 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f9d0c078bc0 0x7f9d0c07b0d0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9d1c0016c0 tx=0x7f9d1c0011b0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:21.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:21.403+0000 7f9d28ff9640 1 -- 172.21.15.112:0/3506062250 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f9d2c06f9b0 con 0x7f9d2c0ccc60 2024-09-21T00:26:21.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:21.592+0000 7f9d31a38640 1 -- 172.21.15.112:0/3506062250 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f9d2c0cd5b0 con 0x7f9d2c0ccc60 2024-09-21T00:26:21.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:21.593+0000 7f9d28ff9640 1 -- 172.21.15.112:0/3506062250 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7f9d14063da0 con 0x7f9d2c0ccc60 2024-09-21T00:26:21.595 INFO:teuthology.orchestra.run.smithi112.stderr:listed 0 entries 2024-09-21T00:26:21.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:21.595+0000 7f9d31a38640 1 -- 172.21.15.112:0/3506062250 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f9d0c078bc0 msgr2=0x7f9d0c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:21.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:21.595+0000 7f9d31a38640 1 --2- 172.21.15.112:0/3506062250 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f9d0c078bc0 0x7f9d0c07b0d0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9d1c0016c0 tx=0x7f9d1c0011b0 comp rx=0 tx=0).stop 2024-09-21T00:26:21.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:21.595+0000 7f9d31a38640 1 -- 172.21.15.112:0/3506062250 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9d2c0ccc60 msgr2=0x7f9d2c0cd070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:21.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:21.595+0000 7f9d31a38640 1 --2- 172.21.15.112:0/3506062250 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9d2c0ccc60 0x7f9d2c0cd070 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f9d14010a70 tx=0x7f9d14010f30 comp rx=0 tx=0).stop 2024-09-21T00:26:21.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:21.595+0000 7f9d31a38640 1 -- 172.21.15.112:0/3506062250 shutdown_connections 2024-09-21T00:26:21.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:21.595+0000 7f9d31a38640 1 --2- 172.21.15.112:0/3506062250 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d2c06f060 0x7f9d2c06f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:21.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:21.595+0000 7f9d31a38640 1 --2- 172.21.15.112:0/3506062250 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f9d0c078bc0 0x7f9d0c07b0d0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:21.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:21.595+0000 7f9d31a38640 1 --2- 172.21.15.112:0/3506062250 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9d2c0ccc60 0x7f9d2c0cd070 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:21.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:21.595+0000 7f9d31a38640 1 --2- 172.21.15.112:0/3506062250 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d18004d40 0x7f9d2c156110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:21.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:21.595+0000 7f9d31a38640 1 -- 172.21.15.112:0/3506062250 >> 172.21.15.112:0/3506062250 conn(0x7f9d2c09d830 msgr2=0x7f9d2c0b47d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:21.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:21.595+0000 7f9d31a38640 1 -- 172.21.15.112:0/3506062250 shutdown_connections 2024-09-21T00:26:21.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:21.595+0000 7f9d31a38640 1 -- 172.21.15.112:0/3506062250 wait complete. 2024-09-21T00:26:21.608 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-21T00:26:21.608 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:26:21.609 DEBUG:teuthology.orchestra.run.smithi112:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-09-21T00:26:21.635 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi112.front.sepia.ceph.com 2024-09-21T00:26:21.636 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:26:21.636 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T00:26:21.691 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-09-21T00:26:21.691 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:26:21.710 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-09-21T00:26:21.711 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:26:21.770 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-09-21T00:26:21.770 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:26:21.829 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T00:26:21.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:21.941+0000 7f22e1fc6640 1 -- 172.21.15.112:0/3354781783 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22dc156a50 msgr2=0x7f22dc074620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:21.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:21.941+0000 7f22e1fc6640 1 --2- 172.21.15.112:0/3354781783 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22dc156a50 0x7f22dc074620 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f22dc074160 tx=0x7f22cc02a560 comp rx=0 tx=0).stop 2024-09-21T00:26:21.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:21.942+0000 7f22e1fc6640 1 -- 172.21.15.112:0/3354781783 shutdown_connections 2024-09-21T00:26:21.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:21.942+0000 7f22e1fc6640 1 --2- 172.21.15.112:0/3354781783 >> v2:172.21.15.187:3300/0 conn(0x7f22dc074b60 0x7f22dc06ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:21.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:21.942+0000 7f22e1fc6640 1 --2- 172.21.15.112:0/3354781783 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22dc156a50 0x7f22dc074620 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:21.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:21.942+0000 7f22e1fc6640 1 -- 172.21.15.112:0/3354781783 >> 172.21.15.112:0/3354781783 conn(0x7f22dc0b1b90 msgr2=0x7f22dc0b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:21.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:21.942+0000 7f22e1fc6640 1 -- 172.21.15.112:0/3354781783 shutdown_connections 2024-09-21T00:26:21.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:21.942+0000 7f22e1fc6640 1 -- 172.21.15.112:0/3354781783 wait complete. 2024-09-21T00:26:21.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:21.943+0000 7f22e1fc6640 1 Processor -- start 2024-09-21T00:26:21.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:21.943+0000 7f22e1fc6640 1 -- start start 2024-09-21T00:26:21.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:21.945+0000 7f22e1fc6640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f22dc06e330 0x7f22dc0c5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:21.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:21.945+0000 7f22e1fc6640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f22dc074b60 0x7f22dc0ca6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:21.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:21.945+0000 7f22e1fc6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22dc156a50 0x7f22dc0cba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:21.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:21.945+0000 7f22e1fc6640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f22dc0ce500 con 0x7f22dc156a50 2024-09-21T00:26:21.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:21.945+0000 7f22e1fc6640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f22dc0ce380 con 0x7f22dc06e330 2024-09-21T00:26:21.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:21.945+0000 7f22e1fc6640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f22dc0ce680 con 0x7f22dc074b60 2024-09-21T00:26:21.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:21.945+0000 7f22dbfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22dc156a50 0x7f22dc0cba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:21.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:21.945+0000 7f22dbfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22dc156a50 0x7f22dc0cba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41480/0 (socket says 172.21.15.112:41480) 2024-09-21T00:26:21.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:21.945+0000 7f22dbfff640 1 -- 172.21.15.112:0/3658588901 learned_addr learned my addr 172.21.15.112:0/3658588901 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:26:21.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:21.945+0000 7f22db7fe640 1 --2- 172.21.15.112:0/3658588901 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f22dc06e330 0x7f22dc0c5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:21.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:21.945+0000 7f22d3fff640 1 --2- 172.21.15.112:0/3658588901 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f22dc074b60 0x7f22dc0ca6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:21.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:21.945+0000 7f22dbfff640 1 -- 172.21.15.112:0/3658588901 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f22dc074b60 msgr2=0x7f22dc0ca6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:21.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:21.945+0000 7f22dbfff640 1 --2- 172.21.15.112:0/3658588901 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f22dc074b60 0x7f22dc0ca6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:21.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:21.945+0000 7f22dbfff640 1 -- 172.21.15.112:0/3658588901 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f22dc06e330 msgr2=0x7f22dc0c5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:21.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:21.945+0000 7f22dbfff640 1 --2- 172.21.15.112:0/3658588901 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f22dc06e330 0x7f22dc0c5fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:21.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:21.945+0000 7f22dbfff640 1 -- 172.21.15.112:0/3658588901 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22dc154b40 con 0x7f22dc156a50 2024-09-21T00:26:21.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:21.945+0000 7f22dbfff640 1 --2- 172.21.15.112:0/3658588901 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22dc156a50 0x7f22dc0cba10 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f22d400c760 tx=0x7f22d400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:21.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:21.945+0000 7f22d8ff9640 1 -- 172.21.15.112:0/3658588901 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22d4017070 con 0x7f22dc156a50 2024-09-21T00:26:21.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:21.945+0000 7f22d8ff9640 1 -- 172.21.15.112:0/3658588901 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f22d400ce10 con 0x7f22dc156a50 2024-09-21T00:26:21.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:21.945+0000 7f22e1fc6640 1 -- 172.21.15.112:0/3658588901 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22dc0cb5b0 con 0x7f22dc156a50 2024-09-21T00:26:21.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:21.945+0000 7f22d8ff9640 1 -- 172.21.15.112:0/3658588901 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22d40052b0 con 0x7f22dc156a50 2024-09-21T00:26:21.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:21.946+0000 7f22e1fc6640 1 -- 172.21.15.112:0/3658588901 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22dc1f8a60 con 0x7f22dc156a50 2024-09-21T00:26:21.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:21.946+0000 7f22e1fc6640 1 -- 172.21.15.112:0/3658588901 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22dc0cb040 con 0x7f22dc156a50 2024-09-21T00:26:21.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:21.947+0000 7f22d8ff9640 1 -- 172.21.15.112:0/3658588901 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f22d401a070 con 0x7f22dc156a50 2024-09-21T00:26:21.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:21.948+0000 7f22d8ff9640 1 --2- 172.21.15.112:0/3658588901 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f22bc078bc0 0x7f22bc07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:21.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:21.948+0000 7f22d8ff9640 1 -- 172.21.15.112:0/3658588901 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f22d409a6b0 con 0x7f22dc156a50 2024-09-21T00:26:21.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:21.948+0000 7f22db7fe640 1 --2- 172.21.15.112:0/3658588901 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f22bc078bc0 0x7f22bc07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:21.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:21.948+0000 7f22db7fe640 1 --2- 172.21.15.112:0/3658588901 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f22bc078bc0 0x7f22bc07b0d0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f22cc034000 tx=0x7f22cc004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:21.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:21.952+0000 7f22d8ff9640 1 -- 172.21.15.112:0/3658588901 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f22dc154b40 con 0x7f22dc156a50 2024-09-21T00:26:22.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:22.139+0000 7f22e1fc6640 1 -- 172.21.15.112:0/3658588901 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f22dc06ddf0 con 0x7f22dc156a50 2024-09-21T00:26:22.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:22.140+0000 7f22d8ff9640 1 -- 172.21.15.112:0/3658588901 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7f22dc06e740 con 0x7f22dc156a50 2024-09-21T00:26:22.142 INFO:teuthology.orchestra.run.smithi112.stderr:listed 0 entries 2024-09-21T00:26:22.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:22.142+0000 7f22e1fc6640 1 -- 172.21.15.112:0/3658588901 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f22bc078bc0 msgr2=0x7f22bc07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:22.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:22.142+0000 7f22e1fc6640 1 --2- 172.21.15.112:0/3658588901 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f22bc078bc0 0x7f22bc07b0d0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f22cc034000 tx=0x7f22cc004730 comp rx=0 tx=0).stop 2024-09-21T00:26:22.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:22.142+0000 7f22e1fc6640 1 -- 172.21.15.112:0/3658588901 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22dc156a50 msgr2=0x7f22dc0cba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:22.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:22.142+0000 7f22e1fc6640 1 --2- 172.21.15.112:0/3658588901 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22dc156a50 0x7f22dc0cba10 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f22d400c760 tx=0x7f22d400cc20 comp rx=0 tx=0).stop 2024-09-21T00:26:22.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:22.142+0000 7f22e1fc6640 1 -- 172.21.15.112:0/3658588901 shutdown_connections 2024-09-21T00:26:22.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:22.142+0000 7f22e1fc6640 1 --2- 172.21.15.112:0/3658588901 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f22bc078bc0 0x7f22bc07b0d0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:22.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:22.142+0000 7f22e1fc6640 1 --2- 172.21.15.112:0/3658588901 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22dc156a50 0x7f22dc0cba10 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:22.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:22.142+0000 7f22e1fc6640 1 --2- 172.21.15.112:0/3658588901 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f22dc074b60 0x7f22dc0ca6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:22.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:22.142+0000 7f22e1fc6640 1 --2- 172.21.15.112:0/3658588901 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f22dc06e330 0x7f22dc0c5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:22.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:22.142+0000 7f22e1fc6640 1 -- 172.21.15.112:0/3658588901 >> 172.21.15.112:0/3658588901 conn(0x7f22dc0b1b90 msgr2=0x7f22dc0cdc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:22.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:22.142+0000 7f22e1fc6640 1 -- 172.21.15.112:0/3658588901 shutdown_connections 2024-09-21T00:26:22.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:22.142+0000 7f22e1fc6640 1 -- 172.21.15.112:0/3658588901 wait complete. 2024-09-21T00:26:22.155 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-09-21T00:26:22.155 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:26:22.155 DEBUG:teuthology.orchestra.run.smithi187:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2024-09-21T00:26:22.188 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi187.front.sepia.ceph.com 2024-09-21T00:26:22.188 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:26:22.188 DEBUG:teuthology.orchestra.run.smithi187:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-21T00:26:22.258 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T00:26:22.258 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:26:22.274 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T00:26:22.274 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:26:22.329 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T00:26:22.330 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:26:22.385 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T00:26:22.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:22.538+0000 7f5a8bc3f640 1 -- 172.21.15.112:0/1538544654 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5a840aeeb0 msgr2=0x7f5a840b2ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:22.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:22.538+0000 7f5a8bc3f640 1 --2- 172.21.15.112:0/1538544654 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5a840aeeb0 0x7f5a840b2ca0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f5a78005fd0 tx=0x7f5a7802a540 comp rx=0 tx=0).stop 2024-09-21T00:26:22.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:22.538+0000 7f5a8bc3f640 1 -- 172.21.15.112:0/1538544654 shutdown_connections 2024-09-21T00:26:22.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:22.538+0000 7f5a8bc3f640 1 --2- 172.21.15.112:0/1538544654 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5a840aeeb0 0x7f5a840b2ca0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:22.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:22.538+0000 7f5a8bc3f640 1 --2- 172.21.15.112:0/1538544654 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5a8415c750 0x7f5a840ae970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:22.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:22.538+0000 7f5a8bc3f640 1 -- 172.21.15.112:0/1538544654 >> 172.21.15.112:0/1538544654 conn(0x7f5a840cdca0 msgr2=0x7f5a840ce0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:22.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:22.538+0000 7f5a8bc3f640 1 -- 172.21.15.112:0/1538544654 shutdown_connections 2024-09-21T00:26:22.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:22.538+0000 7f5a8bc3f640 1 -- 172.21.15.112:0/1538544654 wait complete. 2024-09-21T00:26:22.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:22.539+0000 7f5a8bc3f640 1 Processor -- start 2024-09-21T00:26:22.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:22.540+0000 7f5a8bc3f640 1 -- start start 2024-09-21T00:26:22.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:22.541+0000 7f5a8bc3f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5a840aeeb0 0x7f5a8414b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:22.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:22.541+0000 7f5a8bc3f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a840b31e0 0x7f5a840ccb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:22.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:22.541+0000 7f5a8bc3f640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5a8415c750 0x7f5a8415ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:22.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:22.541+0000 7f5a8bc3f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5a840745b0 con 0x7f5a840b31e0 2024-09-21T00:26:22.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:22.541+0000 7f5a8bc3f640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5a840b5ef0 con 0x7f5a840aeeb0 2024-09-21T00:26:22.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:22.541+0000 7f5a8bc3f640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f5a84074730 con 0x7f5a8415c750 2024-09-21T00:26:22.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:22.541+0000 7f5a891da640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a840b31e0 0x7f5a840ccb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:22.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:22.541+0000 7f5a891da640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a840b31e0 0x7f5a840ccb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41494/0 (socket says 172.21.15.112:41494) 2024-09-21T00:26:22.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:22.541+0000 7f5a891da640 1 -- 172.21.15.112:0/3715790864 learned_addr learned my addr 172.21.15.112:0/3715790864 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:26:22.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:22.541+0000 7f5a8a1dc640 1 --2- 172.21.15.112:0/3715790864 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5a8415c750 0x7f5a8415ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:22.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:22.541+0000 7f5a899db640 1 --2- 172.21.15.112:0/3715790864 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5a840aeeb0 0x7f5a8414b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:22.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:22.541+0000 7f5a891da640 1 -- 172.21.15.112:0/3715790864 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5a8415c750 msgr2=0x7f5a8415ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:22.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:22.541+0000 7f5a891da640 1 --2- 172.21.15.112:0/3715790864 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5a8415c750 0x7f5a8415ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:22.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:22.541+0000 7f5a891da640 1 -- 172.21.15.112:0/3715790864 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5a840aeeb0 msgr2=0x7f5a8414b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:22.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:22.541+0000 7f5a891da640 1 --2- 172.21.15.112:0/3715790864 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5a840aeeb0 0x7f5a8414b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:22.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:22.541+0000 7f5a891da640 1 -- 172.21.15.112:0/3715790864 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a8414baa0 con 0x7f5a840b31e0 2024-09-21T00:26:22.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:22.541+0000 7f5a899db640 1 --2- 172.21.15.112:0/3715790864 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5a840aeeb0 0x7f5a8414b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:26:22.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:22.542+0000 7f5a891da640 1 --2- 172.21.15.112:0/3715790864 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a840b31e0 0x7f5a840ccb20 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f5a78034040 tx=0x7f5a780038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:22.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:22.542+0000 7f5a827fc640 1 -- 172.21.15.112:0/3715790864 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a7803e030 con 0x7f5a840b31e0 2024-09-21T00:26:22.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:22.542+0000 7f5a827fc640 1 -- 172.21.15.112:0/3715790864 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5a78003d90 con 0x7f5a840b31e0 2024-09-21T00:26:22.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:22.542+0000 7f5a827fc640 1 -- 172.21.15.112:0/3715790864 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a78035040 con 0x7f5a840b31e0 2024-09-21T00:26:22.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:22.542+0000 7f5a8bc3f640 1 -- 172.21.15.112:0/3715790864 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a841f7410 con 0x7f5a840b31e0 2024-09-21T00:26:22.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:22.542+0000 7f5a8bc3f640 1 -- 172.21.15.112:0/3715790864 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a841f79d0 con 0x7f5a840b31e0 2024-09-21T00:26:22.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:22.543+0000 7f5a8bc3f640 1 -- 172.21.15.112:0/3715790864 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a840cd5a0 con 0x7f5a840b31e0 2024-09-21T00:26:22.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:22.544+0000 7f5a827fc640 1 -- 172.21.15.112:0/3715790864 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5a78041030 con 0x7f5a840b31e0 2024-09-21T00:26:22.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:22.544+0000 7f5a827fc640 1 --2- 172.21.15.112:0/3715790864 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f5a70078b90 0x7f5a7007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:22.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:22.544+0000 7f5a827fc640 1 -- 172.21.15.112:0/3715790864 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f5a780bb860 con 0x7f5a840b31e0 2024-09-21T00:26:22.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:22.544+0000 7f5a899db640 1 --2- 172.21.15.112:0/3715790864 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f5a70078b90 0x7f5a7007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:22.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:22.545+0000 7f5a899db640 1 --2- 172.21.15.112:0/3715790864 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f5a70078b90 0x7f5a7007b0a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5a74005fd0 tx=0x7f5a74009500 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:22.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:22.549+0000 7f5a827fc640 1 -- 172.21.15.112:0/3715790864 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5a78083960 con 0x7f5a840b31e0 2024-09-21T00:26:22.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:22.736+0000 7f5a8bc3f640 1 -- 172.21.15.112:0/3715790864 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f5a8406acf0 con 0x7f5a840b31e0 2024-09-21T00:26:22.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:22.737+0000 7f5a827fc640 1 -- 172.21.15.112:0/3715790864 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7f5a78083260 con 0x7f5a840b31e0 2024-09-21T00:26:22.739 INFO:teuthology.orchestra.run.smithi112.stderr:listed 0 entries 2024-09-21T00:26:22.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:22.739+0000 7f5a8bc3f640 1 -- 172.21.15.112:0/3715790864 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f5a70078b90 msgr2=0x7f5a7007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:22.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:22.739+0000 7f5a8bc3f640 1 --2- 172.21.15.112:0/3715790864 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f5a70078b90 0x7f5a7007b0a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5a74005fd0 tx=0x7f5a74009500 comp rx=0 tx=0).stop 2024-09-21T00:26:22.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:22.739+0000 7f5a8bc3f640 1 -- 172.21.15.112:0/3715790864 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a840b31e0 msgr2=0x7f5a840ccb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:22.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:22.739+0000 7f5a8bc3f640 1 --2- 172.21.15.112:0/3715790864 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a840b31e0 0x7f5a840ccb20 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f5a78034040 tx=0x7f5a780038a0 comp rx=0 tx=0).stop 2024-09-21T00:26:22.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:22.739+0000 7f5a8bc3f640 1 -- 172.21.15.112:0/3715790864 shutdown_connections 2024-09-21T00:26:22.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:22.739+0000 7f5a8bc3f640 1 --2- 172.21.15.112:0/3715790864 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5a8415c750 0x7f5a8415ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:22.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:22.739+0000 7f5a8bc3f640 1 --2- 172.21.15.112:0/3715790864 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f5a70078b90 0x7f5a7007b0a0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:22.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:22.739+0000 7f5a8bc3f640 1 --2- 172.21.15.112:0/3715790864 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a840b31e0 0x7f5a840ccb20 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:22.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:22.739+0000 7f5a8bc3f640 1 --2- 172.21.15.112:0/3715790864 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5a840aeeb0 0x7f5a8414b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:22.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:22.739+0000 7f5a8bc3f640 1 -- 172.21.15.112:0/3715790864 >> 172.21.15.112:0/3715790864 conn(0x7f5a840cdca0 msgr2=0x7f5a840d1120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:22.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:22.739+0000 7f5a8bc3f640 1 -- 172.21.15.112:0/3715790864 shutdown_connections 2024-09-21T00:26:22.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:22.739+0000 7f5a8bc3f640 1 -- 172.21.15.112:0/3715790864 wait complete. 2024-09-21T00:26:22.752 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2024-09-21T00:26:22.752 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:26:22.752 DEBUG:teuthology.orchestra.run.smithi112:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2024-09-21T00:26:22.778 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi112.front.sepia.ceph.com 2024-09-21T00:26:22.778 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:26:22.778 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-09-21T00:26:22.833 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-09-21T00:26:22.833 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:26:22.853 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-09-21T00:26:22.853 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:26:22.912 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-09-21T00:26:22.912 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:26:22.970 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T00:26:23.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.081+0000 7f5fab21d640 1 -- 172.21.15.112:0/3437818333 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f8c003a50 msgr2=0x7f5f8c003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:23.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.081+0000 7f5fab21d640 1 --2- 172.21.15.112:0/3437818333 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f8c003a50 0x7f5f8c003ec0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f5f940016c0 tx=0x7f5f8c005830 comp rx=0 tx=0).stop 2024-09-21T00:26:23.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.082+0000 7f5fab21d640 1 -- 172.21.15.112:0/3437818333 shutdown_connections 2024-09-21T00:26:23.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.082+0000 7f5fab21d640 1 --2- 172.21.15.112:0/3437818333 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f8c003a50 0x7f5f8c003ec0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:23.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.082+0000 7f5fab21d640 1 -- 172.21.15.112:0/3437818333 >> 172.21.15.112:0/3437818333 conn(0x7f5fa40b1b90 msgr2=0x7f5fa40b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:23.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.082+0000 7f5fab21d640 1 -- 172.21.15.112:0/3437818333 shutdown_connections 2024-09-21T00:26:23.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.082+0000 7f5fab21d640 1 -- 172.21.15.112:0/3437818333 wait complete. 2024-09-21T00:26:23.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.083+0000 7f5fab21d640 1 Processor -- start 2024-09-21T00:26:23.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.083+0000 7f5fab21d640 1 -- start start 2024-09-21T00:26:23.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.085+0000 7f5fab21d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5f8c003a50 0x7f5fa415f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:23.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.085+0000 7f5fab21d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5fa41f6a20 0x7f5fa41f6e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:23.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.085+0000 7f5fab21d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5fa40c9170 0x7f5fa40c9580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:23.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.085+0000 7f5fab21d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5fa406e380 con 0x7f5fa41f6a20 2024-09-21T00:26:23.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.085+0000 7f5fab21d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5fa406e200 con 0x7f5f8c003a50 2024-09-21T00:26:23.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.085+0000 7f5fab21d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f5fa406e500 con 0x7f5fa40c9170 2024-09-21T00:26:23.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.085+0000 7f5fa3fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5fa41f6a20 0x7f5fa41f6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:23.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.085+0000 7f5fa3fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5fa41f6a20 0x7f5fa41f6e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41508/0 (socket says 172.21.15.112:41508) 2024-09-21T00:26:23.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.085+0000 7f5fa3fff640 1 -- 172.21.15.112:0/3682403042 learned_addr learned my addr 172.21.15.112:0/3682403042 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:26:23.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.085+0000 7f5fa8fb9640 1 --2- 172.21.15.112:0/3682403042 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5f8c003a50 0x7f5fa415f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:23.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.085+0000 7f5fa97ba640 1 --2- 172.21.15.112:0/3682403042 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5fa40c9170 0x7f5fa40c9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:23.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.085+0000 7f5fa3fff640 1 -- 172.21.15.112:0/3682403042 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5fa40c9170 msgr2=0x7f5fa40c9580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:23.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.085+0000 7f5fa3fff640 1 --2- 172.21.15.112:0/3682403042 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5fa40c9170 0x7f5fa40c9580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:23.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.085+0000 7f5fa3fff640 1 -- 172.21.15.112:0/3682403042 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5f8c003a50 msgr2=0x7f5fa415f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:23.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.085+0000 7f5fa3fff640 1 --2- 172.21.15.112:0/3682403042 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5f8c003a50 0x7f5fa415f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:23.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.085+0000 7f5fa3fff640 1 -- 172.21.15.112:0/3682403042 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fa41f7370 con 0x7f5fa41f6a20 2024-09-21T00:26:23.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.085+0000 7f5fa3fff640 1 --2- 172.21.15.112:0/3682403042 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5fa41f6a20 0x7f5fa41f6e30 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f5f98010b00 tx=0x7f5f98010f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:23.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.085+0000 7f5fa17fa640 1 -- 172.21.15.112:0/3682403042 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f98003970 con 0x7f5fa41f6a20 2024-09-21T00:26:23.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.086+0000 7f5fa17fa640 1 -- 172.21.15.112:0/3682403042 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5f98003c00 con 0x7f5fa41f6a20 2024-09-21T00:26:23.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.086+0000 7f5fa17fa640 1 -- 172.21.15.112:0/3682403042 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f98004050 con 0x7f5fa41f6a20 2024-09-21T00:26:23.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.086+0000 7f5fab21d640 1 -- 172.21.15.112:0/3682403042 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fa40b4b80 con 0x7f5fa41f6a20 2024-09-21T00:26:23.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.086+0000 7f5fab21d640 1 -- 172.21.15.112:0/3682403042 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fa40b5140 con 0x7f5fa41f6a20 2024-09-21T00:26:23.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.087+0000 7f5fab21d640 1 -- 172.21.15.112:0/3682403042 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fa40749a0 con 0x7f5fa41f6a20 2024-09-21T00:26:23.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.087+0000 7f5fa17fa640 1 -- 172.21.15.112:0/3682403042 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5f98004210 con 0x7f5fa41f6a20 2024-09-21T00:26:23.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.088+0000 7f5fa17fa640 1 --2- 172.21.15.112:0/3682403042 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f5f90078b90 0x7f5f9007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:23.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.088+0000 7f5fa17fa640 1 -- 172.21.15.112:0/3682403042 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f5f9809bf00 con 0x7f5fa41f6a20 2024-09-21T00:26:23.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.088+0000 7f5fa8fb9640 1 --2- 172.21.15.112:0/3682403042 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f5f90078b90 0x7f5f9007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:23.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.088+0000 7f5fa8fb9640 1 --2- 172.21.15.112:0/3682403042 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f5f90078b90 0x7f5f9007b0a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5f94032000 tx=0x7f5f9402c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:23.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.092+0000 7f5fa17fa640 1 -- 172.21.15.112:0/3682403042 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5fa41f7370 con 0x7f5fa41f6a20 2024-09-21T00:26:23.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.280+0000 7f5fab21d640 1 -- 172.21.15.112:0/3682403042 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f5fa415c8f0 con 0x7f5fa41f6a20 2024-09-21T00:26:23.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.280+0000 7f5fa17fa640 1 -- 172.21.15.112:0/3682403042 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7f5f98064000 con 0x7f5fa41f6a20 2024-09-21T00:26:23.282 INFO:teuthology.orchestra.run.smithi112.stderr:listed 0 entries 2024-09-21T00:26:23.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.282+0000 7f5fab21d640 1 -- 172.21.15.112:0/3682403042 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f5f90078b90 msgr2=0x7f5f9007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:23.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.282+0000 7f5fab21d640 1 --2- 172.21.15.112:0/3682403042 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f5f90078b90 0x7f5f9007b0a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5f94032000 tx=0x7f5f9402c040 comp rx=0 tx=0).stop 2024-09-21T00:26:23.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.282+0000 7f5fab21d640 1 -- 172.21.15.112:0/3682403042 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5fa41f6a20 msgr2=0x7f5fa41f6e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:23.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.282+0000 7f5fab21d640 1 --2- 172.21.15.112:0/3682403042 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5fa41f6a20 0x7f5fa41f6e30 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f5f98010b00 tx=0x7f5f98010f00 comp rx=0 tx=0).stop 2024-09-21T00:26:23.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.282+0000 7f5fab21d640 1 -- 172.21.15.112:0/3682403042 shutdown_connections 2024-09-21T00:26:23.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.282+0000 7f5fab21d640 1 --2- 172.21.15.112:0/3682403042 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5fa40c9170 0x7f5fa40c9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:23.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.282+0000 7f5fab21d640 1 --2- 172.21.15.112:0/3682403042 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f5f90078b90 0x7f5f9007b0a0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:23.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.282+0000 7f5fab21d640 1 --2- 172.21.15.112:0/3682403042 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5fa41f6a20 0x7f5fa41f6e30 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:23.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.282+0000 7f5fab21d640 1 --2- 172.21.15.112:0/3682403042 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5f8c003a50 0x7f5fa415f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:23.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.282+0000 7f5fab21d640 1 -- 172.21.15.112:0/3682403042 >> 172.21.15.112:0/3682403042 conn(0x7f5fa40b1b90 msgr2=0x7f5fa406fdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:23.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.282+0000 7f5fab21d640 1 -- 172.21.15.112:0/3682403042 shutdown_connections 2024-09-21T00:26:23.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.282+0000 7f5fab21d640 1 -- 172.21.15.112:0/3682403042 wait complete. 2024-09-21T00:26:23.295 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2024-09-21T00:26:23.298 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:26:23.298 DEBUG:teuthology.orchestra.run.smithi187:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2024-09-21T00:26:23.329 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi187.front.sepia.ceph.com 2024-09-21T00:26:23.329 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:26:23.329 DEBUG:teuthology.orchestra.run.smithi187:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-09-21T00:26:23.388 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-21T00:26:23.388 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T00:26:23.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.499+0000 7f4e77d2b640 1 -- 172.21.15.112:0/1722614254 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e700b2e60 msgr2=0x7f4e700b3270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:23.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.499+0000 7f4e77d2b640 1 --2- 172.21.15.112:0/1722614254 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e700b2e60 0x7f4e700b3270 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f4e7008d3b0 tx=0x7f4e6802a8b0 comp rx=0 tx=0).stop 2024-09-21T00:26:23.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.499+0000 7f4e77d2b640 1 -- 172.21.15.112:0/1722614254 shutdown_connections 2024-09-21T00:26:23.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.499+0000 7f4e77d2b640 1 --2- 172.21.15.112:0/1722614254 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e700b2e60 0x7f4e700b3270 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:23.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.499+0000 7f4e77d2b640 1 --2- 172.21.15.112:0/1722614254 >> v2:172.21.15.187:3300/0 conn(0x7f4e700cfe00 0x7f4e700d2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:23.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.499+0000 7f4e77d2b640 1 -- 172.21.15.112:0/1722614254 >> 172.21.15.112:0/1722614254 conn(0x7f4e700d4440 msgr2=0x7f4e700d4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:23.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.499+0000 7f4e77d2b640 1 -- 172.21.15.112:0/1722614254 shutdown_connections 2024-09-21T00:26:23.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.499+0000 7f4e77d2b640 1 -- 172.21.15.112:0/1722614254 wait complete. 2024-09-21T00:26:23.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.501+0000 7f4e77d2b640 1 Processor -- start 2024-09-21T00:26:23.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.501+0000 7f4e77d2b640 1 -- start start 2024-09-21T00:26:23.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.502+0000 7f4e77d2b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e700b2e60 0x7f4e700c7430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:23.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.502+0000 7f4e77d2b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4e700cfe00 0x7f4e70153530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:23.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.502+0000 7f4e77d2b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4e70162590 0x7f4e70153a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:23.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.502+0000 7f4e77d2b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4e700b3930 con 0x7f4e700b2e60 2024-09-21T00:26:23.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.502+0000 7f4e77d2b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f4e700b37b0 con 0x7f4e700cfe00 2024-09-21T00:26:23.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.502+0000 7f4e77d2b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f4e700b3ab0 con 0x7f4e70162590 2024-09-21T00:26:23.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.502+0000 7f4e75ac7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e700b2e60 0x7f4e700c7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:23.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.502+0000 7f4e75ac7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e700b2e60 0x7f4e700c7430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41522/0 (socket says 172.21.15.112:41522) 2024-09-21T00:26:23.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.503+0000 7f4e75ac7640 1 -- 172.21.15.112:0/166101361 learned_addr learned my addr 172.21.15.112:0/166101361 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:26:23.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.503+0000 7f4e762c8640 1 --2- 172.21.15.112:0/166101361 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4e70162590 0x7f4e70153a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:23.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.503+0000 7f4e752c6640 1 --2- 172.21.15.112:0/166101361 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4e700cfe00 0x7f4e70153530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:23.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.503+0000 7f4e75ac7640 1 -- 172.21.15.112:0/166101361 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4e70162590 msgr2=0x7f4e70153a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:23.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.503+0000 7f4e75ac7640 1 --2- 172.21.15.112:0/166101361 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4e70162590 0x7f4e70153a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:23.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.503+0000 7f4e75ac7640 1 -- 172.21.15.112:0/166101361 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4e700cfe00 msgr2=0x7f4e70153530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:23.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.503+0000 7f4e75ac7640 1 --2- 172.21.15.112:0/166101361 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4e700cfe00 0x7f4e70153530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:23.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.503+0000 7f4e75ac7640 1 -- 172.21.15.112:0/166101361 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e70152920 con 0x7f4e700b2e60 2024-09-21T00:26:23.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.503+0000 7f4e762c8640 1 --2- 172.21.15.112:0/166101361 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4e70162590 0x7f4e70153a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:26:23.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.503+0000 7f4e752c6640 1 --2- 172.21.15.112:0/166101361 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4e700cfe00 0x7f4e70153530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:26:23.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.503+0000 7f4e75ac7640 1 --2- 172.21.15.112:0/166101361 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e700b2e60 0x7f4e700c7430 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c00b410 tx=0x7f4e6c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:23.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.503+0000 7f4e667fc640 1 -- 172.21.15.112:0/166101361 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e6c015020 con 0x7f4e700b2e60 2024-09-21T00:26:23.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.503+0000 7f4e667fc640 1 -- 172.21.15.112:0/166101361 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4e6c011940 con 0x7f4e700b2e60 2024-09-21T00:26:23.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.503+0000 7f4e77d2b640 1 -- 172.21.15.112:0/166101361 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e700b7e40 con 0x7f4e700b2e60 2024-09-21T00:26:23.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.503+0000 7f4e667fc640 1 -- 172.21.15.112:0/166101361 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e6c011d90 con 0x7f4e700b2e60 2024-09-21T00:26:23.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.503+0000 7f4e77d2b640 1 -- 172.21.15.112:0/166101361 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e700b99a0 con 0x7f4e700b2e60 2024-09-21T00:26:23.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.504+0000 7f4e77d2b640 1 -- 172.21.15.112:0/166101361 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e700b9c00 con 0x7f4e700b2e60 2024-09-21T00:26:23.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.505+0000 7f4e667fc640 1 -- 172.21.15.112:0/166101361 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f4e6c018020 con 0x7f4e700b2e60 2024-09-21T00:26:23.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.505+0000 7f4e667fc640 1 --2- 172.21.15.112:0/166101361 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4e54078bc0 0x7f4e5407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:23.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.505+0000 7f4e667fc640 1 -- 172.21.15.112:0/166101361 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f4e6c09aff0 con 0x7f4e700b2e60 2024-09-21T00:26:23.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.506+0000 7f4e752c6640 1 --2- 172.21.15.112:0/166101361 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4e54078bc0 0x7f4e5407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:23.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.506+0000 7f4e752c6640 1 --2- 172.21.15.112:0/166101361 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4e54078bc0 0x7f4e5407b0d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f4e701534e0 tx=0x7f4e60009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:23.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.510+0000 7f4e667fc640 1 -- 172.21.15.112:0/166101361 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f4e6c01f020 con 0x7f4e700b2e60 2024-09-21T00:26:23.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.721+0000 7f4e77d2b640 1 -- 172.21.15.112:0/166101361 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4e700b7c30 con 0x7f4e700b2e60 2024-09-21T00:26:23.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.721+0000 7f4e667fc640 1 -- 172.21.15.112:0/166101361 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5228 (secure 0 0 0) 0x7f4e6c0630f0 con 0x7f4e700b2e60 2024-09-21T00:26:23.723 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:26:23.723 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":9,"btime":"2024-09-21T00:26:01:294823+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4316,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6837/2271985854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6835","nonce":2271985854},{"type":"v1","addr":"172.21.15.187:6837","nonce":2271985854}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4413,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/200348130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":200348130},{"type":"v1","addr":"172.21.15.112:6833","nonce":200348130}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4416,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/3821201561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3821201561},{"type":"v1","addr":"172.21.15.112:6835","nonce":3821201561}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T00:25:47.578960+0000","modified":"2024-09-21T00:26:00.650942+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4425},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4425":{"gid":4425,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6836/2921304789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":2921304789},{"type":"v1","addr":"172.21.15.187:6836","nonce":2921304789}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4425,"qdb_cluster":[4425]},"id":1}]} 2024-09-21T00:26:23.724 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 9 2024-09-21T00:26:23.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.723+0000 7f4e77d2b640 1 -- 172.21.15.112:0/166101361 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4e54078bc0 msgr2=0x7f4e5407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:23.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.724+0000 7f4e77d2b640 1 --2- 172.21.15.112:0/166101361 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4e54078bc0 0x7f4e5407b0d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f4e701534e0 tx=0x7f4e60009040 comp rx=0 tx=0).stop 2024-09-21T00:26:23.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.724+0000 7f4e77d2b640 1 -- 172.21.15.112:0/166101361 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e700b2e60 msgr2=0x7f4e700c7430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:23.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.724+0000 7f4e77d2b640 1 --2- 172.21.15.112:0/166101361 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e700b2e60 0x7f4e700c7430 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c00b410 tx=0x7f4e6c00b8d0 comp rx=0 tx=0).stop 2024-09-21T00:26:23.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.724+0000 7f4e77d2b640 1 -- 172.21.15.112:0/166101361 shutdown_connections 2024-09-21T00:26:23.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.724+0000 7f4e77d2b640 1 --2- 172.21.15.112:0/166101361 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4e70162590 0x7f4e70153a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:23.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.724+0000 7f4e77d2b640 1 --2- 172.21.15.112:0/166101361 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4e700cfe00 0x7f4e70153530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:23.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.724+0000 7f4e77d2b640 1 --2- 172.21.15.112:0/166101361 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4e54078bc0 0x7f4e5407b0d0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:23.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.724+0000 7f4e77d2b640 1 --2- 172.21.15.112:0/166101361 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e700b2e60 0x7f4e700c7430 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:23.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.724+0000 7f4e77d2b640 1 -- 172.21.15.112:0/166101361 >> 172.21.15.112:0/166101361 conn(0x7f4e700d4440 msgr2=0x7f4e7006cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:23.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.724+0000 7f4e77d2b640 1 -- 172.21.15.112:0/166101361 shutdown_connections 2024-09-21T00:26:23.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.724+0000 7f4e77d2b640 1 -- 172.21.15.112:0/166101361 wait complete. 2024-09-21T00:26:23.737 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T00:26:23.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.847+0000 7f06cc5c6640 1 -- 172.21.15.112:0/2362306560 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06c4162280 msgr2=0x7f06c4162690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:23.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.847+0000 7f06cc5c6640 1 --2- 172.21.15.112:0/2362306560 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06c4162280 0x7f06c4162690 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f06b4005fd0 tx=0x7f06b402a5d0 comp rx=0 tx=0).stop 2024-09-21T00:26:23.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.847+0000 7f06cc5c6640 1 -- 172.21.15.112:0/2362306560 shutdown_connections 2024-09-21T00:26:23.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.847+0000 7f06cc5c6640 1 --2- 172.21.15.112:0/2362306560 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06c4162280 0x7f06c4162690 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:23.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.847+0000 7f06cc5c6640 1 --2- 172.21.15.112:0/2362306560 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f06c4162f00 0x7f06c40d24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:23.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.847+0000 7f06cc5c6640 1 -- 172.21.15.112:0/2362306560 >> 172.21.15.112:0/2362306560 conn(0x7f06c40b6b20 msgr2=0x7f06c40d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:23.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.847+0000 7f06cc5c6640 1 -- 172.21.15.112:0/2362306560 shutdown_connections 2024-09-21T00:26:23.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.848+0000 7f06cc5c6640 1 -- 172.21.15.112:0/2362306560 wait complete. 2024-09-21T00:26:23.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.849+0000 7f06cc5c6640 1 Processor -- start 2024-09-21T00:26:23.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.849+0000 7f06cc5c6640 1 -- start start 2024-09-21T00:26:23.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.850+0000 7f06cc5c6640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f06c40b2e40 0x7f06c40abe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:23.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.850+0000 7f06cc5c6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06c4162280 0x7f06c40ced40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:23.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.850+0000 7f06cc5c6640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f06c41f84e0 0x7f06c41f88f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:23.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.850+0000 7f06cc5c6640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f06c415e630 con 0x7f06c4162280 2024-09-21T00:26:23.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.850+0000 7f06cc5c6640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f06c415e4b0 con 0x7f06c40b2e40 2024-09-21T00:26:23.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.850+0000 7f06cc5c6640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f06c415e7b0 con 0x7f06c41f84e0 2024-09-21T00:26:23.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.851+0000 7f06c9b61640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06c4162280 0x7f06c40ced40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:23.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.851+0000 7f06c9b61640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06c4162280 0x7f06c40ced40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41544/0 (socket says 172.21.15.112:41544) 2024-09-21T00:26:23.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.851+0000 7f06c9b61640 1 -- 172.21.15.112:0/2272382930 learned_addr learned my addr 172.21.15.112:0/2272382930 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:26:23.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.851+0000 7f06cab63640 1 --2- 172.21.15.112:0/2272382930 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f06c41f84e0 0x7f06c41f88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:23.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.851+0000 7f06ca362640 1 --2- 172.21.15.112:0/2272382930 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f06c40b2e40 0x7f06c40abe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:23.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.851+0000 7f06c9b61640 1 -- 172.21.15.112:0/2272382930 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f06c41f84e0 msgr2=0x7f06c41f88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:23.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.851+0000 7f06c9b61640 1 --2- 172.21.15.112:0/2272382930 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f06c41f84e0 0x7f06c41f88f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:23.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.851+0000 7f06c9b61640 1 -- 172.21.15.112:0/2272382930 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f06c40b2e40 msgr2=0x7f06c40abe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:23.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.851+0000 7f06c9b61640 1 --2- 172.21.15.112:0/2272382930 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f06c40b2e40 0x7f06c40abe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:23.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.851+0000 7f06c9b61640 1 -- 172.21.15.112:0/2272382930 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06c40ac350 con 0x7f06c4162280 2024-09-21T00:26:23.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.851+0000 7f06ca362640 1 --2- 172.21.15.112:0/2272382930 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f06c40b2e40 0x7f06c40abe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:26:23.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.851+0000 7f06cab63640 1 --2- 172.21.15.112:0/2272382930 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f06c41f84e0 0x7f06c41f88f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:26:23.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.851+0000 7f06c9b61640 1 --2- 172.21.15.112:0/2272382930 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06c4162280 0x7f06c40ced40 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f06b4034040 tx=0x7f06b40038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:23.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.851+0000 7f06baffd640 1 -- 172.21.15.112:0/2272382930 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b403e030 con 0x7f06c4162280 2024-09-21T00:26:23.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.851+0000 7f06baffd640 1 -- 172.21.15.112:0/2272382930 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f06b4003ca0 con 0x7f06c4162280 2024-09-21T00:26:23.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.851+0000 7f06cc5c6640 1 -- 172.21.15.112:0/2272382930 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06c41f8e30 con 0x7f06c4162280 2024-09-21T00:26:23.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.851+0000 7f06baffd640 1 -- 172.21.15.112:0/2272382930 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b4035040 con 0x7f06c4162280 2024-09-21T00:26:23.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.851+0000 7f06cc5c6640 1 -- 172.21.15.112:0/2272382930 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06c40acb40 con 0x7f06c4162280 2024-09-21T00:26:23.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.852+0000 7f06cc5c6640 1 -- 172.21.15.112:0/2272382930 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06c4162f00 con 0x7f06c4162280 2024-09-21T00:26:23.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.853+0000 7f06baffd640 1 -- 172.21.15.112:0/2272382930 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f06b4041030 con 0x7f06c4162280 2024-09-21T00:26:23.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.853+0000 7f06baffd640 1 --2- 172.21.15.112:0/2272382930 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f06a8078b90 0x7f06a807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:23.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.854+0000 7f06baffd640 1 -- 172.21.15.112:0/2272382930 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f06b40bb6b0 con 0x7f06c4162280 2024-09-21T00:26:23.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.854+0000 7f06ca362640 1 --2- 172.21.15.112:0/2272382930 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f06a8078b90 0x7f06a807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:23.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.854+0000 7f06ca362640 1 --2- 172.21.15.112:0/2272382930 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f06a8078b90 0x7f06a807b0a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f06c0009aa0 tx=0x7f06c0005db0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:23.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:23.858+0000 7f06baffd640 1 -- 172.21.15.112:0/2272382930 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f06b4083840 con 0x7f06c4162280 2024-09-21T00:26:24.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.071+0000 7f06cc5c6640 1 -- 172.21.15.112:0/2272382930 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f06c40ac5f0 con 0x7f06c4162280 2024-09-21T00:26:24.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.072+0000 7f06baffd640 1 -- 172.21.15.112:0/2272382930 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5228 (secure 0 0 0) 0x7f06b4083140 con 0x7f06c4162280 2024-09-21T00:26:24.074 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:26:24.074 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":9,"btime":"2024-09-21T00:26:01:294823+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4316,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6837/2271985854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6835","nonce":2271985854},{"type":"v1","addr":"172.21.15.187:6837","nonce":2271985854}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4413,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/200348130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":200348130},{"type":"v1","addr":"172.21.15.112:6833","nonce":200348130}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4416,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/3821201561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3821201561},{"type":"v1","addr":"172.21.15.112:6835","nonce":3821201561}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T00:25:47.578960+0000","modified":"2024-09-21T00:26:00.650942+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4425},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4425":{"gid":4425,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6836/2921304789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":2921304789},{"type":"v1","addr":"172.21.15.187:6836","nonce":2921304789}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4425,"qdb_cluster":[4425]},"id":1}]} 2024-09-21T00:26:24.074 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 9 2024-09-21T00:26:24.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.074+0000 7f06cc5c6640 1 -- 172.21.15.112:0/2272382930 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f06a8078b90 msgr2=0x7f06a807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:24.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.074+0000 7f06cc5c6640 1 --2- 172.21.15.112:0/2272382930 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f06a8078b90 0x7f06a807b0a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f06c0009aa0 tx=0x7f06c0005db0 comp rx=0 tx=0).stop 2024-09-21T00:26:24.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.074+0000 7f06cc5c6640 1 -- 172.21.15.112:0/2272382930 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06c4162280 msgr2=0x7f06c40ced40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:24.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.074+0000 7f06cc5c6640 1 --2- 172.21.15.112:0/2272382930 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06c4162280 0x7f06c40ced40 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f06b4034040 tx=0x7f06b40038a0 comp rx=0 tx=0).stop 2024-09-21T00:26:24.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.074+0000 7f06cc5c6640 1 -- 172.21.15.112:0/2272382930 shutdown_connections 2024-09-21T00:26:24.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.074+0000 7f06cc5c6640 1 --2- 172.21.15.112:0/2272382930 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f06c41f84e0 0x7f06c41f88f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:24.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.074+0000 7f06cc5c6640 1 --2- 172.21.15.112:0/2272382930 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f06a8078b90 0x7f06a807b0a0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:24.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.074+0000 7f06cc5c6640 1 --2- 172.21.15.112:0/2272382930 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06c4162280 0x7f06c40ced40 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:24.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.074+0000 7f06cc5c6640 1 --2- 172.21.15.112:0/2272382930 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f06c40b2e40 0x7f06c40abe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:24.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.074+0000 7f06cc5c6640 1 -- 172.21.15.112:0/2272382930 >> 172.21.15.112:0/2272382930 conn(0x7f06c40b6b20 msgr2=0x7f06c415ec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:24.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.074+0000 7f06cc5c6640 1 -- 172.21.15.112:0/2272382930 shutdown_connections 2024-09-21T00:26:24.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.074+0000 7f06cc5c6640 1 -- 172.21.15.112:0/2272382930 wait complete. 2024-09-21T00:26:24.087 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T00:26:24.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.197+0000 7fe2bdfad640 1 -- 172.21.15.112:0/2700854688 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2b815a6f0 msgr2=0x7fe2b815e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:24.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.197+0000 7fe2bdfad640 1 --2- 172.21.15.112:0/2700854688 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2b815a6f0 0x7fe2b815e230 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac005fd0 tx=0x7fe2ac02a5d0 comp rx=0 tx=0).stop 2024-09-21T00:26:24.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.197+0000 7fe2bdfad640 1 -- 172.21.15.112:0/2700854688 shutdown_connections 2024-09-21T00:26:24.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.197+0000 7fe2bdfad640 1 --2- 172.21.15.112:0/2700854688 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2b815a6f0 0x7fe2b815e230 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:24.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.197+0000 7fe2bdfad640 1 --2- 172.21.15.112:0/2700854688 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2b80d01f0 0x7fe2b815daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:24.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.197+0000 7fe2bdfad640 1 -- 172.21.15.112:0/2700854688 >> 172.21.15.112:0/2700854688 conn(0x7fe2b8072990 msgr2=0x7fe2b8072da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:24.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.197+0000 7fe2bdfad640 1 -- 172.21.15.112:0/2700854688 shutdown_connections 2024-09-21T00:26:24.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.197+0000 7fe2bdfad640 1 -- 172.21.15.112:0/2700854688 wait complete. 2024-09-21T00:26:24.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.198+0000 7fe2bdfad640 1 Processor -- start 2024-09-21T00:26:24.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.199+0000 7fe2bdfad640 1 -- start start 2024-09-21T00:26:24.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.200+0000 7fe2bdfad640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2b80d01f0 0x7fe2b8154940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:24.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.200+0000 7fe2bdfad640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe2b80d0980 0x7fe2b81f9050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:24.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.200+0000 7fe2bdfad640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2b815a6f0 0x7fe2b81f9590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:24.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.200+0000 7fe2bdfad640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b80cd340 con 0x7fe2b80d01f0 2024-09-21T00:26:24.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.200+0000 7fe2bdfad640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b80cd1c0 con 0x7fe2b80d0980 2024-09-21T00:26:24.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.200+0000 7fe2bdfad640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe2b80cd4c0 con 0x7fe2b815a6f0 2024-09-21T00:26:24.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.200+0000 7fe2b77fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2b80d01f0 0x7fe2b8154940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:24.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.200+0000 7fe2b77fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2b80d01f0 0x7fe2b8154940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41550/0 (socket says 172.21.15.112:41550) 2024-09-21T00:26:24.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.200+0000 7fe2b77fe640 1 -- 172.21.15.112:0/684456534 learned_addr learned my addr 172.21.15.112:0/684456534 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:26:24.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.200+0000 7fe2b7fff640 1 --2- 172.21.15.112:0/684456534 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2b815a6f0 0x7fe2b81f9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:24.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.200+0000 7fe2b6ffd640 1 --2- 172.21.15.112:0/684456534 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe2b80d0980 0x7fe2b81f9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:24.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.200+0000 7fe2b77fe640 1 -- 172.21.15.112:0/684456534 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2b815a6f0 msgr2=0x7fe2b81f9590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:24.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.200+0000 7fe2b77fe640 1 --2- 172.21.15.112:0/684456534 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2b815a6f0 0x7fe2b81f9590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:24.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.200+0000 7fe2b77fe640 1 -- 172.21.15.112:0/684456534 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe2b80d0980 msgr2=0x7fe2b81f9050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:24.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.200+0000 7fe2b77fe640 1 --2- 172.21.15.112:0/684456534 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe2b80d0980 0x7fe2b81f9050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:24.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.200+0000 7fe2b77fe640 1 -- 172.21.15.112:0/684456534 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2b806f0a0 con 0x7fe2b80d01f0 2024-09-21T00:26:24.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.200+0000 7fe2b6ffd640 1 --2- 172.21.15.112:0/684456534 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe2b80d0980 0x7fe2b81f9050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:26:24.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.201+0000 7fe2b77fe640 1 --2- 172.21.15.112:0/684456534 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2b80d01f0 0x7fe2b8154940 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fe2a800cd10 tx=0x7fe2a8007610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:24.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.201+0000 7fe29ffff640 1 -- 172.21.15.112:0/684456534 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a8004b20 con 0x7fe2b80d01f0 2024-09-21T00:26:24.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.201+0000 7fe29ffff640 1 -- 172.21.15.112:0/684456534 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe2a8004db0 con 0x7fe2b80d01f0 2024-09-21T00:26:24.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.201+0000 7fe2bdfad640 1 -- 172.21.15.112:0/684456534 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2b806f940 con 0x7fe2b80d01f0 2024-09-21T00:26:24.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.201+0000 7fe29ffff640 1 -- 172.21.15.112:0/684456534 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a8005280 con 0x7fe2b80d01f0 2024-09-21T00:26:24.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.201+0000 7fe2bdfad640 1 -- 172.21.15.112:0/684456534 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2b8155930 con 0x7fe2b80d01f0 2024-09-21T00:26:24.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.202+0000 7fe2bdfad640 1 -- 172.21.15.112:0/684456534 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2b80cd9b0 con 0x7fe2b80d01f0 2024-09-21T00:26:24.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.202+0000 7fe29ffff640 1 -- 172.21.15.112:0/684456534 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe2a8005420 con 0x7fe2b80d01f0 2024-09-21T00:26:24.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.203+0000 7fe29ffff640 1 --2- 172.21.15.112:0/684456534 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fe2a4078bc0 0x7fe2a407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:24.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.203+0000 7fe29ffff640 1 -- 172.21.15.112:0/684456534 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fe2a809b740 con 0x7fe2b80d01f0 2024-09-21T00:26:24.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.203+0000 7fe2b6ffd640 1 --2- 172.21.15.112:0/684456534 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fe2a4078bc0 0x7fe2a407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:24.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.204+0000 7fe2b6ffd640 1 --2- 172.21.15.112:0/684456534 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fe2a4078bc0 0x7fe2a407b0d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac033000 tx=0x7fe2ac004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:24.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.208+0000 7fe29ffff640 1 -- 172.21.15.112:0/684456534 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe2a8014030 con 0x7fe2b80d01f0 2024-09-21T00:26:24.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.389+0000 7fe2bdfad640 1 -- 172.21.15.112:0/684456534 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe2b80713d0 con 0x7fe2b80d01f0 2024-09-21T00:26:24.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.390+0000 7fe29ffff640 1 -- 172.21.15.112:0/684456534 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18622 (secure 0 0 0) 0x7fe2a8063960 con 0x7fe2b80d01f0 2024-09-21T00:26:24.392 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:26:24.392 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":26,"fsid":"8b34692b-dc65-49b2-a08f-26f707b96ac5","created":"2024-09-21T00:25:22.788490+0000","modified":"2024-09-21T00:25:52.623213+0000","last_up_change":"2024-09-21T00:25:36.161835+0000","last_in_change":"2024-09-21T00:25:26.346500+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T00:25:37.105006+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T00:25:45.413895+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T00:25:46.532124+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T00:25:50.244145+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9c575086-67d2-4de2-b01b-d44dcc21d604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6809","nonce":755760621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6811","nonce":755760621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6815","nonce":755760621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6813","nonce":755760621}]},"public_addr":"172.21.15.112:6809/755760621","cluster_addr":"172.21.15.112:6811/755760621","heartbeat_back_addr":"172.21.15.112:6815/755760621","heartbeat_front_addr":"172.21.15.112:6813/755760621","state":["exists","up"]},{"osd":1,"uuid":"a983c227-a3ce-4bdd-82f1-438ba7e3ecd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6801","nonce":344932592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6803","nonce":344932592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6807","nonce":344932592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6805","nonce":344932592}]},"public_addr":"172.21.15.112:6801/344932592","cluster_addr":"172.21.15.112:6803/344932592","heartbeat_back_addr":"172.21.15.112:6807/344932592","heartbeat_front_addr":"172.21.15.112:6805/344932592","state":["exists","up"]},{"osd":2,"uuid":"1193c1d8-bfff-4a57-8200-5f69bf1ea3af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6825","nonce":407933616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6827","nonce":407933616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6831","nonce":407933616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6829","nonce":407933616}]},"public_addr":"172.21.15.112:6825/407933616","cluster_addr":"172.21.15.112:6827/407933616","heartbeat_back_addr":"172.21.15.112:6831/407933616","heartbeat_front_addr":"172.21.15.112:6829/407933616","state":["exists","up"]},{"osd":3,"uuid":"af04330a-36ad-40cc-a8dd-fa259b221de5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6817","nonce":4120164806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6819","nonce":4120164806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6823","nonce":4120164806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6821","nonce":4120164806}]},"public_addr":"172.21.15.112:6817/4120164806","cluster_addr":"172.21.15.112:6819/4120164806","heartbeat_back_addr":"172.21.15.112:6823/4120164806","heartbeat_front_addr":"172.21.15.112:6821/4120164806","state":["exists","up"]},{"osd":4,"uuid":"5b893111-292c-4c5a-b184-53ad38c7199e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6803","nonce":522077560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6805","nonce":522077560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6809","nonce":522077560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6807","nonce":522077560}]},"public_addr":"172.21.15.187:6803/522077560","cluster_addr":"172.21.15.187:6805/522077560","heartbeat_back_addr":"172.21.15.187:6809/522077560","heartbeat_front_addr":"172.21.15.187:6807/522077560","state":["exists","up"]},{"osd":5,"uuid":"c0b8e011-45fe-4c5e-925e-6be2a7f1868a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6827","nonce":3593144721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6829","nonce":3593144721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6833","nonce":3593144721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6831","nonce":3593144721}]},"public_addr":"172.21.15.187:6827/3593144721","cluster_addr":"172.21.15.187:6829/3593144721","heartbeat_back_addr":"172.21.15.187:6833/3593144721","heartbeat_front_addr":"172.21.15.187:6831/3593144721","state":["exists","up"]},{"osd":6,"uuid":"87d2d650-7489-4593-98a3-a1f433ce8dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6811","nonce":1494387657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6813","nonce":1494387657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6817","nonce":1494387657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6815","nonce":1494387657}]},"public_addr":"172.21.15.187:6811/1494387657","cluster_addr":"172.21.15.187:6813/1494387657","heartbeat_back_addr":"172.21.15.187:6817/1494387657","heartbeat_front_addr":"172.21.15.187:6815/1494387657","state":["exists","up"]},{"osd":7,"uuid":"c6b4f8e2-a1ee-4438-aa37-67ff4b6ef1f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6819","nonce":1624699842}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6821","nonce":1624699842}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6825","nonce":1624699842}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6823","nonce":1624699842}]},"public_addr":"172.21.15.187:6819/1624699842","cluster_addr":"172.21.15.187:6821/1624699842","heartbeat_back_addr":"172.21.15.187:6825/1624699842","heartbeat_front_addr":"172.21.15.187:6823/1624699842","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.873162+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.400506+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.664317+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.159791+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.202630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.113212+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.384786+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.448441+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T00:26:24.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.392+0000 7fe2bdfad640 1 -- 172.21.15.112:0/684456534 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fe2a4078bc0 msgr2=0x7fe2a407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:24.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.392+0000 7fe2bdfad640 1 --2- 172.21.15.112:0/684456534 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fe2a4078bc0 0x7fe2a407b0d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac033000 tx=0x7fe2ac004730 comp rx=0 tx=0).stop 2024-09-21T00:26:24.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.392+0000 7fe2bdfad640 1 -- 172.21.15.112:0/684456534 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2b80d01f0 msgr2=0x7fe2b8154940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:24.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.392+0000 7fe2bdfad640 1 --2- 172.21.15.112:0/684456534 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2b80d01f0 0x7fe2b8154940 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fe2a800cd10 tx=0x7fe2a8007610 comp rx=0 tx=0).stop 2024-09-21T00:26:24.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.392+0000 7fe2bdfad640 1 -- 172.21.15.112:0/684456534 shutdown_connections 2024-09-21T00:26:24.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.392+0000 7fe2bdfad640 1 --2- 172.21.15.112:0/684456534 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2b815a6f0 0x7fe2b81f9590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:24.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.392+0000 7fe2bdfad640 1 --2- 172.21.15.112:0/684456534 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe2b80d0980 0x7fe2b81f9050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:24.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.392+0000 7fe2bdfad640 1 --2- 172.21.15.112:0/684456534 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fe2a4078bc0 0x7fe2a407b0d0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:24.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.392+0000 7fe2bdfad640 1 --2- 172.21.15.112:0/684456534 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2b80d01f0 0x7fe2b8154940 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:24.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.392+0000 7fe2bdfad640 1 -- 172.21.15.112:0/684456534 >> 172.21.15.112:0/684456534 conn(0x7fe2b8072990 msgr2=0x7fe2b8070100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:24.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.392+0000 7fe2bdfad640 1 -- 172.21.15.112:0/684456534 shutdown_connections 2024-09-21T00:26:24.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.392+0000 7fe2bdfad640 1 -- 172.21.15.112:0/684456534 wait complete. 2024-09-21T00:26:24.405 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-21T00:26:24.406 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T00:26:24.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.516+0000 7f25f3b57640 1 -- 172.21.15.112:0/3033060117 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25d8004800 msgr2=0x7f25d8004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:24.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.516+0000 7f25f3b57640 1 --2- 172.21.15.112:0/3033060117 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25d8004800 0x7f25d8004c10 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f25dc006fd0 tx=0x7f25d8005830 comp rx=0 tx=0).stop 2024-09-21T00:26:24.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.516+0000 7f25f3b57640 1 -- 172.21.15.112:0/3033060117 shutdown_connections 2024-09-21T00:26:24.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.516+0000 7f25f3b57640 1 --2- 172.21.15.112:0/3033060117 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25d8004800 0x7f25d8004c10 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:24.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.516+0000 7f25f3b57640 1 -- 172.21.15.112:0/3033060117 >> 172.21.15.112:0/3033060117 conn(0x7f25ec0b6b20 msgr2=0x7f25ec0d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:24.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.517+0000 7f25f3b57640 1 -- 172.21.15.112:0/3033060117 shutdown_connections 2024-09-21T00:26:24.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.517+0000 7f25f3b57640 1 -- 172.21.15.112:0/3033060117 wait complete. 2024-09-21T00:26:24.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.518+0000 7f25f3b57640 1 Processor -- start 2024-09-21T00:26:24.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.518+0000 7f25f3b57640 1 -- start start 2024-09-21T00:26:24.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.520+0000 7f25f3b57640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f25d8004800 0x7f25ec1f5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:24.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.520+0000 7f25f3b57640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25ec153540 0x7f25ec0caba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:24.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.520+0000 7f25f3b57640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f25ec0cb0e0 0x7f25ec1f8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:24.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.520+0000 7f25f3b57640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f25ec0d24d0 con 0x7f25ec153540 2024-09-21T00:26:24.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.520+0000 7f25f3b57640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f25ec15fae0 con 0x7f25d8004800 2024-09-21T00:26:24.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.520+0000 7f25f3b57640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f25ec0d2650 con 0x7f25ec0cb0e0 2024-09-21T00:26:24.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.520+0000 7f25f10f2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25ec153540 0x7f25ec0caba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:24.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.520+0000 7f25f10f2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25ec153540 0x7f25ec0caba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41556/0 (socket says 172.21.15.112:41556) 2024-09-21T00:26:24.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.520+0000 7f25f10f2640 1 -- 172.21.15.112:0/1355130122 learned_addr learned my addr 172.21.15.112:0/1355130122 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:26:24.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.520+0000 7f25f20f4640 1 --2- 172.21.15.112:0/1355130122 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f25ec0cb0e0 0x7f25ec1f8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:24.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.520+0000 7f25f18f3640 1 --2- 172.21.15.112:0/1355130122 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f25d8004800 0x7f25ec1f5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:24.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.520+0000 7f25f10f2640 1 -- 172.21.15.112:0/1355130122 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f25ec0cb0e0 msgr2=0x7f25ec1f8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:24.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.520+0000 7f25f10f2640 1 --2- 172.21.15.112:0/1355130122 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f25ec0cb0e0 0x7f25ec1f8b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:24.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.520+0000 7f25f10f2640 1 -- 172.21.15.112:0/1355130122 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f25d8004800 msgr2=0x7f25ec1f5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:24.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.520+0000 7f25f10f2640 1 --2- 172.21.15.112:0/1355130122 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f25d8004800 0x7f25ec1f5530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:24.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.520+0000 7f25f10f2640 1 -- 172.21.15.112:0/1355130122 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25ec0cb4f0 con 0x7f25ec153540 2024-09-21T00:26:24.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.520+0000 7f25f20f4640 1 --2- 172.21.15.112:0/1355130122 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f25ec0cb0e0 0x7f25ec1f8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:26:24.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.520+0000 7f25f10f2640 1 --2- 172.21.15.112:0/1355130122 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25ec153540 0x7f25ec0caba0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f25dc000c00 tx=0x7f25dc001c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:24.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.520+0000 7f25e27fc640 1 -- 172.21.15.112:0/1355130122 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25dc006580 con 0x7f25ec153540 2024-09-21T00:26:24.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.520+0000 7f25e27fc640 1 -- 172.21.15.112:0/1355130122 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f25dc003b10 con 0x7f25ec153540 2024-09-21T00:26:24.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.520+0000 7f25f3b57640 1 -- 172.21.15.112:0/1355130122 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25ec146c60 con 0x7f25ec153540 2024-09-21T00:26:24.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.520+0000 7f25e27fc640 1 -- 172.21.15.112:0/1355130122 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25dc02e480 con 0x7f25ec153540 2024-09-21T00:26:24.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.520+0000 7f25f3b57640 1 -- 172.21.15.112:0/1355130122 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25ec147220 con 0x7f25ec153540 2024-09-21T00:26:24.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.522+0000 7f25f3b57640 1 -- 172.21.15.112:0/1355130122 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25ec0d03d0 con 0x7f25ec153540 2024-09-21T00:26:24.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.522+0000 7f25e27fc640 1 -- 172.21.15.112:0/1355130122 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f25dc035020 con 0x7f25ec153540 2024-09-21T00:26:24.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.523+0000 7f25e27fc640 1 --2- 172.21.15.112:0/1355130122 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f25d0078bc0 0x7f25d007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:24.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.523+0000 7f25e27fc640 1 -- 172.21.15.112:0/1355130122 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f25dc0bbe00 con 0x7f25ec153540 2024-09-21T00:26:24.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.523+0000 7f25f18f3640 1 --2- 172.21.15.112:0/1355130122 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f25d0078bc0 0x7f25d007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:24.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.524+0000 7f25f18f3640 1 --2- 172.21.15.112:0/1355130122 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f25d0078bc0 0x7f25d007b0d0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f25e80065c0 tx=0x7f25e8006b70 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:24.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.527+0000 7f25e27fc640 1 -- 172.21.15.112:0/1355130122 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f25ec0d03d0 con 0x7f25ec153540 2024-09-21T00:26:24.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.741+0000 7f25f3b57640 1 -- 172.21.15.112:0/1355130122 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f25ec162690 con 0x7f25ec153540 2024-09-21T00:26:24.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.741+0000 7f25e27fc640 1 -- 172.21.15.112:0/1355130122 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5228 (secure 0 0 0) 0x7f25dc047060 con 0x7f25ec153540 2024-09-21T00:26:24.743 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:26:24.743 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":9,"btime":"2024-09-21T00:26:01:294823+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4316,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6837/2271985854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6835","nonce":2271985854},{"type":"v1","addr":"172.21.15.187:6837","nonce":2271985854}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4413,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/200348130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":200348130},{"type":"v1","addr":"172.21.15.112:6833","nonce":200348130}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4416,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/3821201561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3821201561},{"type":"v1","addr":"172.21.15.112:6835","nonce":3821201561}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T00:25:47.578960+0000","modified":"2024-09-21T00:26:00.650942+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4425},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4425":{"gid":4425,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6836/2921304789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":2921304789},{"type":"v1","addr":"172.21.15.187:6836","nonce":2921304789}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4425,"qdb_cluster":[4425]},"id":1}]} 2024-09-21T00:26:24.743 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 9 2024-09-21T00:26:24.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.743+0000 7f25f3b57640 1 -- 172.21.15.112:0/1355130122 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f25d0078bc0 msgr2=0x7f25d007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:24.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.743+0000 7f25f3b57640 1 --2- 172.21.15.112:0/1355130122 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f25d0078bc0 0x7f25d007b0d0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f25e80065c0 tx=0x7f25e8006b70 comp rx=0 tx=0).stop 2024-09-21T00:26:24.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.743+0000 7f25f3b57640 1 -- 172.21.15.112:0/1355130122 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25ec153540 msgr2=0x7f25ec0caba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:24.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.743+0000 7f25f3b57640 1 --2- 172.21.15.112:0/1355130122 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25ec153540 0x7f25ec0caba0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f25dc000c00 tx=0x7f25dc001c70 comp rx=0 tx=0).stop 2024-09-21T00:26:24.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.743+0000 7f25f3b57640 1 -- 172.21.15.112:0/1355130122 shutdown_connections 2024-09-21T00:26:24.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.743+0000 7f25f3b57640 1 --2- 172.21.15.112:0/1355130122 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f25ec0cb0e0 0x7f25ec1f8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:24.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.744+0000 7f25f3b57640 1 --2- 172.21.15.112:0/1355130122 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f25d0078bc0 0x7f25d007b0d0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:24.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.744+0000 7f25f3b57640 1 --2- 172.21.15.112:0/1355130122 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25ec153540 0x7f25ec0caba0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:24.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.744+0000 7f25f3b57640 1 --2- 172.21.15.112:0/1355130122 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f25d8004800 0x7f25ec1f5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:24.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.744+0000 7f25f3b57640 1 -- 172.21.15.112:0/1355130122 >> 172.21.15.112:0/1355130122 conn(0x7f25ec0b6b20 msgr2=0x7f25ec0cf360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:24.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.744+0000 7f25f3b57640 1 -- 172.21.15.112:0/1355130122 shutdown_connections 2024-09-21T00:26:24.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.744+0000 7f25f3b57640 1 -- 172.21.15.112:0/1355130122 wait complete. 2024-09-21T00:26:24.756 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T00:26:24.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.867+0000 7f0fa7369640 1 -- 172.21.15.112:0/1018905027 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0fa015cb10 msgr2=0x7f0fa015cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:24.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.867+0000 7f0fa7369640 1 --2- 172.21.15.112:0/1018905027 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0fa015cb10 0x7f0fa015cf20 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f0fa015d7e0 tx=0x7f0f9c02a540 comp rx=0 tx=0).stop 2024-09-21T00:26:24.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.867+0000 7f0fa7369640 1 -- 172.21.15.112:0/1018905027 shutdown_connections 2024-09-21T00:26:24.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.867+0000 7f0fa7369640 1 --2- 172.21.15.112:0/1018905027 >> v2:172.21.15.187:3300/0 conn(0x7f0fa0160100 0x7f0fa0160510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:24.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.867+0000 7f0fa7369640 1 --2- 172.21.15.112:0/1018905027 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0fa00d12e0 0x7f0fa00d16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:24.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.867+0000 7f0fa7369640 1 --2- 172.21.15.112:0/1018905027 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0fa015cb10 0x7f0fa015cf20 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:24.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.867+0000 7f0fa7369640 1 -- 172.21.15.112:0/1018905027 >> 172.21.15.112:0/1018905027 conn(0x7f0fa00d2350 msgr2=0x7f0fa00d2760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:24.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.867+0000 7f0fa7369640 1 -- 172.21.15.112:0/1018905027 shutdown_connections 2024-09-21T00:26:24.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.867+0000 7f0fa7369640 1 -- 172.21.15.112:0/1018905027 wait complete. 2024-09-21T00:26:24.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.869+0000 7f0fa7369640 1 Processor -- start 2024-09-21T00:26:24.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.869+0000 7f0fa7369640 1 -- start start 2024-09-21T00:26:24.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.870+0000 7f0fa7369640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0fa00d12e0 0x7f0fa013f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:24.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.870+0000 7f0fa7369640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0fa015cb10 0x7f0fa013fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:24.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.870+0000 7f0fa7369640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0fa0160100 0x7f0fa0155450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:24.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.870+0000 7f0fa7369640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0fa0160bd0 con 0x7f0fa00d12e0 2024-09-21T00:26:24.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.870+0000 7f0fa7369640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0fa0160a50 con 0x7f0fa0160100 2024-09-21T00:26:24.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.870+0000 7f0fa7369640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f0fa0160d50 con 0x7f0fa015cb10 2024-09-21T00:26:24.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.870+0000 7f0fa5105640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0fa00d12e0 0x7f0fa013f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:24.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.870+0000 7f0fa5105640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0fa00d12e0 0x7f0fa013f620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41570/0 (socket says 172.21.15.112:41570) 2024-09-21T00:26:24.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.870+0000 7f0fa5105640 1 -- 172.21.15.112:0/1409652957 learned_addr learned my addr 172.21.15.112:0/1409652957 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:26:24.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.870+0000 7f0fa4904640 1 --2- 172.21.15.112:0/1409652957 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0fa015cb10 0x7f0fa013fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:24.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.870+0000 7f0fa5906640 1 --2- 172.21.15.112:0/1409652957 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0fa0160100 0x7f0fa0155450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:24.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.870+0000 7f0fa5105640 1 -- 172.21.15.112:0/1409652957 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0fa015cb10 msgr2=0x7f0fa013fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:24.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.870+0000 7f0fa5105640 1 --2- 172.21.15.112:0/1409652957 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0fa015cb10 0x7f0fa013fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:24.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.870+0000 7f0fa5105640 1 -- 172.21.15.112:0/1409652957 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0fa0160100 msgr2=0x7f0fa0155450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:24.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.870+0000 7f0fa5105640 1 --2- 172.21.15.112:0/1409652957 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0fa0160100 0x7f0fa0155450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:24.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.870+0000 7f0fa5105640 1 -- 172.21.15.112:0/1409652957 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fa013fb60 con 0x7f0fa00d12e0 2024-09-21T00:26:24.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.870+0000 7f0fa4904640 1 --2- 172.21.15.112:0/1409652957 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0fa015cb10 0x7f0fa013fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:26:24.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.871+0000 7f0fa5105640 1 --2- 172.21.15.112:0/1409652957 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0fa00d12e0 0x7f0fa013f620 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c034040 tx=0x7f0f9c003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:24.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.871+0000 7f0f95ffb640 1 -- 172.21.15.112:0/1409652957 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f9c03e030 con 0x7f0fa00d12e0 2024-09-21T00:26:24.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.871+0000 7f0f95ffb640 1 -- 172.21.15.112:0/1409652957 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0f9c003e00 con 0x7f0fa00d12e0 2024-09-21T00:26:24.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.871+0000 7f0f95ffb640 1 -- 172.21.15.112:0/1409652957 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f9c035040 con 0x7f0fa00d12e0 2024-09-21T00:26:24.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.871+0000 7f0fa7369640 1 -- 172.21.15.112:0/1409652957 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fa0140a70 con 0x7f0fa00d12e0 2024-09-21T00:26:24.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.871+0000 7f0fa7369640 1 -- 172.21.15.112:0/1409652957 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fa0141030 con 0x7f0fa00d12e0 2024-09-21T00:26:24.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.872+0000 7f0fa7369640 1 -- 172.21.15.112:0/1409652957 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fa015cf20 con 0x7f0fa00d12e0 2024-09-21T00:26:24.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.873+0000 7f0f95ffb640 1 -- 172.21.15.112:0/1409652957 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0f9c041030 con 0x7f0fa00d12e0 2024-09-21T00:26:24.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.873+0000 7f0f95ffb640 1 --2- 172.21.15.112:0/1409652957 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f0f84078b90 0x7f0f8407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:24.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.873+0000 7f0f95ffb640 1 -- 172.21.15.112:0/1409652957 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f0f9c0bb8b0 con 0x7f0fa00d12e0 2024-09-21T00:26:24.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.873+0000 7f0fa4904640 1 --2- 172.21.15.112:0/1409652957 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f0f84078b90 0x7f0f8407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:24.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.874+0000 7f0fa4904640 1 --2- 172.21.15.112:0/1409652957 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f0f84078b90 0x7f0f8407b0a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0fa0153490 tx=0x7f0f90005e70 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:24.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:24.878+0000 7f0f95ffb640 1 -- 172.21.15.112:0/1409652957 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f0f9c0839b0 con 0x7f0fa00d12e0 2024-09-21T00:26:25.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.089+0000 7f0fa7369640 1 -- 172.21.15.112:0/1409652957 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0fa00b56b0 con 0x7f0fa00d12e0 2024-09-21T00:26:25.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.090+0000 7f0f95ffb640 1 -- 172.21.15.112:0/1409652957 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5228 (secure 0 0 0) 0x7f0f9c0832b0 con 0x7f0fa00d12e0 2024-09-21T00:26:25.092 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:26:25.092 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":9,"btime":"2024-09-21T00:26:01:294823+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4316,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6837/2271985854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6835","nonce":2271985854},{"type":"v1","addr":"172.21.15.187:6837","nonce":2271985854}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4413,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/200348130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":200348130},{"type":"v1","addr":"172.21.15.112:6833","nonce":200348130}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4416,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/3821201561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3821201561},{"type":"v1","addr":"172.21.15.112:6835","nonce":3821201561}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T00:25:47.578960+0000","modified":"2024-09-21T00:26:00.650942+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4425},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4425":{"gid":4425,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6836/2921304789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":2921304789},{"type":"v1","addr":"172.21.15.187:6836","nonce":2921304789}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4425,"qdb_cluster":[4425]},"id":1}]} 2024-09-21T00:26:25.092 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 9 2024-09-21T00:26:25.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.092+0000 7f0fa7369640 1 -- 172.21.15.112:0/1409652957 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f0f84078b90 msgr2=0x7f0f8407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:25.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.092+0000 7f0fa7369640 1 --2- 172.21.15.112:0/1409652957 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f0f84078b90 0x7f0f8407b0a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0fa0153490 tx=0x7f0f90005e70 comp rx=0 tx=0).stop 2024-09-21T00:26:25.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.092+0000 7f0fa7369640 1 -- 172.21.15.112:0/1409652957 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0fa00d12e0 msgr2=0x7f0fa013f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:25.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.092+0000 7f0fa7369640 1 --2- 172.21.15.112:0/1409652957 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0fa00d12e0 0x7f0fa013f620 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c034040 tx=0x7f0f9c003910 comp rx=0 tx=0).stop 2024-09-21T00:26:25.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.092+0000 7f0fa7369640 1 -- 172.21.15.112:0/1409652957 shutdown_connections 2024-09-21T00:26:25.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.092+0000 7f0fa7369640 1 --2- 172.21.15.112:0/1409652957 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0fa0160100 0x7f0fa0155450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:25.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.092+0000 7f0fa7369640 1 --2- 172.21.15.112:0/1409652957 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0fa015cb10 0x7f0fa013fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:25.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.092+0000 7f0fa7369640 1 --2- 172.21.15.112:0/1409652957 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f0f84078b90 0x7f0f8407b0a0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:25.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.092+0000 7f0fa7369640 1 --2- 172.21.15.112:0/1409652957 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0fa00d12e0 0x7f0fa013f620 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:25.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.092+0000 7f0fa7369640 1 -- 172.21.15.112:0/1409652957 >> 172.21.15.112:0/1409652957 conn(0x7f0fa00d2350 msgr2=0x7f0fa0070100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:25.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.092+0000 7f0fa7369640 1 -- 172.21.15.112:0/1409652957 shutdown_connections 2024-09-21T00:26:25.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.092+0000 7f0fa7369640 1 -- 172.21.15.112:0/1409652957 wait complete. 2024-09-21T00:26:25.105 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T00:26:25.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.215+0000 7f20c224d640 1 -- 172.21.15.112:0/2486916977 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20bc162280 msgr2=0x7f20bc162690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:25.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.215+0000 7f20c224d640 1 --2- 172.21.15.112:0/2486916977 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20bc162280 0x7f20bc162690 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f20a4005fd0 tx=0x7f20a402a5d0 comp rx=0 tx=0).stop 2024-09-21T00:26:25.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.215+0000 7f20c224d640 1 -- 172.21.15.112:0/2486916977 shutdown_connections 2024-09-21T00:26:25.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.215+0000 7f20c224d640 1 --2- 172.21.15.112:0/2486916977 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f20bc0b2e40 0x7f20bc0b3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:25.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.215+0000 7f20c224d640 1 --2- 172.21.15.112:0/2486916977 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20bc162280 0x7f20bc162690 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:25.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.215+0000 7f20c224d640 1 -- 172.21.15.112:0/2486916977 >> 172.21.15.112:0/2486916977 conn(0x7f20bc0b6b20 msgr2=0x7f20bc0d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:25.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.215+0000 7f20c224d640 1 -- 172.21.15.112:0/2486916977 shutdown_connections 2024-09-21T00:26:25.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.215+0000 7f20c224d640 1 -- 172.21.15.112:0/2486916977 wait complete. 2024-09-21T00:26:25.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.217+0000 7f20c224d640 1 Processor -- start 2024-09-21T00:26:25.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.217+0000 7f20c224d640 1 -- start start 2024-09-21T00:26:25.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.218+0000 7f20c224d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f20bc0b2e40 0x7f20bc15a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:25.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.218+0000 7f20c224d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f20bc162280 0x7f20bc14da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:25.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.218+0000 7f20c224d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20bc0aadc0 0x7f20bc1f88e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:25.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.218+0000 7f20c224d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f20bc15e4b0 con 0x7f20bc0aadc0 2024-09-21T00:26:25.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.218+0000 7f20c224d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f20bc0b39d0 con 0x7f20bc0b2e40 2024-09-21T00:26:25.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.218+0000 7f20c224d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f20bc15e630 con 0x7f20bc162280 2024-09-21T00:26:25.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.218+0000 7f20bbfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20bc0aadc0 0x7f20bc1f88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:25.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.218+0000 7f20bbfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20bc0aadc0 0x7f20bc1f88e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41586/0 (socket says 172.21.15.112:41586) 2024-09-21T00:26:25.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.218+0000 7f20bbfff640 1 -- 172.21.15.112:0/109461231 learned_addr learned my addr 172.21.15.112:0/109461231 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:26:25.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.218+0000 7f20bb7fe640 1 --2- 172.21.15.112:0/109461231 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f20bc0b2e40 0x7f20bc15a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:25.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.218+0000 7f20b3fff640 1 --2- 172.21.15.112:0/109461231 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f20bc162280 0x7f20bc14da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:25.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.219+0000 7f20bbfff640 1 -- 172.21.15.112:0/109461231 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f20bc162280 msgr2=0x7f20bc14da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:25.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.219+0000 7f20bbfff640 1 --2- 172.21.15.112:0/109461231 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f20bc162280 0x7f20bc14da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:25.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.219+0000 7f20bbfff640 1 -- 172.21.15.112:0/109461231 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f20bc0b2e40 msgr2=0x7f20bc15a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:25.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.219+0000 7f20bbfff640 1 --2- 172.21.15.112:0/109461231 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f20bc0b2e40 0x7f20bc15a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:25.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.219+0000 7f20bbfff640 1 -- 172.21.15.112:0/109461231 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20bc14df50 con 0x7f20bc0aadc0 2024-09-21T00:26:25.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.219+0000 7f20bbfff640 1 --2- 172.21.15.112:0/109461231 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20bc0aadc0 0x7f20bc1f88e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f20b400ea40 tx=0x7f20b400ef00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:25.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.219+0000 7f20b8ff9640 1 -- 172.21.15.112:0/109461231 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20b400ce40 con 0x7f20bc0aadc0 2024-09-21T00:26:25.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.219+0000 7f20b8ff9640 1 -- 172.21.15.112:0/109461231 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f20b4004c10 con 0x7f20bc0aadc0 2024-09-21T00:26:25.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.219+0000 7f20c224d640 1 -- 172.21.15.112:0/109461231 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20bc0b1a50 con 0x7f20bc0aadc0 2024-09-21T00:26:25.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.219+0000 7f20b8ff9640 1 -- 172.21.15.112:0/109461231 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20b4005020 con 0x7f20bc0aadc0 2024-09-21T00:26:25.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.219+0000 7f20c224d640 1 -- 172.21.15.112:0/109461231 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20bc14e740 con 0x7f20bc0aadc0 2024-09-21T00:26:25.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.220+0000 7f20c224d640 1 -- 172.21.15.112:0/109461231 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20bc162f00 con 0x7f20bc0aadc0 2024-09-21T00:26:25.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.221+0000 7f20b8ff9640 1 -- 172.21.15.112:0/109461231 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f20b40051c0 con 0x7f20bc0aadc0 2024-09-21T00:26:25.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.221+0000 7f20b8ff9640 1 --2- 172.21.15.112:0/109461231 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f209c078b90 0x7f209c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:25.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.221+0000 7f20b8ff9640 1 -- 172.21.15.112:0/109461231 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f20b409bdc0 con 0x7f20bc0aadc0 2024-09-21T00:26:25.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.222+0000 7f20bb7fe640 1 --2- 172.21.15.112:0/109461231 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f209c078b90 0x7f209c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:25.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.222+0000 7f20bb7fe640 1 --2- 172.21.15.112:0/109461231 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f209c078b90 0x7f209c07b0a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f20ac001720 tx=0x7f20ac009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:25.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.226+0000 7f20b8ff9640 1 -- 172.21.15.112:0/109461231 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f20b4063870 con 0x7f20bc0aadc0 2024-09-21T00:26:25.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.403+0000 7f20c224d640 1 -- 172.21.15.112:0/109461231 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f20bc15dca0 con 0x7f20bc0aadc0 2024-09-21T00:26:25.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.404+0000 7f20b8ff9640 1 -- 172.21.15.112:0/109461231 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18622 (secure 0 0 0) 0x7f20b40682a0 con 0x7f20bc0aadc0 2024-09-21T00:26:25.406 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:26:25.406 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":26,"fsid":"8b34692b-dc65-49b2-a08f-26f707b96ac5","created":"2024-09-21T00:25:22.788490+0000","modified":"2024-09-21T00:25:52.623213+0000","last_up_change":"2024-09-21T00:25:36.161835+0000","last_in_change":"2024-09-21T00:25:26.346500+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T00:25:37.105006+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T00:25:45.413895+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T00:25:46.532124+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T00:25:50.244145+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9c575086-67d2-4de2-b01b-d44dcc21d604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6809","nonce":755760621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6811","nonce":755760621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6815","nonce":755760621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6813","nonce":755760621}]},"public_addr":"172.21.15.112:6809/755760621","cluster_addr":"172.21.15.112:6811/755760621","heartbeat_back_addr":"172.21.15.112:6815/755760621","heartbeat_front_addr":"172.21.15.112:6813/755760621","state":["exists","up"]},{"osd":1,"uuid":"a983c227-a3ce-4bdd-82f1-438ba7e3ecd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6801","nonce":344932592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6803","nonce":344932592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6807","nonce":344932592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6805","nonce":344932592}]},"public_addr":"172.21.15.112:6801/344932592","cluster_addr":"172.21.15.112:6803/344932592","heartbeat_back_addr":"172.21.15.112:6807/344932592","heartbeat_front_addr":"172.21.15.112:6805/344932592","state":["exists","up"]},{"osd":2,"uuid":"1193c1d8-bfff-4a57-8200-5f69bf1ea3af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6825","nonce":407933616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6827","nonce":407933616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6831","nonce":407933616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6829","nonce":407933616}]},"public_addr":"172.21.15.112:6825/407933616","cluster_addr":"172.21.15.112:6827/407933616","heartbeat_back_addr":"172.21.15.112:6831/407933616","heartbeat_front_addr":"172.21.15.112:6829/407933616","state":["exists","up"]},{"osd":3,"uuid":"af04330a-36ad-40cc-a8dd-fa259b221de5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6817","nonce":4120164806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6819","nonce":4120164806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6823","nonce":4120164806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6821","nonce":4120164806}]},"public_addr":"172.21.15.112:6817/4120164806","cluster_addr":"172.21.15.112:6819/4120164806","heartbeat_back_addr":"172.21.15.112:6823/4120164806","heartbeat_front_addr":"172.21.15.112:6821/4120164806","state":["exists","up"]},{"osd":4,"uuid":"5b893111-292c-4c5a-b184-53ad38c7199e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6803","nonce":522077560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6805","nonce":522077560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6809","nonce":522077560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6807","nonce":522077560}]},"public_addr":"172.21.15.187:6803/522077560","cluster_addr":"172.21.15.187:6805/522077560","heartbeat_back_addr":"172.21.15.187:6809/522077560","heartbeat_front_addr":"172.21.15.187:6807/522077560","state":["exists","up"]},{"osd":5,"uuid":"c0b8e011-45fe-4c5e-925e-6be2a7f1868a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6827","nonce":3593144721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6829","nonce":3593144721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6833","nonce":3593144721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6831","nonce":3593144721}]},"public_addr":"172.21.15.187:6827/3593144721","cluster_addr":"172.21.15.187:6829/3593144721","heartbeat_back_addr":"172.21.15.187:6833/3593144721","heartbeat_front_addr":"172.21.15.187:6831/3593144721","state":["exists","up"]},{"osd":6,"uuid":"87d2d650-7489-4593-98a3-a1f433ce8dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6811","nonce":1494387657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6813","nonce":1494387657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6817","nonce":1494387657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6815","nonce":1494387657}]},"public_addr":"172.21.15.187:6811/1494387657","cluster_addr":"172.21.15.187:6813/1494387657","heartbeat_back_addr":"172.21.15.187:6817/1494387657","heartbeat_front_addr":"172.21.15.187:6815/1494387657","state":["exists","up"]},{"osd":7,"uuid":"c6b4f8e2-a1ee-4438-aa37-67ff4b6ef1f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6819","nonce":1624699842}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6821","nonce":1624699842}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6825","nonce":1624699842}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6823","nonce":1624699842}]},"public_addr":"172.21.15.187:6819/1624699842","cluster_addr":"172.21.15.187:6821/1624699842","heartbeat_back_addr":"172.21.15.187:6825/1624699842","heartbeat_front_addr":"172.21.15.187:6823/1624699842","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.873162+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.400506+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.664317+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.159791+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.202630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.113212+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.384786+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.448441+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T00:26:25.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.406+0000 7f20c224d640 1 -- 172.21.15.112:0/109461231 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f209c078b90 msgr2=0x7f209c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:25.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.406+0000 7f20c224d640 1 --2- 172.21.15.112:0/109461231 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f209c078b90 0x7f209c07b0a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f20ac001720 tx=0x7f20ac009040 comp rx=0 tx=0).stop 2024-09-21T00:26:25.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.406+0000 7f20c224d640 1 -- 172.21.15.112:0/109461231 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20bc0aadc0 msgr2=0x7f20bc1f88e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:25.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.406+0000 7f20c224d640 1 --2- 172.21.15.112:0/109461231 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20bc0aadc0 0x7f20bc1f88e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f20b400ea40 tx=0x7f20b400ef00 comp rx=0 tx=0).stop 2024-09-21T00:26:25.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.407+0000 7f20c224d640 1 -- 172.21.15.112:0/109461231 shutdown_connections 2024-09-21T00:26:25.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.407+0000 7f20c224d640 1 --2- 172.21.15.112:0/109461231 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f209c078b90 0x7f209c07b0a0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:25.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.407+0000 7f20c224d640 1 --2- 172.21.15.112:0/109461231 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20bc0aadc0 0x7f20bc1f88e0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:25.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.407+0000 7f20c224d640 1 --2- 172.21.15.112:0/109461231 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f20bc162280 0x7f20bc14da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:25.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.407+0000 7f20c224d640 1 --2- 172.21.15.112:0/109461231 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f20bc0b2e40 0x7f20bc15a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:25.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.407+0000 7f20c224d640 1 -- 172.21.15.112:0/109461231 >> 172.21.15.112:0/109461231 conn(0x7f20bc0b6b20 msgr2=0x7f20bc06cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:25.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.407+0000 7f20c224d640 1 -- 172.21.15.112:0/109461231 shutdown_connections 2024-09-21T00:26:25.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.407+0000 7f20c224d640 1 -- 172.21.15.112:0/109461231 wait complete. 2024-09-21T00:26:25.420 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T00:26:25.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.533+0000 7f151f95e640 1 -- 172.21.15.112:0/44522800 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1504003a50 msgr2=0x7f1504003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:25.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.533+0000 7f151f95e640 1 --2- 172.21.15.112:0/44522800 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1504003a50 0x7f1504003ec0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f1514001dc0 tx=0x7f1504005830 comp rx=0 tx=0).stop 2024-09-21T00:26:25.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.533+0000 7f151f95e640 1 -- 172.21.15.112:0/44522800 shutdown_connections 2024-09-21T00:26:25.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.533+0000 7f151f95e640 1 --2- 172.21.15.112:0/44522800 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1504003a50 0x7f1504003ec0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:25.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.534+0000 7f151f95e640 1 -- 172.21.15.112:0/44522800 >> 172.21.15.112:0/44522800 conn(0x7f15180735d0 msgr2=0x7f15180739e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:25.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.534+0000 7f151f95e640 1 -- 172.21.15.112:0/44522800 shutdown_connections 2024-09-21T00:26:25.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.534+0000 7f151f95e640 1 -- 172.21.15.112:0/44522800 wait complete. 2024-09-21T00:26:25.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.535+0000 7f151f95e640 1 Processor -- start 2024-09-21T00:26:25.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.535+0000 7f151f95e640 1 -- start start 2024-09-21T00:26:25.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.537+0000 7f151f95e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1504003a50 0x7f15180c9360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:25.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.537+0000 7f151f95e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15180c8830 0x7f15180b4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:25.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.537+0000 7f151f95e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f15180cf080 0x7f15180b4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:25.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.537+0000 7f151f95e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f15180caf40 con 0x7f15180c8830 2024-09-21T00:26:25.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.537+0000 7f151f95e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f15180b6010 con 0x7f15180cf080 2024-09-21T00:26:25.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.537+0000 7f151f95e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f15180cb0c0 con 0x7f1504003a50 2024-09-21T00:26:25.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.537+0000 7f151cef9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15180c8830 0x7f15180b4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:25.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.537+0000 7f151cef9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15180c8830 0x7f15180b4770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41594/0 (socket says 172.21.15.112:41594) 2024-09-21T00:26:25.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.537+0000 7f151cef9640 1 -- 172.21.15.112:0/3208379970 learned_addr learned my addr 172.21.15.112:0/3208379970 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:26:25.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.537+0000 7f151d6fa640 1 --2- 172.21.15.112:0/3208379970 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1504003a50 0x7f15180c9360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:25.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.537+0000 7f151defb640 1 --2- 172.21.15.112:0/3208379970 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f15180cf080 0x7f15180b4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:25.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.537+0000 7f151cef9640 1 -- 172.21.15.112:0/3208379970 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1504003a50 msgr2=0x7f15180c9360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:25.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.537+0000 7f151cef9640 1 --2- 172.21.15.112:0/3208379970 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1504003a50 0x7f15180c9360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:25.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.537+0000 7f151cef9640 1 -- 172.21.15.112:0/3208379970 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f15180cf080 msgr2=0x7f15180b4cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:25.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.537+0000 7f151cef9640 1 --2- 172.21.15.112:0/3208379970 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f15180cf080 0x7f15180b4cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:25.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.537+0000 7f151cef9640 1 -- 172.21.15.112:0/3208379970 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15181f5370 con 0x7f15180c8830 2024-09-21T00:26:25.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.537+0000 7f151cef9640 1 --2- 172.21.15.112:0/3208379970 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15180c8830 0x7f15180b4770 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f150800e590 tx=0x7f150800e990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:25.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.537+0000 7f150dffb640 1 -- 172.21.15.112:0/3208379970 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1508015070 con 0x7f15180c8830 2024-09-21T00:26:25.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.537+0000 7f150dffb640 1 -- 172.21.15.112:0/3208379970 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1508010980 con 0x7f15180c8830 2024-09-21T00:26:25.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.537+0000 7f151f95e640 1 -- 172.21.15.112:0/3208379970 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15181f8e30 con 0x7f15180c8830 2024-09-21T00:26:25.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.537+0000 7f150dffb640 1 -- 172.21.15.112:0/3208379970 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1508010db0 con 0x7f15180c8830 2024-09-21T00:26:25.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.538+0000 7f151f95e640 1 -- 172.21.15.112:0/3208379970 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15181f93f0 con 0x7f15180c8830 2024-09-21T00:26:25.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.539+0000 7f151f95e640 1 -- 172.21.15.112:0/3208379970 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15181f8ba0 con 0x7f15180c8830 2024-09-21T00:26:25.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.539+0000 7f150dffb640 1 -- 172.21.15.112:0/3208379970 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1508018070 con 0x7f15180c8830 2024-09-21T00:26:25.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.540+0000 7f150dffb640 1 --2- 172.21.15.112:0/3208379970 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f14fc078b90 0x7f14fc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:25.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.540+0000 7f150dffb640 1 -- 172.21.15.112:0/3208379970 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f150809b540 con 0x7f15180c8830 2024-09-21T00:26:25.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.540+0000 7f151d6fa640 1 --2- 172.21.15.112:0/3208379970 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f14fc078b90 0x7f14fc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:25.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.541+0000 7f151d6fa640 1 --2- 172.21.15.112:0/3208379970 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f14fc078b90 0x7f14fc07b0a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1514032000 tx=0x7f1514001510 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:25.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.544+0000 7f150dffb640 1 -- 172.21.15.112:0/3208379970 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f15080635c0 con 0x7f15180c8830 2024-09-21T00:26:25.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.757+0000 7f151f95e640 1 -- 172.21.15.112:0/3208379970 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f151806ddf0 con 0x7f15180c8830 2024-09-21T00:26:25.759 INFO:tasks.ceph.mon.a.smithi112.stderr:2024-09-21T00:26:25.758+0000 7f7656c13640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T00:26:25.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.761+0000 7f150dffb640 1 -- 172.21.15.112:0/3208379970 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v10) ==== 161+0+0 (secure 0 0 0) 0x7f1508062ec0 con 0x7f15180c8830 2024-09-21T00:26:25.763 INFO:teuthology.orchestra.run.smithi112.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T00:26:25.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.763+0000 7f151f95e640 1 -- 172.21.15.112:0/3208379970 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f14fc078b90 msgr2=0x7f14fc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:25.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.763+0000 7f151f95e640 1 --2- 172.21.15.112:0/3208379970 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f14fc078b90 0x7f14fc07b0a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1514032000 tx=0x7f1514001510 comp rx=0 tx=0).stop 2024-09-21T00:26:25.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.763+0000 7f151f95e640 1 -- 172.21.15.112:0/3208379970 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15180c8830 msgr2=0x7f15180b4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:25.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.763+0000 7f151f95e640 1 --2- 172.21.15.112:0/3208379970 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15180c8830 0x7f15180b4770 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f150800e590 tx=0x7f150800e990 comp rx=0 tx=0).stop 2024-09-21T00:26:25.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.764+0000 7f151f95e640 1 -- 172.21.15.112:0/3208379970 shutdown_connections 2024-09-21T00:26:25.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.764+0000 7f151f95e640 1 --2- 172.21.15.112:0/3208379970 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f15180cf080 0x7f15180b4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:25.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.764+0000 7f151f95e640 1 --2- 172.21.15.112:0/3208379970 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f14fc078b90 0x7f14fc07b0a0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:25.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.764+0000 7f151f95e640 1 --2- 172.21.15.112:0/3208379970 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15180c8830 0x7f15180b4770 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:25.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.764+0000 7f151f95e640 1 --2- 172.21.15.112:0/3208379970 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1504003a50 0x7f15180c9360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:25.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.764+0000 7f151f95e640 1 -- 172.21.15.112:0/3208379970 >> 172.21.15.112:0/3208379970 conn(0x7f15180735d0 msgr2=0x7f151815abb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:25.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.764+0000 7f151f95e640 1 -- 172.21.15.112:0/3208379970 shutdown_connections 2024-09-21T00:26:25.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.764+0000 7f151f95e640 1 -- 172.21.15.112:0/3208379970 wait complete. 2024-09-21T00:26:25.770 INFO:tasks.ceph.mds.b.smithi187.stderr: -4262> 2024-09-21T00:25:53.157+0000 7ff382263640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T00:26:25.777 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-21T00:26:25.828 INFO:tasks.ceph.mds.b.smithi187.stdout:starting mds.b at 2024-09-21T00:26:25.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.890+0000 7fecf89e6640 1 -- 172.21.15.112:0/962816261 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fecf415ee80 msgr2=0x7fecf40d0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:25.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.890+0000 7fecf89e6640 1 --2- 172.21.15.112:0/962816261 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fecf415ee80 0x7fecf40d0170 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fecf40b3c20 tx=0x7fece402a410 comp rx=0 tx=0).stop 2024-09-21T00:26:25.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.891+0000 7fecf89e6640 1 -- 172.21.15.112:0/962816261 shutdown_connections 2024-09-21T00:26:25.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.891+0000 7fecf89e6640 1 --2- 172.21.15.112:0/962816261 >> v2:172.21.15.187:3300/0 conn(0x7fecf40ceb90 0x7fecf40cefa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:25.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.891+0000 7fecf89e6640 1 --2- 172.21.15.112:0/962816261 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fecf415ee80 0x7fecf40d0170 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:25.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.891+0000 7fecf89e6640 1 -- 172.21.15.112:0/962816261 >> 172.21.15.112:0/962816261 conn(0x7fecf409d830 msgr2=0x7fecf40b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:25.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.891+0000 7fecf89e6640 1 -- 172.21.15.112:0/962816261 shutdown_connections 2024-09-21T00:26:25.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.891+0000 7fecf89e6640 1 -- 172.21.15.112:0/962816261 wait complete. 2024-09-21T00:26:25.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.893+0000 7fecf89e6640 1 Processor -- start 2024-09-21T00:26:25.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.893+0000 7fecf89e6640 1 -- start start 2024-09-21T00:26:25.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.894+0000 7fecf89e6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fecf40b9ad0 0x7fecf4158990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:25.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.894+0000 7fecf89e6640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fecf40ceb90 0x7fecf40ccdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:25.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.894+0000 7fecf89e6640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fecf415ee80 0x7fecf406eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:25.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.894+0000 7fecf89e6640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fecf40b6700 con 0x7fecf40b9ad0 2024-09-21T00:26:25.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.894+0000 7fecf89e6640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fecf40b6580 con 0x7fecf415ee80 2024-09-21T00:26:25.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.894+0000 7fecf89e6640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fecf40b6880 con 0x7fecf40ceb90 2024-09-21T00:26:25.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.894+0000 7fecf259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fecf40b9ad0 0x7fecf4158990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:25.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.894+0000 7fecf259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fecf40b9ad0 0x7fecf4158990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41626/0 (socket says 172.21.15.112:41626) 2024-09-21T00:26:25.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.894+0000 7fecf259c640 1 -- 172.21.15.112:0/707586410 learned_addr learned my addr 172.21.15.112:0/707586410 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:26:25.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.894+0000 7fecf2d9d640 1 --2- 172.21.15.112:0/707586410 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fecf415ee80 0x7fecf406eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:25.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.894+0000 7fece9d9b640 1 --2- 172.21.15.112:0/707586410 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fecf40ceb90 0x7fecf40ccdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:25.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.894+0000 7fecf259c640 1 -- 172.21.15.112:0/707586410 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fecf40ceb90 msgr2=0x7fecf40ccdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:25.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.894+0000 7fecf259c640 1 --2- 172.21.15.112:0/707586410 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fecf40ceb90 0x7fecf40ccdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:25.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.894+0000 7fecf259c640 1 -- 172.21.15.112:0/707586410 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fecf415ee80 msgr2=0x7fecf406eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:25.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.894+0000 7fecf259c640 1 --2- 172.21.15.112:0/707586410 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fecf415ee80 0x7fecf406eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:25.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.894+0000 7fecf259c640 1 -- 172.21.15.112:0/707586410 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecf40cd900 con 0x7fecf40b9ad0 2024-09-21T00:26:25.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.894+0000 7fecf2d9d640 1 --2- 172.21.15.112:0/707586410 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fecf415ee80 0x7fecf406eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:26:25.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.895+0000 7fecf259c640 1 --2- 172.21.15.112:0/707586410 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fecf40b9ad0 0x7fecf4158990 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fece4009780 tx=0x7fece4033d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:25.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.895+0000 7feceb7fe640 1 -- 172.21.15.112:0/707586410 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fece4031d60 con 0x7fecf40b9ad0 2024-09-21T00:26:25.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.895+0000 7feceb7fe640 1 -- 172.21.15.112:0/707586410 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fece4005020 con 0x7fecf40b9ad0 2024-09-21T00:26:25.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.895+0000 7fecf89e6640 1 -- 172.21.15.112:0/707586410 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecf41f53f0 con 0x7fecf40b9ad0 2024-09-21T00:26:25.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.895+0000 7feceb7fe640 1 -- 172.21.15.112:0/707586410 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fece4005470 con 0x7fecf40b9ad0 2024-09-21T00:26:25.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.895+0000 7fecf89e6640 1 -- 172.21.15.112:0/707586410 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecf41f59b0 con 0x7fecf40b9ad0 2024-09-21T00:26:25.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.896+0000 7fecf89e6640 1 -- 172.21.15.112:0/707586410 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fecf40b6dd0 con 0x7fecf40b9ad0 2024-09-21T00:26:25.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.897+0000 7feceb7fe640 1 -- 172.21.15.112:0/707586410 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fece40047a0 con 0x7fecf40b9ad0 2024-09-21T00:26:25.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.897+0000 7feceb7fe640 1 --2- 172.21.15.112:0/707586410 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fecd4078b90 0x7fecd407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:25.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.897+0000 7feceb7fe640 1 -- 172.21.15.112:0/707586410 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6618+0+0 (secure 0 0 0) 0x7fece40bc320 con 0x7fecf40b9ad0 2024-09-21T00:26:25.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.897+0000 7fece9d9b640 1 --2- 172.21.15.112:0/707586410 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fecd4078b90 0x7fecd407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:25.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.898+0000 7fece9d9b640 1 --2- 172.21.15.112:0/707586410 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fecd4078b90 0x7fecd407b0a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fecf40ce910 tx=0x7fecdc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:25.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:25.902+0000 7feceb7fe640 1 -- 172.21.15.112:0/707586410 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fece403f030 con 0x7fecf40b9ad0 2024-09-21T00:26:26.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:26.113+0000 7fecf89e6640 1 -- 172.21.15.112:0/707586410 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fecf40b4c50 con 0x7fecf40b9ad0 2024-09-21T00:26:26.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:26.762+0000 7feceb7fe640 1 -- 172.21.15.112:0/707586410 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v11) ==== 104+0+0 (secure 0 0 0) 0x7fece4084460 con 0x7fecf40b9ad0 2024-09-21T00:26:26.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:26.765+0000 7fecf89e6640 1 -- 172.21.15.112:0/707586410 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fecd4078b90 msgr2=0x7fecd407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:26.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:26.765+0000 7fecf89e6640 1 --2- 172.21.15.112:0/707586410 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fecd4078b90 0x7fecd407b0a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fecf40ce910 tx=0x7fecdc009040 comp rx=0 tx=0).stop 2024-09-21T00:26:26.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:26.765+0000 7fecf89e6640 1 -- 172.21.15.112:0/707586410 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fecf40b9ad0 msgr2=0x7fecf4158990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:26.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:26.765+0000 7fecf89e6640 1 --2- 172.21.15.112:0/707586410 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fecf40b9ad0 0x7fecf4158990 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fece4009780 tx=0x7fece4033d60 comp rx=0 tx=0).stop 2024-09-21T00:26:26.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:26.765+0000 7fecf89e6640 1 -- 172.21.15.112:0/707586410 shutdown_connections 2024-09-21T00:26:26.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:26.765+0000 7fecf89e6640 1 --2- 172.21.15.112:0/707586410 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fecf415ee80 0x7fecf406eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:26.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:26.765+0000 7fecf89e6640 1 --2- 172.21.15.112:0/707586410 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fecf40ceb90 0x7fecf40ccdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:26.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:26.765+0000 7fecf89e6640 1 --2- 172.21.15.112:0/707586410 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fecd4078b90 0x7fecd407b0a0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:26.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:26.765+0000 7fecf89e6640 1 --2- 172.21.15.112:0/707586410 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fecf40b9ad0 0x7fecf4158990 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:26.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:26.765+0000 7fecf89e6640 1 -- 172.21.15.112:0/707586410 >> 172.21.15.112:0/707586410 conn(0x7fecf409d830 msgr2=0x7fecf40b31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:26.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:26.765+0000 7fecf89e6640 1 -- 172.21.15.112:0/707586410 shutdown_connections 2024-09-21T00:26:26.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:26.765+0000 7fecf89e6640 1 -- 172.21.15.112:0/707586410 wait complete. 2024-09-21T00:26:26.779 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-21T00:26:26.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:26.896+0000 7f68a9d69640 1 -- 172.21.15.112:0/1977246052 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f68a4070940 msgr2=0x7f68a40ce8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:26.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:26.896+0000 7f68a9d69640 1 --2- 172.21.15.112:0/1977246052 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f68a4070940 0x7f68a40ce8a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f688c009830 tx=0x7f688c02a510 comp rx=0 tx=0).stop 2024-09-21T00:26:26.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:26.897+0000 7f68a9d69640 1 -- 172.21.15.112:0/1977246052 shutdown_connections 2024-09-21T00:26:26.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:26.897+0000 7f68a9d69640 1 --2- 172.21.15.112:0/1977246052 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f68a4070940 0x7f68a40ce8a0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:26.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:26.897+0000 7f68a9d69640 1 --2- 172.21.15.112:0/1977246052 >> v2:172.21.15.187:3300/0 conn(0x7f68a40cd790 0x7f68a40cbd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:26.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:26.897+0000 7f68a9d69640 1 -- 172.21.15.112:0/1977246052 >> 172.21.15.112:0/1977246052 conn(0x7f68a40ccac0 msgr2=0x7f68a40cced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:26.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:26.897+0000 7f68a9d69640 1 -- 172.21.15.112:0/1977246052 shutdown_connections 2024-09-21T00:26:26.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:26.897+0000 7f68a9d69640 1 -- 172.21.15.112:0/1977246052 wait complete. 2024-09-21T00:26:26.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:26.898+0000 7f68a9d69640 1 Processor -- start 2024-09-21T00:26:26.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:26.898+0000 7f68a9d69640 1 -- start start 2024-09-21T00:26:26.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:26.900+0000 7f68a9d69640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68a406def0 0x7f68a41f6680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:26.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:26.900+0000 7f68a9d69640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f68a4070940 0x7f68a40c7300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:26.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:26.900+0000 7f68a9d69640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f68a40cd790 0x7f68a40c7840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:26.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:26.900+0000 7f68a9d69640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f68a406f6b0 con 0x7f68a406def0 2024-09-21T00:26:26.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:26.900+0000 7f68a9d69640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f68a406f530 con 0x7f68a40cd790 2024-09-21T00:26:26.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:26.900+0000 7f68a9d69640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f68a406f830 con 0x7f68a4070940 2024-09-21T00:26:26.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:26.900+0000 7f68a37fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68a406def0 0x7f68a41f6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:26.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:26.900+0000 7f68a37fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68a406def0 0x7f68a41f6680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59702/0 (socket says 172.21.15.112:59702) 2024-09-21T00:26:26.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:26.900+0000 7f68a37fe640 1 -- 172.21.15.112:0/1031835649 learned_addr learned my addr 172.21.15.112:0/1031835649 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:26:26.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:26.900+0000 7f68a2ffd640 1 --2- 172.21.15.112:0/1031835649 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f68a4070940 0x7f68a40c7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:26.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:26.900+0000 7f68a3fff640 1 --2- 172.21.15.112:0/1031835649 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f68a40cd790 0x7f68a40c7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:26.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:26.900+0000 7f68a37fe640 1 -- 172.21.15.112:0/1031835649 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f68a4070940 msgr2=0x7f68a40c7300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:26.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:26.900+0000 7f68a37fe640 1 --2- 172.21.15.112:0/1031835649 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f68a4070940 0x7f68a40c7300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:26.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:26.900+0000 7f68a37fe640 1 -- 172.21.15.112:0/1031835649 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f68a40cd790 msgr2=0x7f68a40c7840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:26.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:26.900+0000 7f68a37fe640 1 --2- 172.21.15.112:0/1031835649 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f68a40cd790 0x7f68a40c7840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:26.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:26.900+0000 7f68a37fe640 1 -- 172.21.15.112:0/1031835649 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68a41f3c00 con 0x7f68a406def0 2024-09-21T00:26:26.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:26.900+0000 7f68a37fe640 1 --2- 172.21.15.112:0/1031835649 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68a406def0 0x7f68a41f6680 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f689400e7c0 tx=0x7f689400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:26.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:26.901+0000 7f689bfff640 1 -- 172.21.15.112:0/1031835649 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f689400cc60 con 0x7f68a406def0 2024-09-21T00:26:26.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:26.901+0000 7f689bfff640 1 -- 172.21.15.112:0/1031835649 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6894010430 con 0x7f68a406def0 2024-09-21T00:26:26.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:26.901+0000 7f68a9d69640 1 -- 172.21.15.112:0/1031835649 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68a41f3ee0 con 0x7f68a406def0 2024-09-21T00:26:26.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:26.901+0000 7f689bfff640 1 -- 172.21.15.112:0/1031835649 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6894010860 con 0x7f68a406def0 2024-09-21T00:26:26.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:26.901+0000 7f68a9d69640 1 -- 172.21.15.112:0/1031835649 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68a41f6bc0 con 0x7f68a406def0 2024-09-21T00:26:26.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:26.902+0000 7f68a9d69640 1 -- 172.21.15.112:0/1031835649 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68a40ce8a0 con 0x7f68a406def0 2024-09-21T00:26:26.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:26.902+0000 7f689bfff640 1 -- 172.21.15.112:0/1031835649 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6894010a00 con 0x7f68a406def0 2024-09-21T00:26:26.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:26.903+0000 7f689bfff640 1 --2- 172.21.15.112:0/1031835649 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f6884078bc0 0x7f688407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:26.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:26.903+0000 7f689bfff640 1 -- 172.21.15.112:0/1031835649 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6618+0+0 (secure 0 0 0) 0x7f689409b950 con 0x7f68a406def0 2024-09-21T00:26:26.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:26.903+0000 7f68a2ffd640 1 --2- 172.21.15.112:0/1031835649 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f6884078bc0 0x7f688407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:26.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:26.904+0000 7f68a2ffd640 1 --2- 172.21.15.112:0/1031835649 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f6884078bc0 0x7f688407b0d0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f688c034000 tx=0x7f688c004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:26.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:26.907+0000 7f689bfff640 1 -- 172.21.15.112:0/1031835649 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f6894063440 con 0x7f68a406def0 2024-09-21T00:26:27.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:27.084+0000 7f68a9d69640 1 -- 172.21.15.112:0/1031835649 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f68a4068fb0 con 0x7f68a406def0 2024-09-21T00:26:27.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:27.767+0000 7f689bfff640 1 -- 172.21.15.112:0/1031835649 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v28) ==== 181+0+0 (secure 0 0 0) 0x7f6894067e70 con 0x7f68a406def0 2024-09-21T00:26:27.769 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'cephfs_metadata' removed 2024-09-21T00:26:27.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:27.771+0000 7f68a9d69640 1 -- 172.21.15.112:0/1031835649 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f6884078bc0 msgr2=0x7f688407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:27.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:27.771+0000 7f68a9d69640 1 --2- 172.21.15.112:0/1031835649 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f6884078bc0 0x7f688407b0d0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f688c034000 tx=0x7f688c004730 comp rx=0 tx=0).stop 2024-09-21T00:26:27.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:27.771+0000 7f68a9d69640 1 -- 172.21.15.112:0/1031835649 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68a406def0 msgr2=0x7f68a41f6680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:27.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:27.771+0000 7f68a9d69640 1 --2- 172.21.15.112:0/1031835649 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68a406def0 0x7f68a41f6680 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f689400e7c0 tx=0x7f689400ec80 comp rx=0 tx=0).stop 2024-09-21T00:26:27.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:27.771+0000 7f68a9d69640 1 -- 172.21.15.112:0/1031835649 shutdown_connections 2024-09-21T00:26:27.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:27.771+0000 7f68a9d69640 1 --2- 172.21.15.112:0/1031835649 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f68a40cd790 0x7f68a40c7840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:27.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:27.771+0000 7f68a9d69640 1 --2- 172.21.15.112:0/1031835649 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f68a4070940 0x7f68a40c7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:27.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:27.771+0000 7f68a9d69640 1 --2- 172.21.15.112:0/1031835649 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f6884078bc0 0x7f688407b0d0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:27.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:27.771+0000 7f68a9d69640 1 --2- 172.21.15.112:0/1031835649 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68a406def0 0x7f68a41f6680 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:27.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:27.771+0000 7f68a9d69640 1 -- 172.21.15.112:0/1031835649 >> 172.21.15.112:0/1031835649 conn(0x7f68a40ccac0 msgr2=0x7f68a40745e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:27.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:27.772+0000 7f68a9d69640 1 -- 172.21.15.112:0/1031835649 shutdown_connections 2024-09-21T00:26:27.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:27.772+0000 7f68a9d69640 1 -- 172.21.15.112:0/1031835649 wait complete. 2024-09-21T00:26:27.793 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-21T00:26:27.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:27.911+0000 7f76576e5640 1 -- 172.21.15.112:0/3028402938 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7650162c10 msgr2=0x7f76500d25d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:27.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:27.911+0000 7f76576e5640 1 --2- 172.21.15.112:0/3028402938 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7650162c10 0x7f76500d25d0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f7648009830 tx=0x7f764802a510 comp rx=0 tx=0).stop 2024-09-21T00:26:27.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:27.911+0000 7f76576e5640 1 -- 172.21.15.112:0/3028402938 shutdown_connections 2024-09-21T00:26:27.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:27.911+0000 7f76576e5640 1 --2- 172.21.15.112:0/3028402938 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7650162c10 0x7f76500d25d0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:27.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:27.911+0000 7f76576e5640 1 --2- 172.21.15.112:0/3028402938 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f76500b6670 0x7f76501626d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:27.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:27.911+0000 7f76576e5640 1 -- 172.21.15.112:0/3028402938 >> 172.21.15.112:0/3028402938 conn(0x7f76500d44e0 msgr2=0x7f76500d48f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:27.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:27.911+0000 7f76576e5640 1 -- 172.21.15.112:0/3028402938 shutdown_connections 2024-09-21T00:26:27.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:27.911+0000 7f76576e5640 1 -- 172.21.15.112:0/3028402938 wait complete. 2024-09-21T00:26:27.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:27.913+0000 7f76576e5640 1 Processor -- start 2024-09-21T00:26:27.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:27.913+0000 7f76576e5640 1 -- start start 2024-09-21T00:26:27.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:27.914+0000 7f76576e5640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f765015e7d0 0x7f76500abdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:27.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:27.914+0000 7f76576e5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7650162c10 0x7f76501f5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:27.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:27.914+0000 7f76576e5640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f76500cdf10 0x7f76500b27c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:27.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:27.914+0000 7f76576e5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f76500d2ea0 con 0x7f7650162c10 2024-09-21T00:26:27.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:27.914+0000 7f76576e5640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f76500d2d20 con 0x7f765015e7d0 2024-09-21T00:26:27.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:27.914+0000 7f76576e5640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f76500d3020 con 0x7f76500cdf10 2024-09-21T00:26:27.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:27.914+0000 7f7654c80640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7650162c10 0x7f76501f5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:27.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:27.914+0000 7f7654c80640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7650162c10 0x7f76501f5880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59734/0 (socket says 172.21.15.112:59734) 2024-09-21T00:26:27.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:27.914+0000 7f7654c80640 1 -- 172.21.15.112:0/123798626 learned_addr learned my addr 172.21.15.112:0/123798626 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:26:27.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:27.914+0000 7f7655c82640 1 --2- 172.21.15.112:0/123798626 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f76500cdf10 0x7f76500b27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:27.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:27.914+0000 7f7655481640 1 --2- 172.21.15.112:0/123798626 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f765015e7d0 0x7f76500abdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:27.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:27.914+0000 7f7654c80640 1 -- 172.21.15.112:0/123798626 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f76500cdf10 msgr2=0x7f76500b27c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:27.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:27.915+0000 7f7654c80640 1 --2- 172.21.15.112:0/123798626 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f76500cdf10 0x7f76500b27c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:27.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:27.915+0000 7f7654c80640 1 -- 172.21.15.112:0/123798626 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f765015e7d0 msgr2=0x7f76500abdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:27.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:27.915+0000 7f7654c80640 1 --2- 172.21.15.112:0/123798626 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f765015e7d0 0x7f76500abdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:27.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:27.915+0000 7f7654c80640 1 -- 172.21.15.112:0/123798626 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76500ab000 con 0x7f7650162c10 2024-09-21T00:26:27.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:27.915+0000 7f7655c82640 1 --2- 172.21.15.112:0/123798626 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f76500cdf10 0x7f76500b27c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:26:27.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:27.915+0000 7f7655481640 1 --2- 172.21.15.112:0/123798626 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f765015e7d0 0x7f76500abdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:26:27.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:27.915+0000 7f7654c80640 1 --2- 172.21.15.112:0/123798626 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7650162c10 0x7f76501f5880 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f764000d740 tx=0x7f764000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:27.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:27.915+0000 7f7645ffb640 1 -- 172.21.15.112:0/123798626 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7640017070 con 0x7f7650162c10 2024-09-21T00:26:27.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:27.915+0000 7f7645ffb640 1 -- 172.21.15.112:0/123798626 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7640009d20 con 0x7f7650162c10 2024-09-21T00:26:27.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:27.915+0000 7f76576e5640 1 -- 172.21.15.112:0/123798626 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76500ab2e0 con 0x7f7650162c10 2024-09-21T00:26:27.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:27.915+0000 7f7645ffb640 1 -- 172.21.15.112:0/123798626 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76400106c0 con 0x7f7650162c10 2024-09-21T00:26:27.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:27.915+0000 7f76576e5640 1 -- 172.21.15.112:0/123798626 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76500ab8a0 con 0x7f7650162c10 2024-09-21T00:26:27.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:27.916+0000 7f76576e5640 1 -- 172.21.15.112:0/123798626 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76500d34b0 con 0x7f7650162c10 2024-09-21T00:26:27.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:27.917+0000 7f7645ffb640 1 -- 172.21.15.112:0/123798626 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f764001a070 con 0x7f7650162c10 2024-09-21T00:26:27.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:27.917+0000 7f7645ffb640 1 --2- 172.21.15.112:0/123798626 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f7634078b90 0x7f763407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:27.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:27.917+0000 7f7645ffb640 1 -- 172.21.15.112:0/123798626 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6383+0+0 (secure 0 0 0) 0x7f764009b120 con 0x7f7650162c10 2024-09-21T00:26:27.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:27.917+0000 7f7655481640 1 --2- 172.21.15.112:0/123798626 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f7634078b90 0x7f763407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:27.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:27.918+0000 7f7655481640 1 --2- 172.21.15.112:0/123798626 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f7634078b90 0x7f763407b0a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f764c0012e0 tx=0x7f764c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:27.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:27.922+0000 7f7645ffb640 1 -- 172.21.15.112:0/123798626 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f7640063360 con 0x7f7650162c10 2024-09-21T00:26:28.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:28.098+0000 7f76576e5640 1 -- 172.21.15.112:0/123798626 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f76500ce380 con 0x7f7650162c10 2024-09-21T00:26:28.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:28.769+0000 7f7645ffb640 1 -- 172.21.15.112:0/123798626 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v29) ==== 169+0+0 (secure 0 0 0) 0x7f7640062c60 con 0x7f7650162c10 2024-09-21T00:26:28.771 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'cephfs_data' removed 2024-09-21T00:26:28.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:28.772+0000 7f76576e5640 1 -- 172.21.15.112:0/123798626 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f7634078b90 msgr2=0x7f763407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:28.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:28.772+0000 7f76576e5640 1 --2- 172.21.15.112:0/123798626 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f7634078b90 0x7f763407b0a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f764c0012e0 tx=0x7f764c009040 comp rx=0 tx=0).stop 2024-09-21T00:26:28.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:28.772+0000 7f76576e5640 1 -- 172.21.15.112:0/123798626 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7650162c10 msgr2=0x7f76501f5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:28.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:28.772+0000 7f76576e5640 1 --2- 172.21.15.112:0/123798626 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7650162c10 0x7f76501f5880 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f764000d740 tx=0x7f764000dc00 comp rx=0 tx=0).stop 2024-09-21T00:26:28.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:28.772+0000 7f76576e5640 1 -- 172.21.15.112:0/123798626 shutdown_connections 2024-09-21T00:26:28.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:28.772+0000 7f76576e5640 1 --2- 172.21.15.112:0/123798626 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f76500cdf10 0x7f76500b27c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:28.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:28.772+0000 7f76576e5640 1 --2- 172.21.15.112:0/123798626 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f7634078b90 0x7f763407b0a0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:28.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:28.772+0000 7f76576e5640 1 --2- 172.21.15.112:0/123798626 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7650162c10 0x7f76501f5880 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:28.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:28.772+0000 7f76576e5640 1 --2- 172.21.15.112:0/123798626 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f765015e7d0 0x7f76500abdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:28.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:28.772+0000 7f76576e5640 1 -- 172.21.15.112:0/123798626 >> 172.21.15.112:0/123798626 conn(0x7f76500d44e0 msgr2=0x7f765015da50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:28.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:28.773+0000 7f76576e5640 1 -- 172.21.15.112:0/123798626 shutdown_connections 2024-09-21T00:26:28.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:28.773+0000 7f76576e5640 1 -- 172.21.15.112:0/123798626 wait complete. 2024-09-21T00:26:28.790 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-09-21T00:26:28.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:28.905+0000 7fb6f4d69640 1 -- 172.21.15.112:0/2561227770 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6f00b7b60 msgr2=0x7fb6f00b7f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:28.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:28.905+0000 7fb6f4d69640 1 --2- 172.21.15.112:0/2561227770 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6f00b7b60 0x7fb6f00b7f70 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fb6f008d790 tx=0x7fb6d802a560 comp rx=0 tx=0).stop 2024-09-21T00:26:28.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:28.905+0000 7fb6f4d69640 1 -- 172.21.15.112:0/2561227770 shutdown_connections 2024-09-21T00:26:28.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:28.905+0000 7fb6f4d69640 1 --2- 172.21.15.112:0/2561227770 >> v2:172.21.15.187:3300/0 conn(0x7fb6f00cf200 0x7fb6f00cf610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:28.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:28.905+0000 7fb6f4d69640 1 --2- 172.21.15.112:0/2561227770 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6f00b7b60 0x7fb6f00b7f70 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:28.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:28.905+0000 7fb6f4d69640 1 --2- 172.21.15.112:0/2561227770 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb6f00d0460 0x7fb6f0161ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:28.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:28.905+0000 7fb6f4d69640 1 -- 172.21.15.112:0/2561227770 >> 172.21.15.112:0/2561227770 conn(0x7fb6f00d4510 msgr2=0x7fb6f00d4920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:28.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:28.905+0000 7fb6f4d69640 1 -- 172.21.15.112:0/2561227770 shutdown_connections 2024-09-21T00:26:28.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:28.905+0000 7fb6f4d69640 1 -- 172.21.15.112:0/2561227770 wait complete. 2024-09-21T00:26:28.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:28.907+0000 7fb6f4d69640 1 Processor -- start 2024-09-21T00:26:28.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:28.907+0000 7fb6f4d69640 1 -- start start 2024-09-21T00:26:28.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:28.908+0000 7fb6f4d69640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb6f00b7b60 0x7fb6f014d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:28.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:28.908+0000 7fb6f4d69640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb6f00cf200 0x7fb6f014a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:28.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:28.908+0000 7fb6f4d69640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6f00d0460 0x7fb6f014cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:28.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:28.908+0000 7fb6f4d69640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f00b6380 con 0x7fb6f00d0460 2024-09-21T00:26:28.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:28.908+0000 7fb6f4d69640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f00b6200 con 0x7fb6f00b7b60 2024-09-21T00:26:28.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:28.908+0000 7fb6f4d69640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fb6f00b6500 con 0x7fb6f00cf200 2024-09-21T00:26:28.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:28.908+0000 7fb6eed9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6f00d0460 0x7fb6f014cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:28.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:28.908+0000 7fb6eed9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6f00d0460 0x7fb6f014cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59742/0 (socket says 172.21.15.112:59742) 2024-09-21T00:26:28.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:28.908+0000 7fb6eed9d640 1 -- 172.21.15.112:0/116088499 learned_addr learned my addr 172.21.15.112:0/116088499 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:26:28.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:28.908+0000 7fb6edd9b640 1 --2- 172.21.15.112:0/116088499 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb6f00cf200 0x7fb6f014a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:28.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:28.908+0000 7fb6ee59c640 1 --2- 172.21.15.112:0/116088499 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb6f00b7b60 0x7fb6f014d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:28.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:28.909+0000 7fb6eed9d640 1 -- 172.21.15.112:0/116088499 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb6f00cf200 msgr2=0x7fb6f014a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:28.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:28.909+0000 7fb6eed9d640 1 --2- 172.21.15.112:0/116088499 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb6f00cf200 0x7fb6f014a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:28.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:28.909+0000 7fb6eed9d640 1 -- 172.21.15.112:0/116088499 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb6f00b7b60 msgr2=0x7fb6f014d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:28.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:28.909+0000 7fb6eed9d640 1 --2- 172.21.15.112:0/116088499 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb6f00b7b60 0x7fb6f014d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:28.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:28.909+0000 7fb6eed9d640 1 -- 172.21.15.112:0/116088499 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6f0159460 con 0x7fb6f00d0460 2024-09-21T00:26:28.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:28.909+0000 7fb6eed9d640 1 --2- 172.21.15.112:0/116088499 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6f00d0460 0x7fb6f014cd70 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fb6e00058c0 tx=0x7fb6e000da70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:28.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:28.909+0000 7fb6e6ffd640 1 -- 172.21.15.112:0/116088499 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e0017070 con 0x7fb6f00d0460 2024-09-21T00:26:28.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:28.909+0000 7fb6e6ffd640 1 -- 172.21.15.112:0/116088499 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb6e0005310 con 0x7fb6f00d0460 2024-09-21T00:26:28.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:28.909+0000 7fb6f4d69640 1 -- 172.21.15.112:0/116088499 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6f0159740 con 0x7fb6f00d0460 2024-09-21T00:26:28.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:28.909+0000 7fb6e6ffd640 1 -- 172.21.15.112:0/116088499 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e00047a0 con 0x7fb6f00d0460 2024-09-21T00:26:28.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:28.909+0000 7fb6f4d69640 1 -- 172.21.15.112:0/116088499 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6f0159a20 con 0x7fb6f00d0460 2024-09-21T00:26:28.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:28.910+0000 7fb6f4d69640 1 -- 172.21.15.112:0/116088499 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6f00cf610 con 0x7fb6f00d0460 2024-09-21T00:26:28.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:28.911+0000 7fb6e6ffd640 1 -- 172.21.15.112:0/116088499 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb6e0004940 con 0x7fb6f00d0460 2024-09-21T00:26:28.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:28.911+0000 7fb6e6ffd640 1 --2- 172.21.15.112:0/116088499 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fb6d0078bc0 0x7fb6d007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:28.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:28.911+0000 7fb6e6ffd640 1 -- 172.21.15.112:0/116088499 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5853+0+0 (secure 0 0 0) 0x7fb6e009acd0 con 0x7fb6f00d0460 2024-09-21T00:26:28.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:28.911+0000 7fb6ee59c640 1 --2- 172.21.15.112:0/116088499 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fb6d0078bc0 0x7fb6d007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:28.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:28.912+0000 7fb6ee59c640 1 --2- 172.21.15.112:0/116088499 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fb6d0078bc0 0x7fb6d007b0d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb6e8005fd0 tx=0x7fb6e80095e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:28.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:28.916+0000 7fb6e6ffd640 1 -- 172.21.15.112:0/116088499 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb6e0062ac0 con 0x7fb6f00d0460 2024-09-21T00:26:29.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:29.101+0000 7fb6f4d69640 1 -- 172.21.15.112:0/116088499 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7fb6f00c8a00 con 0x7fb6f00d0460 2024-09-21T00:26:29.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:29.784+0000 7fb6e6ffd640 1 -- 172.21.15.112:0/116088499 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v30) ==== 178+0+0 (secure 0 0 0) 0x7fb6e00674f0 con 0x7fb6f00d0460 2024-09-21T00:26:29.785 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'cephfs_data_ec' removed 2024-09-21T00:26:29.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:29.787+0000 7fb6f4d69640 1 -- 172.21.15.112:0/116088499 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fb6d0078bc0 msgr2=0x7fb6d007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:29.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:29.787+0000 7fb6f4d69640 1 --2- 172.21.15.112:0/116088499 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fb6d0078bc0 0x7fb6d007b0d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb6e8005fd0 tx=0x7fb6e80095e0 comp rx=0 tx=0).stop 2024-09-21T00:26:29.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:29.787+0000 7fb6f4d69640 1 -- 172.21.15.112:0/116088499 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6f00d0460 msgr2=0x7fb6f014cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:29.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:29.787+0000 7fb6f4d69640 1 --2- 172.21.15.112:0/116088499 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6f00d0460 0x7fb6f014cd70 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fb6e00058c0 tx=0x7fb6e000da70 comp rx=0 tx=0).stop 2024-09-21T00:26:29.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:29.787+0000 7fb6f4d69640 1 -- 172.21.15.112:0/116088499 shutdown_connections 2024-09-21T00:26:29.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:29.787+0000 7fb6f4d69640 1 --2- 172.21.15.112:0/116088499 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fb6d0078bc0 0x7fb6d007b0d0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:29.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:29.787+0000 7fb6f4d69640 1 --2- 172.21.15.112:0/116088499 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6f00d0460 0x7fb6f014cd70 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:29.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:29.787+0000 7fb6f4d69640 1 --2- 172.21.15.112:0/116088499 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb6f00cf200 0x7fb6f014a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:29.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:29.787+0000 7fb6f4d69640 1 --2- 172.21.15.112:0/116088499 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb6f00b7b60 0x7fb6f014d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:29.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:29.787+0000 7fb6f4d69640 1 -- 172.21.15.112:0/116088499 >> 172.21.15.112:0/116088499 conn(0x7fb6f00d4510 msgr2=0x7fb6f00d0f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:29.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:29.787+0000 7fb6f4d69640 1 -- 172.21.15.112:0/116088499 shutdown_connections 2024-09-21T00:26:29.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:29.787+0000 7fb6f4d69640 1 -- 172.21.15.112:0/116088499 wait complete. 2024-09-21T00:26:29.803 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-21T00:26:29.804 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-21T00:26:29.804 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-21T00:26:29.804 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T00:26:29.804 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-21T00:26:29.804 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-21T00:26:29.804 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-21T00:26:29.804 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T00:26:29.805 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-21T00:26:29.805 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-21T00:26:29.805 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-21T00:26:29.805 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T00:26:29.805 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-21T00:26:29.805 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-21T00:26:29.805 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-21T00:26:29.805 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T00:26:29.814 INFO:tasks.ceph.mds.a:Stopped 2024-09-21T00:26:29.815 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-21T00:26:29.815 INFO:tasks.ceph.mds.c:Stopped 2024-09-21T00:26:29.815 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-21T00:26:29.816 INFO:tasks.ceph.mds.d:Stopped 2024-09-21T00:26:29.816 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-21T00:26:29.819 INFO:tasks.ceph.mds.b:Stopped 2024-09-21T00:26:29.819 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-21T00:26:29.857 INFO:tasks.ceph.mds.a:Started 2024-09-21T00:26:29.857 DEBUG:teuthology.parallel:result is None 2024-09-21T00:26:29.857 INFO:tasks.ceph.mds.c:Started 2024-09-21T00:26:29.858 DEBUG:teuthology.parallel:result is None 2024-09-21T00:26:29.861 INFO:tasks.ceph.mds.d:Started 2024-09-21T00:26:29.861 DEBUG:teuthology.parallel:result is None 2024-09-21T00:26:29.862 INFO:tasks.ceph.mds.b:Started 2024-09-21T00:26:29.862 DEBUG:teuthology.parallel:result is None 2024-09-21T00:26:29.863 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-21T00:26:29.946 INFO:tasks.ceph.mds.a.smithi112.stdout:starting mds.a at 2024-09-21T00:26:29.946 INFO:tasks.ceph.mds.c.smithi112.stdout:starting mds.c at 2024-09-21T00:26:29.952 INFO:tasks.ceph.mds.d.smithi187.stdout:starting mds.d at 2024-09-21T00:26:29.952 INFO:tasks.ceph.mds.b.smithi187.stdout:starting mds.b at 2024-09-21T00:26:29.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:29.984+0000 7fa188c8b640 1 -- 172.21.15.112:0/3382150839 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1840d1120 msgr2=0x7fa1840ceb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:29.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:29.984+0000 7fa188c8b640 1 --2- 172.21.15.112:0/3382150839 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1840d1120 0x7fa1840ceb90 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fa1840d2360 tx=0x7fa17c02a790 comp rx=0 tx=0).stop 2024-09-21T00:26:29.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:29.984+0000 7fa188c8b640 1 -- 172.21.15.112:0/3382150839 shutdown_connections 2024-09-21T00:26:29.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:29.984+0000 7fa188c8b640 1 --2- 172.21.15.112:0/3382150839 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1840d1120 0x7fa1840ceb90 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:29.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:29.984+0000 7fa188c8b640 1 --2- 172.21.15.112:0/3382150839 >> v2:172.21.15.187:3300/0 conn(0x7fa1840d0170 0x7fa1840d0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:29.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:29.984+0000 7fa188c8b640 1 -- 172.21.15.112:0/3382150839 >> 172.21.15.112:0/3382150839 conn(0x7fa18409d830 msgr2=0x7fa1840b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:29.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:29.984+0000 7fa188c8b640 1 -- 172.21.15.112:0/3382150839 shutdown_connections 2024-09-21T00:26:29.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:29.984+0000 7fa188c8b640 1 -- 172.21.15.112:0/3382150839 wait complete. 2024-09-21T00:26:29.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:29.986+0000 7fa188c8b640 1 Processor -- start 2024-09-21T00:26:29.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:29.986+0000 7fa188c8b640 1 -- start start 2024-09-21T00:26:29.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:29.987+0000 7fa188c8b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1840d0170 0x7fa18406eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:29.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:29.987+0000 7fa188c8b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa1840d1120 0x7fa1840b2db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:29.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:29.987+0000 7fa188c8b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa18415ee80 0x7fa1840b32f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:29.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:29.987+0000 7fa188c8b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa1840b6700 con 0x7fa1840d0170 2024-09-21T00:26:29.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:29.987+0000 7fa188c8b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa1840b6580 con 0x7fa18415ee80 2024-09-21T00:26:29.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:29.987+0000 7fa188c8b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa1840b6880 con 0x7fa1840d1120 2024-09-21T00:26:29.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:29.987+0000 7fa18259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1840d0170 0x7fa18406eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:29.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:29.987+0000 7fa18259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1840d0170 0x7fa18406eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59792/0 (socket says 172.21.15.112:59792) 2024-09-21T00:26:29.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:29.987+0000 7fa182d9d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa18415ee80 0x7fa1840b32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:29.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:29.987+0000 7fa18259c640 1 -- 172.21.15.112:0/1940436827 learned_addr learned my addr 172.21.15.112:0/1940436827 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:26:29.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:29.987+0000 7fa17bfff640 1 --2- 172.21.15.112:0/1940436827 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa1840d1120 0x7fa1840b2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:29.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:29.988+0000 7fa18259c640 1 -- 172.21.15.112:0/1940436827 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa1840d1120 msgr2=0x7fa1840b2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:29.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:29.988+0000 7fa18259c640 1 --2- 172.21.15.112:0/1940436827 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa1840d1120 0x7fa1840b2db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:29.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:29.988+0000 7fa18259c640 1 -- 172.21.15.112:0/1940436827 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa18415ee80 msgr2=0x7fa1840b32f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:29.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:29.988+0000 7fa18259c640 1 --2- 172.21.15.112:0/1940436827 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa18415ee80 0x7fa1840b32f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:29.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:29.988+0000 7fa18259c640 1 -- 172.21.15.112:0/1940436827 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1840cc300 con 0x7fa1840d0170 2024-09-21T00:26:29.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:29.988+0000 7fa18259c640 1 --2- 172.21.15.112:0/1940436827 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1840d0170 0x7fa18406eb00 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fa17400c3c0 tx=0x7fa17400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:29.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:29.988+0000 7fa17affd640 1 -- 172.21.15.112:0/1940436827 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa174017070 con 0x7fa1840d0170 2024-09-21T00:26:29.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:29.988+0000 7fa17affd640 1 -- 172.21.15.112:0/1940436827 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa1740035f0 con 0x7fa1840d0170 2024-09-21T00:26:29.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:29.988+0000 7fa188c8b640 1 -- 172.21.15.112:0/1940436827 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1840cc5e0 con 0x7fa1840d0170 2024-09-21T00:26:29.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:29.988+0000 7fa17affd640 1 -- 172.21.15.112:0/1940436827 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa174003a80 con 0x7fa1840d0170 2024-09-21T00:26:29.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:29.988+0000 7fa188c8b640 1 -- 172.21.15.112:0/1940436827 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1840ccba0 con 0x7fa1840d0170 2024-09-21T00:26:29.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:29.989+0000 7fa188c8b640 1 -- 172.21.15.112:0/1940436827 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1840b6da0 con 0x7fa1840d0170 2024-09-21T00:26:29.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:29.990+0000 7fa17affd640 1 -- 172.21.15.112:0/1940436827 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa17401a070 con 0x7fa1840d0170 2024-09-21T00:26:29.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:29.990+0000 7fa17affd640 1 --2- 172.21.15.112:0/1940436827 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fa164078b90 0x7fa16407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:29.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:29.990+0000 7fa17affd640 1 -- 172.21.15.112:0/1940436827 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5335+0+0 (secure 0 0 0) 0x7fa174099df0 con 0x7fa1840d0170 2024-09-21T00:26:29.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:29.991+0000 7fa17bfff640 1 --2- 172.21.15.112:0/1940436827 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fa164078b90 0x7fa16407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:29.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:29.991+0000 7fa17bfff640 1 --2- 172.21.15.112:0/1940436827 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fa164078b90 0x7fa16407b0a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa18406ea90 tx=0x7fa16c00b2e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:29.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:29.995+0000 7fa17affd640 1 -- 172.21.15.112:0/1940436827 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa1841592d0 con 0x7fa1840d0170 2024-09-21T00:26:30.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:30.189+0000 7fa188c8b640 1 -- 172.21.15.112:0/1940436827 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fa184158df0 con 0x7fa1840d0170 2024-09-21T00:26:30.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:30.795+0000 7fa17affd640 1 -- 172.21.15.112:0/1940436827 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v32) ==== 97+0+0 (secure 0 0 0) 0x7fa174061d50 con 0x7fa1840d0170 2024-09-21T00:26:30.797 INFO:teuthology.orchestra.run.smithi112.stderr: removed all blocklist entries 2024-09-21T00:26:30.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:30.797+0000 7fa188c8b640 1 -- 172.21.15.112:0/1940436827 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fa164078b90 msgr2=0x7fa16407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:30.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:30.797+0000 7fa188c8b640 1 --2- 172.21.15.112:0/1940436827 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fa164078b90 0x7fa16407b0a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa18406ea90 tx=0x7fa16c00b2e0 comp rx=0 tx=0).stop 2024-09-21T00:26:30.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:30.797+0000 7fa188c8b640 1 -- 172.21.15.112:0/1940436827 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1840d0170 msgr2=0x7fa18406eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:30.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:30.797+0000 7fa188c8b640 1 --2- 172.21.15.112:0/1940436827 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1840d0170 0x7fa18406eb00 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fa17400c3c0 tx=0x7fa17400c880 comp rx=0 tx=0).stop 2024-09-21T00:26:30.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:30.798+0000 7fa188c8b640 1 -- 172.21.15.112:0/1940436827 shutdown_connections 2024-09-21T00:26:30.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:30.798+0000 7fa188c8b640 1 --2- 172.21.15.112:0/1940436827 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa18415ee80 0x7fa1840b32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:30.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:30.798+0000 7fa188c8b640 1 --2- 172.21.15.112:0/1940436827 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa1840d1120 0x7fa1840b2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:30.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:30.798+0000 7fa188c8b640 1 --2- 172.21.15.112:0/1940436827 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fa164078b90 0x7fa16407b0a0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:30.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:30.798+0000 7fa188c8b640 1 --2- 172.21.15.112:0/1940436827 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1840d0170 0x7fa18406eb00 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:30.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:30.798+0000 7fa188c8b640 1 -- 172.21.15.112:0/1940436827 >> 172.21.15.112:0/1940436827 conn(0x7fa18409d830 msgr2=0x7fa1841618d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:30.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:30.798+0000 7fa188c8b640 1 -- 172.21.15.112:0/1940436827 shutdown_connections 2024-09-21T00:26:30.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:30.798+0000 7fa188c8b640 1 -- 172.21.15.112:0/1940436827 wait complete. 2024-09-21T00:26:30.811 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-21T00:26:30.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:30.922+0000 7f64dbcc5640 1 -- 172.21.15.112:0/3734736019 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64d40cbcd0 msgr2=0x7f64d406cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:30.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:30.922+0000 7f64dbcc5640 1 --2- 172.21.15.112:0/3734736019 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64d40cbcd0 0x7f64d406cde0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f64d40738b0 tx=0x7f64cc02a4d0 comp rx=0 tx=0).stop 2024-09-21T00:26:30.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:30.922+0000 7f64dbcc5640 1 -- 172.21.15.112:0/3734736019 shutdown_connections 2024-09-21T00:26:30.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:30.922+0000 7f64dbcc5640 1 --2- 172.21.15.112:0/3734736019 >> v2:172.21.15.187:3300/0 conn(0x7f64d4074a70 0x7f64d40cb790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:30.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:30.922+0000 7f64dbcc5640 1 --2- 172.21.15.112:0/3734736019 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64d40cbcd0 0x7f64d406cde0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:30.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:30.922+0000 7f64dbcc5640 1 --2- 172.21.15.112:0/3734736019 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f64d4070f10 0x7f64d4071630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:30.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:30.922+0000 7f64dbcc5640 1 -- 172.21.15.112:0/3734736019 >> 172.21.15.112:0/3734736019 conn(0x7f64d40d1140 msgr2=0x7f64d40d1550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:30.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:30.923+0000 7f64dbcc5640 1 -- 172.21.15.112:0/3734736019 shutdown_connections 2024-09-21T00:26:30.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:30.923+0000 7f64dbcc5640 1 -- 172.21.15.112:0/3734736019 wait complete. 2024-09-21T00:26:30.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:30.924+0000 7f64dbcc5640 1 Processor -- start 2024-09-21T00:26:30.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:30.924+0000 7f64dbcc5640 1 -- start start 2024-09-21T00:26:30.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:30.925+0000 7f64dbcc5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64d4070f10 0x7f64d41505d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:30.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:30.926+0000 7f64dbcc5640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f64d4074a70 0x7f64d40c9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:30.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:30.926+0000 7f64dbcc5640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f64d40cbcd0 0x7f64d406e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:30.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:30.926+0000 7f64dbcc5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f64d406d6b0 con 0x7f64d4070f10 2024-09-21T00:26:30.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:30.926+0000 7f64dbcc5640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f64d406d530 con 0x7f64d40cbcd0 2024-09-21T00:26:30.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:30.926+0000 7f64dbcc5640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f64d406d830 con 0x7f64d4074a70 2024-09-21T00:26:30.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:30.926+0000 7f64d9a61640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64d4070f10 0x7f64d41505d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:30.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:30.926+0000 7f64d9a61640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64d4070f10 0x7f64d41505d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59808/0 (socket says 172.21.15.112:59808) 2024-09-21T00:26:30.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:30.926+0000 7f64d9a61640 1 -- 172.21.15.112:0/1370593199 learned_addr learned my addr 172.21.15.112:0/1370593199 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:26:30.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:30.926+0000 7f64d9260640 1 --2- 172.21.15.112:0/1370593199 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f64d4074a70 0x7f64d40c9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:30.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:30.926+0000 7f64da262640 1 --2- 172.21.15.112:0/1370593199 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f64d40cbcd0 0x7f64d406e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:30.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:30.926+0000 7f64d9a61640 1 -- 172.21.15.112:0/1370593199 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f64d4074a70 msgr2=0x7f64d40c9fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:30.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:30.926+0000 7f64d9a61640 1 --2- 172.21.15.112:0/1370593199 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f64d4074a70 0x7f64d40c9fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:30.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:30.926+0000 7f64d9a61640 1 -- 172.21.15.112:0/1370593199 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f64d40cbcd0 msgr2=0x7f64d406e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:30.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:30.926+0000 7f64d9a61640 1 --2- 172.21.15.112:0/1370593199 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f64d40cbcd0 0x7f64d406e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:30.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:30.926+0000 7f64d9a61640 1 -- 172.21.15.112:0/1370593199 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64d40c8300 con 0x7f64d4070f10 2024-09-21T00:26:30.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:30.926+0000 7f64d9a61640 1 --2- 172.21.15.112:0/1370593199 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64d4070f10 0x7f64d41505d0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f64d000e990 tx=0x7f64d000ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:30.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:30.926+0000 7f64ca7fc640 1 -- 172.21.15.112:0/1370593199 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64d0004d20 con 0x7f64d4070f10 2024-09-21T00:26:30.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:30.926+0000 7f64ca7fc640 1 -- 172.21.15.112:0/1370593199 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f64d000cb00 con 0x7f64d4070f10 2024-09-21T00:26:30.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:30.926+0000 7f64ca7fc640 1 -- 172.21.15.112:0/1370593199 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64d0005020 con 0x7f64d4070f10 2024-09-21T00:26:30.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:30.926+0000 7f64dbcc5640 1 -- 172.21.15.112:0/1370593199 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64d40c85e0 con 0x7f64d4070f10 2024-09-21T00:26:30.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:30.926+0000 7f64dbcc5640 1 -- 172.21.15.112:0/1370593199 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64d40ca920 con 0x7f64d4070f10 2024-09-21T00:26:30.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:30.927+0000 7f64dbcc5640 1 -- 172.21.15.112:0/1370593199 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64d40cb790 con 0x7f64d4070f10 2024-09-21T00:26:30.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:30.928+0000 7f64ca7fc640 1 -- 172.21.15.112:0/1370593199 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f64d00051c0 con 0x7f64d4070f10 2024-09-21T00:26:30.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:30.928+0000 7f64ca7fc640 1 --2- 172.21.15.112:0/1370593199 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f64b8078b90 0x7f64b807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:30.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:30.929+0000 7f64ca7fc640 1 -- 172.21.15.112:0/1370593199 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7f64d009b430 con 0x7f64d4070f10 2024-09-21T00:26:30.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:30.929+0000 7f64d9260640 1 --2- 172.21.15.112:0/1370593199 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f64b8078b90 0x7f64b807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:30.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:30.929+0000 7f64d9260640 1 --2- 172.21.15.112:0/1370593199 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f64b8078b90 0x7f64b807b0a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f64c4005fd0 tx=0x7f64c4005850 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:30.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:30.933+0000 7f64ca7fc640 1 -- 172.21.15.112:0/1370593199 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f64d0016030 con 0x7f64d4070f10 2024-09-21T00:26:31.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:31.180+0000 7f64dbcc5640 1 -- 172.21.15.112:0/1370593199 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f64d40696c0 con 0x7f64d4070f10 2024-09-21T00:26:31.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:31.181+0000 7f64ca7fc640 1 -- 172.21.15.112:0/1370593199 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v8) ==== 80+0+6645 (secure 0 0 0) 0x7f64d0063ac0 con 0x7f64d4070f10 2024-09-21T00:26:31.183 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:26:31.183 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-21T00:26:31.183 INFO:teuthology.orchestra.run.smithi112.stdout: "auth_dump": [ 2024-09-21T00:26:31.183 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:26:31.183 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "mds.a", 2024-09-21T00:26:31.183 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBPEu5m3capAhAAczRM4L8ldX985NW4ep3w5Q==", 2024-09-21T00:26:31.183 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:26:31.183 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2024-09-21T00:26:31.183 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow *", 2024-09-21T00:26:31.184 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow *", 2024-09-21T00:26:31.184 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2024-09-21T00:26:31.184 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:26:31.184 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:26:31.184 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:26:31.184 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "mds.b", 2024-09-21T00:26:31.184 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBPEu5mxlDlDxAAtqngbTOhPqXDjFhJpSCLHg==", 2024-09-21T00:26:31.184 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:26:31.184 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2024-09-21T00:26:31.184 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow *", 2024-09-21T00:26:31.184 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow *", 2024-09-21T00:26:31.184 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2024-09-21T00:26:31.184 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:26:31.184 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:26:31.185 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:26:31.185 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "mds.c", 2024-09-21T00:26:31.185 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBPEu5mXNVRChAA1qsPbEybmY16jw5Jwk/v1A==", 2024-09-21T00:26:31.185 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:26:31.185 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2024-09-21T00:26:31.185 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow *", 2024-09-21T00:26:31.185 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow *", 2024-09-21T00:26:31.185 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2024-09-21T00:26:31.185 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:26:31.185 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:26:31.185 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:26:31.185 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "mds.d", 2024-09-21T00:26:31.185 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBPEu5mnzySFxAAX3VfTgGYGcDweI7rPf3xvw==", 2024-09-21T00:26:31.185 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:26:31.185 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2024-09-21T00:26:31.186 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow *", 2024-09-21T00:26:31.186 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow *", 2024-09-21T00:26:31.186 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2024-09-21T00:26:31.186 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:26:31.186 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:26:31.186 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:26:31.186 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.0", 2024-09-21T00:26:31.186 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBSEu5moQg4OBAAi8mzLx7tWnORlrL64/MWlw==", 2024-09-21T00:26:31.186 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:26:31.186 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2024-09-21T00:26:31.186 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2024-09-21T00:26:31.186 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2024-09-21T00:26:31.186 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:26:31.186 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:26:31.187 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:26:31.187 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.1", 2024-09-21T00:26:31.187 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBWEu5mxfobDhAAi7I6LYq9udB0W8bnYaBY9Q==", 2024-09-21T00:26:31.187 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:26:31.187 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2024-09-21T00:26:31.187 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2024-09-21T00:26:31.187 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2024-09-21T00:26:31.187 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:26:31.187 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:26:31.187 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:26:31.187 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.2", 2024-09-21T00:26:31.187 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBZEu5mDqC/EhAAwzTbj22ocR4xJ8DMJvN7Lg==", 2024-09-21T00:26:31.187 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:26:31.187 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2024-09-21T00:26:31.187 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2024-09-21T00:26:31.188 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2024-09-21T00:26:31.188 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:26:31.188 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:26:31.188 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:26:31.188 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.3", 2024-09-21T00:26:31.188 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBcEu5mwweGJBAAT3hZn5pto0aWt6BwGuaqcw==", 2024-09-21T00:26:31.188 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:26:31.188 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2024-09-21T00:26:31.188 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2024-09-21T00:26:31.188 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2024-09-21T00:26:31.188 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:26:31.188 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:26:31.188 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:26:31.188 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.4", 2024-09-21T00:26:31.188 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBjEu5mIPd3ARAAkxTYphn38uKh48UUVEB+Fg==", 2024-09-21T00:26:31.189 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:26:31.189 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2024-09-21T00:26:31.189 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2024-09-21T00:26:31.189 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2024-09-21T00:26:31.189 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:26:31.189 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:26:31.189 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:26:31.189 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.5", 2024-09-21T00:26:31.189 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBmEu5mP9IVBBAAftfK5V1K4vYRyFyJS2UzKQ==", 2024-09-21T00:26:31.189 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:26:31.189 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2024-09-21T00:26:31.189 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2024-09-21T00:26:31.189 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2024-09-21T00:26:31.189 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:26:31.189 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:26:31.190 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:26:31.190 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.6", 2024-09-21T00:26:31.190 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBpEu5mmjiyFRAAbiW2M8JE0jJOaNg1u+1kgw==", 2024-09-21T00:26:31.190 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:26:31.190 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2024-09-21T00:26:31.190 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2024-09-21T00:26:31.190 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2024-09-21T00:26:31.190 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:26:31.190 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:26:31.190 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:26:31.190 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.7", 2024-09-21T00:26:31.190 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBsEu5moklPKBAA6hzVvgvpmZMkPaFFce4mqw==", 2024-09-21T00:26:31.190 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:26:31.190 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2024-09-21T00:26:31.190 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2024-09-21T00:26:31.191 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2024-09-21T00:26:31.191 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:26:31.191 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:26:31.191 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:26:31.191 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "client.0", 2024-09-21T00:26:31.191 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBPEu5m49NPHBAAl/rcjiE8xYI+XdWwvZ6FOw==", 2024-09-21T00:26:31.191 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:26:31.191 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2024-09-21T00:26:31.191 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow r", 2024-09-21T00:26:31.191 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow rw", 2024-09-21T00:26:31.191 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow rwx" 2024-09-21T00:26:31.191 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:26:31.191 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:26:31.191 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:26:31.191 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "client.1", 2024-09-21T00:26:31.191 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBPEu5mG33aIxAAirQhTlVyF/pRJx93T/0bIQ==", 2024-09-21T00:26:31.192 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:26:31.192 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2024-09-21T00:26:31.192 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow r", 2024-09-21T00:26:31.192 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow rw", 2024-09-21T00:26:31.192 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow rwx" 2024-09-21T00:26:31.192 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:26:31.192 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:26:31.192 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:26:31.192 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "client.2", 2024-09-21T00:26:31.192 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBPEu5mreUUIBAAELeN1N58srw6GqBrj8UOoQ==", 2024-09-21T00:26:31.192 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:26:31.192 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2024-09-21T00:26:31.192 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow r", 2024-09-21T00:26:31.192 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow rw", 2024-09-21T00:26:31.192 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow rwx" 2024-09-21T00:26:31.192 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:26:31.193 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:26:31.193 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:26:31.193 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "client.3", 2024-09-21T00:26:31.193 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBPEu5mcTmiJxAAY6bmTtow0R0heJHjx0ahDQ==", 2024-09-21T00:26:31.193 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:26:31.193 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2024-09-21T00:26:31.193 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow r", 2024-09-21T00:26:31.193 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow rw", 2024-09-21T00:26:31.193 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow rwx" 2024-09-21T00:26:31.193 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:26:31.193 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:26:31.193 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:26:31.193 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "client.admin", 2024-09-21T00:26:31.193 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBOEu5mTxhNIBAAlStXGsGd/okXhakY6WI/4w==", 2024-09-21T00:26:31.193 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:26:31.194 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow *", 2024-09-21T00:26:31.194 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow *", 2024-09-21T00:26:31.194 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow *", 2024-09-21T00:26:31.194 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2024-09-21T00:26:31.194 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:26:31.194 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:26:31.194 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:26:31.194 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "client.bootstrap-mds", 2024-09-21T00:26:31.194 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQByEu5mTCwTLxAAkd49+JvXmjDnF4ZWTcginQ==", 2024-09-21T00:26:31.194 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:26:31.194 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile bootstrap-mds" 2024-09-21T00:26:31.194 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:26:31.194 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:26:31.194 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:26:31.194 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "client.bootstrap-mgr", 2024-09-21T00:26:31.195 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQByEu5mTzoTLxAAm7DHe3vYZ5jlBhEROJm4og==", 2024-09-21T00:26:31.195 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:26:31.195 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile bootstrap-mgr" 2024-09-21T00:26:31.195 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:26:31.195 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:26:31.195 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:26:31.195 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "client.bootstrap-osd", 2024-09-21T00:26:31.195 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQByEu5mP0cTLxAABYi4mf6yld9/uFfBLaTAEA==", 2024-09-21T00:26:31.195 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:26:31.195 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile bootstrap-osd" 2024-09-21T00:26:31.195 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:26:31.195 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:26:31.195 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:26:31.195 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "client.bootstrap-rbd", 2024-09-21T00:26:31.195 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQByEu5m+FMTLxAAXMxyR5xf7QLKQxhbcgVfyQ==", 2024-09-21T00:26:31.196 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:26:31.196 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile bootstrap-rbd" 2024-09-21T00:26:31.196 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:26:31.196 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:26:31.196 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:26:31.196 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-09-21T00:26:31.196 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQByEu5m5mITLxAAW5VWMD0exp5++18Ep1ldbQ==", 2024-09-21T00:26:31.196 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:26:31.196 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-09-21T00:26:31.196 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:26:31.196 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:26:31.196 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:26:31.196 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "client.bootstrap-rgw", 2024-09-21T00:26:31.196 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQByEu5mXHATLxAATuYnsa1H5JucXp7q3yly5A==", 2024-09-21T00:26:31.196 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:26:31.197 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile bootstrap-rgw" 2024-09-21T00:26:31.197 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:26:31.197 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:26:31.197 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:26:31.197 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "mgr.x", 2024-09-21T00:26:31.197 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBOEu5mjGpDOhAAJFudb2wZlOBjo8Ss65Q+eg==", 2024-09-21T00:26:31.197 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:26:31.197 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow *", 2024-09-21T00:26:31.197 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile mgr", 2024-09-21T00:26:31.197 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2024-09-21T00:26:31.197 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:26:31.197 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:26:31.197 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:26:31.197 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "mgr.y", 2024-09-21T00:26:31.197 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBOEu5myGjKNRAA9cMDObZYYOXMGdv2mOXkJA==", 2024-09-21T00:26:31.197 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:26:31.198 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow *", 2024-09-21T00:26:31.198 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile mgr", 2024-09-21T00:26:31.198 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2024-09-21T00:26:31.198 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:26:31.198 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:26:31.198 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-09-21T00:26:31.198 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-21T00:26:31.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:31.183+0000 7f64dbcc5640 1 -- 172.21.15.112:0/1370593199 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f64b8078b90 msgr2=0x7f64b807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:31.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:31.183+0000 7f64dbcc5640 1 --2- 172.21.15.112:0/1370593199 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f64b8078b90 0x7f64b807b0a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f64c4005fd0 tx=0x7f64c4005850 comp rx=0 tx=0).stop 2024-09-21T00:26:31.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:31.183+0000 7f64dbcc5640 1 -- 172.21.15.112:0/1370593199 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64d4070f10 msgr2=0x7f64d41505d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:31.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:31.183+0000 7f64dbcc5640 1 --2- 172.21.15.112:0/1370593199 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64d4070f10 0x7f64d41505d0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f64d000e990 tx=0x7f64d000ee50 comp rx=0 tx=0).stop 2024-09-21T00:26:31.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:31.183+0000 7f64dbcc5640 1 -- 172.21.15.112:0/1370593199 shutdown_connections 2024-09-21T00:26:31.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:31.183+0000 7f64dbcc5640 1 --2- 172.21.15.112:0/1370593199 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f64d40cbcd0 0x7f64d406e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:31.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:31.183+0000 7f64dbcc5640 1 --2- 172.21.15.112:0/1370593199 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f64d4074a70 0x7f64d40c9fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:31.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:31.183+0000 7f64dbcc5640 1 --2- 172.21.15.112:0/1370593199 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f64b8078b90 0x7f64b807b0a0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:31.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:31.183+0000 7f64dbcc5640 1 --2- 172.21.15.112:0/1370593199 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64d4070f10 0x7f64d41505d0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:31.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:31.183+0000 7f64dbcc5640 1 -- 172.21.15.112:0/1370593199 >> 172.21.15.112:0/1370593199 conn(0x7f64d40d1140 msgr2=0x7f64d4159cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:31.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:31.183+0000 7f64dbcc5640 1 -- 172.21.15.112:0/1370593199 shutdown_connections 2024-09-21T00:26:31.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:31.183+0000 7f64dbcc5640 1 -- 172.21.15.112:0/1370593199 wait complete. 2024-09-21T00:26:31.200 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-09-21T00:26:31.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:31.312+0000 7f04df9c6640 1 -- 172.21.15.112:0/3886045468 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04c4004d40 msgr2=0x7f04c4005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:31.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:31.312+0000 7f04df9c6640 1 --2- 172.21.15.112:0/3886045468 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04c4004d40 0x7f04c4005150 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f04d0009ed0 tx=0x7f04c4005830 comp rx=0 tx=0).stop 2024-09-21T00:26:31.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:31.312+0000 7f04ddf63640 1 -- 172.21.15.112:0/3886045468 reap_dead start 2024-09-21T00:26:31.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:31.312+0000 7f04df9c6640 1 -- 172.21.15.112:0/3886045468 shutdown_connections 2024-09-21T00:26:31.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:31.312+0000 7f04df9c6640 1 -- 172.21.15.112:0/3886045468 >> 172.21.15.112:0/3886045468 conn(0x7f04d809d830 msgr2=0x7f04d80b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:31.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:31.313+0000 7f04df9c6640 1 -- 172.21.15.112:0/3886045468 shutdown_connections 2024-09-21T00:26:31.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:31.313+0000 7f04df9c6640 1 -- 172.21.15.112:0/3886045468 wait complete. 2024-09-21T00:26:31.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:31.314+0000 7f04df9c6640 1 Processor -- start 2024-09-21T00:26:31.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:31.314+0000 7f04df9c6640 1 -- start start 2024-09-21T00:26:31.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:31.316+0000 7f04df9c6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04c4004d40 0x7f04d80b23c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:31.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:31.316+0000 7f04df9c6640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04d80b2900 0x7f04d809e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:31.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:31.316+0000 7f04df9c6640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04d809ed40 0x7f04d80a18f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:31.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:31.316+0000 7f04df9c6640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f04d80cefa0 con 0x7f04c4004d40 2024-09-21T00:26:31.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:31.316+0000 7f04df9c6640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f04d80ba0f0 con 0x7f04d809ed40 2024-09-21T00:26:31.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:31.316+0000 7f04df9c6640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f04d80cf120 con 0x7f04d80b2900 2024-09-21T00:26:31.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:31.316+0000 7f04dd762640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04c4004d40 0x7f04d80b23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:31.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:31.316+0000 7f04dd762640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04c4004d40 0x7f04d80b23c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59824/0 (socket says 172.21.15.112:59824) 2024-09-21T00:26:31.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:31.316+0000 7f04dd762640 1 -- 172.21.15.112:0/3486335412 learned_addr learned my addr 172.21.15.112:0/3486335412 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:26:31.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:31.316+0000 7f04dcf61640 1 --2- 172.21.15.112:0/3486335412 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04d80b2900 0x7f04d809e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:31.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:31.316+0000 7f04ddf63640 1 --2- 172.21.15.112:0/3486335412 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04d809ed40 0x7f04d80a18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:31.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:31.316+0000 7f04dd762640 1 -- 172.21.15.112:0/3486335412 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04d80b2900 msgr2=0x7f04d809e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:31.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:31.316+0000 7f04dd762640 1 --2- 172.21.15.112:0/3486335412 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04d80b2900 0x7f04d809e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:31.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:31.316+0000 7f04dd762640 1 -- 172.21.15.112:0/3486335412 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04d809ed40 msgr2=0x7f04d80a18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:31.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:31.316+0000 7f04dd762640 1 --2- 172.21.15.112:0/3486335412 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04d809ed40 0x7f04d80a18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:31.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:31.316+0000 7f04dd762640 1 -- 172.21.15.112:0/3486335412 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04d80a0b70 con 0x7f04c4004d40 2024-09-21T00:26:31.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:31.316+0000 7f04dcf61640 1 --2- 172.21.15.112:0/3486335412 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04d80b2900 0x7f04d809e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:26:31.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:31.316+0000 7f04dd762640 1 --2- 172.21.15.112:0/3486335412 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04c4004d40 0x7f04d80b23c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f04d4011ae0 tx=0x7f04d4011ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:31.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:31.316+0000 7f04cdffb640 1 -- 172.21.15.112:0/3486335412 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d400fd30 con 0x7f04c4004d40 2024-09-21T00:26:31.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:31.316+0000 7f04cdffb640 1 -- 172.21.15.112:0/3486335412 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f04d4019410 con 0x7f04c4004d40 2024-09-21T00:26:31.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:31.316+0000 7f04df9c6640 1 -- 172.21.15.112:0/3486335412 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04d809f430 con 0x7f04c4004d40 2024-09-21T00:26:31.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:31.316+0000 7f04cdffb640 1 -- 172.21.15.112:0/3486335412 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d4019860 con 0x7f04c4004d40 2024-09-21T00:26:31.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:31.317+0000 7f04df9c6640 1 -- 172.21.15.112:0/3486335412 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04d80a10f0 con 0x7f04c4004d40 2024-09-21T00:26:31.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:31.317+0000 7f04df9c6640 1 -- 172.21.15.112:0/3486335412 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04d80d0170 con 0x7f04c4004d40 2024-09-21T00:26:31.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:31.318+0000 7f04cdffb640 1 -- 172.21.15.112:0/3486335412 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f04d4019a00 con 0x7f04c4004d40 2024-09-21T00:26:31.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:31.319+0000 7f04cdffb640 1 --2- 172.21.15.112:0/3486335412 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f04bc078bc0 0x7f04bc07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:31.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:31.319+0000 7f04cdffb640 1 -- 172.21.15.112:0/3486335412 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7f04d409b7b0 con 0x7f04c4004d40 2024-09-21T00:26:31.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:31.319+0000 7f04dcf61640 1 --2- 172.21.15.112:0/3486335412 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f04bc078bc0 0x7f04bc07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:31.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:31.320+0000 7f04dcf61640 1 --2- 172.21.15.112:0/3486335412 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f04bc078bc0 0x7f04bc07b0d0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f04d80cc940 tx=0x7f04c8006910 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:31.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:31.323+0000 7f04cdffb640 1 -- 172.21.15.112:0/3486335412 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f04d4063db0 con 0x7f04c4004d40 2024-09-21T00:26:31.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:31.572+0000 7f04df9c6640 1 -- 172.21.15.112:0/3486335412 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) -- 0x7f04d80d1800 con 0x7f04c4004d40 2024-09-21T00:26:31.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:31.574+0000 7f04cdffb640 1 -- 172.21.15.112:0/3486335412 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v9) ==== 90+0+0 (secure 0 0 0) 0x7f04d40636b0 con 0x7f04c4004d40 2024-09-21T00:26:31.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:31.577+0000 7f04df9c6640 1 -- 172.21.15.112:0/3486335412 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f04bc078bc0 msgr2=0x7f04bc07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:31.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:31.577+0000 7f04df9c6640 1 --2- 172.21.15.112:0/3486335412 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f04bc078bc0 0x7f04bc07b0d0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f04d80cc940 tx=0x7f04c8006910 comp rx=0 tx=0).stop 2024-09-21T00:26:31.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:31.577+0000 7f04df9c6640 1 -- 172.21.15.112:0/3486335412 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04c4004d40 msgr2=0x7f04d80b23c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:31.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:31.577+0000 7f04df9c6640 1 --2- 172.21.15.112:0/3486335412 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04c4004d40 0x7f04d80b23c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f04d4011ae0 tx=0x7f04d4011ee0 comp rx=0 tx=0).stop 2024-09-21T00:26:31.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:31.577+0000 7f04df9c6640 1 -- 172.21.15.112:0/3486335412 shutdown_connections 2024-09-21T00:26:31.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:31.577+0000 7f04df9c6640 1 --2- 172.21.15.112:0/3486335412 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04d809ed40 0x7f04d80a18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:31.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:31.577+0000 7f04df9c6640 1 --2- 172.21.15.112:0/3486335412 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04d80b2900 0x7f04d809e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:31.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:31.577+0000 7f04df9c6640 1 --2- 172.21.15.112:0/3486335412 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f04bc078bc0 0x7f04bc07b0d0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:31.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:31.577+0000 7f04df9c6640 1 --2- 172.21.15.112:0/3486335412 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04c4004d40 0x7f04d80b23c0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:31.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:31.577+0000 7f04df9c6640 1 -- 172.21.15.112:0/3486335412 >> 172.21.15.112:0/3486335412 conn(0x7f04d809d830 msgr2=0x7f04d806edf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:31.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:31.577+0000 7f04df9c6640 1 -- 172.21.15.112:0/3486335412 shutdown_connections 2024-09-21T00:26:31.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:31.577+0000 7f04df9c6640 1 -- 172.21.15.112:0/3486335412 wait complete. 2024-09-21T00:26:31.590 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-09-21T00:26:31.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:31.701+0000 7f97b0b07640 1 -- 172.21.15.112:0/3724700825 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97ac0ccc40 msgr2=0x7f97ac15ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:31.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:31.701+0000 7f97b0b07640 1 --2- 172.21.15.112:0/3724700825 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97ac0ccc40 0x7f97ac15ca90 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f97ac0d1940 tx=0x7f979c02a4a0 comp rx=0 tx=0).stop 2024-09-21T00:26:31.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:31.701+0000 7f97b0b07640 1 -- 172.21.15.112:0/3724700825 shutdown_connections 2024-09-21T00:26:31.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:31.701+0000 7f97b0b07640 1 --2- 172.21.15.112:0/3724700825 >> v2:172.21.15.187:3300/0 conn(0x7f97ac0cd440 0x7f97ac0d0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:31.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:31.701+0000 7f97b0b07640 1 --2- 172.21.15.112:0/3724700825 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97ac0ccc40 0x7f97ac15ca90 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:31.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:31.701+0000 7f97b0b07640 1 -- 172.21.15.112:0/3724700825 >> 172.21.15.112:0/3724700825 conn(0x7f97ac0b46e0 msgr2=0x7f97ac0d23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:31.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:31.701+0000 7f97b0b07640 1 -- 172.21.15.112:0/3724700825 shutdown_connections 2024-09-21T00:26:31.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:31.701+0000 7f97b0b07640 1 -- 172.21.15.112:0/3724700825 wait complete. 2024-09-21T00:26:31.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:31.703+0000 7f97b0b07640 1 Processor -- start 2024-09-21T00:26:31.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:31.703+0000 7f97b0b07640 1 -- start start 2024-09-21T00:26:31.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:31.705+0000 7f97b0b07640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f97ac0ccc40 0x7f97ac1f3de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:31.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:31.705+0000 7f97b0b07640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97ac0cd440 0x7f97ac1f2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:31.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:31.705+0000 7f97b0b07640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f97ac0d0bb0 0x7f97ac0cb0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:31.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:31.705+0000 7f97b0b07640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f97ac070810 con 0x7f97ac0cd440 2024-09-21T00:26:31.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:31.705+0000 7f97b0b07640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f97ac0b6e00 con 0x7f97ac0ccc40 2024-09-21T00:26:31.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:31.705+0000 7f97b0b07640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f97ac070990 con 0x7f97ac0d0bb0 2024-09-21T00:26:31.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:31.705+0000 7f97a9d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97ac0cd440 0x7f97ac1f2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:31.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:31.705+0000 7f97a9d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97ac0cd440 0x7f97ac1f2bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59844/0 (socket says 172.21.15.112:59844) 2024-09-21T00:26:31.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:31.705+0000 7f97a9d9b640 1 -- 172.21.15.112:0/4148529923 learned_addr learned my addr 172.21.15.112:0/4148529923 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:26:31.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:31.705+0000 7f97a9d9b640 1 -- 172.21.15.112:0/4148529923 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f97ac0d0bb0 msgr2=0x7f97ac0cb0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:31.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:31.705+0000 7f97a9d9b640 1 --2- 172.21.15.112:0/4148529923 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f97ac0d0bb0 0x7f97ac0cb0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:31.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:31.705+0000 7f97a9d9b640 1 -- 172.21.15.112:0/4148529923 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f97ac0ccc40 msgr2=0x7f97ac1f3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:31.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:31.705+0000 7f97aa59c640 1 --2- 172.21.15.112:0/4148529923 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f97ac0ccc40 0x7f97ac1f3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:31.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:31.705+0000 7f97a9d9b640 1 --2- 172.21.15.112:0/4148529923 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f97ac0ccc40 0x7f97ac1f3de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:31.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:31.705+0000 7f97a9d9b640 1 -- 172.21.15.112:0/4148529923 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97ac1f47e0 con 0x7f97ac0cd440 2024-09-21T00:26:31.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:31.705+0000 7f97a9d9b640 1 --2- 172.21.15.112:0/4148529923 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97ac0cd440 0x7f97ac1f2bd0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f979400d6d0 tx=0x7f979400db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:31.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:31.705+0000 7f97a2ffd640 1 -- 172.21.15.112:0/4148529923 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9794017070 con 0x7f97ac0cd440 2024-09-21T00:26:31.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:31.705+0000 7f97a2ffd640 1 -- 172.21.15.112:0/4148529923 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9794004310 con 0x7f97ac0cd440 2024-09-21T00:26:31.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:31.705+0000 7f97b0b07640 1 -- 172.21.15.112:0/4148529923 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97ac1f4ac0 con 0x7f97ac0cd440 2024-09-21T00:26:31.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:31.705+0000 7f97a2ffd640 1 -- 172.21.15.112:0/4148529923 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9794009a80 con 0x7f97ac0cd440 2024-09-21T00:26:31.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:31.705+0000 7f97b0b07640 1 -- 172.21.15.112:0/4148529923 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97ac1f5080 con 0x7f97ac0cd440 2024-09-21T00:26:31.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:31.706+0000 7f97b0b07640 1 -- 172.21.15.112:0/4148529923 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97ac15ca90 con 0x7f97ac0cd440 2024-09-21T00:26:31.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:31.707+0000 7f97a2ffd640 1 -- 172.21.15.112:0/4148529923 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9794009c20 con 0x7f97ac0cd440 2024-09-21T00:26:31.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:31.708+0000 7f97a2ffd640 1 --2- 172.21.15.112:0/4148529923 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f978c078bc0 0x7f978c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:31.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:31.708+0000 7f97a2ffd640 1 -- 172.21.15.112:0/4148529923 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7f979409ab80 con 0x7f97ac0cd440 2024-09-21T00:26:31.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:31.708+0000 7f97aa59c640 1 --2- 172.21.15.112:0/4148529923 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f978c078bc0 0x7f978c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:31.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:31.709+0000 7f97aa59c640 1 --2- 172.21.15.112:0/4148529923 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f978c078bc0 0x7f978c07b0d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f979c034000 tx=0x7f979c004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:31.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:31.712+0000 7f97a2ffd640 1 -- 172.21.15.112:0/4148529923 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f9794063210 con 0x7f97ac0cd440 2024-09-21T00:26:31.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:31.960+0000 7f97b0b07640 1 -- 172.21.15.112:0/4148529923 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) -- 0x7f97ac0b7b30 con 0x7f97ac0cd440 2024-09-21T00:26:31.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:31.962+0000 7f97a2ffd640 1 -- 172.21.15.112:0/4148529923 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v10) ==== 90+0+0 (secure 0 0 0) 0x7f9794062b10 con 0x7f97ac0cd440 2024-09-21T00:26:31.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:31.965+0000 7f97b0b07640 1 -- 172.21.15.112:0/4148529923 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f978c078bc0 msgr2=0x7f978c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:31.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:31.965+0000 7f97b0b07640 1 --2- 172.21.15.112:0/4148529923 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f978c078bc0 0x7f978c07b0d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f979c034000 tx=0x7f979c004730 comp rx=0 tx=0).stop 2024-09-21T00:26:31.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:31.965+0000 7f97b0b07640 1 -- 172.21.15.112:0/4148529923 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97ac0cd440 msgr2=0x7f97ac1f2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:31.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:31.965+0000 7f97b0b07640 1 --2- 172.21.15.112:0/4148529923 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97ac0cd440 0x7f97ac1f2bd0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f979400d6d0 tx=0x7f979400db90 comp rx=0 tx=0).stop 2024-09-21T00:26:31.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:31.965+0000 7f97b0b07640 1 -- 172.21.15.112:0/4148529923 shutdown_connections 2024-09-21T00:26:31.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:31.965+0000 7f97b0b07640 1 --2- 172.21.15.112:0/4148529923 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f97ac0d0bb0 0x7f97ac0cb0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:31.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:31.965+0000 7f97b0b07640 1 --2- 172.21.15.112:0/4148529923 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f978c078bc0 0x7f978c07b0d0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:31.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:31.965+0000 7f97b0b07640 1 --2- 172.21.15.112:0/4148529923 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97ac0cd440 0x7f97ac1f2bd0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:31.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:31.965+0000 7f97b0b07640 1 --2- 172.21.15.112:0/4148529923 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f97ac0ccc40 0x7f97ac1f3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:31.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:31.965+0000 7f97b0b07640 1 -- 172.21.15.112:0/4148529923 >> 172.21.15.112:0/4148529923 conn(0x7f97ac0b46e0 msgr2=0x7f97ac0b6f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:31.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:31.965+0000 7f97b0b07640 1 -- 172.21.15.112:0/4148529923 shutdown_connections 2024-09-21T00:26:31.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:31.965+0000 7f97b0b07640 1 -- 172.21.15.112:0/4148529923 wait complete. 2024-09-21T00:26:31.978 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-09-21T00:26:32.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.088+0000 7fbf58d20640 1 -- 172.21.15.112:0/3122473829 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf540d0bb0 msgr2=0x7fbf540b5be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:32.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.088+0000 7fbf58d20640 1 --2- 172.21.15.112:0/3122473829 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf540d0bb0 0x7fbf540b5be0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c005fd0 tx=0x7fbf4c02a4a0 comp rx=0 tx=0).stop 2024-09-21T00:26:32.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.088+0000 7fbf58d20640 1 -- 172.21.15.112:0/3122473829 shutdown_connections 2024-09-21T00:26:32.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.088+0000 7fbf58d20640 1 --2- 172.21.15.112:0/3122473829 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf540d0bb0 0x7fbf540b5be0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:32.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.088+0000 7fbf58d20640 1 --2- 172.21.15.112:0/3122473829 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbf5415ca90 0x7fbf540d0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:32.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.088+0000 7fbf58d20640 1 -- 172.21.15.112:0/3122473829 >> 172.21.15.112:0/3122473829 conn(0x7fbf540b46e0 msgr2=0x7fbf540d23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:32.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.088+0000 7fbf58d20640 1 -- 172.21.15.112:0/3122473829 shutdown_connections 2024-09-21T00:26:32.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.088+0000 7fbf58d20640 1 -- 172.21.15.112:0/3122473829 wait complete. 2024-09-21T00:26:32.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.090+0000 7fbf58d20640 1 Processor -- start 2024-09-21T00:26:32.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.090+0000 7fbf58d20640 1 -- start start 2024-09-21T00:26:32.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.091+0000 7fbf58d20640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbf540ccc40 0x7fbf54149380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:32.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.091+0000 7fbf58d20640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf540d0bb0 0x7fbf541f6770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:32.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.091+0000 7fbf58d20640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbf5415ca90 0x7fbf541e6780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:32.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.091+0000 7fbf58d20640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbf540b6620 con 0x7fbf540d0bb0 2024-09-21T00:26:32.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.091+0000 7fbf58d20640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fbf540b64a0 con 0x7fbf540ccc40 2024-09-21T00:26:32.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.091+0000 7fbf58d20640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fbf540b67a0 con 0x7fbf5415ca90 2024-09-21T00:26:32.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.092+0000 7fbf51d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf540d0bb0 0x7fbf541f6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:32.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.092+0000 7fbf51d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf540d0bb0 0x7fbf541f6770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59860/0 (socket says 172.21.15.112:59860) 2024-09-21T00:26:32.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.092+0000 7fbf51d9b640 1 -- 172.21.15.112:0/2353413250 learned_addr learned my addr 172.21.15.112:0/2353413250 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:26:32.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.092+0000 7fbf5259c640 1 --2- 172.21.15.112:0/2353413250 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbf540ccc40 0x7fbf54149380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:32.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.092+0000 7fbf52d9d640 1 --2- 172.21.15.112:0/2353413250 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbf5415ca90 0x7fbf541e6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:32.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.092+0000 7fbf51d9b640 1 -- 172.21.15.112:0/2353413250 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbf5415ca90 msgr2=0x7fbf541e6780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:32.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.092+0000 7fbf51d9b640 1 --2- 172.21.15.112:0/2353413250 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbf5415ca90 0x7fbf541e6780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:32.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.092+0000 7fbf51d9b640 1 -- 172.21.15.112:0/2353413250 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbf540ccc40 msgr2=0x7fbf54149380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:32.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.092+0000 7fbf51d9b640 1 --2- 172.21.15.112:0/2353413250 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbf540ccc40 0x7fbf54149380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:32.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.092+0000 7fbf51d9b640 1 -- 172.21.15.112:0/2353413250 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf541f73f0 con 0x7fbf540d0bb0 2024-09-21T00:26:32.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.092+0000 7fbf51d9b640 1 --2- 172.21.15.112:0/2353413250 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf540d0bb0 0x7fbf541f6770 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c00d750 tx=0x7fbf3c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:32.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.092+0000 7fbf4affd640 1 -- 172.21.15.112:0/2353413250 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf3c017070 con 0x7fbf540d0bb0 2024-09-21T00:26:32.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.092+0000 7fbf4affd640 1 -- 172.21.15.112:0/2353413250 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbf3c009b20 con 0x7fbf540d0bb0 2024-09-21T00:26:32.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.092+0000 7fbf58d20640 1 -- 172.21.15.112:0/2353413250 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf541e1e50 con 0x7fbf540d0bb0 2024-09-21T00:26:32.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.092+0000 7fbf4affd640 1 -- 172.21.15.112:0/2353413250 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf3c010460 con 0x7fbf540d0bb0 2024-09-21T00:26:32.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.092+0000 7fbf58d20640 1 -- 172.21.15.112:0/2353413250 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf541e2410 con 0x7fbf540d0bb0 2024-09-21T00:26:32.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.093+0000 7fbf58d20640 1 -- 172.21.15.112:0/2353413250 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf541f7160 con 0x7fbf540d0bb0 2024-09-21T00:26:32.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.094+0000 7fbf4affd640 1 -- 172.21.15.112:0/2353413250 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fbf3c01a070 con 0x7fbf540d0bb0 2024-09-21T00:26:32.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.095+0000 7fbf4affd640 1 --2- 172.21.15.112:0/2353413250 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fbf34078bc0 0x7fbf3407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:32.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.095+0000 7fbf4affd640 1 -- 172.21.15.112:0/2353413250 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7fbf3c09ac90 con 0x7fbf540d0bb0 2024-09-21T00:26:32.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.095+0000 7fbf5259c640 1 --2- 172.21.15.112:0/2353413250 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fbf34078bc0 0x7fbf3407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:32.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.096+0000 7fbf5259c640 1 --2- 172.21.15.112:0/2353413250 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fbf34078bc0 0x7fbf3407b0d0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fbf44009f20 tx=0x7fbf440094d0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:32.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.099+0000 7fbf4affd640 1 -- 172.21.15.112:0/2353413250 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fbf3c063320 con 0x7fbf540d0bb0 2024-09-21T00:26:32.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.344+0000 7fbf58d20640 1 -- 172.21.15.112:0/2353413250 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) -- 0x7fbf540b6d30 con 0x7fbf540d0bb0 2024-09-21T00:26:32.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.346+0000 7fbf4affd640 1 -- 172.21.15.112:0/2353413250 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v11) ==== 90+0+0 (secure 0 0 0) 0x7fbf3c062c20 con 0x7fbf540d0bb0 2024-09-21T00:26:32.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.348+0000 7fbf58d20640 1 -- 172.21.15.112:0/2353413250 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fbf34078bc0 msgr2=0x7fbf3407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:32.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.348+0000 7fbf58d20640 1 --2- 172.21.15.112:0/2353413250 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fbf34078bc0 0x7fbf3407b0d0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fbf44009f20 tx=0x7fbf440094d0 comp rx=0 tx=0).stop 2024-09-21T00:26:32.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.348+0000 7fbf58d20640 1 -- 172.21.15.112:0/2353413250 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf540d0bb0 msgr2=0x7fbf541f6770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:32.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.348+0000 7fbf58d20640 1 --2- 172.21.15.112:0/2353413250 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf540d0bb0 0x7fbf541f6770 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c00d750 tx=0x7fbf3c00dc10 comp rx=0 tx=0).stop 2024-09-21T00:26:32.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.348+0000 7fbf58d20640 1 -- 172.21.15.112:0/2353413250 shutdown_connections 2024-09-21T00:26:32.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.348+0000 7fbf58d20640 1 --2- 172.21.15.112:0/2353413250 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbf5415ca90 0x7fbf541e6780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:32.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.348+0000 7fbf58d20640 1 --2- 172.21.15.112:0/2353413250 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fbf34078bc0 0x7fbf3407b0d0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:32.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.348+0000 7fbf58d20640 1 --2- 172.21.15.112:0/2353413250 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf540d0bb0 0x7fbf541f6770 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:32.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.348+0000 7fbf58d20640 1 --2- 172.21.15.112:0/2353413250 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbf540ccc40 0x7fbf54149380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:32.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.348+0000 7fbf58d20640 1 -- 172.21.15.112:0/2353413250 >> 172.21.15.112:0/2353413250 conn(0x7fbf540b46e0 msgr2=0x7fbf54070100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:32.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.348+0000 7fbf58d20640 1 -- 172.21.15.112:0/2353413250 shutdown_connections 2024-09-21T00:26:32.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.348+0000 7fbf58d20640 1 -- 172.21.15.112:0/2353413250 wait complete. 2024-09-21T00:26:32.361 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-09-21T00:26:32.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.473+0000 7f20a364d640 1 -- 172.21.15.112:0/84228233 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f209c15ee80 msgr2=0x7f209c0d0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:32.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.473+0000 7f20a364d640 1 --2- 172.21.15.112:0/84228233 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f209c15ee80 0x7f209c0d0170 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f208c005fd0 tx=0x7f208c02a4a0 comp rx=0 tx=0).stop 2024-09-21T00:26:32.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.473+0000 7f20a364d640 1 -- 172.21.15.112:0/84228233 shutdown_connections 2024-09-21T00:26:32.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.473+0000 7f20a364d640 1 --2- 172.21.15.112:0/84228233 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f209c0b9ad0 0x7f209c0d0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:32.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.473+0000 7f20a364d640 1 --2- 172.21.15.112:0/84228233 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f209c15ee80 0x7f209c0d0170 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:32.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.473+0000 7f20a364d640 1 -- 172.21.15.112:0/84228233 >> 172.21.15.112:0/84228233 conn(0x7f209c09d830 msgr2=0x7f209c0b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:32.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.473+0000 7f20a364d640 1 -- 172.21.15.112:0/84228233 shutdown_connections 2024-09-21T00:26:32.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.473+0000 7f20a364d640 1 -- 172.21.15.112:0/84228233 wait complete. 2024-09-21T00:26:32.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.475+0000 7f20a364d640 1 Processor -- start 2024-09-21T00:26:32.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.475+0000 7f20a364d640 1 -- start start 2024-09-21T00:26:32.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.476+0000 7f20a364d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f209c0b9ad0 0x7f209c1f85f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:32.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.476+0000 7f20a364d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f209c0d1120 0x7f209c14d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:32.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.476+0000 7f20a364d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f209c15ee80 0x7f209c06f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:32.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.476+0000 7f20a364d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f209c0b6700 con 0x7f209c15ee80 2024-09-21T00:26:32.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.476+0000 7f20a364d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f209c0b6580 con 0x7f209c0b9ad0 2024-09-21T00:26:32.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.476+0000 7f20a364d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f209c0b6880 con 0x7f209c0d1120 2024-09-21T00:26:32.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.476+0000 7f20a1bea640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f209c15ee80 0x7f209c06f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:32.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.476+0000 7f20a1bea640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f209c15ee80 0x7f209c06f1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59878/0 (socket says 172.21.15.112:59878) 2024-09-21T00:26:32.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.476+0000 7f20a1bea640 1 -- 172.21.15.112:0/3388364365 learned_addr learned my addr 172.21.15.112:0/3388364365 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:26:32.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.476+0000 7f20a13e9640 1 --2- 172.21.15.112:0/3388364365 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f209c0b9ad0 0x7f209c1f85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:32.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.476+0000 7f20a0be8640 1 --2- 172.21.15.112:0/3388364365 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f209c0d1120 0x7f209c14d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:32.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.477+0000 7f20a1bea640 1 -- 172.21.15.112:0/3388364365 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f209c0d1120 msgr2=0x7f209c14d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:32.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.477+0000 7f20a1bea640 1 --2- 172.21.15.112:0/3388364365 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f209c0d1120 0x7f209c14d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:32.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.477+0000 7f20a1bea640 1 -- 172.21.15.112:0/3388364365 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f209c0b9ad0 msgr2=0x7f209c1f85f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:32.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.477+0000 7f20a1bea640 1 --2- 172.21.15.112:0/3388364365 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f209c0b9ad0 0x7f209c1f85f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:32.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.477+0000 7f20a1bea640 1 -- 172.21.15.112:0/3388364365 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f209c1f8b30 con 0x7f209c15ee80 2024-09-21T00:26:32.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.477+0000 7f20a1bea640 1 --2- 172.21.15.112:0/3388364365 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f209c15ee80 0x7f209c06f1f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f209400b9e0 tx=0x7f209400bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:32.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.477+0000 7f2099ffb640 1 -- 172.21.15.112:0/3388364365 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2094003b70 con 0x7f209c15ee80 2024-09-21T00:26:32.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.477+0000 7f2099ffb640 1 -- 172.21.15.112:0/3388364365 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2094003e00 con 0x7f209c15ee80 2024-09-21T00:26:32.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.477+0000 7f20a364d640 1 -- 172.21.15.112:0/3388364365 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f209c0cbaf0 con 0x7f209c15ee80 2024-09-21T00:26:32.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.477+0000 7f2099ffb640 1 -- 172.21.15.112:0/3388364365 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f209400cb80 con 0x7f209c15ee80 2024-09-21T00:26:32.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.477+0000 7f20a364d640 1 -- 172.21.15.112:0/3388364365 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f209c0cc0b0 con 0x7f209c15ee80 2024-09-21T00:26:32.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.478+0000 7f20a364d640 1 -- 172.21.15.112:0/3388364365 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f209c0b6db0 con 0x7f209c15ee80 2024-09-21T00:26:32.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.479+0000 7f2099ffb640 1 -- 172.21.15.112:0/3388364365 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2094004000 con 0x7f209c15ee80 2024-09-21T00:26:32.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.480+0000 7f2099ffb640 1 --2- 172.21.15.112:0/3388364365 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2088078b90 0x7f208807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:32.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.480+0000 7f2099ffb640 1 -- 172.21.15.112:0/3388364365 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7f209409a850 con 0x7f209c15ee80 2024-09-21T00:26:32.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.480+0000 7f20a13e9640 1 --2- 172.21.15.112:0/3388364365 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2088078b90 0x7f208807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:32.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.481+0000 7f20a13e9640 1 --2- 172.21.15.112:0/3388364365 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2088078b90 0x7f208807b0a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f208c033000 tx=0x7f208c004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:32.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.484+0000 7f2099ffb640 1 -- 172.21.15.112:0/3388364365 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f209c1f8b30 con 0x7f209c15ee80 2024-09-21T00:26:32.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.730+0000 7f20a364d640 1 -- 172.21.15.112:0/3388364365 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) -- 0x7f209c0caeb0 con 0x7f209c15ee80 2024-09-21T00:26:32.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.732+0000 7f2099ffb640 1 -- 172.21.15.112:0/3388364365 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v12) ==== 90+0+0 (secure 0 0 0) 0x7f2094062920 con 0x7f209c15ee80 2024-09-21T00:26:32.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.735+0000 7f20a364d640 1 -- 172.21.15.112:0/3388364365 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2088078b90 msgr2=0x7f208807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:32.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.735+0000 7f20a364d640 1 --2- 172.21.15.112:0/3388364365 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2088078b90 0x7f208807b0a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f208c033000 tx=0x7f208c004730 comp rx=0 tx=0).stop 2024-09-21T00:26:32.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.735+0000 7f20a364d640 1 -- 172.21.15.112:0/3388364365 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f209c15ee80 msgr2=0x7f209c06f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:32.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.735+0000 7f20a364d640 1 --2- 172.21.15.112:0/3388364365 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f209c15ee80 0x7f209c06f1f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f209400b9e0 tx=0x7f209400bea0 comp rx=0 tx=0).stop 2024-09-21T00:26:32.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.735+0000 7f20a364d640 1 -- 172.21.15.112:0/3388364365 shutdown_connections 2024-09-21T00:26:32.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.735+0000 7f20a364d640 1 --2- 172.21.15.112:0/3388364365 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2088078b90 0x7f208807b0a0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:32.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.735+0000 7f20a364d640 1 --2- 172.21.15.112:0/3388364365 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f209c15ee80 0x7f209c06f1f0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:32.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.735+0000 7f20a364d640 1 --2- 172.21.15.112:0/3388364365 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f209c0d1120 0x7f209c14d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:32.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.735+0000 7f20a364d640 1 --2- 172.21.15.112:0/3388364365 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f209c0b9ad0 0x7f209c1f85f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:32.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.735+0000 7f20a364d640 1 -- 172.21.15.112:0/3388364365 >> 172.21.15.112:0/3388364365 conn(0x7f209c09d830 msgr2=0x7f209c0ceb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:32.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.735+0000 7f20a364d640 1 -- 172.21.15.112:0/3388364365 shutdown_connections 2024-09-21T00:26:32.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.735+0000 7f20a364d640 1 -- 172.21.15.112:0/3388364365 wait complete. 2024-09-21T00:26:32.748 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-09-21T00:26:32.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.860+0000 7fe0caffd640 1 -- 172.21.15.112:0/1523518975 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0bc031bf0 con 0x7fe0cc15ee80 2024-09-21T00:26:32.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.860+0000 7fe0d2425640 1 -- 172.21.15.112:0/1523518975 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0cc15ee80 msgr2=0x7fe0cc0d0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:32.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.860+0000 7fe0d2425640 1 --2- 172.21.15.112:0/1523518975 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0cc15ee80 0x7fe0cc0d0170 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fe0bc009830 tx=0x7fe0bc02a3f0 comp rx=0 tx=0).stop 2024-09-21T00:26:32.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.860+0000 7fe0d2425640 1 -- 172.21.15.112:0/1523518975 shutdown_connections 2024-09-21T00:26:32.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.860+0000 7fe0d2425640 1 --2- 172.21.15.112:0/1523518975 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0cc15ee80 0x7fe0cc0d0170 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:32.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.860+0000 7fe0d2425640 1 -- 172.21.15.112:0/1523518975 >> 172.21.15.112:0/1523518975 conn(0x7fe0cc09d830 msgr2=0x7fe0cc0b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:32.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.860+0000 7fe0d2425640 1 -- 172.21.15.112:0/1523518975 shutdown_connections 2024-09-21T00:26:32.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.860+0000 7fe0d2425640 1 -- 172.21.15.112:0/1523518975 wait complete. 2024-09-21T00:26:32.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.862+0000 7fe0d2425640 1 Processor -- start 2024-09-21T00:26:32.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.862+0000 7fe0d2425640 1 -- start start 2024-09-21T00:26:32.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.863+0000 7fe0d2425640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe0cc0b9ad0 0x7fe0cc1579f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:32.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.863+0000 7fe0d2425640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0cc0ceb90 0x7fe0cc14c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:32.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.863+0000 7fe0d2425640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe0cc15ee80 0x7fe0cc0b2780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:32.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.863+0000 7fe0d2425640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe0cc0b6910 con 0x7fe0cc0ceb90 2024-09-21T00:26:32.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.863+0000 7fe0d2425640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe0cc0b6790 con 0x7fe0cc15ee80 2024-09-21T00:26:32.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.863+0000 7fe0d2425640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe0cc0b6a90 con 0x7fe0cc0b9ad0 2024-09-21T00:26:32.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.863+0000 7fe0cb7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0cc0ceb90 0x7fe0cc14c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:32.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.863+0000 7fe0cb7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0cc0ceb90 0x7fe0cc14c570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59894/0 (socket says 172.21.15.112:59894) 2024-09-21T00:26:32.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.863+0000 7fe0cb7fe640 1 -- 172.21.15.112:0/851793518 learned_addr learned my addr 172.21.15.112:0/851793518 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:26:32.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.864+0000 7fe0d09c2640 1 --2- 172.21.15.112:0/851793518 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe0cc15ee80 0x7fe0cc0b2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:32.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.864+0000 7fe0cbfff640 1 --2- 172.21.15.112:0/851793518 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe0cc0b9ad0 0x7fe0cc1579f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:32.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.864+0000 7fe0cb7fe640 1 -- 172.21.15.112:0/851793518 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe0cc0b9ad0 msgr2=0x7fe0cc1579f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:32.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.864+0000 7fe0cb7fe640 1 --2- 172.21.15.112:0/851793518 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe0cc0b9ad0 0x7fe0cc1579f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:32.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.864+0000 7fe0cb7fe640 1 -- 172.21.15.112:0/851793518 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe0cc15ee80 msgr2=0x7fe0cc0b2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:32.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.864+0000 7fe0cb7fe640 1 --2- 172.21.15.112:0/851793518 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe0cc15ee80 0x7fe0cc0b2780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:32.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.864+0000 7fe0cb7fe640 1 -- 172.21.15.112:0/851793518 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0cc157f30 con 0x7fe0cc0ceb90 2024-09-21T00:26:32.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.864+0000 7fe0cbfff640 1 --2- 172.21.15.112:0/851793518 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe0cc0b9ad0 0x7fe0cc1579f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:26:32.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.864+0000 7fe0d09c2640 1 --2- 172.21.15.112:0/851793518 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe0cc15ee80 0x7fe0cc0b2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:26:32.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.864+0000 7fe0cb7fe640 1 --2- 172.21.15.112:0/851793518 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0cc0ceb90 0x7fe0cc14c570 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fe0c000c3c0 tx=0x7fe0c000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:32.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.864+0000 7fe0c8ff9640 1 -- 172.21.15.112:0/851793518 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0c0017070 con 0x7fe0cc0ceb90 2024-09-21T00:26:32.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.864+0000 7fe0c8ff9640 1 -- 172.21.15.112:0/851793518 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe0c00035f0 con 0x7fe0cc0ceb90 2024-09-21T00:26:32.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.864+0000 7fe0d2425640 1 -- 172.21.15.112:0/851793518 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0cc06fc30 con 0x7fe0cc0ceb90 2024-09-21T00:26:32.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.864+0000 7fe0c8ff9640 1 -- 172.21.15.112:0/851793518 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0c0003a40 con 0x7fe0cc0ceb90 2024-09-21T00:26:32.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.864+0000 7fe0d2425640 1 -- 172.21.15.112:0/851793518 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0cc06d380 con 0x7fe0cc0ceb90 2024-09-21T00:26:32.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.865+0000 7fe0d2425640 1 -- 172.21.15.112:0/851793518 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0cc0d0170 con 0x7fe0cc0ceb90 2024-09-21T00:26:32.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.866+0000 7fe0c8ff9640 1 -- 172.21.15.112:0/851793518 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe0c001a070 con 0x7fe0cc0ceb90 2024-09-21T00:26:32.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.866+0000 7fe0c8ff9640 1 --2- 172.21.15.112:0/851793518 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fe0b8078b90 0x7fe0b807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:32.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.866+0000 7fe0c8ff9640 1 -- 172.21.15.112:0/851793518 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7fe0c0099e80 con 0x7fe0cc0ceb90 2024-09-21T00:26:32.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.866+0000 7fe0cbfff640 1 --2- 172.21.15.112:0/851793518 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fe0b8078b90 0x7fe0b807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:32.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.867+0000 7fe0cbfff640 1 --2- 172.21.15.112:0/851793518 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fe0b8078b90 0x7fe0b807b0a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe0bc034000 tx=0x7fe0bc004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:32.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:32.871+0000 7fe0c8ff9640 1 -- 172.21.15.112:0/851793518 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe0cc06fc30 con 0x7fe0cc0ceb90 2024-09-21T00:26:33.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.115+0000 7fe0d2425640 1 -- 172.21.15.112:0/851793518 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) -- 0x7fe0cc161cf0 con 0x7fe0cc0ceb90 2024-09-21T00:26:33.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.117+0000 7fe0c8ff9640 1 -- 172.21.15.112:0/851793518 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v13) ==== 97+0+0 (secure 0 0 0) 0x7fe0cc1592d0 con 0x7fe0cc0ceb90 2024-09-21T00:26:33.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.120+0000 7fe0d2425640 1 -- 172.21.15.112:0/851793518 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fe0b8078b90 msgr2=0x7fe0b807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:33.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.120+0000 7fe0d2425640 1 --2- 172.21.15.112:0/851793518 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fe0b8078b90 0x7fe0b807b0a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe0bc034000 tx=0x7fe0bc004730 comp rx=0 tx=0).stop 2024-09-21T00:26:33.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.120+0000 7fe0d2425640 1 -- 172.21.15.112:0/851793518 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0cc0ceb90 msgr2=0x7fe0cc14c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:33.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.120+0000 7fe0d2425640 1 --2- 172.21.15.112:0/851793518 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0cc0ceb90 0x7fe0cc14c570 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fe0c000c3c0 tx=0x7fe0c000c880 comp rx=0 tx=0).stop 2024-09-21T00:26:33.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.120+0000 7fe0d2425640 1 -- 172.21.15.112:0/851793518 shutdown_connections 2024-09-21T00:26:33.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.120+0000 7fe0d2425640 1 --2- 172.21.15.112:0/851793518 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe0cc15ee80 0x7fe0cc0b2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:33.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.120+0000 7fe0d2425640 1 --2- 172.21.15.112:0/851793518 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fe0b8078b90 0x7fe0b807b0a0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:33.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.120+0000 7fe0d2425640 1 --2- 172.21.15.112:0/851793518 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0cc0ceb90 0x7fe0cc14c570 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:33.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.120+0000 7fe0d2425640 1 --2- 172.21.15.112:0/851793518 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe0cc0b9ad0 0x7fe0cc1579f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:33.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.120+0000 7fe0d2425640 1 -- 172.21.15.112:0/851793518 >> 172.21.15.112:0/851793518 conn(0x7fe0cc09d830 msgr2=0x7fe0cc15ca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:33.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.120+0000 7fe0d2425640 1 -- 172.21.15.112:0/851793518 shutdown_connections 2024-09-21T00:26:33.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.120+0000 7fe0d2425640 1 -- 172.21.15.112:0/851793518 wait complete. 2024-09-21T00:26:33.133 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-09-21T00:26:33.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.244+0000 7f164696e640 1 -- 172.21.15.112:0/1321748924 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16400d0bb0 msgr2=0x7f1640070100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:33.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.244+0000 7f164696e640 1 --2- 172.21.15.112:0/1321748924 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16400d0bb0 0x7f1640070100 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f1640071ec0 tx=0x7f163802a980 comp rx=0 tx=0).stop 2024-09-21T00:26:33.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.244+0000 7f164696e640 1 -- 172.21.15.112:0/1321748924 shutdown_connections 2024-09-21T00:26:33.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.244+0000 7f164696e640 1 --2- 172.21.15.112:0/1321748924 >> v2:172.21.15.187:3300/0 conn(0x7f16400cd440 0x7f16400d0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:33.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.244+0000 7f164696e640 1 --2- 172.21.15.112:0/1321748924 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16400d0bb0 0x7f1640070100 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:33.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.244+0000 7f164696e640 1 --2- 172.21.15.112:0/1321748924 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f16400ccc40 0x7f164015ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:33.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.244+0000 7f164696e640 1 -- 172.21.15.112:0/1321748924 >> 172.21.15.112:0/1321748924 conn(0x7f16400b46e0 msgr2=0x7f16400d23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:33.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.244+0000 7f164696e640 1 -- 172.21.15.112:0/1321748924 shutdown_connections 2024-09-21T00:26:33.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.244+0000 7f164696e640 1 -- 172.21.15.112:0/1321748924 wait complete. 2024-09-21T00:26:33.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.246+0000 7f164696e640 1 Processor -- start 2024-09-21T00:26:33.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.246+0000 7f164696e640 1 -- start start 2024-09-21T00:26:33.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.247+0000 7f164696e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16400ccc40 0x7f16400b0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:33.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.247+0000 7f164696e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f16400cd440 0x7f16400cb660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:33.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.247+0000 7f164696e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f16400d0bb0 0x7f16400cbba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:33.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.247+0000 7f164696e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1640070a10 con 0x7f16400ccc40 2024-09-21T00:26:33.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.247+0000 7f164696e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f1640070890 con 0x7f16400cd440 2024-09-21T00:26:33.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.247+0000 7f164696e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f1640070b90 con 0x7f16400d0bb0 2024-09-21T00:26:33.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.247+0000 7f163ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16400ccc40 0x7f16400b0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:33.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.247+0000 7f163ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16400ccc40 0x7f16400b0ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59902/0 (socket says 172.21.15.112:59902) 2024-09-21T00:26:33.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.247+0000 7f163ffff640 1 -- 172.21.15.112:0/624319095 learned_addr learned my addr 172.21.15.112:0/624319095 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:26:33.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.247+0000 7f1644f0b640 1 --2- 172.21.15.112:0/624319095 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f16400d0bb0 0x7f16400cbba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:33.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.247+0000 7f163f7fe640 1 --2- 172.21.15.112:0/624319095 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f16400cd440 0x7f16400cb660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:33.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.248+0000 7f163ffff640 1 -- 172.21.15.112:0/624319095 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f16400d0bb0 msgr2=0x7f16400cbba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:33.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.248+0000 7f163ffff640 1 --2- 172.21.15.112:0/624319095 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f16400d0bb0 0x7f16400cbba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:33.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.248+0000 7f163ffff640 1 -- 172.21.15.112:0/624319095 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f16400cd440 msgr2=0x7f16400cb660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:33.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.248+0000 7f163ffff640 1 --2- 172.21.15.112:0/624319095 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f16400cd440 0x7f16400cb660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:33.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.248+0000 7f163ffff640 1 -- 172.21.15.112:0/624319095 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16400adab0 con 0x7f16400ccc40 2024-09-21T00:26:33.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.248+0000 7f163f7fe640 1 --2- 172.21.15.112:0/624319095 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f16400cd440 0x7f16400cb660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:26:33.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.248+0000 7f163ffff640 1 --2- 172.21.15.112:0/624319095 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16400ccc40 0x7f16400b0ee0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f163000c940 tx=0x7f163000ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:33.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.248+0000 7f163cff9640 1 -- 172.21.15.112:0/624319095 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1630007bb0 con 0x7f16400ccc40 2024-09-21T00:26:33.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.248+0000 7f163cff9640 1 -- 172.21.15.112:0/624319095 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1630007e40 con 0x7f16400ccc40 2024-09-21T00:26:33.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.248+0000 7f164696e640 1 -- 172.21.15.112:0/624319095 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16400add90 con 0x7f16400ccc40 2024-09-21T00:26:33.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.248+0000 7f163cff9640 1 -- 172.21.15.112:0/624319095 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f163000f670 con 0x7f16400ccc40 2024-09-21T00:26:33.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.248+0000 7f164696e640 1 -- 172.21.15.112:0/624319095 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16400ae350 con 0x7f16400ccc40 2024-09-21T00:26:33.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.249+0000 7f164696e640 1 -- 172.21.15.112:0/624319095 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16400d0670 con 0x7f16400ccc40 2024-09-21T00:26:33.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.250+0000 7f163cff9640 1 -- 172.21.15.112:0/624319095 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f163000f810 con 0x7f16400ccc40 2024-09-21T00:26:33.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.250+0000 7f163cff9640 1 --2- 172.21.15.112:0/624319095 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f162c078b90 0x7f162c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:33.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.250+0000 7f163cff9640 1 -- 172.21.15.112:0/624319095 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7f163009b3d0 con 0x7f16400ccc40 2024-09-21T00:26:33.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.250+0000 7f163f7fe640 1 --2- 172.21.15.112:0/624319095 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f162c078b90 0x7f162c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:33.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.251+0000 7f163f7fe640 1 --2- 172.21.15.112:0/624319095 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f162c078b90 0x7f162c07b0a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f16400b7950 tx=0x7f1634005d10 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:33.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.255+0000 7f163cff9640 1 -- 172.21.15.112:0/624319095 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1630014030 con 0x7f16400ccc40 2024-09-21T00:26:33.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.498+0000 7f164696e640 1 -- 172.21.15.112:0/624319095 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) -- 0x7f1640071960 con 0x7f16400ccc40 2024-09-21T00:26:33.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.501+0000 7f163cff9640 1 -- 172.21.15.112:0/624319095 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v14) ==== 90+0+0 (secure 0 0 0) 0x7f1630063950 con 0x7f16400ccc40 2024-09-21T00:26:33.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.503+0000 7f164696e640 1 -- 172.21.15.112:0/624319095 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f162c078b90 msgr2=0x7f162c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:33.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.503+0000 7f164696e640 1 --2- 172.21.15.112:0/624319095 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f162c078b90 0x7f162c07b0a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f16400b7950 tx=0x7f1634005d10 comp rx=0 tx=0).stop 2024-09-21T00:26:33.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.503+0000 7f164696e640 1 -- 172.21.15.112:0/624319095 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16400ccc40 msgr2=0x7f16400b0ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:33.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.503+0000 7f164696e640 1 --2- 172.21.15.112:0/624319095 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16400ccc40 0x7f16400b0ee0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f163000c940 tx=0x7f163000ce00 comp rx=0 tx=0).stop 2024-09-21T00:26:33.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.503+0000 7f164696e640 1 -- 172.21.15.112:0/624319095 shutdown_connections 2024-09-21T00:26:33.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.503+0000 7f164696e640 1 --2- 172.21.15.112:0/624319095 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f16400d0bb0 0x7f16400cbba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:33.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.503+0000 7f164696e640 1 --2- 172.21.15.112:0/624319095 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f16400cd440 0x7f16400cb660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:33.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.503+0000 7f164696e640 1 --2- 172.21.15.112:0/624319095 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f162c078b90 0x7f162c07b0a0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:33.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.503+0000 7f164696e640 1 --2- 172.21.15.112:0/624319095 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16400ccc40 0x7f16400b0ee0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:33.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.503+0000 7f164696e640 1 -- 172.21.15.112:0/624319095 >> 172.21.15.112:0/624319095 conn(0x7f16400b46e0 msgr2=0x7f164015fb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:33.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.503+0000 7f164696e640 1 -- 172.21.15.112:0/624319095 shutdown_connections 2024-09-21T00:26:33.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.503+0000 7f164696e640 1 -- 172.21.15.112:0/624319095 wait complete. 2024-09-21T00:26:33.517 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-21T00:26:33.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.627+0000 7f1ad8069640 1 -- 172.21.15.112:0/3446218244 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ad00cee70 msgr2=0x7f1ad015b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:33.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.627+0000 7f1ad8069640 1 --2- 172.21.15.112:0/3446218244 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ad00cee70 0x7f1ad015b350 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f1ad00ce420 tx=0x7f1ac002aa30 comp rx=0 tx=0).stop 2024-09-21T00:26:33.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.627+0000 7f1ad8069640 1 -- 172.21.15.112:0/3446218244 shutdown_connections 2024-09-21T00:26:33.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.627+0000 7f1ad8069640 1 --2- 172.21.15.112:0/3446218244 >> v2:172.21.15.187:3300/0 conn(0x7f1ad015fc70 0x7f1ad00b40c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:33.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.627+0000 7f1ad8069640 1 --2- 172.21.15.112:0/3446218244 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ad00cee70 0x7f1ad015b350 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:33.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.627+0000 7f1ad8069640 1 -- 172.21.15.112:0/3446218244 >> 172.21.15.112:0/3446218244 conn(0x7f1ad0073e30 msgr2=0x7f1ad0074240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:33.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.628+0000 7f1ad8069640 1 -- 172.21.15.112:0/3446218244 shutdown_connections 2024-09-21T00:26:33.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.628+0000 7f1ad8069640 1 -- 172.21.15.112:0/3446218244 wait complete. 2024-09-21T00:26:33.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.629+0000 7f1ad8069640 1 Processor -- start 2024-09-21T00:26:33.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.629+0000 7f1ad8069640 1 -- start start 2024-09-21T00:26:33.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.631+0000 7f1ad8069640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1ad00cee70 0x7f1ad01f7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:33.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.631+0000 7f1ad8069640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ad015f150 0x7f1ad0148a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:33.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.631+0000 7f1ad8069640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1ad015fc70 0x7f1ad006cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:33.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.631+0000 7f1ad8069640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1ad00b4990 con 0x7f1ad015f150 2024-09-21T00:26:33.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.631+0000 7f1ad8069640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f1ad00b4810 con 0x7f1ad00cee70 2024-09-21T00:26:33.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.631+0000 7f1ad8069640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f1ad00b4b10 con 0x7f1ad015fc70 2024-09-21T00:26:33.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.631+0000 7f1ad5604640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ad015f150 0x7f1ad0148a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:33.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.631+0000 7f1ad5604640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ad015f150 0x7f1ad0148a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59920/0 (socket says 172.21.15.112:59920) 2024-09-21T00:26:33.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.631+0000 7f1ad5604640 1 -- 172.21.15.112:0/2888807184 learned_addr learned my addr 172.21.15.112:0/2888807184 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:26:33.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.631+0000 7f1ad5e05640 1 --2- 172.21.15.112:0/2888807184 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1ad00cee70 0x7f1ad01f7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:33.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.631+0000 7f1ad6606640 1 --2- 172.21.15.112:0/2888807184 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1ad015fc70 0x7f1ad006cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:33.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.631+0000 7f1ad5604640 1 -- 172.21.15.112:0/2888807184 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1ad015fc70 msgr2=0x7f1ad006cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:33.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.631+0000 7f1ad5604640 1 --2- 172.21.15.112:0/2888807184 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1ad015fc70 0x7f1ad006cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:33.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.631+0000 7f1ad5604640 1 -- 172.21.15.112:0/2888807184 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1ad00cee70 msgr2=0x7f1ad01f7ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:33.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.631+0000 7f1ad5604640 1 --2- 172.21.15.112:0/2888807184 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1ad00cee70 0x7f1ad01f7ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:33.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.631+0000 7f1ad5604640 1 -- 172.21.15.112:0/2888807184 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ad01f8020 con 0x7f1ad015f150 2024-09-21T00:26:33.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.631+0000 7f1ad5e05640 1 --2- 172.21.15.112:0/2888807184 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1ad00cee70 0x7f1ad01f7ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:26:33.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.631+0000 7f1ad6606640 1 --2- 172.21.15.112:0/2888807184 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1ad015fc70 0x7f1ad006cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:26:33.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.631+0000 7f1ad5604640 1 --2- 172.21.15.112:0/2888807184 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ad015f150 0x7f1ad0148a70 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f1ac400c430 tx=0x7f1ac400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:33.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.631+0000 7f1ace7fc640 1 -- 172.21.15.112:0/2888807184 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac4017070 con 0x7f1ad015f150 2024-09-21T00:26:33.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.632+0000 7f1ace7fc640 1 -- 172.21.15.112:0/2888807184 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1ac40053c0 con 0x7f1ad015f150 2024-09-21T00:26:33.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.632+0000 7f1ad8069640 1 -- 172.21.15.112:0/2888807184 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ad00ae4f0 con 0x7f1ad015f150 2024-09-21T00:26:33.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.632+0000 7f1ace7fc640 1 -- 172.21.15.112:0/2888807184 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac4003850 con 0x7f1ad015f150 2024-09-21T00:26:33.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.632+0000 7f1ad8069640 1 -- 172.21.15.112:0/2888807184 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ad00aeab0 con 0x7f1ad015f150 2024-09-21T00:26:33.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.633+0000 7f1ad8069640 1 -- 172.21.15.112:0/2888807184 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ad00b4fc0 con 0x7f1ad015f150 2024-09-21T00:26:33.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.633+0000 7f1ace7fc640 1 -- 172.21.15.112:0/2888807184 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1ac401a070 con 0x7f1ad015f150 2024-09-21T00:26:33.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.634+0000 7f1ace7fc640 1 --2- 172.21.15.112:0/2888807184 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1abc078b90 0x7f1abc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:33.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.634+0000 7f1ace7fc640 1 -- 172.21.15.112:0/2888807184 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7f1ac4099f80 con 0x7f1ad015f150 2024-09-21T00:26:33.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.634+0000 7f1ad5e05640 1 --2- 172.21.15.112:0/2888807184 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1abc078b90 0x7f1abc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:33.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.635+0000 7f1ad5e05640 1 --2- 172.21.15.112:0/2888807184 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1abc078b90 0x7f1abc07b0a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1ac0035000 tx=0x7f1ac0004ab0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:33.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.638+0000 7f1ace7fc640 1 -- 172.21.15.112:0/2888807184 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1ac4062580 con 0x7f1ad015f150 2024-09-21T00:26:33.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.817+0000 7f1ad8069640 1 -- 172.21.15.112:0/2888807184 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f1ad00b5850 con 0x7f1ad015f150 2024-09-21T00:26:33.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.817+0000 7f1ace7fc640 1 -- 172.21.15.112:0/2888807184 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v34) ==== 84+0+65 (secure 0 0 0) 0x7f1ac4061e80 con 0x7f1ad015f150 2024-09-21T00:26:33.819 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:26:33.819 INFO:teuthology.orchestra.run.smithi112.stdout:[ 2024-09-21T00:26:33.820 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:26:33.820 INFO:teuthology.orchestra.run.smithi112.stdout: "poolnum": 1, 2024-09-21T00:26:33.820 INFO:teuthology.orchestra.run.smithi112.stdout: "poolname": ".mgr" 2024-09-21T00:26:33.820 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:26:33.820 INFO:teuthology.orchestra.run.smithi112.stdout:] 2024-09-21T00:26:33.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.820+0000 7f1ad8069640 1 -- 172.21.15.112:0/2888807184 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1abc078b90 msgr2=0x7f1abc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:33.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.820+0000 7f1ad8069640 1 --2- 172.21.15.112:0/2888807184 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1abc078b90 0x7f1abc07b0a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1ac0035000 tx=0x7f1ac0004ab0 comp rx=0 tx=0).stop 2024-09-21T00:26:33.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.820+0000 7f1ad8069640 1 -- 172.21.15.112:0/2888807184 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ad015f150 msgr2=0x7f1ad0148a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:33.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.820+0000 7f1ad8069640 1 --2- 172.21.15.112:0/2888807184 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ad015f150 0x7f1ad0148a70 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f1ac400c430 tx=0x7f1ac400c8f0 comp rx=0 tx=0).stop 2024-09-21T00:26:33.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.820+0000 7f1ad8069640 1 -- 172.21.15.112:0/2888807184 shutdown_connections 2024-09-21T00:26:33.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.820+0000 7f1ad8069640 1 --2- 172.21.15.112:0/2888807184 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1ad015fc70 0x7f1ad006cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:33.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.820+0000 7f1ad8069640 1 --2- 172.21.15.112:0/2888807184 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1abc078b90 0x7f1abc07b0a0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:33.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.820+0000 7f1ad8069640 1 --2- 172.21.15.112:0/2888807184 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ad015f150 0x7f1ad0148a70 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:33.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.820+0000 7f1ad8069640 1 --2- 172.21.15.112:0/2888807184 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1ad00cee70 0x7f1ad01f7ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:33.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.820+0000 7f1ad8069640 1 -- 172.21.15.112:0/2888807184 >> 172.21.15.112:0/2888807184 conn(0x7f1ad0073e30 msgr2=0x7f1ad00b0a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:33.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.820+0000 7f1ad8069640 1 -- 172.21.15.112:0/2888807184 shutdown_connections 2024-09-21T00:26:33.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.820+0000 7f1ad8069640 1 -- 172.21.15.112:0/2888807184 wait complete. 2024-09-21T00:26:33.833 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-21T00:26:33.833 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-21T00:26:33.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.947+0000 7fe840bcc640 1 -- 172.21.15.112:0/3422798977 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe83c15d040 msgr2=0x7fe83c0d1200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:33.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.947+0000 7fe840bcc640 1 --2- 172.21.15.112:0/3422798977 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe83c15d040 0x7fe83c0d1200 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fe83c073f30 tx=0x7fe82402a560 comp rx=0 tx=0).stop 2024-09-21T00:26:33.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.947+0000 7fe840bcc640 1 -- 172.21.15.112:0/3422798977 shutdown_connections 2024-09-21T00:26:33.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.947+0000 7fe840bcc640 1 --2- 172.21.15.112:0/3422798977 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe83c0d1740 0x7fe83c1600f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:33.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.947+0000 7fe840bcc640 1 --2- 172.21.15.112:0/3422798977 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe83c15d040 0x7fe83c0d1200 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:33.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.947+0000 7fe840bcc640 1 --2- 172.21.15.112:0/3422798977 >> v2:172.21.15.187:3300/0 conn(0x7fe83c145e40 0x7fe83c15cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:33.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.947+0000 7fe840bcc640 1 -- 172.21.15.112:0/3422798977 >> 172.21.15.112:0/3422798977 conn(0x7fe83c0d22e0 msgr2=0x7fe83c0d26f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:33.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.947+0000 7fe840bcc640 1 -- 172.21.15.112:0/3422798977 shutdown_connections 2024-09-21T00:26:33.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.947+0000 7fe840bcc640 1 -- 172.21.15.112:0/3422798977 wait complete. 2024-09-21T00:26:33.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.949+0000 7fe840bcc640 1 Processor -- start 2024-09-21T00:26:33.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.949+0000 7fe840bcc640 1 -- start start 2024-09-21T00:26:33.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.951+0000 7fe840bcc640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe83c0d1740 0x7fe83c0cdf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:33.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.951+0000 7fe840bcc640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe83c145e40 0x7fe83c13ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:33.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.951+0000 7fe840bcc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe83c15d040 0x7fe83c06e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:33.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.951+0000 7fe840bcc640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe83c1607b0 con 0x7fe83c15d040 2024-09-21T00:26:33.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.951+0000 7fe840bcc640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe83c160630 con 0x7fe83c145e40 2024-09-21T00:26:33.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.951+0000 7fe840bcc640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe83c160930 con 0x7fe83c0d1740 2024-09-21T00:26:33.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.951+0000 7fe83ad9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe83c15d040 0x7fe83c06e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:33.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.951+0000 7fe83ad9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe83c15d040 0x7fe83c06e9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59950/0 (socket says 172.21.15.112:59950) 2024-09-21T00:26:33.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.951+0000 7fe83ad9d640 1 -- 172.21.15.112:0/3822676692 learned_addr learned my addr 172.21.15.112:0/3822676692 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:26:33.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.951+0000 7fe833fff640 1 --2- 172.21.15.112:0/3822676692 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe83c145e40 0x7fe83c13ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:33.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.951+0000 7fe83a59c640 1 --2- 172.21.15.112:0/3822676692 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe83c0d1740 0x7fe83c0cdf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:33.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.951+0000 7fe83ad9d640 1 -- 172.21.15.112:0/3822676692 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe83c0d1740 msgr2=0x7fe83c0cdf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:33.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.951+0000 7fe83ad9d640 1 --2- 172.21.15.112:0/3822676692 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe83c0d1740 0x7fe83c0cdf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:33.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.951+0000 7fe83ad9d640 1 -- 172.21.15.112:0/3822676692 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe83c145e40 msgr2=0x7fe83c13ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:33.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.951+0000 7fe83ad9d640 1 --2- 172.21.15.112:0/3822676692 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe83c145e40 0x7fe83c13ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:33.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.951+0000 7fe83ad9d640 1 -- 172.21.15.112:0/3822676692 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe83c0ce460 con 0x7fe83c15d040 2024-09-21T00:26:33.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.951+0000 7fe83a59c640 1 --2- 172.21.15.112:0/3822676692 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe83c0d1740 0x7fe83c0cdf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:26:33.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.951+0000 7fe83ad9d640 1 --2- 172.21.15.112:0/3822676692 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe83c15d040 0x7fe83c06e9e0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fe83400ba10 tx=0x7fe83400bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:33.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.951+0000 7fe832ffd640 1 -- 172.21.15.112:0/3822676692 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe834019070 con 0x7fe83c15d040 2024-09-21T00:26:33.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.951+0000 7fe832ffd640 1 -- 172.21.15.112:0/3822676692 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe83400cc50 con 0x7fe83c15d040 2024-09-21T00:26:33.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.951+0000 7fe840bcc640 1 -- 172.21.15.112:0/3822676692 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe83c0cc1c0 con 0x7fe83c15d040 2024-09-21T00:26:33.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.952+0000 7fe832ffd640 1 -- 172.21.15.112:0/3822676692 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8340146a0 con 0x7fe83c15d040 2024-09-21T00:26:33.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.952+0000 7fe840bcc640 1 -- 172.21.15.112:0/3822676692 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe83c0cc780 con 0x7fe83c15d040 2024-09-21T00:26:33.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.953+0000 7fe840bcc640 1 -- 172.21.15.112:0/3822676692 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe83c160dc0 con 0x7fe83c15d040 2024-09-21T00:26:33.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.953+0000 7fe832ffd640 1 -- 172.21.15.112:0/3822676692 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe83401c070 con 0x7fe83c15d040 2024-09-21T00:26:33.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.954+0000 7fe832ffd640 1 --2- 172.21.15.112:0/3822676692 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fe81c078b90 0x7fe81c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:33.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.954+0000 7fe832ffd640 1 -- 172.21.15.112:0/3822676692 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7fe83409a850 con 0x7fe83c15d040 2024-09-21T00:26:33.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.954+0000 7fe83a59c640 1 --2- 172.21.15.112:0/3822676692 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fe81c078b90 0x7fe81c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:33.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.954+0000 7fe83a59c640 1 --2- 172.21.15.112:0/3822676692 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fe81c078b90 0x7fe81c07b0a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe82c0058c0 tx=0x7fe82c005c90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:33.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:33.958+0000 7fe832ffd640 1 -- 172.21.15.112:0/3822676692 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe834062e50 con 0x7fe83c15d040 2024-09-21T00:26:34.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:34.135+0000 7fe840bcc640 1 -- 172.21.15.112:0/3822676692 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fe83c0b55d0 con 0x7fe83c15d040 2024-09-21T00:26:34.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:34.809+0000 7fe832ffd640 1 -- 172.21.15.112:0/3822676692 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v35) ==== 138+0+0 (secure 0 0 0) 0x7fe834062750 con 0x7fe83c15d040 2024-09-21T00:26:34.811 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'cephfs_metadata' created 2024-09-21T00:26:34.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:34.812+0000 7fe840bcc640 1 -- 172.21.15.112:0/3822676692 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fe81c078b90 msgr2=0x7fe81c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:34.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:34.812+0000 7fe840bcc640 1 --2- 172.21.15.112:0/3822676692 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fe81c078b90 0x7fe81c07b0a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe82c0058c0 tx=0x7fe82c005c90 comp rx=0 tx=0).stop 2024-09-21T00:26:34.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:34.812+0000 7fe840bcc640 1 -- 172.21.15.112:0/3822676692 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe83c15d040 msgr2=0x7fe83c06e9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:34.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:34.812+0000 7fe840bcc640 1 --2- 172.21.15.112:0/3822676692 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe83c15d040 0x7fe83c06e9e0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fe83400ba10 tx=0x7fe83400bed0 comp rx=0 tx=0).stop 2024-09-21T00:26:34.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:34.812+0000 7fe840bcc640 1 -- 172.21.15.112:0/3822676692 shutdown_connections 2024-09-21T00:26:34.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:34.812+0000 7fe840bcc640 1 --2- 172.21.15.112:0/3822676692 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fe81c078b90 0x7fe81c07b0a0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:34.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:34.812+0000 7fe840bcc640 1 --2- 172.21.15.112:0/3822676692 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe83c15d040 0x7fe83c06e9e0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:34.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:34.812+0000 7fe840bcc640 1 --2- 172.21.15.112:0/3822676692 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe83c145e40 0x7fe83c13ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:34.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:34.812+0000 7fe840bcc640 1 --2- 172.21.15.112:0/3822676692 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe83c0d1740 0x7fe83c0cdf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:34.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:34.812+0000 7fe840bcc640 1 -- 172.21.15.112:0/3822676692 >> 172.21.15.112:0/3822676692 conn(0x7fe83c0d22e0 msgr2=0x7fe83c15ddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:34.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:34.812+0000 7fe840bcc640 1 -- 172.21.15.112:0/3822676692 shutdown_connections 2024-09-21T00:26:34.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:34.812+0000 7fe840bcc640 1 -- 172.21.15.112:0/3822676692 wait complete. 2024-09-21T00:26:34.831 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-21T00:26:34.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:34.950+0000 7fdfac0ec640 1 -- 172.21.15.112:0/2830071615 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdfa40d01f0 msgr2=0x7fdfa415daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:34.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:34.950+0000 7fdfac0ec640 1 --2- 172.21.15.112:0/2830071615 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdfa40d01f0 0x7fdfa415daf0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fdfa40cec40 tx=0x7fdfa002aa30 comp rx=0 tx=0).stop 2024-09-21T00:26:34.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:34.950+0000 7fdfac0ec640 1 -- 172.21.15.112:0/2830071615 shutdown_connections 2024-09-21T00:26:34.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:34.950+0000 7fdfac0ec640 1 --2- 172.21.15.112:0/2830071615 >> v2:172.21.15.187:3300/0 conn(0x7fdfa415a6f0 0x7fdfa415e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:34.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:34.950+0000 7fdfac0ec640 1 --2- 172.21.15.112:0/2830071615 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdfa40d01f0 0x7fdfa415daf0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:34.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:34.950+0000 7fdfac0ec640 1 -- 172.21.15.112:0/2830071615 >> 172.21.15.112:0/2830071615 conn(0x7fdfa4072990 msgr2=0x7fdfa4072da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:34.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:34.950+0000 7fdfac0ec640 1 -- 172.21.15.112:0/2830071615 shutdown_connections 2024-09-21T00:26:34.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:34.950+0000 7fdfac0ec640 1 -- 172.21.15.112:0/2830071615 wait complete. 2024-09-21T00:26:34.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:34.952+0000 7fdfac0ec640 1 Processor -- start 2024-09-21T00:26:34.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:34.952+0000 7fdfac0ec640 1 -- start start 2024-09-21T00:26:34.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:34.953+0000 7fdfac0ec640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdfa40d01f0 0x7fdfa41511a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:34.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:34.953+0000 7fdfac0ec640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdfa40d0980 0x7fdfa40c71f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:34.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:34.953+0000 7fdfac0ec640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdfa415a6f0 0x7fdfa4154990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:34.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:34.953+0000 7fdfac0ec640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fdfa40cd130 con 0x7fdfa40d01f0 2024-09-21T00:26:34.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:34.953+0000 7fdfac0ec640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fdfa40ccfb0 con 0x7fdfa40d0980 2024-09-21T00:26:34.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:34.953+0000 7fdfac0ec640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fdfa40cd2b0 con 0x7fdfa415a6f0 2024-09-21T00:26:34.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:34.953+0000 7fdfa9e88640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdfa40d01f0 0x7fdfa41511a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:34.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:34.953+0000 7fdfa9e88640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdfa40d01f0 0x7fdfa41511a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59956/0 (socket says 172.21.15.112:59956) 2024-09-21T00:26:34.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:34.953+0000 7fdfa9e88640 1 -- 172.21.15.112:0/4098727074 learned_addr learned my addr 172.21.15.112:0/4098727074 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:26:34.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:34.953+0000 7fdfaa689640 1 --2- 172.21.15.112:0/4098727074 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdfa415a6f0 0x7fdfa4154990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:34.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:34.953+0000 7fdfa9687640 1 --2- 172.21.15.112:0/4098727074 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdfa40d0980 0x7fdfa40c71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:34.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:34.954+0000 7fdfa9e88640 1 -- 172.21.15.112:0/4098727074 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdfa415a6f0 msgr2=0x7fdfa4154990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:34.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:34.954+0000 7fdfa9e88640 1 --2- 172.21.15.112:0/4098727074 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdfa415a6f0 0x7fdfa4154990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:34.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:34.954+0000 7fdfa9e88640 1 -- 172.21.15.112:0/4098727074 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdfa40d0980 msgr2=0x7fdfa40c71f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:34.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:34.954+0000 7fdfa9e88640 1 --2- 172.21.15.112:0/4098727074 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdfa40d0980 0x7fdfa40c71f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:34.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:34.954+0000 7fdfa9e88640 1 -- 172.21.15.112:0/4098727074 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfa40c7c40 con 0x7fdfa40d01f0 2024-09-21T00:26:34.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:34.954+0000 7fdfa9e88640 1 --2- 172.21.15.112:0/4098727074 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdfa40d01f0 0x7fdfa41511a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fdfa002b020 tx=0x7fdfa0003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:34.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:34.954+0000 7fdf9a7fc640 1 -- 172.21.15.112:0/4098727074 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfa0033050 con 0x7fdfa40d01f0 2024-09-21T00:26:34.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:34.954+0000 7fdf9a7fc640 1 -- 172.21.15.112:0/4098727074 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdfa00044f0 con 0x7fdfa40d01f0 2024-09-21T00:26:34.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:34.954+0000 7fdfac0ec640 1 -- 172.21.15.112:0/4098727074 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfa40c7730 con 0x7fdfa40d01f0 2024-09-21T00:26:34.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:34.954+0000 7fdf9a7fc640 1 -- 172.21.15.112:0/4098727074 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfa0038450 con 0x7fdfa40d01f0 2024-09-21T00:26:34.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:34.954+0000 7fdfac0ec640 1 -- 172.21.15.112:0/4098727074 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfa41580b0 con 0x7fdfa40d01f0 2024-09-21T00:26:34.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:34.955+0000 7fdfac0ec640 1 -- 172.21.15.112:0/4098727074 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfa40cd7c0 con 0x7fdfa40d01f0 2024-09-21T00:26:34.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:34.956+0000 7fdf9a7fc640 1 -- 172.21.15.112:0/4098727074 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fdfa0036070 con 0x7fdfa40d01f0 2024-09-21T00:26:34.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:34.956+0000 7fdf9a7fc640 1 --2- 172.21.15.112:0/4098727074 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fdf88078b90 0x7fdf8807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:34.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:34.957+0000 7fdf9a7fc640 1 -- 172.21.15.112:0/4098727074 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5619+0+0 (secure 0 0 0) 0x7fdfa00bc170 con 0x7fdfa40d01f0 2024-09-21T00:26:34.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:34.957+0000 7fdfa9687640 1 --2- 172.21.15.112:0/4098727074 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fdf88078b90 0x7fdf8807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:34.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:34.957+0000 7fdfa9687640 1 --2- 172.21.15.112:0/4098727074 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fdf88078b90 0x7fdf8807b0a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fdfa40c87d0 tx=0x7fdf9400b2e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:34.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:34.961+0000 7fdf9a7fc640 1 -- 172.21.15.112:0/4098727074 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fdfa0083fb0 con 0x7fdfa40d01f0 2024-09-21T00:26:35.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:35.139+0000 7fdfac0ec640 1 -- 172.21.15.112:0/4098727074 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fdfa415ff80 con 0x7fdfa40d01f0 2024-09-21T00:26:35.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:35.816+0000 7fdf9a7fc640 1 -- 172.21.15.112:0/4098727074 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v36) ==== 170+0+0 (secure 0 0 0) 0x7fdfa00889e0 con 0x7fdfa40d01f0 2024-09-21T00:26:35.818 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'cephfs_data' created 2024-09-21T00:26:35.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:35.819+0000 7fdfac0ec640 1 -- 172.21.15.112:0/4098727074 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fdf88078b90 msgr2=0x7fdf8807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:35.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:35.819+0000 7fdfac0ec640 1 --2- 172.21.15.112:0/4098727074 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fdf88078b90 0x7fdf8807b0a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fdfa40c87d0 tx=0x7fdf9400b2e0 comp rx=0 tx=0).stop 2024-09-21T00:26:35.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:35.819+0000 7fdfac0ec640 1 -- 172.21.15.112:0/4098727074 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdfa40d01f0 msgr2=0x7fdfa41511a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:35.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:35.819+0000 7fdfac0ec640 1 --2- 172.21.15.112:0/4098727074 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdfa40d01f0 0x7fdfa41511a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fdfa002b020 tx=0x7fdfa0003f10 comp rx=0 tx=0).stop 2024-09-21T00:26:35.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:35.819+0000 7fdfac0ec640 1 -- 172.21.15.112:0/4098727074 shutdown_connections 2024-09-21T00:26:35.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:35.819+0000 7fdfac0ec640 1 --2- 172.21.15.112:0/4098727074 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdfa415a6f0 0x7fdfa4154990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:35.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:35.819+0000 7fdfac0ec640 1 --2- 172.21.15.112:0/4098727074 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdfa40d0980 0x7fdfa40c71f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:35.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:35.819+0000 7fdfac0ec640 1 --2- 172.21.15.112:0/4098727074 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fdf88078b90 0x7fdf8807b0a0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:35.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:35.819+0000 7fdfac0ec640 1 --2- 172.21.15.112:0/4098727074 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdfa40d01f0 0x7fdfa41511a0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:35.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:35.819+0000 7fdfac0ec640 1 -- 172.21.15.112:0/4098727074 >> 172.21.15.112:0/4098727074 conn(0x7fdfa4072990 msgr2=0x7fdfa415eb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:35.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:35.819+0000 7fdfac0ec640 1 -- 172.21.15.112:0/4098727074 shutdown_connections 2024-09-21T00:26:35.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:35.820+0000 7fdfac0ec640 1 -- 172.21.15.112:0/4098727074 wait complete. 2024-09-21T00:26:35.844 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-21T00:26:35.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:35.961+0000 7fc9cc08b640 1 -- 172.21.15.112:0/1246167785 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc9c415b730 msgr2=0x7fc9c40b5d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:35.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:35.961+0000 7fc9cc08b640 1 --2- 172.21.15.112:0/1246167785 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc9c415b730 0x7fc9c40b5d20 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fc9c0005fd0 tx=0x7fc9c002a820 comp rx=0 tx=0).stop 2024-09-21T00:26:35.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:35.961+0000 7fc9cc08b640 1 -- 172.21.15.112:0/1246167785 shutdown_connections 2024-09-21T00:26:35.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:35.961+0000 7fc9cc08b640 1 --2- 172.21.15.112:0/1246167785 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc9c415b730 0x7fc9c40b5d20 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:35.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:35.961+0000 7fc9cc08b640 1 -- 172.21.15.112:0/1246167785 >> 172.21.15.112:0/1246167785 conn(0x7fc9c40735d0 msgr2=0x7fc9c40739e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:35.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:35.961+0000 7fc9cc08b640 1 -- 172.21.15.112:0/1246167785 shutdown_connections 2024-09-21T00:26:35.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:35.961+0000 7fc9cc08b640 1 -- 172.21.15.112:0/1246167785 wait complete. 2024-09-21T00:26:35.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:35.963+0000 7fc9cc08b640 1 Processor -- start 2024-09-21T00:26:35.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:35.963+0000 7fc9cc08b640 1 -- start start 2024-09-21T00:26:35.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:35.964+0000 7fc9cc08b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc9c406ddf0 0x7fc9c40ad860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:35.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:35.964+0000 7fc9cc08b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc9c40b6290 0x7fc9c41f5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:35.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:35.964+0000 7fc9cc08b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc9c415b730 0x7fc9c415d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:35.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:35.964+0000 7fc9cc08b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc9c40719e0 con 0x7fc9c406ddf0 2024-09-21T00:26:35.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:35.964+0000 7fc9cc08b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc9c4071860 con 0x7fc9c415b730 2024-09-21T00:26:35.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:35.964+0000 7fc9cc08b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc9c4071b60 con 0x7fc9c40b6290 2024-09-21T00:26:35.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:35.964+0000 7fc9c9e27640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc9c406ddf0 0x7fc9c40ad860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:35.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:35.964+0000 7fc9c9e27640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc9c406ddf0 0x7fc9c40ad860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59984/0 (socket says 172.21.15.112:59984) 2024-09-21T00:26:35.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:35.964+0000 7fc9c9e27640 1 -- 172.21.15.112:0/1967029064 learned_addr learned my addr 172.21.15.112:0/1967029064 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:26:35.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:35.965+0000 7fc9c9626640 1 --2- 172.21.15.112:0/1967029064 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc9c40b6290 0x7fc9c41f5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:35.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:35.965+0000 7fc9ca628640 1 --2- 172.21.15.112:0/1967029064 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc9c415b730 0x7fc9c415d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:35.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:35.965+0000 7fc9c9e27640 1 -- 172.21.15.112:0/1967029064 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc9c40b6290 msgr2=0x7fc9c41f5550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:35.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:35.965+0000 7fc9c9e27640 1 --2- 172.21.15.112:0/1967029064 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc9c40b6290 0x7fc9c41f5550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:35.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:35.965+0000 7fc9c9e27640 1 -- 172.21.15.112:0/1967029064 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc9c415b730 msgr2=0x7fc9c415d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:35.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:35.965+0000 7fc9c9e27640 1 --2- 172.21.15.112:0/1967029064 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc9c415b730 0x7fc9c415d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:35.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:35.965+0000 7fc9c9e27640 1 -- 172.21.15.112:0/1967029064 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9c40adda0 con 0x7fc9c406ddf0 2024-09-21T00:26:35.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:35.965+0000 7fc9ca628640 1 --2- 172.21.15.112:0/1967029064 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc9c415b730 0x7fc9c415d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:26:35.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:35.965+0000 7fc9c9e27640 1 --2- 172.21.15.112:0/1967029064 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc9c406ddf0 0x7fc9c40ad860 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fc9c0009cd0 tx=0x7fc9c0032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:35.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:35.965+0000 7fc9ba7fc640 1 -- 172.21.15.112:0/1967029064 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9c0037040 con 0x7fc9c406ddf0 2024-09-21T00:26:35.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:35.965+0000 7fc9ba7fc640 1 -- 172.21.15.112:0/1967029064 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc9c0042070 con 0x7fc9c406ddf0 2024-09-21T00:26:35.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:35.965+0000 7fc9ba7fc640 1 -- 172.21.15.112:0/1967029064 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9c0004280 con 0x7fc9c406ddf0 2024-09-21T00:26:35.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:35.965+0000 7fc9cc08b640 1 -- 172.21.15.112:0/1967029064 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9c4154f50 con 0x7fc9c406ddf0 2024-09-21T00:26:35.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:35.965+0000 7fc9cc08b640 1 -- 172.21.15.112:0/1967029064 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9c415cf50 con 0x7fc9c406ddf0 2024-09-21T00:26:35.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:35.966+0000 7fc9cc08b640 1 -- 172.21.15.112:0/1967029064 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9c40b2220 con 0x7fc9c406ddf0 2024-09-21T00:26:35.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:35.967+0000 7fc9ba7fc640 1 -- 172.21.15.112:0/1967029064 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc9c0004420 con 0x7fc9c406ddf0 2024-09-21T00:26:35.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:35.967+0000 7fc9ba7fc640 1 --2- 172.21.15.112:0/1967029064 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fc9a8078bc0 0x7fc9a807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:35.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:35.968+0000 7fc9ba7fc640 1 -- 172.21.15.112:0/1967029064 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 6001+0+0 (secure 0 0 0) 0x7fc9c00bc7a0 con 0x7fc9c406ddf0 2024-09-21T00:26:35.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:35.968+0000 7fc9c9626640 1 --2- 172.21.15.112:0/1967029064 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fc9a8078bc0 0x7fc9a807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:35.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:35.968+0000 7fc9c9626640 1 --2- 172.21.15.112:0/1967029064 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fc9a8078bc0 0x7fc9a807b0d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc9c415ce60 tx=0x7fc9b4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:35.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:35.971+0000 7fc9ba7fc640 1 -- 172.21.15.112:0/1967029064 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc9c40b2220 con 0x7fc9c406ddf0 2024-09-21T00:26:36.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:36.186+0000 7fc9cc08b640 1 -- 172.21.15.112:0/1967029064 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fc9c40b5d20 con 0x7fc9c406ddf0 2024-09-21T00:26:36.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:36.818+0000 7fc9ba7fc640 1 -- 172.21.15.112:0/1967029064 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 5 and data pool 6 v14) ==== 172+0+0 (secure 0 0 0) 0x7fc9c0084c20 con 0x7fc9c406ddf0 2024-09-21T00:26:36.820 INFO:teuthology.orchestra.run.smithi112.stderr:new fs with metadata pool 5 and data pool 6 2024-09-21T00:26:36.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:36.822+0000 7fc9cc08b640 1 -- 172.21.15.112:0/1967029064 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fc9a8078bc0 msgr2=0x7fc9a807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:36.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:36.822+0000 7fc9cc08b640 1 --2- 172.21.15.112:0/1967029064 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fc9a8078bc0 0x7fc9a807b0d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc9c415ce60 tx=0x7fc9b4009040 comp rx=0 tx=0).stop 2024-09-21T00:26:36.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:36.822+0000 7fc9cc08b640 1 -- 172.21.15.112:0/1967029064 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc9c406ddf0 msgr2=0x7fc9c40ad860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:36.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:36.822+0000 7fc9cc08b640 1 --2- 172.21.15.112:0/1967029064 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc9c406ddf0 0x7fc9c40ad860 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fc9c0009cd0 tx=0x7fc9c0032d10 comp rx=0 tx=0).stop 2024-09-21T00:26:36.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:36.822+0000 7fc9cc08b640 1 -- 172.21.15.112:0/1967029064 shutdown_connections 2024-09-21T00:26:36.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:36.822+0000 7fc9cc08b640 1 --2- 172.21.15.112:0/1967029064 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc9c415b730 0x7fc9c415d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:36.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:36.822+0000 7fc9cc08b640 1 --2- 172.21.15.112:0/1967029064 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc9c40b6290 0x7fc9c41f5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:36.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:36.822+0000 7fc9cc08b640 1 --2- 172.21.15.112:0/1967029064 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fc9a8078bc0 0x7fc9a807b0d0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:36.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:36.822+0000 7fc9cc08b640 1 --2- 172.21.15.112:0/1967029064 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc9c406ddf0 0x7fc9c40ad860 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:36.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:36.822+0000 7fc9cc08b640 1 -- 172.21.15.112:0/1967029064 >> 172.21.15.112:0/1967029064 conn(0x7fc9c40735d0 msgr2=0x7fc9c40b0350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:36.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:36.823+0000 7fc9cc08b640 1 -- 172.21.15.112:0/1967029064 shutdown_connections 2024-09-21T00:26:36.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:36.823+0000 7fc9cc08b640 1 -- 172.21.15.112:0/1967029064 wait complete. 2024-09-21T00:26:36.859 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T00:26:36.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:36.971+0000 7f1cb1284640 1 -- 172.21.15.112:0/320707141 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1cac15f970 msgr2=0x7f1cac0ce6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:36.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:36.971+0000 7f1cb1284640 1 --2- 172.21.15.112:0/320707141 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1cac15f970 0x7f1cac0ce6d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1c94005fd0 tx=0x7f1c9402a5d0 comp rx=0 tx=0).stop 2024-09-21T00:26:36.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:36.971+0000 7f1cb1284640 1 -- 172.21.15.112:0/320707141 shutdown_connections 2024-09-21T00:26:36.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:36.971+0000 7f1cb1284640 1 --2- 172.21.15.112:0/320707141 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1cac15f970 0x7f1cac0ce6d0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:36.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:36.971+0000 7f1cb1284640 1 --2- 172.21.15.112:0/320707141 >> v2:172.21.15.187:3300/0 conn(0x7f1cac074210 0x7f1cac158f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:36.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:36.971+0000 7f1cb1284640 1 -- 172.21.15.112:0/320707141 >> 172.21.15.112:0/320707141 conn(0x7f1cac073560 msgr2=0x7f1cac073970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:36.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:36.971+0000 7f1cb1284640 1 -- 172.21.15.112:0/320707141 shutdown_connections 2024-09-21T00:26:36.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:36.971+0000 7f1cb1284640 1 -- 172.21.15.112:0/320707141 wait complete. 2024-09-21T00:26:36.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:36.973+0000 7f1cb1284640 1 Processor -- start 2024-09-21T00:26:36.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:36.973+0000 7f1cb1284640 1 -- start start 2024-09-21T00:26:36.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:36.974+0000 7f1cb1284640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1cac074210 0x7f1cac1f6530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:36.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:36.974+0000 7f1cb1284640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1cac15bfa0 0x7f1cac157230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:36.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:36.974+0000 7f1cb1284640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1cac15f970 0x7f1cac06cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:36.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:36.974+0000 7f1cb1284640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1cac06ddf0 con 0x7f1cac15bfa0 2024-09-21T00:26:36.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:36.974+0000 7f1cb1284640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f1cac06df70 con 0x7f1cac074210 2024-09-21T00:26:36.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:36.974+0000 7f1cb1284640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f1cac06e0f0 con 0x7f1cac15f970 2024-09-21T00:26:36.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:36.974+0000 7f1caa7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1cac15bfa0 0x7f1cac157230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:36.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:36.974+0000 7f1caa7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1cac15bfa0 0x7f1cac157230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36698/0 (socket says 172.21.15.112:36698) 2024-09-21T00:26:36.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:36.974+0000 7f1caa7fc640 1 -- 172.21.15.112:0/178933479 learned_addr learned my addr 172.21.15.112:0/178933479 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:26:36.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:36.974+0000 7f1cab7fe640 1 --2- 172.21.15.112:0/178933479 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1cac15f970 0x7f1cac06cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:36.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:36.974+0000 7f1caaffd640 1 --2- 172.21.15.112:0/178933479 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1cac074210 0x7f1cac1f6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:36.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:36.974+0000 7f1caa7fc640 1 -- 172.21.15.112:0/178933479 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1cac15f970 msgr2=0x7f1cac06cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:36.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:36.974+0000 7f1caa7fc640 1 --2- 172.21.15.112:0/178933479 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1cac15f970 0x7f1cac06cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:36.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:36.974+0000 7f1caa7fc640 1 -- 172.21.15.112:0/178933479 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1cac074210 msgr2=0x7f1cac1f6530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:36.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:36.974+0000 7f1caa7fc640 1 --2- 172.21.15.112:0/178933479 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1cac074210 0x7f1cac1f6530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:36.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:36.974+0000 7f1caa7fc640 1 -- 172.21.15.112:0/178933479 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cac1e98a0 con 0x7f1cac15bfa0 2024-09-21T00:26:36.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:36.975+0000 7f1caa7fc640 1 --2- 172.21.15.112:0/178933479 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1cac15bfa0 0x7f1cac157230 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f1c94034040 tx=0x7f1c940035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:36.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:36.975+0000 7f1ca37fe640 1 -- 172.21.15.112:0/178933479 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c9403e030 con 0x7f1cac15bfa0 2024-09-21T00:26:36.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:36.975+0000 7f1ca37fe640 1 -- 172.21.15.112:0/178933479 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1c94003ae0 con 0x7f1cac15bfa0 2024-09-21T00:26:36.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:36.975+0000 7f1cb1284640 1 -- 172.21.15.112:0/178933479 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cac1e9b80 con 0x7f1cac15bfa0 2024-09-21T00:26:36.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:36.975+0000 7f1ca37fe640 1 -- 172.21.15.112:0/178933479 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c94035040 con 0x7f1cac15bfa0 2024-09-21T00:26:36.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:36.975+0000 7f1cb1284640 1 -- 172.21.15.112:0/178933479 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cac0b5860 con 0x7f1cac15bfa0 2024-09-21T00:26:36.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:36.976+0000 7f1cb1284640 1 -- 172.21.15.112:0/178933479 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cac1e9dc0 con 0x7f1cac15bfa0 2024-09-21T00:26:36.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:36.977+0000 7f1ca37fe640 1 -- 172.21.15.112:0/178933479 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1c94041030 con 0x7f1cac15bfa0 2024-09-21T00:26:36.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:36.977+0000 7f1ca37fe640 1 --2- 172.21.15.112:0/178933479 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1c8c078b90 0x7f1c8c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:36.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:36.977+0000 7f1ca37fe640 1 -- 172.21.15.112:0/178933479 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6101+0+0 (secure 0 0 0) 0x7f1c940bb3a0 con 0x7f1cac15bfa0 2024-09-21T00:26:36.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:36.977+0000 7f1caaffd640 1 --2- 172.21.15.112:0/178933479 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1c8c078b90 0x7f1c8c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:36.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:36.978+0000 7f1caaffd640 1 --2- 172.21.15.112:0/178933479 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1c8c078b90 0x7f1c8c07b0a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f1c9c005fd0 tx=0x7f1c9c009500 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:36.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:36.982+0000 7f1ca37fe640 1 -- 172.21.15.112:0/178933479 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1c94082f80 con 0x7f1cac15bfa0 2024-09-21T00:26:37.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:37.159+0000 7f1cb1284640 1 -- 172.21.15.112:0/178933479 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1cac0d0310 con 0x7f1cac15bfa0 2024-09-21T00:26:37.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:37.160+0000 7f1ca37fe640 1 -- 172.21.15.112:0/178933479 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) ==== 74+0+17039 (secure 0 0 0) 0x7f1cac0d0310 con 0x7f1cac15bfa0 2024-09-21T00:26:37.162 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:26:37.163 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":37,"fsid":"8b34692b-dc65-49b2-a08f-26f707b96ac5","created":"2024-09-21T00:25:22.788490+0000","modified":"2024-09-21T00:26:36.187804+0000","last_up_change":"2024-09-21T00:25:36.161835+0000","last_in_change":"2024-09-21T00:25:26.346500+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T00:25:37.105006+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-09-21T00:26:34.140543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-09-21T00:26:35.143821+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9c575086-67d2-4de2-b01b-d44dcc21d604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6809","nonce":755760621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6811","nonce":755760621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6815","nonce":755760621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6813","nonce":755760621}]},"public_addr":"172.21.15.112:6809/755760621","cluster_addr":"172.21.15.112:6811/755760621","heartbeat_back_addr":"172.21.15.112:6815/755760621","heartbeat_front_addr":"172.21.15.112:6813/755760621","state":["exists","up"]},{"osd":1,"uuid":"a983c227-a3ce-4bdd-82f1-438ba7e3ecd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6801","nonce":344932592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6803","nonce":344932592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6807","nonce":344932592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6805","nonce":344932592}]},"public_addr":"172.21.15.112:6801/344932592","cluster_addr":"172.21.15.112:6803/344932592","heartbeat_back_addr":"172.21.15.112:6807/344932592","heartbeat_front_addr":"172.21.15.112:6805/344932592","state":["exists","up"]},{"osd":2,"uuid":"1193c1d8-bfff-4a57-8200-5f69bf1ea3af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6825","nonce":407933616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6827","nonce":407933616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6831","nonce":407933616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6829","nonce":407933616}]},"public_addr":"172.21.15.112:6825/407933616","cluster_addr":"172.21.15.112:6827/407933616","heartbeat_back_addr":"172.21.15.112:6831/407933616","heartbeat_front_addr":"172.21.15.112:6829/407933616","state":["exists","up"]},{"osd":3,"uuid":"af04330a-36ad-40cc-a8dd-fa259b221de5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6817","nonce":4120164806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6819","nonce":4120164806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6823","nonce":4120164806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6821","nonce":4120164806}]},"public_addr":"172.21.15.112:6817/4120164806","cluster_addr":"172.21.15.112:6819/4120164806","heartbeat_back_addr":"172.21.15.112:6823/4120164806","heartbeat_front_addr":"172.21.15.112:6821/4120164806","state":["exists","up"]},{"osd":4,"uuid":"5b893111-292c-4c5a-b184-53ad38c7199e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6803","nonce":522077560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6805","nonce":522077560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6809","nonce":522077560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6807","nonce":522077560}]},"public_addr":"172.21.15.187:6803/522077560","cluster_addr":"172.21.15.187:6805/522077560","heartbeat_back_addr":"172.21.15.187:6809/522077560","heartbeat_front_addr":"172.21.15.187:6807/522077560","state":["exists","up"]},{"osd":5,"uuid":"c0b8e011-45fe-4c5e-925e-6be2a7f1868a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6827","nonce":3593144721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6829","nonce":3593144721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6833","nonce":3593144721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6831","nonce":3593144721}]},"public_addr":"172.21.15.187:6827/3593144721","cluster_addr":"172.21.15.187:6829/3593144721","heartbeat_back_addr":"172.21.15.187:6833/3593144721","heartbeat_front_addr":"172.21.15.187:6831/3593144721","state":["exists","up"]},{"osd":6,"uuid":"87d2d650-7489-4593-98a3-a1f433ce8dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6811","nonce":1494387657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6813","nonce":1494387657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6817","nonce":1494387657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6815","nonce":1494387657}]},"public_addr":"172.21.15.187:6811/1494387657","cluster_addr":"172.21.15.187:6813/1494387657","heartbeat_back_addr":"172.21.15.187:6817/1494387657","heartbeat_front_addr":"172.21.15.187:6815/1494387657","state":["exists","up"]},{"osd":7,"uuid":"c6b4f8e2-a1ee-4438-aa37-67ff4b6ef1f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6819","nonce":1624699842}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6821","nonce":1624699842}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6825","nonce":1624699842}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6823","nonce":1624699842}]},"public_addr":"172.21.15.187:6819/1624699842","cluster_addr":"172.21.15.187:6821/1624699842","heartbeat_back_addr":"172.21.15.187:6825/1624699842","heartbeat_front_addr":"172.21.15.187:6823/1624699842","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.873162+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.400506+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.664317+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.159791+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.202630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.113212+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.384786+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.448441+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T00:26:37.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:37.163+0000 7f1cb1284640 1 -- 172.21.15.112:0/178933479 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1c8c078b90 msgr2=0x7f1c8c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:37.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:37.163+0000 7f1cb1284640 1 --2- 172.21.15.112:0/178933479 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1c8c078b90 0x7f1c8c07b0a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f1c9c005fd0 tx=0x7f1c9c009500 comp rx=0 tx=0).stop 2024-09-21T00:26:37.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:37.163+0000 7f1cb1284640 1 -- 172.21.15.112:0/178933479 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1cac15bfa0 msgr2=0x7f1cac157230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:37.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:37.163+0000 7f1cb1284640 1 --2- 172.21.15.112:0/178933479 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1cac15bfa0 0x7f1cac157230 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f1c94034040 tx=0x7f1c940035f0 comp rx=0 tx=0).stop 2024-09-21T00:26:37.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:37.163+0000 7f1cb1284640 1 -- 172.21.15.112:0/178933479 shutdown_connections 2024-09-21T00:26:37.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:37.163+0000 7f1cb1284640 1 --2- 172.21.15.112:0/178933479 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1cac15f970 0x7f1cac06cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:37.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:37.163+0000 7f1cb1284640 1 --2- 172.21.15.112:0/178933479 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1c8c078b90 0x7f1c8c07b0a0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:37.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:37.163+0000 7f1cb1284640 1 --2- 172.21.15.112:0/178933479 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1cac15bfa0 0x7f1cac157230 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:37.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:37.163+0000 7f1cb1284640 1 --2- 172.21.15.112:0/178933479 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1cac074210 0x7f1cac1f6530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:37.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:37.163+0000 7f1cb1284640 1 -- 172.21.15.112:0/178933479 >> 172.21.15.112:0/178933479 conn(0x7f1cac073560 msgr2=0x7f1cac0c7030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:37.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:37.163+0000 7f1cb1284640 1 -- 172.21.15.112:0/178933479 shutdown_connections 2024-09-21T00:26:37.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:37.163+0000 7f1cb1284640 1 -- 172.21.15.112:0/178933479 wait complete. 2024-09-21T00:26:37.176 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T00:26:37.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:37.287+0000 7f80b5ddf640 1 -- 172.21.15.112:0/3912047186 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f80b015a6f0 msgr2=0x7f80b015e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:37.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:37.287+0000 7f80b5ddf640 1 --2- 172.21.15.112:0/3912047186 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f80b015a6f0 0x7f80b015e230 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f80a4009830 tx=0x7f80a402a510 comp rx=0 tx=0).stop 2024-09-21T00:26:37.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:37.287+0000 7f80b5ddf640 1 -- 172.21.15.112:0/3912047186 shutdown_connections 2024-09-21T00:26:37.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:37.287+0000 7f80b5ddf640 1 --2- 172.21.15.112:0/3912047186 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f80b015a6f0 0x7f80b015e230 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:37.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:37.287+0000 7f80b5ddf640 1 --2- 172.21.15.112:0/3912047186 >> v2:172.21.15.187:3300/0 conn(0x7f80b00d01f0 0x7f80b015daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:37.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:37.287+0000 7f80b5ddf640 1 -- 172.21.15.112:0/3912047186 >> 172.21.15.112:0/3912047186 conn(0x7f80b0072990 msgr2=0x7f80b0072da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:37.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:37.287+0000 7f80b5ddf640 1 -- 172.21.15.112:0/3912047186 shutdown_connections 2024-09-21T00:26:37.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:37.287+0000 7f80b5ddf640 1 -- 172.21.15.112:0/3912047186 wait complete. 2024-09-21T00:26:37.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:37.289+0000 7f80b5ddf640 1 Processor -- start 2024-09-21T00:26:37.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:37.289+0000 7f80b5ddf640 1 -- start start 2024-09-21T00:26:37.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:37.290+0000 7f80b5ddf640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80b00d01f0 0x7f80b00c8080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:37.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:37.290+0000 7f80b5ddf640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f80b00d0980 0x7f80b00cafb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:37.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:37.290+0000 7f80b5ddf640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f80b015a6f0 0x7f80b0071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:37.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:37.290+0000 7f80b5ddf640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f80b00cd340 con 0x7f80b00d01f0 2024-09-21T00:26:37.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:37.290+0000 7f80b5ddf640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f80b00cd1c0 con 0x7f80b00d0980 2024-09-21T00:26:37.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:37.290+0000 7f80b5ddf640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f80b00cd4c0 con 0x7f80b015a6f0 2024-09-21T00:26:37.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:37.291+0000 7f80af7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80b00d01f0 0x7f80b00c8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:37.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:37.291+0000 7f80af7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80b00d01f0 0x7f80b00c8080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36700/0 (socket says 172.21.15.112:36700) 2024-09-21T00:26:37.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:37.291+0000 7f80af7fe640 1 -- 172.21.15.112:0/3000639812 learned_addr learned my addr 172.21.15.112:0/3000639812 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:26:37.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:37.291+0000 7f80affff640 1 --2- 172.21.15.112:0/3000639812 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f80b015a6f0 0x7f80b0071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:37.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:37.291+0000 7f80aeffd640 1 --2- 172.21.15.112:0/3000639812 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f80b00d0980 0x7f80b00cafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:37.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:37.291+0000 7f80af7fe640 1 -- 172.21.15.112:0/3000639812 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f80b015a6f0 msgr2=0x7f80b0071190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:37.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:37.291+0000 7f80af7fe640 1 --2- 172.21.15.112:0/3000639812 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f80b015a6f0 0x7f80b0071190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:37.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:37.291+0000 7f80af7fe640 1 -- 172.21.15.112:0/3000639812 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f80b00d0980 msgr2=0x7f80b00cafb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:37.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:37.291+0000 7f80af7fe640 1 --2- 172.21.15.112:0/3000639812 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f80b00d0980 0x7f80b00cafb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:37.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:37.291+0000 7f80af7fe640 1 -- 172.21.15.112:0/3000639812 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80b00c85c0 con 0x7f80b00d01f0 2024-09-21T00:26:37.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:37.291+0000 7f80af7fe640 1 --2- 172.21.15.112:0/3000639812 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80b00d01f0 0x7f80b00c8080 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f80a000ea20 tx=0x7f80a000eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:37.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:37.291+0000 7f8097fff640 1 -- 172.21.15.112:0/3000639812 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a000ce40 con 0x7f80b00d01f0 2024-09-21T00:26:37.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:37.291+0000 7f80b5ddf640 1 -- 172.21.15.112:0/3000639812 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80b00cbce0 con 0x7f80b00d01f0 2024-09-21T00:26:37.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:37.291+0000 7f8097fff640 1 -- 172.21.15.112:0/3000639812 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f80a0004020 con 0x7f80b00d01f0 2024-09-21T00:26:37.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:37.291+0000 7f8097fff640 1 -- 172.21.15.112:0/3000639812 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a0004470 con 0x7f80b00d01f0 2024-09-21T00:26:37.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:37.291+0000 7f80b5ddf640 1 -- 172.21.15.112:0/3000639812 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80b00cc2a0 con 0x7f80b00d01f0 2024-09-21T00:26:37.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:37.292+0000 7f80b5ddf640 1 -- 172.21.15.112:0/3000639812 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80b006ac50 con 0x7f80b00d01f0 2024-09-21T00:26:37.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:37.293+0000 7f8097fff640 1 -- 172.21.15.112:0/3000639812 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f80a00035f0 con 0x7f80b00d01f0 2024-09-21T00:26:37.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:37.293+0000 7f8097fff640 1 --2- 172.21.15.112:0/3000639812 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f809c078b90 0x7f809c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:37.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:37.294+0000 7f8097fff640 1 -- 172.21.15.112:0/3000639812 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6101+0+0 (secure 0 0 0) 0x7f80a009b7f0 con 0x7f80b00d01f0 2024-09-21T00:26:37.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:37.294+0000 7f80aeffd640 1 --2- 172.21.15.112:0/3000639812 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f809c078b90 0x7f809c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:37.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:37.294+0000 7f80aeffd640 1 --2- 172.21.15.112:0/3000639812 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f809c078b90 0x7f809c07b0a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f80a4034000 tx=0x7f80a4004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:37.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:37.298+0000 7f8097fff640 1 -- 172.21.15.112:0/3000639812 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f80b006ac50 con 0x7f80b00d01f0 2024-09-21T00:26:37.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:37.473+0000 7f80b5ddf640 1 -- 172.21.15.112:0/3000639812 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f80b00cd9b0 con 0x7f80b00d01f0 2024-09-21T00:26:37.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:37.474+0000 7f8097fff640 1 -- 172.21.15.112:0/3000639812 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) ==== 74+0+17039 (secure 0 0 0) 0x7f80b00c85c0 con 0x7f80b00d01f0 2024-09-21T00:26:37.476 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:26:37.477 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":37,"fsid":"8b34692b-dc65-49b2-a08f-26f707b96ac5","created":"2024-09-21T00:25:22.788490+0000","modified":"2024-09-21T00:26:36.187804+0000","last_up_change":"2024-09-21T00:25:36.161835+0000","last_in_change":"2024-09-21T00:25:26.346500+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T00:25:37.105006+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-09-21T00:26:34.140543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-09-21T00:26:35.143821+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9c575086-67d2-4de2-b01b-d44dcc21d604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6809","nonce":755760621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6811","nonce":755760621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6815","nonce":755760621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6813","nonce":755760621}]},"public_addr":"172.21.15.112:6809/755760621","cluster_addr":"172.21.15.112:6811/755760621","heartbeat_back_addr":"172.21.15.112:6815/755760621","heartbeat_front_addr":"172.21.15.112:6813/755760621","state":["exists","up"]},{"osd":1,"uuid":"a983c227-a3ce-4bdd-82f1-438ba7e3ecd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6801","nonce":344932592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6803","nonce":344932592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6807","nonce":344932592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6805","nonce":344932592}]},"public_addr":"172.21.15.112:6801/344932592","cluster_addr":"172.21.15.112:6803/344932592","heartbeat_back_addr":"172.21.15.112:6807/344932592","heartbeat_front_addr":"172.21.15.112:6805/344932592","state":["exists","up"]},{"osd":2,"uuid":"1193c1d8-bfff-4a57-8200-5f69bf1ea3af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6825","nonce":407933616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6827","nonce":407933616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6831","nonce":407933616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6829","nonce":407933616}]},"public_addr":"172.21.15.112:6825/407933616","cluster_addr":"172.21.15.112:6827/407933616","heartbeat_back_addr":"172.21.15.112:6831/407933616","heartbeat_front_addr":"172.21.15.112:6829/407933616","state":["exists","up"]},{"osd":3,"uuid":"af04330a-36ad-40cc-a8dd-fa259b221de5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6817","nonce":4120164806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6819","nonce":4120164806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6823","nonce":4120164806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6821","nonce":4120164806}]},"public_addr":"172.21.15.112:6817/4120164806","cluster_addr":"172.21.15.112:6819/4120164806","heartbeat_back_addr":"172.21.15.112:6823/4120164806","heartbeat_front_addr":"172.21.15.112:6821/4120164806","state":["exists","up"]},{"osd":4,"uuid":"5b893111-292c-4c5a-b184-53ad38c7199e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6803","nonce":522077560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6805","nonce":522077560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6809","nonce":522077560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6807","nonce":522077560}]},"public_addr":"172.21.15.187:6803/522077560","cluster_addr":"172.21.15.187:6805/522077560","heartbeat_back_addr":"172.21.15.187:6809/522077560","heartbeat_front_addr":"172.21.15.187:6807/522077560","state":["exists","up"]},{"osd":5,"uuid":"c0b8e011-45fe-4c5e-925e-6be2a7f1868a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6827","nonce":3593144721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6829","nonce":3593144721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6833","nonce":3593144721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6831","nonce":3593144721}]},"public_addr":"172.21.15.187:6827/3593144721","cluster_addr":"172.21.15.187:6829/3593144721","heartbeat_back_addr":"172.21.15.187:6833/3593144721","heartbeat_front_addr":"172.21.15.187:6831/3593144721","state":["exists","up"]},{"osd":6,"uuid":"87d2d650-7489-4593-98a3-a1f433ce8dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6811","nonce":1494387657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6813","nonce":1494387657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6817","nonce":1494387657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6815","nonce":1494387657}]},"public_addr":"172.21.15.187:6811/1494387657","cluster_addr":"172.21.15.187:6813/1494387657","heartbeat_back_addr":"172.21.15.187:6817/1494387657","heartbeat_front_addr":"172.21.15.187:6815/1494387657","state":["exists","up"]},{"osd":7,"uuid":"c6b4f8e2-a1ee-4438-aa37-67ff4b6ef1f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6819","nonce":1624699842}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6821","nonce":1624699842}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6825","nonce":1624699842}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6823","nonce":1624699842}]},"public_addr":"172.21.15.187:6819/1624699842","cluster_addr":"172.21.15.187:6821/1624699842","heartbeat_back_addr":"172.21.15.187:6825/1624699842","heartbeat_front_addr":"172.21.15.187:6823/1624699842","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.873162+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.400506+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.664317+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.159791+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.202630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.113212+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.384786+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.448441+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T00:26:37.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:37.477+0000 7f80b5ddf640 1 -- 172.21.15.112:0/3000639812 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f809c078b90 msgr2=0x7f809c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:37.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:37.477+0000 7f80b5ddf640 1 --2- 172.21.15.112:0/3000639812 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f809c078b90 0x7f809c07b0a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f80a4034000 tx=0x7f80a4004730 comp rx=0 tx=0).stop 2024-09-21T00:26:37.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:37.477+0000 7f80b5ddf640 1 -- 172.21.15.112:0/3000639812 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80b00d01f0 msgr2=0x7f80b00c8080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:37.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:37.477+0000 7f80b5ddf640 1 --2- 172.21.15.112:0/3000639812 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80b00d01f0 0x7f80b00c8080 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f80a000ea20 tx=0x7f80a000eee0 comp rx=0 tx=0).stop 2024-09-21T00:26:37.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:37.477+0000 7f80b5ddf640 1 -- 172.21.15.112:0/3000639812 shutdown_connections 2024-09-21T00:26:37.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:37.477+0000 7f80b5ddf640 1 --2- 172.21.15.112:0/3000639812 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f80b015a6f0 0x7f80b0071190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:37.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:37.477+0000 7f80b5ddf640 1 --2- 172.21.15.112:0/3000639812 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f80b00d0980 0x7f80b00cafb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:37.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:37.477+0000 7f80b5ddf640 1 --2- 172.21.15.112:0/3000639812 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f809c078b90 0x7f809c07b0a0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:37.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:37.477+0000 7f80b5ddf640 1 --2- 172.21.15.112:0/3000639812 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80b00d01f0 0x7f80b00c8080 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:37.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:37.477+0000 7f80b5ddf640 1 -- 172.21.15.112:0/3000639812 >> 172.21.15.112:0/3000639812 conn(0x7f80b0072990 msgr2=0x7f80b0070100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:37.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:37.477+0000 7f80b5ddf640 1 -- 172.21.15.112:0/3000639812 shutdown_connections 2024-09-21T00:26:37.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:37.477+0000 7f80b5ddf640 1 -- 172.21.15.112:0/3000639812 wait complete. 2024-09-21T00:26:37.490 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-21T00:26:37.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:37.604+0000 7fdf425e6640 1 -- 172.21.15.112:0/687701494 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdf3c0ccc40 msgr2=0x7fdf3c15ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:37.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:37.604+0000 7fdf425e6640 1 --2- 172.21.15.112:0/687701494 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdf3c0ccc40 0x7fdf3c15ca90 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c005fd0 tx=0x7fdf2c02a540 comp rx=0 tx=0).stop 2024-09-21T00:26:37.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:37.604+0000 7fdf425e6640 1 -- 172.21.15.112:0/687701494 shutdown_connections 2024-09-21T00:26:37.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:37.604+0000 7fdf425e6640 1 --2- 172.21.15.112:0/687701494 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdf3c0d0d80 0x7fdf3c0b5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:37.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:37.604+0000 7fdf425e6640 1 --2- 172.21.15.112:0/687701494 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdf3c0ccc40 0x7fdf3c15ca90 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:37.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:37.604+0000 7fdf425e6640 1 -- 172.21.15.112:0/687701494 >> 172.21.15.112:0/687701494 conn(0x7fdf3c0b46e0 msgr2=0x7fdf3c0d23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:37.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:37.604+0000 7fdf425e6640 1 -- 172.21.15.112:0/687701494 shutdown_connections 2024-09-21T00:26:37.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:37.604+0000 7fdf425e6640 1 -- 172.21.15.112:0/687701494 wait complete. 2024-09-21T00:26:37.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:37.606+0000 7fdf425e6640 1 Processor -- start 2024-09-21T00:26:37.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:37.606+0000 7fdf425e6640 1 -- start start 2024-09-21T00:26:37.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:37.607+0000 7fdf425e6640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdf3c0ccc40 0x7fdf3c1f3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:37.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:37.607+0000 7fdf425e6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdf3c0cd440 0x7fdf3c1f5f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:37.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:37.607+0000 7fdf425e6640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdf3c0cb660 0x7fdf3c0cba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:37.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:37.607+0000 7fdf425e6640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fdf3c0b6650 con 0x7fdf3c0cd440 2024-09-21T00:26:37.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:37.607+0000 7fdf425e6640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fdf3c0b64d0 con 0x7fdf3c0ccc40 2024-09-21T00:26:37.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:37.607+0000 7fdf425e6640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fdf3c0b67d0 con 0x7fdf3c0cb660 2024-09-21T00:26:37.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:37.607+0000 7fdf3b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdf3c0cd440 0x7fdf3c1f5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:37.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:37.607+0000 7fdf3b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdf3c0cd440 0x7fdf3c1f5f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36716/0 (socket says 172.21.15.112:36716) 2024-09-21T00:26:37.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:37.607+0000 7fdf3b7fe640 1 -- 172.21.15.112:0/2988916047 learned_addr learned my addr 172.21.15.112:0/2988916047 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:26:37.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:37.608+0000 7fdf3bfff640 1 --2- 172.21.15.112:0/2988916047 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdf3c0ccc40 0x7fdf3c1f3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:37.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:37.608+0000 7fdf40b83640 1 --2- 172.21.15.112:0/2988916047 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdf3c0cb660 0x7fdf3c0cba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:37.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:37.608+0000 7fdf3b7fe640 1 -- 172.21.15.112:0/2988916047 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdf3c0cb660 msgr2=0x7fdf3c0cba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:37.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:37.608+0000 7fdf3b7fe640 1 --2- 172.21.15.112:0/2988916047 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdf3c0cb660 0x7fdf3c0cba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:37.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:37.608+0000 7fdf3b7fe640 1 -- 172.21.15.112:0/2988916047 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdf3c0ccc40 msgr2=0x7fdf3c1f3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:37.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:37.608+0000 7fdf3b7fe640 1 --2- 172.21.15.112:0/2988916047 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdf3c0ccc40 0x7fdf3c1f3bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:37.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:37.608+0000 7fdf3b7fe640 1 -- 172.21.15.112:0/2988916047 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf3c0a9e80 con 0x7fdf3c0cd440 2024-09-21T00:26:37.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:37.608+0000 7fdf3b7fe640 1 --2- 172.21.15.112:0/2988916047 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdf3c0cd440 0x7fdf3c1f5f80 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fdf3000c3c0 tx=0x7fdf3000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:37.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:37.608+0000 7fdf38ff9640 1 -- 172.21.15.112:0/2988916047 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf30017070 con 0x7fdf3c0cd440 2024-09-21T00:26:37.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:37.608+0000 7fdf38ff9640 1 -- 172.21.15.112:0/2988916047 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdf300053e0 con 0x7fdf3c0cd440 2024-09-21T00:26:37.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:37.608+0000 7fdf425e6640 1 -- 172.21.15.112:0/2988916047 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf3c1f4130 con 0x7fdf3c0cd440 2024-09-21T00:26:37.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:37.608+0000 7fdf38ff9640 1 -- 172.21.15.112:0/2988916047 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf30003870 con 0x7fdf3c0cd440 2024-09-21T00:26:37.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:37.608+0000 7fdf425e6640 1 -- 172.21.15.112:0/2988916047 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf3c1f64c0 con 0x7fdf3c0cd440 2024-09-21T00:26:37.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:37.609+0000 7fdf425e6640 1 -- 172.21.15.112:0/2988916047 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf3c15ccc0 con 0x7fdf3c0cd440 2024-09-21T00:26:37.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:37.610+0000 7fdf38ff9640 1 -- 172.21.15.112:0/2988916047 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fdf3001a070 con 0x7fdf3c0cd440 2024-09-21T00:26:37.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:37.610+0000 7fdf38ff9640 1 --2- 172.21.15.112:0/2988916047 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fdf28078b90 0x7fdf2807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:37.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:37.610+0000 7fdf38ff9640 1 -- 172.21.15.112:0/2988916047 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6101+0+0 (secure 0 0 0) 0x7fdf3009a100 con 0x7fdf3c0cd440 2024-09-21T00:26:37.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:37.611+0000 7fdf3bfff640 1 --2- 172.21.15.112:0/2988916047 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fdf28078b90 0x7fdf2807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:37.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:37.611+0000 7fdf3bfff640 1 --2- 172.21.15.112:0/2988916047 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fdf28078b90 0x7fdf2807b0a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c033000 tx=0x7fdf2c004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:37.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:37.615+0000 7fdf38ff9640 1 -- 172.21.15.112:0/2988916047 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fdf3c1f4130 con 0x7fdf3c0cd440 2024-09-21T00:26:37.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:37.829+0000 7fdf425e6640 1 -- 172.21.15.112:0/2988916047 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fdf3c15cf50 con 0x7fdf3c0cd440 2024-09-21T00:26:38.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:38.578+0000 7fdf38ff9640 1 -- 172.21.15.112:0/2988916047 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) ==== 118+0+0 (secure 0 0 0) 0x7fdf30061f40 con 0x7fdf3c0cd440 2024-09-21T00:26:38.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:38.581+0000 7fdf425e6640 1 -- 172.21.15.112:0/2988916047 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fdf28078b90 msgr2=0x7fdf2807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:38.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:38.581+0000 7fdf425e6640 1 --2- 172.21.15.112:0/2988916047 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fdf28078b90 0x7fdf2807b0a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c033000 tx=0x7fdf2c004730 comp rx=0 tx=0).stop 2024-09-21T00:26:38.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:38.581+0000 7fdf425e6640 1 -- 172.21.15.112:0/2988916047 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdf3c0cd440 msgr2=0x7fdf3c1f5f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:38.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:38.581+0000 7fdf425e6640 1 --2- 172.21.15.112:0/2988916047 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdf3c0cd440 0x7fdf3c1f5f80 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fdf3000c3c0 tx=0x7fdf3000c880 comp rx=0 tx=0).stop 2024-09-21T00:26:38.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:38.581+0000 7fdf425e6640 1 -- 172.21.15.112:0/2988916047 shutdown_connections 2024-09-21T00:26:38.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:38.581+0000 7fdf425e6640 1 --2- 172.21.15.112:0/2988916047 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdf3c0cb660 0x7fdf3c0cba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:38.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:38.581+0000 7fdf425e6640 1 --2- 172.21.15.112:0/2988916047 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fdf28078b90 0x7fdf2807b0a0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:38.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:38.581+0000 7fdf425e6640 1 --2- 172.21.15.112:0/2988916047 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdf3c0cd440 0x7fdf3c1f5f80 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:38.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:38.581+0000 7fdf425e6640 1 --2- 172.21.15.112:0/2988916047 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdf3c0ccc40 0x7fdf3c1f3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:38.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:38.581+0000 7fdf425e6640 1 -- 172.21.15.112:0/2988916047 >> 172.21.15.112:0/2988916047 conn(0x7fdf3c0b46e0 msgr2=0x7fdf3c0c8970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:38.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:38.582+0000 7fdf425e6640 1 -- 172.21.15.112:0/2988916047 shutdown_connections 2024-09-21T00:26:38.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:38.582+0000 7fdf425e6640 1 -- 172.21.15.112:0/2988916047 wait complete. 2024-09-21T00:26:38.597 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-21T00:26:38.597 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-21T00:26:38.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:38.717+0000 7f73de256640 1 -- 172.21.15.112:0/3607542159 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73d80d0bb0 msgr2=0x7f73d8070100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:38.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:38.717+0000 7f73de256640 1 --2- 172.21.15.112:0/3607542159 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73d80d0bb0 0x7f73d8070100 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f73d8071cd0 tx=0x7f73d002a980 comp rx=0 tx=0).stop 2024-09-21T00:26:38.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:38.717+0000 7f73de256640 1 -- 172.21.15.112:0/3607542159 shutdown_connections 2024-09-21T00:26:38.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:38.717+0000 7f73de256640 1 --2- 172.21.15.112:0/3607542159 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73d80d0bb0 0x7f73d8070100 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:38.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:38.717+0000 7f73de256640 1 --2- 172.21.15.112:0/3607542159 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f73d80cd440 0x7f73d80d0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:38.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:38.717+0000 7f73de256640 1 --2- 172.21.15.112:0/3607542159 >> v2:172.21.15.187:3300/0 conn(0x7f73d80ccc40 0x7f73d815ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:38.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:38.717+0000 7f73de256640 1 -- 172.21.15.112:0/3607542159 >> 172.21.15.112:0/3607542159 conn(0x7f73d80b46e0 msgr2=0x7f73d80d23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:38.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:38.717+0000 7f73de256640 1 -- 172.21.15.112:0/3607542159 shutdown_connections 2024-09-21T00:26:38.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:38.717+0000 7f73de256640 1 -- 172.21.15.112:0/3607542159 wait complete. 2024-09-21T00:26:38.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:38.719+0000 7f73de256640 1 Processor -- start 2024-09-21T00:26:38.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:38.719+0000 7f73de256640 1 -- start start 2024-09-21T00:26:38.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:38.720+0000 7f73de256640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73d80ccc40 0x7f73d80cc6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:38.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:38.720+0000 7f73de256640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f73d80cd440 0x7f73d81f6b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:38.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:38.720+0000 7f73de256640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f73d80d0bb0 0x7f73d80b0920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:38.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:38.720+0000 7f73de256640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f73d8070a10 con 0x7f73d80ccc40 2024-09-21T00:26:38.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:38.720+0000 7f73de256640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f73d8070890 con 0x7f73d80cd440 2024-09-21T00:26:38.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:38.720+0000 7f73de256640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f73d8070b90 con 0x7f73d80d0bb0 2024-09-21T00:26:38.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:38.720+0000 7f73d77fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73d80ccc40 0x7f73d80cc6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:38.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:38.720+0000 7f73d77fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73d80ccc40 0x7f73d80cc6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36726/0 (socket says 172.21.15.112:36726) 2024-09-21T00:26:38.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:38.720+0000 7f73d77fe640 1 -- 172.21.15.112:0/2349612396 learned_addr learned my addr 172.21.15.112:0/2349612396 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:26:38.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:38.720+0000 7f73d6ffd640 1 --2- 172.21.15.112:0/2349612396 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f73d80cd440 0x7f73d81f6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:38.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:38.720+0000 7f73d7fff640 1 --2- 172.21.15.112:0/2349612396 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f73d80d0bb0 0x7f73d80b0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:38.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:38.721+0000 7f73d77fe640 1 -- 172.21.15.112:0/2349612396 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f73d80d0bb0 msgr2=0x7f73d80b0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:38.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:38.721+0000 7f73d77fe640 1 --2- 172.21.15.112:0/2349612396 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f73d80d0bb0 0x7f73d80b0920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:38.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:38.721+0000 7f73d77fe640 1 -- 172.21.15.112:0/2349612396 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f73d80cd440 msgr2=0x7f73d81f6b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:38.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:38.721+0000 7f73d77fe640 1 --2- 172.21.15.112:0/2349612396 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f73d80cd440 0x7f73d81f6b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:38.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:38.721+0000 7f73d77fe640 1 -- 172.21.15.112:0/2349612396 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73d80ca680 con 0x7f73d80ccc40 2024-09-21T00:26:38.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:38.721+0000 7f73d6ffd640 1 --2- 172.21.15.112:0/2349612396 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f73d80cd440 0x7f73d81f6b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:26:38.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:38.721+0000 7f73d77fe640 1 --2- 172.21.15.112:0/2349612396 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73d80ccc40 0x7f73d80cc6b0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f73c800b410 tx=0x7f73c800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:38.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:38.721+0000 7f73bffff640 1 -- 172.21.15.112:0/2349612396 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73c8015020 con 0x7f73d80ccc40 2024-09-21T00:26:38.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:38.721+0000 7f73bffff640 1 -- 172.21.15.112:0/2349612396 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f73c8011940 con 0x7f73d80ccc40 2024-09-21T00:26:38.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:38.721+0000 7f73de256640 1 -- 172.21.15.112:0/2349612396 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73d80ca960 con 0x7f73d80ccc40 2024-09-21T00:26:38.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:38.721+0000 7f73bffff640 1 -- 172.21.15.112:0/2349612396 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73c8011d90 con 0x7f73d80ccc40 2024-09-21T00:26:38.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:38.721+0000 7f73de256640 1 -- 172.21.15.112:0/2349612396 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73d80a9610 con 0x7f73d80ccc40 2024-09-21T00:26:38.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:38.722+0000 7f73de256640 1 -- 172.21.15.112:0/2349612396 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73d80d0670 con 0x7f73d80ccc40 2024-09-21T00:26:38.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:38.723+0000 7f73bffff640 1 -- 172.21.15.112:0/2349612396 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f73c8018020 con 0x7f73d80ccc40 2024-09-21T00:26:38.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:38.723+0000 7f73bffff640 1 --2- 172.21.15.112:0/2349612396 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f73c4078b90 0x7f73c407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:38.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:38.724+0000 7f73bffff640 1 -- 172.21.15.112:0/2349612396 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f73c809ade0 con 0x7f73d80ccc40 2024-09-21T00:26:38.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:38.724+0000 7f73d6ffd640 1 --2- 172.21.15.112:0/2349612396 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f73c4078b90 0x7f73c407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:38.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:38.724+0000 7f73d6ffd640 1 --2- 172.21.15.112:0/2349612396 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f73c4078b90 0x7f73c407b0a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f73cc005fd0 tx=0x7f73cc00f360 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:38.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:38.728+0000 7f73bffff640 1 -- 172.21.15.112:0/2349612396 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f73c801f020 con 0x7f73d80ccc40 2024-09-21T00:26:38.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:38.946+0000 7f73de256640 1 -- 172.21.15.112:0/2349612396 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f73d80c5cb0 con 0x7f73d80ccc40 2024-09-21T00:26:39.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:39.846+0000 7f73bffff640 1 -- 172.21.15.112:0/2349612396 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v17) ==== 122+0+0 (secure 0 0 0) 0x7f73c8063120 con 0x7f73d80ccc40 2024-09-21T00:26:39.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:39.849+0000 7f73de256640 1 -- 172.21.15.112:0/2349612396 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f73c4078b90 msgr2=0x7f73c407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:39.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:39.849+0000 7f73de256640 1 --2- 172.21.15.112:0/2349612396 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f73c4078b90 0x7f73c407b0a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f73cc005fd0 tx=0x7f73cc00f360 comp rx=0 tx=0).stop 2024-09-21T00:26:39.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:39.849+0000 7f73de256640 1 -- 172.21.15.112:0/2349612396 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73d80ccc40 msgr2=0x7f73d80cc6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:39.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:39.849+0000 7f73de256640 1 --2- 172.21.15.112:0/2349612396 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73d80ccc40 0x7f73d80cc6b0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f73c800b410 tx=0x7f73c800b8d0 comp rx=0 tx=0).stop 2024-09-21T00:26:39.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:39.849+0000 7f73de256640 1 -- 172.21.15.112:0/2349612396 shutdown_connections 2024-09-21T00:26:39.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:39.849+0000 7f73de256640 1 --2- 172.21.15.112:0/2349612396 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f73d80d0bb0 0x7f73d80b0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:39.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:39.849+0000 7f73de256640 1 --2- 172.21.15.112:0/2349612396 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f73d80cd440 0x7f73d81f6b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:39.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:39.849+0000 7f73de256640 1 --2- 172.21.15.112:0/2349612396 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f73c4078b90 0x7f73c407b0a0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:39.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:39.849+0000 7f73de256640 1 --2- 172.21.15.112:0/2349612396 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73d80ccc40 0x7f73d80cc6b0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:39.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:39.849+0000 7f73de256640 1 -- 172.21.15.112:0/2349612396 >> 172.21.15.112:0/2349612396 conn(0x7f73d80b46e0 msgr2=0x7f73d80d24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:39.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:39.849+0000 7f73de256640 1 -- 172.21.15.112:0/2349612396 shutdown_connections 2024-09-21T00:26:39.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:39.849+0000 7f73de256640 1 -- 172.21.15.112:0/2349612396 wait complete. 2024-09-21T00:26:39.864 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-21T00:26:39.864 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T00:26:39.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:39.982+0000 7f80f96c5640 1 -- 172.21.15.112:0/1965753285 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80dc004800 msgr2=0x7f80dc004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:39.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:39.982+0000 7f80f96c5640 1 --2- 172.21.15.112:0/1965753285 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80dc004800 0x7f80dc004c10 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f80e800c260 tx=0x7f80dc005830 comp rx=0 tx=0).stop 2024-09-21T00:26:39.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:39.982+0000 7f80f96c5640 1 -- 172.21.15.112:0/1965753285 shutdown_connections 2024-09-21T00:26:39.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:39.982+0000 7f80f96c5640 1 --2- 172.21.15.112:0/1965753285 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80dc004800 0x7f80dc004c10 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:39.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:39.982+0000 7f80f96c5640 1 -- 172.21.15.112:0/1965753285 >> 172.21.15.112:0/1965753285 conn(0x7f80f40d1140 msgr2=0x7f80f40d1550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:39.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:39.982+0000 7f80f96c5640 1 -- 172.21.15.112:0/1965753285 shutdown_connections 2024-09-21T00:26:39.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:39.982+0000 7f80f96c5640 1 -- 172.21.15.112:0/1965753285 wait complete. 2024-09-21T00:26:39.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:39.984+0000 7f80f96c5640 1 Processor -- start 2024-09-21T00:26:39.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:39.984+0000 7f80f96c5640 1 -- start start 2024-09-21T00:26:39.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:39.985+0000 7f80f96c5640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f80dc004800 0x7f80f40b6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:39.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:39.985+0000 7f80f96c5640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f80f40b6680 0x7f80f41e4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:39.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:39.985+0000 7f80f96c5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80f41e54c0 0x7f80f414b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:39.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:39.985+0000 7f80f96c5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f80f4070290 con 0x7f80f41e54c0 2024-09-21T00:26:39.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:39.985+0000 7f80f96c5640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f80f4070110 con 0x7f80f40b6680 2024-09-21T00:26:39.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:39.985+0000 7f80f96c5640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f80f4073ea0 con 0x7f80dc004800 2024-09-21T00:26:39.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:39.985+0000 7f80f37fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80f41e54c0 0x7f80f414b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:39.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:39.985+0000 7f80f37fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80f41e54c0 0x7f80f414b050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36742/0 (socket says 172.21.15.112:36742) 2024-09-21T00:26:39.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:39.985+0000 7f80f37fe640 1 -- 172.21.15.112:0/3773470947 learned_addr learned my addr 172.21.15.112:0/3773470947 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:26:39.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:39.985+0000 7f80f27fc640 1 --2- 172.21.15.112:0/3773470947 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f80f40b6680 0x7f80f41e4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:39.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:39.985+0000 7f80f2ffd640 1 --2- 172.21.15.112:0/3773470947 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f80dc004800 0x7f80f40b6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:39.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:39.986+0000 7f80f37fe640 1 -- 172.21.15.112:0/3773470947 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f80dc004800 msgr2=0x7f80f40b6140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:39.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:39.986+0000 7f80f37fe640 1 --2- 172.21.15.112:0/3773470947 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f80dc004800 0x7f80f40b6140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:39.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:39.986+0000 7f80f37fe640 1 -- 172.21.15.112:0/3773470947 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f80f40b6680 msgr2=0x7f80f41e4f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:39.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:39.986+0000 7f80f37fe640 1 --2- 172.21.15.112:0/3773470947 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f80f40b6680 0x7f80f41e4f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:39.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:39.986+0000 7f80f37fe640 1 -- 172.21.15.112:0/3773470947 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80f40b6a90 con 0x7f80f41e54c0 2024-09-21T00:26:39.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:39.986+0000 7f80f2ffd640 1 --2- 172.21.15.112:0/3773470947 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f80dc004800 0x7f80f40b6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:26:39.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:39.986+0000 7f80f37fe640 1 --2- 172.21.15.112:0/3773470947 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80f41e54c0 0x7f80f414b050 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f80f4073080 tx=0x7f80ec00d970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:39.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:39.986+0000 7f80db7fe640 1 -- 172.21.15.112:0/3773470947 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80ec00f880 con 0x7f80f41e54c0 2024-09-21T00:26:39.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:39.986+0000 7f80db7fe640 1 -- 172.21.15.112:0/3773470947 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f80ec017070 con 0x7f80f41e54c0 2024-09-21T00:26:39.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:39.986+0000 7f80f96c5640 1 -- 172.21.15.112:0/3773470947 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80f41e58d0 con 0x7f80f41e54c0 2024-09-21T00:26:39.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:39.986+0000 7f80db7fe640 1 -- 172.21.15.112:0/3773470947 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80ec00fdb0 con 0x7f80f41e54c0 2024-09-21T00:26:39.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:39.986+0000 7f80f96c5640 1 -- 172.21.15.112:0/3773470947 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80f409c810 con 0x7f80f41e54c0 2024-09-21T00:26:39.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:39.987+0000 7f80f96c5640 1 -- 172.21.15.112:0/3773470947 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80f41e5aa0 con 0x7f80f41e54c0 2024-09-21T00:26:39.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:39.988+0000 7f80db7fe640 1 -- 172.21.15.112:0/3773470947 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f80ec01c460 con 0x7f80f41e54c0 2024-09-21T00:26:39.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:39.988+0000 7f80db7fe640 1 --2- 172.21.15.112:0/3773470947 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f80e0078bc0 0x7f80e007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:39.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:39.988+0000 7f80db7fe640 1 -- 172.21.15.112:0/3773470947 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f80ec09b060 con 0x7f80f41e54c0 2024-09-21T00:26:39.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:39.989+0000 7f80f2ffd640 1 --2- 172.21.15.112:0/3773470947 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f80e0078bc0 0x7f80e007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:39.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:39.989+0000 7f80f2ffd640 1 --2- 172.21.15.112:0/3773470947 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f80e0078bc0 0x7f80e007b0d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f80e4003720 tx=0x7f80e400b260 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:39.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:39.993+0000 7f80db7fe640 1 -- 172.21.15.112:0/3773470947 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f80f40b6a90 con 0x7f80f41e54c0 2024-09-21T00:26:40.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:40.203+0000 7f80f96c5640 1 -- 172.21.15.112:0/3773470947 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f80f4074f20 con 0x7f80f41e54c0 2024-09-21T00:26:40.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:40.204+0000 7f80db7fe640 1 -- 172.21.15.112:0/3773470947 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5232 (secure 0 0 0) 0x7f80ec0633b0 con 0x7f80f41e54c0 2024-09-21T00:26:40.206 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:26:40.206 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":17,"btime":"2024-09-21T00:26:39:841719+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4846,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6834/2036740840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":2036740840},{"type":"v1","addr":"172.21.15.112:6834","nonce":2036740840}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4855,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6836/3699399640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":3699399640},{"type":"v1","addr":"172.21.15.187:6836","nonce":3699399640}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4858,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6837/694936803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6835","nonce":694936803},{"type":"v1","addr":"172.21.15.187:6837","nonce":694936803}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T00:26:36.187741+0000","modified":"2024-09-21T00:26:38.947874+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5028},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5028":{"gid":5028,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6835/4012316979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":4012316979},{"type":"v1","addr":"172.21.15.112:6835","nonce":4012316979}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5028,"qdb_cluster":[5028]},"id":2}]} 2024-09-21T00:26:40.206 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 17 2024-09-21T00:26:40.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:40.206+0000 7f80f96c5640 1 -- 172.21.15.112:0/3773470947 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f80e0078bc0 msgr2=0x7f80e007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:40.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:40.206+0000 7f80f96c5640 1 --2- 172.21.15.112:0/3773470947 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f80e0078bc0 0x7f80e007b0d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f80e4003720 tx=0x7f80e400b260 comp rx=0 tx=0).stop 2024-09-21T00:26:40.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:40.206+0000 7f80f96c5640 1 -- 172.21.15.112:0/3773470947 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80f41e54c0 msgr2=0x7f80f414b050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:40.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:40.206+0000 7f80f96c5640 1 --2- 172.21.15.112:0/3773470947 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80f41e54c0 0x7f80f414b050 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f80f4073080 tx=0x7f80ec00d970 comp rx=0 tx=0).stop 2024-09-21T00:26:40.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:40.206+0000 7f80f96c5640 1 -- 172.21.15.112:0/3773470947 shutdown_connections 2024-09-21T00:26:40.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:40.206+0000 7f80f96c5640 1 --2- 172.21.15.112:0/3773470947 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f80e0078bc0 0x7f80e007b0d0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:40.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:40.206+0000 7f80f96c5640 1 --2- 172.21.15.112:0/3773470947 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80f41e54c0 0x7f80f414b050 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:40.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:40.206+0000 7f80f96c5640 1 --2- 172.21.15.112:0/3773470947 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f80f40b6680 0x7f80f41e4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:40.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:40.206+0000 7f80f96c5640 1 --2- 172.21.15.112:0/3773470947 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f80dc004800 0x7f80f40b6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:40.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:40.206+0000 7f80f96c5640 1 -- 172.21.15.112:0/3773470947 >> 172.21.15.112:0/3773470947 conn(0x7f80f40d1140 msgr2=0x7f80f415e140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:40.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:40.206+0000 7f80f96c5640 1 -- 172.21.15.112:0/3773470947 shutdown_connections 2024-09-21T00:26:40.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:40.206+0000 7f80f96c5640 1 -- 172.21.15.112:0/3773470947 wait complete. 2024-09-21T00:26:40.220 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T00:26:40.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:40.331+0000 7fe8c01da640 1 -- 172.21.15.112:0/4170622444 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe8a4005150 msgr2=0x7fe8a400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:40.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:40.331+0000 7fe8c01da640 1 --2- 172.21.15.112:0/4170622444 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe8a4005150 0x7fe8a400b9c0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fe8b0001d90 tx=0x7fe8a4005830 comp rx=0 tx=0).stop 2024-09-21T00:26:40.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:40.331+0000 7fe8be777640 1 -- 172.21.15.112:0/4170622444 reap_dead start 2024-09-21T00:26:40.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:40.331+0000 7fe8c01da640 1 -- 172.21.15.112:0/4170622444 shutdown_connections 2024-09-21T00:26:40.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:40.331+0000 7fe8c01da640 1 -- 172.21.15.112:0/4170622444 >> 172.21.15.112:0/4170622444 conn(0x7fe8b80b6b20 msgr2=0x7fe8b80d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:40.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:40.331+0000 7fe8c01da640 1 -- 172.21.15.112:0/4170622444 shutdown_connections 2024-09-21T00:26:40.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:40.331+0000 7fe8c01da640 1 -- 172.21.15.112:0/4170622444 wait complete. 2024-09-21T00:26:40.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:40.333+0000 7fe8c01da640 1 Processor -- start 2024-09-21T00:26:40.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:40.333+0000 7fe8c01da640 1 -- start start 2024-09-21T00:26:40.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:40.334+0000 7fe8c01da640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe8a4005150 0x7fe8b80ca5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:40.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:40.334+0000 7fe8c01da640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe8b80b2590 0x7fe8b814c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:40.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:40.334+0000 7fe8c01da640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe8b80b29a0 0x7fe8b814ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:40.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:40.334+0000 7fe8c01da640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe8b815f640 con 0x7fe8b80b2590 2024-09-21T00:26:40.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:40.334+0000 7fe8c01da640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe8b815f4c0 con 0x7fe8b80b29a0 2024-09-21T00:26:40.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:40.334+0000 7fe8c01da640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe8b815f7c0 con 0x7fe8a4005150 2024-09-21T00:26:40.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:40.335+0000 7fe8bd775640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe8b80b2590 0x7fe8b814c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:40.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:40.335+0000 7fe8bd775640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe8b80b2590 0x7fe8b814c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36752/0 (socket says 172.21.15.112:36752) 2024-09-21T00:26:40.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:40.335+0000 7fe8bd775640 1 -- 172.21.15.112:0/1041944167 learned_addr learned my addr 172.21.15.112:0/1041944167 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:26:40.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:40.335+0000 7fe8be777640 1 --2- 172.21.15.112:0/1041944167 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe8b80b29a0 0x7fe8b814ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:40.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:40.335+0000 7fe8bdf76640 1 --2- 172.21.15.112:0/1041944167 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe8a4005150 0x7fe8b80ca5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:40.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:40.335+0000 7fe8bd775640 1 -- 172.21.15.112:0/1041944167 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe8a4005150 msgr2=0x7fe8b80ca5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:40.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:40.335+0000 7fe8bd775640 1 --2- 172.21.15.112:0/1041944167 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe8a4005150 0x7fe8b80ca5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:40.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:40.335+0000 7fe8bd775640 1 -- 172.21.15.112:0/1041944167 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe8b80b29a0 msgr2=0x7fe8b814ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:40.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:40.335+0000 7fe8bd775640 1 --2- 172.21.15.112:0/1041944167 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe8b80b29a0 0x7fe8b814ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:40.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:40.335+0000 7fe8bd775640 1 -- 172.21.15.112:0/1041944167 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8b80c9550 con 0x7fe8b80b2590 2024-09-21T00:26:40.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:40.335+0000 7fe8be777640 1 --2- 172.21.15.112:0/1041944167 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe8b80b29a0 0x7fe8b814ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:26:40.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:40.335+0000 7fe8bd775640 1 --2- 172.21.15.112:0/1041944167 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe8b80b2590 0x7fe8b814c4f0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fe8a8006630 tx=0x7fe8a80039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:40.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:40.335+0000 7fe8ae7fc640 1 -- 172.21.15.112:0/1041944167 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8a8015070 con 0x7fe8b80b2590 2024-09-21T00:26:40.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:40.335+0000 7fe8ae7fc640 1 -- 172.21.15.112:0/1041944167 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe8a800f960 con 0x7fe8b80b2590 2024-09-21T00:26:40.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:40.335+0000 7fe8ae7fc640 1 -- 172.21.15.112:0/1041944167 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8a800fdb0 con 0x7fe8b80b2590 2024-09-21T00:26:40.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:40.335+0000 7fe8c01da640 1 -- 172.21.15.112:0/1041944167 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8b80c9830 con 0x7fe8b80b2590 2024-09-21T00:26:40.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:40.335+0000 7fe8c01da640 1 -- 172.21.15.112:0/1041944167 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8b80c9df0 con 0x7fe8b80b2590 2024-09-21T00:26:40.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:40.336+0000 7fe8c01da640 1 -- 172.21.15.112:0/1041944167 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8b80b3250 con 0x7fe8b80b2590 2024-09-21T00:26:40.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:40.337+0000 7fe8ae7fc640 1 -- 172.21.15.112:0/1041944167 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe8a8011430 con 0x7fe8b80b2590 2024-09-21T00:26:40.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:40.338+0000 7fe8ae7fc640 1 --2- 172.21.15.112:0/1041944167 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fe89c078b90 0x7fe89c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:40.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:40.338+0000 7fe8ae7fc640 1 -- 172.21.15.112:0/1041944167 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7fe8a809bbc0 con 0x7fe8b80b2590 2024-09-21T00:26:40.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:40.338+0000 7fe8bdf76640 1 --2- 172.21.15.112:0/1041944167 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fe89c078b90 0x7fe89c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:40.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:40.338+0000 7fe8bdf76640 1 --2- 172.21.15.112:0/1041944167 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fe89c078b90 0x7fe89c07b0a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe8b4009aa0 tx=0x7fe8b4005f90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:40.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:40.342+0000 7fe8ae7fc640 1 -- 172.21.15.112:0/1041944167 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe8a8063f10 con 0x7fe8b80b2590 2024-09-21T00:26:40.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:40.521+0000 7fe8c01da640 1 -- 172.21.15.112:0/1041944167 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe8b80c92e0 con 0x7fe8b80b2590 2024-09-21T00:26:40.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:40.522+0000 7fe8ae7fc640 1 -- 172.21.15.112:0/1041944167 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) ==== 74+0+17026 (secure 0 0 0) 0x7fe8a8063810 con 0x7fe8b80b2590 2024-09-21T00:26:40.524 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:26:40.524 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":38,"fsid":"8b34692b-dc65-49b2-a08f-26f707b96ac5","created":"2024-09-21T00:25:22.788490+0000","modified":"2024-09-21T00:26:37.831810+0000","last_up_change":"2024-09-21T00:25:36.161835+0000","last_in_change":"2024-09-21T00:25:26.346500+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T00:25:37.105006+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-09-21T00:26:34.140543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-09-21T00:26:35.143821+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9c575086-67d2-4de2-b01b-d44dcc21d604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6809","nonce":755760621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6811","nonce":755760621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6815","nonce":755760621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6813","nonce":755760621}]},"public_addr":"172.21.15.112:6809/755760621","cluster_addr":"172.21.15.112:6811/755760621","heartbeat_back_addr":"172.21.15.112:6815/755760621","heartbeat_front_addr":"172.21.15.112:6813/755760621","state":["exists","up"]},{"osd":1,"uuid":"a983c227-a3ce-4bdd-82f1-438ba7e3ecd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6801","nonce":344932592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6803","nonce":344932592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6807","nonce":344932592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6805","nonce":344932592}]},"public_addr":"172.21.15.112:6801/344932592","cluster_addr":"172.21.15.112:6803/344932592","heartbeat_back_addr":"172.21.15.112:6807/344932592","heartbeat_front_addr":"172.21.15.112:6805/344932592","state":["exists","up"]},{"osd":2,"uuid":"1193c1d8-bfff-4a57-8200-5f69bf1ea3af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6825","nonce":407933616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6827","nonce":407933616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6831","nonce":407933616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6829","nonce":407933616}]},"public_addr":"172.21.15.112:6825/407933616","cluster_addr":"172.21.15.112:6827/407933616","heartbeat_back_addr":"172.21.15.112:6831/407933616","heartbeat_front_addr":"172.21.15.112:6829/407933616","state":["exists","up"]},{"osd":3,"uuid":"af04330a-36ad-40cc-a8dd-fa259b221de5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6817","nonce":4120164806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6819","nonce":4120164806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6823","nonce":4120164806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6821","nonce":4120164806}]},"public_addr":"172.21.15.112:6817/4120164806","cluster_addr":"172.21.15.112:6819/4120164806","heartbeat_back_addr":"172.21.15.112:6823/4120164806","heartbeat_front_addr":"172.21.15.112:6821/4120164806","state":["exists","up"]},{"osd":4,"uuid":"5b893111-292c-4c5a-b184-53ad38c7199e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6803","nonce":522077560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6805","nonce":522077560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6809","nonce":522077560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6807","nonce":522077560}]},"public_addr":"172.21.15.187:6803/522077560","cluster_addr":"172.21.15.187:6805/522077560","heartbeat_back_addr":"172.21.15.187:6809/522077560","heartbeat_front_addr":"172.21.15.187:6807/522077560","state":["exists","up"]},{"osd":5,"uuid":"c0b8e011-45fe-4c5e-925e-6be2a7f1868a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6827","nonce":3593144721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6829","nonce":3593144721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6833","nonce":3593144721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6831","nonce":3593144721}]},"public_addr":"172.21.15.187:6827/3593144721","cluster_addr":"172.21.15.187:6829/3593144721","heartbeat_back_addr":"172.21.15.187:6833/3593144721","heartbeat_front_addr":"172.21.15.187:6831/3593144721","state":["exists","up"]},{"osd":6,"uuid":"87d2d650-7489-4593-98a3-a1f433ce8dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6811","nonce":1494387657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6813","nonce":1494387657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6817","nonce":1494387657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6815","nonce":1494387657}]},"public_addr":"172.21.15.187:6811/1494387657","cluster_addr":"172.21.15.187:6813/1494387657","heartbeat_back_addr":"172.21.15.187:6817/1494387657","heartbeat_front_addr":"172.21.15.187:6815/1494387657","state":["exists","up"]},{"osd":7,"uuid":"c6b4f8e2-a1ee-4438-aa37-67ff4b6ef1f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6819","nonce":1624699842}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6821","nonce":1624699842}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6825","nonce":1624699842}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6823","nonce":1624699842}]},"public_addr":"172.21.15.187:6819/1624699842","cluster_addr":"172.21.15.187:6821/1624699842","heartbeat_back_addr":"172.21.15.187:6825/1624699842","heartbeat_front_addr":"172.21.15.187:6823/1624699842","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.873162+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.400506+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.664317+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.159791+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.202630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.113212+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.384786+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.448441+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T00:26:40.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:40.524+0000 7fe8c01da640 1 -- 172.21.15.112:0/1041944167 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fe89c078b90 msgr2=0x7fe89c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:40.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:40.524+0000 7fe8c01da640 1 --2- 172.21.15.112:0/1041944167 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fe89c078b90 0x7fe89c07b0a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe8b4009aa0 tx=0x7fe8b4005f90 comp rx=0 tx=0).stop 2024-09-21T00:26:40.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:40.524+0000 7fe8c01da640 1 -- 172.21.15.112:0/1041944167 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe8b80b2590 msgr2=0x7fe8b814c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:40.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:40.524+0000 7fe8c01da640 1 --2- 172.21.15.112:0/1041944167 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe8b80b2590 0x7fe8b814c4f0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fe8a8006630 tx=0x7fe8a80039c0 comp rx=0 tx=0).stop 2024-09-21T00:26:40.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:40.524+0000 7fe8c01da640 1 -- 172.21.15.112:0/1041944167 shutdown_connections 2024-09-21T00:26:40.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:40.524+0000 7fe8c01da640 1 --2- 172.21.15.112:0/1041944167 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe8b80b29a0 0x7fe8b814ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:40.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:40.524+0000 7fe8c01da640 1 --2- 172.21.15.112:0/1041944167 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fe89c078b90 0x7fe89c07b0a0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:40.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:40.524+0000 7fe8c01da640 1 --2- 172.21.15.112:0/1041944167 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe8b80b2590 0x7fe8b814c4f0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:40.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:40.524+0000 7fe8c01da640 1 --2- 172.21.15.112:0/1041944167 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe8a4005150 0x7fe8b80ca5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:40.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:40.524+0000 7fe8c01da640 1 -- 172.21.15.112:0/1041944167 >> 172.21.15.112:0/1041944167 conn(0x7fe8b80b6b20 msgr2=0x7fe8b806dca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:40.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:40.524+0000 7fe8c01da640 1 -- 172.21.15.112:0/1041944167 shutdown_connections 2024-09-21T00:26:40.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:40.524+0000 7fe8c01da640 1 -- 172.21.15.112:0/1041944167 wait complete. 2024-09-21T00:26:40.538 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T00:26:40.538 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T00:26:40.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:40.648+0000 7f27f3f21640 1 -- 172.21.15.112:0/1397568500 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f27ec06e330 msgr2=0x7f27ec0cdb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:40.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:40.648+0000 7f27f3f21640 1 --2- 172.21.15.112:0/1397568500 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f27ec06e330 0x7f27ec0cdb90 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f27e4005fd0 tx=0x7f27e402a8a0 comp rx=0 tx=0).stop 2024-09-21T00:26:40.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:40.649+0000 7f27f3f21640 1 -- 172.21.15.112:0/1397568500 shutdown_connections 2024-09-21T00:26:40.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:40.649+0000 7f27f3f21640 1 --2- 172.21.15.112:0/1397568500 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f27ec06e330 0x7f27ec0cdb90 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:40.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:40.649+0000 7f27f3f21640 1 --2- 172.21.15.112:0/1397568500 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f27ec070030 0x7f27ec06ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:40.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:40.649+0000 7f27f3f21640 1 -- 172.21.15.112:0/1397568500 >> 172.21.15.112:0/1397568500 conn(0x7f27ec0cca40 msgr2=0x7f27ec0cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:40.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:40.649+0000 7f27f3f21640 1 -- 172.21.15.112:0/1397568500 shutdown_connections 2024-09-21T00:26:40.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:40.649+0000 7f27f3f21640 1 -- 172.21.15.112:0/1397568500 wait complete. 2024-09-21T00:26:40.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:40.650+0000 7f27f3f21640 1 Processor -- start 2024-09-21T00:26:40.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:40.650+0000 7f27f3f21640 1 -- start start 2024-09-21T00:26:40.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:40.652+0000 7f27f3f21640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f27ec06e330 0x7f27ec0a0440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:40.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:40.652+0000 7f27f3f21640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f27ec070030 0x7f27ec09b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:40.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:40.652+0000 7f27f3f21640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f27ec074d90 0x7f27ec1f5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:40.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:40.652+0000 7f27f3f21640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f27ec071000 con 0x7f27ec074d90 2024-09-21T00:26:40.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:40.652+0000 7f27f3f21640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f27ec070e80 con 0x7f27ec070030 2024-09-21T00:26:40.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:40.652+0000 7f27f3f21640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f27ec0ce0d0 con 0x7f27ec06e330 2024-09-21T00:26:40.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:40.652+0000 7f27f24be640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f27ec074d90 0x7f27ec1f5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:40.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:40.652+0000 7f27f24be640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f27ec074d90 0x7f27ec1f5900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36774/0 (socket says 172.21.15.112:36774) 2024-09-21T00:26:40.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:40.652+0000 7f27f24be640 1 -- 172.21.15.112:0/1952621220 learned_addr learned my addr 172.21.15.112:0/1952621220 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:26:40.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:40.652+0000 7f27f1cbd640 1 --2- 172.21.15.112:0/1952621220 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f27ec06e330 0x7f27ec0a0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:40.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:40.652+0000 7f27f14bc640 1 --2- 172.21.15.112:0/1952621220 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f27ec070030 0x7f27ec09b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:40.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:40.652+0000 7f27f24be640 1 -- 172.21.15.112:0/1952621220 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f27ec06e330 msgr2=0x7f27ec0a0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:40.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:40.652+0000 7f27f24be640 1 --2- 172.21.15.112:0/1952621220 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f27ec06e330 0x7f27ec0a0440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:40.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:40.652+0000 7f27f24be640 1 -- 172.21.15.112:0/1952621220 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f27ec070030 msgr2=0x7f27ec09b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:40.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:40.652+0000 7f27f24be640 1 --2- 172.21.15.112:0/1952621220 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f27ec070030 0x7f27ec09b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:40.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:40.652+0000 7f27f24be640 1 -- 172.21.15.112:0/1952621220 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27ec1f6560 con 0x7f27ec074d90 2024-09-21T00:26:40.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:40.652+0000 7f27f14bc640 1 --2- 172.21.15.112:0/1952621220 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f27ec070030 0x7f27ec09b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:26:40.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:40.652+0000 7f27f24be640 1 --2- 172.21.15.112:0/1952621220 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f27ec074d90 0x7f27ec1f5900 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f27e4009cd0 tx=0x7f27e4003ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:40.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:40.653+0000 7f27e27fc640 1 -- 172.21.15.112:0/1952621220 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27e4040070 con 0x7f27ec074d90 2024-09-21T00:26:40.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:40.653+0000 7f27e27fc640 1 -- 172.21.15.112:0/1952621220 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f27e4004450 con 0x7f27ec074d90 2024-09-21T00:26:40.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:40.653+0000 7f27f3f21640 1 -- 172.21.15.112:0/1952621220 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27ec09aba0 con 0x7f27ec074d90 2024-09-21T00:26:40.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:40.653+0000 7f27e27fc640 1 -- 172.21.15.112:0/1952621220 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27e4004840 con 0x7f27ec074d90 2024-09-21T00:26:40.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:40.653+0000 7f27f3f21640 1 -- 172.21.15.112:0/1952621220 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27ec09b160 con 0x7f27ec074d90 2024-09-21T00:26:40.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:40.654+0000 7f27f3f21640 1 -- 172.21.15.112:0/1952621220 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27ec06d4f0 con 0x7f27ec074d90 2024-09-21T00:26:40.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:40.654+0000 7f27e27fc640 1 -- 172.21.15.112:0/1952621220 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f27e4039040 con 0x7f27ec074d90 2024-09-21T00:26:40.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:40.655+0000 7f27e27fc640 1 --2- 172.21.15.112:0/1952621220 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f27d0078b90 0x7f27d007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:40.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:40.655+0000 7f27e27fc640 1 -- 172.21.15.112:0/1952621220 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f27e402e030 con 0x7f27ec074d90 2024-09-21T00:26:40.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:40.655+0000 7f27f1cbd640 1 --2- 172.21.15.112:0/1952621220 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f27d0078b90 0x7f27d007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:40.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:40.656+0000 7f27f1cbd640 1 --2- 172.21.15.112:0/1952621220 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f27d0078b90 0x7f27d007b0a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f27e80046b0 tx=0x7f27e8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:40.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:40.660+0000 7f27e27fc640 1 -- 172.21.15.112:0/1952621220 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f27ec06d4f0 con 0x7f27ec074d90 2024-09-21T00:26:40.793 INFO:tasks.ceph.mds.a.smithi112.stderr:2024-09-21T00:26:40.792+0000 7f8c708e3640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T00:26:40.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:40.833+0000 7f27f3f21640 1 -- 172.21.15.112:0/1952621220 --> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f27ec09bea0 con 0x7f27d0078b90 2024-09-21T00:26:40.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:40.839+0000 7f27e27fc640 1 -- 172.21.15.112:0/1952621220 <== mgr.4103 v2:172.21.15.187:6800/2921522906 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247151 (secure 0 0 0) 0x7f27e800e500 con 0x7f27d0078b90 2024-09-21T00:26:40.843 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:26:40.844 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2024-09-21T00:26:40.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:40.841+0000 7f27f3f21640 1 -- 172.21.15.112:0/1952621220 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f27d0078b90 msgr2=0x7f27d007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:40.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:40.841+0000 7f27f3f21640 1 --2- 172.21.15.112:0/1952621220 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f27d0078b90 0x7f27d007b0a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f27e80046b0 tx=0x7f27e8009040 comp rx=0 tx=0).stop 2024-09-21T00:26:40.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:40.841+0000 7f27f3f21640 1 -- 172.21.15.112:0/1952621220 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f27ec074d90 msgr2=0x7f27ec1f5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:40.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:40.841+0000 7f27f3f21640 1 --2- 172.21.15.112:0/1952621220 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f27ec074d90 0x7f27ec1f5900 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f27e4009cd0 tx=0x7f27e4003ef0 comp rx=0 tx=0).stop 2024-09-21T00:26:40.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:40.841+0000 7f27f3f21640 1 -- 172.21.15.112:0/1952621220 shutdown_connections 2024-09-21T00:26:40.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:40.841+0000 7f27f3f21640 1 --2- 172.21.15.112:0/1952621220 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f27d0078b90 0x7f27d007b0a0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:40.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:40.841+0000 7f27f3f21640 1 --2- 172.21.15.112:0/1952621220 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f27ec074d90 0x7f27ec1f5900 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:40.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:40.841+0000 7f27f3f21640 1 --2- 172.21.15.112:0/1952621220 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f27ec070030 0x7f27ec09b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:40.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:40.841+0000 7f27f3f21640 1 --2- 172.21.15.112:0/1952621220 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f27ec06e330 0x7f27ec0a0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:40.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:40.841+0000 7f27f3f21640 1 -- 172.21.15.112:0/1952621220 >> 172.21.15.112:0/1952621220 conn(0x7f27ec0cca40 msgr2=0x7f27ec06cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:40.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:40.841+0000 7f27f3f21640 1 -- 172.21.15.112:0/1952621220 shutdown_connections 2024-09-21T00:26:40.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:40.841+0000 7f27f3f21640 1 -- 172.21.15.112:0/1952621220 wait complete. 2024-09-21T00:26:40.855 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":64,"stamp":"2024-09-21T00:26:39.101262+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":104,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":89,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220516,"kb_used_data":4284,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754204,"statfs":{"total":773094113280,"available":772868304896,"internally_reserved":0,"allocated":4386816,"data_stored":2498882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.289329"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.829346+0000","last_change":"2024-09-21T00:26:35.829346+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.829346+0000","last_undegraded":"2024-09-21T00:26:35.829346+0000","last_fullsized":"2024-09-21T00:26:35.829346+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.827322+0000","last_change":"2024-09-21T00:26:35.827322+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.827322+0000","last_undegraded":"2024-09-21T00:26:35.827322+0000","last_fullsized":"2024-09-21T00:26:35.827322+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.828578+0000","last_change":"2024-09-21T00:26:35.828578+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.828578+0000","last_undegraded":"2024-09-21T00:26:35.828578+0000","last_fullsized":"2024-09-21T00:26:35.828578+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.825190+0000","last_change":"2024-09-21T00:26:35.825190+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.825190+0000","last_undegraded":"2024-09-21T00:26:35.825190+0000","last_fullsized":"2024-09-21T00:26:35.825190+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.825161+0000","last_change":"2024-09-21T00:26:35.825161+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.825161+0000","last_undegraded":"2024-09-21T00:26:35.825161+0000","last_fullsized":"2024-09-21T00:26:35.825161+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-21T00:26:36.844469+0000","last_change":"2024-09-21T00:26:36.841381+0000","last_active":"2024-09-21T00:26:36.844469+0000","last_peered":"2024-09-21T00:26:36.844469+0000","last_clean":"2024-09-21T00:26:36.844469+0000","last_became_active":"2024-09-21T00:26:36.841094+0000","last_became_peered":"2024-09-21T00:26:36.841094+0000","last_unstale":"2024-09-21T00:26:36.844469+0000","last_undegraded":"2024-09-21T00:26:36.844469+0000","last_fullsized":"2024-09-21T00:26:36.844469+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:55:25.224768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.832564+0000","last_change":"2024-09-21T00:26:35.832564+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.832564+0000","last_undegraded":"2024-09-21T00:26:35.832564+0000","last_fullsized":"2024-09-21T00:26:35.832564+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.830845+0000","last_change":"2024-09-21T00:26:35.830845+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.830845+0000","last_undegraded":"2024-09-21T00:26:35.830845+0000","last_fullsized":"2024-09-21T00:26:35.830845+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.830587+0000","last_change":"2024-09-21T00:26:35.830587+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.830587+0000","last_undegraded":"2024-09-21T00:26:35.830587+0000","last_fullsized":"2024-09-21T00:26:35.830587+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.833967+0000","last_change":"2024-09-21T00:26:35.833967+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.833967+0000","last_undegraded":"2024-09-21T00:26:35.833967+0000","last_fullsized":"2024-09-21T00:26:35.833967+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.832457+0000","last_change":"2024-09-21T00:26:35.832457+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.832457+0000","last_undegraded":"2024-09-21T00:26:35.832457+0000","last_fullsized":"2024-09-21T00:26:35.832457+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.828280+0000","last_change":"2024-09-21T00:26:35.828280+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.828280+0000","last_undegraded":"2024-09-21T00:26:35.828280+0000","last_fullsized":"2024-09-21T00:26:35.828280+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.831355+0000","last_change":"2024-09-21T00:26:35.831355+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.831355+0000","last_undegraded":"2024-09-21T00:26:35.831355+0000","last_fullsized":"2024-09-21T00:26:35.831355+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.827439+0000","last_change":"2024-09-21T00:26:35.827439+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.827439+0000","last_undegraded":"2024-09-21T00:26:35.827439+0000","last_fullsized":"2024-09-21T00:26:35.827439+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.829991+0000","last_change":"2024-09-21T00:26:35.829991+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.829991+0000","last_undegraded":"2024-09-21T00:26:35.829991+0000","last_fullsized":"2024-09-21T00:26:35.829991+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.828731+0000","last_change":"2024-09-21T00:26:35.828731+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.828731+0000","last_undegraded":"2024-09-21T00:26:35.828731+0000","last_fullsized":"2024-09-21T00:26:35.828731+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.828322+0000","last_change":"2024-09-21T00:26:35.828322+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.828322+0000","last_undegraded":"2024-09-21T00:26:35.828322+0000","last_fullsized":"2024-09-21T00:26:35.828322+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.827556+0000","last_change":"2024-09-21T00:26:35.827556+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.827556+0000","last_undegraded":"2024-09-21T00:26:35.827556+0000","last_fullsized":"2024-09-21T00:26:35.827556+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.823845+0000","last_change":"2024-09-21T00:26:35.823845+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.823845+0000","last_undegraded":"2024-09-21T00:26:35.823845+0000","last_fullsized":"2024-09-21T00:26:35.823845+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.827691+0000","last_change":"2024-09-21T00:26:35.827691+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.827691+0000","last_undegraded":"2024-09-21T00:26:35.827691+0000","last_fullsized":"2024-09-21T00:26:35.827691+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.831846+0000","last_change":"2024-09-21T00:26:35.831846+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.831846+0000","last_undegraded":"2024-09-21T00:26:35.831846+0000","last_fullsized":"2024-09-21T00:26:35.831846+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.823745+0000","last_change":"2024-09-21T00:26:35.823745+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.823745+0000","last_undegraded":"2024-09-21T00:26:35.823745+0000","last_fullsized":"2024-09-21T00:26:35.823745+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.827636+0000","last_change":"2024-09-21T00:26:35.827636+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.827636+0000","last_undegraded":"2024-09-21T00:26:35.827636+0000","last_fullsized":"2024-09-21T00:26:35.827636+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.823856+0000","last_change":"2024-09-21T00:26:35.823856+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.823856+0000","last_undegraded":"2024-09-21T00:26:35.823856+0000","last_fullsized":"2024-09-21T00:26:35.823856+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.826279+0000","last_change":"2024-09-21T00:26:35.826279+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.826279+0000","last_undegraded":"2024-09-21T00:26:35.826279+0000","last_fullsized":"2024-09-21T00:26:35.826279+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-21T00:26:36.859512+0000","last_change":"2024-09-21T00:26:36.856780+0000","last_active":"2024-09-21T00:26:36.859512+0000","last_peered":"2024-09-21T00:26:36.859512+0000","last_clean":"2024-09-21T00:26:36.859512+0000","last_became_active":"2024-09-21T00:26:36.856482+0000","last_became_peered":"2024-09-21T00:26:36.856482+0000","last_unstale":"2024-09-21T00:26:36.859512+0000","last_undegraded":"2024-09-21T00:26:36.859512+0000","last_fullsized":"2024-09-21T00:26:36.859512+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:58:37.004181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.825808+0000","last_change":"2024-09-21T00:26:35.825808+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.825808+0000","last_undegraded":"2024-09-21T00:26:35.825808+0000","last_fullsized":"2024-09-21T00:26:35.825808+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.830953+0000","last_change":"2024-09-21T00:26:35.830953+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.830953+0000","last_undegraded":"2024-09-21T00:26:35.830953+0000","last_fullsized":"2024-09-21T00:26:35.830953+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.824648+0000","last_change":"2024-09-21T00:26:35.824648+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.824648+0000","last_undegraded":"2024-09-21T00:26:35.824648+0000","last_fullsized":"2024-09-21T00:26:35.824648+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.826507+0000","last_change":"2024-09-21T00:26:35.826507+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.826507+0000","last_undegraded":"2024-09-21T00:26:35.826507+0000","last_fullsized":"2024-09-21T00:26:35.826507+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.826999+0000","last_change":"2024-09-21T00:26:35.826999+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.826999+0000","last_undegraded":"2024-09-21T00:26:35.826999+0000","last_fullsized":"2024-09-21T00:26:35.826999+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-21T00:26:36.838547+0000","last_change":"2024-09-21T00:26:36.836602+0000","last_active":"2024-09-21T00:26:36.838547+0000","last_peered":"2024-09-21T00:26:36.838547+0000","last_clean":"2024-09-21T00:26:36.838547+0000","last_became_active":"2024-09-21T00:26:36.835713+0000","last_became_peered":"2024-09-21T00:26:36.835713+0000","last_unstale":"2024-09-21T00:26:36.838547+0000","last_undegraded":"2024-09-21T00:26:36.838547+0000","last_fullsized":"2024-09-21T00:26:36.838547+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:23:48.458736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.828100+0000","last_change":"2024-09-21T00:26:35.828100+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.828100+0000","last_undegraded":"2024-09-21T00:26:35.828100+0000","last_fullsized":"2024-09-21T00:26:35.828100+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-21T00:26:35.844965+0000","last_change":"2024-09-21T00:26:35.838321+0000","last_active":"2024-09-21T00:26:35.844965+0000","last_peered":"2024-09-21T00:26:35.844965+0000","last_clean":"2024-09-21T00:26:35.844965+0000","last_became_active":"2024-09-21T00:26:35.838022+0000","last_became_peered":"2024-09-21T00:26:35.838022+0000","last_unstale":"2024-09-21T00:26:35.844965+0000","last_undegraded":"2024-09-21T00:26:35.844965+0000","last_fullsized":"2024-09-21T00:26:35.844965+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:05:48.525312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-21T00:26:36.859717+0000","last_change":"2024-09-21T00:26:36.856958+0000","last_active":"2024-09-21T00:26:36.859717+0000","last_peered":"2024-09-21T00:26:36.859717+0000","last_clean":"2024-09-21T00:26:36.859717+0000","last_became_active":"2024-09-21T00:26:36.856395+0000","last_became_peered":"2024-09-21T00:26:36.856395+0000","last_unstale":"2024-09-21T00:26:36.859717+0000","last_undegraded":"2024-09-21T00:26:36.859717+0000","last_fullsized":"2024-09-21T00:26:36.859717+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:15:12.241543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-21T00:26:35.839447+0000","last_change":"2024-09-21T00:26:35.837664+0000","last_active":"2024-09-21T00:26:35.839447+0000","last_peered":"2024-09-21T00:26:35.839447+0000","last_clean":"2024-09-21T00:26:35.839447+0000","last_became_active":"2024-09-21T00:26:35.837355+0000","last_became_peered":"2024-09-21T00:26:35.837355+0000","last_unstale":"2024-09-21T00:26:35.839447+0000","last_undegraded":"2024-09-21T00:26:35.839447+0000","last_fullsized":"2024-09-21T00:26:35.839447+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T00:41:22.116310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.824468+0000","last_change":"2024-09-21T00:26:35.824468+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.824468+0000","last_undegraded":"2024-09-21T00:26:35.824468+0000","last_fullsized":"2024-09-21T00:26:35.824468+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"37'2","reported_seq":15,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-21T00:26:36.853187+0000","last_change":"2024-09-21T00:26:35.840905+0000","last_active":"2024-09-21T00:26:36.853187+0000","last_peered":"2024-09-21T00:26:36.853187+0000","last_clean":"2024-09-21T00:26:36.853187+0000","last_became_active":"2024-09-21T00:26:35.840594+0000","last_became_peered":"2024-09-21T00:26:35.840594+0000","last_unstale":"2024-09-21T00:26:36.853187+0000","last_undegraded":"2024-09-21T00:26:36.853187+0000","last_fullsized":"2024-09-21T00:26:36.853187+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:54:48.838959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.829845+0000","last_change":"2024-09-21T00:26:35.829845+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.829845+0000","last_undegraded":"2024-09-21T00:26:35.829845+0000","last_fullsized":"2024-09-21T00:26:35.829845+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-21T00:26:35.842277+0000","last_change":"2024-09-21T00:26:35.836092+0000","last_active":"2024-09-21T00:26:35.842277+0000","last_peered":"2024-09-21T00:26:35.842277+0000","last_clean":"2024-09-21T00:26:35.842277+0000","last_became_active":"2024-09-21T00:26:35.835424+0000","last_became_peered":"2024-09-21T00:26:35.835424+0000","last_unstale":"2024-09-21T00:26:35.842277+0000","last_undegraded":"2024-09-21T00:26:35.842277+0000","last_fullsized":"2024-09-21T00:26:35.842277+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:21:21.377354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.826825+0000","last_change":"2024-09-21T00:26:35.826825+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.826825+0000","last_undegraded":"2024-09-21T00:26:35.826825+0000","last_fullsized":"2024-09-21T00:26:35.826825+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-21T00:26:35.841980+0000","last_change":"2024-09-21T00:26:35.835778+0000","last_active":"2024-09-21T00:26:35.841980+0000","last_peered":"2024-09-21T00:26:35.841980+0000","last_clean":"2024-09-21T00:26:35.841980+0000","last_became_active":"2024-09-21T00:26:35.835426+0000","last_became_peered":"2024-09-21T00:26:35.835426+0000","last_unstale":"2024-09-21T00:26:35.841980+0000","last_undegraded":"2024-09-21T00:26:35.841980+0000","last_fullsized":"2024-09-21T00:26:35.841980+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:49:14.000042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.829387+0000","last_change":"2024-09-21T00:26:35.829387+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.829387+0000","last_undegraded":"2024-09-21T00:26:35.829387+0000","last_fullsized":"2024-09-21T00:26:35.829387+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-21T00:26:35.850340+0000","last_change":"2024-09-21T00:26:35.846072+0000","last_active":"2024-09-21T00:26:35.850340+0000","last_peered":"2024-09-21T00:26:35.850340+0000","last_clean":"2024-09-21T00:26:35.850340+0000","last_became_active":"2024-09-21T00:26:35.845724+0000","last_became_peered":"2024-09-21T00:26:35.845724+0000","last_unstale":"2024-09-21T00:26:35.850340+0000","last_undegraded":"2024-09-21T00:26:35.850340+0000","last_fullsized":"2024-09-21T00:26:35.850340+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:49:14.000042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.825895+0000","last_change":"2024-09-21T00:26:35.825895+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.825895+0000","last_undegraded":"2024-09-21T00:26:35.825895+0000","last_fullsized":"2024-09-21T00:26:35.825895+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-21T00:26:36.831276+0000","last_change":"2024-09-21T00:26:35.842674+0000","last_active":"2024-09-21T00:26:36.831276+0000","last_peered":"2024-09-21T00:26:36.831276+0000","last_clean":"2024-09-21T00:26:36.831276+0000","last_became_active":"2024-09-21T00:26:35.842339+0000","last_became_peered":"2024-09-21T00:26:35.842339+0000","last_unstale":"2024-09-21T00:26:36.831276+0000","last_undegraded":"2024-09-21T00:26:36.831276+0000","last_fullsized":"2024-09-21T00:26:36.831276+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:56:47.623787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.828735+0000","last_change":"2024-09-21T00:26:35.828735+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.828735+0000","last_undegraded":"2024-09-21T00:26:35.828735+0000","last_fullsized":"2024-09-21T00:26:35.828735+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-21T00:26:35.839220+0000","last_change":"2024-09-21T00:26:35.835252+0000","last_active":"2024-09-21T00:26:35.839220+0000","last_peered":"2024-09-21T00:26:35.839220+0000","last_clean":"2024-09-21T00:26:35.839220+0000","last_became_active":"2024-09-21T00:26:35.834768+0000","last_became_peered":"2024-09-21T00:26:35.834768+0000","last_unstale":"2024-09-21T00:26:35.839220+0000","last_undegraded":"2024-09-21T00:26:35.839220+0000","last_fullsized":"2024-09-21T00:26:35.839220+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:29:33.119014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.824310+0000","last_change":"2024-09-21T00:26:35.824310+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.824310+0000","last_undegraded":"2024-09-21T00:26:35.824310+0000","last_fullsized":"2024-09-21T00:26:35.824310+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-21T00:26:35.836982+0000","last_change":"2024-09-21T00:26:35.834813+0000","last_active":"2024-09-21T00:26:35.836982+0000","last_peered":"2024-09-21T00:26:35.836982+0000","last_clean":"2024-09-21T00:26:35.836982+0000","last_became_active":"2024-09-21T00:26:35.834218+0000","last_became_peered":"2024-09-21T00:26:35.834218+0000","last_unstale":"2024-09-21T00:26:35.836982+0000","last_undegraded":"2024-09-21T00:26:35.836982+0000","last_fullsized":"2024-09-21T00:26:35.836982+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:41:29.077144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.830680+0000","last_change":"2024-09-21T00:26:35.830680+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.830680+0000","last_undegraded":"2024-09-21T00:26:35.830680+0000","last_fullsized":"2024-09-21T00:26:35.830680+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-21T00:26:35.842651+0000","last_change":"2024-09-21T00:26:35.841364+0000","last_active":"2024-09-21T00:26:35.842651+0000","last_peered":"2024-09-21T00:26:35.842651+0000","last_clean":"2024-09-21T00:26:35.842651+0000","last_became_active":"2024-09-21T00:26:35.840989+0000","last_became_peered":"2024-09-21T00:26:35.840989+0000","last_unstale":"2024-09-21T00:26:35.842651+0000","last_undegraded":"2024-09-21T00:26:35.842651+0000","last_fullsized":"2024-09-21T00:26:35.842651+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T00:47:37.835274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.829676+0000","last_change":"2024-09-21T00:26:35.829676+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.829676+0000","last_undegraded":"2024-09-21T00:26:35.829676+0000","last_fullsized":"2024-09-21T00:26:35.829676+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-21T00:26:35.842468+0000","last_change":"2024-09-21T00:26:35.836960+0000","last_active":"2024-09-21T00:26:35.842468+0000","last_peered":"2024-09-21T00:26:35.842468+0000","last_clean":"2024-09-21T00:26:35.842468+0000","last_became_active":"2024-09-21T00:26:35.836641+0000","last_became_peered":"2024-09-21T00:26:35.836641+0000","last_unstale":"2024-09-21T00:26:35.842468+0000","last_undegraded":"2024-09-21T00:26:35.842468+0000","last_fullsized":"2024-09-21T00:26:35.842468+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:50:54.008029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.825454+0000","last_change":"2024-09-21T00:26:35.825454+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.825454+0000","last_undegraded":"2024-09-21T00:26:35.825454+0000","last_fullsized":"2024-09-21T00:26:35.825454+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-21T00:26:35.836435+0000","last_change":"2024-09-21T00:26:35.834520+0000","last_active":"2024-09-21T00:26:35.836435+0000","last_peered":"2024-09-21T00:26:35.836435+0000","last_clean":"2024-09-21T00:26:35.836435+0000","last_became_active":"2024-09-21T00:26:35.834146+0000","last_became_peered":"2024-09-21T00:26:35.834146+0000","last_unstale":"2024-09-21T00:26:35.836435+0000","last_undegraded":"2024-09-21T00:26:35.836435+0000","last_fullsized":"2024-09-21T00:26:35.836435+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T00:47:37.835274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.826334+0000","last_change":"2024-09-21T00:26:35.826334+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.826334+0000","last_undegraded":"2024-09-21T00:26:35.826334+0000","last_fullsized":"2024-09-21T00:26:35.826334+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-21T00:26:35.843305+0000","last_change":"2024-09-21T00:26:35.840502+0000","last_active":"2024-09-21T00:26:35.843305+0000","last_peered":"2024-09-21T00:26:35.843305+0000","last_clean":"2024-09-21T00:26:35.843305+0000","last_became_active":"2024-09-21T00:26:35.840189+0000","last_became_peered":"2024-09-21T00:26:35.840189+0000","last_unstale":"2024-09-21T00:26:35.843305+0000","last_undegraded":"2024-09-21T00:26:35.843305+0000","last_fullsized":"2024-09-21T00:26:35.843305+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:41:13.666350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.827602+0000","last_change":"2024-09-21T00:26:35.827602+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.827602+0000","last_undegraded":"2024-09-21T00:26:35.827602+0000","last_fullsized":"2024-09-21T00:26:35.827602+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-21T00:26:36.831536+0000","last_change":"2024-09-21T00:26:35.842688+0000","last_active":"2024-09-21T00:26:36.831536+0000","last_peered":"2024-09-21T00:26:36.831536+0000","last_clean":"2024-09-21T00:26:36.831536+0000","last_became_active":"2024-09-21T00:26:35.842375+0000","last_became_peered":"2024-09-21T00:26:35.842375+0000","last_unstale":"2024-09-21T00:26:36.831536+0000","last_undegraded":"2024-09-21T00:26:36.831536+0000","last_fullsized":"2024-09-21T00:26:36.831536+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:58:30.617062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.828170+0000","last_change":"2024-09-21T00:26:35.828170+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.828170+0000","last_undegraded":"2024-09-21T00:26:35.828170+0000","last_fullsized":"2024-09-21T00:26:35.828170+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-21T00:26:35.844992+0000","last_change":"2024-09-21T00:26:35.839140+0000","last_active":"2024-09-21T00:26:35.844992+0000","last_peered":"2024-09-21T00:26:35.844992+0000","last_clean":"2024-09-21T00:26:35.844992+0000","last_became_active":"2024-09-21T00:26:35.838817+0000","last_became_peered":"2024-09-21T00:26:35.838817+0000","last_unstale":"2024-09-21T00:26:35.844992+0000","last_undegraded":"2024-09-21T00:26:35.844992+0000","last_fullsized":"2024-09-21T00:26:35.844992+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:07:46.709919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.828288+0000","last_change":"2024-09-21T00:26:35.828288+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.828288+0000","last_undegraded":"2024-09-21T00:26:35.828288+0000","last_fullsized":"2024-09-21T00:26:35.828288+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-21T00:26:35.850329+0000","last_change":"2024-09-21T00:26:35.846027+0000","last_active":"2024-09-21T00:26:35.850329+0000","last_peered":"2024-09-21T00:26:35.850329+0000","last_clean":"2024-09-21T00:26:35.850329+0000","last_became_active":"2024-09-21T00:26:35.845681+0000","last_became_peered":"2024-09-21T00:26:35.845681+0000","last_unstale":"2024-09-21T00:26:35.850329+0000","last_undegraded":"2024-09-21T00:26:35.850329+0000","last_fullsized":"2024-09-21T00:26:35.850329+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:11:52.985613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.827957+0000","last_change":"2024-09-21T00:26:35.827957+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.827957+0000","last_undegraded":"2024-09-21T00:26:35.827957+0000","last_fullsized":"2024-09-21T00:26:35.827957+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-21T00:26:35.850340+0000","last_change":"2024-09-21T00:26:35.845969+0000","last_active":"2024-09-21T00:26:35.850340+0000","last_peered":"2024-09-21T00:26:35.850340+0000","last_clean":"2024-09-21T00:26:35.850340+0000","last_became_active":"2024-09-21T00:26:35.845633+0000","last_became_peered":"2024-09-21T00:26:35.845633+0000","last_unstale":"2024-09-21T00:26:35.850340+0000","last_undegraded":"2024-09-21T00:26:35.850340+0000","last_fullsized":"2024-09-21T00:26:35.850340+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:41:29.077144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.830853+0000","last_change":"2024-09-21T00:26:35.830853+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.830853+0000","last_undegraded":"2024-09-21T00:26:35.830853+0000","last_fullsized":"2024-09-21T00:26:35.830853+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-21T00:26:35.845305+0000","last_change":"2024-09-21T00:26:35.840529+0000","last_active":"2024-09-21T00:26:35.845305+0000","last_peered":"2024-09-21T00:26:35.845305+0000","last_clean":"2024-09-21T00:26:35.845305+0000","last_became_active":"2024-09-21T00:26:35.840115+0000","last_became_peered":"2024-09-21T00:26:35.840115+0000","last_unstale":"2024-09-21T00:26:35.845305+0000","last_undegraded":"2024-09-21T00:26:35.845305+0000","last_fullsized":"2024-09-21T00:26:35.845305+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:33:02.079644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.824938+0000","last_change":"2024-09-21T00:26:35.824938+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.824938+0000","last_undegraded":"2024-09-21T00:26:35.824938+0000","last_fullsized":"2024-09-21T00:26:35.824938+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-21T00:26:35.838164+0000","last_change":"2024-09-21T00:26:35.835450+0000","last_active":"2024-09-21T00:26:35.838164+0000","last_peered":"2024-09-21T00:26:35.838164+0000","last_clean":"2024-09-21T00:26:35.838164+0000","last_became_active":"2024-09-21T00:26:35.835141+0000","last_became_peered":"2024-09-21T00:26:35.835141+0000","last_unstale":"2024-09-21T00:26:35.838164+0000","last_undegraded":"2024-09-21T00:26:35.838164+0000","last_fullsized":"2024-09-21T00:26:35.838164+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:11:09.781135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.824556+0000","last_change":"2024-09-21T00:26:35.824556+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.824556+0000","last_undegraded":"2024-09-21T00:26:35.824556+0000","last_fullsized":"2024-09-21T00:26:35.824556+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-21T00:26:36.831408+0000","last_change":"2024-09-21T00:26:35.841410+0000","last_active":"2024-09-21T00:26:36.831408+0000","last_peered":"2024-09-21T00:26:36.831408+0000","last_clean":"2024-09-21T00:26:36.831408+0000","last_became_active":"2024-09-21T00:26:35.841094+0000","last_became_peered":"2024-09-21T00:26:35.841094+0000","last_unstale":"2024-09-21T00:26:36.831408+0000","last_undegraded":"2024-09-21T00:26:36.831408+0000","last_fullsized":"2024-09-21T00:26:36.831408+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:48:22.317860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.827023+0000","last_change":"2024-09-21T00:26:35.827023+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.827023+0000","last_undegraded":"2024-09-21T00:26:35.827023+0000","last_fullsized":"2024-09-21T00:26:35.827023+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-21T00:26:35.843267+0000","last_change":"2024-09-21T00:26:35.836023+0000","last_active":"2024-09-21T00:26:35.843267+0000","last_peered":"2024-09-21T00:26:35.843267+0000","last_clean":"2024-09-21T00:26:35.843267+0000","last_became_active":"2024-09-21T00:26:35.835721+0000","last_became_peered":"2024-09-21T00:26:35.835721+0000","last_unstale":"2024-09-21T00:26:35.843267+0000","last_undegraded":"2024-09-21T00:26:35.843267+0000","last_fullsized":"2024-09-21T00:26:35.843267+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:43:45.222366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.825112+0000","last_change":"2024-09-21T00:26:35.825112+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.825112+0000","last_undegraded":"2024-09-21T00:26:35.825112+0000","last_fullsized":"2024-09-21T00:26:35.825112+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-21T00:26:35.843093+0000","last_change":"2024-09-21T00:26:35.837224+0000","last_active":"2024-09-21T00:26:35.843093+0000","last_peered":"2024-09-21T00:26:35.843093+0000","last_clean":"2024-09-21T00:26:35.843093+0000","last_became_active":"2024-09-21T00:26:35.836849+0000","last_became_peered":"2024-09-21T00:26:35.836849+0000","last_unstale":"2024-09-21T00:26:35.843093+0000","last_undegraded":"2024-09-21T00:26:35.843093+0000","last_fullsized":"2024-09-21T00:26:35.843093+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:59:09.929281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.829525+0000","last_change":"2024-09-21T00:26:35.829525+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.829525+0000","last_undegraded":"2024-09-21T00:26:35.829525+0000","last_fullsized":"2024-09-21T00:26:35.829525+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-21T00:26:35.842275+0000","last_change":"2024-09-21T00:26:35.837934+0000","last_active":"2024-09-21T00:26:35.842275+0000","last_peered":"2024-09-21T00:26:35.842275+0000","last_clean":"2024-09-21T00:26:35.842275+0000","last_became_active":"2024-09-21T00:26:35.837198+0000","last_became_peered":"2024-09-21T00:26:35.837198+0000","last_unstale":"2024-09-21T00:26:35.842275+0000","last_undegraded":"2024-09-21T00:26:35.842275+0000","last_fullsized":"2024-09-21T00:26:35.842275+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:04:35.145795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.827774+0000","last_change":"2024-09-21T00:26:35.827774+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.827774+0000","last_undegraded":"2024-09-21T00:26:35.827774+0000","last_fullsized":"2024-09-21T00:26:35.827774+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-21T00:26:35.836465+0000","last_change":"2024-09-21T00:26:35.832534+0000","last_active":"2024-09-21T00:26:35.836465+0000","last_peered":"2024-09-21T00:26:35.836465+0000","last_clean":"2024-09-21T00:26:35.836465+0000","last_became_active":"2024-09-21T00:26:35.832197+0000","last_became_peered":"2024-09-21T00:26:35.832197+0000","last_unstale":"2024-09-21T00:26:35.836465+0000","last_undegraded":"2024-09-21T00:26:35.836465+0000","last_fullsized":"2024-09-21T00:26:35.836465+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:28:49.838026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-21T00:26:36.844617+0000","last_change":"2024-09-21T00:26:36.841191+0000","last_active":"2024-09-21T00:26:36.844617+0000","last_peered":"2024-09-21T00:26:36.844617+0000","last_clean":"2024-09-21T00:26:36.844617+0000","last_became_active":"2024-09-21T00:26:36.840676+0000","last_became_peered":"2024-09-21T00:26:36.840676+0000","last_unstale":"2024-09-21T00:26:36.844617+0000","last_undegraded":"2024-09-21T00:26:36.844617+0000","last_fullsized":"2024-09-21T00:26:36.844617+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:16:54.958692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-21T00:26:35.842557+0000","last_change":"2024-09-21T00:26:35.840350+0000","last_active":"2024-09-21T00:26:35.842557+0000","last_peered":"2024-09-21T00:26:35.842557+0000","last_clean":"2024-09-21T00:26:35.842557+0000","last_became_active":"2024-09-21T00:26:35.840048+0000","last_became_peered":"2024-09-21T00:26:35.840048+0000","last_unstale":"2024-09-21T00:26:35.842557+0000","last_undegraded":"2024-09-21T00:26:35.842557+0000","last_fullsized":"2024-09-21T00:26:35.842557+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:19:54.299718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.829609+0000","last_change":"2024-09-21T00:26:35.829609+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.829609+0000","last_undegraded":"2024-09-21T00:26:35.829609+0000","last_fullsized":"2024-09-21T00:26:35.829609+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-21T00:26:35.846174+0000","last_change":"2024-09-21T00:26:35.838940+0000","last_active":"2024-09-21T00:26:35.846174+0000","last_peered":"2024-09-21T00:26:35.846174+0000","last_clean":"2024-09-21T00:26:35.846174+0000","last_became_active":"2024-09-21T00:26:35.838616+0000","last_became_peered":"2024-09-21T00:26:35.838616+0000","last_unstale":"2024-09-21T00:26:35.846174+0000","last_undegraded":"2024-09-21T00:26:35.846174+0000","last_fullsized":"2024-09-21T00:26:35.846174+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:01:07.494519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.826730+0000","last_change":"2024-09-21T00:26:35.826730+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.826730+0000","last_undegraded":"2024-09-21T00:26:35.826730+0000","last_fullsized":"2024-09-21T00:26:35.826730+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-21T00:26:35.841029+0000","last_change":"2024-09-21T00:26:35.838748+0000","last_active":"2024-09-21T00:26:35.841029+0000","last_peered":"2024-09-21T00:26:35.841029+0000","last_clean":"2024-09-21T00:26:35.841029+0000","last_became_active":"2024-09-21T00:26:35.838445+0000","last_became_peered":"2024-09-21T00:26:35.838445+0000","last_unstale":"2024-09-21T00:26:35.841029+0000","last_undegraded":"2024-09-21T00:26:35.841029+0000","last_fullsized":"2024-09-21T00:26:35.841029+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:44:34.009179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.825823+0000","last_change":"2024-09-21T00:26:35.825823+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.825823+0000","last_undegraded":"2024-09-21T00:26:35.825823+0000","last_fullsized":"2024-09-21T00:26:35.825823+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-21T00:26:35.843172+0000","last_change":"2024-09-21T00:26:35.840565+0000","last_active":"2024-09-21T00:26:35.843172+0000","last_peered":"2024-09-21T00:26:35.843172+0000","last_clean":"2024-09-21T00:26:35.843172+0000","last_became_active":"2024-09-21T00:26:35.840265+0000","last_became_peered":"2024-09-21T00:26:35.840265+0000","last_unstale":"2024-09-21T00:26:35.843172+0000","last_undegraded":"2024-09-21T00:26:35.843172+0000","last_fullsized":"2024-09-21T00:26:35.843172+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:14:15.665953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-21T00:26:36.848314+0000","last_change":"2024-09-21T00:26:36.842397+0000","last_active":"2024-09-21T00:26:36.848314+0000","last_peered":"2024-09-21T00:26:36.848314+0000","last_clean":"2024-09-21T00:26:36.848314+0000","last_became_active":"2024-09-21T00:26:36.842095+0000","last_became_peered":"2024-09-21T00:26:36.842095+0000","last_unstale":"2024-09-21T00:26:36.848314+0000","last_undegraded":"2024-09-21T00:26:36.848314+0000","last_fullsized":"2024-09-21T00:26:36.848314+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:50:12.134645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-21T00:26:35.837296+0000","last_change":"2024-09-21T00:26:35.835982+0000","last_active":"2024-09-21T00:26:35.837296+0000","last_peered":"2024-09-21T00:26:35.837296+0000","last_clean":"2024-09-21T00:26:35.837296+0000","last_became_active":"2024-09-21T00:26:35.835677+0000","last_became_peered":"2024-09-21T00:26:35.835677+0000","last_unstale":"2024-09-21T00:26:35.837296+0000","last_undegraded":"2024-09-21T00:26:35.837296+0000","last_fullsized":"2024-09-21T00:26:35.837296+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:46:16.688217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.826853+0000","last_change":"2024-09-21T00:26:35.826853+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.826853+0000","last_undegraded":"2024-09-21T00:26:35.826853+0000","last_fullsized":"2024-09-21T00:26:35.826853+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-21T00:26:35.842371+0000","last_change":"2024-09-21T00:26:35.838318+0000","last_active":"2024-09-21T00:26:35.842371+0000","last_peered":"2024-09-21T00:26:35.842371+0000","last_clean":"2024-09-21T00:26:35.842371+0000","last_became_active":"2024-09-21T00:26:35.836426+0000","last_became_peered":"2024-09-21T00:26:35.836426+0000","last_unstale":"2024-09-21T00:26:35.842371+0000","last_undegraded":"2024-09-21T00:26:35.842371+0000","last_fullsized":"2024-09-21T00:26:35.842371+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:48:50.639166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.827493+0000","last_change":"2024-09-21T00:26:35.827493+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.827493+0000","last_undegraded":"2024-09-21T00:26:35.827493+0000","last_fullsized":"2024-09-21T00:26:35.827493+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-21T00:26:35.841994+0000","last_change":"2024-09-21T00:26:35.836745+0000","last_active":"2024-09-21T00:26:35.841994+0000","last_peered":"2024-09-21T00:26:35.841994+0000","last_clean":"2024-09-21T00:26:35.841994+0000","last_became_active":"2024-09-21T00:26:35.836427+0000","last_became_peered":"2024-09-21T00:26:35.836427+0000","last_unstale":"2024-09-21T00:26:35.841994+0000","last_undegraded":"2024-09-21T00:26:35.841994+0000","last_fullsized":"2024-09-21T00:26:35.841994+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:33:15.124204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.830688+0000","last_change":"2024-09-21T00:26:35.830688+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.830688+0000","last_undegraded":"2024-09-21T00:26:35.830688+0000","last_fullsized":"2024-09-21T00:26:35.830688+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-21T00:26:35.845045+0000","last_change":"2024-09-21T00:26:35.839244+0000","last_active":"2024-09-21T00:26:35.845045+0000","last_peered":"2024-09-21T00:26:35.845045+0000","last_clean":"2024-09-21T00:26:35.845045+0000","last_became_active":"2024-09-21T00:26:35.838876+0000","last_became_peered":"2024-09-21T00:26:35.838876+0000","last_unstale":"2024-09-21T00:26:35.845045+0000","last_undegraded":"2024-09-21T00:26:35.845045+0000","last_fullsized":"2024-09-21T00:26:35.845045+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:02:08.223701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":136,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-21T00:26:35.831437+0000","last_change":"2024-09-21T00:25:38.725792+0000","last_active":"2024-09-21T00:26:35.831437+0000","last_peered":"2024-09-21T00:26:35.831437+0000","last_clean":"2024-09-21T00:26:35.831437+0000","last_became_active":"2024-09-21T00:25:38.724668+0000","last_became_peered":"2024-09-21T00:25:38.724668+0000","last_unstale":"2024-09-21T00:26:35.831437+0000","last_undegraded":"2024-09-21T00:26:35.831437+0000","last_fullsized":"2024-09-21T00:26:35.831437+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:37.163216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:37.163216+0000","last_clean_scrub_stamp":"2024-09-21T00:25:37.163216+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:14:36.588660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509455,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28008,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343832,"statfs":{"total":96636764160,"available":96608083968,"internally_reserved":0,"allocated":999424,"data_stored":758396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509456,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":393216,"data_stored":161361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509455,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27428,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":405504,"data_stored":168028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509454,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":393216,"data_stored":161361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509456,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":393216,"data_stored":161361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509456,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":393216,"data_stored":161361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509456,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27992,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343848,"statfs":{"total":96636764160,"available":96608100352,"internally_reserved":0,"allocated":987136,"data_stored":751729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574864,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27440,"kb_used_data":412,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":421888,"data_stored":175285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T00:26:40.859 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T00:26:41.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.007+0000 7fc8763c6640 1 -- 172.21.15.112:0/2918498091 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc85c004d40 msgr2=0x7fc85c005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:41.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.007+0000 7fc8763c6640 1 --2- 172.21.15.112:0/2918498091 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc85c004d40 0x7fc85c005150 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fc868005fd0 tx=0x7fc85c005830 comp rx=0 tx=0).stop 2024-09-21T00:26:41.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.007+0000 7fc874963640 1 -- 172.21.15.112:0/2918498091 reap_dead start 2024-09-21T00:26:41.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.007+0000 7fc8763c6640 1 -- 172.21.15.112:0/2918498091 shutdown_connections 2024-09-21T00:26:41.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.007+0000 7fc8763c6640 1 -- 172.21.15.112:0/2918498091 >> 172.21.15.112:0/2918498091 conn(0x7fc8700b6b20 msgr2=0x7fc8700d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:41.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.007+0000 7fc8763c6640 1 -- 172.21.15.112:0/2918498091 shutdown_connections 2024-09-21T00:26:41.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.007+0000 7fc8763c6640 1 -- 172.21.15.112:0/2918498091 wait complete. 2024-09-21T00:26:41.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.009+0000 7fc8763c6640 1 Processor -- start 2024-09-21T00:26:41.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.009+0000 7fc8763c6640 1 -- start start 2024-09-21T00:26:41.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.010+0000 7fc8763c6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc85c004d40 0x7fc87014bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:41.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.010+0000 7fc8763c6640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc8700c9a80 0x7fc8701f3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:41.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.010+0000 7fc8763c6640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc8701f4420 0x7fc8701f82c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:41.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.010+0000 7fc8763c6640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc8700b35e0 con 0x7fc85c004d40 2024-09-21T00:26:41.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.010+0000 7fc8763c6640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc8700b3460 con 0x7fc8700c9a80 2024-09-21T00:26:41.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.010+0000 7fc8763c6640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc8700b3760 con 0x7fc8701f4420 2024-09-21T00:26:41.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.010+0000 7fc86ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc85c004d40 0x7fc87014bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:41.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.010+0000 7fc86ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc85c004d40 0x7fc87014bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36794/0 (socket says 172.21.15.112:36794) 2024-09-21T00:26:41.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.010+0000 7fc86ffff640 1 -- 172.21.15.112:0/2809725821 learned_addr learned my addr 172.21.15.112:0/2809725821 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:26:41.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.010+0000 7fc874963640 1 --2- 172.21.15.112:0/2809725821 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc8701f4420 0x7fc8701f82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:41.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.010+0000 7fc867fff640 1 --2- 172.21.15.112:0/2809725821 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc8700c9a80 0x7fc8701f3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:41.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.010+0000 7fc86ffff640 1 -- 172.21.15.112:0/2809725821 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc8701f4420 msgr2=0x7fc8701f82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:41.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.010+0000 7fc86ffff640 1 --2- 172.21.15.112:0/2809725821 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc8701f4420 0x7fc8701f82c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:41.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.010+0000 7fc86ffff640 1 -- 172.21.15.112:0/2809725821 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc8700c9a80 msgr2=0x7fc8701f3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:41.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.010+0000 7fc86ffff640 1 --2- 172.21.15.112:0/2809725821 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc8700c9a80 0x7fc8701f3ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:41.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.010+0000 7fc86ffff640 1 -- 172.21.15.112:0/2809725821 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8700ca220 con 0x7fc85c004d40 2024-09-21T00:26:41.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.010+0000 7fc867fff640 1 --2- 172.21.15.112:0/2809725821 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc8700c9a80 0x7fc8701f3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:26:41.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.010+0000 7fc874963640 1 --2- 172.21.15.112:0/2809725821 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc8701f4420 0x7fc8701f82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:26:41.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.011+0000 7fc86ffff640 1 --2- 172.21.15.112:0/2809725821 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc85c004d40 0x7fc87014bf80 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fc860006680 tx=0x7fc8600039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:41.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.011+0000 7fc86d7fa640 1 -- 172.21.15.112:0/2809725821 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc860015070 con 0x7fc85c004d40 2024-09-21T00:26:41.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.011+0000 7fc86d7fa640 1 -- 172.21.15.112:0/2809725821 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc860011920 con 0x7fc85c004d40 2024-09-21T00:26:41.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.011+0000 7fc8763c6640 1 -- 172.21.15.112:0/2809725821 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8700ca500 con 0x7fc85c004d40 2024-09-21T00:26:41.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.011+0000 7fc86d7fa640 1 -- 172.21.15.112:0/2809725821 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc860011d70 con 0x7fc85c004d40 2024-09-21T00:26:41.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.011+0000 7fc8763c6640 1 -- 172.21.15.112:0/2809725821 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8700caac0 con 0x7fc85c004d40 2024-09-21T00:26:41.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.012+0000 7fc8763c6640 1 -- 172.21.15.112:0/2809725821 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc870162f00 con 0x7fc85c004d40 2024-09-21T00:26:41.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.013+0000 7fc86d7fa640 1 -- 172.21.15.112:0/2809725821 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc86000f430 con 0x7fc85c004d40 2024-09-21T00:26:41.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.013+0000 7fc86d7fa640 1 --2- 172.21.15.112:0/2809725821 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fc850078bc0 0x7fc85007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:41.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.013+0000 7fc86d7fa640 1 -- 172.21.15.112:0/2809725821 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7fc86009bc30 con 0x7fc85c004d40 2024-09-21T00:26:41.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.013+0000 7fc867fff640 1 --2- 172.21.15.112:0/2809725821 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fc850078bc0 0x7fc85007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:41.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.014+0000 7fc867fff640 1 --2- 172.21.15.112:0/2809725821 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fc850078bc0 0x7fc85007b0d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc8701f9380 tx=0x7fc858005f90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:41.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.018+0000 7fc86d7fa640 1 -- 172.21.15.112:0/2809725821 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc870162bd0 con 0x7fc85c004d40 2024-09-21T00:26:41.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.191+0000 7fc8763c6640 1 -- 172.21.15.112:0/2809725821 --> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc87006a680 con 0x7fc850078bc0 2024-09-21T00:26:41.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.196+0000 7fc86d7fa640 1 -- 172.21.15.112:0/2809725821 <== mgr.4103 v2:172.21.15.187:6800/2921522906 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247151 (secure 0 0 0) 0x7fc87006a680 con 0x7fc850078bc0 2024-09-21T00:26:41.200 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:26:41.201 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2024-09-21T00:26:41.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.198+0000 7fc8763c6640 1 -- 172.21.15.112:0/2809725821 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fc850078bc0 msgr2=0x7fc85007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:41.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.198+0000 7fc8763c6640 1 --2- 172.21.15.112:0/2809725821 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fc850078bc0 0x7fc85007b0d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc8701f9380 tx=0x7fc858005f90 comp rx=0 tx=0).stop 2024-09-21T00:26:41.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.198+0000 7fc8763c6640 1 -- 172.21.15.112:0/2809725821 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc85c004d40 msgr2=0x7fc87014bf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:41.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.198+0000 7fc8763c6640 1 --2- 172.21.15.112:0/2809725821 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc85c004d40 0x7fc87014bf80 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fc860006680 tx=0x7fc8600039c0 comp rx=0 tx=0).stop 2024-09-21T00:26:41.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.198+0000 7fc8763c6640 1 -- 172.21.15.112:0/2809725821 shutdown_connections 2024-09-21T00:26:41.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.198+0000 7fc8763c6640 1 --2- 172.21.15.112:0/2809725821 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc8701f4420 0x7fc8701f82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:41.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.198+0000 7fc8763c6640 1 --2- 172.21.15.112:0/2809725821 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc8700c9a80 0x7fc8701f3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:41.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.198+0000 7fc8763c6640 1 --2- 172.21.15.112:0/2809725821 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fc850078bc0 0x7fc85007b0d0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:41.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.198+0000 7fc8763c6640 1 --2- 172.21.15.112:0/2809725821 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc85c004d40 0x7fc87014bf80 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:41.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.198+0000 7fc8763c6640 1 -- 172.21.15.112:0/2809725821 >> 172.21.15.112:0/2809725821 conn(0x7fc8700b6b20 msgr2=0x7fc87015d9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:41.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.199+0000 7fc8763c6640 1 -- 172.21.15.112:0/2809725821 shutdown_connections 2024-09-21T00:26:41.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.199+0000 7fc8763c6640 1 -- 172.21.15.112:0/2809725821 wait complete. 2024-09-21T00:26:41.211 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":65,"stamp":"2024-09-21T00:26:41.101777+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":104,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":89,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220516,"kb_used_data":4284,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754204,"statfs":{"total":773094113280,"available":772868304896,"internally_reserved":0,"allocated":4386816,"data_stored":2498882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001572"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.829346+0000","last_change":"2024-09-21T00:26:35.829346+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.829346+0000","last_undegraded":"2024-09-21T00:26:35.829346+0000","last_fullsized":"2024-09-21T00:26:35.829346+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.827322+0000","last_change":"2024-09-21T00:26:35.827322+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.827322+0000","last_undegraded":"2024-09-21T00:26:35.827322+0000","last_fullsized":"2024-09-21T00:26:35.827322+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.828578+0000","last_change":"2024-09-21T00:26:35.828578+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.828578+0000","last_undegraded":"2024-09-21T00:26:35.828578+0000","last_fullsized":"2024-09-21T00:26:35.828578+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.825190+0000","last_change":"2024-09-21T00:26:35.825190+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.825190+0000","last_undegraded":"2024-09-21T00:26:35.825190+0000","last_fullsized":"2024-09-21T00:26:35.825190+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.825161+0000","last_change":"2024-09-21T00:26:35.825161+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.825161+0000","last_undegraded":"2024-09-21T00:26:35.825161+0000","last_fullsized":"2024-09-21T00:26:35.825161+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-21T00:26:36.844469+0000","last_change":"2024-09-21T00:26:36.841381+0000","last_active":"2024-09-21T00:26:36.844469+0000","last_peered":"2024-09-21T00:26:36.844469+0000","last_clean":"2024-09-21T00:26:36.844469+0000","last_became_active":"2024-09-21T00:26:36.841094+0000","last_became_peered":"2024-09-21T00:26:36.841094+0000","last_unstale":"2024-09-21T00:26:36.844469+0000","last_undegraded":"2024-09-21T00:26:36.844469+0000","last_fullsized":"2024-09-21T00:26:36.844469+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:55:25.224768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.832564+0000","last_change":"2024-09-21T00:26:35.832564+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.832564+0000","last_undegraded":"2024-09-21T00:26:35.832564+0000","last_fullsized":"2024-09-21T00:26:35.832564+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.830845+0000","last_change":"2024-09-21T00:26:35.830845+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.830845+0000","last_undegraded":"2024-09-21T00:26:35.830845+0000","last_fullsized":"2024-09-21T00:26:35.830845+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.830587+0000","last_change":"2024-09-21T00:26:35.830587+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.830587+0000","last_undegraded":"2024-09-21T00:26:35.830587+0000","last_fullsized":"2024-09-21T00:26:35.830587+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.833967+0000","last_change":"2024-09-21T00:26:35.833967+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.833967+0000","last_undegraded":"2024-09-21T00:26:35.833967+0000","last_fullsized":"2024-09-21T00:26:35.833967+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.832457+0000","last_change":"2024-09-21T00:26:35.832457+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.832457+0000","last_undegraded":"2024-09-21T00:26:35.832457+0000","last_fullsized":"2024-09-21T00:26:35.832457+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.828280+0000","last_change":"2024-09-21T00:26:35.828280+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.828280+0000","last_undegraded":"2024-09-21T00:26:35.828280+0000","last_fullsized":"2024-09-21T00:26:35.828280+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.831355+0000","last_change":"2024-09-21T00:26:35.831355+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.831355+0000","last_undegraded":"2024-09-21T00:26:35.831355+0000","last_fullsized":"2024-09-21T00:26:35.831355+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.827439+0000","last_change":"2024-09-21T00:26:35.827439+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.827439+0000","last_undegraded":"2024-09-21T00:26:35.827439+0000","last_fullsized":"2024-09-21T00:26:35.827439+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.829991+0000","last_change":"2024-09-21T00:26:35.829991+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.829991+0000","last_undegraded":"2024-09-21T00:26:35.829991+0000","last_fullsized":"2024-09-21T00:26:35.829991+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.828731+0000","last_change":"2024-09-21T00:26:35.828731+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.828731+0000","last_undegraded":"2024-09-21T00:26:35.828731+0000","last_fullsized":"2024-09-21T00:26:35.828731+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.828322+0000","last_change":"2024-09-21T00:26:35.828322+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.828322+0000","last_undegraded":"2024-09-21T00:26:35.828322+0000","last_fullsized":"2024-09-21T00:26:35.828322+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.827556+0000","last_change":"2024-09-21T00:26:35.827556+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.827556+0000","last_undegraded":"2024-09-21T00:26:35.827556+0000","last_fullsized":"2024-09-21T00:26:35.827556+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.823845+0000","last_change":"2024-09-21T00:26:35.823845+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.823845+0000","last_undegraded":"2024-09-21T00:26:35.823845+0000","last_fullsized":"2024-09-21T00:26:35.823845+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.827691+0000","last_change":"2024-09-21T00:26:35.827691+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.827691+0000","last_undegraded":"2024-09-21T00:26:35.827691+0000","last_fullsized":"2024-09-21T00:26:35.827691+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.831846+0000","last_change":"2024-09-21T00:26:35.831846+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.831846+0000","last_undegraded":"2024-09-21T00:26:35.831846+0000","last_fullsized":"2024-09-21T00:26:35.831846+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.823745+0000","last_change":"2024-09-21T00:26:35.823745+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.823745+0000","last_undegraded":"2024-09-21T00:26:35.823745+0000","last_fullsized":"2024-09-21T00:26:35.823745+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.827636+0000","last_change":"2024-09-21T00:26:35.827636+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.827636+0000","last_undegraded":"2024-09-21T00:26:35.827636+0000","last_fullsized":"2024-09-21T00:26:35.827636+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.823856+0000","last_change":"2024-09-21T00:26:35.823856+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.823856+0000","last_undegraded":"2024-09-21T00:26:35.823856+0000","last_fullsized":"2024-09-21T00:26:35.823856+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.826279+0000","last_change":"2024-09-21T00:26:35.826279+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.826279+0000","last_undegraded":"2024-09-21T00:26:35.826279+0000","last_fullsized":"2024-09-21T00:26:35.826279+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-21T00:26:36.859512+0000","last_change":"2024-09-21T00:26:36.856780+0000","last_active":"2024-09-21T00:26:36.859512+0000","last_peered":"2024-09-21T00:26:36.859512+0000","last_clean":"2024-09-21T00:26:36.859512+0000","last_became_active":"2024-09-21T00:26:36.856482+0000","last_became_peered":"2024-09-21T00:26:36.856482+0000","last_unstale":"2024-09-21T00:26:36.859512+0000","last_undegraded":"2024-09-21T00:26:36.859512+0000","last_fullsized":"2024-09-21T00:26:36.859512+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:58:37.004181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.825808+0000","last_change":"2024-09-21T00:26:35.825808+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.825808+0000","last_undegraded":"2024-09-21T00:26:35.825808+0000","last_fullsized":"2024-09-21T00:26:35.825808+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.830953+0000","last_change":"2024-09-21T00:26:35.830953+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.830953+0000","last_undegraded":"2024-09-21T00:26:35.830953+0000","last_fullsized":"2024-09-21T00:26:35.830953+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.824648+0000","last_change":"2024-09-21T00:26:35.824648+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.824648+0000","last_undegraded":"2024-09-21T00:26:35.824648+0000","last_fullsized":"2024-09-21T00:26:35.824648+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.826507+0000","last_change":"2024-09-21T00:26:35.826507+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.826507+0000","last_undegraded":"2024-09-21T00:26:35.826507+0000","last_fullsized":"2024-09-21T00:26:35.826507+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.826999+0000","last_change":"2024-09-21T00:26:35.826999+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.826999+0000","last_undegraded":"2024-09-21T00:26:35.826999+0000","last_fullsized":"2024-09-21T00:26:35.826999+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-21T00:26:36.838547+0000","last_change":"2024-09-21T00:26:36.836602+0000","last_active":"2024-09-21T00:26:36.838547+0000","last_peered":"2024-09-21T00:26:36.838547+0000","last_clean":"2024-09-21T00:26:36.838547+0000","last_became_active":"2024-09-21T00:26:36.835713+0000","last_became_peered":"2024-09-21T00:26:36.835713+0000","last_unstale":"2024-09-21T00:26:36.838547+0000","last_undegraded":"2024-09-21T00:26:36.838547+0000","last_fullsized":"2024-09-21T00:26:36.838547+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:23:48.458736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.828100+0000","last_change":"2024-09-21T00:26:35.828100+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.828100+0000","last_undegraded":"2024-09-21T00:26:35.828100+0000","last_fullsized":"2024-09-21T00:26:35.828100+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-21T00:26:35.844965+0000","last_change":"2024-09-21T00:26:35.838321+0000","last_active":"2024-09-21T00:26:35.844965+0000","last_peered":"2024-09-21T00:26:35.844965+0000","last_clean":"2024-09-21T00:26:35.844965+0000","last_became_active":"2024-09-21T00:26:35.838022+0000","last_became_peered":"2024-09-21T00:26:35.838022+0000","last_unstale":"2024-09-21T00:26:35.844965+0000","last_undegraded":"2024-09-21T00:26:35.844965+0000","last_fullsized":"2024-09-21T00:26:35.844965+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:05:48.525312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-21T00:26:36.859717+0000","last_change":"2024-09-21T00:26:36.856958+0000","last_active":"2024-09-21T00:26:36.859717+0000","last_peered":"2024-09-21T00:26:36.859717+0000","last_clean":"2024-09-21T00:26:36.859717+0000","last_became_active":"2024-09-21T00:26:36.856395+0000","last_became_peered":"2024-09-21T00:26:36.856395+0000","last_unstale":"2024-09-21T00:26:36.859717+0000","last_undegraded":"2024-09-21T00:26:36.859717+0000","last_fullsized":"2024-09-21T00:26:36.859717+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:15:12.241543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-21T00:26:35.839447+0000","last_change":"2024-09-21T00:26:35.837664+0000","last_active":"2024-09-21T00:26:35.839447+0000","last_peered":"2024-09-21T00:26:35.839447+0000","last_clean":"2024-09-21T00:26:35.839447+0000","last_became_active":"2024-09-21T00:26:35.837355+0000","last_became_peered":"2024-09-21T00:26:35.837355+0000","last_unstale":"2024-09-21T00:26:35.839447+0000","last_undegraded":"2024-09-21T00:26:35.839447+0000","last_fullsized":"2024-09-21T00:26:35.839447+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T00:41:22.116310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.824468+0000","last_change":"2024-09-21T00:26:35.824468+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.824468+0000","last_undegraded":"2024-09-21T00:26:35.824468+0000","last_fullsized":"2024-09-21T00:26:35.824468+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"37'2","reported_seq":15,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-21T00:26:36.853187+0000","last_change":"2024-09-21T00:26:35.840905+0000","last_active":"2024-09-21T00:26:36.853187+0000","last_peered":"2024-09-21T00:26:36.853187+0000","last_clean":"2024-09-21T00:26:36.853187+0000","last_became_active":"2024-09-21T00:26:35.840594+0000","last_became_peered":"2024-09-21T00:26:35.840594+0000","last_unstale":"2024-09-21T00:26:36.853187+0000","last_undegraded":"2024-09-21T00:26:36.853187+0000","last_fullsized":"2024-09-21T00:26:36.853187+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:54:48.838959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.829845+0000","last_change":"2024-09-21T00:26:35.829845+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.829845+0000","last_undegraded":"2024-09-21T00:26:35.829845+0000","last_fullsized":"2024-09-21T00:26:35.829845+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-21T00:26:35.842277+0000","last_change":"2024-09-21T00:26:35.836092+0000","last_active":"2024-09-21T00:26:35.842277+0000","last_peered":"2024-09-21T00:26:35.842277+0000","last_clean":"2024-09-21T00:26:35.842277+0000","last_became_active":"2024-09-21T00:26:35.835424+0000","last_became_peered":"2024-09-21T00:26:35.835424+0000","last_unstale":"2024-09-21T00:26:35.842277+0000","last_undegraded":"2024-09-21T00:26:35.842277+0000","last_fullsized":"2024-09-21T00:26:35.842277+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:21:21.377354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.826825+0000","last_change":"2024-09-21T00:26:35.826825+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.826825+0000","last_undegraded":"2024-09-21T00:26:35.826825+0000","last_fullsized":"2024-09-21T00:26:35.826825+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-21T00:26:35.841980+0000","last_change":"2024-09-21T00:26:35.835778+0000","last_active":"2024-09-21T00:26:35.841980+0000","last_peered":"2024-09-21T00:26:35.841980+0000","last_clean":"2024-09-21T00:26:35.841980+0000","last_became_active":"2024-09-21T00:26:35.835426+0000","last_became_peered":"2024-09-21T00:26:35.835426+0000","last_unstale":"2024-09-21T00:26:35.841980+0000","last_undegraded":"2024-09-21T00:26:35.841980+0000","last_fullsized":"2024-09-21T00:26:35.841980+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:49:14.000042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.829387+0000","last_change":"2024-09-21T00:26:35.829387+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.829387+0000","last_undegraded":"2024-09-21T00:26:35.829387+0000","last_fullsized":"2024-09-21T00:26:35.829387+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-21T00:26:35.850340+0000","last_change":"2024-09-21T00:26:35.846072+0000","last_active":"2024-09-21T00:26:35.850340+0000","last_peered":"2024-09-21T00:26:35.850340+0000","last_clean":"2024-09-21T00:26:35.850340+0000","last_became_active":"2024-09-21T00:26:35.845724+0000","last_became_peered":"2024-09-21T00:26:35.845724+0000","last_unstale":"2024-09-21T00:26:35.850340+0000","last_undegraded":"2024-09-21T00:26:35.850340+0000","last_fullsized":"2024-09-21T00:26:35.850340+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:49:14.000042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.825895+0000","last_change":"2024-09-21T00:26:35.825895+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.825895+0000","last_undegraded":"2024-09-21T00:26:35.825895+0000","last_fullsized":"2024-09-21T00:26:35.825895+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-21T00:26:36.831276+0000","last_change":"2024-09-21T00:26:35.842674+0000","last_active":"2024-09-21T00:26:36.831276+0000","last_peered":"2024-09-21T00:26:36.831276+0000","last_clean":"2024-09-21T00:26:36.831276+0000","last_became_active":"2024-09-21T00:26:35.842339+0000","last_became_peered":"2024-09-21T00:26:35.842339+0000","last_unstale":"2024-09-21T00:26:36.831276+0000","last_undegraded":"2024-09-21T00:26:36.831276+0000","last_fullsized":"2024-09-21T00:26:36.831276+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:56:47.623787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.828735+0000","last_change":"2024-09-21T00:26:35.828735+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.828735+0000","last_undegraded":"2024-09-21T00:26:35.828735+0000","last_fullsized":"2024-09-21T00:26:35.828735+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-21T00:26:35.839220+0000","last_change":"2024-09-21T00:26:35.835252+0000","last_active":"2024-09-21T00:26:35.839220+0000","last_peered":"2024-09-21T00:26:35.839220+0000","last_clean":"2024-09-21T00:26:35.839220+0000","last_became_active":"2024-09-21T00:26:35.834768+0000","last_became_peered":"2024-09-21T00:26:35.834768+0000","last_unstale":"2024-09-21T00:26:35.839220+0000","last_undegraded":"2024-09-21T00:26:35.839220+0000","last_fullsized":"2024-09-21T00:26:35.839220+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:29:33.119014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.824310+0000","last_change":"2024-09-21T00:26:35.824310+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.824310+0000","last_undegraded":"2024-09-21T00:26:35.824310+0000","last_fullsized":"2024-09-21T00:26:35.824310+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-21T00:26:35.836982+0000","last_change":"2024-09-21T00:26:35.834813+0000","last_active":"2024-09-21T00:26:35.836982+0000","last_peered":"2024-09-21T00:26:35.836982+0000","last_clean":"2024-09-21T00:26:35.836982+0000","last_became_active":"2024-09-21T00:26:35.834218+0000","last_became_peered":"2024-09-21T00:26:35.834218+0000","last_unstale":"2024-09-21T00:26:35.836982+0000","last_undegraded":"2024-09-21T00:26:35.836982+0000","last_fullsized":"2024-09-21T00:26:35.836982+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:41:29.077144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.830680+0000","last_change":"2024-09-21T00:26:35.830680+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.830680+0000","last_undegraded":"2024-09-21T00:26:35.830680+0000","last_fullsized":"2024-09-21T00:26:35.830680+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-21T00:26:35.842651+0000","last_change":"2024-09-21T00:26:35.841364+0000","last_active":"2024-09-21T00:26:35.842651+0000","last_peered":"2024-09-21T00:26:35.842651+0000","last_clean":"2024-09-21T00:26:35.842651+0000","last_became_active":"2024-09-21T00:26:35.840989+0000","last_became_peered":"2024-09-21T00:26:35.840989+0000","last_unstale":"2024-09-21T00:26:35.842651+0000","last_undegraded":"2024-09-21T00:26:35.842651+0000","last_fullsized":"2024-09-21T00:26:35.842651+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T00:47:37.835274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.829676+0000","last_change":"2024-09-21T00:26:35.829676+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.829676+0000","last_undegraded":"2024-09-21T00:26:35.829676+0000","last_fullsized":"2024-09-21T00:26:35.829676+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-21T00:26:35.842468+0000","last_change":"2024-09-21T00:26:35.836960+0000","last_active":"2024-09-21T00:26:35.842468+0000","last_peered":"2024-09-21T00:26:35.842468+0000","last_clean":"2024-09-21T00:26:35.842468+0000","last_became_active":"2024-09-21T00:26:35.836641+0000","last_became_peered":"2024-09-21T00:26:35.836641+0000","last_unstale":"2024-09-21T00:26:35.842468+0000","last_undegraded":"2024-09-21T00:26:35.842468+0000","last_fullsized":"2024-09-21T00:26:35.842468+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:50:54.008029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.825454+0000","last_change":"2024-09-21T00:26:35.825454+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.825454+0000","last_undegraded":"2024-09-21T00:26:35.825454+0000","last_fullsized":"2024-09-21T00:26:35.825454+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-21T00:26:35.836435+0000","last_change":"2024-09-21T00:26:35.834520+0000","last_active":"2024-09-21T00:26:35.836435+0000","last_peered":"2024-09-21T00:26:35.836435+0000","last_clean":"2024-09-21T00:26:35.836435+0000","last_became_active":"2024-09-21T00:26:35.834146+0000","last_became_peered":"2024-09-21T00:26:35.834146+0000","last_unstale":"2024-09-21T00:26:35.836435+0000","last_undegraded":"2024-09-21T00:26:35.836435+0000","last_fullsized":"2024-09-21T00:26:35.836435+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T00:47:37.835274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.826334+0000","last_change":"2024-09-21T00:26:35.826334+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.826334+0000","last_undegraded":"2024-09-21T00:26:35.826334+0000","last_fullsized":"2024-09-21T00:26:35.826334+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-21T00:26:35.843305+0000","last_change":"2024-09-21T00:26:35.840502+0000","last_active":"2024-09-21T00:26:35.843305+0000","last_peered":"2024-09-21T00:26:35.843305+0000","last_clean":"2024-09-21T00:26:35.843305+0000","last_became_active":"2024-09-21T00:26:35.840189+0000","last_became_peered":"2024-09-21T00:26:35.840189+0000","last_unstale":"2024-09-21T00:26:35.843305+0000","last_undegraded":"2024-09-21T00:26:35.843305+0000","last_fullsized":"2024-09-21T00:26:35.843305+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:41:13.666350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.827602+0000","last_change":"2024-09-21T00:26:35.827602+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.827602+0000","last_undegraded":"2024-09-21T00:26:35.827602+0000","last_fullsized":"2024-09-21T00:26:35.827602+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-21T00:26:36.831536+0000","last_change":"2024-09-21T00:26:35.842688+0000","last_active":"2024-09-21T00:26:36.831536+0000","last_peered":"2024-09-21T00:26:36.831536+0000","last_clean":"2024-09-21T00:26:36.831536+0000","last_became_active":"2024-09-21T00:26:35.842375+0000","last_became_peered":"2024-09-21T00:26:35.842375+0000","last_unstale":"2024-09-21T00:26:36.831536+0000","last_undegraded":"2024-09-21T00:26:36.831536+0000","last_fullsized":"2024-09-21T00:26:36.831536+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:58:30.617062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.828170+0000","last_change":"2024-09-21T00:26:35.828170+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.828170+0000","last_undegraded":"2024-09-21T00:26:35.828170+0000","last_fullsized":"2024-09-21T00:26:35.828170+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-21T00:26:35.844992+0000","last_change":"2024-09-21T00:26:35.839140+0000","last_active":"2024-09-21T00:26:35.844992+0000","last_peered":"2024-09-21T00:26:35.844992+0000","last_clean":"2024-09-21T00:26:35.844992+0000","last_became_active":"2024-09-21T00:26:35.838817+0000","last_became_peered":"2024-09-21T00:26:35.838817+0000","last_unstale":"2024-09-21T00:26:35.844992+0000","last_undegraded":"2024-09-21T00:26:35.844992+0000","last_fullsized":"2024-09-21T00:26:35.844992+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:07:46.709919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.828288+0000","last_change":"2024-09-21T00:26:35.828288+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.828288+0000","last_undegraded":"2024-09-21T00:26:35.828288+0000","last_fullsized":"2024-09-21T00:26:35.828288+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-21T00:26:35.850329+0000","last_change":"2024-09-21T00:26:35.846027+0000","last_active":"2024-09-21T00:26:35.850329+0000","last_peered":"2024-09-21T00:26:35.850329+0000","last_clean":"2024-09-21T00:26:35.850329+0000","last_became_active":"2024-09-21T00:26:35.845681+0000","last_became_peered":"2024-09-21T00:26:35.845681+0000","last_unstale":"2024-09-21T00:26:35.850329+0000","last_undegraded":"2024-09-21T00:26:35.850329+0000","last_fullsized":"2024-09-21T00:26:35.850329+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:11:52.985613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.827957+0000","last_change":"2024-09-21T00:26:35.827957+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.827957+0000","last_undegraded":"2024-09-21T00:26:35.827957+0000","last_fullsized":"2024-09-21T00:26:35.827957+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-21T00:26:35.850340+0000","last_change":"2024-09-21T00:26:35.845969+0000","last_active":"2024-09-21T00:26:35.850340+0000","last_peered":"2024-09-21T00:26:35.850340+0000","last_clean":"2024-09-21T00:26:35.850340+0000","last_became_active":"2024-09-21T00:26:35.845633+0000","last_became_peered":"2024-09-21T00:26:35.845633+0000","last_unstale":"2024-09-21T00:26:35.850340+0000","last_undegraded":"2024-09-21T00:26:35.850340+0000","last_fullsized":"2024-09-21T00:26:35.850340+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:41:29.077144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.830853+0000","last_change":"2024-09-21T00:26:35.830853+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.830853+0000","last_undegraded":"2024-09-21T00:26:35.830853+0000","last_fullsized":"2024-09-21T00:26:35.830853+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-21T00:26:35.845305+0000","last_change":"2024-09-21T00:26:35.840529+0000","last_active":"2024-09-21T00:26:35.845305+0000","last_peered":"2024-09-21T00:26:35.845305+0000","last_clean":"2024-09-21T00:26:35.845305+0000","last_became_active":"2024-09-21T00:26:35.840115+0000","last_became_peered":"2024-09-21T00:26:35.840115+0000","last_unstale":"2024-09-21T00:26:35.845305+0000","last_undegraded":"2024-09-21T00:26:35.845305+0000","last_fullsized":"2024-09-21T00:26:35.845305+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:33:02.079644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.824938+0000","last_change":"2024-09-21T00:26:35.824938+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.824938+0000","last_undegraded":"2024-09-21T00:26:35.824938+0000","last_fullsized":"2024-09-21T00:26:35.824938+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-21T00:26:35.838164+0000","last_change":"2024-09-21T00:26:35.835450+0000","last_active":"2024-09-21T00:26:35.838164+0000","last_peered":"2024-09-21T00:26:35.838164+0000","last_clean":"2024-09-21T00:26:35.838164+0000","last_became_active":"2024-09-21T00:26:35.835141+0000","last_became_peered":"2024-09-21T00:26:35.835141+0000","last_unstale":"2024-09-21T00:26:35.838164+0000","last_undegraded":"2024-09-21T00:26:35.838164+0000","last_fullsized":"2024-09-21T00:26:35.838164+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:11:09.781135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.824556+0000","last_change":"2024-09-21T00:26:35.824556+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.824556+0000","last_undegraded":"2024-09-21T00:26:35.824556+0000","last_fullsized":"2024-09-21T00:26:35.824556+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-21T00:26:36.831408+0000","last_change":"2024-09-21T00:26:35.841410+0000","last_active":"2024-09-21T00:26:36.831408+0000","last_peered":"2024-09-21T00:26:36.831408+0000","last_clean":"2024-09-21T00:26:36.831408+0000","last_became_active":"2024-09-21T00:26:35.841094+0000","last_became_peered":"2024-09-21T00:26:35.841094+0000","last_unstale":"2024-09-21T00:26:36.831408+0000","last_undegraded":"2024-09-21T00:26:36.831408+0000","last_fullsized":"2024-09-21T00:26:36.831408+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:48:22.317860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.827023+0000","last_change":"2024-09-21T00:26:35.827023+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.827023+0000","last_undegraded":"2024-09-21T00:26:35.827023+0000","last_fullsized":"2024-09-21T00:26:35.827023+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-21T00:26:35.843267+0000","last_change":"2024-09-21T00:26:35.836023+0000","last_active":"2024-09-21T00:26:35.843267+0000","last_peered":"2024-09-21T00:26:35.843267+0000","last_clean":"2024-09-21T00:26:35.843267+0000","last_became_active":"2024-09-21T00:26:35.835721+0000","last_became_peered":"2024-09-21T00:26:35.835721+0000","last_unstale":"2024-09-21T00:26:35.843267+0000","last_undegraded":"2024-09-21T00:26:35.843267+0000","last_fullsized":"2024-09-21T00:26:35.843267+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:43:45.222366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.825112+0000","last_change":"2024-09-21T00:26:35.825112+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.825112+0000","last_undegraded":"2024-09-21T00:26:35.825112+0000","last_fullsized":"2024-09-21T00:26:35.825112+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-21T00:26:35.843093+0000","last_change":"2024-09-21T00:26:35.837224+0000","last_active":"2024-09-21T00:26:35.843093+0000","last_peered":"2024-09-21T00:26:35.843093+0000","last_clean":"2024-09-21T00:26:35.843093+0000","last_became_active":"2024-09-21T00:26:35.836849+0000","last_became_peered":"2024-09-21T00:26:35.836849+0000","last_unstale":"2024-09-21T00:26:35.843093+0000","last_undegraded":"2024-09-21T00:26:35.843093+0000","last_fullsized":"2024-09-21T00:26:35.843093+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:59:09.929281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.829525+0000","last_change":"2024-09-21T00:26:35.829525+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.829525+0000","last_undegraded":"2024-09-21T00:26:35.829525+0000","last_fullsized":"2024-09-21T00:26:35.829525+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-21T00:26:35.842275+0000","last_change":"2024-09-21T00:26:35.837934+0000","last_active":"2024-09-21T00:26:35.842275+0000","last_peered":"2024-09-21T00:26:35.842275+0000","last_clean":"2024-09-21T00:26:35.842275+0000","last_became_active":"2024-09-21T00:26:35.837198+0000","last_became_peered":"2024-09-21T00:26:35.837198+0000","last_unstale":"2024-09-21T00:26:35.842275+0000","last_undegraded":"2024-09-21T00:26:35.842275+0000","last_fullsized":"2024-09-21T00:26:35.842275+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:04:35.145795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.827774+0000","last_change":"2024-09-21T00:26:35.827774+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.827774+0000","last_undegraded":"2024-09-21T00:26:35.827774+0000","last_fullsized":"2024-09-21T00:26:35.827774+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-21T00:26:35.836465+0000","last_change":"2024-09-21T00:26:35.832534+0000","last_active":"2024-09-21T00:26:35.836465+0000","last_peered":"2024-09-21T00:26:35.836465+0000","last_clean":"2024-09-21T00:26:35.836465+0000","last_became_active":"2024-09-21T00:26:35.832197+0000","last_became_peered":"2024-09-21T00:26:35.832197+0000","last_unstale":"2024-09-21T00:26:35.836465+0000","last_undegraded":"2024-09-21T00:26:35.836465+0000","last_fullsized":"2024-09-21T00:26:35.836465+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:28:49.838026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-21T00:26:36.844617+0000","last_change":"2024-09-21T00:26:36.841191+0000","last_active":"2024-09-21T00:26:36.844617+0000","last_peered":"2024-09-21T00:26:36.844617+0000","last_clean":"2024-09-21T00:26:36.844617+0000","last_became_active":"2024-09-21T00:26:36.840676+0000","last_became_peered":"2024-09-21T00:26:36.840676+0000","last_unstale":"2024-09-21T00:26:36.844617+0000","last_undegraded":"2024-09-21T00:26:36.844617+0000","last_fullsized":"2024-09-21T00:26:36.844617+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:16:54.958692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-21T00:26:35.842557+0000","last_change":"2024-09-21T00:26:35.840350+0000","last_active":"2024-09-21T00:26:35.842557+0000","last_peered":"2024-09-21T00:26:35.842557+0000","last_clean":"2024-09-21T00:26:35.842557+0000","last_became_active":"2024-09-21T00:26:35.840048+0000","last_became_peered":"2024-09-21T00:26:35.840048+0000","last_unstale":"2024-09-21T00:26:35.842557+0000","last_undegraded":"2024-09-21T00:26:35.842557+0000","last_fullsized":"2024-09-21T00:26:35.842557+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:19:54.299718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.829609+0000","last_change":"2024-09-21T00:26:35.829609+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.829609+0000","last_undegraded":"2024-09-21T00:26:35.829609+0000","last_fullsized":"2024-09-21T00:26:35.829609+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-21T00:26:35.846174+0000","last_change":"2024-09-21T00:26:35.838940+0000","last_active":"2024-09-21T00:26:35.846174+0000","last_peered":"2024-09-21T00:26:35.846174+0000","last_clean":"2024-09-21T00:26:35.846174+0000","last_became_active":"2024-09-21T00:26:35.838616+0000","last_became_peered":"2024-09-21T00:26:35.838616+0000","last_unstale":"2024-09-21T00:26:35.846174+0000","last_undegraded":"2024-09-21T00:26:35.846174+0000","last_fullsized":"2024-09-21T00:26:35.846174+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:01:07.494519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.826730+0000","last_change":"2024-09-21T00:26:35.826730+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.826730+0000","last_undegraded":"2024-09-21T00:26:35.826730+0000","last_fullsized":"2024-09-21T00:26:35.826730+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-21T00:26:35.841029+0000","last_change":"2024-09-21T00:26:35.838748+0000","last_active":"2024-09-21T00:26:35.841029+0000","last_peered":"2024-09-21T00:26:35.841029+0000","last_clean":"2024-09-21T00:26:35.841029+0000","last_became_active":"2024-09-21T00:26:35.838445+0000","last_became_peered":"2024-09-21T00:26:35.838445+0000","last_unstale":"2024-09-21T00:26:35.841029+0000","last_undegraded":"2024-09-21T00:26:35.841029+0000","last_fullsized":"2024-09-21T00:26:35.841029+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:44:34.009179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.825823+0000","last_change":"2024-09-21T00:26:35.825823+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.825823+0000","last_undegraded":"2024-09-21T00:26:35.825823+0000","last_fullsized":"2024-09-21T00:26:35.825823+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-21T00:26:35.843172+0000","last_change":"2024-09-21T00:26:35.840565+0000","last_active":"2024-09-21T00:26:35.843172+0000","last_peered":"2024-09-21T00:26:35.843172+0000","last_clean":"2024-09-21T00:26:35.843172+0000","last_became_active":"2024-09-21T00:26:35.840265+0000","last_became_peered":"2024-09-21T00:26:35.840265+0000","last_unstale":"2024-09-21T00:26:35.843172+0000","last_undegraded":"2024-09-21T00:26:35.843172+0000","last_fullsized":"2024-09-21T00:26:35.843172+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:14:15.665953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-21T00:26:36.848314+0000","last_change":"2024-09-21T00:26:36.842397+0000","last_active":"2024-09-21T00:26:36.848314+0000","last_peered":"2024-09-21T00:26:36.848314+0000","last_clean":"2024-09-21T00:26:36.848314+0000","last_became_active":"2024-09-21T00:26:36.842095+0000","last_became_peered":"2024-09-21T00:26:36.842095+0000","last_unstale":"2024-09-21T00:26:36.848314+0000","last_undegraded":"2024-09-21T00:26:36.848314+0000","last_fullsized":"2024-09-21T00:26:36.848314+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:50:12.134645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-21T00:26:35.837296+0000","last_change":"2024-09-21T00:26:35.835982+0000","last_active":"2024-09-21T00:26:35.837296+0000","last_peered":"2024-09-21T00:26:35.837296+0000","last_clean":"2024-09-21T00:26:35.837296+0000","last_became_active":"2024-09-21T00:26:35.835677+0000","last_became_peered":"2024-09-21T00:26:35.835677+0000","last_unstale":"2024-09-21T00:26:35.837296+0000","last_undegraded":"2024-09-21T00:26:35.837296+0000","last_fullsized":"2024-09-21T00:26:35.837296+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:46:16.688217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.826853+0000","last_change":"2024-09-21T00:26:35.826853+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.826853+0000","last_undegraded":"2024-09-21T00:26:35.826853+0000","last_fullsized":"2024-09-21T00:26:35.826853+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-21T00:26:35.842371+0000","last_change":"2024-09-21T00:26:35.838318+0000","last_active":"2024-09-21T00:26:35.842371+0000","last_peered":"2024-09-21T00:26:35.842371+0000","last_clean":"2024-09-21T00:26:35.842371+0000","last_became_active":"2024-09-21T00:26:35.836426+0000","last_became_peered":"2024-09-21T00:26:35.836426+0000","last_unstale":"2024-09-21T00:26:35.842371+0000","last_undegraded":"2024-09-21T00:26:35.842371+0000","last_fullsized":"2024-09-21T00:26:35.842371+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:48:50.639166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.827493+0000","last_change":"2024-09-21T00:26:35.827493+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.827493+0000","last_undegraded":"2024-09-21T00:26:35.827493+0000","last_fullsized":"2024-09-21T00:26:35.827493+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-21T00:26:35.841994+0000","last_change":"2024-09-21T00:26:35.836745+0000","last_active":"2024-09-21T00:26:35.841994+0000","last_peered":"2024-09-21T00:26:35.841994+0000","last_clean":"2024-09-21T00:26:35.841994+0000","last_became_active":"2024-09-21T00:26:35.836427+0000","last_became_peered":"2024-09-21T00:26:35.836427+0000","last_unstale":"2024-09-21T00:26:35.841994+0000","last_undegraded":"2024-09-21T00:26:35.841994+0000","last_fullsized":"2024-09-21T00:26:35.841994+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:33:15.124204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.830688+0000","last_change":"2024-09-21T00:26:35.830688+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.830688+0000","last_undegraded":"2024-09-21T00:26:35.830688+0000","last_fullsized":"2024-09-21T00:26:35.830688+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-21T00:26:35.845045+0000","last_change":"2024-09-21T00:26:35.839244+0000","last_active":"2024-09-21T00:26:35.845045+0000","last_peered":"2024-09-21T00:26:35.845045+0000","last_clean":"2024-09-21T00:26:35.845045+0000","last_became_active":"2024-09-21T00:26:35.838876+0000","last_became_peered":"2024-09-21T00:26:35.838876+0000","last_unstale":"2024-09-21T00:26:35.845045+0000","last_undegraded":"2024-09-21T00:26:35.845045+0000","last_fullsized":"2024-09-21T00:26:35.845045+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:02:08.223701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":136,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-21T00:26:35.831437+0000","last_change":"2024-09-21T00:25:38.725792+0000","last_active":"2024-09-21T00:26:35.831437+0000","last_peered":"2024-09-21T00:26:35.831437+0000","last_clean":"2024-09-21T00:26:35.831437+0000","last_became_active":"2024-09-21T00:25:38.724668+0000","last_became_peered":"2024-09-21T00:25:38.724668+0000","last_unstale":"2024-09-21T00:26:35.831437+0000","last_undegraded":"2024-09-21T00:26:35.831437+0000","last_fullsized":"2024-09-21T00:26:35.831437+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:37.163216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:37.163216+0000","last_clean_scrub_stamp":"2024-09-21T00:25:37.163216+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:14:36.588660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509455,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28008,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343832,"statfs":{"total":96636764160,"available":96608083968,"internally_reserved":0,"allocated":999424,"data_stored":758396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509456,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":393216,"data_stored":161361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509455,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27428,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":405504,"data_stored":168028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509454,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":393216,"data_stored":161361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509456,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":393216,"data_stored":161361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509456,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":393216,"data_stored":161361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509456,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27992,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343848,"statfs":{"total":96636764160,"available":96608100352,"internally_reserved":0,"allocated":987136,"data_stored":751729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574864,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27440,"kb_used_data":412,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":421888,"data_stored":175285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T00:26:41.215 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T00:26:41.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.361+0000 7fdf8c5ad640 1 -- 172.21.15.112:0/1463580822 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdf6c0039e0 msgr2=0x7fdf6c003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:41.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.361+0000 7fdf8c5ad640 1 --2- 172.21.15.112:0/1463580822 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdf6c0039e0 0x7fdf6c003e10 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fdf74009ed0 tx=0x7fdf6c005830 comp rx=0 tx=0).stop 2024-09-21T00:26:41.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.361+0000 7fdf8ab4a640 1 -- 172.21.15.112:0/1463580822 reap_dead start 2024-09-21T00:26:41.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.362+0000 7fdf8c5ad640 1 -- 172.21.15.112:0/1463580822 shutdown_connections 2024-09-21T00:26:41.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.362+0000 7fdf8c5ad640 1 -- 172.21.15.112:0/1463580822 >> 172.21.15.112:0/1463580822 conn(0x7fdf840d1140 msgr2=0x7fdf840d1550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:41.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.362+0000 7fdf8c5ad640 1 -- 172.21.15.112:0/1463580822 shutdown_connections 2024-09-21T00:26:41.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.362+0000 7fdf8c5ad640 1 -- 172.21.15.112:0/1463580822 wait complete. 2024-09-21T00:26:41.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.363+0000 7fdf8c5ad640 1 Processor -- start 2024-09-21T00:26:41.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.363+0000 7fdf8c5ad640 1 -- start start 2024-09-21T00:26:41.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.365+0000 7fdf8c5ad640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdf6c0039e0 0x7fdf840b5690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:41.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.365+0000 7fdf8c5ad640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdf84158620 0x7fdf841e57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:41.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.365+0000 7fdf8c5ad640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdf841e5d00 0x7fdf840b3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:41.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.365+0000 7fdf8c5ad640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fdf84070370 con 0x7fdf841e5d00 2024-09-21T00:26:41.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.365+0000 7fdf8c5ad640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fdf840701f0 con 0x7fdf84158620 2024-09-21T00:26:41.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.365+0000 7fdf8c5ad640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fdf8406ec50 con 0x7fdf6c0039e0 2024-09-21T00:26:41.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.365+0000 7fdf8ab4a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdf841e5d00 0x7fdf840b3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:41.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.365+0000 7fdf8ab4a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdf841e5d00 0x7fdf840b3f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36812/0 (socket says 172.21.15.112:36812) 2024-09-21T00:26:41.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.365+0000 7fdf8ab4a640 1 -- 172.21.15.112:0/4267709433 learned_addr learned my addr 172.21.15.112:0/4267709433 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:26:41.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.365+0000 7fdf8a349640 1 --2- 172.21.15.112:0/4267709433 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdf6c0039e0 0x7fdf840b5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:41.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.365+0000 7fdf89b48640 1 --2- 172.21.15.112:0/4267709433 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdf84158620 0x7fdf841e57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:41.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.365+0000 7fdf8ab4a640 1 -- 172.21.15.112:0/4267709433 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdf6c0039e0 msgr2=0x7fdf840b5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:41.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.365+0000 7fdf8ab4a640 1 --2- 172.21.15.112:0/4267709433 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdf6c0039e0 0x7fdf840b5690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:41.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.365+0000 7fdf8ab4a640 1 -- 172.21.15.112:0/4267709433 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdf84158620 msgr2=0x7fdf841e57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:41.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.365+0000 7fdf8ab4a640 1 --2- 172.21.15.112:0/4267709433 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdf84158620 0x7fdf841e57c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:41.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.365+0000 7fdf8ab4a640 1 -- 172.21.15.112:0/4267709433 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf840b4c30 con 0x7fdf841e5d00 2024-09-21T00:26:41.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.365+0000 7fdf8ab4a640 1 --2- 172.21.15.112:0/4267709433 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdf841e5d00 0x7fdf840b3f70 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fdf6c004780 tx=0x7fdf7c001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:41.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.365+0000 7fdf82ffd640 1 -- 172.21.15.112:0/4267709433 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf7c0107d0 con 0x7fdf841e5d00 2024-09-21T00:26:41.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.365+0000 7fdf82ffd640 1 -- 172.21.15.112:0/4267709433 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdf7c016070 con 0x7fdf841e5d00 2024-09-21T00:26:41.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.365+0000 7fdf8c5ad640 1 -- 172.21.15.112:0/4267709433 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf84157b40 con 0x7fdf841e5d00 2024-09-21T00:26:41.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.365+0000 7fdf82ffd640 1 -- 172.21.15.112:0/4267709433 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf7c010d00 con 0x7fdf841e5d00 2024-09-21T00:26:41.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.365+0000 7fdf8c5ad640 1 -- 172.21.15.112:0/4267709433 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf84158100 con 0x7fdf841e5d00 2024-09-21T00:26:41.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.367+0000 7fdf8c5ad640 1 -- 172.21.15.112:0/4267709433 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf8406f0f0 con 0x7fdf841e5d00 2024-09-21T00:26:41.370 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.367+0000 7fdf82ffd640 1 -- 172.21.15.112:0/4267709433 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fdf7c01b460 con 0x7fdf841e5d00 2024-09-21T00:26:41.370 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.368+0000 7fdf82ffd640 1 --2- 172.21.15.112:0/4267709433 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fdf70078bc0 0x7fdf7007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:41.370 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.368+0000 7fdf82ffd640 1 -- 172.21.15.112:0/4267709433 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7fdf7c09b700 con 0x7fdf841e5d00 2024-09-21T00:26:41.370 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.368+0000 7fdf8a349640 1 --2- 172.21.15.112:0/4267709433 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fdf70078bc0 0x7fdf7007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:41.370 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.369+0000 7fdf8a349640 1 --2- 172.21.15.112:0/4267709433 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fdf70078bc0 0x7fdf7007b0d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fdf74035000 tx=0x7fdf74004a40 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:41.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.372+0000 7fdf82ffd640 1 -- 172.21.15.112:0/4267709433 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fdf7c063a50 con 0x7fdf841e5d00 2024-09-21T00:26:41.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.546+0000 7fdf8c5ad640 1 -- 172.21.15.112:0/4267709433 --> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fdf840b4f10 con 0x7fdf70078bc0 2024-09-21T00:26:41.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.551+0000 7fdf82ffd640 1 -- 172.21.15.112:0/4267709433 <== mgr.4103 v2:172.21.15.187:6800/2921522906 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247151 (secure 0 0 0) 0x7fdf74039040 con 0x7fdf70078bc0 2024-09-21T00:26:41.555 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:26:41.556 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2024-09-21T00:26:41.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.554+0000 7fdf8c5ad640 1 -- 172.21.15.112:0/4267709433 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fdf70078bc0 msgr2=0x7fdf7007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:41.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.554+0000 7fdf8c5ad640 1 --2- 172.21.15.112:0/4267709433 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fdf70078bc0 0x7fdf7007b0d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fdf74035000 tx=0x7fdf74004a40 comp rx=0 tx=0).stop 2024-09-21T00:26:41.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.554+0000 7fdf8c5ad640 1 -- 172.21.15.112:0/4267709433 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdf841e5d00 msgr2=0x7fdf840b3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:41.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.554+0000 7fdf8c5ad640 1 --2- 172.21.15.112:0/4267709433 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdf841e5d00 0x7fdf840b3f70 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fdf6c004780 tx=0x7fdf7c001710 comp rx=0 tx=0).stop 2024-09-21T00:26:41.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.554+0000 7fdf8c5ad640 1 -- 172.21.15.112:0/4267709433 shutdown_connections 2024-09-21T00:26:41.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.554+0000 7fdf8c5ad640 1 --2- 172.21.15.112:0/4267709433 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fdf70078bc0 0x7fdf7007b0d0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:41.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.554+0000 7fdf8c5ad640 1 --2- 172.21.15.112:0/4267709433 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdf841e5d00 0x7fdf840b3f70 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:41.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.554+0000 7fdf8c5ad640 1 --2- 172.21.15.112:0/4267709433 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdf84158620 0x7fdf841e57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:41.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.554+0000 7fdf8c5ad640 1 --2- 172.21.15.112:0/4267709433 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdf6c0039e0 0x7fdf840b5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:41.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.554+0000 7fdf8c5ad640 1 -- 172.21.15.112:0/4267709433 >> 172.21.15.112:0/4267709433 conn(0x7fdf840d1140 msgr2=0x7fdf841593e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:41.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.554+0000 7fdf8c5ad640 1 -- 172.21.15.112:0/4267709433 shutdown_connections 2024-09-21T00:26:41.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.554+0000 7fdf8c5ad640 1 -- 172.21.15.112:0/4267709433 wait complete. 2024-09-21T00:26:41.567 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":65,"stamp":"2024-09-21T00:26:41.101777+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":104,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":89,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220516,"kb_used_data":4284,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754204,"statfs":{"total":773094113280,"available":772868304896,"internally_reserved":0,"allocated":4386816,"data_stored":2498882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001572"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.829346+0000","last_change":"2024-09-21T00:26:35.829346+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.829346+0000","last_undegraded":"2024-09-21T00:26:35.829346+0000","last_fullsized":"2024-09-21T00:26:35.829346+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.827322+0000","last_change":"2024-09-21T00:26:35.827322+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.827322+0000","last_undegraded":"2024-09-21T00:26:35.827322+0000","last_fullsized":"2024-09-21T00:26:35.827322+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.828578+0000","last_change":"2024-09-21T00:26:35.828578+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.828578+0000","last_undegraded":"2024-09-21T00:26:35.828578+0000","last_fullsized":"2024-09-21T00:26:35.828578+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.825190+0000","last_change":"2024-09-21T00:26:35.825190+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.825190+0000","last_undegraded":"2024-09-21T00:26:35.825190+0000","last_fullsized":"2024-09-21T00:26:35.825190+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.825161+0000","last_change":"2024-09-21T00:26:35.825161+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.825161+0000","last_undegraded":"2024-09-21T00:26:35.825161+0000","last_fullsized":"2024-09-21T00:26:35.825161+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-21T00:26:36.844469+0000","last_change":"2024-09-21T00:26:36.841381+0000","last_active":"2024-09-21T00:26:36.844469+0000","last_peered":"2024-09-21T00:26:36.844469+0000","last_clean":"2024-09-21T00:26:36.844469+0000","last_became_active":"2024-09-21T00:26:36.841094+0000","last_became_peered":"2024-09-21T00:26:36.841094+0000","last_unstale":"2024-09-21T00:26:36.844469+0000","last_undegraded":"2024-09-21T00:26:36.844469+0000","last_fullsized":"2024-09-21T00:26:36.844469+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:55:25.224768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.832564+0000","last_change":"2024-09-21T00:26:35.832564+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.832564+0000","last_undegraded":"2024-09-21T00:26:35.832564+0000","last_fullsized":"2024-09-21T00:26:35.832564+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.830845+0000","last_change":"2024-09-21T00:26:35.830845+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.830845+0000","last_undegraded":"2024-09-21T00:26:35.830845+0000","last_fullsized":"2024-09-21T00:26:35.830845+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.830587+0000","last_change":"2024-09-21T00:26:35.830587+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.830587+0000","last_undegraded":"2024-09-21T00:26:35.830587+0000","last_fullsized":"2024-09-21T00:26:35.830587+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.833967+0000","last_change":"2024-09-21T00:26:35.833967+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.833967+0000","last_undegraded":"2024-09-21T00:26:35.833967+0000","last_fullsized":"2024-09-21T00:26:35.833967+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.832457+0000","last_change":"2024-09-21T00:26:35.832457+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.832457+0000","last_undegraded":"2024-09-21T00:26:35.832457+0000","last_fullsized":"2024-09-21T00:26:35.832457+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.828280+0000","last_change":"2024-09-21T00:26:35.828280+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.828280+0000","last_undegraded":"2024-09-21T00:26:35.828280+0000","last_fullsized":"2024-09-21T00:26:35.828280+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.831355+0000","last_change":"2024-09-21T00:26:35.831355+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.831355+0000","last_undegraded":"2024-09-21T00:26:35.831355+0000","last_fullsized":"2024-09-21T00:26:35.831355+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.827439+0000","last_change":"2024-09-21T00:26:35.827439+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.827439+0000","last_undegraded":"2024-09-21T00:26:35.827439+0000","last_fullsized":"2024-09-21T00:26:35.827439+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.829991+0000","last_change":"2024-09-21T00:26:35.829991+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.829991+0000","last_undegraded":"2024-09-21T00:26:35.829991+0000","last_fullsized":"2024-09-21T00:26:35.829991+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.828731+0000","last_change":"2024-09-21T00:26:35.828731+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.828731+0000","last_undegraded":"2024-09-21T00:26:35.828731+0000","last_fullsized":"2024-09-21T00:26:35.828731+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.828322+0000","last_change":"2024-09-21T00:26:35.828322+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.828322+0000","last_undegraded":"2024-09-21T00:26:35.828322+0000","last_fullsized":"2024-09-21T00:26:35.828322+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.827556+0000","last_change":"2024-09-21T00:26:35.827556+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.827556+0000","last_undegraded":"2024-09-21T00:26:35.827556+0000","last_fullsized":"2024-09-21T00:26:35.827556+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.823845+0000","last_change":"2024-09-21T00:26:35.823845+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.823845+0000","last_undegraded":"2024-09-21T00:26:35.823845+0000","last_fullsized":"2024-09-21T00:26:35.823845+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.827691+0000","last_change":"2024-09-21T00:26:35.827691+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.827691+0000","last_undegraded":"2024-09-21T00:26:35.827691+0000","last_fullsized":"2024-09-21T00:26:35.827691+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.831846+0000","last_change":"2024-09-21T00:26:35.831846+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.831846+0000","last_undegraded":"2024-09-21T00:26:35.831846+0000","last_fullsized":"2024-09-21T00:26:35.831846+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.823745+0000","last_change":"2024-09-21T00:26:35.823745+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.823745+0000","last_undegraded":"2024-09-21T00:26:35.823745+0000","last_fullsized":"2024-09-21T00:26:35.823745+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.827636+0000","last_change":"2024-09-21T00:26:35.827636+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.827636+0000","last_undegraded":"2024-09-21T00:26:35.827636+0000","last_fullsized":"2024-09-21T00:26:35.827636+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.823856+0000","last_change":"2024-09-21T00:26:35.823856+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.823856+0000","last_undegraded":"2024-09-21T00:26:35.823856+0000","last_fullsized":"2024-09-21T00:26:35.823856+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.826279+0000","last_change":"2024-09-21T00:26:35.826279+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.826279+0000","last_undegraded":"2024-09-21T00:26:35.826279+0000","last_fullsized":"2024-09-21T00:26:35.826279+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-21T00:26:36.859512+0000","last_change":"2024-09-21T00:26:36.856780+0000","last_active":"2024-09-21T00:26:36.859512+0000","last_peered":"2024-09-21T00:26:36.859512+0000","last_clean":"2024-09-21T00:26:36.859512+0000","last_became_active":"2024-09-21T00:26:36.856482+0000","last_became_peered":"2024-09-21T00:26:36.856482+0000","last_unstale":"2024-09-21T00:26:36.859512+0000","last_undegraded":"2024-09-21T00:26:36.859512+0000","last_fullsized":"2024-09-21T00:26:36.859512+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:58:37.004181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.825808+0000","last_change":"2024-09-21T00:26:35.825808+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.825808+0000","last_undegraded":"2024-09-21T00:26:35.825808+0000","last_fullsized":"2024-09-21T00:26:35.825808+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.830953+0000","last_change":"2024-09-21T00:26:35.830953+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.830953+0000","last_undegraded":"2024-09-21T00:26:35.830953+0000","last_fullsized":"2024-09-21T00:26:35.830953+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.824648+0000","last_change":"2024-09-21T00:26:35.824648+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.824648+0000","last_undegraded":"2024-09-21T00:26:35.824648+0000","last_fullsized":"2024-09-21T00:26:35.824648+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.826507+0000","last_change":"2024-09-21T00:26:35.826507+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.826507+0000","last_undegraded":"2024-09-21T00:26:35.826507+0000","last_fullsized":"2024-09-21T00:26:35.826507+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.826999+0000","last_change":"2024-09-21T00:26:35.826999+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.826999+0000","last_undegraded":"2024-09-21T00:26:35.826999+0000","last_fullsized":"2024-09-21T00:26:35.826999+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-21T00:26:36.838547+0000","last_change":"2024-09-21T00:26:36.836602+0000","last_active":"2024-09-21T00:26:36.838547+0000","last_peered":"2024-09-21T00:26:36.838547+0000","last_clean":"2024-09-21T00:26:36.838547+0000","last_became_active":"2024-09-21T00:26:36.835713+0000","last_became_peered":"2024-09-21T00:26:36.835713+0000","last_unstale":"2024-09-21T00:26:36.838547+0000","last_undegraded":"2024-09-21T00:26:36.838547+0000","last_fullsized":"2024-09-21T00:26:36.838547+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:23:48.458736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.828100+0000","last_change":"2024-09-21T00:26:35.828100+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.828100+0000","last_undegraded":"2024-09-21T00:26:35.828100+0000","last_fullsized":"2024-09-21T00:26:35.828100+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-21T00:26:35.844965+0000","last_change":"2024-09-21T00:26:35.838321+0000","last_active":"2024-09-21T00:26:35.844965+0000","last_peered":"2024-09-21T00:26:35.844965+0000","last_clean":"2024-09-21T00:26:35.844965+0000","last_became_active":"2024-09-21T00:26:35.838022+0000","last_became_peered":"2024-09-21T00:26:35.838022+0000","last_unstale":"2024-09-21T00:26:35.844965+0000","last_undegraded":"2024-09-21T00:26:35.844965+0000","last_fullsized":"2024-09-21T00:26:35.844965+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:05:48.525312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-21T00:26:36.859717+0000","last_change":"2024-09-21T00:26:36.856958+0000","last_active":"2024-09-21T00:26:36.859717+0000","last_peered":"2024-09-21T00:26:36.859717+0000","last_clean":"2024-09-21T00:26:36.859717+0000","last_became_active":"2024-09-21T00:26:36.856395+0000","last_became_peered":"2024-09-21T00:26:36.856395+0000","last_unstale":"2024-09-21T00:26:36.859717+0000","last_undegraded":"2024-09-21T00:26:36.859717+0000","last_fullsized":"2024-09-21T00:26:36.859717+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:15:12.241543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-21T00:26:35.839447+0000","last_change":"2024-09-21T00:26:35.837664+0000","last_active":"2024-09-21T00:26:35.839447+0000","last_peered":"2024-09-21T00:26:35.839447+0000","last_clean":"2024-09-21T00:26:35.839447+0000","last_became_active":"2024-09-21T00:26:35.837355+0000","last_became_peered":"2024-09-21T00:26:35.837355+0000","last_unstale":"2024-09-21T00:26:35.839447+0000","last_undegraded":"2024-09-21T00:26:35.839447+0000","last_fullsized":"2024-09-21T00:26:35.839447+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T00:41:22.116310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.824468+0000","last_change":"2024-09-21T00:26:35.824468+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.824468+0000","last_undegraded":"2024-09-21T00:26:35.824468+0000","last_fullsized":"2024-09-21T00:26:35.824468+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"37'2","reported_seq":15,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-21T00:26:36.853187+0000","last_change":"2024-09-21T00:26:35.840905+0000","last_active":"2024-09-21T00:26:36.853187+0000","last_peered":"2024-09-21T00:26:36.853187+0000","last_clean":"2024-09-21T00:26:36.853187+0000","last_became_active":"2024-09-21T00:26:35.840594+0000","last_became_peered":"2024-09-21T00:26:35.840594+0000","last_unstale":"2024-09-21T00:26:36.853187+0000","last_undegraded":"2024-09-21T00:26:36.853187+0000","last_fullsized":"2024-09-21T00:26:36.853187+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:54:48.838959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.829845+0000","last_change":"2024-09-21T00:26:35.829845+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.829845+0000","last_undegraded":"2024-09-21T00:26:35.829845+0000","last_fullsized":"2024-09-21T00:26:35.829845+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-21T00:26:35.842277+0000","last_change":"2024-09-21T00:26:35.836092+0000","last_active":"2024-09-21T00:26:35.842277+0000","last_peered":"2024-09-21T00:26:35.842277+0000","last_clean":"2024-09-21T00:26:35.842277+0000","last_became_active":"2024-09-21T00:26:35.835424+0000","last_became_peered":"2024-09-21T00:26:35.835424+0000","last_unstale":"2024-09-21T00:26:35.842277+0000","last_undegraded":"2024-09-21T00:26:35.842277+0000","last_fullsized":"2024-09-21T00:26:35.842277+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:21:21.377354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.826825+0000","last_change":"2024-09-21T00:26:35.826825+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.826825+0000","last_undegraded":"2024-09-21T00:26:35.826825+0000","last_fullsized":"2024-09-21T00:26:35.826825+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-21T00:26:35.841980+0000","last_change":"2024-09-21T00:26:35.835778+0000","last_active":"2024-09-21T00:26:35.841980+0000","last_peered":"2024-09-21T00:26:35.841980+0000","last_clean":"2024-09-21T00:26:35.841980+0000","last_became_active":"2024-09-21T00:26:35.835426+0000","last_became_peered":"2024-09-21T00:26:35.835426+0000","last_unstale":"2024-09-21T00:26:35.841980+0000","last_undegraded":"2024-09-21T00:26:35.841980+0000","last_fullsized":"2024-09-21T00:26:35.841980+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:49:14.000042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.829387+0000","last_change":"2024-09-21T00:26:35.829387+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.829387+0000","last_undegraded":"2024-09-21T00:26:35.829387+0000","last_fullsized":"2024-09-21T00:26:35.829387+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-21T00:26:35.850340+0000","last_change":"2024-09-21T00:26:35.846072+0000","last_active":"2024-09-21T00:26:35.850340+0000","last_peered":"2024-09-21T00:26:35.850340+0000","last_clean":"2024-09-21T00:26:35.850340+0000","last_became_active":"2024-09-21T00:26:35.845724+0000","last_became_peered":"2024-09-21T00:26:35.845724+0000","last_unstale":"2024-09-21T00:26:35.850340+0000","last_undegraded":"2024-09-21T00:26:35.850340+0000","last_fullsized":"2024-09-21T00:26:35.850340+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:49:14.000042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.825895+0000","last_change":"2024-09-21T00:26:35.825895+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.825895+0000","last_undegraded":"2024-09-21T00:26:35.825895+0000","last_fullsized":"2024-09-21T00:26:35.825895+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-21T00:26:36.831276+0000","last_change":"2024-09-21T00:26:35.842674+0000","last_active":"2024-09-21T00:26:36.831276+0000","last_peered":"2024-09-21T00:26:36.831276+0000","last_clean":"2024-09-21T00:26:36.831276+0000","last_became_active":"2024-09-21T00:26:35.842339+0000","last_became_peered":"2024-09-21T00:26:35.842339+0000","last_unstale":"2024-09-21T00:26:36.831276+0000","last_undegraded":"2024-09-21T00:26:36.831276+0000","last_fullsized":"2024-09-21T00:26:36.831276+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:56:47.623787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.828735+0000","last_change":"2024-09-21T00:26:35.828735+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.828735+0000","last_undegraded":"2024-09-21T00:26:35.828735+0000","last_fullsized":"2024-09-21T00:26:35.828735+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-21T00:26:35.839220+0000","last_change":"2024-09-21T00:26:35.835252+0000","last_active":"2024-09-21T00:26:35.839220+0000","last_peered":"2024-09-21T00:26:35.839220+0000","last_clean":"2024-09-21T00:26:35.839220+0000","last_became_active":"2024-09-21T00:26:35.834768+0000","last_became_peered":"2024-09-21T00:26:35.834768+0000","last_unstale":"2024-09-21T00:26:35.839220+0000","last_undegraded":"2024-09-21T00:26:35.839220+0000","last_fullsized":"2024-09-21T00:26:35.839220+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:29:33.119014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.824310+0000","last_change":"2024-09-21T00:26:35.824310+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.824310+0000","last_undegraded":"2024-09-21T00:26:35.824310+0000","last_fullsized":"2024-09-21T00:26:35.824310+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-21T00:26:35.836982+0000","last_change":"2024-09-21T00:26:35.834813+0000","last_active":"2024-09-21T00:26:35.836982+0000","last_peered":"2024-09-21T00:26:35.836982+0000","last_clean":"2024-09-21T00:26:35.836982+0000","last_became_active":"2024-09-21T00:26:35.834218+0000","last_became_peered":"2024-09-21T00:26:35.834218+0000","last_unstale":"2024-09-21T00:26:35.836982+0000","last_undegraded":"2024-09-21T00:26:35.836982+0000","last_fullsized":"2024-09-21T00:26:35.836982+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:41:29.077144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.830680+0000","last_change":"2024-09-21T00:26:35.830680+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.830680+0000","last_undegraded":"2024-09-21T00:26:35.830680+0000","last_fullsized":"2024-09-21T00:26:35.830680+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-21T00:26:35.842651+0000","last_change":"2024-09-21T00:26:35.841364+0000","last_active":"2024-09-21T00:26:35.842651+0000","last_peered":"2024-09-21T00:26:35.842651+0000","last_clean":"2024-09-21T00:26:35.842651+0000","last_became_active":"2024-09-21T00:26:35.840989+0000","last_became_peered":"2024-09-21T00:26:35.840989+0000","last_unstale":"2024-09-21T00:26:35.842651+0000","last_undegraded":"2024-09-21T00:26:35.842651+0000","last_fullsized":"2024-09-21T00:26:35.842651+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T00:47:37.835274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.829676+0000","last_change":"2024-09-21T00:26:35.829676+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.829676+0000","last_undegraded":"2024-09-21T00:26:35.829676+0000","last_fullsized":"2024-09-21T00:26:35.829676+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-21T00:26:35.842468+0000","last_change":"2024-09-21T00:26:35.836960+0000","last_active":"2024-09-21T00:26:35.842468+0000","last_peered":"2024-09-21T00:26:35.842468+0000","last_clean":"2024-09-21T00:26:35.842468+0000","last_became_active":"2024-09-21T00:26:35.836641+0000","last_became_peered":"2024-09-21T00:26:35.836641+0000","last_unstale":"2024-09-21T00:26:35.842468+0000","last_undegraded":"2024-09-21T00:26:35.842468+0000","last_fullsized":"2024-09-21T00:26:35.842468+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:50:54.008029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.825454+0000","last_change":"2024-09-21T00:26:35.825454+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.825454+0000","last_undegraded":"2024-09-21T00:26:35.825454+0000","last_fullsized":"2024-09-21T00:26:35.825454+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-21T00:26:35.836435+0000","last_change":"2024-09-21T00:26:35.834520+0000","last_active":"2024-09-21T00:26:35.836435+0000","last_peered":"2024-09-21T00:26:35.836435+0000","last_clean":"2024-09-21T00:26:35.836435+0000","last_became_active":"2024-09-21T00:26:35.834146+0000","last_became_peered":"2024-09-21T00:26:35.834146+0000","last_unstale":"2024-09-21T00:26:35.836435+0000","last_undegraded":"2024-09-21T00:26:35.836435+0000","last_fullsized":"2024-09-21T00:26:35.836435+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T00:47:37.835274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.826334+0000","last_change":"2024-09-21T00:26:35.826334+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.826334+0000","last_undegraded":"2024-09-21T00:26:35.826334+0000","last_fullsized":"2024-09-21T00:26:35.826334+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-21T00:26:35.843305+0000","last_change":"2024-09-21T00:26:35.840502+0000","last_active":"2024-09-21T00:26:35.843305+0000","last_peered":"2024-09-21T00:26:35.843305+0000","last_clean":"2024-09-21T00:26:35.843305+0000","last_became_active":"2024-09-21T00:26:35.840189+0000","last_became_peered":"2024-09-21T00:26:35.840189+0000","last_unstale":"2024-09-21T00:26:35.843305+0000","last_undegraded":"2024-09-21T00:26:35.843305+0000","last_fullsized":"2024-09-21T00:26:35.843305+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:41:13.666350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.827602+0000","last_change":"2024-09-21T00:26:35.827602+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.827602+0000","last_undegraded":"2024-09-21T00:26:35.827602+0000","last_fullsized":"2024-09-21T00:26:35.827602+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-21T00:26:36.831536+0000","last_change":"2024-09-21T00:26:35.842688+0000","last_active":"2024-09-21T00:26:36.831536+0000","last_peered":"2024-09-21T00:26:36.831536+0000","last_clean":"2024-09-21T00:26:36.831536+0000","last_became_active":"2024-09-21T00:26:35.842375+0000","last_became_peered":"2024-09-21T00:26:35.842375+0000","last_unstale":"2024-09-21T00:26:36.831536+0000","last_undegraded":"2024-09-21T00:26:36.831536+0000","last_fullsized":"2024-09-21T00:26:36.831536+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:58:30.617062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.828170+0000","last_change":"2024-09-21T00:26:35.828170+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.828170+0000","last_undegraded":"2024-09-21T00:26:35.828170+0000","last_fullsized":"2024-09-21T00:26:35.828170+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-21T00:26:35.844992+0000","last_change":"2024-09-21T00:26:35.839140+0000","last_active":"2024-09-21T00:26:35.844992+0000","last_peered":"2024-09-21T00:26:35.844992+0000","last_clean":"2024-09-21T00:26:35.844992+0000","last_became_active":"2024-09-21T00:26:35.838817+0000","last_became_peered":"2024-09-21T00:26:35.838817+0000","last_unstale":"2024-09-21T00:26:35.844992+0000","last_undegraded":"2024-09-21T00:26:35.844992+0000","last_fullsized":"2024-09-21T00:26:35.844992+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:07:46.709919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.828288+0000","last_change":"2024-09-21T00:26:35.828288+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.828288+0000","last_undegraded":"2024-09-21T00:26:35.828288+0000","last_fullsized":"2024-09-21T00:26:35.828288+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-21T00:26:35.850329+0000","last_change":"2024-09-21T00:26:35.846027+0000","last_active":"2024-09-21T00:26:35.850329+0000","last_peered":"2024-09-21T00:26:35.850329+0000","last_clean":"2024-09-21T00:26:35.850329+0000","last_became_active":"2024-09-21T00:26:35.845681+0000","last_became_peered":"2024-09-21T00:26:35.845681+0000","last_unstale":"2024-09-21T00:26:35.850329+0000","last_undegraded":"2024-09-21T00:26:35.850329+0000","last_fullsized":"2024-09-21T00:26:35.850329+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:11:52.985613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.827957+0000","last_change":"2024-09-21T00:26:35.827957+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.827957+0000","last_undegraded":"2024-09-21T00:26:35.827957+0000","last_fullsized":"2024-09-21T00:26:35.827957+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-21T00:26:35.850340+0000","last_change":"2024-09-21T00:26:35.845969+0000","last_active":"2024-09-21T00:26:35.850340+0000","last_peered":"2024-09-21T00:26:35.850340+0000","last_clean":"2024-09-21T00:26:35.850340+0000","last_became_active":"2024-09-21T00:26:35.845633+0000","last_became_peered":"2024-09-21T00:26:35.845633+0000","last_unstale":"2024-09-21T00:26:35.850340+0000","last_undegraded":"2024-09-21T00:26:35.850340+0000","last_fullsized":"2024-09-21T00:26:35.850340+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:41:29.077144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.830853+0000","last_change":"2024-09-21T00:26:35.830853+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.830853+0000","last_undegraded":"2024-09-21T00:26:35.830853+0000","last_fullsized":"2024-09-21T00:26:35.830853+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-21T00:26:35.845305+0000","last_change":"2024-09-21T00:26:35.840529+0000","last_active":"2024-09-21T00:26:35.845305+0000","last_peered":"2024-09-21T00:26:35.845305+0000","last_clean":"2024-09-21T00:26:35.845305+0000","last_became_active":"2024-09-21T00:26:35.840115+0000","last_became_peered":"2024-09-21T00:26:35.840115+0000","last_unstale":"2024-09-21T00:26:35.845305+0000","last_undegraded":"2024-09-21T00:26:35.845305+0000","last_fullsized":"2024-09-21T00:26:35.845305+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:33:02.079644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.824938+0000","last_change":"2024-09-21T00:26:35.824938+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.824938+0000","last_undegraded":"2024-09-21T00:26:35.824938+0000","last_fullsized":"2024-09-21T00:26:35.824938+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-21T00:26:35.838164+0000","last_change":"2024-09-21T00:26:35.835450+0000","last_active":"2024-09-21T00:26:35.838164+0000","last_peered":"2024-09-21T00:26:35.838164+0000","last_clean":"2024-09-21T00:26:35.838164+0000","last_became_active":"2024-09-21T00:26:35.835141+0000","last_became_peered":"2024-09-21T00:26:35.835141+0000","last_unstale":"2024-09-21T00:26:35.838164+0000","last_undegraded":"2024-09-21T00:26:35.838164+0000","last_fullsized":"2024-09-21T00:26:35.838164+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:11:09.781135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.824556+0000","last_change":"2024-09-21T00:26:35.824556+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.824556+0000","last_undegraded":"2024-09-21T00:26:35.824556+0000","last_fullsized":"2024-09-21T00:26:35.824556+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-21T00:26:36.831408+0000","last_change":"2024-09-21T00:26:35.841410+0000","last_active":"2024-09-21T00:26:36.831408+0000","last_peered":"2024-09-21T00:26:36.831408+0000","last_clean":"2024-09-21T00:26:36.831408+0000","last_became_active":"2024-09-21T00:26:35.841094+0000","last_became_peered":"2024-09-21T00:26:35.841094+0000","last_unstale":"2024-09-21T00:26:36.831408+0000","last_undegraded":"2024-09-21T00:26:36.831408+0000","last_fullsized":"2024-09-21T00:26:36.831408+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:48:22.317860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.827023+0000","last_change":"2024-09-21T00:26:35.827023+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.827023+0000","last_undegraded":"2024-09-21T00:26:35.827023+0000","last_fullsized":"2024-09-21T00:26:35.827023+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-21T00:26:35.843267+0000","last_change":"2024-09-21T00:26:35.836023+0000","last_active":"2024-09-21T00:26:35.843267+0000","last_peered":"2024-09-21T00:26:35.843267+0000","last_clean":"2024-09-21T00:26:35.843267+0000","last_became_active":"2024-09-21T00:26:35.835721+0000","last_became_peered":"2024-09-21T00:26:35.835721+0000","last_unstale":"2024-09-21T00:26:35.843267+0000","last_undegraded":"2024-09-21T00:26:35.843267+0000","last_fullsized":"2024-09-21T00:26:35.843267+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:43:45.222366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.825112+0000","last_change":"2024-09-21T00:26:35.825112+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.825112+0000","last_undegraded":"2024-09-21T00:26:35.825112+0000","last_fullsized":"2024-09-21T00:26:35.825112+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-21T00:26:35.843093+0000","last_change":"2024-09-21T00:26:35.837224+0000","last_active":"2024-09-21T00:26:35.843093+0000","last_peered":"2024-09-21T00:26:35.843093+0000","last_clean":"2024-09-21T00:26:35.843093+0000","last_became_active":"2024-09-21T00:26:35.836849+0000","last_became_peered":"2024-09-21T00:26:35.836849+0000","last_unstale":"2024-09-21T00:26:35.843093+0000","last_undegraded":"2024-09-21T00:26:35.843093+0000","last_fullsized":"2024-09-21T00:26:35.843093+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:59:09.929281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.829525+0000","last_change":"2024-09-21T00:26:35.829525+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.829525+0000","last_undegraded":"2024-09-21T00:26:35.829525+0000","last_fullsized":"2024-09-21T00:26:35.829525+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-21T00:26:35.842275+0000","last_change":"2024-09-21T00:26:35.837934+0000","last_active":"2024-09-21T00:26:35.842275+0000","last_peered":"2024-09-21T00:26:35.842275+0000","last_clean":"2024-09-21T00:26:35.842275+0000","last_became_active":"2024-09-21T00:26:35.837198+0000","last_became_peered":"2024-09-21T00:26:35.837198+0000","last_unstale":"2024-09-21T00:26:35.842275+0000","last_undegraded":"2024-09-21T00:26:35.842275+0000","last_fullsized":"2024-09-21T00:26:35.842275+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:04:35.145795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.827774+0000","last_change":"2024-09-21T00:26:35.827774+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.827774+0000","last_undegraded":"2024-09-21T00:26:35.827774+0000","last_fullsized":"2024-09-21T00:26:35.827774+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-21T00:26:35.836465+0000","last_change":"2024-09-21T00:26:35.832534+0000","last_active":"2024-09-21T00:26:35.836465+0000","last_peered":"2024-09-21T00:26:35.836465+0000","last_clean":"2024-09-21T00:26:35.836465+0000","last_became_active":"2024-09-21T00:26:35.832197+0000","last_became_peered":"2024-09-21T00:26:35.832197+0000","last_unstale":"2024-09-21T00:26:35.836465+0000","last_undegraded":"2024-09-21T00:26:35.836465+0000","last_fullsized":"2024-09-21T00:26:35.836465+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:28:49.838026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-21T00:26:36.844617+0000","last_change":"2024-09-21T00:26:36.841191+0000","last_active":"2024-09-21T00:26:36.844617+0000","last_peered":"2024-09-21T00:26:36.844617+0000","last_clean":"2024-09-21T00:26:36.844617+0000","last_became_active":"2024-09-21T00:26:36.840676+0000","last_became_peered":"2024-09-21T00:26:36.840676+0000","last_unstale":"2024-09-21T00:26:36.844617+0000","last_undegraded":"2024-09-21T00:26:36.844617+0000","last_fullsized":"2024-09-21T00:26:36.844617+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:16:54.958692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-21T00:26:35.842557+0000","last_change":"2024-09-21T00:26:35.840350+0000","last_active":"2024-09-21T00:26:35.842557+0000","last_peered":"2024-09-21T00:26:35.842557+0000","last_clean":"2024-09-21T00:26:35.842557+0000","last_became_active":"2024-09-21T00:26:35.840048+0000","last_became_peered":"2024-09-21T00:26:35.840048+0000","last_unstale":"2024-09-21T00:26:35.842557+0000","last_undegraded":"2024-09-21T00:26:35.842557+0000","last_fullsized":"2024-09-21T00:26:35.842557+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:19:54.299718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.829609+0000","last_change":"2024-09-21T00:26:35.829609+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.829609+0000","last_undegraded":"2024-09-21T00:26:35.829609+0000","last_fullsized":"2024-09-21T00:26:35.829609+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-21T00:26:35.846174+0000","last_change":"2024-09-21T00:26:35.838940+0000","last_active":"2024-09-21T00:26:35.846174+0000","last_peered":"2024-09-21T00:26:35.846174+0000","last_clean":"2024-09-21T00:26:35.846174+0000","last_became_active":"2024-09-21T00:26:35.838616+0000","last_became_peered":"2024-09-21T00:26:35.838616+0000","last_unstale":"2024-09-21T00:26:35.846174+0000","last_undegraded":"2024-09-21T00:26:35.846174+0000","last_fullsized":"2024-09-21T00:26:35.846174+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:01:07.494519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.826730+0000","last_change":"2024-09-21T00:26:35.826730+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.826730+0000","last_undegraded":"2024-09-21T00:26:35.826730+0000","last_fullsized":"2024-09-21T00:26:35.826730+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-21T00:26:35.841029+0000","last_change":"2024-09-21T00:26:35.838748+0000","last_active":"2024-09-21T00:26:35.841029+0000","last_peered":"2024-09-21T00:26:35.841029+0000","last_clean":"2024-09-21T00:26:35.841029+0000","last_became_active":"2024-09-21T00:26:35.838445+0000","last_became_peered":"2024-09-21T00:26:35.838445+0000","last_unstale":"2024-09-21T00:26:35.841029+0000","last_undegraded":"2024-09-21T00:26:35.841029+0000","last_fullsized":"2024-09-21T00:26:35.841029+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:44:34.009179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.825823+0000","last_change":"2024-09-21T00:26:35.825823+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.825823+0000","last_undegraded":"2024-09-21T00:26:35.825823+0000","last_fullsized":"2024-09-21T00:26:35.825823+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-21T00:26:35.843172+0000","last_change":"2024-09-21T00:26:35.840565+0000","last_active":"2024-09-21T00:26:35.843172+0000","last_peered":"2024-09-21T00:26:35.843172+0000","last_clean":"2024-09-21T00:26:35.843172+0000","last_became_active":"2024-09-21T00:26:35.840265+0000","last_became_peered":"2024-09-21T00:26:35.840265+0000","last_unstale":"2024-09-21T00:26:35.843172+0000","last_undegraded":"2024-09-21T00:26:35.843172+0000","last_fullsized":"2024-09-21T00:26:35.843172+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:14:15.665953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-21T00:26:36.848314+0000","last_change":"2024-09-21T00:26:36.842397+0000","last_active":"2024-09-21T00:26:36.848314+0000","last_peered":"2024-09-21T00:26:36.848314+0000","last_clean":"2024-09-21T00:26:36.848314+0000","last_became_active":"2024-09-21T00:26:36.842095+0000","last_became_peered":"2024-09-21T00:26:36.842095+0000","last_unstale":"2024-09-21T00:26:36.848314+0000","last_undegraded":"2024-09-21T00:26:36.848314+0000","last_fullsized":"2024-09-21T00:26:36.848314+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:50:12.134645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-21T00:26:35.837296+0000","last_change":"2024-09-21T00:26:35.835982+0000","last_active":"2024-09-21T00:26:35.837296+0000","last_peered":"2024-09-21T00:26:35.837296+0000","last_clean":"2024-09-21T00:26:35.837296+0000","last_became_active":"2024-09-21T00:26:35.835677+0000","last_became_peered":"2024-09-21T00:26:35.835677+0000","last_unstale":"2024-09-21T00:26:35.837296+0000","last_undegraded":"2024-09-21T00:26:35.837296+0000","last_fullsized":"2024-09-21T00:26:35.837296+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:46:16.688217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.826853+0000","last_change":"2024-09-21T00:26:35.826853+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.826853+0000","last_undegraded":"2024-09-21T00:26:35.826853+0000","last_fullsized":"2024-09-21T00:26:35.826853+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-21T00:26:35.842371+0000","last_change":"2024-09-21T00:26:35.838318+0000","last_active":"2024-09-21T00:26:35.842371+0000","last_peered":"2024-09-21T00:26:35.842371+0000","last_clean":"2024-09-21T00:26:35.842371+0000","last_became_active":"2024-09-21T00:26:35.836426+0000","last_became_peered":"2024-09-21T00:26:35.836426+0000","last_unstale":"2024-09-21T00:26:35.842371+0000","last_undegraded":"2024-09-21T00:26:35.842371+0000","last_fullsized":"2024-09-21T00:26:35.842371+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:48:50.639166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.827493+0000","last_change":"2024-09-21T00:26:35.827493+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.827493+0000","last_undegraded":"2024-09-21T00:26:35.827493+0000","last_fullsized":"2024-09-21T00:26:35.827493+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-21T00:26:35.841994+0000","last_change":"2024-09-21T00:26:35.836745+0000","last_active":"2024-09-21T00:26:35.841994+0000","last_peered":"2024-09-21T00:26:35.841994+0000","last_clean":"2024-09-21T00:26:35.841994+0000","last_became_active":"2024-09-21T00:26:35.836427+0000","last_became_peered":"2024-09-21T00:26:35.836427+0000","last_unstale":"2024-09-21T00:26:35.841994+0000","last_undegraded":"2024-09-21T00:26:35.841994+0000","last_fullsized":"2024-09-21T00:26:35.841994+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:33:15.124204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.830688+0000","last_change":"2024-09-21T00:26:35.830688+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.830688+0000","last_undegraded":"2024-09-21T00:26:35.830688+0000","last_fullsized":"2024-09-21T00:26:35.830688+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-21T00:26:35.845045+0000","last_change":"2024-09-21T00:26:35.839244+0000","last_active":"2024-09-21T00:26:35.845045+0000","last_peered":"2024-09-21T00:26:35.845045+0000","last_clean":"2024-09-21T00:26:35.845045+0000","last_became_active":"2024-09-21T00:26:35.838876+0000","last_became_peered":"2024-09-21T00:26:35.838876+0000","last_unstale":"2024-09-21T00:26:35.845045+0000","last_undegraded":"2024-09-21T00:26:35.845045+0000","last_fullsized":"2024-09-21T00:26:35.845045+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:02:08.223701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":136,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-21T00:26:35.831437+0000","last_change":"2024-09-21T00:25:38.725792+0000","last_active":"2024-09-21T00:26:35.831437+0000","last_peered":"2024-09-21T00:26:35.831437+0000","last_clean":"2024-09-21T00:26:35.831437+0000","last_became_active":"2024-09-21T00:25:38.724668+0000","last_became_peered":"2024-09-21T00:25:38.724668+0000","last_unstale":"2024-09-21T00:26:35.831437+0000","last_undegraded":"2024-09-21T00:26:35.831437+0000","last_fullsized":"2024-09-21T00:26:35.831437+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:37.163216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:37.163216+0000","last_clean_scrub_stamp":"2024-09-21T00:25:37.163216+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:14:36.588660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509455,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28008,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343832,"statfs":{"total":96636764160,"available":96608083968,"internally_reserved":0,"allocated":999424,"data_stored":758396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509456,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":393216,"data_stored":161361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509455,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27428,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":405504,"data_stored":168028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509454,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":393216,"data_stored":161361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509456,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":393216,"data_stored":161361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509456,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":393216,"data_stored":161361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509456,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27992,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343848,"statfs":{"total":96636764160,"available":96608100352,"internally_reserved":0,"allocated":987136,"data_stored":751729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574864,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27440,"kb_used_data":412,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":421888,"data_stored":175285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T00:26:41.570 INFO:tasks.ceph.ceph_manager.ceph:PG 6.3f is not active+clean 2024-09-21T00:26:41.570 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-09-21T00:26:35.829346+0000', 'last_change': '2024-09-21T00:26:35.829346+0000', 'last_active': '2024-09-21T00:26:35.811406+0000', 'last_peered': '2024-09-21T00:26:35.811406+0000', 'last_clean': '2024-09-21T00:26:35.811406+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:26:35.829346+0000', 'last_undegraded': '2024-09-21T00:26:35.829346+0000', 'last_fullsized': '2024-09-21T00:26:35.829346+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'last_clean_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-21T00:26:41.570 INFO:tasks.ceph.ceph_manager.ceph:PG 6.3e is not active+clean 2024-09-21T00:26:41.570 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-09-21T00:26:35.827322+0000', 'last_change': '2024-09-21T00:26:35.827322+0000', 'last_active': '2024-09-21T00:26:35.811406+0000', 'last_peered': '2024-09-21T00:26:35.811406+0000', 'last_clean': '2024-09-21T00:26:35.811406+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:26:35.827322+0000', 'last_undegraded': '2024-09-21T00:26:35.827322+0000', 'last_fullsized': '2024-09-21T00:26:35.827322+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'last_clean_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-21T00:26:41.570 INFO:tasks.ceph.ceph_manager.ceph:PG 6.3d is not active+clean 2024-09-21T00:26:41.570 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-09-21T00:26:35.828578+0000', 'last_change': '2024-09-21T00:26:35.828578+0000', 'last_active': '2024-09-21T00:26:35.811406+0000', 'last_peered': '2024-09-21T00:26:35.811406+0000', 'last_clean': '2024-09-21T00:26:35.811406+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:26:35.828578+0000', 'last_undegraded': '2024-09-21T00:26:35.828578+0000', 'last_fullsized': '2024-09-21T00:26:35.828578+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'last_clean_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-21T00:26:41.571 INFO:tasks.ceph.ceph_manager.ceph:PG 6.3c is not active+clean 2024-09-21T00:26:41.571 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-09-21T00:26:35.825190+0000', 'last_change': '2024-09-21T00:26:35.825190+0000', 'last_active': '2024-09-21T00:26:35.811406+0000', 'last_peered': '2024-09-21T00:26:35.811406+0000', 'last_clean': '2024-09-21T00:26:35.811406+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:26:35.825190+0000', 'last_undegraded': '2024-09-21T00:26:35.825190+0000', 'last_fullsized': '2024-09-21T00:26:35.825190+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'last_clean_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-21T00:26:41.571 INFO:tasks.ceph.ceph_manager.ceph:PG 6.3b is not active+clean 2024-09-21T00:26:41.571 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-09-21T00:26:35.825161+0000', 'last_change': '2024-09-21T00:26:35.825161+0000', 'last_active': '2024-09-21T00:26:35.811406+0000', 'last_peered': '2024-09-21T00:26:35.811406+0000', 'last_clean': '2024-09-21T00:26:35.811406+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:26:35.825161+0000', 'last_undegraded': '2024-09-21T00:26:35.825161+0000', 'last_fullsized': '2024-09-21T00:26:35.825161+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'last_clean_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-21T00:26:41.571 INFO:tasks.ceph.ceph_manager.ceph:PG 6.39 is not active+clean 2024-09-21T00:26:41.571 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-09-21T00:26:35.832564+0000', 'last_change': '2024-09-21T00:26:35.832564+0000', 'last_active': '2024-09-21T00:26:35.811406+0000', 'last_peered': '2024-09-21T00:26:35.811406+0000', 'last_clean': '2024-09-21T00:26:35.811406+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:26:35.832564+0000', 'last_undegraded': '2024-09-21T00:26:35.832564+0000', 'last_fullsized': '2024-09-21T00:26:35.832564+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'last_clean_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-21T00:26:41.571 INFO:tasks.ceph.ceph_manager.ceph:PG 6.38 is not active+clean 2024-09-21T00:26:41.571 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-09-21T00:26:35.830845+0000', 'last_change': '2024-09-21T00:26:35.830845+0000', 'last_active': '2024-09-21T00:26:35.811406+0000', 'last_peered': '2024-09-21T00:26:35.811406+0000', 'last_clean': '2024-09-21T00:26:35.811406+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:26:35.830845+0000', 'last_undegraded': '2024-09-21T00:26:35.830845+0000', 'last_fullsized': '2024-09-21T00:26:35.830845+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'last_clean_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-21T00:26:41.571 INFO:tasks.ceph.ceph_manager.ceph:PG 6.37 is not active+clean 2024-09-21T00:26:41.571 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-09-21T00:26:35.830587+0000', 'last_change': '2024-09-21T00:26:35.830587+0000', 'last_active': '2024-09-21T00:26:35.811406+0000', 'last_peered': '2024-09-21T00:26:35.811406+0000', 'last_clean': '2024-09-21T00:26:35.811406+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:26:35.830587+0000', 'last_undegraded': '2024-09-21T00:26:35.830587+0000', 'last_fullsized': '2024-09-21T00:26:35.830587+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'last_clean_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-21T00:26:41.571 INFO:tasks.ceph.ceph_manager.ceph:PG 6.36 is not active+clean 2024-09-21T00:26:41.572 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-09-21T00:26:35.833967+0000', 'last_change': '2024-09-21T00:26:35.833967+0000', 'last_active': '2024-09-21T00:26:35.811406+0000', 'last_peered': '2024-09-21T00:26:35.811406+0000', 'last_clean': '2024-09-21T00:26:35.811406+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:26:35.833967+0000', 'last_undegraded': '2024-09-21T00:26:35.833967+0000', 'last_fullsized': '2024-09-21T00:26:35.833967+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'last_clean_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-21T00:26:41.572 INFO:tasks.ceph.ceph_manager.ceph:PG 6.35 is not active+clean 2024-09-21T00:26:41.572 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-09-21T00:26:35.832457+0000', 'last_change': '2024-09-21T00:26:35.832457+0000', 'last_active': '2024-09-21T00:26:35.811406+0000', 'last_peered': '2024-09-21T00:26:35.811406+0000', 'last_clean': '2024-09-21T00:26:35.811406+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:26:35.832457+0000', 'last_undegraded': '2024-09-21T00:26:35.832457+0000', 'last_fullsized': '2024-09-21T00:26:35.832457+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'last_clean_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-21T00:26:41.572 INFO:tasks.ceph.ceph_manager.ceph:PG 6.34 is not active+clean 2024-09-21T00:26:41.572 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-09-21T00:26:35.828280+0000', 'last_change': '2024-09-21T00:26:35.828280+0000', 'last_active': '2024-09-21T00:26:35.811406+0000', 'last_peered': '2024-09-21T00:26:35.811406+0000', 'last_clean': '2024-09-21T00:26:35.811406+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:26:35.828280+0000', 'last_undegraded': '2024-09-21T00:26:35.828280+0000', 'last_fullsized': '2024-09-21T00:26:35.828280+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'last_clean_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-21T00:26:41.572 INFO:tasks.ceph.ceph_manager.ceph:PG 6.33 is not active+clean 2024-09-21T00:26:41.572 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-09-21T00:26:35.831355+0000', 'last_change': '2024-09-21T00:26:35.831355+0000', 'last_active': '2024-09-21T00:26:35.811406+0000', 'last_peered': '2024-09-21T00:26:35.811406+0000', 'last_clean': '2024-09-21T00:26:35.811406+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:26:35.831355+0000', 'last_undegraded': '2024-09-21T00:26:35.831355+0000', 'last_fullsized': '2024-09-21T00:26:35.831355+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'last_clean_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-21T00:26:41.572 INFO:tasks.ceph.ceph_manager.ceph:PG 6.32 is not active+clean 2024-09-21T00:26:41.572 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-09-21T00:26:35.827439+0000', 'last_change': '2024-09-21T00:26:35.827439+0000', 'last_active': '2024-09-21T00:26:35.811406+0000', 'last_peered': '2024-09-21T00:26:35.811406+0000', 'last_clean': '2024-09-21T00:26:35.811406+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:26:35.827439+0000', 'last_undegraded': '2024-09-21T00:26:35.827439+0000', 'last_fullsized': '2024-09-21T00:26:35.827439+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'last_clean_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-21T00:26:41.572 INFO:tasks.ceph.ceph_manager.ceph:PG 6.31 is not active+clean 2024-09-21T00:26:41.572 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-09-21T00:26:35.829991+0000', 'last_change': '2024-09-21T00:26:35.829991+0000', 'last_active': '2024-09-21T00:26:35.811406+0000', 'last_peered': '2024-09-21T00:26:35.811406+0000', 'last_clean': '2024-09-21T00:26:35.811406+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:26:35.829991+0000', 'last_undegraded': '2024-09-21T00:26:35.829991+0000', 'last_fullsized': '2024-09-21T00:26:35.829991+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'last_clean_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-21T00:26:41.572 INFO:tasks.ceph.ceph_manager.ceph:PG 6.30 is not active+clean 2024-09-21T00:26:41.572 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-09-21T00:26:35.828731+0000', 'last_change': '2024-09-21T00:26:35.828731+0000', 'last_active': '2024-09-21T00:26:35.811406+0000', 'last_peered': '2024-09-21T00:26:35.811406+0000', 'last_clean': '2024-09-21T00:26:35.811406+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:26:35.828731+0000', 'last_undegraded': '2024-09-21T00:26:35.828731+0000', 'last_fullsized': '2024-09-21T00:26:35.828731+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'last_clean_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-21T00:26:41.573 INFO:tasks.ceph.ceph_manager.ceph:PG 6.2f is not active+clean 2024-09-21T00:26:41.573 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-09-21T00:26:35.828322+0000', 'last_change': '2024-09-21T00:26:35.828322+0000', 'last_active': '2024-09-21T00:26:35.811406+0000', 'last_peered': '2024-09-21T00:26:35.811406+0000', 'last_clean': '2024-09-21T00:26:35.811406+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:26:35.828322+0000', 'last_undegraded': '2024-09-21T00:26:35.828322+0000', 'last_fullsized': '2024-09-21T00:26:35.828322+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'last_clean_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-21T00:26:41.573 INFO:tasks.ceph.ceph_manager.ceph:PG 6.2e is not active+clean 2024-09-21T00:26:41.573 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-09-21T00:26:35.827556+0000', 'last_change': '2024-09-21T00:26:35.827556+0000', 'last_active': '2024-09-21T00:26:35.811406+0000', 'last_peered': '2024-09-21T00:26:35.811406+0000', 'last_clean': '2024-09-21T00:26:35.811406+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:26:35.827556+0000', 'last_undegraded': '2024-09-21T00:26:35.827556+0000', 'last_fullsized': '2024-09-21T00:26:35.827556+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'last_clean_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-21T00:26:41.573 INFO:tasks.ceph.ceph_manager.ceph:PG 6.2d is not active+clean 2024-09-21T00:26:41.573 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-09-21T00:26:35.823845+0000', 'last_change': '2024-09-21T00:26:35.823845+0000', 'last_active': '2024-09-21T00:26:35.811406+0000', 'last_peered': '2024-09-21T00:26:35.811406+0000', 'last_clean': '2024-09-21T00:26:35.811406+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:26:35.823845+0000', 'last_undegraded': '2024-09-21T00:26:35.823845+0000', 'last_fullsized': '2024-09-21T00:26:35.823845+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'last_clean_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-21T00:26:41.573 INFO:tasks.ceph.ceph_manager.ceph:PG 6.2c is not active+clean 2024-09-21T00:26:41.573 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-09-21T00:26:35.827691+0000', 'last_change': '2024-09-21T00:26:35.827691+0000', 'last_active': '2024-09-21T00:26:35.811406+0000', 'last_peered': '2024-09-21T00:26:35.811406+0000', 'last_clean': '2024-09-21T00:26:35.811406+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:26:35.827691+0000', 'last_undegraded': '2024-09-21T00:26:35.827691+0000', 'last_fullsized': '2024-09-21T00:26:35.827691+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'last_clean_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-21T00:26:41.573 INFO:tasks.ceph.ceph_manager.ceph:PG 6.2b is not active+clean 2024-09-21T00:26:41.573 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-09-21T00:26:35.831846+0000', 'last_change': '2024-09-21T00:26:35.831846+0000', 'last_active': '2024-09-21T00:26:35.811406+0000', 'last_peered': '2024-09-21T00:26:35.811406+0000', 'last_clean': '2024-09-21T00:26:35.811406+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:26:35.831846+0000', 'last_undegraded': '2024-09-21T00:26:35.831846+0000', 'last_fullsized': '2024-09-21T00:26:35.831846+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'last_clean_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-21T00:26:41.573 INFO:tasks.ceph.ceph_manager.ceph:PG 6.2a is not active+clean 2024-09-21T00:26:41.573 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-09-21T00:26:35.823745+0000', 'last_change': '2024-09-21T00:26:35.823745+0000', 'last_active': '2024-09-21T00:26:35.811406+0000', 'last_peered': '2024-09-21T00:26:35.811406+0000', 'last_clean': '2024-09-21T00:26:35.811406+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:26:35.823745+0000', 'last_undegraded': '2024-09-21T00:26:35.823745+0000', 'last_fullsized': '2024-09-21T00:26:35.823745+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'last_clean_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-21T00:26:41.574 INFO:tasks.ceph.ceph_manager.ceph:PG 6.29 is not active+clean 2024-09-21T00:26:41.574 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-09-21T00:26:35.827636+0000', 'last_change': '2024-09-21T00:26:35.827636+0000', 'last_active': '2024-09-21T00:26:35.811406+0000', 'last_peered': '2024-09-21T00:26:35.811406+0000', 'last_clean': '2024-09-21T00:26:35.811406+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:26:35.827636+0000', 'last_undegraded': '2024-09-21T00:26:35.827636+0000', 'last_fullsized': '2024-09-21T00:26:35.827636+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'last_clean_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-21T00:26:41.574 INFO:tasks.ceph.ceph_manager.ceph:PG 6.28 is not active+clean 2024-09-21T00:26:41.574 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-09-21T00:26:35.823856+0000', 'last_change': '2024-09-21T00:26:35.823856+0000', 'last_active': '2024-09-21T00:26:35.811406+0000', 'last_peered': '2024-09-21T00:26:35.811406+0000', 'last_clean': '2024-09-21T00:26:35.811406+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:26:35.823856+0000', 'last_undegraded': '2024-09-21T00:26:35.823856+0000', 'last_fullsized': '2024-09-21T00:26:35.823856+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'last_clean_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-21T00:26:41.574 INFO:tasks.ceph.ceph_manager.ceph:PG 6.27 is not active+clean 2024-09-21T00:26:41.574 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-09-21T00:26:35.826279+0000', 'last_change': '2024-09-21T00:26:35.826279+0000', 'last_active': '2024-09-21T00:26:35.811406+0000', 'last_peered': '2024-09-21T00:26:35.811406+0000', 'last_clean': '2024-09-21T00:26:35.811406+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:26:35.826279+0000', 'last_undegraded': '2024-09-21T00:26:35.826279+0000', 'last_fullsized': '2024-09-21T00:26:35.826279+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'last_clean_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-21T00:26:41.574 INFO:tasks.ceph.ceph_manager.ceph:PG 6.25 is not active+clean 2024-09-21T00:26:41.574 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-09-21T00:26:35.825808+0000', 'last_change': '2024-09-21T00:26:35.825808+0000', 'last_active': '2024-09-21T00:26:35.811406+0000', 'last_peered': '2024-09-21T00:26:35.811406+0000', 'last_clean': '2024-09-21T00:26:35.811406+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:26:35.825808+0000', 'last_undegraded': '2024-09-21T00:26:35.825808+0000', 'last_fullsized': '2024-09-21T00:26:35.825808+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'last_clean_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-21T00:26:41.574 INFO:tasks.ceph.ceph_manager.ceph:PG 6.24 is not active+clean 2024-09-21T00:26:41.574 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-09-21T00:26:35.830953+0000', 'last_change': '2024-09-21T00:26:35.830953+0000', 'last_active': '2024-09-21T00:26:35.811406+0000', 'last_peered': '2024-09-21T00:26:35.811406+0000', 'last_clean': '2024-09-21T00:26:35.811406+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:26:35.830953+0000', 'last_undegraded': '2024-09-21T00:26:35.830953+0000', 'last_fullsized': '2024-09-21T00:26:35.830953+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'last_clean_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-21T00:26:41.574 INFO:tasks.ceph.ceph_manager.ceph:PG 6.23 is not active+clean 2024-09-21T00:26:41.575 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-09-21T00:26:35.824648+0000', 'last_change': '2024-09-21T00:26:35.824648+0000', 'last_active': '2024-09-21T00:26:35.811406+0000', 'last_peered': '2024-09-21T00:26:35.811406+0000', 'last_clean': '2024-09-21T00:26:35.811406+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:26:35.824648+0000', 'last_undegraded': '2024-09-21T00:26:35.824648+0000', 'last_fullsized': '2024-09-21T00:26:35.824648+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'last_clean_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-21T00:26:41.575 INFO:tasks.ceph.ceph_manager.ceph:PG 6.22 is not active+clean 2024-09-21T00:26:41.575 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-09-21T00:26:35.826507+0000', 'last_change': '2024-09-21T00:26:35.826507+0000', 'last_active': '2024-09-21T00:26:35.811406+0000', 'last_peered': '2024-09-21T00:26:35.811406+0000', 'last_clean': '2024-09-21T00:26:35.811406+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:26:35.826507+0000', 'last_undegraded': '2024-09-21T00:26:35.826507+0000', 'last_fullsized': '2024-09-21T00:26:35.826507+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'last_clean_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-21T00:26:41.575 INFO:tasks.ceph.ceph_manager.ceph:PG 6.21 is not active+clean 2024-09-21T00:26:41.575 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-09-21T00:26:35.826999+0000', 'last_change': '2024-09-21T00:26:35.826999+0000', 'last_active': '2024-09-21T00:26:35.811406+0000', 'last_peered': '2024-09-21T00:26:35.811406+0000', 'last_clean': '2024-09-21T00:26:35.811406+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:26:35.826999+0000', 'last_undegraded': '2024-09-21T00:26:35.826999+0000', 'last_fullsized': '2024-09-21T00:26:35.826999+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'last_clean_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-21T00:26:41.575 INFO:tasks.ceph.ceph_manager.ceph:PG 6.1c is not active+clean 2024-09-21T00:26:41.575 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-09-21T00:26:35.828100+0000', 'last_change': '2024-09-21T00:26:35.828100+0000', 'last_active': '2024-09-21T00:26:35.811406+0000', 'last_peered': '2024-09-21T00:26:35.811406+0000', 'last_clean': '2024-09-21T00:26:35.811406+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:26:35.828100+0000', 'last_undegraded': '2024-09-21T00:26:35.828100+0000', 'last_fullsized': '2024-09-21T00:26:35.828100+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'last_clean_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-21T00:26:41.575 INFO:tasks.ceph.ceph_manager.ceph:PG 6.1e is not active+clean 2024-09-21T00:26:41.575 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-09-21T00:26:35.824468+0000', 'last_change': '2024-09-21T00:26:35.824468+0000', 'last_active': '2024-09-21T00:26:35.811406+0000', 'last_peered': '2024-09-21T00:26:35.811406+0000', 'last_clean': '2024-09-21T00:26:35.811406+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:26:35.824468+0000', 'last_undegraded': '2024-09-21T00:26:35.824468+0000', 'last_fullsized': '2024-09-21T00:26:35.824468+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'last_clean_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-21T00:26:41.575 INFO:tasks.ceph.ceph_manager.ceph:PG 6.1f is not active+clean 2024-09-21T00:26:41.575 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-09-21T00:26:35.829845+0000', 'last_change': '2024-09-21T00:26:35.829845+0000', 'last_active': '2024-09-21T00:26:35.811406+0000', 'last_peered': '2024-09-21T00:26:35.811406+0000', 'last_clean': '2024-09-21T00:26:35.811406+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:26:35.829845+0000', 'last_undegraded': '2024-09-21T00:26:35.829845+0000', 'last_fullsized': '2024-09-21T00:26:35.829845+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'last_clean_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-21T00:26:41.575 INFO:tasks.ceph.ceph_manager.ceph:PG 6.18 is not active+clean 2024-09-21T00:26:41.576 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-09-21T00:26:35.826825+0000', 'last_change': '2024-09-21T00:26:35.826825+0000', 'last_active': '2024-09-21T00:26:35.811406+0000', 'last_peered': '2024-09-21T00:26:35.811406+0000', 'last_clean': '2024-09-21T00:26:35.811406+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:26:35.826825+0000', 'last_undegraded': '2024-09-21T00:26:35.826825+0000', 'last_fullsized': '2024-09-21T00:26:35.826825+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'last_clean_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-21T00:26:41.576 INFO:tasks.ceph.ceph_manager.ceph:PG 6.19 is not active+clean 2024-09-21T00:26:41.576 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-09-21T00:26:35.829387+0000', 'last_change': '2024-09-21T00:26:35.829387+0000', 'last_active': '2024-09-21T00:26:35.811406+0000', 'last_peered': '2024-09-21T00:26:35.811406+0000', 'last_clean': '2024-09-21T00:26:35.811406+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:26:35.829387+0000', 'last_undegraded': '2024-09-21T00:26:35.829387+0000', 'last_fullsized': '2024-09-21T00:26:35.829387+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'last_clean_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-21T00:26:41.576 INFO:tasks.ceph.ceph_manager.ceph:PG 6.1a is not active+clean 2024-09-21T00:26:41.576 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-09-21T00:26:35.825895+0000', 'last_change': '2024-09-21T00:26:35.825895+0000', 'last_active': '2024-09-21T00:26:35.811406+0000', 'last_peered': '2024-09-21T00:26:35.811406+0000', 'last_clean': '2024-09-21T00:26:35.811406+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:26:35.825895+0000', 'last_undegraded': '2024-09-21T00:26:35.825895+0000', 'last_fullsized': '2024-09-21T00:26:35.825895+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'last_clean_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-21T00:26:41.576 INFO:tasks.ceph.ceph_manager.ceph:PG 6.1b is not active+clean 2024-09-21T00:26:41.576 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-09-21T00:26:35.828735+0000', 'last_change': '2024-09-21T00:26:35.828735+0000', 'last_active': '2024-09-21T00:26:35.811406+0000', 'last_peered': '2024-09-21T00:26:35.811406+0000', 'last_clean': '2024-09-21T00:26:35.811406+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:26:35.828735+0000', 'last_undegraded': '2024-09-21T00:26:35.828735+0000', 'last_fullsized': '2024-09-21T00:26:35.828735+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'last_clean_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-21T00:26:41.576 INFO:tasks.ceph.ceph_manager.ceph:PG 6.14 is not active+clean 2024-09-21T00:26:41.576 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-09-21T00:26:35.824310+0000', 'last_change': '2024-09-21T00:26:35.824310+0000', 'last_active': '2024-09-21T00:26:35.811406+0000', 'last_peered': '2024-09-21T00:26:35.811406+0000', 'last_clean': '2024-09-21T00:26:35.811406+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:26:35.824310+0000', 'last_undegraded': '2024-09-21T00:26:35.824310+0000', 'last_fullsized': '2024-09-21T00:26:35.824310+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'last_clean_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-21T00:26:41.576 INFO:tasks.ceph.ceph_manager.ceph:PG 6.15 is not active+clean 2024-09-21T00:26:41.576 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-09-21T00:26:35.830680+0000', 'last_change': '2024-09-21T00:26:35.830680+0000', 'last_active': '2024-09-21T00:26:35.811406+0000', 'last_peered': '2024-09-21T00:26:35.811406+0000', 'last_clean': '2024-09-21T00:26:35.811406+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:26:35.830680+0000', 'last_undegraded': '2024-09-21T00:26:35.830680+0000', 'last_fullsized': '2024-09-21T00:26:35.830680+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'last_clean_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-21T00:26:41.576 INFO:tasks.ceph.ceph_manager.ceph:PG 6.16 is not active+clean 2024-09-21T00:26:41.577 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-09-21T00:26:35.829676+0000', 'last_change': '2024-09-21T00:26:35.829676+0000', 'last_active': '2024-09-21T00:26:35.811406+0000', 'last_peered': '2024-09-21T00:26:35.811406+0000', 'last_clean': '2024-09-21T00:26:35.811406+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:26:35.829676+0000', 'last_undegraded': '2024-09-21T00:26:35.829676+0000', 'last_fullsized': '2024-09-21T00:26:35.829676+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'last_clean_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-21T00:26:41.577 INFO:tasks.ceph.ceph_manager.ceph:PG 6.17 is not active+clean 2024-09-21T00:26:41.577 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-09-21T00:26:35.825454+0000', 'last_change': '2024-09-21T00:26:35.825454+0000', 'last_active': '2024-09-21T00:26:35.811406+0000', 'last_peered': '2024-09-21T00:26:35.811406+0000', 'last_clean': '2024-09-21T00:26:35.811406+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:26:35.825454+0000', 'last_undegraded': '2024-09-21T00:26:35.825454+0000', 'last_fullsized': '2024-09-21T00:26:35.825454+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'last_clean_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-21T00:26:41.577 INFO:tasks.ceph.ceph_manager.ceph:PG 6.10 is not active+clean 2024-09-21T00:26:41.577 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-09-21T00:26:35.826334+0000', 'last_change': '2024-09-21T00:26:35.826334+0000', 'last_active': '2024-09-21T00:26:35.811406+0000', 'last_peered': '2024-09-21T00:26:35.811406+0000', 'last_clean': '2024-09-21T00:26:35.811406+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:26:35.826334+0000', 'last_undegraded': '2024-09-21T00:26:35.826334+0000', 'last_fullsized': '2024-09-21T00:26:35.826334+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'last_clean_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-21T00:26:41.577 INFO:tasks.ceph.ceph_manager.ceph:PG 6.11 is not active+clean 2024-09-21T00:26:41.577 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-09-21T00:26:35.827602+0000', 'last_change': '2024-09-21T00:26:35.827602+0000', 'last_active': '2024-09-21T00:26:35.811406+0000', 'last_peered': '2024-09-21T00:26:35.811406+0000', 'last_clean': '2024-09-21T00:26:35.811406+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:26:35.827602+0000', 'last_undegraded': '2024-09-21T00:26:35.827602+0000', 'last_fullsized': '2024-09-21T00:26:35.827602+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'last_clean_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-21T00:26:41.577 INFO:tasks.ceph.ceph_manager.ceph:PG 6.12 is not active+clean 2024-09-21T00:26:41.577 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-09-21T00:26:35.828170+0000', 'last_change': '2024-09-21T00:26:35.828170+0000', 'last_active': '2024-09-21T00:26:35.811406+0000', 'last_peered': '2024-09-21T00:26:35.811406+0000', 'last_clean': '2024-09-21T00:26:35.811406+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:26:35.828170+0000', 'last_undegraded': '2024-09-21T00:26:35.828170+0000', 'last_fullsized': '2024-09-21T00:26:35.828170+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'last_clean_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-21T00:26:41.577 INFO:tasks.ceph.ceph_manager.ceph:PG 6.13 is not active+clean 2024-09-21T00:26:41.577 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-09-21T00:26:35.828288+0000', 'last_change': '2024-09-21T00:26:35.828288+0000', 'last_active': '2024-09-21T00:26:35.811406+0000', 'last_peered': '2024-09-21T00:26:35.811406+0000', 'last_clean': '2024-09-21T00:26:35.811406+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:26:35.828288+0000', 'last_undegraded': '2024-09-21T00:26:35.828288+0000', 'last_fullsized': '2024-09-21T00:26:35.828288+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'last_clean_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-21T00:26:41.577 INFO:tasks.ceph.ceph_manager.ceph:PG 6.c is not active+clean 2024-09-21T00:26:41.577 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-09-21T00:26:35.827957+0000', 'last_change': '2024-09-21T00:26:35.827957+0000', 'last_active': '2024-09-21T00:26:35.811406+0000', 'last_peered': '2024-09-21T00:26:35.811406+0000', 'last_clean': '2024-09-21T00:26:35.811406+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:26:35.827957+0000', 'last_undegraded': '2024-09-21T00:26:35.827957+0000', 'last_fullsized': '2024-09-21T00:26:35.827957+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'last_clean_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-21T00:26:41.578 INFO:tasks.ceph.ceph_manager.ceph:PG 6.d is not active+clean 2024-09-21T00:26:41.578 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-09-21T00:26:35.830853+0000', 'last_change': '2024-09-21T00:26:35.830853+0000', 'last_active': '2024-09-21T00:26:35.811406+0000', 'last_peered': '2024-09-21T00:26:35.811406+0000', 'last_clean': '2024-09-21T00:26:35.811406+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:26:35.830853+0000', 'last_undegraded': '2024-09-21T00:26:35.830853+0000', 'last_fullsized': '2024-09-21T00:26:35.830853+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'last_clean_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-21T00:26:41.578 INFO:tasks.ceph.ceph_manager.ceph:PG 6.e is not active+clean 2024-09-21T00:26:41.578 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-09-21T00:26:35.824938+0000', 'last_change': '2024-09-21T00:26:35.824938+0000', 'last_active': '2024-09-21T00:26:35.811406+0000', 'last_peered': '2024-09-21T00:26:35.811406+0000', 'last_clean': '2024-09-21T00:26:35.811406+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:26:35.824938+0000', 'last_undegraded': '2024-09-21T00:26:35.824938+0000', 'last_fullsized': '2024-09-21T00:26:35.824938+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'last_clean_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-21T00:26:41.578 INFO:tasks.ceph.ceph_manager.ceph:PG 6.f is not active+clean 2024-09-21T00:26:41.578 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-09-21T00:26:35.824556+0000', 'last_change': '2024-09-21T00:26:35.824556+0000', 'last_active': '2024-09-21T00:26:35.811406+0000', 'last_peered': '2024-09-21T00:26:35.811406+0000', 'last_clean': '2024-09-21T00:26:35.811406+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:26:35.824556+0000', 'last_undegraded': '2024-09-21T00:26:35.824556+0000', 'last_fullsized': '2024-09-21T00:26:35.824556+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'last_clean_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-21T00:26:41.578 INFO:tasks.ceph.ceph_manager.ceph:PG 6.8 is not active+clean 2024-09-21T00:26:41.578 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-09-21T00:26:35.827023+0000', 'last_change': '2024-09-21T00:26:35.827023+0000', 'last_active': '2024-09-21T00:26:35.811406+0000', 'last_peered': '2024-09-21T00:26:35.811406+0000', 'last_clean': '2024-09-21T00:26:35.811406+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:26:35.827023+0000', 'last_undegraded': '2024-09-21T00:26:35.827023+0000', 'last_fullsized': '2024-09-21T00:26:35.827023+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'last_clean_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-21T00:26:41.578 INFO:tasks.ceph.ceph_manager.ceph:PG 6.9 is not active+clean 2024-09-21T00:26:41.578 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-09-21T00:26:35.825112+0000', 'last_change': '2024-09-21T00:26:35.825112+0000', 'last_active': '2024-09-21T00:26:35.811406+0000', 'last_peered': '2024-09-21T00:26:35.811406+0000', 'last_clean': '2024-09-21T00:26:35.811406+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:26:35.825112+0000', 'last_undegraded': '2024-09-21T00:26:35.825112+0000', 'last_fullsized': '2024-09-21T00:26:35.825112+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'last_clean_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-21T00:26:41.579 INFO:tasks.ceph.ceph_manager.ceph:PG 6.a is not active+clean 2024-09-21T00:26:41.579 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-09-21T00:26:35.829525+0000', 'last_change': '2024-09-21T00:26:35.829525+0000', 'last_active': '2024-09-21T00:26:35.811406+0000', 'last_peered': '2024-09-21T00:26:35.811406+0000', 'last_clean': '2024-09-21T00:26:35.811406+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:26:35.829525+0000', 'last_undegraded': '2024-09-21T00:26:35.829525+0000', 'last_fullsized': '2024-09-21T00:26:35.829525+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'last_clean_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-21T00:26:41.579 INFO:tasks.ceph.ceph_manager.ceph:PG 6.b is not active+clean 2024-09-21T00:26:41.579 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-09-21T00:26:35.827774+0000', 'last_change': '2024-09-21T00:26:35.827774+0000', 'last_active': '2024-09-21T00:26:35.811406+0000', 'last_peered': '2024-09-21T00:26:35.811406+0000', 'last_clean': '2024-09-21T00:26:35.811406+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:26:35.827774+0000', 'last_undegraded': '2024-09-21T00:26:35.827774+0000', 'last_fullsized': '2024-09-21T00:26:35.827774+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'last_clean_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-21T00:26:41.579 INFO:tasks.ceph.ceph_manager.ceph:PG 6.5 is not active+clean 2024-09-21T00:26:41.579 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-09-21T00:26:35.829609+0000', 'last_change': '2024-09-21T00:26:35.829609+0000', 'last_active': '2024-09-21T00:26:35.811406+0000', 'last_peered': '2024-09-21T00:26:35.811406+0000', 'last_clean': '2024-09-21T00:26:35.811406+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:26:35.829609+0000', 'last_undegraded': '2024-09-21T00:26:35.829609+0000', 'last_fullsized': '2024-09-21T00:26:35.829609+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'last_clean_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-21T00:26:41.579 INFO:tasks.ceph.ceph_manager.ceph:PG 6.6 is not active+clean 2024-09-21T00:26:41.579 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-09-21T00:26:35.826730+0000', 'last_change': '2024-09-21T00:26:35.826730+0000', 'last_active': '2024-09-21T00:26:35.811406+0000', 'last_peered': '2024-09-21T00:26:35.811406+0000', 'last_clean': '2024-09-21T00:26:35.811406+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:26:35.826730+0000', 'last_undegraded': '2024-09-21T00:26:35.826730+0000', 'last_fullsized': '2024-09-21T00:26:35.826730+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'last_clean_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-21T00:26:41.579 INFO:tasks.ceph.ceph_manager.ceph:PG 6.0 is not active+clean 2024-09-21T00:26:41.579 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-09-21T00:26:35.825823+0000', 'last_change': '2024-09-21T00:26:35.825823+0000', 'last_active': '2024-09-21T00:26:35.811406+0000', 'last_peered': '2024-09-21T00:26:35.811406+0000', 'last_clean': '2024-09-21T00:26:35.811406+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:26:35.825823+0000', 'last_undegraded': '2024-09-21T00:26:35.825823+0000', 'last_fullsized': '2024-09-21T00:26:35.825823+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'last_clean_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-21T00:26:41.579 INFO:tasks.ceph.ceph_manager.ceph:PG 6.2 is not active+clean 2024-09-21T00:26:41.579 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-09-21T00:26:35.826853+0000', 'last_change': '2024-09-21T00:26:35.826853+0000', 'last_active': '2024-09-21T00:26:35.811406+0000', 'last_peered': '2024-09-21T00:26:35.811406+0000', 'last_clean': '2024-09-21T00:26:35.811406+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:26:35.826853+0000', 'last_undegraded': '2024-09-21T00:26:35.826853+0000', 'last_fullsized': '2024-09-21T00:26:35.826853+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'last_clean_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-21T00:26:41.580 INFO:tasks.ceph.ceph_manager.ceph:PG 6.3 is not active+clean 2024-09-21T00:26:41.580 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-09-21T00:26:35.827493+0000', 'last_change': '2024-09-21T00:26:35.827493+0000', 'last_active': '2024-09-21T00:26:35.811406+0000', 'last_peered': '2024-09-21T00:26:35.811406+0000', 'last_clean': '2024-09-21T00:26:35.811406+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:26:35.827493+0000', 'last_undegraded': '2024-09-21T00:26:35.827493+0000', 'last_fullsized': '2024-09-21T00:26:35.827493+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'last_clean_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-21T00:26:41.580 INFO:tasks.ceph.ceph_manager.ceph:PG 6.7 is not active+clean 2024-09-21T00:26:41.580 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-09-21T00:26:35.830688+0000', 'last_change': '2024-09-21T00:26:35.830688+0000', 'last_active': '2024-09-21T00:26:35.811406+0000', 'last_peered': '2024-09-21T00:26:35.811406+0000', 'last_clean': '2024-09-21T00:26:35.811406+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:26:35.830688+0000', 'last_undegraded': '2024-09-21T00:26:35.830688+0000', 'last_fullsized': '2024-09-21T00:26:35.830688+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'last_clean_scrub_stamp': '2024-09-21T00:26:35.811406+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-21T00:26:41.580 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-21T00:26:41.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.716+0000 7ffaaa4da640 1 -- 172.21.15.112:0/1587844246 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffaa40b2e40 msgr2=0x7ffaa40b3250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:41.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.716+0000 7ffaaa4da640 1 --2- 172.21.15.112:0/1587844246 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffaa40b2e40 0x7ffaa40b3250 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c005fd0 tx=0x7ffa9c02a8b0 comp rx=0 tx=0).stop 2024-09-21T00:26:41.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.716+0000 7ffaaa4da640 1 -- 172.21.15.112:0/1587844246 shutdown_connections 2024-09-21T00:26:41.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.716+0000 7ffaaa4da640 1 --2- 172.21.15.112:0/1587844246 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffaa40b2e40 0x7ffaa40b3250 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:41.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.716+0000 7ffaaa4da640 1 --2- 172.21.15.112:0/1587844246 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffaa4162f00 0x7ffaa40d24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:41.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.716+0000 7ffaaa4da640 1 -- 172.21.15.112:0/1587844246 >> 172.21.15.112:0/1587844246 conn(0x7ffaa40b6b20 msgr2=0x7ffaa40d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:41.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.716+0000 7ffaaa4da640 1 -- 172.21.15.112:0/1587844246 shutdown_connections 2024-09-21T00:26:41.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.716+0000 7ffaaa4da640 1 -- 172.21.15.112:0/1587844246 wait complete. 2024-09-21T00:26:41.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.718+0000 7ffaaa4da640 1 Processor -- start 2024-09-21T00:26:41.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.718+0000 7ffaaa4da640 1 -- start start 2024-09-21T00:26:41.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.719+0000 7ffaaa4da640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffaa40b2e40 0x7ffaa40cede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:41.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.719+0000 7ffaaa4da640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffaa4162280 0x7ffaa4149c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:41.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.719+0000 7ffaaa4da640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffaa414a180 0x7ffaa414a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:41.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.719+0000 7ffaaa4da640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ffaa415e4b0 con 0x7ffaa414a180 2024-09-21T00:26:41.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.719+0000 7ffaaa4da640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ffaa40b39a0 con 0x7ffaa40b2e40 2024-09-21T00:26:41.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.719+0000 7ffaaa4da640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ffaa415e630 con 0x7ffaa4162280 2024-09-21T00:26:41.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.719+0000 7ffaa8a77640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffaa414a180 0x7ffaa414a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:41.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.719+0000 7ffaa8a77640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffaa414a180 0x7ffaa414a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36832/0 (socket says 172.21.15.112:36832) 2024-09-21T00:26:41.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.719+0000 7ffaa8a77640 1 -- 172.21.15.112:0/3876470730 learned_addr learned my addr 172.21.15.112:0/3876470730 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:26:41.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.720+0000 7ffaa37fe640 1 --2- 172.21.15.112:0/3876470730 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffaa4162280 0x7ffaa4149c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:41.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.720+0000 7ffaa3fff640 1 --2- 172.21.15.112:0/3876470730 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffaa40b2e40 0x7ffaa40cede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:41.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.720+0000 7ffaa8a77640 1 -- 172.21.15.112:0/3876470730 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffaa4162280 msgr2=0x7ffaa4149c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:41.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.720+0000 7ffaa8a77640 1 --2- 172.21.15.112:0/3876470730 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffaa4162280 0x7ffaa4149c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:41.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.720+0000 7ffaa8a77640 1 -- 172.21.15.112:0/3876470730 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffaa40b2e40 msgr2=0x7ffaa40cede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:41.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.720+0000 7ffaa8a77640 1 --2- 172.21.15.112:0/3876470730 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffaa40b2e40 0x7ffaa40cede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:41.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.720+0000 7ffaa8a77640 1 -- 172.21.15.112:0/3876470730 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffaa414aad0 con 0x7ffaa414a180 2024-09-21T00:26:41.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.720+0000 7ffaa3fff640 1 --2- 172.21.15.112:0/3876470730 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffaa40b2e40 0x7ffaa40cede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:26:41.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.720+0000 7ffaa8a77640 1 --2- 172.21.15.112:0/3876470730 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffaa414a180 0x7ffaa414a590 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c009cd0 tx=0x7ffa9c031760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:41.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.720+0000 7ffaa0ff9640 1 -- 172.21.15.112:0/3876470730 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa9c031990 con 0x7ffaa414a180 2024-09-21T00:26:41.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.720+0000 7ffaa0ff9640 1 -- 172.21.15.112:0/3876470730 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffa9c031c20 con 0x7ffaa414a180 2024-09-21T00:26:41.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.720+0000 7ffaaa4da640 1 -- 172.21.15.112:0/3876470730 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffaa4161220 con 0x7ffaa414a180 2024-09-21T00:26:41.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.720+0000 7ffaa0ff9640 1 -- 172.21.15.112:0/3876470730 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa9c0330d0 con 0x7ffaa414a180 2024-09-21T00:26:41.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.720+0000 7ffaaa4da640 1 -- 172.21.15.112:0/3876470730 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffaa4149440 con 0x7ffaa414a180 2024-09-21T00:26:41.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.721+0000 7ffaaa4da640 1 -- 172.21.15.112:0/3876470730 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffaa4160f90 con 0x7ffaa414a180 2024-09-21T00:26:41.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.722+0000 7ffaa0ff9640 1 -- 172.21.15.112:0/3876470730 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ffa9c033270 con 0x7ffaa414a180 2024-09-21T00:26:41.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.722+0000 7ffaa0ff9640 1 --2- 172.21.15.112:0/3876470730 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ffa90078b90 0x7ffa9007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:41.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.723+0000 7ffaa0ff9640 1 -- 172.21.15.112:0/3876470730 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7ffa9c0bc380 con 0x7ffaa414a180 2024-09-21T00:26:41.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.723+0000 7ffaa3fff640 1 --2- 172.21.15.112:0/3876470730 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ffa90078b90 0x7ffa9007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:41.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.723+0000 7ffaa3fff640 1 --2- 172.21.15.112:0/3876470730 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ffa90078b90 0x7ffa9007b0a0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ffa9400bbc0 tx=0x7ffa94005ef0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:41.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.727+0000 7ffaa0ff9640 1 -- 172.21.15.112:0/3876470730 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ffa9c03f030 con 0x7ffaa414a180 2024-09-21T00:26:41.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.978+0000 7ffaaa4da640 1 -- 172.21.15.112:0/3876470730 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7ffaa41e9e30 con 0x7ffaa414a180 2024-09-21T00:26:41.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.978+0000 7ffaa0ff9640 1 -- 172.21.15.112:0/3876470730 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1684 (secure 0 0 0) 0x7ffa9c0846c0 con 0x7ffaa414a180 2024-09-21T00:26:41.980 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:26:41.980 INFO:teuthology.orchestra.run.smithi112.stdout:{"fsid":"8b34692b-dc65-49b2-a08f-26f707b96ac5","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":79,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":38,"num_osds":8,"num_up_osds":8,"osd_up_since":1726878336,"num_in_osds":8,"osd_in_since":1726878326,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":58},{"state_name":"active+clean","count":39}],"num_pgs":97,"num_pools":3,"num_objects":3,"data_bytes":590368,"bytes_used":225808384,"bytes_avail":772868304896,"bytes_total":773094113280,"inactive_pgs_ratio":0.59793812036514282,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":0},"fsmap":{"epoch":17,"btime":"2024-09-21T00:26:39:841719+0000","id":2,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":2,"rank":0,"name":"a","status":"up:active","gid":5028}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":4,"modified":"2024-09-21T00:26:37.100727+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-21T00:26:41.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.980+0000 7ffaaa4da640 1 -- 172.21.15.112:0/3876470730 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ffa90078b90 msgr2=0x7ffa9007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:41.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.980+0000 7ffaaa4da640 1 --2- 172.21.15.112:0/3876470730 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ffa90078b90 0x7ffa9007b0a0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ffa9400bbc0 tx=0x7ffa94005ef0 comp rx=0 tx=0).stop 2024-09-21T00:26:41.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.980+0000 7ffaaa4da640 1 -- 172.21.15.112:0/3876470730 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffaa414a180 msgr2=0x7ffaa414a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:41.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.980+0000 7ffaaa4da640 1 --2- 172.21.15.112:0/3876470730 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffaa414a180 0x7ffaa414a590 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c009cd0 tx=0x7ffa9c031760 comp rx=0 tx=0).stop 2024-09-21T00:26:41.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.980+0000 7ffaaa4da640 1 -- 172.21.15.112:0/3876470730 shutdown_connections 2024-09-21T00:26:41.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.980+0000 7ffaaa4da640 1 --2- 172.21.15.112:0/3876470730 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ffa90078b90 0x7ffa9007b0a0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:41.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.980+0000 7ffaaa4da640 1 --2- 172.21.15.112:0/3876470730 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffaa414a180 0x7ffaa414a590 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:41.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.980+0000 7ffaaa4da640 1 --2- 172.21.15.112:0/3876470730 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffaa4162280 0x7ffaa4149c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:41.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.980+0000 7ffaaa4da640 1 --2- 172.21.15.112:0/3876470730 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffaa40b2e40 0x7ffaa40cede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:41.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.980+0000 7ffaaa4da640 1 -- 172.21.15.112:0/3876470730 >> 172.21.15.112:0/3876470730 conn(0x7ffaa40b6b20 msgr2=0x7ffaa4162690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:41.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.981+0000 7ffaaa4da640 1 -- 172.21.15.112:0/3876470730 shutdown_connections 2024-09-21T00:26:41.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:41.981+0000 7ffaaa4da640 1 -- 172.21.15.112:0/3876470730 wait complete. 2024-09-21T00:26:41.994 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-21T00:26:41.995 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T00:26:42.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:42.105+0000 7f144e484640 1 -- 172.21.15.112:0/3886234094 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14480d0980 msgr2=0x7f14480d0d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:42.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:42.105+0000 7f144e484640 1 --2- 172.21.15.112:0/3886234094 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14480d0980 0x7f14480d0d90 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f144806dda0 tx=0x7f144002aa10 comp rx=0 tx=0).stop 2024-09-21T00:26:42.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:42.105+0000 7f144e484640 1 -- 172.21.15.112:0/3886234094 shutdown_connections 2024-09-21T00:26:42.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:42.105+0000 7f144e484640 1 --2- 172.21.15.112:0/3886234094 >> v2:172.21.15.187:3300/0 conn(0x7f144815a6f0 0x7f144815e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:42.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:42.105+0000 7f144e484640 1 --2- 172.21.15.112:0/3886234094 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14480d0980 0x7f14480d0d90 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:42.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:42.105+0000 7f144e484640 1 --2- 172.21.15.112:0/3886234094 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f14480d01f0 0x7f144815daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:42.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:42.105+0000 7f144e484640 1 -- 172.21.15.112:0/3886234094 >> 172.21.15.112:0/3886234094 conn(0x7f1448072990 msgr2=0x7f1448072da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:42.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:42.105+0000 7f144e484640 1 -- 172.21.15.112:0/3886234094 shutdown_connections 2024-09-21T00:26:42.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:42.105+0000 7f144e484640 1 -- 172.21.15.112:0/3886234094 wait complete. 2024-09-21T00:26:42.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:42.106+0000 7f144e484640 1 Processor -- start 2024-09-21T00:26:42.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:42.107+0000 7f144e484640 1 -- start start 2024-09-21T00:26:42.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:42.108+0000 7f144e484640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f14480d01f0 0x7f144806f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:42.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:42.108+0000 7f144e484640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f14480d0980 0x7f1448155250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:42.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:42.108+0000 7f144e484640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f144815a6f0 0x7f1448157380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:42.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:42.108+0000 7f144e484640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f14480ccfb0 con 0x7f144815a6f0 2024-09-21T00:26:42.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:42.108+0000 7f144e484640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f144815e980 con 0x7f14480d01f0 2024-09-21T00:26:42.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:42.108+0000 7f144e484640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f14480cd130 con 0x7f14480d0980 2024-09-21T00:26:42.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:42.108+0000 7f144ca21640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f144815a6f0 0x7f1448157380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:42.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:42.108+0000 7f144ca21640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f144815a6f0 0x7f1448157380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36848/0 (socket says 172.21.15.112:36848) 2024-09-21T00:26:42.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:42.108+0000 7f144ca21640 1 -- 172.21.15.112:0/2980950438 learned_addr learned my addr 172.21.15.112:0/2980950438 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:26:42.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:42.108+0000 7f143ffff640 1 --2- 172.21.15.112:0/2980950438 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f14480d0980 0x7f1448155250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:42.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:42.108+0000 7f1447fff640 1 --2- 172.21.15.112:0/2980950438 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f14480d01f0 0x7f144806f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:42.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:42.108+0000 7f144ca21640 1 -- 172.21.15.112:0/2980950438 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f14480d0980 msgr2=0x7f1448155250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:42.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:42.108+0000 7f144ca21640 1 --2- 172.21.15.112:0/2980950438 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f14480d0980 0x7f1448155250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:42.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:42.108+0000 7f144ca21640 1 -- 172.21.15.112:0/2980950438 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f14480d01f0 msgr2=0x7f144806f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:42.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:42.108+0000 7f144ca21640 1 --2- 172.21.15.112:0/2980950438 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f14480d01f0 0x7f144806f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:42.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:42.108+0000 7f144ca21640 1 -- 172.21.15.112:0/2980950438 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1448155b10 con 0x7f144815a6f0 2024-09-21T00:26:42.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:42.108+0000 7f144ca21640 1 --2- 172.21.15.112:0/2980950438 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f144815a6f0 0x7f1448157380 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f144002afd0 tx=0x7f1440003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:42.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:42.109+0000 7f14457fa640 1 -- 172.21.15.112:0/2980950438 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1440033050 con 0x7f144815a6f0 2024-09-21T00:26:42.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:42.109+0000 7f14457fa640 1 -- 172.21.15.112:0/2980950438 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f14400042d0 con 0x7f144815a6f0 2024-09-21T00:26:42.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:42.109+0000 7f14457fa640 1 -- 172.21.15.112:0/2980950438 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1440004720 con 0x7f144815a6f0 2024-09-21T00:26:42.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:42.109+0000 7f144e484640 1 -- 172.21.15.112:0/2980950438 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1448155df0 con 0x7f144815a6f0 2024-09-21T00:26:42.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:42.109+0000 7f144e484640 1 -- 172.21.15.112:0/2980950438 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14481f6050 con 0x7f144815a6f0 2024-09-21T00:26:42.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:42.110+0000 7f144e484640 1 -- 172.21.15.112:0/2980950438 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14480d0d90 con 0x7f144815a6f0 2024-09-21T00:26:42.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:42.110+0000 7f14457fa640 1 -- 172.21.15.112:0/2980950438 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1440036070 con 0x7f144815a6f0 2024-09-21T00:26:42.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:42.111+0000 7f14457fa640 1 --2- 172.21.15.112:0/2980950438 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1428078b90 0x7f142807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:42.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:42.111+0000 7f14457fa640 1 -- 172.21.15.112:0/2980950438 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f14400bbfe0 con 0x7f144815a6f0 2024-09-21T00:26:42.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:42.111+0000 7f1447fff640 1 --2- 172.21.15.112:0/2980950438 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1428078b90 0x7f142807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:42.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:42.112+0000 7f1447fff640 1 --2- 172.21.15.112:0/2980950438 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1428078b90 0x7f142807b0a0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1438005fd0 tx=0x7f143800b260 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:42.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:42.115+0000 7f14457fa640 1 -- 172.21.15.112:0/2980950438 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1440083c40 con 0x7f144815a6f0 2024-09-21T00:26:42.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:42.287+0000 7f144e484640 1 -- 172.21.15.112:0/2980950438 --> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f14480cc620 con 0x7f1428078b90 2024-09-21T00:26:42.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:42.292+0000 7f14457fa640 1 -- 172.21.15.112:0/2980950438 <== mgr.4103 v2:172.21.15.187:6800/2921522906 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247151 (secure 0 0 0) 0x7f14480cc620 con 0x7f1428078b90 2024-09-21T00:26:42.296 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:26:42.297 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2024-09-21T00:26:42.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:42.295+0000 7f144e484640 1 -- 172.21.15.112:0/2980950438 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1428078b90 msgr2=0x7f142807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:42.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:42.295+0000 7f144e484640 1 --2- 172.21.15.112:0/2980950438 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1428078b90 0x7f142807b0a0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1438005fd0 tx=0x7f143800b260 comp rx=0 tx=0).stop 2024-09-21T00:26:42.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:42.295+0000 7f144e484640 1 -- 172.21.15.112:0/2980950438 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f144815a6f0 msgr2=0x7f1448157380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:42.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:42.295+0000 7f144e484640 1 --2- 172.21.15.112:0/2980950438 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f144815a6f0 0x7f1448157380 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f144002afd0 tx=0x7f1440003f10 comp rx=0 tx=0).stop 2024-09-21T00:26:42.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:42.295+0000 7f144e484640 1 -- 172.21.15.112:0/2980950438 shutdown_connections 2024-09-21T00:26:42.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:42.295+0000 7f144e484640 1 --2- 172.21.15.112:0/2980950438 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1428078b90 0x7f142807b0a0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:42.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:42.295+0000 7f144e484640 1 --2- 172.21.15.112:0/2980950438 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f144815a6f0 0x7f1448157380 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:42.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:42.295+0000 7f144e484640 1 --2- 172.21.15.112:0/2980950438 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f14480d0980 0x7f1448155250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:42.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:42.295+0000 7f144e484640 1 --2- 172.21.15.112:0/2980950438 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f14480d01f0 0x7f144806f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:42.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:42.295+0000 7f144e484640 1 -- 172.21.15.112:0/2980950438 >> 172.21.15.112:0/2980950438 conn(0x7f1448072990 msgr2=0x7f144815c920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:42.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:42.295+0000 7f144e484640 1 -- 172.21.15.112:0/2980950438 shutdown_connections 2024-09-21T00:26:42.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:42.295+0000 7f144e484640 1 -- 172.21.15.112:0/2980950438 wait complete. 2024-09-21T00:26:42.308 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":65,"stamp":"2024-09-21T00:26:41.101777+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":104,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":89,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220516,"kb_used_data":4284,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754204,"statfs":{"total":773094113280,"available":772868304896,"internally_reserved":0,"allocated":4386816,"data_stored":2498882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001572"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.829346+0000","last_change":"2024-09-21T00:26:35.829346+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.829346+0000","last_undegraded":"2024-09-21T00:26:35.829346+0000","last_fullsized":"2024-09-21T00:26:35.829346+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.827322+0000","last_change":"2024-09-21T00:26:35.827322+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.827322+0000","last_undegraded":"2024-09-21T00:26:35.827322+0000","last_fullsized":"2024-09-21T00:26:35.827322+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.828578+0000","last_change":"2024-09-21T00:26:35.828578+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.828578+0000","last_undegraded":"2024-09-21T00:26:35.828578+0000","last_fullsized":"2024-09-21T00:26:35.828578+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.825190+0000","last_change":"2024-09-21T00:26:35.825190+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.825190+0000","last_undegraded":"2024-09-21T00:26:35.825190+0000","last_fullsized":"2024-09-21T00:26:35.825190+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.825161+0000","last_change":"2024-09-21T00:26:35.825161+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.825161+0000","last_undegraded":"2024-09-21T00:26:35.825161+0000","last_fullsized":"2024-09-21T00:26:35.825161+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-21T00:26:36.844469+0000","last_change":"2024-09-21T00:26:36.841381+0000","last_active":"2024-09-21T00:26:36.844469+0000","last_peered":"2024-09-21T00:26:36.844469+0000","last_clean":"2024-09-21T00:26:36.844469+0000","last_became_active":"2024-09-21T00:26:36.841094+0000","last_became_peered":"2024-09-21T00:26:36.841094+0000","last_unstale":"2024-09-21T00:26:36.844469+0000","last_undegraded":"2024-09-21T00:26:36.844469+0000","last_fullsized":"2024-09-21T00:26:36.844469+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:55:25.224768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.832564+0000","last_change":"2024-09-21T00:26:35.832564+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.832564+0000","last_undegraded":"2024-09-21T00:26:35.832564+0000","last_fullsized":"2024-09-21T00:26:35.832564+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.830845+0000","last_change":"2024-09-21T00:26:35.830845+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.830845+0000","last_undegraded":"2024-09-21T00:26:35.830845+0000","last_fullsized":"2024-09-21T00:26:35.830845+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.830587+0000","last_change":"2024-09-21T00:26:35.830587+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.830587+0000","last_undegraded":"2024-09-21T00:26:35.830587+0000","last_fullsized":"2024-09-21T00:26:35.830587+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.833967+0000","last_change":"2024-09-21T00:26:35.833967+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.833967+0000","last_undegraded":"2024-09-21T00:26:35.833967+0000","last_fullsized":"2024-09-21T00:26:35.833967+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.832457+0000","last_change":"2024-09-21T00:26:35.832457+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.832457+0000","last_undegraded":"2024-09-21T00:26:35.832457+0000","last_fullsized":"2024-09-21T00:26:35.832457+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.828280+0000","last_change":"2024-09-21T00:26:35.828280+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.828280+0000","last_undegraded":"2024-09-21T00:26:35.828280+0000","last_fullsized":"2024-09-21T00:26:35.828280+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.831355+0000","last_change":"2024-09-21T00:26:35.831355+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.831355+0000","last_undegraded":"2024-09-21T00:26:35.831355+0000","last_fullsized":"2024-09-21T00:26:35.831355+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.827439+0000","last_change":"2024-09-21T00:26:35.827439+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.827439+0000","last_undegraded":"2024-09-21T00:26:35.827439+0000","last_fullsized":"2024-09-21T00:26:35.827439+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.829991+0000","last_change":"2024-09-21T00:26:35.829991+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.829991+0000","last_undegraded":"2024-09-21T00:26:35.829991+0000","last_fullsized":"2024-09-21T00:26:35.829991+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.828731+0000","last_change":"2024-09-21T00:26:35.828731+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.828731+0000","last_undegraded":"2024-09-21T00:26:35.828731+0000","last_fullsized":"2024-09-21T00:26:35.828731+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.828322+0000","last_change":"2024-09-21T00:26:35.828322+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.828322+0000","last_undegraded":"2024-09-21T00:26:35.828322+0000","last_fullsized":"2024-09-21T00:26:35.828322+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.827556+0000","last_change":"2024-09-21T00:26:35.827556+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.827556+0000","last_undegraded":"2024-09-21T00:26:35.827556+0000","last_fullsized":"2024-09-21T00:26:35.827556+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.823845+0000","last_change":"2024-09-21T00:26:35.823845+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.823845+0000","last_undegraded":"2024-09-21T00:26:35.823845+0000","last_fullsized":"2024-09-21T00:26:35.823845+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.827691+0000","last_change":"2024-09-21T00:26:35.827691+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.827691+0000","last_undegraded":"2024-09-21T00:26:35.827691+0000","last_fullsized":"2024-09-21T00:26:35.827691+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.831846+0000","last_change":"2024-09-21T00:26:35.831846+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.831846+0000","last_undegraded":"2024-09-21T00:26:35.831846+0000","last_fullsized":"2024-09-21T00:26:35.831846+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.823745+0000","last_change":"2024-09-21T00:26:35.823745+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.823745+0000","last_undegraded":"2024-09-21T00:26:35.823745+0000","last_fullsized":"2024-09-21T00:26:35.823745+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.827636+0000","last_change":"2024-09-21T00:26:35.827636+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.827636+0000","last_undegraded":"2024-09-21T00:26:35.827636+0000","last_fullsized":"2024-09-21T00:26:35.827636+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.823856+0000","last_change":"2024-09-21T00:26:35.823856+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.823856+0000","last_undegraded":"2024-09-21T00:26:35.823856+0000","last_fullsized":"2024-09-21T00:26:35.823856+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.826279+0000","last_change":"2024-09-21T00:26:35.826279+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.826279+0000","last_undegraded":"2024-09-21T00:26:35.826279+0000","last_fullsized":"2024-09-21T00:26:35.826279+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-21T00:26:36.859512+0000","last_change":"2024-09-21T00:26:36.856780+0000","last_active":"2024-09-21T00:26:36.859512+0000","last_peered":"2024-09-21T00:26:36.859512+0000","last_clean":"2024-09-21T00:26:36.859512+0000","last_became_active":"2024-09-21T00:26:36.856482+0000","last_became_peered":"2024-09-21T00:26:36.856482+0000","last_unstale":"2024-09-21T00:26:36.859512+0000","last_undegraded":"2024-09-21T00:26:36.859512+0000","last_fullsized":"2024-09-21T00:26:36.859512+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:58:37.004181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.825808+0000","last_change":"2024-09-21T00:26:35.825808+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.825808+0000","last_undegraded":"2024-09-21T00:26:35.825808+0000","last_fullsized":"2024-09-21T00:26:35.825808+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.830953+0000","last_change":"2024-09-21T00:26:35.830953+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.830953+0000","last_undegraded":"2024-09-21T00:26:35.830953+0000","last_fullsized":"2024-09-21T00:26:35.830953+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.824648+0000","last_change":"2024-09-21T00:26:35.824648+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.824648+0000","last_undegraded":"2024-09-21T00:26:35.824648+0000","last_fullsized":"2024-09-21T00:26:35.824648+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.826507+0000","last_change":"2024-09-21T00:26:35.826507+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.826507+0000","last_undegraded":"2024-09-21T00:26:35.826507+0000","last_fullsized":"2024-09-21T00:26:35.826507+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.826999+0000","last_change":"2024-09-21T00:26:35.826999+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.826999+0000","last_undegraded":"2024-09-21T00:26:35.826999+0000","last_fullsized":"2024-09-21T00:26:35.826999+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-21T00:26:36.838547+0000","last_change":"2024-09-21T00:26:36.836602+0000","last_active":"2024-09-21T00:26:36.838547+0000","last_peered":"2024-09-21T00:26:36.838547+0000","last_clean":"2024-09-21T00:26:36.838547+0000","last_became_active":"2024-09-21T00:26:36.835713+0000","last_became_peered":"2024-09-21T00:26:36.835713+0000","last_unstale":"2024-09-21T00:26:36.838547+0000","last_undegraded":"2024-09-21T00:26:36.838547+0000","last_fullsized":"2024-09-21T00:26:36.838547+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:23:48.458736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.828100+0000","last_change":"2024-09-21T00:26:35.828100+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.828100+0000","last_undegraded":"2024-09-21T00:26:35.828100+0000","last_fullsized":"2024-09-21T00:26:35.828100+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-21T00:26:35.844965+0000","last_change":"2024-09-21T00:26:35.838321+0000","last_active":"2024-09-21T00:26:35.844965+0000","last_peered":"2024-09-21T00:26:35.844965+0000","last_clean":"2024-09-21T00:26:35.844965+0000","last_became_active":"2024-09-21T00:26:35.838022+0000","last_became_peered":"2024-09-21T00:26:35.838022+0000","last_unstale":"2024-09-21T00:26:35.844965+0000","last_undegraded":"2024-09-21T00:26:35.844965+0000","last_fullsized":"2024-09-21T00:26:35.844965+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:05:48.525312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-21T00:26:36.859717+0000","last_change":"2024-09-21T00:26:36.856958+0000","last_active":"2024-09-21T00:26:36.859717+0000","last_peered":"2024-09-21T00:26:36.859717+0000","last_clean":"2024-09-21T00:26:36.859717+0000","last_became_active":"2024-09-21T00:26:36.856395+0000","last_became_peered":"2024-09-21T00:26:36.856395+0000","last_unstale":"2024-09-21T00:26:36.859717+0000","last_undegraded":"2024-09-21T00:26:36.859717+0000","last_fullsized":"2024-09-21T00:26:36.859717+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:15:12.241543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-21T00:26:35.839447+0000","last_change":"2024-09-21T00:26:35.837664+0000","last_active":"2024-09-21T00:26:35.839447+0000","last_peered":"2024-09-21T00:26:35.839447+0000","last_clean":"2024-09-21T00:26:35.839447+0000","last_became_active":"2024-09-21T00:26:35.837355+0000","last_became_peered":"2024-09-21T00:26:35.837355+0000","last_unstale":"2024-09-21T00:26:35.839447+0000","last_undegraded":"2024-09-21T00:26:35.839447+0000","last_fullsized":"2024-09-21T00:26:35.839447+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T00:41:22.116310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.824468+0000","last_change":"2024-09-21T00:26:35.824468+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.824468+0000","last_undegraded":"2024-09-21T00:26:35.824468+0000","last_fullsized":"2024-09-21T00:26:35.824468+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"37'2","reported_seq":15,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-21T00:26:36.853187+0000","last_change":"2024-09-21T00:26:35.840905+0000","last_active":"2024-09-21T00:26:36.853187+0000","last_peered":"2024-09-21T00:26:36.853187+0000","last_clean":"2024-09-21T00:26:36.853187+0000","last_became_active":"2024-09-21T00:26:35.840594+0000","last_became_peered":"2024-09-21T00:26:35.840594+0000","last_unstale":"2024-09-21T00:26:36.853187+0000","last_undegraded":"2024-09-21T00:26:36.853187+0000","last_fullsized":"2024-09-21T00:26:36.853187+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:54:48.838959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.829845+0000","last_change":"2024-09-21T00:26:35.829845+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.829845+0000","last_undegraded":"2024-09-21T00:26:35.829845+0000","last_fullsized":"2024-09-21T00:26:35.829845+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-21T00:26:35.842277+0000","last_change":"2024-09-21T00:26:35.836092+0000","last_active":"2024-09-21T00:26:35.842277+0000","last_peered":"2024-09-21T00:26:35.842277+0000","last_clean":"2024-09-21T00:26:35.842277+0000","last_became_active":"2024-09-21T00:26:35.835424+0000","last_became_peered":"2024-09-21T00:26:35.835424+0000","last_unstale":"2024-09-21T00:26:35.842277+0000","last_undegraded":"2024-09-21T00:26:35.842277+0000","last_fullsized":"2024-09-21T00:26:35.842277+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:21:21.377354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.826825+0000","last_change":"2024-09-21T00:26:35.826825+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.826825+0000","last_undegraded":"2024-09-21T00:26:35.826825+0000","last_fullsized":"2024-09-21T00:26:35.826825+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-21T00:26:35.841980+0000","last_change":"2024-09-21T00:26:35.835778+0000","last_active":"2024-09-21T00:26:35.841980+0000","last_peered":"2024-09-21T00:26:35.841980+0000","last_clean":"2024-09-21T00:26:35.841980+0000","last_became_active":"2024-09-21T00:26:35.835426+0000","last_became_peered":"2024-09-21T00:26:35.835426+0000","last_unstale":"2024-09-21T00:26:35.841980+0000","last_undegraded":"2024-09-21T00:26:35.841980+0000","last_fullsized":"2024-09-21T00:26:35.841980+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:49:14.000042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.829387+0000","last_change":"2024-09-21T00:26:35.829387+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.829387+0000","last_undegraded":"2024-09-21T00:26:35.829387+0000","last_fullsized":"2024-09-21T00:26:35.829387+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-21T00:26:35.850340+0000","last_change":"2024-09-21T00:26:35.846072+0000","last_active":"2024-09-21T00:26:35.850340+0000","last_peered":"2024-09-21T00:26:35.850340+0000","last_clean":"2024-09-21T00:26:35.850340+0000","last_became_active":"2024-09-21T00:26:35.845724+0000","last_became_peered":"2024-09-21T00:26:35.845724+0000","last_unstale":"2024-09-21T00:26:35.850340+0000","last_undegraded":"2024-09-21T00:26:35.850340+0000","last_fullsized":"2024-09-21T00:26:35.850340+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:49:14.000042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.825895+0000","last_change":"2024-09-21T00:26:35.825895+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.825895+0000","last_undegraded":"2024-09-21T00:26:35.825895+0000","last_fullsized":"2024-09-21T00:26:35.825895+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-21T00:26:36.831276+0000","last_change":"2024-09-21T00:26:35.842674+0000","last_active":"2024-09-21T00:26:36.831276+0000","last_peered":"2024-09-21T00:26:36.831276+0000","last_clean":"2024-09-21T00:26:36.831276+0000","last_became_active":"2024-09-21T00:26:35.842339+0000","last_became_peered":"2024-09-21T00:26:35.842339+0000","last_unstale":"2024-09-21T00:26:36.831276+0000","last_undegraded":"2024-09-21T00:26:36.831276+0000","last_fullsized":"2024-09-21T00:26:36.831276+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:56:47.623787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.828735+0000","last_change":"2024-09-21T00:26:35.828735+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.828735+0000","last_undegraded":"2024-09-21T00:26:35.828735+0000","last_fullsized":"2024-09-21T00:26:35.828735+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-21T00:26:35.839220+0000","last_change":"2024-09-21T00:26:35.835252+0000","last_active":"2024-09-21T00:26:35.839220+0000","last_peered":"2024-09-21T00:26:35.839220+0000","last_clean":"2024-09-21T00:26:35.839220+0000","last_became_active":"2024-09-21T00:26:35.834768+0000","last_became_peered":"2024-09-21T00:26:35.834768+0000","last_unstale":"2024-09-21T00:26:35.839220+0000","last_undegraded":"2024-09-21T00:26:35.839220+0000","last_fullsized":"2024-09-21T00:26:35.839220+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:29:33.119014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.824310+0000","last_change":"2024-09-21T00:26:35.824310+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.824310+0000","last_undegraded":"2024-09-21T00:26:35.824310+0000","last_fullsized":"2024-09-21T00:26:35.824310+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-21T00:26:35.836982+0000","last_change":"2024-09-21T00:26:35.834813+0000","last_active":"2024-09-21T00:26:35.836982+0000","last_peered":"2024-09-21T00:26:35.836982+0000","last_clean":"2024-09-21T00:26:35.836982+0000","last_became_active":"2024-09-21T00:26:35.834218+0000","last_became_peered":"2024-09-21T00:26:35.834218+0000","last_unstale":"2024-09-21T00:26:35.836982+0000","last_undegraded":"2024-09-21T00:26:35.836982+0000","last_fullsized":"2024-09-21T00:26:35.836982+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:41:29.077144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.830680+0000","last_change":"2024-09-21T00:26:35.830680+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.830680+0000","last_undegraded":"2024-09-21T00:26:35.830680+0000","last_fullsized":"2024-09-21T00:26:35.830680+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-21T00:26:35.842651+0000","last_change":"2024-09-21T00:26:35.841364+0000","last_active":"2024-09-21T00:26:35.842651+0000","last_peered":"2024-09-21T00:26:35.842651+0000","last_clean":"2024-09-21T00:26:35.842651+0000","last_became_active":"2024-09-21T00:26:35.840989+0000","last_became_peered":"2024-09-21T00:26:35.840989+0000","last_unstale":"2024-09-21T00:26:35.842651+0000","last_undegraded":"2024-09-21T00:26:35.842651+0000","last_fullsized":"2024-09-21T00:26:35.842651+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T00:47:37.835274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.829676+0000","last_change":"2024-09-21T00:26:35.829676+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.829676+0000","last_undegraded":"2024-09-21T00:26:35.829676+0000","last_fullsized":"2024-09-21T00:26:35.829676+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-21T00:26:35.842468+0000","last_change":"2024-09-21T00:26:35.836960+0000","last_active":"2024-09-21T00:26:35.842468+0000","last_peered":"2024-09-21T00:26:35.842468+0000","last_clean":"2024-09-21T00:26:35.842468+0000","last_became_active":"2024-09-21T00:26:35.836641+0000","last_became_peered":"2024-09-21T00:26:35.836641+0000","last_unstale":"2024-09-21T00:26:35.842468+0000","last_undegraded":"2024-09-21T00:26:35.842468+0000","last_fullsized":"2024-09-21T00:26:35.842468+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:50:54.008029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.825454+0000","last_change":"2024-09-21T00:26:35.825454+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.825454+0000","last_undegraded":"2024-09-21T00:26:35.825454+0000","last_fullsized":"2024-09-21T00:26:35.825454+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-21T00:26:35.836435+0000","last_change":"2024-09-21T00:26:35.834520+0000","last_active":"2024-09-21T00:26:35.836435+0000","last_peered":"2024-09-21T00:26:35.836435+0000","last_clean":"2024-09-21T00:26:35.836435+0000","last_became_active":"2024-09-21T00:26:35.834146+0000","last_became_peered":"2024-09-21T00:26:35.834146+0000","last_unstale":"2024-09-21T00:26:35.836435+0000","last_undegraded":"2024-09-21T00:26:35.836435+0000","last_fullsized":"2024-09-21T00:26:35.836435+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T00:47:37.835274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.826334+0000","last_change":"2024-09-21T00:26:35.826334+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.826334+0000","last_undegraded":"2024-09-21T00:26:35.826334+0000","last_fullsized":"2024-09-21T00:26:35.826334+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-21T00:26:35.843305+0000","last_change":"2024-09-21T00:26:35.840502+0000","last_active":"2024-09-21T00:26:35.843305+0000","last_peered":"2024-09-21T00:26:35.843305+0000","last_clean":"2024-09-21T00:26:35.843305+0000","last_became_active":"2024-09-21T00:26:35.840189+0000","last_became_peered":"2024-09-21T00:26:35.840189+0000","last_unstale":"2024-09-21T00:26:35.843305+0000","last_undegraded":"2024-09-21T00:26:35.843305+0000","last_fullsized":"2024-09-21T00:26:35.843305+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:41:13.666350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.827602+0000","last_change":"2024-09-21T00:26:35.827602+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.827602+0000","last_undegraded":"2024-09-21T00:26:35.827602+0000","last_fullsized":"2024-09-21T00:26:35.827602+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-21T00:26:36.831536+0000","last_change":"2024-09-21T00:26:35.842688+0000","last_active":"2024-09-21T00:26:36.831536+0000","last_peered":"2024-09-21T00:26:36.831536+0000","last_clean":"2024-09-21T00:26:36.831536+0000","last_became_active":"2024-09-21T00:26:35.842375+0000","last_became_peered":"2024-09-21T00:26:35.842375+0000","last_unstale":"2024-09-21T00:26:36.831536+0000","last_undegraded":"2024-09-21T00:26:36.831536+0000","last_fullsized":"2024-09-21T00:26:36.831536+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:58:30.617062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.828170+0000","last_change":"2024-09-21T00:26:35.828170+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.828170+0000","last_undegraded":"2024-09-21T00:26:35.828170+0000","last_fullsized":"2024-09-21T00:26:35.828170+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-21T00:26:35.844992+0000","last_change":"2024-09-21T00:26:35.839140+0000","last_active":"2024-09-21T00:26:35.844992+0000","last_peered":"2024-09-21T00:26:35.844992+0000","last_clean":"2024-09-21T00:26:35.844992+0000","last_became_active":"2024-09-21T00:26:35.838817+0000","last_became_peered":"2024-09-21T00:26:35.838817+0000","last_unstale":"2024-09-21T00:26:35.844992+0000","last_undegraded":"2024-09-21T00:26:35.844992+0000","last_fullsized":"2024-09-21T00:26:35.844992+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:07:46.709919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.828288+0000","last_change":"2024-09-21T00:26:35.828288+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.828288+0000","last_undegraded":"2024-09-21T00:26:35.828288+0000","last_fullsized":"2024-09-21T00:26:35.828288+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-21T00:26:35.850329+0000","last_change":"2024-09-21T00:26:35.846027+0000","last_active":"2024-09-21T00:26:35.850329+0000","last_peered":"2024-09-21T00:26:35.850329+0000","last_clean":"2024-09-21T00:26:35.850329+0000","last_became_active":"2024-09-21T00:26:35.845681+0000","last_became_peered":"2024-09-21T00:26:35.845681+0000","last_unstale":"2024-09-21T00:26:35.850329+0000","last_undegraded":"2024-09-21T00:26:35.850329+0000","last_fullsized":"2024-09-21T00:26:35.850329+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:11:52.985613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.827957+0000","last_change":"2024-09-21T00:26:35.827957+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.827957+0000","last_undegraded":"2024-09-21T00:26:35.827957+0000","last_fullsized":"2024-09-21T00:26:35.827957+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-21T00:26:35.850340+0000","last_change":"2024-09-21T00:26:35.845969+0000","last_active":"2024-09-21T00:26:35.850340+0000","last_peered":"2024-09-21T00:26:35.850340+0000","last_clean":"2024-09-21T00:26:35.850340+0000","last_became_active":"2024-09-21T00:26:35.845633+0000","last_became_peered":"2024-09-21T00:26:35.845633+0000","last_unstale":"2024-09-21T00:26:35.850340+0000","last_undegraded":"2024-09-21T00:26:35.850340+0000","last_fullsized":"2024-09-21T00:26:35.850340+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:41:29.077144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.830853+0000","last_change":"2024-09-21T00:26:35.830853+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.830853+0000","last_undegraded":"2024-09-21T00:26:35.830853+0000","last_fullsized":"2024-09-21T00:26:35.830853+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-21T00:26:35.845305+0000","last_change":"2024-09-21T00:26:35.840529+0000","last_active":"2024-09-21T00:26:35.845305+0000","last_peered":"2024-09-21T00:26:35.845305+0000","last_clean":"2024-09-21T00:26:35.845305+0000","last_became_active":"2024-09-21T00:26:35.840115+0000","last_became_peered":"2024-09-21T00:26:35.840115+0000","last_unstale":"2024-09-21T00:26:35.845305+0000","last_undegraded":"2024-09-21T00:26:35.845305+0000","last_fullsized":"2024-09-21T00:26:35.845305+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:33:02.079644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.824938+0000","last_change":"2024-09-21T00:26:35.824938+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.824938+0000","last_undegraded":"2024-09-21T00:26:35.824938+0000","last_fullsized":"2024-09-21T00:26:35.824938+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-21T00:26:35.838164+0000","last_change":"2024-09-21T00:26:35.835450+0000","last_active":"2024-09-21T00:26:35.838164+0000","last_peered":"2024-09-21T00:26:35.838164+0000","last_clean":"2024-09-21T00:26:35.838164+0000","last_became_active":"2024-09-21T00:26:35.835141+0000","last_became_peered":"2024-09-21T00:26:35.835141+0000","last_unstale":"2024-09-21T00:26:35.838164+0000","last_undegraded":"2024-09-21T00:26:35.838164+0000","last_fullsized":"2024-09-21T00:26:35.838164+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:11:09.781135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.824556+0000","last_change":"2024-09-21T00:26:35.824556+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.824556+0000","last_undegraded":"2024-09-21T00:26:35.824556+0000","last_fullsized":"2024-09-21T00:26:35.824556+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-21T00:26:36.831408+0000","last_change":"2024-09-21T00:26:35.841410+0000","last_active":"2024-09-21T00:26:36.831408+0000","last_peered":"2024-09-21T00:26:36.831408+0000","last_clean":"2024-09-21T00:26:36.831408+0000","last_became_active":"2024-09-21T00:26:35.841094+0000","last_became_peered":"2024-09-21T00:26:35.841094+0000","last_unstale":"2024-09-21T00:26:36.831408+0000","last_undegraded":"2024-09-21T00:26:36.831408+0000","last_fullsized":"2024-09-21T00:26:36.831408+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:48:22.317860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.827023+0000","last_change":"2024-09-21T00:26:35.827023+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.827023+0000","last_undegraded":"2024-09-21T00:26:35.827023+0000","last_fullsized":"2024-09-21T00:26:35.827023+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-21T00:26:35.843267+0000","last_change":"2024-09-21T00:26:35.836023+0000","last_active":"2024-09-21T00:26:35.843267+0000","last_peered":"2024-09-21T00:26:35.843267+0000","last_clean":"2024-09-21T00:26:35.843267+0000","last_became_active":"2024-09-21T00:26:35.835721+0000","last_became_peered":"2024-09-21T00:26:35.835721+0000","last_unstale":"2024-09-21T00:26:35.843267+0000","last_undegraded":"2024-09-21T00:26:35.843267+0000","last_fullsized":"2024-09-21T00:26:35.843267+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:43:45.222366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.825112+0000","last_change":"2024-09-21T00:26:35.825112+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.825112+0000","last_undegraded":"2024-09-21T00:26:35.825112+0000","last_fullsized":"2024-09-21T00:26:35.825112+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-21T00:26:35.843093+0000","last_change":"2024-09-21T00:26:35.837224+0000","last_active":"2024-09-21T00:26:35.843093+0000","last_peered":"2024-09-21T00:26:35.843093+0000","last_clean":"2024-09-21T00:26:35.843093+0000","last_became_active":"2024-09-21T00:26:35.836849+0000","last_became_peered":"2024-09-21T00:26:35.836849+0000","last_unstale":"2024-09-21T00:26:35.843093+0000","last_undegraded":"2024-09-21T00:26:35.843093+0000","last_fullsized":"2024-09-21T00:26:35.843093+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:59:09.929281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.829525+0000","last_change":"2024-09-21T00:26:35.829525+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.829525+0000","last_undegraded":"2024-09-21T00:26:35.829525+0000","last_fullsized":"2024-09-21T00:26:35.829525+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-21T00:26:35.842275+0000","last_change":"2024-09-21T00:26:35.837934+0000","last_active":"2024-09-21T00:26:35.842275+0000","last_peered":"2024-09-21T00:26:35.842275+0000","last_clean":"2024-09-21T00:26:35.842275+0000","last_became_active":"2024-09-21T00:26:35.837198+0000","last_became_peered":"2024-09-21T00:26:35.837198+0000","last_unstale":"2024-09-21T00:26:35.842275+0000","last_undegraded":"2024-09-21T00:26:35.842275+0000","last_fullsized":"2024-09-21T00:26:35.842275+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:04:35.145795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.827774+0000","last_change":"2024-09-21T00:26:35.827774+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.827774+0000","last_undegraded":"2024-09-21T00:26:35.827774+0000","last_fullsized":"2024-09-21T00:26:35.827774+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-21T00:26:35.836465+0000","last_change":"2024-09-21T00:26:35.832534+0000","last_active":"2024-09-21T00:26:35.836465+0000","last_peered":"2024-09-21T00:26:35.836465+0000","last_clean":"2024-09-21T00:26:35.836465+0000","last_became_active":"2024-09-21T00:26:35.832197+0000","last_became_peered":"2024-09-21T00:26:35.832197+0000","last_unstale":"2024-09-21T00:26:35.836465+0000","last_undegraded":"2024-09-21T00:26:35.836465+0000","last_fullsized":"2024-09-21T00:26:35.836465+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:28:49.838026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-21T00:26:36.844617+0000","last_change":"2024-09-21T00:26:36.841191+0000","last_active":"2024-09-21T00:26:36.844617+0000","last_peered":"2024-09-21T00:26:36.844617+0000","last_clean":"2024-09-21T00:26:36.844617+0000","last_became_active":"2024-09-21T00:26:36.840676+0000","last_became_peered":"2024-09-21T00:26:36.840676+0000","last_unstale":"2024-09-21T00:26:36.844617+0000","last_undegraded":"2024-09-21T00:26:36.844617+0000","last_fullsized":"2024-09-21T00:26:36.844617+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:16:54.958692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-21T00:26:35.842557+0000","last_change":"2024-09-21T00:26:35.840350+0000","last_active":"2024-09-21T00:26:35.842557+0000","last_peered":"2024-09-21T00:26:35.842557+0000","last_clean":"2024-09-21T00:26:35.842557+0000","last_became_active":"2024-09-21T00:26:35.840048+0000","last_became_peered":"2024-09-21T00:26:35.840048+0000","last_unstale":"2024-09-21T00:26:35.842557+0000","last_undegraded":"2024-09-21T00:26:35.842557+0000","last_fullsized":"2024-09-21T00:26:35.842557+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:19:54.299718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.829609+0000","last_change":"2024-09-21T00:26:35.829609+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.829609+0000","last_undegraded":"2024-09-21T00:26:35.829609+0000","last_fullsized":"2024-09-21T00:26:35.829609+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-21T00:26:35.846174+0000","last_change":"2024-09-21T00:26:35.838940+0000","last_active":"2024-09-21T00:26:35.846174+0000","last_peered":"2024-09-21T00:26:35.846174+0000","last_clean":"2024-09-21T00:26:35.846174+0000","last_became_active":"2024-09-21T00:26:35.838616+0000","last_became_peered":"2024-09-21T00:26:35.838616+0000","last_unstale":"2024-09-21T00:26:35.846174+0000","last_undegraded":"2024-09-21T00:26:35.846174+0000","last_fullsized":"2024-09-21T00:26:35.846174+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:01:07.494519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.826730+0000","last_change":"2024-09-21T00:26:35.826730+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.826730+0000","last_undegraded":"2024-09-21T00:26:35.826730+0000","last_fullsized":"2024-09-21T00:26:35.826730+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-21T00:26:35.841029+0000","last_change":"2024-09-21T00:26:35.838748+0000","last_active":"2024-09-21T00:26:35.841029+0000","last_peered":"2024-09-21T00:26:35.841029+0000","last_clean":"2024-09-21T00:26:35.841029+0000","last_became_active":"2024-09-21T00:26:35.838445+0000","last_became_peered":"2024-09-21T00:26:35.838445+0000","last_unstale":"2024-09-21T00:26:35.841029+0000","last_undegraded":"2024-09-21T00:26:35.841029+0000","last_fullsized":"2024-09-21T00:26:35.841029+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:44:34.009179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.825823+0000","last_change":"2024-09-21T00:26:35.825823+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.825823+0000","last_undegraded":"2024-09-21T00:26:35.825823+0000","last_fullsized":"2024-09-21T00:26:35.825823+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-21T00:26:35.843172+0000","last_change":"2024-09-21T00:26:35.840565+0000","last_active":"2024-09-21T00:26:35.843172+0000","last_peered":"2024-09-21T00:26:35.843172+0000","last_clean":"2024-09-21T00:26:35.843172+0000","last_became_active":"2024-09-21T00:26:35.840265+0000","last_became_peered":"2024-09-21T00:26:35.840265+0000","last_unstale":"2024-09-21T00:26:35.843172+0000","last_undegraded":"2024-09-21T00:26:35.843172+0000","last_fullsized":"2024-09-21T00:26:35.843172+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:14:15.665953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-21T00:26:36.848314+0000","last_change":"2024-09-21T00:26:36.842397+0000","last_active":"2024-09-21T00:26:36.848314+0000","last_peered":"2024-09-21T00:26:36.848314+0000","last_clean":"2024-09-21T00:26:36.848314+0000","last_became_active":"2024-09-21T00:26:36.842095+0000","last_became_peered":"2024-09-21T00:26:36.842095+0000","last_unstale":"2024-09-21T00:26:36.848314+0000","last_undegraded":"2024-09-21T00:26:36.848314+0000","last_fullsized":"2024-09-21T00:26:36.848314+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:50:12.134645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-21T00:26:35.837296+0000","last_change":"2024-09-21T00:26:35.835982+0000","last_active":"2024-09-21T00:26:35.837296+0000","last_peered":"2024-09-21T00:26:35.837296+0000","last_clean":"2024-09-21T00:26:35.837296+0000","last_became_active":"2024-09-21T00:26:35.835677+0000","last_became_peered":"2024-09-21T00:26:35.835677+0000","last_unstale":"2024-09-21T00:26:35.837296+0000","last_undegraded":"2024-09-21T00:26:35.837296+0000","last_fullsized":"2024-09-21T00:26:35.837296+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:46:16.688217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.826853+0000","last_change":"2024-09-21T00:26:35.826853+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.826853+0000","last_undegraded":"2024-09-21T00:26:35.826853+0000","last_fullsized":"2024-09-21T00:26:35.826853+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-21T00:26:35.842371+0000","last_change":"2024-09-21T00:26:35.838318+0000","last_active":"2024-09-21T00:26:35.842371+0000","last_peered":"2024-09-21T00:26:35.842371+0000","last_clean":"2024-09-21T00:26:35.842371+0000","last_became_active":"2024-09-21T00:26:35.836426+0000","last_became_peered":"2024-09-21T00:26:35.836426+0000","last_unstale":"2024-09-21T00:26:35.842371+0000","last_undegraded":"2024-09-21T00:26:35.842371+0000","last_fullsized":"2024-09-21T00:26:35.842371+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:48:50.639166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.827493+0000","last_change":"2024-09-21T00:26:35.827493+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.827493+0000","last_undegraded":"2024-09-21T00:26:35.827493+0000","last_fullsized":"2024-09-21T00:26:35.827493+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-21T00:26:35.841994+0000","last_change":"2024-09-21T00:26:35.836745+0000","last_active":"2024-09-21T00:26:35.841994+0000","last_peered":"2024-09-21T00:26:35.841994+0000","last_clean":"2024-09-21T00:26:35.841994+0000","last_became_active":"2024-09-21T00:26:35.836427+0000","last_became_peered":"2024-09-21T00:26:35.836427+0000","last_unstale":"2024-09-21T00:26:35.841994+0000","last_undegraded":"2024-09-21T00:26:35.841994+0000","last_fullsized":"2024-09-21T00:26:35.841994+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:33:15.124204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-09-21T00:26:35.830688+0000","last_change":"2024-09-21T00:26:35.830688+0000","last_active":"2024-09-21T00:26:35.811406+0000","last_peered":"2024-09-21T00:26:35.811406+0000","last_clean":"2024-09-21T00:26:35.811406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:26:35.830688+0000","last_undegraded":"2024-09-21T00:26:35.830688+0000","last_fullsized":"2024-09-21T00:26:35.830688+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-21T00:26:35.845045+0000","last_change":"2024-09-21T00:26:35.839244+0000","last_active":"2024-09-21T00:26:35.845045+0000","last_peered":"2024-09-21T00:26:35.845045+0000","last_clean":"2024-09-21T00:26:35.845045+0000","last_became_active":"2024-09-21T00:26:35.838876+0000","last_became_peered":"2024-09-21T00:26:35.838876+0000","last_unstale":"2024-09-21T00:26:35.845045+0000","last_undegraded":"2024-09-21T00:26:35.845045+0000","last_fullsized":"2024-09-21T00:26:35.845045+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:02:08.223701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":136,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-21T00:26:35.831437+0000","last_change":"2024-09-21T00:25:38.725792+0000","last_active":"2024-09-21T00:26:35.831437+0000","last_peered":"2024-09-21T00:26:35.831437+0000","last_clean":"2024-09-21T00:26:35.831437+0000","last_became_active":"2024-09-21T00:25:38.724668+0000","last_became_peered":"2024-09-21T00:25:38.724668+0000","last_unstale":"2024-09-21T00:26:35.831437+0000","last_undegraded":"2024-09-21T00:26:35.831437+0000","last_fullsized":"2024-09-21T00:26:35.831437+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:37.163216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:37.163216+0000","last_clean_scrub_stamp":"2024-09-21T00:25:37.163216+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:14:36.588660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509455,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28008,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343832,"statfs":{"total":96636764160,"available":96608083968,"internally_reserved":0,"allocated":999424,"data_stored":758396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509456,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":393216,"data_stored":161361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509455,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27428,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":405504,"data_stored":168028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509454,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":393216,"data_stored":161361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509456,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":393216,"data_stored":161361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509456,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":393216,"data_stored":161361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509456,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27992,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343848,"statfs":{"total":96636764160,"available":96608100352,"internally_reserved":0,"allocated":987136,"data_stored":751729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574864,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27440,"kb_used_data":412,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":421888,"data_stored":175285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T00:26:44.724 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T00:26:44.727 DEBUG:teuthology.orchestra.run.smithi187:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T00:26:45.312 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T00:26:45.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:45.436+0000 7f92913c3640 1 -- 172.21.15.112:0/4287866580 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f928c0b4500 msgr2=0x7f928c073d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:45.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:45.436+0000 7f92913c3640 1 --2- 172.21.15.112:0/4287866580 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f928c0b4500 0x7f928c073d50 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f928c0b3af0 tx=0x7f928402aa30 comp rx=0 tx=0).stop 2024-09-21T00:26:45.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:45.436+0000 7f92913c3640 1 -- 172.21.15.112:0/4287866580 shutdown_connections 2024-09-21T00:26:45.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:45.436+0000 7f92913c3640 1 --2- 172.21.15.112:0/4287866580 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f928c0b4500 0x7f928c073d50 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:45.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:45.436+0000 7f92913c3640 1 --2- 172.21.15.112:0/4287866580 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f928c06ec30 0x7f928c0b3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:45.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:45.436+0000 7f92913c3640 1 --2- 172.21.15.112:0/4287866580 >> v2:172.21.15.187:3300/0 conn(0x7f928c0d0b40 0x7f928c06e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:45.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:45.436+0000 7f92913c3640 1 -- 172.21.15.112:0/4287866580 >> 172.21.15.112:0/4287866580 conn(0x7f928c0b2cc0 msgr2=0x7f928c0b30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:45.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:45.436+0000 7f92913c3640 1 -- 172.21.15.112:0/4287866580 shutdown_connections 2024-09-21T00:26:45.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:45.436+0000 7f92913c3640 1 -- 172.21.15.112:0/4287866580 wait complete. 2024-09-21T00:26:45.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:45.438+0000 7f92913c3640 1 Processor -- start 2024-09-21T00:26:45.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:45.438+0000 7f92913c3640 1 -- start start 2024-09-21T00:26:45.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:45.441+0000 7f92913c3640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f928c06ec30 0x7f928c14bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:45.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:45.441+0000 7f92913c3640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f928c0b4500 0x7f928c1f6a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:45.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:45.441+0000 7f92913c3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f928c0d0b40 0x7f928c1f6f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:45.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:45.441+0000 7f92913c3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f928c15b5d0 con 0x7f928c0d0b40 2024-09-21T00:26:45.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:45.441+0000 7f92913c3640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f928c074610 con 0x7f928c06ec30 2024-09-21T00:26:45.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:45.441+0000 7f92913c3640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f928c15b750 con 0x7f928c0b4500 2024-09-21T00:26:45.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:45.441+0000 7f928b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f928c0d0b40 0x7f928c1f6f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:45.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:45.441+0000 7f928b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f928c0d0b40 0x7f928c1f6f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36872/0 (socket says 172.21.15.112:36872) 2024-09-21T00:26:45.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:45.441+0000 7f928b7fe640 1 -- 172.21.15.112:0/1428876876 learned_addr learned my addr 172.21.15.112:0/1428876876 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:26:45.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:45.441+0000 7f928affd640 1 --2- 172.21.15.112:0/1428876876 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f928c06ec30 0x7f928c14bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:45.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:45.441+0000 7f92827fc640 1 --2- 172.21.15.112:0/1428876876 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f928c0b4500 0x7f928c1f6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:45.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:45.441+0000 7f928b7fe640 1 -- 172.21.15.112:0/1428876876 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f928c0b4500 msgr2=0x7f928c1f6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:45.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:45.441+0000 7f928b7fe640 1 --2- 172.21.15.112:0/1428876876 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f928c0b4500 0x7f928c1f6a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:45.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:45.441+0000 7f928b7fe640 1 -- 172.21.15.112:0/1428876876 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f928c06ec30 msgr2=0x7f928c14bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:45.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:45.441+0000 7f928b7fe640 1 --2- 172.21.15.112:0/1428876876 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f928c06ec30 0x7f928c14bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:45.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:45.441+0000 7f928b7fe640 1 -- 172.21.15.112:0/1428876876 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f928c14c120 con 0x7f928c0d0b40 2024-09-21T00:26:45.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:45.441+0000 7f928b7fe640 1 --2- 172.21.15.112:0/1428876876 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f928c0d0b40 0x7f928c1f6f50 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f9284005fd0 tx=0x7f9284003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:45.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:45.441+0000 7f9283fff640 1 -- 172.21.15.112:0/1428876876 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9284033050 con 0x7f928c0d0b40 2024-09-21T00:26:45.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:45.441+0000 7f9283fff640 1 -- 172.21.15.112:0/1428876876 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9284004450 con 0x7f928c0d0b40 2024-09-21T00:26:45.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:45.441+0000 7f92913c3640 1 -- 172.21.15.112:0/1428876876 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f928c1f7670 con 0x7f928c0d0b40 2024-09-21T00:26:45.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:45.441+0000 7f9283fff640 1 -- 172.21.15.112:0/1428876876 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9284038450 con 0x7f928c0d0b40 2024-09-21T00:26:45.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:45.441+0000 7f92913c3640 1 -- 172.21.15.112:0/1428876876 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f928c0aa1f0 con 0x7f928c0d0b40 2024-09-21T00:26:45.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:45.441+0000 7f92913c3640 1 -- 172.21.15.112:0/1428876876 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f928c0750c0 con 0x7f928c0d0b40 2024-09-21T00:26:45.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:45.442+0000 7f9283fff640 1 -- 172.21.15.112:0/1428876876 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9284036070 con 0x7f928c0d0b40 2024-09-21T00:26:45.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:45.443+0000 7f9283fff640 1 --2- 172.21.15.112:0/1428876876 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f926c078b90 0x7f926c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:45.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:45.443+0000 7f9283fff640 1 -- 172.21.15.112:0/1428876876 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f92840bc050 con 0x7f928c0d0b40 2024-09-21T00:26:45.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:45.443+0000 7f928affd640 1 --2- 172.21.15.112:0/1428876876 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f926c078b90 0x7f926c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:45.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:45.443+0000 7f928affd640 1 --2- 172.21.15.112:0/1428876876 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f926c078b90 0x7f926c07b0a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f927c009ab0 tx=0x7f927c005f90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:45.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:45.448+0000 7f9283fff640 1 -- 172.21.15.112:0/1428876876 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f928c0750c0 con 0x7f928c0d0b40 2024-09-21T00:26:45.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:45.622+0000 7f92913c3640 1 -- 172.21.15.112:0/1428876876 --> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f928c15a340 con 0x7f926c078b90 2024-09-21T00:26:45.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:45.627+0000 7f9283fff640 1 -- 172.21.15.112:0/1428876876 <== mgr.4103 v2:172.21.15.187:6800/2921522906 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251936 (secure 0 0 0) 0x7f928c15a340 con 0x7f926c078b90 2024-09-21T00:26:45.631 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:26:45.632 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2024-09-21T00:26:45.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:45.629+0000 7f92913c3640 1 -- 172.21.15.112:0/1428876876 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f926c078b90 msgr2=0x7f926c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:45.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:45.629+0000 7f92913c3640 1 --2- 172.21.15.112:0/1428876876 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f926c078b90 0x7f926c07b0a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f927c009ab0 tx=0x7f927c005f90 comp rx=0 tx=0).stop 2024-09-21T00:26:45.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:45.629+0000 7f92913c3640 1 -- 172.21.15.112:0/1428876876 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f928c0d0b40 msgr2=0x7f928c1f6f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:45.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:45.629+0000 7f92913c3640 1 --2- 172.21.15.112:0/1428876876 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f928c0d0b40 0x7f928c1f6f50 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f9284005fd0 tx=0x7f9284003f90 comp rx=0 tx=0).stop 2024-09-21T00:26:45.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:45.630+0000 7f92913c3640 1 -- 172.21.15.112:0/1428876876 shutdown_connections 2024-09-21T00:26:45.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:45.630+0000 7f92913c3640 1 --2- 172.21.15.112:0/1428876876 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f926c078b90 0x7f926c07b0a0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:45.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:45.630+0000 7f92913c3640 1 --2- 172.21.15.112:0/1428876876 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f928c0d0b40 0x7f928c1f6f50 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:45.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:45.630+0000 7f92913c3640 1 --2- 172.21.15.112:0/1428876876 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f928c0b4500 0x7f928c1f6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:45.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:45.630+0000 7f92913c3640 1 --2- 172.21.15.112:0/1428876876 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f928c06ec30 0x7f928c14bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:45.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:45.630+0000 7f92913c3640 1 -- 172.21.15.112:0/1428876876 >> 172.21.15.112:0/1428876876 conn(0x7f928c0b2cc0 msgr2=0x7f928c06cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:45.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:45.630+0000 7f92913c3640 1 -- 172.21.15.112:0/1428876876 shutdown_connections 2024-09-21T00:26:45.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:45.630+0000 7f92913c3640 1 -- 172.21.15.112:0/1428876876 wait complete. 2024-09-21T00:26:45.643 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":67,"stamp":"2024-09-21T00:26:45.103007+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":187,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220812,"kb_used_data":4584,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753908,"statfs":{"total":773094113280,"available":772868001792,"internally_reserved":0,"allocated":4694016,"data_stored":2640652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.280528"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T00:26:38.901075+0000","last_change":"2024-09-21T00:26:36.843855+0000","last_active":"2024-09-21T00:26:38.901075+0000","last_peered":"2024-09-21T00:26:38.901075+0000","last_clean":"2024-09-21T00:26:38.901075+0000","last_became_active":"2024-09-21T00:26:36.843548+0000","last_became_peered":"2024-09-21T00:26:36.843548+0000","last_unstale":"2024-09-21T00:26:38.901075+0000","last_undegraded":"2024-09-21T00:26:38.901075+0000","last_fullsized":"2024-09-21T00:26:38.901075+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:11:01.088738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T00:26:38.900839+0000","last_change":"2024-09-21T00:26:36.833875+0000","last_active":"2024-09-21T00:26:38.900839+0000","last_peered":"2024-09-21T00:26:38.900839+0000","last_clean":"2024-09-21T00:26:38.900839+0000","last_became_active":"2024-09-21T00:26:36.833580+0000","last_became_peered":"2024-09-21T00:26:36.833580+0000","last_unstale":"2024-09-21T00:26:38.900839+0000","last_undegraded":"2024-09-21T00:26:38.900839+0000","last_fullsized":"2024-09-21T00:26:38.900839+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:02:58.597500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T00:26:38.214475+0000","last_change":"2024-09-21T00:26:36.844176+0000","last_active":"2024-09-21T00:26:38.214475+0000","last_peered":"2024-09-21T00:26:38.214475+0000","last_clean":"2024-09-21T00:26:38.214475+0000","last_became_active":"2024-09-21T00:26:36.843804+0000","last_became_peered":"2024-09-21T00:26:36.843804+0000","last_unstale":"2024-09-21T00:26:38.214475+0000","last_undegraded":"2024-09-21T00:26:38.214475+0000","last_fullsized":"2024-09-21T00:26:38.214475+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:58:01.448309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T00:26:37.844267+0000","last_change":"2024-09-21T00:26:36.838249+0000","last_active":"2024-09-21T00:26:37.844267+0000","last_peered":"2024-09-21T00:26:37.844267+0000","last_clean":"2024-09-21T00:26:37.844267+0000","last_became_active":"2024-09-21T00:26:36.837963+0000","last_became_peered":"2024-09-21T00:26:36.837963+0000","last_unstale":"2024-09-21T00:26:37.844267+0000","last_undegraded":"2024-09-21T00:26:37.844267+0000","last_fullsized":"2024-09-21T00:26:37.844267+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:55:55.071953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T00:26:37.844043+0000","last_change":"2024-09-21T00:26:36.837966+0000","last_active":"2024-09-21T00:26:37.844043+0000","last_peered":"2024-09-21T00:26:37.844043+0000","last_clean":"2024-09-21T00:26:37.844043+0000","last_became_active":"2024-09-21T00:26:36.837669+0000","last_became_peered":"2024-09-21T00:26:36.837669+0000","last_unstale":"2024-09-21T00:26:37.844043+0000","last_undegraded":"2024-09-21T00:26:37.844043+0000","last_fullsized":"2024-09-21T00:26:37.844043+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:01:58.935290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T00:26:38.937200+0000","last_change":"2024-09-21T00:26:36.841381+0000","last_active":"2024-09-21T00:26:38.937200+0000","last_peered":"2024-09-21T00:26:38.937200+0000","last_clean":"2024-09-21T00:26:38.937200+0000","last_became_active":"2024-09-21T00:26:36.841094+0000","last_became_peered":"2024-09-21T00:26:36.841094+0000","last_unstale":"2024-09-21T00:26:38.937200+0000","last_undegraded":"2024-09-21T00:26:38.937200+0000","last_fullsized":"2024-09-21T00:26:38.937200+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:14:35.306236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T00:26:38.938157+0000","last_change":"2024-09-21T00:26:36.851184+0000","last_active":"2024-09-21T00:26:38.938157+0000","last_peered":"2024-09-21T00:26:38.938157+0000","last_clean":"2024-09-21T00:26:38.938157+0000","last_became_active":"2024-09-21T00:26:36.850806+0000","last_became_peered":"2024-09-21T00:26:36.850806+0000","last_unstale":"2024-09-21T00:26:38.938157+0000","last_undegraded":"2024-09-21T00:26:38.938157+0000","last_fullsized":"2024-09-21T00:26:38.938157+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:53:43.474732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T00:26:38.900557+0000","last_change":"2024-09-21T00:26:36.837498+0000","last_active":"2024-09-21T00:26:38.900557+0000","last_peered":"2024-09-21T00:26:38.900557+0000","last_clean":"2024-09-21T00:26:38.900557+0000","last_became_active":"2024-09-21T00:26:36.837185+0000","last_became_peered":"2024-09-21T00:26:36.837185+0000","last_unstale":"2024-09-21T00:26:38.900557+0000","last_undegraded":"2024-09-21T00:26:38.900557+0000","last_fullsized":"2024-09-21T00:26:38.900557+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:35:47.775173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T00:26:38.214719+0000","last_change":"2024-09-21T00:26:36.849007+0000","last_active":"2024-09-21T00:26:38.214719+0000","last_peered":"2024-09-21T00:26:38.214719+0000","last_clean":"2024-09-21T00:26:38.214719+0000","last_became_active":"2024-09-21T00:26:36.848668+0000","last_became_peered":"2024-09-21T00:26:36.848668+0000","last_unstale":"2024-09-21T00:26:38.214719+0000","last_undegraded":"2024-09-21T00:26:38.214719+0000","last_fullsized":"2024-09-21T00:26:38.214719+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:08:08.195297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T00:26:38.938618+0000","last_change":"2024-09-21T00:26:36.847050+0000","last_active":"2024-09-21T00:26:38.938618+0000","last_peered":"2024-09-21T00:26:38.938618+0000","last_clean":"2024-09-21T00:26:38.938618+0000","last_became_active":"2024-09-21T00:26:36.846709+0000","last_became_peered":"2024-09-21T00:26:36.846709+0000","last_unstale":"2024-09-21T00:26:38.938618+0000","last_undegraded":"2024-09-21T00:26:38.938618+0000","last_fullsized":"2024-09-21T00:26:38.938618+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:47:02.792405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T00:26:38.938941+0000","last_change":"2024-09-21T00:26:36.839076+0000","last_active":"2024-09-21T00:26:38.938941+0000","last_peered":"2024-09-21T00:26:38.938941+0000","last_clean":"2024-09-21T00:26:38.938941+0000","last_became_active":"2024-09-21T00:26:36.838729+0000","last_became_peered":"2024-09-21T00:26:36.838729+0000","last_unstale":"2024-09-21T00:26:38.938941+0000","last_undegraded":"2024-09-21T00:26:38.938941+0000","last_fullsized":"2024-09-21T00:26:38.938941+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:55:00.792478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T00:26:38.214448+0000","last_change":"2024-09-21T00:26:36.844436+0000","last_active":"2024-09-21T00:26:38.214448+0000","last_peered":"2024-09-21T00:26:38.214448+0000","last_clean":"2024-09-21T00:26:38.214448+0000","last_became_active":"2024-09-21T00:26:36.844074+0000","last_became_peered":"2024-09-21T00:26:36.844074+0000","last_unstale":"2024-09-21T00:26:38.214448+0000","last_undegraded":"2024-09-21T00:26:38.214448+0000","last_fullsized":"2024-09-21T00:26:38.214448+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:51:25.834331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T00:26:38.939152+0000","last_change":"2024-09-21T00:26:36.847751+0000","last_active":"2024-09-21T00:26:38.939152+0000","last_peered":"2024-09-21T00:26:38.939152+0000","last_clean":"2024-09-21T00:26:38.939152+0000","last_became_active":"2024-09-21T00:26:36.847424+0000","last_became_peered":"2024-09-21T00:26:36.847424+0000","last_unstale":"2024-09-21T00:26:38.939152+0000","last_undegraded":"2024-09-21T00:26:38.939152+0000","last_fullsized":"2024-09-21T00:26:38.939152+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:11:01.088738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T00:26:37.841670+0000","last_change":"2024-09-21T00:26:36.846757+0000","last_active":"2024-09-21T00:26:37.841670+0000","last_peered":"2024-09-21T00:26:37.841670+0000","last_clean":"2024-09-21T00:26:37.841670+0000","last_became_active":"2024-09-21T00:26:36.846262+0000","last_became_peered":"2024-09-21T00:26:36.846262+0000","last_unstale":"2024-09-21T00:26:37.841670+0000","last_undegraded":"2024-09-21T00:26:37.841670+0000","last_fullsized":"2024-09-21T00:26:37.841670+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:28:02.701637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T00:26:38.938416+0000","last_change":"2024-09-21T00:26:36.839469+0000","last_active":"2024-09-21T00:26:38.938416+0000","last_peered":"2024-09-21T00:26:38.938416+0000","last_clean":"2024-09-21T00:26:38.938416+0000","last_became_active":"2024-09-21T00:26:36.839156+0000","last_became_peered":"2024-09-21T00:26:36.839156+0000","last_unstale":"2024-09-21T00:26:38.938416+0000","last_undegraded":"2024-09-21T00:26:38.938416+0000","last_fullsized":"2024-09-21T00:26:38.938416+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:57:25.274588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T00:26:38.937812+0000","last_change":"2024-09-21T00:26:36.849813+0000","last_active":"2024-09-21T00:26:38.937812+0000","last_peered":"2024-09-21T00:26:38.937812+0000","last_clean":"2024-09-21T00:26:38.937812+0000","last_became_active":"2024-09-21T00:26:36.849469+0000","last_became_peered":"2024-09-21T00:26:36.849469+0000","last_unstale":"2024-09-21T00:26:38.937812+0000","last_undegraded":"2024-09-21T00:26:38.937812+0000","last_fullsized":"2024-09-21T00:26:38.937812+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:49:58.147854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T00:26:38.939116+0000","last_change":"2024-09-21T00:26:36.853897+0000","last_active":"2024-09-21T00:26:38.939116+0000","last_peered":"2024-09-21T00:26:38.939116+0000","last_clean":"2024-09-21T00:26:38.939116+0000","last_became_active":"2024-09-21T00:26:36.853585+0000","last_became_peered":"2024-09-21T00:26:36.853585+0000","last_unstale":"2024-09-21T00:26:38.939116+0000","last_undegraded":"2024-09-21T00:26:38.939116+0000","last_fullsized":"2024-09-21T00:26:38.939116+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:20:30.580419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T00:26:37.842063+0000","last_change":"2024-09-21T00:26:36.855801+0000","last_active":"2024-09-21T00:26:37.842063+0000","last_peered":"2024-09-21T00:26:37.842063+0000","last_clean":"2024-09-21T00:26:37.842063+0000","last_became_active":"2024-09-21T00:26:36.855460+0000","last_became_peered":"2024-09-21T00:26:36.855460+0000","last_unstale":"2024-09-21T00:26:37.842063+0000","last_undegraded":"2024-09-21T00:26:37.842063+0000","last_fullsized":"2024-09-21T00:26:37.842063+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:15:12.241543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T00:26:37.844589+0000","last_change":"2024-09-21T00:26:36.837241+0000","last_active":"2024-09-21T00:26:37.844589+0000","last_peered":"2024-09-21T00:26:37.844589+0000","last_clean":"2024-09-21T00:26:37.844589+0000","last_became_active":"2024-09-21T00:26:36.836938+0000","last_became_peered":"2024-09-21T00:26:36.836938+0000","last_unstale":"2024-09-21T00:26:37.844589+0000","last_undegraded":"2024-09-21T00:26:37.844589+0000","last_fullsized":"2024-09-21T00:26:37.844589+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T00:41:23.122063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T00:26:37.841131+0000","last_change":"2024-09-21T00:26:36.854001+0000","last_active":"2024-09-21T00:26:37.841131+0000","last_peered":"2024-09-21T00:26:37.841131+0000","last_clean":"2024-09-21T00:26:37.841131+0000","last_became_active":"2024-09-21T00:26:36.853595+0000","last_became_peered":"2024-09-21T00:26:36.853595+0000","last_unstale":"2024-09-21T00:26:37.841131+0000","last_undegraded":"2024-09-21T00:26:37.841131+0000","last_fullsized":"2024-09-21T00:26:37.841131+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:16:54.958692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T00:26:38.938368+0000","last_change":"2024-09-21T00:26:36.854376+0000","last_active":"2024-09-21T00:26:38.938368+0000","last_peered":"2024-09-21T00:26:38.938368+0000","last_clean":"2024-09-21T00:26:38.938368+0000","last_became_active":"2024-09-21T00:26:36.852751+0000","last_became_peered":"2024-09-21T00:26:36.852751+0000","last_unstale":"2024-09-21T00:26:38.938368+0000","last_undegraded":"2024-09-21T00:26:38.938368+0000","last_fullsized":"2024-09-21T00:26:38.938368+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:35:47.775173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T00:26:37.842354+0000","last_change":"2024-09-21T00:26:36.843921+0000","last_active":"2024-09-21T00:26:37.842354+0000","last_peered":"2024-09-21T00:26:37.842354+0000","last_clean":"2024-09-21T00:26:37.842354+0000","last_became_active":"2024-09-21T00:26:36.843605+0000","last_became_peered":"2024-09-21T00:26:36.843605+0000","last_unstale":"2024-09-21T00:26:37.842354+0000","last_undegraded":"2024-09-21T00:26:37.842354+0000","last_fullsized":"2024-09-21T00:26:37.842354+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:16:22.159727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T00:26:38.937848+0000","last_change":"2024-09-21T00:26:36.842661+0000","last_active":"2024-09-21T00:26:38.937848+0000","last_peered":"2024-09-21T00:26:38.937848+0000","last_clean":"2024-09-21T00:26:38.937848+0000","last_became_active":"2024-09-21T00:26:36.842156+0000","last_became_peered":"2024-09-21T00:26:36.842156+0000","last_unstale":"2024-09-21T00:26:38.937848+0000","last_undegraded":"2024-09-21T00:26:38.937848+0000","last_fullsized":"2024-09-21T00:26:38.937848+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:52:53.387318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T00:26:37.843995+0000","last_change":"2024-09-21T00:26:36.840053+0000","last_active":"2024-09-21T00:26:37.843995+0000","last_peered":"2024-09-21T00:26:37.843995+0000","last_clean":"2024-09-21T00:26:37.843995+0000","last_became_active":"2024-09-21T00:26:36.839510+0000","last_became_peered":"2024-09-21T00:26:36.839510+0000","last_unstale":"2024-09-21T00:26:37.843995+0000","last_undegraded":"2024-09-21T00:26:37.843995+0000","last_fullsized":"2024-09-21T00:26:37.843995+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:55:33.711400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T00:26:37.841778+0000","last_change":"2024-09-21T00:26:36.845405+0000","last_active":"2024-09-21T00:26:37.841778+0000","last_peered":"2024-09-21T00:26:37.841778+0000","last_clean":"2024-09-21T00:26:37.841778+0000","last_became_active":"2024-09-21T00:26:36.845064+0000","last_became_peered":"2024-09-21T00:26:36.845064+0000","last_unstale":"2024-09-21T00:26:37.841778+0000","last_undegraded":"2024-09-21T00:26:37.841778+0000","last_fullsized":"2024-09-21T00:26:37.841778+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:58:37.004181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T00:26:38.937730+0000","last_change":"2024-09-21T00:26:36.856780+0000","last_active":"2024-09-21T00:26:38.937730+0000","last_peered":"2024-09-21T00:26:38.937730+0000","last_clean":"2024-09-21T00:26:38.937730+0000","last_became_active":"2024-09-21T00:26:36.856482+0000","last_became_peered":"2024-09-21T00:26:36.856482+0000","last_unstale":"2024-09-21T00:26:38.937730+0000","last_undegraded":"2024-09-21T00:26:38.937730+0000","last_fullsized":"2024-09-21T00:26:38.937730+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:38:18.623907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T00:26:37.842261+0000","last_change":"2024-09-21T00:26:36.844711+0000","last_active":"2024-09-21T00:26:37.842261+0000","last_peered":"2024-09-21T00:26:37.842261+0000","last_clean":"2024-09-21T00:26:37.842261+0000","last_became_active":"2024-09-21T00:26:36.844390+0000","last_became_peered":"2024-09-21T00:26:36.844390+0000","last_unstale":"2024-09-21T00:26:37.842261+0000","last_undegraded":"2024-09-21T00:26:37.842261+0000","last_fullsized":"2024-09-21T00:26:37.842261+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:36:47.818388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T00:26:38.938763+0000","last_change":"2024-09-21T00:26:36.852510+0000","last_active":"2024-09-21T00:26:38.938763+0000","last_peered":"2024-09-21T00:26:38.938763+0000","last_clean":"2024-09-21T00:26:38.938763+0000","last_became_active":"2024-09-21T00:26:36.848060+0000","last_became_peered":"2024-09-21T00:26:36.848060+0000","last_unstale":"2024-09-21T00:26:38.938763+0000","last_undegraded":"2024-09-21T00:26:38.938763+0000","last_fullsized":"2024-09-21T00:26:38.938763+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:01:43.398894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T00:26:37.841261+0000","last_change":"2024-09-21T00:26:36.841220+0000","last_active":"2024-09-21T00:26:37.841261+0000","last_peered":"2024-09-21T00:26:37.841261+0000","last_clean":"2024-09-21T00:26:37.841261+0000","last_became_active":"2024-09-21T00:26:36.840895+0000","last_became_peered":"2024-09-21T00:26:36.840895+0000","last_unstale":"2024-09-21T00:26:37.841261+0000","last_undegraded":"2024-09-21T00:26:37.841261+0000","last_fullsized":"2024-09-21T00:26:37.841261+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:55:25.224768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T00:26:37.841531+0000","last_change":"2024-09-21T00:26:36.841663+0000","last_active":"2024-09-21T00:26:37.841531+0000","last_peered":"2024-09-21T00:26:37.841531+0000","last_clean":"2024-09-21T00:26:37.841531+0000","last_became_active":"2024-09-21T00:26:36.841370+0000","last_became_peered":"2024-09-21T00:26:36.841370+0000","last_unstale":"2024-09-21T00:26:37.841531+0000","last_undegraded":"2024-09-21T00:26:37.841531+0000","last_fullsized":"2024-09-21T00:26:37.841531+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:50:12.134645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T00:26:38.937926+0000","last_change":"2024-09-21T00:26:36.843303+0000","last_active":"2024-09-21T00:26:38.937926+0000","last_peered":"2024-09-21T00:26:38.937926+0000","last_clean":"2024-09-21T00:26:38.937926+0000","last_became_active":"2024-09-21T00:26:36.842517+0000","last_became_peered":"2024-09-21T00:26:36.842517+0000","last_unstale":"2024-09-21T00:26:38.937926+0000","last_undegraded":"2024-09-21T00:26:38.937926+0000","last_fullsized":"2024-09-21T00:26:38.937926+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:56:38.653789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T00:26:38.937249+0000","last_change":"2024-09-21T00:26:36.836602+0000","last_active":"2024-09-21T00:26:38.937249+0000","last_peered":"2024-09-21T00:26:38.937249+0000","last_clean":"2024-09-21T00:26:38.937249+0000","last_became_active":"2024-09-21T00:26:36.835713+0000","last_became_peered":"2024-09-21T00:26:36.835713+0000","last_unstale":"2024-09-21T00:26:38.937249+0000","last_undegraded":"2024-09-21T00:26:38.937249+0000","last_fullsized":"2024-09-21T00:26:38.937249+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:43:21.485371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T00:26:38.938585+0000","last_change":"2024-09-21T00:26:36.853706+0000","last_active":"2024-09-21T00:26:38.938585+0000","last_peered":"2024-09-21T00:26:38.938585+0000","last_clean":"2024-09-21T00:26:38.938585+0000","last_became_active":"2024-09-21T00:26:36.853323+0000","last_became_peered":"2024-09-21T00:26:36.853323+0000","last_unstale":"2024-09-21T00:26:38.938585+0000","last_undegraded":"2024-09-21T00:26:38.938585+0000","last_fullsized":"2024-09-21T00:26:38.938585+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:35:50.889880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"37'4","reported_seq":19,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T00:26:37.841947+0000","last_change":"2024-09-21T00:26:35.838321+0000","last_active":"2024-09-21T00:26:37.841947+0000","last_peered":"2024-09-21T00:26:37.841947+0000","last_clean":"2024-09-21T00:26:37.841947+0000","last_became_active":"2024-09-21T00:26:35.838022+0000","last_became_peered":"2024-09-21T00:26:35.838022+0000","last_unstale":"2024-09-21T00:26:37.841947+0000","last_undegraded":"2024-09-21T00:26:37.841947+0000","last_fullsized":"2024-09-21T00:26:37.841947+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:07:42.890061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T00:26:38.937659+0000","last_change":"2024-09-21T00:26:36.856958+0000","last_active":"2024-09-21T00:26:38.937659+0000","last_peered":"2024-09-21T00:26:38.937659+0000","last_clean":"2024-09-21T00:26:38.937659+0000","last_became_active":"2024-09-21T00:26:36.856395+0000","last_became_peered":"2024-09-21T00:26:36.856395+0000","last_unstale":"2024-09-21T00:26:38.937659+0000","last_undegraded":"2024-09-21T00:26:38.937659+0000","last_fullsized":"2024-09-21T00:26:38.937659+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:17:03.455104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"37'1","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T00:26:38.214566+0000","last_change":"2024-09-21T00:26:35.837664+0000","last_active":"2024-09-21T00:26:38.214566+0000","last_peered":"2024-09-21T00:26:38.214566+0000","last_clean":"2024-09-21T00:26:38.214566+0000","last_became_active":"2024-09-21T00:26:35.837355+0000","last_became_peered":"2024-09-21T00:26:35.837355+0000","last_unstale":"2024-09-21T00:26:38.214566+0000","last_undegraded":"2024-09-21T00:26:38.214566+0000","last_fullsized":"2024-09-21T00:26:38.214566+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:01:08.134613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T00:26:37.842138+0000","last_change":"2024-09-21T00:26:36.850969+0000","last_active":"2024-09-21T00:26:37.842138+0000","last_peered":"2024-09-21T00:26:37.842138+0000","last_clean":"2024-09-21T00:26:37.842138+0000","last_became_active":"2024-09-21T00:26:36.850649+0000","last_became_peered":"2024-09-21T00:26:36.850649+0000","last_unstale":"2024-09-21T00:26:37.842138+0000","last_undegraded":"2024-09-21T00:26:37.842138+0000","last_fullsized":"2024-09-21T00:26:37.842138+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:10:20.588029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"37'2","reported_seq":17,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T00:26:38.937777+0000","last_change":"2024-09-21T00:26:35.840905+0000","last_active":"2024-09-21T00:26:38.937777+0000","last_peered":"2024-09-21T00:26:38.937777+0000","last_clean":"2024-09-21T00:26:38.937777+0000","last_became_active":"2024-09-21T00:26:35.840594+0000","last_became_peered":"2024-09-21T00:26:35.840594+0000","last_unstale":"2024-09-21T00:26:38.937777+0000","last_undegraded":"2024-09-21T00:26:38.937777+0000","last_fullsized":"2024-09-21T00:26:38.937777+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:54:48.838959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T00:26:38.900966+0000","last_change":"2024-09-21T00:26:36.849207+0000","last_active":"2024-09-21T00:26:38.900966+0000","last_peered":"2024-09-21T00:26:38.900966+0000","last_clean":"2024-09-21T00:26:38.900966+0000","last_became_active":"2024-09-21T00:26:36.848880+0000","last_became_peered":"2024-09-21T00:26:36.848880+0000","last_unstale":"2024-09-21T00:26:38.900966+0000","last_undegraded":"2024-09-21T00:26:38.900966+0000","last_fullsized":"2024-09-21T00:26:38.900966+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:51:29.747215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"37'2","reported_seq":17,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T00:26:37.841462+0000","last_change":"2024-09-21T00:26:35.836092+0000","last_active":"2024-09-21T00:26:37.841462+0000","last_peered":"2024-09-21T00:26:37.841462+0000","last_clean":"2024-09-21T00:26:37.841462+0000","last_became_active":"2024-09-21T00:26:35.835424+0000","last_became_peered":"2024-09-21T00:26:35.835424+0000","last_unstale":"2024-09-21T00:26:37.841462+0000","last_undegraded":"2024-09-21T00:26:37.841462+0000","last_fullsized":"2024-09-21T00:26:37.841462+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:45:54.889994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T00:26:38.214393+0000","last_change":"2024-09-21T00:26:36.843668+0000","last_active":"2024-09-21T00:26:38.214393+0000","last_peered":"2024-09-21T00:26:38.214393+0000","last_clean":"2024-09-21T00:26:38.214393+0000","last_became_active":"2024-09-21T00:26:36.843094+0000","last_became_peered":"2024-09-21T00:26:36.843094+0000","last_unstale":"2024-09-21T00:26:38.214393+0000","last_undegraded":"2024-09-21T00:26:38.214393+0000","last_fullsized":"2024-09-21T00:26:38.214393+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:48:24.357715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"37'1","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T00:26:38.939386+0000","last_change":"2024-09-21T00:26:35.835778+0000","last_active":"2024-09-21T00:26:38.939386+0000","last_peered":"2024-09-21T00:26:38.939386+0000","last_clean":"2024-09-21T00:26:38.939386+0000","last_became_active":"2024-09-21T00:26:35.835426+0000","last_became_peered":"2024-09-21T00:26:35.835426+0000","last_unstale":"2024-09-21T00:26:38.939386+0000","last_undegraded":"2024-09-21T00:26:38.939386+0000","last_fullsized":"2024-09-21T00:26:38.939386+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:57:20.956431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T00:26:38.938522+0000","last_change":"2024-09-21T00:26:36.852092+0000","last_active":"2024-09-21T00:26:38.938522+0000","last_peered":"2024-09-21T00:26:38.938522+0000","last_clean":"2024-09-21T00:26:38.938522+0000","last_became_active":"2024-09-21T00:26:36.851782+0000","last_became_peered":"2024-09-21T00:26:36.851782+0000","last_unstale":"2024-09-21T00:26:38.938522+0000","last_undegraded":"2024-09-21T00:26:38.938522+0000","last_fullsized":"2024-09-21T00:26:38.938522+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:38:14.768235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"37'1","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T00:26:38.938115+0000","last_change":"2024-09-21T00:26:35.846072+0000","last_active":"2024-09-21T00:26:38.938115+0000","last_peered":"2024-09-21T00:26:38.938115+0000","last_clean":"2024-09-21T00:26:38.938115+0000","last_became_active":"2024-09-21T00:26:35.845724+0000","last_became_peered":"2024-09-21T00:26:35.845724+0000","last_unstale":"2024-09-21T00:26:38.938115+0000","last_undegraded":"2024-09-21T00:26:38.938115+0000","last_fullsized":"2024-09-21T00:26:38.938115+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:15:04.026410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T00:26:37.841995+0000","last_change":"2024-09-21T00:26:36.839740+0000","last_active":"2024-09-21T00:26:37.841995+0000","last_peered":"2024-09-21T00:26:37.841995+0000","last_clean":"2024-09-21T00:26:37.841995+0000","last_became_active":"2024-09-21T00:26:36.839421+0000","last_became_peered":"2024-09-21T00:26:36.839421+0000","last_unstale":"2024-09-21T00:26:37.841995+0000","last_undegraded":"2024-09-21T00:26:37.841995+0000","last_fullsized":"2024-09-21T00:26:37.841995+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:33:26.630805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T00:26:38.937163+0000","last_change":"2024-09-21T00:26:35.842674+0000","last_active":"2024-09-21T00:26:38.937163+0000","last_peered":"2024-09-21T00:26:38.937163+0000","last_clean":"2024-09-21T00:26:38.937163+0000","last_became_active":"2024-09-21T00:26:35.842339+0000","last_became_peered":"2024-09-21T00:26:35.842339+0000","last_unstale":"2024-09-21T00:26:38.937163+0000","last_undegraded":"2024-09-21T00:26:38.937163+0000","last_fullsized":"2024-09-21T00:26:38.937163+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:56:47.623787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T00:26:38.900680+0000","last_change":"2024-09-21T00:26:36.842297+0000","last_active":"2024-09-21T00:26:38.900680+0000","last_peered":"2024-09-21T00:26:38.900680+0000","last_clean":"2024-09-21T00:26:38.900680+0000","last_became_active":"2024-09-21T00:26:36.841576+0000","last_became_peered":"2024-09-21T00:26:36.841576+0000","last_unstale":"2024-09-21T00:26:38.900680+0000","last_undegraded":"2024-09-21T00:26:38.900680+0000","last_fullsized":"2024-09-21T00:26:38.900680+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:41:23.289334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T00:26:37.841087+0000","last_change":"2024-09-21T00:26:35.835252+0000","last_active":"2024-09-21T00:26:37.841087+0000","last_peered":"2024-09-21T00:26:37.841087+0000","last_clean":"2024-09-21T00:26:37.841087+0000","last_became_active":"2024-09-21T00:26:35.834768+0000","last_became_peered":"2024-09-21T00:26:35.834768+0000","last_unstale":"2024-09-21T00:26:37.841087+0000","last_undegraded":"2024-09-21T00:26:37.841087+0000","last_fullsized":"2024-09-21T00:26:37.841087+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T00:41:22.116310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T00:26:37.844437+0000","last_change":"2024-09-21T00:26:36.835034+0000","last_active":"2024-09-21T00:26:37.844437+0000","last_peered":"2024-09-21T00:26:37.844437+0000","last_clean":"2024-09-21T00:26:37.844437+0000","last_became_active":"2024-09-21T00:26:36.834752+0000","last_became_peered":"2024-09-21T00:26:36.834752+0000","last_unstale":"2024-09-21T00:26:37.844437+0000","last_undegraded":"2024-09-21T00:26:37.844437+0000","last_fullsized":"2024-09-21T00:26:37.844437+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:11:29.663546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T00:26:38.901118+0000","last_change":"2024-09-21T00:26:35.834813+0000","last_active":"2024-09-21T00:26:38.901118+0000","last_peered":"2024-09-21T00:26:38.901118+0000","last_clean":"2024-09-21T00:26:38.901118+0000","last_became_active":"2024-09-21T00:26:35.834218+0000","last_became_peered":"2024-09-21T00:26:35.834218+0000","last_unstale":"2024-09-21T00:26:38.901118+0000","last_undegraded":"2024-09-21T00:26:38.901118+0000","last_fullsized":"2024-09-21T00:26:38.901118+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:26:28.101505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T00:26:38.939446+0000","last_change":"2024-09-21T00:26:36.850515+0000","last_active":"2024-09-21T00:26:38.939446+0000","last_peered":"2024-09-21T00:26:38.939446+0000","last_clean":"2024-09-21T00:26:38.939446+0000","last_became_active":"2024-09-21T00:26:36.850217+0000","last_became_peered":"2024-09-21T00:26:36.850217+0000","last_unstale":"2024-09-21T00:26:38.939446+0000","last_undegraded":"2024-09-21T00:26:38.939446+0000","last_fullsized":"2024-09-21T00:26:38.939446+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:06:33.925951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T00:26:38.939079+0000","last_change":"2024-09-21T00:26:35.841364+0000","last_active":"2024-09-21T00:26:38.939079+0000","last_peered":"2024-09-21T00:26:38.939079+0000","last_clean":"2024-09-21T00:26:38.939079+0000","last_became_active":"2024-09-21T00:26:35.840989+0000","last_became_peered":"2024-09-21T00:26:35.840989+0000","last_unstale":"2024-09-21T00:26:38.939079+0000","last_undegraded":"2024-09-21T00:26:38.939079+0000","last_fullsized":"2024-09-21T00:26:38.939079+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:48:23.351963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T00:26:38.214653+0000","last_change":"2024-09-21T00:26:36.855723+0000","last_active":"2024-09-21T00:26:38.214653+0000","last_peered":"2024-09-21T00:26:38.214653+0000","last_clean":"2024-09-21T00:26:38.214653+0000","last_became_active":"2024-09-21T00:26:36.855400+0000","last_became_peered":"2024-09-21T00:26:36.855400+0000","last_unstale":"2024-09-21T00:26:38.214653+0000","last_undegraded":"2024-09-21T00:26:38.214653+0000","last_fullsized":"2024-09-21T00:26:38.214653+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:30:14.192930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"37'3","reported_seq":18,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T00:26:38.939220+0000","last_change":"2024-09-21T00:26:35.836960+0000","last_active":"2024-09-21T00:26:38.939220+0000","last_peered":"2024-09-21T00:26:38.939220+0000","last_clean":"2024-09-21T00:26:38.939220+0000","last_became_active":"2024-09-21T00:26:35.836641+0000","last_became_peered":"2024-09-21T00:26:35.836641+0000","last_unstale":"2024-09-21T00:26:38.939220+0000","last_undegraded":"2024-09-21T00:26:38.939220+0000","last_fullsized":"2024-09-21T00:26:38.939220+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:20:00.827514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T00:26:37.841443+0000","last_change":"2024-09-21T00:26:36.842783+0000","last_active":"2024-09-21T00:26:37.841443+0000","last_peered":"2024-09-21T00:26:37.841443+0000","last_clean":"2024-09-21T00:26:37.841443+0000","last_became_active":"2024-09-21T00:26:36.842441+0000","last_became_peered":"2024-09-21T00:26:36.842441+0000","last_unstale":"2024-09-21T00:26:37.841443+0000","last_undegraded":"2024-09-21T00:26:37.841443+0000","last_fullsized":"2024-09-21T00:26:37.841443+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:44:28.396278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"37'6","reported_seq":21,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T00:26:38.900851+0000","last_change":"2024-09-21T00:26:35.834520+0000","last_active":"2024-09-21T00:26:38.900851+0000","last_peered":"2024-09-21T00:26:38.900851+0000","last_clean":"2024-09-21T00:26:38.900851+0000","last_became_active":"2024-09-21T00:26:35.834146+0000","last_became_peered":"2024-09-21T00:26:35.834146+0000","last_unstale":"2024-09-21T00:26:38.900851+0000","last_undegraded":"2024-09-21T00:26:38.900851+0000","last_fullsized":"2024-09-21T00:26:38.900851+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:03:29.499842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T00:26:38.214586+0000","last_change":"2024-09-21T00:26:36.847551+0000","last_active":"2024-09-21T00:26:38.214586+0000","last_peered":"2024-09-21T00:26:38.214586+0000","last_clean":"2024-09-21T00:26:38.214586+0000","last_became_active":"2024-09-21T00:26:36.846339+0000","last_became_peered":"2024-09-21T00:26:36.846339+0000","last_unstale":"2024-09-21T00:26:38.214586+0000","last_undegraded":"2024-09-21T00:26:38.214586+0000","last_fullsized":"2024-09-21T00:26:38.214586+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T00:29:26.174453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"37'2","reported_seq":17,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T00:26:38.901154+0000","last_change":"2024-09-21T00:26:35.840502+0000","last_active":"2024-09-21T00:26:38.901154+0000","last_peered":"2024-09-21T00:26:38.901154+0000","last_clean":"2024-09-21T00:26:38.901154+0000","last_became_active":"2024-09-21T00:26:35.840189+0000","last_became_peered":"2024-09-21T00:26:35.840189+0000","last_unstale":"2024-09-21T00:26:38.901154+0000","last_undegraded":"2024-09-21T00:26:38.901154+0000","last_fullsized":"2024-09-21T00:26:38.901154+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:33:25.625052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T00:26:38.900565+0000","last_change":"2024-09-21T00:26:36.846354+0000","last_active":"2024-09-21T00:26:38.900565+0000","last_peered":"2024-09-21T00:26:38.900565+0000","last_clean":"2024-09-21T00:26:38.900565+0000","last_became_active":"2024-09-21T00:26:36.845933+0000","last_became_peered":"2024-09-21T00:26:36.845933+0000","last_unstale":"2024-09-21T00:26:38.900565+0000","last_undegraded":"2024-09-21T00:26:38.900565+0000","last_fullsized":"2024-09-21T00:26:38.900565+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:57:25.274588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T00:26:38.937364+0000","last_change":"2024-09-21T00:26:35.842688+0000","last_active":"2024-09-21T00:26:38.937364+0000","last_peered":"2024-09-21T00:26:38.937364+0000","last_clean":"2024-09-21T00:26:38.937364+0000","last_became_active":"2024-09-21T00:26:35.842375+0000","last_became_peered":"2024-09-21T00:26:35.842375+0000","last_unstale":"2024-09-21T00:26:38.937364+0000","last_undegraded":"2024-09-21T00:26:38.937364+0000","last_fullsized":"2024-09-21T00:26:38.937364+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:58:30.617062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T00:26:38.937900+0000","last_change":"2024-09-21T00:26:36.842600+0000","last_active":"2024-09-21T00:26:38.937900+0000","last_peered":"2024-09-21T00:26:38.937900+0000","last_clean":"2024-09-21T00:26:38.937900+0000","last_became_active":"2024-09-21T00:26:36.842314+0000","last_became_peered":"2024-09-21T00:26:36.842314+0000","last_unstale":"2024-09-21T00:26:38.937900+0000","last_undegraded":"2024-09-21T00:26:38.937900+0000","last_fullsized":"2024-09-21T00:26:38.937900+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:47:09.496573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T00:26:37.842046+0000","last_change":"2024-09-21T00:26:35.839140+0000","last_active":"2024-09-21T00:26:37.842046+0000","last_peered":"2024-09-21T00:26:37.842046+0000","last_clean":"2024-09-21T00:26:37.842046+0000","last_became_active":"2024-09-21T00:26:35.838817+0000","last_became_peered":"2024-09-21T00:26:35.838817+0000","last_unstale":"2024-09-21T00:26:37.842046+0000","last_undegraded":"2024-09-21T00:26:37.842046+0000","last_fullsized":"2024-09-21T00:26:37.842046+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:04:22.293481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T00:26:38.900782+0000","last_change":"2024-09-21T00:26:36.845664+0000","last_active":"2024-09-21T00:26:38.900782+0000","last_peered":"2024-09-21T00:26:38.900782+0000","last_clean":"2024-09-21T00:26:38.900782+0000","last_became_active":"2024-09-21T00:26:36.845335+0000","last_became_peered":"2024-09-21T00:26:36.845335+0000","last_unstale":"2024-09-21T00:26:38.900782+0000","last_undegraded":"2024-09-21T00:26:38.900782+0000","last_fullsized":"2024-09-21T00:26:38.900782+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:13:24.745330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"37'8","reported_seq":23,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T00:26:38.937830+0000","last_change":"2024-09-21T00:26:35.846027+0000","last_active":"2024-09-21T00:26:38.937830+0000","last_peered":"2024-09-21T00:26:38.937830+0000","last_clean":"2024-09-21T00:26:38.937830+0000","last_became_active":"2024-09-21T00:26:35.845681+0000","last_became_peered":"2024-09-21T00:26:35.845681+0000","last_unstale":"2024-09-21T00:26:38.937830+0000","last_undegraded":"2024-09-21T00:26:38.937830+0000","last_fullsized":"2024-09-21T00:26:38.937830+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:05:50.542466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T00:26:38.900760+0000","last_change":"2024-09-21T00:26:36.849360+0000","last_active":"2024-09-21T00:26:38.900760+0000","last_peered":"2024-09-21T00:26:38.900760+0000","last_clean":"2024-09-21T00:26:38.900760+0000","last_became_active":"2024-09-21T00:26:36.849013+0000","last_became_peered":"2024-09-21T00:26:36.849013+0000","last_unstale":"2024-09-21T00:26:38.900760+0000","last_undegraded":"2024-09-21T00:26:38.900760+0000","last_fullsized":"2024-09-21T00:26:38.900760+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:47:02.792405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"37'1","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T00:26:38.939250+0000","last_change":"2024-09-21T00:26:35.845969+0000","last_active":"2024-09-21T00:26:38.939250+0000","last_peered":"2024-09-21T00:26:38.939250+0000","last_clean":"2024-09-21T00:26:38.939250+0000","last_became_active":"2024-09-21T00:26:35.845633+0000","last_became_peered":"2024-09-21T00:26:35.845633+0000","last_unstale":"2024-09-21T00:26:38.939250+0000","last_undegraded":"2024-09-21T00:26:38.939250+0000","last_fullsized":"2024-09-21T00:26:38.939250+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:41:13.666350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T00:26:38.939032+0000","last_change":"2024-09-21T00:26:36.853027+0000","last_active":"2024-09-21T00:26:38.939032+0000","last_peered":"2024-09-21T00:26:38.939032+0000","last_clean":"2024-09-21T00:26:38.939032+0000","last_became_active":"2024-09-21T00:26:36.852678+0000","last_became_peered":"2024-09-21T00:26:36.852678+0000","last_unstale":"2024-09-21T00:26:38.939032+0000","last_undegraded":"2024-09-21T00:26:38.939032+0000","last_fullsized":"2024-09-21T00:26:38.939032+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:41:23.289334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"37'2","reported_seq":17,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T00:26:37.842158+0000","last_change":"2024-09-21T00:26:35.840529+0000","last_active":"2024-09-21T00:26:37.842158+0000","last_peered":"2024-09-21T00:26:37.842158+0000","last_clean":"2024-09-21T00:26:37.842158+0000","last_became_active":"2024-09-21T00:26:35.840115+0000","last_became_peered":"2024-09-21T00:26:35.840115+0000","last_unstale":"2024-09-21T00:26:37.842158+0000","last_undegraded":"2024-09-21T00:26:37.842158+0000","last_fullsized":"2024-09-21T00:26:37.842158+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:14:15.665953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T00:26:37.841971+0000","last_change":"2024-09-21T00:26:36.847056+0000","last_active":"2024-09-21T00:26:37.841971+0000","last_peered":"2024-09-21T00:26:37.841971+0000","last_clean":"2024-09-21T00:26:37.841971+0000","last_became_active":"2024-09-21T00:26:36.845928+0000","last_became_peered":"2024-09-21T00:26:36.845928+0000","last_unstale":"2024-09-21T00:26:37.841971+0000","last_undegraded":"2024-09-21T00:26:37.841971+0000","last_fullsized":"2024-09-21T00:26:37.841971+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:26:29.107258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"37'1","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T00:26:37.844221+0000","last_change":"2024-09-21T00:26:35.835450+0000","last_active":"2024-09-21T00:26:37.844221+0000","last_peered":"2024-09-21T00:26:37.844221+0000","last_clean":"2024-09-21T00:26:37.844221+0000","last_became_active":"2024-09-21T00:26:35.835141+0000","last_became_peered":"2024-09-21T00:26:35.835141+0000","last_unstale":"2024-09-21T00:26:37.844221+0000","last_undegraded":"2024-09-21T00:26:37.844221+0000","last_fullsized":"2024-09-21T00:26:37.844221+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:14:27.363401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T00:26:37.844259+0000","last_change":"2024-09-21T00:26:36.832064+0000","last_active":"2024-09-21T00:26:37.844259+0000","last_peered":"2024-09-21T00:26:37.844259+0000","last_clean":"2024-09-21T00:26:37.844259+0000","last_became_active":"2024-09-21T00:26:36.831715+0000","last_became_peered":"2024-09-21T00:26:36.831715+0000","last_unstale":"2024-09-21T00:26:37.844259+0000","last_undegraded":"2024-09-21T00:26:37.844259+0000","last_fullsized":"2024-09-21T00:26:37.844259+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:02:09.248415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T00:26:38.937482+0000","last_change":"2024-09-21T00:26:35.841410+0000","last_active":"2024-09-21T00:26:38.937482+0000","last_peered":"2024-09-21T00:26:38.937482+0000","last_clean":"2024-09-21T00:26:38.937482+0000","last_became_active":"2024-09-21T00:26:35.841094+0000","last_became_peered":"2024-09-21T00:26:35.841094+0000","last_unstale":"2024-09-21T00:26:38.937482+0000","last_undegraded":"2024-09-21T00:26:38.937482+0000","last_fullsized":"2024-09-21T00:26:38.937482+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:48:22.317860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T00:26:38.938089+0000","last_change":"2024-09-21T00:26:36.845717+0000","last_active":"2024-09-21T00:26:38.938089+0000","last_peered":"2024-09-21T00:26:38.938089+0000","last_clean":"2024-09-21T00:26:38.938089+0000","last_became_active":"2024-09-21T00:26:36.845239+0000","last_became_peered":"2024-09-21T00:26:36.845239+0000","last_unstale":"2024-09-21T00:26:38.938089+0000","last_undegraded":"2024-09-21T00:26:38.938089+0000","last_fullsized":"2024-09-21T00:26:38.938089+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:26:35.982930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"37'2","reported_seq":17,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T00:26:37.844221+0000","last_change":"2024-09-21T00:26:35.836023+0000","last_active":"2024-09-21T00:26:37.844221+0000","last_peered":"2024-09-21T00:26:37.844221+0000","last_clean":"2024-09-21T00:26:37.844221+0000","last_became_active":"2024-09-21T00:26:35.835721+0000","last_became_peered":"2024-09-21T00:26:35.835721+0000","last_unstale":"2024-09-21T00:26:37.844221+0000","last_undegraded":"2024-09-21T00:26:37.844221+0000","last_fullsized":"2024-09-21T00:26:37.844221+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T00:46:35.942269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T00:26:38.214404+0000","last_change":"2024-09-21T00:26:36.850817+0000","last_active":"2024-09-21T00:26:38.214404+0000","last_peered":"2024-09-21T00:26:38.214404+0000","last_clean":"2024-09-21T00:26:38.214404+0000","last_became_active":"2024-09-21T00:26:36.850485+0000","last_became_peered":"2024-09-21T00:26:36.850485+0000","last_unstale":"2024-09-21T00:26:38.214404+0000","last_undegraded":"2024-09-21T00:26:38.214404+0000","last_fullsized":"2024-09-21T00:26:38.214404+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T00:41:16.417895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T00:26:37.844245+0000","last_change":"2024-09-21T00:26:35.837224+0000","last_active":"2024-09-21T00:26:37.844245+0000","last_peered":"2024-09-21T00:26:37.844245+0000","last_clean":"2024-09-21T00:26:37.844245+0000","last_became_active":"2024-09-21T00:26:35.836849+0000","last_became_peered":"2024-09-21T00:26:35.836849+0000","last_unstale":"2024-09-21T00:26:37.844245+0000","last_undegraded":"2024-09-21T00:26:37.844245+0000","last_fullsized":"2024-09-21T00:26:37.844245+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:33:20.213727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T00:26:38.938974+0000","last_change":"2024-09-21T00:26:36.851736+0000","last_active":"2024-09-21T00:26:38.938974+0000","last_peered":"2024-09-21T00:26:38.938974+0000","last_clean":"2024-09-21T00:26:38.938974+0000","last_became_active":"2024-09-21T00:26:36.847404+0000","last_became_peered":"2024-09-21T00:26:36.847404+0000","last_unstale":"2024-09-21T00:26:38.938974+0000","last_undegraded":"2024-09-21T00:26:38.938974+0000","last_fullsized":"2024-09-21T00:26:38.938974+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:51:29.747215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T00:26:38.938126+0000","last_change":"2024-09-21T00:26:35.837934+0000","last_active":"2024-09-21T00:26:38.938126+0000","last_peered":"2024-09-21T00:26:38.938126+0000","last_clean":"2024-09-21T00:26:38.938126+0000","last_became_active":"2024-09-21T00:26:35.837198+0000","last_became_peered":"2024-09-21T00:26:35.837198+0000","last_unstale":"2024-09-21T00:26:38.938126+0000","last_undegraded":"2024-09-21T00:26:38.938126+0000","last_fullsized":"2024-09-21T00:26:38.938126+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:39:14.371644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T00:26:38.901027+0000","last_change":"2024-09-21T00:26:36.841327+0000","last_active":"2024-09-21T00:26:38.901027+0000","last_peered":"2024-09-21T00:26:38.901027+0000","last_clean":"2024-09-21T00:26:38.901027+0000","last_became_active":"2024-09-21T00:26:36.841007+0000","last_became_peered":"2024-09-21T00:26:36.841007+0000","last_unstale":"2024-09-21T00:26:38.901027+0000","last_undegraded":"2024-09-21T00:26:38.901027+0000","last_fullsized":"2024-09-21T00:26:38.901027+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:55:00.792478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T00:26:37.843950+0000","last_change":"2024-09-21T00:26:35.832534+0000","last_active":"2024-09-21T00:26:37.843950+0000","last_peered":"2024-09-21T00:26:37.843950+0000","last_clean":"2024-09-21T00:26:37.843950+0000","last_became_active":"2024-09-21T00:26:35.832197+0000","last_became_peered":"2024-09-21T00:26:35.832197+0000","last_unstale":"2024-09-21T00:26:37.843950+0000","last_undegraded":"2024-09-21T00:26:37.843950+0000","last_fullsized":"2024-09-21T00:26:37.843950+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:03:37.183826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T00:26:38.937435+0000","last_change":"2024-09-21T00:26:36.841191+0000","last_active":"2024-09-21T00:26:38.937435+0000","last_peered":"2024-09-21T00:26:38.937435+0000","last_clean":"2024-09-21T00:26:38.937435+0000","last_became_active":"2024-09-21T00:26:36.840676+0000","last_became_peered":"2024-09-21T00:26:36.840676+0000","last_unstale":"2024-09-21T00:26:38.937435+0000","last_undegraded":"2024-09-21T00:26:38.937435+0000","last_fullsized":"2024-09-21T00:26:38.937435+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:49:57.619424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"37'2","reported_seq":17,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T00:26:38.939355+0000","last_change":"2024-09-21T00:26:35.840350+0000","last_active":"2024-09-21T00:26:38.939355+0000","last_peered":"2024-09-21T00:26:38.939355+0000","last_clean":"2024-09-21T00:26:38.939355+0000","last_became_active":"2024-09-21T00:26:35.840048+0000","last_became_peered":"2024-09-21T00:26:35.840048+0000","last_unstale":"2024-09-21T00:26:38.939355+0000","last_undegraded":"2024-09-21T00:26:38.939355+0000","last_fullsized":"2024-09-21T00:26:38.939355+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:33:15.124204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T00:26:38.939474+0000","last_change":"2024-09-21T00:26:36.851474+0000","last_active":"2024-09-21T00:26:38.939474+0000","last_peered":"2024-09-21T00:26:38.939474+0000","last_clean":"2024-09-21T00:26:38.939474+0000","last_became_active":"2024-09-21T00:26:36.851006+0000","last_became_peered":"2024-09-21T00:26:36.851006+0000","last_unstale":"2024-09-21T00:26:38.939474+0000","last_undegraded":"2024-09-21T00:26:38.939474+0000","last_fullsized":"2024-09-21T00:26:38.939474+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:19:13.820898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"37'1","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T00:26:37.844096+0000","last_change":"2024-09-21T00:26:35.838940+0000","last_active":"2024-09-21T00:26:37.844096+0000","last_peered":"2024-09-21T00:26:37.844096+0000","last_clean":"2024-09-21T00:26:37.844096+0000","last_became_active":"2024-09-21T00:26:35.838616+0000","last_became_peered":"2024-09-21T00:26:35.838616+0000","last_unstale":"2024-09-21T00:26:37.844096+0000","last_undegraded":"2024-09-21T00:26:37.844096+0000","last_fullsized":"2024-09-21T00:26:37.844096+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:46:53.236827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T00:26:38.900958+0000","last_change":"2024-09-21T00:26:36.849765+0000","last_active":"2024-09-21T00:26:38.900958+0000","last_peered":"2024-09-21T00:26:38.900958+0000","last_clean":"2024-09-21T00:26:38.900958+0000","last_became_active":"2024-09-21T00:26:36.849493+0000","last_became_peered":"2024-09-21T00:26:36.849493+0000","last_unstale":"2024-09-21T00:26:38.900958+0000","last_undegraded":"2024-09-21T00:26:38.900958+0000","last_fullsized":"2024-09-21T00:26:38.900958+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:34:24.234404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T00:26:38.214437+0000","last_change":"2024-09-21T00:26:35.838748+0000","last_active":"2024-09-21T00:26:38.214437+0000","last_peered":"2024-09-21T00:26:38.214437+0000","last_clean":"2024-09-21T00:26:38.214437+0000","last_became_active":"2024-09-21T00:26:35.838445+0000","last_became_peered":"2024-09-21T00:26:35.838445+0000","last_unstale":"2024-09-21T00:26:38.214437+0000","last_undegraded":"2024-09-21T00:26:38.214437+0000","last_fullsized":"2024-09-21T00:26:38.214437+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:45:54.889994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T00:26:38.214734+0000","last_change":"2024-09-21T00:26:36.843385+0000","last_active":"2024-09-21T00:26:38.214734+0000","last_peered":"2024-09-21T00:26:38.214734+0000","last_clean":"2024-09-21T00:26:38.214734+0000","last_became_active":"2024-09-21T00:26:36.842770+0000","last_became_peered":"2024-09-21T00:26:36.842770+0000","last_unstale":"2024-09-21T00:26:38.214734+0000","last_undegraded":"2024-09-21T00:26:38.214734+0000","last_fullsized":"2024-09-21T00:26:38.214734+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:56:38.653789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T00:26:38.214795+0000","last_change":"2024-09-21T00:26:35.840565+0000","last_active":"2024-09-21T00:26:38.214795+0000","last_peered":"2024-09-21T00:26:38.214795+0000","last_clean":"2024-09-21T00:26:38.214795+0000","last_became_active":"2024-09-21T00:26:35.840265+0000","last_became_peered":"2024-09-21T00:26:35.840265+0000","last_unstale":"2024-09-21T00:26:38.214795+0000","last_undegraded":"2024-09-21T00:26:38.214795+0000","last_fullsized":"2024-09-21T00:26:38.214795+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:27:22.824110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T00:26:38.937170+0000","last_change":"2024-09-21T00:26:36.842397+0000","last_active":"2024-09-21T00:26:38.937170+0000","last_peered":"2024-09-21T00:26:38.937170+0000","last_clean":"2024-09-21T00:26:38.937170+0000","last_became_active":"2024-09-21T00:26:36.842095+0000","last_became_peered":"2024-09-21T00:26:36.842095+0000","last_unstale":"2024-09-21T00:26:38.937170+0000","last_undegraded":"2024-09-21T00:26:38.937170+0000","last_fullsized":"2024-09-21T00:26:38.937170+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:11:17.622409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T00:26:37.841711+0000","last_change":"2024-09-21T00:26:35.835982+0000","last_active":"2024-09-21T00:26:37.841711+0000","last_peered":"2024-09-21T00:26:37.841711+0000","last_clean":"2024-09-21T00:26:37.841711+0000","last_became_active":"2024-09-21T00:26:35.835677+0000","last_became_peered":"2024-09-21T00:26:35.835677+0000","last_unstale":"2024-09-21T00:26:37.841711+0000","last_undegraded":"2024-09-21T00:26:37.841711+0000","last_fullsized":"2024-09-21T00:26:37.841711+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:59:38.916045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T00:26:37.841903+0000","last_change":"2024-09-21T00:26:36.848205+0000","last_active":"2024-09-21T00:26:37.841903+0000","last_peered":"2024-09-21T00:26:37.841903+0000","last_clean":"2024-09-21T00:26:37.841903+0000","last_became_active":"2024-09-21T00:26:36.847492+0000","last_became_peered":"2024-09-21T00:26:36.847492+0000","last_unstale":"2024-09-21T00:26:37.841903+0000","last_undegraded":"2024-09-21T00:26:37.841903+0000","last_fullsized":"2024-09-21T00:26:37.841903+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:03:30.505595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T00:26:37.840948+0000","last_change":"2024-09-21T00:26:35.838318+0000","last_active":"2024-09-21T00:26:37.840948+0000","last_peered":"2024-09-21T00:26:37.840948+0000","last_clean":"2024-09-21T00:26:37.840948+0000","last_became_active":"2024-09-21T00:26:35.836426+0000","last_became_peered":"2024-09-21T00:26:35.836426+0000","last_unstale":"2024-09-21T00:26:37.840948+0000","last_undegraded":"2024-09-21T00:26:37.840948+0000","last_fullsized":"2024-09-21T00:26:37.840948+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:44:34.009179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T00:26:38.938747+0000","last_change":"2024-09-21T00:26:36.849704+0000","last_active":"2024-09-21T00:26:38.938747+0000","last_peered":"2024-09-21T00:26:38.938747+0000","last_clean":"2024-09-21T00:26:38.938747+0000","last_became_active":"2024-09-21T00:26:36.849313+0000","last_became_peered":"2024-09-21T00:26:36.849313+0000","last_unstale":"2024-09-21T00:26:38.938747+0000","last_undegraded":"2024-09-21T00:26:38.938747+0000","last_fullsized":"2024-09-21T00:26:38.938747+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:34:20.884941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T00:26:38.900730+0000","last_change":"2024-09-21T00:26:35.836745+0000","last_active":"2024-09-21T00:26:38.900730+0000","last_peered":"2024-09-21T00:26:38.900730+0000","last_clean":"2024-09-21T00:26:38.900730+0000","last_became_active":"2024-09-21T00:26:35.836427+0000","last_became_peered":"2024-09-21T00:26:35.836427+0000","last_unstale":"2024-09-21T00:26:38.900730+0000","last_undegraded":"2024-09-21T00:26:38.900730+0000","last_fullsized":"2024-09-21T00:26:38.900730+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:10:19.582276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T00:26:38.938735+0000","last_change":"2024-09-21T00:26:36.847316+0000","last_active":"2024-09-21T00:26:38.938735+0000","last_peered":"2024-09-21T00:26:38.938735+0000","last_clean":"2024-09-21T00:26:38.938735+0000","last_became_active":"2024-09-21T00:26:36.846640+0000","last_became_peered":"2024-09-21T00:26:36.846640+0000","last_unstale":"2024-09-21T00:26:38.938735+0000","last_undegraded":"2024-09-21T00:26:38.938735+0000","last_fullsized":"2024-09-21T00:26:38.938735+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:35.811406+0000","last_clean_scrub_stamp":"2024-09-21T00:26:35.811406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:19:13.820898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T00:26:38.938759+0000","last_change":"2024-09-21T00:26:35.839244+0000","last_active":"2024-09-21T00:26:38.938759+0000","last_peered":"2024-09-21T00:26:38.938759+0000","last_clean":"2024-09-21T00:26:38.938759+0000","last_became_active":"2024-09-21T00:26:35.838876+0000","last_became_peered":"2024-09-21T00:26:35.838876+0000","last_unstale":"2024-09-21T00:26:38.938759+0000","last_undegraded":"2024-09-21T00:26:38.938759+0000","last_fullsized":"2024-09-21T00:26:38.938759+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:26:34.805653+0000","last_clean_scrub_stamp":"2024-09-21T00:26:34.805653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:04:17.224210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":140,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T00:26:38.938296+0000","last_change":"2024-09-21T00:25:38.725792+0000","last_active":"2024-09-21T00:26:38.938296+0000","last_peered":"2024-09-21T00:26:38.938296+0000","last_clean":"2024-09-21T00:26:38.938296+0000","last_became_active":"2024-09-21T00:25:38.724668+0000","last_became_peered":"2024-09-21T00:25:38.724668+0000","last_unstale":"2024-09-21T00:26:38.938296+0000","last_undegraded":"2024-09-21T00:26:38.938296+0000","last_fullsized":"2024-09-21T00:26:38.938296+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:37.163216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:37.163216+0000","last_clean_scrub_stamp":"2024-09-21T00:25:37.163216+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:14:36.588660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509456,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28044,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343796,"statfs":{"total":96636764160,"available":96608047104,"internally_reserved":0,"allocated":1040384,"data_stored":773937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509457,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27452,"kb_used_data":424,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344388,"statfs":{"total":96636764160,"available":96608653312,"internally_reserved":0,"allocated":434176,"data_stored":182469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509456,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27456,"kb_used_data":428,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344384,"statfs":{"total":96636764160,"available":96608649216,"internally_reserved":0,"allocated":438272,"data_stored":181999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509455,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27456,"kb_used_data":428,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344384,"statfs":{"total":96636764160,"available":96608649216,"internally_reserved":0,"allocated":438272,"data_stored":182491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509457,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27464,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344376,"statfs":{"total":96636764160,"available":96608641024,"internally_reserved":0,"allocated":446464,"data_stored":183533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509457,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27452,"kb_used_data":424,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344388,"statfs":{"total":96636764160,"available":96608653312,"internally_reserved":0,"allocated":434176,"data_stored":181957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509457,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28032,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343808,"statfs":{"total":96636764160,"available":96608059392,"internally_reserved":0,"allocated":1028096,"data_stored":772289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574865,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27456,"kb_used_data":424,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344384,"statfs":{"total":96636764160,"available":96608649216,"internally_reserved":0,"allocated":434176,"data_stored":181977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T00:26:45.647 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T00:26:45.647 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-21T00:26:45.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:45.800+0000 7fc1de8dd640 1 -- 172.21.15.112:0/4132830703 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1d80d0980 msgr2=0x7fc1d80d0d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:45.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:45.800+0000 7fc1de8dd640 1 --2- 172.21.15.112:0/4132830703 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1d80d0980 0x7fc1d80d0d90 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fc1d8073e10 tx=0x7fc1d002aa10 comp rx=0 tx=0).stop 2024-09-21T00:26:45.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:45.800+0000 7fc1de8dd640 1 -- 172.21.15.112:0/4132830703 shutdown_connections 2024-09-21T00:26:45.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:45.800+0000 7fc1de8dd640 1 --2- 172.21.15.112:0/4132830703 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1d80d0980 0x7fc1d80d0d90 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:45.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:45.800+0000 7fc1de8dd640 1 --2- 172.21.15.112:0/4132830703 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc1d815a6f0 0x7fc1d815e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:45.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:45.800+0000 7fc1de8dd640 1 --2- 172.21.15.112:0/4132830703 >> v2:172.21.15.187:3300/0 conn(0x7fc1d80d01f0 0x7fc1d815daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:45.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:45.800+0000 7fc1de8dd640 1 -- 172.21.15.112:0/4132830703 >> 172.21.15.112:0/4132830703 conn(0x7fc1d8072990 msgr2=0x7fc1d8072da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:45.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:45.800+0000 7fc1de8dd640 1 -- 172.21.15.112:0/4132830703 shutdown_connections 2024-09-21T00:26:45.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:45.800+0000 7fc1de8dd640 1 -- 172.21.15.112:0/4132830703 wait complete. 2024-09-21T00:26:45.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:45.802+0000 7fc1de8dd640 1 Processor -- start 2024-09-21T00:26:45.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:45.802+0000 7fc1de8dd640 1 -- start start 2024-09-21T00:26:45.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:45.803+0000 7fc1de8dd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1d80d01f0 0x7fc1d81f9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:45.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:45.803+0000 7fc1de8dd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc1d80d0980 0x7fc1d8148f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:45.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:45.803+0000 7fc1de8dd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc1d815a6f0 0x7fc1d80cbc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:45.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:45.803+0000 7fc1de8dd640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc1d80ccfb0 con 0x7fc1d80d01f0 2024-09-21T00:26:45.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:45.803+0000 7fc1de8dd640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc1d815e980 con 0x7fc1d815a6f0 2024-09-21T00:26:45.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:45.803+0000 7fc1de8dd640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc1d80cd130 con 0x7fc1d80d0980 2024-09-21T00:26:45.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:45.803+0000 7fc1d7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1d80d01f0 0x7fc1d81f9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:45.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:45.803+0000 7fc1d7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1d80d01f0 0x7fc1d81f9cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36888/0 (socket says 172.21.15.112:36888) 2024-09-21T00:26:45.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:45.803+0000 7fc1d7fff640 1 -- 172.21.15.112:0/3449249873 learned_addr learned my addr 172.21.15.112:0/3449249873 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:26:45.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:45.803+0000 7fc1cffff640 1 --2- 172.21.15.112:0/3449249873 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc1d80d0980 0x7fc1d8148f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:45.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:45.803+0000 7fc1dce7a640 1 --2- 172.21.15.112:0/3449249873 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc1d815a6f0 0x7fc1d80cbc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:45.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:45.804+0000 7fc1d7fff640 1 -- 172.21.15.112:0/3449249873 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc1d80d0980 msgr2=0x7fc1d8148f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:45.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:45.804+0000 7fc1d7fff640 1 --2- 172.21.15.112:0/3449249873 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc1d80d0980 0x7fc1d8148f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:45.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:45.804+0000 7fc1d7fff640 1 -- 172.21.15.112:0/3449249873 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc1d815a6f0 msgr2=0x7fc1d80cbc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:45.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:45.804+0000 7fc1d7fff640 1 --2- 172.21.15.112:0/3449249873 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc1d815a6f0 0x7fc1d80cbc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:45.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:45.804+0000 7fc1d7fff640 1 -- 172.21.15.112:0/3449249873 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1d80c8ba0 con 0x7fc1d80d01f0 2024-09-21T00:26:45.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:45.804+0000 7fc1d7fff640 1 --2- 172.21.15.112:0/3449249873 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1d80d01f0 0x7fc1d81f9cd0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fc1c800e740 tx=0x7fc1c800ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:45.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:45.804+0000 7fc1d57fa640 1 -- 172.21.15.112:0/3449249873 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1c800cc60 con 0x7fc1d80d01f0 2024-09-21T00:26:45.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:45.804+0000 7fc1d57fa640 1 -- 172.21.15.112:0/3449249873 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc1c800ee50 con 0x7fc1d80d01f0 2024-09-21T00:26:45.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:45.804+0000 7fc1d57fa640 1 -- 172.21.15.112:0/3449249873 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1c8010670 con 0x7fc1d80d01f0 2024-09-21T00:26:45.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:45.804+0000 7fc1de8dd640 1 -- 172.21.15.112:0/3449249873 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1d81494d0 con 0x7fc1d80d01f0 2024-09-21T00:26:45.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:45.804+0000 7fc1de8dd640 1 -- 172.21.15.112:0/3449249873 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1d80c88c0 con 0x7fc1d80d01f0 2024-09-21T00:26:45.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:45.805+0000 7fc1de8dd640 1 -- 172.21.15.112:0/3449249873 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1d80d0d90 con 0x7fc1d80d01f0 2024-09-21T00:26:45.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:45.806+0000 7fc1d57fa640 1 -- 172.21.15.112:0/3449249873 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc1c8010810 con 0x7fc1d80d01f0 2024-09-21T00:26:45.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:45.806+0000 7fc1d57fa640 1 --2- 172.21.15.112:0/3449249873 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fc1b8078b90 0x7fc1b807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:45.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:45.806+0000 7fc1d57fa640 1 -- 172.21.15.112:0/3449249873 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7fc1c809b7e0 con 0x7fc1d80d01f0 2024-09-21T00:26:45.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:45.806+0000 7fc1cffff640 1 --2- 172.21.15.112:0/3449249873 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fc1b8078b90 0x7fc1b807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:45.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:45.807+0000 7fc1cffff640 1 --2- 172.21.15.112:0/3449249873 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fc1b8078b90 0x7fc1b807b0a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc1d81463c0 tx=0x7fc1c000b2e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:45.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:45.811+0000 7fc1d57fa640 1 -- 172.21.15.112:0/3449249873 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc1d81494d0 con 0x7fc1d80d01f0 2024-09-21T00:26:46.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.059+0000 7fc1de8dd640 1 -- 172.21.15.112:0/3449249873 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fc1d80cc3b0 con 0x7fc1d80d01f0 2024-09-21T00:26:46.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.061+0000 7fc1d57fa640 1 -- 172.21.15.112:0/3449249873 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v15) ==== 190+0+147 (secure 0 0 0) 0x7fc1c8063440 con 0x7fc1d80d01f0 2024-09-21T00:26:46.063 INFO:teuthology.orchestra.run.smithi112.stdout:[client.0] 2024-09-21T00:26:46.064 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQBPEu5m49NPHBAAl/rcjiE8xYI+XdWwvZ6FOw== 2024-09-21T00:26:46.064 INFO:teuthology.orchestra.run.smithi112.stdout: caps mds = "allow" 2024-09-21T00:26:46.064 INFO:teuthology.orchestra.run.smithi112.stdout: caps mon = "allow r" 2024-09-21T00:26:46.064 INFO:teuthology.orchestra.run.smithi112.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-21T00:26:46.064 INFO:teuthology.orchestra.run.smithi112.stderr:updated caps for client.0 2024-09-21T00:26:46.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.064+0000 7fc1de8dd640 1 -- 172.21.15.112:0/3449249873 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fc1b8078b90 msgr2=0x7fc1b807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:46.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.064+0000 7fc1de8dd640 1 --2- 172.21.15.112:0/3449249873 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fc1b8078b90 0x7fc1b807b0a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc1d81463c0 tx=0x7fc1c000b2e0 comp rx=0 tx=0).stop 2024-09-21T00:26:46.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.064+0000 7fc1de8dd640 1 -- 172.21.15.112:0/3449249873 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1d80d01f0 msgr2=0x7fc1d81f9cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:46.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.064+0000 7fc1de8dd640 1 --2- 172.21.15.112:0/3449249873 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1d80d01f0 0x7fc1d81f9cd0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fc1c800e740 tx=0x7fc1c800ec00 comp rx=0 tx=0).stop 2024-09-21T00:26:46.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.064+0000 7fc1de8dd640 1 -- 172.21.15.112:0/3449249873 shutdown_connections 2024-09-21T00:26:46.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.064+0000 7fc1de8dd640 1 --2- 172.21.15.112:0/3449249873 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc1d815a6f0 0x7fc1d80cbc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:46.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.064+0000 7fc1de8dd640 1 --2- 172.21.15.112:0/3449249873 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc1d80d0980 0x7fc1d8148f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:46.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.064+0000 7fc1de8dd640 1 --2- 172.21.15.112:0/3449249873 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fc1b8078b90 0x7fc1b807b0a0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:46.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.064+0000 7fc1de8dd640 1 --2- 172.21.15.112:0/3449249873 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1d80d01f0 0x7fc1d81f9cd0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:46.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.064+0000 7fc1de8dd640 1 -- 172.21.15.112:0/3449249873 >> 172.21.15.112:0/3449249873 conn(0x7fc1d8072990 msgr2=0x7fc1d815d4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:46.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.064+0000 7fc1de8dd640 1 -- 172.21.15.112:0/3449249873 shutdown_connections 2024-09-21T00:26:46.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.064+0000 7fc1de8dd640 1 -- 172.21.15.112:0/3449249873 wait complete. 2024-09-21T00:26:46.077 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T00:26:46.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.189+0000 7f11f5eb4640 1 -- 172.21.15.112:0/1854898563 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f11f00cbea0 msgr2=0x7f11f006cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:46.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.189+0000 7f11f5eb4640 1 --2- 172.21.15.112:0/1854898563 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f11f00cbea0 0x7f11f006cde0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f11e8009860 tx=0x7f11e802a4a0 comp rx=0 tx=0).stop 2024-09-21T00:26:46.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.189+0000 7f11f5eb4640 1 -- 172.21.15.112:0/1854898563 shutdown_connections 2024-09-21T00:26:46.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.189+0000 7f11f5eb4640 1 --2- 172.21.15.112:0/1854898563 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f11f00cbea0 0x7f11f006cde0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:46.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.189+0000 7f11f5eb4640 1 -- 172.21.15.112:0/1854898563 >> 172.21.15.112:0/1854898563 conn(0x7f11f00d1140 msgr2=0x7f11f00d1550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:46.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.189+0000 7f11f5eb4640 1 -- 172.21.15.112:0/1854898563 shutdown_connections 2024-09-21T00:26:46.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.189+0000 7f11f5eb4640 1 -- 172.21.15.112:0/1854898563 wait complete. 2024-09-21T00:26:46.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.191+0000 7f11f5eb4640 1 Processor -- start 2024-09-21T00:26:46.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.191+0000 7f11f5eb4640 1 -- start start 2024-09-21T00:26:46.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.192+0000 7f11f5eb4640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f11f0070f10 0x7f11f0157840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:46.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.192+0000 7f11f5eb4640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f11f0074a70 0x7f11f015a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:46.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.192+0000 7f11f5eb4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11f0157d80 0x7f11f01f6a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:46.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.192+0000 7f11f5eb4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f11f006daf0 con 0x7f11f0157d80 2024-09-21T00:26:46.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.192+0000 7f11f5eb4640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f11f006d970 con 0x7f11f0070f10 2024-09-21T00:26:46.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.192+0000 7f11f5eb4640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f11f0167d00 con 0x7f11f0074a70 2024-09-21T00:26:46.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.192+0000 7f11effff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11f0157d80 0x7f11f01f6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:46.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.192+0000 7f11effff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11f0157d80 0x7f11f01f6a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60690/0 (socket says 172.21.15.112:60690) 2024-09-21T00:26:46.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.192+0000 7f11effff640 1 -- 172.21.15.112:0/2293304605 learned_addr learned my addr 172.21.15.112:0/2293304605 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:26:46.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.192+0000 7f11ef7fe640 1 --2- 172.21.15.112:0/2293304605 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f11f0070f10 0x7f11f0157840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:46.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.192+0000 7f11eeffd640 1 --2- 172.21.15.112:0/2293304605 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f11f0074a70 0x7f11f015a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:46.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.193+0000 7f11effff640 1 -- 172.21.15.112:0/2293304605 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f11f0074a70 msgr2=0x7f11f015a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:46.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.193+0000 7f11effff640 1 --2- 172.21.15.112:0/2293304605 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f11f0074a70 0x7f11f015a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:46.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.193+0000 7f11effff640 1 -- 172.21.15.112:0/2293304605 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f11f0070f10 msgr2=0x7f11f0157840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:46.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.193+0000 7f11effff640 1 --2- 172.21.15.112:0/2293304605 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f11f0070f10 0x7f11f0157840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:46.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.193+0000 7f11effff640 1 -- 172.21.15.112:0/2293304605 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11f013d4c0 con 0x7f11f0157d80 2024-09-21T00:26:46.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.193+0000 7f11effff640 1 --2- 172.21.15.112:0/2293304605 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11f0157d80 0x7f11f01f6a10 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f11e8009980 tx=0x7f11e8003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:46.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.193+0000 7f11d7fff640 1 -- 172.21.15.112:0/2293304605 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11e8005c90 con 0x7f11f0157d80 2024-09-21T00:26:46.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.193+0000 7f11d7fff640 1 -- 172.21.15.112:0/2293304605 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f11e8005020 con 0x7f11f0157d80 2024-09-21T00:26:46.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.193+0000 7f11d7fff640 1 -- 172.21.15.112:0/2293304605 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11e8005450 con 0x7f11f0157d80 2024-09-21T00:26:46.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.193+0000 7f11f5eb4640 1 -- 172.21.15.112:0/2293304605 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11f015aff0 con 0x7f11f0157d80 2024-09-21T00:26:46.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.193+0000 7f11f5eb4640 1 -- 172.21.15.112:0/2293304605 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11f00b5300 con 0x7f11f0157d80 2024-09-21T00:26:46.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.194+0000 7f11f5eb4640 1 -- 172.21.15.112:0/2293304605 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11f015ad60 con 0x7f11f0157d80 2024-09-21T00:26:46.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.195+0000 7f11d7fff640 1 -- 172.21.15.112:0/2293304605 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f11e80317c0 con 0x7f11f0157d80 2024-09-21T00:26:46.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.195+0000 7f11d7fff640 1 --2- 172.21.15.112:0/2293304605 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f11dc078b90 0x7f11dc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:46.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.195+0000 7f11d7fff640 1 -- 172.21.15.112:0/2293304605 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f11e80bbe60 con 0x7f11f0157d80 2024-09-21T00:26:46.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.195+0000 7f11ef7fe640 1 --2- 172.21.15.112:0/2293304605 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f11dc078b90 0x7f11dc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:46.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.196+0000 7f11ef7fe640 1 --2- 172.21.15.112:0/2293304605 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f11dc078b90 0x7f11dc07b0a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f11e0001740 tx=0x7f11e0001aa0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:46.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.200+0000 7f11d7fff640 1 -- 172.21.15.112:0/2293304605 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f11e803f030 con 0x7f11f0157d80 2024-09-21T00:26:46.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.413+0000 7f11f5eb4640 1 -- 172.21.15.112:0/2293304605 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f11f013d690 con 0x7f11f0157d80 2024-09-21T00:26:46.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.413+0000 7f11d7fff640 1 -- 172.21.15.112:0/2293304605 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5232 (secure 0 0 0) 0x7f11e8083ac0 con 0x7f11f0157d80 2024-09-21T00:26:46.414 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:26:46.414 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":17,"btime":"2024-09-21T00:26:39:841719+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4846,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6834/2036740840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":2036740840},{"type":"v1","addr":"172.21.15.112:6834","nonce":2036740840}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4855,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6836/3699399640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":3699399640},{"type":"v1","addr":"172.21.15.187:6836","nonce":3699399640}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4858,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6837/694936803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6835","nonce":694936803},{"type":"v1","addr":"172.21.15.187:6837","nonce":694936803}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T00:26:36.187741+0000","modified":"2024-09-21T00:26:38.947874+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5028},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5028":{"gid":5028,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6835/4012316979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":4012316979},{"type":"v1","addr":"172.21.15.112:6835","nonce":4012316979}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5028,"qdb_cluster":[5028]},"id":2}]} 2024-09-21T00:26:46.415 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 17 2024-09-21T00:26:46.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.415+0000 7f11f5eb4640 1 -- 172.21.15.112:0/2293304605 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f11dc078b90 msgr2=0x7f11dc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:46.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.415+0000 7f11f5eb4640 1 --2- 172.21.15.112:0/2293304605 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f11dc078b90 0x7f11dc07b0a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f11e0001740 tx=0x7f11e0001aa0 comp rx=0 tx=0).stop 2024-09-21T00:26:46.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.415+0000 7f11f5eb4640 1 -- 172.21.15.112:0/2293304605 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11f0157d80 msgr2=0x7f11f01f6a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:46.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.415+0000 7f11f5eb4640 1 --2- 172.21.15.112:0/2293304605 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11f0157d80 0x7f11f01f6a10 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f11e8009980 tx=0x7f11e8003c10 comp rx=0 tx=0).stop 2024-09-21T00:26:46.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.415+0000 7f11f5eb4640 1 -- 172.21.15.112:0/2293304605 shutdown_connections 2024-09-21T00:26:46.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.415+0000 7f11f5eb4640 1 --2- 172.21.15.112:0/2293304605 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f11dc078b90 0x7f11dc07b0a0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:46.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.416+0000 7f11f5eb4640 1 --2- 172.21.15.112:0/2293304605 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11f0157d80 0x7f11f01f6a10 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:46.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.416+0000 7f11f5eb4640 1 --2- 172.21.15.112:0/2293304605 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f11f0074a70 0x7f11f015a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:46.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.416+0000 7f11f5eb4640 1 --2- 172.21.15.112:0/2293304605 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f11f0070f10 0x7f11f0157840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:46.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.416+0000 7f11f5eb4640 1 -- 172.21.15.112:0/2293304605 >> 172.21.15.112:0/2293304605 conn(0x7f11f00d1140 msgr2=0x7f11f00b1910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:46.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.416+0000 7f11f5eb4640 1 -- 172.21.15.112:0/2293304605 shutdown_connections 2024-09-21T00:26:46.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.416+0000 7f11f5eb4640 1 -- 172.21.15.112:0/2293304605 wait complete. 2024-09-21T00:26:46.830 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T00:26:36.187741+0000', 'modified': '2024-09-21T00:26:38.947874+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5028}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5028': {'gid': 5028, 'name': 'a', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.112:6835/4012316979', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6833', 'nonce': 4012316979}, {'type': 'v1', 'addr': '172.21.15.112:6835', 'nonce': 4012316979}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5028, 'qdb_cluster': [5028]} 2024-09-21T00:26:46.830 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-21T00:26:46.830 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-21T00:26:46.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.945+0000 7f591f19d640 1 -- 172.21.15.112:0/3773717900 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f59180d2290 msgr2=0x7f59180d26a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:46.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.945+0000 7f591f19d640 1 --2- 172.21.15.112:0/3773717900 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f59180d2290 0x7f59180d26a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5910005fd0 tx=0x7f591002aa90 comp rx=0 tx=0).stop 2024-09-21T00:26:46.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.945+0000 7f591f19d640 1 -- 172.21.15.112:0/3773717900 shutdown_connections 2024-09-21T00:26:46.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.945+0000 7f591f19d640 1 --2- 172.21.15.112:0/3773717900 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f59180d2290 0x7f59180d26a0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:46.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.945+0000 7f591f19d640 1 --2- 172.21.15.112:0/3773717900 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f591815f6d0 0x7f591815e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:46.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.945+0000 7f591f19d640 1 -- 172.21.15.112:0/3773717900 >> 172.21.15.112:0/3773717900 conn(0x7f59180d41e0 msgr2=0x7f59180d45f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:46.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.945+0000 7f591f19d640 1 -- 172.21.15.112:0/3773717900 shutdown_connections 2024-09-21T00:26:46.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.945+0000 7f591f19d640 1 -- 172.21.15.112:0/3773717900 wait complete. 2024-09-21T00:26:46.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.947+0000 7f591f19d640 1 Processor -- start 2024-09-21T00:26:46.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.947+0000 7f591f19d640 1 -- start start 2024-09-21T00:26:46.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.948+0000 7f591f19d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f59180cef00 0x7f591814c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:46.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.948+0000 7f591f19d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f59180d2290 0x7f5918159e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:46.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.948+0000 7f591f19d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f591815f6d0 0x7f59181607b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:46.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.948+0000 7f591f19d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f59180d2be0 con 0x7f591815f6d0 2024-09-21T00:26:46.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.948+0000 7f591f19d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f591806dce0 con 0x7f59180cef00 2024-09-21T00:26:46.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.948+0000 7f591f19d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f59180d2d60 con 0x7f59180d2290 2024-09-21T00:26:46.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.948+0000 7f591e99c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f591815f6d0 0x7f59181607b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:46.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.948+0000 7f591e99c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f591815f6d0 0x7f59181607b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60692/0 (socket says 172.21.15.112:60692) 2024-09-21T00:26:46.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.948+0000 7f591e99c640 1 -- 172.21.15.112:0/839678220 learned_addr learned my addr 172.21.15.112:0/839678220 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:26:46.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.948+0000 7f591e19b640 1 --2- 172.21.15.112:0/839678220 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f59180cef00 0x7f591814c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:46.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.948+0000 7f591d99a640 1 --2- 172.21.15.112:0/839678220 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f59180d2290 0x7f5918159e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:46.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.948+0000 7f591e99c640 1 -- 172.21.15.112:0/839678220 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f59180d2290 msgr2=0x7f5918159e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:46.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.948+0000 7f591e99c640 1 --2- 172.21.15.112:0/839678220 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f59180d2290 0x7f5918159e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:46.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.948+0000 7f591e99c640 1 -- 172.21.15.112:0/839678220 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f59180cef00 msgr2=0x7f591814c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:46.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.948+0000 7f591e99c640 1 --2- 172.21.15.112:0/839678220 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f59180cef00 0x7f591814c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:46.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.948+0000 7f591e99c640 1 -- 172.21.15.112:0/839678220 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59181f8a10 con 0x7f591815f6d0 2024-09-21T00:26:46.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.949+0000 7f591e99c640 1 --2- 172.21.15.112:0/839678220 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f591815f6d0 0x7f59181607b0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f5910009bf0 tx=0x7f5910003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:46.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.949+0000 7f590effd640 1 -- 172.21.15.112:0/839678220 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5910044070 con 0x7f591815f6d0 2024-09-21T00:26:46.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.949+0000 7f590effd640 1 -- 172.21.15.112:0/839678220 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5910003e00 con 0x7f591815f6d0 2024-09-21T00:26:46.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.949+0000 7f590effd640 1 -- 172.21.15.112:0/839678220 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59100044b0 con 0x7f591815f6d0 2024-09-21T00:26:46.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.949+0000 7f591f19d640 1 -- 172.21.15.112:0/839678220 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59181f8cf0 con 0x7f591815f6d0 2024-09-21T00:26:46.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.949+0000 7f591f19d640 1 -- 172.21.15.112:0/839678220 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59181f92b0 con 0x7f591815f6d0 2024-09-21T00:26:46.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.951+0000 7f590effd640 1 -- 172.21.15.112:0/839678220 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5910047070 con 0x7f591815f6d0 2024-09-21T00:26:46.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.952+0000 7f590effd640 1 --2- 172.21.15.112:0/839678220 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f58fc078bc0 0x7f58fc07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:46.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.952+0000 7f590effd640 1 -- 172.21.15.112:0/839678220 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f59100bbcb0 con 0x7f591815f6d0 2024-09-21T00:26:46.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.952+0000 7f591e19b640 1 --2- 172.21.15.112:0/839678220 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f58fc078bc0 0x7f58fc07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:46.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.952+0000 7f591e19b640 1 --2- 172.21.15.112:0/839678220 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f58fc078bc0 0x7f58fc07b0d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f59140095f0 tx=0x7f5914005f90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:46.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.952+0000 7f5927443740 1 Processor -- start 2024-09-21T00:26:46.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.953+0000 7f5927443740 1 -- start start 2024-09-21T00:26:46.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.954+0000 7f5927443740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d206b25270 0x55d206ef7170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:46.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.954+0000 7f5927443740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d206afc3b0 0x55d206ef76b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:46.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.954+0000 7f5927443740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d206b04590 0x55d206f145b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:46.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.954+0000 7f5927443740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d206e3f820 con 0x55d206b25270 2024-09-21T00:26:46.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.954+0000 7f5927443740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55d206e2ec50 con 0x55d206b04590 2024-09-21T00:26:46.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.954+0000 7f5927443740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x55d206e32a60 con 0x55d206afc3b0 2024-09-21T00:26:46.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.954+0000 7f591e19b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d206b25270 0x55d206ef7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:46.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.954+0000 7f591d99a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d206afc3b0 0x55d206ef76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:46.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.954+0000 7f591e19b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d206b25270 0x55d206ef7170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60698/0 (socket says 172.21.15.112:60698) 2024-09-21T00:26:46.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.954+0000 7f591e19b640 1 -- 172.21.15.112:0/3729190679 learned_addr learned my addr 172.21.15.112:0/3729190679 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:26:46.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.954+0000 7f591e99c640 1 --2- 172.21.15.112:0/3729190679 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d206b04590 0x55d206f145b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:46.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.954+0000 7f591d99a640 1 -- 172.21.15.112:0/3729190679 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d206b04590 msgr2=0x55d206f145b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:26:46.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.954+0000 7f591d99a640 1 --2- 172.21.15.112:0/3729190679 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d206b04590 0x55d206f145b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:46.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.954+0000 7f591d99a640 1 -- 172.21.15.112:0/3729190679 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d206b25270 msgr2=0x55d206ef7170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:26:46.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.954+0000 7f591d99a640 1 --2- 172.21.15.112:0/3729190679 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d206b25270 0x55d206ef7170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:46.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.954+0000 7f591d99a640 1 -- 172.21.15.112:0/3729190679 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d206b90850 con 0x55d206afc3b0 2024-09-21T00:26:46.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.954+0000 7f591e19b640 1 --2- 172.21.15.112:0/3729190679 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d206b25270 0x55d206ef7170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:26:46.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.954+0000 7f591d99a640 1 --2- 172.21.15.112:0/3729190679 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d206afc3b0 0x55d206ef76b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f59180ce9c0 tx=0x7f5908001b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=11c22c3efec55b14 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:46.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.955+0000 7f58fbfff640 1 -- 172.21.15.112:0/3729190679 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59080052e0 con 0x55d206afc3b0 2024-09-21T00:26:46.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.955+0000 7f58fbfff640 1 -- 172.21.15.112:0/3729190679 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5908018070 con 0x55d206afc3b0 2024-09-21T00:26:46.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.955+0000 7f58fbfff640 1 -- 172.21.15.112:0/3729190679 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f590800f9e0 con 0x55d206afc3b0 2024-09-21T00:26:46.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.955+0000 7f5927443740 1 -- 172.21.15.112:0/3729190679 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d206afc3b0 msgr2=0x55d206ef76b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:46.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.955+0000 7f5927443740 1 --2- 172.21.15.112:0/3729190679 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d206afc3b0 0x55d206ef76b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f59180ce9c0 tx=0x7f5908001b50 comp rx=0 tx=0).stop 2024-09-21T00:26:46.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.955+0000 7f5927443740 1 -- 172.21.15.112:0/3729190679 shutdown_connections 2024-09-21T00:26:46.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.955+0000 7f5927443740 1 --2- 172.21.15.112:0/3729190679 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d206b04590 0x55d206f145b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:46.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.955+0000 7f5927443740 1 --2- 172.21.15.112:0/3729190679 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d206afc3b0 0x55d206ef76b0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:46.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.955+0000 7f5927443740 1 --2- 172.21.15.112:0/3729190679 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d206b25270 0x55d206ef7170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:46.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.956+0000 7f5927443740 1 -- 172.21.15.112:0/3729190679 >> 172.21.15.112:0/3729190679 conn(0x55d206b193c0 msgr2=0x55d206ecd430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:46.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.956+0000 7f5927443740 1 -- 172.21.15.112:0/3729190679 shutdown_connections 2024-09-21T00:26:46.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.956+0000 7f5927443740 1 -- 172.21.15.112:0/3729190679 wait complete. 2024-09-21T00:26:46.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.956+0000 7f5927443740 1 -- start start 2024-09-21T00:26:46.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.957+0000 7f5927443740 1 Processor -- start 2024-09-21T00:26:46.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.958+0000 7f5927443740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d206afc3b0 0x55d206f24fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:46.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.958+0000 7f5927443740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d206b04590 0x55d206ee28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:46.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.958+0000 7f5927443740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d206b25270 0x55d206ef5e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:46.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.958+0000 7f5927443740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d206b9f620 con 0x55d206afc3b0 2024-09-21T00:26:46.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.958+0000 7f5927443740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55d206dd45f0 con 0x55d206b25270 2024-09-21T00:26:46.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.958+0000 7f5927443740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x55d206ef7bf0 con 0x55d206b04590 2024-09-21T00:26:46.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.958+0000 7f5927443740 10 client.0 _ll_register_callbacks cb 0x55d206dd80a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:26:46.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.958+0000 7f591e19b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d206afc3b0 0x55d206f24fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:46.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.958+0000 7f591e19b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d206afc3b0 0x55d206f24fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60706/0 (socket says 172.21.15.112:60706) 2024-09-21T00:26:46.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.958+0000 7f591e19b640 1 -- 172.21.15.112:0/2573608614 learned_addr learned my addr 172.21.15.112:0/2573608614 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:26:46.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.958+0000 7f591e99c640 1 --2- 172.21.15.112:0/2573608614 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d206b25270 0x55d206ef5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:46.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.958+0000 7f591d99a640 1 --2- 172.21.15.112:0/2573608614 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d206b04590 0x55d206ee28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:46.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.958+0000 7f591e19b640 1 -- 172.21.15.112:0/2573608614 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d206b04590 msgr2=0x55d206ee28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:26:46.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.958+0000 7f591e19b640 1 --2- 172.21.15.112:0/2573608614 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d206b04590 0x55d206ee28a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:46.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.958+0000 7f591e19b640 1 -- 172.21.15.112:0/2573608614 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d206b25270 msgr2=0x55d206ef5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:26:46.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.958+0000 7f591e19b640 1 --2- 172.21.15.112:0/2573608614 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d206b25270 0x55d206ef5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:46.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.958+0000 7f591e19b640 1 -- 172.21.15.112:0/2573608614 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d206c076b0 con 0x55d206afc3b0 2024-09-21T00:26:46.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.959+0000 7f591e19b640 1 --2- 172.21.15.112:0/2573608614 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d206afc3b0 0x55d206f24fe0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f5914011fd0 tx=0x7f5914004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=621e29e2cc28afdc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:46.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.959+0000 7f58f97fa640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T00:26:46.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.959+0000 7f58f97fa640 1 -- 172.21.15.112:0/2573608614 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5914027420 con 0x55d206afc3b0 2024-09-21T00:26:46.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.959+0000 7f58f97fa640 1 -- 172.21.15.112:0/2573608614 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f5914028e00 con 0x55d206afc3b0 2024-09-21T00:26:46.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.959+0000 7f5927443740 1 -- 172.21.15.112:0/2573608614 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5908018070 con 0x55d206afc3b0 2024-09-21T00:26:46.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.959+0000 7f58f97fa640 1 -- 172.21.15.112:0/2573608614 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f591401d410 con 0x55d206afc3b0 2024-09-21T00:26:46.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.959+0000 7f5927443740 10 client.5211 fetch_fsmap learned FSMap version 17 2024-09-21T00:26:46.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.959+0000 7f5927443740 1 -- 172.21.15.112:0/2573608614 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x55d206ee2de0 con 0x55d206afc3b0 2024-09-21T00:26:46.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.959+0000 7f58f97fa640 1 -- 172.21.15.112:0/2573608614 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5908018070 con 0x55d206afc3b0 2024-09-21T00:26:46.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.959+0000 7f58f97fa640 1 -- 172.21.15.112:0/2573608614 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 17) ==== 3502+0+0 (secure 0 0 0) 0x7f591402f050 con 0x55d206afc3b0 2024-09-21T00:26:46.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.959+0000 7f5927443740 10 client.5211 fetch_fsmap finished waiting for FSMap version 17 2024-09-21T00:26:46.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.959+0000 7f5927443740 10 client.5211 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-21T00:26:46.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.959+0000 7f5927443740 20 client.5211 populate_metadata read hostname 'smithi112' 2024-09-21T00:26:46.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.959+0000 7f5927443740 1 --2- 172.21.15.112:0/2573608614 >> [v2:172.21.15.112:6833/4012316979,v1:172.21.15.112:6835/4012316979] conn(0x55d206ae3e40 0x55d206f090a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:46.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.959+0000 7f5927443740 4 client.5211 mds_command: new command op to 5028 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T00:26:46.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.959+0000 7f5927443740 1 -- 172.21.15.112:0/2573608614 --> [v2:172.21.15.112:6833/4012316979,v1:172.21.15.112:6835/4012316979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d206e41770 con 0x55d206ae3e40 2024-09-21T00:26:46.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.959+0000 7f591d99a640 1 --2- 172.21.15.112:0/2573608614 >> [v2:172.21.15.112:6833/4012316979,v1:172.21.15.112:6835/4012316979] conn(0x55d206ae3e40 0x55d206f090a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:46.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.960+0000 7f58f97fa640 10 client.5211 ms_handle_connect on v2:172.21.15.112:6833/4012316979 2024-09-21T00:26:46.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.960+0000 7f591d99a640 1 --2- 172.21.15.112:0/2573608614 >> [v2:172.21.15.112:6833/4012316979,v1:172.21.15.112:6835/4012316979] conn(0x55d206ae3e40 0x55d206f090a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8c111b6b3e39532 server_cookie=6f0f750d035f36ec in_seq=0 out_seq=0 2024-09-21T00:26:46.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.961+0000 7f58f97fa640 1 -- 172.21.15.112:0/2573608614 <== mds.0 v2:172.21.15.112:6833/4012316979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d206e41770 con 0x55d206ae3e40 2024-09-21T00:26:46.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.961+0000 7f58f97fa640 10 client.5211 handle_command_reply: tid=0 2024-09-21T00:26:46.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.961+0000 7f5927443740 1 client.5211 shutdown 2024-09-21T00:26:46.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.961+0000 7f5927443740 1 -- 172.21.15.112:0/2573608614 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d206afc3b0 msgr2=0x55d206f24fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:46.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.961+0000 7f5927443740 1 --2- 172.21.15.112:0/2573608614 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d206afc3b0 0x55d206f24fe0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f5914011fd0 tx=0x7f5914004730 comp rx=0 tx=0).stop 2024-09-21T00:26:46.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.961+0000 7f5927443740 1 -- 172.21.15.112:0/2573608614 shutdown_connections 2024-09-21T00:26:46.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.961+0000 7f5927443740 1 --2- 172.21.15.112:0/2573608614 >> [v2:172.21.15.112:6833/4012316979,v1:172.21.15.112:6835/4012316979] conn(0x55d206ae3e40 0x55d206f090a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:46.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.961+0000 7f5927443740 1 --2- 172.21.15.112:0/2573608614 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d206b25270 0x55d206ef5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:46.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.961+0000 7f5927443740 1 --2- 172.21.15.112:0/2573608614 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d206b04590 0x55d206ee28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:46.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.961+0000 7f58f97fa640 0 client.5211 ms_handle_reset on v2:172.21.15.112:6833/4012316979 2024-09-21T00:26:46.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.961+0000 7f5927443740 1 --2- 172.21.15.112:0/2573608614 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d206afc3b0 0x55d206f24fe0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:46.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.961+0000 7f5927443740 1 -- 172.21.15.112:0/2573608614 >> 172.21.15.112:0/2573608614 conn(0x55d206b193c0 msgr2=0x55d206ec6760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:46.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.961+0000 7f5927443740 1 -- 172.21.15.112:0/2573608614 shutdown_connections 2024-09-21T00:26:46.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.961+0000 7f5927443740 1 -- 172.21.15.112:0/2573608614 wait complete. 2024-09-21T00:26:46.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.961+0000 7f5927443740 20 client.5211 trim_cache size 0 max 16384 2024-09-21T00:26:46.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.979+0000 7f5927443740 1 Processor -- start 2024-09-21T00:26:46.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.979+0000 7f5927443740 1 -- start start 2024-09-21T00:26:46.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.980+0000 7f5927443740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d206afc3b0 0x55d206ee5860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:46.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.980+0000 7f5927443740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d206b04590 0x55d206ee5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:46.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.980+0000 7f5927443740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d206b25270 0x55d206f34ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:46.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.980+0000 7f5927443740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f59182013d0 con 0x55d206b25270 2024-09-21T00:26:46.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.980+0000 7f5927443740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5918159060 con 0x55d206b04590 2024-09-21T00:26:46.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.980+0000 7f5927443740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f5918200d80 con 0x55d206afc3b0 2024-09-21T00:26:46.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.981+0000 7f591e99c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d206b25270 0x55d206f34ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:46.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.981+0000 7f591e99c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d206b25270 0x55d206f34ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60710/0 (socket says 172.21.15.112:60710) 2024-09-21T00:26:46.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.981+0000 7f591e99c640 1 -- 172.21.15.112:0/567662298 learned_addr learned my addr 172.21.15.112:0/567662298 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:26:46.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.981+0000 7f591e99c640 1 -- 172.21.15.112:0/567662298 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d206afc3b0 msgr2=0x55d206ee5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:26:46.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.981+0000 7f591d99a640 1 --2- 172.21.15.112:0/567662298 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d206b04590 0x55d206ee5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:46.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.981+0000 7f591e99c640 1 --2- 172.21.15.112:0/567662298 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d206afc3b0 0x55d206ee5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:46.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.981+0000 7f591e99c640 1 -- 172.21.15.112:0/567662298 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d206b04590 msgr2=0x55d206ee5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:26:46.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.981+0000 7f591e99c640 1 --2- 172.21.15.112:0/567662298 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d206b04590 0x55d206ee5da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:46.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.981+0000 7f591e99c640 1 -- 172.21.15.112:0/567662298 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d206f0c4e0 con 0x55d206b25270 2024-09-21T00:26:46.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.981+0000 7f591e99c640 1 --2- 172.21.15.112:0/567662298 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d206b25270 0x55d206f34ab0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f59100838b0 tx=0x7f59100095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2dc1dba56d9cc11 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:46.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.981+0000 7f58fb7fe640 1 -- 172.21.15.112:0/567662298 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5910083920 con 0x55d206b25270 2024-09-21T00:26:46.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.981+0000 7f58fb7fe640 1 -- 172.21.15.112:0/567662298 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5910088350 con 0x55d206b25270 2024-09-21T00:26:46.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.981+0000 7f58fb7fe640 1 -- 172.21.15.112:0/567662298 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f591008d200 con 0x55d206b25270 2024-09-21T00:26:46.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.982+0000 7f5927443740 1 -- 172.21.15.112:0/567662298 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d206b25270 msgr2=0x55d206f34ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:46.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.982+0000 7f5927443740 1 --2- 172.21.15.112:0/567662298 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d206b25270 0x55d206f34ab0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f59100838b0 tx=0x7f59100095c0 comp rx=0 tx=0).stop 2024-09-21T00:26:46.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.982+0000 7f5927443740 1 -- 172.21.15.112:0/567662298 shutdown_connections 2024-09-21T00:26:46.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.982+0000 7f5927443740 1 --2- 172.21.15.112:0/567662298 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d206b25270 0x55d206f34ab0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:46.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.982+0000 7f5927443740 1 --2- 172.21.15.112:0/567662298 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d206b04590 0x55d206ee5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:46.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.982+0000 7f5927443740 1 --2- 172.21.15.112:0/567662298 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d206afc3b0 0x55d206ee5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:46.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.982+0000 7f5927443740 1 -- 172.21.15.112:0/567662298 >> 172.21.15.112:0/567662298 conn(0x55d206b193c0 msgr2=0x55d206ec8770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:46.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.982+0000 7f5927443740 1 -- 172.21.15.112:0/567662298 shutdown_connections 2024-09-21T00:26:46.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.982+0000 7f5927443740 1 -- 172.21.15.112:0/567662298 wait complete. 2024-09-21T00:26:46.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.983+0000 7f5927443740 1 -- start start 2024-09-21T00:26:46.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.983+0000 7f5927443740 1 Processor -- start 2024-09-21T00:26:46.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.984+0000 7f5927443740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d206afc3b0 0x55d206f8f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:46.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.984+0000 7f5927443740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d206b04590 0x55d206f10ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:46.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.984+0000 7f5927443740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d206b25270 0x55d206f11020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:46.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.984+0000 7f5927443740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d206ee30c0 con 0x55d206afc3b0 2024-09-21T00:26:46.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.984+0000 7f5927443740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55d206ee34d0 con 0x55d206b04590 2024-09-21T00:26:46.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.984+0000 7f5927443740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x55d206f0ad00 con 0x55d206b25270 2024-09-21T00:26:46.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.984+0000 7f5927443740 10 client.0 _ll_register_callbacks cb 0x7f5914011bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:26:46.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.984+0000 7f591e19b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d206afc3b0 0x55d206f8f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:46.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.984+0000 7f591e19b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d206afc3b0 0x55d206f8f920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60722/0 (socket says 172.21.15.112:60722) 2024-09-21T00:26:46.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.984+0000 7f591e19b640 1 -- 172.21.15.112:0/2147029368 learned_addr learned my addr 172.21.15.112:0/2147029368 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:26:46.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.984+0000 7f591d99a640 1 --2- 172.21.15.112:0/2147029368 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d206b04590 0x55d206f10ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:46.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.984+0000 7f591e99c640 1 --2- 172.21.15.112:0/2147029368 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d206b25270 0x55d206f11020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:46.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.985+0000 7f591e19b640 1 -- 172.21.15.112:0/2147029368 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d206b25270 msgr2=0x55d206f11020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:26:46.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.985+0000 7f591e19b640 1 --2- 172.21.15.112:0/2147029368 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d206b25270 0x55d206f11020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:46.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.985+0000 7f591e19b640 1 -- 172.21.15.112:0/2147029368 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d206b04590 msgr2=0x55d206f10ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:26:46.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.985+0000 7f591e19b640 1 --2- 172.21.15.112:0/2147029368 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d206b04590 0x55d206f10ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:46.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.985+0000 7f591e19b640 1 -- 172.21.15.112:0/2147029368 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d206ee6500 con 0x55d206afc3b0 2024-09-21T00:26:46.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.985+0000 7f591e19b640 1 --2- 172.21.15.112:0/2147029368 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d206afc3b0 0x55d206f8f920 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f591401d860 tx=0x7f591401d890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4179729a12fbb02 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:46.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.985+0000 7f58f97fa640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T00:26:46.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.985+0000 7f58f97fa640 1 -- 172.21.15.112:0/2147029368 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59140043e0 con 0x55d206afc3b0 2024-09-21T00:26:46.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.985+0000 7f58f97fa640 1 -- 172.21.15.112:0/2147029368 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f5914023040 con 0x55d206afc3b0 2024-09-21T00:26:46.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.985+0000 7f5927443740 1 -- 172.21.15.112:0/2147029368 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5910088350 con 0x55d206afc3b0 2024-09-21T00:26:46.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.985+0000 7f58f97fa640 1 -- 172.21.15.112:0/2147029368 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f591401dcc0 con 0x55d206afc3b0 2024-09-21T00:26:46.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.985+0000 7f5927443740 10 client.5217 fetch_fsmap learned FSMap version 17 2024-09-21T00:26:46.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.985+0000 7f5927443740 1 -- 172.21.15.112:0/2147029368 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x55d206ee2090 con 0x55d206afc3b0 2024-09-21T00:26:46.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.985+0000 7f58f97fa640 1 -- 172.21.15.112:0/2147029368 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5910088350 con 0x55d206afc3b0 2024-09-21T00:26:46.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.985+0000 7f58f97fa640 1 -- 172.21.15.112:0/2147029368 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 17) ==== 3502+0+0 (secure 0 0 0) 0x7f591402f070 con 0x55d206afc3b0 2024-09-21T00:26:46.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.986+0000 7f5927443740 10 client.5217 fetch_fsmap finished waiting for FSMap version 17 2024-09-21T00:26:46.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.986+0000 7f5927443740 10 client.5217 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-21T00:26:46.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.986+0000 7f5927443740 20 client.5217 populate_metadata read hostname 'smithi112' 2024-09-21T00:26:46.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.986+0000 7f5927443740 1 --2- 172.21.15.112:0/2147029368 >> [v2:172.21.15.112:6833/4012316979,v1:172.21.15.112:6835/4012316979] conn(0x55d206f20580 0x55d206f2a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:46.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.986+0000 7f5927443740 4 client.5217 mds_command: new command op to 5028 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T00:26:46.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.986+0000 7f5927443740 1 -- 172.21.15.112:0/2147029368 --> [v2:172.21.15.112:6833/4012316979,v1:172.21.15.112:6835/4012316979] -- command(tid 0: {"prefix": "status"}) -- 0x55d206f0c140 con 0x55d206f20580 2024-09-21T00:26:46.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.986+0000 7f591d99a640 1 --2- 172.21.15.112:0/2147029368 >> [v2:172.21.15.112:6833/4012316979,v1:172.21.15.112:6835/4012316979] conn(0x55d206f20580 0x55d206f2a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:46.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.986+0000 7f58f97fa640 10 client.5217 ms_handle_connect on v2:172.21.15.112:6833/4012316979 2024-09-21T00:26:46.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.986+0000 7f591d99a640 1 --2- 172.21.15.112:0/2147029368 >> [v2:172.21.15.112:6833/4012316979,v1:172.21.15.112:6835/4012316979] conn(0x55d206f20580 0x55d206f2a0a0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ed2ee5c18f40693 server_cookie=d8c019fb74d6a405 in_seq=0 out_seq=0 2024-09-21T00:26:46.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.986+0000 7f58f97fa640 1 -- 172.21.15.112:0/2147029368 <== mds.0 v2:172.21.15.112:6833/4012316979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x55d206f0c140 con 0x55d206f20580 2024-09-21T00:26:46.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.986+0000 7f58f97fa640 10 client.5217 handle_command_reply: tid=0 2024-09-21T00:26:46.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.986+0000 7f5927443740 1 client.5217 shutdown 2024-09-21T00:26:46.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.986+0000 7f5927443740 1 -- 172.21.15.112:0/2147029368 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d206afc3b0 msgr2=0x55d206f8f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:46.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.986+0000 7f5927443740 1 --2- 172.21.15.112:0/2147029368 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d206afc3b0 0x55d206f8f920 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f591401d860 tx=0x7f591401d890 comp rx=0 tx=0).stop 2024-09-21T00:26:46.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.987+0000 7f5927443740 1 -- 172.21.15.112:0/2147029368 shutdown_connections 2024-09-21T00:26:46.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.987+0000 7f5927443740 1 --2- 172.21.15.112:0/2147029368 >> [v2:172.21.15.112:6833/4012316979,v1:172.21.15.112:6835/4012316979] conn(0x55d206f20580 0x55d206f2a0a0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:46.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.987+0000 7f5927443740 1 --2- 172.21.15.112:0/2147029368 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d206b25270 0x55d206f11020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:46.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.987+0000 7f5927443740 1 --2- 172.21.15.112:0/2147029368 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d206b04590 0x55d206f10ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:46.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.987+0000 7f5927443740 1 --2- 172.21.15.112:0/2147029368 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d206afc3b0 0x55d206f8f920 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:46.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.987+0000 7f58f97fa640 0 client.5217 ms_handle_reset on v2:172.21.15.112:6833/4012316979 2024-09-21T00:26:46.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.987+0000 7f5927443740 1 -- 172.21.15.112:0/2147029368 >> 172.21.15.112:0/2147029368 conn(0x55d206b193c0 msgr2=0x55d206ec6960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:46.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.987+0000 7f5927443740 1 -- 172.21.15.112:0/2147029368 shutdown_connections 2024-09-21T00:26:46.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.987+0000 7f5927443740 1 -- 172.21.15.112:0/2147029368 wait complete. 2024-09-21T00:26:46.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.987+0000 7f5927443740 20 client.5217 trim_cache size 0 max 16384 2024-09-21T00:26:46.993 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-21T00:26:46.993 INFO:teuthology.orchestra.run.smithi112.stdout: "cluster_fsid": "8b34692b-dc65-49b2-a08f-26f707b96ac5", 2024-09-21T00:26:46.993 INFO:teuthology.orchestra.run.smithi112.stdout: "whoami": 0, 2024-09-21T00:26:46.993 INFO:teuthology.orchestra.run.smithi112.stdout: "id": 5028, 2024-09-21T00:26:46.993 INFO:teuthology.orchestra.run.smithi112.stdout: "want_state": "up:active", 2024-09-21T00:26:46.993 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "up:active", 2024-09-21T00:26:46.993 INFO:teuthology.orchestra.run.smithi112.stdout: "fs_name": "cephfs", 2024-09-21T00:26:46.993 INFO:teuthology.orchestra.run.smithi112.stdout: "rank_uptime": 10.167170836, 2024-09-21T00:26:46.993 INFO:teuthology.orchestra.run.smithi112.stdout: "mdsmap_epoch": 17, 2024-09-21T00:26:46.993 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch": 38, 2024-09-21T00:26:46.993 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch_barrier": 38, 2024-09-21T00:26:46.993 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 17.039020576999999 2024-09-21T00:26:46.994 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-21T00:26:46.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.987+0000 7f591f19d640 1 -- 172.21.15.112:0/839678220 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f58fc078bc0 msgr2=0x7f58fc07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:46.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.987+0000 7f591f19d640 1 --2- 172.21.15.112:0/839678220 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f58fc078bc0 0x7f58fc07b0d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f59140095f0 tx=0x7f5914005f90 comp rx=0 tx=0).stop 2024-09-21T00:26:46.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.987+0000 7f591f19d640 1 -- 172.21.15.112:0/839678220 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f591815f6d0 msgr2=0x7f59181607b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:46.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.987+0000 7f591f19d640 1 --2- 172.21.15.112:0/839678220 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f591815f6d0 0x7f59181607b0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f5910009bf0 tx=0x7f5910003c10 comp rx=0 tx=0).stop 2024-09-21T00:26:46.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.987+0000 7f591f19d640 1 -- 172.21.15.112:0/839678220 shutdown_connections 2024-09-21T00:26:46.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.987+0000 7f591f19d640 1 --2- 172.21.15.112:0/839678220 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f58fc078bc0 0x7f58fc07b0d0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:46.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.987+0000 7f591f19d640 1 --2- 172.21.15.112:0/839678220 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f591815f6d0 0x7f59181607b0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:46.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.987+0000 7f591f19d640 1 --2- 172.21.15.112:0/839678220 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f59180d2290 0x7f5918159e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:46.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.987+0000 7f591f19d640 1 --2- 172.21.15.112:0/839678220 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f59180cef00 0x7f591814c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:46.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.987+0000 7f591f19d640 1 -- 172.21.15.112:0/839678220 >> 172.21.15.112:0/839678220 conn(0x7f59180d41e0 msgr2=0x7f59180d45f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:46.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.987+0000 7f591f19d640 1 -- 172.21.15.112:0/839678220 shutdown_connections 2024-09-21T00:26:46.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:46.987+0000 7f591f19d640 1 -- 172.21.15.112:0/839678220 wait complete. 2024-09-21T00:26:46.999 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T00:26:47.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:47.110+0000 7f1106bad640 1 -- 172.21.15.112:0/3700371475 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11000d1d00 msgr2=0x7f11000cc930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:47.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:47.110+0000 7f1106bad640 1 --2- 172.21.15.112:0/3700371475 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11000d1d00 0x7f11000cc930 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f10f0005fd0 tx=0x7f10f002a520 comp rx=0 tx=0).stop 2024-09-21T00:26:47.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:47.110+0000 7f1106bad640 1 -- 172.21.15.112:0/3700371475 shutdown_connections 2024-09-21T00:26:47.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:47.110+0000 7f1106bad640 1 --2- 172.21.15.112:0/3700371475 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11000d1d00 0x7f11000cc930 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:47.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:47.110+0000 7f1106bad640 1 --2- 172.21.15.112:0/3700371475 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f11001536e0 0x7f11000d17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:47.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:47.110+0000 7f1106bad640 1 -- 172.21.15.112:0/3700371475 >> 172.21.15.112:0/3700371475 conn(0x7f11000cef40 msgr2=0x7f11000cf350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:47.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:47.110+0000 7f1106bad640 1 -- 172.21.15.112:0/3700371475 shutdown_connections 2024-09-21T00:26:47.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:47.110+0000 7f1106bad640 1 -- 172.21.15.112:0/3700371475 wait complete. 2024-09-21T00:26:47.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:47.111+0000 7f1106bad640 1 Processor -- start 2024-09-21T00:26:47.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:47.111+0000 7f1106bad640 1 -- start start 2024-09-21T00:26:47.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:47.113+0000 7f1106bad640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f11000cce70 0x7f110014b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:47.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:47.113+0000 7f1106bad640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f11000d1d00 0x7f11001567c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:47.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:47.113+0000 7f1106bad640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11001536e0 0x7f11001f2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:47.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:47.113+0000 7f1106bad640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1100160a20 con 0x7f11001536e0 2024-09-21T00:26:47.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:47.113+0000 7f1106bad640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f11001608a0 con 0x7f11000d1d00 2024-09-21T00:26:47.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:47.113+0000 7f1106bad640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f1100160ba0 con 0x7f11000cce70 2024-09-21T00:26:47.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:47.113+0000 7f110514a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11001536e0 0x7f11001f2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:47.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:47.113+0000 7f110514a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11001536e0 0x7f11001f2eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60754/0 (socket says 172.21.15.112:60754) 2024-09-21T00:26:47.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:47.113+0000 7f110514a640 1 -- 172.21.15.112:0/2785343838 learned_addr learned my addr 172.21.15.112:0/2785343838 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:26:47.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:47.113+0000 7f1104949640 1 --2- 172.21.15.112:0/2785343838 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f11000cce70 0x7f110014b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:47.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:47.113+0000 7f10f7fff640 1 --2- 172.21.15.112:0/2785343838 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f11000d1d00 0x7f11001567c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:47.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:47.113+0000 7f110514a640 1 -- 172.21.15.112:0/2785343838 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f11000cce70 msgr2=0x7f110014b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:47.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:47.113+0000 7f110514a640 1 --2- 172.21.15.112:0/2785343838 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f11000cce70 0x7f110014b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:47.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:47.113+0000 7f110514a640 1 -- 172.21.15.112:0/2785343838 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f11000d1d00 msgr2=0x7f11001567c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:47.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:47.113+0000 7f110514a640 1 --2- 172.21.15.112:0/2785343838 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f11000d1d00 0x7f11001567c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:47.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:47.113+0000 7f110514a640 1 -- 172.21.15.112:0/2785343838 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f110014da90 con 0x7f11001536e0 2024-09-21T00:26:47.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:47.113+0000 7f110514a640 1 --2- 172.21.15.112:0/2785343838 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11001536e0 0x7f11001f2eb0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f10f800b9c0 tx=0x7f10f800be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:47.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:47.114+0000 7f10f57fa640 1 -- 172.21.15.112:0/2785343838 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10f8019070 con 0x7f11001536e0 2024-09-21T00:26:47.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:47.114+0000 7f10f57fa640 1 -- 172.21.15.112:0/2785343838 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f10f800cc90 con 0x7f11001536e0 2024-09-21T00:26:47.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:47.114+0000 7f1106bad640 1 -- 172.21.15.112:0/2785343838 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f110014dd30 con 0x7f11001536e0 2024-09-21T00:26:47.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:47.114+0000 7f10f57fa640 1 -- 172.21.15.112:0/2785343838 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10f80146a0 con 0x7f11001536e0 2024-09-21T00:26:47.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:47.114+0000 7f1106bad640 1 -- 172.21.15.112:0/2785343838 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11000b6c80 con 0x7f11001536e0 2024-09-21T00:26:47.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:47.115+0000 7f1106bad640 1 -- 172.21.15.112:0/2785343838 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11000cc930 con 0x7f11001536e0 2024-09-21T00:26:47.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:47.115+0000 7f10f57fa640 1 -- 172.21.15.112:0/2785343838 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f10f801c070 con 0x7f11001536e0 2024-09-21T00:26:47.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:47.116+0000 7f10f57fa640 1 --2- 172.21.15.112:0/2785343838 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f10ec078b90 0x7f10ec07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:47.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:47.116+0000 7f10f57fa640 1 -- 172.21.15.112:0/2785343838 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f10f809ab40 con 0x7f11001536e0 2024-09-21T00:26:47.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:47.116+0000 7f1104949640 1 --2- 172.21.15.112:0/2785343838 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f10ec078b90 0x7f10ec07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:47.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:47.116+0000 7f1104949640 1 --2- 172.21.15.112:0/2785343838 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f10ec078b90 0x7f10ec07b0a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f10fc0098e0 tx=0x7f10fc009060 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:47.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:47.120+0000 7f10f57fa640 1 -- 172.21.15.112:0/2785343838 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f10f8062830 con 0x7f11001536e0 2024-09-21T00:26:47.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:47.334+0000 7f1106bad640 1 -- 172.21.15.112:0/2785343838 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f11000cf820 con 0x7f11001536e0 2024-09-21T00:26:47.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:47.334+0000 7f10f57fa640 1 -- 172.21.15.112:0/2785343838 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+73 (secure 0 0 0) 0x7f10f8067260 con 0x7f11001536e0 2024-09-21T00:26:47.335 INFO:teuthology.orchestra.run.smithi112.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T00:26:47.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:47.336+0000 7f1106bad640 1 -- 172.21.15.112:0/2785343838 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f10ec078b90 msgr2=0x7f10ec07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:47.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:47.336+0000 7f1106bad640 1 --2- 172.21.15.112:0/2785343838 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f10ec078b90 0x7f10ec07b0a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f10fc0098e0 tx=0x7f10fc009060 comp rx=0 tx=0).stop 2024-09-21T00:26:47.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:47.336+0000 7f1106bad640 1 -- 172.21.15.112:0/2785343838 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11001536e0 msgr2=0x7f11001f2eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:47.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:47.336+0000 7f1106bad640 1 --2- 172.21.15.112:0/2785343838 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11001536e0 0x7f11001f2eb0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f10f800b9c0 tx=0x7f10f800be80 comp rx=0 tx=0).stop 2024-09-21T00:26:47.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:47.336+0000 7f1106bad640 1 -- 172.21.15.112:0/2785343838 shutdown_connections 2024-09-21T00:26:47.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:47.336+0000 7f1106bad640 1 --2- 172.21.15.112:0/2785343838 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f10ec078b90 0x7f10ec07b0a0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:47.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:47.336+0000 7f1106bad640 1 --2- 172.21.15.112:0/2785343838 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11001536e0 0x7f11001f2eb0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:47.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:47.336+0000 7f1106bad640 1 --2- 172.21.15.112:0/2785343838 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f11000d1d00 0x7f11001567c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:47.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:47.336+0000 7f1106bad640 1 --2- 172.21.15.112:0/2785343838 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f11000cce70 0x7f110014b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:47.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:47.336+0000 7f1106bad640 1 -- 172.21.15.112:0/2785343838 >> 172.21.15.112:0/2785343838 conn(0x7f11000cef40 msgr2=0x7f1100071220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:47.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:47.336+0000 7f1106bad640 1 -- 172.21.15.112:0/2785343838 shutdown_connections 2024-09-21T00:26:47.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:47.336+0000 7f1106bad640 1 -- 172.21.15.112:0/2785343838 wait complete. 2024-09-21T00:26:47.348 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T00:26:47.349 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T00:26:47.349 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi112.front.sepia.ceph.com 2024-09-21T00:26:47.349 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T00:26:47.349 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T00:26:47.349 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T00:26:47.349 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-21T00:26:47.349 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T00:26:47.349 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T00:26:47.349 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:26:47.349 DEBUG:teuthology.orchestra.run.smithi112:> ip addr 2024-09-21T00:26:47.365 INFO:teuthology.orchestra.run.smithi112.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T00:26:47.365 INFO:teuthology.orchestra.run.smithi112.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T00:26:47.365 INFO:teuthology.orchestra.run.smithi112.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T00:26:47.365 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:26:47.365 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 ::1/128 scope host 2024-09-21T00:26:47.366 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:26:47.366 INFO:teuthology.orchestra.run.smithi112.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T00:26:47.366 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:d8:a5:2c brd ff:ff:ff:ff:ff:ff 2024-09-21T00:26:47.366 INFO:teuthology.orchestra.run.smithi112.stdout: altname enp7s0f0 2024-09-21T00:26:47.366 INFO:teuthology.orchestra.run.smithi112.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T00:26:47.366 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:d8:a5:2d brd ff:ff:ff:ff:ff:ff 2024-09-21T00:26:47.366 INFO:teuthology.orchestra.run.smithi112.stdout: altname enp7s0f1 2024-09-21T00:26:47.366 INFO:teuthology.orchestra.run.smithi112.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T00:26:47.366 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:8f:cb:68 brd ff:ff:ff:ff:ff:ff 2024-09-21T00:26:47.366 INFO:teuthology.orchestra.run.smithi112.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T00:26:47.366 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:8f:cb:69 brd ff:ff:ff:ff:ff:ff 2024-09-21T00:26:47.366 INFO:teuthology.orchestra.run.smithi112.stdout: inet 172.21.15.112/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-21T00:26:47.366 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft 42715sec preferred_lft 42715sec 2024-09-21T00:26:47.367 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::ec4:7aff:fe8f:cb69/64 scope link noprefixroute 2024-09-21T00:26:47.367 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:26:47.367 INFO:teuthology.orchestra.run.smithi112.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T00:26:47.367 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 1a:39:64:e7:fe:d7 brd ff:ff:ff:ff:ff:ff 2024-09-21T00:26:47.367 INFO:teuthology.orchestra.run.smithi112.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T00:26:47.367 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:26:47.367 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::5858:3dff:feaf:1774/64 scope link 2024-09-21T00:26:47.367 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:26:47.367 INFO:teuthology.orchestra.run.smithi112.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T00:26:47.367 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 1a:39:64:e7:fe:d7 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T00:26:47.367 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::1839:64ff:fee7:fed7/64 scope link 2024-09-21T00:26:47.367 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:26:47.367 INFO:teuthology.orchestra.run.smithi112.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T00:26:47.367 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 86:c9:6c:ef:e4:e9 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T00:26:47.368 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::84c9:6cff:feef:e4e9/64 scope link 2024-09-21T00:26:47.368 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:26:47.368 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:26:47.368 DEBUG:teuthology.orchestra.run.smithi112:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T00:26:47.428 INFO:teuthology.orchestra.run.smithi112.stdout:1 2024-09-21T00:26:47.430 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:26:47.430 DEBUG:teuthology.orchestra.run.smithi112:> route 2024-09-21T00:26:47.486 INFO:teuthology.orchestra.run.smithi112.stdout:Kernel IP routing table 2024-09-21T00:26:47.487 INFO:teuthology.orchestra.run.smithi112.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T00:26:47.487 INFO:teuthology.orchestra.run.smithi112.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-21T00:26:47.487 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-21T00:26:47.487 INFO:teuthology.orchestra.run.smithi112.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T00:26:47.487 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T00:26:47.487 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2024-09-21T00:26:47.487 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-21T00:26:47.487 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-21T00:26:47.487 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-21T00:26:47.488 DEBUG:teuthology.orchestra.run.smithi112:> ') 2024-09-21T00:26:47.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:47 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T00:26:47.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T00:26:47.612 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:26:47.612 DEBUG:teuthology.orchestra.run.smithi112:> ip netns list 2024-09-21T00:26:47.628 INFO:teuthology.orchestra.run.smithi112.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-21T00:26:47.628 INFO:teuthology.orchestra.run.smithi112.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T00:26:47.628 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T00:26:47.628 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:26:47.628 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link set brx.0 up 2024-09-21T00:26:47.693 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T00:26:47.693 DEBUG:teuthology.orchestra.run.smithi112:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T00:26:47.749 INFO:teuthology.orchestra.run.smithi112.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T00:26:47.749 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T00:26:47.750 DEBUG:teuthology.orchestra.run.smithi112:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T00:26:47.804 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-21T00:26:47.804 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:26:47.804 DEBUG:teuthology.orchestra.run.smithi112:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T00:26:47.898 INFO:teuthology.orchestra.run.smithi112.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T00:26:47.898 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T00:26:47.898 INFO:teuthology.orchestra.run.smithi112.stdout:invalid new device string format 2024-09-21T00:26:47.909 INFO:teuthology.orchestra.run.smithi112.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T00:26:47.909 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T00:26:47.909 INFO:teuthology.orchestra.run.smithi112.stdout:invalid new device string format 2024-09-21T00:26:47.909 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: resolved to: "172.21.15.112:3300,172.21.15.187:3300,172.21.15.187:3301" 2024-09-21T00:26:47.909 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T00:26:47.916 INFO:teuthology.orchestra.run.smithi112.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T00:26:47.916 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T00:26:47.916 INFO:teuthology.orchestra.run.smithi112.stdout:invalid new device string format 2024-09-21T00:26:47.916 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: resolved to: "172.21.15.112:3300,172.21.15.187:3300,172.21.15.187:3301" 2024-09-21T00:26:47.916 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T00:26:47.916 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: resolved to: "172.21.15.112:6789,172.21.15.187:6789,172.21.15.187:6790" 2024-09-21T00:26:47.916 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.112:6789,172.21.15.187:6789,172.21.15.187:6790:/ 2024-09-21T00:26:47.916 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=8b34692b-dc65-49b2-a08f-26f707b96ac5" will pass to kernel 2024-09-21T00:26:47.916 INFO:teuthology.orchestra.run.smithi112.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-21T00:26:47.917 INFO:teuthology.orchestra.run.smithi112.stdout: You just mounted a file system that supports labels which does not 2024-09-21T00:26:47.917 INFO:teuthology.orchestra.run.smithi112.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T00:26:47.917 INFO:teuthology.orchestra.run.smithi112.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T00:26:47.917 INFO:teuthology.orchestra.run.smithi112.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T00:26:47.918 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T00:26:47.918 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:26:47.918 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T00:26:47.943 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T00:26:48.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:48 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T00:26:48.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T00:26:48.043 INFO:teuthology.orchestra.run.smithi112.stdout:client4859 2024-09-21T00:26:48.044 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T00:26:48.044 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:26:48.060 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T00:26:48.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:48.213+0000 7f16f95ed640 1 -- 172.21.15.112:0/503185103 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16dc005910 msgr2=0x7f16dc005d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:48.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:48.213+0000 7f16f95ed640 1 --2- 172.21.15.112:0/503185103 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16dc005910 0x7f16dc005d20 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f16e8001760 tx=0x7f16dc006940 comp rx=0 tx=0).stop 2024-09-21T00:26:48.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:48.213+0000 7f16f37fe640 1 -- 172.21.15.112:0/503185103 reap_dead start 2024-09-21T00:26:48.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:48.213+0000 7f16f95ed640 1 -- 172.21.15.112:0/503185103 shutdown_connections 2024-09-21T00:26:48.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:48.213+0000 7f16f95ed640 1 -- 172.21.15.112:0/503185103 >> 172.21.15.112:0/503185103 conn(0x7f16f40cca40 msgr2=0x7f16f40cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:48.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:48.213+0000 7f16f95ed640 1 -- 172.21.15.112:0/503185103 shutdown_connections 2024-09-21T00:26:48.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:48.213+0000 7f16f95ed640 1 -- 172.21.15.112:0/503185103 wait complete. 2024-09-21T00:26:48.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:48.215+0000 7f16f95ed640 1 Processor -- start 2024-09-21T00:26:48.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:48.215+0000 7f16f95ed640 1 -- start start 2024-09-21T00:26:48.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:48.216+0000 7f16f95ed640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16dc005910 0x7f16f40b2e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:48.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:48.216+0000 7f16f95ed640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f16f415ae60 0x7f16f41f29a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:48.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:48.216+0000 7f16f95ed640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f16f40a0e60 0x7f16f40c5b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:48.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:48.216+0000 7f16f95ed640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f16f415ace0 con 0x7f16dc005910 2024-09-21T00:26:48.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:48.216+0000 7f16f95ed640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f16f406e710 con 0x7f16f40a0e60 2024-09-21T00:26:48.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:48.216+0000 7f16f95ed640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f16f4070e80 con 0x7f16f415ae60 2024-09-21T00:26:48.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:48.216+0000 7f16f2ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16dc005910 0x7f16f40b2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:48.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:48.216+0000 7f16f2ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16dc005910 0x7f16f40b2e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60778/0 (socket says 172.21.15.112:60778) 2024-09-21T00:26:48.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:48.216+0000 7f16f2ffd640 1 -- 172.21.15.112:0/783499395 learned_addr learned my addr 172.21.15.112:0/783499395 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:26:48.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:48.216+0000 7f16f27fc640 1 --2- 172.21.15.112:0/783499395 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f16f415ae60 0x7f16f41f29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:48.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:48.216+0000 7f16f37fe640 1 --2- 172.21.15.112:0/783499395 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f16f40a0e60 0x7f16f40c5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:48.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:48.216+0000 7f16f2ffd640 1 -- 172.21.15.112:0/783499395 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f16f415ae60 msgr2=0x7f16f41f29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:48.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:48.216+0000 7f16f2ffd640 1 --2- 172.21.15.112:0/783499395 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f16f415ae60 0x7f16f41f29a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:48.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:48.216+0000 7f16f2ffd640 1 -- 172.21.15.112:0/783499395 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f16f40a0e60 msgr2=0x7f16f40c5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:48.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:48.216+0000 7f16f2ffd640 1 --2- 172.21.15.112:0/783499395 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f16f40a0e60 0x7f16f40c5b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:48.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:48.216+0000 7f16f2ffd640 1 -- 172.21.15.112:0/783499395 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16f40b3640 con 0x7f16dc005910 2024-09-21T00:26:48.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:48.216+0000 7f16f37fe640 1 --2- 172.21.15.112:0/783499395 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f16f40a0e60 0x7f16f40c5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:26:48.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:48.216+0000 7f16f27fc640 1 --2- 172.21.15.112:0/783499395 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f16f415ae60 0x7f16f41f29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:26:48.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:48.217+0000 7f16f2ffd640 1 --2- 172.21.15.112:0/783499395 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16dc005910 0x7f16f40b2e10 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f16e4001ec0 tx=0x7f16e400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:48.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:48.217+0000 7f16db7fe640 1 -- 172.21.15.112:0/783499395 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16e4015070 con 0x7f16dc005910 2024-09-21T00:26:48.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:48.217+0000 7f16db7fe640 1 -- 172.21.15.112:0/783499395 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f16e400cdd0 con 0x7f16dc005910 2024-09-21T00:26:48.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:48.217+0000 7f16f95ed640 1 -- 172.21.15.112:0/783499395 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16f40b3920 con 0x7f16dc005910 2024-09-21T00:26:48.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:48.217+0000 7f16db7fe640 1 -- 172.21.15.112:0/783499395 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16e40116c0 con 0x7f16dc005910 2024-09-21T00:26:48.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:48.217+0000 7f16f95ed640 1 -- 172.21.15.112:0/783499395 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16f415b2a0 con 0x7f16dc005910 2024-09-21T00:26:48.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:48.218+0000 7f16f95ed640 1 -- 172.21.15.112:0/783499395 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16f415b500 con 0x7f16dc005910 2024-09-21T00:26:48.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:48.219+0000 7f16db7fe640 1 -- 172.21.15.112:0/783499395 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f16e4018070 con 0x7f16dc005910 2024-09-21T00:26:48.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:48.219+0000 7f16db7fe640 1 --2- 172.21.15.112:0/783499395 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f16e0078b90 0x7f16e007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:48.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:48.219+0000 7f16db7fe640 1 -- 172.21.15.112:0/783499395 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f16e409b5b0 con 0x7f16dc005910 2024-09-21T00:26:48.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:48.219+0000 7f16f27fc640 1 --2- 172.21.15.112:0/783499395 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f16e0078b90 0x7f16e007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:48.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:48.220+0000 7f16f27fc640 1 --2- 172.21.15.112:0/783499395 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f16e0078b90 0x7f16e007b0a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f16e8001730 tx=0x7f16e80012c0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:48.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:48.224+0000 7f16db7fe640 1 -- 172.21.15.112:0/783499395 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f16e4063980 con 0x7f16dc005910 2024-09-21T00:26:48.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:48.437+0000 7f16f95ed640 1 -- 172.21.15.112:0/783499395 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f16f406ddf0 con 0x7f16dc005910 2024-09-21T00:26:48.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:48.438+0000 7f16db7fe640 1 -- 172.21.15.112:0/783499395 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+73 (secure 0 0 0) 0x7f16e4063280 con 0x7f16dc005910 2024-09-21T00:26:48.438 INFO:teuthology.orchestra.run.smithi112.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T00:26:48.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:48.440+0000 7f16f95ed640 1 -- 172.21.15.112:0/783499395 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f16e0078b90 msgr2=0x7f16e007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:48.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:48.440+0000 7f16f95ed640 1 --2- 172.21.15.112:0/783499395 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f16e0078b90 0x7f16e007b0a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f16e8001730 tx=0x7f16e80012c0 comp rx=0 tx=0).stop 2024-09-21T00:26:48.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:48.440+0000 7f16f95ed640 1 -- 172.21.15.112:0/783499395 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16dc005910 msgr2=0x7f16f40b2e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:48.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:48.440+0000 7f16f95ed640 1 --2- 172.21.15.112:0/783499395 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16dc005910 0x7f16f40b2e10 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f16e4001ec0 tx=0x7f16e400ee20 comp rx=0 tx=0).stop 2024-09-21T00:26:48.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:48.440+0000 7f16f95ed640 1 -- 172.21.15.112:0/783499395 shutdown_connections 2024-09-21T00:26:48.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:48.440+0000 7f16f95ed640 1 --2- 172.21.15.112:0/783499395 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f16f40a0e60 0x7f16f40c5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:48.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:48.440+0000 7f16f95ed640 1 --2- 172.21.15.112:0/783499395 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f16f415ae60 0x7f16f41f29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:48.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:48.440+0000 7f16f95ed640 1 --2- 172.21.15.112:0/783499395 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f16e0078b90 0x7f16e007b0a0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:48.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:48.440+0000 7f16f95ed640 1 --2- 172.21.15.112:0/783499395 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16dc005910 0x7f16f40b2e10 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:48.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:48.440+0000 7f16f95ed640 1 -- 172.21.15.112:0/783499395 >> 172.21.15.112:0/783499395 conn(0x7f16f40cca40 msgr2=0x7f16f40cd000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:48.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:48.440+0000 7f16f95ed640 1 -- 172.21.15.112:0/783499395 shutdown_connections 2024-09-21T00:26:48.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:48.440+0000 7f16f95ed640 1 -- 172.21.15.112:0/783499395 wait complete. 2024-09-21T00:26:48.452 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T00:26:48.452 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T00:26:48.452 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi187.front.sepia.ceph.com 2024-09-21T00:26:48.452 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-21T00:26:48.452 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-21T00:26:48.452 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T00:26:48.452 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-21T00:26:48.452 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T00:26:48.452 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-21T00:26:48.452 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:26:48.453 DEBUG:teuthology.orchestra.run.smithi187:> ip addr 2024-09-21T00:26:48.473 INFO:teuthology.orchestra.run.smithi187.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T00:26:48.473 INFO:teuthology.orchestra.run.smithi187.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T00:26:48.473 INFO:teuthology.orchestra.run.smithi187.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T00:26:48.473 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:26:48.473 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 ::1/128 scope host 2024-09-21T00:26:48.473 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:26:48.473 INFO:teuthology.orchestra.run.smithi187.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T00:26:48.473 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 0c:c4:7a:d9:36:78 brd ff:ff:ff:ff:ff:ff 2024-09-21T00:26:48.473 INFO:teuthology.orchestra.run.smithi187.stdout: altname enp7s0f0 2024-09-21T00:26:48.473 INFO:teuthology.orchestra.run.smithi187.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T00:26:48.474 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 0c:c4:7a:d9:36:79 brd ff:ff:ff:ff:ff:ff 2024-09-21T00:26:48.474 INFO:teuthology.orchestra.run.smithi187.stdout: altname enp7s0f1 2024-09-21T00:26:48.474 INFO:teuthology.orchestra.run.smithi187.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T00:26:48.474 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 0c:c4:7a:88:77:b0 brd ff:ff:ff:ff:ff:ff 2024-09-21T00:26:48.474 INFO:teuthology.orchestra.run.smithi187.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T00:26:48.474 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 0c:c4:7a:88:77:b1 brd ff:ff:ff:ff:ff:ff 2024-09-21T00:26:48.474 INFO:teuthology.orchestra.run.smithi187.stdout: inet 172.21.15.187/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-21T00:26:48.474 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft 42702sec preferred_lft 42702sec 2024-09-21T00:26:48.474 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 fe80::ec4:7aff:fe88:77b1/64 scope link noprefixroute 2024-09-21T00:26:48.474 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:26:48.474 INFO:teuthology.orchestra.run.smithi187.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T00:26:48.474 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 86:3b:f6:5b:1b:32 brd ff:ff:ff:ff:ff:ff 2024-09-21T00:26:48.474 INFO:teuthology.orchestra.run.smithi187.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T00:26:48.475 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:26:48.475 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 fe80::ac6f:97ff:fe09:b82e/64 scope link 2024-09-21T00:26:48.475 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:26:48.475 INFO:teuthology.orchestra.run.smithi187.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T00:26:48.475 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether ca:0b:a0:f5:2b:b5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T00:26:48.475 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 fe80::c80b:a0ff:fef5:2bb5/64 scope link 2024-09-21T00:26:48.475 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:26:48.475 INFO:teuthology.orchestra.run.smithi187.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T00:26:48.475 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 86:3b:f6:5b:1b:32 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-21T00:26:48.475 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 fe80::843b:f6ff:fe5b:1b32/64 scope link 2024-09-21T00:26:48.475 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:26:48.475 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:26:48.476 DEBUG:teuthology.orchestra.run.smithi187:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T00:26:48.541 INFO:teuthology.orchestra.run.smithi187.stdout:1 2024-09-21T00:26:48.543 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:26:48.543 DEBUG:teuthology.orchestra.run.smithi187:> route 2024-09-21T00:26:48.601 INFO:teuthology.orchestra.run.smithi187.stdout:Kernel IP routing table 2024-09-21T00:26:48.601 INFO:teuthology.orchestra.run.smithi187.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T00:26:48.601 INFO:teuthology.orchestra.run.smithi187.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-21T00:26:48.601 INFO:teuthology.orchestra.run.smithi187.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-21T00:26:48.601 INFO:teuthology.orchestra.run.smithi187.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T00:26:48.602 DEBUG:teuthology.orchestra.run.smithi187:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T00:26:48.602 DEBUG:teuthology.orchestra.run.smithi187:> set -e 2024-09-21T00:26:48.602 DEBUG:teuthology.orchestra.run.smithi187:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-21T00:26:48.602 DEBUG:teuthology.orchestra.run.smithi187:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-21T00:26:48.602 DEBUG:teuthology.orchestra.run.smithi187:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-21T00:26:48.602 DEBUG:teuthology.orchestra.run.smithi187:> ') 2024-09-21T00:26:48.693 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:26:48 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T00:26:48.727 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:26:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T00:26:48.733 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:26:48.733 DEBUG:teuthology.orchestra.run.smithi187:> ip netns list 2024-09-21T00:26:48.748 INFO:teuthology.orchestra.run.smithi187.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-21T00:26:48.748 INFO:teuthology.orchestra.run.smithi187.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-21T00:26:48.748 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-21T00:26:48.748 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:26:48.748 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip link set brx.0 up 2024-09-21T00:26:48.813 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T00:26:48.813 DEBUG:teuthology.orchestra.run.smithi187:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-21T00:26:48.868 INFO:teuthology.orchestra.run.smithi187.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-21T00:26:48.868 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T00:26:48.868 DEBUG:teuthology.orchestra.run.smithi187:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-21T00:26:48.923 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-21T00:26:48.923 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:26:48.923 DEBUG:teuthology.orchestra.run.smithi187:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-21T00:26:49.017 INFO:teuthology.orchestra.run.smithi187.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-21T00:26:49.017 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-21T00:26:49.017 INFO:teuthology.orchestra.run.smithi187.stdout:invalid new device string format 2024-09-21T00:26:49.028 INFO:teuthology.orchestra.run.smithi187.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-21T00:26:49.028 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-21T00:26:49.028 INFO:teuthology.orchestra.run.smithi187.stdout:invalid new device string format 2024-09-21T00:26:49.028 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: resolved to: "172.21.15.112:3300,172.21.15.187:3300,172.21.15.187:3301" 2024-09-21T00:26:49.028 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T00:26:49.034 INFO:teuthology.orchestra.run.smithi187.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-21T00:26:49.034 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-21T00:26:49.034 INFO:teuthology.orchestra.run.smithi187.stdout:invalid new device string format 2024-09-21T00:26:49.034 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: resolved to: "172.21.15.112:3300,172.21.15.187:3300,172.21.15.187:3301" 2024-09-21T00:26:49.034 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T00:26:49.034 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: resolved to: "172.21.15.112:6789,172.21.15.187:6789,172.21.15.187:6790" 2024-09-21T00:26:49.035 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.112:6789,172.21.15.187:6789,172.21.15.187:6790:/ 2024-09-21T00:26:49.035 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=8b34692b-dc65-49b2-a08f-26f707b96ac5" will pass to kernel 2024-09-21T00:26:49.035 INFO:teuthology.orchestra.run.smithi187.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-09-21T00:26:49.035 INFO:teuthology.orchestra.run.smithi187.stdout: You just mounted a file system that supports labels which does not 2024-09-21T00:26:49.035 INFO:teuthology.orchestra.run.smithi187.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T00:26:49.035 INFO:teuthology.orchestra.run.smithi187.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T00:26:49.035 INFO:teuthology.orchestra.run.smithi187.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T00:26:49.037 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T00:26:49.037 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:26:49.037 DEBUG:teuthology.orchestra.run.smithi187:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-21T00:26:49.062 DEBUG:teuthology.orchestra.run.smithi187:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T00:26:49.150 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:26:49 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T00:26:49.153 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:26:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T00:26:49.158 INFO:teuthology.orchestra.run.smithi187.stdout:client4868 2024-09-21T00:26:49.158 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-09-21T00:26:49.159 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:26:49.175 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds ms_async_reap_threshold 1 2024-09-21T00:26:49.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:49.294+0000 7fe81745c640 1 -- 172.21.15.112:0/1001782848 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe81015ee80 msgr2=0x7fe8100d0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:49.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:49.294+0000 7fe81745c640 1 --2- 172.21.15.112:0/1001782848 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe81015ee80 0x7fe8100d0170 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe80c009830 tx=0x7fe80c02a420 comp rx=0 tx=0).stop 2024-09-21T00:26:49.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:49.294+0000 7fe81745c640 1 -- 172.21.15.112:0/1001782848 shutdown_connections 2024-09-21T00:26:49.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:49.294+0000 7fe81745c640 1 --2- 172.21.15.112:0/1001782848 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe81015ee80 0x7fe8100d0170 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:49.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:49.294+0000 7fe81745c640 1 -- 172.21.15.112:0/1001782848 >> 172.21.15.112:0/1001782848 conn(0x7fe81009d830 msgr2=0x7fe8100b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:49.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:49.295+0000 7fe81745c640 1 -- 172.21.15.112:0/1001782848 shutdown_connections 2024-09-21T00:26:49.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:49.295+0000 7fe81745c640 1 -- 172.21.15.112:0/1001782848 wait complete. 2024-09-21T00:26:49.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:49.296+0000 7fe81745c640 1 Processor -- start 2024-09-21T00:26:49.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:49.296+0000 7fe81745c640 1 -- start start 2024-09-21T00:26:49.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:49.298+0000 7fe81745c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe8100b9ad0 0x7fe8101569e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:49.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:49.298+0000 7fe81745c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe8100ceb90 0x7fe8101fd050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:49.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:49.298+0000 7fe81745c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe81015ee80 0x7fe8101fd590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:49.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:49.298+0000 7fe81745c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe8100b6910 con 0x7fe8100ceb90 2024-09-21T00:26:49.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:49.298+0000 7fe81745c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe8100b6790 con 0x7fe81015ee80 2024-09-21T00:26:49.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:49.298+0000 7fe81745c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe8100b6a90 con 0x7fe8100b9ad0 2024-09-21T00:26:49.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:49.298+0000 7fe8149f7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe8100ceb90 0x7fe8101fd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:49.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:49.298+0000 7fe8149f7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe8100ceb90 0x7fe8101fd050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60794/0 (socket says 172.21.15.112:60794) 2024-09-21T00:26:49.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:49.298+0000 7fe8149f7640 1 -- 172.21.15.112:0/1738664638 learned_addr learned my addr 172.21.15.112:0/1738664638 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:26:49.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:49.298+0000 7fe8151f8640 1 --2- 172.21.15.112:0/1738664638 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe8100b9ad0 0x7fe8101569e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:49.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:49.298+0000 7fe8159f9640 1 --2- 172.21.15.112:0/1738664638 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe81015ee80 0x7fe8101fd590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:49.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:49.298+0000 7fe8149f7640 1 -- 172.21.15.112:0/1738664638 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe8100b9ad0 msgr2=0x7fe8101569e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:49.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:49.298+0000 7fe8149f7640 1 --2- 172.21.15.112:0/1738664638 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe8100b9ad0 0x7fe8101569e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:49.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:49.298+0000 7fe8149f7640 1 -- 172.21.15.112:0/1738664638 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe81015ee80 msgr2=0x7fe8101fd590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:49.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:49.298+0000 7fe8149f7640 1 --2- 172.21.15.112:0/1738664638 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe81015ee80 0x7fe8101fd590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:49.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:49.298+0000 7fe8149f7640 1 -- 172.21.15.112:0/1738664638 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8100ce880 con 0x7fe8100ceb90 2024-09-21T00:26:49.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:49.298+0000 7fe8149f7640 1 --2- 172.21.15.112:0/1738664638 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe8100ceb90 0x7fe8101fd050 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fe80000c760 tx=0x7fe80000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:49.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:49.298+0000 7fe805ffb640 1 -- 172.21.15.112:0/1738664638 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe800017070 con 0x7fe8100ceb90 2024-09-21T00:26:49.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:49.298+0000 7fe805ffb640 1 -- 172.21.15.112:0/1738664638 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe80000ce10 con 0x7fe8100ceb90 2024-09-21T00:26:49.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:49.298+0000 7fe805ffb640 1 -- 172.21.15.112:0/1738664638 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe80000a8a0 con 0x7fe8100ceb90 2024-09-21T00:26:49.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:49.298+0000 7fe81745c640 1 -- 172.21.15.112:0/1738664638 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8101fdad0 con 0x7fe8100ceb90 2024-09-21T00:26:49.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:49.299+0000 7fe81745c640 1 -- 172.21.15.112:0/1738664638 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe81006d6e0 con 0x7fe8100ceb90 2024-09-21T00:26:49.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:49.300+0000 7fe81745c640 1 -- 172.21.15.112:0/1738664638 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8100d0170 con 0x7fe8100ceb90 2024-09-21T00:26:49.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:49.300+0000 7fe805ffb640 1 -- 172.21.15.112:0/1738664638 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe80001a070 con 0x7fe8100ceb90 2024-09-21T00:26:49.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:49.301+0000 7fe805ffb640 1 --2- 172.21.15.112:0/1738664638 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fe7f4078bc0 0x7fe7f407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:49.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:49.301+0000 7fe805ffb640 1 -- 172.21.15.112:0/1738664638 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7fe80009a3a0 con 0x7fe8100ceb90 2024-09-21T00:26:49.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:49.301+0000 7fe8151f8640 1 --2- 172.21.15.112:0/1738664638 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fe7f4078bc0 0x7fe7f407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:49.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:49.301+0000 7fe8151f8640 1 --2- 172.21.15.112:0/1738664638 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fe7f4078bc0 0x7fe7f407b0d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fe80c034000 tx=0x7fe80c004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:49.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:49.306+0000 7fe805ffb640 1 -- 172.21.15.112:0/1738664638 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe8101592d0 con 0x7fe8100ceb90 2024-09-21T00:26:49.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:49.480+0000 7fe81745c640 1 -- 172.21.15.112:0/1738664638 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command([{prefix=config set, name=ms_async_reap_threshold}] v 0) -- 0x7fe8101620e0 con 0x7fe8100ceb90 2024-09-21T00:26:49.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:49.483+0000 7fe805ffb640 1 -- 172.21.15.112:0/1738664638 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{prefix=config set, name=ms_async_reap_threshold}]=0 v33)=0 v33) ==== 121+0+0 (secure 0 0 0) 0x7fe800062010 con 0x7fe8100ceb90 2024-09-21T00:26:49.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:49.486+0000 7fe81745c640 1 -- 172.21.15.112:0/1738664638 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fe7f4078bc0 msgr2=0x7fe7f407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:49.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:49.486+0000 7fe81745c640 1 --2- 172.21.15.112:0/1738664638 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fe7f4078bc0 0x7fe7f407b0d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fe80c034000 tx=0x7fe80c004730 comp rx=0 tx=0).stop 2024-09-21T00:26:49.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:49.486+0000 7fe81745c640 1 -- 172.21.15.112:0/1738664638 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe8100ceb90 msgr2=0x7fe8101fd050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:49.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:49.486+0000 7fe81745c640 1 --2- 172.21.15.112:0/1738664638 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe8100ceb90 0x7fe8101fd050 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fe80000c760 tx=0x7fe80000cc20 comp rx=0 tx=0).stop 2024-09-21T00:26:49.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:49.486+0000 7fe81745c640 1 -- 172.21.15.112:0/1738664638 shutdown_connections 2024-09-21T00:26:49.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:49.486+0000 7fe81745c640 1 --2- 172.21.15.112:0/1738664638 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe81015ee80 0x7fe8101fd590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:49.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:49.486+0000 7fe81745c640 1 --2- 172.21.15.112:0/1738664638 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fe7f4078bc0 0x7fe7f407b0d0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:49.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:49.486+0000 7fe81745c640 1 --2- 172.21.15.112:0/1738664638 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe8100ceb90 0x7fe8101fd050 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:49.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:49.486+0000 7fe81745c640 1 --2- 172.21.15.112:0/1738664638 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe8100b9ad0 0x7fe8101569e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:49.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:49.486+0000 7fe81745c640 1 -- 172.21.15.112:0/1738664638 >> 172.21.15.112:0/1738664638 conn(0x7fe81009d830 msgr2=0x7fe810158a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:49.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:49.486+0000 7fe81745c640 1 -- 172.21.15.112:0/1738664638 shutdown_connections 2024-09-21T00:26:49.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:49.486+0000 7fe81745c640 1 -- 172.21.15.112:0/1738664638 wait complete. 2024-09-21T00:26:49.498 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T00:26:49.498 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:26:49.513 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T00:26:49.513 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:26:49.569 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T00:26:49.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:49.721+0000 7f9c354b2640 1 -- 172.21.15.112:0/1327179277 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c1c003a00 msgr2=0x7f9c1c003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:49.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:49.721+0000 7f9c354b2640 1 --2- 172.21.15.112:0/1327179277 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c1c003a00 0x7f9c1c003e70 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f9c28009d50 tx=0x7f9c1c005830 comp rx=0 tx=0).stop 2024-09-21T00:26:49.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:49.721+0000 7f9c2f7fe640 1 -- 172.21.15.112:0/1327179277 reap_dead start 2024-09-21T00:26:49.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:49.721+0000 7f9c354b2640 1 -- 172.21.15.112:0/1327179277 shutdown_connections 2024-09-21T00:26:49.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:49.721+0000 7f9c354b2640 1 -- 172.21.15.112:0/1327179277 >> 172.21.15.112:0/1327179277 conn(0x7f9c300b46e0 msgr2=0x7f9c300d23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:49.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:49.721+0000 7f9c354b2640 1 -- 172.21.15.112:0/1327179277 shutdown_connections 2024-09-21T00:26:49.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:49.721+0000 7f9c354b2640 1 -- 172.21.15.112:0/1327179277 wait complete. 2024-09-21T00:26:49.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:49.723+0000 7f9c354b2640 1 Processor -- start 2024-09-21T00:26:49.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:49.723+0000 7f9c354b2640 1 -- start start 2024-09-21T00:26:49.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:49.724+0000 7f9c354b2640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9c1c003a00 0x7f9c3014fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:49.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:49.724+0000 7f9c354b2640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9c30150220 0x7f9c3015ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:49.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:49.724+0000 7f9c354b2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c3015f4b0 0x7f9c3015d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:49.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:49.724+0000 7f9c354b2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9c300b5d60 con 0x7f9c3015f4b0 2024-09-21T00:26:49.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:49.724+0000 7f9c354b2640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9c300b5be0 con 0x7f9c1c003a00 2024-09-21T00:26:49.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:49.724+0000 7f9c354b2640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f9c300b5ee0 con 0x7f9c30150220 2024-09-21T00:26:49.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:49.724+0000 7f9c2f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c3015f4b0 0x7f9c3015d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:49.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:49.724+0000 7f9c2f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c3015f4b0 0x7f9c3015d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60806/0 (socket says 172.21.15.112:60806) 2024-09-21T00:26:49.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:49.724+0000 7f9c2f7fe640 1 -- 172.21.15.112:0/1352231247 learned_addr learned my addr 172.21.15.112:0/1352231247 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:26:49.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:49.724+0000 7f9c2effd640 1 --2- 172.21.15.112:0/1352231247 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9c1c003a00 0x7f9c3014fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:49.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:49.724+0000 7f9c2e7fc640 1 --2- 172.21.15.112:0/1352231247 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9c30150220 0x7f9c3015ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:49.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:49.724+0000 7f9c2f7fe640 1 -- 172.21.15.112:0/1352231247 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9c30150220 msgr2=0x7f9c3015ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:49.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:49.724+0000 7f9c2f7fe640 1 --2- 172.21.15.112:0/1352231247 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9c30150220 0x7f9c3015ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:49.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:49.724+0000 7f9c2f7fe640 1 -- 172.21.15.112:0/1352231247 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9c1c003a00 msgr2=0x7f9c3014fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:49.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:49.724+0000 7f9c2f7fe640 1 --2- 172.21.15.112:0/1352231247 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9c1c003a00 0x7f9c3014fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:49.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:49.724+0000 7f9c2f7fe640 1 -- 172.21.15.112:0/1352231247 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c3015e6c0 con 0x7f9c3015f4b0 2024-09-21T00:26:49.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:49.724+0000 7f9c2e7fc640 1 --2- 172.21.15.112:0/1352231247 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9c30150220 0x7f9c3015ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:26:49.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:49.725+0000 7f9c2f7fe640 1 --2- 172.21.15.112:0/1352231247 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c3015f4b0 0x7f9c3015d640 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f9c20005fd0 tx=0x7f9c2000f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:49.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:49.725+0000 7f9c277fe640 1 -- 172.21.15.112:0/1352231247 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c20004b20 con 0x7f9c3015f4b0 2024-09-21T00:26:49.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:49.725+0000 7f9c277fe640 1 -- 172.21.15.112:0/1352231247 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9c20004cc0 con 0x7f9c3015f4b0 2024-09-21T00:26:49.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:49.725+0000 7f9c354b2640 1 -- 172.21.15.112:0/1352231247 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c3015f8c0 con 0x7f9c3015f4b0 2024-09-21T00:26:49.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:49.725+0000 7f9c277fe640 1 -- 172.21.15.112:0/1352231247 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c20003bd0 con 0x7f9c3015f4b0 2024-09-21T00:26:49.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:49.725+0000 7f9c354b2640 1 -- 172.21.15.112:0/1352231247 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c3015ce40 con 0x7f9c3015f4b0 2024-09-21T00:26:49.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:49.726+0000 7f9c354b2640 1 -- 172.21.15.112:0/1352231247 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c301544c0 con 0x7f9c3015f4b0 2024-09-21T00:26:49.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:49.727+0000 7f9c277fe640 1 -- 172.21.15.112:0/1352231247 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9c20004030 con 0x7f9c3015f4b0 2024-09-21T00:26:49.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:49.727+0000 7f9c277fe640 1 --2- 172.21.15.112:0/1352231247 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f9c10078bc0 0x7f9c1007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:49.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:49.727+0000 7f9c277fe640 1 -- 172.21.15.112:0/1352231247 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f9c2009b710 con 0x7f9c3015f4b0 2024-09-21T00:26:49.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:49.728+0000 7f9c2effd640 1 --2- 172.21.15.112:0/1352231247 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f9c10078bc0 0x7f9c1007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:49.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:49.728+0000 7f9c2effd640 1 --2- 172.21.15.112:0/1352231247 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f9c10078bc0 0x7f9c1007b0d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f9c28034000 tx=0x7f9c280046c0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:49.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:49.732+0000 7f9c277fe640 1 -- 172.21.15.112:0/1352231247 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f9c301544c0 con 0x7f9c3015f4b0 2024-09-21T00:26:49.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:49.921+0000 7f9c354b2640 1 -- 172.21.15.112:0/1352231247 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f9c300ca940 con 0x7f9c3015f4b0 2024-09-21T00:26:49.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:49.922+0000 7f9c277fe640 1 -- 172.21.15.112:0/1352231247 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v38) ==== 80+0+0 (secure 0 0 0) 0x7f9c300ca940 con 0x7f9c3015f4b0 2024-09-21T00:26:49.922 INFO:teuthology.orchestra.run.smithi112.stderr:listed 0 entries 2024-09-21T00:26:49.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:49.924+0000 7f9c354b2640 1 -- 172.21.15.112:0/1352231247 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f9c10078bc0 msgr2=0x7f9c1007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:49.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:49.924+0000 7f9c354b2640 1 --2- 172.21.15.112:0/1352231247 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f9c10078bc0 0x7f9c1007b0d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f9c28034000 tx=0x7f9c280046c0 comp rx=0 tx=0).stop 2024-09-21T00:26:49.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:49.924+0000 7f9c354b2640 1 -- 172.21.15.112:0/1352231247 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c3015f4b0 msgr2=0x7f9c3015d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:49.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:49.924+0000 7f9c354b2640 1 --2- 172.21.15.112:0/1352231247 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c3015f4b0 0x7f9c3015d640 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f9c20005fd0 tx=0x7f9c2000f3c0 comp rx=0 tx=0).stop 2024-09-21T00:26:49.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:49.924+0000 7f9c354b2640 1 -- 172.21.15.112:0/1352231247 shutdown_connections 2024-09-21T00:26:49.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:49.924+0000 7f9c354b2640 1 --2- 172.21.15.112:0/1352231247 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f9c10078bc0 0x7f9c1007b0d0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:49.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:49.924+0000 7f9c354b2640 1 --2- 172.21.15.112:0/1352231247 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c3015f4b0 0x7f9c3015d640 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:49.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:49.924+0000 7f9c354b2640 1 --2- 172.21.15.112:0/1352231247 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9c30150220 0x7f9c3015ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:49.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:49.924+0000 7f9c354b2640 1 --2- 172.21.15.112:0/1352231247 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9c1c003a00 0x7f9c3014fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:49.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:49.924+0000 7f9c354b2640 1 -- 172.21.15.112:0/1352231247 >> 172.21.15.112:0/1352231247 conn(0x7f9c300b46e0 msgr2=0x7f9c300b37f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:49.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:49.924+0000 7f9c354b2640 1 -- 172.21.15.112:0/1352231247 shutdown_connections 2024-09-21T00:26:49.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:49.924+0000 7f9c354b2640 1 -- 172.21.15.112:0/1352231247 wait complete. 2024-09-21T00:26:49.936 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-21T00:26:49.936 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:26:49.936 DEBUG:teuthology.orchestra.run.smithi112:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-21T00:26:49.964 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi112.front.sepia.ceph.com 2024-09-21T00:26:49.964 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:26:49.964 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T00:26:50.020 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-09-21T00:26:50.020 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:26:50.039 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-09-21T00:26:50.039 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:26:50.097 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T00:26:50.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:50.210+0000 7f8a2c12b640 1 -- 172.21.15.112:0/270290878 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a2406def0 msgr2=0x7f8a2406f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:50.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:50.210+0000 7f8a2c12b640 1 --2- 172.21.15.112:0/270290878 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a2406def0 0x7f8a2406f410 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f8a24075150 tx=0x7f8a2002aac0 comp rx=0 tx=0).stop 2024-09-21T00:26:50.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:50.210+0000 7f8a2c12b640 1 -- 172.21.15.112:0/270290878 shutdown_connections 2024-09-21T00:26:50.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:50.210+0000 7f8a2c12b640 1 --2- 172.21.15.112:0/270290878 >> v2:172.21.15.187:3300/0 conn(0x7f8a240b5260 0x7f8a2415ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:50.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:50.210+0000 7f8a2c12b640 1 --2- 172.21.15.112:0/270290878 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a2406def0 0x7f8a2406f410 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:50.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:50.210+0000 7f8a2c12b640 1 -- 172.21.15.112:0/270290878 >> 172.21.15.112:0/270290878 conn(0x7f8a240cdca0 msgr2=0x7f8a240ce0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:50.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:50.210+0000 7f8a2c12b640 1 -- 172.21.15.112:0/270290878 shutdown_connections 2024-09-21T00:26:50.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:50.210+0000 7f8a2c12b640 1 -- 172.21.15.112:0/270290878 wait complete. 2024-09-21T00:26:50.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:50.212+0000 7f8a2c12b640 1 Processor -- start 2024-09-21T00:26:50.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:50.212+0000 7f8a2c12b640 1 -- start start 2024-09-21T00:26:50.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:50.213+0000 7f8a2c12b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8a2406def0 0x7f8a2409dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:50.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:50.213+0000 7f8a2c12b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a240b5260 0x7f8a24148500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:50.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:50.213+0000 7f8a2c12b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8a2415bfc0 0x7f8a24073e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:50.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:50.213+0000 7f8a2c12b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8a240b5a00 con 0x7f8a240b5260 2024-09-21T00:26:50.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:50.213+0000 7f8a2c12b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8a240b5880 con 0x7f8a2415bfc0 2024-09-21T00:26:50.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:50.213+0000 7f8a2c12b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f8a24071300 con 0x7f8a2406def0 2024-09-21T00:26:50.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:50.213+0000 7f8a296c6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a240b5260 0x7f8a24148500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:50.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:50.214+0000 7f8a296c6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a240b5260 0x7f8a24148500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60836/0 (socket says 172.21.15.112:60836) 2024-09-21T00:26:50.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:50.214+0000 7f8a296c6640 1 -- 172.21.15.112:0/2581196103 learned_addr learned my addr 172.21.15.112:0/2581196103 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:26:50.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:50.214+0000 7f8a29ec7640 1 --2- 172.21.15.112:0/2581196103 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8a2406def0 0x7f8a2409dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:50.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:50.214+0000 7f8a2a6c8640 1 --2- 172.21.15.112:0/2581196103 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8a2415bfc0 0x7f8a24073e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:50.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:50.214+0000 7f8a296c6640 1 -- 172.21.15.112:0/2581196103 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8a2406def0 msgr2=0x7f8a2409dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:50.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:50.214+0000 7f8a296c6640 1 --2- 172.21.15.112:0/2581196103 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8a2406def0 0x7f8a2409dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:50.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:50.214+0000 7f8a296c6640 1 -- 172.21.15.112:0/2581196103 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8a2415bfc0 msgr2=0x7f8a24073e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:50.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:50.214+0000 7f8a296c6640 1 --2- 172.21.15.112:0/2581196103 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8a2415bfc0 0x7f8a24073e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:50.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:50.214+0000 7f8a296c6640 1 -- 172.21.15.112:0/2581196103 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a241490c0 con 0x7f8a240b5260 2024-09-21T00:26:50.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:50.214+0000 7f8a296c6640 1 --2- 172.21.15.112:0/2581196103 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a240b5260 0x7f8a24148500 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f8a1400d6a0 tx=0x7f8a1400db60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:50.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:50.214+0000 7f8a1a7fc640 1 -- 172.21.15.112:0/2581196103 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a14017070 con 0x7f8a240b5260 2024-09-21T00:26:50.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:50.214+0000 7f8a1a7fc640 1 -- 172.21.15.112:0/2581196103 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8a14005310 con 0x7f8a240b5260 2024-09-21T00:26:50.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:50.214+0000 7f8a1a7fc640 1 -- 172.21.15.112:0/2581196103 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a14009820 con 0x7f8a240b5260 2024-09-21T00:26:50.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:50.214+0000 7f8a2c12b640 1 -- 172.21.15.112:0/2581196103 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a241493a0 con 0x7f8a240b5260 2024-09-21T00:26:50.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:50.214+0000 7f8a2c12b640 1 -- 172.21.15.112:0/2581196103 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a241467a0 con 0x7f8a240b5260 2024-09-21T00:26:50.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:50.215+0000 7f8a2c12b640 1 -- 172.21.15.112:0/2581196103 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a2415ba80 con 0x7f8a240b5260 2024-09-21T00:26:50.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:50.216+0000 7f8a1a7fc640 1 -- 172.21.15.112:0/2581196103 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8a140099c0 con 0x7f8a240b5260 2024-09-21T00:26:50.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:50.216+0000 7f8a1a7fc640 1 --2- 172.21.15.112:0/2581196103 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f8a08078b90 0x7f8a0807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:50.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:50.216+0000 7f8a1a7fc640 1 -- 172.21.15.112:0/2581196103 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f8a1409af30 con 0x7f8a240b5260 2024-09-21T00:26:50.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:50.217+0000 7f8a29ec7640 1 --2- 172.21.15.112:0/2581196103 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f8a08078b90 0x7f8a0807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:50.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:50.217+0000 7f8a29ec7640 1 --2- 172.21.15.112:0/2581196103 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f8a08078b90 0x7f8a0807b0a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8a20035000 tx=0x7f8a20038040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:50.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:50.221+0000 7f8a1a7fc640 1 -- 172.21.15.112:0/2581196103 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f8a14062b90 con 0x7f8a240b5260 2024-09-21T00:26:50.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:50.409+0000 7f8a2c12b640 1 -- 172.21.15.112:0/2581196103 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f8a24148e10 con 0x7f8a240b5260 2024-09-21T00:26:50.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:50.409+0000 7f8a1a7fc640 1 -- 172.21.15.112:0/2581196103 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v38) ==== 80+0+0 (secure 0 0 0) 0x7f8a140675c0 con 0x7f8a240b5260 2024-09-21T00:26:50.410 INFO:teuthology.orchestra.run.smithi112.stderr:listed 0 entries 2024-09-21T00:26:50.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:50.411+0000 7f8a2c12b640 1 -- 172.21.15.112:0/2581196103 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f8a08078b90 msgr2=0x7f8a0807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:50.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:50.411+0000 7f8a2c12b640 1 --2- 172.21.15.112:0/2581196103 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f8a08078b90 0x7f8a0807b0a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8a20035000 tx=0x7f8a20038040 comp rx=0 tx=0).stop 2024-09-21T00:26:50.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:50.412+0000 7f8a2c12b640 1 -- 172.21.15.112:0/2581196103 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a240b5260 msgr2=0x7f8a24148500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:50.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:50.412+0000 7f8a2c12b640 1 --2- 172.21.15.112:0/2581196103 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a240b5260 0x7f8a24148500 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f8a1400d6a0 tx=0x7f8a1400db60 comp rx=0 tx=0).stop 2024-09-21T00:26:50.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:50.412+0000 7f8a2c12b640 1 -- 172.21.15.112:0/2581196103 shutdown_connections 2024-09-21T00:26:50.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:50.412+0000 7f8a2c12b640 1 --2- 172.21.15.112:0/2581196103 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8a2415bfc0 0x7f8a24073e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:50.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:50.412+0000 7f8a2c12b640 1 --2- 172.21.15.112:0/2581196103 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f8a08078b90 0x7f8a0807b0a0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:50.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:50.412+0000 7f8a2c12b640 1 --2- 172.21.15.112:0/2581196103 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a240b5260 0x7f8a24148500 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:50.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:50.412+0000 7f8a2c12b640 1 --2- 172.21.15.112:0/2581196103 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8a2406def0 0x7f8a2409dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:50.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:50.412+0000 7f8a2c12b640 1 -- 172.21.15.112:0/2581196103 >> 172.21.15.112:0/2581196103 conn(0x7f8a240cdca0 msgr2=0x7f8a24071830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:50.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:50.412+0000 7f8a2c12b640 1 -- 172.21.15.112:0/2581196103 shutdown_connections 2024-09-21T00:26:50.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:50.412+0000 7f8a2c12b640 1 -- 172.21.15.112:0/2581196103 wait complete. 2024-09-21T00:26:50.423 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-09-21T00:26:50.423 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:26:50.424 DEBUG:teuthology.orchestra.run.smithi187:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-09-21T00:26:50.458 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi187.front.sepia.ceph.com 2024-09-21T00:26:50.458 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:26:50.458 DEBUG:teuthology.orchestra.run.smithi187:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-21T00:26:50.516 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T00:26:50.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:50.629+0000 7f3fb64d3640 1 -- 172.21.15.112:0/1554365431 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3fb0162280 msgr2=0x7f3fb0162690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:50.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:50.629+0000 7f3fb64d3640 1 --2- 172.21.15.112:0/1554365431 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3fb0162280 0x7f3fb0162690 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3f98005fd0 tx=0x7f3f9802a5d0 comp rx=0 tx=0).stop 2024-09-21T00:26:50.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:50.629+0000 7f3fb64d3640 1 -- 172.21.15.112:0/1554365431 shutdown_connections 2024-09-21T00:26:50.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:50.629+0000 7f3fb64d3640 1 --2- 172.21.15.112:0/1554365431 >> v2:172.21.15.187:3300/0 conn(0x7f3fb00b2e40 0x7f3fb00b3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:50.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:50.629+0000 7f3fb64d3640 1 --2- 172.21.15.112:0/1554365431 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3fb0162280 0x7f3fb0162690 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:50.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:50.629+0000 7f3fb64d3640 1 -- 172.21.15.112:0/1554365431 >> 172.21.15.112:0/1554365431 conn(0x7f3fb00b6b20 msgr2=0x7f3fb00d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:50.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:50.629+0000 7f3fb64d3640 1 -- 172.21.15.112:0/1554365431 shutdown_connections 2024-09-21T00:26:50.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:50.629+0000 7f3fb64d3640 1 -- 172.21.15.112:0/1554365431 wait complete. 2024-09-21T00:26:50.632 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:50.631+0000 7f3fb64d3640 1 Processor -- start 2024-09-21T00:26:50.632 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:50.631+0000 7f3fb64d3640 1 -- start start 2024-09-21T00:26:50.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:50.632+0000 7f3fb64d3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3fb00b2e40 0x7f3fb00ceb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:50.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:50.632+0000 7f3fb64d3640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3fb0162280 0x7f3fb015baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:50.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:50.632+0000 7f3fb64d3640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3fb015c030 0x7f3fb01f87d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:50.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:50.632+0000 7f3fb64d3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3fb015e4b0 con 0x7f3fb00b2e40 2024-09-21T00:26:50.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:50.632+0000 7f3fb64d3640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3fb00b39a0 con 0x7f3fb015c030 2024-09-21T00:26:50.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:50.632+0000 7f3fb64d3640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3fb015e630 con 0x7f3fb0162280 2024-09-21T00:26:50.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:50.633+0000 7f3faffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3fb00b2e40 0x7f3fb00ceb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:50.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:50.633+0000 7f3faffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3fb00b2e40 0x7f3fb00ceb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60848/0 (socket says 172.21.15.112:60848) 2024-09-21T00:26:50.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:50.633+0000 7f3faffff640 1 -- 172.21.15.112:0/3744188136 learned_addr learned my addr 172.21.15.112:0/3744188136 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:26:50.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:50.633+0000 7f3fa7fff640 1 --2- 172.21.15.112:0/3744188136 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3fb0162280 0x7f3fb015baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:50.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:50.633+0000 7f3fb4a70640 1 --2- 172.21.15.112:0/3744188136 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3fb015c030 0x7f3fb01f87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:50.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:50.633+0000 7f3faffff640 1 -- 172.21.15.112:0/3744188136 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3fb0162280 msgr2=0x7f3fb015baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:50.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:50.633+0000 7f3faffff640 1 --2- 172.21.15.112:0/3744188136 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3fb0162280 0x7f3fb015baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:50.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:50.633+0000 7f3faffff640 1 -- 172.21.15.112:0/3744188136 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3fb015c030 msgr2=0x7f3fb01f87d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:50.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:50.633+0000 7f3faffff640 1 --2- 172.21.15.112:0/3744188136 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3fb015c030 0x7f3fb01f87d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:50.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:50.633+0000 7f3faffff640 1 -- 172.21.15.112:0/3744188136 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fb015c6d0 con 0x7f3fb00b2e40 2024-09-21T00:26:50.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:50.633+0000 7f3faffff640 1 --2- 172.21.15.112:0/3744188136 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3fb00b2e40 0x7f3fb00ceb20 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f3fa000c3a0 tx=0x7f3fa000c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:50.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:50.633+0000 7f3fad7fa640 1 -- 172.21.15.112:0/3744188136 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fa0017070 con 0x7f3fb00b2e40 2024-09-21T00:26:50.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:50.633+0000 7f3fad7fa640 1 -- 172.21.15.112:0/3744188136 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3fa00053c0 con 0x7f3fb00b2e40 2024-09-21T00:26:50.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:50.633+0000 7f3fb64d3640 1 -- 172.21.15.112:0/3744188136 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fb015c9b0 con 0x7f3fb00b2e40 2024-09-21T00:26:50.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:50.633+0000 7f3fad7fa640 1 -- 172.21.15.112:0/3744188136 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fa0003850 con 0x7f3fb00b2e40 2024-09-21T00:26:50.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:50.633+0000 7f3fb64d3640 1 -- 172.21.15.112:0/3744188136 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fb01f6800 con 0x7f3fb00b2e40 2024-09-21T00:26:50.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:50.634+0000 7f3fb64d3640 1 -- 172.21.15.112:0/3744188136 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3fb0162f00 con 0x7f3fb00b2e40 2024-09-21T00:26:50.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:50.635+0000 7f3fad7fa640 1 -- 172.21.15.112:0/3744188136 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3fa001a070 con 0x7f3fb00b2e40 2024-09-21T00:26:50.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:50.635+0000 7f3fad7fa640 1 --2- 172.21.15.112:0/3744188136 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f3f90078b90 0x7f3f9007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:50.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:50.635+0000 7f3fad7fa640 1 -- 172.21.15.112:0/3744188136 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f3fa009a110 con 0x7f3fb00b2e40 2024-09-21T00:26:50.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:50.636+0000 7f3fa7fff640 1 --2- 172.21.15.112:0/3744188136 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f3f90078b90 0x7f3f9007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:50.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:50.636+0000 7f3fa7fff640 1 --2- 172.21.15.112:0/3744188136 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f3f90078b90 0x7f3f9007b0a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f3f98033000 tx=0x7f3f98004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:50.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:50.640+0000 7f3fad7fa640 1 -- 172.21.15.112:0/3744188136 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3fb0162bd0 con 0x7f3fb00b2e40 2024-09-21T00:26:50.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:50.852+0000 7f3fb64d3640 1 -- 172.21.15.112:0/3744188136 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3fb00cf060 con 0x7f3fb00b2e40 2024-09-21T00:26:50.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:50.853+0000 7f3fad7fa640 1 -- 172.21.15.112:0/3744188136 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5232 (secure 0 0 0) 0x7f3fa0061d70 con 0x7f3fb00b2e40 2024-09-21T00:26:50.854 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:26:50.854 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":17,"btime":"2024-09-21T00:26:39:841719+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4846,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6834/2036740840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":2036740840},{"type":"v1","addr":"172.21.15.112:6834","nonce":2036740840}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4855,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6836/3699399640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":3699399640},{"type":"v1","addr":"172.21.15.187:6836","nonce":3699399640}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4858,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6837/694936803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6835","nonce":694936803},{"type":"v1","addr":"172.21.15.187:6837","nonce":694936803}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T00:26:36.187741+0000","modified":"2024-09-21T00:26:38.947874+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5028},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5028":{"gid":5028,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6835/4012316979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":4012316979},{"type":"v1","addr":"172.21.15.112:6835","nonce":4012316979}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5028,"qdb_cluster":[5028]},"id":2}]} 2024-09-21T00:26:50.854 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 17 2024-09-21T00:26:50.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:50.855+0000 7f3fb64d3640 1 -- 172.21.15.112:0/3744188136 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f3f90078b90 msgr2=0x7f3f9007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:50.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:50.855+0000 7f3fb64d3640 1 --2- 172.21.15.112:0/3744188136 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f3f90078b90 0x7f3f9007b0a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f3f98033000 tx=0x7f3f98004730 comp rx=0 tx=0).stop 2024-09-21T00:26:50.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:50.855+0000 7f3fb64d3640 1 -- 172.21.15.112:0/3744188136 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3fb00b2e40 msgr2=0x7f3fb00ceb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:50.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:50.855+0000 7f3fb64d3640 1 --2- 172.21.15.112:0/3744188136 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3fb00b2e40 0x7f3fb00ceb20 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f3fa000c3a0 tx=0x7f3fa000c7a0 comp rx=0 tx=0).stop 2024-09-21T00:26:50.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:50.855+0000 7f3fb64d3640 1 -- 172.21.15.112:0/3744188136 shutdown_connections 2024-09-21T00:26:50.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:50.855+0000 7f3fb64d3640 1 --2- 172.21.15.112:0/3744188136 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3fb015c030 0x7f3fb01f87d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:50.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:50.855+0000 7f3fb64d3640 1 --2- 172.21.15.112:0/3744188136 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3fb0162280 0x7f3fb015baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:50.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:50.855+0000 7f3fb64d3640 1 --2- 172.21.15.112:0/3744188136 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f3f90078b90 0x7f3f9007b0a0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:50.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:50.855+0000 7f3fb64d3640 1 --2- 172.21.15.112:0/3744188136 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3fb00b2e40 0x7f3fb00ceb20 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:50.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:50.855+0000 7f3fb64d3640 1 -- 172.21.15.112:0/3744188136 >> 172.21.15.112:0/3744188136 conn(0x7f3fb00b6b20 msgr2=0x7f3fb0068f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:50.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:50.855+0000 7f3fb64d3640 1 -- 172.21.15.112:0/3744188136 shutdown_connections 2024-09-21T00:26:50.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:50.855+0000 7f3fb64d3640 1 -- 172.21.15.112:0/3744188136 wait complete. 2024-09-21T00:26:50.867 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json perf dump 2024-09-21T00:26:50.990 INFO:teuthology.orchestra.run.smithi112.stdout:{"AsyncMessenger::Worker-0":{"msgr_recv_messages":33,"msgr_send_messages":33,"msgr_recv_bytes":8510,"msgr_send_bytes":21728,"msgr_created_connections":8,"msgr_active_connections":2,"msgr_running_total_time":0.005975124,"msgr_running_send_time":0.001050503,"msgr_running_recv_time":0.002581857,"msgr_running_fast_dispatch_time":0.000749456,"msgr_send_messages_queue_lat":{"avgcount":32,"sum":0.109690603,"avgtime":0.003427831},"msgr_handle_ack_lat":{"avgcount":5,"sum":0.000002848,"avgtime":0.000000569},"msgr_recv_encrypted_bytes":0,"msgr_send_encrypted_bytes":0},"AsyncMessenger::Worker-1":{"msgr_recv_messages":11,"msgr_send_messages":18,"msgr_recv_bytes":2872,"msgr_send_bytes":102574,"msgr_created_connections":6,"msgr_active_connections":4,"msgr_running_total_time":0.006597288,"msgr_running_send_time":0.000962036,"msgr_running_recv_time":0.005186016,"msgr_running_fast_dispatch_time":0.000302422,"msgr_send_messages_queue_lat":{"avgcount":16,"sum":0.053647297,"avgtime":0.003352956},"msgr_handle_ack_lat":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"msgr_recv_encrypted_bytes":160,"msgr_send_encrypted_bytes":97920},"AsyncMessenger::Worker-2":{"msgr_recv_messages":100,"msgr_send_messages":92,"msgr_recv_bytes":136913,"msgr_send_bytes":54084,"msgr_created_connections":7,"msgr_active_connections":3,"msgr_running_total_time":0.010941180,"msgr_running_send_time":0.003425852,"msgr_running_recv_time":0.009948389,"msgr_running_fast_dispatch_time":0.000209334,"msgr_send_messages_queue_lat":{"avgcount":92,"sum":0.093926659,"avgtime":0.001020941},"msgr_handle_ack_lat":{"avgcount":29,"sum":0.000008831,"avgtime":0.000000304},"msgr_recv_encrypted_bytes":127232,"msgr_send_encrypted_bytes":12176},"cct":{"total_workers":1,"unhealthy_workers":0},"finisher-MDSRank":{"queue_len":0,"complete_latency":{"avgcount":51,"sum":0.002583280,"avgtime":0.000050652}},"finisher-PurgeQueue":{"queue_len":0,"complete_latency":{"avgcount":2,"sum":0.000022767,"avgtime":0.000011383}},"mds":{"request":29,"reply":29,"reply_latency":{"avgcount":29,"sum":0.004663383,"avgtime":0.000160806},"slow_reply":0,"forward":0,"dir_fetch_complete":0,"dir_fetch_keys":0,"dir_commit":12,"dir_split":0,"dir_merge":0,"inodes":11,"inodes_top":0,"inodes_bottom":11,"inodes_pin_tail":0,"inodes_pinned":10,"inodes_expired":0,"inodes_with_caps":1,"caps":2,"subtrees":2,"traverse":55,"traverse_hit":53,"traverse_forward":0,"traverse_discover":0,"traverse_dir_fetch":0,"traverse_remote_ino":0,"traverse_lock":0,"load_cent":0,"q":0,"exported":0,"exported_inodes":0,"imported":0,"imported_inodes":0,"openino_dir_fetch":0,"openino_backtrace_fetch":0,"openino_peer_discover":0,"root_rfiles":0,"root_rbytes":0,"root_rsnaps":0,"scrub_backtrace_fetch":0,"scrub_set_tag":0,"scrub_backtrace_repaired":0,"scrub_inotable_repaired":0,"scrub_dir_inodes":0,"scrub_dir_base_inodes":0,"scrub_dirfrag_rstats":0,"scrub_file_inodes":0,"handle_inode_file_caps":0,"ceph_cap_op_revoke":0,"ceph_cap_op_grant":0,"ceph_cap_op_trunc":0,"ceph_cap_op_flushsnap_ack":0,"ceph_cap_op_flush_ack":0,"handle_client_caps":0,"handle_client_caps_dirty":0,"handle_client_cap_release":0,"process_request_cap_release":1},"mds_cache":{"dir_update":0,"dir_update_receipt":0,"dir_try_discover":0,"dir_send_discover":0,"dir_handle_discover":0,"num_strays":0,"num_strays_delayed":0,"num_strays_enqueuing":0,"strays_created":0,"strays_enqueued":0,"strays_reintegrated":0,"strays_migrated":0,"num_recovering_processing":0,"num_recovering_enqueued":0,"num_recovering_prioritized":0,"recovery_started":0,"recovery_completed":0,"ireq_quiesce_path":0,"ireq_quiesce_inode":0,"ireq_enqueue_scrub":0,"ireq_exportdir":0,"ireq_flush":0,"ireq_fragmentdir":0,"ireq_fragstats":0,"ireq_inodestats":0,"uninline_started":0,"uninline_succeeded":0,"uninline_write_failed":0},"mds_log":{"evlrg":0,"evadd":8,"evex":0,"evtrm":0,"ev":8,"evexg":0,"evexd":0,"segadd":2,"segex":0,"segtrm":0,"seg":2,"segmjr":2,"segexg":0,"segexd":0,"expos":4194304,"wrpos":4197025,"rdpos":0,"jlat":{"avgcount":5,"sum":0.016960368,"avgtime":0.003392073},"replayed":0},"mds_mem":{"ino":13,"ino+":13,"ino-":0,"dir":12,"dir+":12,"dir-":0,"dn":11,"dn+":11,"dn-":0,"cap":0,"cap+":2,"cap-":2,"rss":110896,"heap":332226560},"mds_server":{"dispatch_client_request":29,"dispatch_server_request":0,"handle_client_request":29,"handle_client_session":6,"handle_peer_request":0,"req_create_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_getattr_latency":{"avgcount":26,"sum":0.003974428,"avgtime":0.000152862},"req_getfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_link_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookup_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookuphash_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupino_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupname_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupparent_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lssnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mkdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mknod_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mksnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_open_latency":{"avgcount":2,"sum":0.000370292,"avgtime":0.000185146},"req_readdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rename_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_renamesnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_snapdiff_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setattr_latency":{"avgcount":1,"sum":0.000318663,"avgtime":0.000318663},"req_setdirlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_symlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_unlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"cap_revoke_eviction":0,"cap_acquisition_throttle":0,"req_getvxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"mds_sessions":{"session_count":0,"session_add":2,"session_remove":2,"sessions_open":0,"sessions_stale":0,"total_load":28,"average_load":14,"avg_session_uptime":1,"mdthresh_evicted":0},"mempool":{"bloom_filter_bytes":0,"bloom_filter_items":0,"bluestore_alloc_bytes":0,"bluestore_alloc_items":0,"bluestore_cache_data_bytes":0,"bluestore_cache_data_items":0,"bluestore_cache_onode_bytes":0,"bluestore_cache_onode_items":0,"bluestore_cache_meta_bytes":0,"bluestore_cache_meta_items":0,"bluestore_cache_other_bytes":0,"bluestore_cache_other_items":0,"bluestore_cache_buffer_bytes":0,"bluestore_cache_buffer_items":0,"bluestore_extent_bytes":0,"bluestore_extent_items":0,"bluestore_blob_bytes":0,"bluestore_blob_items":0,"bluestore_shared_blob_bytes":0,"bluestore_shared_blob_items":0,"bluestore_inline_bl_bytes":0,"bluestore_inline_bl_items":0,"bluestore_fsck_bytes":0,"bluestore_fsck_items":0,"bluestore_txc_bytes":0,"bluestore_txc_items":0,"bluestore_writing_deferred_bytes":0,"bluestore_writing_deferred_items":0,"bluestore_writing_bytes":0,"bluestore_writing_items":0,"bluefs_bytes":0,"bluefs_items":0,"bluefs_file_reader_bytes":0,"bluefs_file_reader_items":0,"bluefs_file_writer_bytes":0,"bluefs_file_writer_items":0,"buffer_anon_bytes":225169,"buffer_anon_items":89,"buffer_meta_bytes":0,"buffer_meta_items":0,"osd_bytes":0,"osd_items":0,"osd_mapbl_bytes":0,"osd_mapbl_items":0,"osd_pglog_bytes":0,"osd_pglog_items":0,"osdmap_bytes":5424,"osdmap_items":76,"osdmap_mapping_bytes":0,"osdmap_mapping_items":0,"pgmap_bytes":0,"pgmap_items":0,"mds_co_bytes":53984,"mds_co_items":218,"unittest_1_bytes":0,"unittest_1_items":0,"unittest_2_bytes":0,"unittest_2_items":0},"objecter":{"op_active":0,"op_laggy":0,"op_send":43,"op_send_bytes":13841,"op_resend":37,"op_reply":43,"op_latency":{"avgcount":43,"sum":0.957976936,"avgtime":0.022278533},"op_inflight":0,"oplen_avg":{"avgcount":43,"sum":67},"op":43,"op_r":0,"op_w":43,"op_rmw":0,"op_pg":0,"osdop_stat":0,"osdop_create":11,"osdop_read":0,"osdop_write":5,"osdop_writefull":7,"osdop_writesame":0,"osdop_append":0,"osdop_zero":0,"osdop_truncate":0,"osdop_delete":6,"osdop_mapext":0,"osdop_sparse_read":0,"osdop_clonerange":0,"osdop_getxattr":0,"osdop_setxattr":22,"osdop_cmpxattr":0,"osdop_rmxattr":0,"osdop_resetxattrs":0,"osdop_call":0,"osdop_watch":0,"osdop_notify":0,"osdop_src_cmpxattr":0,"osdop_pgls":0,"osdop_pgls_filter":0,"osdop_other":0,"linger_active":0,"linger_send":0,"linger_resend":0,"linger_ping":0,"poolop_active":0,"poolop_send":0,"poolop_resend":0,"poolstat_active":0,"poolstat_send":0,"poolstat_resend":0,"statfs_active":0,"statfs_send":0,"statfs_resend":0,"command_active":0,"command_send":0,"command_resend":0,"map_epoch":38,"map_full":0,"map_inc":1,"osd_sessions":8,"osd_session_open":8,"osd_session_close":0,"osd_laggy":0,"omap_wr":15,"omap_rd":0,"omap_del":1},"oft":{"omap_total_objs":1,"omap_total_kv_pairs":0,"omap_total_updates":0,"omap_total_removes":0},"purge_queue":{"pq_executing_ops":0,"pq_executing_ops_high_water":0,"pq_executing":0,"pq_executing_high_water":0,"pq_executed_ops":0,"pq_executed":0,"pq_item_in_journal":0},"throttle-msgr_dispatch_throttler-mds":{"val":0,"max":104857600,"get_started":0,"get":140,"get_sum":131704,"get_or_fail_fail":0,"get_or_fail_success":140,"take":0,"take_sum":0,"put":140,"put_sum":131704,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_bytes":{"val":0,"max":104857600,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":43,"take_sum":13841,"put":43,"put_sum":13841,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_ops":{"val":0,"max":1024,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":43,"take_sum":43,"put":43,"put_sum":43,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle":{"val":0,"max":3758096384,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":0,"take_sum":0,"put":0,"put_sum":0,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle-0x555fdc7e5380":{"val":0,"max":3758096384,"get_started":0,"get":8,"get_sum":2721,"get_or_fail_fail":0,"get_or_fail_success":8,"take":0,"take_sum":0,"put":5,"put_sum":2721,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"trackedop":{"slow_ops_count":0}} 2024-09-21T00:26:50.991 DEBUG:tasks.cephfs.filesystem:_json_asok output { "AsyncMessenger::Worker-0": { "msgr_active_connections": 2, "msgr_created_connections": 8, "msgr_handle_ack_lat": { "avgcount": 5, "avgtime": 5.69e-07, "sum": 2.848e-06 }, "msgr_recv_bytes": 8510, "msgr_recv_encrypted_bytes": 0, "msgr_recv_messages": 33, "msgr_running_fast_dispatch_time": 0.000749456, "msgr_running_recv_time": 0.002581857, "msgr_running_send_time": 0.001050503, "msgr_running_total_time": 0.005975124, "msgr_send_bytes": 21728, "msgr_send_encrypted_bytes": 0, "msgr_send_messages": 33, "msgr_send_messages_queue_lat": { "avgcount": 32, "avgtime": 0.003427831, "sum": 0.109690603 } }, "AsyncMessenger::Worker-1": { "msgr_active_connections": 4, "msgr_created_connections": 6, "msgr_handle_ack_lat": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "msgr_recv_bytes": 2872, "msgr_recv_encrypted_bytes": 160, "msgr_recv_messages": 11, "msgr_running_fast_dispatch_time": 0.000302422, "msgr_running_recv_time": 0.005186016, "msgr_running_send_time": 0.000962036, "msgr_running_total_time": 0.006597288, "msgr_send_bytes": 102574, "msgr_send_encrypted_bytes": 97920, "msgr_send_messages": 18, "msgr_send_messages_queue_lat": { "avgcount": 16, "avgtime": 0.003352956, "sum": 0.053647297 } }, "AsyncMessenger::Worker-2": { "msgr_active_connections": 3, "msgr_created_connections": 7, "msgr_handle_ack_lat": { "avgcount": 29, "avgtime": 3.04e-07, "sum": 8.831e-06 }, "msgr_recv_bytes": 136913, "msgr_recv_encrypted_bytes": 127232, "msgr_recv_messages": 100, "msgr_running_fast_dispatch_time": 0.000209334, "msgr_running_recv_time": 0.009948389, "msgr_running_send_time": 0.003425852, "msgr_running_total_time": 0.01094118, "msgr_send_bytes": 54084, "msgr_send_encrypted_bytes": 12176, "msgr_send_messages": 92, "msgr_send_messages_queue_lat": { "avgcount": 92, "avgtime": 0.001020941, "sum": 0.093926659 } }, "cct": { "total_workers": 1, "unhealthy_workers": 0 }, "finisher-MDSRank": { "complete_latency": { "avgcount": 51, "avgtime": 5.0652e-05, "sum": 0.00258328 }, "queue_len": 0 }, "finisher-PurgeQueue": { "complete_latency": { "avgcount": 2, "avgtime": 1.1383e-05, "sum": 2.2767e-05 }, "queue_len": 0 }, "mds": { "caps": 2, "ceph_cap_op_flush_ack": 0, "ceph_cap_op_flushsnap_ack": 0, "ceph_cap_op_grant": 0, "ceph_cap_op_revoke": 0, "ceph_cap_op_trunc": 0, "dir_commit": 12, "dir_fetch_complete": 0, "dir_fetch_keys": 0, "dir_merge": 0, "dir_split": 0, "exported": 0, "exported_inodes": 0, "forward": 0, "handle_client_cap_release": 0, "handle_client_caps": 0, "handle_client_caps_dirty": 0, "handle_inode_file_caps": 0, "imported": 0, "imported_inodes": 0, "inodes": 11, "inodes_bottom": 11, "inodes_expired": 0, "inodes_pin_tail": 0, "inodes_pinned": 10, "inodes_top": 0, "inodes_with_caps": 1, "load_cent": 0, "openino_backtrace_fetch": 0, "openino_dir_fetch": 0, "openino_peer_discover": 0, "process_request_cap_release": 1, "q": 0, "reply": 29, "reply_latency": { "avgcount": 29, "avgtime": 0.000160806, "sum": 0.004663383 }, "request": 29, "root_rbytes": 0, "root_rfiles": 0, "root_rsnaps": 0, "scrub_backtrace_fetch": 0, "scrub_backtrace_repaired": 0, "scrub_dir_base_inodes": 0, "scrub_dir_inodes": 0, "scrub_dirfrag_rstats": 0, "scrub_file_inodes": 0, "scrub_inotable_repaired": 0, "scrub_set_tag": 0, "slow_reply": 0, "subtrees": 2, "traverse": 55, "traverse_dir_fetch": 0, "traverse_discover": 0, "traverse_forward": 0, "traverse_hit": 53, "traverse_lock": 0, "traverse_remote_ino": 0 }, "mds_cache": { "dir_handle_discover": 0, "dir_send_discover": 0, "dir_try_discover": 0, "dir_update": 0, "dir_update_receipt": 0, "ireq_enqueue_scrub": 0, "ireq_exportdir": 0, "ireq_flush": 0, "ireq_fragmentdir": 0, "ireq_fragstats": 0, "ireq_inodestats": 0, "ireq_quiesce_inode": 0, "ireq_quiesce_path": 0, "num_recovering_enqueued": 0, "num_recovering_prioritized": 0, "num_recovering_processing": 0, "num_strays": 0, "num_strays_delayed": 0, "num_strays_enqueuing": 0, "recovery_completed": 0, "recovery_started": 0, "strays_created": 0, "strays_enqueued": 0, "strays_migrated": 0, "strays_reintegrated": 0, "uninline_started": 0, "uninline_succeeded": 0, "uninline_write_failed": 0 }, "mds_log": { "ev": 8, "evadd": 8, "evex": 0, "evexd": 0, "evexg": 0, "evlrg": 0, "evtrm": 0, "expos": 4194304, "jlat": { "avgcount": 5, "avgtime": 0.003392073, "sum": 0.016960368 }, "rdpos": 0, "replayed": 0, "seg": 2, "segadd": 2, "segex": 0, "segexd": 0, "segexg": 0, "segmjr": 2, "segtrm": 0, "wrpos": 4197025 }, "mds_mem": { "cap": 0, "cap+": 2, "cap-": 2, "dir": 12, "dir+": 12, "dir-": 0, "dn": 11, "dn+": 11, "dn-": 0, "heap": 332226560, "ino": 13, "ino+": 13, "ino-": 0, "rss": 110896 }, "mds_server": { "cap_acquisition_throttle": 0, "cap_revoke_eviction": 0, "dispatch_client_request": 29, "dispatch_server_request": 0, "handle_client_request": 29, "handle_client_session": 6, "handle_peer_request": 0, "req_create_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_getattr_latency": { "avgcount": 26, "avgtime": 0.000152862, "sum": 0.003974428 }, "req_getfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_getvxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_link_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookup_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookuphash_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupino_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupname_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupparent_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lssnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mkdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mknod_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mksnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_open_latency": { "avgcount": 2, "avgtime": 0.000185146, "sum": 0.000370292 }, "req_readdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rename_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_renamesnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setattr_latency": { "avgcount": 1, "avgtime": 0.000318663, "sum": 0.000318663 }, "req_setdirlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_snapdiff_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_symlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_unlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "mds_sessions": { "average_load": 14, "avg_session_uptime": 1, "mdthresh_evicted": 0, "session_add": 2, "session_count": 0, "session_remove": 2, "sessions_open": 0, "sessions_stale": 0, "total_load": 28 }, "mempool": { "bloom_filter_bytes": 0, "bloom_filter_items": 0, "bluefs_bytes": 0, "bluefs_file_reader_bytes": 0, "bluefs_file_reader_items": 0, "bluefs_file_writer_bytes": 0, "bluefs_file_writer_items": 0, "bluefs_items": 0, "bluestore_alloc_bytes": 0, "bluestore_alloc_items": 0, "bluestore_blob_bytes": 0, "bluestore_blob_items": 0, "bluestore_cache_buffer_bytes": 0, "bluestore_cache_buffer_items": 0, "bluestore_cache_data_bytes": 0, "bluestore_cache_data_items": 0, "bluestore_cache_meta_bytes": 0, "bluestore_cache_meta_items": 0, "bluestore_cache_onode_bytes": 0, "bluestore_cache_onode_items": 0, "bluestore_cache_other_bytes": 0, "bluestore_cache_other_items": 0, "bluestore_extent_bytes": 0, "bluestore_extent_items": 0, "bluestore_fsck_bytes": 0, "bluestore_fsck_items": 0, "bluestore_inline_bl_bytes": 0, "bluestore_inline_bl_items": 0, "bluestore_shared_blob_bytes": 0, "bluestore_shared_blob_items": 0, "bluestore_txc_bytes": 0, "bluestore_txc_items": 0, "bluestore_writing_bytes": 0, "bluestore_writing_deferred_bytes": 0, "bluestore_writing_deferred_items": 0, "bluestore_writing_items": 0, "buffer_anon_bytes": 225169, "buffer_anon_items": 89, "buffer_meta_bytes": 0, "buffer_meta_items": 0, "mds_co_bytes": 53984, "mds_co_items": 218, "osd_bytes": 0, "osd_items": 0, "osd_mapbl_bytes": 0, "osd_mapbl_items": 0, "osd_pglog_bytes": 0, "osd_pglog_items": 0, "osdmap_bytes": 5424, "osdmap_items": 76, "osdmap_mapping_bytes": 0, "osdmap_mapping_items": 0, "pgmap_bytes": 0, "pgmap_items": 0, "unittest_1_bytes": 0, "unittest_1_items": 0, "unittest_2_bytes": 0, "unittest_2_items": 0 }, "objecter": { "command_active": 0, "command_resend": 0, "command_send": 0, "linger_active": 0, "linger_ping": 0, "linger_resend": 0, "linger_send": 0, "map_epoch": 38, "map_full": 0, "map_inc": 1, "omap_del": 1, "omap_rd": 0, "omap_wr": 15, "op": 43, "op_Infinitylight": 0, "op_active": 0, "op_laggy": 0, "op_latency": { "avgcount": 43, "avgtime": 0.022278533, "sum": 0.957976936 }, "op_pg": 0, "op_r": 0, "op_reply": 43, "op_resend": 37, "op_rmw": 0, "op_send": 43, "op_send_bytes": 13841, "op_w": 43, "oplen_avg": { "avgcount": 43, "sum": 67 }, "osd_laggy": 0, "osd_session_close": 0, "osd_session_open": 8, "osd_sessions": 8, "osdop_append": 0, "osdop_call": 0, "osdop_clonerange": 0, "osdop_cmpxattr": 0, "osdop_create": 11, "osdop_delete": 6, "osdop_getxattr": 0, "osdop_mapext": 0, "osdop_notify": 0, "osdop_other": 0, "osdop_pgls": 0, "osdop_pgls_filter": 0, "osdop_read": 0, "osdop_resetxattrs": 0, "osdop_rmxattr": 0, "osdop_setxattr": 22, "osdop_sparse_read": 0, "osdop_src_cmpxattr": 0, "osdop_stat": 0, "osdop_truncate": 0, "osdop_watch": 0, "osdop_write": 5, "osdop_writefull": 7, "osdop_writesame": 0, "osdop_zero": 0, "poolop_active": 0, "poolop_resend": 0, "poolop_send": 0, "poolstat_active": 0, "poolstat_resend": 0, "poolstat_send": 0, "statfs_active": 0, "statfs_resend": 0, "statfs_send": 0 }, "oft": { "omap_total_kv_pairs": 0, "omap_total_objs": 1, "omap_total_removes": 0, "omap_total_updates": 0 }, "purge_queue": { "pq_executed": 0, "pq_executed_ops": 0, "pq_executing": 0, "pq_executing_high_water": 0, "pq_executing_ops": 0, "pq_executing_ops_high_water": 0, "pq_item_in_journal": 0 }, "throttle-msgr_dispatch_throttler-mds": { "get": 140, "get_or_fail_fail": 0, "get_or_fail_success": 140, "get_started": 0, "get_sum": 131704, "max": 104857600, "put": 140, "put_sum": 131704, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_bytes": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 104857600, "put": 43, "put_sum": 13841, "take": 43, "take_sum": 13841, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_ops": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 1024, "put": 43, "put_sum": 43, "take": 43, "take_sum": 43, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 3758096384, "put": 0, "put_sum": 0, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle-0x555fdc7e5380": { "get": 8, "get_or_fail_fail": 0, "get_or_fail_success": 8, "get_started": 0, "get_sum": 2721, "max": 3758096384, "put": 5, "put_sum": 2721, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "trackedop": { "slow_ops_count": 0 } } 2024-09-21T00:26:50.991 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T00:26:51.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:51.102+0000 7f6628ae5640 1 -- 172.21.15.112:0/17515168 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6624162f00 msgr2=0x7f66240d24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:51.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:51.102+0000 7f6628ae5640 1 --2- 172.21.15.112:0/17515168 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6624162f00 0x7f66240d24d0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f6614009830 tx=0x7f661402a510 comp rx=0 tx=0).stop 2024-09-21T00:26:51.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:51.102+0000 7f6628ae5640 1 -- 172.21.15.112:0/17515168 shutdown_connections 2024-09-21T00:26:51.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:51.102+0000 7f6628ae5640 1 --2- 172.21.15.112:0/17515168 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6624162280 0x7f6624162690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:51.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:51.102+0000 7f6628ae5640 1 --2- 172.21.15.112:0/17515168 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6624162f00 0x7f66240d24d0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:51.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:51.102+0000 7f6628ae5640 1 -- 172.21.15.112:0/17515168 >> 172.21.15.112:0/17515168 conn(0x7f66240b6b20 msgr2=0x7f66240d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:51.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:51.102+0000 7f6628ae5640 1 -- 172.21.15.112:0/17515168 shutdown_connections 2024-09-21T00:26:51.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:51.102+0000 7f6628ae5640 1 -- 172.21.15.112:0/17515168 wait complete. 2024-09-21T00:26:51.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:51.104+0000 7f6628ae5640 1 Processor -- start 2024-09-21T00:26:51.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:51.104+0000 7f6628ae5640 1 -- start start 2024-09-21T00:26:51.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:51.105+0000 7f6628ae5640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f66240b2e40 0x7f662414eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:51.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:51.105+0000 7f6628ae5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6624162280 0x7f662414f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:51.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:51.105+0000 7f6628ae5640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f66240b14a0 0x7f66240b18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:51.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:51.105+0000 7f6628ae5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f662415e630 con 0x7f6624162280 2024-09-21T00:26:51.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:51.105+0000 7f6628ae5640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f662415e4b0 con 0x7f66240b14a0 2024-09-21T00:26:51.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:51.105+0000 7f6628ae5640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f662415e7b0 con 0x7f66240b2e40 2024-09-21T00:26:51.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:51.105+0000 7f6621d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6624162280 0x7f662414f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:51.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:51.105+0000 7f6621d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6624162280 0x7f662414f3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60880/0 (socket says 172.21.15.112:60880) 2024-09-21T00:26:51.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:51.105+0000 7f6621d9b640 1 -- 172.21.15.112:0/3396171579 learned_addr learned my addr 172.21.15.112:0/3396171579 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:26:51.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:51.105+0000 7f662259c640 1 --2- 172.21.15.112:0/3396171579 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f66240b2e40 0x7f662414eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:51.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:51.106+0000 7f6622d9d640 1 --2- 172.21.15.112:0/3396171579 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f66240b14a0 0x7f66240b18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:51.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:51.106+0000 7f6621d9b640 1 -- 172.21.15.112:0/3396171579 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f66240b2e40 msgr2=0x7f662414eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:51.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:51.106+0000 7f6621d9b640 1 --2- 172.21.15.112:0/3396171579 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f66240b2e40 0x7f662414eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:51.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:51.106+0000 7f6621d9b640 1 -- 172.21.15.112:0/3396171579 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f66240b14a0 msgr2=0x7f66240b18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:51.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:51.106+0000 7f6621d9b640 1 --2- 172.21.15.112:0/3396171579 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f66240b14a0 0x7f66240b18b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:51.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:51.106+0000 7f6621d9b640 1 -- 172.21.15.112:0/3396171579 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66240b1df0 con 0x7f6624162280 2024-09-21T00:26:51.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:51.106+0000 7f6621d9b640 1 --2- 172.21.15.112:0/3396171579 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6624162280 0x7f662414f3e0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f6618009fd0 tx=0x7f661800ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:51.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:51.106+0000 7f6612ffd640 1 -- 172.21.15.112:0/3396171579 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f661800ccc0 con 0x7f6624162280 2024-09-21T00:26:51.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:51.106+0000 7f6612ffd640 1 -- 172.21.15.112:0/3396171579 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f661800ee70 con 0x7f6624162280 2024-09-21T00:26:51.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:51.106+0000 7f6628ae5640 1 -- 172.21.15.112:0/3396171579 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f662414fc50 con 0x7f6624162280 2024-09-21T00:26:51.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:51.107+0000 7f6612ffd640 1 -- 172.21.15.112:0/3396171579 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f661800ce60 con 0x7f6624162280 2024-09-21T00:26:51.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:51.107+0000 7f6628ae5640 1 -- 172.21.15.112:0/3396171579 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66240ac6a0 con 0x7f6624162280 2024-09-21T00:26:51.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:51.107+0000 7f6628ae5640 1 -- 172.21.15.112:0/3396171579 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6624162f00 con 0x7f6624162280 2024-09-21T00:26:51.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:51.108+0000 7f6612ffd640 1 -- 172.21.15.112:0/3396171579 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6618004280 con 0x7f6624162280 2024-09-21T00:26:51.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:51.109+0000 7f6612ffd640 1 --2- 172.21.15.112:0/3396171579 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f6604078b90 0x7f660407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:51.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:51.109+0000 7f6612ffd640 1 -- 172.21.15.112:0/3396171579 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f661809b640 con 0x7f6624162280 2024-09-21T00:26:51.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:51.109+0000 7f662259c640 1 --2- 172.21.15.112:0/3396171579 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f6604078b90 0x7f660407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:51.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:51.109+0000 7f662259c640 1 --2- 172.21.15.112:0/3396171579 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f6604078b90 0x7f660407b0a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f6614034000 tx=0x7f6614004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:51.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:51.113+0000 7f6612ffd640 1 -- 172.21.15.112:0/3396171579 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f6618063330 con 0x7f6624162280 2024-09-21T00:26:51.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:51.326+0000 7f6628ae5640 1 -- 172.21.15.112:0/3396171579 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f66240b9670 con 0x7f6624162280 2024-09-21T00:26:51.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:51.327+0000 7f6612ffd640 1 -- 172.21.15.112:0/3396171579 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+73 (secure 0 0 0) 0x7f6618067d60 con 0x7f6624162280 2024-09-21T00:26:51.328 INFO:teuthology.orchestra.run.smithi112.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T00:26:51.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:51.329+0000 7f6628ae5640 1 -- 172.21.15.112:0/3396171579 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f6604078b90 msgr2=0x7f660407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:51.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:51.329+0000 7f6628ae5640 1 --2- 172.21.15.112:0/3396171579 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f6604078b90 0x7f660407b0a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f6614034000 tx=0x7f6614004730 comp rx=0 tx=0).stop 2024-09-21T00:26:51.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:51.329+0000 7f6628ae5640 1 -- 172.21.15.112:0/3396171579 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6624162280 msgr2=0x7f662414f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:51.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:51.329+0000 7f6628ae5640 1 --2- 172.21.15.112:0/3396171579 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6624162280 0x7f662414f3e0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f6618009fd0 tx=0x7f661800ec20 comp rx=0 tx=0).stop 2024-09-21T00:26:51.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:51.329+0000 7f6628ae5640 1 -- 172.21.15.112:0/3396171579 shutdown_connections 2024-09-21T00:26:51.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:51.329+0000 7f6628ae5640 1 --2- 172.21.15.112:0/3396171579 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f66240b14a0 0x7f66240b18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:51.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:51.329+0000 7f6628ae5640 1 --2- 172.21.15.112:0/3396171579 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f6604078b90 0x7f660407b0a0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:51.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:51.329+0000 7f6628ae5640 1 --2- 172.21.15.112:0/3396171579 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6624162280 0x7f662414f3e0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:51.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:51.329+0000 7f6628ae5640 1 --2- 172.21.15.112:0/3396171579 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f66240b2e40 0x7f662414eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:51.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:51.329+0000 7f6628ae5640 1 -- 172.21.15.112:0/3396171579 >> 172.21.15.112:0/3396171579 conn(0x7f66240b6b20 msgr2=0x7f662406ded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:51.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:51.329+0000 7f6628ae5640 1 -- 172.21.15.112:0/3396171579 shutdown_connections 2024-09-21T00:26:51.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:51.329+0000 7f6628ae5640 1 -- 172.21.15.112:0/3396171579 wait complete. 2024-09-21T00:26:51.340 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T00:26:51.341 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T00:26:51.341 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi112.front.sepia.ceph.com 2024-09-21T00:26:51.341 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T00:26:51.341 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T00:26:51.341 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T00:26:51.341 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-21T00:26:51.341 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T00:26:51.341 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T00:26:51.341 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:26:51.342 DEBUG:teuthology.orchestra.run.smithi112:> ip addr 2024-09-21T00:26:51.358 INFO:teuthology.orchestra.run.smithi112.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T00:26:51.358 INFO:teuthology.orchestra.run.smithi112.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T00:26:51.358 INFO:teuthology.orchestra.run.smithi112.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T00:26:51.358 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:26:51.358 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 ::1/128 scope host 2024-09-21T00:26:51.358 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:26:51.358 INFO:teuthology.orchestra.run.smithi112.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T00:26:51.358 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:d8:a5:2c brd ff:ff:ff:ff:ff:ff 2024-09-21T00:26:51.358 INFO:teuthology.orchestra.run.smithi112.stdout: altname enp7s0f0 2024-09-21T00:26:51.358 INFO:teuthology.orchestra.run.smithi112.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T00:26:51.358 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:d8:a5:2d brd ff:ff:ff:ff:ff:ff 2024-09-21T00:26:51.359 INFO:teuthology.orchestra.run.smithi112.stdout: altname enp7s0f1 2024-09-21T00:26:51.359 INFO:teuthology.orchestra.run.smithi112.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T00:26:51.359 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:8f:cb:68 brd ff:ff:ff:ff:ff:ff 2024-09-21T00:26:51.359 INFO:teuthology.orchestra.run.smithi112.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T00:26:51.359 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:8f:cb:69 brd ff:ff:ff:ff:ff:ff 2024-09-21T00:26:51.359 INFO:teuthology.orchestra.run.smithi112.stdout: inet 172.21.15.112/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-21T00:26:51.359 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft 42711sec preferred_lft 42711sec 2024-09-21T00:26:51.359 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::ec4:7aff:fe8f:cb69/64 scope link noprefixroute 2024-09-21T00:26:51.359 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:26:51.359 INFO:teuthology.orchestra.run.smithi112.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T00:26:51.359 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 1a:39:64:e7:fe:d7 brd ff:ff:ff:ff:ff:ff 2024-09-21T00:26:51.359 INFO:teuthology.orchestra.run.smithi112.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T00:26:51.359 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:26:51.359 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::5858:3dff:feaf:1774/64 scope link 2024-09-21T00:26:51.360 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:26:51.360 INFO:teuthology.orchestra.run.smithi112.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T00:26:51.360 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 1a:39:64:e7:fe:d7 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T00:26:51.360 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::1839:64ff:fee7:fed7/64 scope link 2024-09-21T00:26:51.360 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:26:51.360 INFO:teuthology.orchestra.run.smithi112.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T00:26:51.360 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 86:c9:6c:ef:e4:e9 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T00:26:51.360 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::84c9:6cff:feef:e4e9/64 scope link 2024-09-21T00:26:51.360 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:26:51.360 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:26:51.360 DEBUG:teuthology.orchestra.run.smithi112:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T00:26:51.421 INFO:teuthology.orchestra.run.smithi112.stdout:1 2024-09-21T00:26:51.423 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:26:51.423 DEBUG:teuthology.orchestra.run.smithi112:> route 2024-09-21T00:26:51.478 INFO:teuthology.orchestra.run.smithi112.stdout:Kernel IP routing table 2024-09-21T00:26:51.478 INFO:teuthology.orchestra.run.smithi112.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T00:26:51.478 INFO:teuthology.orchestra.run.smithi112.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-21T00:26:51.478 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-21T00:26:51.478 INFO:teuthology.orchestra.run.smithi112.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T00:26:51.479 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T00:26:51.479 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2024-09-21T00:26:51.479 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-21T00:26:51.479 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-21T00:26:51.479 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-21T00:26:51.479 DEBUG:teuthology.orchestra.run.smithi112:> ') 2024-09-21T00:26:51.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T00:26:51.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T00:26:51.605 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:26:51.605 DEBUG:teuthology.orchestra.run.smithi112:> ip netns list 2024-09-21T00:26:51.620 INFO:teuthology.orchestra.run.smithi112.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-21T00:26:51.620 INFO:teuthology.orchestra.run.smithi112.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T00:26:51.620 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T00:26:51.620 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:26:51.621 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link set brx.0 up 2024-09-21T00:26:51.685 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T00:26:51.685 DEBUG:teuthology.orchestra.run.smithi112:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T00:26:51.741 INFO:teuthology.orchestra.run.smithi112.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T00:26:51.741 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T00:26:51.741 DEBUG:teuthology.orchestra.run.smithi112:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T00:26:51.796 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-21T00:26:51.796 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:26:51.796 DEBUG:teuthology.orchestra.run.smithi112:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T00:26:51.890 INFO:teuthology.orchestra.run.smithi112.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T00:26:51.890 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T00:26:51.890 INFO:teuthology.orchestra.run.smithi112.stdout:invalid new device string format 2024-09-21T00:26:51.901 INFO:teuthology.orchestra.run.smithi112.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T00:26:51.902 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T00:26:51.902 INFO:teuthology.orchestra.run.smithi112.stdout:invalid new device string format 2024-09-21T00:26:51.902 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: resolved to: "172.21.15.112:3300,172.21.15.187:3300,172.21.15.187:3301" 2024-09-21T00:26:51.902 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T00:26:51.907 INFO:teuthology.orchestra.run.smithi112.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T00:26:51.907 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T00:26:51.907 INFO:teuthology.orchestra.run.smithi112.stdout:invalid new device string format 2024-09-21T00:26:51.907 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: resolved to: "172.21.15.112:3300,172.21.15.187:3300,172.21.15.187:3301" 2024-09-21T00:26:51.907 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T00:26:51.908 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: resolved to: "172.21.15.112:6789,172.21.15.187:6789,172.21.15.187:6790" 2024-09-21T00:26:51.908 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.112:6789,172.21.15.187:6789,172.21.15.187:6790:/ 2024-09-21T00:26:51.908 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=8b34692b-dc65-49b2-a08f-26f707b96ac5" will pass to kernel 2024-09-21T00:26:51.910 INFO:teuthology.orchestra.run.smithi112.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-21T00:26:51.910 INFO:teuthology.orchestra.run.smithi112.stdout: You just mounted a file system that supports labels which does not 2024-09-21T00:26:51.910 INFO:teuthology.orchestra.run.smithi112.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T00:26:51.910 INFO:teuthology.orchestra.run.smithi112.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T00:26:51.910 INFO:teuthology.orchestra.run.smithi112.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T00:26:51.910 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T00:26:51.910 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:26:51.910 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T00:26:51.934 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T00:26:52.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:52 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T00:26:52.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T00:26:52.030 INFO:teuthology.orchestra.run.smithi112.stdout:client5265 2024-09-21T00:26:52.031 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T00:26:52.031 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:26:52.046 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json perf dump 2024-09-21T00:26:52.210 INFO:teuthology.orchestra.run.smithi112.stdout:{"AsyncMessenger::Worker-0":{"msgr_recv_messages":35,"msgr_send_messages":35,"msgr_recv_bytes":9002,"msgr_send_bytes":22710,"msgr_created_connections":8,"msgr_active_connections":2,"msgr_running_total_time":0.006154884,"msgr_running_send_time":0.001110484,"msgr_running_recv_time":0.002615006,"msgr_running_fast_dispatch_time":0.000795344,"msgr_send_messages_queue_lat":{"avgcount":34,"sum":0.109734381,"avgtime":0.003227481},"msgr_handle_ack_lat":{"avgcount":5,"sum":0.000002848,"avgtime":0.000000569},"msgr_recv_encrypted_bytes":0,"msgr_send_encrypted_bytes":0},"AsyncMessenger::Worker-1":{"msgr_recv_messages":11,"msgr_send_messages":18,"msgr_recv_bytes":2872,"msgr_send_bytes":102574,"msgr_created_connections":6,"msgr_active_connections":4,"msgr_running_total_time":0.006597288,"msgr_running_send_time":0.000962036,"msgr_running_recv_time":0.005186016,"msgr_running_fast_dispatch_time":0.000302422,"msgr_send_messages_queue_lat":{"avgcount":16,"sum":0.053647297,"avgtime":0.003352956},"msgr_handle_ack_lat":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"msgr_recv_encrypted_bytes":160,"msgr_send_encrypted_bytes":97920},"AsyncMessenger::Worker-2":{"msgr_recv_messages":105,"msgr_send_messages":97,"msgr_recv_bytes":138150,"msgr_send_bytes":56248,"msgr_created_connections":8,"msgr_active_connections":4,"msgr_running_total_time":0.011546134,"msgr_running_send_time":0.003687946,"msgr_running_recv_time":0.010377769,"msgr_running_fast_dispatch_time":0.000209334,"msgr_send_messages_queue_lat":{"avgcount":97,"sum":0.093991555,"avgtime":0.000968985},"msgr_handle_ack_lat":{"avgcount":33,"sum":0.000010741,"avgtime":0.000000325},"msgr_recv_encrypted_bytes":127232,"msgr_send_encrypted_bytes":12176},"cct":{"total_workers":1,"unhealthy_workers":0},"finisher-MDSRank":{"queue_len":0,"complete_latency":{"avgcount":54,"sum":0.002628443,"avgtime":0.000048674}},"finisher-PurgeQueue":{"queue_len":0,"complete_latency":{"avgcount":2,"sum":0.000022767,"avgtime":0.000011383}},"mds":{"request":33,"reply":33,"reply_latency":{"avgcount":33,"sum":0.005359070,"avgtime":0.000162396},"slow_reply":0,"forward":0,"dir_fetch_complete":0,"dir_fetch_keys":0,"dir_commit":12,"dir_split":0,"dir_merge":0,"inodes":11,"inodes_top":0,"inodes_bottom":11,"inodes_pin_tail":0,"inodes_pinned":10,"inodes_expired":0,"inodes_with_caps":1,"caps":1,"subtrees":2,"traverse":62,"traverse_hit":59,"traverse_forward":0,"traverse_discover":0,"traverse_dir_fetch":0,"traverse_remote_ino":0,"traverse_lock":0,"load_cent":0,"q":0,"exported":0,"exported_inodes":0,"imported":0,"imported_inodes":0,"openino_dir_fetch":0,"openino_backtrace_fetch":0,"openino_peer_discover":0,"root_rfiles":0,"root_rbytes":0,"root_rsnaps":0,"scrub_backtrace_fetch":0,"scrub_set_tag":0,"scrub_backtrace_repaired":0,"scrub_inotable_repaired":0,"scrub_dir_inodes":0,"scrub_dir_base_inodes":0,"scrub_dirfrag_rstats":0,"scrub_file_inodes":0,"handle_inode_file_caps":0,"ceph_cap_op_revoke":0,"ceph_cap_op_grant":0,"ceph_cap_op_trunc":0,"ceph_cap_op_flushsnap_ack":0,"ceph_cap_op_flush_ack":0,"handle_client_caps":0,"handle_client_caps_dirty":0,"handle_client_cap_release":0,"process_request_cap_release":1},"mds_cache":{"dir_update":0,"dir_update_receipt":0,"dir_try_discover":0,"dir_send_discover":0,"dir_handle_discover":0,"num_strays":0,"num_strays_delayed":0,"num_strays_enqueuing":0,"strays_created":0,"strays_enqueued":0,"strays_reintegrated":0,"strays_migrated":0,"num_recovering_processing":0,"num_recovering_enqueued":0,"num_recovering_prioritized":0,"recovery_started":0,"recovery_completed":0,"ireq_quiesce_path":0,"ireq_quiesce_inode":0,"ireq_enqueue_scrub":0,"ireq_exportdir":0,"ireq_flush":0,"ireq_fragmentdir":0,"ireq_fragstats":0,"ireq_inodestats":0,"uninline_started":0,"uninline_succeeded":0,"uninline_write_failed":0},"mds_log":{"evlrg":0,"evadd":9,"evex":0,"evtrm":0,"ev":9,"evexg":0,"evexd":0,"segadd":2,"segex":0,"segtrm":0,"seg":2,"segmjr":2,"segexg":0,"segexd":0,"expos":4194304,"wrpos":4197281,"rdpos":0,"jlat":{"avgcount":6,"sum":0.019120806,"avgtime":0.003186801},"replayed":0},"mds_mem":{"ino":13,"ino+":13,"ino-":0,"dir":12,"dir+":12,"dir-":0,"dn":11,"dn+":11,"dn-":0,"cap":1,"cap+":3,"cap-":2,"rss":110896,"heap":332226560},"mds_server":{"dispatch_client_request":33,"dispatch_server_request":0,"handle_client_request":33,"handle_client_session":7,"handle_peer_request":0,"req_create_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_getattr_latency":{"avgcount":29,"sum":0.004452988,"avgtime":0.000153551},"req_getfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_link_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookup_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookuphash_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupino_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupname_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupparent_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lssnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mkdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mknod_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mksnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_open_latency":{"avgcount":3,"sum":0.000587419,"avgtime":0.000195806},"req_readdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rename_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_renamesnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_snapdiff_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setattr_latency":{"avgcount":1,"sum":0.000318663,"avgtime":0.000318663},"req_setdirlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_symlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_unlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"cap_revoke_eviction":0,"cap_acquisition_throttle":0,"req_getvxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"mds_sessions":{"session_count":1,"session_add":3,"session_remove":2,"sessions_open":1,"sessions_stale":0,"total_load":31,"average_load":31,"avg_session_uptime":1,"mdthresh_evicted":0},"mempool":{"bloom_filter_bytes":0,"bloom_filter_items":0,"bluestore_alloc_bytes":0,"bluestore_alloc_items":0,"bluestore_cache_data_bytes":0,"bluestore_cache_data_items":0,"bluestore_cache_onode_bytes":0,"bluestore_cache_onode_items":0,"bluestore_cache_meta_bytes":0,"bluestore_cache_meta_items":0,"bluestore_cache_other_bytes":0,"bluestore_cache_other_items":0,"bluestore_cache_buffer_bytes":0,"bluestore_cache_buffer_items":0,"bluestore_extent_bytes":0,"bluestore_extent_items":0,"bluestore_blob_bytes":0,"bluestore_blob_items":0,"bluestore_shared_blob_bytes":0,"bluestore_shared_blob_items":0,"bluestore_inline_bl_bytes":0,"bluestore_inline_bl_items":0,"bluestore_fsck_bytes":0,"bluestore_fsck_items":0,"bluestore_txc_bytes":0,"bluestore_txc_items":0,"bluestore_writing_deferred_bytes":0,"bluestore_writing_deferred_items":0,"bluestore_writing_bytes":0,"bluestore_writing_items":0,"bluefs_bytes":0,"bluefs_items":0,"bluefs_file_reader_bytes":0,"bluefs_file_reader_items":0,"bluefs_file_writer_bytes":0,"bluefs_file_writer_items":0,"buffer_anon_bytes":225169,"buffer_anon_items":89,"buffer_meta_bytes":0,"buffer_meta_items":0,"osd_bytes":0,"osd_items":0,"osd_mapbl_bytes":0,"osd_mapbl_items":0,"osd_pglog_bytes":0,"osd_pglog_items":0,"osdmap_bytes":5424,"osdmap_items":76,"osdmap_mapping_bytes":0,"osdmap_mapping_items":0,"pgmap_bytes":0,"pgmap_items":0,"mds_co_bytes":53984,"mds_co_items":218,"unittest_1_bytes":0,"unittest_1_items":0,"unittest_2_bytes":0,"unittest_2_items":0},"objecter":{"op_active":0,"op_laggy":0,"op_send":45,"op_send_bytes":14187,"op_resend":37,"op_reply":45,"op_latency":{"avgcount":45,"sum":0.962976816,"avgtime":0.021399484},"op_inflight":0,"oplen_avg":{"avgcount":45,"sum":69},"op":45,"op_r":0,"op_w":45,"op_rmw":0,"op_pg":0,"osdop_stat":0,"osdop_create":11,"osdop_read":0,"osdop_write":6,"osdop_writefull":8,"osdop_writesame":0,"osdop_append":0,"osdop_zero":0,"osdop_truncate":0,"osdop_delete":6,"osdop_mapext":0,"osdop_sparse_read":0,"osdop_clonerange":0,"osdop_getxattr":0,"osdop_setxattr":22,"osdop_cmpxattr":0,"osdop_rmxattr":0,"osdop_resetxattrs":0,"osdop_call":0,"osdop_watch":0,"osdop_notify":0,"osdop_src_cmpxattr":0,"osdop_pgls":0,"osdop_pgls_filter":0,"osdop_other":0,"linger_active":0,"linger_send":0,"linger_resend":0,"linger_ping":0,"poolop_active":0,"poolop_send":0,"poolop_resend":0,"poolstat_active":0,"poolstat_send":0,"poolstat_resend":0,"statfs_active":0,"statfs_send":0,"statfs_resend":0,"command_active":0,"command_send":0,"command_resend":0,"map_epoch":38,"map_full":0,"map_inc":1,"osd_sessions":8,"osd_session_open":8,"osd_session_close":0,"osd_laggy":0,"omap_wr":15,"omap_rd":0,"omap_del":1},"oft":{"omap_total_objs":1,"omap_total_kv_pairs":0,"omap_total_updates":0,"omap_total_removes":0},"purge_queue":{"pq_executing_ops":0,"pq_executing_ops_high_water":0,"pq_executing":0,"pq_executing_high_water":0,"pq_executed_ops":0,"pq_executed":0,"pq_item_in_journal":0},"throttle-msgr_dispatch_throttler-mds":{"val":0,"max":104857600,"get_started":0,"get":147,"get_sum":132883,"get_or_fail_fail":0,"get_or_fail_success":147,"take":0,"take_sum":0,"put":147,"put_sum":132883,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_bytes":{"val":0,"max":104857600,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":45,"take_sum":14187,"put":45,"put_sum":14187,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_ops":{"val":0,"max":1024,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":45,"take_sum":45,"put":45,"put_sum":45,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle":{"val":0,"max":3758096384,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":0,"take_sum":0,"put":0,"put_sum":0,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle-0x555fdc7e5380":{"val":0,"max":3758096384,"get_started":0,"get":9,"get_sum":2977,"get_or_fail_fail":0,"get_or_fail_success":9,"take":0,"take_sum":0,"put":6,"put_sum":2977,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"trackedop":{"slow_ops_count":0}} 2024-09-21T00:26:52.212 DEBUG:tasks.cephfs.filesystem:_json_asok output { "AsyncMessenger::Worker-0": { "msgr_active_connections": 2, "msgr_created_connections": 8, "msgr_handle_ack_lat": { "avgcount": 5, "avgtime": 5.69e-07, "sum": 2.848e-06 }, "msgr_recv_bytes": 9002, "msgr_recv_encrypted_bytes": 0, "msgr_recv_messages": 35, "msgr_running_fast_dispatch_time": 0.000795344, "msgr_running_recv_time": 0.002615006, "msgr_running_send_time": 0.001110484, "msgr_running_total_time": 0.006154884, "msgr_send_bytes": 22710, "msgr_send_encrypted_bytes": 0, "msgr_send_messages": 35, "msgr_send_messages_queue_lat": { "avgcount": 34, "avgtime": 0.003227481, "sum": 0.109734381 } }, "AsyncMessenger::Worker-1": { "msgr_active_connections": 4, "msgr_created_connections": 6, "msgr_handle_ack_lat": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "msgr_recv_bytes": 2872, "msgr_recv_encrypted_bytes": 160, "msgr_recv_messages": 11, "msgr_running_fast_dispatch_time": 0.000302422, "msgr_running_recv_time": 0.005186016, "msgr_running_send_time": 0.000962036, "msgr_running_total_time": 0.006597288, "msgr_send_bytes": 102574, "msgr_send_encrypted_bytes": 97920, "msgr_send_messages": 18, "msgr_send_messages_queue_lat": { "avgcount": 16, "avgtime": 0.003352956, "sum": 0.053647297 } }, "AsyncMessenger::Worker-2": { "msgr_active_connections": 4, "msgr_created_connections": 8, "msgr_handle_ack_lat": { "avgcount": 33, "avgtime": 3.25e-07, "sum": 1.0741e-05 }, "msgr_recv_bytes": 138150, "msgr_recv_encrypted_bytes": 127232, "msgr_recv_messages": 105, "msgr_running_fast_dispatch_time": 0.000209334, "msgr_running_recv_time": 0.010377769, "msgr_running_send_time": 0.003687946, "msgr_running_total_time": 0.011546134, "msgr_send_bytes": 56248, "msgr_send_encrypted_bytes": 12176, "msgr_send_messages": 97, "msgr_send_messages_queue_lat": { "avgcount": 97, "avgtime": 0.000968985, "sum": 0.093991555 } }, "cct": { "total_workers": 1, "unhealthy_workers": 0 }, "finisher-MDSRank": { "complete_latency": { "avgcount": 54, "avgtime": 4.8674e-05, "sum": 0.002628443 }, "queue_len": 0 }, "finisher-PurgeQueue": { "complete_latency": { "avgcount": 2, "avgtime": 1.1383e-05, "sum": 2.2767e-05 }, "queue_len": 0 }, "mds": { "caps": 1, "ceph_cap_op_flush_ack": 0, "ceph_cap_op_flushsnap_ack": 0, "ceph_cap_op_grant": 0, "ceph_cap_op_revoke": 0, "ceph_cap_op_trunc": 0, "dir_commit": 12, "dir_fetch_complete": 0, "dir_fetch_keys": 0, "dir_merge": 0, "dir_split": 0, "exported": 0, "exported_inodes": 0, "forward": 0, "handle_client_cap_release": 0, "handle_client_caps": 0, "handle_client_caps_dirty": 0, "handle_inode_file_caps": 0, "imported": 0, "imported_inodes": 0, "inodes": 11, "inodes_bottom": 11, "inodes_expired": 0, "inodes_pin_tail": 0, "inodes_pinned": 10, "inodes_top": 0, "inodes_with_caps": 1, "load_cent": 0, "openino_backtrace_fetch": 0, "openino_dir_fetch": 0, "openino_peer_discover": 0, "process_request_cap_release": 1, "q": 0, "reply": 33, "reply_latency": { "avgcount": 33, "avgtime": 0.000162396, "sum": 0.00535907 }, "request": 33, "root_rbytes": 0, "root_rfiles": 0, "root_rsnaps": 0, "scrub_backtrace_fetch": 0, "scrub_backtrace_repaired": 0, "scrub_dir_base_inodes": 0, "scrub_dir_inodes": 0, "scrub_dirfrag_rstats": 0, "scrub_file_inodes": 0, "scrub_inotable_repaired": 0, "scrub_set_tag": 0, "slow_reply": 0, "subtrees": 2, "traverse": 62, "traverse_dir_fetch": 0, "traverse_discover": 0, "traverse_forward": 0, "traverse_hit": 59, "traverse_lock": 0, "traverse_remote_ino": 0 }, "mds_cache": { "dir_handle_discover": 0, "dir_send_discover": 0, "dir_try_discover": 0, "dir_update": 0, "dir_update_receipt": 0, "ireq_enqueue_scrub": 0, "ireq_exportdir": 0, "ireq_flush": 0, "ireq_fragmentdir": 0, "ireq_fragstats": 0, "ireq_inodestats": 0, "ireq_quiesce_inode": 0, "ireq_quiesce_path": 0, "num_recovering_enqueued": 0, "num_recovering_prioritized": 0, "num_recovering_processing": 0, "num_strays": 0, "num_strays_delayed": 0, "num_strays_enqueuing": 0, "recovery_completed": 0, "recovery_started": 0, "strays_created": 0, "strays_enqueued": 0, "strays_migrated": 0, "strays_reintegrated": 0, "uninline_started": 0, "uninline_succeeded": 0, "uninline_write_failed": 0 }, "mds_log": { "ev": 9, "evadd": 9, "evex": 0, "evexd": 0, "evexg": 0, "evlrg": 0, "evtrm": 0, "expos": 4194304, "jlat": { "avgcount": 6, "avgtime": 0.003186801, "sum": 0.019120806 }, "rdpos": 0, "replayed": 0, "seg": 2, "segadd": 2, "segex": 0, "segexd": 0, "segexg": 0, "segmjr": 2, "segtrm": 0, "wrpos": 4197281 }, "mds_mem": { "cap": 1, "cap+": 3, "cap-": 2, "dir": 12, "dir+": 12, "dir-": 0, "dn": 11, "dn+": 11, "dn-": 0, "heap": 332226560, "ino": 13, "ino+": 13, "ino-": 0, "rss": 110896 }, "mds_server": { "cap_acquisition_throttle": 0, "cap_revoke_eviction": 0, "dispatch_client_request": 33, "dispatch_server_request": 0, "handle_client_request": 33, "handle_client_session": 7, "handle_peer_request": 0, "req_create_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_getattr_latency": { "avgcount": 29, "avgtime": 0.000153551, "sum": 0.004452988 }, "req_getfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_getvxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_link_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookup_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookuphash_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupino_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupname_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupparent_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lssnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mkdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mknod_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mksnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_open_latency": { "avgcount": 3, "avgtime": 0.000195806, "sum": 0.000587419 }, "req_readdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rename_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_renamesnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setattr_latency": { "avgcount": 1, "avgtime": 0.000318663, "sum": 0.000318663 }, "req_setdirlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_snapdiff_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_symlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_unlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "mds_sessions": { "average_load": 31, "avg_session_uptime": 1, "mdthresh_evicted": 0, "session_add": 3, "session_count": 1, "session_remove": 2, "sessions_open": 1, "sessions_stale": 0, "total_load": 31 }, "mempool": { "bloom_filter_bytes": 0, "bloom_filter_items": 0, "bluefs_bytes": 0, "bluefs_file_reader_bytes": 0, "bluefs_file_reader_items": 0, "bluefs_file_writer_bytes": 0, "bluefs_file_writer_items": 0, "bluefs_items": 0, "bluestore_alloc_bytes": 0, "bluestore_alloc_items": 0, "bluestore_blob_bytes": 0, "bluestore_blob_items": 0, "bluestore_cache_buffer_bytes": 0, "bluestore_cache_buffer_items": 0, "bluestore_cache_data_bytes": 0, "bluestore_cache_data_items": 0, "bluestore_cache_meta_bytes": 0, "bluestore_cache_meta_items": 0, "bluestore_cache_onode_bytes": 0, "bluestore_cache_onode_items": 0, "bluestore_cache_other_bytes": 0, "bluestore_cache_other_items": 0, "bluestore_extent_bytes": 0, "bluestore_extent_items": 0, "bluestore_fsck_bytes": 0, "bluestore_fsck_items": 0, "bluestore_inline_bl_bytes": 0, "bluestore_inline_bl_items": 0, "bluestore_shared_blob_bytes": 0, "bluestore_shared_blob_items": 0, "bluestore_txc_bytes": 0, "bluestore_txc_items": 0, "bluestore_writing_bytes": 0, "bluestore_writing_deferred_bytes": 0, "bluestore_writing_deferred_items": 0, "bluestore_writing_items": 0, "buffer_anon_bytes": 225169, "buffer_anon_items": 89, "buffer_meta_bytes": 0, "buffer_meta_items": 0, "mds_co_bytes": 53984, "mds_co_items": 218, "osd_bytes": 0, "osd_items": 0, "osd_mapbl_bytes": 0, "osd_mapbl_items": 0, "osd_pglog_bytes": 0, "osd_pglog_items": 0, "osdmap_bytes": 5424, "osdmap_items": 76, "osdmap_mapping_bytes": 0, "osdmap_mapping_items": 0, "pgmap_bytes": 0, "pgmap_items": 0, "unittest_1_bytes": 0, "unittest_1_items": 0, "unittest_2_bytes": 0, "unittest_2_items": 0 }, "objecter": { "command_active": 0, "command_resend": 0, "command_send": 0, "linger_active": 0, "linger_ping": 0, "linger_resend": 0, "linger_send": 0, "map_epoch": 38, "map_full": 0, "map_inc": 1, "omap_del": 1, "omap_rd": 0, "omap_wr": 15, "op": 45, "op_Infinitylight": 0, "op_active": 0, "op_laggy": 0, "op_latency": { "avgcount": 45, "avgtime": 0.021399484, "sum": 0.962976816 }, "op_pg": 0, "op_r": 0, "op_reply": 45, "op_resend": 37, "op_rmw": 0, "op_send": 45, "op_send_bytes": 14187, "op_w": 45, "oplen_avg": { "avgcount": 45, "sum": 69 }, "osd_laggy": 0, "osd_session_close": 0, "osd_session_open": 8, "osd_sessions": 8, "osdop_append": 0, "osdop_call": 0, "osdop_clonerange": 0, "osdop_cmpxattr": 0, "osdop_create": 11, "osdop_delete": 6, "osdop_getxattr": 0, "osdop_mapext": 0, "osdop_notify": 0, "osdop_other": 0, "osdop_pgls": 0, "osdop_pgls_filter": 0, "osdop_read": 0, "osdop_resetxattrs": 0, "osdop_rmxattr": 0, "osdop_setxattr": 22, "osdop_sparse_read": 0, "osdop_src_cmpxattr": 0, "osdop_stat": 0, "osdop_truncate": 0, "osdop_watch": 0, "osdop_write": 6, "osdop_writefull": 8, "osdop_writesame": 0, "osdop_zero": 0, "poolop_active": 0, "poolop_resend": 0, "poolop_send": 0, "poolstat_active": 0, "poolstat_resend": 0, "poolstat_send": 0, "statfs_active": 0, "statfs_resend": 0, "statfs_send": 0 }, "oft": { "omap_total_kv_pairs": 0, "omap_total_objs": 1, "omap_total_removes": 0, "omap_total_updates": 0 }, "purge_queue": { "pq_executed": 0, "pq_executed_ops": 0, "pq_executing": 0, "pq_executing_high_water": 0, "pq_executing_ops": 0, "pq_executing_ops_high_water": 0, "pq_item_in_journal": 0 }, "throttle-msgr_dispatch_throttler-mds": { "get": 147, "get_or_fail_fail": 0, "get_or_fail_success": 147, "get_started": 0, "get_sum": 132883, "max": 104857600, "put": 147, "put_sum": 132883, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_bytes": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 104857600, "put": 45, "put_sum": 14187, "take": 45, "take_sum": 14187, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_ops": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 1024, "put": 45, "put_sum": 45, "take": 45, "take_sum": 45, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 3758096384, "put": 0, "put_sum": 0, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle-0x555fdc7e5380": { "get": 9, "get_or_fail_fail": 0, "get_or_fail_success": 9, "get_started": 0, "get_sum": 2977, "max": 3758096384, "put": 6, "put_sum": 2977, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "trackedop": { "slow_ops_count": 0 } } 2024-09-21T00:26:52.212 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T00:26:52.212 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:26:52.228 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T00:26:52.228 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:26:52.284 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T00:26:52.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:52.437+0000 7f20be562640 1 -- 172.21.15.112:0/1115688956 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20a0004d40 msgr2=0x7f20a0005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:52.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:52.437+0000 7f20be562640 1 --2- 172.21.15.112:0/1115688956 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20a0004d40 0x7f20a0005210 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f20b0001d90 tx=0x7f20a0005830 comp rx=0 tx=0).stop 2024-09-21T00:26:52.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:52.437+0000 7f20bcaff640 1 -- 172.21.15.112:0/1115688956 reap_dead start 2024-09-21T00:26:52.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:52.437+0000 7f20be562640 1 -- 172.21.15.112:0/1115688956 shutdown_connections 2024-09-21T00:26:52.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:52.437+0000 7f20be562640 1 -- 172.21.15.112:0/1115688956 >> 172.21.15.112:0/1115688956 conn(0x7f20b80b46e0 msgr2=0x7f20b80d23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:52.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:52.437+0000 7f20be562640 1 -- 172.21.15.112:0/1115688956 shutdown_connections 2024-09-21T00:26:52.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:52.437+0000 7f20be562640 1 -- 172.21.15.112:0/1115688956 wait complete. 2024-09-21T00:26:52.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:52.438+0000 7f20be562640 1 Processor -- start 2024-09-21T00:26:52.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:52.439+0000 7f20be562640 1 -- start start 2024-09-21T00:26:52.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:52.440+0000 7f20be562640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f20b80b0a20 0x7f20b80cb660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:52.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:52.440+0000 7f20be562640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20b80cbba0 0x7f20b81f3450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:52.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:52.440+0000 7f20be562640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f20b81f3990 0x7f20b80a9450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:52.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:52.440+0000 7f20be562640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f20b80b62f0 con 0x7f20b80cbba0 2024-09-21T00:26:52.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:52.440+0000 7f20be562640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f20b80d09e0 con 0x7f20b81f3990 2024-09-21T00:26:52.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:52.440+0000 7f20be562640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f20b80b6470 con 0x7f20b80b0a20 2024-09-21T00:26:52.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:52.440+0000 7f20b77fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20b80cbba0 0x7f20b81f3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:52.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:52.440+0000 7f20b77fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20b80cbba0 0x7f20b81f3450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60900/0 (socket says 172.21.15.112:60900) 2024-09-21T00:26:52.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:52.440+0000 7f20b77fe640 1 -- 172.21.15.112:0/1099011325 learned_addr learned my addr 172.21.15.112:0/1099011325 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:26:52.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:52.440+0000 7f20b7fff640 1 --2- 172.21.15.112:0/1099011325 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f20b80b0a20 0x7f20b80cb660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:52.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:52.440+0000 7f20bcaff640 1 --2- 172.21.15.112:0/1099011325 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f20b81f3990 0x7f20b80a9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:52.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:52.440+0000 7f20b77fe640 1 -- 172.21.15.112:0/1099011325 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f20b80b0a20 msgr2=0x7f20b80cb660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:52.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:52.440+0000 7f20b77fe640 1 --2- 172.21.15.112:0/1099011325 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f20b80b0a20 0x7f20b80cb660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:52.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:52.440+0000 7f20b77fe640 1 -- 172.21.15.112:0/1099011325 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f20b81f3990 msgr2=0x7f20b80a9450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:52.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:52.440+0000 7f20b77fe640 1 --2- 172.21.15.112:0/1099011325 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f20b81f3990 0x7f20b80a9450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:52.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:52.440+0000 7f20b77fe640 1 -- 172.21.15.112:0/1099011325 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20b80cbfb0 con 0x7f20b80cbba0 2024-09-21T00:26:52.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:52.440+0000 7f20b7fff640 1 --2- 172.21.15.112:0/1099011325 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f20b80b0a20 0x7f20b80cb660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:26:52.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:52.440+0000 7f20b77fe640 1 --2- 172.21.15.112:0/1099011325 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20b80cbba0 0x7f20b81f3450 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f20ac009fd0 tx=0x7f20ac011cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:52.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:52.440+0000 7f20b4ff9640 1 -- 172.21.15.112:0/1099011325 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20ac004460 con 0x7f20b80cbba0 2024-09-21T00:26:52.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:52.441+0000 7f20b4ff9640 1 -- 172.21.15.112:0/1099011325 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f20ac016070 con 0x7f20b80cbba0 2024-09-21T00:26:52.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:52.441+0000 7f20be562640 1 -- 172.21.15.112:0/1099011325 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20b81f77a0 con 0x7f20b80cbba0 2024-09-21T00:26:52.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:52.441+0000 7f20b4ff9640 1 -- 172.21.15.112:0/1099011325 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20ac00fb60 con 0x7f20b80cbba0 2024-09-21T00:26:52.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:52.441+0000 7f20be562640 1 -- 172.21.15.112:0/1099011325 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20b81f6cc0 con 0x7f20b80cbba0 2024-09-21T00:26:52.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:52.442+0000 7f20be562640 1 -- 172.21.15.112:0/1099011325 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20b80b5df0 con 0x7f20b80cbba0 2024-09-21T00:26:52.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:52.442+0000 7f20b4ff9640 1 -- 172.21.15.112:0/1099011325 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f20ac018030 con 0x7f20b80cbba0 2024-09-21T00:26:52.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:52.443+0000 7f20b4ff9640 1 --2- 172.21.15.112:0/1099011325 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f20a4078b90 0x7f20a407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:52.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:52.443+0000 7f20b4ff9640 1 -- 172.21.15.112:0/1099011325 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f20ac09b650 con 0x7f20b80cbba0 2024-09-21T00:26:52.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:52.443+0000 7f20b7fff640 1 --2- 172.21.15.112:0/1099011325 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f20a4078b90 0x7f20a407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:52.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:52.444+0000 7f20b7fff640 1 --2- 172.21.15.112:0/1099011325 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f20a4078b90 0x7f20a407b0a0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f20b80b1240 tx=0x7f20a8006910 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:52.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:52.448+0000 7f20b4ff9640 1 -- 172.21.15.112:0/1099011325 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f20b80b5df0 con 0x7f20b80cbba0 2024-09-21T00:26:52.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:52.637+0000 7f20be562640 1 -- 172.21.15.112:0/1099011325 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f20b815c3b0 con 0x7f20b80cbba0 2024-09-21T00:26:52.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:52.638+0000 7f20b4ff9640 1 -- 172.21.15.112:0/1099011325 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v38) ==== 80+0+0 (secure 0 0 0) 0x7f20b80cbfb0 con 0x7f20b80cbba0 2024-09-21T00:26:52.638 INFO:teuthology.orchestra.run.smithi112.stderr:listed 0 entries 2024-09-21T00:26:52.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:52.640+0000 7f20be562640 1 -- 172.21.15.112:0/1099011325 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f20a4078b90 msgr2=0x7f20a407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:52.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:52.640+0000 7f20be562640 1 --2- 172.21.15.112:0/1099011325 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f20a4078b90 0x7f20a407b0a0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f20b80b1240 tx=0x7f20a8006910 comp rx=0 tx=0).stop 2024-09-21T00:26:52.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:52.640+0000 7f20be562640 1 -- 172.21.15.112:0/1099011325 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20b80cbba0 msgr2=0x7f20b81f3450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:52.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:52.640+0000 7f20be562640 1 --2- 172.21.15.112:0/1099011325 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20b80cbba0 0x7f20b81f3450 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f20ac009fd0 tx=0x7f20ac011cb0 comp rx=0 tx=0).stop 2024-09-21T00:26:52.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:52.640+0000 7f20be562640 1 -- 172.21.15.112:0/1099011325 shutdown_connections 2024-09-21T00:26:52.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:52.640+0000 7f20be562640 1 --2- 172.21.15.112:0/1099011325 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f20b81f3990 0x7f20b80a9450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:52.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:52.640+0000 7f20be562640 1 --2- 172.21.15.112:0/1099011325 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f20a4078b90 0x7f20a407b0a0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:52.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:52.640+0000 7f20be562640 1 --2- 172.21.15.112:0/1099011325 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20b80cbba0 0x7f20b81f3450 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:52.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:52.640+0000 7f20be562640 1 --2- 172.21.15.112:0/1099011325 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f20b80b0a20 0x7f20b80cb660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:52.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:52.640+0000 7f20be562640 1 -- 172.21.15.112:0/1099011325 >> 172.21.15.112:0/1099011325 conn(0x7f20b80b46e0 msgr2=0x7f20b80b79e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:52.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:52.640+0000 7f20be562640 1 -- 172.21.15.112:0/1099011325 shutdown_connections 2024-09-21T00:26:52.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:52.640+0000 7f20be562640 1 -- 172.21.15.112:0/1099011325 wait complete. 2024-09-21T00:26:52.652 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-21T00:26:52.652 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:26:52.652 DEBUG:teuthology.orchestra.run.smithi112:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-21T00:26:52.680 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi112.front.sepia.ceph.com 2024-09-21T00:26:52.680 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:26:52.680 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T00:26:52.736 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json perf dump 2024-09-21T00:26:52.902 INFO:teuthology.orchestra.run.smithi112.stdout:{"AsyncMessenger::Worker-0":{"msgr_recv_messages":36,"msgr_send_messages":36,"msgr_recv_bytes":9248,"msgr_send_bytes":23169,"msgr_created_connections":8,"msgr_active_connections":2,"msgr_running_total_time":0.006277813,"msgr_running_send_time":0.001164580,"msgr_running_recv_time":0.002637477,"msgr_running_fast_dispatch_time":0.000819785,"msgr_send_messages_queue_lat":{"avgcount":35,"sum":0.109760153,"avgtime":0.003136004},"msgr_handle_ack_lat":{"avgcount":5,"sum":0.000002848,"avgtime":0.000000569},"msgr_recv_encrypted_bytes":0,"msgr_send_encrypted_bytes":0},"AsyncMessenger::Worker-1":{"msgr_recv_messages":11,"msgr_send_messages":18,"msgr_recv_bytes":2872,"msgr_send_bytes":102574,"msgr_created_connections":6,"msgr_active_connections":4,"msgr_running_total_time":0.006597288,"msgr_running_send_time":0.000962036,"msgr_running_recv_time":0.005186016,"msgr_running_fast_dispatch_time":0.000302422,"msgr_send_messages_queue_lat":{"avgcount":16,"sum":0.053647297,"avgtime":0.003352956},"msgr_handle_ack_lat":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"msgr_recv_encrypted_bytes":160,"msgr_send_encrypted_bytes":97920},"AsyncMessenger::Worker-2":{"msgr_recv_messages":108,"msgr_send_messages":99,"msgr_recv_bytes":138626,"msgr_send_bytes":56991,"msgr_created_connections":8,"msgr_active_connections":3,"msgr_running_total_time":0.011816434,"msgr_running_send_time":0.003795838,"msgr_running_recv_time":0.010551923,"msgr_running_fast_dispatch_time":0.000209334,"msgr_send_messages_queue_lat":{"avgcount":99,"sum":0.094029546,"avgtime":0.000949793},"msgr_handle_ack_lat":{"avgcount":34,"sum":0.000011421,"avgtime":0.000000335},"msgr_recv_encrypted_bytes":127504,"msgr_send_encrypted_bytes":12816},"cct":{"total_workers":1,"unhealthy_workers":0},"finisher-MDSRank":{"queue_len":0,"complete_latency":{"avgcount":56,"sum":0.002691075,"avgtime":0.000048054}},"finisher-PurgeQueue":{"queue_len":0,"complete_latency":{"avgcount":2,"sum":0.000022767,"avgtime":0.000011383}},"mds":{"request":33,"reply":33,"reply_latency":{"avgcount":33,"sum":0.005359070,"avgtime":0.000162396},"slow_reply":0,"forward":0,"dir_fetch_complete":0,"dir_fetch_keys":0,"dir_commit":12,"dir_split":0,"dir_merge":0,"inodes":11,"inodes_top":0,"inodes_bottom":11,"inodes_pin_tail":0,"inodes_pinned":10,"inodes_expired":0,"inodes_with_caps":1,"caps":1,"subtrees":2,"traverse":62,"traverse_hit":59,"traverse_forward":0,"traverse_discover":0,"traverse_dir_fetch":0,"traverse_remote_ino":0,"traverse_lock":0,"load_cent":0,"q":0,"exported":0,"exported_inodes":0,"imported":0,"imported_inodes":0,"openino_dir_fetch":0,"openino_backtrace_fetch":0,"openino_peer_discover":0,"root_rfiles":0,"root_rbytes":0,"root_rsnaps":0,"scrub_backtrace_fetch":0,"scrub_set_tag":0,"scrub_backtrace_repaired":0,"scrub_inotable_repaired":0,"scrub_dir_inodes":0,"scrub_dir_base_inodes":0,"scrub_dirfrag_rstats":0,"scrub_file_inodes":0,"handle_inode_file_caps":0,"ceph_cap_op_revoke":0,"ceph_cap_op_grant":0,"ceph_cap_op_trunc":0,"ceph_cap_op_flushsnap_ack":0,"ceph_cap_op_flush_ack":0,"handle_client_caps":0,"handle_client_caps_dirty":0,"handle_client_cap_release":0,"process_request_cap_release":1},"mds_cache":{"dir_update":0,"dir_update_receipt":0,"dir_try_discover":0,"dir_send_discover":0,"dir_handle_discover":0,"num_strays":0,"num_strays_delayed":0,"num_strays_enqueuing":0,"strays_created":0,"strays_enqueued":0,"strays_reintegrated":0,"strays_migrated":0,"num_recovering_processing":0,"num_recovering_enqueued":0,"num_recovering_prioritized":0,"recovery_started":0,"recovery_completed":0,"ireq_quiesce_path":0,"ireq_quiesce_inode":0,"ireq_enqueue_scrub":0,"ireq_exportdir":0,"ireq_flush":0,"ireq_fragmentdir":0,"ireq_fragstats":0,"ireq_inodestats":0,"uninline_started":0,"uninline_succeeded":0,"uninline_write_failed":0},"mds_log":{"evlrg":0,"evadd":10,"evex":0,"evtrm":0,"ev":10,"evexg":0,"evexd":0,"segadd":2,"segex":0,"segtrm":0,"seg":2,"segmjr":2,"segexg":0,"segexd":0,"expos":4194304,"wrpos":4197422,"rdpos":0,"jlat":{"avgcount":7,"sum":0.021303759,"avgtime":0.003043394},"replayed":0},"mds_mem":{"ino":13,"ino+":13,"ino-":0,"dir":12,"dir+":12,"dir-":0,"dn":11,"dn+":11,"dn-":0,"cap":0,"cap+":3,"cap-":3,"rss":111024,"heap":332226560},"mds_server":{"dispatch_client_request":33,"dispatch_server_request":0,"handle_client_request":33,"handle_client_session":9,"handle_peer_request":0,"req_create_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_getattr_latency":{"avgcount":29,"sum":0.004452988,"avgtime":0.000153551},"req_getfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_link_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookup_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookuphash_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupino_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupname_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupparent_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lssnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mkdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mknod_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mksnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_open_latency":{"avgcount":3,"sum":0.000587419,"avgtime":0.000195806},"req_readdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rename_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_renamesnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_snapdiff_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setattr_latency":{"avgcount":1,"sum":0.000318663,"avgtime":0.000318663},"req_setdirlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_symlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_unlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"cap_revoke_eviction":0,"cap_acquisition_throttle":0,"req_getvxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"mds_sessions":{"session_count":0,"session_add":3,"session_remove":3,"sessions_open":0,"sessions_stale":0,"total_load":31,"average_load":31,"avg_session_uptime":1,"mdthresh_evicted":0},"mempool":{"bloom_filter_bytes":0,"bloom_filter_items":0,"bluestore_alloc_bytes":0,"bluestore_alloc_items":0,"bluestore_cache_data_bytes":0,"bluestore_cache_data_items":0,"bluestore_cache_onode_bytes":0,"bluestore_cache_onode_items":0,"bluestore_cache_meta_bytes":0,"bluestore_cache_meta_items":0,"bluestore_cache_other_bytes":0,"bluestore_cache_other_items":0,"bluestore_cache_buffer_bytes":0,"bluestore_cache_buffer_items":0,"bluestore_extent_bytes":0,"bluestore_extent_items":0,"bluestore_blob_bytes":0,"bluestore_blob_items":0,"bluestore_shared_blob_bytes":0,"bluestore_shared_blob_items":0,"bluestore_inline_bl_bytes":0,"bluestore_inline_bl_items":0,"bluestore_fsck_bytes":0,"bluestore_fsck_items":0,"bluestore_txc_bytes":0,"bluestore_txc_items":0,"bluestore_writing_deferred_bytes":0,"bluestore_writing_deferred_items":0,"bluestore_writing_bytes":0,"bluestore_writing_items":0,"bluefs_bytes":0,"bluefs_items":0,"bluefs_file_reader_bytes":0,"bluefs_file_reader_items":0,"bluefs_file_writer_bytes":0,"bluefs_file_writer_items":0,"buffer_anon_bytes":225169,"buffer_anon_items":89,"buffer_meta_bytes":0,"buffer_meta_items":0,"osd_bytes":0,"osd_items":0,"osd_mapbl_bytes":0,"osd_mapbl_items":0,"osd_pglog_bytes":0,"osd_pglog_items":0,"osdmap_bytes":5424,"osdmap_items":76,"osdmap_mapping_bytes":0,"osdmap_mapping_items":0,"pgmap_bytes":0,"pgmap_items":0,"mds_co_bytes":53984,"mds_co_items":218,"unittest_1_bytes":0,"unittest_1_items":0,"unittest_2_bytes":0,"unittest_2_items":0},"objecter":{"op_active":0,"op_laggy":0,"op_send":46,"op_send_bytes":14328,"op_resend":37,"op_reply":46,"op_latency":{"avgcount":46,"sum":0.964976768,"avgtime":0.020977755},"op_inflight":0,"oplen_avg":{"avgcount":46,"sum":70},"op":46,"op_r":0,"op_w":46,"op_rmw":0,"op_pg":0,"osdop_stat":0,"osdop_create":11,"osdop_read":0,"osdop_write":7,"osdop_writefull":8,"osdop_writesame":0,"osdop_append":0,"osdop_zero":0,"osdop_truncate":0,"osdop_delete":6,"osdop_mapext":0,"osdop_sparse_read":0,"osdop_clonerange":0,"osdop_getxattr":0,"osdop_setxattr":22,"osdop_cmpxattr":0,"osdop_rmxattr":0,"osdop_resetxattrs":0,"osdop_call":0,"osdop_watch":0,"osdop_notify":0,"osdop_src_cmpxattr":0,"osdop_pgls":0,"osdop_pgls_filter":0,"osdop_other":0,"linger_active":0,"linger_send":0,"linger_resend":0,"linger_ping":0,"poolop_active":0,"poolop_send":0,"poolop_resend":0,"poolstat_active":0,"poolstat_send":0,"poolstat_resend":0,"statfs_active":0,"statfs_send":0,"statfs_resend":0,"command_active":0,"command_send":0,"command_resend":0,"map_epoch":38,"map_full":0,"map_inc":1,"osd_sessions":8,"osd_session_open":8,"osd_session_close":0,"osd_laggy":0,"omap_wr":15,"omap_rd":0,"omap_del":1},"oft":{"omap_total_objs":1,"omap_total_kv_pairs":0,"omap_total_updates":0,"omap_total_removes":0},"purge_queue":{"pq_executing_ops":0,"pq_executing_ops_high_water":0,"pq_executing":0,"pq_executing_high_water":0,"pq_executed_ops":0,"pq_executed":0,"pq_item_in_journal":0},"throttle-msgr_dispatch_throttler-mds":{"val":0,"max":104857600,"get_started":0,"get":151,"get_sum":133225,"get_or_fail_fail":0,"get_or_fail_success":151,"take":0,"take_sum":0,"put":151,"put_sum":133225,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_bytes":{"val":0,"max":104857600,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":46,"take_sum":14328,"put":46,"put_sum":14328,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_ops":{"val":0,"max":1024,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":46,"take_sum":46,"put":46,"put_sum":46,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle":{"val":0,"max":3758096384,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":0,"take_sum":0,"put":0,"put_sum":0,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle-0x555fdc7e5380":{"val":0,"max":3758096384,"get_started":0,"get":10,"get_sum":3118,"get_or_fail_fail":0,"get_or_fail_success":10,"take":0,"take_sum":0,"put":7,"put_sum":3118,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"trackedop":{"slow_ops_count":0}} 2024-09-21T00:26:52.903 DEBUG:tasks.cephfs.filesystem:_json_asok output { "AsyncMessenger::Worker-0": { "msgr_active_connections": 2, "msgr_created_connections": 8, "msgr_handle_ack_lat": { "avgcount": 5, "avgtime": 5.69e-07, "sum": 2.848e-06 }, "msgr_recv_bytes": 9248, "msgr_recv_encrypted_bytes": 0, "msgr_recv_messages": 36, "msgr_running_fast_dispatch_time": 0.000819785, "msgr_running_recv_time": 0.002637477, "msgr_running_send_time": 0.00116458, "msgr_running_total_time": 0.006277813, "msgr_send_bytes": 23169, "msgr_send_encrypted_bytes": 0, "msgr_send_messages": 36, "msgr_send_messages_queue_lat": { "avgcount": 35, "avgtime": 0.003136004, "sum": 0.109760153 } }, "AsyncMessenger::Worker-1": { "msgr_active_connections": 4, "msgr_created_connections": 6, "msgr_handle_ack_lat": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "msgr_recv_bytes": 2872, "msgr_recv_encrypted_bytes": 160, "msgr_recv_messages": 11, "msgr_running_fast_dispatch_time": 0.000302422, "msgr_running_recv_time": 0.005186016, "msgr_running_send_time": 0.000962036, "msgr_running_total_time": 0.006597288, "msgr_send_bytes": 102574, "msgr_send_encrypted_bytes": 97920, "msgr_send_messages": 18, "msgr_send_messages_queue_lat": { "avgcount": 16, "avgtime": 0.003352956, "sum": 0.053647297 } }, "AsyncMessenger::Worker-2": { "msgr_active_connections": 3, "msgr_created_connections": 8, "msgr_handle_ack_lat": { "avgcount": 34, "avgtime": 3.35e-07, "sum": 1.1421e-05 }, "msgr_recv_bytes": 138626, "msgr_recv_encrypted_bytes": 127504, "msgr_recv_messages": 108, "msgr_running_fast_dispatch_time": 0.000209334, "msgr_running_recv_time": 0.010551923, "msgr_running_send_time": 0.003795838, "msgr_running_total_time": 0.011816434, "msgr_send_bytes": 56991, "msgr_send_encrypted_bytes": 12816, "msgr_send_messages": 99, "msgr_send_messages_queue_lat": { "avgcount": 99, "avgtime": 0.000949793, "sum": 0.094029546 } }, "cct": { "total_workers": 1, "unhealthy_workers": 0 }, "finisher-MDSRank": { "complete_latency": { "avgcount": 56, "avgtime": 4.8054e-05, "sum": 0.002691075 }, "queue_len": 0 }, "finisher-PurgeQueue": { "complete_latency": { "avgcount": 2, "avgtime": 1.1383e-05, "sum": 2.2767e-05 }, "queue_len": 0 }, "mds": { "caps": 1, "ceph_cap_op_flush_ack": 0, "ceph_cap_op_flushsnap_ack": 0, "ceph_cap_op_grant": 0, "ceph_cap_op_revoke": 0, "ceph_cap_op_trunc": 0, "dir_commit": 12, "dir_fetch_complete": 0, "dir_fetch_keys": 0, "dir_merge": 0, "dir_split": 0, "exported": 0, "exported_inodes": 0, "forward": 0, "handle_client_cap_release": 0, "handle_client_caps": 0, "handle_client_caps_dirty": 0, "handle_inode_file_caps": 0, "imported": 0, "imported_inodes": 0, "inodes": 11, "inodes_bottom": 11, "inodes_expired": 0, "inodes_pin_tail": 0, "inodes_pinned": 10, "inodes_top": 0, "inodes_with_caps": 1, "load_cent": 0, "openino_backtrace_fetch": 0, "openino_dir_fetch": 0, "openino_peer_discover": 0, "process_request_cap_release": 1, "q": 0, "reply": 33, "reply_latency": { "avgcount": 33, "avgtime": 0.000162396, "sum": 0.00535907 }, "request": 33, "root_rbytes": 0, "root_rfiles": 0, "root_rsnaps": 0, "scrub_backtrace_fetch": 0, "scrub_backtrace_repaired": 0, "scrub_dir_base_inodes": 0, "scrub_dir_inodes": 0, "scrub_dirfrag_rstats": 0, "scrub_file_inodes": 0, "scrub_inotable_repaired": 0, "scrub_set_tag": 0, "slow_reply": 0, "subtrees": 2, "traverse": 62, "traverse_dir_fetch": 0, "traverse_discover": 0, "traverse_forward": 0, "traverse_hit": 59, "traverse_lock": 0, "traverse_remote_ino": 0 }, "mds_cache": { "dir_handle_discover": 0, "dir_send_discover": 0, "dir_try_discover": 0, "dir_update": 0, "dir_update_receipt": 0, "ireq_enqueue_scrub": 0, "ireq_exportdir": 0, "ireq_flush": 0, "ireq_fragmentdir": 0, "ireq_fragstats": 0, "ireq_inodestats": 0, "ireq_quiesce_inode": 0, "ireq_quiesce_path": 0, "num_recovering_enqueued": 0, "num_recovering_prioritized": 0, "num_recovering_processing": 0, "num_strays": 0, "num_strays_delayed": 0, "num_strays_enqueuing": 0, "recovery_completed": 0, "recovery_started": 0, "strays_created": 0, "strays_enqueued": 0, "strays_migrated": 0, "strays_reintegrated": 0, "uninline_started": 0, "uninline_succeeded": 0, "uninline_write_failed": 0 }, "mds_log": { "ev": 10, "evadd": 10, "evex": 0, "evexd": 0, "evexg": 0, "evlrg": 0, "evtrm": 0, "expos": 4194304, "jlat": { "avgcount": 7, "avgtime": 0.003043394, "sum": 0.021303759 }, "rdpos": 0, "replayed": 0, "seg": 2, "segadd": 2, "segex": 0, "segexd": 0, "segexg": 0, "segmjr": 2, "segtrm": 0, "wrpos": 4197422 }, "mds_mem": { "cap": 0, "cap+": 3, "cap-": 3, "dir": 12, "dir+": 12, "dir-": 0, "dn": 11, "dn+": 11, "dn-": 0, "heap": 332226560, "ino": 13, "ino+": 13, "ino-": 0, "rss": 111024 }, "mds_server": { "cap_acquisition_throttle": 0, "cap_revoke_eviction": 0, "dispatch_client_request": 33, "dispatch_server_request": 0, "handle_client_request": 33, "handle_client_session": 9, "handle_peer_request": 0, "req_create_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_getattr_latency": { "avgcount": 29, "avgtime": 0.000153551, "sum": 0.004452988 }, "req_getfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_getvxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_link_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookup_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookuphash_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupino_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupname_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupparent_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lssnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mkdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mknod_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mksnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_open_latency": { "avgcount": 3, "avgtime": 0.000195806, "sum": 0.000587419 }, "req_readdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rename_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_renamesnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setattr_latency": { "avgcount": 1, "avgtime": 0.000318663, "sum": 0.000318663 }, "req_setdirlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_snapdiff_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_symlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_unlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "mds_sessions": { "average_load": 31, "avg_session_uptime": 1, "mdthresh_evicted": 0, "session_add": 3, "session_count": 0, "session_remove": 3, "sessions_open": 0, "sessions_stale": 0, "total_load": 31 }, "mempool": { "bloom_filter_bytes": 0, "bloom_filter_items": 0, "bluefs_bytes": 0, "bluefs_file_reader_bytes": 0, "bluefs_file_reader_items": 0, "bluefs_file_writer_bytes": 0, "bluefs_file_writer_items": 0, "bluefs_items": 0, "bluestore_alloc_bytes": 0, "bluestore_alloc_items": 0, "bluestore_blob_bytes": 0, "bluestore_blob_items": 0, "bluestore_cache_buffer_bytes": 0, "bluestore_cache_buffer_items": 0, "bluestore_cache_data_bytes": 0, "bluestore_cache_data_items": 0, "bluestore_cache_meta_bytes": 0, "bluestore_cache_meta_items": 0, "bluestore_cache_onode_bytes": 0, "bluestore_cache_onode_items": 0, "bluestore_cache_other_bytes": 0, "bluestore_cache_other_items": 0, "bluestore_extent_bytes": 0, "bluestore_extent_items": 0, "bluestore_fsck_bytes": 0, "bluestore_fsck_items": 0, "bluestore_inline_bl_bytes": 0, "bluestore_inline_bl_items": 0, "bluestore_shared_blob_bytes": 0, "bluestore_shared_blob_items": 0, "bluestore_txc_bytes": 0, "bluestore_txc_items": 0, "bluestore_writing_bytes": 0, "bluestore_writing_deferred_bytes": 0, "bluestore_writing_deferred_items": 0, "bluestore_writing_items": 0, "buffer_anon_bytes": 225169, "buffer_anon_items": 89, "buffer_meta_bytes": 0, "buffer_meta_items": 0, "mds_co_bytes": 53984, "mds_co_items": 218, "osd_bytes": 0, "osd_items": 0, "osd_mapbl_bytes": 0, "osd_mapbl_items": 0, "osd_pglog_bytes": 0, "osd_pglog_items": 0, "osdmap_bytes": 5424, "osdmap_items": 76, "osdmap_mapping_bytes": 0, "osdmap_mapping_items": 0, "pgmap_bytes": 0, "pgmap_items": 0, "unittest_1_bytes": 0, "unittest_1_items": 0, "unittest_2_bytes": 0, "unittest_2_items": 0 }, "objecter": { "command_active": 0, "command_resend": 0, "command_send": 0, "linger_active": 0, "linger_ping": 0, "linger_resend": 0, "linger_send": 0, "map_epoch": 38, "map_full": 0, "map_inc": 1, "omap_del": 1, "omap_rd": 0, "omap_wr": 15, "op": 46, "op_Infinitylight": 0, "op_active": 0, "op_laggy": 0, "op_latency": { "avgcount": 46, "avgtime": 0.020977755, "sum": 0.964976768 }, "op_pg": 0, "op_r": 0, "op_reply": 46, "op_resend": 37, "op_rmw": 0, "op_send": 46, "op_send_bytes": 14328, "op_w": 46, "oplen_avg": { "avgcount": 46, "sum": 70 }, "osd_laggy": 0, "osd_session_close": 0, "osd_session_open": 8, "osd_sessions": 8, "osdop_append": 0, "osdop_call": 0, "osdop_clonerange": 0, "osdop_cmpxattr": 0, "osdop_create": 11, "osdop_delete": 6, "osdop_getxattr": 0, "osdop_mapext": 0, "osdop_notify": 0, "osdop_other": 0, "osdop_pgls": 0, "osdop_pgls_filter": 0, "osdop_read": 0, "osdop_resetxattrs": 0, "osdop_rmxattr": 0, "osdop_setxattr": 22, "osdop_sparse_read": 0, "osdop_src_cmpxattr": 0, "osdop_stat": 0, "osdop_truncate": 0, "osdop_watch": 0, "osdop_write": 7, "osdop_writefull": 8, "osdop_writesame": 0, "osdop_zero": 0, "poolop_active": 0, "poolop_resend": 0, "poolop_send": 0, "poolstat_active": 0, "poolstat_resend": 0, "poolstat_send": 0, "statfs_active": 0, "statfs_resend": 0, "statfs_send": 0 }, "oft": { "omap_total_kv_pairs": 0, "omap_total_objs": 1, "omap_total_removes": 0, "omap_total_updates": 0 }, "purge_queue": { "pq_executed": 0, "pq_executed_ops": 0, "pq_executing": 0, "pq_executing_high_water": 0, "pq_executing_ops": 0, "pq_executing_ops_high_water": 0, "pq_item_in_journal": 0 }, "throttle-msgr_dispatch_throttler-mds": { "get": 151, "get_or_fail_fail": 0, "get_or_fail_success": 151, "get_started": 0, "get_sum": 133225, "max": 104857600, "put": 151, "put_sum": 133225, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_bytes": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 104857600, "put": 46, "put_sum": 14328, "take": 46, "take_sum": 14328, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_ops": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 1024, "put": 46, "put_sum": 46, "take": 46, "take_sum": 46, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 3758096384, "put": 0, "put_sum": 0, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle-0x555fdc7e5380": { "get": 10, "get_or_fail_fail": 0, "get_or_fail_success": 10, "get_started": 0, "get_sum": 3118, "max": 3758096384, "put": 7, "put_sum": 3118, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "trackedop": { "slow_ops_count": 0 } } 2024-09-21T00:26:52.903 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-21T00:26:52.904 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T00:26:52.904 DEBUG:teuthology.orchestra.run.smithi112:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T00:26:52.941 DEBUG:teuthology.orchestra.run.smithi187:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T00:26:52.985 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T00:26:52.985 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T00:26:52.985 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T00:26:52.985 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:26:53.001 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056010,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1439 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248716k,nr_inodes=812179,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T00:26:53.001 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T00:26:53.001 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-09-21T00:26:53.001 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:26:53.044 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056013,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497440k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16835 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248720k,nr_inodes=812180,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497440k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T00:26:53.044 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T00:26:53.044 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T00:26:53.044 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:26:53.060 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056010,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1439 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248716k,nr_inodes=812179,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T00:26:53.060 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T00:26:53.060 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-09-21T00:26:53.060 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:26:53.103 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056013,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497440k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16835 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248720k,nr_inodes=812180,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497440k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T00:26:53.103 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T00:26:53.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.214+0000 7f918414a640 1 -- 172.21.15.112:0/2656606004 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9164005150 msgr2=0x7f916400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:53.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.214+0000 7f918414a640 1 --2- 172.21.15.112:0/2656606004 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9164005150 0x7f916400b9c0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f91740016c0 tx=0x7f9164005830 comp rx=0 tx=0).stop 2024-09-21T00:26:53.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.214+0000 7f91826e7640 1 -- 172.21.15.112:0/2656606004 reap_dead start 2024-09-21T00:26:53.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.214+0000 7f918414a640 1 -- 172.21.15.112:0/2656606004 shutdown_connections 2024-09-21T00:26:53.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.215+0000 7f918414a640 1 -- 172.21.15.112:0/2656606004 >> 172.21.15.112:0/2656606004 conn(0x7f917c0d1140 msgr2=0x7f917c0d1550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:53.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.215+0000 7f918414a640 1 -- 172.21.15.112:0/2656606004 shutdown_connections 2024-09-21T00:26:53.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.215+0000 7f918414a640 1 -- 172.21.15.112:0/2656606004 wait complete. 2024-09-21T00:26:53.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.216+0000 7f918414a640 1 Processor -- start 2024-09-21T00:26:53.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.216+0000 7f918414a640 1 -- start start 2024-09-21T00:26:53.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.218+0000 7f918414a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9164005150 0x7f917c15c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:53.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.218+0000 7f918414a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f917c0cf800 0x7f917c0cfc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:53.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.218+0000 7f918414a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f917c15e280 0x7f917c15e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:53.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.218+0000 7f918414a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f917c06ff00 con 0x7f917c15e280 2024-09-21T00:26:53.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.218+0000 7f918414a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f917c0cf4a0 con 0x7f917c0cf800 2024-09-21T00:26:53.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.218+0000 7f918414a640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f917c070080 con 0x7f9164005150 2024-09-21T00:26:53.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.218+0000 7f91826e7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f917c15e280 0x7f917c15e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:53.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.218+0000 7f91826e7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f917c15e280 0x7f917c15e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60922/0 (socket says 172.21.15.112:60922) 2024-09-21T00:26:53.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.218+0000 7f91826e7640 1 -- 172.21.15.112:0/185690962 learned_addr learned my addr 172.21.15.112:0/185690962 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:26:53.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.218+0000 7f9181ee6640 1 --2- 172.21.15.112:0/185690962 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9164005150 0x7f917c15c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:53.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.218+0000 7f91816e5640 1 --2- 172.21.15.112:0/185690962 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f917c0cf800 0x7f917c0cfc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:53.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.218+0000 7f91826e7640 1 -- 172.21.15.112:0/185690962 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9164005150 msgr2=0x7f917c15c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:53.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.218+0000 7f91826e7640 1 --2- 172.21.15.112:0/185690962 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9164005150 0x7f917c15c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:53.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.218+0000 7f91826e7640 1 -- 172.21.15.112:0/185690962 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f917c0cf800 msgr2=0x7f917c0cfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:53.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.218+0000 7f91826e7640 1 --2- 172.21.15.112:0/185690962 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f917c0cf800 0x7f917c0cfc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:53.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.218+0000 7f91826e7640 1 -- 172.21.15.112:0/185690962 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f917c156ff0 con 0x7f917c15e280 2024-09-21T00:26:53.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.218+0000 7f91826e7640 1 --2- 172.21.15.112:0/185690962 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f917c15e280 0x7f917c15e690 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f91740017b0 tx=0x7f9174006450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:53.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.218+0000 7f917a7fc640 1 -- 172.21.15.112:0/185690962 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9174006920 con 0x7f917c15e280 2024-09-21T00:26:53.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.218+0000 7f917a7fc640 1 -- 172.21.15.112:0/185690962 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9174003a00 con 0x7f917c15e280 2024-09-21T00:26:53.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.218+0000 7f918414a640 1 -- 172.21.15.112:0/185690962 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f917c1572d0 con 0x7f917c15e280 2024-09-21T00:26:53.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.218+0000 7f917a7fc640 1 -- 172.21.15.112:0/185690962 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9174003e10 con 0x7f917c15e280 2024-09-21T00:26:53.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.218+0000 7f918414a640 1 -- 172.21.15.112:0/185690962 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f917c157890 con 0x7f917c15e280 2024-09-21T00:26:53.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.219+0000 7f918414a640 1 -- 172.21.15.112:0/185690962 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f917c073ea0 con 0x7f917c15e280 2024-09-21T00:26:53.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.220+0000 7f917a7fc640 1 -- 172.21.15.112:0/185690962 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9174030050 con 0x7f917c15e280 2024-09-21T00:26:53.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.220+0000 7f917a7fc640 1 --2- 172.21.15.112:0/185690962 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f9168078b90 0x7f916807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:53.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.221+0000 7f917a7fc640 1 -- 172.21.15.112:0/185690962 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f91740bc680 con 0x7f917c15e280 2024-09-21T00:26:53.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.221+0000 7f9181ee6640 1 --2- 172.21.15.112:0/185690962 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f9168078b90 0x7f916807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:53.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.221+0000 7f9181ee6640 1 --2- 172.21.15.112:0/185690962 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f9168078b90 0x7f916807b0a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f916c0015a0 tx=0x7f916c006b20 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:53.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.225+0000 7f917a7fc640 1 -- 172.21.15.112:0/185690962 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f9174084930 con 0x7f917c15e280 2024-09-21T00:26:53.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.439+0000 7f918414a640 1 -- 172.21.15.112:0/185690962 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f917c15b050 con 0x7f917c15e280 2024-09-21T00:26:53.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.440+0000 7f917a7fc640 1 -- 172.21.15.112:0/185690962 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5232 (secure 0 0 0) 0x7f9174084230 con 0x7f917c15e280 2024-09-21T00:26:53.441 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:26:53.441 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":17,"btime":"2024-09-21T00:26:39:841719+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4846,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6834/2036740840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":2036740840},{"type":"v1","addr":"172.21.15.112:6834","nonce":2036740840}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4855,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6836/3699399640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":3699399640},{"type":"v1","addr":"172.21.15.187:6836","nonce":3699399640}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4858,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6837/694936803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6835","nonce":694936803},{"type":"v1","addr":"172.21.15.187:6837","nonce":694936803}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T00:26:36.187741+0000","modified":"2024-09-21T00:26:38.947874+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5028},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5028":{"gid":5028,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6835/4012316979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":4012316979},{"type":"v1","addr":"172.21.15.112:6835","nonce":4012316979}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5028,"qdb_cluster":[5028]},"id":2}]} 2024-09-21T00:26:53.441 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 17 2024-09-21T00:26:53.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.442+0000 7f918414a640 1 -- 172.21.15.112:0/185690962 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f9168078b90 msgr2=0x7f916807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:53.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.442+0000 7f918414a640 1 --2- 172.21.15.112:0/185690962 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f9168078b90 0x7f916807b0a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f916c0015a0 tx=0x7f916c006b20 comp rx=0 tx=0).stop 2024-09-21T00:26:53.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.442+0000 7f918414a640 1 -- 172.21.15.112:0/185690962 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f917c15e280 msgr2=0x7f917c15e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:53.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.442+0000 7f918414a640 1 --2- 172.21.15.112:0/185690962 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f917c15e280 0x7f917c15e690 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f91740017b0 tx=0x7f9174006450 comp rx=0 tx=0).stop 2024-09-21T00:26:53.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.442+0000 7f918414a640 1 -- 172.21.15.112:0/185690962 shutdown_connections 2024-09-21T00:26:53.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.442+0000 7f918414a640 1 --2- 172.21.15.112:0/185690962 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f9168078b90 0x7f916807b0a0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:53.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.442+0000 7f918414a640 1 --2- 172.21.15.112:0/185690962 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f917c15e280 0x7f917c15e690 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:53.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.442+0000 7f918414a640 1 --2- 172.21.15.112:0/185690962 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f917c0cf800 0x7f917c0cfc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:53.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.442+0000 7f918414a640 1 --2- 172.21.15.112:0/185690962 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9164005150 0x7f917c15c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:53.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.442+0000 7f918414a640 1 -- 172.21.15.112:0/185690962 >> 172.21.15.112:0/185690962 conn(0x7f917c0d1140 msgr2=0x7f917c0d1550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:53.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.442+0000 7f918414a640 1 -- 172.21.15.112:0/185690962 shutdown_connections 2024-09-21T00:26:53.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.442+0000 7f918414a640 1 -- 172.21.15.112:0/185690962 wait complete. 2024-09-21T00:26:53.454 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T00:26:53.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.566+0000 7f2260d69640 1 -- 172.21.15.112:0/3121338437 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f225c0d12e0 msgr2=0x7f225c0d16f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:53.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.566+0000 7f2260d69640 1 --2- 172.21.15.112:0/3121338437 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f225c0d12e0 0x7f225c0d16f0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f2244009830 tx=0x7f224402a510 comp rx=0 tx=0).stop 2024-09-21T00:26:53.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.566+0000 7f2260d69640 1 -- 172.21.15.112:0/3121338437 shutdown_connections 2024-09-21T00:26:53.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.566+0000 7f2260d69640 1 --2- 172.21.15.112:0/3121338437 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f225c0d12e0 0x7f225c0d16f0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:53.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.566+0000 7f2260d69640 1 --2- 172.21.15.112:0/3121338437 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f225c15cb10 0x7f225c15cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:53.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.566+0000 7f2260d69640 1 -- 172.21.15.112:0/3121338437 >> 172.21.15.112:0/3121338437 conn(0x7f225c0d2350 msgr2=0x7f225c0d2760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:53.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.566+0000 7f2260d69640 1 -- 172.21.15.112:0/3121338437 shutdown_connections 2024-09-21T00:26:53.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.566+0000 7f2260d69640 1 -- 172.21.15.112:0/3121338437 wait complete. 2024-09-21T00:26:53.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.568+0000 7f2260d69640 1 Processor -- start 2024-09-21T00:26:53.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.568+0000 7f2260d69640 1 -- start start 2024-09-21T00:26:53.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.569+0000 7f2260d69640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f225c0d12e0 0x7f225c14ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:53.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.569+0000 7f2260d69640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f225c15cb10 0x7f225c0b61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:53.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.569+0000 7f2260d69640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f225c160100 0x7f225c0b6700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:53.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.569+0000 7f2260d69640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f225c0b73e0 con 0x7f225c0d12e0 2024-09-21T00:26:53.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.569+0000 7f2260d69640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f225c0b7260 con 0x7f225c160100 2024-09-21T00:26:53.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.569+0000 7f2260d69640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f225c160c20 con 0x7f225c15cb10 2024-09-21T00:26:53.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.569+0000 7f225a59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f225c0d12e0 0x7f225c14ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:53.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.569+0000 7f225a59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f225c0d12e0 0x7f225c14ffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60930/0 (socket says 172.21.15.112:60930) 2024-09-21T00:26:53.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.570+0000 7f225a59c640 1 -- 172.21.15.112:0/1096519087 learned_addr learned my addr 172.21.15.112:0/1096519087 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:26:53.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.570+0000 7f225ad9d640 1 --2- 172.21.15.112:0/1096519087 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f225c160100 0x7f225c0b6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:53.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.570+0000 7f2251d9b640 1 --2- 172.21.15.112:0/1096519087 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f225c15cb10 0x7f225c0b61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:53.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.570+0000 7f225a59c640 1 -- 172.21.15.112:0/1096519087 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f225c15cb10 msgr2=0x7f225c0b61c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:53.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.570+0000 7f225a59c640 1 --2- 172.21.15.112:0/1096519087 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f225c15cb10 0x7f225c0b61c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:53.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.570+0000 7f225a59c640 1 -- 172.21.15.112:0/1096519087 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f225c160100 msgr2=0x7f225c0b6700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:53.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.570+0000 7f225a59c640 1 --2- 172.21.15.112:0/1096519087 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f225c160100 0x7f225c0b6700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:53.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.570+0000 7f225a59c640 1 -- 172.21.15.112:0/1096519087 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f225c0cb300 con 0x7f225c0d12e0 2024-09-21T00:26:53.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.570+0000 7f225ad9d640 1 --2- 172.21.15.112:0/1096519087 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f225c160100 0x7f225c0b6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:26:53.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.570+0000 7f225a59c640 1 --2- 172.21.15.112:0/1096519087 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f225c0d12e0 0x7f225c14ffe0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f224c00c9a0 tx=0x7f224c00ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:53.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.570+0000 7f22537fe640 1 -- 172.21.15.112:0/1096519087 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f224c007bb0 con 0x7f225c0d12e0 2024-09-21T00:26:53.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.570+0000 7f22537fe640 1 -- 172.21.15.112:0/1096519087 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f224c007e40 con 0x7f225c0d12e0 2024-09-21T00:26:53.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.570+0000 7f2260d69640 1 -- 172.21.15.112:0/1096519087 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f225c06e380 con 0x7f225c0d12e0 2024-09-21T00:26:53.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.570+0000 7f22537fe640 1 -- 172.21.15.112:0/1096519087 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f224c00f690 con 0x7f225c0d12e0 2024-09-21T00:26:53.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.570+0000 7f2260d69640 1 -- 172.21.15.112:0/1096519087 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f225c06e940 con 0x7f225c0d12e0 2024-09-21T00:26:53.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.571+0000 7f2260d69640 1 -- 172.21.15.112:0/1096519087 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f225c0cb4d0 con 0x7f225c0d12e0 2024-09-21T00:26:53.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.572+0000 7f22537fe640 1 -- 172.21.15.112:0/1096519087 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f224c00f830 con 0x7f225c0d12e0 2024-09-21T00:26:53.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.573+0000 7f22537fe640 1 --2- 172.21.15.112:0/1096519087 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f223c078b90 0x7f223c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:53.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.573+0000 7f22537fe640 1 -- 172.21.15.112:0/1096519087 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f224c09b6a0 con 0x7f225c0d12e0 2024-09-21T00:26:53.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.573+0000 7f2251d9b640 1 --2- 172.21.15.112:0/1096519087 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f223c078b90 0x7f223c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:53.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.574+0000 7f2251d9b640 1 --2- 172.21.15.112:0/1096519087 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f223c078b90 0x7f223c07b0a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2244034000 tx=0x7f2244033040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:53.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.577+0000 7f22537fe640 1 -- 172.21.15.112:0/1096519087 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f224c014030 con 0x7f225c0d12e0 2024-09-21T00:26:53.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.791+0000 7f2260d69640 1 -- 172.21.15.112:0/1096519087 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f225c0d16f0 con 0x7f225c0d12e0 2024-09-21T00:26:53.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.791+0000 7f22537fe640 1 -- 172.21.15.112:0/1096519087 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5232 (secure 0 0 0) 0x7f224c063960 con 0x7f225c0d12e0 2024-09-21T00:26:53.792 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:26:53.792 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":17,"btime":"2024-09-21T00:26:39:841719+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4846,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6834/2036740840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":2036740840},{"type":"v1","addr":"172.21.15.112:6834","nonce":2036740840}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4855,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6836/3699399640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":3699399640},{"type":"v1","addr":"172.21.15.187:6836","nonce":3699399640}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4858,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6837/694936803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6835","nonce":694936803},{"type":"v1","addr":"172.21.15.187:6837","nonce":694936803}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T00:26:36.187741+0000","modified":"2024-09-21T00:26:38.947874+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5028},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5028":{"gid":5028,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6835/4012316979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":4012316979},{"type":"v1","addr":"172.21.15.112:6835","nonce":4012316979}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5028,"qdb_cluster":[5028]},"id":2}]} 2024-09-21T00:26:53.793 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 17 2024-09-21T00:26:53.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.793+0000 7f2260d69640 1 -- 172.21.15.112:0/1096519087 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f223c078b90 msgr2=0x7f223c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:53.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.793+0000 7f2260d69640 1 --2- 172.21.15.112:0/1096519087 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f223c078b90 0x7f223c07b0a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2244034000 tx=0x7f2244033040 comp rx=0 tx=0).stop 2024-09-21T00:26:53.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.793+0000 7f2260d69640 1 -- 172.21.15.112:0/1096519087 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f225c0d12e0 msgr2=0x7f225c14ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:53.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.793+0000 7f2260d69640 1 --2- 172.21.15.112:0/1096519087 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f225c0d12e0 0x7f225c14ffe0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f224c00c9a0 tx=0x7f224c00ce60 comp rx=0 tx=0).stop 2024-09-21T00:26:53.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.793+0000 7f2260d69640 1 -- 172.21.15.112:0/1096519087 shutdown_connections 2024-09-21T00:26:53.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.793+0000 7f2260d69640 1 --2- 172.21.15.112:0/1096519087 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f225c160100 0x7f225c0b6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:53.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.793+0000 7f2260d69640 1 --2- 172.21.15.112:0/1096519087 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f225c15cb10 0x7f225c0b61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:53.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.793+0000 7f2260d69640 1 --2- 172.21.15.112:0/1096519087 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f223c078b90 0x7f223c07b0a0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:53.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.793+0000 7f2260d69640 1 --2- 172.21.15.112:0/1096519087 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f225c0d12e0 0x7f225c14ffe0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:53.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.793+0000 7f2260d69640 1 -- 172.21.15.112:0/1096519087 >> 172.21.15.112:0/1096519087 conn(0x7f225c0d2350 msgr2=0x7f225c152300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:53.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.794+0000 7f2260d69640 1 -- 172.21.15.112:0/1096519087 shutdown_connections 2024-09-21T00:26:53.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.794+0000 7f2260d69640 1 -- 172.21.15.112:0/1096519087 wait complete. 2024-09-21T00:26:53.806 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T00:26:53.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.917+0000 7f478a6de640 1 -- 172.21.15.112:0/2395695289 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f47841536e0 msgr2=0x7f47840d17c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:53.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.917+0000 7f478a6de640 1 --2- 172.21.15.112:0/2395695289 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f47841536e0 0x7f47840d17c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4774005fd0 tx=0x7f477402a5c0 comp rx=0 tx=0).stop 2024-09-21T00:26:53.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.917+0000 7f478a6de640 1 -- 172.21.15.112:0/2395695289 shutdown_connections 2024-09-21T00:26:53.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.917+0000 7f478a6de640 1 --2- 172.21.15.112:0/2395695289 >> v2:172.21.15.187:3300/0 conn(0x7f47840d1d00 0x7f47840cc930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:53.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.917+0000 7f478a6de640 1 --2- 172.21.15.112:0/2395695289 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f47841536e0 0x7f47840d17c0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:53.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.917+0000 7f478a6de640 1 -- 172.21.15.112:0/2395695289 >> 172.21.15.112:0/2395695289 conn(0x7f47840cef40 msgr2=0x7f47840cf350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:53.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.917+0000 7f478a6de640 1 -- 172.21.15.112:0/2395695289 shutdown_connections 2024-09-21T00:26:53.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.917+0000 7f478a6de640 1 -- 172.21.15.112:0/2395695289 wait complete. 2024-09-21T00:26:53.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.918+0000 7f478a6de640 1 Processor -- start 2024-09-21T00:26:53.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.919+0000 7f478a6de640 1 -- start start 2024-09-21T00:26:53.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.920+0000 7f478a6de640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f47840cce70 0x7f47841566b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:53.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.920+0000 7f478a6de640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47840d1d00 0x7f478415c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:53.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.920+0000 7f478a6de640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f47841536e0 0x7f47841544b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:53.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.920+0000 7f478a6de640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f47841608a0 con 0x7f47840d1d00 2024-09-21T00:26:53.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.920+0000 7f478a6de640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f4784160a20 con 0x7f47840cce70 2024-09-21T00:26:53.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.920+0000 7f478a6de640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f4784160ba0 con 0x7f47841536e0 2024-09-21T00:26:53.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.920+0000 7f47837fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47840d1d00 0x7f478415c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:53.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.920+0000 7f47837fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47840d1d00 0x7f478415c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60936/0 (socket says 172.21.15.112:60936) 2024-09-21T00:26:53.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.920+0000 7f47837fe640 1 -- 172.21.15.112:0/986022003 learned_addr learned my addr 172.21.15.112:0/986022003 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:26:53.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.920+0000 7f4783fff640 1 --2- 172.21.15.112:0/986022003 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f47840cce70 0x7f47841566b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:53.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.920+0000 7f4788c7b640 1 --2- 172.21.15.112:0/986022003 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f47841536e0 0x7f47841544b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:53.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.920+0000 7f47837fe640 1 -- 172.21.15.112:0/986022003 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f47841536e0 msgr2=0x7f47841544b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:53.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.920+0000 7f47837fe640 1 --2- 172.21.15.112:0/986022003 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f47841536e0 0x7f47841544b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:53.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.920+0000 7f47837fe640 1 -- 172.21.15.112:0/986022003 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f47840cce70 msgr2=0x7f47841566b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:53.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.920+0000 7f47837fe640 1 --2- 172.21.15.112:0/986022003 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f47840cce70 0x7f47841566b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:53.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.920+0000 7f47837fe640 1 -- 172.21.15.112:0/986022003 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47840ca2c0 con 0x7f47840d1d00 2024-09-21T00:26:53.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.920+0000 7f4783fff640 1 --2- 172.21.15.112:0/986022003 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f47840cce70 0x7f47841566b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:26:53.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.921+0000 7f47837fe640 1 --2- 172.21.15.112:0/986022003 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47840d1d00 0x7f478415c450 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f477800ea20 tx=0x7f477800eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:53.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.921+0000 7f4780ff9640 1 -- 172.21.15.112:0/986022003 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f477800cd80 con 0x7f47840d1d00 2024-09-21T00:26:53.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.921+0000 7f4780ff9640 1 -- 172.21.15.112:0/986022003 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4778004020 con 0x7f47840d1d00 2024-09-21T00:26:53.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.921+0000 7f478a6de640 1 -- 172.21.15.112:0/986022003 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47841f5d50 con 0x7f47840d1d00 2024-09-21T00:26:53.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.921+0000 7f4780ff9640 1 -- 172.21.15.112:0/986022003 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4778004470 con 0x7f47840d1d00 2024-09-21T00:26:53.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.921+0000 7f478a6de640 1 -- 172.21.15.112:0/986022003 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47841f6310 con 0x7f47840d1d00 2024-09-21T00:26:53.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.922+0000 7f478a6de640 1 -- 172.21.15.112:0/986022003 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47840cc930 con 0x7f47840d1d00 2024-09-21T00:26:53.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.922+0000 7f4780ff9640 1 -- 172.21.15.112:0/986022003 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f47780035f0 con 0x7f47840d1d00 2024-09-21T00:26:53.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.923+0000 7f4780ff9640 1 --2- 172.21.15.112:0/986022003 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4770078b90 0x7f477007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:53.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.923+0000 7f4780ff9640 1 -- 172.21.15.112:0/986022003 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f477809b780 con 0x7f47840d1d00 2024-09-21T00:26:53.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.923+0000 7f4783fff640 1 --2- 172.21.15.112:0/986022003 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4770078b90 0x7f477007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:53.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.924+0000 7f4783fff640 1 --2- 172.21.15.112:0/986022003 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4770078b90 0x7f477007b0a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f4774033000 tx=0x7f4774004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:53.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:53.928+0000 7f4780ff9640 1 -- 172.21.15.112:0/986022003 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f4778063a30 con 0x7f47840d1d00 2024-09-21T00:26:54.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.104+0000 7f478a6de640 1 -- 172.21.15.112:0/986022003 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4784160280 con 0x7f47840d1d00 2024-09-21T00:26:54.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.105+0000 7f4780ff9640 1 -- 172.21.15.112:0/986022003 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) ==== 74+0+17026 (secure 0 0 0) 0x7f4778063330 con 0x7f47840d1d00 2024-09-21T00:26:54.106 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:26:54.106 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":38,"fsid":"8b34692b-dc65-49b2-a08f-26f707b96ac5","created":"2024-09-21T00:25:22.788490+0000","modified":"2024-09-21T00:26:37.831810+0000","last_up_change":"2024-09-21T00:25:36.161835+0000","last_in_change":"2024-09-21T00:25:26.346500+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T00:25:37.105006+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-09-21T00:26:34.140543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-09-21T00:26:35.143821+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9c575086-67d2-4de2-b01b-d44dcc21d604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6809","nonce":755760621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6811","nonce":755760621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6815","nonce":755760621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6813","nonce":755760621}]},"public_addr":"172.21.15.112:6809/755760621","cluster_addr":"172.21.15.112:6811/755760621","heartbeat_back_addr":"172.21.15.112:6815/755760621","heartbeat_front_addr":"172.21.15.112:6813/755760621","state":["exists","up"]},{"osd":1,"uuid":"a983c227-a3ce-4bdd-82f1-438ba7e3ecd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6801","nonce":344932592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6803","nonce":344932592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6807","nonce":344932592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6805","nonce":344932592}]},"public_addr":"172.21.15.112:6801/344932592","cluster_addr":"172.21.15.112:6803/344932592","heartbeat_back_addr":"172.21.15.112:6807/344932592","heartbeat_front_addr":"172.21.15.112:6805/344932592","state":["exists","up"]},{"osd":2,"uuid":"1193c1d8-bfff-4a57-8200-5f69bf1ea3af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6825","nonce":407933616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6827","nonce":407933616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6831","nonce":407933616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6829","nonce":407933616}]},"public_addr":"172.21.15.112:6825/407933616","cluster_addr":"172.21.15.112:6827/407933616","heartbeat_back_addr":"172.21.15.112:6831/407933616","heartbeat_front_addr":"172.21.15.112:6829/407933616","state":["exists","up"]},{"osd":3,"uuid":"af04330a-36ad-40cc-a8dd-fa259b221de5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6817","nonce":4120164806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6819","nonce":4120164806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6823","nonce":4120164806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6821","nonce":4120164806}]},"public_addr":"172.21.15.112:6817/4120164806","cluster_addr":"172.21.15.112:6819/4120164806","heartbeat_back_addr":"172.21.15.112:6823/4120164806","heartbeat_front_addr":"172.21.15.112:6821/4120164806","state":["exists","up"]},{"osd":4,"uuid":"5b893111-292c-4c5a-b184-53ad38c7199e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6803","nonce":522077560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6805","nonce":522077560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6809","nonce":522077560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6807","nonce":522077560}]},"public_addr":"172.21.15.187:6803/522077560","cluster_addr":"172.21.15.187:6805/522077560","heartbeat_back_addr":"172.21.15.187:6809/522077560","heartbeat_front_addr":"172.21.15.187:6807/522077560","state":["exists","up"]},{"osd":5,"uuid":"c0b8e011-45fe-4c5e-925e-6be2a7f1868a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6827","nonce":3593144721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6829","nonce":3593144721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6833","nonce":3593144721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6831","nonce":3593144721}]},"public_addr":"172.21.15.187:6827/3593144721","cluster_addr":"172.21.15.187:6829/3593144721","heartbeat_back_addr":"172.21.15.187:6833/3593144721","heartbeat_front_addr":"172.21.15.187:6831/3593144721","state":["exists","up"]},{"osd":6,"uuid":"87d2d650-7489-4593-98a3-a1f433ce8dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6811","nonce":1494387657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6813","nonce":1494387657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6817","nonce":1494387657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6815","nonce":1494387657}]},"public_addr":"172.21.15.187:6811/1494387657","cluster_addr":"172.21.15.187:6813/1494387657","heartbeat_back_addr":"172.21.15.187:6817/1494387657","heartbeat_front_addr":"172.21.15.187:6815/1494387657","state":["exists","up"]},{"osd":7,"uuid":"c6b4f8e2-a1ee-4438-aa37-67ff4b6ef1f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6819","nonce":1624699842}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6821","nonce":1624699842}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6825","nonce":1624699842}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6823","nonce":1624699842}]},"public_addr":"172.21.15.187:6819/1624699842","cluster_addr":"172.21.15.187:6821/1624699842","heartbeat_back_addr":"172.21.15.187:6825/1624699842","heartbeat_front_addr":"172.21.15.187:6823/1624699842","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.873162+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.400506+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.664317+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.159791+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.202630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.113212+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.384786+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.448441+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T00:26:54.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.107+0000 7f478a6de640 1 -- 172.21.15.112:0/986022003 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4770078b90 msgr2=0x7f477007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:54.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.107+0000 7f478a6de640 1 --2- 172.21.15.112:0/986022003 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4770078b90 0x7f477007b0a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f4774033000 tx=0x7f4774004730 comp rx=0 tx=0).stop 2024-09-21T00:26:54.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.107+0000 7f478a6de640 1 -- 172.21.15.112:0/986022003 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47840d1d00 msgr2=0x7f478415c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:54.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.107+0000 7f478a6de640 1 --2- 172.21.15.112:0/986022003 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47840d1d00 0x7f478415c450 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f477800ea20 tx=0x7f477800eee0 comp rx=0 tx=0).stop 2024-09-21T00:26:54.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.108+0000 7f478a6de640 1 -- 172.21.15.112:0/986022003 shutdown_connections 2024-09-21T00:26:54.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.108+0000 7f478a6de640 1 --2- 172.21.15.112:0/986022003 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f47841536e0 0x7f47841544b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:54.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.108+0000 7f478a6de640 1 --2- 172.21.15.112:0/986022003 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4770078b90 0x7f477007b0a0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:54.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.108+0000 7f478a6de640 1 --2- 172.21.15.112:0/986022003 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47840d1d00 0x7f478415c450 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:54.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.108+0000 7f478a6de640 1 --2- 172.21.15.112:0/986022003 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f47840cce70 0x7f47841566b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:54.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.108+0000 7f478a6de640 1 -- 172.21.15.112:0/986022003 >> 172.21.15.112:0/986022003 conn(0x7f47840cef40 msgr2=0x7f4784070100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:54.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.108+0000 7f478a6de640 1 -- 172.21.15.112:0/986022003 shutdown_connections 2024-09-21T00:26:54.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.108+0000 7f478a6de640 1 -- 172.21.15.112:0/986022003 wait complete. 2024-09-21T00:26:54.120 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-21T00:26:54.120 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T00:26:54.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.231+0000 7f4f19739640 1 -- 172.21.15.112:0/600275200 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4f14162590 msgr2=0x7f4f141629a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:54.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.231+0000 7f4f19739640 1 --2- 172.21.15.112:0/600275200 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4f14162590 0x7f4f141629a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4f08005fd0 tx=0x7f4f0802a5d0 comp rx=0 tx=0).stop 2024-09-21T00:26:54.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.231+0000 7f4f19739640 1 -- 172.21.15.112:0/600275200 shutdown_connections 2024-09-21T00:26:54.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.231+0000 7f4f19739640 1 --2- 172.21.15.112:0/600275200 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4f14162590 0x7f4f141629a0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:54.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.231+0000 7f4f19739640 1 --2- 172.21.15.112:0/600275200 >> v2:172.21.15.187:3300/0 conn(0x7f4f140cfe00 0x7f4f140d2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:54.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.231+0000 7f4f19739640 1 -- 172.21.15.112:0/600275200 >> 172.21.15.112:0/600275200 conn(0x7f4f140d4440 msgr2=0x7f4f140d4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:54.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.231+0000 7f4f19739640 1 -- 172.21.15.112:0/600275200 shutdown_connections 2024-09-21T00:26:54.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.231+0000 7f4f19739640 1 -- 172.21.15.112:0/600275200 wait complete. 2024-09-21T00:26:54.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.233+0000 7f4f19739640 1 Processor -- start 2024-09-21T00:26:54.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.233+0000 7f4f19739640 1 -- start start 2024-09-21T00:26:54.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.234+0000 7f4f19739640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4f140b2e60 0x7f4f14158440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:54.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.234+0000 7f4f19739640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4f140cfe00 0x7f4f1415dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:54.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.234+0000 7f4f19739640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4f14162590 0x7f4f141f8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:54.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.234+0000 7f4f19739640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4f140b3980 con 0x7f4f140cfe00 2024-09-21T00:26:54.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.234+0000 7f4f19739640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f4f140cf5a0 con 0x7f4f14162590 2024-09-21T00:26:54.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.234+0000 7f4f19739640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f4f140b3b00 con 0x7f4f140b2e60 2024-09-21T00:26:54.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.234+0000 7f4f127fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4f140cfe00 0x7f4f1415dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:54.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.234+0000 7f4f127fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4f140cfe00 0x7f4f1415dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60944/0 (socket says 172.21.15.112:60944) 2024-09-21T00:26:54.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.234+0000 7f4f127fc640 1 -- 172.21.15.112:0/169262143 learned_addr learned my addr 172.21.15.112:0/169262143 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:26:54.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.234+0000 7f4f137fe640 1 --2- 172.21.15.112:0/169262143 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4f14162590 0x7f4f141f8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:54.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.234+0000 7f4f12ffd640 1 --2- 172.21.15.112:0/169262143 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4f140b2e60 0x7f4f14158440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:54.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.235+0000 7f4f127fc640 1 -- 172.21.15.112:0/169262143 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4f140b2e60 msgr2=0x7f4f14158440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:54.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.235+0000 7f4f127fc640 1 --2- 172.21.15.112:0/169262143 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4f140b2e60 0x7f4f14158440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:54.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.235+0000 7f4f127fc640 1 -- 172.21.15.112:0/169262143 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4f14162590 msgr2=0x7f4f141f8a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:54.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.235+0000 7f4f127fc640 1 --2- 172.21.15.112:0/169262143 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4f14162590 0x7f4f141f8a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:54.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.235+0000 7f4f127fc640 1 -- 172.21.15.112:0/169262143 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f1415ddd0 con 0x7f4f140cfe00 2024-09-21T00:26:54.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.235+0000 7f4f127fc640 1 --2- 172.21.15.112:0/169262143 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4f140cfe00 0x7f4f1415dfc0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f4f08034040 tx=0x7f4f08005770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:54.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.235+0000 7f4efb7fe640 1 -- 172.21.15.112:0/169262143 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f0803e030 con 0x7f4f140cfe00 2024-09-21T00:26:54.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.235+0000 7f4f19739640 1 -- 172.21.15.112:0/169262143 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f140ce830 con 0x7f4f140cfe00 2024-09-21T00:26:54.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.235+0000 7f4efb7fe640 1 -- 172.21.15.112:0/169262143 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4f08003a70 con 0x7f4f140cfe00 2024-09-21T00:26:54.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.235+0000 7f4efb7fe640 1 -- 172.21.15.112:0/169262143 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f08035040 con 0x7f4f140cfe00 2024-09-21T00:26:54.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.235+0000 7f4f19739640 1 -- 172.21.15.112:0/169262143 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f140ce1e0 con 0x7f4f140cfe00 2024-09-21T00:26:54.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.236+0000 7f4f19739640 1 -- 172.21.15.112:0/169262143 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f140d2a20 con 0x7f4f140cfe00 2024-09-21T00:26:54.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.237+0000 7f4efb7fe640 1 -- 172.21.15.112:0/169262143 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f4f08041030 con 0x7f4f140cfe00 2024-09-21T00:26:54.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.237+0000 7f4efb7fe640 1 --2- 172.21.15.112:0/169262143 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4f00078bc0 0x7f4f0007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:54.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.237+0000 7f4efb7fe640 1 -- 172.21.15.112:0/169262143 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f4f080bb230 con 0x7f4f140cfe00 2024-09-21T00:26:54.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.238+0000 7f4f12ffd640 1 --2- 172.21.15.112:0/169262143 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4f00078bc0 0x7f4f0007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:54.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.238+0000 7f4f12ffd640 1 --2- 172.21.15.112:0/169262143 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4f00078bc0 0x7f4f0007b0d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f4f04009b30 tx=0x7f4f04005f90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:54.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.242+0000 7f4efb7fe640 1 -- 172.21.15.112:0/169262143 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f4f08082f30 con 0x7f4f140cfe00 2024-09-21T00:26:54.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.453+0000 7f4f19739640 1 -- 172.21.15.112:0/169262143 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4f140c7ef0 con 0x7f4f140cfe00 2024-09-21T00:26:54.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.453+0000 7f4efb7fe640 1 -- 172.21.15.112:0/169262143 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5232 (secure 0 0 0) 0x7f4f08087960 con 0x7f4f140cfe00 2024-09-21T00:26:54.454 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:26:54.454 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":17,"btime":"2024-09-21T00:26:39:841719+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4846,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6834/2036740840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":2036740840},{"type":"v1","addr":"172.21.15.112:6834","nonce":2036740840}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4855,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6836/3699399640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":3699399640},{"type":"v1","addr":"172.21.15.187:6836","nonce":3699399640}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4858,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6837/694936803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6835","nonce":694936803},{"type":"v1","addr":"172.21.15.187:6837","nonce":694936803}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T00:26:36.187741+0000","modified":"2024-09-21T00:26:38.947874+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5028},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5028":{"gid":5028,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6835/4012316979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":4012316979},{"type":"v1","addr":"172.21.15.112:6835","nonce":4012316979}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5028,"qdb_cluster":[5028]},"id":2}]} 2024-09-21T00:26:54.455 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 17 2024-09-21T00:26:54.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.456+0000 7f4f19739640 1 -- 172.21.15.112:0/169262143 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4f00078bc0 msgr2=0x7f4f0007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:54.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.456+0000 7f4f19739640 1 --2- 172.21.15.112:0/169262143 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4f00078bc0 0x7f4f0007b0d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f4f04009b30 tx=0x7f4f04005f90 comp rx=0 tx=0).stop 2024-09-21T00:26:54.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.456+0000 7f4f19739640 1 -- 172.21.15.112:0/169262143 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4f140cfe00 msgr2=0x7f4f1415dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:54.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.456+0000 7f4f19739640 1 --2- 172.21.15.112:0/169262143 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4f140cfe00 0x7f4f1415dfc0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f4f08034040 tx=0x7f4f08005770 comp rx=0 tx=0).stop 2024-09-21T00:26:54.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.456+0000 7f4f19739640 1 -- 172.21.15.112:0/169262143 shutdown_connections 2024-09-21T00:26:54.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.456+0000 7f4f19739640 1 --2- 172.21.15.112:0/169262143 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4f14162590 0x7f4f141f8a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:54.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.456+0000 7f4f19739640 1 --2- 172.21.15.112:0/169262143 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4f00078bc0 0x7f4f0007b0d0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:54.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.456+0000 7f4f19739640 1 --2- 172.21.15.112:0/169262143 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4f140cfe00 0x7f4f1415dfc0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:54.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.456+0000 7f4f19739640 1 --2- 172.21.15.112:0/169262143 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4f140b2e60 0x7f4f14158440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:54.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.456+0000 7f4f19739640 1 -- 172.21.15.112:0/169262143 >> 172.21.15.112:0/169262143 conn(0x7f4f140d4440 msgr2=0x7f4f14160240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:54.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.456+0000 7f4f19739640 1 -- 172.21.15.112:0/169262143 shutdown_connections 2024-09-21T00:26:54.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.456+0000 7f4f19739640 1 -- 172.21.15.112:0/169262143 wait complete. 2024-09-21T00:26:54.468 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T00:26:54.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.579+0000 7fc14c0a9640 1 -- 172.21.15.112:0/4098942695 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1440ca610 msgr2=0x7fc1440cdb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:54.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.579+0000 7fc14c0a9640 1 --2- 172.21.15.112:0/4098942695 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1440ca610 0x7fc1440cdb90 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fc1440cd750 tx=0x7fc14002a560 comp rx=0 tx=0).stop 2024-09-21T00:26:54.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.579+0000 7fc14c0a9640 1 -- 172.21.15.112:0/4098942695 shutdown_connections 2024-09-21T00:26:54.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.579+0000 7fc14c0a9640 1 --2- 172.21.15.112:0/4098942695 >> v2:172.21.15.187:3300/0 conn(0x7fc1440ce0d0 0x7fc144073e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:54.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.579+0000 7fc14c0a9640 1 --2- 172.21.15.112:0/4098942695 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1440ca610 0x7fc1440cdb90 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:54.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.579+0000 7fc14c0a9640 1 -- 172.21.15.112:0/4098942695 >> 172.21.15.112:0/4098942695 conn(0x7fc1440cca40 msgr2=0x7fc1440cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:54.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.579+0000 7fc14c0a9640 1 -- 172.21.15.112:0/4098942695 shutdown_connections 2024-09-21T00:26:54.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.579+0000 7fc14c0a9640 1 -- 172.21.15.112:0/4098942695 wait complete. 2024-09-21T00:26:54.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.581+0000 7fc14c0a9640 1 Processor -- start 2024-09-21T00:26:54.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.581+0000 7fc14c0a9640 1 -- start start 2024-09-21T00:26:54.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.582+0000 7fc14c0a9640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc144074360 0x7fc1440b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:54.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.582+0000 7fc14c0a9640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc1440ca610 0x7fc144154210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:54.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.582+0000 7fc14c0a9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1440ce0d0 0x7fc14409a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:54.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.582+0000 7fc14c0a9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc14415baa0 con 0x7fc1440ce0d0 2024-09-21T00:26:54.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.582+0000 7fc14c0a9640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc14415b920 con 0x7fc1440ca610 2024-09-21T00:26:54.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.582+0000 7fc14c0a9640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc14415bc20 con 0x7fc144074360 2024-09-21T00:26:54.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.582+0000 7fc14a646640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1440ce0d0 0x7fc14409a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:54.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.582+0000 7fc14a646640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1440ce0d0 0x7fc14409a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60958/0 (socket says 172.21.15.112:60958) 2024-09-21T00:26:54.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.582+0000 7fc14a646640 1 -- 172.21.15.112:0/3060627108 learned_addr learned my addr 172.21.15.112:0/3060627108 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:26:54.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.582+0000 7fc149e45640 1 --2- 172.21.15.112:0/3060627108 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc144074360 0x7fc1440b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:54.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.582+0000 7fc149644640 1 --2- 172.21.15.112:0/3060627108 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc1440ca610 0x7fc144154210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:54.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.583+0000 7fc14a646640 1 -- 172.21.15.112:0/3060627108 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc144074360 msgr2=0x7fc1440b4f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:54.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.583+0000 7fc14a646640 1 --2- 172.21.15.112:0/3060627108 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc144074360 0x7fc1440b4f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:54.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.583+0000 7fc14a646640 1 -- 172.21.15.112:0/3060627108 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc1440ca610 msgr2=0x7fc144154210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:54.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.583+0000 7fc14a646640 1 --2- 172.21.15.112:0/3060627108 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc1440ca610 0x7fc144154210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:54.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.583+0000 7fc14a646640 1 -- 172.21.15.112:0/3060627108 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1440b5c10 con 0x7fc1440ce0d0 2024-09-21T00:26:54.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.583+0000 7fc14a646640 1 --2- 172.21.15.112:0/3060627108 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1440ce0d0 0x7fc14409a9e0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fc13c00ba10 tx=0x7fc13c00bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:54.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.583+0000 7fc13a7fc640 1 -- 172.21.15.112:0/3060627108 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc13c019070 con 0x7fc1440ce0d0 2024-09-21T00:26:54.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.583+0000 7fc13a7fc640 1 -- 172.21.15.112:0/3060627108 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc13c00cc30 con 0x7fc1440ce0d0 2024-09-21T00:26:54.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.583+0000 7fc14c0a9640 1 -- 172.21.15.112:0/3060627108 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1440b4190 con 0x7fc1440ce0d0 2024-09-21T00:26:54.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.583+0000 7fc13a7fc640 1 -- 172.21.15.112:0/3060627108 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc13c0146a0 con 0x7fc1440ce0d0 2024-09-21T00:26:54.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.583+0000 7fc14c0a9640 1 -- 172.21.15.112:0/3060627108 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1440b4750 con 0x7fc1440ce0d0 2024-09-21T00:26:54.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.584+0000 7fc14c0a9640 1 -- 172.21.15.112:0/3060627108 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1440cdb90 con 0x7fc1440ce0d0 2024-09-21T00:26:54.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.585+0000 7fc13a7fc640 1 -- 172.21.15.112:0/3060627108 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc13c01c070 con 0x7fc1440ce0d0 2024-09-21T00:26:54.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.585+0000 7fc13a7fc640 1 --2- 172.21.15.112:0/3060627108 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fc128078bc0 0x7fc12807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:54.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.586+0000 7fc13a7fc640 1 -- 172.21.15.112:0/3060627108 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7fc13c09aab0 con 0x7fc1440ce0d0 2024-09-21T00:26:54.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.586+0000 7fc149e45640 1 --2- 172.21.15.112:0/3060627108 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fc128078bc0 0x7fc12807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:54.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.586+0000 7fc149e45640 1 --2- 172.21.15.112:0/3060627108 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fc128078bc0 0x7fc12807b0d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc140033000 tx=0x7fc140004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:54.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.590+0000 7fc13a7fc640 1 -- 172.21.15.112:0/3060627108 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc13c00ce20 con 0x7fc1440ce0d0 2024-09-21T00:26:54.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.804+0000 7fc14c0a9640 1 -- 172.21.15.112:0/3060627108 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc144075050 con 0x7fc1440ce0d0 2024-09-21T00:26:54.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.805+0000 7fc13a7fc640 1 -- 172.21.15.112:0/3060627108 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5232 (secure 0 0 0) 0x7fc13c0627a0 con 0x7fc1440ce0d0 2024-09-21T00:26:54.806 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:26:54.806 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":17,"btime":"2024-09-21T00:26:39:841719+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4846,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6834/2036740840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":2036740840},{"type":"v1","addr":"172.21.15.112:6834","nonce":2036740840}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4855,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6836/3699399640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":3699399640},{"type":"v1","addr":"172.21.15.187:6836","nonce":3699399640}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4858,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6837/694936803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6835","nonce":694936803},{"type":"v1","addr":"172.21.15.187:6837","nonce":694936803}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T00:26:36.187741+0000","modified":"2024-09-21T00:26:38.947874+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5028},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5028":{"gid":5028,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6835/4012316979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":4012316979},{"type":"v1","addr":"172.21.15.112:6835","nonce":4012316979}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5028,"qdb_cluster":[5028]},"id":2}]} 2024-09-21T00:26:54.806 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 17 2024-09-21T00:26:54.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.807+0000 7fc14c0a9640 1 -- 172.21.15.112:0/3060627108 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fc128078bc0 msgr2=0x7fc12807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:54.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.807+0000 7fc14c0a9640 1 --2- 172.21.15.112:0/3060627108 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fc128078bc0 0x7fc12807b0d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc140033000 tx=0x7fc140004730 comp rx=0 tx=0).stop 2024-09-21T00:26:54.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.807+0000 7fc14c0a9640 1 -- 172.21.15.112:0/3060627108 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1440ce0d0 msgr2=0x7fc14409a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:54.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.807+0000 7fc14c0a9640 1 --2- 172.21.15.112:0/3060627108 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1440ce0d0 0x7fc14409a9e0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fc13c00ba10 tx=0x7fc13c00bed0 comp rx=0 tx=0).stop 2024-09-21T00:26:54.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.807+0000 7fc14c0a9640 1 -- 172.21.15.112:0/3060627108 shutdown_connections 2024-09-21T00:26:54.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.807+0000 7fc14c0a9640 1 --2- 172.21.15.112:0/3060627108 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fc128078bc0 0x7fc12807b0d0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:54.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.807+0000 7fc14c0a9640 1 --2- 172.21.15.112:0/3060627108 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1440ce0d0 0x7fc14409a9e0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:54.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.807+0000 7fc14c0a9640 1 --2- 172.21.15.112:0/3060627108 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc1440ca610 0x7fc144154210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:54.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.807+0000 7fc14c0a9640 1 --2- 172.21.15.112:0/3060627108 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc144074360 0x7fc1440b4f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:54.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.807+0000 7fc14c0a9640 1 -- 172.21.15.112:0/3060627108 >> 172.21.15.112:0/3060627108 conn(0x7fc1440cca40 msgr2=0x7fc1440ce4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:54.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.807+0000 7fc14c0a9640 1 -- 172.21.15.112:0/3060627108 shutdown_connections 2024-09-21T00:26:54.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.807+0000 7fc14c0a9640 1 -- 172.21.15.112:0/3060627108 wait complete. 2024-09-21T00:26:54.819 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T00:26:54.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.929+0000 7f3e34830640 1 -- 172.21.15.112:0/3264113108 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e30074cb0 msgr2=0x7f3e30073dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:54.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.929+0000 7f3e34830640 1 --2- 172.21.15.112:0/3264113108 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e30074cb0 0x7f3e30073dd0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f3e20005fd0 tx=0x7f3e2002a5d0 comp rx=0 tx=0).stop 2024-09-21T00:26:54.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.929+0000 7f3e34830640 1 -- 172.21.15.112:0/3264113108 shutdown_connections 2024-09-21T00:26:54.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.929+0000 7f3e34830640 1 --2- 172.21.15.112:0/3264113108 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3e300cbc10 0x7f3e300cc020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:54.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.929+0000 7f3e34830640 1 --2- 172.21.15.112:0/3264113108 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e30074cb0 0x7f3e30073dd0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:54.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.929+0000 7f3e34830640 1 -- 172.21.15.112:0/3264113108 >> 172.21.15.112:0/3264113108 conn(0x7f3e300cca40 msgr2=0x7f3e300cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:54.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.929+0000 7f3e34830640 1 -- 172.21.15.112:0/3264113108 shutdown_connections 2024-09-21T00:26:54.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.929+0000 7f3e34830640 1 -- 172.21.15.112:0/3264113108 wait complete. 2024-09-21T00:26:54.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.931+0000 7f3e34830640 1 Processor -- start 2024-09-21T00:26:54.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.931+0000 7f3e34830640 1 -- start start 2024-09-21T00:26:54.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.933+0000 7f3e34830640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3e3006ddf0 0x7f3e300b0e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:54.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.933+0000 7f3e34830640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3e30074cb0 0x7f3e30157580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:54.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.933+0000 7f3e34830640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e300cbc10 0x7f3e30157ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:54.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.933+0000 7f3e34830640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3e3006f360 con 0x7f3e300cbc10 2024-09-21T00:26:54.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.933+0000 7f3e34830640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3e3006f1e0 con 0x7f3e3006ddf0 2024-09-21T00:26:54.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.933+0000 7f3e34830640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3e3006f4e0 con 0x7f3e30074cb0 2024-09-21T00:26:54.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.933+0000 7f3e2ed9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e300cbc10 0x7f3e30157ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:54.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.933+0000 7f3e2ed9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e300cbc10 0x7f3e30157ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60984/0 (socket says 172.21.15.112:60984) 2024-09-21T00:26:54.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.933+0000 7f3e2ed9d640 1 -- 172.21.15.112:0/1254361153 learned_addr learned my addr 172.21.15.112:0/1254361153 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:26:54.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.933+0000 7f3e2e59c640 1 --2- 172.21.15.112:0/1254361153 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3e3006ddf0 0x7f3e300b0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:54.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.933+0000 7f3e25d9b640 1 --2- 172.21.15.112:0/1254361153 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3e30074cb0 0x7f3e30157580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:54.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.933+0000 7f3e2ed9d640 1 -- 172.21.15.112:0/1254361153 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3e30074cb0 msgr2=0x7f3e30157580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:54.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.933+0000 7f3e2ed9d640 1 --2- 172.21.15.112:0/1254361153 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3e30074cb0 0x7f3e30157580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:54.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.933+0000 7f3e2ed9d640 1 -- 172.21.15.112:0/1254361153 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3e3006ddf0 msgr2=0x7f3e300b0e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:54.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.933+0000 7f3e2ed9d640 1 --2- 172.21.15.112:0/1254361153 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3e3006ddf0 0x7f3e300b0e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:54.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.933+0000 7f3e2ed9d640 1 -- 172.21.15.112:0/1254361153 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e301f4a10 con 0x7f3e300cbc10 2024-09-21T00:26:54.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.933+0000 7f3e2ed9d640 1 --2- 172.21.15.112:0/1254361153 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e300cbc10 0x7f3e30157ac0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f3e2800c760 tx=0x7f3e2800cb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:54.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.934+0000 7f3e277fe640 1 -- 172.21.15.112:0/1254361153 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e28003b70 con 0x7f3e300cbc10 2024-09-21T00:26:54.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.934+0000 7f3e277fe640 1 -- 172.21.15.112:0/1254361153 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3e28003d10 con 0x7f3e300cbc10 2024-09-21T00:26:54.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.934+0000 7f3e277fe640 1 -- 172.21.15.112:0/1254361153 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e280052d0 con 0x7f3e300cbc10 2024-09-21T00:26:54.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.934+0000 7f3e34830640 1 -- 172.21.15.112:0/1254361153 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e301544f0 con 0x7f3e300cbc10 2024-09-21T00:26:54.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.934+0000 7f3e34830640 1 -- 172.21.15.112:0/1254361153 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e301f3c50 con 0x7f3e300cbc10 2024-09-21T00:26:54.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.935+0000 7f3e34830640 1 -- 172.21.15.112:0/1254361153 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e30154760 con 0x7f3e300cbc10 2024-09-21T00:26:54.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.935+0000 7f3e277fe640 1 -- 172.21.15.112:0/1254361153 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3e28005470 con 0x7f3e300cbc10 2024-09-21T00:26:54.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.936+0000 7f3e277fe640 1 --2- 172.21.15.112:0/1254361153 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f3e10078bc0 0x7f3e1007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:54.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.936+0000 7f3e277fe640 1 -- 172.21.15.112:0/1254361153 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f3e2809ab20 con 0x7f3e300cbc10 2024-09-21T00:26:54.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.936+0000 7f3e2e59c640 1 --2- 172.21.15.112:0/1254361153 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f3e10078bc0 0x7f3e1007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:54.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.937+0000 7f3e2e59c640 1 --2- 172.21.15.112:0/1254361153 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f3e10078bc0 0x7f3e1007b0d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f3e20033000 tx=0x7f3e20004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:54.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:54.940+0000 7f3e277fe640 1 -- 172.21.15.112:0/1254361153 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3e28062780 con 0x7f3e300cbc10 2024-09-21T00:26:55.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:55.117+0000 7f3e34830640 1 -- 172.21.15.112:0/1254361153 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3e3006a780 con 0x7f3e300cbc10 2024-09-21T00:26:55.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:55.118+0000 7f3e277fe640 1 -- 172.21.15.112:0/1254361153 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) ==== 74+0+17026 (secure 0 0 0) 0x7f3e3006a780 con 0x7f3e300cbc10 2024-09-21T00:26:55.119 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:26:55.119 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":38,"fsid":"8b34692b-dc65-49b2-a08f-26f707b96ac5","created":"2024-09-21T00:25:22.788490+0000","modified":"2024-09-21T00:26:37.831810+0000","last_up_change":"2024-09-21T00:25:36.161835+0000","last_in_change":"2024-09-21T00:25:26.346500+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T00:25:37.105006+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-09-21T00:26:34.140543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-09-21T00:26:35.143821+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9c575086-67d2-4de2-b01b-d44dcc21d604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6809","nonce":755760621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6811","nonce":755760621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6815","nonce":755760621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6813","nonce":755760621}]},"public_addr":"172.21.15.112:6809/755760621","cluster_addr":"172.21.15.112:6811/755760621","heartbeat_back_addr":"172.21.15.112:6815/755760621","heartbeat_front_addr":"172.21.15.112:6813/755760621","state":["exists","up"]},{"osd":1,"uuid":"a983c227-a3ce-4bdd-82f1-438ba7e3ecd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6801","nonce":344932592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6803","nonce":344932592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6807","nonce":344932592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6805","nonce":344932592}]},"public_addr":"172.21.15.112:6801/344932592","cluster_addr":"172.21.15.112:6803/344932592","heartbeat_back_addr":"172.21.15.112:6807/344932592","heartbeat_front_addr":"172.21.15.112:6805/344932592","state":["exists","up"]},{"osd":2,"uuid":"1193c1d8-bfff-4a57-8200-5f69bf1ea3af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6825","nonce":407933616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6827","nonce":407933616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6831","nonce":407933616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6829","nonce":407933616}]},"public_addr":"172.21.15.112:6825/407933616","cluster_addr":"172.21.15.112:6827/407933616","heartbeat_back_addr":"172.21.15.112:6831/407933616","heartbeat_front_addr":"172.21.15.112:6829/407933616","state":["exists","up"]},{"osd":3,"uuid":"af04330a-36ad-40cc-a8dd-fa259b221de5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6817","nonce":4120164806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6819","nonce":4120164806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6823","nonce":4120164806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6821","nonce":4120164806}]},"public_addr":"172.21.15.112:6817/4120164806","cluster_addr":"172.21.15.112:6819/4120164806","heartbeat_back_addr":"172.21.15.112:6823/4120164806","heartbeat_front_addr":"172.21.15.112:6821/4120164806","state":["exists","up"]},{"osd":4,"uuid":"5b893111-292c-4c5a-b184-53ad38c7199e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6803","nonce":522077560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6805","nonce":522077560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6809","nonce":522077560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6807","nonce":522077560}]},"public_addr":"172.21.15.187:6803/522077560","cluster_addr":"172.21.15.187:6805/522077560","heartbeat_back_addr":"172.21.15.187:6809/522077560","heartbeat_front_addr":"172.21.15.187:6807/522077560","state":["exists","up"]},{"osd":5,"uuid":"c0b8e011-45fe-4c5e-925e-6be2a7f1868a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6827","nonce":3593144721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6829","nonce":3593144721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6833","nonce":3593144721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6831","nonce":3593144721}]},"public_addr":"172.21.15.187:6827/3593144721","cluster_addr":"172.21.15.187:6829/3593144721","heartbeat_back_addr":"172.21.15.187:6833/3593144721","heartbeat_front_addr":"172.21.15.187:6831/3593144721","state":["exists","up"]},{"osd":6,"uuid":"87d2d650-7489-4593-98a3-a1f433ce8dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6811","nonce":1494387657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6813","nonce":1494387657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6817","nonce":1494387657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6815","nonce":1494387657}]},"public_addr":"172.21.15.187:6811/1494387657","cluster_addr":"172.21.15.187:6813/1494387657","heartbeat_back_addr":"172.21.15.187:6817/1494387657","heartbeat_front_addr":"172.21.15.187:6815/1494387657","state":["exists","up"]},{"osd":7,"uuid":"c6b4f8e2-a1ee-4438-aa37-67ff4b6ef1f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6819","nonce":1624699842}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6821","nonce":1624699842}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6825","nonce":1624699842}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6823","nonce":1624699842}]},"public_addr":"172.21.15.187:6819/1624699842","cluster_addr":"172.21.15.187:6821/1624699842","heartbeat_back_addr":"172.21.15.187:6825/1624699842","heartbeat_front_addr":"172.21.15.187:6823/1624699842","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.873162+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.400506+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.664317+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.159791+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.202630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.113212+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.384786+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.448441+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T00:26:55.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:55.120+0000 7f3e34830640 1 -- 172.21.15.112:0/1254361153 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f3e10078bc0 msgr2=0x7f3e1007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:55.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:55.120+0000 7f3e34830640 1 --2- 172.21.15.112:0/1254361153 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f3e10078bc0 0x7f3e1007b0d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f3e20033000 tx=0x7f3e20004730 comp rx=0 tx=0).stop 2024-09-21T00:26:55.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:55.120+0000 7f3e34830640 1 -- 172.21.15.112:0/1254361153 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e300cbc10 msgr2=0x7f3e30157ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:55.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:55.120+0000 7f3e34830640 1 --2- 172.21.15.112:0/1254361153 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e300cbc10 0x7f3e30157ac0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f3e2800c760 tx=0x7f3e2800cb60 comp rx=0 tx=0).stop 2024-09-21T00:26:55.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:55.120+0000 7f3e34830640 1 -- 172.21.15.112:0/1254361153 shutdown_connections 2024-09-21T00:26:55.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:55.120+0000 7f3e34830640 1 --2- 172.21.15.112:0/1254361153 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f3e10078bc0 0x7f3e1007b0d0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:55.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:55.120+0000 7f3e34830640 1 --2- 172.21.15.112:0/1254361153 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e300cbc10 0x7f3e30157ac0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:55.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:55.120+0000 7f3e34830640 1 --2- 172.21.15.112:0/1254361153 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3e30074cb0 0x7f3e30157580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:55.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:55.120+0000 7f3e34830640 1 --2- 172.21.15.112:0/1254361153 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3e3006ddf0 0x7f3e300b0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:55.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:55.120+0000 7f3e34830640 1 -- 172.21.15.112:0/1254361153 >> 172.21.15.112:0/1254361153 conn(0x7f3e300cca40 msgr2=0x7f3e3015a5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:55.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:55.120+0000 7f3e34830640 1 -- 172.21.15.112:0/1254361153 shutdown_connections 2024-09-21T00:26:55.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:55.120+0000 7f3e34830640 1 -- 172.21.15.112:0/1254361153 wait complete. 2024-09-21T00:26:55.133 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T00:26:55.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:55.245+0000 7f3046389640 1 -- 172.21.15.112:0/121440536 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f30400ccc40 msgr2=0x7f304015ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:55.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:55.245+0000 7f3046389640 1 --2- 172.21.15.112:0/121440536 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f30400ccc40 0x7f304015ca90 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f3040071cd0 tx=0x7f303002a4b0 comp rx=0 tx=0).stop 2024-09-21T00:26:55.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:55.245+0000 7f3046389640 1 -- 172.21.15.112:0/121440536 shutdown_connections 2024-09-21T00:26:55.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:55.245+0000 7f3046389640 1 --2- 172.21.15.112:0/121440536 >> v2:172.21.15.187:3300/0 conn(0x7f30400d0d80 0x7f30400b5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:55.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:55.245+0000 7f3046389640 1 --2- 172.21.15.112:0/121440536 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f30400ccc40 0x7f304015ca90 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:55.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:55.245+0000 7f3046389640 1 -- 172.21.15.112:0/121440536 >> 172.21.15.112:0/121440536 conn(0x7f30400b46e0 msgr2=0x7f30400d23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:55.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:55.245+0000 7f3046389640 1 -- 172.21.15.112:0/121440536 shutdown_connections 2024-09-21T00:26:55.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:55.245+0000 7f3046389640 1 -- 172.21.15.112:0/121440536 wait complete. 2024-09-21T00:26:55.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:55.247+0000 7f3046389640 1 Processor -- start 2024-09-21T00:26:55.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:55.247+0000 7f3046389640 1 -- start start 2024-09-21T00:26:55.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:55.248+0000 7f3046389640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f30400ccc40 0x7f30400cbdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:55.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:55.248+0000 7f3046389640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f30400cd440 0x7f3040152150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:55.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:55.248+0000 7f3046389640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f30400ae750 0x7f30400b6f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:55.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:55.248+0000 7f3046389640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f30400b6650 con 0x7f30400ccc40 2024-09-21T00:26:55.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:55.248+0000 7f3046389640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f30400b64d0 con 0x7f30400cd440 2024-09-21T00:26:55.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:55.248+0000 7f3046389640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f30400b67d0 con 0x7f30400ae750 2024-09-21T00:26:55.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:55.248+0000 7f303ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f30400ccc40 0x7f30400cbdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:55.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:55.248+0000 7f303ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f30400ccc40 0x7f30400cbdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:32774/0 (socket says 172.21.15.112:32774) 2024-09-21T00:26:55.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:55.248+0000 7f303ffff640 1 -- 172.21.15.112:0/3040475566 learned_addr learned my addr 172.21.15.112:0/3040475566 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:26:55.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:55.248+0000 7f3044926640 1 --2- 172.21.15.112:0/3040475566 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f30400ae750 0x7f30400b6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:55.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:55.248+0000 7f303f7fe640 1 --2- 172.21.15.112:0/3040475566 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f30400cd440 0x7f3040152150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:55.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:55.248+0000 7f303ffff640 1 -- 172.21.15.112:0/3040475566 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f30400ae750 msgr2=0x7f30400b6f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:55.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:55.248+0000 7f303ffff640 1 --2- 172.21.15.112:0/3040475566 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f30400ae750 0x7f30400b6f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:55.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:55.248+0000 7f303ffff640 1 -- 172.21.15.112:0/3040475566 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f30400cd440 msgr2=0x7f3040152150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:55.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:55.248+0000 7f303ffff640 1 --2- 172.21.15.112:0/3040475566 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f30400cd440 0x7f3040152150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:55.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:55.248+0000 7f303ffff640 1 -- 172.21.15.112:0/3040475566 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30400cc300 con 0x7f30400ccc40 2024-09-21T00:26:55.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:55.249+0000 7f3044926640 1 --2- 172.21.15.112:0/3040475566 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f30400ae750 0x7f30400b6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:26:55.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:55.249+0000 7f303f7fe640 1 --2- 172.21.15.112:0/3040475566 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f30400cd440 0x7f3040152150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:26:55.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:55.249+0000 7f303ffff640 1 --2- 172.21.15.112:0/3040475566 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f30400ccc40 0x7f30400cbdc0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f3030034040 tx=0x7f3030003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:55.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:55.249+0000 7f303cff9640 1 -- 172.21.15.112:0/3040475566 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f303003e030 con 0x7f30400ccc40 2024-09-21T00:26:55.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:55.249+0000 7f303cff9640 1 -- 172.21.15.112:0/3040475566 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3030003e00 con 0x7f30400ccc40 2024-09-21T00:26:55.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:55.249+0000 7f3046389640 1 -- 172.21.15.112:0/3040475566 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3040152690 con 0x7f30400ccc40 2024-09-21T00:26:55.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:55.249+0000 7f303cff9640 1 -- 172.21.15.112:0/3040475566 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3030035040 con 0x7f30400ccc40 2024-09-21T00:26:55.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:55.249+0000 7f3046389640 1 -- 172.21.15.112:0/3040475566 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f304014fd80 con 0x7f30400ccc40 2024-09-21T00:26:55.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:55.250+0000 7f3046389640 1 -- 172.21.15.112:0/3040475566 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30400b6d10 con 0x7f30400ccc40 2024-09-21T00:26:55.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:55.251+0000 7f303cff9640 1 -- 172.21.15.112:0/3040475566 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3030041030 con 0x7f30400ccc40 2024-09-21T00:26:55.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:55.251+0000 7f303cff9640 1 --2- 172.21.15.112:0/3040475566 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f302c078bc0 0x7f302c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:55.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:55.251+0000 7f303cff9640 1 -- 172.21.15.112:0/3040475566 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f30300bb700 con 0x7f30400ccc40 2024-09-21T00:26:55.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:55.251+0000 7f303f7fe640 1 --2- 172.21.15.112:0/3040475566 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f302c078bc0 0x7f302c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:55.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:55.252+0000 7f303f7fe640 1 --2- 172.21.15.112:0/3040475566 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f302c078bc0 0x7f302c07b0d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f30400cc990 tx=0x7f3034008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:55.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:55.256+0000 7f303cff9640 1 -- 172.21.15.112:0/3040475566 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f30300839b0 con 0x7f30400ccc40 2024-09-21T00:26:55.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:55.468+0000 7f3046389640 1 -- 172.21.15.112:0/3040475566 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f304015c460 con 0x7f30400ccc40 2024-09-21T00:26:55.469 INFO:tasks.ceph.mon.a.smithi112.stderr:2024-09-21T00:26:55.469+0000 7f7656c13640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T00:26:55.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:55.470+0000 7f303cff9640 1 -- 172.21.15.112:0/3040475566 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v18) ==== 161+0+0 (secure 0 0 0) 0x7f30300832b0 con 0x7f30400ccc40 2024-09-21T00:26:55.471 INFO:teuthology.orchestra.run.smithi112.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T00:26:55.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:55.473+0000 7f3046389640 1 -- 172.21.15.112:0/3040475566 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f302c078bc0 msgr2=0x7f302c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:55.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:55.473+0000 7f3046389640 1 --2- 172.21.15.112:0/3040475566 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f302c078bc0 0x7f302c07b0d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f30400cc990 tx=0x7f3034008040 comp rx=0 tx=0).stop 2024-09-21T00:26:55.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:55.473+0000 7f3046389640 1 -- 172.21.15.112:0/3040475566 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f30400ccc40 msgr2=0x7f30400cbdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:55.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:55.473+0000 7f3046389640 1 --2- 172.21.15.112:0/3040475566 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f30400ccc40 0x7f30400cbdc0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f3030034040 tx=0x7f3030003910 comp rx=0 tx=0).stop 2024-09-21T00:26:55.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:55.473+0000 7f3046389640 1 -- 172.21.15.112:0/3040475566 shutdown_connections 2024-09-21T00:26:55.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:55.473+0000 7f3046389640 1 --2- 172.21.15.112:0/3040475566 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f30400ae750 0x7f30400b6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:55.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:55.473+0000 7f3046389640 1 --2- 172.21.15.112:0/3040475566 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f30400cd440 0x7f3040152150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:55.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:55.473+0000 7f3046389640 1 --2- 172.21.15.112:0/3040475566 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f302c078bc0 0x7f302c07b0d0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:55.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:55.473+0000 7f3046389640 1 --2- 172.21.15.112:0/3040475566 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f30400ccc40 0x7f30400cbdc0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:55.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:55.473+0000 7f3046389640 1 -- 172.21.15.112:0/3040475566 >> 172.21.15.112:0/3040475566 conn(0x7f30400b46e0 msgr2=0x7f30400714e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:55.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:55.473+0000 7f3046389640 1 -- 172.21.15.112:0/3040475566 shutdown_connections 2024-09-21T00:26:55.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:55.474+0000 7f3046389640 1 -- 172.21.15.112:0/3040475566 wait complete. 2024-09-21T00:26:55.476 INFO:tasks.ceph.mds.a.smithi112.stderr: -2521> 2024-09-21T00:26:40.792+0000 7f8c708e3640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T00:26:55.487 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-21T00:26:55.529 INFO:tasks.ceph.mds.a.smithi112.stdout:starting mds.a at 2024-09-21T00:26:55.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:55.603+0000 7f6fbf920640 1 -- 172.21.15.112:0/306180017 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6fb80b5260 msgr2=0x7f6fb815ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:55.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:55.603+0000 7f6fbf920640 1 --2- 172.21.15.112:0/306180017 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6fb80b5260 0x7f6fb815ba80 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f6fb8073190 tx=0x7f6fa802a560 comp rx=0 tx=0).stop 2024-09-21T00:26:55.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:55.603+0000 7f6fbf920640 1 -- 172.21.15.112:0/306180017 shutdown_connections 2024-09-21T00:26:55.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:55.603+0000 7f6fbf920640 1 --2- 172.21.15.112:0/306180017 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6fb815bfc0 0x7f6fb815f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:55.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:55.603+0000 7f6fbf920640 1 --2- 172.21.15.112:0/306180017 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6fb80b5260 0x7f6fb815ba80 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:55.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:55.603+0000 7f6fbf920640 1 --2- 172.21.15.112:0/306180017 >> v2:172.21.15.187:3300/0 conn(0x7f6fb806def0 0x7f6fb806f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:55.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:55.603+0000 7f6fbf920640 1 -- 172.21.15.112:0/306180017 >> 172.21.15.112:0/306180017 conn(0x7f6fb80cdca0 msgr2=0x7f6fb80ce0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:55.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:55.603+0000 7f6fbf920640 1 -- 172.21.15.112:0/306180017 shutdown_connections 2024-09-21T00:26:55.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:55.603+0000 7f6fbf920640 1 -- 172.21.15.112:0/306180017 wait complete. 2024-09-21T00:26:55.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:55.605+0000 7f6fbf920640 1 Processor -- start 2024-09-21T00:26:55.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:55.605+0000 7f6fbf920640 1 -- start start 2024-09-21T00:26:55.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:55.606+0000 7f6fbf920640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6fb806def0 0x7f6fb81f55f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:55.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:55.606+0000 7f6fbf920640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6fb80b5260 0x7f6fb81f5b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:55.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:55.606+0000 7f6fbf920640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6fb815bfc0 0x7f6fb81eef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:55.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:55.606+0000 7f6fbf920640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6fb815ff00 con 0x7f6fb815bfc0 2024-09-21T00:26:55.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:55.606+0000 7f6fbf920640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f6fb815fd80 con 0x7f6fb80b5260 2024-09-21T00:26:55.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:55.606+0000 7f6fbf920640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f6fb80b59d0 con 0x7f6fb806def0 2024-09-21T00:26:55.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:55.606+0000 7f6fbdebd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6fb815bfc0 0x7f6fb81eef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:55.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:55.606+0000 7f6fbdebd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6fb815bfc0 0x7f6fb81eef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:32812/0 (socket says 172.21.15.112:32812) 2024-09-21T00:26:55.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:55.606+0000 7f6fbdebd640 1 -- 172.21.15.112:0/3376051663 learned_addr learned my addr 172.21.15.112:0/3376051663 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:26:55.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:55.606+0000 7f6fbcebb640 1 --2- 172.21.15.112:0/3376051663 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6fb80b5260 0x7f6fb81f5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:55.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:55.606+0000 7f6fbd6bc640 1 --2- 172.21.15.112:0/3376051663 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6fb806def0 0x7f6fb81f55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:55.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:55.607+0000 7f6fbdebd640 1 -- 172.21.15.112:0/3376051663 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6fb806def0 msgr2=0x7f6fb81f55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:55.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:55.607+0000 7f6fbdebd640 1 --2- 172.21.15.112:0/3376051663 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6fb806def0 0x7f6fb81f55f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:55.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:55.607+0000 7f6fbdebd640 1 -- 172.21.15.112:0/3376051663 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6fb80b5260 msgr2=0x7f6fb81f5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:55.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:55.607+0000 7f6fbdebd640 1 --2- 172.21.15.112:0/3376051663 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6fb80b5260 0x7f6fb81f5b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:55.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:55.607+0000 7f6fbdebd640 1 -- 172.21.15.112:0/3376051663 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fb81f6420 con 0x7f6fb815bfc0 2024-09-21T00:26:55.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:55.607+0000 7f6fbdebd640 1 --2- 172.21.15.112:0/3376051663 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6fb815bfc0 0x7f6fb81eef20 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f6fb000e990 tx=0x7f6fb000ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:55.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:55.607+0000 7f6fadffb640 1 -- 172.21.15.112:0/3376051663 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fb0004d20 con 0x7f6fb815bfc0 2024-09-21T00:26:55.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:55.607+0000 7f6fadffb640 1 -- 172.21.15.112:0/3376051663 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6fb000caa0 con 0x7f6fb815bfc0 2024-09-21T00:26:55.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:55.607+0000 7f6fbf920640 1 -- 172.21.15.112:0/3376051663 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fb80cb640 con 0x7f6fb815bfc0 2024-09-21T00:26:55.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:55.607+0000 7f6fadffb640 1 -- 172.21.15.112:0/3376051663 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fb0005020 con 0x7f6fb815bfc0 2024-09-21T00:26:55.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:55.607+0000 7f6fbf920640 1 -- 172.21.15.112:0/3376051663 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fb8149050 con 0x7f6fb815bfc0 2024-09-21T00:26:55.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:55.609+0000 7f6fbf920640 1 -- 172.21.15.112:0/3376051663 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fb8148ae0 con 0x7f6fb815bfc0 2024-09-21T00:26:55.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:55.609+0000 7f6fadffb640 1 -- 172.21.15.112:0/3376051663 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6fb00051c0 con 0x7f6fb815bfc0 2024-09-21T00:26:55.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:55.609+0000 7f6fadffb640 1 --2- 172.21.15.112:0/3376051663 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f6f9c078bc0 0x7f6f9c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:55.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:55.610+0000 7f6fadffb640 1 -- 172.21.15.112:0/3376051663 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6187+0+0 (secure 0 0 0) 0x7f6fb009b650 con 0x7f6fb815bfc0 2024-09-21T00:26:55.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:55.610+0000 7f6fbd6bc640 1 --2- 172.21.15.112:0/3376051663 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f6f9c078bc0 0x7f6f9c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:55.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:55.610+0000 7f6fbd6bc640 1 --2- 172.21.15.112:0/3376051663 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f6f9c078bc0 0x7f6f9c07b0d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f6fb815efd0 tx=0x7f6fb400b2e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:55.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:55.614+0000 7f6fadffb640 1 -- 172.21.15.112:0/3376051663 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f6fb0016030 con 0x7f6fb815bfc0 2024-09-21T00:26:55.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:55.829+0000 7f6fbf920640 1 -- 172.21.15.112:0/3376051663 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f6fb815ba80 con 0x7f6fb815bfc0 2024-09-21T00:26:56.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:56.502+0000 7f6fadffb640 1 -- 172.21.15.112:0/3376051663 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v19) ==== 104+0+0 (secure 0 0 0) 0x7f6fb00632f0 con 0x7f6fb815bfc0 2024-09-21T00:26:56.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:56.505+0000 7f6fbf920640 1 -- 172.21.15.112:0/3376051663 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f6f9c078bc0 msgr2=0x7f6f9c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:56.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:56.505+0000 7f6fbf920640 1 --2- 172.21.15.112:0/3376051663 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f6f9c078bc0 0x7f6f9c07b0d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f6fb815efd0 tx=0x7f6fb400b2e0 comp rx=0 tx=0).stop 2024-09-21T00:26:56.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:56.505+0000 7f6fbf920640 1 -- 172.21.15.112:0/3376051663 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6fb815bfc0 msgr2=0x7f6fb81eef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:56.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:56.505+0000 7f6fbf920640 1 --2- 172.21.15.112:0/3376051663 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6fb815bfc0 0x7f6fb81eef20 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f6fb000e990 tx=0x7f6fb000ee50 comp rx=0 tx=0).stop 2024-09-21T00:26:56.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:56.505+0000 7f6fbf920640 1 -- 172.21.15.112:0/3376051663 shutdown_connections 2024-09-21T00:26:56.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:56.505+0000 7f6fbf920640 1 --2- 172.21.15.112:0/3376051663 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f6f9c078bc0 0x7f6f9c07b0d0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:56.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:56.505+0000 7f6fbf920640 1 --2- 172.21.15.112:0/3376051663 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6fb815bfc0 0x7f6fb81eef20 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:56.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:56.505+0000 7f6fbf920640 1 --2- 172.21.15.112:0/3376051663 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6fb80b5260 0x7f6fb81f5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:56.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:56.505+0000 7f6fbf920640 1 --2- 172.21.15.112:0/3376051663 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6fb806def0 0x7f6fb81f55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:56.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:56.505+0000 7f6fbf920640 1 -- 172.21.15.112:0/3376051663 >> 172.21.15.112:0/3376051663 conn(0x7f6fb80cdca0 msgr2=0x7f6fb815cd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:56.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:56.505+0000 7f6fbf920640 1 -- 172.21.15.112:0/3376051663 shutdown_connections 2024-09-21T00:26:56.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:56.505+0000 7f6fbf920640 1 -- 172.21.15.112:0/3376051663 wait complete. 2024-09-21T00:26:56.519 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-21T00:26:56.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:56.680+0000 7fd51ac12640 1 -- 172.21.15.112:0/1649333012 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd514163350 msgr2=0x7fd51415c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:56.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:56.680+0000 7fd51ac12640 1 --2- 172.21.15.112:0/1649333012 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd514163350 0x7fd51415c650 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd504005fd0 tx=0x7fd50402a980 comp rx=0 tx=0).stop 2024-09-21T00:26:56.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:56.680+0000 7fd51ac12640 1 -- 172.21.15.112:0/1649333012 shutdown_connections 2024-09-21T00:26:56.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:56.680+0000 7fd51ac12640 1 --2- 172.21.15.112:0/1649333012 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd514162ec0 0x7fd5140b3000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:56.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:56.680+0000 7fd51ac12640 1 --2- 172.21.15.112:0/1649333012 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd514163350 0x7fd51415c650 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:56.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:56.680+0000 7fd51ac12640 1 -- 172.21.15.112:0/1649333012 >> 172.21.15.112:0/1649333012 conn(0x7fd51406cde0 msgr2=0x7fd51406d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:56.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:56.680+0000 7fd51ac12640 1 -- 172.21.15.112:0/1649333012 shutdown_connections 2024-09-21T00:26:56.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:56.680+0000 7fd51ac12640 1 -- 172.21.15.112:0/1649333012 wait complete. 2024-09-21T00:26:56.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:56.682+0000 7fd51ac12640 1 Processor -- start 2024-09-21T00:26:56.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:56.682+0000 7fd51ac12640 1 -- start start 2024-09-21T00:26:56.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:56.684+0000 7fd51ac12640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd5140d0e40 0x7fd5140cc250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:56.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:56.684+0000 7fd51ac12640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd514163350 0x7fd5141f9430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:56.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:56.684+0000 7fd51ac12640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd5141e95b0 0x7fd5141e99c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:56.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:56.684+0000 7fd51ac12640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd5140b39e0 con 0x7fd514163350 2024-09-21T00:26:56.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:56.684+0000 7fd51ac12640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd5140b3860 con 0x7fd5141e95b0 2024-09-21T00:26:56.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:56.684+0000 7fd51ac12640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd5140b3b60 con 0x7fd5140d0e40 2024-09-21T00:26:56.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:56.684+0000 7fd513fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd514163350 0x7fd5141f9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:56.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:56.684+0000 7fd513fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd514163350 0x7fd5141f9430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40738/0 (socket says 172.21.15.112:40738) 2024-09-21T00:26:56.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:56.684+0000 7fd513fff640 1 -- 172.21.15.112:0/870874789 learned_addr learned my addr 172.21.15.112:0/870874789 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:26:56.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:56.684+0000 7fd5191af640 1 --2- 172.21.15.112:0/870874789 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd5141e95b0 0x7fd5141e99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:56.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:56.684+0000 7fd5189ae640 1 --2- 172.21.15.112:0/870874789 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd5140d0e40 0x7fd5140cc250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:56.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:56.684+0000 7fd513fff640 1 -- 172.21.15.112:0/870874789 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd5140d0e40 msgr2=0x7fd5140cc250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:56.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:56.684+0000 7fd513fff640 1 --2- 172.21.15.112:0/870874789 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd5140d0e40 0x7fd5140cc250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:56.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:56.684+0000 7fd513fff640 1 -- 172.21.15.112:0/870874789 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd5141e95b0 msgr2=0x7fd5141e99c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:56.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:56.684+0000 7fd513fff640 1 --2- 172.21.15.112:0/870874789 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd5141e95b0 0x7fd5141e99c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:56.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:56.684+0000 7fd513fff640 1 -- 172.21.15.112:0/870874789 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5141e9f00 con 0x7fd514163350 2024-09-21T00:26:56.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:56.684+0000 7fd513fff640 1 --2- 172.21.15.112:0/870874789 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd514163350 0x7fd5141f9430 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fd4fc00c400 tx=0x7fd4fc00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:56.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:56.684+0000 7fd5117fa640 1 -- 172.21.15.112:0/870874789 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4fc017070 con 0x7fd514163350 2024-09-21T00:26:56.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:56.684+0000 7fd51ac12640 1 -- 172.21.15.112:0/870874789 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5141f5af0 con 0x7fd514163350 2024-09-21T00:26:56.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:56.684+0000 7fd5117fa640 1 -- 172.21.15.112:0/870874789 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd4fc0053c0 con 0x7fd514163350 2024-09-21T00:26:56.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:56.685+0000 7fd5117fa640 1 -- 172.21.15.112:0/870874789 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4fc003830 con 0x7fd514163350 2024-09-21T00:26:56.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:56.685+0000 7fd51ac12640 1 -- 172.21.15.112:0/870874789 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5141f60b0 con 0x7fd514163350 2024-09-21T00:26:56.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:56.685+0000 7fd51ac12640 1 -- 172.21.15.112:0/870874789 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5141f5860 con 0x7fd514163350 2024-09-21T00:26:56.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:56.686+0000 7fd5117fa640 1 -- 172.21.15.112:0/870874789 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd4fc01a070 con 0x7fd514163350 2024-09-21T00:26:56.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:56.687+0000 7fd5117fa640 1 --2- 172.21.15.112:0/870874789 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fd4f4078bc0 0x7fd4f407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:56.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:56.687+0000 7fd5117fa640 1 -- 172.21.15.112:0/870874789 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6187+0+0 (secure 0 0 0) 0x7fd4fc09a230 con 0x7fd514163350 2024-09-21T00:26:56.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:56.687+0000 7fd5189ae640 1 --2- 172.21.15.112:0/870874789 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fd4f4078bc0 0x7fd4f407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:56.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:56.687+0000 7fd5189ae640 1 --2- 172.21.15.112:0/870874789 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fd4f4078bc0 0x7fd4f407b0d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd504035000 tx=0x7fd504004ab0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:56.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:56.691+0000 7fd5117fa640 1 -- 172.21.15.112:0/870874789 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd5141e9f00 con 0x7fd514163350 2024-09-21T00:26:56.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:56.870+0000 7fd51ac12640 1 -- 172.21.15.112:0/870874789 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fd514160bc0 con 0x7fd514163350 2024-09-21T00:26:57.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:57.512+0000 7fd5117fa640 1 -- 172.21.15.112:0/870874789 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v40) ==== 181+0+0 (secure 0 0 0) 0x7fd4fc062530 con 0x7fd514163350 2024-09-21T00:26:57.513 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'cephfs_metadata' removed 2024-09-21T00:26:57.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:57.515+0000 7fd51ac12640 1 -- 172.21.15.112:0/870874789 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fd4f4078bc0 msgr2=0x7fd4f407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:57.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:57.515+0000 7fd51ac12640 1 --2- 172.21.15.112:0/870874789 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fd4f4078bc0 0x7fd4f407b0d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd504035000 tx=0x7fd504004ab0 comp rx=0 tx=0).stop 2024-09-21T00:26:57.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:57.515+0000 7fd51ac12640 1 -- 172.21.15.112:0/870874789 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd514163350 msgr2=0x7fd5141f9430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:57.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:57.515+0000 7fd51ac12640 1 --2- 172.21.15.112:0/870874789 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd514163350 0x7fd5141f9430 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fd4fc00c400 tx=0x7fd4fc00c8c0 comp rx=0 tx=0).stop 2024-09-21T00:26:57.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:57.516+0000 7fd51ac12640 1 -- 172.21.15.112:0/870874789 shutdown_connections 2024-09-21T00:26:57.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:57.516+0000 7fd51ac12640 1 --2- 172.21.15.112:0/870874789 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd5141e95b0 0x7fd5141e99c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:57.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:57.516+0000 7fd51ac12640 1 --2- 172.21.15.112:0/870874789 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fd4f4078bc0 0x7fd4f407b0d0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:57.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:57.516+0000 7fd51ac12640 1 --2- 172.21.15.112:0/870874789 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd514163350 0x7fd5141f9430 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:57.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:57.516+0000 7fd51ac12640 1 --2- 172.21.15.112:0/870874789 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd5140d0e40 0x7fd5140cc250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:57.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:57.516+0000 7fd51ac12640 1 -- 172.21.15.112:0/870874789 >> 172.21.15.112:0/870874789 conn(0x7fd51406cde0 msgr2=0x7fd51415e5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:57.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:57.516+0000 7fd51ac12640 1 -- 172.21.15.112:0/870874789 shutdown_connections 2024-09-21T00:26:57.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:57.516+0000 7fd51ac12640 1 -- 172.21.15.112:0/870874789 wait complete. 2024-09-21T00:26:57.529 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-21T00:26:57.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:57.647+0000 7fd64cd8b640 1 -- 172.21.15.112:0/617083172 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd648162c10 msgr2=0x7fd64806e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:57.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:57.647+0000 7fd64cd8b640 1 --2- 172.21.15.112:0/617083172 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd648162c10 0x7fd64806e300 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fd64806f360 tx=0x7fd64002aa30 comp rx=0 tx=0).stop 2024-09-21T00:26:57.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:57.647+0000 7fd64cd8b640 1 -- 172.21.15.112:0/617083172 shutdown_connections 2024-09-21T00:26:57.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:57.647+0000 7fd64cd8b640 1 --2- 172.21.15.112:0/617083172 >> v2:172.21.15.187:3300/0 conn(0x7fd6480b64a0 0x7fd6481626d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:57.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:57.647+0000 7fd64cd8b640 1 --2- 172.21.15.112:0/617083172 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd648162c10 0x7fd64806e300 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:57.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:57.647+0000 7fd64cd8b640 1 --2- 172.21.15.112:0/617083172 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd64815e7d0 0x7fd6480b5f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:57.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:57.647+0000 7fd64cd8b640 1 -- 172.21.15.112:0/617083172 >> 172.21.15.112:0/617083172 conn(0x7fd6480d44e0 msgr2=0x7fd6480d48f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:57.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:57.647+0000 7fd64cd8b640 1 -- 172.21.15.112:0/617083172 shutdown_connections 2024-09-21T00:26:57.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:57.647+0000 7fd64cd8b640 1 -- 172.21.15.112:0/617083172 wait complete. 2024-09-21T00:26:57.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:57.649+0000 7fd64cd8b640 1 Processor -- start 2024-09-21T00:26:57.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:57.649+0000 7fd64cd8b640 1 -- start start 2024-09-21T00:26:57.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:57.650+0000 7fd64cd8b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd6480b64a0 0x7fd6480aae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:57.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:57.650+0000 7fd64cd8b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd64815e7d0 0x7fd6480cdf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:57.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:57.650+0000 7fd64cd8b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd648162c10 0x7fd64815c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:57.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:57.650+0000 7fd64cd8b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd64806e9c0 con 0x7fd648162c10 2024-09-21T00:26:57.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:57.650+0000 7fd64cd8b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd64806e840 con 0x7fd6480b64a0 2024-09-21T00:26:57.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:57.650+0000 7fd64cd8b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd64806eb40 con 0x7fd64815e7d0 2024-09-21T00:26:57.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:57.650+0000 7fd646d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd648162c10 0x7fd64815c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:57.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:57.650+0000 7fd646d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd648162c10 0x7fd64815c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40754/0 (socket says 172.21.15.112:40754) 2024-09-21T00:26:57.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:57.650+0000 7fd646d9d640 1 -- 172.21.15.112:0/573831731 learned_addr learned my addr 172.21.15.112:0/573831731 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:26:57.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:57.650+0000 7fd63ffff640 1 --2- 172.21.15.112:0/573831731 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd64815e7d0 0x7fd6480cdf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:57.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:57.650+0000 7fd64659c640 1 --2- 172.21.15.112:0/573831731 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd6480b64a0 0x7fd6480aae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:57.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:57.650+0000 7fd646d9d640 1 -- 172.21.15.112:0/573831731 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd64815e7d0 msgr2=0x7fd6480cdf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:57.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:57.650+0000 7fd646d9d640 1 --2- 172.21.15.112:0/573831731 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd64815e7d0 0x7fd6480cdf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:57.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:57.650+0000 7fd646d9d640 1 -- 172.21.15.112:0/573831731 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd6480b64a0 msgr2=0x7fd6480aae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:57.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:57.650+0000 7fd646d9d640 1 --2- 172.21.15.112:0/573831731 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd6480b64a0 0x7fd6480aae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:57.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:57.650+0000 7fd646d9d640 1 -- 172.21.15.112:0/573831731 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6481f8d90 con 0x7fd648162c10 2024-09-21T00:26:57.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:57.650+0000 7fd646d9d640 1 --2- 172.21.15.112:0/573831731 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd648162c10 0x7fd64815c9e0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fd640005fd0 tx=0x7fd640003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:57.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:57.651+0000 7fd63effd640 1 -- 172.21.15.112:0/573831731 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd640033050 con 0x7fd648162c10 2024-09-21T00:26:57.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:57.651+0000 7fd63effd640 1 -- 172.21.15.112:0/573831731 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd640004450 con 0x7fd648162c10 2024-09-21T00:26:57.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:57.651+0000 7fd63effd640 1 -- 172.21.15.112:0/573831731 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd640004840 con 0x7fd648162c10 2024-09-21T00:26:57.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:57.651+0000 7fd64cd8b640 1 -- 172.21.15.112:0/573831731 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6481f9070 con 0x7fd648162c10 2024-09-21T00:26:57.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:57.651+0000 7fd64cd8b640 1 -- 172.21.15.112:0/573831731 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6480ab430 con 0x7fd648162c10 2024-09-21T00:26:57.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:57.652+0000 7fd64cd8b640 1 -- 172.21.15.112:0/573831731 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6480ab690 con 0x7fd648162c10 2024-09-21T00:26:57.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:57.652+0000 7fd63effd640 1 -- 172.21.15.112:0/573831731 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd640036070 con 0x7fd648162c10 2024-09-21T00:26:57.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:57.653+0000 7fd63effd640 1 --2- 172.21.15.112:0/573831731 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fd628078b90 0x7fd62807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:57.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:57.653+0000 7fd63effd640 1 -- 172.21.15.112:0/573831731 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 5749+0+0 (secure 0 0 0) 0x7fd6400bc170 con 0x7fd648162c10 2024-09-21T00:26:57.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:57.653+0000 7fd64659c640 1 --2- 172.21.15.112:0/573831731 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fd628078b90 0x7fd62807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:57.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:57.654+0000 7fd64659c640 1 --2- 172.21.15.112:0/573831731 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fd628078b90 0x7fd62807b0a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd638004630 tx=0x7fd63800b2e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:57.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:57.658+0000 7fd63effd640 1 -- 172.21.15.112:0/573831731 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd640083f30 con 0x7fd648162c10 2024-09-21T00:26:57.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:57.836+0000 7fd64cd8b640 1 -- 172.21.15.112:0/573831731 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fd6481f8b20 con 0x7fd648162c10 2024-09-21T00:26:58.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:58.513+0000 7fd63effd640 1 -- 172.21.15.112:0/573831731 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v41) ==== 169+0+0 (secure 0 0 0) 0x7fd640088960 con 0x7fd648162c10 2024-09-21T00:26:58.514 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'cephfs_data' removed 2024-09-21T00:26:58.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:58.518+0000 7fd64cd8b640 1 -- 172.21.15.112:0/573831731 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fd628078b90 msgr2=0x7fd62807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:58.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:58.518+0000 7fd64cd8b640 1 --2- 172.21.15.112:0/573831731 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fd628078b90 0x7fd62807b0a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd638004630 tx=0x7fd63800b2e0 comp rx=0 tx=0).stop 2024-09-21T00:26:58.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:58.518+0000 7fd64cd8b640 1 -- 172.21.15.112:0/573831731 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd648162c10 msgr2=0x7fd64815c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:58.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:58.518+0000 7fd64cd8b640 1 --2- 172.21.15.112:0/573831731 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd648162c10 0x7fd64815c9e0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fd640005fd0 tx=0x7fd640003f10 comp rx=0 tx=0).stop 2024-09-21T00:26:58.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:58.518+0000 7fd64cd8b640 1 -- 172.21.15.112:0/573831731 shutdown_connections 2024-09-21T00:26:58.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:58.518+0000 7fd64cd8b640 1 --2- 172.21.15.112:0/573831731 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fd628078b90 0x7fd62807b0a0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:58.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:58.518+0000 7fd64cd8b640 1 --2- 172.21.15.112:0/573831731 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd648162c10 0x7fd64815c9e0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:58.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:58.518+0000 7fd64cd8b640 1 --2- 172.21.15.112:0/573831731 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd64815e7d0 0x7fd6480cdf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:58.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:58.518+0000 7fd64cd8b640 1 --2- 172.21.15.112:0/573831731 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd6480b64a0 0x7fd6480aae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:58.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:58.518+0000 7fd64cd8b640 1 -- 172.21.15.112:0/573831731 >> 172.21.15.112:0/573831731 conn(0x7fd6480d44e0 msgr2=0x7fd6480b9a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:58.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:58.518+0000 7fd64cd8b640 1 -- 172.21.15.112:0/573831731 shutdown_connections 2024-09-21T00:26:58.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:58.518+0000 7fd64cd8b640 1 -- 172.21.15.112:0/573831731 wait complete. 2024-09-21T00:26:58.533 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 31 2024-09-21T00:26:58.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:58.650+0000 7fcb2567e640 1 -- 172.21.15.112:0/3769487946 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb20074b00 msgr2=0x7fcb2015aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:58.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:58.650+0000 7fcb2567e640 1 --2- 172.21.15.112:0/3769487946 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb20074b00 0x7fcb2015aa90 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fcb200c3ba0 tx=0x7fcb1002a4a0 comp rx=0 tx=0).stop 2024-09-21T00:26:58.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:58.650+0000 7fcb2567e640 1 -- 172.21.15.112:0/3769487946 shutdown_connections 2024-09-21T00:26:58.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:58.650+0000 7fcb2567e640 1 --2- 172.21.15.112:0/3769487946 >> v2:172.21.15.187:3300/0 conn(0x7fcb20070370 0x7fcb200cdc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:58.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:58.650+0000 7fcb2567e640 1 --2- 172.21.15.112:0/3769487946 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb20074b00 0x7fcb2015aa90 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:58.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:58.650+0000 7fcb2567e640 1 -- 172.21.15.112:0/3769487946 >> 172.21.15.112:0/3769487946 conn(0x7fcb200b1b90 msgr2=0x7fcb200b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:58.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:58.650+0000 7fcb2567e640 1 -- 172.21.15.112:0/3769487946 shutdown_connections 2024-09-21T00:26:58.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:58.650+0000 7fcb2567e640 1 -- 172.21.15.112:0/3769487946 wait complete. 2024-09-21T00:26:58.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:58.652+0000 7fcb2567e640 1 Processor -- start 2024-09-21T00:26:58.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:58.652+0000 7fcb2567e640 1 -- start start 2024-09-21T00:26:58.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:58.653+0000 7fcb2567e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcb20070370 0x7fcb200c5690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:58.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:58.653+0000 7fcb2567e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcb20074b00 0x7fcb200c3340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:58.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:58.653+0000 7fcb2567e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb2015afd0 0x7fcb200c3880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:58.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:58.653+0000 7fcb2567e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcb2006e360 con 0x7fcb2015afd0 2024-09-21T00:26:58.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:58.653+0000 7fcb2567e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fcb2006e1e0 con 0x7fcb20074b00 2024-09-21T00:26:58.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:58.653+0000 7fcb2567e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fcb2006e4e0 con 0x7fcb20070370 2024-09-21T00:26:58.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:58.653+0000 7fcb1f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb2015afd0 0x7fcb200c3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:58.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:58.653+0000 7fcb1f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb2015afd0 0x7fcb200c3880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40774/0 (socket says 172.21.15.112:40774) 2024-09-21T00:26:58.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:58.653+0000 7fcb1f7fe640 1 -- 172.21.15.112:0/3989893324 learned_addr learned my addr 172.21.15.112:0/3989893324 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:26:58.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:58.654+0000 7fcb167fc640 1 --2- 172.21.15.112:0/3989893324 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcb20074b00 0x7fcb200c3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:58.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:58.654+0000 7fcb1effd640 1 --2- 172.21.15.112:0/3989893324 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcb20070370 0x7fcb200c5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:58.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:58.654+0000 7fcb1f7fe640 1 -- 172.21.15.112:0/3989893324 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcb20070370 msgr2=0x7fcb200c5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:58.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:58.654+0000 7fcb1f7fe640 1 --2- 172.21.15.112:0/3989893324 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcb20070370 0x7fcb200c5690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:58.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:58.654+0000 7fcb1f7fe640 1 -- 172.21.15.112:0/3989893324 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcb20074b00 msgr2=0x7fcb200c3340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:58.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:58.654+0000 7fcb1f7fe640 1 --2- 172.21.15.112:0/3989893324 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcb20074b00 0x7fcb200c3340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:58.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:58.654+0000 7fcb1f7fe640 1 -- 172.21.15.112:0/3989893324 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb200c4950 con 0x7fcb2015afd0 2024-09-21T00:26:58.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:58.654+0000 7fcb167fc640 1 --2- 172.21.15.112:0/3989893324 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcb20074b00 0x7fcb200c3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:26:58.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:58.654+0000 7fcb1f7fe640 1 --2- 172.21.15.112:0/3989893324 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb2015afd0 0x7fcb200c3880 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fcb1800cad0 tx=0x7fcb1800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:58.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:58.654+0000 7fcb17fff640 1 -- 172.21.15.112:0/3989893324 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb1800adc0 con 0x7fcb2015afd0 2024-09-21T00:26:58.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:58.654+0000 7fcb17fff640 1 -- 172.21.15.112:0/3989893324 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcb1800f540 con 0x7fcb2015afd0 2024-09-21T00:26:58.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:58.654+0000 7fcb2567e640 1 -- 172.21.15.112:0/3989893324 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb2015a170 con 0x7fcb2015afd0 2024-09-21T00:26:58.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:58.654+0000 7fcb17fff640 1 -- 172.21.15.112:0/3989893324 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb1800f990 con 0x7fcb2015afd0 2024-09-21T00:26:58.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:58.654+0000 7fcb2567e640 1 -- 172.21.15.112:0/3989893324 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb200c8640 con 0x7fcb2015afd0 2024-09-21T00:26:58.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:58.655+0000 7fcb2567e640 1 -- 172.21.15.112:0/3989893324 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb200a0e00 con 0x7fcb2015afd0 2024-09-21T00:26:58.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:58.656+0000 7fcb17fff640 1 -- 172.21.15.112:0/3989893324 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fcb1800fb30 con 0x7fcb2015afd0 2024-09-21T00:26:58.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:58.656+0000 7fcb17fff640 1 --2- 172.21.15.112:0/3989893324 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fcb00078bc0 0x7fcb0007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:58.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:58.656+0000 7fcb17fff640 1 -- 172.21.15.112:0/3989893324 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 5335+0+0 (secure 0 0 0) 0x7fcb1809b5f0 con 0x7fcb2015afd0 2024-09-21T00:26:58.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:58.656+0000 7fcb1effd640 1 --2- 172.21.15.112:0/3989893324 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fcb00078bc0 0x7fcb0007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:58.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:58.657+0000 7fcb1effd640 1 --2- 172.21.15.112:0/3989893324 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fcb00078bc0 0x7fcb0007b0d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fcb10034000 tx=0x7fcb10004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:58.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:58.661+0000 7fcb17fff640 1 -- 172.21.15.112:0/3989893324 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fcb200a0e00 con 0x7fcb2015afd0 2024-09-21T00:26:58.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:58.842+0000 7fcb2567e640 1 -- 172.21.15.112:0/3989893324 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "config reset", "num": 31} v 0) -- 0x7fcb200c88a0 con 0x7fcb2015afd0 2024-09-21T00:26:58.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:58.846+0000 7fcb17fff640 1 -- 172.21.15.112:0/3989893324 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 31}]=0 v34) ==== 71+0+0 (secure 0 0 0) 0x7fcb18014030 con 0x7fcb2015afd0 2024-09-21T00:26:58.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:58.848+0000 7fcb2567e640 1 -- 172.21.15.112:0/3989893324 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fcb00078bc0 msgr2=0x7fcb0007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:58.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:58.848+0000 7fcb2567e640 1 --2- 172.21.15.112:0/3989893324 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fcb00078bc0 0x7fcb0007b0d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fcb10034000 tx=0x7fcb10004730 comp rx=0 tx=0).stop 2024-09-21T00:26:58.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:58.848+0000 7fcb2567e640 1 -- 172.21.15.112:0/3989893324 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb2015afd0 msgr2=0x7fcb200c3880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:58.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:58.848+0000 7fcb2567e640 1 --2- 172.21.15.112:0/3989893324 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb2015afd0 0x7fcb200c3880 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fcb1800cad0 tx=0x7fcb1800cf90 comp rx=0 tx=0).stop 2024-09-21T00:26:58.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:58.848+0000 7fcb2567e640 1 -- 172.21.15.112:0/3989893324 shutdown_connections 2024-09-21T00:26:58.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:58.848+0000 7fcb2567e640 1 --2- 172.21.15.112:0/3989893324 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fcb00078bc0 0x7fcb0007b0d0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:58.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:58.848+0000 7fcb2567e640 1 --2- 172.21.15.112:0/3989893324 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb2015afd0 0x7fcb200c3880 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:58.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:58.848+0000 7fcb2567e640 1 --2- 172.21.15.112:0/3989893324 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcb20074b00 0x7fcb200c3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:58.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:58.848+0000 7fcb2567e640 1 --2- 172.21.15.112:0/3989893324 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcb20070370 0x7fcb200c5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:58.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:58.848+0000 7fcb2567e640 1 -- 172.21.15.112:0/3989893324 >> 172.21.15.112:0/3989893324 conn(0x7fcb200b1b90 msgr2=0x7fcb2006ec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:58.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:58.849+0000 7fcb2567e640 1 -- 172.21.15.112:0/3989893324 shutdown_connections 2024-09-21T00:26:58.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:58.849+0000 7fcb2567e640 1 -- 172.21.15.112:0/3989893324 wait complete. 2024-09-21T00:26:58.860 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_sessionmap.TestSessionMap.test_mount_conn_close' 2024-09-21T00:26:58.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:58.972+0000 7fcbbdc9d640 1 -- 172.21.15.112:0/3984697585 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcbb80b2e60 msgr2=0x7fcbb80b3270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:58.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:58.972+0000 7fcbbdc9d640 1 --2- 172.21.15.112:0/3984697585 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcbb80b2e60 0x7fcbb80b3270 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fcbb0005fd0 tx=0x7fcbb002aa90 comp rx=0 tx=0).stop 2024-09-21T00:26:58.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:58.972+0000 7fcbbdc9d640 1 -- 172.21.15.112:0/3984697585 shutdown_connections 2024-09-21T00:26:58.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:58.972+0000 7fcbbdc9d640 1 --2- 172.21.15.112:0/3984697585 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcbb80b2e60 0x7fcbb80b3270 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:58.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:58.972+0000 7fcbbdc9d640 1 --2- 172.21.15.112:0/3984697585 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcbb80cfe00 0x7fcbb80d2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:58.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:58.972+0000 7fcbbdc9d640 1 -- 172.21.15.112:0/3984697585 >> 172.21.15.112:0/3984697585 conn(0x7fcbb80d4440 msgr2=0x7fcbb80d4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:58.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:58.972+0000 7fcbbdc9d640 1 -- 172.21.15.112:0/3984697585 shutdown_connections 2024-09-21T00:26:58.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:58.972+0000 7fcbbdc9d640 1 -- 172.21.15.112:0/3984697585 wait complete. 2024-09-21T00:26:58.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:58.974+0000 7fcbbdc9d640 1 Processor -- start 2024-09-21T00:26:58.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:58.974+0000 7fcbbdc9d640 1 -- start start 2024-09-21T00:26:58.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:58.975+0000 7fcbbdc9d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcbb80b2e60 0x7fcbb81581f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:58.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:58.976+0000 7fcbbdc9d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcbb80cfe00 0x7fcbb80c73c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:58.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:58.976+0000 7fcbbdc9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcbb8162590 0x7fcbb8157490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:58.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:58.976+0000 7fcbbdc9d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcbb80b3930 con 0x7fcbb8162590 2024-09-21T00:26:58.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:58.976+0000 7fcbbdc9d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fcbb80b37b0 con 0x7fcbb80b2e60 2024-09-21T00:26:58.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:58.976+0000 7fcbbdc9d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fcbb80b3ab0 con 0x7fcbb80cfe00 2024-09-21T00:26:58.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:58.976+0000 7fcbb7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcbb8162590 0x7fcbb8157490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:58.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:58.976+0000 7fcbb7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcbb8162590 0x7fcbb8157490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40784/0 (socket says 172.21.15.112:40784) 2024-09-21T00:26:58.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:58.976+0000 7fcbb7fff640 1 -- 172.21.15.112:0/1114228823 learned_addr learned my addr 172.21.15.112:0/1114228823 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:26:58.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:58.976+0000 7fcbb6ffd640 1 --2- 172.21.15.112:0/1114228823 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcbb80cfe00 0x7fcbb80c73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:58.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:58.976+0000 7fcbb77fe640 1 --2- 172.21.15.112:0/1114228823 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcbb80b2e60 0x7fcbb81581f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:58.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:58.976+0000 7fcbb7fff640 1 -- 172.21.15.112:0/1114228823 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcbb80cfe00 msgr2=0x7fcbb80c73c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:58.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:58.976+0000 7fcbb7fff640 1 --2- 172.21.15.112:0/1114228823 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcbb80cfe00 0x7fcbb80c73c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:58.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:58.976+0000 7fcbb7fff640 1 -- 172.21.15.112:0/1114228823 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcbb80b2e60 msgr2=0x7fcbb81581f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:58.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:58.976+0000 7fcbb7fff640 1 --2- 172.21.15.112:0/1114228823 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcbb80b2e60 0x7fcbb81581f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:58.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:58.976+0000 7fcbb7fff640 1 -- 172.21.15.112:0/1114228823 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbb81f9970 con 0x7fcbb8162590 2024-09-21T00:26:58.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:58.976+0000 7fcbb6ffd640 1 --2- 172.21.15.112:0/1114228823 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcbb80cfe00 0x7fcbb80c73c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:26:58.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:58.976+0000 7fcbb77fe640 1 --2- 172.21.15.112:0/1114228823 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcbb80b2e60 0x7fcbb81581f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:26:58.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:58.976+0000 7fcbb7fff640 1 --2- 172.21.15.112:0/1114228823 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcbb8162590 0x7fcbb8157490 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fcbb0009d90 tx=0x7fcbb0003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:58.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:58.976+0000 7fcbaffff640 1 -- 172.21.15.112:0/1114228823 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb0044070 con 0x7fcbb8162590 2024-09-21T00:26:58.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:58.976+0000 7fcbaffff640 1 -- 172.21.15.112:0/1114228823 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcbb0004040 con 0x7fcbb8162590 2024-09-21T00:26:58.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:58.976+0000 7fcbbdc9d640 1 -- 172.21.15.112:0/1114228823 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbb814fb50 con 0x7fcbb8162590 2024-09-21T00:26:58.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:58.976+0000 7fcbaffff640 1 -- 172.21.15.112:0/1114228823 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb0004430 con 0x7fcbb8162590 2024-09-21T00:26:58.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:58.976+0000 7fcbbdc9d640 1 -- 172.21.15.112:0/1114228823 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbb8152ac0 con 0x7fcbb8162590 2024-09-21T00:26:58.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:58.977+0000 7fcbbdc9d640 1 -- 172.21.15.112:0/1114228823 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbb81f96e0 con 0x7fcbb8162590 2024-09-21T00:26:58.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:58.978+0000 7fcbaffff640 1 -- 172.21.15.112:0/1114228823 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fcbb0047070 con 0x7fcbb8162590 2024-09-21T00:26:58.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:58.978+0000 7fcbaffff640 1 --2- 172.21.15.112:0/1114228823 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fcb98078bc0 0x7fcb9807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:58.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:58.979+0000 7fcbaffff640 1 -- 172.21.15.112:0/1114228823 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 5335+0+0 (secure 0 0 0) 0x7fcbb00bba60 con 0x7fcbb8162590 2024-09-21T00:26:58.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:58.979+0000 7fcbb77fe640 1 --2- 172.21.15.112:0/1114228823 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fcb98078bc0 0x7fcb9807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:58.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:58.979+0000 7fcbb77fe640 1 --2- 172.21.15.112:0/1114228823 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fcb98078bc0 0x7fcb9807b0d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fcba8009b80 tx=0x7fcba8005f90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:58.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:58.983+0000 7fcbaffff640 1 -- 172.21.15.112:0/1114228823 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fcbb0032030 con 0x7fcbb8162590 2024-09-21T00:26:59.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:59.228+0000 7fcbbdc9d640 1 -- 172.21.15.112:0/1114228823 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_sessionmap.TestSessionMap.test_mount_conn_close"]} v 0) -- 0x7fcbb814f940 con 0x7fcbb8162590 2024-09-21T00:26:59.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:59.521+0000 7fcbaffff640 1 -- 172.21.15.112:0/1114228823 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_sessionmap.TestSessionMap.test_mount_conn_close"]}]=0 v91) ==== 144+0+0 (secure 0 0 0) 0x7fcbb0084010 con 0x7fcbb8162590 2024-09-21T00:26:59.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:59.524+0000 7fcbbdc9d640 1 -- 172.21.15.112:0/1114228823 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fcb98078bc0 msgr2=0x7fcb9807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:59.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:59.524+0000 7fcbbdc9d640 1 --2- 172.21.15.112:0/1114228823 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fcb98078bc0 0x7fcb9807b0d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fcba8009b80 tx=0x7fcba8005f90 comp rx=0 tx=0).stop 2024-09-21T00:26:59.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:59.524+0000 7fcbbdc9d640 1 -- 172.21.15.112:0/1114228823 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcbb8162590 msgr2=0x7fcbb8157490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:59.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:59.524+0000 7fcbbdc9d640 1 --2- 172.21.15.112:0/1114228823 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcbb8162590 0x7fcbb8157490 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fcbb0009d90 tx=0x7fcbb0003ce0 comp rx=0 tx=0).stop 2024-09-21T00:26:59.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:59.524+0000 7fcbbdc9d640 1 -- 172.21.15.112:0/1114228823 shutdown_connections 2024-09-21T00:26:59.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:59.524+0000 7fcbbdc9d640 1 --2- 172.21.15.112:0/1114228823 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fcb98078bc0 0x7fcb9807b0d0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:59.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:59.524+0000 7fcbbdc9d640 1 --2- 172.21.15.112:0/1114228823 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcbb8162590 0x7fcbb8157490 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:59.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:59.524+0000 7fcbbdc9d640 1 --2- 172.21.15.112:0/1114228823 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcbb80cfe00 0x7fcbb80c73c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:59.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:59.524+0000 7fcbbdc9d640 1 --2- 172.21.15.112:0/1114228823 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcbb80b2e60 0x7fcbb81581f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:59.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:59.524+0000 7fcbbdc9d640 1 -- 172.21.15.112:0/1114228823 >> 172.21.15.112:0/1114228823 conn(0x7fcbb80d4440 msgr2=0x7fcbb81605c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:59.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:59.524+0000 7fcbbdc9d640 1 -- 172.21.15.112:0/1114228823 shutdown_connections 2024-09-21T00:26:59.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:59.524+0000 7fcbbdc9d640 1 -- 172.21.15.112:0/1114228823 wait complete. 2024-09-21T00:26:59.536 INFO:tasks.cephfs_test_runner:That when a client unmounts, the thread count on the MDS goes back ... ok 2024-09-21T00:26:59.537 INFO:tasks.cephfs_test_runner:Starting test: test_session_evict_blocklisted (tasks.cephfs.test_sessionmap.TestSessionMap) Check that mds evicts blocklisted client 2024-09-21T00:26:59.537 INFO:tasks.cephfs_test_runner:test_session_evict_blocklisted (tasks.cephfs.test_sessionmap.TestSessionMap) 2024-09-21T00:26:59.537 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T00:26:59.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:59.649+0000 7ff366571640 1 -- 172.21.15.112:0/3024138014 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff348004800 msgr2=0x7ff348004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:59.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:59.649+0000 7ff366571640 1 --2- 172.21.15.112:0/3024138014 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff348004800 0x7ff348004c10 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7ff354006fc0 tx=0x7ff348005830 comp rx=0 tx=0).stop 2024-09-21T00:26:59.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:59.649+0000 7ff366571640 1 -- 172.21.15.112:0/3024138014 shutdown_connections 2024-09-21T00:26:59.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:59.649+0000 7ff366571640 1 --2- 172.21.15.112:0/3024138014 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff348004800 0x7ff348004c10 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:59.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:59.650+0000 7ff366571640 1 -- 172.21.15.112:0/3024138014 >> 172.21.15.112:0/3024138014 conn(0x7ff360072990 msgr2=0x7ff360072da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:59.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:59.650+0000 7ff366571640 1 -- 172.21.15.112:0/3024138014 shutdown_connections 2024-09-21T00:26:59.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:59.650+0000 7ff366571640 1 -- 172.21.15.112:0/3024138014 wait complete. 2024-09-21T00:26:59.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:59.651+0000 7ff366571640 1 Processor -- start 2024-09-21T00:26:59.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:59.651+0000 7ff366571640 1 -- start start 2024-09-21T00:26:59.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:59.653+0000 7ff366571640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff348004800 0x7ff3600b2e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:59.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:59.653+0000 7ff366571640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff360149620 0x7ff3600c8180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:59.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:59.653+0000 7ff366571640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff3601f6b00 0x7ff3601f6f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:59.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:59.653+0000 7ff366571640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff36015de80 con 0x7ff348004800 2024-09-21T00:26:59.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:59.653+0000 7ff366571640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff36015dd00 con 0x7ff360149620 2024-09-21T00:26:59.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:59.653+0000 7ff366571640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff36015e230 con 0x7ff3601f6b00 2024-09-21T00:26:59.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:59.653+0000 7ff35ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff348004800 0x7ff3600b2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:59.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:59.653+0000 7ff35ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff348004800 0x7ff3600b2e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40800/0 (socket says 172.21.15.112:40800) 2024-09-21T00:26:59.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:59.653+0000 7ff35ffff640 1 -- 172.21.15.112:0/828660956 learned_addr learned my addr 172.21.15.112:0/828660956 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:26:59.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:59.653+0000 7ff35f7fe640 1 --2- 172.21.15.112:0/828660956 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff360149620 0x7ff3600c8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:59.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:59.653+0000 7ff364b0e640 1 --2- 172.21.15.112:0/828660956 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff3601f6b00 0x7ff3601f6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:59.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:59.653+0000 7ff35ffff640 1 -- 172.21.15.112:0/828660956 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff3601f6b00 msgr2=0x7ff3601f6f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:59.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:59.653+0000 7ff35ffff640 1 --2- 172.21.15.112:0/828660956 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff3601f6b00 0x7ff3601f6f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:59.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:59.653+0000 7ff35ffff640 1 -- 172.21.15.112:0/828660956 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff360149620 msgr2=0x7ff3600c8180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:59.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:59.653+0000 7ff35ffff640 1 --2- 172.21.15.112:0/828660956 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff360149620 0x7ff3600c8180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:59.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:59.653+0000 7ff35ffff640 1 -- 172.21.15.112:0/828660956 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3600c70e0 con 0x7ff348004800 2024-09-21T00:26:59.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:59.653+0000 7ff35ffff640 1 --2- 172.21.15.112:0/828660956 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff348004800 0x7ff3600b2e20 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7ff350011a80 tx=0x7ff350011f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:59.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:59.653+0000 7ff35cff9640 1 -- 172.21.15.112:0/828660956 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff35000fe10 con 0x7ff348004800 2024-09-21T00:26:59.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:59.653+0000 7ff35cff9640 1 -- 172.21.15.112:0/828660956 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff350016070 con 0x7ff348004800 2024-09-21T00:26:59.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:59.654+0000 7ff366571640 1 -- 172.21.15.112:0/828660956 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3600c73c0 con 0x7ff348004800 2024-09-21T00:26:59.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:59.654+0000 7ff35cff9640 1 -- 172.21.15.112:0/828660956 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff350012780 con 0x7ff348004800 2024-09-21T00:26:59.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:59.654+0000 7ff366571640 1 -- 172.21.15.112:0/828660956 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3600c7980 con 0x7ff348004800 2024-09-21T00:26:59.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:59.655+0000 7ff366571640 1 -- 172.21.15.112:0/828660956 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff36015e790 con 0x7ff348004800 2024-09-21T00:26:59.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:59.655+0000 7ff35cff9640 1 -- 172.21.15.112:0/828660956 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff350018030 con 0x7ff348004800 2024-09-21T00:26:59.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:59.656+0000 7ff35cff9640 1 --2- 172.21.15.112:0/828660956 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff34c078bc0 0x7ff34c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:59.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:59.656+0000 7ff35cff9640 1 -- 172.21.15.112:0/828660956 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 5335+0+0 (secure 0 0 0) 0x7ff35009b510 con 0x7ff348004800 2024-09-21T00:26:59.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:59.656+0000 7ff35f7fe640 1 --2- 172.21.15.112:0/828660956 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff34c078bc0 0x7ff34c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:59.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:59.656+0000 7ff35f7fe640 1 --2- 172.21.15.112:0/828660956 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff34c078bc0 0x7ff34c07b0d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff354033000 tx=0x7ff354000fe0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:59.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:59.660+0000 7ff35cff9640 1 -- 172.21.15.112:0/828660956 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ff350063ac0 con 0x7ff348004800 2024-09-21T00:26:59.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:59.843+0000 7ff366571640 1 -- 172.21.15.112:0/828660956 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff360070c50 con 0x7ff348004800 2024-09-21T00:26:59.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:59.844+0000 7ff35cff9640 1 -- 172.21.15.112:0/828660956 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) ==== 74+0+13331 (secure 0 0 0) 0x7ff3500633c0 con 0x7ff348004800 2024-09-21T00:26:59.845 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:26:59.845 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":41,"fsid":"8b34692b-dc65-49b2-a08f-26f707b96ac5","created":"2024-09-21T00:25:22.788490+0000","modified":"2024-09-21T00:26:58.512511+0000","last_up_change":"2024-09-21T00:25:36.161835+0000","last_in_change":"2024-09-21T00:25:26.346500+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T00:25:37.105006+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9c575086-67d2-4de2-b01b-d44dcc21d604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6809","nonce":755760621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6811","nonce":755760621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6815","nonce":755760621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6813","nonce":755760621}]},"public_addr":"172.21.15.112:6809/755760621","cluster_addr":"172.21.15.112:6811/755760621","heartbeat_back_addr":"172.21.15.112:6815/755760621","heartbeat_front_addr":"172.21.15.112:6813/755760621","state":["exists","up"]},{"osd":1,"uuid":"a983c227-a3ce-4bdd-82f1-438ba7e3ecd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6801","nonce":344932592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6803","nonce":344932592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6807","nonce":344932592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6805","nonce":344932592}]},"public_addr":"172.21.15.112:6801/344932592","cluster_addr":"172.21.15.112:6803/344932592","heartbeat_back_addr":"172.21.15.112:6807/344932592","heartbeat_front_addr":"172.21.15.112:6805/344932592","state":["exists","up"]},{"osd":2,"uuid":"1193c1d8-bfff-4a57-8200-5f69bf1ea3af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6825","nonce":407933616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6827","nonce":407933616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6831","nonce":407933616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6829","nonce":407933616}]},"public_addr":"172.21.15.112:6825/407933616","cluster_addr":"172.21.15.112:6827/407933616","heartbeat_back_addr":"172.21.15.112:6831/407933616","heartbeat_front_addr":"172.21.15.112:6829/407933616","state":["exists","up"]},{"osd":3,"uuid":"af04330a-36ad-40cc-a8dd-fa259b221de5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6817","nonce":4120164806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6819","nonce":4120164806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6823","nonce":4120164806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6821","nonce":4120164806}]},"public_addr":"172.21.15.112:6817/4120164806","cluster_addr":"172.21.15.112:6819/4120164806","heartbeat_back_addr":"172.21.15.112:6823/4120164806","heartbeat_front_addr":"172.21.15.112:6821/4120164806","state":["exists","up"]},{"osd":4,"uuid":"5b893111-292c-4c5a-b184-53ad38c7199e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6803","nonce":522077560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6805","nonce":522077560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6809","nonce":522077560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6807","nonce":522077560}]},"public_addr":"172.21.15.187:6803/522077560","cluster_addr":"172.21.15.187:6805/522077560","heartbeat_back_addr":"172.21.15.187:6809/522077560","heartbeat_front_addr":"172.21.15.187:6807/522077560","state":["exists","up"]},{"osd":5,"uuid":"c0b8e011-45fe-4c5e-925e-6be2a7f1868a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6827","nonce":3593144721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6829","nonce":3593144721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6833","nonce":3593144721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6831","nonce":3593144721}]},"public_addr":"172.21.15.187:6827/3593144721","cluster_addr":"172.21.15.187:6829/3593144721","heartbeat_back_addr":"172.21.15.187:6833/3593144721","heartbeat_front_addr":"172.21.15.187:6831/3593144721","state":["exists","up"]},{"osd":6,"uuid":"87d2d650-7489-4593-98a3-a1f433ce8dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6811","nonce":1494387657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6813","nonce":1494387657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6817","nonce":1494387657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6815","nonce":1494387657}]},"public_addr":"172.21.15.187:6811/1494387657","cluster_addr":"172.21.15.187:6813/1494387657","heartbeat_back_addr":"172.21.15.187:6817/1494387657","heartbeat_front_addr":"172.21.15.187:6815/1494387657","state":["exists","up"]},{"osd":7,"uuid":"c6b4f8e2-a1ee-4438-aa37-67ff4b6ef1f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6819","nonce":1624699842}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6821","nonce":1624699842}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6825","nonce":1624699842}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6823","nonce":1624699842}]},"public_addr":"172.21.15.187:6819/1624699842","cluster_addr":"172.21.15.187:6821/1624699842","heartbeat_back_addr":"172.21.15.187:6825/1624699842","heartbeat_front_addr":"172.21.15.187:6823/1624699842","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.873162+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.400506+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.664317+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.159791+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.202630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.113212+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.384786+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.448441+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:6835/4012316979":"2024-09-22T00:26:55.469618+0000","172.21.15.112:6833/4012316979":"2024-09-22T00:26:55.469618+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T00:26:59.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:59.846+0000 7ff366571640 1 -- 172.21.15.112:0/828660956 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff34c078bc0 msgr2=0x7ff34c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:59.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:59.846+0000 7ff366571640 1 --2- 172.21.15.112:0/828660956 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff34c078bc0 0x7ff34c07b0d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff354033000 tx=0x7ff354000fe0 comp rx=0 tx=0).stop 2024-09-21T00:26:59.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:59.846+0000 7ff366571640 1 -- 172.21.15.112:0/828660956 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff348004800 msgr2=0x7ff3600b2e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:59.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:59.846+0000 7ff366571640 1 --2- 172.21.15.112:0/828660956 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff348004800 0x7ff3600b2e20 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7ff350011a80 tx=0x7ff350011f40 comp rx=0 tx=0).stop 2024-09-21T00:26:59.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:59.846+0000 7ff366571640 1 -- 172.21.15.112:0/828660956 shutdown_connections 2024-09-21T00:26:59.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:59.846+0000 7ff366571640 1 --2- 172.21.15.112:0/828660956 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff3601f6b00 0x7ff3601f6f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:59.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:59.846+0000 7ff366571640 1 --2- 172.21.15.112:0/828660956 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff360149620 0x7ff3600c8180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:59.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:59.846+0000 7ff366571640 1 --2- 172.21.15.112:0/828660956 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff34c078bc0 0x7ff34c07b0d0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:59.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:59.846+0000 7ff366571640 1 --2- 172.21.15.112:0/828660956 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff348004800 0x7ff3600b2e20 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:59.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:59.846+0000 7ff366571640 1 -- 172.21.15.112:0/828660956 >> 172.21.15.112:0/828660956 conn(0x7ff360072990 msgr2=0x7ff3601594a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:59.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:59.847+0000 7ff366571640 1 -- 172.21.15.112:0/828660956 shutdown_connections 2024-09-21T00:26:59.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:59.847+0000 7ff366571640 1 -- 172.21.15.112:0/828660956 wait complete. 2024-09-21T00:26:59.859 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T00:25:37.105006+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-21T00:26:59.859 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-21T00:26:59.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:59.973+0000 7f7e3d2dc640 1 -- 172.21.15.112:0/1293294168 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e200039e0 msgr2=0x7f7e20003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:59.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:59.973+0000 7f7e3d2dc640 1 --2- 172.21.15.112:0/1293294168 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e200039e0 0x7f7e20003e10 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f7e2800c760 tx=0x7f7e20005830 comp rx=0 tx=0).stop 2024-09-21T00:26:59.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:59.973+0000 7f7e377fe640 1 -- 172.21.15.112:0/1293294168 reap_dead start 2024-09-21T00:26:59.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:59.973+0000 7f7e3d2dc640 1 -- 172.21.15.112:0/1293294168 shutdown_connections 2024-09-21T00:26:59.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:59.973+0000 7f7e3d2dc640 1 -- 172.21.15.112:0/1293294168 >> 172.21.15.112:0/1293294168 conn(0x7f7e3809d830 msgr2=0x7f7e380b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:26:59.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:59.973+0000 7f7e3d2dc640 1 -- 172.21.15.112:0/1293294168 shutdown_connections 2024-09-21T00:26:59.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:59.973+0000 7f7e3d2dc640 1 -- 172.21.15.112:0/1293294168 wait complete. 2024-09-21T00:26:59.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:59.975+0000 7f7e3d2dc640 1 Processor -- start 2024-09-21T00:26:59.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:59.975+0000 7f7e3d2dc640 1 -- start start 2024-09-21T00:26:59.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:59.977+0000 7f7e3d2dc640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7e200039e0 0x7f7e3806cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:59.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:59.977+0000 7f7e3d2dc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e38157bc0 0x7f7e38157300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:59.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:59.977+0000 7f7e3d2dc640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7e3815da60 0x7f7e3815de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:59.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:59.977+0000 7f7e3d2dc640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7e380cf330 con 0x7f7e38157bc0 2024-09-21T00:26:59.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:59.977+0000 7f7e3d2dc640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7e380cf1b0 con 0x7f7e200039e0 2024-09-21T00:26:59.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:59.977+0000 7f7e3d2dc640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f7e380b6580 con 0x7f7e3815da60 2024-09-21T00:26:59.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:59.977+0000 7f7e367fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e38157bc0 0x7f7e38157300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:59.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:59.977+0000 7f7e367fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e38157bc0 0x7f7e38157300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40814/0 (socket says 172.21.15.112:40814) 2024-09-21T00:26:59.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:59.977+0000 7f7e367fc640 1 -- 172.21.15.112:0/2561816767 learned_addr learned my addr 172.21.15.112:0/2561816767 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:26:59.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:59.977+0000 7f7e367fc640 1 -- 172.21.15.112:0/2561816767 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7e3815da60 msgr2=0x7f7e3815de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:59.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:59.977+0000 7f7e367fc640 1 --2- 172.21.15.112:0/2561816767 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7e3815da60 0x7f7e3815de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:59.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:59.977+0000 7f7e367fc640 1 -- 172.21.15.112:0/2561816767 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7e200039e0 msgr2=0x7f7e3806cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:26:59.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:59.977+0000 7f7e367fc640 1 --2- 172.21.15.112:0/2561816767 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7e200039e0 0x7f7e3806cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:26:59.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:59.977+0000 7f7e367fc640 1 -- 172.21.15.112:0/2561816767 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e3815e3b0 con 0x7f7e38157bc0 2024-09-21T00:26:59.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:59.977+0000 7f7e367fc640 1 --2- 172.21.15.112:0/2561816767 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e38157bc0 0x7f7e38157300 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c003cb0 tx=0x7f7e2c0018d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:59.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:59.977+0000 7f7e1f7fe640 1 -- 172.21.15.112:0/2561816767 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e2c015070 con 0x7f7e38157bc0 2024-09-21T00:26:59.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:59.977+0000 7f7e1f7fe640 1 -- 172.21.15.112:0/2561816767 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7e2c011840 con 0x7f7e38157bc0 2024-09-21T00:26:59.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:59.977+0000 7f7e1f7fe640 1 -- 172.21.15.112:0/2561816767 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e2c011c10 con 0x7f7e38157bc0 2024-09-21T00:26:59.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:59.977+0000 7f7e3d2dc640 1 -- 172.21.15.112:0/2561816767 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e3806d630 con 0x7f7e38157bc0 2024-09-21T00:26:59.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:59.977+0000 7f7e3d2dc640 1 -- 172.21.15.112:0/2561816767 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e3806dbf0 con 0x7f7e38157bc0 2024-09-21T00:26:59.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:59.978+0000 7f7e3d2dc640 1 -- 172.21.15.112:0/2561816767 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e380b2940 con 0x7f7e38157bc0 2024-09-21T00:26:59.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:59.979+0000 7f7e1f7fe640 1 -- 172.21.15.112:0/2561816767 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f7e2c018070 con 0x7f7e38157bc0 2024-09-21T00:26:59.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:59.980+0000 7f7e1f7fe640 1 --2- 172.21.15.112:0/2561816767 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f7e24078b90 0x7f7e2407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:26:59.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:59.980+0000 7f7e1f7fe640 1 -- 172.21.15.112:0/2561816767 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 5335+0+0 (secure 0 0 0) 0x7f7e2c09b7d0 con 0x7f7e38157bc0 2024-09-21T00:26:59.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:59.980+0000 7f7e36ffd640 1 --2- 172.21.15.112:0/2561816767 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f7e24078b90 0x7f7e2407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:26:59.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:59.981+0000 7f7e36ffd640 1 --2- 172.21.15.112:0/2561816767 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f7e24078b90 0x7f7e2407b0a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7e28033000 tx=0x7f7e3815f630 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:26:59.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:26:59.984+0000 7f7e1f7fe640 1 -- 172.21.15.112:0/2561816767 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f7e380b2940 con 0x7f7e38157bc0 2024-09-21T00:27:00.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:00.160+0000 7f7e3d2dc640 1 -- 172.21.15.112:0/2561816767 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7e380b6ae0 con 0x7f7e38157bc0 2024-09-21T00:27:00.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:00.160+0000 7f7e1f7fe640 1 -- 172.21.15.112:0/2561816767 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v41) ==== 93+0+10 (secure 0 0 0) 0x7f7e3815e3b0 con 0x7f7e38157bc0 2024-09-21T00:27:00.161 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2024-09-21T00:27:00.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:00.162+0000 7f7e3d2dc640 1 -- 172.21.15.112:0/2561816767 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f7e24078b90 msgr2=0x7f7e2407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:00.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:00.162+0000 7f7e3d2dc640 1 --2- 172.21.15.112:0/2561816767 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f7e24078b90 0x7f7e2407b0a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7e28033000 tx=0x7f7e3815f630 comp rx=0 tx=0).stop 2024-09-21T00:27:00.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:00.162+0000 7f7e3d2dc640 1 -- 172.21.15.112:0/2561816767 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e38157bc0 msgr2=0x7f7e38157300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:00.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:00.162+0000 7f7e3d2dc640 1 --2- 172.21.15.112:0/2561816767 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e38157bc0 0x7f7e38157300 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c003cb0 tx=0x7f7e2c0018d0 comp rx=0 tx=0).stop 2024-09-21T00:27:00.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:00.162+0000 7f7e3d2dc640 1 -- 172.21.15.112:0/2561816767 shutdown_connections 2024-09-21T00:27:00.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:00.162+0000 7f7e3d2dc640 1 --2- 172.21.15.112:0/2561816767 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7e3815da60 0x7f7e3815de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:00.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:00.162+0000 7f7e3d2dc640 1 --2- 172.21.15.112:0/2561816767 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f7e24078b90 0x7f7e2407b0a0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:00.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:00.162+0000 7f7e3d2dc640 1 --2- 172.21.15.112:0/2561816767 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e38157bc0 0x7f7e38157300 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:00.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:00.162+0000 7f7e3d2dc640 1 --2- 172.21.15.112:0/2561816767 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7e200039e0 0x7f7e3806cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:00.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:00.162+0000 7f7e3d2dc640 1 -- 172.21.15.112:0/2561816767 >> 172.21.15.112:0/2561816767 conn(0x7f7e3809d830 msgr2=0x7f7e38068d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:00.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:00.162+0000 7f7e3d2dc640 1 -- 172.21.15.112:0/2561816767 shutdown_connections 2024-09-21T00:27:00.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:00.162+0000 7f7e3d2dc640 1 -- 172.21.15.112:0/2561816767 wait complete. 2024-09-21T00:27:00.174 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_sessionmap.TestSessionMap.test_session_evict_blocklisted' 2024-09-21T00:27:00.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:00.288+0000 7f10dedf1640 1 -- 172.21.15.112:0/1147999272 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10c0004800 msgr2=0x7f10c0004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:00.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:00.288+0000 7f10dedf1640 1 --2- 172.21.15.112:0/1147999272 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10c0004800 0x7f10c0004c10 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f10c80016c0 tx=0x7f10c0005830 comp rx=0 tx=0).stop 2024-09-21T00:27:00.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:00.288+0000 7f10dedf1640 1 -- 172.21.15.112:0/1147999272 shutdown_connections 2024-09-21T00:27:00.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:00.288+0000 7f10dedf1640 1 --2- 172.21.15.112:0/1147999272 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10c0004800 0x7f10c0004c10 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:00.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:00.288+0000 7f10dedf1640 1 -- 172.21.15.112:0/1147999272 >> 172.21.15.112:0/1147999272 conn(0x7f10d80cef40 msgr2=0x7f10d80cf350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:00.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:00.288+0000 7f10dedf1640 1 -- 172.21.15.112:0/1147999272 shutdown_connections 2024-09-21T00:27:00.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:00.288+0000 7f10dedf1640 1 -- 172.21.15.112:0/1147999272 wait complete. 2024-09-21T00:27:00.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:00.290+0000 7f10dedf1640 1 Processor -- start 2024-09-21T00:27:00.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:00.290+0000 7f10dedf1640 1 -- start start 2024-09-21T00:27:00.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:00.291+0000 7f10dedf1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10c0004800 0x7f10d814b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:00.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:00.291+0000 7f10dedf1640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f10d81552a0 0x7f10d81e71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:00.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:00.291+0000 7f10dedf1640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f10d81556b0 0x7f10d81e76f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:00.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:00.291+0000 7f10dedf1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f10d80d1b50 con 0x7f10c0004800 2024-09-21T00:27:00.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:00.291+0000 7f10dedf1640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f10d80d19d0 con 0x7f10d81556b0 2024-09-21T00:27:00.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:00.291+0000 7f10dedf1640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f10d80d1cd0 con 0x7f10d81552a0 2024-09-21T00:27:00.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:00.291+0000 7f10dcb8d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10c0004800 0x7f10d814b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:00.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:00.291+0000 7f10dcb8d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10c0004800 0x7f10d814b920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40830/0 (socket says 172.21.15.112:40830) 2024-09-21T00:27:00.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:00.291+0000 7f10dcb8d640 1 -- 172.21.15.112:0/2669688035 learned_addr learned my addr 172.21.15.112:0/2669688035 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:27:00.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:00.292+0000 7f10cffff640 1 --2- 172.21.15.112:0/2669688035 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f10d81552a0 0x7f10d81e71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:00.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:00.292+0000 7f10dd38e640 1 --2- 172.21.15.112:0/2669688035 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f10d81556b0 0x7f10d81e76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:00.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:00.292+0000 7f10dcb8d640 1 -- 172.21.15.112:0/2669688035 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f10d81552a0 msgr2=0x7f10d81e71b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:00.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:00.292+0000 7f10dcb8d640 1 --2- 172.21.15.112:0/2669688035 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f10d81552a0 0x7f10d81e71b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:00.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:00.292+0000 7f10dcb8d640 1 -- 172.21.15.112:0/2669688035 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f10d81556b0 msgr2=0x7f10d81e76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:00.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:00.292+0000 7f10dcb8d640 1 --2- 172.21.15.112:0/2669688035 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f10d81556b0 0x7f10d81e76f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:00.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:00.292+0000 7f10dcb8d640 1 -- 172.21.15.112:0/2669688035 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10d806fed0 con 0x7f10c0004800 2024-09-21T00:27:00.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:00.292+0000 7f10dd38e640 1 --2- 172.21.15.112:0/2669688035 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f10d81556b0 0x7f10d81e76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:27:00.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:00.292+0000 7f10dcb8d640 1 --2- 172.21.15.112:0/2669688035 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10c0004800 0x7f10d814b920 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f10d4000fb0 tx=0x7f10d40039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:00.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:00.292+0000 7f10cd7fa640 1 -- 172.21.15.112:0/2669688035 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d4015070 con 0x7f10c0004800 2024-09-21T00:27:00.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:00.292+0000 7f10cd7fa640 1 -- 172.21.15.112:0/2669688035 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f10d4011920 con 0x7f10c0004800 2024-09-21T00:27:00.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:00.292+0000 7f10dedf1640 1 -- 172.21.15.112:0/2669688035 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10d8155e10 con 0x7f10c0004800 2024-09-21T00:27:00.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:00.292+0000 7f10cd7fa640 1 -- 172.21.15.112:0/2669688035 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d4011d70 con 0x7f10c0004800 2024-09-21T00:27:00.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:00.292+0000 7f10dedf1640 1 -- 172.21.15.112:0/2669688035 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10d806e1d0 con 0x7f10c0004800 2024-09-21T00:27:00.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:00.293+0000 7f10dedf1640 1 -- 172.21.15.112:0/2669688035 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10d80cccb0 con 0x7f10c0004800 2024-09-21T00:27:00.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:00.294+0000 7f10cd7fa640 1 -- 172.21.15.112:0/2669688035 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f10d4018070 con 0x7f10c0004800 2024-09-21T00:27:00.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:00.294+0000 7f10cd7fa640 1 --2- 172.21.15.112:0/2669688035 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f10c4078bc0 0x7f10c407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:00.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:00.295+0000 7f10cd7fa640 1 -- 172.21.15.112:0/2669688035 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 5335+0+0 (secure 0 0 0) 0x7f10d409b8b0 con 0x7f10c0004800 2024-09-21T00:27:00.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:00.295+0000 7f10cffff640 1 --2- 172.21.15.112:0/2669688035 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f10c4078bc0 0x7f10c407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:00.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:00.295+0000 7f10cffff640 1 --2- 172.21.15.112:0/2669688035 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f10c4078bc0 0x7f10c407b0d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f10c8032000 tx=0x7f10c8001190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:00.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:00.299+0000 7f10cd7fa640 1 -- 172.21.15.112:0/2669688035 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f10d4063ef0 con 0x7f10c0004800 2024-09-21T00:27:00.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:00.544+0000 7f10dedf1640 1 -- 172.21.15.112:0/2669688035 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_sessionmap.TestSessionMap.test_session_evict_blocklisted"]} v 0) -- 0x7f10d80b5ed0 con 0x7f10c0004800 2024-09-21T00:27:01.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:01.527+0000 7f10cd7fa640 1 -- 172.21.15.112:0/2669688035 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_sessionmap.TestSessionMap.test_session_evict_blocklisted"]}]=0 v93) ==== 156+0+0 (secure 0 0 0) 0x7f10d40637f0 con 0x7f10c0004800 2024-09-21T00:27:01.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:01.531+0000 7f10dedf1640 1 -- 172.21.15.112:0/2669688035 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f10c4078bc0 msgr2=0x7f10c407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:01.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:01.531+0000 7f10dedf1640 1 --2- 172.21.15.112:0/2669688035 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f10c4078bc0 0x7f10c407b0d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f10c8032000 tx=0x7f10c8001190 comp rx=0 tx=0).stop 2024-09-21T00:27:01.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:01.532+0000 7f10dedf1640 1 -- 172.21.15.112:0/2669688035 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10c0004800 msgr2=0x7f10d814b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:01.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:01.532+0000 7f10dedf1640 1 --2- 172.21.15.112:0/2669688035 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10c0004800 0x7f10d814b920 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f10d4000fb0 tx=0x7f10d40039c0 comp rx=0 tx=0).stop 2024-09-21T00:27:01.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:01.532+0000 7f10dedf1640 1 -- 172.21.15.112:0/2669688035 shutdown_connections 2024-09-21T00:27:01.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:01.532+0000 7f10dedf1640 1 --2- 172.21.15.112:0/2669688035 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f10d81556b0 0x7f10d81e76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:01.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:01.532+0000 7f10dedf1640 1 --2- 172.21.15.112:0/2669688035 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f10d81552a0 0x7f10d81e71b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:01.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:01.532+0000 7f10dedf1640 1 --2- 172.21.15.112:0/2669688035 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f10c4078bc0 0x7f10c407b0d0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:01.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:01.532+0000 7f10dedf1640 1 --2- 172.21.15.112:0/2669688035 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10c0004800 0x7f10d814b920 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:01.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:01.532+0000 7f10dedf1640 1 -- 172.21.15.112:0/2669688035 >> 172.21.15.112:0/2669688035 conn(0x7f10d80cef40 msgr2=0x7f10d80cbcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:01.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:01.532+0000 7f10dedf1640 1 -- 172.21.15.112:0/2669688035 shutdown_connections 2024-09-21T00:27:01.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:01.532+0000 7f10dedf1640 1 -- 172.21.15.112:0/2669688035 wait complete. 2024-09-21T00:27:01.547 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-21T00:27:01.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:01.665+0000 7f73e1fc3640 1 -- 172.21.15.112:0/3284250771 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73c4004800 msgr2=0x7f73c4004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:01.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:01.665+0000 7f73e1fc3640 1 --2- 172.21.15.112:0/3284250771 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73c4004800 0x7f73c4004c10 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f73dc0cd6d0 tx=0x7f73c4005830 comp rx=0 tx=0).stop 2024-09-21T00:27:01.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:01.665+0000 7f73dbfff640 1 -- 172.21.15.112:0/3284250771 reap_dead start 2024-09-21T00:27:01.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:01.665+0000 7f73e1fc3640 1 -- 172.21.15.112:0/3284250771 shutdown_connections 2024-09-21T00:27:01.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:01.665+0000 7f73e1fc3640 1 -- 172.21.15.112:0/3284250771 >> 172.21.15.112:0/3284250771 conn(0x7f73dc0cef40 msgr2=0x7f73dc0cf350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:01.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:01.665+0000 7f73e1fc3640 1 -- 172.21.15.112:0/3284250771 shutdown_connections 2024-09-21T00:27:01.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:01.665+0000 7f73e1fc3640 1 -- 172.21.15.112:0/3284250771 wait complete. 2024-09-21T00:27:01.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:01.667+0000 7f73e1fc3640 1 Processor -- start 2024-09-21T00:27:01.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:01.667+0000 7f73e1fc3640 1 -- start start 2024-09-21T00:27:01.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:01.668+0000 7f73e1fc3640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f73c4004800 0x7f73dc1f64c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:01.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:01.668+0000 7f73e1fc3640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f73dc0c67a0 0x7f73dc0c6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:01.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:01.668+0000 7f73e1fc3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73dc15b5e0 0x7f73dc15b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:01.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:01.668+0000 7f73e1fc3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f73dc0cccc0 con 0x7f73dc15b5e0 2024-09-21T00:27:01.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:01.668+0000 7f73e1fc3640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f73dc0ccb40 con 0x7f73c4004800 2024-09-21T00:27:01.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:01.668+0000 7f73e1fc3640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f73dc0d0bd0 con 0x7f73dc0c67a0 2024-09-21T00:27:01.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:01.668+0000 7f73dbfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73dc15b5e0 0x7f73dc15b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:01.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:01.668+0000 7f73dbfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73dc15b5e0 0x7f73dc15b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40848/0 (socket says 172.21.15.112:40848) 2024-09-21T00:27:01.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:01.668+0000 7f73dbfff640 1 -- 172.21.15.112:0/1506468450 learned_addr learned my addr 172.21.15.112:0/1506468450 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:27:01.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:01.668+0000 7f73daffd640 1 --2- 172.21.15.112:0/1506468450 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f73dc0c67a0 0x7f73dc0c6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:01.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:01.668+0000 7f73db7fe640 1 --2- 172.21.15.112:0/1506468450 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f73c4004800 0x7f73dc1f64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:01.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:01.669+0000 7f73dbfff640 1 -- 172.21.15.112:0/1506468450 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f73dc0c67a0 msgr2=0x7f73dc0c6bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:01.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:01.669+0000 7f73dbfff640 1 --2- 172.21.15.112:0/1506468450 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f73dc0c67a0 0x7f73dc0c6bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:01.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:01.669+0000 7f73dbfff640 1 -- 172.21.15.112:0/1506468450 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f73c4004800 msgr2=0x7f73dc1f64c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:01.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:01.669+0000 7f73dbfff640 1 --2- 172.21.15.112:0/1506468450 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f73c4004800 0x7f73dc1f64c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:01.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:01.669+0000 7f73dbfff640 1 -- 172.21.15.112:0/1506468450 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73dc15bf30 con 0x7f73dc15b5e0 2024-09-21T00:27:01.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:01.669+0000 7f73dbfff640 1 --2- 172.21.15.112:0/1506468450 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73dc15b5e0 0x7f73dc15b9f0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f73dc15df60 tx=0x7f73d4006430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:01.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:01.669+0000 7f73c3fff640 1 -- 172.21.15.112:0/1506468450 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d4011840 con 0x7f73dc15b5e0 2024-09-21T00:27:01.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:01.669+0000 7f73c3fff640 1 -- 172.21.15.112:0/1506468450 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f73d4018070 con 0x7f73dc15b5e0 2024-09-21T00:27:01.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:01.669+0000 7f73c3fff640 1 -- 172.21.15.112:0/1506468450 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d4011c80 con 0x7f73dc15b5e0 2024-09-21T00:27:01.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:01.669+0000 7f73e1fc3640 1 -- 172.21.15.112:0/1506468450 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73dc0c3320 con 0x7f73dc15b5e0 2024-09-21T00:27:01.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:01.669+0000 7f73e1fc3640 1 -- 172.21.15.112:0/1506468450 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73dc0c38c0 con 0x7f73dc15b5e0 2024-09-21T00:27:01.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:01.670+0000 7f73e1fc3640 1 -- 172.21.15.112:0/1506468450 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73dc0d10c0 con 0x7f73dc15b5e0 2024-09-21T00:27:01.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:01.671+0000 7f73c3fff640 1 -- 172.21.15.112:0/1506468450 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f73d401d460 con 0x7f73dc15b5e0 2024-09-21T00:27:01.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:01.671+0000 7f73c3fff640 1 --2- 172.21.15.112:0/1506468450 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f73c8078bc0 0x7f73c807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:01.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:01.671+0000 7f73c3fff640 1 -- 172.21.15.112:0/1506468450 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 5335+0+0 (secure 0 0 0) 0x7f73d409b340 con 0x7f73dc15b5e0 2024-09-21T00:27:01.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:01.672+0000 7f73db7fe640 1 --2- 172.21.15.112:0/1506468450 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f73c8078bc0 0x7f73c807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:01.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:01.672+0000 7f73db7fe640 1 --2- 172.21.15.112:0/1506468450 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f73c8078bc0 0x7f73c807b0d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f73cc001eb0 tx=0x7f73cc006970 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:01.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:01.676+0000 7f73c3fff640 1 -- 172.21.15.112:0/1506468450 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f73dc15bf30 con 0x7f73dc15b5e0 2024-09-21T00:27:01.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:01.850+0000 7f73e1fc3640 1 -- 172.21.15.112:0/1506468450 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f73dc0b5110 con 0x7f73dc15b5e0 2024-09-21T00:27:01.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:01.851+0000 7f73c3fff640 1 -- 172.21.15.112:0/1506468450 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v34) ==== 59+0+2294 (secure 0 0 0) 0x7f73d4011e20 con 0x7f73dc15b5e0 2024-09-21T00:27:01.853 INFO:teuthology.orchestra.run.smithi112.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-21T00:27:01.853 INFO:teuthology.orchestra.run.smithi112.stdout:mon advanced mon_op_complaint_time 120 2024-09-21T00:27:01.853 INFO:teuthology.orchestra.run.smithi112.stdout:mgr advanced client_mount_timeout 30 2024-09-21T00:27:01.853 INFO:teuthology.orchestra.run.smithi112.stdout:mgr advanced debug_client 20/20 2024-09-21T00:27:01.853 INFO:teuthology.orchestra.run.smithi112.stdout:mgr advanced debug_mgr 20/20 2024-09-21T00:27:01.853 INFO:teuthology.orchestra.run.smithi112.stdout:mgr advanced debug_ms 1/1 2024-09-21T00:27:01.853 INFO:teuthology.orchestra.run.smithi112.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-21T00:27:01.853 INFO:teuthology.orchestra.run.smithi112.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-21T00:27:01.853 INFO:teuthology.orchestra.run.smithi112.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 5090.401743 2024-09-21T00:27:01.853 INFO:teuthology.orchestra.run.smithi112.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 7876.743127 2024-09-21T00:27:01.854 INFO:teuthology.orchestra.run.smithi112.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 4865.073913 2024-09-21T00:27:01.854 INFO:teuthology.orchestra.run.smithi112.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 5117.206240 2024-09-21T00:27:01.854 INFO:teuthology.orchestra.run.smithi112.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 4329.674604 2024-09-21T00:27:01.854 INFO:teuthology.orchestra.run.smithi112.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4322.652151 2024-09-21T00:27:01.854 INFO:teuthology.orchestra.run.smithi112.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 4244.674337 2024-09-21T00:27:01.854 INFO:teuthology.orchestra.run.smithi112.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 4472.316624 2024-09-21T00:27:01.854 INFO:teuthology.orchestra.run.smithi112.stdout:mds advanced debug_mds 20/20 2024-09-21T00:27:01.854 INFO:teuthology.orchestra.run.smithi112.stdout:mds advanced debug_mds_balancer 20/20 2024-09-21T00:27:01.854 INFO:teuthology.orchestra.run.smithi112.stdout:mds advanced debug_ms 2/2 2024-09-21T00:27:01.854 INFO:teuthology.orchestra.run.smithi112.stdout:mds dev mds_debug_frag true 2024-09-21T00:27:01.854 INFO:teuthology.orchestra.run.smithi112.stdout:mds dev mds_debug_scatterstat true 2024-09-21T00:27:01.854 INFO:teuthology.orchestra.run.smithi112.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-21T00:27:01.855 INFO:teuthology.orchestra.run.smithi112.stdout:mds dev mds_verify_scatter true 2024-09-21T00:27:01.855 INFO:teuthology.orchestra.run.smithi112.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-21T00:27:01.855 INFO:teuthology.orchestra.run.smithi112.stdout:mds advanced rados_mon_op_timeout 900 2024-09-21T00:27:01.855 INFO:teuthology.orchestra.run.smithi112.stdout:mds advanced rados_osd_op_timeout 900 2024-09-21T00:27:01.855 INFO:teuthology.orchestra.run.smithi112.stdout:client advanced client_mount_timeout 600 2024-09-21T00:27:01.855 INFO:teuthology.orchestra.run.smithi112.stdout:client advanced debug_client 20/20 2024-09-21T00:27:01.855 INFO:teuthology.orchestra.run.smithi112.stdout:client advanced debug_ms 1/1 2024-09-21T00:27:01.855 INFO:teuthology.orchestra.run.smithi112.stdout:client advanced rados_mon_op_timeout 900 2024-09-21T00:27:01.855 INFO:teuthology.orchestra.run.smithi112.stdout:client advanced rados_osd_op_timeout 900 2024-09-21T00:27:01.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:01.853+0000 7f73e1fc3640 1 -- 172.21.15.112:0/1506468450 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f73c8078bc0 msgr2=0x7f73c807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:01.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:01.853+0000 7f73e1fc3640 1 --2- 172.21.15.112:0/1506468450 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f73c8078bc0 0x7f73c807b0d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f73cc001eb0 tx=0x7f73cc006970 comp rx=0 tx=0).stop 2024-09-21T00:27:01.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:01.853+0000 7f73e1fc3640 1 -- 172.21.15.112:0/1506468450 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73dc15b5e0 msgr2=0x7f73dc15b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:01.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:01.853+0000 7f73e1fc3640 1 --2- 172.21.15.112:0/1506468450 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73dc15b5e0 0x7f73dc15b9f0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f73dc15df60 tx=0x7f73d4006430 comp rx=0 tx=0).stop 2024-09-21T00:27:01.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:01.853+0000 7f73e1fc3640 1 -- 172.21.15.112:0/1506468450 shutdown_connections 2024-09-21T00:27:01.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:01.853+0000 7f73e1fc3640 1 --2- 172.21.15.112:0/1506468450 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f73c8078bc0 0x7f73c807b0d0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:01.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:01.853+0000 7f73e1fc3640 1 --2- 172.21.15.112:0/1506468450 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73dc15b5e0 0x7f73dc15b9f0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:01.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:01.853+0000 7f73e1fc3640 1 --2- 172.21.15.112:0/1506468450 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f73dc0c67a0 0x7f73dc0c6bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:01.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:01.853+0000 7f73e1fc3640 1 --2- 172.21.15.112:0/1506468450 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f73c4004800 0x7f73dc1f64c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:01.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:01.853+0000 7f73e1fc3640 1 -- 172.21.15.112:0/1506468450 >> 172.21.15.112:0/1506468450 conn(0x7f73dc0cef40 msgr2=0x7f73dc0d12d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:01.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:01.853+0000 7f73e1fc3640 1 -- 172.21.15.112:0/1506468450 shutdown_connections 2024-09-21T00:27:01.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:01.853+0000 7f73e1fc3640 1 -- 172.21.15.112:0/1506468450 wait complete. 2024-09-21T00:27:01.866 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-21T00:27:01.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:01.980+0000 7f2f01c1d640 1 -- 172.21.15.112:0/1011431631 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ee4004d40 msgr2=0x7f2ee40051f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:01.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:01.980+0000 7f2f01c1d640 1 --2- 172.21.15.112:0/1011431631 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ee4004d40 0x7f2ee40051f0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f2efc0b1d70 tx=0x7f2ee4005830 comp rx=0 tx=0).stop 2024-09-21T00:27:01.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:01.980+0000 7f2efbfff640 1 -- 172.21.15.112:0/1011431631 reap_dead start 2024-09-21T00:27:01.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:01.981+0000 7f2f01c1d640 1 -- 172.21.15.112:0/1011431631 shutdown_connections 2024-09-21T00:27:01.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:01.981+0000 7f2f01c1d640 1 -- 172.21.15.112:0/1011431631 >> 172.21.15.112:0/1011431631 conn(0x7f2efc0b46e0 msgr2=0x7f2efc0d23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:01.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:01.981+0000 7f2f01c1d640 1 -- 172.21.15.112:0/1011431631 shutdown_connections 2024-09-21T00:27:01.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:01.981+0000 7f2f01c1d640 1 -- 172.21.15.112:0/1011431631 wait complete. 2024-09-21T00:27:01.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:01.982+0000 7f2f01c1d640 1 Processor -- start 2024-09-21T00:27:01.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:01.983+0000 7f2f01c1d640 1 -- start start 2024-09-21T00:27:01.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:01.984+0000 7f2f01c1d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2efc15a6e0 0x7f2efc0b1220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:01.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:01.984+0000 7f2f01c1d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2efc159eb0 0x7f2efc1f5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:01.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:01.984+0000 7f2f01c1d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2efc1f5e90 0x7f2efc071100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:01.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:01.984+0000 7f2f01c1d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2efc0cdb60 con 0x7f2efc1f5e90 2024-09-21T00:27:01.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:01.984+0000 7f2f01c1d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f2efc15ee50 con 0x7f2efc15a6e0 2024-09-21T00:27:01.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:01.984+0000 7f2f01c1d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f2efc0cdce0 con 0x7f2efc159eb0 2024-09-21T00:27:01.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:01.984+0000 7f2efbfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2efc1f5e90 0x7f2efc071100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:01.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:01.984+0000 7f2efbfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2efc1f5e90 0x7f2efc071100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40856/0 (socket says 172.21.15.112:40856) 2024-09-21T00:27:01.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:01.984+0000 7f2efbfff640 1 -- 172.21.15.112:0/3954652231 learned_addr learned my addr 172.21.15.112:0/3954652231 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:27:01.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:01.984+0000 7f2efb7fe640 1 --2- 172.21.15.112:0/3954652231 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2efc15a6e0 0x7f2efc0b1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:01.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:01.984+0000 7f2efaffd640 1 --2- 172.21.15.112:0/3954652231 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2efc159eb0 0x7f2efc1f5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:01.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:01.984+0000 7f2efbfff640 1 -- 172.21.15.112:0/3954652231 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2efc159eb0 msgr2=0x7f2efc1f5950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:01.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:01.984+0000 7f2efbfff640 1 --2- 172.21.15.112:0/3954652231 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2efc159eb0 0x7f2efc1f5950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:01.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:01.984+0000 7f2efbfff640 1 -- 172.21.15.112:0/3954652231 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2efc15a6e0 msgr2=0x7f2efc0b1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:01.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:01.984+0000 7f2efbfff640 1 --2- 172.21.15.112:0/3954652231 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2efc15a6e0 0x7f2efc0b1220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:01.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:01.984+0000 7f2efbfff640 1 -- 172.21.15.112:0/3954652231 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2efc06f020 con 0x7f2efc1f5e90 2024-09-21T00:27:01.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:01.984+0000 7f2efbfff640 1 --2- 172.21.15.112:0/3954652231 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2efc1f5e90 0x7f2efc071100 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f2ef400c700 tx=0x7f2ef4003c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:01.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:01.985+0000 7f2ee3fff640 1 -- 172.21.15.112:0/3954652231 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef4039070 con 0x7f2efc1f5e90 2024-09-21T00:27:01.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:01.985+0000 7f2ee3fff640 1 -- 172.21.15.112:0/3954652231 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2ef402ba50 con 0x7f2efc1f5e90 2024-09-21T00:27:01.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:01.985+0000 7f2ee3fff640 1 -- 172.21.15.112:0/3954652231 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef4002030 con 0x7f2efc1f5e90 2024-09-21T00:27:01.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:01.985+0000 7f2f01c1d640 1 -- 172.21.15.112:0/3954652231 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2efc06f300 con 0x7f2efc1f5e90 2024-09-21T00:27:01.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:01.985+0000 7f2f01c1d640 1 -- 172.21.15.112:0/3954652231 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2efc06f8c0 con 0x7f2efc1f5e90 2024-09-21T00:27:01.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:01.986+0000 7f2f01c1d640 1 -- 172.21.15.112:0/3954652231 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2efc0cea30 con 0x7f2efc1f5e90 2024-09-21T00:27:01.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:01.987+0000 7f2ee3fff640 1 -- 172.21.15.112:0/3954652231 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2ef403c020 con 0x7f2efc1f5e90 2024-09-21T00:27:01.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:01.987+0000 7f2ee3fff640 1 --2- 172.21.15.112:0/3954652231 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2ee8078b90 0x7f2ee807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:01.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:01.987+0000 7f2ee3fff640 1 -- 172.21.15.112:0/3954652231 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 5335+0+0 (secure 0 0 0) 0x7f2ef40ba090 con 0x7f2efc1f5e90 2024-09-21T00:27:01.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:01.987+0000 7f2efb7fe640 1 --2- 172.21.15.112:0/3954652231 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2ee8078b90 0x7f2ee807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:01.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:01.988+0000 7f2efb7fe640 1 --2- 172.21.15.112:0/3954652231 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2ee8078b90 0x7f2ee807b0a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2efc0c5930 tx=0x7f2eec005f90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:01.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:01.992+0000 7f2ee3fff640 1 -- 172.21.15.112:0/3954652231 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2ef4045030 con 0x7f2efc1f5e90 2024-09-21T00:27:02.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:02.169+0000 7f2f01c1d640 1 -- 172.21.15.112:0/3954652231 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f2efc15b3b0 con 0x7f2efc1f5e90 2024-09-21T00:27:02.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:02.173+0000 7f2ee3fff640 1 -- 172.21.15.112:0/3954652231 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v35)=0 v35) ==== 122+0+0 (secure 0 0 0) 0x7f2ef402be20 con 0x7f2efc1f5e90 2024-09-21T00:27:02.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:02.175+0000 7f2f01c1d640 1 -- 172.21.15.112:0/3954652231 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2ee8078b90 msgr2=0x7f2ee807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:02.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:02.175+0000 7f2f01c1d640 1 --2- 172.21.15.112:0/3954652231 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2ee8078b90 0x7f2ee807b0a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2efc0c5930 tx=0x7f2eec005f90 comp rx=0 tx=0).stop 2024-09-21T00:27:02.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:02.175+0000 7f2f01c1d640 1 -- 172.21.15.112:0/3954652231 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2efc1f5e90 msgr2=0x7f2efc071100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:02.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:02.175+0000 7f2f01c1d640 1 --2- 172.21.15.112:0/3954652231 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2efc1f5e90 0x7f2efc071100 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f2ef400c700 tx=0x7f2ef4003c60 comp rx=0 tx=0).stop 2024-09-21T00:27:02.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:02.175+0000 7f2f01c1d640 1 -- 172.21.15.112:0/3954652231 shutdown_connections 2024-09-21T00:27:02.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:02.175+0000 7f2f01c1d640 1 --2- 172.21.15.112:0/3954652231 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2ee8078b90 0x7f2ee807b0a0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:02.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:02.175+0000 7f2f01c1d640 1 --2- 172.21.15.112:0/3954652231 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2efc1f5e90 0x7f2efc071100 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:02.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:02.175+0000 7f2f01c1d640 1 --2- 172.21.15.112:0/3954652231 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2efc159eb0 0x7f2efc1f5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:02.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:02.175+0000 7f2f01c1d640 1 --2- 172.21.15.112:0/3954652231 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2efc15a6e0 0x7f2efc0b1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:02.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:02.175+0000 7f2f01c1d640 1 -- 172.21.15.112:0/3954652231 >> 172.21.15.112:0/3954652231 conn(0x7f2efc0b46e0 msgr2=0x7f2efc068d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:02.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:02.175+0000 7f2f01c1d640 1 -- 172.21.15.112:0/3954652231 shutdown_connections 2024-09-21T00:27:02.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:02.175+0000 7f2f01c1d640 1 -- 172.21.15.112:0/3954652231 wait complete. 2024-09-21T00:27:02.188 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T00:27:02.188 DEBUG:teuthology.orchestra.run.smithi112:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T00:27:02.235 DEBUG:teuthology.orchestra.run.smithi187:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T00:27:02.277 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T00:27:02.277 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T00:27:02.277 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:27:02.293 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056010,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1439 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248716k,nr_inodes=812179,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T00:27:02.293 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-09-21T00:27:02.293 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:27:02.336 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056013,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497440k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16835 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248720k,nr_inodes=812180,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497440k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T00:27:02.336 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T00:27:02.336 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:27:02.352 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056010,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1439 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248716k,nr_inodes=812179,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T00:27:02.352 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-09-21T00:27:02.352 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:27:02.394 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056013,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497440k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16835 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248720k,nr_inodes=812180,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497440k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T00:27:02.394 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-21T00:27:02.394 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T00:27:02.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:02.503+0000 7f060d73a640 1 -- 172.21.15.112:0/1871926449 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06080cf930 msgr2=0x7f06080ce320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:02.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:02.503+0000 7f060d73a640 1 --2- 172.21.15.112:0/1871926449 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06080cf930 0x7f06080ce320 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f05f0005fd0 tx=0x7f05f002a5d0 comp rx=0 tx=0).stop 2024-09-21T00:27:02.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:02.503+0000 7f060d73a640 1 -- 172.21.15.112:0/1871926449 shutdown_connections 2024-09-21T00:27:02.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:02.503+0000 7f060d73a640 1 --2- 172.21.15.112:0/1871926449 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f06080ce860 0x7f060815f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:02.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:02.503+0000 7f060d73a640 1 --2- 172.21.15.112:0/1871926449 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06080cf930 0x7f06080ce320 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:02.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:02.503+0000 7f060d73a640 1 -- 172.21.15.112:0/1871926449 >> 172.21.15.112:0/1871926449 conn(0x7f06080cca40 msgr2=0x7f06080cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:02.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:02.503+0000 7f060d73a640 1 -- 172.21.15.112:0/1871926449 shutdown_connections 2024-09-21T00:27:02.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:02.503+0000 7f060d73a640 1 -- 172.21.15.112:0/1871926449 wait complete. 2024-09-21T00:27:02.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:02.505+0000 7f060d73a640 1 Processor -- start 2024-09-21T00:27:02.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:02.505+0000 7f060d73a640 1 -- start start 2024-09-21T00:27:02.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:02.506+0000 7f060d73a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f06080ce860 0x7f06080c2960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:02.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:02.506+0000 7f060d73a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f06080cf930 0x7f06080c5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:02.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:02.506+0000 7f060d73a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f060815b9f0 0x7f06080c6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:02.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:02.506+0000 7f060d73a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f060806df70 con 0x7f060815b9f0 2024-09-21T00:27:02.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:02.506+0000 7f060d73a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f060806ddf0 con 0x7f06080ce860 2024-09-21T00:27:02.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:02.506+0000 7f060d73a640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f060806e0f0 con 0x7f06080cf930 2024-09-21T00:27:02.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:02.507+0000 7f06077fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f060815b9f0 0x7f06080c6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:02.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:02.507+0000 7f06077fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f060815b9f0 0x7f06080c6220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40876/0 (socket says 172.21.15.112:40876) 2024-09-21T00:27:02.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:02.507+0000 7f06077fe640 1 -- 172.21.15.112:0/3863710159 learned_addr learned my addr 172.21.15.112:0/3863710159 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:27:02.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:02.507+0000 7f05fe7fc640 1 --2- 172.21.15.112:0/3863710159 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f06080cf930 0x7f06080c5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:02.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:02.507+0000 7f0606ffd640 1 --2- 172.21.15.112:0/3863710159 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f06080ce860 0x7f06080c2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:02.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:02.507+0000 7f06077fe640 1 -- 172.21.15.112:0/3863710159 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f06080cf930 msgr2=0x7f06080c5ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:02.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:02.507+0000 7f06077fe640 1 --2- 172.21.15.112:0/3863710159 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f06080cf930 0x7f06080c5ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:02.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:02.507+0000 7f06077fe640 1 -- 172.21.15.112:0/3863710159 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f06080ce860 msgr2=0x7f06080c2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:02.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:02.507+0000 7f06077fe640 1 --2- 172.21.15.112:0/3863710159 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f06080ce860 0x7f06080c2960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:02.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:02.507+0000 7f06077fe640 1 -- 172.21.15.112:0/3863710159 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06080b5d00 con 0x7f060815b9f0 2024-09-21T00:27:02.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:02.507+0000 7f06077fe640 1 --2- 172.21.15.112:0/3863710159 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f060815b9f0 0x7f06080c6220 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f060000e990 tx=0x7f060000ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:02.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:02.507+0000 7f05fffff640 1 -- 172.21.15.112:0/3863710159 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0600004d20 con 0x7f060815b9f0 2024-09-21T00:27:02.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:02.507+0000 7f05fffff640 1 -- 172.21.15.112:0/3863710159 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f060000cb00 con 0x7f060815b9f0 2024-09-21T00:27:02.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:02.507+0000 7f060d73a640 1 -- 172.21.15.112:0/3863710159 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06080b5a20 con 0x7f060815b9f0 2024-09-21T00:27:02.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:02.507+0000 7f05fffff640 1 -- 172.21.15.112:0/3863710159 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0600005020 con 0x7f060815b9f0 2024-09-21T00:27:02.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:02.507+0000 7f060d73a640 1 -- 172.21.15.112:0/3863710159 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06081f7650 con 0x7f060815b9f0 2024-09-21T00:27:02.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:02.508+0000 7f060d73a640 1 -- 172.21.15.112:0/3863710159 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f060806e610 con 0x7f060815b9f0 2024-09-21T00:27:02.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:02.509+0000 7f05fffff640 1 -- 172.21.15.112:0/3863710159 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f06000051c0 con 0x7f060815b9f0 2024-09-21T00:27:02.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:02.510+0000 7f05fffff640 1 --2- 172.21.15.112:0/3863710159 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f05e8078b90 0x7f05e807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:02.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:02.510+0000 7f05fffff640 1 -- 172.21.15.112:0/3863710159 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 5335+0+0 (secure 0 0 0) 0x7f060009b390 con 0x7f060815b9f0 2024-09-21T00:27:02.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:02.510+0000 7f0606ffd640 1 --2- 172.21.15.112:0/3863710159 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f05e8078b90 0x7f05e807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:02.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:02.510+0000 7f0606ffd640 1 --2- 172.21.15.112:0/3863710159 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f05e8078b90 0x7f05e807b0a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f06080ca5c0 tx=0x7f05f8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:02.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:02.514+0000 7f05fffff640 1 -- 172.21.15.112:0/3863710159 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f0600016030 con 0x7f060815b9f0 2024-09-21T00:27:02.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:02.730+0000 7f060d73a640 1 -- 172.21.15.112:0/3863710159 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f06080b57b0 con 0x7f060815b9f0 2024-09-21T00:27:02.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:02.731+0000 7f05fffff640 1 -- 172.21.15.112:0/3863710159 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+3920 (secure 0 0 0) 0x7f0600063380 con 0x7f060815b9f0 2024-09-21T00:27:02.732 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:27:02.732 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":19,"btime":"2024-09-21T00:26:56:501432+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4846,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6834/2036740840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":2036740840},{"type":"v1","addr":"172.21.15.112:6834","nonce":2036740840}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4855,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6836/3699399640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":3699399640},{"type":"v1","addr":"172.21.15.187:6836","nonce":3699399640}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4858,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6837/694936803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6835","nonce":694936803},{"type":"v1","addr":"172.21.15.187:6837","nonce":694936803}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5322,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/1123974079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":1123974079},{"type":"v1","addr":"172.21.15.112:6835","nonce":1123974079}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[]} 2024-09-21T00:27:02.733 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 19 2024-09-21T00:27:02.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:02.733+0000 7f060d73a640 1 -- 172.21.15.112:0/3863710159 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f05e8078b90 msgr2=0x7f05e807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:02.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:02.733+0000 7f060d73a640 1 --2- 172.21.15.112:0/3863710159 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f05e8078b90 0x7f05e807b0a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f06080ca5c0 tx=0x7f05f8009040 comp rx=0 tx=0).stop 2024-09-21T00:27:02.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:02.733+0000 7f060d73a640 1 -- 172.21.15.112:0/3863710159 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f060815b9f0 msgr2=0x7f06080c6220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:02.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:02.733+0000 7f060d73a640 1 --2- 172.21.15.112:0/3863710159 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f060815b9f0 0x7f06080c6220 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f060000e990 tx=0x7f060000ee50 comp rx=0 tx=0).stop 2024-09-21T00:27:02.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:02.733+0000 7f060d73a640 1 -- 172.21.15.112:0/3863710159 shutdown_connections 2024-09-21T00:27:02.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:02.733+0000 7f060d73a640 1 --2- 172.21.15.112:0/3863710159 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f05e8078b90 0x7f05e807b0a0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:02.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:02.733+0000 7f060d73a640 1 --2- 172.21.15.112:0/3863710159 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f060815b9f0 0x7f06080c6220 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:02.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:02.733+0000 7f060d73a640 1 --2- 172.21.15.112:0/3863710159 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f06080cf930 0x7f06080c5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:02.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:02.733+0000 7f060d73a640 1 --2- 172.21.15.112:0/3863710159 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f06080ce860 0x7f06080c2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:02.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:02.733+0000 7f060d73a640 1 -- 172.21.15.112:0/3863710159 >> 172.21.15.112:0/3863710159 conn(0x7f06080cca40 msgr2=0x7f060806fd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:02.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:02.733+0000 7f060d73a640 1 -- 172.21.15.112:0/3863710159 shutdown_connections 2024-09-21T00:27:02.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:02.733+0000 7f060d73a640 1 -- 172.21.15.112:0/3863710159 wait complete. 2024-09-21T00:27:02.746 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-21T00:27:02.746 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-21T00:27:02.746 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-21T00:27:02.746 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T00:27:02.746 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-21T00:27:02.746 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-21T00:27:02.747 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-21T00:27:02.747 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T00:27:02.747 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-21T00:27:02.747 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-21T00:27:02.747 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-21T00:27:02.747 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T00:27:02.747 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-21T00:27:02.747 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-21T00:27:02.748 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-21T00:27:02.748 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T00:27:02.756 INFO:tasks.ceph.mds.c:Stopped 2024-09-21T00:27:02.757 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-21T00:27:02.758 INFO:tasks.ceph.mds.a:Stopped 2024-09-21T00:27:02.758 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-21T00:27:02.762 INFO:tasks.ceph.mds.d:Stopped 2024-09-21T00:27:02.762 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-21T00:27:02.763 INFO:tasks.ceph.mds.b:Stopped 2024-09-21T00:27:02.763 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-21T00:27:02.800 INFO:tasks.ceph.mds.c:Started 2024-09-21T00:27:02.801 DEBUG:teuthology.parallel:result is None 2024-09-21T00:27:02.802 INFO:tasks.ceph.mds.a:Started 2024-09-21T00:27:02.802 DEBUG:teuthology.parallel:result is None 2024-09-21T00:27:02.805 INFO:tasks.ceph.mds.d:Started 2024-09-21T00:27:02.805 DEBUG:teuthology.parallel:result is None 2024-09-21T00:27:02.805 INFO:tasks.ceph.mds.b:Started 2024-09-21T00:27:02.806 DEBUG:teuthology.parallel:result is None 2024-09-21T00:27:02.806 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-21T00:27:02.887 INFO:tasks.ceph.mds.a.smithi112.stdout:starting mds.a at 2024-09-21T00:27:02.887 INFO:tasks.ceph.mds.c.smithi112.stdout:starting mds.c at 2024-09-21T00:27:02.899 INFO:tasks.ceph.mds.b.smithi187.stdout:starting mds.b at 2024-09-21T00:27:02.899 INFO:tasks.ceph.mds.d.smithi187.stdout:starting mds.d at 2024-09-21T00:27:02.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:02.924+0000 7f311fef2640 1 -- 172.21.15.112:0/1538826790 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f31180b9ad0 msgr2=0x7f31180d0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:02.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:02.924+0000 7f311fef2640 1 --2- 172.21.15.112:0/1538826790 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f31180b9ad0 0x7f31180d0be0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f31180cfba0 tx=0x7f310c02a4b0 comp rx=0 tx=0).stop 2024-09-21T00:27:02.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:02.925+0000 7f311fef2640 1 -- 172.21.15.112:0/1538826790 shutdown_connections 2024-09-21T00:27:02.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:02.925+0000 7f311fef2640 1 --2- 172.21.15.112:0/1538826790 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f31180d1120 0x7f31180ceb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:02.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:02.925+0000 7f311fef2640 1 --2- 172.21.15.112:0/1538826790 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f31180b9ad0 0x7f31180d0be0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:02.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:02.925+0000 7f311fef2640 1 --2- 172.21.15.112:0/1538826790 >> v2:172.21.15.187:3300/0 conn(0x7f311815ee80 0x7f31180d0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:02.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:02.925+0000 7f311fef2640 1 -- 172.21.15.112:0/1538826790 >> 172.21.15.112:0/1538826790 conn(0x7f311809d830 msgr2=0x7f31180b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:02.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:02.925+0000 7f311fef2640 1 -- 172.21.15.112:0/1538826790 shutdown_connections 2024-09-21T00:27:02.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:02.925+0000 7f311fef2640 1 -- 172.21.15.112:0/1538826790 wait complete. 2024-09-21T00:27:02.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:02.926+0000 7f311fef2640 1 Processor -- start 2024-09-21T00:27:02.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:02.926+0000 7f311fef2640 1 -- start start 2024-09-21T00:27:02.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:02.928+0000 7f311fef2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f31180b9ad0 0x7f311814fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:02.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:02.928+0000 7f311fef2640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f31180d1120 0x7f31180d3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:02.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:02.928+0000 7f311fef2640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f311815ee80 0x7f31180d4130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:02.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:02.928+0000 7f311fef2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f31180cf520 con 0x7f31180b9ad0 2024-09-21T00:27:02.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:02.928+0000 7f311fef2640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f31180cf3a0 con 0x7f311815ee80 2024-09-21T00:27:02.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:02.928+0000 7f311fef2640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f31180b6580 con 0x7f31180d1120 2024-09-21T00:27:02.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:02.928+0000 7f311dc8e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f31180b9ad0 0x7f311814fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:02.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:02.928+0000 7f311dc8e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f31180b9ad0 0x7f311814fa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40912/0 (socket says 172.21.15.112:40912) 2024-09-21T00:27:02.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:02.928+0000 7f311dc8e640 1 -- 172.21.15.112:0/415501092 learned_addr learned my addr 172.21.15.112:0/415501092 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:27:02.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:02.928+0000 7f311e48f640 1 --2- 172.21.15.112:0/415501092 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f311815ee80 0x7f31180d4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:02.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:02.928+0000 7f311d48d640 1 --2- 172.21.15.112:0/415501092 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f31180d1120 0x7f31180d3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:02.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:02.928+0000 7f311dc8e640 1 -- 172.21.15.112:0/415501092 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f31180d1120 msgr2=0x7f31180d3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:02.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:02.928+0000 7f311dc8e640 1 --2- 172.21.15.112:0/415501092 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f31180d1120 0x7f31180d3bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:02.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:02.928+0000 7f311dc8e640 1 -- 172.21.15.112:0/415501092 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f311815ee80 msgr2=0x7f31180d4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:02.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:02.928+0000 7f311dc8e640 1 --2- 172.21.15.112:0/415501092 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f311815ee80 0x7f31180d4130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:02.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:02.928+0000 7f311dc8e640 1 -- 172.21.15.112:0/415501092 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f311806e750 con 0x7f31180b9ad0 2024-09-21T00:27:02.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:02.928+0000 7f311dc8e640 1 --2- 172.21.15.112:0/415501092 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f31180b9ad0 0x7f311814fa20 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f311400d750 tx=0x7f311400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:02.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:02.928+0000 7f310a7fc640 1 -- 172.21.15.112:0/415501092 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3114017070 con 0x7f31180b9ad0 2024-09-21T00:27:02.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:02.929+0000 7f310a7fc640 1 -- 172.21.15.112:0/415501092 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3114004310 con 0x7f31180b9ad0 2024-09-21T00:27:02.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:02.929+0000 7f311fef2640 1 -- 172.21.15.112:0/415501092 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f311806ea30 con 0x7f31180b9ad0 2024-09-21T00:27:02.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:02.929+0000 7f310a7fc640 1 -- 172.21.15.112:0/415501092 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f311400de60 con 0x7f31180b9ad0 2024-09-21T00:27:02.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:02.929+0000 7f311fef2640 1 -- 172.21.15.112:0/415501092 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f311806eff0 con 0x7f31180b9ad0 2024-09-21T00:27:02.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:02.929+0000 7f311fef2640 1 -- 172.21.15.112:0/415501092 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31180b5bd0 con 0x7f31180b9ad0 2024-09-21T00:27:02.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:02.930+0000 7f310a7fc640 1 -- 172.21.15.112:0/415501092 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3114009c30 con 0x7f31180b9ad0 2024-09-21T00:27:02.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:02.931+0000 7f310a7fc640 1 --2- 172.21.15.112:0/415501092 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f30fc078bc0 0x7f30fc07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:02.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:02.931+0000 7f310a7fc640 1 -- 172.21.15.112:0/415501092 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 5335+0+0 (secure 0 0 0) 0x7f311409aea0 con 0x7f31180b9ad0 2024-09-21T00:27:02.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:02.931+0000 7f311d48d640 1 --2- 172.21.15.112:0/415501092 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f30fc078bc0 0x7f30fc07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:02.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:02.931+0000 7f311d48d640 1 --2- 172.21.15.112:0/415501092 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f30fc078bc0 0x7f30fc07b0d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f310c034000 tx=0x7f310c004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:02.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:02.935+0000 7f310a7fc640 1 -- 172.21.15.112:0/415501092 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f31180b5bd0 con 0x7f31180b9ad0 2024-09-21T00:27:03.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:03.122+0000 7f311fef2640 1 -- 172.21.15.112:0/415501092 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f31180ceb70 con 0x7f31180b9ad0 2024-09-21T00:27:03.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:03.539+0000 7f310a7fc640 1 -- 172.21.15.112:0/415501092 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v43) ==== 97+0+0 (secure 0 0 0) 0x7f3114020020 con 0x7f31180b9ad0 2024-09-21T00:27:03.541 INFO:teuthology.orchestra.run.smithi112.stderr: removed all blocklist entries 2024-09-21T00:27:03.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:03.542+0000 7f311fef2640 1 -- 172.21.15.112:0/415501092 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f30fc078bc0 msgr2=0x7f30fc07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:03.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:03.542+0000 7f311fef2640 1 --2- 172.21.15.112:0/415501092 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f30fc078bc0 0x7f30fc07b0d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f310c034000 tx=0x7f310c004730 comp rx=0 tx=0).stop 2024-09-21T00:27:03.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:03.542+0000 7f311fef2640 1 -- 172.21.15.112:0/415501092 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f31180b9ad0 msgr2=0x7f311814fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:03.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:03.542+0000 7f311fef2640 1 --2- 172.21.15.112:0/415501092 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f31180b9ad0 0x7f311814fa20 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f311400d750 tx=0x7f311400dc10 comp rx=0 tx=0).stop 2024-09-21T00:27:03.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:03.542+0000 7f311fef2640 1 -- 172.21.15.112:0/415501092 shutdown_connections 2024-09-21T00:27:03.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:03.542+0000 7f311fef2640 1 --2- 172.21.15.112:0/415501092 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f311815ee80 0x7f31180d4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:03.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:03.542+0000 7f311fef2640 1 --2- 172.21.15.112:0/415501092 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f31180d1120 0x7f31180d3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:03.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:03.542+0000 7f311fef2640 1 --2- 172.21.15.112:0/415501092 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f30fc078bc0 0x7f30fc07b0d0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:03.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:03.542+0000 7f311fef2640 1 --2- 172.21.15.112:0/415501092 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f31180b9ad0 0x7f311814fa20 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:03.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:03.542+0000 7f311fef2640 1 -- 172.21.15.112:0/415501092 >> 172.21.15.112:0/415501092 conn(0x7f311809d830 msgr2=0x7f3118161b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:03.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:03.542+0000 7f311fef2640 1 -- 172.21.15.112:0/415501092 shutdown_connections 2024-09-21T00:27:03.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:03.542+0000 7f311fef2640 1 -- 172.21.15.112:0/415501092 wait complete. 2024-09-21T00:27:03.556 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-21T00:27:03.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:03.668+0000 7fa32858b640 1 -- 172.21.15.112:0/2241964491 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa320162f00 msgr2=0x7fa3200d24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:03.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:03.668+0000 7fa32858b640 1 --2- 172.21.15.112:0/2241964491 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa320162f00 0x7fa3200d24d0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fa31c005fd0 tx=0x7fa31c02a540 comp rx=0 tx=0).stop 2024-09-21T00:27:03.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:03.668+0000 7fa32858b640 1 -- 172.21.15.112:0/2241964491 shutdown_connections 2024-09-21T00:27:03.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:03.668+0000 7fa32858b640 1 --2- 172.21.15.112:0/2241964491 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa320162f00 0x7fa3200d24d0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:03.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:03.668+0000 7fa32858b640 1 -- 172.21.15.112:0/2241964491 >> 172.21.15.112:0/2241964491 conn(0x7fa3200b6b20 msgr2=0x7fa3200d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:03.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:03.668+0000 7fa32858b640 1 -- 172.21.15.112:0/2241964491 shutdown_connections 2024-09-21T00:27:03.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:03.668+0000 7fa32858b640 1 -- 172.21.15.112:0/2241964491 wait complete. 2024-09-21T00:27:03.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:03.670+0000 7fa32858b640 1 Processor -- start 2024-09-21T00:27:03.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:03.670+0000 7fa32858b640 1 -- start start 2024-09-21T00:27:03.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:03.671+0000 7fa32858b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa3200b2e40 0x7fa3200ac7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:03.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:03.671+0000 7fa32858b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa320162280 0x7fa3200acd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:03.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:03.671+0000 7fa32858b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa3201f51c0 0x7fa3201f8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:03.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:03.671+0000 7fa32858b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa32015f6d0 con 0x7fa320162280 2024-09-21T00:27:03.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:03.671+0000 7fa32858b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa3200d2c00 con 0x7fa3200b2e40 2024-09-21T00:27:03.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:03.671+0000 7fa32858b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa32015f850 con 0x7fa3201f51c0 2024-09-21T00:27:03.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:03.671+0000 7fa325b26640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa320162280 0x7fa3200acd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:03.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:03.671+0000 7fa325b26640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa320162280 0x7fa3200acd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40926/0 (socket says 172.21.15.112:40926) 2024-09-21T00:27:03.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:03.671+0000 7fa325b26640 1 -- 172.21.15.112:0/510669120 learned_addr learned my addr 172.21.15.112:0/510669120 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:27:03.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:03.672+0000 7fa326327640 1 --2- 172.21.15.112:0/510669120 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa3200b2e40 0x7fa3200ac7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:03.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:03.672+0000 7fa326b28640 1 --2- 172.21.15.112:0/510669120 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa3201f51c0 0x7fa3201f8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:03.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:03.672+0000 7fa325b26640 1 -- 172.21.15.112:0/510669120 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa3201f51c0 msgr2=0x7fa3201f8990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:03.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:03.672+0000 7fa325b26640 1 --2- 172.21.15.112:0/510669120 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa3201f51c0 0x7fa3201f8990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:03.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:03.672+0000 7fa325b26640 1 -- 172.21.15.112:0/510669120 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa3200b2e40 msgr2=0x7fa3200ac7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:03.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:03.672+0000 7fa325b26640 1 --2- 172.21.15.112:0/510669120 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa3200b2e40 0x7fa3200ac7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:03.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:03.672+0000 7fa325b26640 1 -- 172.21.15.112:0/510669120 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3201f5930 con 0x7fa320162280 2024-09-21T00:27:03.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:03.672+0000 7fa325b26640 1 --2- 172.21.15.112:0/510669120 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa320162280 0x7fa3200acd00 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fa31000b3f0 tx=0x7fa31000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:03.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:03.672+0000 7fa316ffd640 1 -- 172.21.15.112:0/510669120 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa310015020 con 0x7fa320162280 2024-09-21T00:27:03.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:03.672+0000 7fa316ffd640 1 -- 172.21.15.112:0/510669120 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa310011920 con 0x7fa320162280 2024-09-21T00:27:03.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:03.672+0000 7fa32858b640 1 -- 172.21.15.112:0/510669120 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3201f5c10 con 0x7fa320162280 2024-09-21T00:27:03.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:03.672+0000 7fa316ffd640 1 -- 172.21.15.112:0/510669120 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa310011d70 con 0x7fa320162280 2024-09-21T00:27:03.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:03.672+0000 7fa32858b640 1 -- 172.21.15.112:0/510669120 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa32015a8d0 con 0x7fa320162280 2024-09-21T00:27:03.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:03.673+0000 7fa32858b640 1 -- 172.21.15.112:0/510669120 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa320162f00 con 0x7fa320162280 2024-09-21T00:27:03.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:03.674+0000 7fa316ffd640 1 -- 172.21.15.112:0/510669120 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa310018020 con 0x7fa320162280 2024-09-21T00:27:03.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:03.674+0000 7fa316ffd640 1 --2- 172.21.15.112:0/510669120 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fa304078b90 0x7fa30407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:03.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:03.675+0000 7fa316ffd640 1 -- 172.21.15.112:0/510669120 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5249+0+0 (secure 0 0 0) 0x7fa31009a740 con 0x7fa320162280 2024-09-21T00:27:03.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:03.675+0000 7fa326327640 1 --2- 172.21.15.112:0/510669120 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fa304078b90 0x7fa30407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:03.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:03.675+0000 7fa326327640 1 --2- 172.21.15.112:0/510669120 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fa304078b90 0x7fa30407b0a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa31c033000 tx=0x7fa31c004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:03.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:03.679+0000 7fa316ffd640 1 -- 172.21.15.112:0/510669120 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa31001f030 con 0x7fa320162280 2024-09-21T00:27:03.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:03.925+0000 7fa32858b640 1 -- 172.21.15.112:0/510669120 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fa3201f56c0 con 0x7fa320162280 2024-09-21T00:27:03.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:03.926+0000 7fa316ffd640 1 -- 172.21.15.112:0/510669120 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v15) ==== 80+0+5304 (secure 0 0 0) 0x7fa3100626e0 con 0x7fa320162280 2024-09-21T00:27:03.928 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:27:03.928 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-21T00:27:03.928 INFO:teuthology.orchestra.run.smithi112.stdout: "auth_dump": [ 2024-09-21T00:27:03.928 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:27:03.928 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "mds.a", 2024-09-21T00:27:03.928 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBPEu5m3capAhAAczRM4L8ldX985NW4ep3w5Q==", 2024-09-21T00:27:03.928 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:27:03.928 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2024-09-21T00:27:03.929 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow *", 2024-09-21T00:27:03.929 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow *", 2024-09-21T00:27:03.929 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2024-09-21T00:27:03.929 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:27:03.929 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:27:03.929 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:27:03.929 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "mds.b", 2024-09-21T00:27:03.929 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBPEu5mxlDlDxAAtqngbTOhPqXDjFhJpSCLHg==", 2024-09-21T00:27:03.929 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:27:03.929 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2024-09-21T00:27:03.929 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow *", 2024-09-21T00:27:03.929 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow *", 2024-09-21T00:27:03.929 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2024-09-21T00:27:03.929 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:27:03.929 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:27:03.930 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:27:03.930 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "mds.c", 2024-09-21T00:27:03.930 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBPEu5mXNVRChAA1qsPbEybmY16jw5Jwk/v1A==", 2024-09-21T00:27:03.930 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:27:03.930 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2024-09-21T00:27:03.930 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow *", 2024-09-21T00:27:03.930 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow *", 2024-09-21T00:27:03.930 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2024-09-21T00:27:03.930 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:27:03.930 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:27:03.930 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:27:03.930 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "mds.d", 2024-09-21T00:27:03.930 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBPEu5mnzySFxAAX3VfTgGYGcDweI7rPf3xvw==", 2024-09-21T00:27:03.931 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:27:03.931 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2024-09-21T00:27:03.931 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow *", 2024-09-21T00:27:03.931 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow *", 2024-09-21T00:27:03.931 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2024-09-21T00:27:03.931 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:27:03.931 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:27:03.931 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:27:03.931 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.0", 2024-09-21T00:27:03.931 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBSEu5moQg4OBAAi8mzLx7tWnORlrL64/MWlw==", 2024-09-21T00:27:03.931 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:27:03.931 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2024-09-21T00:27:03.931 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2024-09-21T00:27:03.931 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2024-09-21T00:27:03.932 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:27:03.932 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:27:03.932 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:27:03.932 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.1", 2024-09-21T00:27:03.932 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBWEu5mxfobDhAAi7I6LYq9udB0W8bnYaBY9Q==", 2024-09-21T00:27:03.932 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:27:03.932 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2024-09-21T00:27:03.932 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2024-09-21T00:27:03.932 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2024-09-21T00:27:03.932 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:27:03.932 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:27:03.932 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:27:03.933 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.2", 2024-09-21T00:27:03.933 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBZEu5mDqC/EhAAwzTbj22ocR4xJ8DMJvN7Lg==", 2024-09-21T00:27:03.933 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:27:03.933 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2024-09-21T00:27:03.933 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2024-09-21T00:27:03.933 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2024-09-21T00:27:03.933 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:27:03.933 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:27:03.933 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:27:03.933 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.3", 2024-09-21T00:27:03.933 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBcEu5mwweGJBAAT3hZn5pto0aWt6BwGuaqcw==", 2024-09-21T00:27:03.933 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:27:03.933 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2024-09-21T00:27:03.933 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2024-09-21T00:27:03.934 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2024-09-21T00:27:03.934 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:27:03.934 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:27:03.934 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:27:03.934 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.4", 2024-09-21T00:27:03.934 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBjEu5mIPd3ARAAkxTYphn38uKh48UUVEB+Fg==", 2024-09-21T00:27:03.934 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:27:03.934 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2024-09-21T00:27:03.934 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2024-09-21T00:27:03.934 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2024-09-21T00:27:03.934 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:27:03.934 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:27:03.934 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:27:03.934 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.5", 2024-09-21T00:27:03.935 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBmEu5mP9IVBBAAftfK5V1K4vYRyFyJS2UzKQ==", 2024-09-21T00:27:03.935 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:27:03.935 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2024-09-21T00:27:03.935 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2024-09-21T00:27:03.935 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2024-09-21T00:27:03.935 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:27:03.935 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:27:03.935 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:27:03.935 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.6", 2024-09-21T00:27:03.935 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBpEu5mmjiyFRAAbiW2M8JE0jJOaNg1u+1kgw==", 2024-09-21T00:27:03.935 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:27:03.935 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2024-09-21T00:27:03.935 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2024-09-21T00:27:03.935 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2024-09-21T00:27:03.936 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:27:03.936 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:27:03.936 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:27:03.936 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.7", 2024-09-21T00:27:03.936 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBsEu5moklPKBAA6hzVvgvpmZMkPaFFce4mqw==", 2024-09-21T00:27:03.936 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:27:03.936 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2024-09-21T00:27:03.936 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2024-09-21T00:27:03.936 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2024-09-21T00:27:03.936 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:27:03.936 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:27:03.937 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:27:03.937 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "client.0", 2024-09-21T00:27:03.937 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBPEu5m49NPHBAAl/rcjiE8xYI+XdWwvZ6FOw==", 2024-09-21T00:27:03.937 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:27:03.937 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2024-09-21T00:27:03.937 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow r", 2024-09-21T00:27:03.937 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-21T00:27:03.937 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:27:03.937 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:27:03.937 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:27:03.937 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "client.1", 2024-09-21T00:27:03.937 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBPEu5mG33aIxAAirQhTlVyF/pRJx93T/0bIQ==", 2024-09-21T00:27:03.937 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:27:03.937 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2024-09-21T00:27:03.937 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow r", 2024-09-21T00:27:03.938 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow rw", 2024-09-21T00:27:03.938 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow rwx" 2024-09-21T00:27:03.938 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:27:03.938 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:27:03.938 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:27:03.938 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "client.2", 2024-09-21T00:27:03.938 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBPEu5mreUUIBAAELeN1N58srw6GqBrj8UOoQ==", 2024-09-21T00:27:03.938 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:27:03.938 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2024-09-21T00:27:03.938 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow r", 2024-09-21T00:27:03.938 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow rw", 2024-09-21T00:27:03.938 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow rwx" 2024-09-21T00:27:03.938 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:27:03.939 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:27:03.939 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:27:03.939 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "client.3", 2024-09-21T00:27:03.939 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBPEu5mcTmiJxAAY6bmTtow0R0heJHjx0ahDQ==", 2024-09-21T00:27:03.939 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:27:03.939 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2024-09-21T00:27:03.939 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow r", 2024-09-21T00:27:03.939 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow rw", 2024-09-21T00:27:03.939 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow rwx" 2024-09-21T00:27:03.939 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:27:03.939 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:27:03.939 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:27:03.939 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "client.admin", 2024-09-21T00:27:03.939 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBOEu5mTxhNIBAAlStXGsGd/okXhakY6WI/4w==", 2024-09-21T00:27:03.940 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:27:03.940 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow *", 2024-09-21T00:27:03.940 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow *", 2024-09-21T00:27:03.940 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow *", 2024-09-21T00:27:03.940 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2024-09-21T00:27:03.940 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:27:03.940 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:27:03.940 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:27:03.940 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "mgr.x", 2024-09-21T00:27:03.940 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBOEu5mjGpDOhAAJFudb2wZlOBjo8Ss65Q+eg==", 2024-09-21T00:27:03.940 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:27:03.940 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow *", 2024-09-21T00:27:03.940 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile mgr", 2024-09-21T00:27:03.940 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2024-09-21T00:27:03.940 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:27:03.941 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:27:03.941 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:27:03.941 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "mgr.y", 2024-09-21T00:27:03.941 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBOEu5myGjKNRAA9cMDObZYYOXMGdv2mOXkJA==", 2024-09-21T00:27:03.941 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:27:03.941 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow *", 2024-09-21T00:27:03.941 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile mgr", 2024-09-21T00:27:03.941 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2024-09-21T00:27:03.941 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:27:03.941 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:27:03.941 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-09-21T00:27:03.941 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-21T00:27:03.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:03.928+0000 7fa32858b640 1 -- 172.21.15.112:0/510669120 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fa304078b90 msgr2=0x7fa30407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:03.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:03.928+0000 7fa32858b640 1 --2- 172.21.15.112:0/510669120 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fa304078b90 0x7fa30407b0a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa31c033000 tx=0x7fa31c004730 comp rx=0 tx=0).stop 2024-09-21T00:27:03.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:03.928+0000 7fa32858b640 1 -- 172.21.15.112:0/510669120 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa320162280 msgr2=0x7fa3200acd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:03.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:03.928+0000 7fa32858b640 1 --2- 172.21.15.112:0/510669120 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa320162280 0x7fa3200acd00 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fa31000b3f0 tx=0x7fa31000b8b0 comp rx=0 tx=0).stop 2024-09-21T00:27:03.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:03.928+0000 7fa32858b640 1 -- 172.21.15.112:0/510669120 shutdown_connections 2024-09-21T00:27:03.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:03.928+0000 7fa32858b640 1 --2- 172.21.15.112:0/510669120 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa3201f51c0 0x7fa3201f8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:03.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:03.928+0000 7fa32858b640 1 --2- 172.21.15.112:0/510669120 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fa304078b90 0x7fa30407b0a0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:03.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:03.928+0000 7fa32858b640 1 --2- 172.21.15.112:0/510669120 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa320162280 0x7fa3200acd00 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:03.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:03.928+0000 7fa32858b640 1 --2- 172.21.15.112:0/510669120 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa3200b2e40 0x7fa3200ac7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:03.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:03.928+0000 7fa32858b640 1 -- 172.21.15.112:0/510669120 >> 172.21.15.112:0/510669120 conn(0x7fa3200b6b20 msgr2=0x7fa320068d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:03.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:03.928+0000 7fa32858b640 1 -- 172.21.15.112:0/510669120 shutdown_connections 2024-09-21T00:27:03.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:03.928+0000 7fa32858b640 1 -- 172.21.15.112:0/510669120 wait complete. 2024-09-21T00:27:03.944 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-21T00:27:04.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:04.054+0000 7f2f3f70b640 1 -- 172.21.15.112:0/2778618066 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f38074d90 msgr2=0x7f2f3806f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:04.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:04.054+0000 7f2f3f70b640 1 --2- 172.21.15.112:0/2778618066 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f38074d90 0x7f2f3806f710 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f2f38071fa0 tx=0x7f2f3402a560 comp rx=0 tx=0).stop 2024-09-21T00:27:04.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:04.055+0000 7f2f3f70b640 1 -- 172.21.15.112:0/2778618066 shutdown_connections 2024-09-21T00:27:04.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:04.055+0000 7f2f3f70b640 1 --2- 172.21.15.112:0/2778618066 >> v2:172.21.15.187:3300/0 conn(0x7f2f38070030 0x7f2f3806ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:04.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:04.055+0000 7f2f3f70b640 1 --2- 172.21.15.112:0/2778618066 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f38074d90 0x7f2f3806f710 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:04.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:04.055+0000 7f2f3f70b640 1 -- 172.21.15.112:0/2778618066 >> 172.21.15.112:0/2778618066 conn(0x7f2f380cca40 msgr2=0x7f2f380cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:04.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:04.055+0000 7f2f3f70b640 1 -- 172.21.15.112:0/2778618066 shutdown_connections 2024-09-21T00:27:04.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:04.055+0000 7f2f3f70b640 1 -- 172.21.15.112:0/2778618066 wait complete. 2024-09-21T00:27:04.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:04.056+0000 7f2f3f70b640 1 Processor -- start 2024-09-21T00:27:04.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:04.056+0000 7f2f3f70b640 1 -- start start 2024-09-21T00:27:04.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:04.058+0000 7f2f3f70b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2f3806e330 0x7f2f380aeae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:04.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:04.058+0000 7f2f3f70b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f38070030 0x7f2f380c8b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:04.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:04.058+0000 7f2f3f70b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2f38074d90 0x7f2f380c9050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:04.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:04.058+0000 7f2f3f70b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2f3815ad50 con 0x7f2f38070030 2024-09-21T00:27:04.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:04.058+0000 7f2f3f70b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f2f3815abd0 con 0x7f2f3806e330 2024-09-21T00:27:04.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:04.058+0000 7f2f3f70b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f2f38070e80 con 0x7f2f38074d90 2024-09-21T00:27:04.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:04.058+0000 7f2f3cca6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f38070030 0x7f2f380c8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:04.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:04.058+0000 7f2f3cca6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f38070030 0x7f2f380c8b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40946/0 (socket says 172.21.15.112:40946) 2024-09-21T00:27:04.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:04.058+0000 7f2f3cca6640 1 -- 172.21.15.112:0/4234896860 learned_addr learned my addr 172.21.15.112:0/4234896860 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:27:04.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:04.058+0000 7f2f3dca8640 1 --2- 172.21.15.112:0/4234896860 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2f38074d90 0x7f2f380c9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:04.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:04.058+0000 7f2f3d4a7640 1 --2- 172.21.15.112:0/4234896860 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2f3806e330 0x7f2f380aeae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:04.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:04.058+0000 7f2f3cca6640 1 -- 172.21.15.112:0/4234896860 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2f38074d90 msgr2=0x7f2f380c9050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:04.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:04.058+0000 7f2f3cca6640 1 --2- 172.21.15.112:0/4234896860 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2f38074d90 0x7f2f380c9050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:04.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:04.058+0000 7f2f3cca6640 1 -- 172.21.15.112:0/4234896860 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2f3806e330 msgr2=0x7f2f380aeae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:04.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:04.058+0000 7f2f3cca6640 1 --2- 172.21.15.112:0/4234896860 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2f3806e330 0x7f2f380aeae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:04.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:04.058+0000 7f2f3cca6640 1 -- 172.21.15.112:0/4234896860 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f380b5a40 con 0x7f2f38070030 2024-09-21T00:27:04.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:04.058+0000 7f2f3cca6640 1 --2- 172.21.15.112:0/4234896860 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f38070030 0x7f2f380c8b10 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f2f2800b410 tx=0x7f2f2800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:04.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:04.058+0000 7f2f2dffb640 1 -- 172.21.15.112:0/4234896860 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f28015020 con 0x7f2f38070030 2024-09-21T00:27:04.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:04.059+0000 7f2f2dffb640 1 -- 172.21.15.112:0/4234896860 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2f28011940 con 0x7f2f38070030 2024-09-21T00:27:04.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:04.059+0000 7f2f3f70b640 1 -- 172.21.15.112:0/4234896860 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f380c8030 con 0x7f2f38070030 2024-09-21T00:27:04.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:04.059+0000 7f2f2dffb640 1 -- 172.21.15.112:0/4234896860 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f28011d90 con 0x7f2f38070030 2024-09-21T00:27:04.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:04.059+0000 7f2f3f70b640 1 -- 172.21.15.112:0/4234896860 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f380c85f0 con 0x7f2f38070030 2024-09-21T00:27:04.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:04.060+0000 7f2f3f70b640 1 -- 172.21.15.112:0/4234896860 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f380b57b0 con 0x7f2f38070030 2024-09-21T00:27:04.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:04.060+0000 7f2f2dffb640 1 -- 172.21.15.112:0/4234896860 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2f28018020 con 0x7f2f38070030 2024-09-21T00:27:04.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:04.061+0000 7f2f2dffb640 1 --2- 172.21.15.112:0/4234896860 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2f1c078bc0 0x7f2f1c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:04.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:04.061+0000 7f2f2dffb640 1 -- 172.21.15.112:0/4234896860 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5249+0+0 (secure 0 0 0) 0x7f2f2809ab20 con 0x7f2f38070030 2024-09-21T00:27:04.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:04.061+0000 7f2f3d4a7640 1 --2- 172.21.15.112:0/4234896860 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2f1c078bc0 0x7f2f1c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:04.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:04.062+0000 7f2f3d4a7640 1 --2- 172.21.15.112:0/4234896860 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2f1c078bc0 0x7f2f1c07b0d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f2f34033000 tx=0x7f2f34004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:04.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:04.065+0000 7f2f2dffb640 1 -- 172.21.15.112:0/4234896860 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2f2801f020 con 0x7f2f38070030 2024-09-21T00:27:04.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:04.243+0000 7f2f3f70b640 1 -- 172.21.15.112:0/4234896860 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f2f3806ddf0 con 0x7f2f38070030 2024-09-21T00:27:04.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:04.244+0000 7f2f2dffb640 1 -- 172.21.15.112:0/4234896860 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v45) ==== 84+0+65 (secure 0 0 0) 0x7f2f280631b0 con 0x7f2f38070030 2024-09-21T00:27:04.245 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:27:04.246 INFO:teuthology.orchestra.run.smithi112.stdout:[ 2024-09-21T00:27:04.246 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:27:04.246 INFO:teuthology.orchestra.run.smithi112.stdout: "poolnum": 1, 2024-09-21T00:27:04.246 INFO:teuthology.orchestra.run.smithi112.stdout: "poolname": ".mgr" 2024-09-21T00:27:04.246 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:27:04.246 INFO:teuthology.orchestra.run.smithi112.stdout:] 2024-09-21T00:27:04.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:04.246+0000 7f2f3f70b640 1 -- 172.21.15.112:0/4234896860 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2f1c078bc0 msgr2=0x7f2f1c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:04.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:04.246+0000 7f2f3f70b640 1 --2- 172.21.15.112:0/4234896860 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2f1c078bc0 0x7f2f1c07b0d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f2f34033000 tx=0x7f2f34004730 comp rx=0 tx=0).stop 2024-09-21T00:27:04.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:04.246+0000 7f2f3f70b640 1 -- 172.21.15.112:0/4234896860 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f38070030 msgr2=0x7f2f380c8b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:04.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:04.246+0000 7f2f3f70b640 1 --2- 172.21.15.112:0/4234896860 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f38070030 0x7f2f380c8b10 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f2f2800b410 tx=0x7f2f2800b8d0 comp rx=0 tx=0).stop 2024-09-21T00:27:04.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:04.246+0000 7f2f3f70b640 1 -- 172.21.15.112:0/4234896860 shutdown_connections 2024-09-21T00:27:04.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:04.246+0000 7f2f3f70b640 1 --2- 172.21.15.112:0/4234896860 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2f38074d90 0x7f2f380c9050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:04.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:04.246+0000 7f2f3f70b640 1 --2- 172.21.15.112:0/4234896860 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2f1c078bc0 0x7f2f1c07b0d0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:04.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:04.246+0000 7f2f3f70b640 1 --2- 172.21.15.112:0/4234896860 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f38070030 0x7f2f380c8b10 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:04.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:04.246+0000 7f2f3f70b640 1 --2- 172.21.15.112:0/4234896860 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2f3806e330 0x7f2f380aeae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:04.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:04.246+0000 7f2f3f70b640 1 -- 172.21.15.112:0/4234896860 >> 172.21.15.112:0/4234896860 conn(0x7f2f380cca40 msgr2=0x7f2f38073dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:04.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:04.246+0000 7f2f3f70b640 1 -- 172.21.15.112:0/4234896860 shutdown_connections 2024-09-21T00:27:04.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:04.246+0000 7f2f3f70b640 1 -- 172.21.15.112:0/4234896860 wait complete. 2024-09-21T00:27:04.259 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-21T00:27:04.259 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-21T00:27:04.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:04.374+0000 7fa123d57640 1 -- 172.21.15.112:0/2057426883 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa108005210 msgr2=0x7fa10800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:04.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:04.374+0000 7fa123d57640 1 --2- 172.21.15.112:0/2057426883 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa108005210 0x7fa10800b9c0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fa11c0c7d90 tx=0x7fa108005830 comp rx=0 tx=0).stop 2024-09-21T00:27:04.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:04.374+0000 7fa1222f4640 1 -- 172.21.15.112:0/2057426883 reap_dead start 2024-09-21T00:27:04.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:04.374+0000 7fa123d57640 1 -- 172.21.15.112:0/2057426883 shutdown_connections 2024-09-21T00:27:04.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:04.374+0000 7fa123d57640 1 -- 172.21.15.112:0/2057426883 >> 172.21.15.112:0/2057426883 conn(0x7fa11c0d4510 msgr2=0x7fa11c0d4920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:04.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:04.375+0000 7fa123d57640 1 -- 172.21.15.112:0/2057426883 shutdown_connections 2024-09-21T00:27:04.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:04.375+0000 7fa123d57640 1 -- 172.21.15.112:0/2057426883 wait complete. 2024-09-21T00:27:04.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:04.376+0000 7fa123d57640 1 Processor -- start 2024-09-21T00:27:04.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:04.376+0000 7fa123d57640 1 -- start start 2024-09-21T00:27:04.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:04.378+0000 7fa123d57640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa108005210 0x7fa11c1671b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:04.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:04.378+0000 7fa123d57640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa11c1f6770 0x7fa11c1f6b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:04.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:04.378+0000 7fa123d57640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa11c1f5550 0x7fa11c1f5960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:04.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:04.378+0000 7fa123d57640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa11c167aa0 con 0x7fa11c1f5550 2024-09-21T00:27:04.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:04.378+0000 7fa123d57640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa11c167920 con 0x7fa108005210 2024-09-21T00:27:04.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:04.378+0000 7fa123d57640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa11c167c20 con 0x7fa11c1f6770 2024-09-21T00:27:04.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:04.378+0000 7fa1222f4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa11c1f5550 0x7fa11c1f5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:04.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:04.378+0000 7fa1222f4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa11c1f5550 0x7fa11c1f5960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40974/0 (socket says 172.21.15.112:40974) 2024-09-21T00:27:04.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:04.378+0000 7fa1222f4640 1 -- 172.21.15.112:0/3339598997 learned_addr learned my addr 172.21.15.112:0/3339598997 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:27:04.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:04.378+0000 7fa121af3640 1 --2- 172.21.15.112:0/3339598997 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa108005210 0x7fa11c1671b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:04.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:04.378+0000 7fa1222f4640 1 -- 172.21.15.112:0/3339598997 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa11c1f6770 msgr2=0x7fa11c1f6b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:04.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:04.378+0000 7fa1222f4640 1 --2- 172.21.15.112:0/3339598997 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa11c1f6770 0x7fa11c1f6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:04.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:04.378+0000 7fa1222f4640 1 -- 172.21.15.112:0/3339598997 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa108005210 msgr2=0x7fa11c1671b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:04.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:04.378+0000 7fa1222f4640 1 --2- 172.21.15.112:0/3339598997 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa108005210 0x7fa11c1671b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:04.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:04.378+0000 7fa1222f4640 1 -- 172.21.15.112:0/3339598997 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa11c0ccff0 con 0x7fa11c1f5550 2024-09-21T00:27:04.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:04.378+0000 7fa1222f4640 1 --2- 172.21.15.112:0/3339598997 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa11c1f5550 0x7fa11c1f5960 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fa114035040 tx=0x7fa114033c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:04.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:04.378+0000 7fa1127fc640 1 -- 172.21.15.112:0/3339598997 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa114009c10 con 0x7fa11c1f5550 2024-09-21T00:27:04.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:04.378+0000 7fa1127fc640 1 -- 172.21.15.112:0/3339598997 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa11402bba0 con 0x7fa11c1f5550 2024-09-21T00:27:04.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:04.378+0000 7fa1127fc640 1 -- 172.21.15.112:0/3339598997 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa11403e6d0 con 0x7fa11c1f5550 2024-09-21T00:27:04.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:04.378+0000 7fa123d57640 1 -- 172.21.15.112:0/3339598997 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa11c1612e0 con 0x7fa11c1f5550 2024-09-21T00:27:04.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:04.378+0000 7fa123d57640 1 -- 172.21.15.112:0/3339598997 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa11c0cd570 con 0x7fa11c1f5550 2024-09-21T00:27:04.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:04.379+0000 7fa123d57640 1 -- 172.21.15.112:0/3339598997 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa11c0b7b60 con 0x7fa11c1f5550 2024-09-21T00:27:04.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:04.380+0000 7fa1127fc640 1 -- 172.21.15.112:0/3339598997 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa114046020 con 0x7fa11c1f5550 2024-09-21T00:27:04.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:04.380+0000 7fa1127fc640 1 --2- 172.21.15.112:0/3339598997 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fa100078b90 0x7fa10007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:04.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:04.381+0000 7fa1127fc640 1 -- 172.21.15.112:0/3339598997 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5249+0+0 (secure 0 0 0) 0x7fa1140bbdb0 con 0x7fa11c1f5550 2024-09-21T00:27:04.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:04.381+0000 7fa121af3640 1 --2- 172.21.15.112:0/3339598997 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fa100078b90 0x7fa10007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:04.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:04.381+0000 7fa121af3640 1 --2- 172.21.15.112:0/3339598997 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fa100078b90 0x7fa10007b0a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa118006560 tx=0x7fa118006b10 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:04.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:04.385+0000 7fa1127fc640 1 -- 172.21.15.112:0/3339598997 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa11404d020 con 0x7fa11c1f5550 2024-09-21T00:27:04.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:04.564+0000 7fa123d57640 1 -- 172.21.15.112:0/3339598997 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fa11c161070 con 0x7fa11c1f5550 2024-09-21T00:27:05.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:05.547+0000 7fa1127fc640 1 -- 172.21.15.112:0/3339598997 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v46) ==== 138+0+0 (secure 0 0 0) 0x7fa1140843b0 con 0x7fa11c1f5550 2024-09-21T00:27:05.549 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'cephfs_metadata' created 2024-09-21T00:27:05.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:05.550+0000 7fa123d57640 1 -- 172.21.15.112:0/3339598997 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fa100078b90 msgr2=0x7fa10007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:05.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:05.550+0000 7fa123d57640 1 --2- 172.21.15.112:0/3339598997 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fa100078b90 0x7fa10007b0a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa118006560 tx=0x7fa118006b10 comp rx=0 tx=0).stop 2024-09-21T00:27:05.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:05.550+0000 7fa123d57640 1 -- 172.21.15.112:0/3339598997 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa11c1f5550 msgr2=0x7fa11c1f5960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:05.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:05.550+0000 7fa123d57640 1 --2- 172.21.15.112:0/3339598997 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa11c1f5550 0x7fa11c1f5960 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fa114035040 tx=0x7fa114033c30 comp rx=0 tx=0).stop 2024-09-21T00:27:05.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:05.550+0000 7fa123d57640 1 -- 172.21.15.112:0/3339598997 shutdown_connections 2024-09-21T00:27:05.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:05.550+0000 7fa123d57640 1 --2- 172.21.15.112:0/3339598997 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fa100078b90 0x7fa10007b0a0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:05.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:05.550+0000 7fa123d57640 1 --2- 172.21.15.112:0/3339598997 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa11c1f5550 0x7fa11c1f5960 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:05.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:05.550+0000 7fa123d57640 1 --2- 172.21.15.112:0/3339598997 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa11c1f6770 0x7fa11c1f6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:05.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:05.550+0000 7fa123d57640 1 --2- 172.21.15.112:0/3339598997 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa108005210 0x7fa11c1671b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:05.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:05.550+0000 7fa123d57640 1 -- 172.21.15.112:0/3339598997 >> 172.21.15.112:0/3339598997 conn(0x7fa11c0d4510 msgr2=0x7fa11c0d2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:05.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:05.550+0000 7fa123d57640 1 -- 172.21.15.112:0/3339598997 shutdown_connections 2024-09-21T00:27:05.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:05.550+0000 7fa123d57640 1 -- 172.21.15.112:0/3339598997 wait complete. 2024-09-21T00:27:05.568 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-21T00:27:05.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:05.688+0000 7fcef2598640 1 -- 172.21.15.112:0/795638621 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fceec15b9f0 msgr2=0x7fceec0cf930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:05.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:05.688+0000 7fcef2598640 1 --2- 172.21.15.112:0/795638621 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fceec15b9f0 0x7fceec0cf930 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fceec072d40 tx=0x7fcedc02a560 comp rx=0 tx=0).stop 2024-09-21T00:27:05.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:05.688+0000 7fcef2598640 1 -- 172.21.15.112:0/795638621 shutdown_connections 2024-09-21T00:27:05.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:05.688+0000 7fcef2598640 1 --2- 172.21.15.112:0/795638621 >> v2:172.21.15.187:3300/0 conn(0x7fceec0cefa0 0x7fceec0ce320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:05.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:05.688+0000 7fcef2598640 1 --2- 172.21.15.112:0/795638621 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fceec15b9f0 0x7fceec0cf930 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:05.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:05.688+0000 7fcef2598640 1 -- 172.21.15.112:0/795638621 >> 172.21.15.112:0/795638621 conn(0x7fceec0cca40 msgr2=0x7fceec0cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:05.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:05.688+0000 7fcef2598640 1 -- 172.21.15.112:0/795638621 shutdown_connections 2024-09-21T00:27:05.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:05.688+0000 7fcef2598640 1 -- 172.21.15.112:0/795638621 wait complete. 2024-09-21T00:27:05.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:05.689+0000 7fcef2598640 1 Processor -- start 2024-09-21T00:27:05.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:05.690+0000 7fcef2598640 1 -- start start 2024-09-21T00:27:05.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:05.691+0000 7fcef2598640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fceec0ce860 0x7fceec0c5d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:05.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:05.691+0000 7fcef2598640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fceec0cefa0 0x7fceec0c6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:05.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:05.691+0000 7fcef2598640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fceec15b9f0 0x7fceec1f3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:05.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:05.691+0000 7fcef2598640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fceec06ddf0 con 0x7fceec15b9f0 2024-09-21T00:27:05.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:05.691+0000 7fcef2598640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fceec06df70 con 0x7fceec0ce860 2024-09-21T00:27:05.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:05.691+0000 7fcef2598640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fceec06e0f0 con 0x7fceec0cefa0 2024-09-21T00:27:05.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:05.691+0000 7fcef0b35640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fceec15b9f0 0x7fceec1f3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:05.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:05.691+0000 7fcef0b35640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fceec15b9f0 0x7fceec1f3710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40992/0 (socket says 172.21.15.112:40992) 2024-09-21T00:27:05.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:05.691+0000 7fcef0b35640 1 -- 172.21.15.112:0/2132542359 learned_addr learned my addr 172.21.15.112:0/2132542359 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:27:05.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:05.691+0000 7fceeb7fe640 1 --2- 172.21.15.112:0/2132542359 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fceec0cefa0 0x7fceec0c6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:05.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:05.691+0000 7fceebfff640 1 --2- 172.21.15.112:0/2132542359 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fceec0ce860 0x7fceec0c5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:05.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:05.691+0000 7fcef0b35640 1 -- 172.21.15.112:0/2132542359 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fceec0cefa0 msgr2=0x7fceec0c6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:05.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:05.691+0000 7fcef0b35640 1 --2- 172.21.15.112:0/2132542359 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fceec0cefa0 0x7fceec0c6240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:05.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:05.691+0000 7fcef0b35640 1 -- 172.21.15.112:0/2132542359 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fceec0ce860 msgr2=0x7fceec0c5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:05.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:05.691+0000 7fcef0b35640 1 --2- 172.21.15.112:0/2132542359 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fceec0ce860 0x7fceec0c5d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:05.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:05.691+0000 7fcef0b35640 1 -- 172.21.15.112:0/2132542359 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fceec1f73c0 con 0x7fceec15b9f0 2024-09-21T00:27:05.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:05.692+0000 7fcef0b35640 1 --2- 172.21.15.112:0/2132542359 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fceec15b9f0 0x7fceec1f3710 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fcee400ba10 tx=0x7fcee400bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:05.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:05.692+0000 7fcee8ff9640 1 -- 172.21.15.112:0/2132542359 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcee4019070 con 0x7fceec15b9f0 2024-09-21T00:27:05.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:05.692+0000 7fcef2598640 1 -- 172.21.15.112:0/2132542359 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fceec06efd0 con 0x7fceec15b9f0 2024-09-21T00:27:05.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:05.692+0000 7fcee8ff9640 1 -- 172.21.15.112:0/2132542359 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcee400cc30 con 0x7fceec15b9f0 2024-09-21T00:27:05.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:05.692+0000 7fcee8ff9640 1 -- 172.21.15.112:0/2132542359 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcee40146a0 con 0x7fceec15b9f0 2024-09-21T00:27:05.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:05.692+0000 7fcef2598640 1 -- 172.21.15.112:0/2132542359 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fceec06f590 con 0x7fceec15b9f0 2024-09-21T00:27:05.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:05.693+0000 7fcef2598640 1 -- 172.21.15.112:0/2132542359 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fceec06e620 con 0x7fceec15b9f0 2024-09-21T00:27:05.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:05.694+0000 7fcee8ff9640 1 -- 172.21.15.112:0/2132542359 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fcee401c070 con 0x7fceec15b9f0 2024-09-21T00:27:05.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:05.694+0000 7fcee8ff9640 1 --2- 172.21.15.112:0/2132542359 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fced8078bc0 0x7fced807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:05.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:05.694+0000 7fcee8ff9640 1 -- 172.21.15.112:0/2132542359 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5619+0+0 (secure 0 0 0) 0x7fcee409ae60 con 0x7fceec15b9f0 2024-09-21T00:27:05.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:05.694+0000 7fceebfff640 1 --2- 172.21.15.112:0/2132542359 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fced8078bc0 0x7fced807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:05.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:05.695+0000 7fceebfff640 1 --2- 172.21.15.112:0/2132542359 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fced8078bc0 0x7fced807b0d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fcedc033000 tx=0x7fcedc004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:05.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:05.698+0000 7fcee8ff9640 1 -- 172.21.15.112:0/2132542359 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fcee4023020 con 0x7fceec15b9f0 2024-09-21T00:27:05.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:05.876+0000 7fcef2598640 1 -- 172.21.15.112:0/2132542359 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fceec075050 con 0x7fceec15b9f0 2024-09-21T00:27:06.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:06.555+0000 7fcee8ff9640 1 -- 172.21.15.112:0/2132542359 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v47) ==== 170+0+0 (secure 0 0 0) 0x7fcee400ce20 con 0x7fceec15b9f0 2024-09-21T00:27:06.557 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'cephfs_data' created 2024-09-21T00:27:06.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:06.559+0000 7fcef2598640 1 -- 172.21.15.112:0/2132542359 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fced8078bc0 msgr2=0x7fced807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:06.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:06.559+0000 7fcef2598640 1 --2- 172.21.15.112:0/2132542359 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fced8078bc0 0x7fced807b0d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fcedc033000 tx=0x7fcedc004730 comp rx=0 tx=0).stop 2024-09-21T00:27:06.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:06.559+0000 7fcef2598640 1 -- 172.21.15.112:0/2132542359 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fceec15b9f0 msgr2=0x7fceec1f3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:06.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:06.559+0000 7fcef2598640 1 --2- 172.21.15.112:0/2132542359 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fceec15b9f0 0x7fceec1f3710 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fcee400ba10 tx=0x7fcee400bed0 comp rx=0 tx=0).stop 2024-09-21T00:27:06.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:06.559+0000 7fcef2598640 1 -- 172.21.15.112:0/2132542359 shutdown_connections 2024-09-21T00:27:06.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:06.559+0000 7fcef2598640 1 --2- 172.21.15.112:0/2132542359 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fced8078bc0 0x7fced807b0d0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:06.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:06.559+0000 7fcef2598640 1 --2- 172.21.15.112:0/2132542359 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fceec15b9f0 0x7fceec1f3710 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:06.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:06.559+0000 7fcef2598640 1 --2- 172.21.15.112:0/2132542359 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fceec0cefa0 0x7fceec0c6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:06.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:06.559+0000 7fcef2598640 1 --2- 172.21.15.112:0/2132542359 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fceec0ce860 0x7fceec0c5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:06.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:06.559+0000 7fcef2598640 1 -- 172.21.15.112:0/2132542359 >> 172.21.15.112:0/2132542359 conn(0x7fceec0cca40 msgr2=0x7fceec0afb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:06.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:06.559+0000 7fcef2598640 1 -- 172.21.15.112:0/2132542359 shutdown_connections 2024-09-21T00:27:06.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:06.559+0000 7fcef2598640 1 -- 172.21.15.112:0/2132542359 wait complete. 2024-09-21T00:27:06.587 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-21T00:27:06.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:06.701+0000 7f9d2727f640 1 -- 172.21.15.112:0/3535277471 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9d200ccc40 msgr2=0x7f9d2015ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:06.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:06.701+0000 7f9d2727f640 1 --2- 172.21.15.112:0/3535277471 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9d200ccc40 0x7f9d2015ca90 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f9d20071cd0 tx=0x7f9d1c02a4b0 comp rx=0 tx=0).stop 2024-09-21T00:27:06.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:06.701+0000 7f9d2727f640 1 -- 172.21.15.112:0/3535277471 shutdown_connections 2024-09-21T00:27:06.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:06.701+0000 7f9d2727f640 1 --2- 172.21.15.112:0/3535277471 >> v2:172.21.15.187:3300/0 conn(0x7f9d200d0d80 0x7f9d200b5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:06.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:06.701+0000 7f9d2727f640 1 --2- 172.21.15.112:0/3535277471 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9d200ccc40 0x7f9d2015ca90 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:06.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:06.701+0000 7f9d2727f640 1 -- 172.21.15.112:0/3535277471 >> 172.21.15.112:0/3535277471 conn(0x7f9d200b46e0 msgr2=0x7f9d200d23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:06.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:06.701+0000 7f9d2727f640 1 -- 172.21.15.112:0/3535277471 shutdown_connections 2024-09-21T00:27:06.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:06.701+0000 7f9d2727f640 1 -- 172.21.15.112:0/3535277471 wait complete. 2024-09-21T00:27:06.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:06.703+0000 7f9d2727f640 1 Processor -- start 2024-09-21T00:27:06.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:06.703+0000 7f9d2727f640 1 -- start start 2024-09-21T00:27:06.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:06.704+0000 7f9d2727f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d200ccc40 0x7f9d2006f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:06.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:06.704+0000 7f9d2727f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9d200cd440 0x7f9d2014ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:06.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:06.704+0000 7f9d2727f640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d2015b990 0x7f9d2014f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:06.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:06.704+0000 7f9d2727f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9d200b6650 con 0x7f9d200cd440 2024-09-21T00:27:06.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:06.704+0000 7f9d2727f640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9d200b64d0 con 0x7f9d200ccc40 2024-09-21T00:27:06.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:06.704+0000 7f9d2727f640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f9d200b67d0 con 0x7f9d2015b990 2024-09-21T00:27:06.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:06.704+0000 7f9d2481a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9d200cd440 0x7f9d2014ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:06.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:06.704+0000 7f9d2481a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9d200cd440 0x7f9d2014ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55260/0 (socket says 172.21.15.112:55260) 2024-09-21T00:27:06.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:06.704+0000 7f9d2481a640 1 -- 172.21.15.112:0/551863076 learned_addr learned my addr 172.21.15.112:0/551863076 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:27:06.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:06.704+0000 7f9d2501b640 1 --2- 172.21.15.112:0/551863076 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d200ccc40 0x7f9d2006f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:06.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:06.704+0000 7f9d2581c640 1 --2- 172.21.15.112:0/551863076 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d2015b990 0x7f9d2014f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:06.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:06.704+0000 7f9d2481a640 1 -- 172.21.15.112:0/551863076 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d2015b990 msgr2=0x7f9d2014f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:06.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:06.704+0000 7f9d2481a640 1 --2- 172.21.15.112:0/551863076 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d2015b990 0x7f9d2014f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:06.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:06.704+0000 7f9d2481a640 1 -- 172.21.15.112:0/551863076 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d200ccc40 msgr2=0x7f9d2006f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:06.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:06.704+0000 7f9d2481a640 1 --2- 172.21.15.112:0/551863076 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d200ccc40 0x7f9d2006f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:06.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:06.704+0000 7f9d2481a640 1 -- 172.21.15.112:0/551863076 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d2006f840 con 0x7f9d200cd440 2024-09-21T00:27:06.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:06.705+0000 7f9d2481a640 1 --2- 172.21.15.112:0/551863076 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9d200cd440 0x7f9d2014ef60 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f9d10009fd0 tx=0x7f9d1000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:06.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:06.705+0000 7f9d15ffb640 1 -- 172.21.15.112:0/551863076 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d10015020 con 0x7f9d200cd440 2024-09-21T00:27:06.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:06.705+0000 7f9d15ffb640 1 -- 172.21.15.112:0/551863076 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9d10011920 con 0x7f9d200cd440 2024-09-21T00:27:06.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:06.705+0000 7f9d2727f640 1 -- 172.21.15.112:0/551863076 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d2006e260 con 0x7f9d200cd440 2024-09-21T00:27:06.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:06.705+0000 7f9d15ffb640 1 -- 172.21.15.112:0/551863076 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d10011d50 con 0x7f9d200cd440 2024-09-21T00:27:06.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:06.705+0000 7f9d2727f640 1 -- 172.21.15.112:0/551863076 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d2006e820 con 0x7f9d200cd440 2024-09-21T00:27:06.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:06.706+0000 7f9d2727f640 1 -- 172.21.15.112:0/551863076 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d2006dfd0 con 0x7f9d200cd440 2024-09-21T00:27:06.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:06.707+0000 7f9d15ffb640 1 -- 172.21.15.112:0/551863076 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9d10018020 con 0x7f9d200cd440 2024-09-21T00:27:06.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:06.708+0000 7f9d15ffb640 1 --2- 172.21.15.112:0/551863076 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f9d04078bc0 0x7f9d0407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:06.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:06.708+0000 7f9d15ffb640 1 -- 172.21.15.112:0/551863076 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 6001+0+0 (secure 0 0 0) 0x7f9d1009a7d0 con 0x7f9d200cd440 2024-09-21T00:27:06.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:06.708+0000 7f9d2501b640 1 --2- 172.21.15.112:0/551863076 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f9d04078bc0 0x7f9d0407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:06.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:06.708+0000 7f9d2501b640 1 --2- 172.21.15.112:0/551863076 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f9d04078bc0 0x7f9d0407b0d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f9d1c033000 tx=0x7f9d1c004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:06.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:06.712+0000 7f9d15ffb640 1 -- 172.21.15.112:0/551863076 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f9d1001f070 con 0x7f9d200cd440 2024-09-21T00:27:06.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:06.927+0000 7f9d2727f640 1 -- 172.21.15.112:0/551863076 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f9d200b6d70 con 0x7f9d200cd440 2024-09-21T00:27:07.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:07.566+0000 7f9d15ffb640 1 -- 172.21.15.112:0/551863076 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 7 and data pool 8 v22) ==== 172+0+0 (secure 0 0 0) 0x7f9d10062ae0 con 0x7f9d200cd440 2024-09-21T00:27:07.568 INFO:teuthology.orchestra.run.smithi112.stderr:new fs with metadata pool 7 and data pool 8 2024-09-21T00:27:07.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:07.574+0000 7f9d2727f640 1 -- 172.21.15.112:0/551863076 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f9d04078bc0 msgr2=0x7f9d0407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:07.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:07.574+0000 7f9d2727f640 1 --2- 172.21.15.112:0/551863076 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f9d04078bc0 0x7f9d0407b0d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f9d1c033000 tx=0x7f9d1c004730 comp rx=0 tx=0).stop 2024-09-21T00:27:07.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:07.574+0000 7f9d2727f640 1 -- 172.21.15.112:0/551863076 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9d200cd440 msgr2=0x7f9d2014ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:07.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:07.574+0000 7f9d2727f640 1 --2- 172.21.15.112:0/551863076 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9d200cd440 0x7f9d2014ef60 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f9d10009fd0 tx=0x7f9d1000b870 comp rx=0 tx=0).stop 2024-09-21T00:27:07.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:07.575+0000 7f9d2727f640 1 -- 172.21.15.112:0/551863076 shutdown_connections 2024-09-21T00:27:07.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:07.575+0000 7f9d2727f640 1 --2- 172.21.15.112:0/551863076 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d2015b990 0x7f9d2014f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:07.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:07.575+0000 7f9d2727f640 1 --2- 172.21.15.112:0/551863076 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f9d04078bc0 0x7f9d0407b0d0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:07.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:07.575+0000 7f9d2727f640 1 --2- 172.21.15.112:0/551863076 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9d200cd440 0x7f9d2014ef60 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:07.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:07.575+0000 7f9d2727f640 1 --2- 172.21.15.112:0/551863076 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d200ccc40 0x7f9d2006f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:07.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:07.575+0000 7f9d2727f640 1 -- 172.21.15.112:0/551863076 >> 172.21.15.112:0/551863076 conn(0x7f9d200b46e0 msgr2=0x7f9d200b2a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:07.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:07.578+0000 7f9d2727f640 1 -- 172.21.15.112:0/551863076 shutdown_connections 2024-09-21T00:27:07.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:07.578+0000 7f9d2727f640 1 -- 172.21.15.112:0/551863076 wait complete. 2024-09-21T00:27:07.607 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T00:27:07.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:07.720+0000 7f390f1da640 1 -- 172.21.15.112:0/829859219 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38f0003a00 msgr2=0x7f38f0003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:07.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:07.720+0000 7f390f1da640 1 --2- 172.21.15.112:0/829859219 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38f0003a00 0x7f38f0003e70 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f38f800c470 tx=0x7f38f0005830 comp rx=0 tx=0).stop 2024-09-21T00:27:07.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:07.720+0000 7f390d777640 1 -- 172.21.15.112:0/829859219 reap_dead start 2024-09-21T00:27:07.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:07.720+0000 7f390f1da640 1 -- 172.21.15.112:0/829859219 shutdown_connections 2024-09-21T00:27:07.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:07.720+0000 7f390f1da640 1 -- 172.21.15.112:0/829859219 >> 172.21.15.112:0/829859219 conn(0x7f3908072990 msgr2=0x7f3908072da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:07.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:07.720+0000 7f390f1da640 1 -- 172.21.15.112:0/829859219 shutdown_connections 2024-09-21T00:27:07.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:07.720+0000 7f390f1da640 1 -- 172.21.15.112:0/829859219 wait complete. 2024-09-21T00:27:07.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:07.722+0000 7f390f1da640 1 Processor -- start 2024-09-21T00:27:07.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:07.722+0000 7f390f1da640 1 -- start start 2024-09-21T00:27:07.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:07.723+0000 7f390f1da640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f38f0003a00 0x7f39080caf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:07.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:07.723+0000 7f390f1da640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3908155530 0x7f3908153590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:07.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:07.723+0000 7f390f1da640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3908155940 0x7f3908153ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:07.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:07.723+0000 7f390f1da640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f390815aa80 con 0x7f3908155530 2024-09-21T00:27:07.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:07.723+0000 7f390f1da640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f390815a900 con 0x7f3908155940 2024-09-21T00:27:07.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:07.723+0000 7f390f1da640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f39080c86c0 con 0x7f38f0003a00 2024-09-21T00:27:07.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:07.723+0000 7f3907fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3908155530 0x7f3908153590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:07.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:07.723+0000 7f390cf76640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f38f0003a00 0x7f39080caf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:07.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:07.723+0000 7f3907fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3908155530 0x7f3908153590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55276/0 (socket says 172.21.15.112:55276) 2024-09-21T00:27:07.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:07.723+0000 7f3907fff640 1 -- 172.21.15.112:0/472399067 learned_addr learned my addr 172.21.15.112:0/472399067 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:27:07.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:07.723+0000 7f390d777640 1 --2- 172.21.15.112:0/472399067 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3908155940 0x7f3908153ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:07.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:07.724+0000 7f3907fff640 1 -- 172.21.15.112:0/472399067 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f38f0003a00 msgr2=0x7f39080caf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:07.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:07.724+0000 7f3907fff640 1 --2- 172.21.15.112:0/472399067 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f38f0003a00 0x7f39080caf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:07.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:07.724+0000 7f3907fff640 1 -- 172.21.15.112:0/472399067 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3908155940 msgr2=0x7f3908153ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:07.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:07.724+0000 7f3907fff640 1 --2- 172.21.15.112:0/472399067 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3908155940 0x7f3908153ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:07.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:07.724+0000 7f3907fff640 1 -- 172.21.15.112:0/472399067 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3908156900 con 0x7f3908155530 2024-09-21T00:27:07.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:07.724+0000 7f390cf76640 1 --2- 172.21.15.112:0/472399067 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f38f0003a00 0x7f39080caf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:27:07.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:07.724+0000 7f390d777640 1 --2- 172.21.15.112:0/472399067 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3908155940 0x7f3908153ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:27:07.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:07.724+0000 7f3907fff640 1 --2- 172.21.15.112:0/472399067 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3908155530 0x7f3908153590 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f38fc011d90 tx=0x7f38fc00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:07.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:07.724+0000 7f39057fa640 1 -- 172.21.15.112:0/472399067 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38fc003d80 con 0x7f3908155530 2024-09-21T00:27:07.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:07.724+0000 7f39057fa640 1 -- 172.21.15.112:0/472399067 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f38fc004b20 con 0x7f3908155530 2024-09-21T00:27:07.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:07.724+0000 7f39057fa640 1 -- 172.21.15.112:0/472399067 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38fc004050 con 0x7f3908155530 2024-09-21T00:27:07.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:07.724+0000 7f390f1da640 1 -- 172.21.15.112:0/472399067 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39080c6ea0 con 0x7f3908155530 2024-09-21T00:27:07.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:07.724+0000 7f390f1da640 1 -- 172.21.15.112:0/472399067 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3908156340 con 0x7f3908155530 2024-09-21T00:27:07.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:07.725+0000 7f390f1da640 1 -- 172.21.15.112:0/472399067 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39080d0d90 con 0x7f3908155530 2024-09-21T00:27:07.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:07.726+0000 7f39057fa640 1 -- 172.21.15.112:0/472399067 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f38fc0041f0 con 0x7f3908155530 2024-09-21T00:27:07.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:07.726+0000 7f39057fa640 1 --2- 172.21.15.112:0/472399067 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f38f4078b90 0x7f38f407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:07.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:07.726+0000 7f39057fa640 1 -- 172.21.15.112:0/472399067 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 6101+0+0 (secure 0 0 0) 0x7f38fc09c000 con 0x7f3908155530 2024-09-21T00:27:07.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:07.727+0000 7f390cf76640 1 --2- 172.21.15.112:0/472399067 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f38f4078b90 0x7f38f407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:07.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:07.727+0000 7f390cf76640 1 --2- 172.21.15.112:0/472399067 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f38f4078b90 0x7f38f407b0a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f38f8033000 tx=0x7f38f802d070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:07.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:07.731+0000 7f39057fa640 1 -- 172.21.15.112:0/472399067 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f38fc064340 con 0x7f3908155530 2024-09-21T00:27:07.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:07.908+0000 7f390f1da640 1 -- 172.21.15.112:0/472399067 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f39080cb4b0 con 0x7f3908155530 2024-09-21T00:27:07.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:07.910+0000 7f39057fa640 1 -- 172.21.15.112:0/472399067 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v48) ==== 74+0+17039 (secure 0 0 0) 0x7f38fc063c40 con 0x7f3908155530 2024-09-21T00:27:07.911 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:27:07.912 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":48,"fsid":"8b34692b-dc65-49b2-a08f-26f707b96ac5","created":"2024-09-21T00:25:22.788490+0000","modified":"2024-09-21T00:27:06.929503+0000","last_up_change":"2024-09-21T00:25:36.161835+0000","last_in_change":"2024-09-21T00:25:26.346500+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T00:25:37.105006+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-09-21T00:27:04.569521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-09-21T00:27:05.881104+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9c575086-67d2-4de2-b01b-d44dcc21d604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6809","nonce":755760621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6811","nonce":755760621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6815","nonce":755760621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6813","nonce":755760621}]},"public_addr":"172.21.15.112:6809/755760621","cluster_addr":"172.21.15.112:6811/755760621","heartbeat_back_addr":"172.21.15.112:6815/755760621","heartbeat_front_addr":"172.21.15.112:6813/755760621","state":["exists","up"]},{"osd":1,"uuid":"a983c227-a3ce-4bdd-82f1-438ba7e3ecd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6801","nonce":344932592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6803","nonce":344932592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6807","nonce":344932592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6805","nonce":344932592}]},"public_addr":"172.21.15.112:6801/344932592","cluster_addr":"172.21.15.112:6803/344932592","heartbeat_back_addr":"172.21.15.112:6807/344932592","heartbeat_front_addr":"172.21.15.112:6805/344932592","state":["exists","up"]},{"osd":2,"uuid":"1193c1d8-bfff-4a57-8200-5f69bf1ea3af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6825","nonce":407933616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6827","nonce":407933616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6831","nonce":407933616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6829","nonce":407933616}]},"public_addr":"172.21.15.112:6825/407933616","cluster_addr":"172.21.15.112:6827/407933616","heartbeat_back_addr":"172.21.15.112:6831/407933616","heartbeat_front_addr":"172.21.15.112:6829/407933616","state":["exists","up"]},{"osd":3,"uuid":"af04330a-36ad-40cc-a8dd-fa259b221de5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6817","nonce":4120164806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6819","nonce":4120164806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6823","nonce":4120164806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6821","nonce":4120164806}]},"public_addr":"172.21.15.112:6817/4120164806","cluster_addr":"172.21.15.112:6819/4120164806","heartbeat_back_addr":"172.21.15.112:6823/4120164806","heartbeat_front_addr":"172.21.15.112:6821/4120164806","state":["exists","up"]},{"osd":4,"uuid":"5b893111-292c-4c5a-b184-53ad38c7199e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6803","nonce":522077560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6805","nonce":522077560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6809","nonce":522077560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6807","nonce":522077560}]},"public_addr":"172.21.15.187:6803/522077560","cluster_addr":"172.21.15.187:6805/522077560","heartbeat_back_addr":"172.21.15.187:6809/522077560","heartbeat_front_addr":"172.21.15.187:6807/522077560","state":["exists","up"]},{"osd":5,"uuid":"c0b8e011-45fe-4c5e-925e-6be2a7f1868a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6827","nonce":3593144721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6829","nonce":3593144721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6833","nonce":3593144721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6831","nonce":3593144721}]},"public_addr":"172.21.15.187:6827/3593144721","cluster_addr":"172.21.15.187:6829/3593144721","heartbeat_back_addr":"172.21.15.187:6833/3593144721","heartbeat_front_addr":"172.21.15.187:6831/3593144721","state":["exists","up"]},{"osd":6,"uuid":"87d2d650-7489-4593-98a3-a1f433ce8dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6811","nonce":1494387657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6813","nonce":1494387657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6817","nonce":1494387657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6815","nonce":1494387657}]},"public_addr":"172.21.15.187:6811/1494387657","cluster_addr":"172.21.15.187:6813/1494387657","heartbeat_back_addr":"172.21.15.187:6817/1494387657","heartbeat_front_addr":"172.21.15.187:6815/1494387657","state":["exists","up"]},{"osd":7,"uuid":"c6b4f8e2-a1ee-4438-aa37-67ff4b6ef1f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6819","nonce":1624699842}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6821","nonce":1624699842}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6825","nonce":1624699842}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6823","nonce":1624699842}]},"public_addr":"172.21.15.187:6819/1624699842","cluster_addr":"172.21.15.187:6821/1624699842","heartbeat_back_addr":"172.21.15.187:6825/1624699842","heartbeat_front_addr":"172.21.15.187:6823/1624699842","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.873162+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.400506+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.664317+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.159791+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.202630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.113212+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.384786+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.448441+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T00:27:07.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:07.912+0000 7f390f1da640 1 -- 172.21.15.112:0/472399067 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f38f4078b90 msgr2=0x7f38f407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:07.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:07.912+0000 7f390f1da640 1 --2- 172.21.15.112:0/472399067 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f38f4078b90 0x7f38f407b0a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f38f8033000 tx=0x7f38f802d070 comp rx=0 tx=0).stop 2024-09-21T00:27:07.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:07.912+0000 7f390f1da640 1 -- 172.21.15.112:0/472399067 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3908155530 msgr2=0x7f3908153590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:07.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:07.912+0000 7f390f1da640 1 --2- 172.21.15.112:0/472399067 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3908155530 0x7f3908153590 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f38fc011d90 tx=0x7f38fc00f790 comp rx=0 tx=0).stop 2024-09-21T00:27:07.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:07.912+0000 7f390f1da640 1 -- 172.21.15.112:0/472399067 shutdown_connections 2024-09-21T00:27:07.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:07.912+0000 7f390f1da640 1 --2- 172.21.15.112:0/472399067 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3908155940 0x7f3908153ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:07.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:07.912+0000 7f390f1da640 1 --2- 172.21.15.112:0/472399067 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f38f4078b90 0x7f38f407b0a0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:07.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:07.912+0000 7f390f1da640 1 --2- 172.21.15.112:0/472399067 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3908155530 0x7f3908153590 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:07.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:07.912+0000 7f390f1da640 1 --2- 172.21.15.112:0/472399067 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f38f0003a00 0x7f39080caf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:07.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:07.912+0000 7f390f1da640 1 -- 172.21.15.112:0/472399067 >> 172.21.15.112:0/472399067 conn(0x7f3908072990 msgr2=0x7f3908159a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:07.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:07.912+0000 7f390f1da640 1 -- 172.21.15.112:0/472399067 shutdown_connections 2024-09-21T00:27:07.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:07.912+0000 7f390f1da640 1 -- 172.21.15.112:0/472399067 wait complete. 2024-09-21T00:27:07.925 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T00:27:08.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:08.036+0000 7fc8466a4640 1 -- 172.21.15.112:0/901913597 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc840162280 msgr2=0x7fc840162690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:08.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:08.036+0000 7fc8466a4640 1 --2- 172.21.15.112:0/901913597 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc840162280 0x7fc840162690 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fc834009830 tx=0x7fc83402a4b0 comp rx=0 tx=0).stop 2024-09-21T00:27:08.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:08.036+0000 7fc8466a4640 1 -- 172.21.15.112:0/901913597 shutdown_connections 2024-09-21T00:27:08.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:08.036+0000 7fc8466a4640 1 --2- 172.21.15.112:0/901913597 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc840162280 0x7fc840162690 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:08.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:08.036+0000 7fc8466a4640 1 -- 172.21.15.112:0/901913597 >> 172.21.15.112:0/901913597 conn(0x7fc8400b6b20 msgr2=0x7fc8400d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:08.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:08.036+0000 7fc8466a4640 1 -- 172.21.15.112:0/901913597 shutdown_connections 2024-09-21T00:27:08.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:08.036+0000 7fc8466a4640 1 -- 172.21.15.112:0/901913597 wait complete. 2024-09-21T00:27:08.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:08.038+0000 7fc8466a4640 1 Processor -- start 2024-09-21T00:27:08.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:08.038+0000 7fc8466a4640 1 -- start start 2024-09-21T00:27:08.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:08.039+0000 7fc8466a4640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc8400b2e40 0x7fc8400ce910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:08.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:08.039+0000 7fc8466a4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc840162280 0x7fc8400cee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:08.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:08.039+0000 7fc8466a4640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc8401f88f0 0x7fc8401f8d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:08.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:08.039+0000 7fc8466a4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc84015f640 con 0x7fc840162280 2024-09-21T00:27:08.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:08.039+0000 7fc8466a4640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc84015f4c0 con 0x7fc8401f88f0 2024-09-21T00:27:08.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:08.039+0000 7fc8466a4640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc84015f7c0 con 0x7fc8400b2e40 2024-09-21T00:27:08.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:08.040+0000 7fc83f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc840162280 0x7fc8400cee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:08.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:08.040+0000 7fc83f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc840162280 0x7fc8400cee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55290/0 (socket says 172.21.15.112:55290) 2024-09-21T00:27:08.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:08.040+0000 7fc83f7fe640 1 -- 172.21.15.112:0/3752135000 learned_addr learned my addr 172.21.15.112:0/3752135000 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:27:08.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:08.040+0000 7fc83ffff640 1 --2- 172.21.15.112:0/3752135000 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc8400b2e40 0x7fc8400ce910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:08.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:08.040+0000 7fc844c41640 1 --2- 172.21.15.112:0/3752135000 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc8401f88f0 0x7fc8401f8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:08.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:08.040+0000 7fc83f7fe640 1 -- 172.21.15.112:0/3752135000 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc8400b2e40 msgr2=0x7fc8400ce910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:08.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:08.040+0000 7fc83f7fe640 1 --2- 172.21.15.112:0/3752135000 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc8400b2e40 0x7fc8400ce910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:08.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:08.040+0000 7fc83f7fe640 1 -- 172.21.15.112:0/3752135000 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc8401f88f0 msgr2=0x7fc8401f8d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:08.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:08.040+0000 7fc83f7fe640 1 --2- 172.21.15.112:0/3752135000 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc8401f88f0 0x7fc8401f8d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:08.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:08.040+0000 7fc83f7fe640 1 -- 172.21.15.112:0/3752135000 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8400ad380 con 0x7fc840162280 2024-09-21T00:27:08.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:08.040+0000 7fc83f7fe640 1 --2- 172.21.15.112:0/3752135000 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc840162280 0x7fc8400cee50 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fc834009950 tx=0x7fc834003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:08.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:08.040+0000 7fc83cff9640 1 -- 172.21.15.112:0/3752135000 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc834005c90 con 0x7fc840162280 2024-09-21T00:27:08.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:08.040+0000 7fc83cff9640 1 -- 172.21.15.112:0/3752135000 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc834005020 con 0x7fc840162280 2024-09-21T00:27:08.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:08.040+0000 7fc8466a4640 1 -- 172.21.15.112:0/3752135000 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8400ad660 con 0x7fc840162280 2024-09-21T00:27:08.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:08.040+0000 7fc83cff9640 1 -- 172.21.15.112:0/3752135000 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc834005450 con 0x7fc840162280 2024-09-21T00:27:08.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:08.040+0000 7fc8466a4640 1 -- 172.21.15.112:0/3752135000 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8400adc20 con 0x7fc840162280 2024-09-21T00:27:08.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:08.041+0000 7fc8466a4640 1 -- 172.21.15.112:0/3752135000 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8400ad0f0 con 0x7fc840162280 2024-09-21T00:27:08.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:08.042+0000 7fc83cff9640 1 -- 172.21.15.112:0/3752135000 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc8340317c0 con 0x7fc840162280 2024-09-21T00:27:08.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:08.042+0000 7fc83cff9640 1 --2- 172.21.15.112:0/3752135000 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fc82c078b90 0x7fc82c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:08.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:08.043+0000 7fc83cff9640 1 -- 172.21.15.112:0/3752135000 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 6101+0+0 (secure 0 0 0) 0x7fc8340bbf30 con 0x7fc840162280 2024-09-21T00:27:08.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:08.043+0000 7fc83ffff640 1 --2- 172.21.15.112:0/3752135000 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fc82c078b90 0x7fc82c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:08.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:08.043+0000 7fc83ffff640 1 --2- 172.21.15.112:0/3752135000 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fc82c078b90 0x7fc82c07b0a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc830001590 tx=0x7fc830001830 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:08.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:08.047+0000 7fc83cff9640 1 -- 172.21.15.112:0/3752135000 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc83403f030 con 0x7fc840162280 2024-09-21T00:27:08.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:08.227+0000 7fc8466a4640 1 -- 172.21.15.112:0/3752135000 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc840162f00 con 0x7fc840162280 2024-09-21T00:27:08.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:08.228+0000 7fc83cff9640 1 -- 172.21.15.112:0/3752135000 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v48) ==== 74+0+17039 (secure 0 0 0) 0x7fc834083ba0 con 0x7fc840162280 2024-09-21T00:27:08.230 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:27:08.231 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":48,"fsid":"8b34692b-dc65-49b2-a08f-26f707b96ac5","created":"2024-09-21T00:25:22.788490+0000","modified":"2024-09-21T00:27:06.929503+0000","last_up_change":"2024-09-21T00:25:36.161835+0000","last_in_change":"2024-09-21T00:25:26.346500+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T00:25:37.105006+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-09-21T00:27:04.569521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-09-21T00:27:05.881104+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9c575086-67d2-4de2-b01b-d44dcc21d604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6809","nonce":755760621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6811","nonce":755760621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6815","nonce":755760621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6813","nonce":755760621}]},"public_addr":"172.21.15.112:6809/755760621","cluster_addr":"172.21.15.112:6811/755760621","heartbeat_back_addr":"172.21.15.112:6815/755760621","heartbeat_front_addr":"172.21.15.112:6813/755760621","state":["exists","up"]},{"osd":1,"uuid":"a983c227-a3ce-4bdd-82f1-438ba7e3ecd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6801","nonce":344932592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6803","nonce":344932592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6807","nonce":344932592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6805","nonce":344932592}]},"public_addr":"172.21.15.112:6801/344932592","cluster_addr":"172.21.15.112:6803/344932592","heartbeat_back_addr":"172.21.15.112:6807/344932592","heartbeat_front_addr":"172.21.15.112:6805/344932592","state":["exists","up"]},{"osd":2,"uuid":"1193c1d8-bfff-4a57-8200-5f69bf1ea3af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6825","nonce":407933616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6827","nonce":407933616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6831","nonce":407933616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6829","nonce":407933616}]},"public_addr":"172.21.15.112:6825/407933616","cluster_addr":"172.21.15.112:6827/407933616","heartbeat_back_addr":"172.21.15.112:6831/407933616","heartbeat_front_addr":"172.21.15.112:6829/407933616","state":["exists","up"]},{"osd":3,"uuid":"af04330a-36ad-40cc-a8dd-fa259b221de5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6817","nonce":4120164806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6819","nonce":4120164806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6823","nonce":4120164806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6821","nonce":4120164806}]},"public_addr":"172.21.15.112:6817/4120164806","cluster_addr":"172.21.15.112:6819/4120164806","heartbeat_back_addr":"172.21.15.112:6823/4120164806","heartbeat_front_addr":"172.21.15.112:6821/4120164806","state":["exists","up"]},{"osd":4,"uuid":"5b893111-292c-4c5a-b184-53ad38c7199e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6803","nonce":522077560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6805","nonce":522077560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6809","nonce":522077560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6807","nonce":522077560}]},"public_addr":"172.21.15.187:6803/522077560","cluster_addr":"172.21.15.187:6805/522077560","heartbeat_back_addr":"172.21.15.187:6809/522077560","heartbeat_front_addr":"172.21.15.187:6807/522077560","state":["exists","up"]},{"osd":5,"uuid":"c0b8e011-45fe-4c5e-925e-6be2a7f1868a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6827","nonce":3593144721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6829","nonce":3593144721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6833","nonce":3593144721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6831","nonce":3593144721}]},"public_addr":"172.21.15.187:6827/3593144721","cluster_addr":"172.21.15.187:6829/3593144721","heartbeat_back_addr":"172.21.15.187:6833/3593144721","heartbeat_front_addr":"172.21.15.187:6831/3593144721","state":["exists","up"]},{"osd":6,"uuid":"87d2d650-7489-4593-98a3-a1f433ce8dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6811","nonce":1494387657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6813","nonce":1494387657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6817","nonce":1494387657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6815","nonce":1494387657}]},"public_addr":"172.21.15.187:6811/1494387657","cluster_addr":"172.21.15.187:6813/1494387657","heartbeat_back_addr":"172.21.15.187:6817/1494387657","heartbeat_front_addr":"172.21.15.187:6815/1494387657","state":["exists","up"]},{"osd":7,"uuid":"c6b4f8e2-a1ee-4438-aa37-67ff4b6ef1f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6819","nonce":1624699842}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6821","nonce":1624699842}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6825","nonce":1624699842}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6823","nonce":1624699842}]},"public_addr":"172.21.15.187:6819/1624699842","cluster_addr":"172.21.15.187:6821/1624699842","heartbeat_back_addr":"172.21.15.187:6825/1624699842","heartbeat_front_addr":"172.21.15.187:6823/1624699842","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.873162+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.400506+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.664317+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.159791+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.202630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.113212+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.384786+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.448441+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T00:27:08.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:08.231+0000 7fc8466a4640 1 -- 172.21.15.112:0/3752135000 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fc82c078b90 msgr2=0x7fc82c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:08.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:08.231+0000 7fc8466a4640 1 --2- 172.21.15.112:0/3752135000 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fc82c078b90 0x7fc82c07b0a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc830001590 tx=0x7fc830001830 comp rx=0 tx=0).stop 2024-09-21T00:27:08.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:08.231+0000 7fc8466a4640 1 -- 172.21.15.112:0/3752135000 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc840162280 msgr2=0x7fc8400cee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:08.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:08.231+0000 7fc8466a4640 1 --2- 172.21.15.112:0/3752135000 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc840162280 0x7fc8400cee50 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fc834009950 tx=0x7fc834003c10 comp rx=0 tx=0).stop 2024-09-21T00:27:08.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:08.231+0000 7fc8466a4640 1 -- 172.21.15.112:0/3752135000 shutdown_connections 2024-09-21T00:27:08.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:08.231+0000 7fc8466a4640 1 --2- 172.21.15.112:0/3752135000 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc8401f88f0 0x7fc8401f8d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:08.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:08.231+0000 7fc8466a4640 1 --2- 172.21.15.112:0/3752135000 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fc82c078b90 0x7fc82c07b0a0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:08.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:08.231+0000 7fc8466a4640 1 --2- 172.21.15.112:0/3752135000 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc840162280 0x7fc8400cee50 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:08.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:08.231+0000 7fc8466a4640 1 --2- 172.21.15.112:0/3752135000 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc8400b2e40 0x7fc8400ce910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:08.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:08.231+0000 7fc8466a4640 1 -- 172.21.15.112:0/3752135000 >> 172.21.15.112:0/3752135000 conn(0x7fc8400b6b20 msgr2=0x7fc840068f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:08.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:08.231+0000 7fc8466a4640 1 -- 172.21.15.112:0/3752135000 shutdown_connections 2024-09-21T00:27:08.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:08.231+0000 7fc8466a4640 1 -- 172.21.15.112:0/3752135000 wait complete. 2024-09-21T00:27:08.244 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-21T00:27:08.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:08.357+0000 7f0298c7e640 1 -- 172.21.15.112:0/3055054098 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02940d0170 msgr2=0x7f02940d0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:08.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:08.357+0000 7f0298c7e640 1 --2- 172.21.15.112:0/3055054098 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02940d0170 0x7f02940d0be0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f02940d2080 tx=0x7f028802a420 comp rx=0 tx=0).stop 2024-09-21T00:27:08.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:08.357+0000 7f0298c7e640 1 -- 172.21.15.112:0/3055054098 shutdown_connections 2024-09-21T00:27:08.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:08.357+0000 7f0298c7e640 1 --2- 172.21.15.112:0/3055054098 >> v2:172.21.15.187:3300/0 conn(0x7f02940d1120 0x7f02940ceb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:08.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:08.357+0000 7f0298c7e640 1 --2- 172.21.15.112:0/3055054098 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02940d0170 0x7f02940d0be0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:08.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:08.357+0000 7f0298c7e640 1 -- 172.21.15.112:0/3055054098 >> 172.21.15.112:0/3055054098 conn(0x7f029409d830 msgr2=0x7f02940b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:08.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:08.357+0000 7f0298c7e640 1 -- 172.21.15.112:0/3055054098 shutdown_connections 2024-09-21T00:27:08.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:08.357+0000 7f0298c7e640 1 -- 172.21.15.112:0/3055054098 wait complete. 2024-09-21T00:27:08.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:08.359+0000 7f0298c7e640 1 Processor -- start 2024-09-21T00:27:08.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:08.359+0000 7f0298c7e640 1 -- start start 2024-09-21T00:27:08.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:08.360+0000 7f0298c7e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02940d0170 0x7f02940cb610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:08.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:08.360+0000 7f0298c7e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f02940d1120 0x7f029414f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:08.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:08.360+0000 7f0298c7e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f029415ee80 0x7f02941f5df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:08.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:08.360+0000 7f0298c7e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f02940b6700 con 0x7f02940d0170 2024-09-21T00:27:08.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:08.360+0000 7f0298c7e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f02940b6580 con 0x7f02940d1120 2024-09-21T00:27:08.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:08.360+0000 7f0298c7e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f02940b6880 con 0x7f029415ee80 2024-09-21T00:27:08.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:08.361+0000 7f029259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02940d0170 0x7f02940cb610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:08.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:08.361+0000 7f029259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02940d0170 0x7f02940cb610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55320/0 (socket says 172.21.15.112:55320) 2024-09-21T00:27:08.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:08.361+0000 7f029259c640 1 -- 172.21.15.112:0/3813352547 learned_addr learned my addr 172.21.15.112:0/3813352547 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:27:08.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:08.361+0000 7f0292d9d640 1 --2- 172.21.15.112:0/3813352547 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f029415ee80 0x7f02941f5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:08.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:08.361+0000 7f0291d9b640 1 --2- 172.21.15.112:0/3813352547 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f02940d1120 0x7f029414f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:08.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:08.361+0000 7f029259c640 1 -- 172.21.15.112:0/3813352547 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f029415ee80 msgr2=0x7f02941f5df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:08.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:08.361+0000 7f029259c640 1 --2- 172.21.15.112:0/3813352547 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f029415ee80 0x7f02941f5df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:08.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:08.361+0000 7f029259c640 1 -- 172.21.15.112:0/3813352547 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f02940d1120 msgr2=0x7f029414f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:08.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:08.361+0000 7f029259c640 1 --2- 172.21.15.112:0/3813352547 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f02940d1120 0x7f029414f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:08.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:08.361+0000 7f029259c640 1 -- 172.21.15.112:0/3813352547 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02941f93b0 con 0x7f02940d0170 2024-09-21T00:27:08.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:08.361+0000 7f0292d9d640 1 --2- 172.21.15.112:0/3813352547 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f029415ee80 0x7f02941f5df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:27:08.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:08.361+0000 7f0291d9b640 1 --2- 172.21.15.112:0/3813352547 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f02940d1120 0x7f029414f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:27:08.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:08.361+0000 7f029259c640 1 --2- 172.21.15.112:0/3813352547 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02940d0170 0x7f02940cb610 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f028400b3f0 tx=0x7f028400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:08.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:08.361+0000 7f0282ffd640 1 -- 172.21.15.112:0/3813352547 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0284015020 con 0x7f02940d0170 2024-09-21T00:27:08.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:08.361+0000 7f0282ffd640 1 -- 172.21.15.112:0/3813352547 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0284011920 con 0x7f02940d0170 2024-09-21T00:27:08.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:08.361+0000 7f0298c7e640 1 -- 172.21.15.112:0/3813352547 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02941f9690 con 0x7f02940d0170 2024-09-21T00:27:08.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:08.361+0000 7f0282ffd640 1 -- 172.21.15.112:0/3813352547 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0284011d70 con 0x7f02940d0170 2024-09-21T00:27:08.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:08.361+0000 7f0298c7e640 1 -- 172.21.15.112:0/3813352547 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02940cbc50 con 0x7f02940d0170 2024-09-21T00:27:08.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:08.362+0000 7f0298c7e640 1 -- 172.21.15.112:0/3813352547 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02941f9bb0 con 0x7f02940d0170 2024-09-21T00:27:08.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:08.363+0000 7f0282ffd640 1 -- 172.21.15.112:0/3813352547 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0284018020 con 0x7f02940d0170 2024-09-21T00:27:08.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:08.364+0000 7f0282ffd640 1 --2- 172.21.15.112:0/3813352547 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f0274078b90 0x7f027407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:08.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:08.364+0000 7f0282ffd640 1 -- 172.21.15.112:0/3813352547 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 6101+0+0 (secure 0 0 0) 0x7f028409aee0 con 0x7f02940d0170 2024-09-21T00:27:08.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:08.364+0000 7f0291d9b640 1 --2- 172.21.15.112:0/3813352547 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f0274078b90 0x7f027407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:08.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:08.365+0000 7f0291d9b640 1 --2- 172.21.15.112:0/3813352547 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f0274078b90 0x7f027407b0a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f0288034000 tx=0x7f0288004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:08.370 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:08.369+0000 7f0282ffd640 1 -- 172.21.15.112:0/3813352547 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f02941f9bb0 con 0x7f02940d0170 2024-09-21T00:27:08.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:08.586+0000 7f0298c7e640 1 -- 172.21.15.112:0/3813352547 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f02940b9d80 con 0x7f02940d0170 2024-09-21T00:27:09.550 INFO:tasks.ceph.mds.a.smithi112.stderr:2024-09-21T00:27:09.548+0000 7f53a341a640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T00:27:09.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:09.579+0000 7f0282ffd640 1 -- 172.21.15.112:0/3813352547 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v25) ==== 118+0+0 (secure 0 0 0) 0x7f02940b9d80 con 0x7f02940d0170 2024-09-21T00:27:09.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:09.583+0000 7f0298c7e640 1 -- 172.21.15.112:0/3813352547 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f0274078b90 msgr2=0x7f027407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:09.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:09.583+0000 7f0298c7e640 1 --2- 172.21.15.112:0/3813352547 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f0274078b90 0x7f027407b0a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f0288034000 tx=0x7f0288004730 comp rx=0 tx=0).stop 2024-09-21T00:27:09.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:09.583+0000 7f0298c7e640 1 -- 172.21.15.112:0/3813352547 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02940d0170 msgr2=0x7f02940cb610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:09.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:09.583+0000 7f0298c7e640 1 --2- 172.21.15.112:0/3813352547 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02940d0170 0x7f02940cb610 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f028400b3f0 tx=0x7f028400b8b0 comp rx=0 tx=0).stop 2024-09-21T00:27:09.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:09.583+0000 7f0298c7e640 1 -- 172.21.15.112:0/3813352547 shutdown_connections 2024-09-21T00:27:09.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:09.583+0000 7f0298c7e640 1 --2- 172.21.15.112:0/3813352547 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f029415ee80 0x7f02941f5df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:09.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:09.583+0000 7f0298c7e640 1 --2- 172.21.15.112:0/3813352547 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f02940d1120 0x7f029414f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:09.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:09.583+0000 7f0298c7e640 1 --2- 172.21.15.112:0/3813352547 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f0274078b90 0x7f027407b0a0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:09.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:09.583+0000 7f0298c7e640 1 --2- 172.21.15.112:0/3813352547 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02940d0170 0x7f02940cb610 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:09.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:09.583+0000 7f0298c7e640 1 -- 172.21.15.112:0/3813352547 >> 172.21.15.112:0/3813352547 conn(0x7f029409d830 msgr2=0x7f029415df50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:09.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:09.583+0000 7f0298c7e640 1 -- 172.21.15.112:0/3813352547 shutdown_connections 2024-09-21T00:27:09.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:09.583+0000 7f0298c7e640 1 -- 172.21.15.112:0/3813352547 wait complete. 2024-09-21T00:27:09.599 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-21T00:27:09.599 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-21T00:27:09.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:09.720+0000 7feb9a884640 1 -- 172.21.15.112:0/4273482495 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb940b5260 msgr2=0x7feb9415ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:09.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:09.720+0000 7feb9a884640 1 --2- 172.21.15.112:0/4273482495 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb940b5260 0x7feb9415ba80 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7feb94073190 tx=0x7feb7c02a530 comp rx=0 tx=0).stop 2024-09-21T00:27:09.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:09.720+0000 7feb9a884640 1 -- 172.21.15.112:0/4273482495 shutdown_connections 2024-09-21T00:27:09.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:09.720+0000 7feb9a884640 1 --2- 172.21.15.112:0/4273482495 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb940b5260 0x7feb9415ba80 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:09.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:09.720+0000 7feb9a884640 1 --2- 172.21.15.112:0/4273482495 >> v2:172.21.15.187:3300/0 conn(0x7feb9406def0 0x7feb9406f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:09.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:09.720+0000 7feb9a884640 1 -- 172.21.15.112:0/4273482495 >> 172.21.15.112:0/4273482495 conn(0x7feb940cdca0 msgr2=0x7feb940ce0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:09.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:09.720+0000 7feb9a884640 1 -- 172.21.15.112:0/4273482495 shutdown_connections 2024-09-21T00:27:09.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:09.720+0000 7feb9a884640 1 -- 172.21.15.112:0/4273482495 wait complete. 2024-09-21T00:27:09.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:09.722+0000 7feb9a884640 1 Processor -- start 2024-09-21T00:27:09.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:09.722+0000 7feb9a884640 1 -- start start 2024-09-21T00:27:09.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:09.723+0000 7feb9a884640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feb9406def0 0x7feb941f6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:09.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:09.723+0000 7feb9a884640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb940b5260 0x7feb94149160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:09.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:09.723+0000 7feb9a884640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feb9415bfc0 0x7feb940b20e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:09.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:09.723+0000 7feb9a884640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7feb940b5a00 con 0x7feb940b5260 2024-09-21T00:27:09.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:09.723+0000 7feb9a884640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7feb940b5880 con 0x7feb9415bfc0 2024-09-21T00:27:09.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:09.723+0000 7feb9a884640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7feb94071300 con 0x7feb9406def0 2024-09-21T00:27:09.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:09.723+0000 7feb8bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb940b5260 0x7feb94149160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:09.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:09.723+0000 7feb8bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb940b5260 0x7feb94149160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55332/0 (socket says 172.21.15.112:55332) 2024-09-21T00:27:09.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:09.723+0000 7feb8bfff640 1 -- 172.21.15.112:0/1478373197 learned_addr learned my addr 172.21.15.112:0/1478373197 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:27:09.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:09.724+0000 7feb93fff640 1 --2- 172.21.15.112:0/1478373197 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feb9406def0 0x7feb941f6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:09.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:09.724+0000 7feb98e21640 1 --2- 172.21.15.112:0/1478373197 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feb9415bfc0 0x7feb940b20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:09.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:09.724+0000 7feb8bfff640 1 -- 172.21.15.112:0/1478373197 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feb9406def0 msgr2=0x7feb941f6460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:09.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:09.724+0000 7feb8bfff640 1 --2- 172.21.15.112:0/1478373197 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feb9406def0 0x7feb941f6460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:09.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:09.724+0000 7feb8bfff640 1 -- 172.21.15.112:0/1478373197 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feb9415bfc0 msgr2=0x7feb940b20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:09.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:09.724+0000 7feb8bfff640 1 --2- 172.21.15.112:0/1478373197 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feb9415bfc0 0x7feb940b20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:09.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:09.724+0000 7feb8bfff640 1 -- 172.21.15.112:0/1478373197 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb941f8f90 con 0x7feb940b5260 2024-09-21T00:27:09.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:09.724+0000 7feb8bfff640 1 --2- 172.21.15.112:0/1478373197 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb940b5260 0x7feb94149160 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7feb7c02ab20 tx=0x7feb7c033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:09.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:09.724+0000 7feb917fa640 1 -- 172.21.15.112:0/1478373197 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb7c031d60 con 0x7feb940b5260 2024-09-21T00:27:09.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:09.724+0000 7feb917fa640 1 -- 172.21.15.112:0/1478373197 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feb7c033e10 con 0x7feb940b5260 2024-09-21T00:27:09.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:09.724+0000 7feb9a884640 1 -- 172.21.15.112:0/1478373197 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb94156990 con 0x7feb940b5260 2024-09-21T00:27:09.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:09.724+0000 7feb917fa640 1 -- 172.21.15.112:0/1478373197 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb7c005260 con 0x7feb940b5260 2024-09-21T00:27:09.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:09.724+0000 7feb9a884640 1 -- 172.21.15.112:0/1478373197 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb941f9510 con 0x7feb940b5260 2024-09-21T00:27:09.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:09.725+0000 7feb9a884640 1 -- 172.21.15.112:0/1478373197 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb9415ba80 con 0x7feb940b5260 2024-09-21T00:27:09.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:09.726+0000 7feb917fa640 1 -- 172.21.15.112:0/1478373197 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7feb7c005400 con 0x7feb940b5260 2024-09-21T00:27:09.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:09.726+0000 7feb917fa640 1 --2- 172.21.15.112:0/1478373197 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7feb74078b90 0x7feb7407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:09.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:09.727+0000 7feb917fa640 1 -- 172.21.15.112:0/1478373197 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 6101+0+0 (secure 0 0 0) 0x7feb7c0bc530 con 0x7feb940b5260 2024-09-21T00:27:09.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:09.727+0000 7feb93fff640 1 --2- 172.21.15.112:0/1478373197 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7feb74078b90 0x7feb7407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:09.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:09.727+0000 7feb93fff640 1 --2- 172.21.15.112:0/1478373197 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7feb74078b90 0x7feb7407b0a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7feb84005d10 tx=0x7feb84005ca0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:09.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:09.730+0000 7feb917fa640 1 -- 172.21.15.112:0/1478373197 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7feb7c03f030 con 0x7feb940b5260 2024-09-21T00:27:09.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:09.943+0000 7feb9a884640 1 -- 172.21.15.112:0/1478373197 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7feb94156720 con 0x7feb940b5260 2024-09-21T00:27:10.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:10.580+0000 7feb917fa640 1 -- 172.21.15.112:0/1478373197 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v26) ==== 122+0+0 (secure 0 0 0) 0x7feb7c0849e0 con 0x7feb940b5260 2024-09-21T00:27:10.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:10.583+0000 7feb9a884640 1 -- 172.21.15.112:0/1478373197 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7feb74078b90 msgr2=0x7feb7407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:10.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:10.583+0000 7feb9a884640 1 --2- 172.21.15.112:0/1478373197 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7feb74078b90 0x7feb7407b0a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7feb84005d10 tx=0x7feb84005ca0 comp rx=0 tx=0).stop 2024-09-21T00:27:10.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:10.583+0000 7feb9a884640 1 -- 172.21.15.112:0/1478373197 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb940b5260 msgr2=0x7feb94149160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:10.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:10.583+0000 7feb9a884640 1 --2- 172.21.15.112:0/1478373197 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb940b5260 0x7feb94149160 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7feb7c02ab20 tx=0x7feb7c033c20 comp rx=0 tx=0).stop 2024-09-21T00:27:10.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:10.583+0000 7feb9a884640 1 -- 172.21.15.112:0/1478373197 shutdown_connections 2024-09-21T00:27:10.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:10.583+0000 7feb9a884640 1 --2- 172.21.15.112:0/1478373197 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feb9415bfc0 0x7feb940b20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:10.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:10.583+0000 7feb9a884640 1 --2- 172.21.15.112:0/1478373197 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7feb74078b90 0x7feb7407b0a0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:10.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:10.583+0000 7feb9a884640 1 --2- 172.21.15.112:0/1478373197 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb940b5260 0x7feb94149160 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:10.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:10.583+0000 7feb9a884640 1 --2- 172.21.15.112:0/1478373197 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feb9406def0 0x7feb941f6460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:10.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:10.583+0000 7feb9a884640 1 -- 172.21.15.112:0/1478373197 >> 172.21.15.112:0/1478373197 conn(0x7feb940cdca0 msgr2=0x7feb940717f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:10.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:10.583+0000 7feb9a884640 1 -- 172.21.15.112:0/1478373197 shutdown_connections 2024-09-21T00:27:10.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:10.583+0000 7feb9a884640 1 -- 172.21.15.112:0/1478373197 wait complete. 2024-09-21T00:27:10.599 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-21T00:27:10.600 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T00:27:10.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:10.716+0000 7f437bc1f640 1 -- 172.21.15.112:0/2672528838 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f437415cb10 msgr2=0x7f437415cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:10.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:10.716+0000 7f437bc1f640 1 --2- 172.21.15.112:0/2672528838 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f437415cb10 0x7f437415cf20 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f4370005fd0 tx=0x7f437002a5d0 comp rx=0 tx=0).stop 2024-09-21T00:27:10.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:10.716+0000 7f437bc1f640 1 -- 172.21.15.112:0/2672528838 shutdown_connections 2024-09-21T00:27:10.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:10.716+0000 7f437bc1f640 1 --2- 172.21.15.112:0/2672528838 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f43740b6250 0x7f43740b6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:10.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:10.716+0000 7f437bc1f640 1 --2- 172.21.15.112:0/2672528838 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f437415cb10 0x7f437415cf20 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:10.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:10.716+0000 7f437bc1f640 1 -- 172.21.15.112:0/2672528838 >> 172.21.15.112:0/2672528838 conn(0x7f43740d2350 msgr2=0x7f43740d2760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:10.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:10.717+0000 7f437bc1f640 1 -- 172.21.15.112:0/2672528838 shutdown_connections 2024-09-21T00:27:10.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:10.717+0000 7f437bc1f640 1 -- 172.21.15.112:0/2672528838 wait complete. 2024-09-21T00:27:10.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:10.718+0000 7f437bc1f640 1 Processor -- start 2024-09-21T00:27:10.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:10.718+0000 7f437bc1f640 1 -- start start 2024-09-21T00:27:10.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:10.720+0000 7f437bc1f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f43740b6250 0x7f437415beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:10.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:10.720+0000 7f437bc1f640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f43740d12e0 0x7f4374159160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:10.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:10.720+0000 7f437bc1f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f437415cb10 0x7f437415e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:10.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:10.720+0000 7f437bc1f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f43740b6f30 con 0x7f43740b6250 2024-09-21T00:27:10.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:10.720+0000 7f437bc1f640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f43740b6db0 con 0x7f437415cb10 2024-09-21T00:27:10.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:10.720+0000 7f437bc1f640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f43740b70b0 con 0x7f43740d12e0 2024-09-21T00:27:10.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:10.720+0000 7f43799bb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f43740b6250 0x7f437415beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:10.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:10.720+0000 7f43799bb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f43740b6250 0x7f437415beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55352/0 (socket says 172.21.15.112:55352) 2024-09-21T00:27:10.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:10.720+0000 7f43799bb640 1 -- 172.21.15.112:0/812667799 learned_addr learned my addr 172.21.15.112:0/812667799 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:27:10.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:10.720+0000 7f43791ba640 1 --2- 172.21.15.112:0/812667799 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f43740d12e0 0x7f4374159160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:10.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:10.720+0000 7f437a1bc640 1 --2- 172.21.15.112:0/812667799 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f437415cb10 0x7f437415e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:10.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:10.720+0000 7f43799bb640 1 -- 172.21.15.112:0/812667799 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f43740d12e0 msgr2=0x7f4374159160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:10.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:10.720+0000 7f43799bb640 1 --2- 172.21.15.112:0/812667799 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f43740d12e0 0x7f4374159160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:10.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:10.720+0000 7f43799bb640 1 -- 172.21.15.112:0/812667799 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f437415cb10 msgr2=0x7f437415e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:10.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:10.720+0000 7f43799bb640 1 --2- 172.21.15.112:0/812667799 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f437415cb10 0x7f437415e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:10.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:10.720+0000 7f43799bb640 1 -- 172.21.15.112:0/812667799 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43741f7420 con 0x7f43740b6250 2024-09-21T00:27:10.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:10.720+0000 7f43799bb640 1 --2- 172.21.15.112:0/812667799 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f43740b6250 0x7f437415beb0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f4370006010 tx=0x7f43700038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:10.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:10.720+0000 7f436a7fc640 1 -- 172.21.15.112:0/812667799 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f437003e030 con 0x7f43740b6250 2024-09-21T00:27:10.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:10.720+0000 7f436a7fc640 1 -- 172.21.15.112:0/812667799 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4370003ca0 con 0x7f43740b6250 2024-09-21T00:27:10.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:10.720+0000 7f437bc1f640 1 -- 172.21.15.112:0/812667799 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43741f7700 con 0x7f43740b6250 2024-09-21T00:27:10.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:10.720+0000 7f436a7fc640 1 -- 172.21.15.112:0/812667799 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4370035040 con 0x7f43740b6250 2024-09-21T00:27:10.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:10.721+0000 7f437bc1f640 1 -- 172.21.15.112:0/812667799 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43741f7cc0 con 0x7f43740b6250 2024-09-21T00:27:10.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:10.721+0000 7f437bc1f640 1 -- 172.21.15.112:0/812667799 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f437415f7f0 con 0x7f43740b6250 2024-09-21T00:27:10.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:10.722+0000 7f436a7fc640 1 -- 172.21.15.112:0/812667799 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f4370041030 con 0x7f43740b6250 2024-09-21T00:27:10.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:10.723+0000 7f436a7fc640 1 --2- 172.21.15.112:0/812667799 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4358078bc0 0x7f435807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:10.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:10.723+0000 7f436a7fc640 1 -- 172.21.15.112:0/812667799 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 6101+0+0 (secure 0 0 0) 0x7f43700bb760 con 0x7f43740b6250 2024-09-21T00:27:10.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:10.723+0000 7f43791ba640 1 --2- 172.21.15.112:0/812667799 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4358078bc0 0x7f435807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:10.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:10.724+0000 7f43791ba640 1 --2- 172.21.15.112:0/812667799 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4358078bc0 0x7f435807b0d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f43741f6e10 tx=0x7f4364009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:10.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:10.727+0000 7f436a7fc640 1 -- 172.21.15.112:0/812667799 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f437415f7f0 con 0x7f43740b6250 2024-09-21T00:27:10.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:10.939+0000 7f437bc1f640 1 -- 172.21.15.112:0/812667799 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f43740b6660 con 0x7f43740b6250 2024-09-21T00:27:10.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:10.940+0000 7f436a7fc640 1 -- 172.21.15.112:0/812667799 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5229 (secure 0 0 0) 0x7f4370083530 con 0x7f43740b6250 2024-09-21T00:27:10.942 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:27:10.942 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":26,"btime":"2024-09-21T00:27:10:580658+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5158,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6836/2010548530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":2010548530},{"type":"v1","addr":"172.21.15.187:6836","nonce":2010548530}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5161,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6837/2290459931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6835","nonce":2290459931},{"type":"v1","addr":"172.21.15.187:6837","nonce":2290459931}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5400,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6834/158782340","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":158782340},{"type":"v1","addr":"172.21.15.112:6834","nonce":158782340}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T00:27:06.929464+0000","modified":"2024-09-21T00:27:09.945226+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5403},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5403":{"gid":5403,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6835/722271164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":722271164},{"type":"v1","addr":"172.21.15.112:6835","nonce":722271164}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5403,"qdb_cluster":[5403]},"id":3}]} 2024-09-21T00:27:10.943 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 26 2024-09-21T00:27:10.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:10.942+0000 7f437bc1f640 1 -- 172.21.15.112:0/812667799 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4358078bc0 msgr2=0x7f435807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:10.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:10.942+0000 7f437bc1f640 1 --2- 172.21.15.112:0/812667799 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4358078bc0 0x7f435807b0d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f43741f6e10 tx=0x7f4364009040 comp rx=0 tx=0).stop 2024-09-21T00:27:10.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:10.942+0000 7f437bc1f640 1 -- 172.21.15.112:0/812667799 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f43740b6250 msgr2=0x7f437415beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:10.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:10.942+0000 7f437bc1f640 1 --2- 172.21.15.112:0/812667799 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f43740b6250 0x7f437415beb0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f4370006010 tx=0x7f43700038a0 comp rx=0 tx=0).stop 2024-09-21T00:27:10.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:10.942+0000 7f437bc1f640 1 -- 172.21.15.112:0/812667799 shutdown_connections 2024-09-21T00:27:10.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:10.942+0000 7f437bc1f640 1 --2- 172.21.15.112:0/812667799 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f437415cb10 0x7f437415e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:10.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:10.942+0000 7f437bc1f640 1 --2- 172.21.15.112:0/812667799 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f43740d12e0 0x7f4374159160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:10.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:10.942+0000 7f437bc1f640 1 --2- 172.21.15.112:0/812667799 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4358078bc0 0x7f435807b0d0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:10.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:10.942+0000 7f437bc1f640 1 --2- 172.21.15.112:0/812667799 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f43740b6250 0x7f437415beb0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:10.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:10.942+0000 7f437bc1f640 1 -- 172.21.15.112:0/812667799 >> 172.21.15.112:0/812667799 conn(0x7f43740d2350 msgr2=0x7f43740d2760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:10.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:10.942+0000 7f437bc1f640 1 -- 172.21.15.112:0/812667799 shutdown_connections 2024-09-21T00:27:10.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:10.942+0000 7f437bc1f640 1 -- 172.21.15.112:0/812667799 wait complete. 2024-09-21T00:27:10.955 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T00:27:11.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.066+0000 7fbf67319640 1 -- 172.21.15.112:0/3542191231 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf4c003a50 msgr2=0x7fbf4c003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:11.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.066+0000 7fbf67319640 1 --2- 172.21.15.112:0/3542191231 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf4c003a50 0x7fbf4c003ec0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c0016c0 tx=0x7fbf4c005830 comp rx=0 tx=0).stop 2024-09-21T00:27:11.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.066+0000 7fbf67319640 1 -- 172.21.15.112:0/3542191231 shutdown_connections 2024-09-21T00:27:11.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.066+0000 7fbf67319640 1 --2- 172.21.15.112:0/3542191231 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf4c003a50 0x7fbf4c003ec0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:11.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.066+0000 7fbf67319640 1 -- 172.21.15.112:0/3542191231 >> 172.21.15.112:0/3542191231 conn(0x7fbf600b6b20 msgr2=0x7fbf600d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:11.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.066+0000 7fbf67319640 1 -- 172.21.15.112:0/3542191231 shutdown_connections 2024-09-21T00:27:11.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.066+0000 7fbf67319640 1 -- 172.21.15.112:0/3542191231 wait complete. 2024-09-21T00:27:11.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.068+0000 7fbf67319640 1 Processor -- start 2024-09-21T00:27:11.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.068+0000 7fbf67319640 1 -- start start 2024-09-21T00:27:11.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.069+0000 7fbf67319640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbf4c003a50 0x7fbf600b4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:11.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.069+0000 7fbf67319640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbf6015ba60 0x7fbf600ab6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:11.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.069+0000 7fbf67319640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf600abc10 0x7fbf601f8570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:11.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.069+0000 7fbf67319640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbf600d24d0 con 0x7fbf600abc10 2024-09-21T00:27:11.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.069+0000 7fbf67319640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fbf6015fae0 con 0x7fbf6015ba60 2024-09-21T00:27:11.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.069+0000 7fbf67319640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fbf600d2650 con 0x7fbf4c003a50 2024-09-21T00:27:11.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.070+0000 7fbf658b6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf600abc10 0x7fbf601f8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:11.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.070+0000 7fbf658b6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf600abc10 0x7fbf601f8570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55382/0 (socket says 172.21.15.112:55382) 2024-09-21T00:27:11.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.070+0000 7fbf658b6640 1 -- 172.21.15.112:0/2286034623 learned_addr learned my addr 172.21.15.112:0/2286034623 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:27:11.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.070+0000 7fbf650b5640 1 --2- 172.21.15.112:0/2286034623 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbf4c003a50 0x7fbf600b4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:11.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.070+0000 7fbf648b4640 1 --2- 172.21.15.112:0/2286034623 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbf6015ba60 0x7fbf600ab6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:11.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.070+0000 7fbf658b6640 1 -- 172.21.15.112:0/2286034623 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbf4c003a50 msgr2=0x7fbf600b4df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:11.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.070+0000 7fbf658b6640 1 --2- 172.21.15.112:0/2286034623 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbf4c003a50 0x7fbf600b4df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:11.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.070+0000 7fbf658b6640 1 -- 172.21.15.112:0/2286034623 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbf6015ba60 msgr2=0x7fbf600ab6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:11.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.070+0000 7fbf658b6640 1 --2- 172.21.15.112:0/2286034623 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbf6015ba60 0x7fbf600ab6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:11.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.070+0000 7fbf658b6640 1 -- 172.21.15.112:0/2286034623 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf600cea10 con 0x7fbf600abc10 2024-09-21T00:27:11.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.070+0000 7fbf648b4640 1 --2- 172.21.15.112:0/2286034623 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbf6015ba60 0x7fbf600ab6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:27:11.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.070+0000 7fbf658b6640 1 --2- 172.21.15.112:0/2286034623 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf600abc10 0x7fbf601f8570 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fbf58005fd0 tx=0x7fbf5800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:11.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.070+0000 7fbf55ffb640 1 -- 172.21.15.112:0/2286034623 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf58003d80 con 0x7fbf600abc10 2024-09-21T00:27:11.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.070+0000 7fbf55ffb640 1 -- 172.21.15.112:0/2286034623 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbf58004b20 con 0x7fbf600abc10 2024-09-21T00:27:11.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.070+0000 7fbf67319640 1 -- 172.21.15.112:0/2286034623 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf600ce450 con 0x7fbf600abc10 2024-09-21T00:27:11.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.070+0000 7fbf55ffb640 1 -- 172.21.15.112:0/2286034623 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf58004030 con 0x7fbf600abc10 2024-09-21T00:27:11.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.071+0000 7fbf67319640 1 -- 172.21.15.112:0/2286034623 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf600aaed0 con 0x7fbf600abc10 2024-09-21T00:27:11.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.071+0000 7fbf67319640 1 -- 172.21.15.112:0/2286034623 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf60162690 con 0x7fbf600abc10 2024-09-21T00:27:11.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.072+0000 7fbf55ffb640 1 -- 172.21.15.112:0/2286034623 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fbf580041d0 con 0x7fbf600abc10 2024-09-21T00:27:11.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.072+0000 7fbf55ffb640 1 --2- 172.21.15.112:0/2286034623 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fbf44078b90 0x7fbf4407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:11.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.073+0000 7fbf55ffb640 1 -- 172.21.15.112:0/2286034623 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 6101+0+0 (secure 0 0 0) 0x7fbf5809bcf0 con 0x7fbf600abc10 2024-09-21T00:27:11.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.073+0000 7fbf650b5640 1 --2- 172.21.15.112:0/2286034623 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fbf44078b90 0x7fbf4407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:11.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.073+0000 7fbf650b5640 1 --2- 172.21.15.112:0/2286034623 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fbf44078b90 0x7fbf4407b0a0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c032000 tx=0x7fbf5c02c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:11.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.077+0000 7fbf55ffb640 1 -- 172.21.15.112:0/2286034623 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fbf580a0070 con 0x7fbf600abc10 2024-09-21T00:27:11.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.254+0000 7fbf67319640 1 -- 172.21.15.112:0/2286034623 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbf60160d60 con 0x7fbf600abc10 2024-09-21T00:27:11.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.255+0000 7fbf55ffb640 1 -- 172.21.15.112:0/2286034623 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) ==== 74+0+17026 (secure 0 0 0) 0x7fbf58063fa0 con 0x7fbf600abc10 2024-09-21T00:27:11.257 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:27:11.257 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":49,"fsid":"8b34692b-dc65-49b2-a08f-26f707b96ac5","created":"2024-09-21T00:25:22.788490+0000","modified":"2024-09-21T00:27:08.558791+0000","last_up_change":"2024-09-21T00:25:36.161835+0000","last_in_change":"2024-09-21T00:25:26.346500+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T00:25:37.105006+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-09-21T00:27:04.569521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-09-21T00:27:05.881104+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9c575086-67d2-4de2-b01b-d44dcc21d604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6809","nonce":755760621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6811","nonce":755760621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6815","nonce":755760621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6813","nonce":755760621}]},"public_addr":"172.21.15.112:6809/755760621","cluster_addr":"172.21.15.112:6811/755760621","heartbeat_back_addr":"172.21.15.112:6815/755760621","heartbeat_front_addr":"172.21.15.112:6813/755760621","state":["exists","up"]},{"osd":1,"uuid":"a983c227-a3ce-4bdd-82f1-438ba7e3ecd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6801","nonce":344932592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6803","nonce":344932592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6807","nonce":344932592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6805","nonce":344932592}]},"public_addr":"172.21.15.112:6801/344932592","cluster_addr":"172.21.15.112:6803/344932592","heartbeat_back_addr":"172.21.15.112:6807/344932592","heartbeat_front_addr":"172.21.15.112:6805/344932592","state":["exists","up"]},{"osd":2,"uuid":"1193c1d8-bfff-4a57-8200-5f69bf1ea3af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6825","nonce":407933616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6827","nonce":407933616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6831","nonce":407933616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6829","nonce":407933616}]},"public_addr":"172.21.15.112:6825/407933616","cluster_addr":"172.21.15.112:6827/407933616","heartbeat_back_addr":"172.21.15.112:6831/407933616","heartbeat_front_addr":"172.21.15.112:6829/407933616","state":["exists","up"]},{"osd":3,"uuid":"af04330a-36ad-40cc-a8dd-fa259b221de5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6817","nonce":4120164806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6819","nonce":4120164806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6823","nonce":4120164806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6821","nonce":4120164806}]},"public_addr":"172.21.15.112:6817/4120164806","cluster_addr":"172.21.15.112:6819/4120164806","heartbeat_back_addr":"172.21.15.112:6823/4120164806","heartbeat_front_addr":"172.21.15.112:6821/4120164806","state":["exists","up"]},{"osd":4,"uuid":"5b893111-292c-4c5a-b184-53ad38c7199e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6803","nonce":522077560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6805","nonce":522077560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6809","nonce":522077560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6807","nonce":522077560}]},"public_addr":"172.21.15.187:6803/522077560","cluster_addr":"172.21.15.187:6805/522077560","heartbeat_back_addr":"172.21.15.187:6809/522077560","heartbeat_front_addr":"172.21.15.187:6807/522077560","state":["exists","up"]},{"osd":5,"uuid":"c0b8e011-45fe-4c5e-925e-6be2a7f1868a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6827","nonce":3593144721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6829","nonce":3593144721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6833","nonce":3593144721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6831","nonce":3593144721}]},"public_addr":"172.21.15.187:6827/3593144721","cluster_addr":"172.21.15.187:6829/3593144721","heartbeat_back_addr":"172.21.15.187:6833/3593144721","heartbeat_front_addr":"172.21.15.187:6831/3593144721","state":["exists","up"]},{"osd":6,"uuid":"87d2d650-7489-4593-98a3-a1f433ce8dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6811","nonce":1494387657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6813","nonce":1494387657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6817","nonce":1494387657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6815","nonce":1494387657}]},"public_addr":"172.21.15.187:6811/1494387657","cluster_addr":"172.21.15.187:6813/1494387657","heartbeat_back_addr":"172.21.15.187:6817/1494387657","heartbeat_front_addr":"172.21.15.187:6815/1494387657","state":["exists","up"]},{"osd":7,"uuid":"c6b4f8e2-a1ee-4438-aa37-67ff4b6ef1f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6819","nonce":1624699842}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6821","nonce":1624699842}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6825","nonce":1624699842}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6823","nonce":1624699842}]},"public_addr":"172.21.15.187:6819/1624699842","cluster_addr":"172.21.15.187:6821/1624699842","heartbeat_back_addr":"172.21.15.187:6825/1624699842","heartbeat_front_addr":"172.21.15.187:6823/1624699842","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.873162+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.400506+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.664317+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.159791+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.202630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.113212+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.384786+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.448441+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T00:27:11.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.258+0000 7fbf67319640 1 -- 172.21.15.112:0/2286034623 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fbf44078b90 msgr2=0x7fbf4407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:11.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.258+0000 7fbf67319640 1 --2- 172.21.15.112:0/2286034623 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fbf44078b90 0x7fbf4407b0a0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c032000 tx=0x7fbf5c02c040 comp rx=0 tx=0).stop 2024-09-21T00:27:11.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.258+0000 7fbf67319640 1 -- 172.21.15.112:0/2286034623 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf600abc10 msgr2=0x7fbf601f8570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:11.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.258+0000 7fbf67319640 1 --2- 172.21.15.112:0/2286034623 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf600abc10 0x7fbf601f8570 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fbf58005fd0 tx=0x7fbf5800f790 comp rx=0 tx=0).stop 2024-09-21T00:27:11.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.258+0000 7fbf67319640 1 -- 172.21.15.112:0/2286034623 shutdown_connections 2024-09-21T00:27:11.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.258+0000 7fbf67319640 1 --2- 172.21.15.112:0/2286034623 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fbf44078b90 0x7fbf4407b0a0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:11.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.258+0000 7fbf67319640 1 --2- 172.21.15.112:0/2286034623 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf600abc10 0x7fbf601f8570 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:11.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.258+0000 7fbf67319640 1 --2- 172.21.15.112:0/2286034623 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbf6015ba60 0x7fbf600ab6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:11.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.258+0000 7fbf67319640 1 --2- 172.21.15.112:0/2286034623 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbf4c003a50 0x7fbf600b4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:11.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.258+0000 7fbf67319640 1 -- 172.21.15.112:0/2286034623 >> 172.21.15.112:0/2286034623 conn(0x7fbf600b6b20 msgr2=0x7fbf6015c540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:11.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.258+0000 7fbf67319640 1 -- 172.21.15.112:0/2286034623 shutdown_connections 2024-09-21T00:27:11.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.258+0000 7fbf67319640 1 -- 172.21.15.112:0/2286034623 wait complete. 2024-09-21T00:27:11.271 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T00:27:11.271 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T00:27:11.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.381+0000 7f78b76bd640 1 -- 172.21.15.112:0/2142033593 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f789c003a00 msgr2=0x7f789c003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:11.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.381+0000 7f78b76bd640 1 --2- 172.21.15.112:0/2142033593 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f789c003a00 0x7f789c003e70 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f78ac009ed0 tx=0x7f789c005830 comp rx=0 tx=0).stop 2024-09-21T00:27:11.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.381+0000 7f78b5c5a640 1 -- 172.21.15.112:0/2142033593 reap_dead start 2024-09-21T00:27:11.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.381+0000 7f78b76bd640 1 -- 172.21.15.112:0/2142033593 shutdown_connections 2024-09-21T00:27:11.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.381+0000 7f78b76bd640 1 -- 172.21.15.112:0/2142033593 >> 172.21.15.112:0/2142033593 conn(0x7f78b00d2350 msgr2=0x7f78b00d2760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:11.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.382+0000 7f78b76bd640 1 -- 172.21.15.112:0/2142033593 shutdown_connections 2024-09-21T00:27:11.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.382+0000 7f78b76bd640 1 -- 172.21.15.112:0/2142033593 wait complete. 2024-09-21T00:27:11.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.383+0000 7f78b76bd640 1 Processor -- start 2024-09-21T00:27:11.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.383+0000 7f78b76bd640 1 -- start start 2024-09-21T00:27:11.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.385+0000 7f78b76bd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f789c003a00 0x7f78b00ca0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:11.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.385+0000 7f78b76bd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f78b0143a80 0x7f78b0143e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:11.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.385+0000 7f78b76bd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f78b01f3330 0x7f78b01f3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:11.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.385+0000 7f78b76bd640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f78b0160c20 con 0x7f78b0143a80 2024-09-21T00:27:11.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.385+0000 7f78b76bd640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f78b00b7470 con 0x7f789c003a00 2024-09-21T00:27:11.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.385+0000 7f78b76bd640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f78b0160da0 con 0x7f78b01f3330 2024-09-21T00:27:11.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.385+0000 7f78b4c58640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f78b0143a80 0x7f78b0143e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:11.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.385+0000 7f78b4c58640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f78b0143a80 0x7f78b0143e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55406/0 (socket says 172.21.15.112:55406) 2024-09-21T00:27:11.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.385+0000 7f78b4c58640 1 -- 172.21.15.112:0/3463623041 learned_addr learned my addr 172.21.15.112:0/3463623041 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:27:11.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.385+0000 7f78b5459640 1 --2- 172.21.15.112:0/3463623041 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f789c003a00 0x7f78b00ca0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:11.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.385+0000 7f78b5c5a640 1 --2- 172.21.15.112:0/3463623041 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f78b01f3330 0x7f78b01f3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:11.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.385+0000 7f78b4c58640 1 -- 172.21.15.112:0/3463623041 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f78b01f3330 msgr2=0x7f78b01f3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:11.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.385+0000 7f78b4c58640 1 --2- 172.21.15.112:0/3463623041 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f78b01f3330 0x7f78b01f3740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:11.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.385+0000 7f78b4c58640 1 -- 172.21.15.112:0/3463623041 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f789c003a00 msgr2=0x7f78b00ca0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:11.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.385+0000 7f78b4c58640 1 --2- 172.21.15.112:0/3463623041 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f789c003a00 0x7f78b00ca0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:11.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.385+0000 7f78b4c58640 1 -- 172.21.15.112:0/3463623041 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78b015b490 con 0x7f78b0143a80 2024-09-21T00:27:11.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.385+0000 7f78b5459640 1 --2- 172.21.15.112:0/3463623041 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f789c003a00 0x7f78b00ca0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:27:11.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.385+0000 7f78b4c58640 1 --2- 172.21.15.112:0/3463623041 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f78b0143a80 0x7f78b0143e90 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f78a0011830 tx=0x7f78a0011cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:11.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.385+0000 7f78a5ffb640 1 -- 172.21.15.112:0/3463623041 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78a0004460 con 0x7f78b0143a80 2024-09-21T00:27:11.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.385+0000 7f78a5ffb640 1 -- 172.21.15.112:0/3463623041 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f78a000f9a0 con 0x7f78b0143a80 2024-09-21T00:27:11.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.385+0000 7f78b76bd640 1 -- 172.21.15.112:0/3463623041 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78b015b770 con 0x7f78b0143a80 2024-09-21T00:27:11.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.385+0000 7f78a5ffb640 1 -- 172.21.15.112:0/3463623041 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78a000fdf0 con 0x7f78b0143a80 2024-09-21T00:27:11.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.386+0000 7f78b76bd640 1 -- 172.21.15.112:0/3463623041 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78b015bd30 con 0x7f78b0143a80 2024-09-21T00:27:11.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.387+0000 7f78b76bd640 1 -- 172.21.15.112:0/3463623041 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78b015b200 con 0x7f78b0143a80 2024-09-21T00:27:11.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.387+0000 7f78a5ffb640 1 -- 172.21.15.112:0/3463623041 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f78a00035f0 con 0x7f78b0143a80 2024-09-21T00:27:11.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.388+0000 7f78a5ffb640 1 --2- 172.21.15.112:0/3463623041 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f7894078bc0 0x7f789407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:11.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.388+0000 7f78a5ffb640 1 -- 172.21.15.112:0/3463623041 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 6101+0+0 (secure 0 0 0) 0x7f78a009be10 con 0x7f78b0143a80 2024-09-21T00:27:11.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.388+0000 7f78b5459640 1 --2- 172.21.15.112:0/3463623041 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f7894078bc0 0x7f789407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:11.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.389+0000 7f78b5459640 1 --2- 172.21.15.112:0/3463623041 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f7894078bc0 0x7f789407b0d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f78ac035000 tx=0x7f78ac004a40 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:11.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.393+0000 7f78a5ffb640 1 -- 172.21.15.112:0/3463623041 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f78a00641e0 con 0x7f78b0143a80 2024-09-21T00:27:11.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.569+0000 7f78b76bd640 1 -- 172.21.15.112:0/3463623041 --> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f78b015cf20 con 0x7f7894078bc0 2024-09-21T00:27:11.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.574+0000 7f78a5ffb640 1 -- 172.21.15.112:0/3463623041 <== mgr.4103 v2:172.21.15.187:6800/2921522906 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+241701 (secure 0 0 0) 0x7f78b015cf20 con 0x7f7894078bc0 2024-09-21T00:27:11.578 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:27:11.579 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2024-09-21T00:27:11.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.577+0000 7f78b76bd640 1 -- 172.21.15.112:0/3463623041 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f7894078bc0 msgr2=0x7f789407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:11.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.577+0000 7f78b76bd640 1 --2- 172.21.15.112:0/3463623041 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f7894078bc0 0x7f789407b0d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f78ac035000 tx=0x7f78ac004a40 comp rx=0 tx=0).stop 2024-09-21T00:27:11.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.577+0000 7f78b76bd640 1 -- 172.21.15.112:0/3463623041 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f78b0143a80 msgr2=0x7f78b0143e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:11.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.577+0000 7f78b76bd640 1 --2- 172.21.15.112:0/3463623041 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f78b0143a80 0x7f78b0143e90 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f78a0011830 tx=0x7f78a0011cf0 comp rx=0 tx=0).stop 2024-09-21T00:27:11.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.577+0000 7f78b76bd640 1 -- 172.21.15.112:0/3463623041 shutdown_connections 2024-09-21T00:27:11.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.578+0000 7f78b76bd640 1 --2- 172.21.15.112:0/3463623041 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f78b01f3330 0x7f78b01f3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:11.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.578+0000 7f78b76bd640 1 --2- 172.21.15.112:0/3463623041 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f7894078bc0 0x7f789407b0d0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:11.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.578+0000 7f78b76bd640 1 --2- 172.21.15.112:0/3463623041 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f78b0143a80 0x7f78b0143e90 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:11.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.578+0000 7f78b76bd640 1 --2- 172.21.15.112:0/3463623041 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f789c003a00 0x7f78b00ca0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:11.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.578+0000 7f78b76bd640 1 -- 172.21.15.112:0/3463623041 >> 172.21.15.112:0/3463623041 conn(0x7f78b00d2350 msgr2=0x7f78b0070ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:11.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.578+0000 7f78b76bd640 1 -- 172.21.15.112:0/3463623041 shutdown_connections 2024-09-21T00:27:11.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.578+0000 7f78b76bd640 1 -- 172.21.15.112:0/3463623041 wait complete. 2024-09-21T00:27:11.590 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":91,"stamp":"2024-09-21T00:27:11.109326+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":105,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":77,"ondisk_log_size":77,"up":86,"acting":86,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":7,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":221600,"kb_used_data":5376,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753120,"statfs":{"total":773094113280,"available":772867194880,"internally_reserved":0,"allocated":5505024,"data_stored":3025554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.557642"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-21T00:27:07.579407+0000","last_change":"2024-09-21T00:27:07.574598+0000","last_active":"2024-09-21T00:27:07.579407+0000","last_peered":"2024-09-21T00:27:07.579407+0000","last_clean":"2024-09-21T00:27:07.579407+0000","last_became_active":"2024-09-21T00:27:07.574242+0000","last_became_peered":"2024-09-21T00:27:07.574242+0000","last_unstale":"2024-09-21T00:27:07.579407+0000","last_undegraded":"2024-09-21T00:27:07.579407+0000","last_fullsized":"2024-09-21T00:27:07.579407+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T00:55:01.680636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-21T00:27:07.593752+0000","last_change":"2024-09-21T00:27:07.581689+0000","last_active":"2024-09-21T00:27:07.593752+0000","last_peered":"2024-09-21T00:27:07.593752+0000","last_clean":"2024-09-21T00:27:07.593752+0000","last_became_active":"2024-09-21T00:27:07.581012+0000","last_became_peered":"2024-09-21T00:27:07.581012+0000","last_unstale":"2024-09-21T00:27:07.593752+0000","last_undegraded":"2024-09-21T00:27:07.593752+0000","last_fullsized":"2024-09-21T00:27:07.593752+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:08:00.101537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-21T00:27:07.579280+0000","last_change":"2024-09-21T00:27:07.567996+0000","last_active":"2024-09-21T00:27:07.579280+0000","last_peered":"2024-09-21T00:27:07.579280+0000","last_clean":"2024-09-21T00:27:07.579280+0000","last_became_active":"2024-09-21T00:27:07.567702+0000","last_became_peered":"2024-09-21T00:27:07.567702+0000","last_unstale":"2024-09-21T00:27:07.579280+0000","last_undegraded":"2024-09-21T00:27:07.579280+0000","last_fullsized":"2024-09-21T00:27:07.579280+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:29:32.175256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-21T00:27:07.574128+0000","last_change":"2024-09-21T00:27:07.567050+0000","last_active":"2024-09-21T00:27:07.574128+0000","last_peered":"2024-09-21T00:27:07.574128+0000","last_clean":"2024-09-21T00:27:07.574128+0000","last_became_active":"2024-09-21T00:27:07.566711+0000","last_became_peered":"2024-09-21T00:27:07.566711+0000","last_unstale":"2024-09-21T00:27:07.574128+0000","last_undegraded":"2024-09-21T00:27:07.574128+0000","last_fullsized":"2024-09-21T00:27:07.574128+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:53:27.777241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-21T00:27:07.574359+0000","last_change":"2024-09-21T00:27:07.567067+0000","last_active":"2024-09-21T00:27:07.574359+0000","last_peered":"2024-09-21T00:27:07.574359+0000","last_clean":"2024-09-21T00:27:07.574359+0000","last_became_active":"2024-09-21T00:27:07.566709+0000","last_became_peered":"2024-09-21T00:27:07.566709+0000","last_unstale":"2024-09-21T00:27:07.574359+0000","last_undegraded":"2024-09-21T00:27:07.574359+0000","last_fullsized":"2024-09-21T00:27:07.574359+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:36:49.037229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-21T00:27:07.595592+0000","last_change":"2024-09-21T00:27:07.586170+0000","last_active":"2024-09-21T00:27:07.595592+0000","last_peered":"2024-09-21T00:27:07.595592+0000","last_clean":"2024-09-21T00:27:07.595592+0000","last_became_active":"2024-09-21T00:27:07.585790+0000","last_became_peered":"2024-09-21T00:27:07.585790+0000","last_unstale":"2024-09-21T00:27:07.595592+0000","last_undegraded":"2024-09-21T00:27:07.595592+0000","last_fullsized":"2024-09-21T00:27:07.595592+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:27:32.413929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-21T00:27:05.556637+0000","last_change":"2024-09-21T00:27:05.556637+0000","last_active":"2024-09-21T00:27:05.543789+0000","last_peered":"2024-09-21T00:27:05.543789+0000","last_clean":"2024-09-21T00:27:05.543789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:05.556637+0000","last_undegraded":"2024-09-21T00:27:05.556637+0000","last_fullsized":"2024-09-21T00:27:05.556637+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1e","version":"48'1","reported_seq":14,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-21T00:27:07.600346+0000","last_change":"2024-09-21T00:27:06.578147+0000","last_active":"2024-09-21T00:27:07.600346+0000","last_peered":"2024-09-21T00:27:07.600346+0000","last_clean":"2024-09-21T00:27:07.600346+0000","last_became_active":"2024-09-21T00:27:06.577845+0000","last_became_peered":"2024-09-21T00:27:06.577845+0000","last_unstale":"2024-09-21T00:27:07.600346+0000","last_undegraded":"2024-09-21T00:27:07.600346+0000","last_fullsized":"2024-09-21T00:27:07.600346+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:18:27.263054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1d","version":"48'2","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-21T00:27:07.599338+0000","last_change":"2024-09-21T00:27:06.573813+0000","last_active":"2024-09-21T00:27:07.599338+0000","last_peered":"2024-09-21T00:27:07.599338+0000","last_clean":"2024-09-21T00:27:07.599338+0000","last_became_active":"2024-09-21T00:27:06.573532+0000","last_became_peered":"2024-09-21T00:27:06.573532+0000","last_unstale":"2024-09-21T00:27:07.599338+0000","last_undegraded":"2024-09-21T00:27:07.599338+0000","last_fullsized":"2024-09-21T00:27:07.599338+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:31:15.501365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-21T00:27:05.555296+0000","last_change":"2024-09-21T00:27:05.555296+0000","last_active":"2024-09-21T00:27:05.543789+0000","last_peered":"2024-09-21T00:27:05.543789+0000","last_clean":"2024-09-21T00:27:05.543789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:05.555296+0000","last_undegraded":"2024-09-21T00:27:05.555296+0000","last_fullsized":"2024-09-21T00:27:05.555296+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-21T00:27:05.562604+0000","last_change":"2024-09-21T00:27:05.562604+0000","last_active":"2024-09-21T00:27:05.543789+0000","last_peered":"2024-09-21T00:27:05.543789+0000","last_clean":"2024-09-21T00:27:05.543789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:05.562604+0000","last_undegraded":"2024-09-21T00:27:05.562604+0000","last_fullsized":"2024-09-21T00:27:05.562604+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-21T00:27:05.557122+0000","last_change":"2024-09-21T00:27:05.557122+0000","last_active":"2024-09-21T00:27:05.543789+0000","last_peered":"2024-09-21T00:27:05.543789+0000","last_clean":"2024-09-21T00:27:05.543789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:05.557122+0000","last_undegraded":"2024-09-21T00:27:05.557122+0000","last_fullsized":"2024-09-21T00:27:05.557122+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-21T00:27:05.555459+0000","last_change":"2024-09-21T00:27:05.555459+0000","last_active":"2024-09-21T00:27:05.543789+0000","last_peered":"2024-09-21T00:27:05.543789+0000","last_clean":"2024-09-21T00:27:05.543789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:05.555459+0000","last_undegraded":"2024-09-21T00:27:05.555459+0000","last_fullsized":"2024-09-21T00:27:05.555459+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-21T00:27:05.562527+0000","last_change":"2024-09-21T00:27:05.562527+0000","last_active":"2024-09-21T00:27:05.543789+0000","last_peered":"2024-09-21T00:27:05.543789+0000","last_clean":"2024-09-21T00:27:05.543789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:05.562527+0000","last_undegraded":"2024-09-21T00:27:05.562527+0000","last_fullsized":"2024-09-21T00:27:05.562527+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-21T00:27:05.562841+0000","last_change":"2024-09-21T00:27:05.562841+0000","last_active":"2024-09-21T00:27:05.543789+0000","last_peered":"2024-09-21T00:27:05.543789+0000","last_clean":"2024-09-21T00:27:05.543789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:05.562841+0000","last_undegraded":"2024-09-21T00:27:05.562841+0000","last_fullsized":"2024-09-21T00:27:05.562841+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-21T00:27:05.558519+0000","last_change":"2024-09-21T00:27:05.558519+0000","last_active":"2024-09-21T00:27:05.543789+0000","last_peered":"2024-09-21T00:27:05.543789+0000","last_clean":"2024-09-21T00:27:05.543789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:05.558519+0000","last_undegraded":"2024-09-21T00:27:05.558519+0000","last_fullsized":"2024-09-21T00:27:05.558519+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-21T00:27:05.563390+0000","last_change":"2024-09-21T00:27:05.563390+0000","last_active":"2024-09-21T00:27:05.543789+0000","last_peered":"2024-09-21T00:27:05.543789+0000","last_clean":"2024-09-21T00:27:05.543789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:05.563390+0000","last_undegraded":"2024-09-21T00:27:05.563390+0000","last_fullsized":"2024-09-21T00:27:05.563390+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-21T00:27:05.554843+0000","last_change":"2024-09-21T00:27:05.554843+0000","last_active":"2024-09-21T00:27:05.543789+0000","last_peered":"2024-09-21T00:27:05.543789+0000","last_clean":"2024-09-21T00:27:05.543789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:05.554843+0000","last_undegraded":"2024-09-21T00:27:05.554843+0000","last_fullsized":"2024-09-21T00:27:05.554843+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-21T00:27:05.562134+0000","last_change":"2024-09-21T00:27:05.562134+0000","last_active":"2024-09-21T00:27:05.543789+0000","last_peered":"2024-09-21T00:27:05.543789+0000","last_clean":"2024-09-21T00:27:05.543789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:05.562134+0000","last_undegraded":"2024-09-21T00:27:05.562134+0000","last_fullsized":"2024-09-21T00:27:05.562134+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-21T00:27:05.556088+0000","last_change":"2024-09-21T00:27:05.556088+0000","last_active":"2024-09-21T00:27:05.543789+0000","last_peered":"2024-09-21T00:27:05.543789+0000","last_clean":"2024-09-21T00:27:05.543789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:05.556088+0000","last_undegraded":"2024-09-21T00:27:05.556088+0000","last_fullsized":"2024-09-21T00:27:05.556088+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-21T00:27:05.562785+0000","last_change":"2024-09-21T00:27:05.562785+0000","last_active":"2024-09-21T00:27:05.543789+0000","last_peered":"2024-09-21T00:27:05.543789+0000","last_clean":"2024-09-21T00:27:05.543789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:05.562785+0000","last_undegraded":"2024-09-21T00:27:05.562785+0000","last_fullsized":"2024-09-21T00:27:05.562785+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-21T00:27:05.554830+0000","last_change":"2024-09-21T00:27:05.554830+0000","last_active":"2024-09-21T00:27:05.543789+0000","last_peered":"2024-09-21T00:27:05.543789+0000","last_clean":"2024-09-21T00:27:05.543789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:05.554830+0000","last_undegraded":"2024-09-21T00:27:05.554830+0000","last_fullsized":"2024-09-21T00:27:05.554830+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-21T00:27:05.561823+0000","last_change":"2024-09-21T00:27:05.561823+0000","last_active":"2024-09-21T00:27:05.543789+0000","last_peered":"2024-09-21T00:27:05.543789+0000","last_clean":"2024-09-21T00:27:05.543789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:05.561823+0000","last_undegraded":"2024-09-21T00:27:05.561823+0000","last_fullsized":"2024-09-21T00:27:05.561823+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-21T00:27:07.594119+0000","last_change":"2024-09-21T00:27:07.585242+0000","last_active":"2024-09-21T00:27:07.594119+0000","last_peered":"2024-09-21T00:27:07.594119+0000","last_clean":"2024-09-21T00:27:07.594119+0000","last_became_active":"2024-09-21T00:27:07.584922+0000","last_became_peered":"2024-09-21T00:27:07.584922+0000","last_unstale":"2024-09-21T00:27:07.594119+0000","last_undegraded":"2024-09-21T00:27:07.594119+0000","last_fullsized":"2024-09-21T00:27:07.594119+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:52:07.026811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-21T00:27:05.562632+0000","last_change":"2024-09-21T00:27:05.562632+0000","last_active":"2024-09-21T00:27:05.543789+0000","last_peered":"2024-09-21T00:27:05.543789+0000","last_clean":"2024-09-21T00:27:05.543789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:05.562632+0000","last_undegraded":"2024-09-21T00:27:05.562632+0000","last_fullsized":"2024-09-21T00:27:05.562632+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-21T00:27:05.555437+0000","last_change":"2024-09-21T00:27:05.555437+0000","last_active":"2024-09-21T00:27:05.543789+0000","last_peered":"2024-09-21T00:27:05.543789+0000","last_clean":"2024-09-21T00:27:05.543789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:05.555437+0000","last_undegraded":"2024-09-21T00:27:05.555437+0000","last_fullsized":"2024-09-21T00:27:05.555437+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-21T00:27:07.580102+0000","last_change":"2024-09-21T00:27:07.569099+0000","last_active":"2024-09-21T00:27:07.580102+0000","last_peered":"2024-09-21T00:27:07.580102+0000","last_clean":"2024-09-21T00:27:07.580102+0000","last_became_active":"2024-09-21T00:27:07.568739+0000","last_became_peered":"2024-09-21T00:27:07.568739+0000","last_unstale":"2024-09-21T00:27:07.580102+0000","last_undegraded":"2024-09-21T00:27:07.580102+0000","last_fullsized":"2024-09-21T00:27:07.580102+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:18:56.106026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-21T00:27:05.556442+0000","last_change":"2024-09-21T00:27:05.556442+0000","last_active":"2024-09-21T00:27:05.543789+0000","last_peered":"2024-09-21T00:27:05.543789+0000","last_clean":"2024-09-21T00:27:05.543789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:05.556442+0000","last_undegraded":"2024-09-21T00:27:05.556442+0000","last_fullsized":"2024-09-21T00:27:05.556442+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.b","version":"48'2","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-21T00:27:07.599703+0000","last_change":"2024-09-21T00:27:06.580120+0000","last_active":"2024-09-21T00:27:07.599703+0000","last_peered":"2024-09-21T00:27:07.599703+0000","last_clean":"2024-09-21T00:27:07.599703+0000","last_became_active":"2024-09-21T00:27:06.579795+0000","last_became_peered":"2024-09-21T00:27:06.579795+0000","last_unstale":"2024-09-21T00:27:07.599703+0000","last_undegraded":"2024-09-21T00:27:07.599703+0000","last_fullsized":"2024-09-21T00:27:07.599703+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:31:18.308257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-21T00:27:05.556012+0000","last_change":"2024-09-21T00:27:05.556012+0000","last_active":"2024-09-21T00:27:05.543789+0000","last_peered":"2024-09-21T00:27:05.543789+0000","last_clean":"2024-09-21T00:27:05.543789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:05.556012+0000","last_undegraded":"2024-09-21T00:27:05.556012+0000","last_fullsized":"2024-09-21T00:27:05.556012+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-21T00:27:05.561999+0000","last_change":"2024-09-21T00:27:05.561999+0000","last_active":"2024-09-21T00:27:05.543789+0000","last_peered":"2024-09-21T00:27:05.543789+0000","last_clean":"2024-09-21T00:27:05.543789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:05.561999+0000","last_undegraded":"2024-09-21T00:27:05.561999+0000","last_fullsized":"2024-09-21T00:27:05.561999+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-21T00:27:07.595184+0000","last_change":"2024-09-21T00:27:07.590250+0000","last_active":"2024-09-21T00:27:07.595184+0000","last_peered":"2024-09-21T00:27:07.595184+0000","last_clean":"2024-09-21T00:27:07.595184+0000","last_became_active":"2024-09-21T00:27:07.589930+0000","last_became_peered":"2024-09-21T00:27:07.589930+0000","last_unstale":"2024-09-21T00:27:07.595184+0000","last_undegraded":"2024-09-21T00:27:07.595184+0000","last_fullsized":"2024-09-21T00:27:07.595184+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:55:47.579955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-21T00:27:05.561751+0000","last_change":"2024-09-21T00:27:05.561751+0000","last_active":"2024-09-21T00:27:05.543789+0000","last_peered":"2024-09-21T00:27:05.543789+0000","last_clean":"2024-09-21T00:27:05.543789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:05.561751+0000","last_undegraded":"2024-09-21T00:27:05.561751+0000","last_fullsized":"2024-09-21T00:27:05.561751+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-21T00:27:05.556126+0000","last_change":"2024-09-21T00:27:05.556126+0000","last_active":"2024-09-21T00:27:05.543789+0000","last_peered":"2024-09-21T00:27:05.543789+0000","last_clean":"2024-09-21T00:27:05.543789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:05.556126+0000","last_undegraded":"2024-09-21T00:27:05.556126+0000","last_fullsized":"2024-09-21T00:27:05.556126+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-21T00:27:07.593806+0000","last_change":"2024-09-21T00:27:07.581995+0000","last_active":"2024-09-21T00:27:07.593806+0000","last_peered":"2024-09-21T00:27:07.593806+0000","last_clean":"2024-09-21T00:27:07.593806+0000","last_became_active":"2024-09-21T00:27:07.581400+0000","last_became_peered":"2024-09-21T00:27:07.581400+0000","last_unstale":"2024-09-21T00:27:07.593806+0000","last_undegraded":"2024-09-21T00:27:07.593806+0000","last_fullsized":"2024-09-21T00:27:07.593806+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:20:00.370047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-21T00:27:05.562808+0000","last_change":"2024-09-21T00:27:05.562808+0000","last_active":"2024-09-21T00:27:05.543789+0000","last_peered":"2024-09-21T00:27:05.543789+0000","last_clean":"2024-09-21T00:27:05.543789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:05.562808+0000","last_undegraded":"2024-09-21T00:27:05.562808+0000","last_fullsized":"2024-09-21T00:27:05.562808+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-21T00:27:05.562127+0000","last_change":"2024-09-21T00:27:05.562127+0000","last_active":"2024-09-21T00:27:05.543789+0000","last_peered":"2024-09-21T00:27:05.543789+0000","last_clean":"2024-09-21T00:27:05.543789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:05.562127+0000","last_undegraded":"2024-09-21T00:27:05.562127+0000","last_fullsized":"2024-09-21T00:27:05.562127+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-21T00:27:05.554822+0000","last_change":"2024-09-21T00:27:05.554822+0000","last_active":"2024-09-21T00:27:05.543789+0000","last_peered":"2024-09-21T00:27:05.543789+0000","last_clean":"2024-09-21T00:27:05.543789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:05.554822+0000","last_undegraded":"2024-09-21T00:27:05.554822+0000","last_fullsized":"2024-09-21T00:27:05.554822+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-21T00:27:05.554816+0000","last_change":"2024-09-21T00:27:05.554816+0000","last_active":"2024-09-21T00:27:05.543789+0000","last_peered":"2024-09-21T00:27:05.543789+0000","last_clean":"2024-09-21T00:27:05.543789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:05.554816+0000","last_undegraded":"2024-09-21T00:27:05.554816+0000","last_fullsized":"2024-09-21T00:27:05.554816+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-21T00:27:05.557773+0000","last_change":"2024-09-21T00:27:05.557773+0000","last_active":"2024-09-21T00:27:05.543789+0000","last_peered":"2024-09-21T00:27:05.543789+0000","last_clean":"2024-09-21T00:27:05.543789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:05.557773+0000","last_undegraded":"2024-09-21T00:27:05.557773+0000","last_fullsized":"2024-09-21T00:27:05.557773+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-21T00:27:05.555561+0000","last_change":"2024-09-21T00:27:05.555561+0000","last_active":"2024-09-21T00:27:05.543789+0000","last_peered":"2024-09-21T00:27:05.543789+0000","last_clean":"2024-09-21T00:27:05.543789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:05.555561+0000","last_undegraded":"2024-09-21T00:27:05.555561+0000","last_fullsized":"2024-09-21T00:27:05.555561+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-21T00:27:05.561883+0000","last_change":"2024-09-21T00:27:05.561883+0000","last_active":"2024-09-21T00:27:05.543789+0000","last_peered":"2024-09-21T00:27:05.543789+0000","last_clean":"2024-09-21T00:27:05.543789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:05.561883+0000","last_undegraded":"2024-09-21T00:27:05.561883+0000","last_fullsized":"2024-09-21T00:27:05.561883+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":153,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T00:27:05.563691+0000","last_change":"2024-09-21T00:25:38.725792+0000","last_active":"2024-09-21T00:27:05.563691+0000","last_peered":"2024-09-21T00:27:05.563691+0000","last_clean":"2024-09-21T00:27:05.563691+0000","last_became_active":"2024-09-21T00:25:38.724668+0000","last_became_peered":"2024-09-21T00:25:38.724668+0000","last_unstale":"2024-09-21T00:27:05.563691+0000","last_undegraded":"2024-09-21T00:27:05.563691+0000","last_fullsized":"2024-09-21T00:27:05.563691+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:37.163216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:37.163216+0000","last_clean_scrub_stamp":"2024-09-21T00:25:37.163216+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:14:36.588660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":20,"acting":20,"num_store_stats":1},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509461,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28132,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343708,"statfs":{"total":96636764160,"available":96607956992,"internally_reserved":0,"allocated":1130496,"data_stored":819241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509462,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27552,"kb_used_data":524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344288,"statfs":{"total":96636764160,"available":96608550912,"internally_reserved":0,"allocated":536576,"data_stored":228873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509461,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27552,"kb_used_data":524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344288,"statfs":{"total":96636764160,"available":96608550912,"internally_reserved":0,"allocated":536576,"data_stored":228873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509460,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27552,"kb_used_data":524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344288,"statfs":{"total":96636764160,"available":96608550912,"internally_reserved":0,"allocated":536576,"data_stored":228873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509462,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27552,"kb_used_data":524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344288,"statfs":{"total":96636764160,"available":96608550912,"internally_reserved":0,"allocated":536576,"data_stored":228873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509462,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27552,"kb_used_data":524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344288,"statfs":{"total":96636764160,"available":96608550912,"internally_reserved":0,"allocated":536576,"data_stored":228873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509462,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28132,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343708,"statfs":{"total":96636764160,"available":96607956992,"internally_reserved":0,"allocated":1130496,"data_stored":819241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27576,"kb_used_data":548,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344264,"statfs":{"total":96636764160,"available":96608526336,"internally_reserved":0,"allocated":561152,"data_stored":242707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T00:27:11.595 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T00:27:11.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.742+0000 7fc7755f1640 1 -- 172.21.15.112:0/17872195 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc7700b2e60 msgr2=0x7fc7700b3270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:11.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.742+0000 7fc7755f1640 1 --2- 172.21.15.112:0/17872195 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc7700b2e60 0x7fc7700b3270 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fc7700cef90 tx=0x7fc76802aa30 comp rx=0 tx=0).stop 2024-09-21T00:27:11.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.742+0000 7fc7755f1640 1 -- 172.21.15.112:0/17872195 shutdown_connections 2024-09-21T00:27:11.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.742+0000 7fc7755f1640 1 --2- 172.21.15.112:0/17872195 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc7700b2e60 0x7fc7700b3270 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:11.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.742+0000 7fc7755f1640 1 --2- 172.21.15.112:0/17872195 >> v2:172.21.15.187:3300/0 conn(0x7fc770162590 0x7fc7701629a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:11.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.742+0000 7fc7755f1640 1 -- 172.21.15.112:0/17872195 >> 172.21.15.112:0/17872195 conn(0x7fc7700d4440 msgr2=0x7fc7700d4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:11.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.742+0000 7fc7755f1640 1 -- 172.21.15.112:0/17872195 shutdown_connections 2024-09-21T00:27:11.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.742+0000 7fc7755f1640 1 -- 172.21.15.112:0/17872195 wait complete. 2024-09-21T00:27:11.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.744+0000 7fc7755f1640 1 Processor -- start 2024-09-21T00:27:11.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.744+0000 7fc7755f1640 1 -- start start 2024-09-21T00:27:11.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.745+0000 7fc7755f1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc7700b2e60 0x7fc770152d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:11.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.745+0000 7fc7755f1640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc7700cfe00 0x7fc770160790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:11.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.745+0000 7fc7755f1640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc770162590 0x7fc770160cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:11.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.745+0000 7fc7755f1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc7700b3930 con 0x7fc7700b2e60 2024-09-21T00:27:11.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.745+0000 7fc7755f1640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc7700b37b0 con 0x7fc7700cfe00 2024-09-21T00:27:11.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.745+0000 7fc7755f1640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc7700b3ab0 con 0x7fc770162590 2024-09-21T00:27:11.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.745+0000 7fc76effd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc7700b2e60 0x7fc770152d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:11.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.745+0000 7fc76effd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc7700b2e60 0x7fc770152d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55428/0 (socket says 172.21.15.112:55428) 2024-09-21T00:27:11.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.745+0000 7fc76effd640 1 -- 172.21.15.112:0/1944980545 learned_addr learned my addr 172.21.15.112:0/1944980545 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:27:11.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.745+0000 7fc76e7fc640 1 --2- 172.21.15.112:0/1944980545 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc7700cfe00 0x7fc770160790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:11.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.745+0000 7fc76f7fe640 1 --2- 172.21.15.112:0/1944980545 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc770162590 0x7fc770160cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:11.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.745+0000 7fc76effd640 1 -- 172.21.15.112:0/1944980545 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc770162590 msgr2=0x7fc770160cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:11.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.745+0000 7fc76effd640 1 --2- 172.21.15.112:0/1944980545 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc770162590 0x7fc770160cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:11.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.745+0000 7fc76effd640 1 -- 172.21.15.112:0/1944980545 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc7700cfe00 msgr2=0x7fc770160790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:11.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.745+0000 7fc76effd640 1 --2- 172.21.15.112:0/1944980545 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc7700cfe00 0x7fc770160790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:11.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.745+0000 7fc76effd640 1 -- 172.21.15.112:0/1944980545 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7701532c0 con 0x7fc7700b2e60 2024-09-21T00:27:11.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.746+0000 7fc76effd640 1 --2- 172.21.15.112:0/1944980545 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc7700b2e60 0x7fc770152d80 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fc76000b420 tx=0x7fc76000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:11.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.746+0000 7fc7577fe640 1 -- 172.21.15.112:0/1944980545 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc760011830 con 0x7fc7700b2e60 2024-09-21T00:27:11.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.746+0000 7fc7577fe640 1 -- 172.21.15.112:0/1944980545 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc760016070 con 0x7fc7700b2e60 2024-09-21T00:27:11.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.746+0000 7fc7755f1640 1 -- 172.21.15.112:0/1944980545 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc770157300 con 0x7fc7700b2e60 2024-09-21T00:27:11.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.746+0000 7fc7577fe640 1 -- 172.21.15.112:0/1944980545 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc760011d60 con 0x7fc7700b2e60 2024-09-21T00:27:11.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.746+0000 7fc7755f1640 1 -- 172.21.15.112:0/1944980545 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7701f8fd0 con 0x7fc7700b2e60 2024-09-21T00:27:11.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.747+0000 7fc7755f1640 1 -- 172.21.15.112:0/1944980545 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7700c7da0 con 0x7fc7700b2e60 2024-09-21T00:27:11.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.747+0000 7fc7577fe640 1 -- 172.21.15.112:0/1944980545 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc76001b460 con 0x7fc7700b2e60 2024-09-21T00:27:11.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.748+0000 7fc7577fe640 1 --2- 172.21.15.112:0/1944980545 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fc75c078b90 0x7fc75c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:11.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.748+0000 7fc7577fe640 1 -- 172.21.15.112:0/1944980545 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 6101+0+0 (secure 0 0 0) 0x7fc76009a850 con 0x7fc7700b2e60 2024-09-21T00:27:11.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.748+0000 7fc76e7fc640 1 --2- 172.21.15.112:0/1944980545 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fc75c078b90 0x7fc75c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:11.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.749+0000 7fc76e7fc640 1 --2- 172.21.15.112:0/1944980545 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fc75c078b90 0x7fc75c07b0a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc770163550 tx=0x7fc76400a250 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:11.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.752+0000 7fc7577fe640 1 -- 172.21.15.112:0/1944980545 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc760020020 con 0x7fc7700b2e60 2024-09-21T00:27:11.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.926+0000 7fc7755f1640 1 -- 172.21.15.112:0/1944980545 --> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc7700b3270 con 0x7fc75c078b90 2024-09-21T00:27:11.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.930+0000 7fc7577fe640 1 -- 172.21.15.112:0/1944980545 <== mgr.4103 v2:172.21.15.187:6800/2921522906 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+241701 (secure 0 0 0) 0x7fc7700b3270 con 0x7fc75c078b90 2024-09-21T00:27:11.935 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:27:11.936 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2024-09-21T00:27:11.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.933+0000 7fc7755f1640 1 -- 172.21.15.112:0/1944980545 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fc75c078b90 msgr2=0x7fc75c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:11.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.933+0000 7fc7755f1640 1 --2- 172.21.15.112:0/1944980545 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fc75c078b90 0x7fc75c07b0a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc770163550 tx=0x7fc76400a250 comp rx=0 tx=0).stop 2024-09-21T00:27:11.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.933+0000 7fc7755f1640 1 -- 172.21.15.112:0/1944980545 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc7700b2e60 msgr2=0x7fc770152d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:11.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.933+0000 7fc7755f1640 1 --2- 172.21.15.112:0/1944980545 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc7700b2e60 0x7fc770152d80 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fc76000b420 tx=0x7fc76000b8e0 comp rx=0 tx=0).stop 2024-09-21T00:27:11.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.933+0000 7fc7755f1640 1 -- 172.21.15.112:0/1944980545 shutdown_connections 2024-09-21T00:27:11.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.933+0000 7fc7755f1640 1 --2- 172.21.15.112:0/1944980545 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc770162590 0x7fc770160cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:11.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.933+0000 7fc7755f1640 1 --2- 172.21.15.112:0/1944980545 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc7700cfe00 0x7fc770160790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:11.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.933+0000 7fc7755f1640 1 --2- 172.21.15.112:0/1944980545 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fc75c078b90 0x7fc75c07b0a0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:11.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.933+0000 7fc7755f1640 1 --2- 172.21.15.112:0/1944980545 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc7700b2e60 0x7fc770152d80 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:11.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.933+0000 7fc7755f1640 1 -- 172.21.15.112:0/1944980545 >> 172.21.15.112:0/1944980545 conn(0x7fc7700d4440 msgr2=0x7fc77006dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:11.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.933+0000 7fc7755f1640 1 -- 172.21.15.112:0/1944980545 shutdown_connections 2024-09-21T00:27:11.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:11.933+0000 7fc7755f1640 1 -- 172.21.15.112:0/1944980545 wait complete. 2024-09-21T00:27:11.946 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":91,"stamp":"2024-09-21T00:27:11.109326+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":105,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":77,"ondisk_log_size":77,"up":86,"acting":86,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":7,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":221600,"kb_used_data":5376,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753120,"statfs":{"total":773094113280,"available":772867194880,"internally_reserved":0,"allocated":5505024,"data_stored":3025554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.557642"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-21T00:27:07.579407+0000","last_change":"2024-09-21T00:27:07.574598+0000","last_active":"2024-09-21T00:27:07.579407+0000","last_peered":"2024-09-21T00:27:07.579407+0000","last_clean":"2024-09-21T00:27:07.579407+0000","last_became_active":"2024-09-21T00:27:07.574242+0000","last_became_peered":"2024-09-21T00:27:07.574242+0000","last_unstale":"2024-09-21T00:27:07.579407+0000","last_undegraded":"2024-09-21T00:27:07.579407+0000","last_fullsized":"2024-09-21T00:27:07.579407+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T00:55:01.680636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-21T00:27:07.593752+0000","last_change":"2024-09-21T00:27:07.581689+0000","last_active":"2024-09-21T00:27:07.593752+0000","last_peered":"2024-09-21T00:27:07.593752+0000","last_clean":"2024-09-21T00:27:07.593752+0000","last_became_active":"2024-09-21T00:27:07.581012+0000","last_became_peered":"2024-09-21T00:27:07.581012+0000","last_unstale":"2024-09-21T00:27:07.593752+0000","last_undegraded":"2024-09-21T00:27:07.593752+0000","last_fullsized":"2024-09-21T00:27:07.593752+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:08:00.101537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-21T00:27:07.579280+0000","last_change":"2024-09-21T00:27:07.567996+0000","last_active":"2024-09-21T00:27:07.579280+0000","last_peered":"2024-09-21T00:27:07.579280+0000","last_clean":"2024-09-21T00:27:07.579280+0000","last_became_active":"2024-09-21T00:27:07.567702+0000","last_became_peered":"2024-09-21T00:27:07.567702+0000","last_unstale":"2024-09-21T00:27:07.579280+0000","last_undegraded":"2024-09-21T00:27:07.579280+0000","last_fullsized":"2024-09-21T00:27:07.579280+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:29:32.175256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-21T00:27:07.574128+0000","last_change":"2024-09-21T00:27:07.567050+0000","last_active":"2024-09-21T00:27:07.574128+0000","last_peered":"2024-09-21T00:27:07.574128+0000","last_clean":"2024-09-21T00:27:07.574128+0000","last_became_active":"2024-09-21T00:27:07.566711+0000","last_became_peered":"2024-09-21T00:27:07.566711+0000","last_unstale":"2024-09-21T00:27:07.574128+0000","last_undegraded":"2024-09-21T00:27:07.574128+0000","last_fullsized":"2024-09-21T00:27:07.574128+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:53:27.777241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-21T00:27:07.574359+0000","last_change":"2024-09-21T00:27:07.567067+0000","last_active":"2024-09-21T00:27:07.574359+0000","last_peered":"2024-09-21T00:27:07.574359+0000","last_clean":"2024-09-21T00:27:07.574359+0000","last_became_active":"2024-09-21T00:27:07.566709+0000","last_became_peered":"2024-09-21T00:27:07.566709+0000","last_unstale":"2024-09-21T00:27:07.574359+0000","last_undegraded":"2024-09-21T00:27:07.574359+0000","last_fullsized":"2024-09-21T00:27:07.574359+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:36:49.037229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-21T00:27:07.595592+0000","last_change":"2024-09-21T00:27:07.586170+0000","last_active":"2024-09-21T00:27:07.595592+0000","last_peered":"2024-09-21T00:27:07.595592+0000","last_clean":"2024-09-21T00:27:07.595592+0000","last_became_active":"2024-09-21T00:27:07.585790+0000","last_became_peered":"2024-09-21T00:27:07.585790+0000","last_unstale":"2024-09-21T00:27:07.595592+0000","last_undegraded":"2024-09-21T00:27:07.595592+0000","last_fullsized":"2024-09-21T00:27:07.595592+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:27:32.413929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-21T00:27:05.556637+0000","last_change":"2024-09-21T00:27:05.556637+0000","last_active":"2024-09-21T00:27:05.543789+0000","last_peered":"2024-09-21T00:27:05.543789+0000","last_clean":"2024-09-21T00:27:05.543789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:05.556637+0000","last_undegraded":"2024-09-21T00:27:05.556637+0000","last_fullsized":"2024-09-21T00:27:05.556637+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1e","version":"48'1","reported_seq":14,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-21T00:27:07.600346+0000","last_change":"2024-09-21T00:27:06.578147+0000","last_active":"2024-09-21T00:27:07.600346+0000","last_peered":"2024-09-21T00:27:07.600346+0000","last_clean":"2024-09-21T00:27:07.600346+0000","last_became_active":"2024-09-21T00:27:06.577845+0000","last_became_peered":"2024-09-21T00:27:06.577845+0000","last_unstale":"2024-09-21T00:27:07.600346+0000","last_undegraded":"2024-09-21T00:27:07.600346+0000","last_fullsized":"2024-09-21T00:27:07.600346+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:18:27.263054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1d","version":"48'2","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-21T00:27:07.599338+0000","last_change":"2024-09-21T00:27:06.573813+0000","last_active":"2024-09-21T00:27:07.599338+0000","last_peered":"2024-09-21T00:27:07.599338+0000","last_clean":"2024-09-21T00:27:07.599338+0000","last_became_active":"2024-09-21T00:27:06.573532+0000","last_became_peered":"2024-09-21T00:27:06.573532+0000","last_unstale":"2024-09-21T00:27:07.599338+0000","last_undegraded":"2024-09-21T00:27:07.599338+0000","last_fullsized":"2024-09-21T00:27:07.599338+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:31:15.501365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-21T00:27:05.555296+0000","last_change":"2024-09-21T00:27:05.555296+0000","last_active":"2024-09-21T00:27:05.543789+0000","last_peered":"2024-09-21T00:27:05.543789+0000","last_clean":"2024-09-21T00:27:05.543789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:05.555296+0000","last_undegraded":"2024-09-21T00:27:05.555296+0000","last_fullsized":"2024-09-21T00:27:05.555296+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-21T00:27:05.562604+0000","last_change":"2024-09-21T00:27:05.562604+0000","last_active":"2024-09-21T00:27:05.543789+0000","last_peered":"2024-09-21T00:27:05.543789+0000","last_clean":"2024-09-21T00:27:05.543789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:05.562604+0000","last_undegraded":"2024-09-21T00:27:05.562604+0000","last_fullsized":"2024-09-21T00:27:05.562604+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-21T00:27:05.557122+0000","last_change":"2024-09-21T00:27:05.557122+0000","last_active":"2024-09-21T00:27:05.543789+0000","last_peered":"2024-09-21T00:27:05.543789+0000","last_clean":"2024-09-21T00:27:05.543789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:05.557122+0000","last_undegraded":"2024-09-21T00:27:05.557122+0000","last_fullsized":"2024-09-21T00:27:05.557122+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-21T00:27:05.555459+0000","last_change":"2024-09-21T00:27:05.555459+0000","last_active":"2024-09-21T00:27:05.543789+0000","last_peered":"2024-09-21T00:27:05.543789+0000","last_clean":"2024-09-21T00:27:05.543789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:05.555459+0000","last_undegraded":"2024-09-21T00:27:05.555459+0000","last_fullsized":"2024-09-21T00:27:05.555459+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-21T00:27:05.562527+0000","last_change":"2024-09-21T00:27:05.562527+0000","last_active":"2024-09-21T00:27:05.543789+0000","last_peered":"2024-09-21T00:27:05.543789+0000","last_clean":"2024-09-21T00:27:05.543789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:05.562527+0000","last_undegraded":"2024-09-21T00:27:05.562527+0000","last_fullsized":"2024-09-21T00:27:05.562527+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-21T00:27:05.562841+0000","last_change":"2024-09-21T00:27:05.562841+0000","last_active":"2024-09-21T00:27:05.543789+0000","last_peered":"2024-09-21T00:27:05.543789+0000","last_clean":"2024-09-21T00:27:05.543789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:05.562841+0000","last_undegraded":"2024-09-21T00:27:05.562841+0000","last_fullsized":"2024-09-21T00:27:05.562841+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-21T00:27:05.558519+0000","last_change":"2024-09-21T00:27:05.558519+0000","last_active":"2024-09-21T00:27:05.543789+0000","last_peered":"2024-09-21T00:27:05.543789+0000","last_clean":"2024-09-21T00:27:05.543789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:05.558519+0000","last_undegraded":"2024-09-21T00:27:05.558519+0000","last_fullsized":"2024-09-21T00:27:05.558519+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-21T00:27:05.563390+0000","last_change":"2024-09-21T00:27:05.563390+0000","last_active":"2024-09-21T00:27:05.543789+0000","last_peered":"2024-09-21T00:27:05.543789+0000","last_clean":"2024-09-21T00:27:05.543789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:05.563390+0000","last_undegraded":"2024-09-21T00:27:05.563390+0000","last_fullsized":"2024-09-21T00:27:05.563390+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-21T00:27:05.554843+0000","last_change":"2024-09-21T00:27:05.554843+0000","last_active":"2024-09-21T00:27:05.543789+0000","last_peered":"2024-09-21T00:27:05.543789+0000","last_clean":"2024-09-21T00:27:05.543789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:05.554843+0000","last_undegraded":"2024-09-21T00:27:05.554843+0000","last_fullsized":"2024-09-21T00:27:05.554843+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-21T00:27:05.562134+0000","last_change":"2024-09-21T00:27:05.562134+0000","last_active":"2024-09-21T00:27:05.543789+0000","last_peered":"2024-09-21T00:27:05.543789+0000","last_clean":"2024-09-21T00:27:05.543789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:05.562134+0000","last_undegraded":"2024-09-21T00:27:05.562134+0000","last_fullsized":"2024-09-21T00:27:05.562134+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-21T00:27:05.556088+0000","last_change":"2024-09-21T00:27:05.556088+0000","last_active":"2024-09-21T00:27:05.543789+0000","last_peered":"2024-09-21T00:27:05.543789+0000","last_clean":"2024-09-21T00:27:05.543789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:05.556088+0000","last_undegraded":"2024-09-21T00:27:05.556088+0000","last_fullsized":"2024-09-21T00:27:05.556088+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-21T00:27:05.562785+0000","last_change":"2024-09-21T00:27:05.562785+0000","last_active":"2024-09-21T00:27:05.543789+0000","last_peered":"2024-09-21T00:27:05.543789+0000","last_clean":"2024-09-21T00:27:05.543789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:05.562785+0000","last_undegraded":"2024-09-21T00:27:05.562785+0000","last_fullsized":"2024-09-21T00:27:05.562785+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-21T00:27:05.554830+0000","last_change":"2024-09-21T00:27:05.554830+0000","last_active":"2024-09-21T00:27:05.543789+0000","last_peered":"2024-09-21T00:27:05.543789+0000","last_clean":"2024-09-21T00:27:05.543789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:05.554830+0000","last_undegraded":"2024-09-21T00:27:05.554830+0000","last_fullsized":"2024-09-21T00:27:05.554830+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-21T00:27:05.561823+0000","last_change":"2024-09-21T00:27:05.561823+0000","last_active":"2024-09-21T00:27:05.543789+0000","last_peered":"2024-09-21T00:27:05.543789+0000","last_clean":"2024-09-21T00:27:05.543789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:05.561823+0000","last_undegraded":"2024-09-21T00:27:05.561823+0000","last_fullsized":"2024-09-21T00:27:05.561823+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-21T00:27:07.594119+0000","last_change":"2024-09-21T00:27:07.585242+0000","last_active":"2024-09-21T00:27:07.594119+0000","last_peered":"2024-09-21T00:27:07.594119+0000","last_clean":"2024-09-21T00:27:07.594119+0000","last_became_active":"2024-09-21T00:27:07.584922+0000","last_became_peered":"2024-09-21T00:27:07.584922+0000","last_unstale":"2024-09-21T00:27:07.594119+0000","last_undegraded":"2024-09-21T00:27:07.594119+0000","last_fullsized":"2024-09-21T00:27:07.594119+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:52:07.026811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-21T00:27:05.562632+0000","last_change":"2024-09-21T00:27:05.562632+0000","last_active":"2024-09-21T00:27:05.543789+0000","last_peered":"2024-09-21T00:27:05.543789+0000","last_clean":"2024-09-21T00:27:05.543789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:05.562632+0000","last_undegraded":"2024-09-21T00:27:05.562632+0000","last_fullsized":"2024-09-21T00:27:05.562632+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-21T00:27:05.555437+0000","last_change":"2024-09-21T00:27:05.555437+0000","last_active":"2024-09-21T00:27:05.543789+0000","last_peered":"2024-09-21T00:27:05.543789+0000","last_clean":"2024-09-21T00:27:05.543789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:05.555437+0000","last_undegraded":"2024-09-21T00:27:05.555437+0000","last_fullsized":"2024-09-21T00:27:05.555437+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-21T00:27:07.580102+0000","last_change":"2024-09-21T00:27:07.569099+0000","last_active":"2024-09-21T00:27:07.580102+0000","last_peered":"2024-09-21T00:27:07.580102+0000","last_clean":"2024-09-21T00:27:07.580102+0000","last_became_active":"2024-09-21T00:27:07.568739+0000","last_became_peered":"2024-09-21T00:27:07.568739+0000","last_unstale":"2024-09-21T00:27:07.580102+0000","last_undegraded":"2024-09-21T00:27:07.580102+0000","last_fullsized":"2024-09-21T00:27:07.580102+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:18:56.106026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-21T00:27:05.556442+0000","last_change":"2024-09-21T00:27:05.556442+0000","last_active":"2024-09-21T00:27:05.543789+0000","last_peered":"2024-09-21T00:27:05.543789+0000","last_clean":"2024-09-21T00:27:05.543789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:05.556442+0000","last_undegraded":"2024-09-21T00:27:05.556442+0000","last_fullsized":"2024-09-21T00:27:05.556442+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.b","version":"48'2","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-21T00:27:07.599703+0000","last_change":"2024-09-21T00:27:06.580120+0000","last_active":"2024-09-21T00:27:07.599703+0000","last_peered":"2024-09-21T00:27:07.599703+0000","last_clean":"2024-09-21T00:27:07.599703+0000","last_became_active":"2024-09-21T00:27:06.579795+0000","last_became_peered":"2024-09-21T00:27:06.579795+0000","last_unstale":"2024-09-21T00:27:07.599703+0000","last_undegraded":"2024-09-21T00:27:07.599703+0000","last_fullsized":"2024-09-21T00:27:07.599703+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:31:18.308257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-21T00:27:05.556012+0000","last_change":"2024-09-21T00:27:05.556012+0000","last_active":"2024-09-21T00:27:05.543789+0000","last_peered":"2024-09-21T00:27:05.543789+0000","last_clean":"2024-09-21T00:27:05.543789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:05.556012+0000","last_undegraded":"2024-09-21T00:27:05.556012+0000","last_fullsized":"2024-09-21T00:27:05.556012+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-21T00:27:05.561999+0000","last_change":"2024-09-21T00:27:05.561999+0000","last_active":"2024-09-21T00:27:05.543789+0000","last_peered":"2024-09-21T00:27:05.543789+0000","last_clean":"2024-09-21T00:27:05.543789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:05.561999+0000","last_undegraded":"2024-09-21T00:27:05.561999+0000","last_fullsized":"2024-09-21T00:27:05.561999+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-21T00:27:07.595184+0000","last_change":"2024-09-21T00:27:07.590250+0000","last_active":"2024-09-21T00:27:07.595184+0000","last_peered":"2024-09-21T00:27:07.595184+0000","last_clean":"2024-09-21T00:27:07.595184+0000","last_became_active":"2024-09-21T00:27:07.589930+0000","last_became_peered":"2024-09-21T00:27:07.589930+0000","last_unstale":"2024-09-21T00:27:07.595184+0000","last_undegraded":"2024-09-21T00:27:07.595184+0000","last_fullsized":"2024-09-21T00:27:07.595184+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:55:47.579955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-21T00:27:05.561751+0000","last_change":"2024-09-21T00:27:05.561751+0000","last_active":"2024-09-21T00:27:05.543789+0000","last_peered":"2024-09-21T00:27:05.543789+0000","last_clean":"2024-09-21T00:27:05.543789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:05.561751+0000","last_undegraded":"2024-09-21T00:27:05.561751+0000","last_fullsized":"2024-09-21T00:27:05.561751+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-21T00:27:05.556126+0000","last_change":"2024-09-21T00:27:05.556126+0000","last_active":"2024-09-21T00:27:05.543789+0000","last_peered":"2024-09-21T00:27:05.543789+0000","last_clean":"2024-09-21T00:27:05.543789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:05.556126+0000","last_undegraded":"2024-09-21T00:27:05.556126+0000","last_fullsized":"2024-09-21T00:27:05.556126+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-21T00:27:07.593806+0000","last_change":"2024-09-21T00:27:07.581995+0000","last_active":"2024-09-21T00:27:07.593806+0000","last_peered":"2024-09-21T00:27:07.593806+0000","last_clean":"2024-09-21T00:27:07.593806+0000","last_became_active":"2024-09-21T00:27:07.581400+0000","last_became_peered":"2024-09-21T00:27:07.581400+0000","last_unstale":"2024-09-21T00:27:07.593806+0000","last_undegraded":"2024-09-21T00:27:07.593806+0000","last_fullsized":"2024-09-21T00:27:07.593806+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:20:00.370047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-21T00:27:05.562808+0000","last_change":"2024-09-21T00:27:05.562808+0000","last_active":"2024-09-21T00:27:05.543789+0000","last_peered":"2024-09-21T00:27:05.543789+0000","last_clean":"2024-09-21T00:27:05.543789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:05.562808+0000","last_undegraded":"2024-09-21T00:27:05.562808+0000","last_fullsized":"2024-09-21T00:27:05.562808+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-21T00:27:05.562127+0000","last_change":"2024-09-21T00:27:05.562127+0000","last_active":"2024-09-21T00:27:05.543789+0000","last_peered":"2024-09-21T00:27:05.543789+0000","last_clean":"2024-09-21T00:27:05.543789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:05.562127+0000","last_undegraded":"2024-09-21T00:27:05.562127+0000","last_fullsized":"2024-09-21T00:27:05.562127+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-21T00:27:05.554822+0000","last_change":"2024-09-21T00:27:05.554822+0000","last_active":"2024-09-21T00:27:05.543789+0000","last_peered":"2024-09-21T00:27:05.543789+0000","last_clean":"2024-09-21T00:27:05.543789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:05.554822+0000","last_undegraded":"2024-09-21T00:27:05.554822+0000","last_fullsized":"2024-09-21T00:27:05.554822+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-21T00:27:05.554816+0000","last_change":"2024-09-21T00:27:05.554816+0000","last_active":"2024-09-21T00:27:05.543789+0000","last_peered":"2024-09-21T00:27:05.543789+0000","last_clean":"2024-09-21T00:27:05.543789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:05.554816+0000","last_undegraded":"2024-09-21T00:27:05.554816+0000","last_fullsized":"2024-09-21T00:27:05.554816+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-21T00:27:05.557773+0000","last_change":"2024-09-21T00:27:05.557773+0000","last_active":"2024-09-21T00:27:05.543789+0000","last_peered":"2024-09-21T00:27:05.543789+0000","last_clean":"2024-09-21T00:27:05.543789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:05.557773+0000","last_undegraded":"2024-09-21T00:27:05.557773+0000","last_fullsized":"2024-09-21T00:27:05.557773+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-21T00:27:05.555561+0000","last_change":"2024-09-21T00:27:05.555561+0000","last_active":"2024-09-21T00:27:05.543789+0000","last_peered":"2024-09-21T00:27:05.543789+0000","last_clean":"2024-09-21T00:27:05.543789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:05.555561+0000","last_undegraded":"2024-09-21T00:27:05.555561+0000","last_fullsized":"2024-09-21T00:27:05.555561+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-21T00:27:05.561883+0000","last_change":"2024-09-21T00:27:05.561883+0000","last_active":"2024-09-21T00:27:05.543789+0000","last_peered":"2024-09-21T00:27:05.543789+0000","last_clean":"2024-09-21T00:27:05.543789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:05.561883+0000","last_undegraded":"2024-09-21T00:27:05.561883+0000","last_fullsized":"2024-09-21T00:27:05.561883+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":153,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T00:27:05.563691+0000","last_change":"2024-09-21T00:25:38.725792+0000","last_active":"2024-09-21T00:27:05.563691+0000","last_peered":"2024-09-21T00:27:05.563691+0000","last_clean":"2024-09-21T00:27:05.563691+0000","last_became_active":"2024-09-21T00:25:38.724668+0000","last_became_peered":"2024-09-21T00:25:38.724668+0000","last_unstale":"2024-09-21T00:27:05.563691+0000","last_undegraded":"2024-09-21T00:27:05.563691+0000","last_fullsized":"2024-09-21T00:27:05.563691+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:37.163216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:37.163216+0000","last_clean_scrub_stamp":"2024-09-21T00:25:37.163216+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:14:36.588660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":20,"acting":20,"num_store_stats":1},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509461,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28132,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343708,"statfs":{"total":96636764160,"available":96607956992,"internally_reserved":0,"allocated":1130496,"data_stored":819241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509462,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27552,"kb_used_data":524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344288,"statfs":{"total":96636764160,"available":96608550912,"internally_reserved":0,"allocated":536576,"data_stored":228873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509461,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27552,"kb_used_data":524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344288,"statfs":{"total":96636764160,"available":96608550912,"internally_reserved":0,"allocated":536576,"data_stored":228873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509460,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27552,"kb_used_data":524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344288,"statfs":{"total":96636764160,"available":96608550912,"internally_reserved":0,"allocated":536576,"data_stored":228873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509462,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27552,"kb_used_data":524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344288,"statfs":{"total":96636764160,"available":96608550912,"internally_reserved":0,"allocated":536576,"data_stored":228873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509462,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27552,"kb_used_data":524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344288,"statfs":{"total":96636764160,"available":96608550912,"internally_reserved":0,"allocated":536576,"data_stored":228873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509462,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28132,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343708,"statfs":{"total":96636764160,"available":96607956992,"internally_reserved":0,"allocated":1130496,"data_stored":819241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27576,"kb_used_data":548,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344264,"statfs":{"total":96636764160,"available":96608526336,"internally_reserved":0,"allocated":561152,"data_stored":242707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T00:27:11.949 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T00:27:12.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.098+0000 7f8dac990640 1 -- 172.21.15.112:0/2775426122 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8d94003a50 msgr2=0x7f8d94003ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:12.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.098+0000 7f8dac990640 1 --2- 172.21.15.112:0/2775426122 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8d94003a50 0x7f8d94003ee0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f8d98009b50 tx=0x7f8d94005830 comp rx=0 tx=0).stop 2024-09-21T00:27:12.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.098+0000 7f8dac990640 1 -- 172.21.15.112:0/2775426122 shutdown_connections 2024-09-21T00:27:12.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.098+0000 7f8dac990640 1 --2- 172.21.15.112:0/2775426122 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8d94003a50 0x7f8d94003ee0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:12.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.098+0000 7f8dac990640 1 -- 172.21.15.112:0/2775426122 >> 172.21.15.112:0/2775426122 conn(0x7f8da80b2cc0 msgr2=0x7f8da80b30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:12.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.098+0000 7f8dac990640 1 -- 172.21.15.112:0/2775426122 shutdown_connections 2024-09-21T00:27:12.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.098+0000 7f8dac990640 1 -- 172.21.15.112:0/2775426122 wait complete. 2024-09-21T00:27:12.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.100+0000 7f8dac990640 1 Processor -- start 2024-09-21T00:27:12.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.100+0000 7f8dac990640 1 -- start start 2024-09-21T00:27:12.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.101+0000 7f8dac990640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8d94003a50 0x7f8da814df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:12.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.101+0000 7f8dac990640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8da814ec00 0x7f8da81e72d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:12.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.101+0000 7f8dac990640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8da8154f70 0x7f8da8155380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:12.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.101+0000 7f8dac990640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8da80742e0 con 0x7f8da814ec00 2024-09-21T00:27:12.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.101+0000 7f8dac990640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8da8074160 con 0x7f8d94003a50 2024-09-21T00:27:12.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.101+0000 7f8dac990640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f8da8074460 con 0x7f8da8154f70 2024-09-21T00:27:12.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.101+0000 7f8d9dd9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8da814ec00 0x7f8da81e72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:12.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.101+0000 7f8d9dd9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8da814ec00 0x7f8da81e72d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55452/0 (socket says 172.21.15.112:55452) 2024-09-21T00:27:12.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.101+0000 7f8d9dd9b640 1 -- 172.21.15.112:0/3931318984 learned_addr learned my addr 172.21.15.112:0/3931318984 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:27:12.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.101+0000 7f8da6d9d640 1 --2- 172.21.15.112:0/3931318984 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8da8154f70 0x7f8da8155380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:12.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.101+0000 7f8da659c640 1 --2- 172.21.15.112:0/3931318984 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8d94003a50 0x7f8da814df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:12.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.101+0000 7f8d9dd9b640 1 -- 172.21.15.112:0/3931318984 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8da8154f70 msgr2=0x7f8da8155380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:12.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.101+0000 7f8d9dd9b640 1 --2- 172.21.15.112:0/3931318984 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8da8154f70 0x7f8da8155380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:12.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.101+0000 7f8d9dd9b640 1 -- 172.21.15.112:0/3931318984 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8d94003a50 msgr2=0x7f8da814df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:12.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.101+0000 7f8d9dd9b640 1 --2- 172.21.15.112:0/3931318984 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8d94003a50 0x7f8da814df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:12.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.101+0000 7f8d9dd9b640 1 -- 172.21.15.112:0/3931318984 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8da815fe20 con 0x7f8da814ec00 2024-09-21T00:27:12.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.102+0000 7f8d9dd9b640 1 --2- 172.21.15.112:0/3931318984 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8da814ec00 0x7f8da81e72d0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f8d9000e6d0 tx=0x7f8d9000ead0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:12.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.102+0000 7f8d9f7fe640 1 -- 172.21.15.112:0/3931318984 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d9000a6f0 con 0x7f8da814ec00 2024-09-21T00:27:12.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.102+0000 7f8d9f7fe640 1 -- 172.21.15.112:0/3931318984 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8d9000a890 con 0x7f8da814ec00 2024-09-21T00:27:12.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.102+0000 7f8dac990640 1 -- 172.21.15.112:0/3931318984 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8da814e4b0 con 0x7f8da814ec00 2024-09-21T00:27:12.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.102+0000 7f8d9f7fe640 1 -- 172.21.15.112:0/3931318984 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d9000ace0 con 0x7f8da814ec00 2024-09-21T00:27:12.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.102+0000 7f8dac990640 1 -- 172.21.15.112:0/3931318984 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8da81e7fc0 con 0x7f8da814ec00 2024-09-21T00:27:12.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.103+0000 7f8dac990640 1 -- 172.21.15.112:0/3931318984 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8da80b4010 con 0x7f8da814ec00 2024-09-21T00:27:12.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.103+0000 7f8d9f7fe640 1 -- 172.21.15.112:0/3931318984 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8d90018420 con 0x7f8da814ec00 2024-09-21T00:27:12.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.104+0000 7f8d9f7fe640 1 --2- 172.21.15.112:0/3931318984 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f8d88078b90 0x7f8d8807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:12.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.104+0000 7f8d9f7fe640 1 -- 172.21.15.112:0/3931318984 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 6101+0+0 (secure 0 0 0) 0x7f8d9009c8a0 con 0x7f8da814ec00 2024-09-21T00:27:12.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.104+0000 7f8da659c640 1 --2- 172.21.15.112:0/3931318984 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f8d88078b90 0x7f8d8807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:12.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.105+0000 7f8da659c640 1 --2- 172.21.15.112:0/3931318984 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f8d88078b90 0x7f8d8807b0a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f8d98035000 tx=0x7f8d980046c0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:12.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.109+0000 7f8d9f7fe640 1 -- 172.21.15.112:0/3931318984 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f8da814e4b0 con 0x7f8da814ec00 2024-09-21T00:27:12.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.281+0000 7f8dac990640 1 -- 172.21.15.112:0/3931318984 --> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8da815af90 con 0x7f8d88078b90 2024-09-21T00:27:12.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.286+0000 7f8d9f7fe640 1 -- 172.21.15.112:0/3931318984 <== mgr.4103 v2:172.21.15.187:6800/2921522906 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+241701 (secure 0 0 0) 0x7f8da815af90 con 0x7f8d88078b90 2024-09-21T00:27:12.290 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:27:12.291 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2024-09-21T00:27:12.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.290+0000 7f8dac990640 1 -- 172.21.15.112:0/3931318984 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f8d88078b90 msgr2=0x7f8d8807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:12.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.290+0000 7f8dac990640 1 --2- 172.21.15.112:0/3931318984 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f8d88078b90 0x7f8d8807b0a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f8d98035000 tx=0x7f8d980046c0 comp rx=0 tx=0).stop 2024-09-21T00:27:12.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.290+0000 7f8dac990640 1 -- 172.21.15.112:0/3931318984 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8da814ec00 msgr2=0x7f8da81e72d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:12.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.290+0000 7f8dac990640 1 --2- 172.21.15.112:0/3931318984 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8da814ec00 0x7f8da81e72d0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f8d9000e6d0 tx=0x7f8d9000ead0 comp rx=0 tx=0).stop 2024-09-21T00:27:12.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.290+0000 7f8dac990640 1 -- 172.21.15.112:0/3931318984 shutdown_connections 2024-09-21T00:27:12.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.290+0000 7f8dac990640 1 --2- 172.21.15.112:0/3931318984 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8da8154f70 0x7f8da8155380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:12.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.290+0000 7f8dac990640 1 --2- 172.21.15.112:0/3931318984 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f8d88078b90 0x7f8d8807b0a0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:12.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.290+0000 7f8dac990640 1 --2- 172.21.15.112:0/3931318984 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8da814ec00 0x7f8da81e72d0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:12.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.290+0000 7f8dac990640 1 --2- 172.21.15.112:0/3931318984 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8d94003a50 0x7f8da814df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:12.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.290+0000 7f8dac990640 1 -- 172.21.15.112:0/3931318984 >> 172.21.15.112:0/3931318984 conn(0x7f8da80b2cc0 msgr2=0x7f8da81509b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:12.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.290+0000 7f8dac990640 1 -- 172.21.15.112:0/3931318984 shutdown_connections 2024-09-21T00:27:12.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.290+0000 7f8dac990640 1 -- 172.21.15.112:0/3931318984 wait complete. 2024-09-21T00:27:12.303 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":91,"stamp":"2024-09-21T00:27:11.109326+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":105,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":77,"ondisk_log_size":77,"up":86,"acting":86,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":7,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":221600,"kb_used_data":5376,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753120,"statfs":{"total":773094113280,"available":772867194880,"internally_reserved":0,"allocated":5505024,"data_stored":3025554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.557642"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-21T00:27:07.579407+0000","last_change":"2024-09-21T00:27:07.574598+0000","last_active":"2024-09-21T00:27:07.579407+0000","last_peered":"2024-09-21T00:27:07.579407+0000","last_clean":"2024-09-21T00:27:07.579407+0000","last_became_active":"2024-09-21T00:27:07.574242+0000","last_became_peered":"2024-09-21T00:27:07.574242+0000","last_unstale":"2024-09-21T00:27:07.579407+0000","last_undegraded":"2024-09-21T00:27:07.579407+0000","last_fullsized":"2024-09-21T00:27:07.579407+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T00:55:01.680636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-21T00:27:07.593752+0000","last_change":"2024-09-21T00:27:07.581689+0000","last_active":"2024-09-21T00:27:07.593752+0000","last_peered":"2024-09-21T00:27:07.593752+0000","last_clean":"2024-09-21T00:27:07.593752+0000","last_became_active":"2024-09-21T00:27:07.581012+0000","last_became_peered":"2024-09-21T00:27:07.581012+0000","last_unstale":"2024-09-21T00:27:07.593752+0000","last_undegraded":"2024-09-21T00:27:07.593752+0000","last_fullsized":"2024-09-21T00:27:07.593752+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:08:00.101537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-21T00:27:07.579280+0000","last_change":"2024-09-21T00:27:07.567996+0000","last_active":"2024-09-21T00:27:07.579280+0000","last_peered":"2024-09-21T00:27:07.579280+0000","last_clean":"2024-09-21T00:27:07.579280+0000","last_became_active":"2024-09-21T00:27:07.567702+0000","last_became_peered":"2024-09-21T00:27:07.567702+0000","last_unstale":"2024-09-21T00:27:07.579280+0000","last_undegraded":"2024-09-21T00:27:07.579280+0000","last_fullsized":"2024-09-21T00:27:07.579280+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:29:32.175256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-21T00:27:07.574128+0000","last_change":"2024-09-21T00:27:07.567050+0000","last_active":"2024-09-21T00:27:07.574128+0000","last_peered":"2024-09-21T00:27:07.574128+0000","last_clean":"2024-09-21T00:27:07.574128+0000","last_became_active":"2024-09-21T00:27:07.566711+0000","last_became_peered":"2024-09-21T00:27:07.566711+0000","last_unstale":"2024-09-21T00:27:07.574128+0000","last_undegraded":"2024-09-21T00:27:07.574128+0000","last_fullsized":"2024-09-21T00:27:07.574128+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:53:27.777241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-21T00:27:07.574359+0000","last_change":"2024-09-21T00:27:07.567067+0000","last_active":"2024-09-21T00:27:07.574359+0000","last_peered":"2024-09-21T00:27:07.574359+0000","last_clean":"2024-09-21T00:27:07.574359+0000","last_became_active":"2024-09-21T00:27:07.566709+0000","last_became_peered":"2024-09-21T00:27:07.566709+0000","last_unstale":"2024-09-21T00:27:07.574359+0000","last_undegraded":"2024-09-21T00:27:07.574359+0000","last_fullsized":"2024-09-21T00:27:07.574359+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:36:49.037229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-21T00:27:07.595592+0000","last_change":"2024-09-21T00:27:07.586170+0000","last_active":"2024-09-21T00:27:07.595592+0000","last_peered":"2024-09-21T00:27:07.595592+0000","last_clean":"2024-09-21T00:27:07.595592+0000","last_became_active":"2024-09-21T00:27:07.585790+0000","last_became_peered":"2024-09-21T00:27:07.585790+0000","last_unstale":"2024-09-21T00:27:07.595592+0000","last_undegraded":"2024-09-21T00:27:07.595592+0000","last_fullsized":"2024-09-21T00:27:07.595592+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:27:32.413929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-21T00:27:05.556637+0000","last_change":"2024-09-21T00:27:05.556637+0000","last_active":"2024-09-21T00:27:05.543789+0000","last_peered":"2024-09-21T00:27:05.543789+0000","last_clean":"2024-09-21T00:27:05.543789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:05.556637+0000","last_undegraded":"2024-09-21T00:27:05.556637+0000","last_fullsized":"2024-09-21T00:27:05.556637+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1e","version":"48'1","reported_seq":14,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-21T00:27:07.600346+0000","last_change":"2024-09-21T00:27:06.578147+0000","last_active":"2024-09-21T00:27:07.600346+0000","last_peered":"2024-09-21T00:27:07.600346+0000","last_clean":"2024-09-21T00:27:07.600346+0000","last_became_active":"2024-09-21T00:27:06.577845+0000","last_became_peered":"2024-09-21T00:27:06.577845+0000","last_unstale":"2024-09-21T00:27:07.600346+0000","last_undegraded":"2024-09-21T00:27:07.600346+0000","last_fullsized":"2024-09-21T00:27:07.600346+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:18:27.263054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1d","version":"48'2","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-21T00:27:07.599338+0000","last_change":"2024-09-21T00:27:06.573813+0000","last_active":"2024-09-21T00:27:07.599338+0000","last_peered":"2024-09-21T00:27:07.599338+0000","last_clean":"2024-09-21T00:27:07.599338+0000","last_became_active":"2024-09-21T00:27:06.573532+0000","last_became_peered":"2024-09-21T00:27:06.573532+0000","last_unstale":"2024-09-21T00:27:07.599338+0000","last_undegraded":"2024-09-21T00:27:07.599338+0000","last_fullsized":"2024-09-21T00:27:07.599338+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:31:15.501365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-21T00:27:05.555296+0000","last_change":"2024-09-21T00:27:05.555296+0000","last_active":"2024-09-21T00:27:05.543789+0000","last_peered":"2024-09-21T00:27:05.543789+0000","last_clean":"2024-09-21T00:27:05.543789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:05.555296+0000","last_undegraded":"2024-09-21T00:27:05.555296+0000","last_fullsized":"2024-09-21T00:27:05.555296+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-21T00:27:05.562604+0000","last_change":"2024-09-21T00:27:05.562604+0000","last_active":"2024-09-21T00:27:05.543789+0000","last_peered":"2024-09-21T00:27:05.543789+0000","last_clean":"2024-09-21T00:27:05.543789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:05.562604+0000","last_undegraded":"2024-09-21T00:27:05.562604+0000","last_fullsized":"2024-09-21T00:27:05.562604+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-21T00:27:05.557122+0000","last_change":"2024-09-21T00:27:05.557122+0000","last_active":"2024-09-21T00:27:05.543789+0000","last_peered":"2024-09-21T00:27:05.543789+0000","last_clean":"2024-09-21T00:27:05.543789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:05.557122+0000","last_undegraded":"2024-09-21T00:27:05.557122+0000","last_fullsized":"2024-09-21T00:27:05.557122+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-21T00:27:05.555459+0000","last_change":"2024-09-21T00:27:05.555459+0000","last_active":"2024-09-21T00:27:05.543789+0000","last_peered":"2024-09-21T00:27:05.543789+0000","last_clean":"2024-09-21T00:27:05.543789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:05.555459+0000","last_undegraded":"2024-09-21T00:27:05.555459+0000","last_fullsized":"2024-09-21T00:27:05.555459+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-21T00:27:05.562527+0000","last_change":"2024-09-21T00:27:05.562527+0000","last_active":"2024-09-21T00:27:05.543789+0000","last_peered":"2024-09-21T00:27:05.543789+0000","last_clean":"2024-09-21T00:27:05.543789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:05.562527+0000","last_undegraded":"2024-09-21T00:27:05.562527+0000","last_fullsized":"2024-09-21T00:27:05.562527+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-21T00:27:05.562841+0000","last_change":"2024-09-21T00:27:05.562841+0000","last_active":"2024-09-21T00:27:05.543789+0000","last_peered":"2024-09-21T00:27:05.543789+0000","last_clean":"2024-09-21T00:27:05.543789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:05.562841+0000","last_undegraded":"2024-09-21T00:27:05.562841+0000","last_fullsized":"2024-09-21T00:27:05.562841+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-21T00:27:05.558519+0000","last_change":"2024-09-21T00:27:05.558519+0000","last_active":"2024-09-21T00:27:05.543789+0000","last_peered":"2024-09-21T00:27:05.543789+0000","last_clean":"2024-09-21T00:27:05.543789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:05.558519+0000","last_undegraded":"2024-09-21T00:27:05.558519+0000","last_fullsized":"2024-09-21T00:27:05.558519+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-21T00:27:05.563390+0000","last_change":"2024-09-21T00:27:05.563390+0000","last_active":"2024-09-21T00:27:05.543789+0000","last_peered":"2024-09-21T00:27:05.543789+0000","last_clean":"2024-09-21T00:27:05.543789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:05.563390+0000","last_undegraded":"2024-09-21T00:27:05.563390+0000","last_fullsized":"2024-09-21T00:27:05.563390+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-21T00:27:05.554843+0000","last_change":"2024-09-21T00:27:05.554843+0000","last_active":"2024-09-21T00:27:05.543789+0000","last_peered":"2024-09-21T00:27:05.543789+0000","last_clean":"2024-09-21T00:27:05.543789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:05.554843+0000","last_undegraded":"2024-09-21T00:27:05.554843+0000","last_fullsized":"2024-09-21T00:27:05.554843+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-21T00:27:05.562134+0000","last_change":"2024-09-21T00:27:05.562134+0000","last_active":"2024-09-21T00:27:05.543789+0000","last_peered":"2024-09-21T00:27:05.543789+0000","last_clean":"2024-09-21T00:27:05.543789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:05.562134+0000","last_undegraded":"2024-09-21T00:27:05.562134+0000","last_fullsized":"2024-09-21T00:27:05.562134+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-21T00:27:05.556088+0000","last_change":"2024-09-21T00:27:05.556088+0000","last_active":"2024-09-21T00:27:05.543789+0000","last_peered":"2024-09-21T00:27:05.543789+0000","last_clean":"2024-09-21T00:27:05.543789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:05.556088+0000","last_undegraded":"2024-09-21T00:27:05.556088+0000","last_fullsized":"2024-09-21T00:27:05.556088+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-21T00:27:05.562785+0000","last_change":"2024-09-21T00:27:05.562785+0000","last_active":"2024-09-21T00:27:05.543789+0000","last_peered":"2024-09-21T00:27:05.543789+0000","last_clean":"2024-09-21T00:27:05.543789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:05.562785+0000","last_undegraded":"2024-09-21T00:27:05.562785+0000","last_fullsized":"2024-09-21T00:27:05.562785+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-21T00:27:05.554830+0000","last_change":"2024-09-21T00:27:05.554830+0000","last_active":"2024-09-21T00:27:05.543789+0000","last_peered":"2024-09-21T00:27:05.543789+0000","last_clean":"2024-09-21T00:27:05.543789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:05.554830+0000","last_undegraded":"2024-09-21T00:27:05.554830+0000","last_fullsized":"2024-09-21T00:27:05.554830+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-21T00:27:05.561823+0000","last_change":"2024-09-21T00:27:05.561823+0000","last_active":"2024-09-21T00:27:05.543789+0000","last_peered":"2024-09-21T00:27:05.543789+0000","last_clean":"2024-09-21T00:27:05.543789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:05.561823+0000","last_undegraded":"2024-09-21T00:27:05.561823+0000","last_fullsized":"2024-09-21T00:27:05.561823+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-21T00:27:07.594119+0000","last_change":"2024-09-21T00:27:07.585242+0000","last_active":"2024-09-21T00:27:07.594119+0000","last_peered":"2024-09-21T00:27:07.594119+0000","last_clean":"2024-09-21T00:27:07.594119+0000","last_became_active":"2024-09-21T00:27:07.584922+0000","last_became_peered":"2024-09-21T00:27:07.584922+0000","last_unstale":"2024-09-21T00:27:07.594119+0000","last_undegraded":"2024-09-21T00:27:07.594119+0000","last_fullsized":"2024-09-21T00:27:07.594119+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:52:07.026811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-21T00:27:05.562632+0000","last_change":"2024-09-21T00:27:05.562632+0000","last_active":"2024-09-21T00:27:05.543789+0000","last_peered":"2024-09-21T00:27:05.543789+0000","last_clean":"2024-09-21T00:27:05.543789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:05.562632+0000","last_undegraded":"2024-09-21T00:27:05.562632+0000","last_fullsized":"2024-09-21T00:27:05.562632+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-21T00:27:05.555437+0000","last_change":"2024-09-21T00:27:05.555437+0000","last_active":"2024-09-21T00:27:05.543789+0000","last_peered":"2024-09-21T00:27:05.543789+0000","last_clean":"2024-09-21T00:27:05.543789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:05.555437+0000","last_undegraded":"2024-09-21T00:27:05.555437+0000","last_fullsized":"2024-09-21T00:27:05.555437+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-21T00:27:07.580102+0000","last_change":"2024-09-21T00:27:07.569099+0000","last_active":"2024-09-21T00:27:07.580102+0000","last_peered":"2024-09-21T00:27:07.580102+0000","last_clean":"2024-09-21T00:27:07.580102+0000","last_became_active":"2024-09-21T00:27:07.568739+0000","last_became_peered":"2024-09-21T00:27:07.568739+0000","last_unstale":"2024-09-21T00:27:07.580102+0000","last_undegraded":"2024-09-21T00:27:07.580102+0000","last_fullsized":"2024-09-21T00:27:07.580102+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:18:56.106026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-21T00:27:05.556442+0000","last_change":"2024-09-21T00:27:05.556442+0000","last_active":"2024-09-21T00:27:05.543789+0000","last_peered":"2024-09-21T00:27:05.543789+0000","last_clean":"2024-09-21T00:27:05.543789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:05.556442+0000","last_undegraded":"2024-09-21T00:27:05.556442+0000","last_fullsized":"2024-09-21T00:27:05.556442+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.b","version":"48'2","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-21T00:27:07.599703+0000","last_change":"2024-09-21T00:27:06.580120+0000","last_active":"2024-09-21T00:27:07.599703+0000","last_peered":"2024-09-21T00:27:07.599703+0000","last_clean":"2024-09-21T00:27:07.599703+0000","last_became_active":"2024-09-21T00:27:06.579795+0000","last_became_peered":"2024-09-21T00:27:06.579795+0000","last_unstale":"2024-09-21T00:27:07.599703+0000","last_undegraded":"2024-09-21T00:27:07.599703+0000","last_fullsized":"2024-09-21T00:27:07.599703+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:31:18.308257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-21T00:27:05.556012+0000","last_change":"2024-09-21T00:27:05.556012+0000","last_active":"2024-09-21T00:27:05.543789+0000","last_peered":"2024-09-21T00:27:05.543789+0000","last_clean":"2024-09-21T00:27:05.543789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:05.556012+0000","last_undegraded":"2024-09-21T00:27:05.556012+0000","last_fullsized":"2024-09-21T00:27:05.556012+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-21T00:27:05.561999+0000","last_change":"2024-09-21T00:27:05.561999+0000","last_active":"2024-09-21T00:27:05.543789+0000","last_peered":"2024-09-21T00:27:05.543789+0000","last_clean":"2024-09-21T00:27:05.543789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:05.561999+0000","last_undegraded":"2024-09-21T00:27:05.561999+0000","last_fullsized":"2024-09-21T00:27:05.561999+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-21T00:27:07.595184+0000","last_change":"2024-09-21T00:27:07.590250+0000","last_active":"2024-09-21T00:27:07.595184+0000","last_peered":"2024-09-21T00:27:07.595184+0000","last_clean":"2024-09-21T00:27:07.595184+0000","last_became_active":"2024-09-21T00:27:07.589930+0000","last_became_peered":"2024-09-21T00:27:07.589930+0000","last_unstale":"2024-09-21T00:27:07.595184+0000","last_undegraded":"2024-09-21T00:27:07.595184+0000","last_fullsized":"2024-09-21T00:27:07.595184+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:55:47.579955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-21T00:27:05.561751+0000","last_change":"2024-09-21T00:27:05.561751+0000","last_active":"2024-09-21T00:27:05.543789+0000","last_peered":"2024-09-21T00:27:05.543789+0000","last_clean":"2024-09-21T00:27:05.543789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:05.561751+0000","last_undegraded":"2024-09-21T00:27:05.561751+0000","last_fullsized":"2024-09-21T00:27:05.561751+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-21T00:27:05.556126+0000","last_change":"2024-09-21T00:27:05.556126+0000","last_active":"2024-09-21T00:27:05.543789+0000","last_peered":"2024-09-21T00:27:05.543789+0000","last_clean":"2024-09-21T00:27:05.543789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:05.556126+0000","last_undegraded":"2024-09-21T00:27:05.556126+0000","last_fullsized":"2024-09-21T00:27:05.556126+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-21T00:27:07.593806+0000","last_change":"2024-09-21T00:27:07.581995+0000","last_active":"2024-09-21T00:27:07.593806+0000","last_peered":"2024-09-21T00:27:07.593806+0000","last_clean":"2024-09-21T00:27:07.593806+0000","last_became_active":"2024-09-21T00:27:07.581400+0000","last_became_peered":"2024-09-21T00:27:07.581400+0000","last_unstale":"2024-09-21T00:27:07.593806+0000","last_undegraded":"2024-09-21T00:27:07.593806+0000","last_fullsized":"2024-09-21T00:27:07.593806+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:20:00.370047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-21T00:27:05.562808+0000","last_change":"2024-09-21T00:27:05.562808+0000","last_active":"2024-09-21T00:27:05.543789+0000","last_peered":"2024-09-21T00:27:05.543789+0000","last_clean":"2024-09-21T00:27:05.543789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:05.562808+0000","last_undegraded":"2024-09-21T00:27:05.562808+0000","last_fullsized":"2024-09-21T00:27:05.562808+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-21T00:27:05.562127+0000","last_change":"2024-09-21T00:27:05.562127+0000","last_active":"2024-09-21T00:27:05.543789+0000","last_peered":"2024-09-21T00:27:05.543789+0000","last_clean":"2024-09-21T00:27:05.543789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:05.562127+0000","last_undegraded":"2024-09-21T00:27:05.562127+0000","last_fullsized":"2024-09-21T00:27:05.562127+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-21T00:27:05.554822+0000","last_change":"2024-09-21T00:27:05.554822+0000","last_active":"2024-09-21T00:27:05.543789+0000","last_peered":"2024-09-21T00:27:05.543789+0000","last_clean":"2024-09-21T00:27:05.543789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:05.554822+0000","last_undegraded":"2024-09-21T00:27:05.554822+0000","last_fullsized":"2024-09-21T00:27:05.554822+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-21T00:27:05.554816+0000","last_change":"2024-09-21T00:27:05.554816+0000","last_active":"2024-09-21T00:27:05.543789+0000","last_peered":"2024-09-21T00:27:05.543789+0000","last_clean":"2024-09-21T00:27:05.543789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:05.554816+0000","last_undegraded":"2024-09-21T00:27:05.554816+0000","last_fullsized":"2024-09-21T00:27:05.554816+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-21T00:27:05.557773+0000","last_change":"2024-09-21T00:27:05.557773+0000","last_active":"2024-09-21T00:27:05.543789+0000","last_peered":"2024-09-21T00:27:05.543789+0000","last_clean":"2024-09-21T00:27:05.543789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:05.557773+0000","last_undegraded":"2024-09-21T00:27:05.557773+0000","last_fullsized":"2024-09-21T00:27:05.557773+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-21T00:27:05.555561+0000","last_change":"2024-09-21T00:27:05.555561+0000","last_active":"2024-09-21T00:27:05.543789+0000","last_peered":"2024-09-21T00:27:05.543789+0000","last_clean":"2024-09-21T00:27:05.543789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:05.555561+0000","last_undegraded":"2024-09-21T00:27:05.555561+0000","last_fullsized":"2024-09-21T00:27:05.555561+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-21T00:27:05.561883+0000","last_change":"2024-09-21T00:27:05.561883+0000","last_active":"2024-09-21T00:27:05.543789+0000","last_peered":"2024-09-21T00:27:05.543789+0000","last_clean":"2024-09-21T00:27:05.543789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:05.561883+0000","last_undegraded":"2024-09-21T00:27:05.561883+0000","last_fullsized":"2024-09-21T00:27:05.561883+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":153,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T00:27:05.563691+0000","last_change":"2024-09-21T00:25:38.725792+0000","last_active":"2024-09-21T00:27:05.563691+0000","last_peered":"2024-09-21T00:27:05.563691+0000","last_clean":"2024-09-21T00:27:05.563691+0000","last_became_active":"2024-09-21T00:25:38.724668+0000","last_became_peered":"2024-09-21T00:25:38.724668+0000","last_unstale":"2024-09-21T00:27:05.563691+0000","last_undegraded":"2024-09-21T00:27:05.563691+0000","last_fullsized":"2024-09-21T00:27:05.563691+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:37.163216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:37.163216+0000","last_clean_scrub_stamp":"2024-09-21T00:25:37.163216+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:14:36.588660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":20,"acting":20,"num_store_stats":1},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509461,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28132,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343708,"statfs":{"total":96636764160,"available":96607956992,"internally_reserved":0,"allocated":1130496,"data_stored":819241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509462,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27552,"kb_used_data":524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344288,"statfs":{"total":96636764160,"available":96608550912,"internally_reserved":0,"allocated":536576,"data_stored":228873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509461,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27552,"kb_used_data":524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344288,"statfs":{"total":96636764160,"available":96608550912,"internally_reserved":0,"allocated":536576,"data_stored":228873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509460,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27552,"kb_used_data":524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344288,"statfs":{"total":96636764160,"available":96608550912,"internally_reserved":0,"allocated":536576,"data_stored":228873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509462,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27552,"kb_used_data":524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344288,"statfs":{"total":96636764160,"available":96608550912,"internally_reserved":0,"allocated":536576,"data_stored":228873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509462,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27552,"kb_used_data":524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344288,"statfs":{"total":96636764160,"available":96608550912,"internally_reserved":0,"allocated":536576,"data_stored":228873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509462,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28132,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343708,"statfs":{"total":96636764160,"available":96607956992,"internally_reserved":0,"allocated":1130496,"data_stored":819241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27576,"kb_used_data":548,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344264,"statfs":{"total":96636764160,"available":96608526336,"internally_reserved":0,"allocated":561152,"data_stored":242707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T00:27:12.307 INFO:tasks.ceph.ceph_manager.ceph:PG 8.3e is not active+clean 2024-09-21T00:27:12.307 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:27:06.549347+0000', 'last_change': '2024-09-21T00:27:06.549347+0000', 'last_active': '2024-09-21T00:27:06.549347+0000', 'last_peered': '2024-09-21T00:27:06.549347+0000', 'last_clean': '2024-09-21T00:27:06.549347+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:27:06.549347+0000', 'last_undegraded': '2024-09-21T00:27:06.549347+0000', 'last_fullsized': '2024-09-21T00:27:06.549347+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'last_clean_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:27:12.307 INFO:tasks.ceph.ceph_manager.ceph:PG 8.3d is not active+clean 2024-09-21T00:27:12.307 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:27:06.549347+0000', 'last_change': '2024-09-21T00:27:06.549347+0000', 'last_active': '2024-09-21T00:27:06.549347+0000', 'last_peered': '2024-09-21T00:27:06.549347+0000', 'last_clean': '2024-09-21T00:27:06.549347+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:27:06.549347+0000', 'last_undegraded': '2024-09-21T00:27:06.549347+0000', 'last_fullsized': '2024-09-21T00:27:06.549347+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'last_clean_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:27:12.307 INFO:tasks.ceph.ceph_manager.ceph:PG 8.3c is not active+clean 2024-09-21T00:27:12.307 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:27:06.549347+0000', 'last_change': '2024-09-21T00:27:06.549347+0000', 'last_active': '2024-09-21T00:27:06.549347+0000', 'last_peered': '2024-09-21T00:27:06.549347+0000', 'last_clean': '2024-09-21T00:27:06.549347+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:27:06.549347+0000', 'last_undegraded': '2024-09-21T00:27:06.549347+0000', 'last_fullsized': '2024-09-21T00:27:06.549347+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'last_clean_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:27:12.307 INFO:tasks.ceph.ceph_manager.ceph:PG 8.3b is not active+clean 2024-09-21T00:27:12.307 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:27:06.549347+0000', 'last_change': '2024-09-21T00:27:06.549347+0000', 'last_active': '2024-09-21T00:27:06.549347+0000', 'last_peered': '2024-09-21T00:27:06.549347+0000', 'last_clean': '2024-09-21T00:27:06.549347+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:27:06.549347+0000', 'last_undegraded': '2024-09-21T00:27:06.549347+0000', 'last_fullsized': '2024-09-21T00:27:06.549347+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'last_clean_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:27:12.307 INFO:tasks.ceph.ceph_manager.ceph:PG 8.39 is not active+clean 2024-09-21T00:27:12.307 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:27:06.549347+0000', 'last_change': '2024-09-21T00:27:06.549347+0000', 'last_active': '2024-09-21T00:27:06.549347+0000', 'last_peered': '2024-09-21T00:27:06.549347+0000', 'last_clean': '2024-09-21T00:27:06.549347+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:27:06.549347+0000', 'last_undegraded': '2024-09-21T00:27:06.549347+0000', 'last_fullsized': '2024-09-21T00:27:06.549347+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'last_clean_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:27:12.308 INFO:tasks.ceph.ceph_manager.ceph:PG 8.38 is not active+clean 2024-09-21T00:27:12.308 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:27:06.549347+0000', 'last_change': '2024-09-21T00:27:06.549347+0000', 'last_active': '2024-09-21T00:27:06.549347+0000', 'last_peered': '2024-09-21T00:27:06.549347+0000', 'last_clean': '2024-09-21T00:27:06.549347+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:27:06.549347+0000', 'last_undegraded': '2024-09-21T00:27:06.549347+0000', 'last_fullsized': '2024-09-21T00:27:06.549347+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'last_clean_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:27:12.308 INFO:tasks.ceph.ceph_manager.ceph:PG 8.37 is not active+clean 2024-09-21T00:27:12.308 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:27:06.549347+0000', 'last_change': '2024-09-21T00:27:06.549347+0000', 'last_active': '2024-09-21T00:27:06.549347+0000', 'last_peered': '2024-09-21T00:27:06.549347+0000', 'last_clean': '2024-09-21T00:27:06.549347+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:27:06.549347+0000', 'last_undegraded': '2024-09-21T00:27:06.549347+0000', 'last_fullsized': '2024-09-21T00:27:06.549347+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'last_clean_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:27:12.308 INFO:tasks.ceph.ceph_manager.ceph:PG 8.35 is not active+clean 2024-09-21T00:27:12.308 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:27:06.549347+0000', 'last_change': '2024-09-21T00:27:06.549347+0000', 'last_active': '2024-09-21T00:27:06.549347+0000', 'last_peered': '2024-09-21T00:27:06.549347+0000', 'last_clean': '2024-09-21T00:27:06.549347+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:27:06.549347+0000', 'last_undegraded': '2024-09-21T00:27:06.549347+0000', 'last_fullsized': '2024-09-21T00:27:06.549347+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'last_clean_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:27:12.308 INFO:tasks.ceph.ceph_manager.ceph:PG 8.34 is not active+clean 2024-09-21T00:27:12.308 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:27:06.549347+0000', 'last_change': '2024-09-21T00:27:06.549347+0000', 'last_active': '2024-09-21T00:27:06.549347+0000', 'last_peered': '2024-09-21T00:27:06.549347+0000', 'last_clean': '2024-09-21T00:27:06.549347+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:27:06.549347+0000', 'last_undegraded': '2024-09-21T00:27:06.549347+0000', 'last_fullsized': '2024-09-21T00:27:06.549347+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'last_clean_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:27:12.308 INFO:tasks.ceph.ceph_manager.ceph:PG 8.32 is not active+clean 2024-09-21T00:27:12.308 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:27:06.549347+0000', 'last_change': '2024-09-21T00:27:06.549347+0000', 'last_active': '2024-09-21T00:27:06.549347+0000', 'last_peered': '2024-09-21T00:27:06.549347+0000', 'last_clean': '2024-09-21T00:27:06.549347+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:27:06.549347+0000', 'last_undegraded': '2024-09-21T00:27:06.549347+0000', 'last_fullsized': '2024-09-21T00:27:06.549347+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'last_clean_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:27:12.308 INFO:tasks.ceph.ceph_manager.ceph:PG 8.31 is not active+clean 2024-09-21T00:27:12.309 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:27:06.549347+0000', 'last_change': '2024-09-21T00:27:06.549347+0000', 'last_active': '2024-09-21T00:27:06.549347+0000', 'last_peered': '2024-09-21T00:27:06.549347+0000', 'last_clean': '2024-09-21T00:27:06.549347+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:27:06.549347+0000', 'last_undegraded': '2024-09-21T00:27:06.549347+0000', 'last_fullsized': '2024-09-21T00:27:06.549347+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'last_clean_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:27:12.309 INFO:tasks.ceph.ceph_manager.ceph:PG 8.2f is not active+clean 2024-09-21T00:27:12.309 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:27:06.549347+0000', 'last_change': '2024-09-21T00:27:06.549347+0000', 'last_active': '2024-09-21T00:27:06.549347+0000', 'last_peered': '2024-09-21T00:27:06.549347+0000', 'last_clean': '2024-09-21T00:27:06.549347+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:27:06.549347+0000', 'last_undegraded': '2024-09-21T00:27:06.549347+0000', 'last_fullsized': '2024-09-21T00:27:06.549347+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'last_clean_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:27:12.309 INFO:tasks.ceph.ceph_manager.ceph:PG 8.2e is not active+clean 2024-09-21T00:27:12.309 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:27:06.549347+0000', 'last_change': '2024-09-21T00:27:06.549347+0000', 'last_active': '2024-09-21T00:27:06.549347+0000', 'last_peered': '2024-09-21T00:27:06.549347+0000', 'last_clean': '2024-09-21T00:27:06.549347+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:27:06.549347+0000', 'last_undegraded': '2024-09-21T00:27:06.549347+0000', 'last_fullsized': '2024-09-21T00:27:06.549347+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'last_clean_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:27:12.309 INFO:tasks.ceph.ceph_manager.ceph:PG 8.2d is not active+clean 2024-09-21T00:27:12.309 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:27:06.549347+0000', 'last_change': '2024-09-21T00:27:06.549347+0000', 'last_active': '2024-09-21T00:27:06.549347+0000', 'last_peered': '2024-09-21T00:27:06.549347+0000', 'last_clean': '2024-09-21T00:27:06.549347+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:27:06.549347+0000', 'last_undegraded': '2024-09-21T00:27:06.549347+0000', 'last_fullsized': '2024-09-21T00:27:06.549347+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'last_clean_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:27:12.309 INFO:tasks.ceph.ceph_manager.ceph:PG 8.2c is not active+clean 2024-09-21T00:27:12.309 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:27:06.549347+0000', 'last_change': '2024-09-21T00:27:06.549347+0000', 'last_active': '2024-09-21T00:27:06.549347+0000', 'last_peered': '2024-09-21T00:27:06.549347+0000', 'last_clean': '2024-09-21T00:27:06.549347+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:27:06.549347+0000', 'last_undegraded': '2024-09-21T00:27:06.549347+0000', 'last_fullsized': '2024-09-21T00:27:06.549347+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'last_clean_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:27:12.309 INFO:tasks.ceph.ceph_manager.ceph:PG 8.2b is not active+clean 2024-09-21T00:27:12.309 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:27:06.549347+0000', 'last_change': '2024-09-21T00:27:06.549347+0000', 'last_active': '2024-09-21T00:27:06.549347+0000', 'last_peered': '2024-09-21T00:27:06.549347+0000', 'last_clean': '2024-09-21T00:27:06.549347+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:27:06.549347+0000', 'last_undegraded': '2024-09-21T00:27:06.549347+0000', 'last_fullsized': '2024-09-21T00:27:06.549347+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'last_clean_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:27:12.309 INFO:tasks.ceph.ceph_manager.ceph:PG 8.2a is not active+clean 2024-09-21T00:27:12.309 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:27:06.549347+0000', 'last_change': '2024-09-21T00:27:06.549347+0000', 'last_active': '2024-09-21T00:27:06.549347+0000', 'last_peered': '2024-09-21T00:27:06.549347+0000', 'last_clean': '2024-09-21T00:27:06.549347+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:27:06.549347+0000', 'last_undegraded': '2024-09-21T00:27:06.549347+0000', 'last_fullsized': '2024-09-21T00:27:06.549347+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'last_clean_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:27:12.310 INFO:tasks.ceph.ceph_manager.ceph:PG 8.29 is not active+clean 2024-09-21T00:27:12.310 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:27:06.549347+0000', 'last_change': '2024-09-21T00:27:06.549347+0000', 'last_active': '2024-09-21T00:27:06.549347+0000', 'last_peered': '2024-09-21T00:27:06.549347+0000', 'last_clean': '2024-09-21T00:27:06.549347+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:27:06.549347+0000', 'last_undegraded': '2024-09-21T00:27:06.549347+0000', 'last_fullsized': '2024-09-21T00:27:06.549347+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'last_clean_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:27:12.310 INFO:tasks.ceph.ceph_manager.ceph:PG 8.28 is not active+clean 2024-09-21T00:27:12.310 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:27:06.549347+0000', 'last_change': '2024-09-21T00:27:06.549347+0000', 'last_active': '2024-09-21T00:27:06.549347+0000', 'last_peered': '2024-09-21T00:27:06.549347+0000', 'last_clean': '2024-09-21T00:27:06.549347+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:27:06.549347+0000', 'last_undegraded': '2024-09-21T00:27:06.549347+0000', 'last_fullsized': '2024-09-21T00:27:06.549347+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'last_clean_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:27:12.310 INFO:tasks.ceph.ceph_manager.ceph:PG 8.27 is not active+clean 2024-09-21T00:27:12.310 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:27:06.549347+0000', 'last_change': '2024-09-21T00:27:06.549347+0000', 'last_active': '2024-09-21T00:27:06.549347+0000', 'last_peered': '2024-09-21T00:27:06.549347+0000', 'last_clean': '2024-09-21T00:27:06.549347+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:27:06.549347+0000', 'last_undegraded': '2024-09-21T00:27:06.549347+0000', 'last_fullsized': '2024-09-21T00:27:06.549347+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'last_clean_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:27:12.310 INFO:tasks.ceph.ceph_manager.ceph:PG 8.26 is not active+clean 2024-09-21T00:27:12.310 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:27:06.549347+0000', 'last_change': '2024-09-21T00:27:06.549347+0000', 'last_active': '2024-09-21T00:27:06.549347+0000', 'last_peered': '2024-09-21T00:27:06.549347+0000', 'last_clean': '2024-09-21T00:27:06.549347+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:27:06.549347+0000', 'last_undegraded': '2024-09-21T00:27:06.549347+0000', 'last_fullsized': '2024-09-21T00:27:06.549347+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'last_clean_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:27:12.310 INFO:tasks.ceph.ceph_manager.ceph:PG 8.24 is not active+clean 2024-09-21T00:27:12.310 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:27:06.549347+0000', 'last_change': '2024-09-21T00:27:06.549347+0000', 'last_active': '2024-09-21T00:27:06.549347+0000', 'last_peered': '2024-09-21T00:27:06.549347+0000', 'last_clean': '2024-09-21T00:27:06.549347+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:27:06.549347+0000', 'last_undegraded': '2024-09-21T00:27:06.549347+0000', 'last_fullsized': '2024-09-21T00:27:06.549347+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'last_clean_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:27:12.310 INFO:tasks.ceph.ceph_manager.ceph:PG 8.23 is not active+clean 2024-09-21T00:27:12.311 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:27:06.549347+0000', 'last_change': '2024-09-21T00:27:06.549347+0000', 'last_active': '2024-09-21T00:27:06.549347+0000', 'last_peered': '2024-09-21T00:27:06.549347+0000', 'last_clean': '2024-09-21T00:27:06.549347+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:27:06.549347+0000', 'last_undegraded': '2024-09-21T00:27:06.549347+0000', 'last_fullsized': '2024-09-21T00:27:06.549347+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'last_clean_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:27:12.311 INFO:tasks.ceph.ceph_manager.ceph:PG 8.22 is not active+clean 2024-09-21T00:27:12.311 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:27:06.549347+0000', 'last_change': '2024-09-21T00:27:06.549347+0000', 'last_active': '2024-09-21T00:27:06.549347+0000', 'last_peered': '2024-09-21T00:27:06.549347+0000', 'last_clean': '2024-09-21T00:27:06.549347+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:27:06.549347+0000', 'last_undegraded': '2024-09-21T00:27:06.549347+0000', 'last_fullsized': '2024-09-21T00:27:06.549347+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'last_clean_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:27:12.311 INFO:tasks.ceph.ceph_manager.ceph:PG 8.21 is not active+clean 2024-09-21T00:27:12.311 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:27:06.549347+0000', 'last_change': '2024-09-21T00:27:06.549347+0000', 'last_active': '2024-09-21T00:27:06.549347+0000', 'last_peered': '2024-09-21T00:27:06.549347+0000', 'last_clean': '2024-09-21T00:27:06.549347+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:27:06.549347+0000', 'last_undegraded': '2024-09-21T00:27:06.549347+0000', 'last_fullsized': '2024-09-21T00:27:06.549347+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'last_clean_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:27:12.311 INFO:tasks.ceph.ceph_manager.ceph:PG 8.20 is not active+clean 2024-09-21T00:27:12.311 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:27:06.549347+0000', 'last_change': '2024-09-21T00:27:06.549347+0000', 'last_active': '2024-09-21T00:27:06.549347+0000', 'last_peered': '2024-09-21T00:27:06.549347+0000', 'last_clean': '2024-09-21T00:27:06.549347+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:27:06.549347+0000', 'last_undegraded': '2024-09-21T00:27:06.549347+0000', 'last_fullsized': '2024-09-21T00:27:06.549347+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'last_clean_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:27:12.311 INFO:tasks.ceph.ceph_manager.ceph:PG 8.10 is not active+clean 2024-09-21T00:27:12.311 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:27:06.549347+0000', 'last_change': '2024-09-21T00:27:06.549347+0000', 'last_active': '2024-09-21T00:27:06.549347+0000', 'last_peered': '2024-09-21T00:27:06.549347+0000', 'last_clean': '2024-09-21T00:27:06.549347+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:27:06.549347+0000', 'last_undegraded': '2024-09-21T00:27:06.549347+0000', 'last_fullsized': '2024-09-21T00:27:06.549347+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'last_clean_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:27:12.312 INFO:tasks.ceph.ceph_manager.ceph:PG 7.1f is not active+clean 2024-09-21T00:27:12.312 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-09-21T00:27:05.556637+0000', 'last_change': '2024-09-21T00:27:05.556637+0000', 'last_active': '2024-09-21T00:27:05.543789+0000', 'last_peered': '2024-09-21T00:27:05.543789+0000', 'last_clean': '2024-09-21T00:27:05.543789+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:27:05.556637+0000', 'last_undegraded': '2024-09-21T00:27:05.556637+0000', 'last_fullsized': '2024-09-21T00:27:05.556637+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:27:05.543789+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:27:05.543789+0000', 'last_clean_scrub_stamp': '2024-09-21T00:27:05.543789+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-21T00:27:12.312 INFO:tasks.ceph.ceph_manager.ceph:PG 8.11 is not active+clean 2024-09-21T00:27:12.312 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:27:06.549347+0000', 'last_change': '2024-09-21T00:27:06.549347+0000', 'last_active': '2024-09-21T00:27:06.549347+0000', 'last_peered': '2024-09-21T00:27:06.549347+0000', 'last_clean': '2024-09-21T00:27:06.549347+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:27:06.549347+0000', 'last_undegraded': '2024-09-21T00:27:06.549347+0000', 'last_fullsized': '2024-09-21T00:27:06.549347+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'last_clean_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:27:12.312 INFO:tasks.ceph.ceph_manager.ceph:PG 8.12 is not active+clean 2024-09-21T00:27:12.312 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:27:06.549347+0000', 'last_change': '2024-09-21T00:27:06.549347+0000', 'last_active': '2024-09-21T00:27:06.549347+0000', 'last_peered': '2024-09-21T00:27:06.549347+0000', 'last_clean': '2024-09-21T00:27:06.549347+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:27:06.549347+0000', 'last_undegraded': '2024-09-21T00:27:06.549347+0000', 'last_fullsized': '2024-09-21T00:27:06.549347+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'last_clean_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:27:12.312 INFO:tasks.ceph.ceph_manager.ceph:PG 8.13 is not active+clean 2024-09-21T00:27:12.312 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:27:06.549347+0000', 'last_change': '2024-09-21T00:27:06.549347+0000', 'last_active': '2024-09-21T00:27:06.549347+0000', 'last_peered': '2024-09-21T00:27:06.549347+0000', 'last_clean': '2024-09-21T00:27:06.549347+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:27:06.549347+0000', 'last_undegraded': '2024-09-21T00:27:06.549347+0000', 'last_fullsized': '2024-09-21T00:27:06.549347+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'last_clean_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:27:12.312 INFO:tasks.ceph.ceph_manager.ceph:PG 7.1c is not active+clean 2024-09-21T00:27:12.312 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-09-21T00:27:05.555296+0000', 'last_change': '2024-09-21T00:27:05.555296+0000', 'last_active': '2024-09-21T00:27:05.543789+0000', 'last_peered': '2024-09-21T00:27:05.543789+0000', 'last_clean': '2024-09-21T00:27:05.543789+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:27:05.555296+0000', 'last_undegraded': '2024-09-21T00:27:05.555296+0000', 'last_fullsized': '2024-09-21T00:27:05.555296+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:27:05.543789+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:27:05.543789+0000', 'last_clean_scrub_stamp': '2024-09-21T00:27:05.543789+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-21T00:27:12.312 INFO:tasks.ceph.ceph_manager.ceph:PG 8.14 is not active+clean 2024-09-21T00:27:12.312 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:27:06.549347+0000', 'last_change': '2024-09-21T00:27:06.549347+0000', 'last_active': '2024-09-21T00:27:06.549347+0000', 'last_peered': '2024-09-21T00:27:06.549347+0000', 'last_clean': '2024-09-21T00:27:06.549347+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:27:06.549347+0000', 'last_undegraded': '2024-09-21T00:27:06.549347+0000', 'last_fullsized': '2024-09-21T00:27:06.549347+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'last_clean_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:27:12.312 INFO:tasks.ceph.ceph_manager.ceph:PG 7.1b is not active+clean 2024-09-21T00:27:12.313 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-09-21T00:27:05.562604+0000', 'last_change': '2024-09-21T00:27:05.562604+0000', 'last_active': '2024-09-21T00:27:05.543789+0000', 'last_peered': '2024-09-21T00:27:05.543789+0000', 'last_clean': '2024-09-21T00:27:05.543789+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:27:05.562604+0000', 'last_undegraded': '2024-09-21T00:27:05.562604+0000', 'last_fullsized': '2024-09-21T00:27:05.562604+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:27:05.543789+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:27:05.543789+0000', 'last_clean_scrub_stamp': '2024-09-21T00:27:05.543789+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-21T00:27:12.313 INFO:tasks.ceph.ceph_manager.ceph:PG 8.15 is not active+clean 2024-09-21T00:27:12.313 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:27:06.549347+0000', 'last_change': '2024-09-21T00:27:06.549347+0000', 'last_active': '2024-09-21T00:27:06.549347+0000', 'last_peered': '2024-09-21T00:27:06.549347+0000', 'last_clean': '2024-09-21T00:27:06.549347+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:27:06.549347+0000', 'last_undegraded': '2024-09-21T00:27:06.549347+0000', 'last_fullsized': '2024-09-21T00:27:06.549347+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'last_clean_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:27:12.313 INFO:tasks.ceph.ceph_manager.ceph:PG 7.1a is not active+clean 2024-09-21T00:27:12.313 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-09-21T00:27:05.557122+0000', 'last_change': '2024-09-21T00:27:05.557122+0000', 'last_active': '2024-09-21T00:27:05.543789+0000', 'last_peered': '2024-09-21T00:27:05.543789+0000', 'last_clean': '2024-09-21T00:27:05.543789+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:27:05.557122+0000', 'last_undegraded': '2024-09-21T00:27:05.557122+0000', 'last_fullsized': '2024-09-21T00:27:05.557122+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:27:05.543789+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:27:05.543789+0000', 'last_clean_scrub_stamp': '2024-09-21T00:27:05.543789+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-21T00:27:12.313 INFO:tasks.ceph.ceph_manager.ceph:PG 8.16 is not active+clean 2024-09-21T00:27:12.313 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:27:06.549347+0000', 'last_change': '2024-09-21T00:27:06.549347+0000', 'last_active': '2024-09-21T00:27:06.549347+0000', 'last_peered': '2024-09-21T00:27:06.549347+0000', 'last_clean': '2024-09-21T00:27:06.549347+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:27:06.549347+0000', 'last_undegraded': '2024-09-21T00:27:06.549347+0000', 'last_fullsized': '2024-09-21T00:27:06.549347+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'last_clean_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:27:12.313 INFO:tasks.ceph.ceph_manager.ceph:PG 7.19 is not active+clean 2024-09-21T00:27:12.313 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-09-21T00:27:05.555459+0000', 'last_change': '2024-09-21T00:27:05.555459+0000', 'last_active': '2024-09-21T00:27:05.543789+0000', 'last_peered': '2024-09-21T00:27:05.543789+0000', 'last_clean': '2024-09-21T00:27:05.543789+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:27:05.555459+0000', 'last_undegraded': '2024-09-21T00:27:05.555459+0000', 'last_fullsized': '2024-09-21T00:27:05.555459+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:27:05.543789+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:27:05.543789+0000', 'last_clean_scrub_stamp': '2024-09-21T00:27:05.543789+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-21T00:27:12.314 INFO:tasks.ceph.ceph_manager.ceph:PG 8.17 is not active+clean 2024-09-21T00:27:12.314 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:27:06.549347+0000', 'last_change': '2024-09-21T00:27:06.549347+0000', 'last_active': '2024-09-21T00:27:06.549347+0000', 'last_peered': '2024-09-21T00:27:06.549347+0000', 'last_clean': '2024-09-21T00:27:06.549347+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:27:06.549347+0000', 'last_undegraded': '2024-09-21T00:27:06.549347+0000', 'last_fullsized': '2024-09-21T00:27:06.549347+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'last_clean_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:27:12.314 INFO:tasks.ceph.ceph_manager.ceph:PG 7.18 is not active+clean 2024-09-21T00:27:12.314 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-09-21T00:27:05.562527+0000', 'last_change': '2024-09-21T00:27:05.562527+0000', 'last_active': '2024-09-21T00:27:05.543789+0000', 'last_peered': '2024-09-21T00:27:05.543789+0000', 'last_clean': '2024-09-21T00:27:05.543789+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:27:05.562527+0000', 'last_undegraded': '2024-09-21T00:27:05.562527+0000', 'last_fullsized': '2024-09-21T00:27:05.562527+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:27:05.543789+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:27:05.543789+0000', 'last_clean_scrub_stamp': '2024-09-21T00:27:05.543789+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-21T00:27:12.314 INFO:tasks.ceph.ceph_manager.ceph:PG 8.18 is not active+clean 2024-09-21T00:27:12.314 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:27:06.549347+0000', 'last_change': '2024-09-21T00:27:06.549347+0000', 'last_active': '2024-09-21T00:27:06.549347+0000', 'last_peered': '2024-09-21T00:27:06.549347+0000', 'last_clean': '2024-09-21T00:27:06.549347+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:27:06.549347+0000', 'last_undegraded': '2024-09-21T00:27:06.549347+0000', 'last_fullsized': '2024-09-21T00:27:06.549347+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'last_clean_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:27:12.315 INFO:tasks.ceph.ceph_manager.ceph:PG 7.17 is not active+clean 2024-09-21T00:27:12.315 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-09-21T00:27:05.562841+0000', 'last_change': '2024-09-21T00:27:05.562841+0000', 'last_active': '2024-09-21T00:27:05.543789+0000', 'last_peered': '2024-09-21T00:27:05.543789+0000', 'last_clean': '2024-09-21T00:27:05.543789+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:27:05.562841+0000', 'last_undegraded': '2024-09-21T00:27:05.562841+0000', 'last_fullsized': '2024-09-21T00:27:05.562841+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:27:05.543789+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:27:05.543789+0000', 'last_clean_scrub_stamp': '2024-09-21T00:27:05.543789+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-21T00:27:12.315 INFO:tasks.ceph.ceph_manager.ceph:PG 8.19 is not active+clean 2024-09-21T00:27:12.315 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:27:06.549347+0000', 'last_change': '2024-09-21T00:27:06.549347+0000', 'last_active': '2024-09-21T00:27:06.549347+0000', 'last_peered': '2024-09-21T00:27:06.549347+0000', 'last_clean': '2024-09-21T00:27:06.549347+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:27:06.549347+0000', 'last_undegraded': '2024-09-21T00:27:06.549347+0000', 'last_fullsized': '2024-09-21T00:27:06.549347+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'last_clean_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:27:12.315 INFO:tasks.ceph.ceph_manager.ceph:PG 7.16 is not active+clean 2024-09-21T00:27:12.315 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-09-21T00:27:05.558519+0000', 'last_change': '2024-09-21T00:27:05.558519+0000', 'last_active': '2024-09-21T00:27:05.543789+0000', 'last_peered': '2024-09-21T00:27:05.543789+0000', 'last_clean': '2024-09-21T00:27:05.543789+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:27:05.558519+0000', 'last_undegraded': '2024-09-21T00:27:05.558519+0000', 'last_fullsized': '2024-09-21T00:27:05.558519+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:27:05.543789+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:27:05.543789+0000', 'last_clean_scrub_stamp': '2024-09-21T00:27:05.543789+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-21T00:27:12.315 INFO:tasks.ceph.ceph_manager.ceph:PG 8.1a is not active+clean 2024-09-21T00:27:12.315 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:27:06.549347+0000', 'last_change': '2024-09-21T00:27:06.549347+0000', 'last_active': '2024-09-21T00:27:06.549347+0000', 'last_peered': '2024-09-21T00:27:06.549347+0000', 'last_clean': '2024-09-21T00:27:06.549347+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:27:06.549347+0000', 'last_undegraded': '2024-09-21T00:27:06.549347+0000', 'last_fullsized': '2024-09-21T00:27:06.549347+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'last_clean_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:27:12.315 INFO:tasks.ceph.ceph_manager.ceph:PG 7.15 is not active+clean 2024-09-21T00:27:12.315 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-09-21T00:27:05.563390+0000', 'last_change': '2024-09-21T00:27:05.563390+0000', 'last_active': '2024-09-21T00:27:05.543789+0000', 'last_peered': '2024-09-21T00:27:05.543789+0000', 'last_clean': '2024-09-21T00:27:05.543789+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:27:05.563390+0000', 'last_undegraded': '2024-09-21T00:27:05.563390+0000', 'last_fullsized': '2024-09-21T00:27:05.563390+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:27:05.543789+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:27:05.543789+0000', 'last_clean_scrub_stamp': '2024-09-21T00:27:05.543789+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-21T00:27:12.316 INFO:tasks.ceph.ceph_manager.ceph:PG 8.1b is not active+clean 2024-09-21T00:27:12.316 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:27:06.549347+0000', 'last_change': '2024-09-21T00:27:06.549347+0000', 'last_active': '2024-09-21T00:27:06.549347+0000', 'last_peered': '2024-09-21T00:27:06.549347+0000', 'last_clean': '2024-09-21T00:27:06.549347+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:27:06.549347+0000', 'last_undegraded': '2024-09-21T00:27:06.549347+0000', 'last_fullsized': '2024-09-21T00:27:06.549347+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'last_clean_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:27:12.316 INFO:tasks.ceph.ceph_manager.ceph:PG 7.14 is not active+clean 2024-09-21T00:27:12.316 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-09-21T00:27:05.554843+0000', 'last_change': '2024-09-21T00:27:05.554843+0000', 'last_active': '2024-09-21T00:27:05.543789+0000', 'last_peered': '2024-09-21T00:27:05.543789+0000', 'last_clean': '2024-09-21T00:27:05.543789+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:27:05.554843+0000', 'last_undegraded': '2024-09-21T00:27:05.554843+0000', 'last_fullsized': '2024-09-21T00:27:05.554843+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:27:05.543789+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:27:05.543789+0000', 'last_clean_scrub_stamp': '2024-09-21T00:27:05.543789+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-21T00:27:12.316 INFO:tasks.ceph.ceph_manager.ceph:PG 8.1c is not active+clean 2024-09-21T00:27:12.316 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:27:06.549347+0000', 'last_change': '2024-09-21T00:27:06.549347+0000', 'last_active': '2024-09-21T00:27:06.549347+0000', 'last_peered': '2024-09-21T00:27:06.549347+0000', 'last_clean': '2024-09-21T00:27:06.549347+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:27:06.549347+0000', 'last_undegraded': '2024-09-21T00:27:06.549347+0000', 'last_fullsized': '2024-09-21T00:27:06.549347+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'last_clean_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:27:12.316 INFO:tasks.ceph.ceph_manager.ceph:PG 7.13 is not active+clean 2024-09-21T00:27:12.316 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-09-21T00:27:05.562134+0000', 'last_change': '2024-09-21T00:27:05.562134+0000', 'last_active': '2024-09-21T00:27:05.543789+0000', 'last_peered': '2024-09-21T00:27:05.543789+0000', 'last_clean': '2024-09-21T00:27:05.543789+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:27:05.562134+0000', 'last_undegraded': '2024-09-21T00:27:05.562134+0000', 'last_fullsized': '2024-09-21T00:27:05.562134+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:27:05.543789+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:27:05.543789+0000', 'last_clean_scrub_stamp': '2024-09-21T00:27:05.543789+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-21T00:27:12.316 INFO:tasks.ceph.ceph_manager.ceph:PG 8.1d is not active+clean 2024-09-21T00:27:12.316 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:27:06.549347+0000', 'last_change': '2024-09-21T00:27:06.549347+0000', 'last_active': '2024-09-21T00:27:06.549347+0000', 'last_peered': '2024-09-21T00:27:06.549347+0000', 'last_clean': '2024-09-21T00:27:06.549347+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:27:06.549347+0000', 'last_undegraded': '2024-09-21T00:27:06.549347+0000', 'last_fullsized': '2024-09-21T00:27:06.549347+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'last_clean_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:27:12.316 INFO:tasks.ceph.ceph_manager.ceph:PG 7.12 is not active+clean 2024-09-21T00:27:12.316 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-09-21T00:27:05.556088+0000', 'last_change': '2024-09-21T00:27:05.556088+0000', 'last_active': '2024-09-21T00:27:05.543789+0000', 'last_peered': '2024-09-21T00:27:05.543789+0000', 'last_clean': '2024-09-21T00:27:05.543789+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:27:05.556088+0000', 'last_undegraded': '2024-09-21T00:27:05.556088+0000', 'last_fullsized': '2024-09-21T00:27:05.556088+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:27:05.543789+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:27:05.543789+0000', 'last_clean_scrub_stamp': '2024-09-21T00:27:05.543789+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-21T00:27:12.317 INFO:tasks.ceph.ceph_manager.ceph:PG 8.1e is not active+clean 2024-09-21T00:27:12.317 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:27:06.549347+0000', 'last_change': '2024-09-21T00:27:06.549347+0000', 'last_active': '2024-09-21T00:27:06.549347+0000', 'last_peered': '2024-09-21T00:27:06.549347+0000', 'last_clean': '2024-09-21T00:27:06.549347+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:27:06.549347+0000', 'last_undegraded': '2024-09-21T00:27:06.549347+0000', 'last_fullsized': '2024-09-21T00:27:06.549347+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'last_clean_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:27:12.317 INFO:tasks.ceph.ceph_manager.ceph:PG 7.11 is not active+clean 2024-09-21T00:27:12.317 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-09-21T00:27:05.562785+0000', 'last_change': '2024-09-21T00:27:05.562785+0000', 'last_active': '2024-09-21T00:27:05.543789+0000', 'last_peered': '2024-09-21T00:27:05.543789+0000', 'last_clean': '2024-09-21T00:27:05.543789+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:27:05.562785+0000', 'last_undegraded': '2024-09-21T00:27:05.562785+0000', 'last_fullsized': '2024-09-21T00:27:05.562785+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:27:05.543789+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:27:05.543789+0000', 'last_clean_scrub_stamp': '2024-09-21T00:27:05.543789+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-21T00:27:12.317 INFO:tasks.ceph.ceph_manager.ceph:PG 8.1f is not active+clean 2024-09-21T00:27:12.317 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:27:06.549347+0000', 'last_change': '2024-09-21T00:27:06.549347+0000', 'last_active': '2024-09-21T00:27:06.549347+0000', 'last_peered': '2024-09-21T00:27:06.549347+0000', 'last_clean': '2024-09-21T00:27:06.549347+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:27:06.549347+0000', 'last_undegraded': '2024-09-21T00:27:06.549347+0000', 'last_fullsized': '2024-09-21T00:27:06.549347+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'last_clean_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:27:12.317 INFO:tasks.ceph.ceph_manager.ceph:PG 7.10 is not active+clean 2024-09-21T00:27:12.317 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-09-21T00:27:05.554830+0000', 'last_change': '2024-09-21T00:27:05.554830+0000', 'last_active': '2024-09-21T00:27:05.543789+0000', 'last_peered': '2024-09-21T00:27:05.543789+0000', 'last_clean': '2024-09-21T00:27:05.543789+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:27:05.554830+0000', 'last_undegraded': '2024-09-21T00:27:05.554830+0000', 'last_fullsized': '2024-09-21T00:27:05.554830+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:27:05.543789+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:27:05.543789+0000', 'last_clean_scrub_stamp': '2024-09-21T00:27:05.543789+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-21T00:27:12.318 INFO:tasks.ceph.ceph_manager.ceph:PG 8.0 is not active+clean 2024-09-21T00:27:12.318 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:27:06.549347+0000', 'last_change': '2024-09-21T00:27:06.549347+0000', 'last_active': '2024-09-21T00:27:06.549347+0000', 'last_peered': '2024-09-21T00:27:06.549347+0000', 'last_clean': '2024-09-21T00:27:06.549347+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:27:06.549347+0000', 'last_undegraded': '2024-09-21T00:27:06.549347+0000', 'last_fullsized': '2024-09-21T00:27:06.549347+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'last_clean_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:27:12.318 INFO:tasks.ceph.ceph_manager.ceph:PG 7.f is not active+clean 2024-09-21T00:27:12.318 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-09-21T00:27:05.561823+0000', 'last_change': '2024-09-21T00:27:05.561823+0000', 'last_active': '2024-09-21T00:27:05.543789+0000', 'last_peered': '2024-09-21T00:27:05.543789+0000', 'last_clean': '2024-09-21T00:27:05.543789+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:27:05.561823+0000', 'last_undegraded': '2024-09-21T00:27:05.561823+0000', 'last_fullsized': '2024-09-21T00:27:05.561823+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:27:05.543789+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:27:05.543789+0000', 'last_clean_scrub_stamp': '2024-09-21T00:27:05.543789+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-21T00:27:12.318 INFO:tasks.ceph.ceph_manager.ceph:PG 7.e is not active+clean 2024-09-21T00:27:12.318 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-09-21T00:27:05.562632+0000', 'last_change': '2024-09-21T00:27:05.562632+0000', 'last_active': '2024-09-21T00:27:05.543789+0000', 'last_peered': '2024-09-21T00:27:05.543789+0000', 'last_clean': '2024-09-21T00:27:05.543789+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:27:05.562632+0000', 'last_undegraded': '2024-09-21T00:27:05.562632+0000', 'last_fullsized': '2024-09-21T00:27:05.562632+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:27:05.543789+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:27:05.543789+0000', 'last_clean_scrub_stamp': '2024-09-21T00:27:05.543789+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-21T00:27:12.318 INFO:tasks.ceph.ceph_manager.ceph:PG 8.2 is not active+clean 2024-09-21T00:27:12.318 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:27:06.549347+0000', 'last_change': '2024-09-21T00:27:06.549347+0000', 'last_active': '2024-09-21T00:27:06.549347+0000', 'last_peered': '2024-09-21T00:27:06.549347+0000', 'last_clean': '2024-09-21T00:27:06.549347+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:27:06.549347+0000', 'last_undegraded': '2024-09-21T00:27:06.549347+0000', 'last_fullsized': '2024-09-21T00:27:06.549347+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'last_clean_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:27:12.318 INFO:tasks.ceph.ceph_manager.ceph:PG 7.d is not active+clean 2024-09-21T00:27:12.318 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-09-21T00:27:05.555437+0000', 'last_change': '2024-09-21T00:27:05.555437+0000', 'last_active': '2024-09-21T00:27:05.543789+0000', 'last_peered': '2024-09-21T00:27:05.543789+0000', 'last_clean': '2024-09-21T00:27:05.543789+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:27:05.555437+0000', 'last_undegraded': '2024-09-21T00:27:05.555437+0000', 'last_fullsized': '2024-09-21T00:27:05.555437+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:27:05.543789+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:27:05.543789+0000', 'last_clean_scrub_stamp': '2024-09-21T00:27:05.543789+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-21T00:27:12.319 INFO:tasks.ceph.ceph_manager.ceph:PG 7.c is not active+clean 2024-09-21T00:27:12.319 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-09-21T00:27:05.556442+0000', 'last_change': '2024-09-21T00:27:05.556442+0000', 'last_active': '2024-09-21T00:27:05.543789+0000', 'last_peered': '2024-09-21T00:27:05.543789+0000', 'last_clean': '2024-09-21T00:27:05.543789+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:27:05.556442+0000', 'last_undegraded': '2024-09-21T00:27:05.556442+0000', 'last_fullsized': '2024-09-21T00:27:05.556442+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:27:05.543789+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:27:05.543789+0000', 'last_clean_scrub_stamp': '2024-09-21T00:27:05.543789+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-21T00:27:12.319 INFO:tasks.ceph.ceph_manager.ceph:PG 8.4 is not active+clean 2024-09-21T00:27:12.319 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:27:06.549347+0000', 'last_change': '2024-09-21T00:27:06.549347+0000', 'last_active': '2024-09-21T00:27:06.549347+0000', 'last_peered': '2024-09-21T00:27:06.549347+0000', 'last_clean': '2024-09-21T00:27:06.549347+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:27:06.549347+0000', 'last_undegraded': '2024-09-21T00:27:06.549347+0000', 'last_fullsized': '2024-09-21T00:27:06.549347+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'last_clean_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:27:12.319 INFO:tasks.ceph.ceph_manager.ceph:PG 8.5 is not active+clean 2024-09-21T00:27:12.319 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:27:06.549347+0000', 'last_change': '2024-09-21T00:27:06.549347+0000', 'last_active': '2024-09-21T00:27:06.549347+0000', 'last_peered': '2024-09-21T00:27:06.549347+0000', 'last_clean': '2024-09-21T00:27:06.549347+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:27:06.549347+0000', 'last_undegraded': '2024-09-21T00:27:06.549347+0000', 'last_fullsized': '2024-09-21T00:27:06.549347+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'last_clean_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:27:12.319 INFO:tasks.ceph.ceph_manager.ceph:PG 7.a is not active+clean 2024-09-21T00:27:12.319 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-09-21T00:27:05.556012+0000', 'last_change': '2024-09-21T00:27:05.556012+0000', 'last_active': '2024-09-21T00:27:05.543789+0000', 'last_peered': '2024-09-21T00:27:05.543789+0000', 'last_clean': '2024-09-21T00:27:05.543789+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:27:05.556012+0000', 'last_undegraded': '2024-09-21T00:27:05.556012+0000', 'last_fullsized': '2024-09-21T00:27:05.556012+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:27:05.543789+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:27:05.543789+0000', 'last_clean_scrub_stamp': '2024-09-21T00:27:05.543789+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-21T00:27:12.319 INFO:tasks.ceph.ceph_manager.ceph:PG 8.6 is not active+clean 2024-09-21T00:27:12.319 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:27:06.549347+0000', 'last_change': '2024-09-21T00:27:06.549347+0000', 'last_active': '2024-09-21T00:27:06.549347+0000', 'last_peered': '2024-09-21T00:27:06.549347+0000', 'last_clean': '2024-09-21T00:27:06.549347+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:27:06.549347+0000', 'last_undegraded': '2024-09-21T00:27:06.549347+0000', 'last_fullsized': '2024-09-21T00:27:06.549347+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'last_clean_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:27:12.320 INFO:tasks.ceph.ceph_manager.ceph:PG 7.9 is not active+clean 2024-09-21T00:27:12.320 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-09-21T00:27:05.561999+0000', 'last_change': '2024-09-21T00:27:05.561999+0000', 'last_active': '2024-09-21T00:27:05.543789+0000', 'last_peered': '2024-09-21T00:27:05.543789+0000', 'last_clean': '2024-09-21T00:27:05.543789+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:27:05.561999+0000', 'last_undegraded': '2024-09-21T00:27:05.561999+0000', 'last_fullsized': '2024-09-21T00:27:05.561999+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:27:05.543789+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:27:05.543789+0000', 'last_clean_scrub_stamp': '2024-09-21T00:27:05.543789+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-21T00:27:12.320 INFO:tasks.ceph.ceph_manager.ceph:PG 7.8 is not active+clean 2024-09-21T00:27:12.320 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-09-21T00:27:05.561751+0000', 'last_change': '2024-09-21T00:27:05.561751+0000', 'last_active': '2024-09-21T00:27:05.543789+0000', 'last_peered': '2024-09-21T00:27:05.543789+0000', 'last_clean': '2024-09-21T00:27:05.543789+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:27:05.561751+0000', 'last_undegraded': '2024-09-21T00:27:05.561751+0000', 'last_fullsized': '2024-09-21T00:27:05.561751+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:27:05.543789+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:27:05.543789+0000', 'last_clean_scrub_stamp': '2024-09-21T00:27:05.543789+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-21T00:27:12.320 INFO:tasks.ceph.ceph_manager.ceph:PG 8.8 is not active+clean 2024-09-21T00:27:12.320 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:27:06.549347+0000', 'last_change': '2024-09-21T00:27:06.549347+0000', 'last_active': '2024-09-21T00:27:06.549347+0000', 'last_peered': '2024-09-21T00:27:06.549347+0000', 'last_clean': '2024-09-21T00:27:06.549347+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:27:06.549347+0000', 'last_undegraded': '2024-09-21T00:27:06.549347+0000', 'last_fullsized': '2024-09-21T00:27:06.549347+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'last_clean_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:27:12.320 INFO:tasks.ceph.ceph_manager.ceph:PG 7.7 is not active+clean 2024-09-21T00:27:12.320 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-09-21T00:27:05.556126+0000', 'last_change': '2024-09-21T00:27:05.556126+0000', 'last_active': '2024-09-21T00:27:05.543789+0000', 'last_peered': '2024-09-21T00:27:05.543789+0000', 'last_clean': '2024-09-21T00:27:05.543789+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:27:05.556126+0000', 'last_undegraded': '2024-09-21T00:27:05.556126+0000', 'last_fullsized': '2024-09-21T00:27:05.556126+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:27:05.543789+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:27:05.543789+0000', 'last_clean_scrub_stamp': '2024-09-21T00:27:05.543789+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-21T00:27:12.320 INFO:tasks.ceph.ceph_manager.ceph:PG 7.5 is not active+clean 2024-09-21T00:27:12.320 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-09-21T00:27:05.562808+0000', 'last_change': '2024-09-21T00:27:05.562808+0000', 'last_active': '2024-09-21T00:27:05.543789+0000', 'last_peered': '2024-09-21T00:27:05.543789+0000', 'last_clean': '2024-09-21T00:27:05.543789+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:27:05.562808+0000', 'last_undegraded': '2024-09-21T00:27:05.562808+0000', 'last_fullsized': '2024-09-21T00:27:05.562808+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:27:05.543789+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:27:05.543789+0000', 'last_clean_scrub_stamp': '2024-09-21T00:27:05.543789+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-21T00:27:12.321 INFO:tasks.ceph.ceph_manager.ceph:PG 8.b is not active+clean 2024-09-21T00:27:12.321 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:27:06.549347+0000', 'last_change': '2024-09-21T00:27:06.549347+0000', 'last_active': '2024-09-21T00:27:06.549347+0000', 'last_peered': '2024-09-21T00:27:06.549347+0000', 'last_clean': '2024-09-21T00:27:06.549347+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:27:06.549347+0000', 'last_undegraded': '2024-09-21T00:27:06.549347+0000', 'last_fullsized': '2024-09-21T00:27:06.549347+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'last_clean_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:27:12.321 INFO:tasks.ceph.ceph_manager.ceph:PG 7.4 is not active+clean 2024-09-21T00:27:12.321 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-09-21T00:27:05.562127+0000', 'last_change': '2024-09-21T00:27:05.562127+0000', 'last_active': '2024-09-21T00:27:05.543789+0000', 'last_peered': '2024-09-21T00:27:05.543789+0000', 'last_clean': '2024-09-21T00:27:05.543789+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:27:05.562127+0000', 'last_undegraded': '2024-09-21T00:27:05.562127+0000', 'last_fullsized': '2024-09-21T00:27:05.562127+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:27:05.543789+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:27:05.543789+0000', 'last_clean_scrub_stamp': '2024-09-21T00:27:05.543789+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-21T00:27:12.321 INFO:tasks.ceph.ceph_manager.ceph:PG 8.c is not active+clean 2024-09-21T00:27:12.321 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:27:06.549347+0000', 'last_change': '2024-09-21T00:27:06.549347+0000', 'last_active': '2024-09-21T00:27:06.549347+0000', 'last_peered': '2024-09-21T00:27:06.549347+0000', 'last_clean': '2024-09-21T00:27:06.549347+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:27:06.549347+0000', 'last_undegraded': '2024-09-21T00:27:06.549347+0000', 'last_fullsized': '2024-09-21T00:27:06.549347+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'last_clean_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:27:12.321 INFO:tasks.ceph.ceph_manager.ceph:PG 7.3 is not active+clean 2024-09-21T00:27:12.321 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-09-21T00:27:05.554822+0000', 'last_change': '2024-09-21T00:27:05.554822+0000', 'last_active': '2024-09-21T00:27:05.543789+0000', 'last_peered': '2024-09-21T00:27:05.543789+0000', 'last_clean': '2024-09-21T00:27:05.543789+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:27:05.554822+0000', 'last_undegraded': '2024-09-21T00:27:05.554822+0000', 'last_fullsized': '2024-09-21T00:27:05.554822+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:27:05.543789+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:27:05.543789+0000', 'last_clean_scrub_stamp': '2024-09-21T00:27:05.543789+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-21T00:27:12.321 INFO:tasks.ceph.ceph_manager.ceph:PG 8.d is not active+clean 2024-09-21T00:27:12.322 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:27:06.549347+0000', 'last_change': '2024-09-21T00:27:06.549347+0000', 'last_active': '2024-09-21T00:27:06.549347+0000', 'last_peered': '2024-09-21T00:27:06.549347+0000', 'last_clean': '2024-09-21T00:27:06.549347+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:27:06.549347+0000', 'last_undegraded': '2024-09-21T00:27:06.549347+0000', 'last_fullsized': '2024-09-21T00:27:06.549347+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'last_clean_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:27:12.322 INFO:tasks.ceph.ceph_manager.ceph:PG 7.2 is not active+clean 2024-09-21T00:27:12.322 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-09-21T00:27:05.554816+0000', 'last_change': '2024-09-21T00:27:05.554816+0000', 'last_active': '2024-09-21T00:27:05.543789+0000', 'last_peered': '2024-09-21T00:27:05.543789+0000', 'last_clean': '2024-09-21T00:27:05.543789+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:27:05.554816+0000', 'last_undegraded': '2024-09-21T00:27:05.554816+0000', 'last_fullsized': '2024-09-21T00:27:05.554816+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:27:05.543789+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:27:05.543789+0000', 'last_clean_scrub_stamp': '2024-09-21T00:27:05.543789+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-21T00:27:12.322 INFO:tasks.ceph.ceph_manager.ceph:PG 8.e is not active+clean 2024-09-21T00:27:12.322 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:27:06.549347+0000', 'last_change': '2024-09-21T00:27:06.549347+0000', 'last_active': '2024-09-21T00:27:06.549347+0000', 'last_peered': '2024-09-21T00:27:06.549347+0000', 'last_clean': '2024-09-21T00:27:06.549347+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:27:06.549347+0000', 'last_undegraded': '2024-09-21T00:27:06.549347+0000', 'last_fullsized': '2024-09-21T00:27:06.549347+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'last_clean_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:27:12.322 INFO:tasks.ceph.ceph_manager.ceph:PG 7.1 is not active+clean 2024-09-21T00:27:12.322 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-09-21T00:27:05.557773+0000', 'last_change': '2024-09-21T00:27:05.557773+0000', 'last_active': '2024-09-21T00:27:05.543789+0000', 'last_peered': '2024-09-21T00:27:05.543789+0000', 'last_clean': '2024-09-21T00:27:05.543789+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:27:05.557773+0000', 'last_undegraded': '2024-09-21T00:27:05.557773+0000', 'last_fullsized': '2024-09-21T00:27:05.557773+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:27:05.543789+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:27:05.543789+0000', 'last_clean_scrub_stamp': '2024-09-21T00:27:05.543789+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-21T00:27:12.322 INFO:tasks.ceph.ceph_manager.ceph:PG 8.f is not active+clean 2024-09-21T00:27:12.322 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:27:06.549347+0000', 'last_change': '2024-09-21T00:27:06.549347+0000', 'last_active': '2024-09-21T00:27:06.549347+0000', 'last_peered': '2024-09-21T00:27:06.549347+0000', 'last_clean': '2024-09-21T00:27:06.549347+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:27:06.549347+0000', 'last_undegraded': '2024-09-21T00:27:06.549347+0000', 'last_fullsized': '2024-09-21T00:27:06.549347+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'last_clean_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:27:12.322 INFO:tasks.ceph.ceph_manager.ceph:PG 7.0 is not active+clean 2024-09-21T00:27:12.322 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-09-21T00:27:05.555561+0000', 'last_change': '2024-09-21T00:27:05.555561+0000', 'last_active': '2024-09-21T00:27:05.543789+0000', 'last_peered': '2024-09-21T00:27:05.543789+0000', 'last_clean': '2024-09-21T00:27:05.543789+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:27:05.555561+0000', 'last_undegraded': '2024-09-21T00:27:05.555561+0000', 'last_fullsized': '2024-09-21T00:27:05.555561+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:27:05.543789+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:27:05.543789+0000', 'last_clean_scrub_stamp': '2024-09-21T00:27:05.543789+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-21T00:27:12.323 INFO:tasks.ceph.ceph_manager.ceph:PG 8.9 is not active+clean 2024-09-21T00:27:12.323 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:27:06.549347+0000', 'last_change': '2024-09-21T00:27:06.549347+0000', 'last_active': '2024-09-21T00:27:06.549347+0000', 'last_peered': '2024-09-21T00:27:06.549347+0000', 'last_clean': '2024-09-21T00:27:06.549347+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:27:06.549347+0000', 'last_undegraded': '2024-09-21T00:27:06.549347+0000', 'last_fullsized': '2024-09-21T00:27:06.549347+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'last_clean_scrub_stamp': '2024-09-21T00:27:06.549347+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:27:12.323 INFO:tasks.ceph.ceph_manager.ceph:PG 7.6 is not active+clean 2024-09-21T00:27:12.323 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-09-21T00:27:05.561883+0000', 'last_change': '2024-09-21T00:27:05.561883+0000', 'last_active': '2024-09-21T00:27:05.543789+0000', 'last_peered': '2024-09-21T00:27:05.543789+0000', 'last_clean': '2024-09-21T00:27:05.543789+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:27:05.561883+0000', 'last_undegraded': '2024-09-21T00:27:05.561883+0000', 'last_fullsized': '2024-09-21T00:27:05.561883+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:27:05.543789+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:27:05.543789+0000', 'last_clean_scrub_stamp': '2024-09-21T00:27:05.543789+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-21T00:27:12.323 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-21T00:27:12.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.452+0000 7f511180a640 1 -- 172.21.15.112:0/2515924660 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f510c15a6f0 msgr2=0x7f510c15e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:12.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.452+0000 7f511180a640 1 --2- 172.21.15.112:0/2515924660 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f510c15a6f0 0x7f510c15e230 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f510c0b5180 tx=0x7f510002a560 comp rx=0 tx=0).stop 2024-09-21T00:27:12.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.452+0000 7f511180a640 1 -- 172.21.15.112:0/2515924660 shutdown_connections 2024-09-21T00:27:12.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.452+0000 7f511180a640 1 --2- 172.21.15.112:0/2515924660 >> v2:172.21.15.187:3300/0 conn(0x7f510c0d0980 0x7f510c0d0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:12.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.452+0000 7f511180a640 1 --2- 172.21.15.112:0/2515924660 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f510c15a6f0 0x7f510c15e230 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:12.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.452+0000 7f511180a640 1 --2- 172.21.15.112:0/2515924660 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f510c0d01f0 0x7f510c15daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:12.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.452+0000 7f511180a640 1 -- 172.21.15.112:0/2515924660 >> 172.21.15.112:0/2515924660 conn(0x7f510c072990 msgr2=0x7f510c072da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:12.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.452+0000 7f511180a640 1 -- 172.21.15.112:0/2515924660 shutdown_connections 2024-09-21T00:27:12.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.452+0000 7f511180a640 1 -- 172.21.15.112:0/2515924660 wait complete. 2024-09-21T00:27:12.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.453+0000 7f511180a640 1 Processor -- start 2024-09-21T00:27:12.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.454+0000 7f511180a640 1 -- start start 2024-09-21T00:27:12.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.455+0000 7f511180a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f510c0d01f0 0x7f510c0cba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:12.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.455+0000 7f511180a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f510c0d0980 0x7f510c070100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:12.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.455+0000 7f511180a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f510c15a6f0 0x7f510c070640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:12.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.455+0000 7f511180a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f510c0ccfb0 con 0x7f510c0d01f0 2024-09-21T00:27:12.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.455+0000 7f511180a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f510c15e980 con 0x7f510c15a6f0 2024-09-21T00:27:12.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.455+0000 7f511180a640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f510c0cd130 con 0x7f510c0d0980 2024-09-21T00:27:12.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.455+0000 7f510affd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f510c0d01f0 0x7f510c0cba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:12.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.455+0000 7f510affd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f510c0d01f0 0x7f510c0cba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55470/0 (socket says 172.21.15.112:55470) 2024-09-21T00:27:12.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.455+0000 7f510affd640 1 -- 172.21.15.112:0/55986794 learned_addr learned my addr 172.21.15.112:0/55986794 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:27:12.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.455+0000 7f510a7fc640 1 --2- 172.21.15.112:0/55986794 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f510c0d0980 0x7f510c070100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:12.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.455+0000 7f510b7fe640 1 --2- 172.21.15.112:0/55986794 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f510c15a6f0 0x7f510c070640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:12.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.455+0000 7f510affd640 1 -- 172.21.15.112:0/55986794 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f510c0d0980 msgr2=0x7f510c070100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:12.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.455+0000 7f510affd640 1 --2- 172.21.15.112:0/55986794 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f510c0d0980 0x7f510c070100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:12.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.455+0000 7f510affd640 1 -- 172.21.15.112:0/55986794 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f510c15a6f0 msgr2=0x7f510c070640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:12.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.455+0000 7f510affd640 1 --2- 172.21.15.112:0/55986794 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f510c15a6f0 0x7f510c070640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:12.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.455+0000 7f510affd640 1 -- 172.21.15.112:0/55986794 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f510c1f71d0 con 0x7f510c0d01f0 2024-09-21T00:27:12.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.455+0000 7f510b7fe640 1 --2- 172.21.15.112:0/55986794 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f510c15a6f0 0x7f510c070640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:27:12.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.455+0000 7f510affd640 1 --2- 172.21.15.112:0/55986794 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f510c0d01f0 0x7f510c0cba20 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f50fc00e9c0 tx=0x7f50fc00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:12.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.456+0000 7f50f3fff640 1 -- 172.21.15.112:0/55986794 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50fc00cd80 con 0x7f510c0d01f0 2024-09-21T00:27:12.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.456+0000 7f511180a640 1 -- 172.21.15.112:0/55986794 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f510c1f74b0 con 0x7f510c0d01f0 2024-09-21T00:27:12.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.456+0000 7f50f3fff640 1 -- 172.21.15.112:0/55986794 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f50fc011470 con 0x7f510c0d01f0 2024-09-21T00:27:12.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.456+0000 7f50f3fff640 1 -- 172.21.15.112:0/55986794 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50fc0118c0 con 0x7f510c0d01f0 2024-09-21T00:27:12.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.456+0000 7f511180a640 1 -- 172.21.15.112:0/55986794 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f510c1f7a70 con 0x7f510c0d01f0 2024-09-21T00:27:12.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.457+0000 7f511180a640 1 -- 172.21.15.112:0/55986794 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f510c0d0d90 con 0x7f510c0d01f0 2024-09-21T00:27:12.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.457+0000 7f50f3fff640 1 -- 172.21.15.112:0/55986794 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f50fc011a60 con 0x7f510c0d01f0 2024-09-21T00:27:12.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.458+0000 7f50f3fff640 1 --2- 172.21.15.112:0/55986794 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f50f8078b90 0x7f50f807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:12.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.458+0000 7f50f3fff640 1 -- 172.21.15.112:0/55986794 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 6101+0+0 (secure 0 0 0) 0x7f50fc09bad0 con 0x7f510c0d01f0 2024-09-21T00:27:12.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.458+0000 7f510a7fc640 1 --2- 172.21.15.112:0/55986794 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f50f8078b90 0x7f50f807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:12.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.459+0000 7f510a7fc640 1 --2- 172.21.15.112:0/55986794 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f50f8078b90 0x7f50f807b0a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f5100033000 tx=0x7f5100004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:12.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.462+0000 7f50f3fff640 1 -- 172.21.15.112:0/55986794 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f50fc016030 con 0x7f510c0d01f0 2024-09-21T00:27:12.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.706+0000 7f511180a640 1 -- 172.21.15.112:0/55986794 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f510c0b74c0 con 0x7f510c0d01f0 2024-09-21T00:27:12.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.707+0000 7f50f3fff640 1 -- 172.21.15.112:0/55986794 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1761 (secure 0 0 0) 0x7f50fc063d00 con 0x7f510c0d01f0 2024-09-21T00:27:12.709 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:27:12.709 INFO:teuthology.orchestra.run.smithi112.stdout:{"fsid":"8b34692b-dc65-49b2-a08f-26f707b96ac5","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":109,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":49,"num_osds":8,"num_up_osds":8,"osd_up_since":1726878336,"num_in_osds":8,"osd_in_since":1726878326,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":54},{"state_name":"creating+peering","count":29},{"state_name":"active+clean","count":14}],"num_pgs":97,"num_pools":3,"num_objects":4,"data_bytes":590368,"bytes_used":226918400,"bytes_avail":772867194880,"bytes_total":773094113280,"unknown_pgs_ratio":0.55670100450515747,"inactive_pgs_ratio":0.29896906018257141,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":0},"fsmap":{"epoch":26,"btime":"2024-09-21T00:27:10:580658+0000","id":3,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":3,"rank":0,"name":"a","status":"up:active","gid":5403}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":4,"modified":"2024-09-21T00:26:37.100727+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-21T00:27:12.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.709+0000 7f511180a640 1 -- 172.21.15.112:0/55986794 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f50f8078b90 msgr2=0x7f50f807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:12.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.709+0000 7f511180a640 1 --2- 172.21.15.112:0/55986794 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f50f8078b90 0x7f50f807b0a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f5100033000 tx=0x7f5100004730 comp rx=0 tx=0).stop 2024-09-21T00:27:12.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.709+0000 7f511180a640 1 -- 172.21.15.112:0/55986794 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f510c0d01f0 msgr2=0x7f510c0cba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:12.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.709+0000 7f511180a640 1 --2- 172.21.15.112:0/55986794 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f510c0d01f0 0x7f510c0cba20 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f50fc00e9c0 tx=0x7f50fc00ee80 comp rx=0 tx=0).stop 2024-09-21T00:27:12.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.709+0000 7f511180a640 1 -- 172.21.15.112:0/55986794 shutdown_connections 2024-09-21T00:27:12.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.709+0000 7f511180a640 1 --2- 172.21.15.112:0/55986794 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f510c15a6f0 0x7f510c070640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:12.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.709+0000 7f511180a640 1 --2- 172.21.15.112:0/55986794 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f510c0d0980 0x7f510c070100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:12.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.709+0000 7f511180a640 1 --2- 172.21.15.112:0/55986794 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f50f8078b90 0x7f50f807b0a0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:12.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.709+0000 7f511180a640 1 --2- 172.21.15.112:0/55986794 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f510c0d01f0 0x7f510c0cba20 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:12.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.709+0000 7f511180a640 1 -- 172.21.15.112:0/55986794 >> 172.21.15.112:0/55986794 conn(0x7f510c072990 msgr2=0x7f510c15ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:12.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.709+0000 7f511180a640 1 -- 172.21.15.112:0/55986794 shutdown_connections 2024-09-21T00:27:12.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.709+0000 7f511180a640 1 -- 172.21.15.112:0/55986794 wait complete. 2024-09-21T00:27:12.722 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-21T00:27:12.722 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T00:27:12.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.833+0000 7fde1272b640 1 -- 172.21.15.112:0/2832284323 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fddf4004d40 msgr2=0x7fddf4005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:12.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.833+0000 7fde1272b640 1 --2- 172.21.15.112:0/2832284323 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fddf4004d40 0x7fddf4005150 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fde04005fd0 tx=0x7fddf4005830 comp rx=0 tx=0).stop 2024-09-21T00:27:12.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.833+0000 7fde10cc8640 1 -- 172.21.15.112:0/2832284323 reap_dead start 2024-09-21T00:27:12.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.833+0000 7fde1272b640 1 -- 172.21.15.112:0/2832284323 shutdown_connections 2024-09-21T00:27:12.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.833+0000 7fde1272b640 1 -- 172.21.15.112:0/2832284323 >> 172.21.15.112:0/2832284323 conn(0x7fde0c0d2350 msgr2=0x7fde0c0d2760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:12.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.833+0000 7fde1272b640 1 -- 172.21.15.112:0/2832284323 shutdown_connections 2024-09-21T00:27:12.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.833+0000 7fde1272b640 1 -- 172.21.15.112:0/2832284323 wait complete. 2024-09-21T00:27:12.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.835+0000 7fde1272b640 1 Processor -- start 2024-09-21T00:27:12.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.835+0000 7fde1272b640 1 -- start start 2024-09-21T00:27:12.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.836+0000 7fde1272b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fddf4004d40 0x7fde0c1f2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:12.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.836+0000 7fde1272b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fde0c1f39a0 0x7fde0c15a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:12.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.836+0000 7fde1272b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde0c15a760 0x7fde0c1f65a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:12.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.836+0000 7fde1272b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fde0c15d2b0 con 0x7fde0c15a760 2024-09-21T00:27:12.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.836+0000 7fde1272b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fde0c15d130 con 0x7fde0c1f39a0 2024-09-21T00:27:12.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.836+0000 7fde1272b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fde0c15fef0 con 0x7fddf4004d40 2024-09-21T00:27:12.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.837+0000 7fde10cc8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde0c15a760 0x7fde0c1f65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:12.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.837+0000 7fde10cc8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde0c15a760 0x7fde0c1f65a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55490/0 (socket says 172.21.15.112:55490) 2024-09-21T00:27:12.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.837+0000 7fde10cc8640 1 -- 172.21.15.112:0/1921406503 learned_addr learned my addr 172.21.15.112:0/1921406503 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:27:12.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.837+0000 7fde0bfff640 1 --2- 172.21.15.112:0/1921406503 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fddf4004d40 0x7fde0c1f2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:12.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.837+0000 7fde0b7fe640 1 --2- 172.21.15.112:0/1921406503 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fde0c1f39a0 0x7fde0c15a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:12.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.837+0000 7fde10cc8640 1 -- 172.21.15.112:0/1921406503 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fddf4004d40 msgr2=0x7fde0c1f2fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:12.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.837+0000 7fde10cc8640 1 --2- 172.21.15.112:0/1921406503 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fddf4004d40 0x7fde0c1f2fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:12.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.837+0000 7fde10cc8640 1 -- 172.21.15.112:0/1921406503 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fde0c1f39a0 msgr2=0x7fde0c15a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:12.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.837+0000 7fde10cc8640 1 --2- 172.21.15.112:0/1921406503 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fde0c1f39a0 0x7fde0c15a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:12.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.837+0000 7fde10cc8640 1 -- 172.21.15.112:0/1921406503 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde0c15ab70 con 0x7fde0c15a760 2024-09-21T00:27:12.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.837+0000 7fde0b7fe640 1 --2- 172.21.15.112:0/1921406503 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fde0c1f39a0 0x7fde0c15a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:27:12.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.837+0000 7fde10cc8640 1 --2- 172.21.15.112:0/1921406503 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde0c15a760 0x7fde0c1f65a0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fde0402aec0 tx=0x7fde04033bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:12.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.837+0000 7fde08ff9640 1 -- 172.21.15.112:0/1921406503 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde04042070 con 0x7fde0c15a760 2024-09-21T00:27:12.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.837+0000 7fde08ff9640 1 -- 172.21.15.112:0/1921406503 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fde04003970 con 0x7fde0c15a760 2024-09-21T00:27:12.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.837+0000 7fde1272b640 1 -- 172.21.15.112:0/1921406503 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde0c156470 con 0x7fde0c15a760 2024-09-21T00:27:12.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.837+0000 7fde08ff9640 1 -- 172.21.15.112:0/1921406503 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde04038040 con 0x7fde0c15a760 2024-09-21T00:27:12.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.837+0000 7fde1272b640 1 -- 172.21.15.112:0/1921406503 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde0c156a30 con 0x7fde0c15a760 2024-09-21T00:27:12.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.838+0000 7fde1272b640 1 -- 172.21.15.112:0/1921406503 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde0c1561e0 con 0x7fde0c15a760 2024-09-21T00:27:12.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.839+0000 7fde08ff9640 1 -- 172.21.15.112:0/1921406503 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fde04004030 con 0x7fde0c15a760 2024-09-21T00:27:12.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.840+0000 7fde08ff9640 1 --2- 172.21.15.112:0/1921406503 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fddf8078b90 0x7fddf807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:12.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.840+0000 7fde08ff9640 1 -- 172.21.15.112:0/1921406503 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 6101+0+0 (secure 0 0 0) 0x7fde040bb2f0 con 0x7fde0c15a760 2024-09-21T00:27:12.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.840+0000 7fde0bfff640 1 --2- 172.21.15.112:0/1921406503 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fddf8078b90 0x7fddf807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:12.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.840+0000 7fde0bfff640 1 --2- 172.21.15.112:0/1921406503 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fddf8078b90 0x7fddf807b0a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fddfc001220 tx=0x7fddfc000ed0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:12.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:12.844+0000 7fde08ff9640 1 -- 172.21.15.112:0/1921406503 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fde0c15ab70 con 0x7fde0c15a760 2024-09-21T00:27:13.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:13.017+0000 7fde1272b640 1 -- 172.21.15.112:0/1921406503 --> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fde0c1f4500 con 0x7fddf8078b90 2024-09-21T00:27:13.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:13.022+0000 7fde08ff9640 1 -- 172.21.15.112:0/1921406503 <== mgr.4103 v2:172.21.15.187:6800/2921522906 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+241701 (secure 0 0 0) 0x7fde0c1f4500 con 0x7fddf8078b90 2024-09-21T00:27:13.025 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:27:13.026 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2024-09-21T00:27:13.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:13.024+0000 7fde1272b640 1 -- 172.21.15.112:0/1921406503 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fddf8078b90 msgr2=0x7fddf807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:13.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:13.024+0000 7fde1272b640 1 --2- 172.21.15.112:0/1921406503 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fddf8078b90 0x7fddf807b0a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fddfc001220 tx=0x7fddfc000ed0 comp rx=0 tx=0).stop 2024-09-21T00:27:13.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:13.024+0000 7fde1272b640 1 -- 172.21.15.112:0/1921406503 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde0c15a760 msgr2=0x7fde0c1f65a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:13.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:13.024+0000 7fde1272b640 1 --2- 172.21.15.112:0/1921406503 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde0c15a760 0x7fde0c1f65a0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fde0402aec0 tx=0x7fde04033bf0 comp rx=0 tx=0).stop 2024-09-21T00:27:13.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:13.024+0000 7fde1272b640 1 -- 172.21.15.112:0/1921406503 shutdown_connections 2024-09-21T00:27:13.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:13.024+0000 7fde1272b640 1 --2- 172.21.15.112:0/1921406503 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fddf8078b90 0x7fddf807b0a0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:13.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:13.024+0000 7fde1272b640 1 --2- 172.21.15.112:0/1921406503 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde0c15a760 0x7fde0c1f65a0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:13.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:13.024+0000 7fde1272b640 1 --2- 172.21.15.112:0/1921406503 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fde0c1f39a0 0x7fde0c15a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:13.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:13.024+0000 7fde1272b640 1 --2- 172.21.15.112:0/1921406503 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fddf4004d40 0x7fde0c1f2fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:13.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:13.024+0000 7fde1272b640 1 -- 172.21.15.112:0/1921406503 >> 172.21.15.112:0/1921406503 conn(0x7fde0c0d2350 msgr2=0x7fde0c153130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:13.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:13.024+0000 7fde1272b640 1 -- 172.21.15.112:0/1921406503 shutdown_connections 2024-09-21T00:27:13.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:13.024+0000 7fde1272b640 1 -- 172.21.15.112:0/1921406503 wait complete. 2024-09-21T00:27:13.037 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":91,"stamp":"2024-09-21T00:27:11.109326+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":105,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":77,"ondisk_log_size":77,"up":86,"acting":86,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":7,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":221600,"kb_used_data":5376,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753120,"statfs":{"total":773094113280,"available":772867194880,"internally_reserved":0,"allocated":5505024,"data_stored":3025554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.557642"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-21T00:27:07.579407+0000","last_change":"2024-09-21T00:27:07.574598+0000","last_active":"2024-09-21T00:27:07.579407+0000","last_peered":"2024-09-21T00:27:07.579407+0000","last_clean":"2024-09-21T00:27:07.579407+0000","last_became_active":"2024-09-21T00:27:07.574242+0000","last_became_peered":"2024-09-21T00:27:07.574242+0000","last_unstale":"2024-09-21T00:27:07.579407+0000","last_undegraded":"2024-09-21T00:27:07.579407+0000","last_fullsized":"2024-09-21T00:27:07.579407+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T00:55:01.680636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-21T00:27:07.593752+0000","last_change":"2024-09-21T00:27:07.581689+0000","last_active":"2024-09-21T00:27:07.593752+0000","last_peered":"2024-09-21T00:27:07.593752+0000","last_clean":"2024-09-21T00:27:07.593752+0000","last_became_active":"2024-09-21T00:27:07.581012+0000","last_became_peered":"2024-09-21T00:27:07.581012+0000","last_unstale":"2024-09-21T00:27:07.593752+0000","last_undegraded":"2024-09-21T00:27:07.593752+0000","last_fullsized":"2024-09-21T00:27:07.593752+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:08:00.101537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-21T00:27:07.579280+0000","last_change":"2024-09-21T00:27:07.567996+0000","last_active":"2024-09-21T00:27:07.579280+0000","last_peered":"2024-09-21T00:27:07.579280+0000","last_clean":"2024-09-21T00:27:07.579280+0000","last_became_active":"2024-09-21T00:27:07.567702+0000","last_became_peered":"2024-09-21T00:27:07.567702+0000","last_unstale":"2024-09-21T00:27:07.579280+0000","last_undegraded":"2024-09-21T00:27:07.579280+0000","last_fullsized":"2024-09-21T00:27:07.579280+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:29:32.175256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-21T00:27:07.574128+0000","last_change":"2024-09-21T00:27:07.567050+0000","last_active":"2024-09-21T00:27:07.574128+0000","last_peered":"2024-09-21T00:27:07.574128+0000","last_clean":"2024-09-21T00:27:07.574128+0000","last_became_active":"2024-09-21T00:27:07.566711+0000","last_became_peered":"2024-09-21T00:27:07.566711+0000","last_unstale":"2024-09-21T00:27:07.574128+0000","last_undegraded":"2024-09-21T00:27:07.574128+0000","last_fullsized":"2024-09-21T00:27:07.574128+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:53:27.777241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-21T00:27:07.574359+0000","last_change":"2024-09-21T00:27:07.567067+0000","last_active":"2024-09-21T00:27:07.574359+0000","last_peered":"2024-09-21T00:27:07.574359+0000","last_clean":"2024-09-21T00:27:07.574359+0000","last_became_active":"2024-09-21T00:27:07.566709+0000","last_became_peered":"2024-09-21T00:27:07.566709+0000","last_unstale":"2024-09-21T00:27:07.574359+0000","last_undegraded":"2024-09-21T00:27:07.574359+0000","last_fullsized":"2024-09-21T00:27:07.574359+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:36:49.037229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-21T00:27:07.595592+0000","last_change":"2024-09-21T00:27:07.586170+0000","last_active":"2024-09-21T00:27:07.595592+0000","last_peered":"2024-09-21T00:27:07.595592+0000","last_clean":"2024-09-21T00:27:07.595592+0000","last_became_active":"2024-09-21T00:27:07.585790+0000","last_became_peered":"2024-09-21T00:27:07.585790+0000","last_unstale":"2024-09-21T00:27:07.595592+0000","last_undegraded":"2024-09-21T00:27:07.595592+0000","last_fullsized":"2024-09-21T00:27:07.595592+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:27:32.413929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-21T00:27:05.556637+0000","last_change":"2024-09-21T00:27:05.556637+0000","last_active":"2024-09-21T00:27:05.543789+0000","last_peered":"2024-09-21T00:27:05.543789+0000","last_clean":"2024-09-21T00:27:05.543789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:05.556637+0000","last_undegraded":"2024-09-21T00:27:05.556637+0000","last_fullsized":"2024-09-21T00:27:05.556637+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1e","version":"48'1","reported_seq":14,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-21T00:27:07.600346+0000","last_change":"2024-09-21T00:27:06.578147+0000","last_active":"2024-09-21T00:27:07.600346+0000","last_peered":"2024-09-21T00:27:07.600346+0000","last_clean":"2024-09-21T00:27:07.600346+0000","last_became_active":"2024-09-21T00:27:06.577845+0000","last_became_peered":"2024-09-21T00:27:06.577845+0000","last_unstale":"2024-09-21T00:27:07.600346+0000","last_undegraded":"2024-09-21T00:27:07.600346+0000","last_fullsized":"2024-09-21T00:27:07.600346+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:18:27.263054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1d","version":"48'2","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-21T00:27:07.599338+0000","last_change":"2024-09-21T00:27:06.573813+0000","last_active":"2024-09-21T00:27:07.599338+0000","last_peered":"2024-09-21T00:27:07.599338+0000","last_clean":"2024-09-21T00:27:07.599338+0000","last_became_active":"2024-09-21T00:27:06.573532+0000","last_became_peered":"2024-09-21T00:27:06.573532+0000","last_unstale":"2024-09-21T00:27:07.599338+0000","last_undegraded":"2024-09-21T00:27:07.599338+0000","last_fullsized":"2024-09-21T00:27:07.599338+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:31:15.501365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-21T00:27:05.555296+0000","last_change":"2024-09-21T00:27:05.555296+0000","last_active":"2024-09-21T00:27:05.543789+0000","last_peered":"2024-09-21T00:27:05.543789+0000","last_clean":"2024-09-21T00:27:05.543789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:05.555296+0000","last_undegraded":"2024-09-21T00:27:05.555296+0000","last_fullsized":"2024-09-21T00:27:05.555296+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-21T00:27:05.562604+0000","last_change":"2024-09-21T00:27:05.562604+0000","last_active":"2024-09-21T00:27:05.543789+0000","last_peered":"2024-09-21T00:27:05.543789+0000","last_clean":"2024-09-21T00:27:05.543789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:05.562604+0000","last_undegraded":"2024-09-21T00:27:05.562604+0000","last_fullsized":"2024-09-21T00:27:05.562604+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-21T00:27:05.557122+0000","last_change":"2024-09-21T00:27:05.557122+0000","last_active":"2024-09-21T00:27:05.543789+0000","last_peered":"2024-09-21T00:27:05.543789+0000","last_clean":"2024-09-21T00:27:05.543789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:05.557122+0000","last_undegraded":"2024-09-21T00:27:05.557122+0000","last_fullsized":"2024-09-21T00:27:05.557122+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-21T00:27:05.555459+0000","last_change":"2024-09-21T00:27:05.555459+0000","last_active":"2024-09-21T00:27:05.543789+0000","last_peered":"2024-09-21T00:27:05.543789+0000","last_clean":"2024-09-21T00:27:05.543789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:05.555459+0000","last_undegraded":"2024-09-21T00:27:05.555459+0000","last_fullsized":"2024-09-21T00:27:05.555459+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-21T00:27:05.562527+0000","last_change":"2024-09-21T00:27:05.562527+0000","last_active":"2024-09-21T00:27:05.543789+0000","last_peered":"2024-09-21T00:27:05.543789+0000","last_clean":"2024-09-21T00:27:05.543789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:05.562527+0000","last_undegraded":"2024-09-21T00:27:05.562527+0000","last_fullsized":"2024-09-21T00:27:05.562527+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-21T00:27:05.562841+0000","last_change":"2024-09-21T00:27:05.562841+0000","last_active":"2024-09-21T00:27:05.543789+0000","last_peered":"2024-09-21T00:27:05.543789+0000","last_clean":"2024-09-21T00:27:05.543789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:05.562841+0000","last_undegraded":"2024-09-21T00:27:05.562841+0000","last_fullsized":"2024-09-21T00:27:05.562841+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-21T00:27:05.558519+0000","last_change":"2024-09-21T00:27:05.558519+0000","last_active":"2024-09-21T00:27:05.543789+0000","last_peered":"2024-09-21T00:27:05.543789+0000","last_clean":"2024-09-21T00:27:05.543789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:05.558519+0000","last_undegraded":"2024-09-21T00:27:05.558519+0000","last_fullsized":"2024-09-21T00:27:05.558519+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-21T00:27:05.563390+0000","last_change":"2024-09-21T00:27:05.563390+0000","last_active":"2024-09-21T00:27:05.543789+0000","last_peered":"2024-09-21T00:27:05.543789+0000","last_clean":"2024-09-21T00:27:05.543789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:05.563390+0000","last_undegraded":"2024-09-21T00:27:05.563390+0000","last_fullsized":"2024-09-21T00:27:05.563390+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-21T00:27:05.554843+0000","last_change":"2024-09-21T00:27:05.554843+0000","last_active":"2024-09-21T00:27:05.543789+0000","last_peered":"2024-09-21T00:27:05.543789+0000","last_clean":"2024-09-21T00:27:05.543789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:05.554843+0000","last_undegraded":"2024-09-21T00:27:05.554843+0000","last_fullsized":"2024-09-21T00:27:05.554843+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-21T00:27:05.562134+0000","last_change":"2024-09-21T00:27:05.562134+0000","last_active":"2024-09-21T00:27:05.543789+0000","last_peered":"2024-09-21T00:27:05.543789+0000","last_clean":"2024-09-21T00:27:05.543789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:05.562134+0000","last_undegraded":"2024-09-21T00:27:05.562134+0000","last_fullsized":"2024-09-21T00:27:05.562134+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-21T00:27:05.556088+0000","last_change":"2024-09-21T00:27:05.556088+0000","last_active":"2024-09-21T00:27:05.543789+0000","last_peered":"2024-09-21T00:27:05.543789+0000","last_clean":"2024-09-21T00:27:05.543789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:05.556088+0000","last_undegraded":"2024-09-21T00:27:05.556088+0000","last_fullsized":"2024-09-21T00:27:05.556088+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-21T00:27:05.562785+0000","last_change":"2024-09-21T00:27:05.562785+0000","last_active":"2024-09-21T00:27:05.543789+0000","last_peered":"2024-09-21T00:27:05.543789+0000","last_clean":"2024-09-21T00:27:05.543789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:05.562785+0000","last_undegraded":"2024-09-21T00:27:05.562785+0000","last_fullsized":"2024-09-21T00:27:05.562785+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-21T00:27:05.554830+0000","last_change":"2024-09-21T00:27:05.554830+0000","last_active":"2024-09-21T00:27:05.543789+0000","last_peered":"2024-09-21T00:27:05.543789+0000","last_clean":"2024-09-21T00:27:05.543789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:05.554830+0000","last_undegraded":"2024-09-21T00:27:05.554830+0000","last_fullsized":"2024-09-21T00:27:05.554830+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-21T00:27:05.561823+0000","last_change":"2024-09-21T00:27:05.561823+0000","last_active":"2024-09-21T00:27:05.543789+0000","last_peered":"2024-09-21T00:27:05.543789+0000","last_clean":"2024-09-21T00:27:05.543789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:05.561823+0000","last_undegraded":"2024-09-21T00:27:05.561823+0000","last_fullsized":"2024-09-21T00:27:05.561823+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-21T00:27:07.594119+0000","last_change":"2024-09-21T00:27:07.585242+0000","last_active":"2024-09-21T00:27:07.594119+0000","last_peered":"2024-09-21T00:27:07.594119+0000","last_clean":"2024-09-21T00:27:07.594119+0000","last_became_active":"2024-09-21T00:27:07.584922+0000","last_became_peered":"2024-09-21T00:27:07.584922+0000","last_unstale":"2024-09-21T00:27:07.594119+0000","last_undegraded":"2024-09-21T00:27:07.594119+0000","last_fullsized":"2024-09-21T00:27:07.594119+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:52:07.026811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-21T00:27:05.562632+0000","last_change":"2024-09-21T00:27:05.562632+0000","last_active":"2024-09-21T00:27:05.543789+0000","last_peered":"2024-09-21T00:27:05.543789+0000","last_clean":"2024-09-21T00:27:05.543789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:05.562632+0000","last_undegraded":"2024-09-21T00:27:05.562632+0000","last_fullsized":"2024-09-21T00:27:05.562632+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-21T00:27:05.555437+0000","last_change":"2024-09-21T00:27:05.555437+0000","last_active":"2024-09-21T00:27:05.543789+0000","last_peered":"2024-09-21T00:27:05.543789+0000","last_clean":"2024-09-21T00:27:05.543789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:05.555437+0000","last_undegraded":"2024-09-21T00:27:05.555437+0000","last_fullsized":"2024-09-21T00:27:05.555437+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-21T00:27:07.580102+0000","last_change":"2024-09-21T00:27:07.569099+0000","last_active":"2024-09-21T00:27:07.580102+0000","last_peered":"2024-09-21T00:27:07.580102+0000","last_clean":"2024-09-21T00:27:07.580102+0000","last_became_active":"2024-09-21T00:27:07.568739+0000","last_became_peered":"2024-09-21T00:27:07.568739+0000","last_unstale":"2024-09-21T00:27:07.580102+0000","last_undegraded":"2024-09-21T00:27:07.580102+0000","last_fullsized":"2024-09-21T00:27:07.580102+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:18:56.106026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-21T00:27:05.556442+0000","last_change":"2024-09-21T00:27:05.556442+0000","last_active":"2024-09-21T00:27:05.543789+0000","last_peered":"2024-09-21T00:27:05.543789+0000","last_clean":"2024-09-21T00:27:05.543789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:05.556442+0000","last_undegraded":"2024-09-21T00:27:05.556442+0000","last_fullsized":"2024-09-21T00:27:05.556442+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.b","version":"48'2","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-21T00:27:07.599703+0000","last_change":"2024-09-21T00:27:06.580120+0000","last_active":"2024-09-21T00:27:07.599703+0000","last_peered":"2024-09-21T00:27:07.599703+0000","last_clean":"2024-09-21T00:27:07.599703+0000","last_became_active":"2024-09-21T00:27:06.579795+0000","last_became_peered":"2024-09-21T00:27:06.579795+0000","last_unstale":"2024-09-21T00:27:07.599703+0000","last_undegraded":"2024-09-21T00:27:07.599703+0000","last_fullsized":"2024-09-21T00:27:07.599703+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:31:18.308257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-21T00:27:05.556012+0000","last_change":"2024-09-21T00:27:05.556012+0000","last_active":"2024-09-21T00:27:05.543789+0000","last_peered":"2024-09-21T00:27:05.543789+0000","last_clean":"2024-09-21T00:27:05.543789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:05.556012+0000","last_undegraded":"2024-09-21T00:27:05.556012+0000","last_fullsized":"2024-09-21T00:27:05.556012+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-21T00:27:05.561999+0000","last_change":"2024-09-21T00:27:05.561999+0000","last_active":"2024-09-21T00:27:05.543789+0000","last_peered":"2024-09-21T00:27:05.543789+0000","last_clean":"2024-09-21T00:27:05.543789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:05.561999+0000","last_undegraded":"2024-09-21T00:27:05.561999+0000","last_fullsized":"2024-09-21T00:27:05.561999+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-21T00:27:07.595184+0000","last_change":"2024-09-21T00:27:07.590250+0000","last_active":"2024-09-21T00:27:07.595184+0000","last_peered":"2024-09-21T00:27:07.595184+0000","last_clean":"2024-09-21T00:27:07.595184+0000","last_became_active":"2024-09-21T00:27:07.589930+0000","last_became_peered":"2024-09-21T00:27:07.589930+0000","last_unstale":"2024-09-21T00:27:07.595184+0000","last_undegraded":"2024-09-21T00:27:07.595184+0000","last_fullsized":"2024-09-21T00:27:07.595184+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:55:47.579955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-21T00:27:05.561751+0000","last_change":"2024-09-21T00:27:05.561751+0000","last_active":"2024-09-21T00:27:05.543789+0000","last_peered":"2024-09-21T00:27:05.543789+0000","last_clean":"2024-09-21T00:27:05.543789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:05.561751+0000","last_undegraded":"2024-09-21T00:27:05.561751+0000","last_fullsized":"2024-09-21T00:27:05.561751+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-21T00:27:05.556126+0000","last_change":"2024-09-21T00:27:05.556126+0000","last_active":"2024-09-21T00:27:05.543789+0000","last_peered":"2024-09-21T00:27:05.543789+0000","last_clean":"2024-09-21T00:27:05.543789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:05.556126+0000","last_undegraded":"2024-09-21T00:27:05.556126+0000","last_fullsized":"2024-09-21T00:27:05.556126+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":10,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-21T00:27:07.593806+0000","last_change":"2024-09-21T00:27:07.581995+0000","last_active":"2024-09-21T00:27:07.593806+0000","last_peered":"2024-09-21T00:27:07.593806+0000","last_clean":"2024-09-21T00:27:07.593806+0000","last_became_active":"2024-09-21T00:27:07.581400+0000","last_became_peered":"2024-09-21T00:27:07.581400+0000","last_unstale":"2024-09-21T00:27:07.593806+0000","last_undegraded":"2024-09-21T00:27:07.593806+0000","last_fullsized":"2024-09-21T00:27:07.593806+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:20:00.370047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-21T00:27:05.562808+0000","last_change":"2024-09-21T00:27:05.562808+0000","last_active":"2024-09-21T00:27:05.543789+0000","last_peered":"2024-09-21T00:27:05.543789+0000","last_clean":"2024-09-21T00:27:05.543789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:05.562808+0000","last_undegraded":"2024-09-21T00:27:05.562808+0000","last_fullsized":"2024-09-21T00:27:05.562808+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-21T00:27:05.562127+0000","last_change":"2024-09-21T00:27:05.562127+0000","last_active":"2024-09-21T00:27:05.543789+0000","last_peered":"2024-09-21T00:27:05.543789+0000","last_clean":"2024-09-21T00:27:05.543789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:05.562127+0000","last_undegraded":"2024-09-21T00:27:05.562127+0000","last_fullsized":"2024-09-21T00:27:05.562127+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-21T00:27:05.554822+0000","last_change":"2024-09-21T00:27:05.554822+0000","last_active":"2024-09-21T00:27:05.543789+0000","last_peered":"2024-09-21T00:27:05.543789+0000","last_clean":"2024-09-21T00:27:05.543789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:05.554822+0000","last_undegraded":"2024-09-21T00:27:05.554822+0000","last_fullsized":"2024-09-21T00:27:05.554822+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-21T00:27:05.554816+0000","last_change":"2024-09-21T00:27:05.554816+0000","last_active":"2024-09-21T00:27:05.543789+0000","last_peered":"2024-09-21T00:27:05.543789+0000","last_clean":"2024-09-21T00:27:05.543789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:05.554816+0000","last_undegraded":"2024-09-21T00:27:05.554816+0000","last_fullsized":"2024-09-21T00:27:05.554816+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-21T00:27:05.557773+0000","last_change":"2024-09-21T00:27:05.557773+0000","last_active":"2024-09-21T00:27:05.543789+0000","last_peered":"2024-09-21T00:27:05.543789+0000","last_clean":"2024-09-21T00:27:05.543789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:05.557773+0000","last_undegraded":"2024-09-21T00:27:05.557773+0000","last_fullsized":"2024-09-21T00:27:05.557773+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-21T00:27:05.555561+0000","last_change":"2024-09-21T00:27:05.555561+0000","last_active":"2024-09-21T00:27:05.543789+0000","last_peered":"2024-09-21T00:27:05.543789+0000","last_clean":"2024-09-21T00:27:05.543789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:05.555561+0000","last_undegraded":"2024-09-21T00:27:05.555561+0000","last_fullsized":"2024-09-21T00:27:05.555561+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:27:06.549347+0000","last_change":"2024-09-21T00:27:06.549347+0000","last_active":"2024-09-21T00:27:06.549347+0000","last_peered":"2024-09-21T00:27:06.549347+0000","last_clean":"2024-09-21T00:27:06.549347+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:06.549347+0000","last_undegraded":"2024-09-21T00:27:06.549347+0000","last_fullsized":"2024-09-21T00:27:06.549347+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-21T00:27:05.561883+0000","last_change":"2024-09-21T00:27:05.561883+0000","last_active":"2024-09-21T00:27:05.543789+0000","last_peered":"2024-09-21T00:27:05.543789+0000","last_clean":"2024-09-21T00:27:05.543789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:27:05.561883+0000","last_undegraded":"2024-09-21T00:27:05.561883+0000","last_fullsized":"2024-09-21T00:27:05.561883+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":153,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T00:27:05.563691+0000","last_change":"2024-09-21T00:25:38.725792+0000","last_active":"2024-09-21T00:27:05.563691+0000","last_peered":"2024-09-21T00:27:05.563691+0000","last_clean":"2024-09-21T00:27:05.563691+0000","last_became_active":"2024-09-21T00:25:38.724668+0000","last_became_peered":"2024-09-21T00:25:38.724668+0000","last_unstale":"2024-09-21T00:27:05.563691+0000","last_undegraded":"2024-09-21T00:27:05.563691+0000","last_fullsized":"2024-09-21T00:27:05.563691+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:37.163216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:37.163216+0000","last_clean_scrub_stamp":"2024-09-21T00:25:37.163216+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:14:36.588660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":20,"acting":20,"num_store_stats":1},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509461,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28132,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343708,"statfs":{"total":96636764160,"available":96607956992,"internally_reserved":0,"allocated":1130496,"data_stored":819241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509462,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27552,"kb_used_data":524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344288,"statfs":{"total":96636764160,"available":96608550912,"internally_reserved":0,"allocated":536576,"data_stored":228873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509461,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27552,"kb_used_data":524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344288,"statfs":{"total":96636764160,"available":96608550912,"internally_reserved":0,"allocated":536576,"data_stored":228873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509460,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27552,"kb_used_data":524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344288,"statfs":{"total":96636764160,"available":96608550912,"internally_reserved":0,"allocated":536576,"data_stored":228873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509462,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27552,"kb_used_data":524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344288,"statfs":{"total":96636764160,"available":96608550912,"internally_reserved":0,"allocated":536576,"data_stored":228873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509462,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27552,"kb_used_data":524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344288,"statfs":{"total":96636764160,"available":96608550912,"internally_reserved":0,"allocated":536576,"data_stored":228873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509462,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28132,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343708,"statfs":{"total":96636764160,"available":96607956992,"internally_reserved":0,"allocated":1130496,"data_stored":819241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27576,"kb_used_data":548,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344264,"statfs":{"total":96636764160,"available":96608526336,"internally_reserved":0,"allocated":561152,"data_stored":242707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T00:27:14.926 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T00:27:14.929 DEBUG:teuthology.orchestra.run.smithi187:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T00:27:16.041 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T00:27:16.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.166+0000 7f556b3eb640 1 -- 172.21.15.112:0/2619587062 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f55640cd040 msgr2=0x7f55640d04c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:16.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.166+0000 7f556b3eb640 1 --2- 172.21.15.112:0/2619587062 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f55640cd040 0x7f55640d04c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f555c005fd0 tx=0x7f555c02a8b0 comp rx=0 tx=0).stop 2024-09-21T00:27:16.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.167+0000 7f556b3eb640 1 -- 172.21.15.112:0/2619587062 shutdown_connections 2024-09-21T00:27:16.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.167+0000 7f556b3eb640 1 --2- 172.21.15.112:0/2619587062 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f55640cd040 0x7f55640d04c0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:16.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.167+0000 7f556b3eb640 1 --2- 172.21.15.112:0/2619587062 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f55641536e0 0x7f55640d17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:16.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.167+0000 7f556b3eb640 1 -- 172.21.15.112:0/2619587062 >> 172.21.15.112:0/2619587062 conn(0x7f55640cef40 msgr2=0x7f55640cf350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:16.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.167+0000 7f556b3eb640 1 -- 172.21.15.112:0/2619587062 shutdown_connections 2024-09-21T00:27:16.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.167+0000 7f556b3eb640 1 -- 172.21.15.112:0/2619587062 wait complete. 2024-09-21T00:27:16.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.168+0000 7f556b3eb640 1 Processor -- start 2024-09-21T00:27:16.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.168+0000 7f556b3eb640 1 -- start start 2024-09-21T00:27:16.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.170+0000 7f556b3eb640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f55640d1d00 0x7f55641f9010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:16.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.170+0000 7f556b3eb640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f55641536e0 0x7f55640bf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:16.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.170+0000 7f556b3eb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55640c25c0 0x7f55641e73c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:16.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.170+0000 7f556b3eb640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f55640d0d90 con 0x7f55640c25c0 2024-09-21T00:27:16.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.170+0000 7f556b3eb640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f55640d0c10 con 0x7f55641536e0 2024-09-21T00:27:16.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.170+0000 7f556b3eb640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f55640d0f10 con 0x7f55640d1d00 2024-09-21T00:27:16.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.170+0000 7f5569988640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55640c25c0 0x7f55641e73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:16.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.170+0000 7f5569988640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55640c25c0 0x7f55641e73c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47604/0 (socket says 172.21.15.112:47604) 2024-09-21T00:27:16.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.170+0000 7f5569988640 1 -- 172.21.15.112:0/3674802437 learned_addr learned my addr 172.21.15.112:0/3674802437 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:27:16.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.170+0000 7f5569187640 1 --2- 172.21.15.112:0/3674802437 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f55640d1d00 0x7f55641f9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:16.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.170+0000 7f5568986640 1 --2- 172.21.15.112:0/3674802437 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f55641536e0 0x7f55640bf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:16.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.170+0000 7f5569988640 1 -- 172.21.15.112:0/3674802437 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f55640d1d00 msgr2=0x7f55641f9010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:16.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.170+0000 7f5569988640 1 --2- 172.21.15.112:0/3674802437 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f55640d1d00 0x7f55641f9010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:16.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.170+0000 7f5569988640 1 -- 172.21.15.112:0/3674802437 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f55641536e0 msgr2=0x7f55640bf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:16.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.170+0000 7f5569988640 1 --2- 172.21.15.112:0/3674802437 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f55641536e0 0x7f55640bf3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:16.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.170+0000 7f5569988640 1 -- 172.21.15.112:0/3674802437 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f556414f880 con 0x7f55640c25c0 2024-09-21T00:27:16.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.170+0000 7f5568986640 1 --2- 172.21.15.112:0/3674802437 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f55641536e0 0x7f55640bf3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:27:16.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.171+0000 7f5569988640 1 --2- 172.21.15.112:0/3674802437 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55640c25c0 0x7f55641e73c0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f555c009cd0 tx=0x7f555c003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:16.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.171+0000 7f5559ffb640 1 -- 172.21.15.112:0/3674802437 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f555c004040 con 0x7f55640c25c0 2024-09-21T00:27:16.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.171+0000 7f5559ffb640 1 -- 172.21.15.112:0/3674802437 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f555c0042d0 con 0x7f55640c25c0 2024-09-21T00:27:16.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.171+0000 7f556b3eb640 1 -- 172.21.15.112:0/3674802437 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f556414fb60 con 0x7f55640c25c0 2024-09-21T00:27:16.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.171+0000 7f5559ffb640 1 -- 172.21.15.112:0/3674802437 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f555c004720 con 0x7f55640c25c0 2024-09-21T00:27:16.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.171+0000 7f556b3eb640 1 -- 172.21.15.112:0/3674802437 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55641f8af0 con 0x7f55640c25c0 2024-09-21T00:27:16.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.172+0000 7f556b3eb640 1 -- 172.21.15.112:0/3674802437 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55640d04c0 con 0x7f55640c25c0 2024-09-21T00:27:16.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.173+0000 7f5559ffb640 1 -- 172.21.15.112:0/3674802437 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f555c03b410 con 0x7f55640c25c0 2024-09-21T00:27:16.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.173+0000 7f5559ffb640 1 --2- 172.21.15.112:0/3674802437 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f5548078b90 0x7f554807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:16.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.174+0000 7f5559ffb640 1 -- 172.21.15.112:0/3674802437 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 6101+0+0 (secure 0 0 0) 0x7f555c0bc530 con 0x7f55640c25c0 2024-09-21T00:27:16.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.174+0000 7f5569187640 1 --2- 172.21.15.112:0/3674802437 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f5548078b90 0x7f554807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:16.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.174+0000 7f5569187640 1 --2- 172.21.15.112:0/3674802437 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f5548078b90 0x7f554807b0a0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f5560009530 tx=0x7f5560009260 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:16.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.178+0000 7f5559ffb640 1 -- 172.21.15.112:0/3674802437 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f555c048060 con 0x7f55640c25c0 2024-09-21T00:27:16.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.353+0000 7f556b3eb640 1 -- 172.21.15.112:0/3674802437 --> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5564071100 con 0x7f5548078b90 2024-09-21T00:27:16.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.358+0000 7f5559ffb640 1 -- 172.21.15.112:0/3674802437 <== mgr.4103 v2:172.21.15.187:6800/2921522906 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251933 (secure 0 0 0) 0x7f5564071100 con 0x7f5548078b90 2024-09-21T00:27:16.362 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:27:16.363 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2024-09-21T00:27:16.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.361+0000 7f556b3eb640 1 -- 172.21.15.112:0/3674802437 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f5548078b90 msgr2=0x7f554807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:16.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.361+0000 7f556b3eb640 1 --2- 172.21.15.112:0/3674802437 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f5548078b90 0x7f554807b0a0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f5560009530 tx=0x7f5560009260 comp rx=0 tx=0).stop 2024-09-21T00:27:16.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.361+0000 7f556b3eb640 1 -- 172.21.15.112:0/3674802437 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55640c25c0 msgr2=0x7f55641e73c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:16.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.361+0000 7f556b3eb640 1 --2- 172.21.15.112:0/3674802437 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55640c25c0 0x7f55641e73c0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f555c009cd0 tx=0x7f555c003970 comp rx=0 tx=0).stop 2024-09-21T00:27:16.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.361+0000 7f556b3eb640 1 -- 172.21.15.112:0/3674802437 shutdown_connections 2024-09-21T00:27:16.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.361+0000 7f556b3eb640 1 --2- 172.21.15.112:0/3674802437 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f5548078b90 0x7f554807b0a0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:16.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.361+0000 7f556b3eb640 1 --2- 172.21.15.112:0/3674802437 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55640c25c0 0x7f55641e73c0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:16.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.361+0000 7f556b3eb640 1 --2- 172.21.15.112:0/3674802437 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f55641536e0 0x7f55640bf3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:16.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.361+0000 7f556b3eb640 1 --2- 172.21.15.112:0/3674802437 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f55640d1d00 0x7f55641f9010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:16.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.361+0000 7f556b3eb640 1 -- 172.21.15.112:0/3674802437 >> 172.21.15.112:0/3674802437 conn(0x7f55640cef40 msgr2=0x7f5564070100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:16.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.361+0000 7f556b3eb640 1 -- 172.21.15.112:0/3674802437 shutdown_connections 2024-09-21T00:27:16.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.361+0000 7f556b3eb640 1 -- 172.21.15.112:0/3674802437 wait complete. 2024-09-21T00:27:16.374 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":93,"stamp":"2024-09-21T00:27:15.110429+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":184,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":221948,"kb_used_data":5704,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754752772,"statfs":{"total":773094113280,"available":772866838528,"internally_reserved":0,"allocated":5840896,"data_stored":3180748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002037"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T00:27:08.637097+0000","last_change":"2024-09-21T00:27:07.574598+0000","last_active":"2024-09-21T00:27:08.637097+0000","last_peered":"2024-09-21T00:27:08.637097+0000","last_clean":"2024-09-21T00:27:08.637097+0000","last_became_active":"2024-09-21T00:27:07.574242+0000","last_became_peered":"2024-09-21T00:27:07.574242+0000","last_unstale":"2024-09-21T00:27:08.637097+0000","last_undegraded":"2024-09-21T00:27:08.637097+0000","last_fullsized":"2024-09-21T00:27:08.637097+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:14:48.992210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T00:27:08.577256+0000","last_change":"2024-09-21T00:27:07.582509+0000","last_active":"2024-09-21T00:27:08.577256+0000","last_peered":"2024-09-21T00:27:08.577256+0000","last_clean":"2024-09-21T00:27:08.577256+0000","last_became_active":"2024-09-21T00:27:07.582156+0000","last_became_peered":"2024-09-21T00:27:07.582156+0000","last_unstale":"2024-09-21T00:27:08.577256+0000","last_undegraded":"2024-09-21T00:27:08.577256+0000","last_fullsized":"2024-09-21T00:27:08.577256+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:48:23.582479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T00:27:08.565144+0000","last_change":"2024-09-21T00:27:07.585860+0000","last_active":"2024-09-21T00:27:08.565144+0000","last_peered":"2024-09-21T00:27:08.565144+0000","last_clean":"2024-09-21T00:27:08.565144+0000","last_became_active":"2024-09-21T00:27:07.583499+0000","last_became_peered":"2024-09-21T00:27:07.583499+0000","last_unstale":"2024-09-21T00:27:08.565144+0000","last_undegraded":"2024-09-21T00:27:08.565144+0000","last_fullsized":"2024-09-21T00:27:08.565144+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T00:35:47.054430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T00:27:08.576993+0000","last_change":"2024-09-21T00:27:07.591866+0000","last_active":"2024-09-21T00:27:08.576993+0000","last_peered":"2024-09-21T00:27:08.576993+0000","last_clean":"2024-09-21T00:27:08.576993+0000","last_became_active":"2024-09-21T00:27:07.591420+0000","last_became_peered":"2024-09-21T00:27:07.591420+0000","last_unstale":"2024-09-21T00:27:08.576993+0000","last_undegraded":"2024-09-21T00:27:08.576993+0000","last_fullsized":"2024-09-21T00:27:08.576993+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:03:47.068456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T00:27:08.564497+0000","last_change":"2024-09-21T00:27:07.573651+0000","last_active":"2024-09-21T00:27:08.564497+0000","last_peered":"2024-09-21T00:27:08.564497+0000","last_clean":"2024-09-21T00:27:08.564497+0000","last_became_active":"2024-09-21T00:27:07.573348+0000","last_became_peered":"2024-09-21T00:27:07.573348+0000","last_unstale":"2024-09-21T00:27:08.564497+0000","last_undegraded":"2024-09-21T00:27:08.564497+0000","last_fullsized":"2024-09-21T00:27:08.564497+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:25:34.372782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T00:27:08.637217+0000","last_change":"2024-09-21T00:27:07.581689+0000","last_active":"2024-09-21T00:27:08.637217+0000","last_peered":"2024-09-21T00:27:08.637217+0000","last_clean":"2024-09-21T00:27:08.637217+0000","last_became_active":"2024-09-21T00:27:07.581012+0000","last_became_peered":"2024-09-21T00:27:07.581012+0000","last_unstale":"2024-09-21T00:27:08.637217+0000","last_undegraded":"2024-09-21T00:27:08.637217+0000","last_fullsized":"2024-09-21T00:27:08.637217+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:40:30.254996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T00:27:08.564830+0000","last_change":"2024-09-21T00:27:07.582525+0000","last_active":"2024-09-21T00:27:08.564830+0000","last_peered":"2024-09-21T00:27:08.564830+0000","last_clean":"2024-09-21T00:27:08.564830+0000","last_became_active":"2024-09-21T00:27:07.582001+0000","last_became_peered":"2024-09-21T00:27:07.582001+0000","last_unstale":"2024-09-21T00:27:08.564830+0000","last_undegraded":"2024-09-21T00:27:08.564830+0000","last_fullsized":"2024-09-21T00:27:08.564830+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:57:21.971706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T00:27:08.637745+0000","last_change":"2024-09-21T00:27:07.591988+0000","last_active":"2024-09-21T00:27:08.637745+0000","last_peered":"2024-09-21T00:27:08.637745+0000","last_clean":"2024-09-21T00:27:08.637745+0000","last_became_active":"2024-09-21T00:27:07.591664+0000","last_became_peered":"2024-09-21T00:27:07.591664+0000","last_unstale":"2024-09-21T00:27:08.637745+0000","last_undegraded":"2024-09-21T00:27:08.637745+0000","last_fullsized":"2024-09-21T00:27:08.637745+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:10:41.248363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T00:27:08.638343+0000","last_change":"2024-09-21T00:27:07.589913+0000","last_active":"2024-09-21T00:27:08.638343+0000","last_peered":"2024-09-21T00:27:08.638343+0000","last_clean":"2024-09-21T00:27:08.638343+0000","last_became_active":"2024-09-21T00:27:07.589578+0000","last_became_peered":"2024-09-21T00:27:07.589578+0000","last_unstale":"2024-09-21T00:27:08.638343+0000","last_undegraded":"2024-09-21T00:27:08.638343+0000","last_fullsized":"2024-09-21T00:27:08.638343+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:06:35.167244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T00:27:08.637359+0000","last_change":"2024-09-21T00:27:07.567996+0000","last_active":"2024-09-21T00:27:08.637359+0000","last_peered":"2024-09-21T00:27:08.637359+0000","last_clean":"2024-09-21T00:27:08.637359+0000","last_became_active":"2024-09-21T00:27:07.567702+0000","last_became_peered":"2024-09-21T00:27:07.567702+0000","last_unstale":"2024-09-21T00:27:08.637359+0000","last_undegraded":"2024-09-21T00:27:08.637359+0000","last_fullsized":"2024-09-21T00:27:08.637359+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:11:14.015975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T00:27:08.638618+0000","last_change":"2024-09-21T00:27:07.573744+0000","last_active":"2024-09-21T00:27:08.638618+0000","last_peered":"2024-09-21T00:27:08.638618+0000","last_clean":"2024-09-21T00:27:08.638618+0000","last_became_active":"2024-09-21T00:27:07.572830+0000","last_became_peered":"2024-09-21T00:27:07.572830+0000","last_unstale":"2024-09-21T00:27:08.638618+0000","last_undegraded":"2024-09-21T00:27:08.638618+0000","last_fullsized":"2024-09-21T00:27:08.638618+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:48:55.095657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T00:27:08.638262+0000","last_change":"2024-09-21T00:27:07.591038+0000","last_active":"2024-09-21T00:27:08.638262+0000","last_peered":"2024-09-21T00:27:08.638262+0000","last_clean":"2024-09-21T00:27:08.638262+0000","last_became_active":"2024-09-21T00:27:07.590601+0000","last_became_peered":"2024-09-21T00:27:07.590601+0000","last_unstale":"2024-09-21T00:27:08.638262+0000","last_undegraded":"2024-09-21T00:27:08.638262+0000","last_fullsized":"2024-09-21T00:27:08.638262+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:25:43.056707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T00:27:08.637746+0000","last_change":"2024-09-21T00:27:07.567050+0000","last_active":"2024-09-21T00:27:08.637746+0000","last_peered":"2024-09-21T00:27:08.637746+0000","last_clean":"2024-09-21T00:27:08.637746+0000","last_became_active":"2024-09-21T00:27:07.566711+0000","last_became_peered":"2024-09-21T00:27:07.566711+0000","last_unstale":"2024-09-21T00:27:08.637746+0000","last_undegraded":"2024-09-21T00:27:08.637746+0000","last_fullsized":"2024-09-21T00:27:08.637746+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:05:35.844093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T00:27:08.565192+0000","last_change":"2024-09-21T00:27:07.587259+0000","last_active":"2024-09-21T00:27:08.565192+0000","last_peered":"2024-09-21T00:27:08.565192+0000","last_clean":"2024-09-21T00:27:08.565192+0000","last_became_active":"2024-09-21T00:27:07.582617+0000","last_became_peered":"2024-09-21T00:27:07.582617+0000","last_unstale":"2024-09-21T00:27:08.565192+0000","last_undegraded":"2024-09-21T00:27:08.565192+0000","last_fullsized":"2024-09-21T00:27:08.565192+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:16:19.467656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T00:27:08.576881+0000","last_change":"2024-09-21T00:27:07.581457+0000","last_active":"2024-09-21T00:27:08.576881+0000","last_peered":"2024-09-21T00:27:08.576881+0000","last_clean":"2024-09-21T00:27:08.576881+0000","last_became_active":"2024-09-21T00:27:07.581087+0000","last_became_peered":"2024-09-21T00:27:07.581087+0000","last_unstale":"2024-09-21T00:27:08.576881+0000","last_undegraded":"2024-09-21T00:27:08.576881+0000","last_fullsized":"2024-09-21T00:27:08.576881+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:44:37.286933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T00:27:08.636999+0000","last_change":"2024-09-21T00:27:07.567067+0000","last_active":"2024-09-21T00:27:08.636999+0000","last_peered":"2024-09-21T00:27:08.636999+0000","last_clean":"2024-09-21T00:27:08.636999+0000","last_became_active":"2024-09-21T00:27:07.566709+0000","last_became_peered":"2024-09-21T00:27:07.566709+0000","last_unstale":"2024-09-21T00:27:08.636999+0000","last_undegraded":"2024-09-21T00:27:08.636999+0000","last_fullsized":"2024-09-21T00:27:08.636999+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:39:53.701668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T00:27:08.566009+0000","last_change":"2024-09-21T00:27:07.586880+0000","last_active":"2024-09-21T00:27:08.566009+0000","last_peered":"2024-09-21T00:27:08.566009+0000","last_clean":"2024-09-21T00:27:08.566009+0000","last_became_active":"2024-09-21T00:27:07.583356+0000","last_became_peered":"2024-09-21T00:27:07.583356+0000","last_unstale":"2024-09-21T00:27:08.566009+0000","last_undegraded":"2024-09-21T00:27:08.566009+0000","last_fullsized":"2024-09-21T00:27:08.566009+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:19:46.136112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T00:27:08.577299+0000","last_change":"2024-09-21T00:27:07.582718+0000","last_active":"2024-09-21T00:27:08.577299+0000","last_peered":"2024-09-21T00:27:08.577299+0000","last_clean":"2024-09-21T00:27:08.577299+0000","last_became_active":"2024-09-21T00:27:07.582398+0000","last_became_peered":"2024-09-21T00:27:07.582398+0000","last_unstale":"2024-09-21T00:27:08.577299+0000","last_undegraded":"2024-09-21T00:27:08.577299+0000","last_fullsized":"2024-09-21T00:27:08.577299+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:13:20.554203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T00:27:08.565302+0000","last_change":"2024-09-21T00:27:07.583854+0000","last_active":"2024-09-21T00:27:08.565302+0000","last_peered":"2024-09-21T00:27:08.565302+0000","last_clean":"2024-09-21T00:27:08.565302+0000","last_became_active":"2024-09-21T00:27:07.583150+0000","last_became_peered":"2024-09-21T00:27:07.583150+0000","last_unstale":"2024-09-21T00:27:08.565302+0000","last_undegraded":"2024-09-21T00:27:08.565302+0000","last_fullsized":"2024-09-21T00:27:08.565302+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:36:22.531060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T00:27:08.637486+0000","last_change":"2024-09-21T00:27:07.581178+0000","last_active":"2024-09-21T00:27:08.637486+0000","last_peered":"2024-09-21T00:27:08.637486+0000","last_clean":"2024-09-21T00:27:08.637486+0000","last_became_active":"2024-09-21T00:27:07.580531+0000","last_became_peered":"2024-09-21T00:27:07.580531+0000","last_unstale":"2024-09-21T00:27:08.637486+0000","last_undegraded":"2024-09-21T00:27:08.637486+0000","last_fullsized":"2024-09-21T00:27:08.637486+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:55:47.579955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T00:27:08.637481+0000","last_change":"2024-09-21T00:27:07.579423+0000","last_active":"2024-09-21T00:27:08.637481+0000","last_peered":"2024-09-21T00:27:08.637481+0000","last_clean":"2024-09-21T00:27:08.637481+0000","last_became_active":"2024-09-21T00:27:07.579032+0000","last_became_peered":"2024-09-21T00:27:07.579032+0000","last_unstale":"2024-09-21T00:27:08.637481+0000","last_undegraded":"2024-09-21T00:27:08.637481+0000","last_fullsized":"2024-09-21T00:27:08.637481+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:27:32.413929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T00:27:08.637247+0000","last_change":"2024-09-21T00:27:07.590112+0000","last_active":"2024-09-21T00:27:08.637247+0000","last_peered":"2024-09-21T00:27:08.637247+0000","last_clean":"2024-09-21T00:27:08.637247+0000","last_became_active":"2024-09-21T00:27:07.589804+0000","last_became_peered":"2024-09-21T00:27:07.589804+0000","last_unstale":"2024-09-21T00:27:08.637247+0000","last_undegraded":"2024-09-21T00:27:08.637247+0000","last_fullsized":"2024-09-21T00:27:08.637247+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:22:00.948618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T00:27:08.637553+0000","last_change":"2024-09-21T00:27:07.578901+0000","last_active":"2024-09-21T00:27:08.637553+0000","last_peered":"2024-09-21T00:27:08.637553+0000","last_clean":"2024-09-21T00:27:08.637553+0000","last_became_active":"2024-09-21T00:27:07.578577+0000","last_became_peered":"2024-09-21T00:27:07.578577+0000","last_unstale":"2024-09-21T00:27:08.637553+0000","last_undegraded":"2024-09-21T00:27:08.637553+0000","last_fullsized":"2024-09-21T00:27:08.637553+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:57:52.700125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T00:27:08.564773+0000","last_change":"2024-09-21T00:27:07.582234+0000","last_active":"2024-09-21T00:27:08.564773+0000","last_peered":"2024-09-21T00:27:08.564773+0000","last_clean":"2024-09-21T00:27:08.564773+0000","last_became_active":"2024-09-21T00:27:07.581808+0000","last_became_peered":"2024-09-21T00:27:07.581808+0000","last_unstale":"2024-09-21T00:27:08.564773+0000","last_undegraded":"2024-09-21T00:27:08.564773+0000","last_fullsized":"2024-09-21T00:27:08.564773+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:46:56.875442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T00:27:08.566152+0000","last_change":"2024-09-21T00:27:07.588853+0000","last_active":"2024-09-21T00:27:08.566152+0000","last_peered":"2024-09-21T00:27:08.566152+0000","last_clean":"2024-09-21T00:27:08.566152+0000","last_became_active":"2024-09-21T00:27:07.588569+0000","last_became_peered":"2024-09-21T00:27:07.588569+0000","last_unstale":"2024-09-21T00:27:08.566152+0000","last_undegraded":"2024-09-21T00:27:08.566152+0000","last_fullsized":"2024-09-21T00:27:08.566152+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:42:14.978562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T00:27:08.637337+0000","last_change":"2024-09-21T00:27:07.586991+0000","last_active":"2024-09-21T00:27:08.637337+0000","last_peered":"2024-09-21T00:27:08.637337+0000","last_clean":"2024-09-21T00:27:08.637337+0000","last_became_active":"2024-09-21T00:27:07.586360+0000","last_became_peered":"2024-09-21T00:27:07.586360+0000","last_unstale":"2024-09-21T00:27:08.637337+0000","last_undegraded":"2024-09-21T00:27:08.637337+0000","last_fullsized":"2024-09-21T00:27:08.637337+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:36:22.531060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T00:27:08.638784+0000","last_change":"2024-09-21T00:27:07.586170+0000","last_active":"2024-09-21T00:27:08.638784+0000","last_peered":"2024-09-21T00:27:08.638784+0000","last_clean":"2024-09-21T00:27:08.638784+0000","last_became_active":"2024-09-21T00:27:07.585790+0000","last_became_peered":"2024-09-21T00:27:07.585790+0000","last_unstale":"2024-09-21T00:27:08.638784+0000","last_undegraded":"2024-09-21T00:27:08.638784+0000","last_fullsized":"2024-09-21T00:27:08.638784+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:22:09.908391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T00:27:08.637559+0000","last_change":"2024-09-21T00:27:07.578640+0000","last_active":"2024-09-21T00:27:08.637559+0000","last_peered":"2024-09-21T00:27:08.637559+0000","last_clean":"2024-09-21T00:27:08.637559+0000","last_became_active":"2024-09-21T00:27:07.578317+0000","last_became_peered":"2024-09-21T00:27:07.578317+0000","last_unstale":"2024-09-21T00:27:08.637559+0000","last_undegraded":"2024-09-21T00:27:08.637559+0000","last_fullsized":"2024-09-21T00:27:08.637559+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:39:53.701668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T00:27:08.565064+0000","last_change":"2024-09-21T00:27:07.583157+0000","last_active":"2024-09-21T00:27:08.565064+0000","last_peered":"2024-09-21T00:27:08.565064+0000","last_clean":"2024-09-21T00:27:08.565064+0000","last_became_active":"2024-09-21T00:27:07.582846+0000","last_became_peered":"2024-09-21T00:27:07.582846+0000","last_unstale":"2024-09-21T00:27:08.565064+0000","last_undegraded":"2024-09-21T00:27:08.565064+0000","last_fullsized":"2024-09-21T00:27:08.565064+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:13:20.554203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T00:27:08.576995+0000","last_change":"2024-09-21T00:27:07.594804+0000","last_active":"2024-09-21T00:27:08.576995+0000","last_peered":"2024-09-21T00:27:08.576995+0000","last_clean":"2024-09-21T00:27:08.576995+0000","last_became_active":"2024-09-21T00:27:07.594492+0000","last_became_peered":"2024-09-21T00:27:07.594492+0000","last_unstale":"2024-09-21T00:27:08.576995+0000","last_undegraded":"2024-09-21T00:27:08.576995+0000","last_fullsized":"2024-09-21T00:27:08.576995+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:41:09.541961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T00:27:08.564872+0000","last_change":"2024-09-21T00:27:07.583022+0000","last_active":"2024-09-21T00:27:08.564872+0000","last_peered":"2024-09-21T00:27:08.564872+0000","last_clean":"2024-09-21T00:27:08.564872+0000","last_became_active":"2024-09-21T00:27:07.582424+0000","last_became_peered":"2024-09-21T00:27:07.582424+0000","last_unstale":"2024-09-21T00:27:08.564872+0000","last_undegraded":"2024-09-21T00:27:08.564872+0000","last_fullsized":"2024-09-21T00:27:08.564872+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:48:23.582479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T00:27:08.563994+0000","last_change":"2024-09-21T00:27:07.592911+0000","last_active":"2024-09-21T00:27:08.563994+0000","last_peered":"2024-09-21T00:27:08.563994+0000","last_clean":"2024-09-21T00:27:08.563994+0000","last_became_active":"2024-09-21T00:27:07.592095+0000","last_became_peered":"2024-09-21T00:27:07.592095+0000","last_unstale":"2024-09-21T00:27:08.563994+0000","last_undegraded":"2024-09-21T00:27:08.563994+0000","last_fullsized":"2024-09-21T00:27:08.563994+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:03:47.068456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T00:27:08.576671+0000","last_change":"2024-09-21T00:27:07.578763+0000","last_active":"2024-09-21T00:27:08.576671+0000","last_peered":"2024-09-21T00:27:08.576671+0000","last_clean":"2024-09-21T00:27:08.576671+0000","last_became_active":"2024-09-21T00:27:07.578129+0000","last_became_peered":"2024-09-21T00:27:07.578129+0000","last_unstale":"2024-09-21T00:27:08.576671+0000","last_undegraded":"2024-09-21T00:27:08.576671+0000","last_fullsized":"2024-09-21T00:27:08.576671+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:02:26.805443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"48'4","reported_seq":19,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T00:27:08.637656+0000","last_change":"2024-09-21T00:27:06.575497+0000","last_active":"2024-09-21T00:27:08.637656+0000","last_peered":"2024-09-21T00:27:08.637656+0000","last_clean":"2024-09-21T00:27:08.637656+0000","last_became_active":"2024-09-21T00:27:06.574937+0000","last_became_peered":"2024-09-21T00:27:06.574937+0000","last_unstale":"2024-09-21T00:27:08.637656+0000","last_undegraded":"2024-09-21T00:27:08.637656+0000","last_fullsized":"2024-09-21T00:27:08.637656+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:52:06.198578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T00:27:08.565081+0000","last_change":"2024-09-21T00:27:07.575616+0000","last_active":"2024-09-21T00:27:08.565081+0000","last_peered":"2024-09-21T00:27:08.565081+0000","last_clean":"2024-09-21T00:27:08.565081+0000","last_became_active":"2024-09-21T00:27:07.574719+0000","last_became_peered":"2024-09-21T00:27:07.574719+0000","last_unstale":"2024-09-21T00:27:08.565081+0000","last_undegraded":"2024-09-21T00:27:08.565081+0000","last_fullsized":"2024-09-21T00:27:08.565081+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:13:20.554203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"48'1","reported_seq":16,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T00:27:08.637545+0000","last_change":"2024-09-21T00:27:06.578147+0000","last_active":"2024-09-21T00:27:08.637545+0000","last_peered":"2024-09-21T00:27:08.637545+0000","last_clean":"2024-09-21T00:27:08.637545+0000","last_became_active":"2024-09-21T00:27:06.577845+0000","last_became_peered":"2024-09-21T00:27:06.577845+0000","last_unstale":"2024-09-21T00:27:08.637545+0000","last_undegraded":"2024-09-21T00:27:08.637545+0000","last_fullsized":"2024-09-21T00:27:08.637545+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:18:27.263054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T00:27:08.637430+0000","last_change":"2024-09-21T00:27:07.578486+0000","last_active":"2024-09-21T00:27:08.637430+0000","last_peered":"2024-09-21T00:27:08.637430+0000","last_clean":"2024-09-21T00:27:08.637430+0000","last_became_active":"2024-09-21T00:27:07.578145+0000","last_became_peered":"2024-09-21T00:27:07.578145+0000","last_unstale":"2024-09-21T00:27:08.637430+0000","last_undegraded":"2024-09-21T00:27:08.637430+0000","last_fullsized":"2024-09-21T00:27:08.637430+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:52:07.026811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"48'2","reported_seq":17,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T00:27:08.638740+0000","last_change":"2024-09-21T00:27:06.573813+0000","last_active":"2024-09-21T00:27:08.638740+0000","last_peered":"2024-09-21T00:27:08.638740+0000","last_clean":"2024-09-21T00:27:08.638740+0000","last_became_active":"2024-09-21T00:27:06.573532+0000","last_became_peered":"2024-09-21T00:27:06.573532+0000","last_unstale":"2024-09-21T00:27:08.638740+0000","last_undegraded":"2024-09-21T00:27:08.638740+0000","last_fullsized":"2024-09-21T00:27:08.638740+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:31:15.501365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T00:27:08.637694+0000","last_change":"2024-09-21T00:27:07.585496+0000","last_active":"2024-09-21T00:27:08.637694+0000","last_peered":"2024-09-21T00:27:08.637694+0000","last_clean":"2024-09-21T00:27:08.637694+0000","last_became_active":"2024-09-21T00:27:07.585201+0000","last_became_peered":"2024-09-21T00:27:07.585201+0000","last_unstale":"2024-09-21T00:27:08.637694+0000","last_undegraded":"2024-09-21T00:27:08.637694+0000","last_fullsized":"2024-09-21T00:27:08.637694+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:45:43.204726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"48'2","reported_seq":17,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T00:27:08.564849+0000","last_change":"2024-09-21T00:27:06.578736+0000","last_active":"2024-09-21T00:27:08.564849+0000","last_peered":"2024-09-21T00:27:08.564849+0000","last_clean":"2024-09-21T00:27:08.564849+0000","last_became_active":"2024-09-21T00:27:06.578417+0000","last_became_peered":"2024-09-21T00:27:06.578417+0000","last_unstale":"2024-09-21T00:27:08.564849+0000","last_undegraded":"2024-09-21T00:27:08.564849+0000","last_fullsized":"2024-09-21T00:27:08.564849+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:41:01.206761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T00:27:08.564410+0000","last_change":"2024-09-21T00:27:07.584295+0000","last_active":"2024-09-21T00:27:08.564410+0000","last_peered":"2024-09-21T00:27:08.564410+0000","last_clean":"2024-09-21T00:27:08.564410+0000","last_became_active":"2024-09-21T00:27:07.583928+0000","last_became_peered":"2024-09-21T00:27:07.583928+0000","last_unstale":"2024-09-21T00:27:08.564410+0000","last_undegraded":"2024-09-21T00:27:08.564410+0000","last_fullsized":"2024-09-21T00:27:08.564410+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:23:26.411146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"48'1","reported_seq":16,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T00:27:08.565129+0000","last_change":"2024-09-21T00:27:06.583543+0000","last_active":"2024-09-21T00:27:08.565129+0000","last_peered":"2024-09-21T00:27:08.565129+0000","last_clean":"2024-09-21T00:27:08.565129+0000","last_became_active":"2024-09-21T00:27:06.583205+0000","last_became_peered":"2024-09-21T00:27:06.583205+0000","last_unstale":"2024-09-21T00:27:08.565129+0000","last_undegraded":"2024-09-21T00:27:08.565129+0000","last_fullsized":"2024-09-21T00:27:08.565129+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:25:33.367223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T00:27:08.577114+0000","last_change":"2024-09-21T00:27:07.594990+0000","last_active":"2024-09-21T00:27:08.577114+0000","last_peered":"2024-09-21T00:27:08.577114+0000","last_clean":"2024-09-21T00:27:08.577114+0000","last_became_active":"2024-09-21T00:27:07.594469+0000","last_became_peered":"2024-09-21T00:27:07.594469+0000","last_unstale":"2024-09-21T00:27:08.577114+0000","last_undegraded":"2024-09-21T00:27:08.577114+0000","last_fullsized":"2024-09-21T00:27:08.577114+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:46:56.875442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"48'1","reported_seq":16,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T00:27:08.563521+0000","last_change":"2024-09-21T00:27:06.573298+0000","last_active":"2024-09-21T00:27:08.563521+0000","last_peered":"2024-09-21T00:27:08.563521+0000","last_clean":"2024-09-21T00:27:08.563521+0000","last_became_active":"2024-09-21T00:27:06.573004+0000","last_became_peered":"2024-09-21T00:27:06.573004+0000","last_unstale":"2024-09-21T00:27:08.563521+0000","last_undegraded":"2024-09-21T00:27:08.563521+0000","last_fullsized":"2024-09-21T00:27:08.563521+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:47:39.228956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T00:27:08.637806+0000","last_change":"2024-09-21T00:27:07.586677+0000","last_active":"2024-09-21T00:27:08.637806+0000","last_peered":"2024-09-21T00:27:08.637806+0000","last_clean":"2024-09-21T00:27:08.637806+0000","last_became_active":"2024-09-21T00:27:07.586032+0000","last_became_peered":"2024-09-21T00:27:07.586032+0000","last_unstale":"2024-09-21T00:27:08.637806+0000","last_undegraded":"2024-09-21T00:27:08.637806+0000","last_fullsized":"2024-09-21T00:27:08.637806+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:49:27.103642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T00:27:08.637326+0000","last_change":"2024-09-21T00:27:06.578211+0000","last_active":"2024-09-21T00:27:08.637326+0000","last_peered":"2024-09-21T00:27:08.637326+0000","last_clean":"2024-09-21T00:27:08.637326+0000","last_became_active":"2024-09-21T00:27:06.577853+0000","last_became_peered":"2024-09-21T00:27:06.577853+0000","last_unstale":"2024-09-21T00:27:08.637326+0000","last_undegraded":"2024-09-21T00:27:08.637326+0000","last_fullsized":"2024-09-21T00:27:08.637326+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:20:28.122438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T00:27:08.577199+0000","last_change":"2024-09-21T00:27:07.591370+0000","last_active":"2024-09-21T00:27:08.577199+0000","last_peered":"2024-09-21T00:27:08.577199+0000","last_clean":"2024-09-21T00:27:08.577199+0000","last_became_active":"2024-09-21T00:27:07.591020+0000","last_became_peered":"2024-09-21T00:27:07.591020+0000","last_unstale":"2024-09-21T00:27:08.577199+0000","last_undegraded":"2024-09-21T00:27:08.577199+0000","last_fullsized":"2024-09-21T00:27:08.577199+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:57:21.971706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T00:27:08.564671+0000","last_change":"2024-09-21T00:27:06.583093+0000","last_active":"2024-09-21T00:27:08.564671+0000","last_peered":"2024-09-21T00:27:08.564671+0000","last_clean":"2024-09-21T00:27:08.564671+0000","last_became_active":"2024-09-21T00:27:06.582779+0000","last_became_peered":"2024-09-21T00:27:06.582779+0000","last_unstale":"2024-09-21T00:27:08.564671+0000","last_undegraded":"2024-09-21T00:27:08.564671+0000","last_fullsized":"2024-09-21T00:27:08.564671+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:01:05.912259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T00:27:08.565208+0000","last_change":"2024-09-21T00:27:07.593103+0000","last_active":"2024-09-21T00:27:08.565208+0000","last_peered":"2024-09-21T00:27:08.565208+0000","last_clean":"2024-09-21T00:27:08.565208+0000","last_became_active":"2024-09-21T00:27:07.592502+0000","last_became_peered":"2024-09-21T00:27:07.592502+0000","last_unstale":"2024-09-21T00:27:08.565208+0000","last_undegraded":"2024-09-21T00:27:08.565208+0000","last_fullsized":"2024-09-21T00:27:08.565208+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:45:14.250767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T00:27:08.577078+0000","last_change":"2024-09-21T00:27:06.575319+0000","last_active":"2024-09-21T00:27:08.577078+0000","last_peered":"2024-09-21T00:27:08.577078+0000","last_clean":"2024-09-21T00:27:08.577078+0000","last_became_active":"2024-09-21T00:27:06.574976+0000","last_became_peered":"2024-09-21T00:27:06.574976+0000","last_unstale":"2024-09-21T00:27:08.577078+0000","last_undegraded":"2024-09-21T00:27:08.577078+0000","last_fullsized":"2024-09-21T00:27:08.577078+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:00:00.540596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T00:27:08.564825+0000","last_change":"2024-09-21T00:27:07.567037+0000","last_active":"2024-09-21T00:27:08.564825+0000","last_peered":"2024-09-21T00:27:08.564825+0000","last_clean":"2024-09-21T00:27:08.564825+0000","last_became_active":"2024-09-21T00:27:07.566717+0000","last_became_peered":"2024-09-21T00:27:07.566717+0000","last_unstale":"2024-09-21T00:27:08.564825+0000","last_undegraded":"2024-09-21T00:27:08.564825+0000","last_fullsized":"2024-09-21T00:27:08.564825+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:48:23.582479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T00:27:08.563622+0000","last_change":"2024-09-21T00:27:06.582888+0000","last_active":"2024-09-21T00:27:08.563622+0000","last_peered":"2024-09-21T00:27:08.563622+0000","last_clean":"2024-09-21T00:27:08.563622+0000","last_became_active":"2024-09-21T00:27:06.582597+0000","last_became_peered":"2024-09-21T00:27:06.582597+0000","last_unstale":"2024-09-21T00:27:08.563622+0000","last_undegraded":"2024-09-21T00:27:08.563622+0000","last_fullsized":"2024-09-21T00:27:08.563622+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:34:50.617324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T00:27:08.565205+0000","last_change":"2024-09-21T00:27:07.583544+0000","last_active":"2024-09-21T00:27:08.565205+0000","last_peered":"2024-09-21T00:27:08.565205+0000","last_clean":"2024-09-21T00:27:08.565205+0000","last_became_active":"2024-09-21T00:27:07.582860+0000","last_became_peered":"2024-09-21T00:27:07.582860+0000","last_unstale":"2024-09-21T00:27:08.565205+0000","last_undegraded":"2024-09-21T00:27:08.565205+0000","last_fullsized":"2024-09-21T00:27:08.565205+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:22:00.948618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"48'3","reported_seq":18,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T00:27:08.577090+0000","last_change":"2024-09-21T00:27:06.573030+0000","last_active":"2024-09-21T00:27:08.577090+0000","last_peered":"2024-09-21T00:27:08.577090+0000","last_clean":"2024-09-21T00:27:08.577090+0000","last_became_active":"2024-09-21T00:27:06.572710+0000","last_became_peered":"2024-09-21T00:27:06.572710+0000","last_unstale":"2024-09-21T00:27:08.577090+0000","last_undegraded":"2024-09-21T00:27:08.577090+0000","last_fullsized":"2024-09-21T00:27:08.577090+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:29:44.042844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T00:27:08.564805+0000","last_change":"2024-09-21T00:27:07.579692+0000","last_active":"2024-09-21T00:27:08.564805+0000","last_peered":"2024-09-21T00:27:08.564805+0000","last_clean":"2024-09-21T00:27:08.564805+0000","last_became_active":"2024-09-21T00:27:07.579366+0000","last_became_peered":"2024-09-21T00:27:07.579366+0000","last_unstale":"2024-09-21T00:27:08.564805+0000","last_undegraded":"2024-09-21T00:27:08.564805+0000","last_fullsized":"2024-09-21T00:27:08.564805+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:50:28.968562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"48'6","reported_seq":21,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T00:27:08.638162+0000","last_change":"2024-09-21T00:27:06.576063+0000","last_active":"2024-09-21T00:27:08.638162+0000","last_peered":"2024-09-21T00:27:08.638162+0000","last_clean":"2024-09-21T00:27:08.638162+0000","last_became_active":"2024-09-21T00:27:06.574974+0000","last_became_peered":"2024-09-21T00:27:06.574974+0000","last_unstale":"2024-09-21T00:27:08.638162+0000","last_undegraded":"2024-09-21T00:27:08.638162+0000","last_fullsized":"2024-09-21T00:27:08.638162+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:41:02.730078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T00:27:08.564533+0000","last_change":"2024-09-21T00:27:07.573085+0000","last_active":"2024-09-21T00:27:08.564533+0000","last_peered":"2024-09-21T00:27:08.564533+0000","last_clean":"2024-09-21T00:27:08.564533+0000","last_became_active":"2024-09-21T00:27:07.572628+0000","last_became_peered":"2024-09-21T00:27:07.572628+0000","last_unstale":"2024-09-21T00:27:08.564533+0000","last_undegraded":"2024-09-21T00:27:08.564533+0000","last_fullsized":"2024-09-21T00:27:08.564533+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:46:56.875442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"48'2","reported_seq":17,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T00:27:08.565248+0000","last_change":"2024-09-21T00:27:06.581810+0000","last_active":"2024-09-21T00:27:08.565248+0000","last_peered":"2024-09-21T00:27:08.565248+0000","last_clean":"2024-09-21T00:27:08.565248+0000","last_became_active":"2024-09-21T00:27:06.581505+0000","last_became_peered":"2024-09-21T00:27:06.581505+0000","last_unstale":"2024-09-21T00:27:08.565248+0000","last_undegraded":"2024-09-21T00:27:08.565248+0000","last_fullsized":"2024-09-21T00:27:08.565248+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:50:27.963003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T00:27:08.565432+0000","last_change":"2024-09-21T00:27:07.586655+0000","last_active":"2024-09-21T00:27:08.565432+0000","last_peered":"2024-09-21T00:27:08.565432+0000","last_clean":"2024-09-21T00:27:08.565432+0000","last_became_active":"2024-09-21T00:27:07.582641+0000","last_became_peered":"2024-09-21T00:27:07.582641+0000","last_unstale":"2024-09-21T00:27:08.565432+0000","last_undegraded":"2024-09-21T00:27:08.565432+0000","last_fullsized":"2024-09-21T00:27:08.565432+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:18:14.208665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T00:27:08.637481+0000","last_change":"2024-09-21T00:27:06.585272+0000","last_active":"2024-09-21T00:27:08.637481+0000","last_peered":"2024-09-21T00:27:08.637481+0000","last_clean":"2024-09-21T00:27:08.637481+0000","last_became_active":"2024-09-21T00:27:06.584933+0000","last_became_peered":"2024-09-21T00:27:06.584933+0000","last_unstale":"2024-09-21T00:27:08.637481+0000","last_undegraded":"2024-09-21T00:27:08.637481+0000","last_fullsized":"2024-09-21T00:27:08.637481+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:31:18.308257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T00:27:08.637573+0000","last_change":"2024-09-21T00:27:07.587662+0000","last_active":"2024-09-21T00:27:08.637573+0000","last_peered":"2024-09-21T00:27:08.637573+0000","last_clean":"2024-09-21T00:27:08.637573+0000","last_became_active":"2024-09-21T00:27:07.587380+0000","last_became_peered":"2024-09-21T00:27:07.587380+0000","last_unstale":"2024-09-21T00:27:08.637573+0000","last_undegraded":"2024-09-21T00:27:08.637573+0000","last_fullsized":"2024-09-21T00:27:08.637573+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:18:14.208665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T00:27:08.576842+0000","last_change":"2024-09-21T00:27:06.573382+0000","last_active":"2024-09-21T00:27:08.576842+0000","last_peered":"2024-09-21T00:27:08.576842+0000","last_clean":"2024-09-21T00:27:08.576842+0000","last_became_active":"2024-09-21T00:27:06.573070+0000","last_became_peered":"2024-09-21T00:27:06.573070+0000","last_unstale":"2024-09-21T00:27:08.576842+0000","last_undegraded":"2024-09-21T00:27:08.576842+0000","last_fullsized":"2024-09-21T00:27:08.576842+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:50:00.458656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T00:27:08.565927+0000","last_change":"2024-09-21T00:27:07.587253+0000","last_active":"2024-09-21T00:27:08.565927+0000","last_peered":"2024-09-21T00:27:08.565927+0000","last_clean":"2024-09-21T00:27:08.565927+0000","last_became_active":"2024-09-21T00:27:07.583819+0000","last_became_peered":"2024-09-21T00:27:07.583819+0000","last_unstale":"2024-09-21T00:27:08.565927+0000","last_undegraded":"2024-09-21T00:27:08.565927+0000","last_fullsized":"2024-09-21T00:27:08.565927+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:49:27.103642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"48'8","reported_seq":23,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T00:27:08.637890+0000","last_change":"2024-09-21T00:27:06.577075+0000","last_active":"2024-09-21T00:27:08.637890+0000","last_peered":"2024-09-21T00:27:08.637890+0000","last_clean":"2024-09-21T00:27:08.637890+0000","last_became_active":"2024-09-21T00:27:06.575526+0000","last_became_peered":"2024-09-21T00:27:06.575526+0000","last_unstale":"2024-09-21T00:27:08.637890+0000","last_undegraded":"2024-09-21T00:27:08.637890+0000","last_fullsized":"2024-09-21T00:27:08.637890+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:46:04.971531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T00:27:08.638028+0000","last_change":"2024-09-21T00:27:07.590155+0000","last_active":"2024-09-21T00:27:08.638028+0000","last_peered":"2024-09-21T00:27:08.638028+0000","last_clean":"2024-09-21T00:27:08.638028+0000","last_became_active":"2024-09-21T00:27:07.589287+0000","last_became_peered":"2024-09-21T00:27:07.589287+0000","last_unstale":"2024-09-21T00:27:08.638028+0000","last_undegraded":"2024-09-21T00:27:08.638028+0000","last_fullsized":"2024-09-21T00:27:08.638028+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:42:14.978562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"48'1","reported_seq":16,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T00:27:08.566070+0000","last_change":"2024-09-21T00:27:06.583726+0000","last_active":"2024-09-21T00:27:08.566070+0000","last_peered":"2024-09-21T00:27:08.566070+0000","last_clean":"2024-09-21T00:27:08.566070+0000","last_became_active":"2024-09-21T00:27:06.583259+0000","last_became_peered":"2024-09-21T00:27:06.583259+0000","last_unstale":"2024-09-21T00:27:08.566070+0000","last_undegraded":"2024-09-21T00:27:08.566070+0000","last_fullsized":"2024-09-21T00:27:08.566070+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:23:37.798796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T00:27:08.637024+0000","last_change":"2024-09-21T00:27:07.585242+0000","last_active":"2024-09-21T00:27:08.637024+0000","last_peered":"2024-09-21T00:27:08.637024+0000","last_clean":"2024-09-21T00:27:08.637024+0000","last_became_active":"2024-09-21T00:27:07.584922+0000","last_became_peered":"2024-09-21T00:27:07.584922+0000","last_unstale":"2024-09-21T00:27:08.637024+0000","last_undegraded":"2024-09-21T00:27:08.637024+0000","last_fullsized":"2024-09-21T00:27:08.637024+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:23:40.281685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"48'2","reported_seq":17,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T00:27:08.638634+0000","last_change":"2024-09-21T00:27:06.576635+0000","last_active":"2024-09-21T00:27:08.638634+0000","last_peered":"2024-09-21T00:27:08.638634+0000","last_clean":"2024-09-21T00:27:08.638634+0000","last_became_active":"2024-09-21T00:27:06.576333+0000","last_became_peered":"2024-09-21T00:27:06.576333+0000","last_unstale":"2024-09-21T00:27:08.638634+0000","last_undegraded":"2024-09-21T00:27:08.638634+0000","last_fullsized":"2024-09-21T00:27:08.638634+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:06:21.280602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T00:27:08.563994+0000","last_change":"2024-09-21T00:27:07.584301+0000","last_active":"2024-09-21T00:27:08.563994+0000","last_peered":"2024-09-21T00:27:08.563994+0000","last_clean":"2024-09-21T00:27:08.563994+0000","last_became_active":"2024-09-21T00:27:07.583977+0000","last_became_peered":"2024-09-21T00:27:07.583977+0000","last_unstale":"2024-09-21T00:27:08.563994+0000","last_undegraded":"2024-09-21T00:27:08.563994+0000","last_fullsized":"2024-09-21T00:27:08.563994+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:01:06.917817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"48'1","reported_seq":16,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T00:27:08.637674+0000","last_change":"2024-09-21T00:27:06.575768+0000","last_active":"2024-09-21T00:27:08.637674+0000","last_peered":"2024-09-21T00:27:08.637674+0000","last_clean":"2024-09-21T00:27:08.637674+0000","last_became_active":"2024-09-21T00:27:06.575023+0000","last_became_peered":"2024-09-21T00:27:06.575023+0000","last_unstale":"2024-09-21T00:27:08.637674+0000","last_undegraded":"2024-09-21T00:27:08.637674+0000","last_fullsized":"2024-09-21T00:27:08.637674+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:31:15.501365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T00:27:08.637959+0000","last_change":"2024-09-21T00:27:07.569099+0000","last_active":"2024-09-21T00:27:08.637959+0000","last_peered":"2024-09-21T00:27:08.637959+0000","last_clean":"2024-09-21T00:27:08.637959+0000","last_became_active":"2024-09-21T00:27:07.568739+0000","last_became_peered":"2024-09-21T00:27:07.568739+0000","last_unstale":"2024-09-21T00:27:08.637959+0000","last_undegraded":"2024-09-21T00:27:08.637959+0000","last_fullsized":"2024-09-21T00:27:08.637959+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:07:09.795024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T00:27:08.637577+0000","last_change":"2024-09-21T00:27:06.577837+0000","last_active":"2024-09-21T00:27:08.637577+0000","last_peered":"2024-09-21T00:27:08.637577+0000","last_clean":"2024-09-21T00:27:08.637577+0000","last_became_active":"2024-09-21T00:27:06.577492+0000","last_became_peered":"2024-09-21T00:27:06.577492+0000","last_unstale":"2024-09-21T00:27:08.637577+0000","last_undegraded":"2024-09-21T00:27:08.637577+0000","last_fullsized":"2024-09-21T00:27:08.637577+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:59:09.729098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T00:27:08.564771+0000","last_change":"2024-09-21T00:27:07.566533+0000","last_active":"2024-09-21T00:27:08.564771+0000","last_peered":"2024-09-21T00:27:08.564771+0000","last_clean":"2024-09-21T00:27:08.564771+0000","last_became_active":"2024-09-21T00:27:07.565880+0000","last_became_peered":"2024-09-21T00:27:07.565880+0000","last_unstale":"2024-09-21T00:27:08.564771+0000","last_undegraded":"2024-09-21T00:27:08.564771+0000","last_fullsized":"2024-09-21T00:27:08.564771+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:57:21.971706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"48'2","reported_seq":17,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T00:27:08.638130+0000","last_change":"2024-09-21T00:27:06.580120+0000","last_active":"2024-09-21T00:27:08.638130+0000","last_peered":"2024-09-21T00:27:08.638130+0000","last_clean":"2024-09-21T00:27:08.638130+0000","last_became_active":"2024-09-21T00:27:06.579795+0000","last_became_peered":"2024-09-21T00:27:06.579795+0000","last_unstale":"2024-09-21T00:27:08.638130+0000","last_undegraded":"2024-09-21T00:27:08.638130+0000","last_fullsized":"2024-09-21T00:27:08.638130+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:31:18.308257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T00:27:08.637689+0000","last_change":"2024-09-21T00:27:07.580772+0000","last_active":"2024-09-21T00:27:08.637689+0000","last_peered":"2024-09-21T00:27:08.637689+0000","last_clean":"2024-09-21T00:27:08.637689+0000","last_became_active":"2024-09-21T00:27:07.580319+0000","last_became_peered":"2024-09-21T00:27:07.580319+0000","last_unstale":"2024-09-21T00:27:08.637689+0000","last_undegraded":"2024-09-21T00:27:08.637689+0000","last_fullsized":"2024-09-21T00:27:08.637689+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:23:40.281685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T00:27:08.637315+0000","last_change":"2024-09-21T00:27:06.574607+0000","last_active":"2024-09-21T00:27:08.637315+0000","last_peered":"2024-09-21T00:27:08.637315+0000","last_clean":"2024-09-21T00:27:08.637315+0000","last_became_active":"2024-09-21T00:27:06.574282+0000","last_became_peered":"2024-09-21T00:27:06.574282+0000","last_unstale":"2024-09-21T00:27:08.637315+0000","last_undegraded":"2024-09-21T00:27:08.637315+0000","last_fullsized":"2024-09-21T00:27:08.637315+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:23:29.070605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T00:27:08.565465+0000","last_change":"2024-09-21T00:27:07.584715+0000","last_active":"2024-09-21T00:27:08.565465+0000","last_peered":"2024-09-21T00:27:08.565465+0000","last_clean":"2024-09-21T00:27:08.565465+0000","last_became_active":"2024-09-21T00:27:07.583529+0000","last_became_peered":"2024-09-21T00:27:07.583529+0000","last_unstale":"2024-09-21T00:27:08.565465+0000","last_undegraded":"2024-09-21T00:27:08.565465+0000","last_fullsized":"2024-09-21T00:27:08.565465+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:45:43.204726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T00:27:08.564904+0000","last_change":"2024-09-21T00:27:06.578555+0000","last_active":"2024-09-21T00:27:08.564904+0000","last_peered":"2024-09-21T00:27:08.564904+0000","last_clean":"2024-09-21T00:27:08.564904+0000","last_became_active":"2024-09-21T00:27:06.578188+0000","last_became_peered":"2024-09-21T00:27:06.578188+0000","last_unstale":"2024-09-21T00:27:08.564904+0000","last_undegraded":"2024-09-21T00:27:08.564904+0000","last_fullsized":"2024-09-21T00:27:08.564904+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:45:13.245209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T00:27:08.637223+0000","last_change":"2024-09-21T00:27:07.590250+0000","last_active":"2024-09-21T00:27:08.637223+0000","last_peered":"2024-09-21T00:27:08.637223+0000","last_clean":"2024-09-21T00:27:08.637223+0000","last_became_active":"2024-09-21T00:27:07.589930+0000","last_became_peered":"2024-09-21T00:27:07.589930+0000","last_unstale":"2024-09-21T00:27:08.637223+0000","last_undegraded":"2024-09-21T00:27:08.637223+0000","last_fullsized":"2024-09-21T00:27:08.637223+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:19:01.756697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T00:27:08.637193+0000","last_change":"2024-09-21T00:27:06.574158+0000","last_active":"2024-09-21T00:27:08.637193+0000","last_peered":"2024-09-21T00:27:08.637193+0000","last_clean":"2024-09-21T00:27:08.637193+0000","last_became_active":"2024-09-21T00:27:06.573854+0000","last_became_peered":"2024-09-21T00:27:06.573854+0000","last_unstale":"2024-09-21T00:27:08.637193+0000","last_undegraded":"2024-09-21T00:27:08.637193+0000","last_fullsized":"2024-09-21T00:27:08.637193+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:04:47.962346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T00:27:08.564230+0000","last_change":"2024-09-21T00:27:07.592939+0000","last_active":"2024-09-21T00:27:08.564230+0000","last_peered":"2024-09-21T00:27:08.564230+0000","last_clean":"2024-09-21T00:27:08.564230+0000","last_became_active":"2024-09-21T00:27:07.591778+0000","last_became_peered":"2024-09-21T00:27:07.591778+0000","last_unstale":"2024-09-21T00:27:08.564230+0000","last_undegraded":"2024-09-21T00:27:08.564230+0000","last_fullsized":"2024-09-21T00:27:08.564230+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:49:11.815396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"48'2","reported_seq":17,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T00:27:08.637719+0000","last_change":"2024-09-21T00:27:06.584062+0000","last_active":"2024-09-21T00:27:08.637719+0000","last_peered":"2024-09-21T00:27:08.637719+0000","last_clean":"2024-09-21T00:27:08.637719+0000","last_became_active":"2024-09-21T00:27:06.583799+0000","last_became_peered":"2024-09-21T00:27:06.583799+0000","last_unstale":"2024-09-21T00:27:08.637719+0000","last_undegraded":"2024-09-21T00:27:08.637719+0000","last_fullsized":"2024-09-21T00:27:08.637719+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:18:27.263054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T00:27:08.637459+0000","last_change":"2024-09-21T00:27:07.581995+0000","last_active":"2024-09-21T00:27:08.637459+0000","last_peered":"2024-09-21T00:27:08.637459+0000","last_clean":"2024-09-21T00:27:08.637459+0000","last_became_active":"2024-09-21T00:27:07.581400+0000","last_became_peered":"2024-09-21T00:27:07.581400+0000","last_unstale":"2024-09-21T00:27:08.637459+0000","last_undegraded":"2024-09-21T00:27:08.637459+0000","last_fullsized":"2024-09-21T00:27:08.637459+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:21:06.635457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T00:27:08.577183+0000","last_change":"2024-09-21T00:27:06.577031+0000","last_active":"2024-09-21T00:27:08.577183+0000","last_peered":"2024-09-21T00:27:08.577183+0000","last_clean":"2024-09-21T00:27:08.577183+0000","last_became_active":"2024-09-21T00:27:06.576704+0000","last_became_peered":"2024-09-21T00:27:06.576704+0000","last_unstale":"2024-09-21T00:27:08.577183+0000","last_undegraded":"2024-09-21T00:27:08.577183+0000","last_fullsized":"2024-09-21T00:27:08.577183+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:06:57.495939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T00:27:08.637902+0000","last_change":"2024-09-21T00:27:07.584652+0000","last_active":"2024-09-21T00:27:08.637902+0000","last_peered":"2024-09-21T00:27:08.637902+0000","last_clean":"2024-09-21T00:27:08.637902+0000","last_became_active":"2024-09-21T00:27:07.584348+0000","last_became_peered":"2024-09-21T00:27:07.584348+0000","last_unstale":"2024-09-21T00:27:08.637902+0000","last_undegraded":"2024-09-21T00:27:08.637902+0000","last_fullsized":"2024-09-21T00:27:08.637902+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:19:46.136112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T00:27:08.565596+0000","last_change":"2024-09-21T00:27:06.581998+0000","last_active":"2024-09-21T00:27:08.565596+0000","last_peered":"2024-09-21T00:27:08.565596+0000","last_clean":"2024-09-21T00:27:08.565596+0000","last_became_active":"2024-09-21T00:27:06.581649+0000","last_became_peered":"2024-09-21T00:27:06.581649+0000","last_unstale":"2024-09-21T00:27:08.565596+0000","last_undegraded":"2024-09-21T00:27:08.565596+0000","last_fullsized":"2024-09-21T00:27:08.565596+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:23:25.405588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T00:27:08.565531+0000","last_change":"2024-09-21T00:27:07.583035+0000","last_active":"2024-09-21T00:27:08.565531+0000","last_peered":"2024-09-21T00:27:08.565531+0000","last_clean":"2024-09-21T00:27:08.565531+0000","last_became_active":"2024-09-21T00:27:07.582352+0000","last_became_peered":"2024-09-21T00:27:07.582352+0000","last_unstale":"2024-09-21T00:27:08.565531+0000","last_undegraded":"2024-09-21T00:27:08.565531+0000","last_fullsized":"2024-09-21T00:27:08.565531+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:10:41.248363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T00:27:08.637962+0000","last_change":"2024-09-21T00:27:06.573856+0000","last_active":"2024-09-21T00:27:08.637962+0000","last_peered":"2024-09-21T00:27:08.637962+0000","last_clean":"2024-09-21T00:27:08.637962+0000","last_became_active":"2024-09-21T00:27:06.573530+0000","last_became_peered":"2024-09-21T00:27:06.573530+0000","last_unstale":"2024-09-21T00:27:08.637962+0000","last_undegraded":"2024-09-21T00:27:08.637962+0000","last_fullsized":"2024-09-21T00:27:08.637962+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:02:25.799885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T00:27:08.563937+0000","last_change":"2024-09-21T00:27:07.573838+0000","last_active":"2024-09-21T00:27:08.563937+0000","last_peered":"2024-09-21T00:27:08.563937+0000","last_clean":"2024-09-21T00:27:08.563937+0000","last_became_active":"2024-09-21T00:27:07.573314+0000","last_became_peered":"2024-09-21T00:27:07.573314+0000","last_unstale":"2024-09-21T00:27:08.563937+0000","last_undegraded":"2024-09-21T00:27:08.563937+0000","last_fullsized":"2024-09-21T00:27:08.563937+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:44:37.286933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T00:27:08.637437+0000","last_change":"2024-09-21T00:27:06.572714+0000","last_active":"2024-09-21T00:27:08.637437+0000","last_peered":"2024-09-21T00:27:08.637437+0000","last_clean":"2024-09-21T00:27:08.637437+0000","last_became_active":"2024-09-21T00:27:06.571259+0000","last_became_peered":"2024-09-21T00:27:06.571259+0000","last_unstale":"2024-09-21T00:27:08.637437+0000","last_undegraded":"2024-09-21T00:27:08.637437+0000","last_fullsized":"2024-09-21T00:27:08.637437+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:59:52.821153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T00:27:08.565131+0000","last_change":"2024-09-21T00:27:07.584894+0000","last_active":"2024-09-21T00:27:08.565131+0000","last_peered":"2024-09-21T00:27:08.565131+0000","last_clean":"2024-09-21T00:27:08.565131+0000","last_became_active":"2024-09-21T00:27:07.583849+0000","last_became_peered":"2024-09-21T00:27:07.583849+0000","last_unstale":"2024-09-21T00:27:08.565131+0000","last_undegraded":"2024-09-21T00:27:08.565131+0000","last_fullsized":"2024-09-21T00:27:08.565131+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:57:03.990531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T00:27:08.563463+0000","last_change":"2024-09-21T00:27:06.571468+0000","last_active":"2024-09-21T00:27:08.563463+0000","last_peered":"2024-09-21T00:27:08.563463+0000","last_clean":"2024-09-21T00:27:08.563463+0000","last_became_active":"2024-09-21T00:27:06.571086+0000","last_became_peered":"2024-09-21T00:27:06.571086+0000","last_unstale":"2024-09-21T00:27:08.563463+0000","last_undegraded":"2024-09-21T00:27:08.563463+0000","last_fullsized":"2024-09-21T00:27:08.563463+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:21:00.312802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T00:27:08.638407+0000","last_change":"2024-09-21T00:27:07.586630+0000","last_active":"2024-09-21T00:27:08.638407+0000","last_peered":"2024-09-21T00:27:08.638407+0000","last_clean":"2024-09-21T00:27:08.638407+0000","last_became_active":"2024-09-21T00:27:07.586268+0000","last_became_peered":"2024-09-21T00:27:07.586268+0000","last_unstale":"2024-09-21T00:27:08.638407+0000","last_undegraded":"2024-09-21T00:27:08.638407+0000","last_fullsized":"2024-09-21T00:27:08.638407+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:54:50.627538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T00:27:08.564183+0000","last_change":"2024-09-21T00:27:06.573982+0000","last_active":"2024-09-21T00:27:08.564183+0000","last_peered":"2024-09-21T00:27:08.564183+0000","last_clean":"2024-09-21T00:27:08.564183+0000","last_became_active":"2024-09-21T00:27:06.573598+0000","last_became_peered":"2024-09-21T00:27:06.573598+0000","last_unstale":"2024-09-21T00:27:08.564183+0000","last_undegraded":"2024-09-21T00:27:08.564183+0000","last_fullsized":"2024-09-21T00:27:08.564183+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:54:28.275019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T00:27:08.637318+0000","last_change":"2024-09-21T00:27:07.577595+0000","last_active":"2024-09-21T00:27:08.637318+0000","last_peered":"2024-09-21T00:27:08.637318+0000","last_clean":"2024-09-21T00:27:08.637318+0000","last_became_active":"2024-09-21T00:27:07.576649+0000","last_became_peered":"2024-09-21T00:27:07.576649+0000","last_unstale":"2024-09-21T00:27:08.637318+0000","last_undegraded":"2024-09-21T00:27:08.637318+0000","last_fullsized":"2024-09-21T00:27:08.637318+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:06.549347+0000","last_clean_scrub_stamp":"2024-09-21T00:27:06.549347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:20:00.370047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"49'1","reported_seq":16,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T00:27:08.565582+0000","last_change":"2024-09-21T00:27:06.579195+0000","last_active":"2024-09-21T00:27:08.565582+0000","last_peered":"2024-09-21T00:27:08.565582+0000","last_clean":"2024-09-21T00:27:08.565582+0000","last_became_active":"2024-09-21T00:27:06.578886+0000","last_became_peered":"2024-09-21T00:27:06.578886+0000","last_unstale":"2024-09-21T00:27:08.565582+0000","last_undegraded":"2024-09-21T00:27:08.565582+0000","last_fullsized":"2024-09-21T00:27:08.565582+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:05.543789+0000","last_clean_scrub_stamp":"2024-09-21T00:27:05.543789+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:50:00.458656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":159,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T00:27:08.576572+0000","last_change":"2024-09-21T00:25:38.725792+0000","last_active":"2024-09-21T00:27:08.576572+0000","last_peered":"2024-09-21T00:27:08.576572+0000","last_clean":"2024-09-21T00:27:08.576572+0000","last_became_active":"2024-09-21T00:25:38.724668+0000","last_became_peered":"2024-09-21T00:25:38.724668+0000","last_unstale":"2024-09-21T00:27:08.576572+0000","last_undegraded":"2024-09-21T00:27:08.576572+0000","last_fullsized":"2024-09-21T00:27:08.576572+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:37.163216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:37.163216+0000","last_clean_scrub_stamp":"2024-09-21T00:25:37.163216+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:14:36.588660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509462,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28176,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343664,"statfs":{"total":96636764160,"available":96607911936,"internally_reserved":0,"allocated":1175552,"data_stored":839891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509463,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27596,"kb_used_data":568,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344244,"statfs":{"total":96636764160,"available":96608505856,"internally_reserved":0,"allocated":581632,"data_stored":250051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509462,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27592,"kb_used_data":560,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344248,"statfs":{"total":96636764160,"available":96608509952,"internally_reserved":0,"allocated":573440,"data_stored":249399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509461,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27596,"kb_used_data":564,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344244,"statfs":{"total":96636764160,"available":96608505856,"internally_reserved":0,"allocated":577536,"data_stored":249981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509463,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27608,"kb_used_data":576,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344232,"statfs":{"total":96636764160,"available":96608493568,"internally_reserved":0,"allocated":589824,"data_stored":251033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509463,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27600,"kb_used_data":572,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344240,"statfs":{"total":96636764160,"available":96608501760,"internally_reserved":0,"allocated":585728,"data_stored":251011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509463,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28180,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343660,"statfs":{"total":96636764160,"available":96607907840,"internally_reserved":0,"allocated":1175552,"data_stored":839823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574871,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27600,"kb_used_data":568,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344240,"statfs":{"total":96636764160,"available":96608501760,"internally_reserved":0,"allocated":581632,"data_stored":249559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T00:27:16.378 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T00:27:16.378 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-21T00:27:16.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.533+0000 7f64df67f640 1 -- 172.21.15.112:0/3300813762 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64d815c4d0 msgr2=0x7f64d80cd650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:16.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.533+0000 7f64df67f640 1 --2- 172.21.15.112:0/3300813762 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64d815c4d0 0x7f64d80cd650 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f64c8009860 tx=0x7f64c802a5a0 comp rx=0 tx=0).stop 2024-09-21T00:27:16.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.533+0000 7f64df67f640 1 -- 172.21.15.112:0/3300813762 shutdown_connections 2024-09-21T00:27:16.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.533+0000 7f64df67f640 1 --2- 172.21.15.112:0/3300813762 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64d815c4d0 0x7f64d80cd650 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:16.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.533+0000 7f64df67f640 1 --2- 172.21.15.112:0/3300813762 >> v2:172.21.15.187:3300/0 conn(0x7f64d80cf9e0 0x7f64d8155b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:16.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.533+0000 7f64df67f640 1 -- 172.21.15.112:0/3300813762 >> 172.21.15.112:0/3300813762 conn(0x7f64d80d0c00 msgr2=0x7f64d80ccf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:16.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.533+0000 7f64df67f640 1 -- 172.21.15.112:0/3300813762 shutdown_connections 2024-09-21T00:27:16.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.533+0000 7f64df67f640 1 -- 172.21.15.112:0/3300813762 wait complete. 2024-09-21T00:27:16.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.535+0000 7f64df67f640 1 Processor -- start 2024-09-21T00:27:16.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.535+0000 7f64df67f640 1 -- start start 2024-09-21T00:27:16.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.536+0000 7f64df67f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64d80cdb90 0x7f64d8159a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:16.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.536+0000 7f64df67f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f64d80cf9e0 0x7f64d815a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:16.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.536+0000 7f64df67f640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f64d815c4d0 0x7f64d806cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:16.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.536+0000 7f64df67f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f64d815f750 con 0x7f64d80cdb90 2024-09-21T00:27:16.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.536+0000 7f64df67f640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f64d8070ed0 con 0x7f64d80cf9e0 2024-09-21T00:27:16.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.536+0000 7f64df67f640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f64d815f8d0 con 0x7f64d815c4d0 2024-09-21T00:27:16.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.536+0000 7f64dd41b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64d80cdb90 0x7f64d8159a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:16.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.537+0000 7f64dd41b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64d80cdb90 0x7f64d8159a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47624/0 (socket says 172.21.15.112:47624) 2024-09-21T00:27:16.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.537+0000 7f64dd41b640 1 -- 172.21.15.112:0/4026518448 learned_addr learned my addr 172.21.15.112:0/4026518448 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:27:16.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.537+0000 7f64dd41b640 1 -- 172.21.15.112:0/4026518448 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f64d815c4d0 msgr2=0x7f64d806cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:16.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.537+0000 7f64dd41b640 1 --2- 172.21.15.112:0/4026518448 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f64d815c4d0 0x7f64d806cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:16.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.537+0000 7f64dd41b640 1 -- 172.21.15.112:0/4026518448 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f64d80cf9e0 msgr2=0x7f64d815a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:16.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.537+0000 7f64dd41b640 1 --2- 172.21.15.112:0/4026518448 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f64d80cf9e0 0x7f64d815a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:16.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.537+0000 7f64dd41b640 1 -- 172.21.15.112:0/4026518448 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64d8157fc0 con 0x7f64d80cdb90 2024-09-21T00:27:16.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.537+0000 7f64dd41b640 1 --2- 172.21.15.112:0/4026518448 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64d80cdb90 0x7f64d8159a70 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f64d400d6d0 tx=0x7f64d400db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:16.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.537+0000 7f64cdffb640 1 -- 172.21.15.112:0/4026518448 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64d4017070 con 0x7f64d80cdb90 2024-09-21T00:27:16.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.537+0000 7f64cdffb640 1 -- 172.21.15.112:0/4026518448 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f64d4005330 con 0x7f64d80cdb90 2024-09-21T00:27:16.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.537+0000 7f64df67f640 1 -- 172.21.15.112:0/4026518448 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64d815aa40 con 0x7f64d80cdb90 2024-09-21T00:27:16.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.537+0000 7f64cdffb640 1 -- 172.21.15.112:0/4026518448 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64d4009820 con 0x7f64d80cdb90 2024-09-21T00:27:16.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.537+0000 7f64df67f640 1 -- 172.21.15.112:0/4026518448 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64d8158580 con 0x7f64d80cdb90 2024-09-21T00:27:16.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.538+0000 7f64df67f640 1 -- 172.21.15.112:0/4026518448 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64d815fe10 con 0x7f64d80cdb90 2024-09-21T00:27:16.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.539+0000 7f64cdffb640 1 -- 172.21.15.112:0/4026518448 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f64d4009bd0 con 0x7f64d80cdb90 2024-09-21T00:27:16.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.539+0000 7f64cdffb640 1 --2- 172.21.15.112:0/4026518448 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f64bc078bc0 0x7f64bc07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:16.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.539+0000 7f64cdffb640 1 -- 172.21.15.112:0/4026518448 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 6101+0+0 (secure 0 0 0) 0x7f64d409b2a0 con 0x7f64d80cdb90 2024-09-21T00:27:16.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.540+0000 7f64dcc1a640 1 --2- 172.21.15.112:0/4026518448 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f64bc078bc0 0x7f64bc07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:16.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.540+0000 7f64dcc1a640 1 --2- 172.21.15.112:0/4026518448 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f64bc078bc0 0x7f64bc07b0d0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f64c8034000 tx=0x7f64c80046c0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:16.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.544+0000 7f64cdffb640 1 -- 172.21.15.112:0/4026518448 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f64d4063550 con 0x7f64d80cdb90 2024-09-21T00:27:16.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.791+0000 7f64df67f640 1 -- 172.21.15.112:0/4026518448 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f64d815cc50 con 0x7f64d80cdb90 2024-09-21T00:27:16.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.795+0000 7f64cdffb640 1 -- 172.21.15.112:0/4026518448 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v16) ==== 190+0+147 (secure 0 0 0) 0x7f64d409b0c0 con 0x7f64d80cdb90 2024-09-21T00:27:16.797 INFO:teuthology.orchestra.run.smithi112.stdout:[client.0] 2024-09-21T00:27:16.797 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQBPEu5m49NPHBAAl/rcjiE8xYI+XdWwvZ6FOw== 2024-09-21T00:27:16.797 INFO:teuthology.orchestra.run.smithi112.stdout: caps mds = "allow" 2024-09-21T00:27:16.797 INFO:teuthology.orchestra.run.smithi112.stdout: caps mon = "allow r" 2024-09-21T00:27:16.797 INFO:teuthology.orchestra.run.smithi112.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-21T00:27:16.798 INFO:teuthology.orchestra.run.smithi112.stderr:updated caps for client.0 2024-09-21T00:27:16.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.798+0000 7f64df67f640 1 -- 172.21.15.112:0/4026518448 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f64bc078bc0 msgr2=0x7f64bc07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:16.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.798+0000 7f64df67f640 1 --2- 172.21.15.112:0/4026518448 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f64bc078bc0 0x7f64bc07b0d0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f64c8034000 tx=0x7f64c80046c0 comp rx=0 tx=0).stop 2024-09-21T00:27:16.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.798+0000 7f64df67f640 1 -- 172.21.15.112:0/4026518448 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64d80cdb90 msgr2=0x7f64d8159a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:16.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.798+0000 7f64df67f640 1 --2- 172.21.15.112:0/4026518448 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64d80cdb90 0x7f64d8159a70 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f64d400d6d0 tx=0x7f64d400db90 comp rx=0 tx=0).stop 2024-09-21T00:27:16.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.798+0000 7f64df67f640 1 -- 172.21.15.112:0/4026518448 shutdown_connections 2024-09-21T00:27:16.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.798+0000 7f64df67f640 1 --2- 172.21.15.112:0/4026518448 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f64d815c4d0 0x7f64d806cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:16.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.798+0000 7f64df67f640 1 --2- 172.21.15.112:0/4026518448 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f64d80cf9e0 0x7f64d815a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:16.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.798+0000 7f64df67f640 1 --2- 172.21.15.112:0/4026518448 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f64bc078bc0 0x7f64bc07b0d0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:16.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.798+0000 7f64df67f640 1 --2- 172.21.15.112:0/4026518448 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64d80cdb90 0x7f64d8159a70 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:16.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.798+0000 7f64df67f640 1 -- 172.21.15.112:0/4026518448 >> 172.21.15.112:0/4026518448 conn(0x7f64d80d0c00 msgr2=0x7f64d8071050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:16.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.798+0000 7f64df67f640 1 -- 172.21.15.112:0/4026518448 shutdown_connections 2024-09-21T00:27:16.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.798+0000 7f64df67f640 1 -- 172.21.15.112:0/4026518448 wait complete. 2024-09-21T00:27:16.811 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T00:27:16.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.922+0000 7fa7000a9640 1 -- 172.21.15.112:0/3499941936 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa6f815cb10 msgr2=0x7fa6f815cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:16.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.922+0000 7fa7000a9640 1 --2- 172.21.15.112:0/3499941936 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa6f815cb10 0x7fa6f815cf20 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa6f4009830 tx=0x7fa6f402a510 comp rx=0 tx=0).stop 2024-09-21T00:27:16.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.922+0000 7fa7000a9640 1 -- 172.21.15.112:0/3499941936 shutdown_connections 2024-09-21T00:27:16.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.922+0000 7fa7000a9640 1 --2- 172.21.15.112:0/3499941936 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa6f80b6250 0x7fa6f80b6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:16.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.922+0000 7fa7000a9640 1 --2- 172.21.15.112:0/3499941936 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa6f815cb10 0x7fa6f815cf20 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:16.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.922+0000 7fa7000a9640 1 -- 172.21.15.112:0/3499941936 >> 172.21.15.112:0/3499941936 conn(0x7fa6f80d2350 msgr2=0x7fa6f80d2760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:16.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.922+0000 7fa7000a9640 1 -- 172.21.15.112:0/3499941936 shutdown_connections 2024-09-21T00:27:16.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.922+0000 7fa7000a9640 1 -- 172.21.15.112:0/3499941936 wait complete. 2024-09-21T00:27:16.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.924+0000 7fa7000a9640 1 Processor -- start 2024-09-21T00:27:16.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.924+0000 7fa7000a9640 1 -- start start 2024-09-21T00:27:16.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.925+0000 7fa7000a9640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa6f80b6250 0x7fa6f81f9f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:16.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.925+0000 7fa7000a9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6f80d12e0 0x7fa6f8152ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:16.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.925+0000 7fa7000a9640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa6f815cb10 0x7fa6f815b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:16.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.925+0000 7fa7000a9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa6f80b6f30 con 0x7fa6f80d12e0 2024-09-21T00:27:16.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.925+0000 7fa7000a9640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa6f80b6db0 con 0x7fa6f80b6250 2024-09-21T00:27:16.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.925+0000 7fa7000a9640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa6f80b70b0 con 0x7fa6f815cb10 2024-09-21T00:27:16.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.925+0000 7fa6fd644640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6f80d12e0 0x7fa6f8152ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:16.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.925+0000 7fa6fd644640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6f80d12e0 0x7fa6f8152ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47640/0 (socket says 172.21.15.112:47640) 2024-09-21T00:27:16.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.925+0000 7fa6fd644640 1 -- 172.21.15.112:0/4039649818 learned_addr learned my addr 172.21.15.112:0/4039649818 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:27:16.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.925+0000 7fa6fde45640 1 --2- 172.21.15.112:0/4039649818 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa6f80b6250 0x7fa6f81f9f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:16.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.925+0000 7fa6fe646640 1 --2- 172.21.15.112:0/4039649818 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa6f815cb10 0x7fa6f815b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:16.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.926+0000 7fa6fd644640 1 -- 172.21.15.112:0/4039649818 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa6f815cb10 msgr2=0x7fa6f815b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:16.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.926+0000 7fa6fd644640 1 --2- 172.21.15.112:0/4039649818 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa6f815cb10 0x7fa6f815b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:16.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.926+0000 7fa6fd644640 1 -- 172.21.15.112:0/4039649818 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa6f80b6250 msgr2=0x7fa6f81f9f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:16.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.926+0000 7fa6fd644640 1 --2- 172.21.15.112:0/4039649818 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa6f80b6250 0x7fa6f81f9f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:16.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.926+0000 7fa6fd644640 1 -- 172.21.15.112:0/4039649818 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6f806f7d0 con 0x7fa6f80d12e0 2024-09-21T00:27:16.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.926+0000 7fa6fd644640 1 --2- 172.21.15.112:0/4039649818 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6f80d12e0 0x7fa6f8152ac0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fa6e800c3d0 tx=0x7fa6e800c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:16.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.926+0000 7fa6ee7fc640 1 -- 172.21.15.112:0/4039649818 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e8011790 con 0x7fa6f80d12e0 2024-09-21T00:27:16.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.926+0000 7fa6ee7fc640 1 -- 172.21.15.112:0/4039649818 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa6e8017070 con 0x7fa6f80d12e0 2024-09-21T00:27:16.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.926+0000 7fa7000a9640 1 -- 172.21.15.112:0/4039649818 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6f806fab0 con 0x7fa6f80d12e0 2024-09-21T00:27:16.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.926+0000 7fa6ee7fc640 1 -- 172.21.15.112:0/4039649818 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e8011cc0 con 0x7fa6f80d12e0 2024-09-21T00:27:16.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.926+0000 7fa7000a9640 1 -- 172.21.15.112:0/4039649818 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6f80c73c0 con 0x7fa6f80d12e0 2024-09-21T00:27:16.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.927+0000 7fa7000a9640 1 -- 172.21.15.112:0/4039649818 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6f80b6660 con 0x7fa6f80d12e0 2024-09-21T00:27:16.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.928+0000 7fa6ee7fc640 1 -- 172.21.15.112:0/4039649818 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa6e801c460 con 0x7fa6f80d12e0 2024-09-21T00:27:16.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.928+0000 7fa6ee7fc640 1 --2- 172.21.15.112:0/4039649818 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fa6dc078b90 0x7fa6dc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:16.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.928+0000 7fa6ee7fc640 1 -- 172.21.15.112:0/4039649818 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 6101+0+0 (secure 0 0 0) 0x7fa6e809aa60 con 0x7fa6f80d12e0 2024-09-21T00:27:16.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.928+0000 7fa6fde45640 1 --2- 172.21.15.112:0/4039649818 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fa6dc078b90 0x7fa6dc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:16.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.929+0000 7fa6fde45640 1 --2- 172.21.15.112:0/4039649818 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fa6dc078b90 0x7fa6dc07b0a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa6f4034000 tx=0x7fa6f4004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:16.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:16.933+0000 7fa6ee7fc640 1 -- 172.21.15.112:0/4039649818 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa6e8062da0 con 0x7fa6f80d12e0 2024-09-21T00:27:17.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.143+0000 7fa7000a9640 1 -- 172.21.15.112:0/4039649818 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa6f806efe0 con 0x7fa6f80d12e0 2024-09-21T00:27:17.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.144+0000 7fa6ee7fc640 1 -- 172.21.15.112:0/4039649818 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5229 (secure 0 0 0) 0x7fa6e80626a0 con 0x7fa6f80d12e0 2024-09-21T00:27:17.146 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:27:17.146 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":26,"btime":"2024-09-21T00:27:10:580658+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5158,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6836/2010548530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":2010548530},{"type":"v1","addr":"172.21.15.187:6836","nonce":2010548530}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5161,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6837/2290459931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6835","nonce":2290459931},{"type":"v1","addr":"172.21.15.187:6837","nonce":2290459931}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5400,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6834/158782340","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":158782340},{"type":"v1","addr":"172.21.15.112:6834","nonce":158782340}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T00:27:06.929464+0000","modified":"2024-09-21T00:27:09.945226+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5403},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5403":{"gid":5403,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6835/722271164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":722271164},{"type":"v1","addr":"172.21.15.112:6835","nonce":722271164}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5403,"qdb_cluster":[5403]},"id":3}]} 2024-09-21T00:27:17.146 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 26 2024-09-21T00:27:17.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.146+0000 7fa7000a9640 1 -- 172.21.15.112:0/4039649818 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fa6dc078b90 msgr2=0x7fa6dc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:17.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.146+0000 7fa7000a9640 1 --2- 172.21.15.112:0/4039649818 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fa6dc078b90 0x7fa6dc07b0a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa6f4034000 tx=0x7fa6f4004730 comp rx=0 tx=0).stop 2024-09-21T00:27:17.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.146+0000 7fa7000a9640 1 -- 172.21.15.112:0/4039649818 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6f80d12e0 msgr2=0x7fa6f8152ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:17.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.146+0000 7fa7000a9640 1 --2- 172.21.15.112:0/4039649818 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6f80d12e0 0x7fa6f8152ac0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fa6e800c3d0 tx=0x7fa6e800c7d0 comp rx=0 tx=0).stop 2024-09-21T00:27:17.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.146+0000 7fa7000a9640 1 -- 172.21.15.112:0/4039649818 shutdown_connections 2024-09-21T00:27:17.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.146+0000 7fa7000a9640 1 --2- 172.21.15.112:0/4039649818 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa6f815cb10 0x7fa6f815b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:17.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.146+0000 7fa7000a9640 1 --2- 172.21.15.112:0/4039649818 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fa6dc078b90 0x7fa6dc07b0a0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:17.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.146+0000 7fa7000a9640 1 --2- 172.21.15.112:0/4039649818 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6f80d12e0 0x7fa6f8152ac0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:17.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.146+0000 7fa7000a9640 1 --2- 172.21.15.112:0/4039649818 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa6f80b6250 0x7fa6f81f9f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:17.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.146+0000 7fa7000a9640 1 -- 172.21.15.112:0/4039649818 >> 172.21.15.112:0/4039649818 conn(0x7fa6f80d2350 msgr2=0x7fa6f815e190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:17.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.146+0000 7fa7000a9640 1 -- 172.21.15.112:0/4039649818 shutdown_connections 2024-09-21T00:27:17.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.146+0000 7fa7000a9640 1 -- 172.21.15.112:0/4039649818 wait complete. 2024-09-21T00:27:17.562 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T00:27:06.929464+0000', 'modified': '2024-09-21T00:27:09.945226+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5403}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5403': {'gid': 5403, 'name': 'a', 'rank': 0, 'incarnation': 22, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.112:6835/722271164', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6833', 'nonce': 722271164}, {'type': 'v1', 'addr': '172.21.15.112:6835', 'nonce': 722271164}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5403, 'qdb_cluster': [5403]} 2024-09-21T00:27:17.563 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-21T00:27:17.563 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-21T00:27:17.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.680+0000 7fd4b9cf9640 1 -- 172.21.15.112:0/2363226689 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4b40d0170 msgr2=0x7fd4b40d0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:17.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.680+0000 7fd4b9cf9640 1 --2- 172.21.15.112:0/2363226689 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4b40d0170 0x7fd4b40d0be0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fd4a8009820 tx=0x7fd4a802a3f0 comp rx=0 tx=0).stop 2024-09-21T00:27:17.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.680+0000 7fd4b9cf9640 1 -- 172.21.15.112:0/2363226689 shutdown_connections 2024-09-21T00:27:17.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.680+0000 7fd4b9cf9640 1 --2- 172.21.15.112:0/2363226689 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4b40d0170 0x7fd4b40d0be0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:17.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.680+0000 7fd4b9cf9640 1 -- 172.21.15.112:0/2363226689 >> 172.21.15.112:0/2363226689 conn(0x7fd4b409d830 msgr2=0x7fd4b40b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:17.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.680+0000 7fd4b9cf9640 1 -- 172.21.15.112:0/2363226689 shutdown_connections 2024-09-21T00:27:17.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.680+0000 7fd4b9cf9640 1 -- 172.21.15.112:0/2363226689 wait complete. 2024-09-21T00:27:17.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.682+0000 7fd4b9cf9640 1 Processor -- start 2024-09-21T00:27:17.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.682+0000 7fd4b9cf9640 1 -- start start 2024-09-21T00:27:17.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.683+0000 7fd4b9cf9640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd4b40d0170 0x7fd4b40ca890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:17.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.683+0000 7fd4b9cf9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4b40d1120 0x7fd4b40cdf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:17.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.683+0000 7fd4b9cf9640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd4b415ee80 0x7fd4b40b1190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:17.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.683+0000 7fd4b9cf9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd4b40b6910 con 0x7fd4b40d1120 2024-09-21T00:27:17.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.683+0000 7fd4b9cf9640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd4b40b6790 con 0x7fd4b415ee80 2024-09-21T00:27:17.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.683+0000 7fd4b9cf9640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd4b40b6a90 con 0x7fd4b40d0170 2024-09-21T00:27:17.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.683+0000 7fd4b2ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4b40d1120 0x7fd4b40cdf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:17.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.683+0000 7fd4b2ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4b40d1120 0x7fd4b40cdf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47652/0 (socket says 172.21.15.112:47652) 2024-09-21T00:27:17.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.683+0000 7fd4b2ffd640 1 -- 172.21.15.112:0/538021846 learned_addr learned my addr 172.21.15.112:0/538021846 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:27:17.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.683+0000 7fd4b3fff640 1 --2- 172.21.15.112:0/538021846 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd4b415ee80 0x7fd4b40b1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:17.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.683+0000 7fd4b37fe640 1 --2- 172.21.15.112:0/538021846 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd4b40d0170 0x7fd4b40ca890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:17.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.684+0000 7fd4b2ffd640 1 -- 172.21.15.112:0/538021846 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd4b40d0170 msgr2=0x7fd4b40ca890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:17.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.684+0000 7fd4b2ffd640 1 --2- 172.21.15.112:0/538021846 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd4b40d0170 0x7fd4b40ca890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:17.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.684+0000 7fd4b2ffd640 1 -- 172.21.15.112:0/538021846 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd4b415ee80 msgr2=0x7fd4b40b1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:17.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.684+0000 7fd4b2ffd640 1 --2- 172.21.15.112:0/538021846 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd4b415ee80 0x7fd4b40b1190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:17.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.684+0000 7fd4b2ffd640 1 -- 172.21.15.112:0/538021846 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4b406cf80 con 0x7fd4b40d1120 2024-09-21T00:27:17.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.684+0000 7fd4b3fff640 1 --2- 172.21.15.112:0/538021846 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd4b415ee80 0x7fd4b40b1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:27:17.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.684+0000 7fd4b37fe640 1 --2- 172.21.15.112:0/538021846 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd4b40d0170 0x7fd4b40ca890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:27:17.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.684+0000 7fd4b2ffd640 1 --2- 172.21.15.112:0/538021846 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4b40d1120 0x7fd4b40cdf70 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fd4a802a9e0 tx=0x7fd4a8004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:17.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.684+0000 7fd49bfff640 1 -- 172.21.15.112:0/538021846 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4a803f070 con 0x7fd4b40d1120 2024-09-21T00:27:17.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.684+0000 7fd49bfff640 1 -- 172.21.15.112:0/538021846 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd4a8004e50 con 0x7fd4b40d1120 2024-09-21T00:27:17.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.684+0000 7fd4b9cf9640 1 -- 172.21.15.112:0/538021846 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4b406d220 con 0x7fd4b40d1120 2024-09-21T00:27:17.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.684+0000 7fd49bfff640 1 -- 172.21.15.112:0/538021846 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4a8031950 con 0x7fd4b40d1120 2024-09-21T00:27:17.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.684+0000 7fd4b9cf9640 1 -- 172.21.15.112:0/538021846 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4b40b1a70 con 0x7fd4b40d1120 2024-09-21T00:27:17.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.686+0000 7fd49bfff640 1 -- 172.21.15.112:0/538021846 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd4a8042070 con 0x7fd4b40d1120 2024-09-21T00:27:17.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.686+0000 7fd49bfff640 1 --2- 172.21.15.112:0/538021846 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fd4a0078bc0 0x7fd4a007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:17.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.686+0000 7fd49bfff640 1 -- 172.21.15.112:0/538021846 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 6101+0+0 (secure 0 0 0) 0x7fd4a80bbf00 con 0x7fd4b40d1120 2024-09-21T00:27:17.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.686+0000 7fd4b37fe640 1 --2- 172.21.15.112:0/538021846 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fd4a0078bc0 0x7fd4a007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:17.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.687+0000 7fd4b37fe640 1 --2- 172.21.15.112:0/538021846 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fd4a0078bc0 0x7fd4a007b0d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fd4a4006fd0 tx=0x7fd4a4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:17.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.687+0000 7fd4c0c68740 1 Processor -- start 2024-09-21T00:27:17.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.687+0000 7fd4c0c68740 1 -- start start 2024-09-21T00:27:17.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.689+0000 7fd4c0c68740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55720c153270 0x55720c524fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:17.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.689+0000 7fd4c0c68740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55720c12a3b0 0x55720c525500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:17.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.689+0000 7fd4c0c68740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55720c132590 0x55720c542400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:17.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.689+0000 7fd4c0c68740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55720c46d820 con 0x55720c132590 2024-09-21T00:27:17.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.689+0000 7fd4c0c68740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55720c45cc50 con 0x55720c12a3b0 2024-09-21T00:27:17.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.689+0000 7fd4c0c68740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x55720c460a60 con 0x55720c153270 2024-09-21T00:27:17.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.689+0000 7fd4b3fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55720c132590 0x55720c542400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:17.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.689+0000 7fd4b3fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55720c132590 0x55720c542400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47662/0 (socket says 172.21.15.112:47662) 2024-09-21T00:27:17.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.689+0000 7fd4b3fff640 1 -- 172.21.15.112:0/701514544 learned_addr learned my addr 172.21.15.112:0/701514544 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:27:17.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.689+0000 7fd4b37fe640 1 --2- 172.21.15.112:0/701514544 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55720c153270 0x55720c524fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:17.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.689+0000 7fd4b2ffd640 1 --2- 172.21.15.112:0/701514544 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55720c12a3b0 0x55720c525500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:17.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.689+0000 7fd4b3fff640 1 -- 172.21.15.112:0/701514544 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55720c153270 msgr2=0x55720c524fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:27:17.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.689+0000 7fd4b3fff640 1 --2- 172.21.15.112:0/701514544 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55720c153270 0x55720c524fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:17.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.689+0000 7fd4b3fff640 1 -- 172.21.15.112:0/701514544 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55720c12a3b0 msgr2=0x55720c525500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:27:17.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.689+0000 7fd4b3fff640 1 --2- 172.21.15.112:0/701514544 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55720c12a3b0 0x55720c525500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:17.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.689+0000 7fd4b3fff640 1 -- 172.21.15.112:0/701514544 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55720c1be850 con 0x55720c132590 2024-09-21T00:27:17.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.689+0000 7fd4b3fff640 1 --2- 172.21.15.112:0/701514544 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55720c132590 0x55720c542400 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fd4ac001c70 tx=0x7fd4ac002130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a1102139cfced4d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:17.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.689+0000 7fd4997fa640 1 -- 172.21.15.112:0/701514544 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4ac019070 con 0x55720c132590 2024-09-21T00:27:17.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.690+0000 7fd4997fa640 1 -- 172.21.15.112:0/701514544 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd4ac003b80 con 0x55720c132590 2024-09-21T00:27:17.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.690+0000 7fd4997fa640 1 -- 172.21.15.112:0/701514544 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4ac00e860 con 0x55720c132590 2024-09-21T00:27:17.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.690+0000 7fd4c0c68740 1 -- 172.21.15.112:0/701514544 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55720c132590 msgr2=0x55720c542400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:17.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.690+0000 7fd4c0c68740 1 --2- 172.21.15.112:0/701514544 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55720c132590 0x55720c542400 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fd4ac001c70 tx=0x7fd4ac002130 comp rx=0 tx=0).stop 2024-09-21T00:27:17.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.690+0000 7fd4c0c68740 1 -- 172.21.15.112:0/701514544 shutdown_connections 2024-09-21T00:27:17.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.690+0000 7fd4c0c68740 1 --2- 172.21.15.112:0/701514544 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55720c132590 0x55720c542400 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:17.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.690+0000 7fd4c0c68740 1 --2- 172.21.15.112:0/701514544 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55720c12a3b0 0x55720c525500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:17.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.690+0000 7fd4c0c68740 1 --2- 172.21.15.112:0/701514544 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55720c153270 0x55720c524fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:17.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.690+0000 7fd4c0c68740 1 -- 172.21.15.112:0/701514544 >> 172.21.15.112:0/701514544 conn(0x55720c1473c0 msgr2=0x55720c4fb280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:17.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.690+0000 7fd4c0c68740 1 -- 172.21.15.112:0/701514544 shutdown_connections 2024-09-21T00:27:17.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.690+0000 7fd4c0c68740 1 -- 172.21.15.112:0/701514544 wait complete. 2024-09-21T00:27:17.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.691+0000 7fd4c0c68740 1 -- start start 2024-09-21T00:27:17.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.691+0000 7fd4c0c68740 1 Processor -- start 2024-09-21T00:27:17.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.693+0000 7fd4c0c68740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55720c12a3b0 0x55720c536bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:17.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.693+0000 7fd4c0c68740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55720c132590 0x55720c54ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:17.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.693+0000 7fd4c0c68740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55720c153270 0x55720c513500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:17.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.693+0000 7fd4c0c68740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55720c1cd620 con 0x55720c132590 2024-09-21T00:27:17.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.693+0000 7fd4c0c68740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55720c4025f0 con 0x55720c12a3b0 2024-09-21T00:27:17.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.693+0000 7fd4c0c68740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x55720c525a40 con 0x55720c153270 2024-09-21T00:27:17.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.693+0000 7fd4c0c68740 10 client.0 _ll_register_callbacks cb 0x55720c4060a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:27:17.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.693+0000 7fd4b2ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55720c132590 0x55720c54ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:17.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.693+0000 7fd4b2ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55720c132590 0x55720c54ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47670/0 (socket says 172.21.15.112:47670) 2024-09-21T00:27:17.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.693+0000 7fd4b2ffd640 1 -- 172.21.15.112:0/3038118576 learned_addr learned my addr 172.21.15.112:0/3038118576 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:27:17.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.693+0000 7fd4b2ffd640 1 -- 172.21.15.112:0/3038118576 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55720c153270 msgr2=0x55720c513500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:27:17.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.693+0000 7fd4b2ffd640 1 --2- 172.21.15.112:0/3038118576 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55720c153270 0x55720c513500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:17.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.693+0000 7fd4b2ffd640 1 -- 172.21.15.112:0/3038118576 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55720c12a3b0 msgr2=0x55720c536bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:27:17.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.693+0000 7fd4b37fe640 1 --2- 172.21.15.112:0/3038118576 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55720c12a3b0 0x55720c536bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:17.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.693+0000 7fd4b2ffd640 1 --2- 172.21.15.112:0/3038118576 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55720c12a3b0 0x55720c536bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:17.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.693+0000 7fd4b2ffd640 1 -- 172.21.15.112:0/3038118576 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55720c2356b0 con 0x55720c132590 2024-09-21T00:27:17.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.693+0000 7fd4b2ffd640 1 --2- 172.21.15.112:0/3038118576 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55720c132590 0x55720c54ee10 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fd4a8080db0 tx=0x7fd4a8084160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f11447281049d2fc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:17.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.693+0000 7fd4927fc640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T00:27:17.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.693+0000 7fd4927fc640 1 -- 172.21.15.112:0/3038118576 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4a8049070 con 0x55720c132590 2024-09-21T00:27:17.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.693+0000 7fd4927fc640 1 -- 172.21.15.112:0/3038118576 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(49..49 src has 1..49) ==== 6101+0+0 (secure 0 0 0) 0x7fd4a808d040 con 0x55720c132590 2024-09-21T00:27:17.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.693+0000 7fd4c0c68740 1 -- 172.21.15.112:0/3038118576 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd4ac003b80 con 0x55720c132590 2024-09-21T00:27:17.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.694+0000 7fd4927fc640 1 -- 172.21.15.112:0/3038118576 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd4a80a2050 con 0x55720c132590 2024-09-21T00:27:17.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.694+0000 7fd4c0c68740 10 client.5547 fetch_fsmap learned FSMap version 26 2024-09-21T00:27:17.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.694+0000 7fd4c0c68740 1 -- 172.21.15.112:0/3038118576 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) -- 0x55720c538790 con 0x55720c132590 2024-09-21T00:27:17.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.694+0000 7fd4927fc640 1 -- 172.21.15.112:0/3038118576 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd4ac003b80 con 0x55720c132590 2024-09-21T00:27:17.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.694+0000 7fd4927fc640 1 -- 172.21.15.112:0/3038118576 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 26) ==== 3502+0+0 (secure 0 0 0) 0x7fd4a80a5020 con 0x55720c132590 2024-09-21T00:27:17.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.694+0000 7fd4c0c68740 10 client.5547 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T00:27:17.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.694+0000 7fd4c0c68740 10 client.5547 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-21T00:27:17.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.694+0000 7fd4c0c68740 20 client.5547 populate_metadata read hostname 'smithi112' 2024-09-21T00:27:17.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.694+0000 7fd4c0c68740 1 --2- 172.21.15.112:0/3038118576 >> [v2:172.21.15.112:6833/722271164,v1:172.21.15.112:6835/722271164] conn(0x55720c111e40 0x55720c552580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:17.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.694+0000 7fd4c0c68740 4 client.5547 mds_command: new command op to 5403 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T00:27:17.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.694+0000 7fd4c0c68740 1 -- 172.21.15.112:0/3038118576 --> [v2:172.21.15.112:6833/722271164,v1:172.21.15.112:6835/722271164] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55720c46f770 con 0x55720c111e40 2024-09-21T00:27:17.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.694+0000 7fd4b37fe640 1 --2- 172.21.15.112:0/3038118576 >> [v2:172.21.15.112:6833/722271164,v1:172.21.15.112:6835/722271164] conn(0x55720c111e40 0x55720c552580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:17.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.694+0000 7fd4927fc640 10 client.5547 ms_handle_connect on v2:172.21.15.112:6833/722271164 2024-09-21T00:27:17.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.694+0000 7fd4b37fe640 1 --2- 172.21.15.112:0/3038118576 >> [v2:172.21.15.112:6833/722271164,v1:172.21.15.112:6835/722271164] conn(0x55720c111e40 0x55720c552580 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8b68d788f2cfd1d server_cookie=71daf12472b11f15 in_seq=0 out_seq=0 2024-09-21T00:27:17.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.695+0000 7fd4927fc640 1 -- 172.21.15.112:0/3038118576 <== mds.0 v2:172.21.15.112:6833/722271164 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55720c46f770 con 0x55720c111e40 2024-09-21T00:27:17.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.695+0000 7fd4927fc640 10 client.5547 handle_command_reply: tid=0 2024-09-21T00:27:17.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.695+0000 7fd4c0c68740 1 client.5547 shutdown 2024-09-21T00:27:17.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.695+0000 7fd4c0c68740 1 -- 172.21.15.112:0/3038118576 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55720c132590 msgr2=0x55720c54ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:17.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.695+0000 7fd4c0c68740 1 --2- 172.21.15.112:0/3038118576 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55720c132590 0x55720c54ee10 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fd4a8080db0 tx=0x7fd4a8084160 comp rx=0 tx=0).stop 2024-09-21T00:27:17.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.695+0000 7fd4c0c68740 1 -- 172.21.15.112:0/3038118576 shutdown_connections 2024-09-21T00:27:17.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.695+0000 7fd4c0c68740 1 --2- 172.21.15.112:0/3038118576 >> [v2:172.21.15.112:6833/722271164,v1:172.21.15.112:6835/722271164] conn(0x55720c111e40 0x55720c552580 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:17.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.695+0000 7fd4c0c68740 1 --2- 172.21.15.112:0/3038118576 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55720c153270 0x55720c513500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:17.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.695+0000 7fd4c0c68740 1 --2- 172.21.15.112:0/3038118576 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55720c132590 0x55720c54ee10 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:17.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.695+0000 7fd4927fc640 0 client.5547 ms_handle_reset on v2:172.21.15.112:6833/722271164 2024-09-21T00:27:17.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.695+0000 7fd4c0c68740 1 --2- 172.21.15.112:0/3038118576 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55720c12a3b0 0x55720c536bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:17.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.695+0000 7fd4c0c68740 1 -- 172.21.15.112:0/3038118576 >> 172.21.15.112:0/3038118576 conn(0x55720c1473c0 msgr2=0x55720c4f5910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:17.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.695+0000 7fd4c0c68740 1 -- 172.21.15.112:0/3038118576 shutdown_connections 2024-09-21T00:27:17.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.695+0000 7fd4c0c68740 1 -- 172.21.15.112:0/3038118576 wait complete. 2024-09-21T00:27:17.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.695+0000 7fd4c0c68740 20 client.5547 trim_cache size 0 max 16384 2024-09-21T00:27:17.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.714+0000 7fd4c0c68740 1 Processor -- start 2024-09-21T00:27:17.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.714+0000 7fd4c0c68740 1 -- start start 2024-09-21T00:27:17.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.715+0000 7fd4c0c68740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55720c12a3b0 0x55720c523b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:17.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.715+0000 7fd4c0c68740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55720c132590 0x55720c513120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:17.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.715+0000 7fd4c0c68740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55720c153270 0x55720c562900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:17.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.715+0000 7fd4c0c68740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd4b4201710 con 0x55720c132590 2024-09-21T00:27:17.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.715+0000 7fd4c0c68740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd4b414b4b0 con 0x55720c12a3b0 2024-09-21T00:27:17.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.715+0000 7fd4c0c68740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd4b42010c0 con 0x55720c153270 2024-09-21T00:27:17.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.715+0000 7fd4b2ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55720c132590 0x55720c513120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:17.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.715+0000 7fd4b2ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55720c132590 0x55720c513120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47684/0 (socket says 172.21.15.112:47684) 2024-09-21T00:27:17.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.715+0000 7fd4b2ffd640 1 -- 172.21.15.112:0/468966577 learned_addr learned my addr 172.21.15.112:0/468966577 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:27:17.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.715+0000 7fd4b2ffd640 1 -- 172.21.15.112:0/468966577 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55720c153270 msgr2=0x55720c562900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:27:17.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.715+0000 7fd4b3fff640 1 --2- 172.21.15.112:0/468966577 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55720c153270 0x55720c562900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:17.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.715+0000 7fd4b37fe640 1 --2- 172.21.15.112:0/468966577 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55720c12a3b0 0x55720c523b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:17.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.715+0000 7fd4b2ffd640 1 --2- 172.21.15.112:0/468966577 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55720c153270 0x55720c562900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:17.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.715+0000 7fd4b2ffd640 1 -- 172.21.15.112:0/468966577 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55720c12a3b0 msgr2=0x55720c523b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:27:17.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.715+0000 7fd4b2ffd640 1 --2- 172.21.15.112:0/468966577 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55720c12a3b0 0x55720c523b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:17.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.716+0000 7fd4b2ffd640 1 -- 172.21.15.112:0/468966577 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55720c541280 con 0x55720c132590 2024-09-21T00:27:17.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.716+0000 7fd4b2ffd640 1 --2- 172.21.15.112:0/468966577 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55720c132590 0x55720c513120 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fd4a80bc150 tx=0x7fd4a80832b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b64eb31b7869a0a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:17.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.716+0000 7fd498ff9640 1 -- 172.21.15.112:0/468966577 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4a8049040 con 0x55720c132590 2024-09-21T00:27:17.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.716+0000 7fd498ff9640 1 -- 172.21.15.112:0/468966577 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd4a8088620 con 0x55720c132590 2024-09-21T00:27:17.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.716+0000 7fd498ff9640 1 -- 172.21.15.112:0/468966577 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4a8083910 con 0x55720c132590 2024-09-21T00:27:17.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.717+0000 7fd4c0c68740 1 -- 172.21.15.112:0/468966577 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55720c132590 msgr2=0x55720c513120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:17.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.717+0000 7fd4c0c68740 1 --2- 172.21.15.112:0/468966577 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55720c132590 0x55720c513120 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fd4a80bc150 tx=0x7fd4a80832b0 comp rx=0 tx=0).stop 2024-09-21T00:27:17.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.717+0000 7fd4c0c68740 1 -- 172.21.15.112:0/468966577 shutdown_connections 2024-09-21T00:27:17.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.717+0000 7fd4c0c68740 1 --2- 172.21.15.112:0/468966577 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55720c153270 0x55720c562900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:17.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.717+0000 7fd4c0c68740 1 --2- 172.21.15.112:0/468966577 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55720c132590 0x55720c513120 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:17.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.717+0000 7fd4c0c68740 1 --2- 172.21.15.112:0/468966577 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55720c12a3b0 0x55720c523b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:17.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.717+0000 7fd4c0c68740 1 -- 172.21.15.112:0/468966577 >> 172.21.15.112:0/468966577 conn(0x55720c1473c0 msgr2=0x55720c521b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:17.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.717+0000 7fd4c0c68740 1 -- 172.21.15.112:0/468966577 shutdown_connections 2024-09-21T00:27:17.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.717+0000 7fd4c0c68740 1 -- 172.21.15.112:0/468966577 wait complete. 2024-09-21T00:27:17.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.717+0000 7fd4c0c68740 1 -- start start 2024-09-21T00:27:17.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.718+0000 7fd4c0c68740 1 Processor -- start 2024-09-21T00:27:17.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.719+0000 7fd4c0c68740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55720c12a3b0 0x55720c5c3200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:17.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.719+0000 7fd4c0c68740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55720c132590 0x55720c54e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:17.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.719+0000 7fd4c0c68740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55720c153270 0x55720c54e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:17.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.719+0000 7fd4c0c68740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55720c5246c0 con 0x55720c12a3b0 2024-09-21T00:27:17.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.719+0000 7fd4c0c68740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55720c538a50 con 0x55720c153270 2024-09-21T00:27:17.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.719+0000 7fd4c0c68740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x55720c53ae50 con 0x55720c132590 2024-09-21T00:27:17.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.719+0000 7fd4c0c68740 10 client.0 _ll_register_callbacks cb 0x7fd4a807f9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:27:17.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.719+0000 7fd4b37fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55720c12a3b0 0x55720c5c3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:17.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.719+0000 7fd4b37fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55720c12a3b0 0x55720c5c3200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47690/0 (socket says 172.21.15.112:47690) 2024-09-21T00:27:17.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.719+0000 7fd4b37fe640 1 -- 172.21.15.112:0/4020145664 learned_addr learned my addr 172.21.15.112:0/4020145664 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:27:17.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.719+0000 7fd4b37fe640 1 -- 172.21.15.112:0/4020145664 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55720c132590 msgr2=0x55720c54e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:27:17.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.719+0000 7fd4b37fe640 1 --2- 172.21.15.112:0/4020145664 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55720c132590 0x55720c54e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:17.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.719+0000 7fd4b37fe640 1 -- 172.21.15.112:0/4020145664 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55720c153270 msgr2=0x55720c54e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:27:17.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.719+0000 7fd4b37fe640 1 --2- 172.21.15.112:0/4020145664 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55720c153270 0x55720c54e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:17.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.719+0000 7fd4b37fe640 1 -- 172.21.15.112:0/4020145664 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55720c5c3740 con 0x55720c12a3b0 2024-09-21T00:27:17.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.719+0000 7fd4b37fe640 1 --2- 172.21.15.112:0/4020145664 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55720c12a3b0 0x55720c5c3200 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fd4a4009150 tx=0x7fd4a4004680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd7ff2f04051212f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:17.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.719+0000 7fd4927fc640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T00:27:17.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.720+0000 7fd4927fc640 1 -- 172.21.15.112:0/4020145664 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4a402d070 con 0x55720c12a3b0 2024-09-21T00:27:17.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.720+0000 7fd4927fc640 1 -- 172.21.15.112:0/4020145664 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(49..49 src has 1..49) ==== 6101+0+0 (secure 0 0 0) 0x7fd4a400f040 con 0x55720c12a3b0 2024-09-21T00:27:17.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.720+0000 7fd4c0c68740 1 -- 172.21.15.112:0/4020145664 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd4a8088620 con 0x55720c12a3b0 2024-09-21T00:27:17.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.720+0000 7fd4927fc640 1 -- 172.21.15.112:0/4020145664 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd4a4038050 con 0x55720c12a3b0 2024-09-21T00:27:17.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.720+0000 7fd4c0c68740 10 client.5553 fetch_fsmap learned FSMap version 26 2024-09-21T00:27:17.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.720+0000 7fd4c0c68740 1 -- 172.21.15.112:0/4020145664 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) -- 0x55720c5c0760 con 0x55720c12a3b0 2024-09-21T00:27:17.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.720+0000 7fd4927fc640 1 -- 172.21.15.112:0/4020145664 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd4a8088620 con 0x55720c12a3b0 2024-09-21T00:27:17.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.720+0000 7fd4927fc640 1 -- 172.21.15.112:0/4020145664 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 26) ==== 3502+0+0 (secure 0 0 0) 0x7fd4a403b020 con 0x55720c12a3b0 2024-09-21T00:27:17.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.720+0000 7fd4c0c68740 10 client.5553 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T00:27:17.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.720+0000 7fd4c0c68740 10 client.5553 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-21T00:27:17.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.720+0000 7fd4c0c68740 20 client.5553 populate_metadata read hostname 'smithi112' 2024-09-21T00:27:17.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.720+0000 7fd4c0c68740 1 --2- 172.21.15.112:0/4020145664 >> [v2:172.21.15.112:6833/722271164,v1:172.21.15.112:6835/722271164] conn(0x55720c539da0 0x55720c514710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:17.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.720+0000 7fd4c0c68740 4 client.5553 mds_command: new command op to 5403 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T00:27:17.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.720+0000 7fd4c0c68740 1 -- 172.21.15.112:0/4020145664 --> [v2:172.21.15.112:6833/722271164,v1:172.21.15.112:6835/722271164] -- command(tid 0: {"prefix": "status"}) -- 0x55720c46b220 con 0x55720c539da0 2024-09-21T00:27:17.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.720+0000 7fd4b2ffd640 1 --2- 172.21.15.112:0/4020145664 >> [v2:172.21.15.112:6833/722271164,v1:172.21.15.112:6835/722271164] conn(0x55720c539da0 0x55720c514710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:17.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.721+0000 7fd4927fc640 10 client.5553 ms_handle_connect on v2:172.21.15.112:6833/722271164 2024-09-21T00:27:17.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.721+0000 7fd4b2ffd640 1 --2- 172.21.15.112:0/4020145664 >> [v2:172.21.15.112:6833/722271164,v1:172.21.15.112:6835/722271164] conn(0x55720c539da0 0x55720c514710 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5420beb11342ea server_cookie=b58533aba1f47975 in_seq=0 out_seq=0 2024-09-21T00:27:17.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.721+0000 7fd4927fc640 1 -- 172.21.15.112:0/4020145664 <== mds.0 v2:172.21.15.112:6833/722271164 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x55720c46b220 con 0x55720c539da0 2024-09-21T00:27:17.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.721+0000 7fd4927fc640 10 client.5553 handle_command_reply: tid=0 2024-09-21T00:27:17.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.721+0000 7fd4c0c68740 1 client.5553 shutdown 2024-09-21T00:27:17.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.721+0000 7fd4c0c68740 1 -- 172.21.15.112:0/4020145664 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55720c12a3b0 msgr2=0x55720c5c3200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:17.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.721+0000 7fd4c0c68740 1 --2- 172.21.15.112:0/4020145664 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55720c12a3b0 0x55720c5c3200 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fd4a4009150 tx=0x7fd4a4004680 comp rx=0 tx=0).stop 2024-09-21T00:27:17.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.721+0000 7fd4c0c68740 1 -- 172.21.15.112:0/4020145664 shutdown_connections 2024-09-21T00:27:17.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.721+0000 7fd4c0c68740 1 --2- 172.21.15.112:0/4020145664 >> [v2:172.21.15.112:6833/722271164,v1:172.21.15.112:6835/722271164] conn(0x55720c539da0 0x55720c514710 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:17.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.721+0000 7fd4c0c68740 1 --2- 172.21.15.112:0/4020145664 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55720c153270 0x55720c54e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:17.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.721+0000 7fd4c0c68740 1 --2- 172.21.15.112:0/4020145664 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55720c132590 0x55720c54e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:17.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.721+0000 7fd4927fc640 0 client.5553 ms_handle_reset on v2:172.21.15.112:6833/722271164 2024-09-21T00:27:17.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.721+0000 7fd4c0c68740 1 --2- 172.21.15.112:0/4020145664 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55720c12a3b0 0x55720c5c3200 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:17.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.721+0000 7fd4c0c68740 1 -- 172.21.15.112:0/4020145664 >> 172.21.15.112:0/4020145664 conn(0x55720c1473c0 msgr2=0x55720c5219e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:17.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.721+0000 7fd4c0c68740 1 -- 172.21.15.112:0/4020145664 shutdown_connections 2024-09-21T00:27:17.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.721+0000 7fd4c0c68740 1 -- 172.21.15.112:0/4020145664 wait complete. 2024-09-21T00:27:17.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.721+0000 7fd4c0c68740 20 client.5553 trim_cache size 0 max 16384 2024-09-21T00:27:17.728 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-21T00:27:17.728 INFO:teuthology.orchestra.run.smithi112.stdout: "cluster_fsid": "8b34692b-dc65-49b2-a08f-26f707b96ac5", 2024-09-21T00:27:17.728 INFO:teuthology.orchestra.run.smithi112.stdout: "whoami": 0, 2024-09-21T00:27:17.728 INFO:teuthology.orchestra.run.smithi112.stdout: "id": 5403, 2024-09-21T00:27:17.728 INFO:teuthology.orchestra.run.smithi112.stdout: "want_state": "up:active", 2024-09-21T00:27:17.728 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "up:active", 2024-09-21T00:27:17.728 INFO:teuthology.orchestra.run.smithi112.stdout: "fs_name": "cephfs", 2024-09-21T00:27:17.728 INFO:teuthology.orchestra.run.smithi112.stdout: "rank_uptime": 10.154016650999999, 2024-09-21T00:27:17.728 INFO:teuthology.orchestra.run.smithi112.stdout: "mdsmap_epoch": 26, 2024-09-21T00:27:17.728 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch": 49, 2024-09-21T00:27:17.728 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch_barrier": 49, 2024-09-21T00:27:17.729 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 14.833629423 2024-09-21T00:27:17.729 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-21T00:27:17.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.722+0000 7fd4b9cf9640 1 -- 172.21.15.112:0/538021846 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fd4a0078bc0 msgr2=0x7fd4a007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:17.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.722+0000 7fd4b9cf9640 1 --2- 172.21.15.112:0/538021846 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fd4a0078bc0 0x7fd4a007b0d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fd4a4006fd0 tx=0x7fd4a4008040 comp rx=0 tx=0).stop 2024-09-21T00:27:17.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.722+0000 7fd4b9cf9640 1 -- 172.21.15.112:0/538021846 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4b40d1120 msgr2=0x7fd4b40cdf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:17.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.722+0000 7fd4b9cf9640 1 --2- 172.21.15.112:0/538021846 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4b40d1120 0x7fd4b40cdf70 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fd4a802a9e0 tx=0x7fd4a8004a50 comp rx=0 tx=0).stop 2024-09-21T00:27:17.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.722+0000 7fd4b9cf9640 1 -- 172.21.15.112:0/538021846 shutdown_connections 2024-09-21T00:27:17.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.722+0000 7fd4b9cf9640 1 --2- 172.21.15.112:0/538021846 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd4b415ee80 0x7fd4b40b1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:17.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.722+0000 7fd4b9cf9640 1 --2- 172.21.15.112:0/538021846 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fd4a0078bc0 0x7fd4a007b0d0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:17.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.722+0000 7fd4b9cf9640 1 --2- 172.21.15.112:0/538021846 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4b40d1120 0x7fd4b40cdf70 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:17.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.722+0000 7fd4b9cf9640 1 --2- 172.21.15.112:0/538021846 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd4b40d0170 0x7fd4b40ca890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:17.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.722+0000 7fd4b9cf9640 1 -- 172.21.15.112:0/538021846 >> 172.21.15.112:0/538021846 conn(0x7fd4b409d830 msgr2=0x7fd4b4157440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:17.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.722+0000 7fd4b9cf9640 1 -- 172.21.15.112:0/538021846 shutdown_connections 2024-09-21T00:27:17.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.722+0000 7fd4b9cf9640 1 -- 172.21.15.112:0/538021846 wait complete. 2024-09-21T00:27:17.734 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T00:27:17.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.846+0000 7f7458bad640 1 -- 172.21.15.112:0/1681200409 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74540d01f0 msgr2=0x7f745415daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:17.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.846+0000 7f7458bad640 1 --2- 172.21.15.112:0/1681200409 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74540d01f0 0x7f745415daf0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f74541605a0 tx=0x7f744402a560 comp rx=0 tx=0).stop 2024-09-21T00:27:17.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.846+0000 7f7458bad640 1 -- 172.21.15.112:0/1681200409 shutdown_connections 2024-09-21T00:27:17.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.846+0000 7f7458bad640 1 --2- 172.21.15.112:0/1681200409 >> v2:172.21.15.187:3300/0 conn(0x7f74540d0980 0x7f74540d0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:17.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.846+0000 7f7458bad640 1 --2- 172.21.15.112:0/1681200409 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74540d01f0 0x7f745415daf0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:17.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.846+0000 7f7458bad640 1 -- 172.21.15.112:0/1681200409 >> 172.21.15.112:0/1681200409 conn(0x7f7454072990 msgr2=0x7f7454072da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:17.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.846+0000 7f7458bad640 1 -- 172.21.15.112:0/1681200409 shutdown_connections 2024-09-21T00:27:17.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.846+0000 7f7458bad640 1 -- 172.21.15.112:0/1681200409 wait complete. 2024-09-21T00:27:17.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.848+0000 7f7458bad640 1 Processor -- start 2024-09-21T00:27:17.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.848+0000 7f7458bad640 1 -- start start 2024-09-21T00:27:17.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.849+0000 7f7458bad640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74540d01f0 0x7f74541f9990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:17.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.849+0000 7f7458bad640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f74540d0980 0x7f74541f6d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:17.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.849+0000 7f7458bad640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f745415a6f0 0x7f745414d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:17.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.849+0000 7f7458bad640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f74540cd340 con 0x7f74540d01f0 2024-09-21T00:27:17.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.849+0000 7f7458bad640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f74540cd1c0 con 0x7f74540d0980 2024-09-21T00:27:17.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.849+0000 7f7458bad640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f74540cd4c0 con 0x7f745415a6f0 2024-09-21T00:27:17.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.849+0000 7f745259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74540d01f0 0x7f74541f9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:17.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.849+0000 7f745259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74540d01f0 0x7f74541f9990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47708/0 (socket says 172.21.15.112:47708) 2024-09-21T00:27:17.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.849+0000 7f745259c640 1 -- 172.21.15.112:0/822028417 learned_addr learned my addr 172.21.15.112:0/822028417 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:27:17.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.849+0000 7f7451d9b640 1 --2- 172.21.15.112:0/822028417 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f74540d0980 0x7f74541f6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:17.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.849+0000 7f7452d9d640 1 --2- 172.21.15.112:0/822028417 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f745415a6f0 0x7f745414d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:17.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.849+0000 7f745259c640 1 -- 172.21.15.112:0/822028417 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f745415a6f0 msgr2=0x7f745414d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:17.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.849+0000 7f745259c640 1 --2- 172.21.15.112:0/822028417 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f745415a6f0 0x7f745414d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:17.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.849+0000 7f745259c640 1 -- 172.21.15.112:0/822028417 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f74540d0980 msgr2=0x7f74541f6d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:17.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.849+0000 7f745259c640 1 --2- 172.21.15.112:0/822028417 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f74540d0980 0x7f74541f6d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:17.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.849+0000 7f745259c640 1 -- 172.21.15.112:0/822028417 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74541f9ed0 con 0x7f74540d01f0 2024-09-21T00:27:17.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.849+0000 7f7452d9d640 1 --2- 172.21.15.112:0/822028417 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f745415a6f0 0x7f745414d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:27:17.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.850+0000 7f745259c640 1 --2- 172.21.15.112:0/822028417 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74540d01f0 0x7f74541f9990 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f7444034040 tx=0x7f7444003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:17.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.850+0000 7f7442ffd640 1 -- 172.21.15.112:0/822028417 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f744403e030 con 0x7f74540d01f0 2024-09-21T00:27:17.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.850+0000 7f7442ffd640 1 -- 172.21.15.112:0/822028417 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7444003e00 con 0x7f74540d01f0 2024-09-21T00:27:17.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.850+0000 7f7458bad640 1 -- 172.21.15.112:0/822028417 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74541fa3e0 con 0x7f74540d01f0 2024-09-21T00:27:17.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.850+0000 7f7442ffd640 1 -- 172.21.15.112:0/822028417 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7444035040 con 0x7f74540d01f0 2024-09-21T00:27:17.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.850+0000 7f7458bad640 1 -- 172.21.15.112:0/822028417 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74541f6590 con 0x7f74540d01f0 2024-09-21T00:27:17.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.851+0000 7f7458bad640 1 -- 172.21.15.112:0/822028417 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f745415dd60 con 0x7f74540d01f0 2024-09-21T00:27:17.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.852+0000 7f7442ffd640 1 -- 172.21.15.112:0/822028417 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f7444041030 con 0x7f74540d01f0 2024-09-21T00:27:17.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.852+0000 7f7442ffd640 1 --2- 172.21.15.112:0/822028417 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f7434078b90 0x7f743407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:17.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.852+0000 7f7442ffd640 1 -- 172.21.15.112:0/822028417 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 6101+0+0 (secure 0 0 0) 0x7f74440bb710 con 0x7f74540d01f0 2024-09-21T00:27:17.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.853+0000 7f7451d9b640 1 --2- 172.21.15.112:0/822028417 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f7434078b90 0x7f743407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:17.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.853+0000 7f7451d9b640 1 --2- 172.21.15.112:0/822028417 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f7434078b90 0x7f743407b0a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f745415a150 tx=0x7f7448009060 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:17.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:17.857+0000 7f7442ffd640 1 -- 172.21.15.112:0/822028417 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f745415dd60 con 0x7f74540d01f0 2024-09-21T00:27:18.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:18.069+0000 7f7458bad640 1 -- 172.21.15.112:0/822028417 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f74540ce490 con 0x7f74540d01f0 2024-09-21T00:27:18.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:18.069+0000 7f7442ffd640 1 -- 172.21.15.112:0/822028417 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v26) ==== 53+0+73 (secure 0 0 0) 0x7f74540ce490 con 0x7f74540d01f0 2024-09-21T00:27:18.071 INFO:teuthology.orchestra.run.smithi112.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T00:27:18.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:18.071+0000 7f7458bad640 1 -- 172.21.15.112:0/822028417 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f7434078b90 msgr2=0x7f743407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:18.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:18.071+0000 7f7458bad640 1 --2- 172.21.15.112:0/822028417 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f7434078b90 0x7f743407b0a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f745415a150 tx=0x7f7448009060 comp rx=0 tx=0).stop 2024-09-21T00:27:18.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:18.071+0000 7f7458bad640 1 -- 172.21.15.112:0/822028417 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74540d01f0 msgr2=0x7f74541f9990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:18.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:18.071+0000 7f7458bad640 1 --2- 172.21.15.112:0/822028417 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74540d01f0 0x7f74541f9990 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f7444034040 tx=0x7f7444003910 comp rx=0 tx=0).stop 2024-09-21T00:27:18.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:18.071+0000 7f7458bad640 1 -- 172.21.15.112:0/822028417 shutdown_connections 2024-09-21T00:27:18.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:18.071+0000 7f7458bad640 1 --2- 172.21.15.112:0/822028417 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f745415a6f0 0x7f745414d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:18.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:18.071+0000 7f7458bad640 1 --2- 172.21.15.112:0/822028417 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f74540d0980 0x7f74541f6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:18.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:18.071+0000 7f7458bad640 1 --2- 172.21.15.112:0/822028417 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f7434078b90 0x7f743407b0a0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:18.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:18.071+0000 7f7458bad640 1 --2- 172.21.15.112:0/822028417 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74540d01f0 0x7f74541f9990 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:18.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:18.071+0000 7f7458bad640 1 -- 172.21.15.112:0/822028417 >> 172.21.15.112:0/822028417 conn(0x7f7454072990 msgr2=0x7f745415c720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:18.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:18.071+0000 7f7458bad640 1 -- 172.21.15.112:0/822028417 shutdown_connections 2024-09-21T00:27:18.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:18.071+0000 7f7458bad640 1 -- 172.21.15.112:0/822028417 wait complete. 2024-09-21T00:27:18.084 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T00:27:18.084 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T00:27:18.084 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi112.front.sepia.ceph.com 2024-09-21T00:27:18.084 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T00:27:18.084 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T00:27:18.084 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T00:27:18.084 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-21T00:27:18.084 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T00:27:18.085 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T00:27:18.085 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:27:18.085 DEBUG:teuthology.orchestra.run.smithi112:> ip addr 2024-09-21T00:27:18.100 INFO:teuthology.orchestra.run.smithi112.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T00:27:18.100 INFO:teuthology.orchestra.run.smithi112.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T00:27:18.101 INFO:teuthology.orchestra.run.smithi112.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T00:27:18.101 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:27:18.101 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 ::1/128 scope host 2024-09-21T00:27:18.101 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:27:18.101 INFO:teuthology.orchestra.run.smithi112.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T00:27:18.101 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:d8:a5:2c brd ff:ff:ff:ff:ff:ff 2024-09-21T00:27:18.101 INFO:teuthology.orchestra.run.smithi112.stdout: altname enp7s0f0 2024-09-21T00:27:18.101 INFO:teuthology.orchestra.run.smithi112.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T00:27:18.101 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:d8:a5:2d brd ff:ff:ff:ff:ff:ff 2024-09-21T00:27:18.101 INFO:teuthology.orchestra.run.smithi112.stdout: altname enp7s0f1 2024-09-21T00:27:18.101 INFO:teuthology.orchestra.run.smithi112.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T00:27:18.101 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:8f:cb:68 brd ff:ff:ff:ff:ff:ff 2024-09-21T00:27:18.101 INFO:teuthology.orchestra.run.smithi112.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T00:27:18.101 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:8f:cb:69 brd ff:ff:ff:ff:ff:ff 2024-09-21T00:27:18.102 INFO:teuthology.orchestra.run.smithi112.stdout: inet 172.21.15.112/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-21T00:27:18.102 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft 42684sec preferred_lft 42684sec 2024-09-21T00:27:18.102 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::ec4:7aff:fe8f:cb69/64 scope link noprefixroute 2024-09-21T00:27:18.102 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:27:18.102 INFO:teuthology.orchestra.run.smithi112.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T00:27:18.102 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 1a:39:64:e7:fe:d7 brd ff:ff:ff:ff:ff:ff 2024-09-21T00:27:18.102 INFO:teuthology.orchestra.run.smithi112.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T00:27:18.102 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:27:18.102 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::5858:3dff:feaf:1774/64 scope link 2024-09-21T00:27:18.102 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:27:18.102 INFO:teuthology.orchestra.run.smithi112.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T00:27:18.102 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 1a:39:64:e7:fe:d7 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T00:27:18.102 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::1839:64ff:fee7:fed7/64 scope link 2024-09-21T00:27:18.103 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:27:18.103 INFO:teuthology.orchestra.run.smithi112.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T00:27:18.103 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 86:c9:6c:ef:e4:e9 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T00:27:18.103 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::84c9:6cff:feef:e4e9/64 scope link 2024-09-21T00:27:18.103 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:27:18.103 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:27:18.103 DEBUG:teuthology.orchestra.run.smithi112:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T00:27:18.165 INFO:teuthology.orchestra.run.smithi112.stdout:1 2024-09-21T00:27:18.166 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:27:18.166 DEBUG:teuthology.orchestra.run.smithi112:> route 2024-09-21T00:27:18.222 INFO:teuthology.orchestra.run.smithi112.stdout:Kernel IP routing table 2024-09-21T00:27:18.222 INFO:teuthology.orchestra.run.smithi112.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T00:27:18.222 INFO:teuthology.orchestra.run.smithi112.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-21T00:27:18.222 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-21T00:27:18.222 INFO:teuthology.orchestra.run.smithi112.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T00:27:18.223 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T00:27:18.223 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2024-09-21T00:27:18.223 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-21T00:27:18.223 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-21T00:27:18.223 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-21T00:27:18.223 DEBUG:teuthology.orchestra.run.smithi112:> ') 2024-09-21T00:27:18.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:18 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T00:27:18.342 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T00:27:18.348 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:27:18.348 DEBUG:teuthology.orchestra.run.smithi112:> ip netns list 2024-09-21T00:27:18.364 INFO:teuthology.orchestra.run.smithi112.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-21T00:27:18.364 INFO:teuthology.orchestra.run.smithi112.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T00:27:18.364 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T00:27:18.364 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:27:18.364 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link set brx.0 up 2024-09-21T00:27:18.429 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T00:27:18.429 DEBUG:teuthology.orchestra.run.smithi112:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T00:27:18.485 INFO:teuthology.orchestra.run.smithi112.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T00:27:18.485 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T00:27:18.485 DEBUG:teuthology.orchestra.run.smithi112:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T00:27:18.540 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-21T00:27:18.541 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:27:18.541 DEBUG:teuthology.orchestra.run.smithi112:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T00:27:18.634 INFO:teuthology.orchestra.run.smithi112.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T00:27:18.634 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T00:27:18.634 INFO:teuthology.orchestra.run.smithi112.stdout:invalid new device string format 2024-09-21T00:27:18.645 INFO:teuthology.orchestra.run.smithi112.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T00:27:18.645 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T00:27:18.645 INFO:teuthology.orchestra.run.smithi112.stdout:invalid new device string format 2024-09-21T00:27:18.645 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: resolved to: "172.21.15.112:3300,172.21.15.187:3300,172.21.15.187:3301" 2024-09-21T00:27:18.645 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T00:27:18.651 INFO:teuthology.orchestra.run.smithi112.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T00:27:18.651 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T00:27:18.651 INFO:teuthology.orchestra.run.smithi112.stdout:invalid new device string format 2024-09-21T00:27:18.651 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: resolved to: "172.21.15.112:3300,172.21.15.187:3300,172.21.15.187:3301" 2024-09-21T00:27:18.651 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T00:27:18.651 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: resolved to: "172.21.15.112:6789,172.21.15.187:6789,172.21.15.187:6790" 2024-09-21T00:27:18.651 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.112:6789,172.21.15.187:6789,172.21.15.187:6790:/ 2024-09-21T00:27:18.651 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=8b34692b-dc65-49b2-a08f-26f707b96ac5" will pass to kernel 2024-09-21T00:27:18.653 INFO:teuthology.orchestra.run.smithi112.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-21T00:27:18.653 INFO:teuthology.orchestra.run.smithi112.stdout: You just mounted a file system that supports labels which does not 2024-09-21T00:27:18.653 INFO:teuthology.orchestra.run.smithi112.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T00:27:18.653 INFO:teuthology.orchestra.run.smithi112.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T00:27:18.653 INFO:teuthology.orchestra.run.smithi112.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T00:27:18.654 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T00:27:18.654 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:27:18.654 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T00:27:18.679 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T00:27:18.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:18 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T00:27:18.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T00:27:18.779 INFO:teuthology.orchestra.run.smithi112.stdout:client5090 2024-09-21T00:27:18.779 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T00:27:18.779 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:27:18.795 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T00:27:18.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:18.946+0000 7fdc90856640 1 -- 172.21.15.112:0/1921178486 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc8c0d12e0 msgr2=0x7fdc8c0d16f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:18.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:18.946+0000 7fdc90856640 1 --2- 172.21.15.112:0/1921178486 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc8c0d12e0 0x7fdc8c0d16f0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c074a40 tx=0x7fdc7402a560 comp rx=0 tx=0).stop 2024-09-21T00:27:18.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:18.946+0000 7fdc90856640 1 -- 172.21.15.112:0/1921178486 shutdown_connections 2024-09-21T00:27:18.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:18.946+0000 7fdc90856640 1 --2- 172.21.15.112:0/1921178486 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdc8c160100 0x7fdc8c160510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:18.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:18.946+0000 7fdc90856640 1 --2- 172.21.15.112:0/1921178486 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc8c0d12e0 0x7fdc8c0d16f0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:18.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:18.946+0000 7fdc90856640 1 --2- 172.21.15.112:0/1921178486 >> v2:172.21.15.187:3300/0 conn(0x7fdc8c15cb10 0x7fdc8c15cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:18.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:18.946+0000 7fdc90856640 1 -- 172.21.15.112:0/1921178486 >> 172.21.15.112:0/1921178486 conn(0x7fdc8c0d2350 msgr2=0x7fdc8c0d2760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:18.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:18.946+0000 7fdc90856640 1 -- 172.21.15.112:0/1921178486 shutdown_connections 2024-09-21T00:27:18.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:18.946+0000 7fdc90856640 1 -- 172.21.15.112:0/1921178486 wait complete. 2024-09-21T00:27:18.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:18.948+0000 7fdc90856640 1 Processor -- start 2024-09-21T00:27:18.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:18.948+0000 7fdc90856640 1 -- start start 2024-09-21T00:27:18.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:18.949+0000 7fdc90856640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc8c0d12e0 0x7fdc8c141ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:18.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:18.949+0000 7fdc90856640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdc8c15cb10 0x7fdc8c142420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:18.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:18.949+0000 7fdc90856640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdc8c160100 0x7fdc8c1e88d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:18.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:18.949+0000 7fdc90856640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fdc8c160bd0 con 0x7fdc8c0d12e0 2024-09-21T00:27:18.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:18.949+0000 7fdc90856640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fdc8c160a50 con 0x7fdc8c15cb10 2024-09-21T00:27:18.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:18.949+0000 7fdc90856640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fdc8c160d50 con 0x7fdc8c160100 2024-09-21T00:27:18.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:18.949+0000 7fdc8a59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc8c0d12e0 0x7fdc8c141ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:18.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:18.949+0000 7fdc8a59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc8c0d12e0 0x7fdc8c141ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47722/0 (socket says 172.21.15.112:47722) 2024-09-21T00:27:18.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:18.949+0000 7fdc8a59c640 1 -- 172.21.15.112:0/3428538377 learned_addr learned my addr 172.21.15.112:0/3428538377 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:27:18.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:18.949+0000 7fdc81d9b640 1 --2- 172.21.15.112:0/3428538377 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdc8c15cb10 0x7fdc8c142420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:18.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:18.949+0000 7fdc8ad9d640 1 --2- 172.21.15.112:0/3428538377 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdc8c160100 0x7fdc8c1e88d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:18.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:18.950+0000 7fdc8a59c640 1 -- 172.21.15.112:0/3428538377 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdc8c160100 msgr2=0x7fdc8c1e88d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:18.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:18.950+0000 7fdc8a59c640 1 --2- 172.21.15.112:0/3428538377 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdc8c160100 0x7fdc8c1e88d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:18.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:18.950+0000 7fdc8a59c640 1 -- 172.21.15.112:0/3428538377 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdc8c15cb10 msgr2=0x7fdc8c142420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:18.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:18.950+0000 7fdc8a59c640 1 --2- 172.21.15.112:0/3428538377 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdc8c15cb10 0x7fdc8c142420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:18.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:18.950+0000 7fdc8a59c640 1 -- 172.21.15.112:0/3428538377 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc8c13fd50 con 0x7fdc8c0d12e0 2024-09-21T00:27:18.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:18.950+0000 7fdc8a59c640 1 --2- 172.21.15.112:0/3428538377 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc8c0d12e0 0x7fdc8c141ee0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fdc7c00c940 tx=0x7fdc7c00ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:18.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:18.950+0000 7fdc837fe640 1 -- 172.21.15.112:0/3428538377 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc7c004220 con 0x7fdc8c0d12e0 2024-09-21T00:27:18.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:18.950+0000 7fdc837fe640 1 -- 172.21.15.112:0/3428538377 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdc7c0079b0 con 0x7fdc8c0d12e0 2024-09-21T00:27:18.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:18.950+0000 7fdc90856640 1 -- 172.21.15.112:0/3428538377 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc8c140030 con 0x7fdc8c0d12e0 2024-09-21T00:27:18.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:18.950+0000 7fdc837fe640 1 -- 172.21.15.112:0/3428538377 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc7c007e20 con 0x7fdc8c0d12e0 2024-09-21T00:27:18.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:18.950+0000 7fdc90856640 1 -- 172.21.15.112:0/3428538377 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc8c1405f0 con 0x7fdc8c0d12e0 2024-09-21T00:27:18.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:18.951+0000 7fdc90856640 1 -- 172.21.15.112:0/3428538377 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc8c15cf20 con 0x7fdc8c0d12e0 2024-09-21T00:27:18.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:18.952+0000 7fdc837fe640 1 -- 172.21.15.112:0/3428538377 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fdc7c005020 con 0x7fdc8c0d12e0 2024-09-21T00:27:18.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:18.952+0000 7fdc837fe640 1 --2- 172.21.15.112:0/3428538377 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fdc6c078bc0 0x7fdc6c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:18.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:18.952+0000 7fdc837fe640 1 -- 172.21.15.112:0/3428538377 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 6101+0+0 (secure 0 0 0) 0x7fdc7c09b210 con 0x7fdc8c0d12e0 2024-09-21T00:27:18.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:18.953+0000 7fdc81d9b640 1 --2- 172.21.15.112:0/3428538377 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fdc6c078bc0 0x7fdc6c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:18.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:18.953+0000 7fdc81d9b640 1 --2- 172.21.15.112:0/3428538377 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fdc6c078bc0 0x7fdc6c07b0d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fdc74033000 tx=0x7fdc74004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:18.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:18.957+0000 7fdc837fe640 1 -- 172.21.15.112:0/3428538377 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fdc7c014030 con 0x7fdc8c0d12e0 2024-09-21T00:27:19.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:19.171+0000 7fdc90856640 1 -- 172.21.15.112:0/3428538377 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fdc8c0b56b0 con 0x7fdc8c0d12e0 2024-09-21T00:27:19.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:19.172+0000 7fdc837fe640 1 -- 172.21.15.112:0/3428538377 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v26) ==== 53+0+73 (secure 0 0 0) 0x7fdc7c062e70 con 0x7fdc8c0d12e0 2024-09-21T00:27:19.174 INFO:teuthology.orchestra.run.smithi112.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T00:27:19.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:19.174+0000 7fdc90856640 1 -- 172.21.15.112:0/3428538377 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fdc6c078bc0 msgr2=0x7fdc6c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:19.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:19.174+0000 7fdc90856640 1 --2- 172.21.15.112:0/3428538377 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fdc6c078bc0 0x7fdc6c07b0d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fdc74033000 tx=0x7fdc74004730 comp rx=0 tx=0).stop 2024-09-21T00:27:19.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:19.174+0000 7fdc90856640 1 -- 172.21.15.112:0/3428538377 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc8c0d12e0 msgr2=0x7fdc8c141ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:19.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:19.174+0000 7fdc90856640 1 --2- 172.21.15.112:0/3428538377 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc8c0d12e0 0x7fdc8c141ee0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fdc7c00c940 tx=0x7fdc7c00ce00 comp rx=0 tx=0).stop 2024-09-21T00:27:19.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:19.174+0000 7fdc90856640 1 -- 172.21.15.112:0/3428538377 shutdown_connections 2024-09-21T00:27:19.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:19.174+0000 7fdc90856640 1 --2- 172.21.15.112:0/3428538377 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdc8c160100 0x7fdc8c1e88d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:19.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:19.174+0000 7fdc90856640 1 --2- 172.21.15.112:0/3428538377 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdc8c15cb10 0x7fdc8c142420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:19.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:19.174+0000 7fdc90856640 1 --2- 172.21.15.112:0/3428538377 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fdc6c078bc0 0x7fdc6c07b0d0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:19.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:19.174+0000 7fdc90856640 1 --2- 172.21.15.112:0/3428538377 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc8c0d12e0 0x7fdc8c141ee0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:19.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:19.174+0000 7fdc90856640 1 -- 172.21.15.112:0/3428538377 >> 172.21.15.112:0/3428538377 conn(0x7fdc8c0d2350 msgr2=0x7fdc8c070a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:19.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:19.174+0000 7fdc90856640 1 -- 172.21.15.112:0/3428538377 shutdown_connections 2024-09-21T00:27:19.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:19.174+0000 7fdc90856640 1 -- 172.21.15.112:0/3428538377 wait complete. 2024-09-21T00:27:19.187 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T00:27:19.187 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T00:27:19.187 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi187.front.sepia.ceph.com 2024-09-21T00:27:19.187 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-21T00:27:19.187 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-21T00:27:19.187 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T00:27:19.188 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-21T00:27:19.188 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T00:27:19.188 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-21T00:27:19.188 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:27:19.188 DEBUG:teuthology.orchestra.run.smithi187:> ip addr 2024-09-21T00:27:19.207 INFO:teuthology.orchestra.run.smithi187.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T00:27:19.207 INFO:teuthology.orchestra.run.smithi187.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T00:27:19.207 INFO:teuthology.orchestra.run.smithi187.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T00:27:19.207 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:27:19.207 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 ::1/128 scope host 2024-09-21T00:27:19.207 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:27:19.207 INFO:teuthology.orchestra.run.smithi187.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T00:27:19.208 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 0c:c4:7a:d9:36:78 brd ff:ff:ff:ff:ff:ff 2024-09-21T00:27:19.208 INFO:teuthology.orchestra.run.smithi187.stdout: altname enp7s0f0 2024-09-21T00:27:19.208 INFO:teuthology.orchestra.run.smithi187.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T00:27:19.208 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 0c:c4:7a:d9:36:79 brd ff:ff:ff:ff:ff:ff 2024-09-21T00:27:19.208 INFO:teuthology.orchestra.run.smithi187.stdout: altname enp7s0f1 2024-09-21T00:27:19.208 INFO:teuthology.orchestra.run.smithi187.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T00:27:19.208 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 0c:c4:7a:88:77:b0 brd ff:ff:ff:ff:ff:ff 2024-09-21T00:27:19.208 INFO:teuthology.orchestra.run.smithi187.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T00:27:19.208 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 0c:c4:7a:88:77:b1 brd ff:ff:ff:ff:ff:ff 2024-09-21T00:27:19.208 INFO:teuthology.orchestra.run.smithi187.stdout: inet 172.21.15.187/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-21T00:27:19.208 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft 42672sec preferred_lft 42672sec 2024-09-21T00:27:19.208 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 fe80::ec4:7aff:fe88:77b1/64 scope link noprefixroute 2024-09-21T00:27:19.208 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:27:19.208 INFO:teuthology.orchestra.run.smithi187.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T00:27:19.209 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 86:3b:f6:5b:1b:32 brd ff:ff:ff:ff:ff:ff 2024-09-21T00:27:19.209 INFO:teuthology.orchestra.run.smithi187.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T00:27:19.209 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:27:19.209 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 fe80::ac6f:97ff:fe09:b82e/64 scope link 2024-09-21T00:27:19.209 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:27:19.209 INFO:teuthology.orchestra.run.smithi187.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T00:27:19.209 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether ca:0b:a0:f5:2b:b5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T00:27:19.209 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 fe80::c80b:a0ff:fef5:2bb5/64 scope link 2024-09-21T00:27:19.209 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:27:19.209 INFO:teuthology.orchestra.run.smithi187.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T00:27:19.210 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 86:3b:f6:5b:1b:32 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-21T00:27:19.210 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 fe80::843b:f6ff:fe5b:1b32/64 scope link 2024-09-21T00:27:19.210 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:27:19.210 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:27:19.210 DEBUG:teuthology.orchestra.run.smithi187:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T00:27:19.274 INFO:teuthology.orchestra.run.smithi187.stdout:1 2024-09-21T00:27:19.276 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:27:19.276 DEBUG:teuthology.orchestra.run.smithi187:> route 2024-09-21T00:27:19.335 INFO:teuthology.orchestra.run.smithi187.stdout:Kernel IP routing table 2024-09-21T00:27:19.335 INFO:teuthology.orchestra.run.smithi187.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T00:27:19.335 INFO:teuthology.orchestra.run.smithi187.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-21T00:27:19.335 INFO:teuthology.orchestra.run.smithi187.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-21T00:27:19.335 INFO:teuthology.orchestra.run.smithi187.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T00:27:19.336 DEBUG:teuthology.orchestra.run.smithi187:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T00:27:19.336 DEBUG:teuthology.orchestra.run.smithi187:> set -e 2024-09-21T00:27:19.336 DEBUG:teuthology.orchestra.run.smithi187:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-21T00:27:19.336 DEBUG:teuthology.orchestra.run.smithi187:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-21T00:27:19.336 DEBUG:teuthology.orchestra.run.smithi187:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-21T00:27:19.336 DEBUG:teuthology.orchestra.run.smithi187:> ') 2024-09-21T00:27:19.427 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:27:19 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T00:27:19.460 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:27:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T00:27:19.467 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:27:19.467 DEBUG:teuthology.orchestra.run.smithi187:> ip netns list 2024-09-21T00:27:19.483 INFO:teuthology.orchestra.run.smithi187.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-21T00:27:19.483 INFO:teuthology.orchestra.run.smithi187.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-21T00:27:19.483 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-21T00:27:19.484 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:27:19.484 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip link set brx.0 up 2024-09-21T00:27:19.547 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T00:27:19.547 DEBUG:teuthology.orchestra.run.smithi187:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-21T00:27:19.603 INFO:teuthology.orchestra.run.smithi187.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-21T00:27:19.603 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T00:27:19.603 DEBUG:teuthology.orchestra.run.smithi187:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-21T00:27:19.657 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-21T00:27:19.658 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:27:19.658 DEBUG:teuthology.orchestra.run.smithi187:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-21T00:27:19.752 INFO:teuthology.orchestra.run.smithi187.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-21T00:27:19.752 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-21T00:27:19.752 INFO:teuthology.orchestra.run.smithi187.stdout:invalid new device string format 2024-09-21T00:27:19.762 INFO:teuthology.orchestra.run.smithi187.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-21T00:27:19.762 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-21T00:27:19.763 INFO:teuthology.orchestra.run.smithi187.stdout:invalid new device string format 2024-09-21T00:27:19.763 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: resolved to: "172.21.15.112:3300,172.21.15.187:3300,172.21.15.187:3301" 2024-09-21T00:27:19.763 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T00:27:19.769 INFO:teuthology.orchestra.run.smithi187.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-21T00:27:19.769 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-21T00:27:19.769 INFO:teuthology.orchestra.run.smithi187.stdout:invalid new device string format 2024-09-21T00:27:19.770 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: resolved to: "172.21.15.112:3300,172.21.15.187:3300,172.21.15.187:3301" 2024-09-21T00:27:19.770 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T00:27:19.770 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: resolved to: "172.21.15.112:6789,172.21.15.187:6789,172.21.15.187:6790" 2024-09-21T00:27:19.770 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.112:6789,172.21.15.187:6789,172.21.15.187:6790:/ 2024-09-21T00:27:19.770 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=8b34692b-dc65-49b2-a08f-26f707b96ac5" will pass to kernel 2024-09-21T00:27:19.770 INFO:teuthology.orchestra.run.smithi187.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-09-21T00:27:19.770 INFO:teuthology.orchestra.run.smithi187.stdout: You just mounted a file system that supports labels which does not 2024-09-21T00:27:19.771 INFO:teuthology.orchestra.run.smithi187.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T00:27:19.771 INFO:teuthology.orchestra.run.smithi187.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T00:27:19.771 INFO:teuthology.orchestra.run.smithi187.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T00:27:19.772 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T00:27:19.772 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:27:19.772 DEBUG:teuthology.orchestra.run.smithi187:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-21T00:27:19.797 DEBUG:teuthology.orchestra.run.smithi187:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T00:27:19.886 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:27:19 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T00:27:19.888 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:27:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T00:27:19.894 INFO:teuthology.orchestra.run.smithi187.stdout:client5568 2024-09-21T00:27:19.894 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-09-21T00:27:19.894 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:27:19.910 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T00:27:19.910 DEBUG:teuthology.orchestra.run.smithi112:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T00:27:19.956 DEBUG:teuthology.orchestra.run.smithi187:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T00:27:19.999 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T00:27:19.999 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T00:27:19.999 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T00:27:19.999 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:27:20.016 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T00:27:20.016 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:27:20.075 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T00:27:20.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:20.235+0000 7fac5b8f2640 1 -- 172.21.15.112:0/2990061125 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac5406ff00 msgr2=0x7fac54070310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:20.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:20.235+0000 7fac5b8f2640 1 --2- 172.21.15.112:0/2990061125 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac5406ff00 0x7fac54070310 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fac54075340 tx=0x7fac4c02a980 comp rx=0 tx=0).stop 2024-09-21T00:27:20.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:20.235+0000 7fac5b8f2640 1 -- 172.21.15.112:0/2990061125 shutdown_connections 2024-09-21T00:27:20.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:20.235+0000 7fac5b8f2640 1 --2- 172.21.15.112:0/2990061125 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac5406ff00 0x7fac54070310 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:20.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:20.235+0000 7fac5b8f2640 1 --2- 172.21.15.112:0/2990061125 >> v2:172.21.15.187:3300/0 conn(0x7fac540b2ca0 0x7fac540b30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:20.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:20.235+0000 7fac5b8f2640 1 -- 172.21.15.112:0/2990061125 >> 172.21.15.112:0/2990061125 conn(0x7fac540cdca0 msgr2=0x7fac540ce0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:20.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:20.235+0000 7fac5b8f2640 1 -- 172.21.15.112:0/2990061125 shutdown_connections 2024-09-21T00:27:20.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:20.235+0000 7fac5b8f2640 1 -- 172.21.15.112:0/2990061125 wait complete. 2024-09-21T00:27:20.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:20.237+0000 7fac5b8f2640 1 Processor -- start 2024-09-21T00:27:20.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:20.237+0000 7fac5b8f2640 1 -- start start 2024-09-21T00:27:20.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:20.239+0000 7fac5b8f2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac5406ff00 0x7fac5409ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:20.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:20.239+0000 7fac5b8f2640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fac540b2ca0 0x7fac541f41a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:20.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:20.239+0000 7fac5b8f2640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fac5415c750 0x7fac5409c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:20.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:20.239+0000 7fac5b8f2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fac540af200 con 0x7fac5406ff00 2024-09-21T00:27:20.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:20.239+0000 7fac5b8f2640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fac540af080 con 0x7fac5415c750 2024-09-21T00:27:20.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:20.239+0000 7fac5b8f2640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fac54070850 con 0x7fac540b2ca0 2024-09-21T00:27:20.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:20.239+0000 7fac5968e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac5406ff00 0x7fac5409ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:20.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:20.239+0000 7fac5968e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac5406ff00 0x7fac5409ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47750/0 (socket says 172.21.15.112:47750) 2024-09-21T00:27:20.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:20.239+0000 7fac5968e640 1 -- 172.21.15.112:0/3567981981 learned_addr learned my addr 172.21.15.112:0/3567981981 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:27:20.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:20.239+0000 7fac59e8f640 1 --2- 172.21.15.112:0/3567981981 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fac5415c750 0x7fac5409c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:20.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:20.239+0000 7fac58e8d640 1 --2- 172.21.15.112:0/3567981981 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fac540b2ca0 0x7fac541f41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:20.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:20.239+0000 7fac5968e640 1 -- 172.21.15.112:0/3567981981 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fac540b2ca0 msgr2=0x7fac541f41a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:20.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:20.239+0000 7fac5968e640 1 --2- 172.21.15.112:0/3567981981 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fac540b2ca0 0x7fac541f41a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:20.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:20.239+0000 7fac5968e640 1 -- 172.21.15.112:0/3567981981 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fac5415c750 msgr2=0x7fac5409c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:20.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:20.239+0000 7fac5968e640 1 --2- 172.21.15.112:0/3567981981 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fac5415c750 0x7fac5409c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:20.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:20.239+0000 7fac5968e640 1 -- 172.21.15.112:0/3567981981 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac541e50e0 con 0x7fac5406ff00 2024-09-21T00:27:20.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:20.239+0000 7fac58e8d640 1 --2- 172.21.15.112:0/3567981981 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fac540b2ca0 0x7fac541f41a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:27:20.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:20.239+0000 7fac5968e640 1 --2- 172.21.15.112:0/3567981981 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac5406ff00 0x7fac5409ecd0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fac5000b420 tx=0x7fac5000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:20.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:20.239+0000 7fac49ffb640 1 -- 172.21.15.112:0/3567981981 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac50011830 con 0x7fac5406ff00 2024-09-21T00:27:20.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:20.239+0000 7fac49ffb640 1 -- 172.21.15.112:0/3567981981 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fac50016070 con 0x7fac5406ff00 2024-09-21T00:27:20.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:20.239+0000 7fac5b8f2640 1 -- 172.21.15.112:0/3567981981 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac5409dc30 con 0x7fac5406ff00 2024-09-21T00:27:20.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:20.239+0000 7fac49ffb640 1 -- 172.21.15.112:0/3567981981 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac50011d60 con 0x7fac5406ff00 2024-09-21T00:27:20.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:20.240+0000 7fac5b8f2640 1 -- 172.21.15.112:0/3567981981 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac5409e1f0 con 0x7fac5406ff00 2024-09-21T00:27:20.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:20.240+0000 7fac5b8f2640 1 -- 172.21.15.112:0/3567981981 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac54070d70 con 0x7fac5406ff00 2024-09-21T00:27:20.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:20.241+0000 7fac49ffb640 1 -- 172.21.15.112:0/3567981981 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fac5001b460 con 0x7fac5406ff00 2024-09-21T00:27:20.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:20.242+0000 7fac49ffb640 1 --2- 172.21.15.112:0/3567981981 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fac38078b90 0x7fac3807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:20.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:20.242+0000 7fac49ffb640 1 -- 172.21.15.112:0/3567981981 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 6101+0+0 (secure 0 0 0) 0x7fac5009a850 con 0x7fac5406ff00 2024-09-21T00:27:20.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:20.242+0000 7fac58e8d640 1 --2- 172.21.15.112:0/3567981981 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fac38078b90 0x7fac3807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:20.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:20.243+0000 7fac58e8d640 1 --2- 172.21.15.112:0/3567981981 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fac38078b90 0x7fac3807b0a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fac541f6c70 tx=0x7fac44005f10 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:20.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:20.246+0000 7fac49ffb640 1 -- 172.21.15.112:0/3567981981 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fac50020020 con 0x7fac5406ff00 2024-09-21T00:27:20.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:20.434+0000 7fac5b8f2640 1 -- 172.21.15.112:0/3567981981 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fac540b38d0 con 0x7fac5406ff00 2024-09-21T00:27:20.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:20.435+0000 7fac49ffb640 1 -- 172.21.15.112:0/3567981981 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v49) ==== 80+0+0 (secure 0 0 0) 0x7fac541e50e0 con 0x7fac5406ff00 2024-09-21T00:27:20.437 INFO:teuthology.orchestra.run.smithi112.stderr:listed 0 entries 2024-09-21T00:27:20.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:20.437+0000 7fac5b8f2640 1 -- 172.21.15.112:0/3567981981 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fac38078b90 msgr2=0x7fac3807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:20.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:20.437+0000 7fac5b8f2640 1 --2- 172.21.15.112:0/3567981981 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fac38078b90 0x7fac3807b0a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fac541f6c70 tx=0x7fac44005f10 comp rx=0 tx=0).stop 2024-09-21T00:27:20.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:20.437+0000 7fac5b8f2640 1 -- 172.21.15.112:0/3567981981 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac5406ff00 msgr2=0x7fac5409ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:20.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:20.437+0000 7fac5b8f2640 1 --2- 172.21.15.112:0/3567981981 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac5406ff00 0x7fac5409ecd0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fac5000b420 tx=0x7fac5000b8e0 comp rx=0 tx=0).stop 2024-09-21T00:27:20.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:20.437+0000 7fac5b8f2640 1 -- 172.21.15.112:0/3567981981 shutdown_connections 2024-09-21T00:27:20.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:20.437+0000 7fac5b8f2640 1 --2- 172.21.15.112:0/3567981981 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fac5415c750 0x7fac5409c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:20.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:20.437+0000 7fac5b8f2640 1 --2- 172.21.15.112:0/3567981981 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fac540b2ca0 0x7fac541f41a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:20.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:20.437+0000 7fac5b8f2640 1 --2- 172.21.15.112:0/3567981981 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fac38078b90 0x7fac3807b0a0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:20.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:20.437+0000 7fac5b8f2640 1 --2- 172.21.15.112:0/3567981981 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac5406ff00 0x7fac5409ecd0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:20.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:20.437+0000 7fac5b8f2640 1 -- 172.21.15.112:0/3567981981 >> 172.21.15.112:0/3567981981 conn(0x7fac540cdca0 msgr2=0x7fac540b1180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:20.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:20.437+0000 7fac5b8f2640 1 -- 172.21.15.112:0/3567981981 shutdown_connections 2024-09-21T00:27:20.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:20.437+0000 7fac5b8f2640 1 -- 172.21.15.112:0/3567981981 wait complete. 2024-09-21T00:27:20.450 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-21T00:27:20.450 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:27:20.450 DEBUG:teuthology.orchestra.run.smithi112:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-21T00:27:20.478 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi112.front.sepia.ceph.com 2024-09-21T00:27:20.478 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:27:20.478 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T00:27:20.534 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T00:27:20.534 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-09-21T00:27:20.534 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:27:20.551 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-09-21T00:27:20.551 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:27:20.608 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T00:27:20.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:20.719+0000 7fb58a6f2640 1 -- 172.21.15.112:0/2268662763 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb5840ccc40 msgr2=0x7fb58415ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:20.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:20.719+0000 7fb58a6f2640 1 --2- 172.21.15.112:0/2268662763 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb5840ccc40 0x7fb58415ca90 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fb574005fd0 tx=0x7fb57402a540 comp rx=0 tx=0).stop 2024-09-21T00:27:20.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:20.719+0000 7fb58a6f2640 1 -- 172.21.15.112:0/2268662763 shutdown_connections 2024-09-21T00:27:20.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:20.719+0000 7fb58a6f2640 1 --2- 172.21.15.112:0/2268662763 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb5840ccc40 0x7fb58415ca90 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:20.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:20.719+0000 7fb58a6f2640 1 -- 172.21.15.112:0/2268662763 >> 172.21.15.112:0/2268662763 conn(0x7fb5840b46e0 msgr2=0x7fb5840d23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:20.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:20.719+0000 7fb58a6f2640 1 -- 172.21.15.112:0/2268662763 shutdown_connections 2024-09-21T00:27:20.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:20.719+0000 7fb58a6f2640 1 -- 172.21.15.112:0/2268662763 wait complete. 2024-09-21T00:27:20.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:20.720+0000 7fb58a6f2640 1 Processor -- start 2024-09-21T00:27:20.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:20.721+0000 7fb58a6f2640 1 -- start start 2024-09-21T00:27:20.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:20.722+0000 7fb58a6f2640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb5840ccc40 0x7fb5841f9b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:20.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:20.722+0000 7fb58a6f2640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb5840cd440 0x7fb5840cb840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:20.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:20.722+0000 7fb58a6f2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb5840cbd80 0x7fb5840b0490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:20.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:20.722+0000 7fb58a6f2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb5840b6850 con 0x7fb5840cbd80 2024-09-21T00:27:20.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:20.722+0000 7fb58a6f2640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb5840b66d0 con 0x7fb5840ccc40 2024-09-21T00:27:20.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:20.722+0000 7fb58a6f2640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fb5840b69d0 con 0x7fb5840cd440 2024-09-21T00:27:20.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:20.722+0000 7fb588c8f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb5840cbd80 0x7fb5840b0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:20.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:20.722+0000 7fb588c8f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb5840cbd80 0x7fb5840b0490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47760/0 (socket says 172.21.15.112:47760) 2024-09-21T00:27:20.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:20.722+0000 7fb588c8f640 1 -- 172.21.15.112:0/1501632358 learned_addr learned my addr 172.21.15.112:0/1501632358 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:27:20.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:20.722+0000 7fb583fff640 1 --2- 172.21.15.112:0/1501632358 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb5840ccc40 0x7fb5841f9b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:20.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:20.722+0000 7fb588c8f640 1 -- 172.21.15.112:0/1501632358 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb5840cd440 msgr2=0x7fb5840cb840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:20.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:20.722+0000 7fb5837fe640 1 --2- 172.21.15.112:0/1501632358 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb5840cd440 0x7fb5840cb840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:20.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:20.722+0000 7fb588c8f640 1 --2- 172.21.15.112:0/1501632358 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb5840cd440 0x7fb5840cb840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:20.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:20.722+0000 7fb588c8f640 1 -- 172.21.15.112:0/1501632358 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb5840ccc40 msgr2=0x7fb5841f9b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:20.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:20.722+0000 7fb588c8f640 1 --2- 172.21.15.112:0/1501632358 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb5840ccc40 0x7fb5841f9b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:20.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:20.722+0000 7fb588c8f640 1 -- 172.21.15.112:0/1501632358 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5841f6190 con 0x7fb5840cbd80 2024-09-21T00:27:20.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:20.722+0000 7fb583fff640 1 --2- 172.21.15.112:0/1501632358 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb5840ccc40 0x7fb5841f9b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:27:20.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:20.723+0000 7fb588c8f640 1 --2- 172.21.15.112:0/1501632358 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb5840cbd80 0x7fb5840b0490 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fb57c00c700 tx=0x7fb57c00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:20.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:20.723+0000 7fb580ff9640 1 -- 172.21.15.112:0/1501632358 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb57c003b70 con 0x7fb5840cbd80 2024-09-21T00:27:20.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:20.723+0000 7fb580ff9640 1 -- 172.21.15.112:0/1501632358 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb57c003d30 con 0x7fb5840cbd80 2024-09-21T00:27:20.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:20.723+0000 7fb58a6f2640 1 -- 172.21.15.112:0/1501632358 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5841f6470 con 0x7fb5840cbd80 2024-09-21T00:27:20.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:20.723+0000 7fb580ff9640 1 -- 172.21.15.112:0/1501632358 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb57c005330 con 0x7fb5840cbd80 2024-09-21T00:27:20.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:20.723+0000 7fb58a6f2640 1 -- 172.21.15.112:0/1501632358 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb58415cca0 con 0x7fb5840cbd80 2024-09-21T00:27:20.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:20.724+0000 7fb58a6f2640 1 -- 172.21.15.112:0/1501632358 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5841f5f00 con 0x7fb5840cbd80 2024-09-21T00:27:20.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:20.725+0000 7fb580ff9640 1 -- 172.21.15.112:0/1501632358 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb57c0054d0 con 0x7fb5840cbd80 2024-09-21T00:27:20.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:20.725+0000 7fb580ff9640 1 --2- 172.21.15.112:0/1501632358 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fb570078b90 0x7fb57007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:20.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:20.725+0000 7fb580ff9640 1 -- 172.21.15.112:0/1501632358 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 6101+0+0 (secure 0 0 0) 0x7fb57c09ab70 con 0x7fb5840cbd80 2024-09-21T00:27:20.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:20.725+0000 7fb583fff640 1 --2- 172.21.15.112:0/1501632358 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fb570078b90 0x7fb57007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:20.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:20.726+0000 7fb583fff640 1 --2- 172.21.15.112:0/1501632358 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fb570078b90 0x7fb57007b0a0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fb574033000 tx=0x7fb574004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:20.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:20.730+0000 7fb580ff9640 1 -- 172.21.15.112:0/1501632358 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb57c062f40 con 0x7fb5840cbd80 2024-09-21T00:27:20.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:20.919+0000 7fb58a6f2640 1 -- 172.21.15.112:0/1501632358 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fb5840cca60 con 0x7fb5840cbd80 2024-09-21T00:27:20.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:20.919+0000 7fb580ff9640 1 -- 172.21.15.112:0/1501632358 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v49) ==== 80+0+0 (secure 0 0 0) 0x7fb5840cca60 con 0x7fb5840cbd80 2024-09-21T00:27:20.921 INFO:teuthology.orchestra.run.smithi112.stderr:listed 0 entries 2024-09-21T00:27:20.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:20.921+0000 7fb58a6f2640 1 -- 172.21.15.112:0/1501632358 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fb570078b90 msgr2=0x7fb57007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:20.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:20.921+0000 7fb58a6f2640 1 --2- 172.21.15.112:0/1501632358 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fb570078b90 0x7fb57007b0a0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fb574033000 tx=0x7fb574004730 comp rx=0 tx=0).stop 2024-09-21T00:27:20.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:20.921+0000 7fb58a6f2640 1 -- 172.21.15.112:0/1501632358 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb5840cbd80 msgr2=0x7fb5840b0490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:20.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:20.921+0000 7fb58a6f2640 1 --2- 172.21.15.112:0/1501632358 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb5840cbd80 0x7fb5840b0490 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fb57c00c700 tx=0x7fb57c00cbc0 comp rx=0 tx=0).stop 2024-09-21T00:27:20.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:20.921+0000 7fb58a6f2640 1 -- 172.21.15.112:0/1501632358 shutdown_connections 2024-09-21T00:27:20.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:20.921+0000 7fb58a6f2640 1 --2- 172.21.15.112:0/1501632358 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fb570078b90 0x7fb57007b0a0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:20.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:20.921+0000 7fb58a6f2640 1 --2- 172.21.15.112:0/1501632358 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb5840cbd80 0x7fb5840b0490 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:20.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:20.921+0000 7fb58a6f2640 1 --2- 172.21.15.112:0/1501632358 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb5840cd440 0x7fb5840cb840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:20.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:20.921+0000 7fb58a6f2640 1 --2- 172.21.15.112:0/1501632358 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb5840ccc40 0x7fb5841f9b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:20.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:20.921+0000 7fb58a6f2640 1 -- 172.21.15.112:0/1501632358 >> 172.21.15.112:0/1501632358 conn(0x7fb5840b46e0 msgr2=0x7fb5840d0c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:20.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:20.922+0000 7fb58a6f2640 1 -- 172.21.15.112:0/1501632358 shutdown_connections 2024-09-21T00:27:20.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:20.922+0000 7fb58a6f2640 1 -- 172.21.15.112:0/1501632358 wait complete. 2024-09-21T00:27:20.934 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-09-21T00:27:20.934 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:27:20.934 DEBUG:teuthology.orchestra.run.smithi187:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-09-21T00:27:20.966 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi187.front.sepia.ceph.com 2024-09-21T00:27:20.966 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:27:20.966 DEBUG:teuthology.orchestra.run.smithi187:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-21T00:27:21.024 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T00:27:21.024 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T00:27:21.024 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:27:21.040 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056010,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1439 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248716k,nr_inodes=812179,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T00:27:21.040 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T00:27:21.040 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-09-21T00:27:21.040 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:27:21.082 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056013,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497440k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16835 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248720k,nr_inodes=812180,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497440k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T00:27:21.082 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T00:27:21.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.193+0000 7fa933fc6640 1 -- 172.21.15.112:0/1228171047 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa92c073d50 msgr2=0x7fa92c074160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:21.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.193+0000 7fa933fc6640 1 --2- 172.21.15.112:0/1228171047 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa92c073d50 0x7fa92c074160 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fa92c0cd2c0 tx=0x7fa92402a8b0 comp rx=0 tx=0).stop 2024-09-21T00:27:21.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.193+0000 7fa933fc6640 1 -- 172.21.15.112:0/1228171047 shutdown_connections 2024-09-21T00:27:21.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.193+0000 7fa933fc6640 1 --2- 172.21.15.112:0/1228171047 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa92c073d50 0x7fa92c074160 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:21.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.193+0000 7fa933fc6640 1 --2- 172.21.15.112:0/1228171047 >> v2:172.21.15.187:3300/0 conn(0x7fa92c0b3fc0 0x7fa92c0b43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:21.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.193+0000 7fa933fc6640 1 -- 172.21.15.112:0/1228171047 >> 172.21.15.112:0/1228171047 conn(0x7fa92c0b2cc0 msgr2=0x7fa92c0b30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:21.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.193+0000 7fa933fc6640 1 -- 172.21.15.112:0/1228171047 shutdown_connections 2024-09-21T00:27:21.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.193+0000 7fa933fc6640 1 -- 172.21.15.112:0/1228171047 wait complete. 2024-09-21T00:27:21.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.195+0000 7fa933fc6640 1 Processor -- start 2024-09-21T00:27:21.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.195+0000 7fa933fc6640 1 -- start start 2024-09-21T00:27:21.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.196+0000 7fa933fc6640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa92c073d50 0x7fa92c1542c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:21.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.196+0000 7fa933fc6640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa92c0b3fc0 0x7fa92c1f15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:21.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.196+0000 7fa933fc6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa92c0d0b40 0x7fa92c0cf770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:21.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.196+0000 7fa933fc6640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa92c15b750 con 0x7fa92c0d0b40 2024-09-21T00:27:21.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.196+0000 7fa933fc6640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa92c15b5d0 con 0x7fa92c0b3fc0 2024-09-21T00:27:21.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.196+0000 7fa933fc6640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa92c15b8d0 con 0x7fa92c073d50 2024-09-21T00:27:21.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.196+0000 7fa932563640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa92c0d0b40 0x7fa92c0cf770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:21.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.196+0000 7fa932563640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa92c0d0b40 0x7fa92c0cf770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47768/0 (socket says 172.21.15.112:47768) 2024-09-21T00:27:21.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.196+0000 7fa932563640 1 -- 172.21.15.112:0/360424332 learned_addr learned my addr 172.21.15.112:0/360424332 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:27:21.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.196+0000 7fa931561640 1 --2- 172.21.15.112:0/360424332 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa92c0b3fc0 0x7fa92c1f15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:21.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.196+0000 7fa931d62640 1 --2- 172.21.15.112:0/360424332 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa92c073d50 0x7fa92c1542c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:21.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.196+0000 7fa932563640 1 -- 172.21.15.112:0/360424332 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa92c073d50 msgr2=0x7fa92c1542c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:21.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.196+0000 7fa932563640 1 --2- 172.21.15.112:0/360424332 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa92c073d50 0x7fa92c1542c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:21.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.196+0000 7fa932563640 1 -- 172.21.15.112:0/360424332 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa92c0b3fc0 msgr2=0x7fa92c1f15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:21.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.196+0000 7fa932563640 1 --2- 172.21.15.112:0/360424332 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa92c0b3fc0 0x7fa92c1f15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:21.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.196+0000 7fa932563640 1 -- 172.21.15.112:0/360424332 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa92c1527a0 con 0x7fa92c0d0b40 2024-09-21T00:27:21.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.197+0000 7fa932563640 1 --2- 172.21.15.112:0/360424332 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa92c0d0b40 0x7fa92c0cf770 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fa92402aea0 tx=0x7fa9240039f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:21.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.197+0000 7fa9227fc640 1 -- 172.21.15.112:0/360424332 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa924031990 con 0x7fa92c0d0b40 2024-09-21T00:27:21.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.197+0000 7fa9227fc640 1 -- 172.21.15.112:0/360424332 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa924031c20 con 0x7fa92c0d0b40 2024-09-21T00:27:21.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.197+0000 7fa9227fc640 1 -- 172.21.15.112:0/360424332 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa924033330 con 0x7fa92c0d0b40 2024-09-21T00:27:21.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.197+0000 7fa933fc6640 1 -- 172.21.15.112:0/360424332 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa92c152a80 con 0x7fa92c0d0b40 2024-09-21T00:27:21.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.197+0000 7fa933fc6640 1 -- 172.21.15.112:0/360424332 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa92c153040 con 0x7fa92c0d0b40 2024-09-21T00:27:21.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.198+0000 7fa933fc6640 1 -- 172.21.15.112:0/360424332 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa92c15bda0 con 0x7fa92c0d0b40 2024-09-21T00:27:21.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.199+0000 7fa9227fc640 1 -- 172.21.15.112:0/360424332 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa9240334d0 con 0x7fa92c0d0b40 2024-09-21T00:27:21.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.199+0000 7fa9227fc640 1 --2- 172.21.15.112:0/360424332 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fa910078bc0 0x7fa91007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:21.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.199+0000 7fa9227fc640 1 -- 172.21.15.112:0/360424332 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 6101+0+0 (secure 0 0 0) 0x7fa9240bc210 con 0x7fa92c0d0b40 2024-09-21T00:27:21.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.199+0000 7fa931d62640 1 --2- 172.21.15.112:0/360424332 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fa910078bc0 0x7fa91007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:21.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.200+0000 7fa931d62640 1 --2- 172.21.15.112:0/360424332 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fa910078bc0 0x7fa91007b0d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa928001720 tx=0x7fa928007e90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:21.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.204+0000 7fa9227fc640 1 -- 172.21.15.112:0/360424332 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa92403f030 con 0x7fa92c0d0b40 2024-09-21T00:27:21.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.418+0000 7fa933fc6640 1 -- 172.21.15.112:0/360424332 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa92c152510 con 0x7fa92c0d0b40 2024-09-21T00:27:21.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.418+0000 7fa9227fc640 1 -- 172.21.15.112:0/360424332 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5229 (secure 0 0 0) 0x7fa924083f00 con 0x7fa92c0d0b40 2024-09-21T00:27:21.420 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:27:21.420 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":26,"btime":"2024-09-21T00:27:10:580658+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5158,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6836/2010548530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":2010548530},{"type":"v1","addr":"172.21.15.187:6836","nonce":2010548530}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5161,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6837/2290459931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6835","nonce":2290459931},{"type":"v1","addr":"172.21.15.187:6837","nonce":2290459931}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5400,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6834/158782340","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":158782340},{"type":"v1","addr":"172.21.15.112:6834","nonce":158782340}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T00:27:06.929464+0000","modified":"2024-09-21T00:27:09.945226+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5403},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5403":{"gid":5403,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6835/722271164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":722271164},{"type":"v1","addr":"172.21.15.112:6835","nonce":722271164}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5403,"qdb_cluster":[5403]},"id":3}]} 2024-09-21T00:27:21.420 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 26 2024-09-21T00:27:21.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.421+0000 7fa933fc6640 1 -- 172.21.15.112:0/360424332 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fa910078bc0 msgr2=0x7fa91007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:21.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.421+0000 7fa933fc6640 1 --2- 172.21.15.112:0/360424332 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fa910078bc0 0x7fa91007b0d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa928001720 tx=0x7fa928007e90 comp rx=0 tx=0).stop 2024-09-21T00:27:21.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.421+0000 7fa933fc6640 1 -- 172.21.15.112:0/360424332 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa92c0d0b40 msgr2=0x7fa92c0cf770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:21.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.421+0000 7fa933fc6640 1 --2- 172.21.15.112:0/360424332 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa92c0d0b40 0x7fa92c0cf770 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fa92402aea0 tx=0x7fa9240039f0 comp rx=0 tx=0).stop 2024-09-21T00:27:21.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.421+0000 7fa933fc6640 1 -- 172.21.15.112:0/360424332 shutdown_connections 2024-09-21T00:27:21.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.421+0000 7fa933fc6640 1 --2- 172.21.15.112:0/360424332 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fa910078bc0 0x7fa91007b0d0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:21.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.421+0000 7fa933fc6640 1 --2- 172.21.15.112:0/360424332 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa92c0d0b40 0x7fa92c0cf770 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:21.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.421+0000 7fa933fc6640 1 --2- 172.21.15.112:0/360424332 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa92c0b3fc0 0x7fa92c1f15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:21.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.421+0000 7fa933fc6640 1 --2- 172.21.15.112:0/360424332 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa92c073d50 0x7fa92c1542c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:21.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.421+0000 7fa933fc6640 1 -- 172.21.15.112:0/360424332 >> 172.21.15.112:0/360424332 conn(0x7fa92c0b2cc0 msgr2=0x7fa92c15eeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:21.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.421+0000 7fa933fc6640 1 -- 172.21.15.112:0/360424332 shutdown_connections 2024-09-21T00:27:21.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.421+0000 7fa933fc6640 1 -- 172.21.15.112:0/360424332 wait complete. 2024-09-21T00:27:21.434 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T00:27:21.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.544+0000 7f9580e4d640 1 -- 172.21.15.112:0/747650542 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f957c0d1ed0 msgr2=0x7f957c0cc930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:21.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.544+0000 7f9580e4d640 1 --2- 172.21.15.112:0/747650542 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f957c0d1ed0 0x7f957c0cc930 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f9564009830 tx=0x7f956402a510 comp rx=0 tx=0).stop 2024-09-21T00:27:21.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.544+0000 7f9580e4d640 1 -- 172.21.15.112:0/747650542 shutdown_connections 2024-09-21T00:27:21.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.544+0000 7f9580e4d640 1 --2- 172.21.15.112:0/747650542 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f957c0d1ed0 0x7f957c0cc930 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:21.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.544+0000 7f9580e4d640 1 -- 172.21.15.112:0/747650542 >> 172.21.15.112:0/747650542 conn(0x7f957c0cef40 msgr2=0x7f957c0cf350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:21.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.544+0000 7f9580e4d640 1 -- 172.21.15.112:0/747650542 shutdown_connections 2024-09-21T00:27:21.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.544+0000 7f9580e4d640 1 -- 172.21.15.112:0/747650542 wait complete. 2024-09-21T00:27:21.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.546+0000 7f9580e4d640 1 Processor -- start 2024-09-21T00:27:21.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.546+0000 7f9580e4d640 1 -- start start 2024-09-21T00:27:21.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.548+0000 7f9580e4d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f957c0cce70 0x7f957c1f9680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:21.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.548+0000 7f9580e4d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f957c0d1ed0 0x7f957c06def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:21.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.548+0000 7f9580e4d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f957c1536e0 0x7f957c06e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:21.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.548+0000 7f9580e4d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f957c0d0f60 con 0x7f957c0cce70 2024-09-21T00:27:21.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.548+0000 7f9580e4d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f957c0d0de0 con 0x7f957c0d1ed0 2024-09-21T00:27:21.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.548+0000 7f9580e4d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f957c0d10e0 con 0x7f957c1536e0 2024-09-21T00:27:21.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.548+0000 7f957a59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f957c0cce70 0x7f957c1f9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:21.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.548+0000 7f957a59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f957c0cce70 0x7f957c1f9680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47788/0 (socket says 172.21.15.112:47788) 2024-09-21T00:27:21.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.548+0000 7f957a59c640 1 -- 172.21.15.112:0/2503945713 learned_addr learned my addr 172.21.15.112:0/2503945713 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:27:21.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.548+0000 7f9571d9b640 1 --2- 172.21.15.112:0/2503945713 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f957c0d1ed0 0x7f957c06def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:21.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.548+0000 7f957ad9d640 1 --2- 172.21.15.112:0/2503945713 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f957c1536e0 0x7f957c06e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:21.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.548+0000 7f957a59c640 1 -- 172.21.15.112:0/2503945713 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f957c1536e0 msgr2=0x7f957c06e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:21.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.548+0000 7f957a59c640 1 --2- 172.21.15.112:0/2503945713 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f957c1536e0 0x7f957c06e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:21.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.548+0000 7f957a59c640 1 -- 172.21.15.112:0/2503945713 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f957c0d1ed0 msgr2=0x7f957c06def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:21.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.548+0000 7f957a59c640 1 --2- 172.21.15.112:0/2503945713 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f957c0d1ed0 0x7f957c06def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:21.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.548+0000 7f957a59c640 1 -- 172.21.15.112:0/2503945713 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f957c1f8c10 con 0x7f957c0cce70 2024-09-21T00:27:21.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.548+0000 7f9571d9b640 1 --2- 172.21.15.112:0/2503945713 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f957c0d1ed0 0x7f957c06def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:27:21.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.548+0000 7f957a59c640 1 --2- 172.21.15.112:0/2503945713 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f957c0cce70 0x7f957c1f9680 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f956c00b420 tx=0x7f956c00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:21.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.548+0000 7f95737fe640 1 -- 172.21.15.112:0/2503945713 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f956c011830 con 0x7f957c0cce70 2024-09-21T00:27:21.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.548+0000 7f95737fe640 1 -- 172.21.15.112:0/2503945713 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f956c016070 con 0x7f957c0cce70 2024-09-21T00:27:21.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.548+0000 7f9580e4d640 1 -- 172.21.15.112:0/2503945713 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f957c0a95d0 con 0x7f957c0cce70 2024-09-21T00:27:21.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.548+0000 7f95737fe640 1 -- 172.21.15.112:0/2503945713 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f956c011d60 con 0x7f957c0cce70 2024-09-21T00:27:21.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.548+0000 7f9580e4d640 1 -- 172.21.15.112:0/2503945713 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f957c06ea30 con 0x7f957c0cce70 2024-09-21T00:27:21.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.550+0000 7f9580e4d640 1 -- 172.21.15.112:0/2503945713 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f957c0cc930 con 0x7f957c0cce70 2024-09-21T00:27:21.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.550+0000 7f95737fe640 1 -- 172.21.15.112:0/2503945713 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f956c01b460 con 0x7f957c0cce70 2024-09-21T00:27:21.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.550+0000 7f95737fe640 1 --2- 172.21.15.112:0/2503945713 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f955c078bc0 0x7f955c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:21.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.551+0000 7f95737fe640 1 -- 172.21.15.112:0/2503945713 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 6101+0+0 (secure 0 0 0) 0x7f956c09a8d0 con 0x7f957c0cce70 2024-09-21T00:27:21.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.551+0000 7f9571d9b640 1 --2- 172.21.15.112:0/2503945713 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f955c078bc0 0x7f955c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:21.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.551+0000 7f9571d9b640 1 --2- 172.21.15.112:0/2503945713 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f955c078bc0 0x7f955c07b0d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f9564034000 tx=0x7f9564004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:21.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.555+0000 7f95737fe640 1 -- 172.21.15.112:0/2503945713 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f956c020020 con 0x7f957c0cce70 2024-09-21T00:27:21.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.769+0000 7f9580e4d640 1 -- 172.21.15.112:0/2503945713 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f957c156ed0 con 0x7f957c0cce70 2024-09-21T00:27:21.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.769+0000 7f95737fe640 1 -- 172.21.15.112:0/2503945713 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5229 (secure 0 0 0) 0x7f956c062c10 con 0x7f957c0cce70 2024-09-21T00:27:21.771 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:27:21.771 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":26,"btime":"2024-09-21T00:27:10:580658+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5158,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6836/2010548530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":2010548530},{"type":"v1","addr":"172.21.15.187:6836","nonce":2010548530}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5161,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6837/2290459931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6835","nonce":2290459931},{"type":"v1","addr":"172.21.15.187:6837","nonce":2290459931}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5400,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6834/158782340","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":158782340},{"type":"v1","addr":"172.21.15.112:6834","nonce":158782340}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T00:27:06.929464+0000","modified":"2024-09-21T00:27:09.945226+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5403},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5403":{"gid":5403,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6835/722271164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":722271164},{"type":"v1","addr":"172.21.15.112:6835","nonce":722271164}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5403,"qdb_cluster":[5403]},"id":3}]} 2024-09-21T00:27:21.771 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 26 2024-09-21T00:27:21.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.771+0000 7f9580e4d640 1 -- 172.21.15.112:0/2503945713 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f955c078bc0 msgr2=0x7f955c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:21.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.771+0000 7f9580e4d640 1 --2- 172.21.15.112:0/2503945713 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f955c078bc0 0x7f955c07b0d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f9564034000 tx=0x7f9564004730 comp rx=0 tx=0).stop 2024-09-21T00:27:21.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.771+0000 7f9580e4d640 1 -- 172.21.15.112:0/2503945713 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f957c0cce70 msgr2=0x7f957c1f9680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:21.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.771+0000 7f9580e4d640 1 --2- 172.21.15.112:0/2503945713 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f957c0cce70 0x7f957c1f9680 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f956c00b420 tx=0x7f956c00b8e0 comp rx=0 tx=0).stop 2024-09-21T00:27:21.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.771+0000 7f9580e4d640 1 -- 172.21.15.112:0/2503945713 shutdown_connections 2024-09-21T00:27:21.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.771+0000 7f9580e4d640 1 --2- 172.21.15.112:0/2503945713 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f957c1536e0 0x7f957c06e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:21.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.771+0000 7f9580e4d640 1 --2- 172.21.15.112:0/2503945713 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f957c0d1ed0 0x7f957c06def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:21.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.771+0000 7f9580e4d640 1 --2- 172.21.15.112:0/2503945713 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f955c078bc0 0x7f955c07b0d0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:21.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.771+0000 7f9580e4d640 1 --2- 172.21.15.112:0/2503945713 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f957c0cce70 0x7f957c1f9680 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:21.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.771+0000 7f9580e4d640 1 -- 172.21.15.112:0/2503945713 >> 172.21.15.112:0/2503945713 conn(0x7f957c0cef40 msgr2=0x7f957c0b6690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:21.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.771+0000 7f9580e4d640 1 -- 172.21.15.112:0/2503945713 shutdown_connections 2024-09-21T00:27:21.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.771+0000 7f9580e4d640 1 -- 172.21.15.112:0/2503945713 wait complete. 2024-09-21T00:27:21.785 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T00:27:21.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.895+0000 7fd35f7ed640 1 -- 172.21.15.112:0/3467655232 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3580d12e0 msgr2=0x7fd3580d16f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:21.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.895+0000 7fd35f7ed640 1 --2- 172.21.15.112:0/3467655232 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3580d12e0 0x7fd3580d16f0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fd358074a40 tx=0x7fd34802a530 comp rx=0 tx=0).stop 2024-09-21T00:27:21.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.895+0000 7fd35f7ed640 1 -- 172.21.15.112:0/3467655232 shutdown_connections 2024-09-21T00:27:21.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.895+0000 7fd35f7ed640 1 --2- 172.21.15.112:0/3467655232 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3580d12e0 0x7fd3580d16f0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:21.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.895+0000 7fd35f7ed640 1 --2- 172.21.15.112:0/3467655232 >> v2:172.21.15.187:3300/0 conn(0x7fd35815cb10 0x7fd35815cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:21.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.895+0000 7fd35f7ed640 1 -- 172.21.15.112:0/3467655232 >> 172.21.15.112:0/3467655232 conn(0x7fd3580d2350 msgr2=0x7fd3580d2760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:21.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.895+0000 7fd35f7ed640 1 -- 172.21.15.112:0/3467655232 shutdown_connections 2024-09-21T00:27:21.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.895+0000 7fd35f7ed640 1 -- 172.21.15.112:0/3467655232 wait complete. 2024-09-21T00:27:21.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.897+0000 7fd35f7ed640 1 Processor -- start 2024-09-21T00:27:21.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.897+0000 7fd35f7ed640 1 -- start start 2024-09-21T00:27:21.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.898+0000 7fd35f7ed640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd3580d12e0 0x7fd3581f4110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:21.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.898+0000 7fd35f7ed640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd35815cb10 0x7fd358150ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:21.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.898+0000 7fd35f7ed640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd358160100 0x7fd358140930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:21.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.898+0000 7fd35f7ed640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd3580b73e0 con 0x7fd358160100 2024-09-21T00:27:21.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.898+0000 7fd35f7ed640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd3580b7260 con 0x7fd3580d12e0 2024-09-21T00:27:21.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.898+0000 7fd35f7ed640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd358160c20 con 0x7fd35815cb10 2024-09-21T00:27:21.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.898+0000 7fd35dd8a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd358160100 0x7fd358140930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:21.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.898+0000 7fd35dd8a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd358160100 0x7fd358140930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47808/0 (socket says 172.21.15.112:47808) 2024-09-21T00:27:21.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.898+0000 7fd35dd8a640 1 -- 172.21.15.112:0/3274900011 learned_addr learned my addr 172.21.15.112:0/3274900011 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:27:21.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.898+0000 7fd35d589640 1 --2- 172.21.15.112:0/3274900011 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd3580d12e0 0x7fd3581f4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:21.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.898+0000 7fd35cd88640 1 --2- 172.21.15.112:0/3274900011 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd35815cb10 0x7fd358150ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:21.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.899+0000 7fd35dd8a640 1 -- 172.21.15.112:0/3274900011 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd35815cb10 msgr2=0x7fd358150ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:21.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.899+0000 7fd35dd8a640 1 --2- 172.21.15.112:0/3274900011 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd35815cb10 0x7fd358150ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:21.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.899+0000 7fd35dd8a640 1 -- 172.21.15.112:0/3274900011 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd3580d12e0 msgr2=0x7fd3581f4110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:21.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.899+0000 7fd35dd8a640 1 --2- 172.21.15.112:0/3274900011 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd3580d12e0 0x7fd3581f4110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:21.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.899+0000 7fd35dd8a640 1 -- 172.21.15.112:0/3274900011 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3581e5f40 con 0x7fd358160100 2024-09-21T00:27:21.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.899+0000 7fd35cd88640 1 --2- 172.21.15.112:0/3274900011 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd35815cb10 0x7fd358150ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:27:21.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.899+0000 7fd35dd8a640 1 --2- 172.21.15.112:0/3274900011 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd358160100 0x7fd358140930 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fd35000c390 tx=0x7fd35000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:21.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.899+0000 7fd34dffb640 1 -- 172.21.15.112:0/3274900011 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd350011790 con 0x7fd358160100 2024-09-21T00:27:21.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.899+0000 7fd34dffb640 1 -- 172.21.15.112:0/3274900011 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd350017070 con 0x7fd358160100 2024-09-21T00:27:21.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.899+0000 7fd35f7ed640 1 -- 172.21.15.112:0/3274900011 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3581f4a10 con 0x7fd358160100 2024-09-21T00:27:21.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.899+0000 7fd34dffb640 1 -- 172.21.15.112:0/3274900011 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd350011c60 con 0x7fd358160100 2024-09-21T00:27:21.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.899+0000 7fd35f7ed640 1 -- 172.21.15.112:0/3274900011 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3581f4fd0 con 0x7fd358160100 2024-09-21T00:27:21.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.900+0000 7fd35f7ed640 1 -- 172.21.15.112:0/3274900011 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3581e6110 con 0x7fd358160100 2024-09-21T00:27:21.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.901+0000 7fd34dffb640 1 -- 172.21.15.112:0/3274900011 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd35001c460 con 0x7fd358160100 2024-09-21T00:27:21.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.901+0000 7fd34dffb640 1 --2- 172.21.15.112:0/3274900011 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fd33c078b90 0x7fd33c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:21.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.902+0000 7fd34dffb640 1 -- 172.21.15.112:0/3274900011 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 6101+0+0 (secure 0 0 0) 0x7fd35009ab70 con 0x7fd358160100 2024-09-21T00:27:21.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.902+0000 7fd35d589640 1 --2- 172.21.15.112:0/3274900011 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fd33c078b90 0x7fd33c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:21.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.902+0000 7fd35d589640 1 --2- 172.21.15.112:0/3274900011 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fd33c078b90 0x7fd33c07b0a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fd354005fd0 tx=0x7fd3540095e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:21.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:21.906+0000 7fd34dffb640 1 -- 172.21.15.112:0/3274900011 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd350062e20 con 0x7fd358160100 2024-09-21T00:27:22.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.082+0000 7fd35f7ed640 1 -- 172.21.15.112:0/3274900011 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd3581f4780 con 0x7fd358160100 2024-09-21T00:27:22.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.083+0000 7fd34dffb640 1 -- 172.21.15.112:0/3274900011 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) ==== 74+0+17026 (secure 0 0 0) 0x7fd350062720 con 0x7fd358160100 2024-09-21T00:27:22.085 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:27:22.085 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":49,"fsid":"8b34692b-dc65-49b2-a08f-26f707b96ac5","created":"2024-09-21T00:25:22.788490+0000","modified":"2024-09-21T00:27:08.558791+0000","last_up_change":"2024-09-21T00:25:36.161835+0000","last_in_change":"2024-09-21T00:25:26.346500+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T00:25:37.105006+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-09-21T00:27:04.569521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-09-21T00:27:05.881104+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9c575086-67d2-4de2-b01b-d44dcc21d604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6809","nonce":755760621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6811","nonce":755760621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6815","nonce":755760621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6813","nonce":755760621}]},"public_addr":"172.21.15.112:6809/755760621","cluster_addr":"172.21.15.112:6811/755760621","heartbeat_back_addr":"172.21.15.112:6815/755760621","heartbeat_front_addr":"172.21.15.112:6813/755760621","state":["exists","up"]},{"osd":1,"uuid":"a983c227-a3ce-4bdd-82f1-438ba7e3ecd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6801","nonce":344932592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6803","nonce":344932592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6807","nonce":344932592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6805","nonce":344932592}]},"public_addr":"172.21.15.112:6801/344932592","cluster_addr":"172.21.15.112:6803/344932592","heartbeat_back_addr":"172.21.15.112:6807/344932592","heartbeat_front_addr":"172.21.15.112:6805/344932592","state":["exists","up"]},{"osd":2,"uuid":"1193c1d8-bfff-4a57-8200-5f69bf1ea3af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6825","nonce":407933616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6827","nonce":407933616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6831","nonce":407933616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6829","nonce":407933616}]},"public_addr":"172.21.15.112:6825/407933616","cluster_addr":"172.21.15.112:6827/407933616","heartbeat_back_addr":"172.21.15.112:6831/407933616","heartbeat_front_addr":"172.21.15.112:6829/407933616","state":["exists","up"]},{"osd":3,"uuid":"af04330a-36ad-40cc-a8dd-fa259b221de5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6817","nonce":4120164806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6819","nonce":4120164806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6823","nonce":4120164806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6821","nonce":4120164806}]},"public_addr":"172.21.15.112:6817/4120164806","cluster_addr":"172.21.15.112:6819/4120164806","heartbeat_back_addr":"172.21.15.112:6823/4120164806","heartbeat_front_addr":"172.21.15.112:6821/4120164806","state":["exists","up"]},{"osd":4,"uuid":"5b893111-292c-4c5a-b184-53ad38c7199e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6803","nonce":522077560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6805","nonce":522077560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6809","nonce":522077560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6807","nonce":522077560}]},"public_addr":"172.21.15.187:6803/522077560","cluster_addr":"172.21.15.187:6805/522077560","heartbeat_back_addr":"172.21.15.187:6809/522077560","heartbeat_front_addr":"172.21.15.187:6807/522077560","state":["exists","up"]},{"osd":5,"uuid":"c0b8e011-45fe-4c5e-925e-6be2a7f1868a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6827","nonce":3593144721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6829","nonce":3593144721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6833","nonce":3593144721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6831","nonce":3593144721}]},"public_addr":"172.21.15.187:6827/3593144721","cluster_addr":"172.21.15.187:6829/3593144721","heartbeat_back_addr":"172.21.15.187:6833/3593144721","heartbeat_front_addr":"172.21.15.187:6831/3593144721","state":["exists","up"]},{"osd":6,"uuid":"87d2d650-7489-4593-98a3-a1f433ce8dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6811","nonce":1494387657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6813","nonce":1494387657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6817","nonce":1494387657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6815","nonce":1494387657}]},"public_addr":"172.21.15.187:6811/1494387657","cluster_addr":"172.21.15.187:6813/1494387657","heartbeat_back_addr":"172.21.15.187:6817/1494387657","heartbeat_front_addr":"172.21.15.187:6815/1494387657","state":["exists","up"]},{"osd":7,"uuid":"c6b4f8e2-a1ee-4438-aa37-67ff4b6ef1f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6819","nonce":1624699842}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6821","nonce":1624699842}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6825","nonce":1624699842}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6823","nonce":1624699842}]},"public_addr":"172.21.15.187:6819/1624699842","cluster_addr":"172.21.15.187:6821/1624699842","heartbeat_back_addr":"172.21.15.187:6825/1624699842","heartbeat_front_addr":"172.21.15.187:6823/1624699842","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.873162+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.400506+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.664317+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.159791+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.202630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.113212+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.384786+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.448441+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T00:27:22.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.085+0000 7fd35f7ed640 1 -- 172.21.15.112:0/3274900011 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fd33c078b90 msgr2=0x7fd33c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:22.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.085+0000 7fd35f7ed640 1 --2- 172.21.15.112:0/3274900011 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fd33c078b90 0x7fd33c07b0a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fd354005fd0 tx=0x7fd3540095e0 comp rx=0 tx=0).stop 2024-09-21T00:27:22.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.085+0000 7fd35f7ed640 1 -- 172.21.15.112:0/3274900011 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd358160100 msgr2=0x7fd358140930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:22.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.085+0000 7fd35f7ed640 1 --2- 172.21.15.112:0/3274900011 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd358160100 0x7fd358140930 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fd35000c390 tx=0x7fd35000c850 comp rx=0 tx=0).stop 2024-09-21T00:27:22.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.085+0000 7fd35f7ed640 1 -- 172.21.15.112:0/3274900011 shutdown_connections 2024-09-21T00:27:22.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.085+0000 7fd35f7ed640 1 --2- 172.21.15.112:0/3274900011 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fd33c078b90 0x7fd33c07b0a0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:22.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.085+0000 7fd35f7ed640 1 --2- 172.21.15.112:0/3274900011 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd358160100 0x7fd358140930 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:22.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.085+0000 7fd35f7ed640 1 --2- 172.21.15.112:0/3274900011 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd35815cb10 0x7fd358150ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:22.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.085+0000 7fd35f7ed640 1 --2- 172.21.15.112:0/3274900011 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd3580d12e0 0x7fd3581f4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:22.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.085+0000 7fd35f7ed640 1 -- 172.21.15.112:0/3274900011 >> 172.21.15.112:0/3274900011 conn(0x7fd3580d2350 msgr2=0x7fd358160510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:22.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.085+0000 7fd35f7ed640 1 -- 172.21.15.112:0/3274900011 shutdown_connections 2024-09-21T00:27:22.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.085+0000 7fd35f7ed640 1 -- 172.21.15.112:0/3274900011 wait complete. 2024-09-21T00:27:22.098 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-21T00:27:22.098 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T00:27:22.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.208+0000 7fb67e313640 1 -- 172.21.15.112:0/2259337327 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6780cf9e0 msgr2=0x7fb678155b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:22.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.208+0000 7fb67e313640 1 --2- 172.21.15.112:0/2259337327 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6780cf9e0 0x7fb678155b60 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fb668009900 tx=0x7fb66802a610 comp rx=0 tx=0).stop 2024-09-21T00:27:22.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.208+0000 7fb67e313640 1 -- 172.21.15.112:0/2259337327 shutdown_connections 2024-09-21T00:27:22.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.208+0000 7fb67e313640 1 --2- 172.21.15.112:0/2259337327 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb67815c4d0 0x7fb6780cd650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:22.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.208+0000 7fb67e313640 1 --2- 172.21.15.112:0/2259337327 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6780cf9e0 0x7fb678155b60 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:22.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.208+0000 7fb67e313640 1 -- 172.21.15.112:0/2259337327 >> 172.21.15.112:0/2259337327 conn(0x7fb6780d0c00 msgr2=0x7fb6780ccf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:22.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.208+0000 7fb67e313640 1 -- 172.21.15.112:0/2259337327 shutdown_connections 2024-09-21T00:27:22.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.208+0000 7fb67e313640 1 -- 172.21.15.112:0/2259337327 wait complete. 2024-09-21T00:27:22.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.210+0000 7fb67e313640 1 Processor -- start 2024-09-21T00:27:22.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.210+0000 7fb67e313640 1 -- start start 2024-09-21T00:27:22.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.211+0000 7fb67e313640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb6780cdb90 0x7fb678158d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:22.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.211+0000 7fb67e313640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb6780cf9e0 0x7fb678143810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:22.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.211+0000 7fb67e313640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb67815c4d0 0x7fb6781f3540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:22.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.211+0000 7fb67e313640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb67815f8d0 con 0x7fb67815c4d0 2024-09-21T00:27:22.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.211+0000 7fb67e313640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb67815f750 con 0x7fb6780cdb90 2024-09-21T00:27:22.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.211+0000 7fb67e313640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fb67815fa50 con 0x7fb6780cf9e0 2024-09-21T00:27:22.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.211+0000 7fb67c8b0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb67815c4d0 0x7fb6781f3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:22.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.211+0000 7fb67c8b0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb67815c4d0 0x7fb6781f3540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47824/0 (socket says 172.21.15.112:47824) 2024-09-21T00:27:22.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.211+0000 7fb67c8b0640 1 -- 172.21.15.112:0/1344030325 learned_addr learned my addr 172.21.15.112:0/1344030325 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:27:22.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.212+0000 7fb6777fe640 1 --2- 172.21.15.112:0/1344030325 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb6780cf9e0 0x7fb678143810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:22.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.212+0000 7fb677fff640 1 --2- 172.21.15.112:0/1344030325 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb6780cdb90 0x7fb678158d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:22.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.212+0000 7fb67c8b0640 1 -- 172.21.15.112:0/1344030325 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb6780cf9e0 msgr2=0x7fb678143810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:22.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.212+0000 7fb67c8b0640 1 --2- 172.21.15.112:0/1344030325 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb6780cf9e0 0x7fb678143810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:22.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.212+0000 7fb67c8b0640 1 -- 172.21.15.112:0/1344030325 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb6780cdb90 msgr2=0x7fb678158d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:22.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.212+0000 7fb67c8b0640 1 --2- 172.21.15.112:0/1344030325 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb6780cdb90 0x7fb678158d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:22.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.212+0000 7fb67c8b0640 1 -- 172.21.15.112:0/1344030325 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6781f40a0 con 0x7fb67815c4d0 2024-09-21T00:27:22.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.212+0000 7fb67c8b0640 1 --2- 172.21.15.112:0/1344030325 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb67815c4d0 0x7fb6781f3540 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fb67000c730 tx=0x7fb67000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:22.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.212+0000 7fb674ff9640 1 -- 172.21.15.112:0/1344030325 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb670003b70 con 0x7fb67815c4d0 2024-09-21T00:27:22.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.212+0000 7fb674ff9640 1 -- 172.21.15.112:0/1344030325 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb670003d30 con 0x7fb67815c4d0 2024-09-21T00:27:22.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.212+0000 7fb67e313640 1 -- 172.21.15.112:0/1344030325 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6781f4380 con 0x7fb67815c4d0 2024-09-21T00:27:22.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.212+0000 7fb674ff9640 1 -- 172.21.15.112:0/1344030325 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6700052d0 con 0x7fb67815c4d0 2024-09-21T00:27:22.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.212+0000 7fb67e313640 1 -- 172.21.15.112:0/1344030325 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb678159240 con 0x7fb67815c4d0 2024-09-21T00:27:22.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.213+0000 7fb67e313640 1 -- 172.21.15.112:0/1344030325 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6781582f0 con 0x7fb67815c4d0 2024-09-21T00:27:22.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.214+0000 7fb674ff9640 1 -- 172.21.15.112:0/1344030325 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb670005470 con 0x7fb67815c4d0 2024-09-21T00:27:22.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.214+0000 7fb674ff9640 1 --2- 172.21.15.112:0/1344030325 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fb664078b90 0x7fb66407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:22.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.215+0000 7fb674ff9640 1 -- 172.21.15.112:0/1344030325 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 6101+0+0 (secure 0 0 0) 0x7fb67009ab60 con 0x7fb67815c4d0 2024-09-21T00:27:22.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.215+0000 7fb677fff640 1 --2- 172.21.15.112:0/1344030325 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fb664078b90 0x7fb66407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:22.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.215+0000 7fb677fff640 1 --2- 172.21.15.112:0/1344030325 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fb664078b90 0x7fb66407b0a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb668033000 tx=0x7fb668004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:22.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.219+0000 7fb674ff9640 1 -- 172.21.15.112:0/1344030325 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb6780cdfa0 con 0x7fb67815c4d0 2024-09-21T00:27:22.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.431+0000 7fb67e313640 1 -- 172.21.15.112:0/1344030325 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb678155b60 con 0x7fb67815c4d0 2024-09-21T00:27:22.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.432+0000 7fb674ff9640 1 -- 172.21.15.112:0/1344030325 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5229 (secure 0 0 0) 0x7fb6700627c0 con 0x7fb67815c4d0 2024-09-21T00:27:22.434 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:27:22.434 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":26,"btime":"2024-09-21T00:27:10:580658+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5158,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6836/2010548530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":2010548530},{"type":"v1","addr":"172.21.15.187:6836","nonce":2010548530}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5161,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6837/2290459931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6835","nonce":2290459931},{"type":"v1","addr":"172.21.15.187:6837","nonce":2290459931}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5400,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6834/158782340","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":158782340},{"type":"v1","addr":"172.21.15.112:6834","nonce":158782340}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T00:27:06.929464+0000","modified":"2024-09-21T00:27:09.945226+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5403},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5403":{"gid":5403,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6835/722271164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":722271164},{"type":"v1","addr":"172.21.15.112:6835","nonce":722271164}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5403,"qdb_cluster":[5403]},"id":3}]} 2024-09-21T00:27:22.434 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 26 2024-09-21T00:27:22.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.434+0000 7fb67e313640 1 -- 172.21.15.112:0/1344030325 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fb664078b90 msgr2=0x7fb66407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:22.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.434+0000 7fb67e313640 1 --2- 172.21.15.112:0/1344030325 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fb664078b90 0x7fb66407b0a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb668033000 tx=0x7fb668004730 comp rx=0 tx=0).stop 2024-09-21T00:27:22.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.434+0000 7fb67e313640 1 -- 172.21.15.112:0/1344030325 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb67815c4d0 msgr2=0x7fb6781f3540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:22.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.434+0000 7fb67e313640 1 --2- 172.21.15.112:0/1344030325 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb67815c4d0 0x7fb6781f3540 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fb67000c730 tx=0x7fb67000cbf0 comp rx=0 tx=0).stop 2024-09-21T00:27:22.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.434+0000 7fb67e313640 1 -- 172.21.15.112:0/1344030325 shutdown_connections 2024-09-21T00:27:22.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.434+0000 7fb67e313640 1 --2- 172.21.15.112:0/1344030325 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fb664078b90 0x7fb66407b0a0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:22.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.434+0000 7fb67e313640 1 --2- 172.21.15.112:0/1344030325 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb67815c4d0 0x7fb6781f3540 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:22.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.434+0000 7fb67e313640 1 --2- 172.21.15.112:0/1344030325 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb6780cf9e0 0x7fb678143810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:22.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.434+0000 7fb67e313640 1 --2- 172.21.15.112:0/1344030325 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb6780cdb90 0x7fb678158d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:22.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.434+0000 7fb67e313640 1 -- 172.21.15.112:0/1344030325 >> 172.21.15.112:0/1344030325 conn(0x7fb6780d0c00 msgr2=0x7fb6780b0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:22.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.434+0000 7fb67e313640 1 -- 172.21.15.112:0/1344030325 shutdown_connections 2024-09-21T00:27:22.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.434+0000 7fb67e313640 1 -- 172.21.15.112:0/1344030325 wait complete. 2024-09-21T00:27:22.447 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T00:27:22.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.557+0000 7fc89d01f640 1 -- 172.21.15.112:0/323716825 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc89815b9f0 msgr2=0x7fc8980cf930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:22.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.557+0000 7fc89d01f640 1 --2- 172.21.15.112:0/323716825 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc89815b9f0 0x7fc8980cf930 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fc888009830 tx=0x7fc88802a510 comp rx=0 tx=0).stop 2024-09-21T00:27:22.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.557+0000 7fc89d01f640 1 -- 172.21.15.112:0/323716825 shutdown_connections 2024-09-21T00:27:22.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.557+0000 7fc89d01f640 1 --2- 172.21.15.112:0/323716825 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc89815f5a0 0x7fc89815f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:22.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.557+0000 7fc89d01f640 1 --2- 172.21.15.112:0/323716825 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc89815b9f0 0x7fc8980cf930 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:22.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.557+0000 7fc89d01f640 1 -- 172.21.15.112:0/323716825 >> 172.21.15.112:0/323716825 conn(0x7fc8980cca40 msgr2=0x7fc8980cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:22.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.557+0000 7fc89d01f640 1 -- 172.21.15.112:0/323716825 shutdown_connections 2024-09-21T00:27:22.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.557+0000 7fc89d01f640 1 -- 172.21.15.112:0/323716825 wait complete. 2024-09-21T00:27:22.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.559+0000 7fc89d01f640 1 Processor -- start 2024-09-21T00:27:22.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.560+0000 7fc89d01f640 1 -- start start 2024-09-21T00:27:22.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.561+0000 7fc89d01f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc8980cefa0 0x7fc89809eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:22.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.561+0000 7fc89d01f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc89815b9f0 0x7fc8981f2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:22.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.561+0000 7fc89d01f640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc89815f5a0 0x7fc89809a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:22.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.561+0000 7fc89d01f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc89806df70 con 0x7fc8980cefa0 2024-09-21T00:27:22.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.561+0000 7fc89d01f640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc89806ddf0 con 0x7fc89815b9f0 2024-09-21T00:27:22.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.561+0000 7fc89d01f640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc89806e0f0 con 0x7fc89815f5a0 2024-09-21T00:27:22.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.561+0000 7fc896d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc8980cefa0 0x7fc89809eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:22.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.561+0000 7fc896d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc8980cefa0 0x7fc89809eca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47838/0 (socket says 172.21.15.112:47838) 2024-09-21T00:27:22.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.561+0000 7fc896d9d640 1 -- 172.21.15.112:0/2214504464 learned_addr learned my addr 172.21.15.112:0/2214504464 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:27:22.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.561+0000 7fc89759e640 1 --2- 172.21.15.112:0/2214504464 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc89815f5a0 0x7fc89809a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:22.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.561+0000 7fc89659c640 1 --2- 172.21.15.112:0/2214504464 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc89815b9f0 0x7fc8981f2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:22.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.561+0000 7fc896d9d640 1 -- 172.21.15.112:0/2214504464 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc89815f5a0 msgr2=0x7fc89809a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:22.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.561+0000 7fc896d9d640 1 --2- 172.21.15.112:0/2214504464 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc89815f5a0 0x7fc89809a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:22.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.561+0000 7fc896d9d640 1 -- 172.21.15.112:0/2214504464 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc89815b9f0 msgr2=0x7fc8981f2f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:22.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.561+0000 7fc896d9d640 1 --2- 172.21.15.112:0/2214504464 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc89815b9f0 0x7fc8981f2f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:22.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.561+0000 7fc896d9d640 1 -- 172.21.15.112:0/2214504464 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc89809f1e0 con 0x7fc8980cefa0 2024-09-21T00:27:22.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.562+0000 7fc896d9d640 1 --2- 172.21.15.112:0/2214504464 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc8980cefa0 0x7fc89809eca0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fc888009950 tx=0x7fc888033c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:22.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.562+0000 7fc87f7fe640 1 -- 172.21.15.112:0/2214504464 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc888005c90 con 0x7fc8980cefa0 2024-09-21T00:27:22.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.562+0000 7fc87f7fe640 1 -- 172.21.15.112:0/2214504464 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc888031e50 con 0x7fc8980cefa0 2024-09-21T00:27:22.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.562+0000 7fc87f7fe640 1 -- 172.21.15.112:0/2214504464 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8880332f0 con 0x7fc8980cefa0 2024-09-21T00:27:22.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.562+0000 7fc89d01f640 1 -- 172.21.15.112:0/2214504464 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc89809f470 con 0x7fc8980cefa0 2024-09-21T00:27:22.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.562+0000 7fc89d01f640 1 -- 172.21.15.112:0/2214504464 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8980b4f10 con 0x7fc8980cefa0 2024-09-21T00:27:22.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.563+0000 7fc89d01f640 1 -- 172.21.15.112:0/2214504464 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc89806e610 con 0x7fc8980cefa0 2024-09-21T00:27:22.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.564+0000 7fc87f7fe640 1 -- 172.21.15.112:0/2214504464 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc888033490 con 0x7fc8980cefa0 2024-09-21T00:27:22.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.564+0000 7fc87f7fe640 1 --2- 172.21.15.112:0/2214504464 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fc884078b90 0x7fc88407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:22.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.564+0000 7fc87f7fe640 1 -- 172.21.15.112:0/2214504464 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 6101+0+0 (secure 0 0 0) 0x7fc8880bbee0 con 0x7fc8980cefa0 2024-09-21T00:27:22.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.564+0000 7fc89659c640 1 --2- 172.21.15.112:0/2214504464 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fc884078b90 0x7fc88407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:22.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.565+0000 7fc89659c640 1 --2- 172.21.15.112:0/2214504464 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fc884078b90 0x7fc88407b0a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc898070110 tx=0x7fc88c007e90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:22.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.569+0000 7fc87f7fe640 1 -- 172.21.15.112:0/2214504464 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc88804a030 con 0x7fc8980cefa0 2024-09-21T00:27:22.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.782+0000 7fc89d01f640 1 -- 172.21.15.112:0/2214504464 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc89815fb50 con 0x7fc8980cefa0 2024-09-21T00:27:22.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.783+0000 7fc87f7fe640 1 -- 172.21.15.112:0/2214504464 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5229 (secure 0 0 0) 0x7fc88803f030 con 0x7fc8980cefa0 2024-09-21T00:27:22.784 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:27:22.785 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":26,"btime":"2024-09-21T00:27:10:580658+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5158,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6836/2010548530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":2010548530},{"type":"v1","addr":"172.21.15.187:6836","nonce":2010548530}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5161,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6837/2290459931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6835","nonce":2290459931},{"type":"v1","addr":"172.21.15.187:6837","nonce":2290459931}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5400,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6834/158782340","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":158782340},{"type":"v1","addr":"172.21.15.112:6834","nonce":158782340}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T00:27:06.929464+0000","modified":"2024-09-21T00:27:09.945226+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5403},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5403":{"gid":5403,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6835/722271164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":722271164},{"type":"v1","addr":"172.21.15.112:6835","nonce":722271164}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5403,"qdb_cluster":[5403]},"id":3}]} 2024-09-21T00:27:22.785 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 26 2024-09-21T00:27:22.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.785+0000 7fc89d01f640 1 -- 172.21.15.112:0/2214504464 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fc884078b90 msgr2=0x7fc88407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:22.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.785+0000 7fc89d01f640 1 --2- 172.21.15.112:0/2214504464 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fc884078b90 0x7fc88407b0a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc898070110 tx=0x7fc88c007e90 comp rx=0 tx=0).stop 2024-09-21T00:27:22.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.785+0000 7fc89d01f640 1 -- 172.21.15.112:0/2214504464 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc8980cefa0 msgr2=0x7fc89809eca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:22.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.785+0000 7fc89d01f640 1 --2- 172.21.15.112:0/2214504464 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc8980cefa0 0x7fc89809eca0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fc888009950 tx=0x7fc888033c90 comp rx=0 tx=0).stop 2024-09-21T00:27:22.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.785+0000 7fc89d01f640 1 -- 172.21.15.112:0/2214504464 shutdown_connections 2024-09-21T00:27:22.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.785+0000 7fc89d01f640 1 --2- 172.21.15.112:0/2214504464 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc89815f5a0 0x7fc89809a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:22.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.785+0000 7fc89d01f640 1 --2- 172.21.15.112:0/2214504464 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc89815b9f0 0x7fc8981f2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:22.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.785+0000 7fc89d01f640 1 --2- 172.21.15.112:0/2214504464 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fc884078b90 0x7fc88407b0a0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:22.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.785+0000 7fc89d01f640 1 --2- 172.21.15.112:0/2214504464 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc8980cefa0 0x7fc89809eca0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:22.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.785+0000 7fc89d01f640 1 -- 172.21.15.112:0/2214504464 >> 172.21.15.112:0/2214504464 conn(0x7fc8980cca40 msgr2=0x7fc8980afa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:22.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.785+0000 7fc89d01f640 1 -- 172.21.15.112:0/2214504464 shutdown_connections 2024-09-21T00:27:22.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.785+0000 7fc89d01f640 1 -- 172.21.15.112:0/2214504464 wait complete. 2024-09-21T00:27:22.798 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T00:27:22.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.909+0000 7ff2562be640 1 -- 172.21.15.112:0/2297855834 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff238003a50 msgr2=0x7ff238003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:22.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.909+0000 7ff2562be640 1 --2- 172.21.15.112:0/2297855834 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff238003a50 0x7ff238003ec0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7ff2500cec40 tx=0x7ff238005830 comp rx=0 tx=0).stop 2024-09-21T00:27:22.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.909+0000 7ff2562be640 1 -- 172.21.15.112:0/2297855834 shutdown_connections 2024-09-21T00:27:22.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.909+0000 7ff2562be640 1 --2- 172.21.15.112:0/2297855834 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff238003a50 0x7ff238003ec0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:22.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.909+0000 7ff2562be640 1 -- 172.21.15.112:0/2297855834 >> 172.21.15.112:0/2297855834 conn(0x7ff250072990 msgr2=0x7ff250072da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:22.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.909+0000 7ff2562be640 1 -- 172.21.15.112:0/2297855834 shutdown_connections 2024-09-21T00:27:22.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.909+0000 7ff2562be640 1 -- 172.21.15.112:0/2297855834 wait complete. 2024-09-21T00:27:22.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.910+0000 7ff2562be640 1 Processor -- start 2024-09-21T00:27:22.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.911+0000 7ff2562be640 1 -- start start 2024-09-21T00:27:22.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.912+0000 7ff2562be640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff238003a50 0x7ff2501f5b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:22.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.912+0000 7ff2562be640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2500ca730 0x7ff2500cab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:22.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.912+0000 7ff2562be640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff2501f2e90 0x7ff2501f32a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:22.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.912+0000 7ff2562be640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff25015e230 con 0x7ff2500ca730 2024-09-21T00:27:22.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.912+0000 7ff2562be640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff25015a950 con 0x7ff2501f2e90 2024-09-21T00:27:22.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.912+0000 7ff2562be640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff25015e3b0 con 0x7ff238003a50 2024-09-21T00:27:22.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.912+0000 7ff24f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2500ca730 0x7ff2500cab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:22.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.912+0000 7ff24f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2500ca730 0x7ff2500cab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47852/0 (socket says 172.21.15.112:47852) 2024-09-21T00:27:22.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.912+0000 7ff24f7fe640 1 -- 172.21.15.112:0/3308178856 learned_addr learned my addr 172.21.15.112:0/3308178856 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:27:22.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.912+0000 7ff24ffff640 1 --2- 172.21.15.112:0/3308178856 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff238003a50 0x7ff2501f5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:22.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.912+0000 7ff25485b640 1 --2- 172.21.15.112:0/3308178856 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff2501f2e90 0x7ff2501f32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:22.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.912+0000 7ff24f7fe640 1 -- 172.21.15.112:0/3308178856 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff238003a50 msgr2=0x7ff2501f5b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:22.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.912+0000 7ff24f7fe640 1 --2- 172.21.15.112:0/3308178856 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff238003a50 0x7ff2501f5b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:22.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.912+0000 7ff24f7fe640 1 -- 172.21.15.112:0/3308178856 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff2501f2e90 msgr2=0x7ff2501f32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:22.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.912+0000 7ff24f7fe640 1 --2- 172.21.15.112:0/3308178856 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff2501f2e90 0x7ff2501f32a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:22.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.912+0000 7ff24f7fe640 1 -- 172.21.15.112:0/3308178856 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2500cb080 con 0x7ff2500ca730 2024-09-21T00:27:22.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.912+0000 7ff24ffff640 1 --2- 172.21.15.112:0/3308178856 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff238003a50 0x7ff2501f5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:27:22.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.912+0000 7ff24f7fe640 1 --2- 172.21.15.112:0/3308178856 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2500ca730 0x7ff2500cab40 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7ff244004eb0 tx=0x7ff244001630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:22.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.913+0000 7ff24cff9640 1 -- 172.21.15.112:0/3308178856 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff244015020 con 0x7ff2500ca730 2024-09-21T00:27:22.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.913+0000 7ff24cff9640 1 -- 172.21.15.112:0/3308178856 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff244011960 con 0x7ff2500ca730 2024-09-21T00:27:22.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.913+0000 7ff2562be640 1 -- 172.21.15.112:0/3308178856 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2500cc0e0 con 0x7ff2500ca730 2024-09-21T00:27:22.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.913+0000 7ff24cff9640 1 -- 172.21.15.112:0/3308178856 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff244001d40 con 0x7ff2500ca730 2024-09-21T00:27:22.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.913+0000 7ff2562be640 1 -- 172.21.15.112:0/3308178856 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2500cb7b0 con 0x7ff2500ca730 2024-09-21T00:27:22.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.914+0000 7ff2562be640 1 -- 172.21.15.112:0/3308178856 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff25015b380 con 0x7ff2500ca730 2024-09-21T00:27:22.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.914+0000 7ff24cff9640 1 -- 172.21.15.112:0/3308178856 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff244018020 con 0x7ff2500ca730 2024-09-21T00:27:22.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.915+0000 7ff24cff9640 1 --2- 172.21.15.112:0/3308178856 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff23c078b90 0x7ff23c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:22.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.915+0000 7ff24cff9640 1 -- 172.21.15.112:0/3308178856 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 6101+0+0 (secure 0 0 0) 0x7ff24409b850 con 0x7ff2500ca730 2024-09-21T00:27:22.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.915+0000 7ff24ffff640 1 --2- 172.21.15.112:0/3308178856 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff23c078b90 0x7ff23c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:22.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.916+0000 7ff24ffff640 1 --2- 172.21.15.112:0/3308178856 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff23c078b90 0x7ff23c07b0a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff240033000 tx=0x7ff2400019f0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:22.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:22.919+0000 7ff24cff9640 1 -- 172.21.15.112:0/3308178856 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ff24401f020 con 0x7ff2500ca730 2024-09-21T00:27:23.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:23.096+0000 7ff2562be640 1 -- 172.21.15.112:0/3308178856 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff2500cb260 con 0x7ff2500ca730 2024-09-21T00:27:23.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:23.097+0000 7ff24cff9640 1 -- 172.21.15.112:0/3308178856 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) ==== 74+0+17026 (secure 0 0 0) 0x7ff2500cb080 con 0x7ff2500ca730 2024-09-21T00:27:23.099 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:27:23.099 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":49,"fsid":"8b34692b-dc65-49b2-a08f-26f707b96ac5","created":"2024-09-21T00:25:22.788490+0000","modified":"2024-09-21T00:27:08.558791+0000","last_up_change":"2024-09-21T00:25:36.161835+0000","last_in_change":"2024-09-21T00:25:26.346500+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T00:25:37.105006+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-09-21T00:27:04.569521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-09-21T00:27:05.881104+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9c575086-67d2-4de2-b01b-d44dcc21d604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6809","nonce":755760621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6811","nonce":755760621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6815","nonce":755760621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6813","nonce":755760621}]},"public_addr":"172.21.15.112:6809/755760621","cluster_addr":"172.21.15.112:6811/755760621","heartbeat_back_addr":"172.21.15.112:6815/755760621","heartbeat_front_addr":"172.21.15.112:6813/755760621","state":["exists","up"]},{"osd":1,"uuid":"a983c227-a3ce-4bdd-82f1-438ba7e3ecd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6801","nonce":344932592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6803","nonce":344932592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6807","nonce":344932592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6805","nonce":344932592}]},"public_addr":"172.21.15.112:6801/344932592","cluster_addr":"172.21.15.112:6803/344932592","heartbeat_back_addr":"172.21.15.112:6807/344932592","heartbeat_front_addr":"172.21.15.112:6805/344932592","state":["exists","up"]},{"osd":2,"uuid":"1193c1d8-bfff-4a57-8200-5f69bf1ea3af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6825","nonce":407933616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6827","nonce":407933616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6831","nonce":407933616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6829","nonce":407933616}]},"public_addr":"172.21.15.112:6825/407933616","cluster_addr":"172.21.15.112:6827/407933616","heartbeat_back_addr":"172.21.15.112:6831/407933616","heartbeat_front_addr":"172.21.15.112:6829/407933616","state":["exists","up"]},{"osd":3,"uuid":"af04330a-36ad-40cc-a8dd-fa259b221de5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6817","nonce":4120164806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6819","nonce":4120164806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6823","nonce":4120164806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6821","nonce":4120164806}]},"public_addr":"172.21.15.112:6817/4120164806","cluster_addr":"172.21.15.112:6819/4120164806","heartbeat_back_addr":"172.21.15.112:6823/4120164806","heartbeat_front_addr":"172.21.15.112:6821/4120164806","state":["exists","up"]},{"osd":4,"uuid":"5b893111-292c-4c5a-b184-53ad38c7199e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6803","nonce":522077560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6805","nonce":522077560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6809","nonce":522077560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6807","nonce":522077560}]},"public_addr":"172.21.15.187:6803/522077560","cluster_addr":"172.21.15.187:6805/522077560","heartbeat_back_addr":"172.21.15.187:6809/522077560","heartbeat_front_addr":"172.21.15.187:6807/522077560","state":["exists","up"]},{"osd":5,"uuid":"c0b8e011-45fe-4c5e-925e-6be2a7f1868a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6827","nonce":3593144721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6829","nonce":3593144721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6833","nonce":3593144721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6831","nonce":3593144721}]},"public_addr":"172.21.15.187:6827/3593144721","cluster_addr":"172.21.15.187:6829/3593144721","heartbeat_back_addr":"172.21.15.187:6833/3593144721","heartbeat_front_addr":"172.21.15.187:6831/3593144721","state":["exists","up"]},{"osd":6,"uuid":"87d2d650-7489-4593-98a3-a1f433ce8dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6811","nonce":1494387657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6813","nonce":1494387657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6817","nonce":1494387657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6815","nonce":1494387657}]},"public_addr":"172.21.15.187:6811/1494387657","cluster_addr":"172.21.15.187:6813/1494387657","heartbeat_back_addr":"172.21.15.187:6817/1494387657","heartbeat_front_addr":"172.21.15.187:6815/1494387657","state":["exists","up"]},{"osd":7,"uuid":"c6b4f8e2-a1ee-4438-aa37-67ff4b6ef1f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6819","nonce":1624699842}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6821","nonce":1624699842}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6825","nonce":1624699842}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6823","nonce":1624699842}]},"public_addr":"172.21.15.187:6819/1624699842","cluster_addr":"172.21.15.187:6821/1624699842","heartbeat_back_addr":"172.21.15.187:6825/1624699842","heartbeat_front_addr":"172.21.15.187:6823/1624699842","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.873162+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.400506+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.664317+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.159791+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.202630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.113212+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.384786+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.448441+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T00:27:23.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:23.100+0000 7ff2562be640 1 -- 172.21.15.112:0/3308178856 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff23c078b90 msgr2=0x7ff23c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:23.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:23.100+0000 7ff2562be640 1 --2- 172.21.15.112:0/3308178856 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff23c078b90 0x7ff23c07b0a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff240033000 tx=0x7ff2400019f0 comp rx=0 tx=0).stop 2024-09-21T00:27:23.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:23.100+0000 7ff2562be640 1 -- 172.21.15.112:0/3308178856 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2500ca730 msgr2=0x7ff2500cab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:23.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:23.100+0000 7ff2562be640 1 --2- 172.21.15.112:0/3308178856 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2500ca730 0x7ff2500cab40 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7ff244004eb0 tx=0x7ff244001630 comp rx=0 tx=0).stop 2024-09-21T00:27:23.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:23.100+0000 7ff2562be640 1 -- 172.21.15.112:0/3308178856 shutdown_connections 2024-09-21T00:27:23.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:23.100+0000 7ff2562be640 1 --2- 172.21.15.112:0/3308178856 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff2501f2e90 0x7ff2501f32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:23.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:23.100+0000 7ff2562be640 1 --2- 172.21.15.112:0/3308178856 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff23c078b90 0x7ff23c07b0a0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:23.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:23.100+0000 7ff2562be640 1 --2- 172.21.15.112:0/3308178856 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2500ca730 0x7ff2500cab40 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:23.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:23.100+0000 7ff2562be640 1 --2- 172.21.15.112:0/3308178856 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff238003a50 0x7ff2501f5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:23.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:23.100+0000 7ff2562be640 1 -- 172.21.15.112:0/3308178856 >> 172.21.15.112:0/3308178856 conn(0x7ff250072990 msgr2=0x7ff2501593d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:23.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:23.100+0000 7ff2562be640 1 -- 172.21.15.112:0/3308178856 shutdown_connections 2024-09-21T00:27:23.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:23.100+0000 7ff2562be640 1 -- 172.21.15.112:0/3308178856 wait complete. 2024-09-21T00:27:23.113 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T00:27:23.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:23.224+0000 7f3485da1640 1 -- 172.21.15.112:0/1725561683 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34800d0170 msgr2=0x7f34800d0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:23.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:23.224+0000 7f3485da1640 1 --2- 172.21.15.112:0/1725561683 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34800d0170 0x7f34800d0be0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f34740098e0 tx=0x7f347402a4a0 comp rx=0 tx=0).stop 2024-09-21T00:27:23.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:23.224+0000 7f3485da1640 1 -- 172.21.15.112:0/1725561683 shutdown_connections 2024-09-21T00:27:23.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:23.224+0000 7f3485da1640 1 --2- 172.21.15.112:0/1725561683 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34800d0170 0x7f34800d0be0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:23.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:23.224+0000 7f3485da1640 1 -- 172.21.15.112:0/1725561683 >> 172.21.15.112:0/1725561683 conn(0x7f348009d830 msgr2=0x7f34800b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:23.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:23.224+0000 7f3485da1640 1 -- 172.21.15.112:0/1725561683 shutdown_connections 2024-09-21T00:27:23.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:23.224+0000 7f3485da1640 1 -- 172.21.15.112:0/1725561683 wait complete. 2024-09-21T00:27:23.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:23.226+0000 7f3485da1640 1 Processor -- start 2024-09-21T00:27:23.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:23.226+0000 7f3485da1640 1 -- start start 2024-09-21T00:27:23.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:23.227+0000 7f3485da1640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f34800d0170 0x7f34800ca890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:23.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:23.227+0000 7f3485da1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34800d1120 0x7f34800cdf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:23.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:23.227+0000 7f3485da1640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f348015ee80 0x7f34800b1190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:23.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:23.227+0000 7f3485da1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f34800b6910 con 0x7f34800d1120 2024-09-21T00:27:23.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:23.227+0000 7f3485da1640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f34800b6790 con 0x7f348015ee80 2024-09-21T00:27:23.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:23.227+0000 7f3485da1640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f34800b6a90 con 0x7f34800d0170 2024-09-21T00:27:23.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:23.227+0000 7f347effd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34800d1120 0x7f34800cdf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:23.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:23.227+0000 7f347effd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34800d1120 0x7f34800cdf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47872/0 (socket says 172.21.15.112:47872) 2024-09-21T00:27:23.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:23.227+0000 7f347effd640 1 -- 172.21.15.112:0/1969214289 learned_addr learned my addr 172.21.15.112:0/1969214289 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:27:23.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:23.227+0000 7f347f7fe640 1 --2- 172.21.15.112:0/1969214289 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f34800d0170 0x7f34800ca890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:23.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:23.227+0000 7f347ffff640 1 --2- 172.21.15.112:0/1969214289 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f348015ee80 0x7f34800b1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:23.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:23.227+0000 7f347effd640 1 -- 172.21.15.112:0/1969214289 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f34800d0170 msgr2=0x7f34800ca890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:23.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:23.227+0000 7f347effd640 1 --2- 172.21.15.112:0/1969214289 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f34800d0170 0x7f34800ca890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:23.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:23.227+0000 7f347effd640 1 -- 172.21.15.112:0/1969214289 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f348015ee80 msgr2=0x7f34800b1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:23.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:23.228+0000 7f347effd640 1 --2- 172.21.15.112:0/1969214289 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f348015ee80 0x7f34800b1190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:23.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:23.228+0000 7f347effd640 1 -- 172.21.15.112:0/1969214289 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f348006cf80 con 0x7f34800d1120 2024-09-21T00:27:23.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:23.228+0000 7f347ffff640 1 --2- 172.21.15.112:0/1969214289 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f348015ee80 0x7f34800b1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:27:23.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:23.228+0000 7f347effd640 1 --2- 172.21.15.112:0/1969214289 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34800d1120 0x7f34800cdf70 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f347402aa90 tx=0x7f3474004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:23.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:23.228+0000 7f3467fff640 1 -- 172.21.15.112:0/1969214289 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3474031b50 con 0x7f34800d1120 2024-09-21T00:27:23.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:23.228+0000 7f3467fff640 1 -- 172.21.15.112:0/1969214289 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3474004e50 con 0x7f34800d1120 2024-09-21T00:27:23.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:23.228+0000 7f3467fff640 1 -- 172.21.15.112:0/1969214289 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3474033310 con 0x7f34800d1120 2024-09-21T00:27:23.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:23.228+0000 7f3485da1640 1 -- 172.21.15.112:0/1969214289 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f348006d220 con 0x7f34800d1120 2024-09-21T00:27:23.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:23.228+0000 7f3485da1640 1 -- 172.21.15.112:0/1969214289 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34800b1a70 con 0x7f34800d1120 2024-09-21T00:27:23.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:23.229+0000 7f3485da1640 1 -- 172.21.15.112:0/1969214289 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f348006fa00 con 0x7f34800d1120 2024-09-21T00:27:23.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:23.230+0000 7f3467fff640 1 -- 172.21.15.112:0/1969214289 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f34740334b0 con 0x7f34800d1120 2024-09-21T00:27:23.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:23.230+0000 7f3467fff640 1 --2- 172.21.15.112:0/1969214289 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f346c078bc0 0x7f346c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:23.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:23.230+0000 7f3467fff640 1 -- 172.21.15.112:0/1969214289 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 6101+0+0 (secure 0 0 0) 0x7f34740bc240 con 0x7f34800d1120 2024-09-21T00:27:23.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:23.231+0000 7f347f7fe640 1 --2- 172.21.15.112:0/1969214289 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f346c078bc0 0x7f346c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:23.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:23.231+0000 7f347f7fe640 1 --2- 172.21.15.112:0/1969214289 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f346c078bc0 0x7f346c07b0d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f3470006fd0 tx=0x7f3470009060 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:23.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:23.235+0000 7f3467fff640 1 -- 172.21.15.112:0/1969214289 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f347403f030 con 0x7f34800d1120 2024-09-21T00:27:23.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:23.450+0000 7f3485da1640 1 -- 172.21.15.112:0/1969214289 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f34800cd8d0 con 0x7f34800d1120 2024-09-21T00:27:23.452 INFO:tasks.ceph.mon.a.smithi112.stderr:2024-09-21T00:27:23.451+0000 7f7656c13640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T00:27:23.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:23.452+0000 7f3467fff640 1 -- 172.21.15.112:0/1969214289 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v27) ==== 161+0+0 (secure 0 0 0) 0x7f3474084580 con 0x7f34800d1120 2024-09-21T00:27:23.454 INFO:teuthology.orchestra.run.smithi112.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T00:27:23.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:23.455+0000 7f3485da1640 1 -- 172.21.15.112:0/1969214289 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f346c078bc0 msgr2=0x7f346c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:23.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:23.455+0000 7f3485da1640 1 --2- 172.21.15.112:0/1969214289 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f346c078bc0 0x7f346c07b0d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f3470006fd0 tx=0x7f3470009060 comp rx=0 tx=0).stop 2024-09-21T00:27:23.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:23.455+0000 7f3485da1640 1 -- 172.21.15.112:0/1969214289 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34800d1120 msgr2=0x7f34800cdf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:23.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:23.455+0000 7f3485da1640 1 --2- 172.21.15.112:0/1969214289 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34800d1120 0x7f34800cdf70 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f347402aa90 tx=0x7f3474004a50 comp rx=0 tx=0).stop 2024-09-21T00:27:23.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:23.455+0000 7f3485da1640 1 -- 172.21.15.112:0/1969214289 shutdown_connections 2024-09-21T00:27:23.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:23.455+0000 7f3485da1640 1 --2- 172.21.15.112:0/1969214289 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f348015ee80 0x7f34800b1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:23.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:23.455+0000 7f3485da1640 1 --2- 172.21.15.112:0/1969214289 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f346c078bc0 0x7f346c07b0d0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:23.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:23.455+0000 7f3485da1640 1 --2- 172.21.15.112:0/1969214289 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34800d1120 0x7f34800cdf70 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:23.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:23.455+0000 7f3485da1640 1 --2- 172.21.15.112:0/1969214289 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f34800d0170 0x7f34800ca890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:23.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:23.455+0000 7f3485da1640 1 -- 172.21.15.112:0/1969214289 >> 172.21.15.112:0/1969214289 conn(0x7f348009d830 msgr2=0x7f3480157440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:23.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:23.455+0000 7f3485da1640 1 -- 172.21.15.112:0/1969214289 shutdown_connections 2024-09-21T00:27:23.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:23.455+0000 7f3485da1640 1 -- 172.21.15.112:0/1969214289 wait complete. 2024-09-21T00:27:23.458 INFO:tasks.ceph.mds.a.smithi112.stderr: -2252> 2024-09-21T00:27:09.548+0000 7f53a341a640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T00:27:23.469 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-21T00:27:23.510 INFO:tasks.ceph.mds.a.smithi112.stdout:starting mds.a at 2024-09-21T00:27:23.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:23.585+0000 7fbd478f9640 1 -- 172.21.15.112:0/3150309202 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbd400cd440 msgr2=0x7fbd400d0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:23.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:23.585+0000 7fbd478f9640 1 --2- 172.21.15.112:0/3150309202 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbd400cd440 0x7fbd400d0670 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fbd400d2e30 tx=0x7fbd3002a4d0 comp rx=0 tx=0).stop 2024-09-21T00:27:23.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:23.585+0000 7fbd478f9640 1 -- 172.21.15.112:0/3150309202 shutdown_connections 2024-09-21T00:27:23.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:23.585+0000 7fbd478f9640 1 --2- 172.21.15.112:0/3150309202 >> v2:172.21.15.187:3300/0 conn(0x7fbd400d0bb0 0x7fbd40070100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:23.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:23.585+0000 7fbd478f9640 1 --2- 172.21.15.112:0/3150309202 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbd400cd440 0x7fbd400d0670 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:23.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:23.585+0000 7fbd478f9640 1 --2- 172.21.15.112:0/3150309202 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbd400ccc40 0x7fbd4015ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:23.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:23.585+0000 7fbd478f9640 1 -- 172.21.15.112:0/3150309202 >> 172.21.15.112:0/3150309202 conn(0x7fbd400b46e0 msgr2=0x7fbd400d23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:23.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:23.585+0000 7fbd478f9640 1 -- 172.21.15.112:0/3150309202 shutdown_connections 2024-09-21T00:27:23.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:23.585+0000 7fbd478f9640 1 -- 172.21.15.112:0/3150309202 wait complete. 2024-09-21T00:27:23.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:23.587+0000 7fbd478f9640 1 Processor -- start 2024-09-21T00:27:23.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:23.587+0000 7fbd478f9640 1 -- start start 2024-09-21T00:27:23.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:23.589+0000 7fbd478f9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbd400ccc40 0x7fbd400cc330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:23.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:23.589+0000 7fbd478f9640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbd400cd440 0x7fbd40153790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:23.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:23.589+0000 7fbd478f9640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbd400d0bb0 0x7fbd400ab5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:23.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:23.589+0000 7fbd478f9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbd40070a10 con 0x7fbd400ccc40 2024-09-21T00:27:23.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:23.589+0000 7fbd478f9640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fbd40070890 con 0x7fbd400d0bb0 2024-09-21T00:27:23.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:23.589+0000 7fbd478f9640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fbd40070b90 con 0x7fbd400cd440 2024-09-21T00:27:23.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:23.589+0000 7fbd45695640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbd400ccc40 0x7fbd400cc330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:23.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:23.589+0000 7fbd45695640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbd400ccc40 0x7fbd400cc330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47908/0 (socket says 172.21.15.112:47908) 2024-09-21T00:27:23.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:23.589+0000 7fbd45695640 1 -- 172.21.15.112:0/2747768625 learned_addr learned my addr 172.21.15.112:0/2747768625 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:27:23.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:23.589+0000 7fbd44e94640 1 --2- 172.21.15.112:0/2747768625 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbd400cd440 0x7fbd40153790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:23.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:23.589+0000 7fbd45e96640 1 --2- 172.21.15.112:0/2747768625 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbd400d0bb0 0x7fbd400ab5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:23.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:23.589+0000 7fbd45695640 1 -- 172.21.15.112:0/2747768625 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbd400cd440 msgr2=0x7fbd40153790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:23.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:23.589+0000 7fbd45695640 1 --2- 172.21.15.112:0/2747768625 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbd400cd440 0x7fbd40153790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:23.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:23.589+0000 7fbd45695640 1 -- 172.21.15.112:0/2747768625 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbd400d0bb0 msgr2=0x7fbd400ab5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:23.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:23.589+0000 7fbd45695640 1 --2- 172.21.15.112:0/2747768625 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbd400d0bb0 0x7fbd400ab5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:23.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:23.589+0000 7fbd45695640 1 -- 172.21.15.112:0/2747768625 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd400c97e0 con 0x7fbd400ccc40 2024-09-21T00:27:23.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:23.589+0000 7fbd45695640 1 --2- 172.21.15.112:0/2747768625 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbd400ccc40 0x7fbd400cc330 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c00e7c0 tx=0x7fbd3c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:23.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:23.589+0000 7fbd35ffb640 1 -- 172.21.15.112:0/2747768625 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd3c00cc60 con 0x7fbd400ccc40 2024-09-21T00:27:23.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:23.589+0000 7fbd35ffb640 1 -- 172.21.15.112:0/2747768625 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbd3c00ce00 con 0x7fbd400ccc40 2024-09-21T00:27:23.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:23.589+0000 7fbd35ffb640 1 -- 172.21.15.112:0/2747768625 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd3c005280 con 0x7fbd400ccc40 2024-09-21T00:27:23.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:23.590+0000 7fbd478f9640 1 -- 172.21.15.112:0/2747768625 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd400aa4f0 con 0x7fbd400ccc40 2024-09-21T00:27:23.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:23.590+0000 7fbd478f9640 1 -- 172.21.15.112:0/2747768625 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd400aaab0 con 0x7fbd400ccc40 2024-09-21T00:27:23.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:23.591+0000 7fbd478f9640 1 -- 172.21.15.112:0/2747768625 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd400d0670 con 0x7fbd400ccc40 2024-09-21T00:27:23.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:23.591+0000 7fbd35ffb640 1 -- 172.21.15.112:0/2747768625 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fbd3c005420 con 0x7fbd400ccc40 2024-09-21T00:27:23.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:23.592+0000 7fbd35ffb640 1 --2- 172.21.15.112:0/2747768625 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fbd24078b90 0x7fbd2407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:23.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:23.592+0000 7fbd35ffb640 1 -- 172.21.15.112:0/2747768625 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 6187+0+0 (secure 0 0 0) 0x7fbd3c09b9b0 con 0x7fbd400ccc40 2024-09-21T00:27:23.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:23.592+0000 7fbd44e94640 1 --2- 172.21.15.112:0/2747768625 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fbd24078b90 0x7fbd2407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:23.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:23.593+0000 7fbd44e94640 1 --2- 172.21.15.112:0/2747768625 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fbd24078b90 0x7fbd2407b0a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fbd30033000 tx=0x7fbd30004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:23.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:23.596+0000 7fbd35ffb640 1 -- 172.21.15.112:0/2747768625 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fbd3c063c10 con 0x7fbd400ccc40 2024-09-21T00:27:23.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:23.811+0000 7fbd478f9640 1 -- 172.21.15.112:0/2747768625 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fbd400b3650 con 0x7fbd400ccc40 2024-09-21T00:27:24.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:24.453+0000 7fbd35ffb640 1 -- 172.21.15.112:0/2747768625 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v28) ==== 104+0+0 (secure 0 0 0) 0x7fbd3c063510 con 0x7fbd400ccc40 2024-09-21T00:27:24.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:24.456+0000 7fbd478f9640 1 -- 172.21.15.112:0/2747768625 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fbd24078b90 msgr2=0x7fbd2407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:24.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:24.456+0000 7fbd478f9640 1 --2- 172.21.15.112:0/2747768625 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fbd24078b90 0x7fbd2407b0a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fbd30033000 tx=0x7fbd30004730 comp rx=0 tx=0).stop 2024-09-21T00:27:24.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:24.456+0000 7fbd478f9640 1 -- 172.21.15.112:0/2747768625 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbd400ccc40 msgr2=0x7fbd400cc330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:24.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:24.456+0000 7fbd478f9640 1 --2- 172.21.15.112:0/2747768625 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbd400ccc40 0x7fbd400cc330 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c00e7c0 tx=0x7fbd3c00ec80 comp rx=0 tx=0).stop 2024-09-21T00:27:24.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:24.456+0000 7fbd478f9640 1 -- 172.21.15.112:0/2747768625 shutdown_connections 2024-09-21T00:27:24.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:24.456+0000 7fbd478f9640 1 --2- 172.21.15.112:0/2747768625 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbd400d0bb0 0x7fbd400ab5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:24.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:24.456+0000 7fbd478f9640 1 --2- 172.21.15.112:0/2747768625 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbd400cd440 0x7fbd40153790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:24.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:24.456+0000 7fbd478f9640 1 --2- 172.21.15.112:0/2747768625 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fbd24078b90 0x7fbd2407b0a0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:24.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:24.456+0000 7fbd478f9640 1 --2- 172.21.15.112:0/2747768625 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbd400ccc40 0x7fbd400cc330 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:24.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:24.456+0000 7fbd478f9640 1 -- 172.21.15.112:0/2747768625 >> 172.21.15.112:0/2747768625 conn(0x7fbd400b46e0 msgr2=0x7fbd4015c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:24.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:24.456+0000 7fbd478f9640 1 -- 172.21.15.112:0/2747768625 shutdown_connections 2024-09-21T00:27:24.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:24.456+0000 7fbd478f9640 1 -- 172.21.15.112:0/2747768625 wait complete. 2024-09-21T00:27:24.469 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-21T00:27:24.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:24.626+0000 7fdbe972b640 1 -- 172.21.15.112:0/2355650117 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdbcc003a00 msgr2=0x7fdbcc003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:24.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:24.626+0000 7fdbe972b640 1 --2- 172.21.15.112:0/2355650117 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdbcc003a00 0x7fdbcc003e70 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fdbd4009b40 tx=0x7fdbcc005830 comp rx=0 tx=0).stop 2024-09-21T00:27:24.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:24.626+0000 7fdbe37fe640 1 -- 172.21.15.112:0/2355650117 reap_dead start 2024-09-21T00:27:24.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:24.626+0000 7fdbe972b640 1 -- 172.21.15.112:0/2355650117 shutdown_connections 2024-09-21T00:27:24.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:24.626+0000 7fdbe972b640 1 -- 172.21.15.112:0/2355650117 >> 172.21.15.112:0/2355650117 conn(0x7fdbe409d800 msgr2=0x7fdbe40b94c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:24.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:24.626+0000 7fdbe972b640 1 -- 172.21.15.112:0/2355650117 shutdown_connections 2024-09-21T00:27:24.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:24.626+0000 7fdbe972b640 1 -- 172.21.15.112:0/2355650117 wait complete. 2024-09-21T00:27:24.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:24.628+0000 7fdbe972b640 1 Processor -- start 2024-09-21T00:27:24.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:24.628+0000 7fdbe972b640 1 -- start start 2024-09-21T00:27:24.632 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:24.629+0000 7fdbe972b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdbcc003a00 0x7fdbe40b25f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:24.632 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:24.629+0000 7fdbe972b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdbe40d1390 0x7fdbe406cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:24.632 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:24.629+0000 7fdbe972b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdbe40d17a0 0x7fdbe406d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:24.632 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:24.629+0000 7fdbe972b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fdbe40b39a0 con 0x7fdbe40d1390 2024-09-21T00:27:24.632 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:24.629+0000 7fdbe972b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fdbe40d3b20 con 0x7fdbcc003a00 2024-09-21T00:27:24.632 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:24.629+0000 7fdbe972b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fdbe40b3b20 con 0x7fdbe40d17a0 2024-09-21T00:27:24.632 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:24.629+0000 7fdbe27fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdbe40d1390 0x7fdbe406cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:24.632 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:24.629+0000 7fdbe27fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdbe40d1390 0x7fdbe406cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47924/0 (socket says 172.21.15.112:47924) 2024-09-21T00:27:24.632 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:24.629+0000 7fdbe27fc640 1 -- 172.21.15.112:0/789708667 learned_addr learned my addr 172.21.15.112:0/789708667 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:27:24.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:24.630+0000 7fdbe2ffd640 1 --2- 172.21.15.112:0/789708667 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdbcc003a00 0x7fdbe40b25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:24.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:24.630+0000 7fdbe37fe640 1 --2- 172.21.15.112:0/789708667 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdbe40d17a0 0x7fdbe406d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:24.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:24.630+0000 7fdbe27fc640 1 -- 172.21.15.112:0/789708667 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdbe40d17a0 msgr2=0x7fdbe406d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:24.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:24.630+0000 7fdbe27fc640 1 --2- 172.21.15.112:0/789708667 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdbe40d17a0 0x7fdbe406d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:24.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:24.630+0000 7fdbe27fc640 1 -- 172.21.15.112:0/789708667 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdbcc003a00 msgr2=0x7fdbe40b25f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:24.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:24.630+0000 7fdbe27fc640 1 --2- 172.21.15.112:0/789708667 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdbcc003a00 0x7fdbe40b25f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:24.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:24.630+0000 7fdbe27fc640 1 -- 172.21.15.112:0/789708667 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbe406fc00 con 0x7fdbe40d1390 2024-09-21T00:27:24.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:24.630+0000 7fdbe2ffd640 1 --2- 172.21.15.112:0/789708667 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdbcc003a00 0x7fdbe40b25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:27:24.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:24.630+0000 7fdbe37fe640 1 --2- 172.21.15.112:0/789708667 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdbe40d17a0 0x7fdbe406d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:27:24.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:24.630+0000 7fdbe27fc640 1 --2- 172.21.15.112:0/789708667 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdbe40d1390 0x7fdbe406cde0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fdbd800e600 tx=0x7fdbd800eac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:24.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:24.630+0000 7fdbcb7fe640 1 -- 172.21.15.112:0/789708667 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbd8015070 con 0x7fdbe40d1390 2024-09-21T00:27:24.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:24.630+0000 7fdbcb7fe640 1 -- 172.21.15.112:0/789708667 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdbd8004310 con 0x7fdbe40d1390 2024-09-21T00:27:24.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:24.630+0000 7fdbe972b640 1 -- 172.21.15.112:0/789708667 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbe41f5950 con 0x7fdbe40d1390 2024-09-21T00:27:24.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:24.630+0000 7fdbcb7fe640 1 -- 172.21.15.112:0/789708667 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbd800ca00 con 0x7fdbe40d1390 2024-09-21T00:27:24.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:24.630+0000 7fdbe972b640 1 -- 172.21.15.112:0/789708667 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbe41f5f10 con 0x7fdbe40d1390 2024-09-21T00:27:24.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:24.631+0000 7fdbe972b640 1 -- 172.21.15.112:0/789708667 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbe40b64a0 con 0x7fdbe40d1390 2024-09-21T00:27:24.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:24.632+0000 7fdbcb7fe640 1 -- 172.21.15.112:0/789708667 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fdbd800cba0 con 0x7fdbe40d1390 2024-09-21T00:27:24.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:24.632+0000 7fdbcb7fe640 1 --2- 172.21.15.112:0/789708667 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fdbd0078b90 0x7fdbd007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:24.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:24.632+0000 7fdbcb7fe640 1 -- 172.21.15.112:0/789708667 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 6187+0+0 (secure 0 0 0) 0x7fdbd809b560 con 0x7fdbe40d1390 2024-09-21T00:27:24.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:24.632+0000 7fdbe2ffd640 1 --2- 172.21.15.112:0/789708667 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fdbd0078b90 0x7fdbd007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:24.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:24.633+0000 7fdbe2ffd640 1 --2- 172.21.15.112:0/789708667 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fdbd0078b90 0x7fdbd007b0a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fdbd4035000 tx=0x7fdbd40046c0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:24.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:24.637+0000 7fdbcb7fe640 1 -- 172.21.15.112:0/789708667 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fdbd80636a0 con 0x7fdbe40d1390 2024-09-21T00:27:24.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:24.814+0000 7fdbe972b640 1 -- 172.21.15.112:0/789708667 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fdbe4159060 con 0x7fdbe40d1390 2024-09-21T00:27:25.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:25.455+0000 7fdbcb7fe640 1 -- 172.21.15.112:0/789708667 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v51) ==== 181+0+0 (secure 0 0 0) 0x7fdbd8062fa0 con 0x7fdbe40d1390 2024-09-21T00:27:25.464 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'cephfs_metadata' removed 2024-09-21T00:27:25.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:25.458+0000 7fdbe972b640 1 -- 172.21.15.112:0/789708667 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fdbd0078b90 msgr2=0x7fdbd007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:25.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:25.458+0000 7fdbe972b640 1 --2- 172.21.15.112:0/789708667 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fdbd0078b90 0x7fdbd007b0a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fdbd4035000 tx=0x7fdbd40046c0 comp rx=0 tx=0).stop 2024-09-21T00:27:25.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:25.458+0000 7fdbe972b640 1 -- 172.21.15.112:0/789708667 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdbe40d1390 msgr2=0x7fdbe406cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:25.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:25.458+0000 7fdbe972b640 1 --2- 172.21.15.112:0/789708667 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdbe40d1390 0x7fdbe406cde0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fdbd800e600 tx=0x7fdbd800eac0 comp rx=0 tx=0).stop 2024-09-21T00:27:25.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:25.459+0000 7fdbe972b640 1 -- 172.21.15.112:0/789708667 shutdown_connections 2024-09-21T00:27:25.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:25.459+0000 7fdbe972b640 1 --2- 172.21.15.112:0/789708667 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdbe40d17a0 0x7fdbe406d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:25.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:25.459+0000 7fdbe972b640 1 --2- 172.21.15.112:0/789708667 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fdbd0078b90 0x7fdbd007b0a0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:25.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:25.459+0000 7fdbe972b640 1 --2- 172.21.15.112:0/789708667 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdbe40d1390 0x7fdbe406cde0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:25.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:25.459+0000 7fdbe972b640 1 --2- 172.21.15.112:0/789708667 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdbcc003a00 0x7fdbe40b25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:25.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:25.459+0000 7fdbe972b640 1 -- 172.21.15.112:0/789708667 >> 172.21.15.112:0/789708667 conn(0x7fdbe409d800 msgr2=0x7fdbe41eafc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:25.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:25.459+0000 7fdbe972b640 1 -- 172.21.15.112:0/789708667 shutdown_connections 2024-09-21T00:27:25.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:25.459+0000 7fdbe972b640 1 -- 172.21.15.112:0/789708667 wait complete. 2024-09-21T00:27:25.474 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-21T00:27:25.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:25.602+0000 7fd23ad13640 1 -- 172.21.15.112:0/1256065151 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd234070940 msgr2=0x7fd2340ce8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:25.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:25.602+0000 7fd23ad13640 1 --2- 172.21.15.112:0/1256065151 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd234070940 0x7fd2340ce8a0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fd234071e00 tx=0x7fd22802a560 comp rx=0 tx=0).stop 2024-09-21T00:27:25.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:25.602+0000 7fd23ad13640 1 -- 172.21.15.112:0/1256065151 shutdown_connections 2024-09-21T00:27:25.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:25.602+0000 7fd23ad13640 1 --2- 172.21.15.112:0/1256065151 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd234070940 0x7fd2340ce8a0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:25.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:25.602+0000 7fd23ad13640 1 --2- 172.21.15.112:0/1256065151 >> v2:172.21.15.187:3300/0 conn(0x7fd2340cd790 0x7fd2340cbd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:25.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:25.602+0000 7fd23ad13640 1 -- 172.21.15.112:0/1256065151 >> 172.21.15.112:0/1256065151 conn(0x7fd2340ccac0 msgr2=0x7fd2340cced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:25.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:25.603+0000 7fd23ad13640 1 -- 172.21.15.112:0/1256065151 shutdown_connections 2024-09-21T00:27:25.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:25.603+0000 7fd23ad13640 1 -- 172.21.15.112:0/1256065151 wait complete. 2024-09-21T00:27:25.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:25.604+0000 7fd23ad13640 1 Processor -- start 2024-09-21T00:27:25.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:25.604+0000 7fd23ad13640 1 -- start start 2024-09-21T00:27:25.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:25.606+0000 7fd23ad13640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd23406def0 0x7fd2340c6780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:25.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:25.606+0000 7fd23ad13640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd234070940 0x7fd2341f3140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:25.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:25.606+0000 7fd23ad13640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd2340cd790 0x7fd23409d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:25.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:25.606+0000 7fd23ad13640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd23406f610 con 0x7fd23406def0 2024-09-21T00:27:25.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:25.606+0000 7fd23ad13640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd23406f490 con 0x7fd2340cd790 2024-09-21T00:27:25.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:25.606+0000 7fd23ad13640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd23406f790 con 0x7fd234070940 2024-09-21T00:27:25.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:25.606+0000 7fd238aaf640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd23406def0 0x7fd2340c6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:25.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:25.606+0000 7fd238aaf640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd23406def0 0x7fd2340c6780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47954/0 (socket says 172.21.15.112:47954) 2024-09-21T00:27:25.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:25.606+0000 7fd238aaf640 1 -- 172.21.15.112:0/3930047137 learned_addr learned my addr 172.21.15.112:0/3930047137 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:27:25.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:25.606+0000 7fd238aaf640 1 -- 172.21.15.112:0/3930047137 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd234070940 msgr2=0x7fd2341f3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:25.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:25.606+0000 7fd238aaf640 1 --2- 172.21.15.112:0/3930047137 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd234070940 0x7fd2341f3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:25.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:25.606+0000 7fd238aaf640 1 -- 172.21.15.112:0/3930047137 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd2340cd790 msgr2=0x7fd23409d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:25.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:25.606+0000 7fd238aaf640 1 --2- 172.21.15.112:0/3930047137 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd2340cd790 0x7fd23409d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:25.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:25.606+0000 7fd238aaf640 1 -- 172.21.15.112:0/3930047137 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2341f3680 con 0x7fd23406def0 2024-09-21T00:27:25.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:25.606+0000 7fd238aaf640 1 --2- 172.21.15.112:0/3930047137 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd23406def0 0x7fd2340c6780 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fd22400b440 tx=0x7fd22400b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:25.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:25.606+0000 7fd2317fa640 1 -- 172.21.15.112:0/3930047137 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd224015020 con 0x7fd23406def0 2024-09-21T00:27:25.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:25.606+0000 7fd2317fa640 1 -- 172.21.15.112:0/3930047137 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd224011980 con 0x7fd23406def0 2024-09-21T00:27:25.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:25.606+0000 7fd23ad13640 1 -- 172.21.15.112:0/3930047137 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2340c8480 con 0x7fd23406def0 2024-09-21T00:27:25.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:25.606+0000 7fd2317fa640 1 -- 172.21.15.112:0/3930047137 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd224011dd0 con 0x7fd23406def0 2024-09-21T00:27:25.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:25.607+0000 7fd23ad13640 1 -- 172.21.15.112:0/3930047137 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2340c5f80 con 0x7fd23406def0 2024-09-21T00:27:25.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:25.607+0000 7fd23ad13640 1 -- 172.21.15.112:0/3930047137 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2340ce8a0 con 0x7fd23406def0 2024-09-21T00:27:25.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:25.608+0000 7fd2317fa640 1 -- 172.21.15.112:0/3930047137 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd224018020 con 0x7fd23406def0 2024-09-21T00:27:25.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:25.609+0000 7fd2317fa640 1 --2- 172.21.15.112:0/3930047137 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fd220078bc0 0x7fd22007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:25.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:25.609+0000 7fd2317fa640 1 -- 172.21.15.112:0/3930047137 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 5749+0+0 (secure 0 0 0) 0x7fd22409af00 con 0x7fd23406def0 2024-09-21T00:27:25.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:25.609+0000 7fd233fff640 1 --2- 172.21.15.112:0/3930047137 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fd220078bc0 0x7fd22007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:25.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:25.610+0000 7fd233fff640 1 --2- 172.21.15.112:0/3930047137 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fd220078bc0 0x7fd22007b0d0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fd228033000 tx=0x7fd2280046c0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:25.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:25.614+0000 7fd2317fa640 1 -- 172.21.15.112:0/3930047137 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd22401f020 con 0x7fd23406def0 2024-09-21T00:27:25.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:25.791+0000 7fd23ad13640 1 -- 172.21.15.112:0/3930047137 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fd2340b5f20 con 0x7fd23406def0 2024-09-21T00:27:26.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:26.460+0000 7fd2317fa640 1 -- 172.21.15.112:0/3930047137 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v52) ==== 169+0+0 (secure 0 0 0) 0x7fd2341f3680 con 0x7fd23406def0 2024-09-21T00:27:26.462 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'cephfs_data' removed 2024-09-21T00:27:26.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:26.464+0000 7fd23ad13640 1 -- 172.21.15.112:0/3930047137 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fd220078bc0 msgr2=0x7fd22007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:26.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:26.464+0000 7fd23ad13640 1 --2- 172.21.15.112:0/3930047137 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fd220078bc0 0x7fd22007b0d0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fd228033000 tx=0x7fd2280046c0 comp rx=0 tx=0).stop 2024-09-21T00:27:26.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:26.464+0000 7fd23ad13640 1 -- 172.21.15.112:0/3930047137 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd23406def0 msgr2=0x7fd2340c6780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:26.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:26.464+0000 7fd23ad13640 1 --2- 172.21.15.112:0/3930047137 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd23406def0 0x7fd2340c6780 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fd22400b440 tx=0x7fd22400b900 comp rx=0 tx=0).stop 2024-09-21T00:27:26.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:26.464+0000 7fd23ad13640 1 -- 172.21.15.112:0/3930047137 shutdown_connections 2024-09-21T00:27:26.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:26.464+0000 7fd23ad13640 1 --2- 172.21.15.112:0/3930047137 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd2340cd790 0x7fd23409d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:26.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:26.464+0000 7fd23ad13640 1 --2- 172.21.15.112:0/3930047137 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd234070940 0x7fd2341f3140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:26.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:26.464+0000 7fd23ad13640 1 --2- 172.21.15.112:0/3930047137 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fd220078bc0 0x7fd22007b0d0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:26.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:26.464+0000 7fd23ad13640 1 --2- 172.21.15.112:0/3930047137 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd23406def0 0x7fd2340c6780 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:26.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:26.464+0000 7fd23ad13640 1 -- 172.21.15.112:0/3930047137 >> 172.21.15.112:0/3930047137 conn(0x7fd2340ccac0 msgr2=0x7fd2340b4f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:26.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:26.464+0000 7fd23ad13640 1 -- 172.21.15.112:0/3930047137 shutdown_connections 2024-09-21T00:27:26.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:26.464+0000 7fd23ad13640 1 -- 172.21.15.112:0/3930047137 wait complete. 2024-09-21T00:27:26.478 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 31 2024-09-21T00:27:26.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:26.588+0000 7f04be80e640 1 -- 172.21.15.112:0/2835961161 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04b80d0d80 msgr2=0x7f04b8167550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:26.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:26.588+0000 7f04be80e640 1 --2- 172.21.15.112:0/2835961161 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04b80d0d80 0x7f04b8167550 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f04b0005fd0 tx=0x7f04b002a8b0 comp rx=0 tx=0).stop 2024-09-21T00:27:26.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:26.588+0000 7f04be80e640 1 -- 172.21.15.112:0/2835961161 shutdown_connections 2024-09-21T00:27:26.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:26.588+0000 7f04be80e640 1 --2- 172.21.15.112:0/2835961161 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04b80d0d80 0x7f04b8167550 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:26.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:26.588+0000 7f04be80e640 1 -- 172.21.15.112:0/2835961161 >> 172.21.15.112:0/2835961161 conn(0x7f04b80b46e0 msgr2=0x7f04b80d23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:26.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:26.588+0000 7f04be80e640 1 -- 172.21.15.112:0/2835961161 shutdown_connections 2024-09-21T00:27:26.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:26.588+0000 7f04be80e640 1 -- 172.21.15.112:0/2835961161 wait complete. 2024-09-21T00:27:26.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:26.590+0000 7f04be80e640 1 Processor -- start 2024-09-21T00:27:26.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:26.590+0000 7f04be80e640 1 -- start start 2024-09-21T00:27:26.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:26.591+0000 7f04be80e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04b80ccc40 0x7f04b81556a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:26.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:26.592+0000 7f04be80e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04b815ca90 0x7f04b806ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:26.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:26.592+0000 7f04be80e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04b8151870 0x7f04b806f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:26.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:26.592+0000 7f04be80e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f04b8167e20 con 0x7f04b815ca90 2024-09-21T00:27:26.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:26.592+0000 7f04be80e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f04b8167ca0 con 0x7f04b8151870 2024-09-21T00:27:26.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:26.592+0000 7f04be80e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f04b8167fa0 con 0x7f04b80ccc40 2024-09-21T00:27:26.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:26.592+0000 7f04b77fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04b815ca90 0x7f04b806ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:26.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:26.592+0000 7f04b77fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04b815ca90 0x7f04b806ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58346/0 (socket says 172.21.15.112:58346) 2024-09-21T00:27:26.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:26.592+0000 7f04b77fe640 1 -- 172.21.15.112:0/1123540975 learned_addr learned my addr 172.21.15.112:0/1123540975 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:27:26.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:26.592+0000 7f04bcdab640 1 --2- 172.21.15.112:0/1123540975 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04b8151870 0x7f04b806f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:26.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:26.592+0000 7f04b7fff640 1 --2- 172.21.15.112:0/1123540975 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04b80ccc40 0x7f04b81556a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:26.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:26.592+0000 7f04b77fe640 1 -- 172.21.15.112:0/1123540975 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04b80ccc40 msgr2=0x7f04b81556a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:26.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:26.592+0000 7f04b77fe640 1 --2- 172.21.15.112:0/1123540975 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04b80ccc40 0x7f04b81556a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:26.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:26.592+0000 7f04b77fe640 1 -- 172.21.15.112:0/1123540975 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04b8151870 msgr2=0x7f04b806f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:26.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:26.592+0000 7f04b77fe640 1 --2- 172.21.15.112:0/1123540975 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04b8151870 0x7f04b806f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:26.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:26.592+0000 7f04b77fe640 1 -- 172.21.15.112:0/1123540975 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04b8167140 con 0x7f04b815ca90 2024-09-21T00:27:26.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:26.592+0000 7f04b7fff640 1 --2- 172.21.15.112:0/1123540975 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04b80ccc40 0x7f04b81556a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:27:26.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:26.592+0000 7f04b77fe640 1 --2- 172.21.15.112:0/1123540975 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04b815ca90 0x7f04b806ec20 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f04ac00c370 tx=0x7f04ac00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:26.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:26.592+0000 7f04b4ff9640 1 -- 172.21.15.112:0/1123540975 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04ac017070 con 0x7f04b815ca90 2024-09-21T00:27:26.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:26.592+0000 7f04b4ff9640 1 -- 172.21.15.112:0/1123540975 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f04ac0053c0 con 0x7f04b815ca90 2024-09-21T00:27:26.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:26.592+0000 7f04be80e640 1 -- 172.21.15.112:0/1123540975 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04b8165340 con 0x7f04b815ca90 2024-09-21T00:27:26.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:26.592+0000 7f04b4ff9640 1 -- 172.21.15.112:0/1123540975 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04ac003850 con 0x7f04b815ca90 2024-09-21T00:27:26.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:26.592+0000 7f04be80e640 1 -- 172.21.15.112:0/1123540975 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04b8165900 con 0x7f04b815ca90 2024-09-21T00:27:26.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:26.593+0000 7f04be80e640 1 -- 172.21.15.112:0/1123540975 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04b8071350 con 0x7f04b815ca90 2024-09-21T00:27:26.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:26.594+0000 7f04b4ff9640 1 -- 172.21.15.112:0/1123540975 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f04ac01a070 con 0x7f04b815ca90 2024-09-21T00:27:26.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:26.594+0000 7f04b4ff9640 1 --2- 172.21.15.112:0/1123540975 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f04a4078bc0 0x7f04a407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:26.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:26.595+0000 7f04b4ff9640 1 -- 172.21.15.112:0/1123540975 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5335+0+0 (secure 0 0 0) 0x7f04ac099ec0 con 0x7f04b815ca90 2024-09-21T00:27:26.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:26.595+0000 7f04b7fff640 1 --2- 172.21.15.112:0/1123540975 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f04a4078bc0 0x7f04a407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:26.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:26.595+0000 7f04b7fff640 1 --2- 172.21.15.112:0/1123540975 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f04a4078bc0 0x7f04a407b0d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f04a80046b0 tx=0x7f04a8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:26.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:26.599+0000 7f04b4ff9640 1 -- 172.21.15.112:0/1123540975 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f04ac062470 con 0x7f04b815ca90 2024-09-21T00:27:26.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:26.775+0000 7f04be80e640 1 -- 172.21.15.112:0/1123540975 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "config reset", "num": 31} v 0) -- 0x7f04b8071610 con 0x7f04b815ca90 2024-09-21T00:27:26.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:26.780+0000 7f04b4ff9640 1 -- 172.21.15.112:0/1123540975 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 31}]=0 v36) ==== 71+0+0 (secure 0 0 0) 0x7f04ac061d70 con 0x7f04b815ca90 2024-09-21T00:27:26.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:26.781+0000 7f04be80e640 1 -- 172.21.15.112:0/1123540975 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f04a4078bc0 msgr2=0x7f04a407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:26.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:26.781+0000 7f04be80e640 1 --2- 172.21.15.112:0/1123540975 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f04a4078bc0 0x7f04a407b0d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f04a80046b0 tx=0x7f04a8009040 comp rx=0 tx=0).stop 2024-09-21T00:27:26.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:26.781+0000 7f04be80e640 1 -- 172.21.15.112:0/1123540975 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04b815ca90 msgr2=0x7f04b806ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:26.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:26.781+0000 7f04be80e640 1 --2- 172.21.15.112:0/1123540975 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04b815ca90 0x7f04b806ec20 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f04ac00c370 tx=0x7f04ac00c830 comp rx=0 tx=0).stop 2024-09-21T00:27:26.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:26.781+0000 7f04be80e640 1 -- 172.21.15.112:0/1123540975 shutdown_connections 2024-09-21T00:27:26.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:26.781+0000 7f04be80e640 1 --2- 172.21.15.112:0/1123540975 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04b8151870 0x7f04b806f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:26.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:26.781+0000 7f04be80e640 1 --2- 172.21.15.112:0/1123540975 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f04a4078bc0 0x7f04a407b0d0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:26.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:26.781+0000 7f04be80e640 1 --2- 172.21.15.112:0/1123540975 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04b815ca90 0x7f04b806ec20 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:26.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:26.781+0000 7f04be80e640 1 --2- 172.21.15.112:0/1123540975 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04b80ccc40 0x7f04b81556a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:26.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:26.781+0000 7f04be80e640 1 -- 172.21.15.112:0/1123540975 >> 172.21.15.112:0/1123540975 conn(0x7f04b80b46e0 msgr2=0x7f04b80d23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:26.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:26.781+0000 7f04be80e640 1 -- 172.21.15.112:0/1123540975 shutdown_connections 2024-09-21T00:27:26.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:26.781+0000 7f04be80e640 1 -- 172.21.15.112:0/1123540975 wait complete. 2024-09-21T00:27:26.794 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_sessionmap.TestSessionMap.test_session_evict_blocklisted' 2024-09-21T00:27:26.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:26.904+0000 7f3afbf39640 1 -- 172.21.15.112:0/1201291493 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3af40b6250 msgr2=0x7f3af40b6660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:26.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:26.904+0000 7f3afbf39640 1 --2- 172.21.15.112:0/1201291493 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3af40b6250 0x7f3af40b6660 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f3aec005fd0 tx=0x7f3aec02a560 comp rx=0 tx=0).stop 2024-09-21T00:27:26.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:26.904+0000 7f3afbf39640 1 -- 172.21.15.112:0/1201291493 shutdown_connections 2024-09-21T00:27:26.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:26.904+0000 7f3afbf39640 1 --2- 172.21.15.112:0/1201291493 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3af40b6250 0x7f3af40b6660 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:26.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:26.904+0000 7f3afbf39640 1 --2- 172.21.15.112:0/1201291493 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3af40d19f0 0x7f3af415cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:26.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:26.904+0000 7f3afbf39640 1 -- 172.21.15.112:0/1201291493 >> 172.21.15.112:0/1201291493 conn(0x7f3af40d2350 msgr2=0x7f3af40d2760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:26.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:26.904+0000 7f3afbf39640 1 -- 172.21.15.112:0/1201291493 shutdown_connections 2024-09-21T00:27:26.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:26.904+0000 7f3afbf39640 1 -- 172.21.15.112:0/1201291493 wait complete. 2024-09-21T00:27:26.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:26.906+0000 7f3afbf39640 1 Processor -- start 2024-09-21T00:27:26.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:26.906+0000 7f3afbf39640 1 -- start start 2024-09-21T00:27:26.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:26.908+0000 7f3afbf39640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3af40b6250 0x7f3af41f7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:26.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:26.908+0000 7f3afbf39640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3af40d19f0 0x7f3af4154b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:26.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:26.908+0000 7f3afbf39640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3af415cb10 0x7f3af41550a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:26.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:26.908+0000 7f3afbf39640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3af40b6f30 con 0x7f3af415cb10 2024-09-21T00:27:26.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:26.908+0000 7f3afbf39640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3af40b6db0 con 0x7f3af40d19f0 2024-09-21T00:27:26.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:26.908+0000 7f3afbf39640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3af40b70b0 con 0x7f3af40b6250 2024-09-21T00:27:26.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:26.908+0000 7f3afa4d6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3af415cb10 0x7f3af41550a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:26.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:26.908+0000 7f3afa4d6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3af415cb10 0x7f3af41550a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58372/0 (socket says 172.21.15.112:58372) 2024-09-21T00:27:26.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:26.908+0000 7f3afa4d6640 1 -- 172.21.15.112:0/4266654328 learned_addr learned my addr 172.21.15.112:0/4266654328 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:27:26.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:26.908+0000 7f3afa4d6640 1 -- 172.21.15.112:0/4266654328 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3af40b6250 msgr2=0x7f3af41f7910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:26.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:26.908+0000 7f3afa4d6640 1 --2- 172.21.15.112:0/4266654328 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3af40b6250 0x7f3af41f7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:26.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:26.908+0000 7f3afa4d6640 1 -- 172.21.15.112:0/4266654328 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3af40d19f0 msgr2=0x7f3af4154b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:26.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:26.908+0000 7f3afa4d6640 1 --2- 172.21.15.112:0/4266654328 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3af40d19f0 0x7f3af4154b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:26.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:26.908+0000 7f3afa4d6640 1 -- 172.21.15.112:0/4266654328 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3af41e6540 con 0x7f3af415cb10 2024-09-21T00:27:26.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:26.908+0000 7f3afa4d6640 1 --2- 172.21.15.112:0/4266654328 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3af415cb10 0x7f3af41550a0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f3aec006010 tx=0x7f3aec003890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:26.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:26.908+0000 7f3aea7fc640 1 -- 172.21.15.112:0/4266654328 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aec03e030 con 0x7f3af415cb10 2024-09-21T00:27:26.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:26.908+0000 7f3aea7fc640 1 -- 172.21.15.112:0/4266654328 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3aec003d80 con 0x7f3af415cb10 2024-09-21T00:27:26.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:26.908+0000 7f3afbf39640 1 -- 172.21.15.112:0/4266654328 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3af41e6820 con 0x7f3af415cb10 2024-09-21T00:27:26.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:26.908+0000 7f3aea7fc640 1 -- 172.21.15.112:0/4266654328 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aec035040 con 0x7f3af415cb10 2024-09-21T00:27:26.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:26.908+0000 7f3afbf39640 1 -- 172.21.15.112:0/4266654328 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3af41e6de0 con 0x7f3af415cb10 2024-09-21T00:27:26.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:26.909+0000 7f3afbf39640 1 -- 172.21.15.112:0/4266654328 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3af40b6660 con 0x7f3af415cb10 2024-09-21T00:27:26.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:26.910+0000 7f3aea7fc640 1 -- 172.21.15.112:0/4266654328 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3aec041030 con 0x7f3af415cb10 2024-09-21T00:27:26.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:26.910+0000 7f3aea7fc640 1 --2- 172.21.15.112:0/4266654328 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f3ad8078b90 0x7f3ad807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:26.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:26.911+0000 7f3aea7fc640 1 -- 172.21.15.112:0/4266654328 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5335+0+0 (secure 0 0 0) 0x7f3aec0bb700 con 0x7f3af415cb10 2024-09-21T00:27:26.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:26.911+0000 7f3af9cd5640 1 --2- 172.21.15.112:0/4266654328 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f3ad8078b90 0x7f3ad807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:26.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:26.911+0000 7f3af9cd5640 1 --2- 172.21.15.112:0/4266654328 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f3ad8078b90 0x7f3ad807b0a0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f3af000ab90 tx=0x7f3af0009230 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:26.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:26.915+0000 7f3aea7fc640 1 -- 172.21.15.112:0/4266654328 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3aec083cb0 con 0x7f3af415cb10 2024-09-21T00:27:27.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:27.169+0000 7f3afbf39640 1 -- 172.21.15.112:0/4266654328 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_sessionmap.TestSessionMap.test_session_evict_blocklisted"]} v 0) -- 0x7f3af40b56b0 con 0x7f3af415cb10 2024-09-21T00:27:27.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:27.462+0000 7f3aea7fc640 1 -- 172.21.15.112:0/4266654328 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_sessionmap.TestSessionMap.test_session_evict_blocklisted"]}]=0 v117) ==== 153+0+0 (secure 0 0 0) 0x7f3aec0835b0 con 0x7f3af415cb10 2024-09-21T00:27:27.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:27.464+0000 7f3afbf39640 1 -- 172.21.15.112:0/4266654328 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f3ad8078b90 msgr2=0x7f3ad807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:27.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:27.464+0000 7f3afbf39640 1 --2- 172.21.15.112:0/4266654328 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f3ad8078b90 0x7f3ad807b0a0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f3af000ab90 tx=0x7f3af0009230 comp rx=0 tx=0).stop 2024-09-21T00:27:27.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:27.464+0000 7f3afbf39640 1 -- 172.21.15.112:0/4266654328 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3af415cb10 msgr2=0x7f3af41550a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:27.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:27.464+0000 7f3afbf39640 1 --2- 172.21.15.112:0/4266654328 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3af415cb10 0x7f3af41550a0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f3aec006010 tx=0x7f3aec003890 comp rx=0 tx=0).stop 2024-09-21T00:27:27.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:27.464+0000 7f3afbf39640 1 -- 172.21.15.112:0/4266654328 shutdown_connections 2024-09-21T00:27:27.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:27.464+0000 7f3afbf39640 1 --2- 172.21.15.112:0/4266654328 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f3ad8078b90 0x7f3ad807b0a0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:27.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:27.464+0000 7f3afbf39640 1 --2- 172.21.15.112:0/4266654328 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3af415cb10 0x7f3af41550a0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:27.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:27.464+0000 7f3afbf39640 1 --2- 172.21.15.112:0/4266654328 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3af40d19f0 0x7f3af4154b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:27.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:27.464+0000 7f3afbf39640 1 --2- 172.21.15.112:0/4266654328 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3af40b6250 0x7f3af41f7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:27.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:27.464+0000 7f3afbf39640 1 -- 172.21.15.112:0/4266654328 >> 172.21.15.112:0/4266654328 conn(0x7f3af40d2350 msgr2=0x7f3af4153ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:27.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:27.464+0000 7f3afbf39640 1 -- 172.21.15.112:0/4266654328 shutdown_connections 2024-09-21T00:27:27.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:27.464+0000 7f3afbf39640 1 -- 172.21.15.112:0/4266654328 wait complete. 2024-09-21T00:27:27.477 INFO:tasks.cephfs_test_runner:Check that mds evicts blocklisted client ... skipped 'Requires FUSE client to use mds_cluster.is_addr_blocklisted()' 2024-09-21T00:27:27.477 INFO:tasks.cephfs_test_runner:Starting test: test_session_evict_non_blocklisted (tasks.cephfs.test_sessionmap.TestSessionMap) Check that mds evicts without blocklisting client 2024-09-21T00:27:27.477 INFO:tasks.cephfs_test_runner:test_session_evict_non_blocklisted (tasks.cephfs.test_sessionmap.TestSessionMap) 2024-09-21T00:27:27.477 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T00:27:27.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:27.587+0000 7f4d6be63640 1 -- 172.21.15.112:0/1862160528 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4d64162f00 msgr2=0x7f4d640d24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:27.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:27.587+0000 7f4d6be63640 1 --2- 172.21.15.112:0/1862160528 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4d64162f00 0x7f4d640d24d0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f4d640b7460 tx=0x7f4d6002a560 comp rx=0 tx=0).stop 2024-09-21T00:27:27.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:27.587+0000 7f4d6be63640 1 -- 172.21.15.112:0/1862160528 shutdown_connections 2024-09-21T00:27:27.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:27.587+0000 7f4d6be63640 1 --2- 172.21.15.112:0/1862160528 >> v2:172.21.15.187:3300/0 conn(0x7f4d64162280 0x7f4d64162690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:27.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:27.587+0000 7f4d6be63640 1 --2- 172.21.15.112:0/1862160528 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4d640b2e40 0x7f4d640b3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:27.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:27.587+0000 7f4d6be63640 1 --2- 172.21.15.112:0/1862160528 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4d64162f00 0x7f4d640d24d0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:27.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:27.587+0000 7f4d6be63640 1 -- 172.21.15.112:0/1862160528 >> 172.21.15.112:0/1862160528 conn(0x7f4d640b6b20 msgr2=0x7f4d640d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:27.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:27.587+0000 7f4d6be63640 1 -- 172.21.15.112:0/1862160528 shutdown_connections 2024-09-21T00:27:27.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:27.587+0000 7f4d6be63640 1 -- 172.21.15.112:0/1862160528 wait complete. 2024-09-21T00:27:27.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:27.589+0000 7f4d6be63640 1 Processor -- start 2024-09-21T00:27:27.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:27.589+0000 7f4d6be63640 1 -- start start 2024-09-21T00:27:27.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:27.590+0000 7f4d6be63640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4d640b2e40 0x7f4d640ac730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:27.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:27.590+0000 7f4d6be63640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4d64162280 0x7f4d6414d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:27.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:27.590+0000 7f4d6be63640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4d6414dec0 0x7f4d6414e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:27.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:27.590+0000 7f4d6be63640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4d640b3dd0 con 0x7f4d64162280 2024-09-21T00:27:27.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:27.590+0000 7f4d6be63640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f4d640b3a20 con 0x7f4d640b2e40 2024-09-21T00:27:27.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:27.590+0000 7f4d6be63640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f4d640b3f50 con 0x7f4d6414dec0 2024-09-21T00:27:27.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:27.591+0000 7f4d693fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4d64162280 0x7f4d6414d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:27.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:27.591+0000 7f4d693fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4d64162280 0x7f4d6414d980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58394/0 (socket says 172.21.15.112:58394) 2024-09-21T00:27:27.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:27.591+0000 7f4d693fe640 1 -- 172.21.15.112:0/2490734607 learned_addr learned my addr 172.21.15.112:0/2490734607 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:27:27.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:27.591+0000 7f4d69bff640 1 --2- 172.21.15.112:0/2490734607 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4d640b2e40 0x7f4d640ac730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:27.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:27.591+0000 7f4d6a400640 1 --2- 172.21.15.112:0/2490734607 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4d6414dec0 0x7f4d6414e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:27.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:27.591+0000 7f4d693fe640 1 -- 172.21.15.112:0/2490734607 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4d6414dec0 msgr2=0x7f4d6414e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:27.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:27.591+0000 7f4d693fe640 1 --2- 172.21.15.112:0/2490734607 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4d6414dec0 0x7f4d6414e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:27.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:27.591+0000 7f4d693fe640 1 -- 172.21.15.112:0/2490734607 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4d640b2e40 msgr2=0x7f4d640ac730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:27.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:27.591+0000 7f4d693fe640 1 --2- 172.21.15.112:0/2490734607 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4d640b2e40 0x7f4d640ac730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:27.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:27.591+0000 7f4d693fe640 1 -- 172.21.15.112:0/2490734607 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d6415b1e0 con 0x7f4d64162280 2024-09-21T00:27:27.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:27.591+0000 7f4d693fe640 1 --2- 172.21.15.112:0/2490734607 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4d64162280 0x7f4d6414d980 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f4d5400b410 tx=0x7f4d5400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:27.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:27.591+0000 7f4d5a7fc640 1 -- 172.21.15.112:0/2490734607 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d54015020 con 0x7f4d64162280 2024-09-21T00:27:27.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:27.591+0000 7f4d5a7fc640 1 -- 172.21.15.112:0/2490734607 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4d54011940 con 0x7f4d64162280 2024-09-21T00:27:27.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:27.591+0000 7f4d6be63640 1 -- 172.21.15.112:0/2490734607 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d640b1d20 con 0x7f4d64162280 2024-09-21T00:27:27.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:27.591+0000 7f4d5a7fc640 1 -- 172.21.15.112:0/2490734607 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d54011d90 con 0x7f4d64162280 2024-09-21T00:27:27.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:27.591+0000 7f4d6be63640 1 -- 172.21.15.112:0/2490734607 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d640b22e0 con 0x7f4d64162280 2024-09-21T00:27:27.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:27.592+0000 7f4d6be63640 1 -- 172.21.15.112:0/2490734607 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d64162f00 con 0x7f4d64162280 2024-09-21T00:27:27.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:27.593+0000 7f4d5a7fc640 1 -- 172.21.15.112:0/2490734607 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f4d54018020 con 0x7f4d64162280 2024-09-21T00:27:27.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:27.594+0000 7f4d5a7fc640 1 --2- 172.21.15.112:0/2490734607 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4d48078b90 0x7f4d4807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:27.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:27.594+0000 7f4d5a7fc640 1 -- 172.21.15.112:0/2490734607 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5335+0+0 (secure 0 0 0) 0x7f4d5409a960 con 0x7f4d64162280 2024-09-21T00:27:27.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:27.594+0000 7f4d69bff640 1 --2- 172.21.15.112:0/2490734607 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4d48078b90 0x7f4d4807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:27.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:27.595+0000 7f4d69bff640 1 --2- 172.21.15.112:0/2490734607 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4d48078b90 0x7f4d4807b0a0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f4d60033000 tx=0x7f4d60004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:27.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:27.599+0000 7f4d5a7fc640 1 -- 172.21.15.112:0/2490734607 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f4d5401f020 con 0x7f4d64162280 2024-09-21T00:27:27.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:27.778+0000 7f4d6be63640 1 -- 172.21.15.112:0/2490734607 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4d640b33f0 con 0x7f4d64162280 2024-09-21T00:27:27.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:27.779+0000 7f4d5a7fc640 1 -- 172.21.15.112:0/2490734607 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v52) ==== 74+0+13329 (secure 0 0 0) 0x7f4d54062950 con 0x7f4d64162280 2024-09-21T00:27:27.780 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:27:27.780 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":52,"fsid":"8b34692b-dc65-49b2-a08f-26f707b96ac5","created":"2024-09-21T00:25:22.788490+0000","modified":"2024-09-21T00:27:26.457130+0000","last_up_change":"2024-09-21T00:25:36.161835+0000","last_in_change":"2024-09-21T00:25:26.346500+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T00:25:37.105006+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9c575086-67d2-4de2-b01b-d44dcc21d604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6809","nonce":755760621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6811","nonce":755760621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6815","nonce":755760621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6813","nonce":755760621}]},"public_addr":"172.21.15.112:6809/755760621","cluster_addr":"172.21.15.112:6811/755760621","heartbeat_back_addr":"172.21.15.112:6815/755760621","heartbeat_front_addr":"172.21.15.112:6813/755760621","state":["exists","up"]},{"osd":1,"uuid":"a983c227-a3ce-4bdd-82f1-438ba7e3ecd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6801","nonce":344932592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6803","nonce":344932592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6807","nonce":344932592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6805","nonce":344932592}]},"public_addr":"172.21.15.112:6801/344932592","cluster_addr":"172.21.15.112:6803/344932592","heartbeat_back_addr":"172.21.15.112:6807/344932592","heartbeat_front_addr":"172.21.15.112:6805/344932592","state":["exists","up"]},{"osd":2,"uuid":"1193c1d8-bfff-4a57-8200-5f69bf1ea3af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6825","nonce":407933616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6827","nonce":407933616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6831","nonce":407933616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6829","nonce":407933616}]},"public_addr":"172.21.15.112:6825/407933616","cluster_addr":"172.21.15.112:6827/407933616","heartbeat_back_addr":"172.21.15.112:6831/407933616","heartbeat_front_addr":"172.21.15.112:6829/407933616","state":["exists","up"]},{"osd":3,"uuid":"af04330a-36ad-40cc-a8dd-fa259b221de5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6817","nonce":4120164806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6819","nonce":4120164806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6823","nonce":4120164806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6821","nonce":4120164806}]},"public_addr":"172.21.15.112:6817/4120164806","cluster_addr":"172.21.15.112:6819/4120164806","heartbeat_back_addr":"172.21.15.112:6823/4120164806","heartbeat_front_addr":"172.21.15.112:6821/4120164806","state":["exists","up"]},{"osd":4,"uuid":"5b893111-292c-4c5a-b184-53ad38c7199e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6803","nonce":522077560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6805","nonce":522077560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6809","nonce":522077560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6807","nonce":522077560}]},"public_addr":"172.21.15.187:6803/522077560","cluster_addr":"172.21.15.187:6805/522077560","heartbeat_back_addr":"172.21.15.187:6809/522077560","heartbeat_front_addr":"172.21.15.187:6807/522077560","state":["exists","up"]},{"osd":5,"uuid":"c0b8e011-45fe-4c5e-925e-6be2a7f1868a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6827","nonce":3593144721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6829","nonce":3593144721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6833","nonce":3593144721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6831","nonce":3593144721}]},"public_addr":"172.21.15.187:6827/3593144721","cluster_addr":"172.21.15.187:6829/3593144721","heartbeat_back_addr":"172.21.15.187:6833/3593144721","heartbeat_front_addr":"172.21.15.187:6831/3593144721","state":["exists","up"]},{"osd":6,"uuid":"87d2d650-7489-4593-98a3-a1f433ce8dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6811","nonce":1494387657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6813","nonce":1494387657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6817","nonce":1494387657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6815","nonce":1494387657}]},"public_addr":"172.21.15.187:6811/1494387657","cluster_addr":"172.21.15.187:6813/1494387657","heartbeat_back_addr":"172.21.15.187:6817/1494387657","heartbeat_front_addr":"172.21.15.187:6815/1494387657","state":["exists","up"]},{"osd":7,"uuid":"c6b4f8e2-a1ee-4438-aa37-67ff4b6ef1f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6819","nonce":1624699842}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6821","nonce":1624699842}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6825","nonce":1624699842}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6823","nonce":1624699842}]},"public_addr":"172.21.15.187:6819/1624699842","cluster_addr":"172.21.15.187:6821/1624699842","heartbeat_back_addr":"172.21.15.187:6825/1624699842","heartbeat_front_addr":"172.21.15.187:6823/1624699842","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.873162+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.400506+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.664317+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.159791+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.202630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.113212+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.384786+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.448441+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:6835/722271164":"2024-09-22T00:27:23.452044+0000","172.21.15.112:6833/722271164":"2024-09-22T00:27:23.452044+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T00:27:27.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:27.781+0000 7f4d6be63640 1 -- 172.21.15.112:0/2490734607 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4d48078b90 msgr2=0x7f4d4807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:27.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:27.781+0000 7f4d6be63640 1 --2- 172.21.15.112:0/2490734607 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4d48078b90 0x7f4d4807b0a0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f4d60033000 tx=0x7f4d60004730 comp rx=0 tx=0).stop 2024-09-21T00:27:27.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:27.781+0000 7f4d6be63640 1 -- 172.21.15.112:0/2490734607 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4d64162280 msgr2=0x7f4d6414d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:27.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:27.781+0000 7f4d6be63640 1 --2- 172.21.15.112:0/2490734607 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4d64162280 0x7f4d6414d980 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f4d5400b410 tx=0x7f4d5400b8d0 comp rx=0 tx=0).stop 2024-09-21T00:27:27.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:27.781+0000 7f4d6be63640 1 -- 172.21.15.112:0/2490734607 shutdown_connections 2024-09-21T00:27:27.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:27.781+0000 7f4d6be63640 1 --2- 172.21.15.112:0/2490734607 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4d6414dec0 0x7f4d6414e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:27.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:27.781+0000 7f4d6be63640 1 --2- 172.21.15.112:0/2490734607 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4d48078b90 0x7f4d4807b0a0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:27.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:27.781+0000 7f4d6be63640 1 --2- 172.21.15.112:0/2490734607 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4d64162280 0x7f4d6414d980 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:27.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:27.781+0000 7f4d6be63640 1 --2- 172.21.15.112:0/2490734607 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4d640b2e40 0x7f4d640ac730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:27.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:27.781+0000 7f4d6be63640 1 -- 172.21.15.112:0/2490734607 >> 172.21.15.112:0/2490734607 conn(0x7f4d640b6b20 msgr2=0x7f4d6415d9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:27.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:27.781+0000 7f4d6be63640 1 -- 172.21.15.112:0/2490734607 shutdown_connections 2024-09-21T00:27:27.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:27.781+0000 7f4d6be63640 1 -- 172.21.15.112:0/2490734607 wait complete. 2024-09-21T00:27:27.793 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T00:25:37.105006+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-21T00:27:27.794 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-21T00:27:27.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:27.907+0000 7f0a75920640 1 -- 172.21.15.112:0/1100459782 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a700b5260 msgr2=0x7f0a7015ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:27.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:27.907+0000 7f0a75920640 1 --2- 172.21.15.112:0/1100459782 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a700b5260 0x7f0a7015ba80 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f0a7006e6e0 tx=0x7f0a5802a540 comp rx=0 tx=0).stop 2024-09-21T00:27:27.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:27.907+0000 7f0a75920640 1 -- 172.21.15.112:0/1100459782 shutdown_connections 2024-09-21T00:27:27.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:27.907+0000 7f0a75920640 1 --2- 172.21.15.112:0/1100459782 >> v2:172.21.15.187:3300/0 conn(0x7f0a7015bfc0 0x7f0a7015f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:27.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:27.907+0000 7f0a75920640 1 --2- 172.21.15.112:0/1100459782 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a700b5260 0x7f0a7015ba80 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:27.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:27.907+0000 7f0a75920640 1 --2- 172.21.15.112:0/1100459782 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0a7006def0 0x7f0a7006f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:27.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:27.907+0000 7f0a75920640 1 -- 172.21.15.112:0/1100459782 >> 172.21.15.112:0/1100459782 conn(0x7f0a700cdca0 msgr2=0x7f0a700ce0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:27.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:27.907+0000 7f0a75920640 1 -- 172.21.15.112:0/1100459782 shutdown_connections 2024-09-21T00:27:27.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:27.907+0000 7f0a75920640 1 -- 172.21.15.112:0/1100459782 wait complete. 2024-09-21T00:27:27.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:27.909+0000 7f0a75920640 1 Processor -- start 2024-09-21T00:27:27.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:27.909+0000 7f0a75920640 1 -- start start 2024-09-21T00:27:27.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:27.910+0000 7f0a75920640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0a7006def0 0x7f0a700c9d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:27.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:27.910+0000 7f0a75920640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0a700b5260 0x7f0a700b0c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:27.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:27.910+0000 7f0a75920640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a7015bfc0 0x7f0a700b11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:27.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:27.910+0000 7f0a75920640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0a7015fcf0 con 0x7f0a7015bfc0 2024-09-21T00:27:27.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:27.910+0000 7f0a75920640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0a7015fb70 con 0x7f0a700b5260 2024-09-21T00:27:27.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:27.910+0000 7f0a75920640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f0a7015fe70 con 0x7f0a7006def0 2024-09-21T00:27:27.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:27.910+0000 7f0a6f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a7015bfc0 0x7f0a700b11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:27.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:27.911+0000 7f0a6f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a7015bfc0 0x7f0a700b11a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58422/0 (socket says 172.21.15.112:58422) 2024-09-21T00:27:27.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:27.911+0000 7f0a6f7fe640 1 -- 172.21.15.112:0/1828687067 learned_addr learned my addr 172.21.15.112:0/1828687067 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:27:27.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:27.911+0000 7f0a6effd640 1 --2- 172.21.15.112:0/1828687067 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0a7006def0 0x7f0a700c9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:27.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:27.911+0000 7f0a6e7fc640 1 --2- 172.21.15.112:0/1828687067 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0a700b5260 0x7f0a700b0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:27.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:27.911+0000 7f0a6f7fe640 1 -- 172.21.15.112:0/1828687067 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0a7006def0 msgr2=0x7f0a700c9d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:27.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:27.911+0000 7f0a6f7fe640 1 --2- 172.21.15.112:0/1828687067 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0a7006def0 0x7f0a700c9d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:27.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:27.911+0000 7f0a6f7fe640 1 -- 172.21.15.112:0/1828687067 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0a700b5260 msgr2=0x7f0a700b0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:27.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:27.911+0000 7f0a6f7fe640 1 --2- 172.21.15.112:0/1828687067 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0a700b5260 0x7f0a700b0c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:27.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:27.911+0000 7f0a6f7fe640 1 -- 172.21.15.112:0/1828687067 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a70156590 con 0x7f0a7015bfc0 2024-09-21T00:27:27.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:27.911+0000 7f0a6f7fe640 1 --2- 172.21.15.112:0/1828687067 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a7015bfc0 0x7f0a700b11a0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f0a6800c8f0 tx=0x7f0a6800cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:27.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:27.911+0000 7f0a67fff640 1 -- 172.21.15.112:0/1828687067 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a68007b50 con 0x7f0a7015bfc0 2024-09-21T00:27:27.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:27.911+0000 7f0a67fff640 1 -- 172.21.15.112:0/1828687067 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0a68007de0 con 0x7f0a7015bfc0 2024-09-21T00:27:27.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:27.911+0000 7f0a75920640 1 -- 172.21.15.112:0/1828687067 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a7015a090 con 0x7f0a7015bfc0 2024-09-21T00:27:27.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:27.911+0000 7f0a67fff640 1 -- 172.21.15.112:0/1828687067 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a68005280 con 0x7f0a7015bfc0 2024-09-21T00:27:27.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:27.911+0000 7f0a75920640 1 -- 172.21.15.112:0/1828687067 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a70074b80 con 0x7f0a7015bfc0 2024-09-21T00:27:27.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:27.912+0000 7f0a75920640 1 -- 172.21.15.112:0/1828687067 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a7006acf0 con 0x7f0a7015bfc0 2024-09-21T00:27:27.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:27.913+0000 7f0a67fff640 1 -- 172.21.15.112:0/1828687067 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0a68005420 con 0x7f0a7015bfc0 2024-09-21T00:27:27.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:27.913+0000 7f0a67fff640 1 --2- 172.21.15.112:0/1828687067 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f0a50078b90 0x7f0a5007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:27.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:27.914+0000 7f0a67fff640 1 -- 172.21.15.112:0/1828687067 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5335+0+0 (secure 0 0 0) 0x7f0a6809ae30 con 0x7f0a7015bfc0 2024-09-21T00:27:27.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:27.914+0000 7f0a6effd640 1 --2- 172.21.15.112:0/1828687067 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f0a50078b90 0x7f0a5007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:27.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:27.914+0000 7f0a6effd640 1 --2- 172.21.15.112:0/1828687067 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f0a50078b90 0x7f0a5007b0a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f0a60005c40 tx=0x7f0a6000a550 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:27.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:27.918+0000 7f0a67fff640 1 -- 172.21.15.112:0/1828687067 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f0a7006acf0 con 0x7f0a7015bfc0 2024-09-21T00:27:28.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:28.094+0000 7f0a75920640 1 -- 172.21.15.112:0/1828687067 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0a70051050 con 0x7f0a7015bfc0 2024-09-21T00:27:28.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:28.095+0000 7f0a67fff640 1 -- 172.21.15.112:0/1828687067 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v52) ==== 93+0+10 (secure 0 0 0) 0x7f0a70051050 con 0x7f0a7015bfc0 2024-09-21T00:27:28.095 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2024-09-21T00:27:28.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:28.097+0000 7f0a75920640 1 -- 172.21.15.112:0/1828687067 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f0a50078b90 msgr2=0x7f0a5007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:28.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:28.097+0000 7f0a75920640 1 --2- 172.21.15.112:0/1828687067 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f0a50078b90 0x7f0a5007b0a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f0a60005c40 tx=0x7f0a6000a550 comp rx=0 tx=0).stop 2024-09-21T00:27:28.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:28.097+0000 7f0a75920640 1 -- 172.21.15.112:0/1828687067 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a7015bfc0 msgr2=0x7f0a700b11a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:28.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:28.097+0000 7f0a75920640 1 --2- 172.21.15.112:0/1828687067 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a7015bfc0 0x7f0a700b11a0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f0a6800c8f0 tx=0x7f0a6800cdb0 comp rx=0 tx=0).stop 2024-09-21T00:27:28.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:28.097+0000 7f0a75920640 1 -- 172.21.15.112:0/1828687067 shutdown_connections 2024-09-21T00:27:28.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:28.097+0000 7f0a75920640 1 --2- 172.21.15.112:0/1828687067 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f0a50078b90 0x7f0a5007b0a0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:28.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:28.097+0000 7f0a75920640 1 --2- 172.21.15.112:0/1828687067 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a7015bfc0 0x7f0a700b11a0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:28.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:28.097+0000 7f0a75920640 1 --2- 172.21.15.112:0/1828687067 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0a700b5260 0x7f0a700b0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:28.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:28.097+0000 7f0a75920640 1 --2- 172.21.15.112:0/1828687067 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0a7006def0 0x7f0a700c9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:28.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:28.097+0000 7f0a75920640 1 -- 172.21.15.112:0/1828687067 >> 172.21.15.112:0/1828687067 conn(0x7f0a700cdca0 msgr2=0x7f0a700d0350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:28.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:28.097+0000 7f0a75920640 1 -- 172.21.15.112:0/1828687067 shutdown_connections 2024-09-21T00:27:28.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:28.097+0000 7f0a75920640 1 -- 172.21.15.112:0/1828687067 wait complete. 2024-09-21T00:27:28.109 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_sessionmap.TestSessionMap.test_session_evict_non_blocklisted' 2024-09-21T00:27:28.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:28.221+0000 7fa734382640 1 -- 172.21.15.112:0/856831542 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa714004800 msgr2=0x7fa714004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:28.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:28.221+0000 7fa734382640 1 --2- 172.21.15.112:0/856831542 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa714004800 0x7fa714004c10 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fa72000c2e0 tx=0x7fa714005830 comp rx=0 tx=0).stop 2024-09-21T00:27:28.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:28.221+0000 7fa734382640 1 -- 172.21.15.112:0/856831542 shutdown_connections 2024-09-21T00:27:28.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:28.221+0000 7fa734382640 1 --2- 172.21.15.112:0/856831542 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa714004800 0x7fa714004c10 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:28.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:28.221+0000 7fa734382640 1 -- 172.21.15.112:0/856831542 >> 172.21.15.112:0/856831542 conn(0x7fa72c0d0c00 msgr2=0x7fa72c0ccf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:28.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:28.221+0000 7fa734382640 1 -- 172.21.15.112:0/856831542 shutdown_connections 2024-09-21T00:27:28.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:28.221+0000 7fa734382640 1 -- 172.21.15.112:0/856831542 wait complete. 2024-09-21T00:27:28.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:28.223+0000 7fa734382640 1 Processor -- start 2024-09-21T00:27:28.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:28.223+0000 7fa734382640 1 -- start start 2024-09-21T00:27:28.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:28.224+0000 7fa734382640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa714004800 0x7fa72c15a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:28.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:28.224+0000 7fa734382640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa72c14c0e0 0x7fa72c154430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:28.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:28.224+0000 7fa734382640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa72c15d9a0 0x7fa72c15ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:28.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:28.224+0000 7fa734382640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa72c151710 con 0x7fa714004800 2024-09-21T00:27:28.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:28.224+0000 7fa734382640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa72c0b4ad0 con 0x7fa72c14c0e0 2024-09-21T00:27:28.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:28.224+0000 7fa734382640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa72c151890 con 0x7fa72c15d9a0 2024-09-21T00:27:28.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:28.224+0000 7fa73211e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa714004800 0x7fa72c15a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:28.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:28.224+0000 7fa73211e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa714004800 0x7fa72c15a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58440/0 (socket says 172.21.15.112:58440) 2024-09-21T00:27:28.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:28.224+0000 7fa73211e640 1 -- 172.21.15.112:0/2250817565 learned_addr learned my addr 172.21.15.112:0/2250817565 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:27:28.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:28.224+0000 7fa73191d640 1 --2- 172.21.15.112:0/2250817565 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa72c14c0e0 0x7fa72c154430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:28.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:28.224+0000 7fa73211e640 1 -- 172.21.15.112:0/2250817565 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa72c15d9a0 msgr2=0x7fa72c15ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:28.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:28.224+0000 7fa73291f640 1 --2- 172.21.15.112:0/2250817565 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa72c15d9a0 0x7fa72c15ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:28.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:28.224+0000 7fa73211e640 1 --2- 172.21.15.112:0/2250817565 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa72c15d9a0 0x7fa72c15ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:28.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:28.224+0000 7fa73211e640 1 -- 172.21.15.112:0/2250817565 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa72c14c0e0 msgr2=0x7fa72c154430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:28.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:28.224+0000 7fa73211e640 1 --2- 172.21.15.112:0/2250817565 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa72c14c0e0 0x7fa72c154430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:28.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:28.224+0000 7fa73211e640 1 -- 172.21.15.112:0/2250817565 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa72c15e2f0 con 0x7fa714004800 2024-09-21T00:27:28.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:28.224+0000 7fa73191d640 1 --2- 172.21.15.112:0/2250817565 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa72c14c0e0 0x7fa72c154430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:27:28.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:28.225+0000 7fa73211e640 1 --2- 172.21.15.112:0/2250817565 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa714004800 0x7fa72c15a320 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fa71c003ea0 tx=0x7fa71c003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:28.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:28.225+0000 7fa72affd640 1 -- 172.21.15.112:0/2250817565 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa71c015070 con 0x7fa714004800 2024-09-21T00:27:28.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:28.225+0000 7fa72affd640 1 -- 172.21.15.112:0/2250817565 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa71c011840 con 0x7fa714004800 2024-09-21T00:27:28.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:28.225+0000 7fa734382640 1 -- 172.21.15.112:0/2250817565 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa72c15a860 con 0x7fa714004800 2024-09-21T00:27:28.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:28.225+0000 7fa72affd640 1 -- 172.21.15.112:0/2250817565 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa71c011c70 con 0x7fa714004800 2024-09-21T00:27:28.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:28.225+0000 7fa734382640 1 -- 172.21.15.112:0/2250817565 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa72c1f6db0 con 0x7fa714004800 2024-09-21T00:27:28.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:28.226+0000 7fa734382640 1 -- 172.21.15.112:0/2250817565 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa72c0cd650 con 0x7fa714004800 2024-09-21T00:27:28.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:28.227+0000 7fa72affd640 1 -- 172.21.15.112:0/2250817565 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa71c018070 con 0x7fa714004800 2024-09-21T00:27:28.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:28.227+0000 7fa72affd640 1 --2- 172.21.15.112:0/2250817565 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fa718078b90 0x7fa71807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:28.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:28.227+0000 7fa72affd640 1 -- 172.21.15.112:0/2250817565 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5335+0+0 (secure 0 0 0) 0x7fa71c09b6e0 con 0x7fa714004800 2024-09-21T00:27:28.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:28.227+0000 7fa73191d640 1 --2- 172.21.15.112:0/2250817565 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fa718078b90 0x7fa71807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:28.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:28.228+0000 7fa73191d640 1 --2- 172.21.15.112:0/2250817565 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fa718078b90 0x7fa71807b0a0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa720033000 tx=0x7fa72c0cf240 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:28.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:28.231+0000 7fa72affd640 1 -- 172.21.15.112:0/2250817565 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa72c15e2f0 con 0x7fa714004800 2024-09-21T00:27:28.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:28.478+0000 7fa734382640 1 -- 172.21.15.112:0/2250817565 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_sessionmap.TestSessionMap.test_session_evict_non_blocklisted"]} v 0) -- 0x7fa72c15ce30 con 0x7fa714004800 2024-09-21T00:27:29.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:29.471+0000 7fa72affd640 1 -- 172.21.15.112:0/2250817565 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_sessionmap.TestSessionMap.test_session_evict_non_blocklisted"]}]=0 v119) ==== 160+0+0 (secure 0 0 0) 0x7fa71c063d20 con 0x7fa714004800 2024-09-21T00:27:29.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:29.474+0000 7fa734382640 1 -- 172.21.15.112:0/2250817565 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fa718078b90 msgr2=0x7fa71807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:29.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:29.474+0000 7fa734382640 1 --2- 172.21.15.112:0/2250817565 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fa718078b90 0x7fa71807b0a0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa720033000 tx=0x7fa72c0cf240 comp rx=0 tx=0).stop 2024-09-21T00:27:29.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:29.474+0000 7fa734382640 1 -- 172.21.15.112:0/2250817565 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa714004800 msgr2=0x7fa72c15a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:29.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:29.474+0000 7fa734382640 1 --2- 172.21.15.112:0/2250817565 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa714004800 0x7fa72c15a320 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fa71c003ea0 tx=0x7fa71c003f90 comp rx=0 tx=0).stop 2024-09-21T00:27:29.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:29.474+0000 7fa734382640 1 -- 172.21.15.112:0/2250817565 shutdown_connections 2024-09-21T00:27:29.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:29.474+0000 7fa734382640 1 --2- 172.21.15.112:0/2250817565 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa72c15d9a0 0x7fa72c15ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:29.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:29.474+0000 7fa734382640 1 --2- 172.21.15.112:0/2250817565 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa72c14c0e0 0x7fa72c154430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:29.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:29.474+0000 7fa734382640 1 --2- 172.21.15.112:0/2250817565 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fa718078b90 0x7fa71807b0a0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:29.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:29.474+0000 7fa734382640 1 --2- 172.21.15.112:0/2250817565 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa714004800 0x7fa72c15a320 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:29.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:29.474+0000 7fa734382640 1 -- 172.21.15.112:0/2250817565 >> 172.21.15.112:0/2250817565 conn(0x7fa72c0d0c00 msgr2=0x7fa72c06f820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:29.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:29.474+0000 7fa734382640 1 -- 172.21.15.112:0/2250817565 shutdown_connections 2024-09-21T00:27:29.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:29.474+0000 7fa734382640 1 -- 172.21.15.112:0/2250817565 wait complete. 2024-09-21T00:27:29.488 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-21T00:27:29.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:29.606+0000 7f75ad713640 1 -- 172.21.15.112:0/2915621514 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7590004d40 msgr2=0x7f7590005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:29.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:29.606+0000 7f75ad713640 1 --2- 172.21.15.112:0/2915621514 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7590004d40 0x7f7590005190 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f75a000baf0 tx=0x7f7590005830 comp rx=0 tx=0).stop 2024-09-21T00:27:29.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:29.606+0000 7f75a77fe640 1 -- 172.21.15.112:0/2915621514 reap_dead start 2024-09-21T00:27:29.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:29.606+0000 7f75ad713640 1 -- 172.21.15.112:0/2915621514 shutdown_connections 2024-09-21T00:27:29.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:29.606+0000 7f75ad713640 1 -- 172.21.15.112:0/2915621514 >> 172.21.15.112:0/2915621514 conn(0x7f75a80b6b20 msgr2=0x7f75a80d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:29.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:29.606+0000 7f75ad713640 1 -- 172.21.15.112:0/2915621514 shutdown_connections 2024-09-21T00:27:29.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:29.606+0000 7f75ad713640 1 -- 172.21.15.112:0/2915621514 wait complete. 2024-09-21T00:27:29.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:29.608+0000 7f75ad713640 1 Processor -- start 2024-09-21T00:27:29.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:29.608+0000 7f75ad713640 1 -- start start 2024-09-21T00:27:29.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:29.610+0000 7f75ad713640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7590004d40 0x7f75a815b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:29.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:29.610+0000 7f75ad713640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f75a80c7f60 0x7f75a8159ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:29.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:29.610+0000 7f75ad713640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f75a80c9310 0x7f75a81f8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:29.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:29.610+0000 7f75ad713640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f75a8162690 con 0x7f7590004d40 2024-09-21T00:27:29.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:29.610+0000 7f75ad713640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f75a80b39a0 con 0x7f75a80c9310 2024-09-21T00:27:29.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:29.610+0000 7f75ad713640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f75a8162810 con 0x7f75a80c7f60 2024-09-21T00:27:29.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:29.610+0000 7f75a6ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7590004d40 0x7f75a815b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:29.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:29.610+0000 7f75a6ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7590004d40 0x7f75a815b1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58462/0 (socket says 172.21.15.112:58462) 2024-09-21T00:27:29.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:29.610+0000 7f75a6ffd640 1 -- 172.21.15.112:0/3205820836 learned_addr learned my addr 172.21.15.112:0/3205820836 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:27:29.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:29.610+0000 7f75a77fe640 1 --2- 172.21.15.112:0/3205820836 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f75a80c9310 0x7f75a81f8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:29.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:29.610+0000 7f75a67fc640 1 --2- 172.21.15.112:0/3205820836 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f75a80c7f60 0x7f75a8159ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:29.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:29.610+0000 7f75a6ffd640 1 -- 172.21.15.112:0/3205820836 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f75a80c7f60 msgr2=0x7f75a8159ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:29.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:29.610+0000 7f75a6ffd640 1 --2- 172.21.15.112:0/3205820836 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f75a80c7f60 0x7f75a8159ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:29.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:29.610+0000 7f75a6ffd640 1 -- 172.21.15.112:0/3205820836 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f75a80c9310 msgr2=0x7f75a81f8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:29.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:29.610+0000 7f75a6ffd640 1 --2- 172.21.15.112:0/3205820836 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f75a80c9310 0x7f75a81f8a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:29.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:29.610+0000 7f75a6ffd640 1 -- 172.21.15.112:0/3205820836 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75a8158a40 con 0x7f7590004d40 2024-09-21T00:27:29.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:29.610+0000 7f75a67fc640 1 --2- 172.21.15.112:0/3205820836 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f75a80c7f60 0x7f75a8159ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:27:29.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:29.610+0000 7f75a6ffd640 1 --2- 172.21.15.112:0/3205820836 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7590004d40 0x7f75a815b1d0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f7598001370 tx=0x7f7598006790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:29.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:29.610+0000 7f758f7fe640 1 -- 172.21.15.112:0/3205820836 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f759800fcb0 con 0x7f7590004d40 2024-09-21T00:27:29.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:29.610+0000 7f758f7fe640 1 -- 172.21.15.112:0/3205820836 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f759800fe50 con 0x7f7590004d40 2024-09-21T00:27:29.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:29.610+0000 7f75ad713640 1 -- 172.21.15.112:0/3205820836 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75a8158d20 con 0x7f7590004d40 2024-09-21T00:27:29.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:29.610+0000 7f758f7fe640 1 -- 172.21.15.112:0/3205820836 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7598011690 con 0x7f7590004d40 2024-09-21T00:27:29.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:29.610+0000 7f75ad713640 1 -- 172.21.15.112:0/3205820836 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75a81592e0 con 0x7f7590004d40 2024-09-21T00:27:29.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:29.611+0000 7f75ad713640 1 -- 172.21.15.112:0/3205820836 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75a8162f00 con 0x7f7590004d40 2024-09-21T00:27:29.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:29.612+0000 7f758f7fe640 1 -- 172.21.15.112:0/3205820836 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f7598018020 con 0x7f7590004d40 2024-09-21T00:27:29.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:29.613+0000 7f758f7fe640 1 --2- 172.21.15.112:0/3205820836 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f7594078b90 0x7f759407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:29.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:29.613+0000 7f758f7fe640 1 -- 172.21.15.112:0/3205820836 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5335+0+0 (secure 0 0 0) 0x7f759809b8e0 con 0x7f7590004d40 2024-09-21T00:27:29.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:29.613+0000 7f75a67fc640 1 --2- 172.21.15.112:0/3205820836 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f7594078b90 0x7f759407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:29.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:29.613+0000 7f75a67fc640 1 --2- 172.21.15.112:0/3205820836 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f7594078b90 0x7f759407b0a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f75a80c7bd0 tx=0x7f759c00b260 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:29.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:29.617+0000 7f758f7fe640 1 -- 172.21.15.112:0/3205820836 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f7598015030 con 0x7f7590004d40 2024-09-21T00:27:29.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:29.794+0000 7f75ad713640 1 -- 172.21.15.112:0/3205820836 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f75a806da40 con 0x7f7590004d40 2024-09-21T00:27:29.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:29.795+0000 7f758f7fe640 1 -- 172.21.15.112:0/3205820836 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v36) ==== 59+0+2294 (secure 0 0 0) 0x7f75a8162bd0 con 0x7f7590004d40 2024-09-21T00:27:29.796 INFO:teuthology.orchestra.run.smithi112.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-21T00:27:29.796 INFO:teuthology.orchestra.run.smithi112.stdout:mon advanced mon_op_complaint_time 120 2024-09-21T00:27:29.796 INFO:teuthology.orchestra.run.smithi112.stdout:mgr advanced client_mount_timeout 30 2024-09-21T00:27:29.796 INFO:teuthology.orchestra.run.smithi112.stdout:mgr advanced debug_client 20/20 2024-09-21T00:27:29.796 INFO:teuthology.orchestra.run.smithi112.stdout:mgr advanced debug_mgr 20/20 2024-09-21T00:27:29.796 INFO:teuthology.orchestra.run.smithi112.stdout:mgr advanced debug_ms 1/1 2024-09-21T00:27:29.796 INFO:teuthology.orchestra.run.smithi112.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-21T00:27:29.796 INFO:teuthology.orchestra.run.smithi112.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-21T00:27:29.796 INFO:teuthology.orchestra.run.smithi112.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 5090.401743 2024-09-21T00:27:29.796 INFO:teuthology.orchestra.run.smithi112.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 7876.743127 2024-09-21T00:27:29.796 INFO:teuthology.orchestra.run.smithi112.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 4865.073913 2024-09-21T00:27:29.796 INFO:teuthology.orchestra.run.smithi112.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 5117.206240 2024-09-21T00:27:29.796 INFO:teuthology.orchestra.run.smithi112.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 4329.674604 2024-09-21T00:27:29.797 INFO:teuthology.orchestra.run.smithi112.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4322.652151 2024-09-21T00:27:29.797 INFO:teuthology.orchestra.run.smithi112.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 4244.674337 2024-09-21T00:27:29.797 INFO:teuthology.orchestra.run.smithi112.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 4472.316624 2024-09-21T00:27:29.797 INFO:teuthology.orchestra.run.smithi112.stdout:mds advanced debug_mds 20/20 2024-09-21T00:27:29.797 INFO:teuthology.orchestra.run.smithi112.stdout:mds advanced debug_mds_balancer 20/20 2024-09-21T00:27:29.797 INFO:teuthology.orchestra.run.smithi112.stdout:mds advanced debug_ms 2/2 2024-09-21T00:27:29.797 INFO:teuthology.orchestra.run.smithi112.stdout:mds dev mds_debug_frag true 2024-09-21T00:27:29.797 INFO:teuthology.orchestra.run.smithi112.stdout:mds dev mds_debug_scatterstat true 2024-09-21T00:27:29.797 INFO:teuthology.orchestra.run.smithi112.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-21T00:27:29.797 INFO:teuthology.orchestra.run.smithi112.stdout:mds dev mds_verify_scatter true 2024-09-21T00:27:29.797 INFO:teuthology.orchestra.run.smithi112.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-21T00:27:29.797 INFO:teuthology.orchestra.run.smithi112.stdout:mds advanced rados_mon_op_timeout 900 2024-09-21T00:27:29.798 INFO:teuthology.orchestra.run.smithi112.stdout:mds advanced rados_osd_op_timeout 900 2024-09-21T00:27:29.798 INFO:teuthology.orchestra.run.smithi112.stdout:client advanced client_mount_timeout 600 2024-09-21T00:27:29.798 INFO:teuthology.orchestra.run.smithi112.stdout:client advanced debug_client 20/20 2024-09-21T00:27:29.798 INFO:teuthology.orchestra.run.smithi112.stdout:client advanced debug_ms 1/1 2024-09-21T00:27:29.798 INFO:teuthology.orchestra.run.smithi112.stdout:client advanced rados_mon_op_timeout 900 2024-09-21T00:27:29.798 INFO:teuthology.orchestra.run.smithi112.stdout:client advanced rados_osd_op_timeout 900 2024-09-21T00:27:29.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:29.797+0000 7f75ad713640 1 -- 172.21.15.112:0/3205820836 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f7594078b90 msgr2=0x7f759407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:29.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:29.797+0000 7f75ad713640 1 --2- 172.21.15.112:0/3205820836 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f7594078b90 0x7f759407b0a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f75a80c7bd0 tx=0x7f759c00b260 comp rx=0 tx=0).stop 2024-09-21T00:27:29.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:29.797+0000 7f75ad713640 1 -- 172.21.15.112:0/3205820836 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7590004d40 msgr2=0x7f75a815b1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:29.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:29.797+0000 7f75ad713640 1 --2- 172.21.15.112:0/3205820836 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7590004d40 0x7f75a815b1d0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f7598001370 tx=0x7f7598006790 comp rx=0 tx=0).stop 2024-09-21T00:27:29.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:29.797+0000 7f75ad713640 1 -- 172.21.15.112:0/3205820836 shutdown_connections 2024-09-21T00:27:29.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:29.797+0000 7f75ad713640 1 --2- 172.21.15.112:0/3205820836 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f75a80c9310 0x7f75a81f8a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:29.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:29.797+0000 7f75ad713640 1 --2- 172.21.15.112:0/3205820836 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f75a80c7f60 0x7f75a8159ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:29.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:29.797+0000 7f75ad713640 1 --2- 172.21.15.112:0/3205820836 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f7594078b90 0x7f759407b0a0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:29.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:29.797+0000 7f75ad713640 1 --2- 172.21.15.112:0/3205820836 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7590004d40 0x7f75a815b1d0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:29.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:29.797+0000 7f75ad713640 1 -- 172.21.15.112:0/3205820836 >> 172.21.15.112:0/3205820836 conn(0x7f75a80b6b20 msgr2=0x7f75a806e640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:29.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:29.797+0000 7f75ad713640 1 -- 172.21.15.112:0/3205820836 shutdown_connections 2024-09-21T00:27:29.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:29.797+0000 7f75ad713640 1 -- 172.21.15.112:0/3205820836 wait complete. 2024-09-21T00:27:29.809 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-21T00:27:29.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:29.924+0000 7ff6944e6640 1 -- 172.21.15.112:0/4112554044 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff674003a00 msgr2=0x7ff674003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:29.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:29.924+0000 7ff6944e6640 1 --2- 172.21.15.112:0/4112554044 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff674003a00 0x7ff674003e70 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7ff67c00c4a0 tx=0x7ff674005830 comp rx=0 tx=0).stop 2024-09-21T00:27:29.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:29.924+0000 7ff692a83640 1 -- 172.21.15.112:0/4112554044 reap_dead start 2024-09-21T00:27:29.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:29.924+0000 7ff6944e6640 1 -- 172.21.15.112:0/4112554044 shutdown_connections 2024-09-21T00:27:29.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:29.924+0000 7ff6944e6640 1 -- 172.21.15.112:0/4112554044 >> 172.21.15.112:0/4112554044 conn(0x7ff68c0b46e0 msgr2=0x7ff68c0d23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:29.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:29.924+0000 7ff6944e6640 1 -- 172.21.15.112:0/4112554044 shutdown_connections 2024-09-21T00:27:29.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:29.924+0000 7ff6944e6640 1 -- 172.21.15.112:0/4112554044 wait complete. 2024-09-21T00:27:29.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:29.925+0000 7ff6944e6640 1 Processor -- start 2024-09-21T00:27:29.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:29.925+0000 7ff6944e6640 1 -- start start 2024-09-21T00:27:29.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:29.927+0000 7ff6944e6640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff674003a00 0x7ff68c1f7320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:29.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:29.927+0000 7ff6944e6640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff68c1e8950 0x7ff68c0b6c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:29.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:29.927+0000 7ff6944e6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff68c1e8d60 0x7ff68c0b7160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:29.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:29.927+0000 7ff6944e6640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff68c0b6120 con 0x7ff68c1e8d60 2024-09-21T00:27:29.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:29.927+0000 7ff6944e6640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff68c0d0910 con 0x7ff68c1e8950 2024-09-21T00:27:29.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:29.927+0000 7ff6944e6640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff68c0b62a0 con 0x7ff674003a00 2024-09-21T00:27:29.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:29.927+0000 7ff692a83640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff68c1e8d60 0x7ff68c0b7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:29.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:29.927+0000 7ff692a83640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff68c1e8d60 0x7ff68c0b7160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58466/0 (socket says 172.21.15.112:58466) 2024-09-21T00:27:29.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:29.927+0000 7ff692a83640 1 -- 172.21.15.112:0/4199712760 learned_addr learned my addr 172.21.15.112:0/4199712760 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:27:29.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:29.927+0000 7ff692a83640 1 -- 172.21.15.112:0/4199712760 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff674003a00 msgr2=0x7ff68c1f7320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:29.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:29.927+0000 7ff692282640 1 --2- 172.21.15.112:0/4199712760 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff674003a00 0x7ff68c1f7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:29.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:29.927+0000 7ff691a81640 1 --2- 172.21.15.112:0/4199712760 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff68c1e8950 0x7ff68c0b6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:29.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:29.927+0000 7ff692a83640 1 --2- 172.21.15.112:0/4199712760 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff674003a00 0x7ff68c1f7320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:29.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:29.927+0000 7ff692a83640 1 -- 172.21.15.112:0/4199712760 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff68c1e8950 msgr2=0x7ff68c0b6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:29.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:29.927+0000 7ff692a83640 1 --2- 172.21.15.112:0/4199712760 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff68c1e8950 0x7ff68c0b6c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:29.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:29.927+0000 7ff692a83640 1 -- 172.21.15.112:0/4199712760 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff68c1f3870 con 0x7ff68c1e8d60 2024-09-21T00:27:29.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:29.927+0000 7ff692a83640 1 --2- 172.21.15.112:0/4199712760 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff68c1e8d60 0x7ff68c0b7160 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7ff68c0b1e90 tx=0x7ff684011f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:29.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:29.927+0000 7ff68affd640 1 -- 172.21.15.112:0/4199712760 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff684003970 con 0x7ff68c1e8d60 2024-09-21T00:27:29.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:29.928+0000 7ff68affd640 1 -- 172.21.15.112:0/4199712760 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff684016030 con 0x7ff68c1e8d60 2024-09-21T00:27:29.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:29.928+0000 7ff6944e6640 1 -- 172.21.15.112:0/4199712760 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff68c1f3b50 con 0x7ff68c1e8d60 2024-09-21T00:27:29.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:29.928+0000 7ff68affd640 1 -- 172.21.15.112:0/4199712760 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff684003d90 con 0x7ff68c1e8d60 2024-09-21T00:27:29.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:29.928+0000 7ff6944e6640 1 -- 172.21.15.112:0/4199712760 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff68c1f4110 con 0x7ff68c1e8d60 2024-09-21T00:27:29.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:29.928+0000 7ff6944e6640 1 -- 172.21.15.112:0/4199712760 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff68c1f35e0 con 0x7ff68c1e8d60 2024-09-21T00:27:29.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:29.929+0000 7ff68affd640 1 -- 172.21.15.112:0/4199712760 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff684004030 con 0x7ff68c1e8d60 2024-09-21T00:27:29.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:29.930+0000 7ff68affd640 1 --2- 172.21.15.112:0/4199712760 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff678078b90 0x7ff67807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:29.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:29.930+0000 7ff68affd640 1 -- 172.21.15.112:0/4199712760 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5335+0+0 (secure 0 0 0) 0x7ff68409b800 con 0x7ff68c1e8d60 2024-09-21T00:27:29.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:29.930+0000 7ff692282640 1 --2- 172.21.15.112:0/4199712760 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff678078b90 0x7ff67807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:29.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:29.930+0000 7ff692282640 1 --2- 172.21.15.112:0/4199712760 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff678078b90 0x7ff67807b0a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff67c00c470 tx=0x7ff67c02d070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:29.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:29.934+0000 7ff68affd640 1 -- 172.21.15.112:0/4199712760 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ff684063e40 con 0x7ff68c1e8d60 2024-09-21T00:27:30.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:30.110+0000 7ff6944e6640 1 -- 172.21.15.112:0/4199712760 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7ff68c0b5be0 con 0x7ff68c1e8d60 2024-09-21T00:27:30.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:30.114+0000 7ff68affd640 1 -- 172.21.15.112:0/4199712760 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v37)=0 v37) ==== 122+0+0 (secure 0 0 0) 0x7ff684063740 con 0x7ff68c1e8d60 2024-09-21T00:27:30.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:30.116+0000 7ff6944e6640 1 -- 172.21.15.112:0/4199712760 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff678078b90 msgr2=0x7ff67807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:30.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:30.116+0000 7ff6944e6640 1 --2- 172.21.15.112:0/4199712760 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff678078b90 0x7ff67807b0a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff67c00c470 tx=0x7ff67c02d070 comp rx=0 tx=0).stop 2024-09-21T00:27:30.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:30.116+0000 7ff6944e6640 1 -- 172.21.15.112:0/4199712760 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff68c1e8d60 msgr2=0x7ff68c0b7160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:30.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:30.116+0000 7ff6944e6640 1 --2- 172.21.15.112:0/4199712760 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff68c1e8d60 0x7ff68c0b7160 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7ff68c0b1e90 tx=0x7ff684011f70 comp rx=0 tx=0).stop 2024-09-21T00:27:30.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:30.117+0000 7ff6944e6640 1 -- 172.21.15.112:0/4199712760 shutdown_connections 2024-09-21T00:27:30.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:30.117+0000 7ff6944e6640 1 --2- 172.21.15.112:0/4199712760 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff678078b90 0x7ff67807b0a0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:30.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:30.117+0000 7ff6944e6640 1 --2- 172.21.15.112:0/4199712760 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff68c1e8d60 0x7ff68c0b7160 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:30.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:30.117+0000 7ff6944e6640 1 --2- 172.21.15.112:0/4199712760 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff68c1e8950 0x7ff68c0b6c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:30.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:30.117+0000 7ff6944e6640 1 --2- 172.21.15.112:0/4199712760 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff674003a00 0x7ff68c1f7320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:30.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:30.117+0000 7ff6944e6640 1 -- 172.21.15.112:0/4199712760 >> 172.21.15.112:0/4199712760 conn(0x7ff68c0b46e0 msgr2=0x7ff68c0713b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:30.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:30.117+0000 7ff6944e6640 1 -- 172.21.15.112:0/4199712760 shutdown_connections 2024-09-21T00:27:30.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:30.117+0000 7ff6944e6640 1 -- 172.21.15.112:0/4199712760 wait complete. 2024-09-21T00:27:30.128 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T00:27:30.128 DEBUG:teuthology.orchestra.run.smithi112:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T00:27:30.173 DEBUG:teuthology.orchestra.run.smithi187:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T00:27:30.224 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T00:27:30.224 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T00:27:30.224 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:27:30.239 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056010,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1439 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248716k,nr_inodes=812179,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T00:27:30.239 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-09-21T00:27:30.239 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:27:30.258 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056013,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497440k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16835 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248720k,nr_inodes=812180,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497440k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T00:27:30.258 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T00:27:30.258 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:27:30.295 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056010,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1439 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248716k,nr_inodes=812179,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T00:27:30.295 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-09-21T00:27:30.296 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:27:30.316 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056013,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497440k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16835 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248720k,nr_inodes=812180,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497440k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T00:27:30.316 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-21T00:27:30.316 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T00:27:30.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:30.446+0000 7f1c0ca3f640 1 -- 172.21.15.112:0/3070273161 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c0815f4c0 msgr2=0x7f1c080b2e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:30.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:30.446+0000 7f1c0ca3f640 1 --2- 172.21.15.112:0/3070273161 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c0815f4c0 0x7f1c080b2e40 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f1c00005fd0 tx=0x7f1c0002a8b0 comp rx=0 tx=0).stop 2024-09-21T00:27:30.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:30.446+0000 7f1c0ca3f640 1 -- 172.21.15.112:0/3070273161 shutdown_connections 2024-09-21T00:27:30.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:30.446+0000 7f1c0ca3f640 1 --2- 172.21.15.112:0/3070273161 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c0815f4c0 0x7f1c080b2e40 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:30.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:30.446+0000 7f1c0ca3f640 1 -- 172.21.15.112:0/3070273161 >> 172.21.15.112:0/3070273161 conn(0x7f1c080b6b20 msgr2=0x7f1c080d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:30.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:30.447+0000 7f1c0ca3f640 1 -- 172.21.15.112:0/3070273161 shutdown_connections 2024-09-21T00:27:30.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:30.447+0000 7f1c0ca3f640 1 -- 172.21.15.112:0/3070273161 wait complete. 2024-09-21T00:27:30.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:30.449+0000 7f1c0ca3f640 1 Processor -- start 2024-09-21T00:27:30.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:30.449+0000 7f1c0ca3f640 1 -- start start 2024-09-21T00:27:30.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:30.450+0000 7f1c0ca3f640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1c0815f4c0 0x7f1c080b7db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:30.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:30.450+0000 7f1c0ca3f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1c08162280 0x7f1c08159740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:30.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:30.450+0000 7f1c0ca3f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c080b82f0 0x7f1c08159c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:30.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:30.450+0000 7f1c0ca3f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1c080b3710 con 0x7f1c080b82f0 2024-09-21T00:27:30.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:30.450+0000 7f1c0ca3f640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f1c080b3590 con 0x7f1c08162280 2024-09-21T00:27:30.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:30.450+0000 7f1c0ca3f640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f1c080b3890 con 0x7f1c0815f4c0 2024-09-21T00:27:30.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:30.450+0000 7f1c06d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c080b82f0 0x7f1c08159c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:30.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:30.450+0000 7f1c06d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c080b82f0 0x7f1c08159c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58474/0 (socket says 172.21.15.112:58474) 2024-09-21T00:27:30.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:30.450+0000 7f1c06d9d640 1 -- 172.21.15.112:0/1629995672 learned_addr learned my addr 172.21.15.112:0/1629995672 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:27:30.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:30.450+0000 7f1bfdd9b640 1 --2- 172.21.15.112:0/1629995672 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1c08162280 0x7f1c08159740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:30.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:30.450+0000 7f1c0659c640 1 --2- 172.21.15.112:0/1629995672 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1c0815f4c0 0x7f1c080b7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:30.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:30.450+0000 7f1c06d9d640 1 -- 172.21.15.112:0/1629995672 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1c0815f4c0 msgr2=0x7f1c080b7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:30.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:30.450+0000 7f1c06d9d640 1 --2- 172.21.15.112:0/1629995672 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1c0815f4c0 0x7f1c080b7db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:30.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:30.450+0000 7f1c06d9d640 1 -- 172.21.15.112:0/1629995672 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1c08162280 msgr2=0x7f1c08159740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:30.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:30.450+0000 7f1c06d9d640 1 --2- 172.21.15.112:0/1629995672 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1c08162280 0x7f1c08159740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:30.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:30.450+0000 7f1c06d9d640 1 -- 172.21.15.112:0/1629995672 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c0815a540 con 0x7f1c080b82f0 2024-09-21T00:27:30.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:30.451+0000 7f1c06d9d640 1 --2- 172.21.15.112:0/1629995672 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c080b82f0 0x7f1c08159c80 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f1c00009cd0 tx=0x7f1c000316f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:30.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:30.451+0000 7f1bff7fe640 1 -- 172.21.15.112:0/1629995672 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c00031990 con 0x7f1c080b82f0 2024-09-21T00:27:30.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:30.451+0000 7f1bff7fe640 1 -- 172.21.15.112:0/1629995672 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1c00031c40 con 0x7f1c080b82f0 2024-09-21T00:27:30.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:30.451+0000 7f1c0ca3f640 1 -- 172.21.15.112:0/1629995672 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c081e8830 con 0x7f1c080b82f0 2024-09-21T00:27:30.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:30.451+0000 7f1bff7fe640 1 -- 172.21.15.112:0/1629995672 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c00033330 con 0x7f1c080b82f0 2024-09-21T00:27:30.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:30.451+0000 7f1c0ca3f640 1 -- 172.21.15.112:0/1629995672 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c081e8df0 con 0x7f1c080b82f0 2024-09-21T00:27:30.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:30.452+0000 7f1bfcd99640 1 -- 172.21.15.112:0/1629995672 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c08162f00 con 0x7f1c080b82f0 2024-09-21T00:27:30.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:30.453+0000 7f1bff7fe640 1 -- 172.21.15.112:0/1629995672 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1c000334d0 con 0x7f1c080b82f0 2024-09-21T00:27:30.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:30.453+0000 7f1bff7fe640 1 --2- 172.21.15.112:0/1629995672 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1be8078b90 0x7f1be807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:30.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:30.453+0000 7f1bff7fe640 1 -- 172.21.15.112:0/1629995672 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5335+0+0 (secure 0 0 0) 0x7f1c000bbfb0 con 0x7f1c080b82f0 2024-09-21T00:27:30.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:30.453+0000 7f1c0659c640 1 --2- 172.21.15.112:0/1629995672 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1be8078b90 0x7f1be807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:30.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:30.454+0000 7f1c0659c640 1 --2- 172.21.15.112:0/1629995672 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1be8078b90 0x7f1be807b0a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f1bf80046b0 tx=0x7f1bf8007cc0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:30.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:30.458+0000 7f1bff7fe640 1 -- 172.21.15.112:0/1629995672 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1c0003f030 con 0x7f1c080b82f0 2024-09-21T00:27:30.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:30.671+0000 7f1bfcd99640 1 -- 172.21.15.112:0/1629995672 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1c080b5790 con 0x7f1c080b82f0 2024-09-21T00:27:30.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:30.672+0000 7f1bff7fe640 1 -- 172.21.15.112:0/1629995672 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+3920 (secure 0 0 0) 0x7f1c00083f10 con 0x7f1c080b82f0 2024-09-21T00:27:30.673 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:27:30.673 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":28,"btime":"2024-09-21T00:27:24:453788+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5158,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6836/2010548530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":2010548530},{"type":"v1","addr":"172.21.15.187:6836","nonce":2010548530}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5161,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6837/2290459931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6835","nonce":2290459931},{"type":"v1","addr":"172.21.15.187:6837","nonce":2290459931}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5400,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6834/158782340","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":158782340},{"type":"v1","addr":"172.21.15.112:6834","nonce":158782340}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/1873032901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":1873032901},{"type":"v1","addr":"172.21.15.112:6835","nonce":1873032901}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[]} 2024-09-21T00:27:30.673 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 28 2024-09-21T00:27:30.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:30.674+0000 7f1bfcd99640 1 -- 172.21.15.112:0/1629995672 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1be8078b90 msgr2=0x7f1be807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:30.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:30.674+0000 7f1bfcd99640 1 --2- 172.21.15.112:0/1629995672 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1be8078b90 0x7f1be807b0a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f1bf80046b0 tx=0x7f1bf8007cc0 comp rx=0 tx=0).stop 2024-09-21T00:27:30.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:30.674+0000 7f1bfcd99640 1 -- 172.21.15.112:0/1629995672 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c080b82f0 msgr2=0x7f1c08159c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:30.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:30.674+0000 7f1bfcd99640 1 --2- 172.21.15.112:0/1629995672 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c080b82f0 0x7f1c08159c80 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f1c00009cd0 tx=0x7f1c000316f0 comp rx=0 tx=0).stop 2024-09-21T00:27:30.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:30.674+0000 7f1bfcd99640 1 -- 172.21.15.112:0/1629995672 shutdown_connections 2024-09-21T00:27:30.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:30.674+0000 7f1bfcd99640 1 --2- 172.21.15.112:0/1629995672 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1be8078b90 0x7f1be807b0a0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:30.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:30.674+0000 7f1bfcd99640 1 --2- 172.21.15.112:0/1629995672 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c080b82f0 0x7f1c08159c80 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:30.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:30.674+0000 7f1bfcd99640 1 --2- 172.21.15.112:0/1629995672 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1c08162280 0x7f1c08159740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:30.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:30.674+0000 7f1bfcd99640 1 --2- 172.21.15.112:0/1629995672 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1c0815f4c0 0x7f1c080b7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:30.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:30.674+0000 7f1bfcd99640 1 -- 172.21.15.112:0/1629995672 >> 172.21.15.112:0/1629995672 conn(0x7f1c080b6b20 msgr2=0x7f1c0806deb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:30.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:30.675+0000 7f1bfcd99640 1 -- 172.21.15.112:0/1629995672 shutdown_connections 2024-09-21T00:27:30.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:30.675+0000 7f1bfcd99640 1 -- 172.21.15.112:0/1629995672 wait complete. 2024-09-21T00:27:30.687 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-21T00:27:30.687 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-21T00:27:30.687 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-21T00:27:30.687 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T00:27:30.687 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-21T00:27:30.687 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-21T00:27:30.688 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-21T00:27:30.688 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T00:27:30.688 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-21T00:27:30.688 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-21T00:27:30.688 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-21T00:27:30.688 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T00:27:30.688 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-21T00:27:30.688 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-21T00:27:30.689 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-21T00:27:30.689 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T00:27:30.697 INFO:tasks.ceph.mds.c:Stopped 2024-09-21T00:27:30.697 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-21T00:27:30.699 INFO:tasks.ceph.mds.a:Stopped 2024-09-21T00:27:30.700 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-21T00:27:30.701 INFO:tasks.ceph.mds.b:Stopped 2024-09-21T00:27:30.701 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-21T00:27:30.702 INFO:tasks.ceph.mds.d:Stopped 2024-09-21T00:27:30.702 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-21T00:27:30.741 INFO:tasks.ceph.mds.c:Started 2024-09-21T00:27:30.742 DEBUG:teuthology.parallel:result is None 2024-09-21T00:27:30.743 INFO:tasks.ceph.mds.a:Started 2024-09-21T00:27:30.743 DEBUG:teuthology.parallel:result is None 2024-09-21T00:27:30.744 INFO:tasks.ceph.mds.b:Started 2024-09-21T00:27:30.744 DEBUG:teuthology.parallel:result is None 2024-09-21T00:27:30.745 INFO:tasks.ceph.mds.d:Started 2024-09-21T00:27:30.745 DEBUG:teuthology.parallel:result is None 2024-09-21T00:27:30.745 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-21T00:27:30.827 INFO:tasks.ceph.mds.c.smithi112.stdout:starting mds.c at 2024-09-21T00:27:30.828 INFO:tasks.ceph.mds.a.smithi112.stdout:starting mds.a at 2024-09-21T00:27:30.837 INFO:tasks.ceph.mds.b.smithi187.stdout:starting mds.b at 2024-09-21T00:27:30.838 INFO:tasks.ceph.mds.d.smithi187.stdout:starting mds.d at 2024-09-21T00:27:30.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:30.863+0000 7fe39044d640 1 -- 172.21.15.112:0/2746591155 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe374003a50 msgr2=0x7fe374003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:30.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:30.863+0000 7fe39044d640 1 --2- 172.21.15.112:0/2746591155 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe374003a50 0x7fe374003ec0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fe3840016c0 tx=0x7fe374005830 comp rx=0 tx=0).stop 2024-09-21T00:27:30.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:30.863+0000 7fe39044d640 1 -- 172.21.15.112:0/2746591155 shutdown_connections 2024-09-21T00:27:30.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:30.863+0000 7fe39044d640 1 --2- 172.21.15.112:0/2746591155 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe374003a50 0x7fe374003ec0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:30.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:30.863+0000 7fe39044d640 1 -- 172.21.15.112:0/2746591155 >> 172.21.15.112:0/2746591155 conn(0x7fe38809d830 msgr2=0x7fe3880b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:30.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:30.863+0000 7fe39044d640 1 -- 172.21.15.112:0/2746591155 shutdown_connections 2024-09-21T00:27:30.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:30.863+0000 7fe39044d640 1 -- 172.21.15.112:0/2746591155 wait complete. 2024-09-21T00:27:30.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:30.865+0000 7fe39044d640 1 Processor -- start 2024-09-21T00:27:30.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:30.865+0000 7fe39044d640 1 -- start start 2024-09-21T00:27:30.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:30.866+0000 7fe39044d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe374003a50 0x7fe388157a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:30.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:30.867+0000 7fe39044d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe38815c320 0x7fe3880d4230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:30.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:30.867+0000 7fe39044d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe3880d4770 0x7fe3881f8930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:30.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:30.867+0000 7fe39044d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe3880b9c50 con 0x7fe38815c320 2024-09-21T00:27:30.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:30.867+0000 7fe39044d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe3880b9ad0 con 0x7fe374003a50 2024-09-21T00:27:30.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:30.867+0000 7fe39044d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe3880b9dd0 con 0x7fe3880d4770 2024-09-21T00:27:30.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:30.867+0000 7fe38d9e8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe38815c320 0x7fe3880d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:30.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:30.867+0000 7fe38d9e8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe38815c320 0x7fe3880d4230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58518/0 (socket says 172.21.15.112:58518) 2024-09-21T00:27:30.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:30.867+0000 7fe38d9e8640 1 -- 172.21.15.112:0/3365022845 learned_addr learned my addr 172.21.15.112:0/3365022845 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:27:30.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:30.867+0000 7fe38e1e9640 1 --2- 172.21.15.112:0/3365022845 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe374003a50 0x7fe388157a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:30.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:30.867+0000 7fe38e9ea640 1 --2- 172.21.15.112:0/3365022845 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe3880d4770 0x7fe3881f8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:30.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:30.867+0000 7fe38d9e8640 1 -- 172.21.15.112:0/3365022845 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe3880d4770 msgr2=0x7fe3881f8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:30.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:30.867+0000 7fe38d9e8640 1 --2- 172.21.15.112:0/3365022845 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe3880d4770 0x7fe3881f8930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:30.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:30.867+0000 7fe38d9e8640 1 -- 172.21.15.112:0/3365022845 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe374003a50 msgr2=0x7fe388157a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:30.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:30.867+0000 7fe38d9e8640 1 --2- 172.21.15.112:0/3365022845 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe374003a50 0x7fe388157a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:30.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:30.867+0000 7fe38d9e8640 1 -- 172.21.15.112:0/3365022845 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3880d4b80 con 0x7fe38815c320 2024-09-21T00:27:30.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:30.867+0000 7fe38e9ea640 1 --2- 172.21.15.112:0/3365022845 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe3880d4770 0x7fe3881f8930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:27:30.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:30.867+0000 7fe38d9e8640 1 --2- 172.21.15.112:0/3365022845 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe38815c320 0x7fe3880d4230 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fe378004eb0 tx=0x7fe37800b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:30.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:30.867+0000 7fe37effd640 1 -- 172.21.15.112:0/3365022845 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe378004440 con 0x7fe38815c320 2024-09-21T00:27:30.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:30.867+0000 7fe37effd640 1 -- 172.21.15.112:0/3365022845 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe378015070 con 0x7fe38815c320 2024-09-21T00:27:30.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:30.867+0000 7fe37effd640 1 -- 172.21.15.112:0/3365022845 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe378003830 con 0x7fe38815c320 2024-09-21T00:27:30.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:30.867+0000 7fe39044d640 1 -- 172.21.15.112:0/3365022845 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe388157f90 con 0x7fe38815c320 2024-09-21T00:27:30.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:30.867+0000 7fe39044d640 1 -- 172.21.15.112:0/3365022845 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe38806d3a0 con 0x7fe38815c320 2024-09-21T00:27:30.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:30.868+0000 7fe39044d640 1 -- 172.21.15.112:0/3365022845 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3880cec70 con 0x7fe38815c320 2024-09-21T00:27:30.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:30.869+0000 7fe37effd640 1 -- 172.21.15.112:0/3365022845 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe3780039d0 con 0x7fe38815c320 2024-09-21T00:27:30.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:30.869+0000 7fe37effd640 1 --2- 172.21.15.112:0/3365022845 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fe36c078b90 0x7fe36c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:30.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:30.870+0000 7fe37effd640 1 -- 172.21.15.112:0/3365022845 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5335+0+0 (secure 0 0 0) 0x7fe37809b570 con 0x7fe38815c320 2024-09-21T00:27:30.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:30.870+0000 7fe38e1e9640 1 --2- 172.21.15.112:0/3365022845 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fe36c078b90 0x7fe36c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:30.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:30.870+0000 7fe38e1e9640 1 --2- 172.21.15.112:0/3365022845 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fe36c078b90 0x7fe36c07b0a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fe384032000 tx=0x7fe38402c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:30.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:30.874+0000 7fe37effd640 1 -- 172.21.15.112:0/3365022845 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe3880d4b80 con 0x7fe38815c320 2024-09-21T00:27:31.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:31.061+0000 7fe39044d640 1 -- 172.21.15.112:0/3365022845 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fe388161f30 con 0x7fe38815c320 2024-09-21T00:27:31.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:31.482+0000 7fe37effd640 1 -- 172.21.15.112:0/3365022845 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v54) ==== 97+0+0 (secure 0 0 0) 0x7fe378063bb0 con 0x7fe38815c320 2024-09-21T00:27:31.483 INFO:teuthology.orchestra.run.smithi112.stderr: removed all blocklist entries 2024-09-21T00:27:31.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:31.486+0000 7fe39044d640 1 -- 172.21.15.112:0/3365022845 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fe36c078b90 msgr2=0x7fe36c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:31.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:31.486+0000 7fe39044d640 1 --2- 172.21.15.112:0/3365022845 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fe36c078b90 0x7fe36c07b0a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fe384032000 tx=0x7fe38402c040 comp rx=0 tx=0).stop 2024-09-21T00:27:31.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:31.486+0000 7fe39044d640 1 -- 172.21.15.112:0/3365022845 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe38815c320 msgr2=0x7fe3880d4230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:31.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:31.486+0000 7fe39044d640 1 --2- 172.21.15.112:0/3365022845 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe38815c320 0x7fe3880d4230 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fe378004eb0 tx=0x7fe37800b9a0 comp rx=0 tx=0).stop 2024-09-21T00:27:31.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:31.486+0000 7fe39044d640 1 -- 172.21.15.112:0/3365022845 shutdown_connections 2024-09-21T00:27:31.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:31.486+0000 7fe39044d640 1 --2- 172.21.15.112:0/3365022845 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe3880d4770 0x7fe3881f8930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:31.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:31.486+0000 7fe39044d640 1 --2- 172.21.15.112:0/3365022845 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fe36c078b90 0x7fe36c07b0a0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:31.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:31.486+0000 7fe39044d640 1 --2- 172.21.15.112:0/3365022845 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe38815c320 0x7fe3880d4230 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:31.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:31.486+0000 7fe39044d640 1 --2- 172.21.15.112:0/3365022845 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe374003a50 0x7fe388157a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:31.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:31.486+0000 7fe39044d640 1 -- 172.21.15.112:0/3365022845 >> 172.21.15.112:0/3365022845 conn(0x7fe38809d830 msgr2=0x7fe3880b7370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:31.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:31.486+0000 7fe39044d640 1 -- 172.21.15.112:0/3365022845 shutdown_connections 2024-09-21T00:27:31.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:31.486+0000 7fe39044d640 1 -- 172.21.15.112:0/3365022845 wait complete. 2024-09-21T00:27:31.501 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-21T00:27:31.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:31.613+0000 7f0d7da15640 1 -- 172.21.15.112:0/955639323 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d78162f00 msgr2=0x7f0d780d24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:31.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:31.613+0000 7f0d7da15640 1 --2- 172.21.15.112:0/955639323 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d78162f00 0x7f0d780d24d0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f0d780b7460 tx=0x7f0d6802a560 comp rx=0 tx=0).stop 2024-09-21T00:27:31.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:31.614+0000 7f0d7da15640 1 -- 172.21.15.112:0/955639323 shutdown_connections 2024-09-21T00:27:31.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:31.614+0000 7f0d7da15640 1 --2- 172.21.15.112:0/955639323 >> v2:172.21.15.187:3300/0 conn(0x7f0d78162280 0x7f0d78162690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:31.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:31.614+0000 7f0d7da15640 1 --2- 172.21.15.112:0/955639323 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0d780b2e40 0x7f0d780b3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:31.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:31.614+0000 7f0d7da15640 1 --2- 172.21.15.112:0/955639323 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d78162f00 0x7f0d780d24d0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:31.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:31.614+0000 7f0d7da15640 1 -- 172.21.15.112:0/955639323 >> 172.21.15.112:0/955639323 conn(0x7f0d780b6b20 msgr2=0x7f0d780d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:31.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:31.614+0000 7f0d7da15640 1 -- 172.21.15.112:0/955639323 shutdown_connections 2024-09-21T00:27:31.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:31.614+0000 7f0d7da15640 1 -- 172.21.15.112:0/955639323 wait complete. 2024-09-21T00:27:31.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:31.615+0000 7f0d7da15640 1 Processor -- start 2024-09-21T00:27:31.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:31.616+0000 7f0d7da15640 1 -- start start 2024-09-21T00:27:31.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:31.617+0000 7f0d7da15640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d780b2e40 0x7f0d7815a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:31.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:31.617+0000 7f0d7da15640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0d78162280 0x7f0d780aadc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:31.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:31.617+0000 7f0d7da15640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0d780b25b0 0x7f0d7814daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:31.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:31.617+0000 7f0d7da15640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0d780b3dd0 con 0x7f0d780b2e40 2024-09-21T00:27:31.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:31.617+0000 7f0d7da15640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0d780b3a20 con 0x7f0d780b25b0 2024-09-21T00:27:31.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:31.617+0000 7f0d7da15640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f0d780b3f50 con 0x7f0d78162280 2024-09-21T00:27:31.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:31.617+0000 7f0d76ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d780b2e40 0x7f0d7815a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:31.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:31.617+0000 7f0d76ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d780b2e40 0x7f0d7815a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58528/0 (socket says 172.21.15.112:58528) 2024-09-21T00:27:31.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:31.617+0000 7f0d76ffd640 1 -- 172.21.15.112:0/4038433718 learned_addr learned my addr 172.21.15.112:0/4038433718 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:27:31.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:31.617+0000 7f0d6e7fc640 1 --2- 172.21.15.112:0/4038433718 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0d78162280 0x7f0d780aadc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:31.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:31.617+0000 7f0d777fe640 1 --2- 172.21.15.112:0/4038433718 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0d780b25b0 0x7f0d7814daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:31.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:31.617+0000 7f0d76ffd640 1 -- 172.21.15.112:0/4038433718 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0d78162280 msgr2=0x7f0d780aadc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:31.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:31.617+0000 7f0d76ffd640 1 --2- 172.21.15.112:0/4038433718 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0d78162280 0x7f0d780aadc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:31.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:31.617+0000 7f0d76ffd640 1 -- 172.21.15.112:0/4038433718 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0d780b25b0 msgr2=0x7f0d7814daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:31.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:31.617+0000 7f0d76ffd640 1 --2- 172.21.15.112:0/4038433718 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0d780b25b0 0x7f0d7814daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:31.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:31.617+0000 7f0d76ffd640 1 -- 172.21.15.112:0/4038433718 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d780b1850 con 0x7f0d780b2e40 2024-09-21T00:27:31.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:31.617+0000 7f0d777fe640 1 --2- 172.21.15.112:0/4038433718 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0d780b25b0 0x7f0d7814daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:27:31.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:31.617+0000 7f0d76ffd640 1 --2- 172.21.15.112:0/4038433718 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d780b2e40 0x7f0d7815a910 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f0d68000c00 tx=0x7f0d680039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:31.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:31.618+0000 7f0d74ff9640 1 -- 172.21.15.112:0/4038433718 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d6803e030 con 0x7f0d780b2e40 2024-09-21T00:27:31.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:31.618+0000 7f0d74ff9640 1 -- 172.21.15.112:0/4038433718 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0d680047a0 con 0x7f0d780b2e40 2024-09-21T00:27:31.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:31.618+0000 7f0d7da15640 1 -- 172.21.15.112:0/4038433718 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d7814e7d0 con 0x7f0d780b2e40 2024-09-21T00:27:31.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:31.618+0000 7f0d74ff9640 1 -- 172.21.15.112:0/4038433718 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d68035040 con 0x7f0d780b2e40 2024-09-21T00:27:31.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:31.618+0000 7f0d7da15640 1 -- 172.21.15.112:0/4038433718 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d780b1db0 con 0x7f0d780b2e40 2024-09-21T00:27:31.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:31.619+0000 7f0d7da15640 1 -- 172.21.15.112:0/4038433718 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d7815ae50 con 0x7f0d780b2e40 2024-09-21T00:27:31.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:31.619+0000 7f0d74ff9640 1 -- 172.21.15.112:0/4038433718 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0d68041030 con 0x7f0d780b2e40 2024-09-21T00:27:31.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:31.620+0000 7f0d74ff9640 1 --2- 172.21.15.112:0/4038433718 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f0d58078b90 0x7f0d5807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:31.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:31.620+0000 7f0d74ff9640 1 -- 172.21.15.112:0/4038433718 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5249+0+0 (secure 0 0 0) 0x7f0d680bb3c0 con 0x7f0d780b2e40 2024-09-21T00:27:31.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:31.620+0000 7f0d6e7fc640 1 --2- 172.21.15.112:0/4038433718 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f0d58078b90 0x7f0d5807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:31.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:31.621+0000 7f0d6e7fc640 1 --2- 172.21.15.112:0/4038433718 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f0d58078b90 0x7f0d5807b0a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f0d6000a490 tx=0x7f0d6000a420 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:31.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:31.625+0000 7f0d74ff9640 1 -- 172.21.15.112:0/4038433718 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f0d780b1850 con 0x7f0d780b2e40 2024-09-21T00:27:31.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:31.869+0000 7f0d7da15640 1 -- 172.21.15.112:0/4038433718 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f0d78162f00 con 0x7f0d780b2e40 2024-09-21T00:27:31.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:31.870+0000 7f0d74ff9640 1 -- 172.21.15.112:0/4038433718 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v16) ==== 80+0+5304 (secure 0 0 0) 0x7f0d68083370 con 0x7f0d780b2e40 2024-09-21T00:27:31.871 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:27:31.871 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-21T00:27:31.871 INFO:teuthology.orchestra.run.smithi112.stdout: "auth_dump": [ 2024-09-21T00:27:31.871 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:27:31.871 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "mds.a", 2024-09-21T00:27:31.871 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBPEu5m3capAhAAczRM4L8ldX985NW4ep3w5Q==", 2024-09-21T00:27:31.871 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:27:31.871 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2024-09-21T00:27:31.871 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow *", 2024-09-21T00:27:31.871 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow *", 2024-09-21T00:27:31.872 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2024-09-21T00:27:31.872 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:27:31.872 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:27:31.872 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:27:31.872 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "mds.b", 2024-09-21T00:27:31.872 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBPEu5mxlDlDxAAtqngbTOhPqXDjFhJpSCLHg==", 2024-09-21T00:27:31.872 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:27:31.872 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2024-09-21T00:27:31.872 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow *", 2024-09-21T00:27:31.872 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow *", 2024-09-21T00:27:31.872 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2024-09-21T00:27:31.872 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:27:31.872 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:27:31.872 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:27:31.872 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "mds.c", 2024-09-21T00:27:31.872 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBPEu5mXNVRChAA1qsPbEybmY16jw5Jwk/v1A==", 2024-09-21T00:27:31.873 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:27:31.873 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2024-09-21T00:27:31.873 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow *", 2024-09-21T00:27:31.873 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow *", 2024-09-21T00:27:31.873 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2024-09-21T00:27:31.873 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:27:31.873 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:27:31.873 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:27:31.873 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "mds.d", 2024-09-21T00:27:31.873 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBPEu5mnzySFxAAX3VfTgGYGcDweI7rPf3xvw==", 2024-09-21T00:27:31.873 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:27:31.873 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2024-09-21T00:27:31.873 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow *", 2024-09-21T00:27:31.873 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow *", 2024-09-21T00:27:31.873 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2024-09-21T00:27:31.874 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:27:31.874 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:27:31.874 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:27:31.874 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.0", 2024-09-21T00:27:31.874 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBSEu5moQg4OBAAi8mzLx7tWnORlrL64/MWlw==", 2024-09-21T00:27:31.874 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:27:31.874 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2024-09-21T00:27:31.874 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2024-09-21T00:27:31.874 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2024-09-21T00:27:31.874 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:27:31.874 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:27:31.874 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:27:31.874 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.1", 2024-09-21T00:27:31.874 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBWEu5mxfobDhAAi7I6LYq9udB0W8bnYaBY9Q==", 2024-09-21T00:27:31.875 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:27:31.875 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2024-09-21T00:27:31.875 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2024-09-21T00:27:31.875 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2024-09-21T00:27:31.875 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:27:31.875 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:27:31.875 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:27:31.875 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.2", 2024-09-21T00:27:31.875 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBZEu5mDqC/EhAAwzTbj22ocR4xJ8DMJvN7Lg==", 2024-09-21T00:27:31.875 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:27:31.875 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2024-09-21T00:27:31.875 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2024-09-21T00:27:31.875 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2024-09-21T00:27:31.875 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:27:31.875 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:27:31.875 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:27:31.876 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.3", 2024-09-21T00:27:31.876 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBcEu5mwweGJBAAT3hZn5pto0aWt6BwGuaqcw==", 2024-09-21T00:27:31.876 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:27:31.876 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2024-09-21T00:27:31.876 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2024-09-21T00:27:31.876 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2024-09-21T00:27:31.876 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:27:31.876 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:27:31.876 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:27:31.876 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.4", 2024-09-21T00:27:31.876 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBjEu5mIPd3ARAAkxTYphn38uKh48UUVEB+Fg==", 2024-09-21T00:27:31.876 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:27:31.876 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2024-09-21T00:27:31.876 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2024-09-21T00:27:31.876 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2024-09-21T00:27:31.877 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:27:31.877 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:27:31.877 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:27:31.877 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.5", 2024-09-21T00:27:31.877 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBmEu5mP9IVBBAAftfK5V1K4vYRyFyJS2UzKQ==", 2024-09-21T00:27:31.877 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:27:31.877 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2024-09-21T00:27:31.877 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2024-09-21T00:27:31.877 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2024-09-21T00:27:31.877 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:27:31.877 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:27:31.877 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:27:31.877 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.6", 2024-09-21T00:27:31.877 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBpEu5mmjiyFRAAbiW2M8JE0jJOaNg1u+1kgw==", 2024-09-21T00:27:31.877 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:27:31.878 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2024-09-21T00:27:31.878 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2024-09-21T00:27:31.878 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2024-09-21T00:27:31.878 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:27:31.878 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:27:31.878 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:27:31.878 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.7", 2024-09-21T00:27:31.878 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBsEu5moklPKBAA6hzVvgvpmZMkPaFFce4mqw==", 2024-09-21T00:27:31.878 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:27:31.878 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2024-09-21T00:27:31.878 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2024-09-21T00:27:31.878 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2024-09-21T00:27:31.878 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:27:31.878 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:27:31.878 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:27:31.879 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "client.0", 2024-09-21T00:27:31.879 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBPEu5m49NPHBAAl/rcjiE8xYI+XdWwvZ6FOw==", 2024-09-21T00:27:31.879 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:27:31.879 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2024-09-21T00:27:31.879 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow r", 2024-09-21T00:27:31.879 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-21T00:27:31.879 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:27:31.879 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:27:31.879 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:27:31.879 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "client.1", 2024-09-21T00:27:31.879 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBPEu5mG33aIxAAirQhTlVyF/pRJx93T/0bIQ==", 2024-09-21T00:27:31.879 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:27:31.879 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2024-09-21T00:27:31.879 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow r", 2024-09-21T00:27:31.879 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow rw", 2024-09-21T00:27:31.880 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow rwx" 2024-09-21T00:27:31.880 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:27:31.880 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:27:31.880 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:27:31.880 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "client.2", 2024-09-21T00:27:31.880 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBPEu5mreUUIBAAELeN1N58srw6GqBrj8UOoQ==", 2024-09-21T00:27:31.880 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:27:31.880 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2024-09-21T00:27:31.880 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow r", 2024-09-21T00:27:31.880 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow rw", 2024-09-21T00:27:31.880 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow rwx" 2024-09-21T00:27:31.880 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:27:31.880 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:27:31.880 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:27:31.880 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "client.3", 2024-09-21T00:27:31.880 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBPEu5mcTmiJxAAY6bmTtow0R0heJHjx0ahDQ==", 2024-09-21T00:27:31.881 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:27:31.881 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2024-09-21T00:27:31.881 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow r", 2024-09-21T00:27:31.881 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow rw", 2024-09-21T00:27:31.881 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow rwx" 2024-09-21T00:27:31.881 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:27:31.881 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:27:31.881 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:27:31.881 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "client.admin", 2024-09-21T00:27:31.881 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBOEu5mTxhNIBAAlStXGsGd/okXhakY6WI/4w==", 2024-09-21T00:27:31.881 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:27:31.881 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow *", 2024-09-21T00:27:31.881 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow *", 2024-09-21T00:27:31.881 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow *", 2024-09-21T00:27:31.881 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2024-09-21T00:27:31.882 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:27:31.882 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:27:31.882 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:27:31.882 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "mgr.x", 2024-09-21T00:27:31.882 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBOEu5mjGpDOhAAJFudb2wZlOBjo8Ss65Q+eg==", 2024-09-21T00:27:31.882 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:27:31.882 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow *", 2024-09-21T00:27:31.882 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile mgr", 2024-09-21T00:27:31.882 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2024-09-21T00:27:31.882 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:27:31.882 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:27:31.882 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:27:31.882 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "mgr.y", 2024-09-21T00:27:31.882 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBOEu5myGjKNRAA9cMDObZYYOXMGdv2mOXkJA==", 2024-09-21T00:27:31.882 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:27:31.883 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow *", 2024-09-21T00:27:31.883 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile mgr", 2024-09-21T00:27:31.883 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2024-09-21T00:27:31.883 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:27:31.883 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:27:31.883 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-09-21T00:27:31.883 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-21T00:27:31.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:31.872+0000 7f0d7da15640 1 -- 172.21.15.112:0/4038433718 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f0d58078b90 msgr2=0x7f0d5807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:31.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:31.872+0000 7f0d7da15640 1 --2- 172.21.15.112:0/4038433718 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f0d58078b90 0x7f0d5807b0a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f0d6000a490 tx=0x7f0d6000a420 comp rx=0 tx=0).stop 2024-09-21T00:27:31.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:31.872+0000 7f0d7da15640 1 -- 172.21.15.112:0/4038433718 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d780b2e40 msgr2=0x7f0d7815a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:31.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:31.872+0000 7f0d7da15640 1 --2- 172.21.15.112:0/4038433718 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d780b2e40 0x7f0d7815a910 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f0d68000c00 tx=0x7f0d680039e0 comp rx=0 tx=0).stop 2024-09-21T00:27:31.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:31.872+0000 7f0d7da15640 1 -- 172.21.15.112:0/4038433718 shutdown_connections 2024-09-21T00:27:31.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:31.872+0000 7f0d7da15640 1 --2- 172.21.15.112:0/4038433718 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0d780b25b0 0x7f0d7814daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:31.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:31.872+0000 7f0d7da15640 1 --2- 172.21.15.112:0/4038433718 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0d78162280 0x7f0d780aadc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:31.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:31.872+0000 7f0d7da15640 1 --2- 172.21.15.112:0/4038433718 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f0d58078b90 0x7f0d5807b0a0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:31.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:31.872+0000 7f0d7da15640 1 --2- 172.21.15.112:0/4038433718 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d780b2e40 0x7f0d7815a910 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:31.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:31.872+0000 7f0d7da15640 1 -- 172.21.15.112:0/4038433718 >> 172.21.15.112:0/4038433718 conn(0x7f0d780b6b20 msgr2=0x7f0d7815d9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:31.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:31.872+0000 7f0d7da15640 1 -- 172.21.15.112:0/4038433718 shutdown_connections 2024-09-21T00:27:31.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:31.872+0000 7f0d7da15640 1 -- 172.21.15.112:0/4038433718 wait complete. 2024-09-21T00:27:31.886 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-21T00:27:31.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:31.996+0000 7fb5dbe16640 1 -- 172.21.15.112:0/2350834418 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb5d415daf0 msgr2=0x7fb5d415a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:31.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:31.996+0000 7fb5dbe16640 1 --2- 172.21.15.112:0/2350834418 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb5d415daf0 0x7fb5d415a6f0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fb5d40c5cc0 tx=0x7fb5c402a530 comp rx=0 tx=0).stop 2024-09-21T00:27:31.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:31.996+0000 7fb5dbe16640 1 -- 172.21.15.112:0/2350834418 shutdown_connections 2024-09-21T00:27:31.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:31.996+0000 7fb5dbe16640 1 --2- 172.21.15.112:0/2350834418 >> v2:172.21.15.187:3300/0 conn(0x7fb5d40d0980 0x7fb5d40d0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:31.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:31.996+0000 7fb5dbe16640 1 --2- 172.21.15.112:0/2350834418 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb5d415daf0 0x7fb5d415a6f0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:31.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:31.996+0000 7fb5dbe16640 1 -- 172.21.15.112:0/2350834418 >> 172.21.15.112:0/2350834418 conn(0x7fb5d4072990 msgr2=0x7fb5d4072da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:31.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:31.996+0000 7fb5dbe16640 1 -- 172.21.15.112:0/2350834418 shutdown_connections 2024-09-21T00:27:31.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:31.996+0000 7fb5dbe16640 1 -- 172.21.15.112:0/2350834418 wait complete. 2024-09-21T00:27:31.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:31.998+0000 7fb5dbe16640 1 Processor -- start 2024-09-21T00:27:31.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:31.998+0000 7fb5dbe16640 1 -- start start 2024-09-21T00:27:32.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:31.999+0000 7fb5dbe16640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb5d40d01f0 0x7fb5d40ca1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:32.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:31.999+0000 7fb5dbe16640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb5d40d0980 0x7fb5d41f7310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:32.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:31.999+0000 7fb5dbe16640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb5d415daf0 0x7fb5d41f7850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:32.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:31.999+0000 7fb5dbe16640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb5d40cd340 con 0x7fb5d40d01f0 2024-09-21T00:27:32.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:31.999+0000 7fb5dbe16640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb5d40cd1c0 con 0x7fb5d40d0980 2024-09-21T00:27:32.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:31.999+0000 7fb5dbe16640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fb5d40cd4c0 con 0x7fb5d415daf0 2024-09-21T00:27:32.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:31.999+0000 7fb5d9bb2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb5d40d01f0 0x7fb5d40ca1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:32.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:31.999+0000 7fb5d9bb2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb5d40d01f0 0x7fb5d40ca1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58554/0 (socket says 172.21.15.112:58554) 2024-09-21T00:27:32.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:31.999+0000 7fb5d9bb2640 1 -- 172.21.15.112:0/2780349732 learned_addr learned my addr 172.21.15.112:0/2780349732 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:27:32.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:31.999+0000 7fb5d93b1640 1 --2- 172.21.15.112:0/2780349732 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb5d40d0980 0x7fb5d41f7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:32.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:31.999+0000 7fb5da3b3640 1 --2- 172.21.15.112:0/2780349732 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb5d415daf0 0x7fb5d41f7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:32.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:32.000+0000 7fb5d9bb2640 1 -- 172.21.15.112:0/2780349732 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb5d415daf0 msgr2=0x7fb5d41f7850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:32.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:32.000+0000 7fb5d9bb2640 1 --2- 172.21.15.112:0/2780349732 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb5d415daf0 0x7fb5d41f7850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:32.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:32.000+0000 7fb5d9bb2640 1 -- 172.21.15.112:0/2780349732 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb5d40d0980 msgr2=0x7fb5d41f7310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:32.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:32.000+0000 7fb5d9bb2640 1 --2- 172.21.15.112:0/2780349732 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb5d40d0980 0x7fb5d41f7310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:32.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:32.000+0000 7fb5d9bb2640 1 -- 172.21.15.112:0/2780349732 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5d40caeb0 con 0x7fb5d40d01f0 2024-09-21T00:27:32.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:32.000+0000 7fb5d9bb2640 1 --2- 172.21.15.112:0/2780349732 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb5d40d01f0 0x7fb5d40ca1f0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fb5d000c390 tx=0x7fb5d000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:32.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:32.000+0000 7fb5ca7fc640 1 -- 172.21.15.112:0/2780349732 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5d0017070 con 0x7fb5d40d01f0 2024-09-21T00:27:32.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:32.000+0000 7fb5ca7fc640 1 -- 172.21.15.112:0/2780349732 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb5d00035f0 con 0x7fb5d40d01f0 2024-09-21T00:27:32.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:32.000+0000 7fb5dbe16640 1 -- 172.21.15.112:0/2780349732 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5d40cb190 con 0x7fb5d40d01f0 2024-09-21T00:27:32.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:32.000+0000 7fb5ca7fc640 1 -- 172.21.15.112:0/2780349732 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5d0003a20 con 0x7fb5d40d01f0 2024-09-21T00:27:32.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:32.000+0000 7fb5dbe16640 1 -- 172.21.15.112:0/2780349732 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5d40cb750 con 0x7fb5d40d01f0 2024-09-21T00:27:32.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:32.001+0000 7fb5dbe16640 1 -- 172.21.15.112:0/2780349732 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5d40cd9e0 con 0x7fb5d40d01f0 2024-09-21T00:27:32.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:32.002+0000 7fb5ca7fc640 1 -- 172.21.15.112:0/2780349732 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb5d001a070 con 0x7fb5d40d01f0 2024-09-21T00:27:32.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:32.002+0000 7fb5ca7fc640 1 --2- 172.21.15.112:0/2780349732 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fb5b8078b90 0x7fb5b807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:32.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:32.002+0000 7fb5ca7fc640 1 -- 172.21.15.112:0/2780349732 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5249+0+0 (secure 0 0 0) 0x7fb5d0099d30 con 0x7fb5d40d01f0 2024-09-21T00:27:32.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:32.002+0000 7fb5d93b1640 1 --2- 172.21.15.112:0/2780349732 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fb5b8078b90 0x7fb5b807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:32.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:32.003+0000 7fb5d93b1640 1 --2- 172.21.15.112:0/2780349732 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fb5b8078b90 0x7fb5b807b0a0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fb5c4034000 tx=0x7fb5c4004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:32.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:32.007+0000 7fb5ca7fc640 1 -- 172.21.15.112:0/2780349732 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb5d40d12d0 con 0x7fb5d40d01f0 2024-09-21T00:27:32.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:32.187+0000 7fb5dbe16640 1 -- 172.21.15.112:0/2780349732 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fb5d40cc620 con 0x7fb5d40d01f0 2024-09-21T00:27:32.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:32.187+0000 7fb5ca7fc640 1 -- 172.21.15.112:0/2780349732 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v56) ==== 84+0+65 (secure 0 0 0) 0x7fb5d0061d70 con 0x7fb5d40d01f0 2024-09-21T00:27:32.188 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:27:32.188 INFO:teuthology.orchestra.run.smithi112.stdout:[ 2024-09-21T00:27:32.188 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:27:32.188 INFO:teuthology.orchestra.run.smithi112.stdout: "poolnum": 1, 2024-09-21T00:27:32.188 INFO:teuthology.orchestra.run.smithi112.stdout: "poolname": ".mgr" 2024-09-21T00:27:32.188 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:27:32.189 INFO:teuthology.orchestra.run.smithi112.stdout:] 2024-09-21T00:27:32.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:32.190+0000 7fb5dbe16640 1 -- 172.21.15.112:0/2780349732 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fb5b8078b90 msgr2=0x7fb5b807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:32.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:32.190+0000 7fb5dbe16640 1 --2- 172.21.15.112:0/2780349732 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fb5b8078b90 0x7fb5b807b0a0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fb5c4034000 tx=0x7fb5c4004730 comp rx=0 tx=0).stop 2024-09-21T00:27:32.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:32.190+0000 7fb5dbe16640 1 -- 172.21.15.112:0/2780349732 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb5d40d01f0 msgr2=0x7fb5d40ca1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:32.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:32.190+0000 7fb5dbe16640 1 --2- 172.21.15.112:0/2780349732 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb5d40d01f0 0x7fb5d40ca1f0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fb5d000c390 tx=0x7fb5d000c850 comp rx=0 tx=0).stop 2024-09-21T00:27:32.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:32.190+0000 7fb5dbe16640 1 -- 172.21.15.112:0/2780349732 shutdown_connections 2024-09-21T00:27:32.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:32.190+0000 7fb5dbe16640 1 --2- 172.21.15.112:0/2780349732 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb5d415daf0 0x7fb5d41f7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:32.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:32.190+0000 7fb5dbe16640 1 --2- 172.21.15.112:0/2780349732 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb5d40d0980 0x7fb5d41f7310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:32.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:32.190+0000 7fb5dbe16640 1 --2- 172.21.15.112:0/2780349732 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fb5b8078b90 0x7fb5b807b0a0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:32.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:32.190+0000 7fb5dbe16640 1 --2- 172.21.15.112:0/2780349732 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb5d40d01f0 0x7fb5d40ca1f0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:32.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:32.190+0000 7fb5dbe16640 1 -- 172.21.15.112:0/2780349732 >> 172.21.15.112:0/2780349732 conn(0x7fb5d4072990 msgr2=0x7fb5d415e210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:32.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:32.190+0000 7fb5dbe16640 1 -- 172.21.15.112:0/2780349732 shutdown_connections 2024-09-21T00:27:32.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:32.190+0000 7fb5dbe16640 1 -- 172.21.15.112:0/2780349732 wait complete. 2024-09-21T00:27:32.202 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-21T00:27:32.202 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-21T00:27:32.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:32.317+0000 7f9b71ca4640 1 -- 172.21.15.112:0/3801897099 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b54004800 msgr2=0x7f9b54004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:32.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:32.317+0000 7f9b71ca4640 1 --2- 172.21.15.112:0/3801897099 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b54004800 0x7f9b54004c10 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f9b6000bb20 tx=0x7f9b54005830 comp rx=0 tx=0).stop 2024-09-21T00:27:32.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:32.317+0000 7f9b6bfff640 1 -- 172.21.15.112:0/3801897099 reap_dead start 2024-09-21T00:27:32.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:32.317+0000 7f9b71ca4640 1 -- 172.21.15.112:0/3801897099 shutdown_connections 2024-09-21T00:27:32.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:32.317+0000 7f9b71ca4640 1 -- 172.21.15.112:0/3801897099 >> 172.21.15.112:0/3801897099 conn(0x7f9b6c0d44e0 msgr2=0x7f9b6c0d48f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:32.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:32.317+0000 7f9b71ca4640 1 -- 172.21.15.112:0/3801897099 shutdown_connections 2024-09-21T00:27:32.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:32.317+0000 7f9b71ca4640 1 -- 172.21.15.112:0/3801897099 wait complete. 2024-09-21T00:27:32.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:32.319+0000 7f9b71ca4640 1 Processor -- start 2024-09-21T00:27:32.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:32.319+0000 7f9b71ca4640 1 -- start start 2024-09-21T00:27:32.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:32.320+0000 7f9b71ca4640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9b54004800 0x7f9b6c1f8120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:32.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:32.320+0000 7f9b71ca4640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9b6c1f53f0 0x7f9b6c0ac0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:32.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:32.320+0000 7f9b71ca4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b6c161ad0 0x7f9b6c0ac610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:32.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:32.320+0000 7f9b71ca4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9b6c1626d0 con 0x7f9b6c161ad0 2024-09-21T00:27:32.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:32.320+0000 7f9b71ca4640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9b6c0b6880 con 0x7f9b6c1f53f0 2024-09-21T00:27:32.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:32.320+0000 7f9b71ca4640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f9b6c162850 con 0x7f9b54004800 2024-09-21T00:27:32.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:32.321+0000 7f9b6bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b6c161ad0 0x7f9b6c0ac610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:32.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:32.321+0000 7f9b6bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b6c161ad0 0x7f9b6c0ac610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58574/0 (socket says 172.21.15.112:58574) 2024-09-21T00:27:32.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:32.321+0000 7f9b6bfff640 1 -- 172.21.15.112:0/4050592070 learned_addr learned my addr 172.21.15.112:0/4050592070 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:27:32.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:32.321+0000 7f9b6affd640 1 --2- 172.21.15.112:0/4050592070 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9b6c1f53f0 0x7f9b6c0ac0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:32.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:32.321+0000 7f9b6b7fe640 1 --2- 172.21.15.112:0/4050592070 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9b54004800 0x7f9b6c1f8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:32.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:32.321+0000 7f9b6bfff640 1 -- 172.21.15.112:0/4050592070 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9b54004800 msgr2=0x7f9b6c1f8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:32.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:32.321+0000 7f9b6bfff640 1 --2- 172.21.15.112:0/4050592070 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9b54004800 0x7f9b6c1f8120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:32.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:32.321+0000 7f9b6bfff640 1 -- 172.21.15.112:0/4050592070 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9b6c1f53f0 msgr2=0x7f9b6c0ac0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:32.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:32.321+0000 7f9b6bfff640 1 --2- 172.21.15.112:0/4050592070 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9b6c1f53f0 0x7f9b6c0ac0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:32.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:32.321+0000 7f9b6bfff640 1 -- 172.21.15.112:0/4050592070 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b6c1f8cc0 con 0x7f9b6c161ad0 2024-09-21T00:27:32.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:32.321+0000 7f9b6bfff640 1 --2- 172.21.15.112:0/4050592070 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b6c161ad0 0x7f9b6c0ac610 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f9b6c08cf10 tx=0x7f9b64010f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:32.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:32.321+0000 7f9b53fff640 1 -- 172.21.15.112:0/4050592070 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b64003970 con 0x7f9b6c161ad0 2024-09-21T00:27:32.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:32.321+0000 7f9b53fff640 1 -- 172.21.15.112:0/4050592070 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9b64015070 con 0x7f9b6c161ad0 2024-09-21T00:27:32.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:32.321+0000 7f9b71ca4640 1 -- 172.21.15.112:0/4050592070 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b6c1f98d0 con 0x7f9b6c161ad0 2024-09-21T00:27:32.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:32.321+0000 7f9b53fff640 1 -- 172.21.15.112:0/4050592070 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b64004030 con 0x7f9b6c161ad0 2024-09-21T00:27:32.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:32.321+0000 7f9b71ca4640 1 -- 172.21.15.112:0/4050592070 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b6c1f9280 con 0x7f9b6c161ad0 2024-09-21T00:27:32.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:32.322+0000 7f9b71ca4640 1 -- 172.21.15.112:0/4050592070 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b6c0b95d0 con 0x7f9b6c161ad0 2024-09-21T00:27:32.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:32.323+0000 7f9b53fff640 1 -- 172.21.15.112:0/4050592070 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9b640041d0 con 0x7f9b6c161ad0 2024-09-21T00:27:32.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:32.323+0000 7f9b53fff640 1 --2- 172.21.15.112:0/4050592070 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f9b58078b90 0x7f9b5807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:32.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:32.324+0000 7f9b53fff640 1 -- 172.21.15.112:0/4050592070 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5249+0+0 (secure 0 0 0) 0x7f9b6409b6b0 con 0x7f9b6c161ad0 2024-09-21T00:27:32.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:32.324+0000 7f9b6b7fe640 1 --2- 172.21.15.112:0/4050592070 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f9b58078b90 0x7f9b5807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:32.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:32.324+0000 7f9b6b7fe640 1 --2- 172.21.15.112:0/4050592070 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f9b58078b90 0x7f9b5807b0a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f9b5c006b70 tx=0x7f9b5c0067a0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:32.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:32.328+0000 7f9b53fff640 1 -- 172.21.15.112:0/4050592070 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f9b64063660 con 0x7f9b6c161ad0 2024-09-21T00:27:32.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:32.506+0000 7f9b71ca4640 1 -- 172.21.15.112:0/4050592070 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f9b6c0b92f0 con 0x7f9b6c161ad0 2024-09-21T00:27:33.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:33.491+0000 7f9b53fff640 1 -- 172.21.15.112:0/4050592070 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v57) ==== 138+0+0 (secure 0 0 0) 0x7f9b64068090 con 0x7f9b6c161ad0 2024-09-21T00:27:33.492 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'cephfs_metadata' created 2024-09-21T00:27:33.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:33.494+0000 7f9b71ca4640 1 -- 172.21.15.112:0/4050592070 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f9b58078b90 msgr2=0x7f9b5807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:33.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:33.494+0000 7f9b71ca4640 1 --2- 172.21.15.112:0/4050592070 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f9b58078b90 0x7f9b5807b0a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f9b5c006b70 tx=0x7f9b5c0067a0 comp rx=0 tx=0).stop 2024-09-21T00:27:33.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:33.494+0000 7f9b71ca4640 1 -- 172.21.15.112:0/4050592070 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b6c161ad0 msgr2=0x7f9b6c0ac610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:33.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:33.494+0000 7f9b71ca4640 1 --2- 172.21.15.112:0/4050592070 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b6c161ad0 0x7f9b6c0ac610 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f9b6c08cf10 tx=0x7f9b64010f40 comp rx=0 tx=0).stop 2024-09-21T00:27:33.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:33.494+0000 7f9b71ca4640 1 -- 172.21.15.112:0/4050592070 shutdown_connections 2024-09-21T00:27:33.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:33.495+0000 7f9b71ca4640 1 --2- 172.21.15.112:0/4050592070 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f9b58078b90 0x7f9b5807b0a0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:33.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:33.495+0000 7f9b71ca4640 1 --2- 172.21.15.112:0/4050592070 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b6c161ad0 0x7f9b6c0ac610 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:33.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:33.495+0000 7f9b71ca4640 1 --2- 172.21.15.112:0/4050592070 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9b6c1f53f0 0x7f9b6c0ac0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:33.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:33.495+0000 7f9b71ca4640 1 --2- 172.21.15.112:0/4050592070 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9b54004800 0x7f9b6c1f8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:33.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:33.495+0000 7f9b71ca4640 1 -- 172.21.15.112:0/4050592070 >> 172.21.15.112:0/4050592070 conn(0x7f9b6c0d44e0 msgr2=0x7f9b6c0b8290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:33.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:33.495+0000 7f9b71ca4640 1 -- 172.21.15.112:0/4050592070 shutdown_connections 2024-09-21T00:27:33.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:33.495+0000 7f9b71ca4640 1 -- 172.21.15.112:0/4050592070 wait complete. 2024-09-21T00:27:33.513 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-21T00:27:33.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:33.637+0000 7f3eafd13640 1 -- 172.21.15.112:0/2905030667 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ea815cb10 msgr2=0x7f3ea815cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:33.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:33.637+0000 7f3eafd13640 1 --2- 172.21.15.112:0/2905030667 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ea815cb10 0x7f3ea815cf20 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f3ea4005fd0 tx=0x7f3ea402a670 comp rx=0 tx=0).stop 2024-09-21T00:27:33.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:33.637+0000 7f3eafd13640 1 -- 172.21.15.112:0/2905030667 shutdown_connections 2024-09-21T00:27:33.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:33.637+0000 7f3eafd13640 1 --2- 172.21.15.112:0/2905030667 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ea815cb10 0x7f3ea815cf20 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:33.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:33.637+0000 7f3eafd13640 1 -- 172.21.15.112:0/2905030667 >> 172.21.15.112:0/2905030667 conn(0x7f3ea80d2350 msgr2=0x7f3ea80d2760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:33.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:33.637+0000 7f3eafd13640 1 -- 172.21.15.112:0/2905030667 shutdown_connections 2024-09-21T00:27:33.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:33.637+0000 7f3eafd13640 1 -- 172.21.15.112:0/2905030667 wait complete. 2024-09-21T00:27:33.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:33.639+0000 7f3eafd13640 1 Processor -- start 2024-09-21T00:27:33.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:33.639+0000 7f3eafd13640 1 -- start start 2024-09-21T00:27:33.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:33.640+0000 7f3eafd13640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3ea80b6250 0x7f3ea8141c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:33.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:33.640+0000 7f3eafd13640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3ea80d12e0 0x7f3ea815b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:33.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:33.640+0000 7f3eafd13640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ea815cb10 0x7f3ea815e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:33.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:33.640+0000 7f3eafd13640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea80b7100 con 0x7f3ea815cb10 2024-09-21T00:27:33.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:33.640+0000 7f3eafd13640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea80b6f80 con 0x7f3ea80b6250 2024-09-21T00:27:33.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:33.640+0000 7f3eafd13640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3ea80b7280 con 0x7f3ea80d12e0 2024-09-21T00:27:33.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:33.640+0000 7f3eae2b0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ea815cb10 0x7f3ea815e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:33.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:33.640+0000 7f3eadaaf640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3ea80b6250 0x7f3ea8141c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:33.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:33.640+0000 7f3eae2b0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ea815cb10 0x7f3ea815e190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58584/0 (socket says 172.21.15.112:58584) 2024-09-21T00:27:33.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:33.640+0000 7f3eae2b0640 1 -- 172.21.15.112:0/2271443736 learned_addr learned my addr 172.21.15.112:0/2271443736 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:27:33.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:33.640+0000 7f3ead2ae640 1 --2- 172.21.15.112:0/2271443736 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3ea80d12e0 0x7f3ea815b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:33.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:33.640+0000 7f3eae2b0640 1 -- 172.21.15.112:0/2271443736 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3ea80d12e0 msgr2=0x7f3ea815b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:33.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:33.640+0000 7f3eae2b0640 1 --2- 172.21.15.112:0/2271443736 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3ea80d12e0 0x7f3ea815b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:33.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:33.640+0000 7f3eae2b0640 1 -- 172.21.15.112:0/2271443736 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3ea80b6250 msgr2=0x7f3ea8141c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:33.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:33.640+0000 7f3eae2b0640 1 --2- 172.21.15.112:0/2271443736 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3ea80b6250 0x7f3ea8141c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:33.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:33.640+0000 7f3eae2b0640 1 -- 172.21.15.112:0/2271443736 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ea81428e0 con 0x7f3ea815cb10 2024-09-21T00:27:33.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:33.640+0000 7f3eae2b0640 1 --2- 172.21.15.112:0/2271443736 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ea815cb10 0x7f3ea815e190 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f3ea000c700 tx=0x7f3ea000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:33.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:33.641+0000 7f3e9e7fc640 1 -- 172.21.15.112:0/2271443736 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ea0003b70 con 0x7f3ea815cb10 2024-09-21T00:27:33.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:33.641+0000 7f3e9e7fc640 1 -- 172.21.15.112:0/2271443736 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3ea0003d10 con 0x7f3ea815cb10 2024-09-21T00:27:33.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:33.641+0000 7f3eafd13640 1 -- 172.21.15.112:0/2271443736 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ea81f6fd0 con 0x7f3ea815cb10 2024-09-21T00:27:33.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:33.641+0000 7f3e9e7fc640 1 -- 172.21.15.112:0/2271443736 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ea000aab0 con 0x7f3ea815cb10 2024-09-21T00:27:33.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:33.641+0000 7f3eafd13640 1 -- 172.21.15.112:0/2271443736 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ea81f7590 con 0x7f3ea815cb10 2024-09-21T00:27:33.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:33.642+0000 7f3eafd13640 1 -- 172.21.15.112:0/2271443736 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ea815cf20 con 0x7f3ea815cb10 2024-09-21T00:27:33.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:33.643+0000 7f3e9e7fc640 1 -- 172.21.15.112:0/2271443736 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3ea000ac50 con 0x7f3ea815cb10 2024-09-21T00:27:33.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:33.643+0000 7f3e9e7fc640 1 --2- 172.21.15.112:0/2271443736 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f3e8c078b90 0x7f3e8c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:33.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:33.643+0000 7f3e9e7fc640 1 -- 172.21.15.112:0/2271443736 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5619+0+0 (secure 0 0 0) 0x7f3ea009b2b0 con 0x7f3ea815cb10 2024-09-21T00:27:33.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:33.643+0000 7f3eadaaf640 1 --2- 172.21.15.112:0/2271443736 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f3e8c078b90 0x7f3e8c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:33.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:33.644+0000 7f3eadaaf640 1 --2- 172.21.15.112:0/2271443736 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f3e8c078b90 0x7f3e8c07b0a0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f3ea4033000 tx=0x7f3ea4004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:33.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:33.647+0000 7f3e9e7fc640 1 -- 172.21.15.112:0/2271443736 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3ea00632f0 con 0x7f3ea815cb10 2024-09-21T00:27:33.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:33.824+0000 7f3eafd13640 1 -- 172.21.15.112:0/2271443736 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f3ea8142670 con 0x7f3ea815cb10 2024-09-21T00:27:34.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:34.499+0000 7f3e9e7fc640 1 -- 172.21.15.112:0/2271443736 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v58) ==== 170+0+0 (secure 0 0 0) 0x7f3ea0067d20 con 0x7f3ea815cb10 2024-09-21T00:27:34.500 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'cephfs_data' created 2024-09-21T00:27:34.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:34.502+0000 7f3eafd13640 1 -- 172.21.15.112:0/2271443736 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f3e8c078b90 msgr2=0x7f3e8c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:34.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:34.502+0000 7f3eafd13640 1 --2- 172.21.15.112:0/2271443736 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f3e8c078b90 0x7f3e8c07b0a0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f3ea4033000 tx=0x7f3ea4004730 comp rx=0 tx=0).stop 2024-09-21T00:27:34.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:34.502+0000 7f3eafd13640 1 -- 172.21.15.112:0/2271443736 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ea815cb10 msgr2=0x7f3ea815e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:34.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:34.502+0000 7f3eafd13640 1 --2- 172.21.15.112:0/2271443736 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ea815cb10 0x7f3ea815e190 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f3ea000c700 tx=0x7f3ea000cbc0 comp rx=0 tx=0).stop 2024-09-21T00:27:34.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:34.502+0000 7f3eafd13640 1 -- 172.21.15.112:0/2271443736 shutdown_connections 2024-09-21T00:27:34.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:34.502+0000 7f3eafd13640 1 --2- 172.21.15.112:0/2271443736 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f3e8c078b90 0x7f3e8c07b0a0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:34.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:34.502+0000 7f3eafd13640 1 --2- 172.21.15.112:0/2271443736 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ea815cb10 0x7f3ea815e190 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:34.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:34.502+0000 7f3eafd13640 1 --2- 172.21.15.112:0/2271443736 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3ea80d12e0 0x7f3ea815b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:34.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:34.502+0000 7f3eafd13640 1 --2- 172.21.15.112:0/2271443736 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3ea80b6250 0x7f3ea8141c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:34.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:34.502+0000 7f3eafd13640 1 -- 172.21.15.112:0/2271443736 >> 172.21.15.112:0/2271443736 conn(0x7f3ea80d2350 msgr2=0x7f3ea815f3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:34.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:34.502+0000 7f3eafd13640 1 -- 172.21.15.112:0/2271443736 shutdown_connections 2024-09-21T00:27:34.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:34.502+0000 7f3eafd13640 1 -- 172.21.15.112:0/2271443736 wait complete. 2024-09-21T00:27:34.525 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-21T00:27:34.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:34.642+0000 7f2b8d2c4640 1 -- 172.21.15.112:0/282845734 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b88074b00 msgr2=0x7f2b8815aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:34.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:34.642+0000 7f2b8d2c4640 1 --2- 172.21.15.112:0/282845734 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b88074b00 0x7f2b8815aa90 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f2b7800a940 tx=0x7f2b7802b7b0 comp rx=0 tx=0).stop 2024-09-21T00:27:34.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:34.642+0000 7f2b8d2c4640 1 -- 172.21.15.112:0/282845734 shutdown_connections 2024-09-21T00:27:34.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:34.642+0000 7f2b8d2c4640 1 --2- 172.21.15.112:0/282845734 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2b8815afd0 0x7f2b8806fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:34.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:34.642+0000 7f2b8d2c4640 1 --2- 172.21.15.112:0/282845734 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b88074b00 0x7f2b8815aa90 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:34.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:34.642+0000 7f2b8d2c4640 1 -- 172.21.15.112:0/282845734 >> 172.21.15.112:0/282845734 conn(0x7f2b880b1b90 msgr2=0x7f2b880b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:34.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:34.642+0000 7f2b8d2c4640 1 -- 172.21.15.112:0/282845734 shutdown_connections 2024-09-21T00:27:34.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:34.642+0000 7f2b8d2c4640 1 -- 172.21.15.112:0/282845734 wait complete. 2024-09-21T00:27:34.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:34.644+0000 7f2b8d2c4640 1 Processor -- start 2024-09-21T00:27:34.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:34.644+0000 7f2b8d2c4640 1 -- start start 2024-09-21T00:27:34.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:34.645+0000 7f2b8d2c4640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2b880701a0 0x7f2b881f5610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:34.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:34.645+0000 7f2b8d2c4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b88074b00 0x7f2b880b0990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:34.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:34.645+0000 7f2b8d2c4640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2b8815afd0 0x7f2b880cbbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:34.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:34.645+0000 7f2b8d2c4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2b8806e180 con 0x7f2b88074b00 2024-09-21T00:27:34.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:34.645+0000 7f2b8d2c4640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f2b8806e000 con 0x7f2b8815afd0 2024-09-21T00:27:34.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:34.645+0000 7f2b8d2c4640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f2b8806e300 con 0x7f2b880701a0 2024-09-21T00:27:34.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:34.645+0000 7f2b867fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b88074b00 0x7f2b880b0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:34.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:34.645+0000 7f2b867fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b88074b00 0x7f2b880b0990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58604/0 (socket says 172.21.15.112:58604) 2024-09-21T00:27:34.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:34.645+0000 7f2b867fc640 1 -- 172.21.15.112:0/986446662 learned_addr learned my addr 172.21.15.112:0/986446662 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:27:34.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:34.646+0000 7f2b86ffd640 1 --2- 172.21.15.112:0/986446662 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2b880701a0 0x7f2b881f5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:34.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:34.646+0000 7f2b877fe640 1 --2- 172.21.15.112:0/986446662 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2b8815afd0 0x7f2b880cbbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:34.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:34.646+0000 7f2b867fc640 1 -- 172.21.15.112:0/986446662 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2b880701a0 msgr2=0x7f2b881f5610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:34.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:34.646+0000 7f2b867fc640 1 --2- 172.21.15.112:0/986446662 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2b880701a0 0x7f2b881f5610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:34.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:34.646+0000 7f2b867fc640 1 -- 172.21.15.112:0/986446662 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2b8815afd0 msgr2=0x7f2b880cbbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:34.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:34.646+0000 7f2b867fc640 1 --2- 172.21.15.112:0/986446662 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2b8815afd0 0x7f2b880cbbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:34.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:34.646+0000 7f2b867fc640 1 -- 172.21.15.112:0/986446662 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b880c9cf0 con 0x7f2b88074b00 2024-09-21T00:27:34.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:34.646+0000 7f2b867fc640 1 --2- 172.21.15.112:0/986446662 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b88074b00 0x7f2b880b0990 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f2b7c00c970 tx=0x7f2b7c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:34.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:34.646+0000 7f2b6f7fe640 1 -- 172.21.15.112:0/986446662 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b7c007bb0 con 0x7f2b88074b00 2024-09-21T00:27:34.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:34.646+0000 7f2b6f7fe640 1 -- 172.21.15.112:0/986446662 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2b7c007e40 con 0x7f2b88074b00 2024-09-21T00:27:34.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:34.646+0000 7f2b8d2c4640 1 -- 172.21.15.112:0/986446662 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b881f5b50 con 0x7f2b88074b00 2024-09-21T00:27:34.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:34.646+0000 7f2b6f7fe640 1 -- 172.21.15.112:0/986446662 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b7c00f6b0 con 0x7f2b88074b00 2024-09-21T00:27:34.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:34.646+0000 7f2b8d2c4640 1 -- 172.21.15.112:0/986446662 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b880c93c0 con 0x7f2b88074b00 2024-09-21T00:27:34.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:34.647+0000 7f2b8d2c4640 1 -- 172.21.15.112:0/986446662 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b88159ea0 con 0x7f2b88074b00 2024-09-21T00:27:34.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:34.648+0000 7f2b6f7fe640 1 -- 172.21.15.112:0/986446662 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2b7c018400 con 0x7f2b88074b00 2024-09-21T00:27:34.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:34.649+0000 7f2b6f7fe640 1 --2- 172.21.15.112:0/986446662 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2b74078bc0 0x7f2b7407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:34.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:34.649+0000 7f2b6f7fe640 1 -- 172.21.15.112:0/986446662 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 6001+0+0 (secure 0 0 0) 0x7f2b7c09b9c0 con 0x7f2b88074b00 2024-09-21T00:27:34.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:34.649+0000 7f2b86ffd640 1 --2- 172.21.15.112:0/986446662 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2b74078bc0 0x7f2b7407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:34.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:34.650+0000 7f2b86ffd640 1 --2- 172.21.15.112:0/986446662 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2b74078bc0 0x7f2b7407b0d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f2b880b29e0 tx=0x7f2b78004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:34.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:34.653+0000 7f2b6f7fe640 1 -- 172.21.15.112:0/986446662 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2b88159ea0 con 0x7f2b88074b00 2024-09-21T00:27:34.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:34.871+0000 7f2b8d2c4640 1 -- 172.21.15.112:0/986446662 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f2b8815aa90 con 0x7f2b88074b00 2024-09-21T00:27:35.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:35.515+0000 7f2b6f7fe640 1 -- 172.21.15.112:0/986446662 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 9 and data pool 10 v31) ==== 173+0+0 (secure 0 0 0) 0x7f2b881f5b50 con 0x7f2b88074b00 2024-09-21T00:27:35.516 INFO:teuthology.orchestra.run.smithi112.stderr:new fs with metadata pool 9 and data pool 10 2024-09-21T00:27:35.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:35.520+0000 7f2b8d2c4640 1 -- 172.21.15.112:0/986446662 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2b74078bc0 msgr2=0x7f2b7407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:35.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:35.520+0000 7f2b8d2c4640 1 --2- 172.21.15.112:0/986446662 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2b74078bc0 0x7f2b7407b0d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f2b880b29e0 tx=0x7f2b78004730 comp rx=0 tx=0).stop 2024-09-21T00:27:35.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:35.521+0000 7f2b8d2c4640 1 -- 172.21.15.112:0/986446662 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b88074b00 msgr2=0x7f2b880b0990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:35.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:35.521+0000 7f2b8d2c4640 1 --2- 172.21.15.112:0/986446662 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b88074b00 0x7f2b880b0990 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f2b7c00c970 tx=0x7f2b7c00ce30 comp rx=0 tx=0).stop 2024-09-21T00:27:35.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:35.521+0000 7f2b8d2c4640 1 -- 172.21.15.112:0/986446662 shutdown_connections 2024-09-21T00:27:35.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:35.521+0000 7f2b8d2c4640 1 --2- 172.21.15.112:0/986446662 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2b8815afd0 0x7f2b880cbbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:35.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:35.521+0000 7f2b8d2c4640 1 --2- 172.21.15.112:0/986446662 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2b74078bc0 0x7f2b7407b0d0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:35.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:35.522+0000 7f2b8d2c4640 1 --2- 172.21.15.112:0/986446662 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b88074b00 0x7f2b880b0990 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:35.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:35.522+0000 7f2b8d2c4640 1 --2- 172.21.15.112:0/986446662 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2b880701a0 0x7f2b881f5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:35.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:35.522+0000 7f2b8d2c4640 1 -- 172.21.15.112:0/986446662 >> 172.21.15.112:0/986446662 conn(0x7f2b880b1b90 msgr2=0x7f2b88159790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:35.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:35.524+0000 7f2b8d2c4640 1 -- 172.21.15.112:0/986446662 shutdown_connections 2024-09-21T00:27:35.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:35.525+0000 7f2b8d2c4640 1 -- 172.21.15.112:0/986446662 wait complete. 2024-09-21T00:27:35.546 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T00:27:35.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:35.661+0000 7f2ca1c30640 1 -- 172.21.15.112:0/2797097601 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2c9c0b4500 msgr2=0x7f2c9c073d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:35.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:35.661+0000 7f2ca1c30640 1 --2- 172.21.15.112:0/2797097601 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2c9c0b4500 0x7f2c9c073d50 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c0b3af0 tx=0x7f2c9402aa30 comp rx=0 tx=0).stop 2024-09-21T00:27:35.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:35.661+0000 7f2ca1c30640 1 -- 172.21.15.112:0/2797097601 shutdown_connections 2024-09-21T00:27:35.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:35.661+0000 7f2ca1c30640 1 --2- 172.21.15.112:0/2797097601 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2c9c0b4500 0x7f2c9c073d50 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:35.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:35.661+0000 7f2ca1c30640 1 --2- 172.21.15.112:0/2797097601 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2c9c06ec30 0x7f2c9c0b3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:35.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:35.661+0000 7f2ca1c30640 1 --2- 172.21.15.112:0/2797097601 >> v2:172.21.15.187:3300/0 conn(0x7f2c9c0d0b40 0x7f2c9c06e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:35.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:35.661+0000 7f2ca1c30640 1 -- 172.21.15.112:0/2797097601 >> 172.21.15.112:0/2797097601 conn(0x7f2c9c0b2cc0 msgr2=0x7f2c9c0b30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:35.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:35.661+0000 7f2ca1c30640 1 -- 172.21.15.112:0/2797097601 shutdown_connections 2024-09-21T00:27:35.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:35.661+0000 7f2ca1c30640 1 -- 172.21.15.112:0/2797097601 wait complete. 2024-09-21T00:27:35.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:35.663+0000 7f2ca1c30640 1 Processor -- start 2024-09-21T00:27:35.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:35.663+0000 7f2ca1c30640 1 -- start start 2024-09-21T00:27:35.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:35.664+0000 7f2ca1c30640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2c9c06ec30 0x7f2c9c0a9440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:35.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:35.664+0000 7f2ca1c30640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2c9c0b4500 0x7f2c9c14e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:35.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:35.664+0000 7f2ca1c30640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2c9c0d0b40 0x7f2c9c14b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:35.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:35.664+0000 7f2ca1c30640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2c9c15b5d0 con 0x7f2c9c0b4500 2024-09-21T00:27:35.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:35.664+0000 7f2ca1c30640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f2c9c074610 con 0x7f2c9c0d0b40 2024-09-21T00:27:35.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:35.664+0000 7f2ca1c30640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f2c9c15b750 con 0x7f2c9c06ec30 2024-09-21T00:27:35.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:35.665+0000 7f2c9affd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2c9c0b4500 0x7f2c9c14e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:35.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:35.665+0000 7f2c9affd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2c9c0b4500 0x7f2c9c14e300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58616/0 (socket says 172.21.15.112:58616) 2024-09-21T00:27:35.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:35.665+0000 7f2c9affd640 1 -- 172.21.15.112:0/1605767004 learned_addr learned my addr 172.21.15.112:0/1605767004 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:27:35.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:35.665+0000 7f2c9bfff640 1 --2- 172.21.15.112:0/1605767004 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2c9c0d0b40 0x7f2c9c14b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:35.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:35.665+0000 7f2c9b7fe640 1 --2- 172.21.15.112:0/1605767004 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2c9c06ec30 0x7f2c9c0a9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:35.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:35.665+0000 7f2c9affd640 1 -- 172.21.15.112:0/1605767004 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2c9c06ec30 msgr2=0x7f2c9c0a9440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:35.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:35.665+0000 7f2c9affd640 1 --2- 172.21.15.112:0/1605767004 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2c9c06ec30 0x7f2c9c0a9440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:35.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:35.665+0000 7f2c9affd640 1 -- 172.21.15.112:0/1605767004 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2c9c0d0b40 msgr2=0x7f2c9c14b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:35.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:35.665+0000 7f2c9affd640 1 --2- 172.21.15.112:0/1605767004 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2c9c0d0b40 0x7f2c9c14b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:35.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:35.665+0000 7f2c9affd640 1 -- 172.21.15.112:0/1605767004 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c9c0a9980 con 0x7f2c9c0b4500 2024-09-21T00:27:35.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:35.665+0000 7f2c9b7fe640 1 --2- 172.21.15.112:0/1605767004 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2c9c06ec30 0x7f2c9c0a9440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:27:35.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:35.665+0000 7f2c9affd640 1 --2- 172.21.15.112:0/1605767004 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2c9c0b4500 0x7f2c9c14e300 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f2c8400c9a0 tx=0x7f2c8400ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:35.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:35.665+0000 7f2c93fff640 1 -- 172.21.15.112:0/1605767004 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c84007bb0 con 0x7f2c9c0b4500 2024-09-21T00:27:35.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:35.665+0000 7f2c93fff640 1 -- 172.21.15.112:0/1605767004 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2c84005020 con 0x7f2c9c0b4500 2024-09-21T00:27:35.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:35.665+0000 7f2ca1c30640 1 -- 172.21.15.112:0/1605767004 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c9c0a9e90 con 0x7f2c9c0b4500 2024-09-21T00:27:35.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:35.665+0000 7f2c93fff640 1 -- 172.21.15.112:0/1605767004 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c84007e60 con 0x7f2c9c0b4500 2024-09-21T00:27:35.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:35.665+0000 7f2ca1c30640 1 -- 172.21.15.112:0/1605767004 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c9c14db00 con 0x7f2c9c0b4500 2024-09-21T00:27:35.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:35.666+0000 7f2ca1c30640 1 -- 172.21.15.112:0/1605767004 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c9c0a9c00 con 0x7f2c9c0b4500 2024-09-21T00:27:35.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:35.667+0000 7f2c93fff640 1 -- 172.21.15.112:0/1605767004 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2c84005470 con 0x7f2c9c0b4500 2024-09-21T00:27:35.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:35.667+0000 7f2c93fff640 1 --2- 172.21.15.112:0/1605767004 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2c7c078b90 0x7f2c7c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:35.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:35.667+0000 7f2c93fff640 1 -- 172.21.15.112:0/1605767004 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 6101+0+0 (secure 0 0 0) 0x7f2c8409b2f0 con 0x7f2c9c0b4500 2024-09-21T00:27:35.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:35.668+0000 7f2c9b7fe640 1 --2- 172.21.15.112:0/1605767004 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2c7c078b90 0x7f2c7c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:35.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:35.668+0000 7f2c9b7fe640 1 --2- 172.21.15.112:0/1605767004 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2c7c078b90 0x7f2c7c07b0a0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c005980 tx=0x7f2c8c0058f0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:35.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:35.672+0000 7f2c93fff640 1 -- 172.21.15.112:0/1605767004 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2c84014030 con 0x7f2c9c0b4500 2024-09-21T00:27:35.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:35.848+0000 7f2ca1c30640 1 -- 172.21.15.112:0/1605767004 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2c9c073d50 con 0x7f2c9c0b4500 2024-09-21T00:27:35.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:35.849+0000 7f2c93fff640 1 -- 172.21.15.112:0/1605767004 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) ==== 74+0+16981 (secure 0 0 0) 0x7f2c84063630 con 0x7f2c9c0b4500 2024-09-21T00:27:35.851 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:27:35.851 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":59,"fsid":"8b34692b-dc65-49b2-a08f-26f707b96ac5","created":"2024-09-21T00:25:22.788490+0000","modified":"2024-09-21T00:27:34.871760+0000","last_up_change":"2024-09-21T00:25:36.161835+0000","last_in_change":"2024-09-21T00:25:26.346500+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T00:25:37.105006+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-09-21T00:27:32.510522+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-09-21T00:27:33.828833+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9c575086-67d2-4de2-b01b-d44dcc21d604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6809","nonce":755760621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6811","nonce":755760621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6815","nonce":755760621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6813","nonce":755760621}]},"public_addr":"172.21.15.112:6809/755760621","cluster_addr":"172.21.15.112:6811/755760621","heartbeat_back_addr":"172.21.15.112:6815/755760621","heartbeat_front_addr":"172.21.15.112:6813/755760621","state":["exists","up"]},{"osd":1,"uuid":"a983c227-a3ce-4bdd-82f1-438ba7e3ecd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6801","nonce":344932592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6803","nonce":344932592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6807","nonce":344932592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6805","nonce":344932592}]},"public_addr":"172.21.15.112:6801/344932592","cluster_addr":"172.21.15.112:6803/344932592","heartbeat_back_addr":"172.21.15.112:6807/344932592","heartbeat_front_addr":"172.21.15.112:6805/344932592","state":["exists","up"]},{"osd":2,"uuid":"1193c1d8-bfff-4a57-8200-5f69bf1ea3af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6825","nonce":407933616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6827","nonce":407933616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6831","nonce":407933616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6829","nonce":407933616}]},"public_addr":"172.21.15.112:6825/407933616","cluster_addr":"172.21.15.112:6827/407933616","heartbeat_back_addr":"172.21.15.112:6831/407933616","heartbeat_front_addr":"172.21.15.112:6829/407933616","state":["exists","up"]},{"osd":3,"uuid":"af04330a-36ad-40cc-a8dd-fa259b221de5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6817","nonce":4120164806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6819","nonce":4120164806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6823","nonce":4120164806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6821","nonce":4120164806}]},"public_addr":"172.21.15.112:6817/4120164806","cluster_addr":"172.21.15.112:6819/4120164806","heartbeat_back_addr":"172.21.15.112:6823/4120164806","heartbeat_front_addr":"172.21.15.112:6821/4120164806","state":["exists","up"]},{"osd":4,"uuid":"5b893111-292c-4c5a-b184-53ad38c7199e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6803","nonce":522077560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6805","nonce":522077560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6809","nonce":522077560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6807","nonce":522077560}]},"public_addr":"172.21.15.187:6803/522077560","cluster_addr":"172.21.15.187:6805/522077560","heartbeat_back_addr":"172.21.15.187:6809/522077560","heartbeat_front_addr":"172.21.15.187:6807/522077560","state":["exists","up"]},{"osd":5,"uuid":"c0b8e011-45fe-4c5e-925e-6be2a7f1868a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6827","nonce":3593144721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6829","nonce":3593144721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6833","nonce":3593144721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6831","nonce":3593144721}]},"public_addr":"172.21.15.187:6827/3593144721","cluster_addr":"172.21.15.187:6829/3593144721","heartbeat_back_addr":"172.21.15.187:6833/3593144721","heartbeat_front_addr":"172.21.15.187:6831/3593144721","state":["exists","up"]},{"osd":6,"uuid":"87d2d650-7489-4593-98a3-a1f433ce8dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6811","nonce":1494387657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6813","nonce":1494387657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6817","nonce":1494387657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6815","nonce":1494387657}]},"public_addr":"172.21.15.187:6811/1494387657","cluster_addr":"172.21.15.187:6813/1494387657","heartbeat_back_addr":"172.21.15.187:6817/1494387657","heartbeat_front_addr":"172.21.15.187:6815/1494387657","state":["exists","up"]},{"osd":7,"uuid":"c6b4f8e2-a1ee-4438-aa37-67ff4b6ef1f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6819","nonce":1624699842}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6821","nonce":1624699842}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6825","nonce":1624699842}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6823","nonce":1624699842}]},"public_addr":"172.21.15.187:6819/1624699842","cluster_addr":"172.21.15.187:6821/1624699842","heartbeat_back_addr":"172.21.15.187:6825/1624699842","heartbeat_front_addr":"172.21.15.187:6823/1624699842","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.873162+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.400506+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.664317+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.159791+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.202630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.113212+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.384786+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.448441+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T00:27:35.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:35.851+0000 7f2ca1c30640 1 -- 172.21.15.112:0/1605767004 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2c7c078b90 msgr2=0x7f2c7c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:35.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:35.851+0000 7f2ca1c30640 1 --2- 172.21.15.112:0/1605767004 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2c7c078b90 0x7f2c7c07b0a0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c005980 tx=0x7f2c8c0058f0 comp rx=0 tx=0).stop 2024-09-21T00:27:35.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:35.852+0000 7f2ca1c30640 1 -- 172.21.15.112:0/1605767004 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2c9c0b4500 msgr2=0x7f2c9c14e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:35.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:35.852+0000 7f2ca1c30640 1 --2- 172.21.15.112:0/1605767004 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2c9c0b4500 0x7f2c9c14e300 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f2c8400c9a0 tx=0x7f2c8400ce60 comp rx=0 tx=0).stop 2024-09-21T00:27:35.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:35.852+0000 7f2ca1c30640 1 -- 172.21.15.112:0/1605767004 shutdown_connections 2024-09-21T00:27:35.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:35.852+0000 7f2ca1c30640 1 --2- 172.21.15.112:0/1605767004 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2c9c0d0b40 0x7f2c9c14b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:35.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:35.852+0000 7f2ca1c30640 1 --2- 172.21.15.112:0/1605767004 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2c7c078b90 0x7f2c7c07b0a0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:35.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:35.852+0000 7f2ca1c30640 1 --2- 172.21.15.112:0/1605767004 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2c9c0b4500 0x7f2c9c14e300 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:35.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:35.852+0000 7f2ca1c30640 1 --2- 172.21.15.112:0/1605767004 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2c9c06ec30 0x7f2c9c0a9440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:35.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:35.852+0000 7f2ca1c30640 1 -- 172.21.15.112:0/1605767004 >> 172.21.15.112:0/1605767004 conn(0x7f2c9c0b2cc0 msgr2=0x7f2c9c15ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:35.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:35.852+0000 7f2ca1c30640 1 -- 172.21.15.112:0/1605767004 shutdown_connections 2024-09-21T00:27:35.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:35.852+0000 7f2ca1c30640 1 -- 172.21.15.112:0/1605767004 wait complete. 2024-09-21T00:27:35.864 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T00:27:35.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:35.975+0000 7fa9cb39b640 1 -- 172.21.15.112:0/2111607291 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9c40ce0d0 msgr2=0x7fa9c4073e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:35.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:35.975+0000 7fa9cb39b640 1 --2- 172.21.15.112:0/2111607291 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9c40ce0d0 0x7fa9c4073e20 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fa9c4072840 tx=0x7fa9b402a5f0 comp rx=0 tx=0).stop 2024-09-21T00:27:35.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:35.976+0000 7fa9cb39b640 1 -- 172.21.15.112:0/2111607291 shutdown_connections 2024-09-21T00:27:35.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:35.976+0000 7fa9cb39b640 1 --2- 172.21.15.112:0/2111607291 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9c40ce0d0 0x7fa9c4073e20 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:35.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:35.976+0000 7fa9cb39b640 1 --2- 172.21.15.112:0/2111607291 >> v2:172.21.15.187:3300/0 conn(0x7fa9c40ca610 0x7fa9c40cdb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:35.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:35.976+0000 7fa9cb39b640 1 -- 172.21.15.112:0/2111607291 >> 172.21.15.112:0/2111607291 conn(0x7fa9c40cca40 msgr2=0x7fa9c40cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:35.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:35.976+0000 7fa9cb39b640 1 -- 172.21.15.112:0/2111607291 shutdown_connections 2024-09-21T00:27:35.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:35.976+0000 7fa9cb39b640 1 -- 172.21.15.112:0/2111607291 wait complete. 2024-09-21T00:27:35.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:35.977+0000 7fa9cb39b640 1 Processor -- start 2024-09-21T00:27:35.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:35.977+0000 7fa9cb39b640 1 -- start start 2024-09-21T00:27:35.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:35.979+0000 7fa9cb39b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa9c4074360 0x7fa9c40b4630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:35.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:35.979+0000 7fa9cb39b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa9c40ca610 0x7fa9c41f5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:35.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:35.979+0000 7fa9cb39b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9c40ce0d0 0x7fa9c40c89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:35.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:35.979+0000 7fa9cb39b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c415baa0 con 0x7fa9c40ce0d0 2024-09-21T00:27:35.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:35.979+0000 7fa9cb39b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c415b920 con 0x7fa9c40ca610 2024-09-21T00:27:35.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:35.979+0000 7fa9cb39b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa9c415bc20 con 0x7fa9c4074360 2024-09-21T00:27:35.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:35.979+0000 7fa9c9938640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9c40ce0d0 0x7fa9c40c89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:35.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:35.979+0000 7fa9c9938640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9c40ce0d0 0x7fa9c40c89e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58648/0 (socket says 172.21.15.112:58648) 2024-09-21T00:27:35.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:35.979+0000 7fa9c9938640 1 -- 172.21.15.112:0/2568591961 learned_addr learned my addr 172.21.15.112:0/2568591961 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:27:35.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:35.979+0000 7fa9c9137640 1 --2- 172.21.15.112:0/2568591961 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa9c4074360 0x7fa9c40b4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:35.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:35.979+0000 7fa9c8936640 1 --2- 172.21.15.112:0/2568591961 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa9c40ca610 0x7fa9c41f5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:35.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:35.979+0000 7fa9c9938640 1 -- 172.21.15.112:0/2568591961 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa9c4074360 msgr2=0x7fa9c40b4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:35.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:35.979+0000 7fa9c9938640 1 --2- 172.21.15.112:0/2568591961 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa9c4074360 0x7fa9c40b4630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:35.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:35.979+0000 7fa9c9938640 1 -- 172.21.15.112:0/2568591961 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa9c40ca610 msgr2=0x7fa9c41f5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:35.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:35.979+0000 7fa9c9938640 1 --2- 172.21.15.112:0/2568591961 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa9c40ca610 0x7fa9c41f5b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:35.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:35.979+0000 7fa9c9938640 1 -- 172.21.15.112:0/2568591961 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9c4153fd0 con 0x7fa9c40ce0d0 2024-09-21T00:27:35.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:35.979+0000 7fa9c9938640 1 --2- 172.21.15.112:0/2568591961 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9c40ce0d0 0x7fa9c40c89e0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fa9bc002230 tx=0x7fa9bc00cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:35.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:35.979+0000 7fa9b9ffb640 1 -- 172.21.15.112:0/2568591961 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9bc003b70 con 0x7fa9c40ce0d0 2024-09-21T00:27:35.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:35.980+0000 7fa9b9ffb640 1 -- 172.21.15.112:0/2568591961 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa9bc003d10 con 0x7fa9c40ce0d0 2024-09-21T00:27:35.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:35.980+0000 7fa9cb39b640 1 -- 172.21.15.112:0/2568591961 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9c41f60a0 con 0x7fa9c40ce0d0 2024-09-21T00:27:35.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:35.980+0000 7fa9b9ffb640 1 -- 172.21.15.112:0/2568591961 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9bc005330 con 0x7fa9c40ce0d0 2024-09-21T00:27:35.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:35.980+0000 7fa9cb39b640 1 -- 172.21.15.112:0/2568591961 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9c40c7f00 con 0x7fa9c40ce0d0 2024-09-21T00:27:35.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:35.981+0000 7fa9cb39b640 1 -- 172.21.15.112:0/2568591961 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9c415b480 con 0x7fa9c40ce0d0 2024-09-21T00:27:35.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:35.981+0000 7fa9b9ffb640 1 -- 172.21.15.112:0/2568591961 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa9bc0056e0 con 0x7fa9c40ce0d0 2024-09-21T00:27:35.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:35.982+0000 7fa9b9ffb640 1 --2- 172.21.15.112:0/2568591961 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fa9a8078b90 0x7fa9a807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:35.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:35.982+0000 7fa9b9ffb640 1 -- 172.21.15.112:0/2568591961 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 6101+0+0 (secure 0 0 0) 0x7fa9bc09ac80 con 0x7fa9c40ce0d0 2024-09-21T00:27:35.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:35.982+0000 7fa9c9137640 1 --2- 172.21.15.112:0/2568591961 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fa9a8078b90 0x7fa9a807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:35.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:35.982+0000 7fa9c9137640 1 --2- 172.21.15.112:0/2568591961 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fa9a8078b90 0x7fa9a807b0a0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa9c4072b40 tx=0x7fa9c0005f90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:35.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:35.986+0000 7fa9b9ffb640 1 -- 172.21.15.112:0/2568591961 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa9c415b480 con 0x7fa9c40ce0d0 2024-09-21T00:27:36.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:36.162+0000 7fa9cb39b640 1 -- 172.21.15.112:0/2568591961 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa9c41541a0 con 0x7fa9c40ce0d0 2024-09-21T00:27:36.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:36.163+0000 7fa9b9ffb640 1 -- 172.21.15.112:0/2568591961 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) ==== 74+0+16981 (secure 0 0 0) 0x7fa9bc062a00 con 0x7fa9c40ce0d0 2024-09-21T00:27:36.164 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:27:36.165 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":59,"fsid":"8b34692b-dc65-49b2-a08f-26f707b96ac5","created":"2024-09-21T00:25:22.788490+0000","modified":"2024-09-21T00:27:34.871760+0000","last_up_change":"2024-09-21T00:25:36.161835+0000","last_in_change":"2024-09-21T00:25:26.346500+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T00:25:37.105006+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-09-21T00:27:32.510522+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-09-21T00:27:33.828833+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9c575086-67d2-4de2-b01b-d44dcc21d604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6809","nonce":755760621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6811","nonce":755760621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6815","nonce":755760621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6813","nonce":755760621}]},"public_addr":"172.21.15.112:6809/755760621","cluster_addr":"172.21.15.112:6811/755760621","heartbeat_back_addr":"172.21.15.112:6815/755760621","heartbeat_front_addr":"172.21.15.112:6813/755760621","state":["exists","up"]},{"osd":1,"uuid":"a983c227-a3ce-4bdd-82f1-438ba7e3ecd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6801","nonce":344932592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6803","nonce":344932592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6807","nonce":344932592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6805","nonce":344932592}]},"public_addr":"172.21.15.112:6801/344932592","cluster_addr":"172.21.15.112:6803/344932592","heartbeat_back_addr":"172.21.15.112:6807/344932592","heartbeat_front_addr":"172.21.15.112:6805/344932592","state":["exists","up"]},{"osd":2,"uuid":"1193c1d8-bfff-4a57-8200-5f69bf1ea3af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6825","nonce":407933616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6827","nonce":407933616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6831","nonce":407933616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6829","nonce":407933616}]},"public_addr":"172.21.15.112:6825/407933616","cluster_addr":"172.21.15.112:6827/407933616","heartbeat_back_addr":"172.21.15.112:6831/407933616","heartbeat_front_addr":"172.21.15.112:6829/407933616","state":["exists","up"]},{"osd":3,"uuid":"af04330a-36ad-40cc-a8dd-fa259b221de5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6817","nonce":4120164806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6819","nonce":4120164806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6823","nonce":4120164806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6821","nonce":4120164806}]},"public_addr":"172.21.15.112:6817/4120164806","cluster_addr":"172.21.15.112:6819/4120164806","heartbeat_back_addr":"172.21.15.112:6823/4120164806","heartbeat_front_addr":"172.21.15.112:6821/4120164806","state":["exists","up"]},{"osd":4,"uuid":"5b893111-292c-4c5a-b184-53ad38c7199e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6803","nonce":522077560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6805","nonce":522077560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6809","nonce":522077560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6807","nonce":522077560}]},"public_addr":"172.21.15.187:6803/522077560","cluster_addr":"172.21.15.187:6805/522077560","heartbeat_back_addr":"172.21.15.187:6809/522077560","heartbeat_front_addr":"172.21.15.187:6807/522077560","state":["exists","up"]},{"osd":5,"uuid":"c0b8e011-45fe-4c5e-925e-6be2a7f1868a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6827","nonce":3593144721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6829","nonce":3593144721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6833","nonce":3593144721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6831","nonce":3593144721}]},"public_addr":"172.21.15.187:6827/3593144721","cluster_addr":"172.21.15.187:6829/3593144721","heartbeat_back_addr":"172.21.15.187:6833/3593144721","heartbeat_front_addr":"172.21.15.187:6831/3593144721","state":["exists","up"]},{"osd":6,"uuid":"87d2d650-7489-4593-98a3-a1f433ce8dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6811","nonce":1494387657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6813","nonce":1494387657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6817","nonce":1494387657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6815","nonce":1494387657}]},"public_addr":"172.21.15.187:6811/1494387657","cluster_addr":"172.21.15.187:6813/1494387657","heartbeat_back_addr":"172.21.15.187:6817/1494387657","heartbeat_front_addr":"172.21.15.187:6815/1494387657","state":["exists","up"]},{"osd":7,"uuid":"c6b4f8e2-a1ee-4438-aa37-67ff4b6ef1f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6819","nonce":1624699842}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6821","nonce":1624699842}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6825","nonce":1624699842}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6823","nonce":1624699842}]},"public_addr":"172.21.15.187:6819/1624699842","cluster_addr":"172.21.15.187:6821/1624699842","heartbeat_back_addr":"172.21.15.187:6825/1624699842","heartbeat_front_addr":"172.21.15.187:6823/1624699842","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.873162+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.400506+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.664317+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.159791+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.202630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.113212+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.384786+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.448441+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T00:27:36.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:36.166+0000 7fa9cb39b640 1 -- 172.21.15.112:0/2568591961 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fa9a8078b90 msgr2=0x7fa9a807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:36.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:36.166+0000 7fa9cb39b640 1 --2- 172.21.15.112:0/2568591961 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fa9a8078b90 0x7fa9a807b0a0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa9c4072b40 tx=0x7fa9c0005f90 comp rx=0 tx=0).stop 2024-09-21T00:27:36.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:36.166+0000 7fa9cb39b640 1 -- 172.21.15.112:0/2568591961 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9c40ce0d0 msgr2=0x7fa9c40c89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:36.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:36.166+0000 7fa9cb39b640 1 --2- 172.21.15.112:0/2568591961 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9c40ce0d0 0x7fa9c40c89e0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fa9bc002230 tx=0x7fa9bc00cb90 comp rx=0 tx=0).stop 2024-09-21T00:27:36.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:36.166+0000 7fa9cb39b640 1 -- 172.21.15.112:0/2568591961 shutdown_connections 2024-09-21T00:27:36.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:36.166+0000 7fa9cb39b640 1 --2- 172.21.15.112:0/2568591961 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fa9a8078b90 0x7fa9a807b0a0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:36.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:36.166+0000 7fa9cb39b640 1 --2- 172.21.15.112:0/2568591961 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9c40ce0d0 0x7fa9c40c89e0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:36.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:36.166+0000 7fa9cb39b640 1 --2- 172.21.15.112:0/2568591961 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa9c40ca610 0x7fa9c41f5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:36.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:36.166+0000 7fa9cb39b640 1 --2- 172.21.15.112:0/2568591961 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa9c4074360 0x7fa9c40b4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:36.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:36.166+0000 7fa9cb39b640 1 -- 172.21.15.112:0/2568591961 >> 172.21.15.112:0/2568591961 conn(0x7fa9c40cca40 msgr2=0x7fa9c40af9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:36.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:36.166+0000 7fa9cb39b640 1 -- 172.21.15.112:0/2568591961 shutdown_connections 2024-09-21T00:27:36.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:36.166+0000 7fa9cb39b640 1 -- 172.21.15.112:0/2568591961 wait complete. 2024-09-21T00:27:36.178 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-21T00:27:36.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:36.294+0000 7f71eab90640 1 -- 172.21.15.112:0/3906021692 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71e40b9ad0 msgr2=0x7f71e40d0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:36.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:36.294+0000 7f71eab90640 1 --2- 172.21.15.112:0/3906021692 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71e40b9ad0 0x7f71e40d0be0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f71d8005fd0 tx=0x7f71d802a980 comp rx=0 tx=0).stop 2024-09-21T00:27:36.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:36.294+0000 7f71eab90640 1 -- 172.21.15.112:0/3906021692 shutdown_connections 2024-09-21T00:27:36.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:36.294+0000 7f71eab90640 1 --2- 172.21.15.112:0/3906021692 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71e40b9ad0 0x7f71e40d0be0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:36.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:36.294+0000 7f71eab90640 1 --2- 172.21.15.112:0/3906021692 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f71e415ee80 0x7f71e40d0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:36.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:36.294+0000 7f71eab90640 1 -- 172.21.15.112:0/3906021692 >> 172.21.15.112:0/3906021692 conn(0x7f71e409d830 msgr2=0x7f71e40b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:36.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:36.294+0000 7f71eab90640 1 -- 172.21.15.112:0/3906021692 shutdown_connections 2024-09-21T00:27:36.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:36.294+0000 7f71eab90640 1 -- 172.21.15.112:0/3906021692 wait complete. 2024-09-21T00:27:36.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:36.296+0000 7f71eab90640 1 Processor -- start 2024-09-21T00:27:36.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:36.296+0000 7f71eab90640 1 -- start start 2024-09-21T00:27:36.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:36.297+0000 7f71eab90640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f71e40b9ad0 0x7f71e414c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:36.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:36.297+0000 7f71eab90640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71e40d1120 0x7f71e40d3e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:36.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:36.297+0000 7f71eab90640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f71e415ee80 0x7f71e406efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:36.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:36.297+0000 7f71eab90640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f71e40b6700 con 0x7f71e40d1120 2024-09-21T00:27:36.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:36.297+0000 7f71eab90640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f71e40b6580 con 0x7f71e415ee80 2024-09-21T00:27:36.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:36.297+0000 7f71eab90640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f71e40b6880 con 0x7f71e40b9ad0 2024-09-21T00:27:36.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:36.297+0000 7f71e3fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71e40d1120 0x7f71e40d3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:36.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:36.297+0000 7f71e3fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71e40d1120 0x7f71e40d3e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48178/0 (socket says 172.21.15.112:48178) 2024-09-21T00:27:36.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:36.297+0000 7f71e3fff640 1 -- 172.21.15.112:0/3758888823 learned_addr learned my addr 172.21.15.112:0/3758888823 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:27:36.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:36.297+0000 7f71e892c640 1 --2- 172.21.15.112:0/3758888823 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f71e40b9ad0 0x7f71e414c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:36.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:36.297+0000 7f71e912d640 1 --2- 172.21.15.112:0/3758888823 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f71e415ee80 0x7f71e406efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:36.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:36.298+0000 7f71e3fff640 1 -- 172.21.15.112:0/3758888823 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f71e40b9ad0 msgr2=0x7f71e414c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:36.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:36.298+0000 7f71e3fff640 1 --2- 172.21.15.112:0/3758888823 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f71e40b9ad0 0x7f71e414c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:36.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:36.298+0000 7f71e3fff640 1 -- 172.21.15.112:0/3758888823 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f71e415ee80 msgr2=0x7f71e406efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:36.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:36.298+0000 7f71e3fff640 1 --2- 172.21.15.112:0/3758888823 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f71e415ee80 0x7f71e406efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:36.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:36.298+0000 7f71e3fff640 1 -- 172.21.15.112:0/3758888823 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71e40a0360 con 0x7f71e40d1120 2024-09-21T00:27:36.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:36.298+0000 7f71e3fff640 1 --2- 172.21.15.112:0/3758888823 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71e40d1120 0x7f71e40d3e90 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f71d8006010 tx=0x7f71d8003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:36.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:36.298+0000 7f71e17fa640 1 -- 172.21.15.112:0/3758888823 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d8034040 con 0x7f71e40d1120 2024-09-21T00:27:36.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:36.298+0000 7f71e17fa640 1 -- 172.21.15.112:0/3758888823 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f71d8004e30 con 0x7f71e40d1120 2024-09-21T00:27:36.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:36.298+0000 7f71eab90640 1 -- 172.21.15.112:0/3758888823 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71e40a0640 con 0x7f71e40d1120 2024-09-21T00:27:36.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:36.298+0000 7f71e17fa640 1 -- 172.21.15.112:0/3758888823 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d8004490 con 0x7f71e40d1120 2024-09-21T00:27:36.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:36.298+0000 7f71eab90640 1 -- 172.21.15.112:0/3758888823 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71e40a0c00 con 0x7f71e40d1120 2024-09-21T00:27:36.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:36.299+0000 7f71eab90640 1 -- 172.21.15.112:0/3758888823 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71e40b6d30 con 0x7f71e40d1120 2024-09-21T00:27:36.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:36.300+0000 7f71e17fa640 1 -- 172.21.15.112:0/3758888823 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f71d8037020 con 0x7f71e40d1120 2024-09-21T00:27:36.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:36.300+0000 7f71e17fa640 1 --2- 172.21.15.112:0/3758888823 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f71d0078bc0 0x7f71d007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:36.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:36.300+0000 7f71e17fa640 1 -- 172.21.15.112:0/3758888823 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 6101+0+0 (secure 0 0 0) 0x7f71d80bc200 con 0x7f71e40d1120 2024-09-21T00:27:36.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:36.300+0000 7f71e892c640 1 --2- 172.21.15.112:0/3758888823 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f71d0078bc0 0x7f71d007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:36.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:36.301+0000 7f71e892c640 1 --2- 172.21.15.112:0/3758888823 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f71d0078bc0 0x7f71d007b0d0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f71d4009b30 tx=0x7f71d4005f90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:36.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:36.305+0000 7f71e17fa640 1 -- 172.21.15.112:0/3758888823 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f71d8083f00 con 0x7f71e40d1120 2024-09-21T00:27:36.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:36.524+0000 7f71eab90640 1 -- 172.21.15.112:0/3758888823 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f71e4161a10 con 0x7f71e40d1120 2024-09-21T00:27:37.491 INFO:tasks.ceph.mds.a.smithi112.stderr:2024-09-21T00:27:37.491+0000 7fd10fa16640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T00:27:37.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:37.527+0000 7f71e17fa640 1 -- 172.21.15.112:0/3758888823 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v34) ==== 118+0+0 (secure 0 0 0) 0x7f71d8088930 con 0x7f71e40d1120 2024-09-21T00:27:37.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:37.530+0000 7f71eab90640 1 -- 172.21.15.112:0/3758888823 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f71d0078bc0 msgr2=0x7f71d007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:37.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:37.530+0000 7f71eab90640 1 --2- 172.21.15.112:0/3758888823 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f71d0078bc0 0x7f71d007b0d0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f71d4009b30 tx=0x7f71d4005f90 comp rx=0 tx=0).stop 2024-09-21T00:27:37.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:37.530+0000 7f71eab90640 1 -- 172.21.15.112:0/3758888823 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71e40d1120 msgr2=0x7f71e40d3e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:37.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:37.530+0000 7f71eab90640 1 --2- 172.21.15.112:0/3758888823 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71e40d1120 0x7f71e40d3e90 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f71d8006010 tx=0x7f71d8003f90 comp rx=0 tx=0).stop 2024-09-21T00:27:37.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:37.530+0000 7f71eab90640 1 -- 172.21.15.112:0/3758888823 shutdown_connections 2024-09-21T00:27:37.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:37.530+0000 7f71eab90640 1 --2- 172.21.15.112:0/3758888823 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f71e415ee80 0x7f71e406efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:37.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:37.530+0000 7f71eab90640 1 --2- 172.21.15.112:0/3758888823 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f71d0078bc0 0x7f71d007b0d0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:37.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:37.530+0000 7f71eab90640 1 --2- 172.21.15.112:0/3758888823 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71e40d1120 0x7f71e40d3e90 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:37.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:37.530+0000 7f71eab90640 1 --2- 172.21.15.112:0/3758888823 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f71e40b9ad0 0x7f71e414c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:37.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:37.530+0000 7f71eab90640 1 -- 172.21.15.112:0/3758888823 >> 172.21.15.112:0/3758888823 conn(0x7f71e409d830 msgr2=0x7f71e40cadb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:37.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:37.530+0000 7f71eab90640 1 -- 172.21.15.112:0/3758888823 shutdown_connections 2024-09-21T00:27:37.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:37.530+0000 7f71eab90640 1 -- 172.21.15.112:0/3758888823 wait complete. 2024-09-21T00:27:37.545 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-21T00:27:37.545 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-21T00:27:37.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:37.666+0000 7fe238932640 1 -- 172.21.15.112:0/587150645 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2340b9ad0 msgr2=0x7fe2340d0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:37.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:37.666+0000 7fe238932640 1 --2- 172.21.15.112:0/587150645 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2340b9ad0 0x7fe2340d0be0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fe2340cfba0 tx=0x7fe22802a410 comp rx=0 tx=0).stop 2024-09-21T00:27:37.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:37.666+0000 7fe238932640 1 -- 172.21.15.112:0/587150645 shutdown_connections 2024-09-21T00:27:37.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:37.666+0000 7fe238932640 1 --2- 172.21.15.112:0/587150645 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2340b9ad0 0x7fe2340d0be0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:37.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:37.666+0000 7fe238932640 1 --2- 172.21.15.112:0/587150645 >> v2:172.21.15.187:3300/0 conn(0x7fe23415ee80 0x7fe2340d0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:37.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:37.666+0000 7fe238932640 1 -- 172.21.15.112:0/587150645 >> 172.21.15.112:0/587150645 conn(0x7fe23409d830 msgr2=0x7fe2340b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:37.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:37.666+0000 7fe238932640 1 -- 172.21.15.112:0/587150645 shutdown_connections 2024-09-21T00:27:37.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:37.666+0000 7fe238932640 1 -- 172.21.15.112:0/587150645 wait complete. 2024-09-21T00:27:37.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:37.668+0000 7fe238932640 1 Processor -- start 2024-09-21T00:27:37.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:37.668+0000 7fe238932640 1 -- start start 2024-09-21T00:27:37.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:37.669+0000 7fe238932640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2340b9ad0 0x7fe2341575a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:37.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:37.669+0000 7fe238932640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2340d1120 0x7fe23415c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:37.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:37.669+0000 7fe238932640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe23415ee80 0x7fe2340b2750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:37.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:37.669+0000 7fe238932640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe2340b6700 con 0x7fe2340b9ad0 2024-09-21T00:27:37.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:37.669+0000 7fe238932640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe2340b6580 con 0x7fe23415ee80 2024-09-21T00:27:37.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:37.669+0000 7fe238932640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe2340b6880 con 0x7fe2340d1120 2024-09-21T00:27:37.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:37.669+0000 7fe23259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2340b9ad0 0x7fe2341575a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:37.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:37.669+0000 7fe23259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2340b9ad0 0x7fe2341575a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48196/0 (socket says 172.21.15.112:48196) 2024-09-21T00:27:37.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:37.669+0000 7fe23259c640 1 -- 172.21.15.112:0/1677261895 learned_addr learned my addr 172.21.15.112:0/1677261895 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:27:37.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:37.669+0000 7fe232d9d640 1 --2- 172.21.15.112:0/1677261895 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe23415ee80 0x7fe2340b2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:37.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:37.669+0000 7fe231d9b640 1 --2- 172.21.15.112:0/1677261895 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2340d1120 0x7fe23415c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:37.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:37.669+0000 7fe23259c640 1 -- 172.21.15.112:0/1677261895 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2340d1120 msgr2=0x7fe23415c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:37.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:37.669+0000 7fe23259c640 1 --2- 172.21.15.112:0/1677261895 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2340d1120 0x7fe23415c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:37.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:37.669+0000 7fe23259c640 1 -- 172.21.15.112:0/1677261895 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe23415ee80 msgr2=0x7fe2340b2750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:37.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:37.669+0000 7fe23259c640 1 --2- 172.21.15.112:0/1677261895 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe23415ee80 0x7fe2340b2750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:37.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:37.669+0000 7fe23259c640 1 -- 172.21.15.112:0/1677261895 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2341f6060 con 0x7fe2340b9ad0 2024-09-21T00:27:37.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:37.669+0000 7fe231d9b640 1 --2- 172.21.15.112:0/1677261895 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2340d1120 0x7fe23415c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:27:37.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:37.670+0000 7fe23259c640 1 --2- 172.21.15.112:0/1677261895 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2340b9ad0 0x7fe2341575a0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fe22400c9a0 tx=0x7fe22400ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:37.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:37.670+0000 7fe222ffd640 1 -- 172.21.15.112:0/1677261895 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe224007bb0 con 0x7fe2340b9ad0 2024-09-21T00:27:37.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:37.670+0000 7fe222ffd640 1 -- 172.21.15.112:0/1677261895 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe224005020 con 0x7fe2340b9ad0 2024-09-21T00:27:37.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:37.670+0000 7fe238932640 1 -- 172.21.15.112:0/1677261895 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2341f6340 con 0x7fe2340b9ad0 2024-09-21T00:27:37.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:37.670+0000 7fe222ffd640 1 -- 172.21.15.112:0/1677261895 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe224007e60 con 0x7fe2340b9ad0 2024-09-21T00:27:37.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:37.670+0000 7fe238932640 1 -- 172.21.15.112:0/1677261895 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2341f6900 con 0x7fe2340b9ad0 2024-09-21T00:27:37.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:37.671+0000 7fe238932640 1 -- 172.21.15.112:0/1677261895 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2340b6d50 con 0x7fe2340b9ad0 2024-09-21T00:27:37.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:37.672+0000 7fe222ffd640 1 -- 172.21.15.112:0/1677261895 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe224005470 con 0x7fe2340b9ad0 2024-09-21T00:27:37.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:37.672+0000 7fe222ffd640 1 --2- 172.21.15.112:0/1677261895 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fe214078b90 0x7fe21407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:37.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:37.673+0000 7fe222ffd640 1 -- 172.21.15.112:0/1677261895 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7fe22409b340 con 0x7fe2340b9ad0 2024-09-21T00:27:37.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:37.673+0000 7fe231d9b640 1 --2- 172.21.15.112:0/1677261895 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fe214078b90 0x7fe21407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:37.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:37.673+0000 7fe231d9b640 1 --2- 172.21.15.112:0/1677261895 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fe214078b90 0x7fe21407b0a0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe228034000 tx=0x7fe228004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:37.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:37.677+0000 7fe222ffd640 1 -- 172.21.15.112:0/1677261895 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe224014030 con 0x7fe2340b9ad0 2024-09-21T00:27:37.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:37.889+0000 7fe238932640 1 -- 172.21.15.112:0/1677261895 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fe2340d4bf0 con 0x7fe2340b9ad0 2024-09-21T00:27:38.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:38.528+0000 7fe222ffd640 1 -- 172.21.15.112:0/1677261895 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v35) ==== 122+0+0 (secure 0 0 0) 0x7fe224063680 con 0x7fe2340b9ad0 2024-09-21T00:27:38.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:38.531+0000 7fe238932640 1 -- 172.21.15.112:0/1677261895 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fe214078b90 msgr2=0x7fe21407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:38.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:38.531+0000 7fe238932640 1 --2- 172.21.15.112:0/1677261895 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fe214078b90 0x7fe21407b0a0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe228034000 tx=0x7fe228004730 comp rx=0 tx=0).stop 2024-09-21T00:27:38.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:38.531+0000 7fe238932640 1 -- 172.21.15.112:0/1677261895 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2340b9ad0 msgr2=0x7fe2341575a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:38.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:38.531+0000 7fe238932640 1 --2- 172.21.15.112:0/1677261895 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2340b9ad0 0x7fe2341575a0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fe22400c9a0 tx=0x7fe22400ce60 comp rx=0 tx=0).stop 2024-09-21T00:27:38.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:38.531+0000 7fe238932640 1 -- 172.21.15.112:0/1677261895 shutdown_connections 2024-09-21T00:27:38.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:38.531+0000 7fe238932640 1 --2- 172.21.15.112:0/1677261895 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe23415ee80 0x7fe2340b2750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:38.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:38.531+0000 7fe238932640 1 --2- 172.21.15.112:0/1677261895 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2340d1120 0x7fe23415c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:38.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:38.531+0000 7fe238932640 1 --2- 172.21.15.112:0/1677261895 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fe214078b90 0x7fe21407b0a0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:38.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:38.531+0000 7fe238932640 1 --2- 172.21.15.112:0/1677261895 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2340b9ad0 0x7fe2341575a0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:38.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:38.531+0000 7fe238932640 1 -- 172.21.15.112:0/1677261895 >> 172.21.15.112:0/1677261895 conn(0x7fe23409d830 msgr2=0x7fe2340b7370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:38.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:38.531+0000 7fe238932640 1 -- 172.21.15.112:0/1677261895 shutdown_connections 2024-09-21T00:27:38.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:38.531+0000 7fe238932640 1 -- 172.21.15.112:0/1677261895 wait complete. 2024-09-21T00:27:38.544 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-21T00:27:38.544 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T00:27:38.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:38.657+0000 7f0920190640 1 -- 172.21.15.112:0/4238015541 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0904003a30 msgr2=0x7f0904003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:38.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:38.657+0000 7f0920190640 1 --2- 172.21.15.112:0/4238015541 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0904003a30 0x7f0904003ea0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f0914006fd0 tx=0x7f0904005830 comp rx=0 tx=0).stop 2024-09-21T00:27:38.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:38.657+0000 7f091e72d640 1 -- 172.21.15.112:0/4238015541 reap_dead start 2024-09-21T00:27:38.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:38.657+0000 7f0920190640 1 -- 172.21.15.112:0/4238015541 shutdown_connections 2024-09-21T00:27:38.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:38.657+0000 7f0920190640 1 -- 172.21.15.112:0/4238015541 >> 172.21.15.112:0/4238015541 conn(0x7f09180b6b20 msgr2=0x7f09180d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:38.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:38.657+0000 7f0920190640 1 -- 172.21.15.112:0/4238015541 shutdown_connections 2024-09-21T00:27:38.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:38.657+0000 7f0920190640 1 -- 172.21.15.112:0/4238015541 wait complete. 2024-09-21T00:27:38.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:38.659+0000 7f0920190640 1 Processor -- start 2024-09-21T00:27:38.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:38.659+0000 7f0920190640 1 -- start start 2024-09-21T00:27:38.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:38.660+0000 7f0920190640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0904003a30 0x7f09180cd000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:38.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:38.660+0000 7f0920190640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f09181556b0 0x7f09181f8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:38.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:38.660+0000 7f0920190640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0918160cf0 0x7f0918161100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:38.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:38.660+0000 7f0920190640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f09180b31d0 con 0x7f09181556b0 2024-09-21T00:27:38.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:38.660+0000 7f0920190640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f09180b3050 con 0x7f0904003a30 2024-09-21T00:27:38.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:38.660+0000 7f0920190640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f09180b3350 con 0x7f0918160cf0 2024-09-21T00:27:38.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:38.660+0000 7f091d72b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f09181556b0 0x7f09181f8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:38.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:38.660+0000 7f091d72b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f09181556b0 0x7f09181f8fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48218/0 (socket says 172.21.15.112:48218) 2024-09-21T00:27:38.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:38.660+0000 7f091d72b640 1 -- 172.21.15.112:0/1626377054 learned_addr learned my addr 172.21.15.112:0/1626377054 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:27:38.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:38.660+0000 7f091df2c640 1 --2- 172.21.15.112:0/1626377054 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0904003a30 0x7f09180cd000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:38.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:38.660+0000 7f091e72d640 1 --2- 172.21.15.112:0/1626377054 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0918160cf0 0x7f0918161100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:38.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:38.660+0000 7f091d72b640 1 -- 172.21.15.112:0/1626377054 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0918160cf0 msgr2=0x7f0918161100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:38.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:38.660+0000 7f091d72b640 1 --2- 172.21.15.112:0/1626377054 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0918160cf0 0x7f0918161100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:38.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:38.660+0000 7f091d72b640 1 -- 172.21.15.112:0/1626377054 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0904003a30 msgr2=0x7f09180cd000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:38.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:38.660+0000 7f091d72b640 1 --2- 172.21.15.112:0/1626377054 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0904003a30 0x7f09180cd000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:38.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:38.660+0000 7f091d72b640 1 -- 172.21.15.112:0/1626377054 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0918161640 con 0x7f09181556b0 2024-09-21T00:27:38.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:38.660+0000 7f091df2c640 1 --2- 172.21.15.112:0/1626377054 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0904003a30 0x7f09180cd000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:27:38.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:38.660+0000 7f091d72b640 1 --2- 172.21.15.112:0/1626377054 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f09181556b0 0x7f09181f8fc0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f0908004e80 tx=0x7f09080015d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:38.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:38.661+0000 7f090e7fc640 1 -- 172.21.15.112:0/1626377054 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0908015020 con 0x7f09181556b0 2024-09-21T00:27:38.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:38.661+0000 7f090e7fc640 1 -- 172.21.15.112:0/1626377054 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0908011980 con 0x7f09181556b0 2024-09-21T00:27:38.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:38.661+0000 7f0920190640 1 -- 172.21.15.112:0/1626377054 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09180caef0 con 0x7f09181556b0 2024-09-21T00:27:38.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:38.661+0000 7f090e7fc640 1 -- 172.21.15.112:0/1626377054 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0908011dd0 con 0x7f09181556b0 2024-09-21T00:27:38.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:38.661+0000 7f0920190640 1 -- 172.21.15.112:0/1626377054 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09180cb670 con 0x7f09181556b0 2024-09-21T00:27:38.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:38.662+0000 7f0920190640 1 -- 172.21.15.112:0/1626377054 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09180b38e0 con 0x7f09181556b0 2024-09-21T00:27:38.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:38.662+0000 7f090e7fc640 1 -- 172.21.15.112:0/1626377054 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0908018020 con 0x7f09181556b0 2024-09-21T00:27:38.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:38.663+0000 7f090e7fc640 1 --2- 172.21.15.112:0/1626377054 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f08fc078b90 0x7f08fc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:38.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:38.663+0000 7f090e7fc640 1 -- 172.21.15.112:0/1626377054 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7f090809b8f0 con 0x7f09181556b0 2024-09-21T00:27:38.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:38.663+0000 7f091df2c640 1 --2- 172.21.15.112:0/1626377054 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f08fc078b90 0x7f08fc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:38.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:38.664+0000 7f091df2c640 1 --2- 172.21.15.112:0/1626377054 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f08fc078b90 0x7f08fc07b0a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f0914033000 tx=0x7f0914036040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:38.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:38.667+0000 7f090e7fc640 1 -- 172.21.15.112:0/1626377054 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f090801f020 con 0x7f09181556b0 2024-09-21T00:27:38.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:38.878+0000 7f0920190640 1 -- 172.21.15.112:0/1626377054 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f09180cf340 con 0x7f09181556b0 2024-09-21T00:27:38.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:38.879+0000 7f090e7fc640 1 -- 172.21.15.112:0/1626377054 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+5233 (secure 0 0 0) 0x7f0918161640 con 0x7f09181556b0 2024-09-21T00:27:38.880 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:27:38.880 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":35,"btime":"2024-09-21T00:27:38:527458+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5389,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6836/3984762342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":3984762342},{"type":"v1","addr":"172.21.15.187:6836","nonce":3984762342}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5392,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6837/763698806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6835","nonce":763698806},{"type":"v1","addr":"172.21.15.187:6837","nonce":763698806}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5736,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6834/1432588455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1432588455},{"type":"v1","addr":"172.21.15.112:6834","nonce":1432588455}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T00:27:34.871721+0000","modified":"2024-09-21T00:27:37.890630+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5739},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5739":{"gid":5739,"name":"a","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6835/3610025869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":3610025869},{"type":"v1","addr":"172.21.15.112:6835","nonce":3610025869}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5739,"qdb_cluster":[5739]},"id":4}]} 2024-09-21T00:27:38.880 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 35 2024-09-21T00:27:38.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:38.881+0000 7f0920190640 1 -- 172.21.15.112:0/1626377054 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f08fc078b90 msgr2=0x7f08fc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:38.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:38.881+0000 7f0920190640 1 --2- 172.21.15.112:0/1626377054 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f08fc078b90 0x7f08fc07b0a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f0914033000 tx=0x7f0914036040 comp rx=0 tx=0).stop 2024-09-21T00:27:38.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:38.881+0000 7f0920190640 1 -- 172.21.15.112:0/1626377054 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f09181556b0 msgr2=0x7f09181f8fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:38.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:38.881+0000 7f0920190640 1 --2- 172.21.15.112:0/1626377054 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f09181556b0 0x7f09181f8fc0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f0908004e80 tx=0x7f09080015d0 comp rx=0 tx=0).stop 2024-09-21T00:27:38.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:38.881+0000 7f0920190640 1 -- 172.21.15.112:0/1626377054 shutdown_connections 2024-09-21T00:27:38.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:38.881+0000 7f0920190640 1 --2- 172.21.15.112:0/1626377054 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0918160cf0 0x7f0918161100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:38.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:38.881+0000 7f0920190640 1 --2- 172.21.15.112:0/1626377054 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f08fc078b90 0x7f08fc07b0a0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:38.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:38.881+0000 7f0920190640 1 --2- 172.21.15.112:0/1626377054 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f09181556b0 0x7f09181f8fc0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:38.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:38.881+0000 7f0920190640 1 --2- 172.21.15.112:0/1626377054 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0904003a30 0x7f09180cd000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:38.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:38.881+0000 7f0920190640 1 -- 172.21.15.112:0/1626377054 >> 172.21.15.112:0/1626377054 conn(0x7f09180b6b20 msgr2=0x7f091806e210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:38.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:38.881+0000 7f0920190640 1 -- 172.21.15.112:0/1626377054 shutdown_connections 2024-09-21T00:27:38.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:38.881+0000 7f0920190640 1 -- 172.21.15.112:0/1626377054 wait complete. 2024-09-21T00:27:38.893 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T00:27:39.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.005+0000 7f83c3369640 1 -- 172.21.15.112:0/3229459618 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83bc073e20 msgr2=0x7f83bc074230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:39.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.005+0000 7f83c3369640 1 --2- 172.21.15.112:0/3229459618 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83bc073e20 0x7f83bc074230 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f83ac009830 tx=0x7f83ac02a510 comp rx=0 tx=0).stop 2024-09-21T00:27:39.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.005+0000 7f83c3369640 1 -- 172.21.15.112:0/3229459618 shutdown_connections 2024-09-21T00:27:39.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.005+0000 7f83c3369640 1 --2- 172.21.15.112:0/3229459618 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83bc073e20 0x7f83bc074230 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:39.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.005+0000 7f83c3369640 1 -- 172.21.15.112:0/3229459618 >> 172.21.15.112:0/3229459618 conn(0x7f83bc0cca40 msgr2=0x7f83bc0cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:39.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.005+0000 7f83c3369640 1 -- 172.21.15.112:0/3229459618 shutdown_connections 2024-09-21T00:27:39.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.005+0000 7f83c3369640 1 -- 172.21.15.112:0/3229459618 wait complete. 2024-09-21T00:27:39.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.007+0000 7f83c3369640 1 Processor -- start 2024-09-21T00:27:39.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.007+0000 7f83c3369640 1 -- start start 2024-09-21T00:27:39.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.008+0000 7f83c3369640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f83bc073e20 0x7f83bc14dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:39.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.008+0000 7f83c3369640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f83bc0ca610 0x7f83bc09bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:39.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.008+0000 7f83c3369640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83bc15e800 0x7f83bc1f3920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:39.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.008+0000 7f83c3369640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f83bc15f4a0 con 0x7f83bc15e800 2024-09-21T00:27:39.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.008+0000 7f83c3369640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f83bc15f320 con 0x7f83bc073e20 2024-09-21T00:27:39.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.008+0000 7f83c3369640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f83bc15f620 con 0x7f83bc0ca610 2024-09-21T00:27:39.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.008+0000 7f83c1906640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83bc15e800 0x7f83bc1f3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:39.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.008+0000 7f83c1906640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83bc15e800 0x7f83bc1f3920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48236/0 (socket says 172.21.15.112:48236) 2024-09-21T00:27:39.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.008+0000 7f83c1906640 1 -- 172.21.15.112:0/1424916151 learned_addr learned my addr 172.21.15.112:0/1424916151 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:27:39.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.008+0000 7f83c1105640 1 --2- 172.21.15.112:0/1424916151 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f83bc073e20 0x7f83bc14dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:39.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.008+0000 7f83c0904640 1 --2- 172.21.15.112:0/1424916151 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f83bc0ca610 0x7f83bc09bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:39.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.009+0000 7f83c1906640 1 -- 172.21.15.112:0/1424916151 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f83bc0ca610 msgr2=0x7f83bc09bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:39.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.009+0000 7f83c1906640 1 --2- 172.21.15.112:0/1424916151 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f83bc0ca610 0x7f83bc09bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:39.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.009+0000 7f83c1906640 1 -- 172.21.15.112:0/1424916151 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f83bc073e20 msgr2=0x7f83bc14dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:39.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.009+0000 7f83c1906640 1 --2- 172.21.15.112:0/1424916151 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f83bc073e20 0x7f83bc14dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:39.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.009+0000 7f83c1906640 1 -- 172.21.15.112:0/1424916151 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83bc0a07f0 con 0x7f83bc15e800 2024-09-21T00:27:39.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.009+0000 7f83c1906640 1 --2- 172.21.15.112:0/1424916151 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83bc15e800 0x7f83bc1f3920 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f83b400b440 tx=0x7f83b400b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:39.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.009+0000 7f83b1ffb640 1 -- 172.21.15.112:0/1424916151 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83b4011830 con 0x7f83bc15e800 2024-09-21T00:27:39.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.009+0000 7f83b1ffb640 1 -- 172.21.15.112:0/1424916151 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f83b4016070 con 0x7f83bc15e800 2024-09-21T00:27:39.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.009+0000 7f83c3369640 1 -- 172.21.15.112:0/1424916151 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83bc0a0510 con 0x7f83bc15e800 2024-09-21T00:27:39.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.009+0000 7f83b1ffb640 1 -- 172.21.15.112:0/1424916151 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83b4011d60 con 0x7f83bc15e800 2024-09-21T00:27:39.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.009+0000 7f83c3369640 1 -- 172.21.15.112:0/1424916151 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83bc09b7e0 con 0x7f83bc15e800 2024-09-21T00:27:39.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.010+0000 7f83c3369640 1 -- 172.21.15.112:0/1424916151 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83bc06df80 con 0x7f83bc15e800 2024-09-21T00:27:39.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.011+0000 7f83b1ffb640 1 -- 172.21.15.112:0/1424916151 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f83b4004020 con 0x7f83bc15e800 2024-09-21T00:27:39.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.011+0000 7f83b1ffb640 1 --2- 172.21.15.112:0/1424916151 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f83a0078b90 0x7f83a007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:39.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.011+0000 7f83b1ffb640 1 -- 172.21.15.112:0/1424916151 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7f83b409a830 con 0x7f83bc15e800 2024-09-21T00:27:39.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.011+0000 7f83c1105640 1 --2- 172.21.15.112:0/1424916151 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f83a0078b90 0x7f83a007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:39.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.012+0000 7f83c1105640 1 --2- 172.21.15.112:0/1424916151 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f83a0078b90 0x7f83a007b0a0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f83b80017a0 tx=0x7f83b8001730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:39.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.016+0000 7f83b1ffb640 1 -- 172.21.15.112:0/1424916151 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f83b4020020 con 0x7f83bc15e800 2024-09-21T00:27:39.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.193+0000 7f83c3369640 1 -- 172.21.15.112:0/1424916151 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f83bc075050 con 0x7f83bc15e800 2024-09-21T00:27:39.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.194+0000 7f83b1ffb640 1 -- 172.21.15.112:0/1424916151 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v60) ==== 74+0+16968 (secure 0 0 0) 0x7f83b4062b70 con 0x7f83bc15e800 2024-09-21T00:27:39.195 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:27:39.195 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":60,"fsid":"8b34692b-dc65-49b2-a08f-26f707b96ac5","created":"2024-09-21T00:25:22.788490+0000","modified":"2024-09-21T00:27:36.511536+0000","last_up_change":"2024-09-21T00:25:36.161835+0000","last_in_change":"2024-09-21T00:25:26.346500+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T00:25:37.105006+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-09-21T00:27:32.510522+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-09-21T00:27:33.828833+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9c575086-67d2-4de2-b01b-d44dcc21d604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6809","nonce":755760621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6811","nonce":755760621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6815","nonce":755760621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6813","nonce":755760621}]},"public_addr":"172.21.15.112:6809/755760621","cluster_addr":"172.21.15.112:6811/755760621","heartbeat_back_addr":"172.21.15.112:6815/755760621","heartbeat_front_addr":"172.21.15.112:6813/755760621","state":["exists","up"]},{"osd":1,"uuid":"a983c227-a3ce-4bdd-82f1-438ba7e3ecd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6801","nonce":344932592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6803","nonce":344932592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6807","nonce":344932592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6805","nonce":344932592}]},"public_addr":"172.21.15.112:6801/344932592","cluster_addr":"172.21.15.112:6803/344932592","heartbeat_back_addr":"172.21.15.112:6807/344932592","heartbeat_front_addr":"172.21.15.112:6805/344932592","state":["exists","up"]},{"osd":2,"uuid":"1193c1d8-bfff-4a57-8200-5f69bf1ea3af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6825","nonce":407933616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6827","nonce":407933616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6831","nonce":407933616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6829","nonce":407933616}]},"public_addr":"172.21.15.112:6825/407933616","cluster_addr":"172.21.15.112:6827/407933616","heartbeat_back_addr":"172.21.15.112:6831/407933616","heartbeat_front_addr":"172.21.15.112:6829/407933616","state":["exists","up"]},{"osd":3,"uuid":"af04330a-36ad-40cc-a8dd-fa259b221de5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6817","nonce":4120164806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6819","nonce":4120164806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6823","nonce":4120164806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6821","nonce":4120164806}]},"public_addr":"172.21.15.112:6817/4120164806","cluster_addr":"172.21.15.112:6819/4120164806","heartbeat_back_addr":"172.21.15.112:6823/4120164806","heartbeat_front_addr":"172.21.15.112:6821/4120164806","state":["exists","up"]},{"osd":4,"uuid":"5b893111-292c-4c5a-b184-53ad38c7199e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6803","nonce":522077560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6805","nonce":522077560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6809","nonce":522077560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6807","nonce":522077560}]},"public_addr":"172.21.15.187:6803/522077560","cluster_addr":"172.21.15.187:6805/522077560","heartbeat_back_addr":"172.21.15.187:6809/522077560","heartbeat_front_addr":"172.21.15.187:6807/522077560","state":["exists","up"]},{"osd":5,"uuid":"c0b8e011-45fe-4c5e-925e-6be2a7f1868a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6827","nonce":3593144721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6829","nonce":3593144721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6833","nonce":3593144721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6831","nonce":3593144721}]},"public_addr":"172.21.15.187:6827/3593144721","cluster_addr":"172.21.15.187:6829/3593144721","heartbeat_back_addr":"172.21.15.187:6833/3593144721","heartbeat_front_addr":"172.21.15.187:6831/3593144721","state":["exists","up"]},{"osd":6,"uuid":"87d2d650-7489-4593-98a3-a1f433ce8dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6811","nonce":1494387657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6813","nonce":1494387657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6817","nonce":1494387657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6815","nonce":1494387657}]},"public_addr":"172.21.15.187:6811/1494387657","cluster_addr":"172.21.15.187:6813/1494387657","heartbeat_back_addr":"172.21.15.187:6817/1494387657","heartbeat_front_addr":"172.21.15.187:6815/1494387657","state":["exists","up"]},{"osd":7,"uuid":"c6b4f8e2-a1ee-4438-aa37-67ff4b6ef1f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6819","nonce":1624699842}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6821","nonce":1624699842}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6825","nonce":1624699842}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6823","nonce":1624699842}]},"public_addr":"172.21.15.187:6819/1624699842","cluster_addr":"172.21.15.187:6821/1624699842","heartbeat_back_addr":"172.21.15.187:6825/1624699842","heartbeat_front_addr":"172.21.15.187:6823/1624699842","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.873162+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.400506+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.664317+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.159791+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.202630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.113212+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.384786+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.448441+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T00:27:39.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.197+0000 7f83c3369640 1 -- 172.21.15.112:0/1424916151 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f83a0078b90 msgr2=0x7f83a007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:39.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.197+0000 7f83c3369640 1 --2- 172.21.15.112:0/1424916151 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f83a0078b90 0x7f83a007b0a0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f83b80017a0 tx=0x7f83b8001730 comp rx=0 tx=0).stop 2024-09-21T00:27:39.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.197+0000 7f83c3369640 1 -- 172.21.15.112:0/1424916151 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83bc15e800 msgr2=0x7f83bc1f3920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:39.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.197+0000 7f83c3369640 1 --2- 172.21.15.112:0/1424916151 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83bc15e800 0x7f83bc1f3920 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f83b400b440 tx=0x7f83b400b900 comp rx=0 tx=0).stop 2024-09-21T00:27:39.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.197+0000 7f83c3369640 1 -- 172.21.15.112:0/1424916151 shutdown_connections 2024-09-21T00:27:39.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.197+0000 7f83c3369640 1 --2- 172.21.15.112:0/1424916151 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f83a0078b90 0x7f83a007b0a0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:39.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.197+0000 7f83c3369640 1 --2- 172.21.15.112:0/1424916151 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83bc15e800 0x7f83bc1f3920 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:39.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.197+0000 7f83c3369640 1 --2- 172.21.15.112:0/1424916151 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f83bc0ca610 0x7f83bc09bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:39.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.197+0000 7f83c3369640 1 --2- 172.21.15.112:0/1424916151 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f83bc073e20 0x7f83bc14dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:39.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.197+0000 7f83c3369640 1 -- 172.21.15.112:0/1424916151 >> 172.21.15.112:0/1424916151 conn(0x7f83bc0cca40 msgr2=0x7f83bc15d150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:39.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.197+0000 7f83c3369640 1 -- 172.21.15.112:0/1424916151 shutdown_connections 2024-09-21T00:27:39.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.197+0000 7f83c3369640 1 -- 172.21.15.112:0/1424916151 wait complete. 2024-09-21T00:27:39.209 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T00:27:39.209 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T00:27:39.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.320+0000 7f361ce4d640 1 -- 172.21.15.112:0/164122447 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f361815efb0 msgr2=0x7f361806e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:39.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.320+0000 7f361ce4d640 1 --2- 172.21.15.112:0/164122447 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f361815efb0 0x7f361806e540 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f3610009830 tx=0x7f361002a480 comp rx=0 tx=0).stop 2024-09-21T00:27:39.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.320+0000 7f361ce4d640 1 -- 172.21.15.112:0/164122447 shutdown_connections 2024-09-21T00:27:39.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.320+0000 7f361ce4d640 1 --2- 172.21.15.112:0/164122447 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f361815efb0 0x7f361806e540 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:39.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.320+0000 7f361ce4d640 1 --2- 172.21.15.112:0/164122447 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f36180745b0 0x7f361806ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:39.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.320+0000 7f361ce4d640 1 -- 172.21.15.112:0/164122447 >> 172.21.15.112:0/164122447 conn(0x7f36180d1140 msgr2=0x7f36180d1550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:39.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.320+0000 7f361ce4d640 1 -- 172.21.15.112:0/164122447 shutdown_connections 2024-09-21T00:27:39.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.320+0000 7f361ce4d640 1 -- 172.21.15.112:0/164122447 wait complete. 2024-09-21T00:27:39.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.322+0000 7f361ce4d640 1 Processor -- start 2024-09-21T00:27:39.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.322+0000 7f361ce4d640 1 -- start start 2024-09-21T00:27:39.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.323+0000 7f361ce4d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f361815b5f0 0x7f361814fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:39.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.323+0000 7f361ce4d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f361815efb0 0x7f36180c5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:39.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.323+0000 7f361ce4d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f36180b1d60 0x7f36180b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:39.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.324+0000 7f361ce4d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f361806ee10 con 0x7f361815efb0 2024-09-21T00:27:39.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.324+0000 7f361ce4d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f361806ec90 con 0x7f36180b1d60 2024-09-21T00:27:39.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.324+0000 7f361ce4d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f361806ef90 con 0x7f361815b5f0 2024-09-21T00:27:39.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.324+0000 7f3615d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f361815efb0 0x7f36180c5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:39.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.324+0000 7f3615d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f361815efb0 0x7f36180c5fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48252/0 (socket says 172.21.15.112:48252) 2024-09-21T00:27:39.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.324+0000 7f3615d9b640 1 -- 172.21.15.112:0/130676910 learned_addr learned my addr 172.21.15.112:0/130676910 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:27:39.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.324+0000 7f3615d9b640 1 -- 172.21.15.112:0/130676910 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f361815b5f0 msgr2=0x7f361814fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:39.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.324+0000 7f3615d9b640 1 --2- 172.21.15.112:0/130676910 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f361815b5f0 0x7f361814fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:39.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.324+0000 7f3615d9b640 1 -- 172.21.15.112:0/130676910 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f36180b1d60 msgr2=0x7f36180b2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:39.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.324+0000 7f3616d9d640 1 --2- 172.21.15.112:0/130676910 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f36180b1d60 0x7f36180b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:39.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.324+0000 7f3615d9b640 1 --2- 172.21.15.112:0/130676910 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f36180b1d60 0x7f36180b2170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:39.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.324+0000 7f3615d9b640 1 -- 172.21.15.112:0/130676910 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f361814ffc0 con 0x7f361815efb0 2024-09-21T00:27:39.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.324+0000 7f3616d9d640 1 --2- 172.21.15.112:0/130676910 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f36180b1d60 0x7f36180b2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:27:39.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.324+0000 7f3615d9b640 1 --2- 172.21.15.112:0/130676910 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f361815efb0 0x7f36180c5fc0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f360c00d700 tx=0x7f360c00dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:39.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.324+0000 7f3606ffd640 1 -- 172.21.15.112:0/130676910 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f360c017070 con 0x7f361815efb0 2024-09-21T00:27:39.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.324+0000 7f3606ffd640 1 -- 172.21.15.112:0/130676910 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f360c004310 con 0x7f361815efb0 2024-09-21T00:27:39.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.324+0000 7f361ce4d640 1 -- 172.21.15.112:0/130676910 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36180b26b0 con 0x7f361815efb0 2024-09-21T00:27:39.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.324+0000 7f3606ffd640 1 -- 172.21.15.112:0/130676910 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f360c009a80 con 0x7f361815efb0 2024-09-21T00:27:39.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.324+0000 7f361ce4d640 1 -- 172.21.15.112:0/130676910 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36180c50d0 con 0x7f361815efb0 2024-09-21T00:27:39.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.325+0000 7f361ce4d640 1 -- 172.21.15.112:0/130676910 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36180c6910 con 0x7f361815efb0 2024-09-21T00:27:39.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.326+0000 7f3606ffd640 1 -- 172.21.15.112:0/130676910 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f360c009c20 con 0x7f361815efb0 2024-09-21T00:27:39.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.326+0000 7f3606ffd640 1 --2- 172.21.15.112:0/130676910 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f35f8078b90 0x7f35f807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:39.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.327+0000 7f3606ffd640 1 -- 172.21.15.112:0/130676910 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7f360c09afb0 con 0x7f361815efb0 2024-09-21T00:27:39.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.327+0000 7f361659c640 1 --2- 172.21.15.112:0/130676910 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f35f8078b90 0x7f35f807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:39.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.327+0000 7f361659c640 1 --2- 172.21.15.112:0/130676910 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f35f8078b90 0x7f35f807b0a0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f3608004a30 tx=0x7f3608001650 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:39.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.331+0000 7f3606ffd640 1 -- 172.21.15.112:0/130676910 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f361814ffc0 con 0x7f361815efb0 2024-09-21T00:27:39.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.505+0000 7f361ce4d640 1 -- 172.21.15.112:0/130676910 --> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3618074f20 con 0x7f35f8078b90 2024-09-21T00:27:39.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.510+0000 7f3606ffd640 1 -- 172.21.15.112:0/130676910 <== mgr.4103 v2:172.21.15.187:6800/2921522906 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252006 (secure 0 0 0) 0x7f3618074f20 con 0x7f35f8078b90 2024-09-21T00:27:39.514 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:27:39.515 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2024-09-21T00:27:39.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.513+0000 7f361ce4d640 1 -- 172.21.15.112:0/130676910 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f35f8078b90 msgr2=0x7f35f807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:39.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.513+0000 7f361ce4d640 1 --2- 172.21.15.112:0/130676910 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f35f8078b90 0x7f35f807b0a0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f3608004a30 tx=0x7f3608001650 comp rx=0 tx=0).stop 2024-09-21T00:27:39.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.513+0000 7f361ce4d640 1 -- 172.21.15.112:0/130676910 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f361815efb0 msgr2=0x7f36180c5fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:39.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.513+0000 7f361ce4d640 1 --2- 172.21.15.112:0/130676910 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f361815efb0 0x7f36180c5fc0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f360c00d700 tx=0x7f360c00dbc0 comp rx=0 tx=0).stop 2024-09-21T00:27:39.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.513+0000 7f361ce4d640 1 -- 172.21.15.112:0/130676910 shutdown_connections 2024-09-21T00:27:39.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.513+0000 7f361ce4d640 1 --2- 172.21.15.112:0/130676910 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f36180b1d60 0x7f36180b2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:39.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.513+0000 7f361ce4d640 1 --2- 172.21.15.112:0/130676910 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f35f8078b90 0x7f35f807b0a0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:39.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.513+0000 7f361ce4d640 1 --2- 172.21.15.112:0/130676910 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f361815efb0 0x7f36180c5fc0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:39.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.513+0000 7f361ce4d640 1 --2- 172.21.15.112:0/130676910 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f361815b5f0 0x7f361814fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:39.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.513+0000 7f361ce4d640 1 -- 172.21.15.112:0/130676910 >> 172.21.15.112:0/130676910 conn(0x7f36180d1140 msgr2=0x7f3618151820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:39.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.514+0000 7f361ce4d640 1 -- 172.21.15.112:0/130676910 shutdown_connections 2024-09-21T00:27:39.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.514+0000 7f361ce4d640 1 -- 172.21.15.112:0/130676910 wait complete. 2024-09-21T00:27:39.525 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":116,"stamp":"2024-09-21T00:27:39.116995+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":152,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":227136,"kb_used_data":6740,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754747584,"statfs":{"total":773094113280,"available":772861526016,"internally_reserved":0,"allocated":6901760,"data_stored":3673936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":225496625},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.619749"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T00:27:37.409263+0000","last_change":"2024-09-21T00:27:35.533183+0000","last_active":"2024-09-21T00:27:37.409263+0000","last_peered":"2024-09-21T00:27:37.409263+0000","last_clean":"2024-09-21T00:27:37.409263+0000","last_became_active":"2024-09-21T00:27:35.532857+0000","last_became_peered":"2024-09-21T00:27:35.532857+0000","last_unstale":"2024-09-21T00:27:37.409263+0000","last_undegraded":"2024-09-21T00:27:37.409263+0000","last_fullsized":"2024-09-21T00:27:37.409263+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_clean_scrub_stamp":"2024-09-21T00:27:34.492209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:56:34.768513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T00:27:37.409269+0000","last_change":"2024-09-21T00:27:35.517476+0000","last_active":"2024-09-21T00:27:37.409269+0000","last_peered":"2024-09-21T00:27:37.409269+0000","last_clean":"2024-09-21T00:27:37.409269+0000","last_became_active":"2024-09-21T00:27:35.517143+0000","last_became_peered":"2024-09-21T00:27:35.517143+0000","last_unstale":"2024-09-21T00:27:37.409269+0000","last_undegraded":"2024-09-21T00:27:37.409269+0000","last_fullsized":"2024-09-21T00:27:37.409269+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_clean_scrub_stamp":"2024-09-21T00:27:34.492209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:16:00.924147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.537995+0000","last_change":"2024-09-21T00:27:35.535773+0000","last_active":"2024-09-21T00:27:35.537995+0000","last_peered":"2024-09-21T00:27:35.537995+0000","last_clean":"2024-09-21T00:27:35.537995+0000","last_became_active":"2024-09-21T00:27:35.535423+0000","last_became_peered":"2024-09-21T00:27:35.535423+0000","last_unstale":"2024-09-21T00:27:35.537995+0000","last_undegraded":"2024-09-21T00:27:35.537995+0000","last_fullsized":"2024-09-21T00:27:35.537995+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_clean_scrub_stamp":"2024-09-21T00:27:34.492209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:08:04.400599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.515905+0000","last_change":"2024-09-21T00:27:35.514118+0000","last_active":"2024-09-21T00:27:35.515905+0000","last_peered":"2024-09-21T00:27:35.515905+0000","last_clean":"2024-09-21T00:27:35.515905+0000","last_became_active":"2024-09-21T00:27:35.513735+0000","last_became_peered":"2024-09-21T00:27:35.513735+0000","last_unstale":"2024-09-21T00:27:35.515905+0000","last_undegraded":"2024-09-21T00:27:35.515905+0000","last_fullsized":"2024-09-21T00:27:35.515905+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_clean_scrub_stamp":"2024-09-21T00:27:34.492209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:23:58.019025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.528293+0000","last_change":"2024-09-21T00:27:35.520986+0000","last_active":"2024-09-21T00:27:35.528293+0000","last_peered":"2024-09-21T00:27:35.528293+0000","last_clean":"2024-09-21T00:27:35.528293+0000","last_became_active":"2024-09-21T00:27:35.520665+0000","last_became_peered":"2024-09-21T00:27:35.520665+0000","last_unstale":"2024-09-21T00:27:35.528293+0000","last_undegraded":"2024-09-21T00:27:35.528293+0000","last_fullsized":"2024-09-21T00:27:35.528293+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_clean_scrub_stamp":"2024-09-21T00:27:34.492209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:54:50.289245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.521363+0000","last_change":"2024-09-21T00:27:35.514410+0000","last_active":"2024-09-21T00:27:35.521363+0000","last_peered":"2024-09-21T00:27:35.521363+0000","last_clean":"2024-09-21T00:27:35.521363+0000","last_became_active":"2024-09-21T00:27:35.514093+0000","last_became_peered":"2024-09-21T00:27:35.514093+0000","last_unstale":"2024-09-21T00:27:35.521363+0000","last_undegraded":"2024-09-21T00:27:35.521363+0000","last_fullsized":"2024-09-21T00:27:35.521363+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_clean_scrub_stamp":"2024-09-21T00:27:34.492209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:35:10.379534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.535764+0000","last_change":"2024-09-21T00:27:35.530803+0000","last_active":"2024-09-21T00:27:35.535764+0000","last_peered":"2024-09-21T00:27:35.535764+0000","last_clean":"2024-09-21T00:27:35.535764+0000","last_became_active":"2024-09-21T00:27:35.530477+0000","last_became_peered":"2024-09-21T00:27:35.530477+0000","last_unstale":"2024-09-21T00:27:35.535764+0000","last_undegraded":"2024-09-21T00:27:35.535764+0000","last_fullsized":"2024-09-21T00:27:35.535764+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_clean_scrub_stamp":"2024-09-21T00:27:34.492209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:27:34.663733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.521171+0000","last_change":"2024-09-21T00:27:35.519053+0000","last_active":"2024-09-21T00:27:35.521171+0000","last_peered":"2024-09-21T00:27:35.521171+0000","last_clean":"2024-09-21T00:27:35.521171+0000","last_became_active":"2024-09-21T00:27:35.518683+0000","last_became_peered":"2024-09-21T00:27:35.518683+0000","last_unstale":"2024-09-21T00:27:35.521171+0000","last_undegraded":"2024-09-21T00:27:35.521171+0000","last_fullsized":"2024-09-21T00:27:35.521171+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_clean_scrub_stamp":"2024-09-21T00:27:34.492209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:13:07.679402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.531775+0000","last_change":"2024-09-21T00:27:35.529654+0000","last_active":"2024-09-21T00:27:35.531775+0000","last_peered":"2024-09-21T00:27:35.531775+0000","last_clean":"2024-09-21T00:27:35.531775+0000","last_became_active":"2024-09-21T00:27:35.529378+0000","last_became_peered":"2024-09-21T00:27:35.529378+0000","last_unstale":"2024-09-21T00:27:35.531775+0000","last_undegraded":"2024-09-21T00:27:35.531775+0000","last_fullsized":"2024-09-21T00:27:35.531775+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_clean_scrub_stamp":"2024-09-21T00:27:34.492209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:40:00.426293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.538402+0000","last_change":"2024-09-21T00:27:35.536010+0000","last_active":"2024-09-21T00:27:35.538402+0000","last_peered":"2024-09-21T00:27:35.538402+0000","last_clean":"2024-09-21T00:27:35.538402+0000","last_became_active":"2024-09-21T00:27:35.535536+0000","last_became_peered":"2024-09-21T00:27:35.535536+0000","last_unstale":"2024-09-21T00:27:35.538402+0000","last_undegraded":"2024-09-21T00:27:35.538402+0000","last_fullsized":"2024-09-21T00:27:35.538402+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_clean_scrub_stamp":"2024-09-21T00:27:34.492209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:52:41.255394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.521696+0000","last_change":"2024-09-21T00:27:35.518652+0000","last_active":"2024-09-21T00:27:35.521696+0000","last_peered":"2024-09-21T00:27:35.521696+0000","last_clean":"2024-09-21T00:27:35.521696+0000","last_became_active":"2024-09-21T00:27:35.518328+0000","last_became_peered":"2024-09-21T00:27:35.518328+0000","last_unstale":"2024-09-21T00:27:35.521696+0000","last_undegraded":"2024-09-21T00:27:35.521696+0000","last_fullsized":"2024-09-21T00:27:35.521696+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_clean_scrub_stamp":"2024-09-21T00:27:34.492209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:46:33.919951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.523776+0000","last_change":"2024-09-21T00:27:35.519935+0000","last_active":"2024-09-21T00:27:35.523776+0000","last_peered":"2024-09-21T00:27:35.523776+0000","last_clean":"2024-09-21T00:27:35.523776+0000","last_became_active":"2024-09-21T00:27:35.519554+0000","last_became_peered":"2024-09-21T00:27:35.519554+0000","last_unstale":"2024-09-21T00:27:35.523776+0000","last_undegraded":"2024-09-21T00:27:35.523776+0000","last_fullsized":"2024-09-21T00:27:35.523776+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_clean_scrub_stamp":"2024-09-21T00:27:34.492209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:30:44.444600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.531949+0000","last_change":"2024-09-21T00:27:35.525423+0000","last_active":"2024-09-21T00:27:35.531949+0000","last_peered":"2024-09-21T00:27:35.531949+0000","last_clean":"2024-09-21T00:27:35.531949+0000","last_became_active":"2024-09-21T00:27:35.524893+0000","last_became_peered":"2024-09-21T00:27:35.524893+0000","last_unstale":"2024-09-21T00:27:35.531949+0000","last_undegraded":"2024-09-21T00:27:35.531949+0000","last_fullsized":"2024-09-21T00:27:35.531949+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_clean_scrub_stamp":"2024-09-21T00:27:34.492209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T00:52:55.971384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.537832+0000","last_change":"2024-09-21T00:27:35.535033+0000","last_active":"2024-09-21T00:27:35.537832+0000","last_peered":"2024-09-21T00:27:35.537832+0000","last_clean":"2024-09-21T00:27:35.537832+0000","last_became_active":"2024-09-21T00:27:35.534707+0000","last_became_peered":"2024-09-21T00:27:35.534707+0000","last_unstale":"2024-09-21T00:27:35.537832+0000","last_undegraded":"2024-09-21T00:27:35.537832+0000","last_fullsized":"2024-09-21T00:27:35.537832+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_clean_scrub_stamp":"2024-09-21T00:27:34.492209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:40:56.908844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.536232+0000","last_change":"2024-09-21T00:27:35.532937+0000","last_active":"2024-09-21T00:27:35.536232+0000","last_peered":"2024-09-21T00:27:35.536232+0000","last_clean":"2024-09-21T00:27:35.536232+0000","last_became_active":"2024-09-21T00:27:35.532608+0000","last_became_peered":"2024-09-21T00:27:35.532608+0000","last_unstale":"2024-09-21T00:27:35.536232+0000","last_undegraded":"2024-09-21T00:27:35.536232+0000","last_fullsized":"2024-09-21T00:27:35.536232+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_clean_scrub_stamp":"2024-09-21T00:27:34.492209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:51:50.534783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.530851+0000","last_change":"2024-09-21T00:27:35.523073+0000","last_active":"2024-09-21T00:27:35.530851+0000","last_peered":"2024-09-21T00:27:35.530851+0000","last_clean":"2024-09-21T00:27:35.530851+0000","last_became_active":"2024-09-21T00:27:35.522764+0000","last_became_peered":"2024-09-21T00:27:35.522764+0000","last_unstale":"2024-09-21T00:27:35.530851+0000","last_undegraded":"2024-09-21T00:27:35.530851+0000","last_fullsized":"2024-09-21T00:27:35.530851+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_clean_scrub_stamp":"2024-09-21T00:27:34.492209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:50:56.828657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T00:27:37.409336+0000","last_change":"2024-09-21T00:27:35.520120+0000","last_active":"2024-09-21T00:27:37.409336+0000","last_peered":"2024-09-21T00:27:37.409336+0000","last_clean":"2024-09-21T00:27:37.409336+0000","last_became_active":"2024-09-21T00:27:35.519768+0000","last_became_peered":"2024-09-21T00:27:35.519768+0000","last_unstale":"2024-09-21T00:27:37.409336+0000","last_undegraded":"2024-09-21T00:27:37.409336+0000","last_fullsized":"2024-09-21T00:27:37.409336+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_clean_scrub_stamp":"2024-09-21T00:27:34.492209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:31:16.795347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.515946+0000","last_change":"2024-09-21T00:27:35.514337+0000","last_active":"2024-09-21T00:27:35.515946+0000","last_peered":"2024-09-21T00:27:35.515946+0000","last_clean":"2024-09-21T00:27:35.515946+0000","last_became_active":"2024-09-21T00:27:35.513984+0000","last_became_peered":"2024-09-21T00:27:35.513984+0000","last_unstale":"2024-09-21T00:27:35.515946+0000","last_undegraded":"2024-09-21T00:27:35.515946+0000","last_fullsized":"2024-09-21T00:27:35.515946+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_clean_scrub_stamp":"2024-09-21T00:27:34.492209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:41:31.678499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.529944+0000","last_change":"2024-09-21T00:27:35.527039+0000","last_active":"2024-09-21T00:27:35.529944+0000","last_peered":"2024-09-21T00:27:35.529944+0000","last_clean":"2024-09-21T00:27:35.529944+0000","last_became_active":"2024-09-21T00:27:35.526629+0000","last_became_peered":"2024-09-21T00:27:35.526629+0000","last_unstale":"2024-09-21T00:27:35.529944+0000","last_undegraded":"2024-09-21T00:27:35.529944+0000","last_fullsized":"2024-09-21T00:27:35.529944+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_clean_scrub_stamp":"2024-09-21T00:27:34.492209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:37:21.770149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T00:27:36.515157+0000","last_change":"2024-09-21T00:27:35.524293+0000","last_active":"2024-09-21T00:27:36.515157+0000","last_peered":"2024-09-21T00:27:36.515157+0000","last_clean":"2024-09-21T00:27:36.515157+0000","last_became_active":"2024-09-21T00:27:35.523950+0000","last_became_peered":"2024-09-21T00:27:35.523950+0000","last_unstale":"2024-09-21T00:27:36.515157+0000","last_undegraded":"2024-09-21T00:27:36.515157+0000","last_fullsized":"2024-09-21T00:27:36.515157+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_clean_scrub_stamp":"2024-09-21T00:27:34.492209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:12:37.705673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.531915+0000","last_change":"2024-09-21T00:27:35.523872+0000","last_active":"2024-09-21T00:27:35.531915+0000","last_peered":"2024-09-21T00:27:35.531915+0000","last_clean":"2024-09-21T00:27:35.531915+0000","last_became_active":"2024-09-21T00:27:35.523547+0000","last_became_peered":"2024-09-21T00:27:35.523547+0000","last_unstale":"2024-09-21T00:27:35.531915+0000","last_undegraded":"2024-09-21T00:27:35.531915+0000","last_fullsized":"2024-09-21T00:27:35.531915+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_clean_scrub_stamp":"2024-09-21T00:27:34.492209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:48:17.322079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T00:27:36.515069+0000","last_change":"2024-09-21T00:27:35.528987+0000","last_active":"2024-09-21T00:27:36.515069+0000","last_peered":"2024-09-21T00:27:36.515069+0000","last_clean":"2024-09-21T00:27:36.515069+0000","last_became_active":"2024-09-21T00:27:35.528649+0000","last_became_peered":"2024-09-21T00:27:35.528649+0000","last_unstale":"2024-09-21T00:27:36.515069+0000","last_undegraded":"2024-09-21T00:27:36.515069+0000","last_fullsized":"2024-09-21T00:27:36.515069+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_clean_scrub_stamp":"2024-09-21T00:27:34.492209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:19:00.754529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T00:27:36.514888+0000","last_change":"2024-09-21T00:27:35.527021+0000","last_active":"2024-09-21T00:27:36.514888+0000","last_peered":"2024-09-21T00:27:36.514888+0000","last_clean":"2024-09-21T00:27:36.514888+0000","last_became_active":"2024-09-21T00:27:35.526717+0000","last_became_peered":"2024-09-21T00:27:35.526717+0000","last_unstale":"2024-09-21T00:27:36.514888+0000","last_undegraded":"2024-09-21T00:27:36.514888+0000","last_fullsized":"2024-09-21T00:27:36.514888+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_clean_scrub_stamp":"2024-09-21T00:27:34.492209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:31:08.019610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.528674+0000","last_change":"2024-09-21T00:27:35.526864+0000","last_active":"2024-09-21T00:27:35.528674+0000","last_peered":"2024-09-21T00:27:35.528674+0000","last_clean":"2024-09-21T00:27:35.528674+0000","last_became_active":"2024-09-21T00:27:35.526200+0000","last_became_peered":"2024-09-21T00:27:35.526200+0000","last_unstale":"2024-09-21T00:27:35.528674+0000","last_undegraded":"2024-09-21T00:27:35.528674+0000","last_fullsized":"2024-09-21T00:27:35.528674+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_clean_scrub_stamp":"2024-09-21T00:27:34.492209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:08:02.078672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.526882+0000","last_change":"2024-09-21T00:27:35.523337+0000","last_active":"2024-09-21T00:27:35.526882+0000","last_peered":"2024-09-21T00:27:35.526882+0000","last_clean":"2024-09-21T00:27:35.526882+0000","last_became_active":"2024-09-21T00:27:35.522210+0000","last_became_peered":"2024-09-21T00:27:35.522210+0000","last_unstale":"2024-09-21T00:27:35.526882+0000","last_undegraded":"2024-09-21T00:27:35.526882+0000","last_fullsized":"2024-09-21T00:27:35.526882+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_clean_scrub_stamp":"2024-09-21T00:27:34.492209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:32:43.120336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.526677+0000","last_change":"2024-09-21T00:27:35.522817+0000","last_active":"2024-09-21T00:27:35.526677+0000","last_peered":"2024-09-21T00:27:35.526677+0000","last_clean":"2024-09-21T00:27:35.526677+0000","last_became_active":"2024-09-21T00:27:35.522477+0000","last_became_peered":"2024-09-21T00:27:35.522477+0000","last_unstale":"2024-09-21T00:27:35.526677+0000","last_undegraded":"2024-09-21T00:27:35.526677+0000","last_fullsized":"2024-09-21T00:27:35.526677+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_clean_scrub_stamp":"2024-09-21T00:27:34.492209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:08:15.835768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.527928+0000","last_change":"2024-09-21T00:27:35.521160+0000","last_active":"2024-09-21T00:27:35.527928+0000","last_peered":"2024-09-21T00:27:35.527928+0000","last_clean":"2024-09-21T00:27:35.527928+0000","last_became_active":"2024-09-21T00:27:35.520844+0000","last_became_peered":"2024-09-21T00:27:35.520844+0000","last_unstale":"2024-09-21T00:27:35.527928+0000","last_undegraded":"2024-09-21T00:27:35.527928+0000","last_fullsized":"2024-09-21T00:27:35.527928+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_clean_scrub_stamp":"2024-09-21T00:27:34.492209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:04:15.011318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T00:27:37.409002+0000","last_change":"2024-09-21T00:27:35.519886+0000","last_active":"2024-09-21T00:27:37.409002+0000","last_peered":"2024-09-21T00:27:37.409002+0000","last_clean":"2024-09-21T00:27:37.409002+0000","last_became_active":"2024-09-21T00:27:35.519582+0000","last_became_peered":"2024-09-21T00:27:35.519582+0000","last_unstale":"2024-09-21T00:27:37.409002+0000","last_undegraded":"2024-09-21T00:27:37.409002+0000","last_fullsized":"2024-09-21T00:27:37.409002+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_clean_scrub_stamp":"2024-09-21T00:27:34.492209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:37:10.390524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.535208+0000","last_change":"2024-09-21T00:27:35.531245+0000","last_active":"2024-09-21T00:27:35.535208+0000","last_peered":"2024-09-21T00:27:35.535208+0000","last_clean":"2024-09-21T00:27:35.535208+0000","last_became_active":"2024-09-21T00:27:35.530833+0000","last_became_peered":"2024-09-21T00:27:35.530833+0000","last_unstale":"2024-09-21T00:27:35.535208+0000","last_undegraded":"2024-09-21T00:27:35.535208+0000","last_fullsized":"2024-09-21T00:27:35.535208+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_clean_scrub_stamp":"2024-09-21T00:27:34.492209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:54:20.410625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.526621+0000","last_change":"2024-09-21T00:27:35.521518+0000","last_active":"2024-09-21T00:27:35.526621+0000","last_peered":"2024-09-21T00:27:35.526621+0000","last_clean":"2024-09-21T00:27:35.526621+0000","last_became_active":"2024-09-21T00:27:35.520434+0000","last_became_peered":"2024-09-21T00:27:35.520434+0000","last_unstale":"2024-09-21T00:27:35.526621+0000","last_undegraded":"2024-09-21T00:27:35.526621+0000","last_fullsized":"2024-09-21T00:27:35.526621+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_clean_scrub_stamp":"2024-09-21T00:27:34.492209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T00:34:05.265950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.528366+0000","last_change":"2024-09-21T00:27:35.523445+0000","last_active":"2024-09-21T00:27:35.528366+0000","last_peered":"2024-09-21T00:27:35.528366+0000","last_clean":"2024-09-21T00:27:35.528366+0000","last_became_active":"2024-09-21T00:27:35.523102+0000","last_became_peered":"2024-09-21T00:27:35.523102+0000","last_unstale":"2024-09-21T00:27:35.528366+0000","last_undegraded":"2024-09-21T00:27:35.528366+0000","last_fullsized":"2024-09-21T00:27:35.528366+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_clean_scrub_stamp":"2024-09-21T00:27:34.492209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:49:39.758258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.527233+0000","last_change":"2024-09-21T00:27:35.523218+0000","last_active":"2024-09-21T00:27:35.527233+0000","last_peered":"2024-09-21T00:27:35.527233+0000","last_clean":"2024-09-21T00:27:35.527233+0000","last_became_active":"2024-09-21T00:27:35.522901+0000","last_became_peered":"2024-09-21T00:27:35.522901+0000","last_unstale":"2024-09-21T00:27:35.527233+0000","last_undegraded":"2024-09-21T00:27:35.527233+0000","last_fullsized":"2024-09-21T00:27:35.527233+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_clean_scrub_stamp":"2024-09-21T00:27:34.492209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:25:21.445951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.524186+0000","last_change":"2024-09-21T00:27:35.518906+0000","last_active":"2024-09-21T00:27:35.524186+0000","last_peered":"2024-09-21T00:27:35.524186+0000","last_clean":"2024-09-21T00:27:35.524186+0000","last_became_active":"2024-09-21T00:27:35.518612+0000","last_became_peered":"2024-09-21T00:27:35.518612+0000","last_unstale":"2024-09-21T00:27:35.524186+0000","last_undegraded":"2024-09-21T00:27:35.524186+0000","last_fullsized":"2024-09-21T00:27:35.524186+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_clean_scrub_stamp":"2024-09-21T00:27:34.492209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:39:24.102737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"59'4","reported_seq":19,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T00:27:37.409328+0000","last_change":"2024-09-21T00:27:34.520898+0000","last_active":"2024-09-21T00:27:37.409328+0000","last_peered":"2024-09-21T00:27:37.409328+0000","last_clean":"2024-09-21T00:27:37.409328+0000","last_became_active":"2024-09-21T00:27:34.520547+0000","last_became_peered":"2024-09-21T00:27:34.520547+0000","last_unstale":"2024-09-21T00:27:37.409328+0000","last_undegraded":"2024-09-21T00:27:37.409328+0000","last_fullsized":"2024-09-21T00:27:37.409328+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:33.485503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:33.485503+0000","last_clean_scrub_stamp":"2024-09-21T00:27:33.485503+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:15:18.184601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.520164+0000","last_change":"2024-09-21T00:27:35.515491+0000","last_active":"2024-09-21T00:27:35.520164+0000","last_peered":"2024-09-21T00:27:35.520164+0000","last_clean":"2024-09-21T00:27:35.520164+0000","last_became_active":"2024-09-21T00:27:35.515183+0000","last_became_peered":"2024-09-21T00:27:35.515183+0000","last_unstale":"2024-09-21T00:27:35.520164+0000","last_undegraded":"2024-09-21T00:27:35.520164+0000","last_fullsized":"2024-09-21T00:27:35.520164+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_clean_scrub_stamp":"2024-09-21T00:27:34.492209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:00:21.769573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"59'1","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.538187+0000","last_change":"2024-09-21T00:27:34.522863+0000","last_active":"2024-09-21T00:27:35.538187+0000","last_peered":"2024-09-21T00:27:35.538187+0000","last_clean":"2024-09-21T00:27:35.538187+0000","last_became_active":"2024-09-21T00:27:34.522533+0000","last_became_peered":"2024-09-21T00:27:34.522533+0000","last_unstale":"2024-09-21T00:27:35.538187+0000","last_undegraded":"2024-09-21T00:27:35.538187+0000","last_fullsized":"2024-09-21T00:27:35.538187+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:33.485503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:33.485503+0000","last_clean_scrub_stamp":"2024-09-21T00:27:33.485503+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:08:03.393892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.526523+0000","last_change":"2024-09-21T00:27:35.523056+0000","last_active":"2024-09-21T00:27:35.526523+0000","last_peered":"2024-09-21T00:27:35.526523+0000","last_clean":"2024-09-21T00:27:35.526523+0000","last_became_active":"2024-09-21T00:27:35.522711+0000","last_became_peered":"2024-09-21T00:27:35.522711+0000","last_unstale":"2024-09-21T00:27:35.526523+0000","last_undegraded":"2024-09-21T00:27:35.526523+0000","last_fullsized":"2024-09-21T00:27:35.526523+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_clean_scrub_stamp":"2024-09-21T00:27:34.492209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:00:32.489280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"59'2","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.535586+0000","last_change":"2024-09-21T00:27:34.525915+0000","last_active":"2024-09-21T00:27:35.535586+0000","last_peered":"2024-09-21T00:27:35.535586+0000","last_clean":"2024-09-21T00:27:35.535586+0000","last_became_active":"2024-09-21T00:27:34.525637+0000","last_became_peered":"2024-09-21T00:27:34.525637+0000","last_unstale":"2024-09-21T00:27:35.535586+0000","last_undegraded":"2024-09-21T00:27:35.535586+0000","last_fullsized":"2024-09-21T00:27:35.535586+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:33.485503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:33.485503+0000","last_clean_scrub_stamp":"2024-09-21T00:27:33.485503+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:25:11.918561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.537705+0000","last_change":"2024-09-21T00:27:35.535006+0000","last_active":"2024-09-21T00:27:35.537705+0000","last_peered":"2024-09-21T00:27:35.537705+0000","last_clean":"2024-09-21T00:27:35.537705+0000","last_became_active":"2024-09-21T00:27:35.534291+0000","last_became_peered":"2024-09-21T00:27:35.534291+0000","last_unstale":"2024-09-21T00:27:35.537705+0000","last_undegraded":"2024-09-21T00:27:35.537705+0000","last_fullsized":"2024-09-21T00:27:35.537705+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_clean_scrub_stamp":"2024-09-21T00:27:34.492209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:55:03.674651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"59'2","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.537543+0000","last_change":"2024-09-21T00:27:34.520042+0000","last_active":"2024-09-21T00:27:35.537543+0000","last_peered":"2024-09-21T00:27:35.537543+0000","last_clean":"2024-09-21T00:27:35.537543+0000","last_became_active":"2024-09-21T00:27:34.519732+0000","last_became_peered":"2024-09-21T00:27:34.519732+0000","last_unstale":"2024-09-21T00:27:35.537543+0000","last_undegraded":"2024-09-21T00:27:35.537543+0000","last_fullsized":"2024-09-21T00:27:35.537543+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:33.485503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:33.485503+0000","last_clean_scrub_stamp":"2024-09-21T00:27:33.485503+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:55:02.667945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.526280+0000","last_change":"2024-09-21T00:27:35.520574+0000","last_active":"2024-09-21T00:27:35.526280+0000","last_peered":"2024-09-21T00:27:35.526280+0000","last_clean":"2024-09-21T00:27:35.526280+0000","last_became_active":"2024-09-21T00:27:35.520070+0000","last_became_peered":"2024-09-21T00:27:35.520070+0000","last_unstale":"2024-09-21T00:27:35.526280+0000","last_undegraded":"2024-09-21T00:27:35.526280+0000","last_fullsized":"2024-09-21T00:27:35.526280+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_clean_scrub_stamp":"2024-09-21T00:27:34.492209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:53:52.068121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"59'1","reported_seq":16,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T00:27:36.515061+0000","last_change":"2024-09-21T00:27:34.520070+0000","last_active":"2024-09-21T00:27:36.515061+0000","last_peered":"2024-09-21T00:27:36.515061+0000","last_clean":"2024-09-21T00:27:36.515061+0000","last_became_active":"2024-09-21T00:27:34.519759+0000","last_became_peered":"2024-09-21T00:27:34.519759+0000","last_unstale":"2024-09-21T00:27:36.515061+0000","last_undegraded":"2024-09-21T00:27:36.515061+0000","last_fullsized":"2024-09-21T00:27:36.515061+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:33.485503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:33.485503+0000","last_clean_scrub_stamp":"2024-09-21T00:27:33.485503+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:35:09.372828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T00:27:37.408748+0000","last_change":"2024-09-21T00:27:35.519269+0000","last_active":"2024-09-21T00:27:37.408748+0000","last_peered":"2024-09-21T00:27:37.408748+0000","last_clean":"2024-09-21T00:27:37.408748+0000","last_became_active":"2024-09-21T00:27:35.518967+0000","last_became_peered":"2024-09-21T00:27:35.518967+0000","last_unstale":"2024-09-21T00:27:37.408748+0000","last_undegraded":"2024-09-21T00:27:37.408748+0000","last_fullsized":"2024-09-21T00:27:37.408748+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_clean_scrub_stamp":"2024-09-21T00:27:34.492209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:26:53.920532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"59'1","reported_seq":16,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T00:27:36.515164+0000","last_change":"2024-09-21T00:27:34.516561+0000","last_active":"2024-09-21T00:27:36.515164+0000","last_peered":"2024-09-21T00:27:36.515164+0000","last_clean":"2024-09-21T00:27:36.515164+0000","last_became_active":"2024-09-21T00:27:34.516207+0000","last_became_peered":"2024-09-21T00:27:34.516207+0000","last_unstale":"2024-09-21T00:27:36.515164+0000","last_undegraded":"2024-09-21T00:27:36.515164+0000","last_fullsized":"2024-09-21T00:27:36.515164+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:33.485503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:33.485503+0000","last_clean_scrub_stamp":"2024-09-21T00:27:33.485503+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:25:20.439245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.521482+0000","last_change":"2024-09-21T00:27:35.517687+0000","last_active":"2024-09-21T00:27:35.521482+0000","last_peered":"2024-09-21T00:27:35.521482+0000","last_clean":"2024-09-21T00:27:35.521482+0000","last_became_active":"2024-09-21T00:27:35.517360+0000","last_became_peered":"2024-09-21T00:27:35.517360+0000","last_unstale":"2024-09-21T00:27:35.521482+0000","last_undegraded":"2024-09-21T00:27:35.521482+0000","last_fullsized":"2024-09-21T00:27:35.521482+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_clean_scrub_stamp":"2024-09-21T00:27:34.492209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:22:37.851253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T00:27:36.514818+0000","last_change":"2024-09-21T00:27:34.517502+0000","last_active":"2024-09-21T00:27:36.514818+0000","last_peered":"2024-09-21T00:27:36.514818+0000","last_clean":"2024-09-21T00:27:36.514818+0000","last_became_active":"2024-09-21T00:27:34.517161+0000","last_became_peered":"2024-09-21T00:27:34.517161+0000","last_unstale":"2024-09-21T00:27:36.514818+0000","last_undegraded":"2024-09-21T00:27:36.514818+0000","last_fullsized":"2024-09-21T00:27:36.514818+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:33.485503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:33.485503+0000","last_clean_scrub_stamp":"2024-09-21T00:27:33.485503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:00:31.482573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T00:27:36.515230+0000","last_change":"2024-09-21T00:27:35.516946+0000","last_active":"2024-09-21T00:27:36.515230+0000","last_peered":"2024-09-21T00:27:36.515230+0000","last_clean":"2024-09-21T00:27:36.515230+0000","last_became_active":"2024-09-21T00:27:35.516613+0000","last_became_peered":"2024-09-21T00:27:35.516613+0000","last_unstale":"2024-09-21T00:27:36.515230+0000","last_undegraded":"2024-09-21T00:27:36.515230+0000","last_fullsized":"2024-09-21T00:27:36.515230+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_clean_scrub_stamp":"2024-09-21T00:27:34.492209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:05:20.690235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.512761+0000","last_change":"2024-09-21T00:27:34.524357+0000","last_active":"2024-09-21T00:27:35.512761+0000","last_peered":"2024-09-21T00:27:35.512761+0000","last_clean":"2024-09-21T00:27:35.512761+0000","last_became_active":"2024-09-21T00:27:34.523998+0000","last_became_peered":"2024-09-21T00:27:34.523998+0000","last_unstale":"2024-09-21T00:27:35.512761+0000","last_undegraded":"2024-09-21T00:27:35.512761+0000","last_fullsized":"2024-09-21T00:27:35.512761+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:33.485503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:33.485503+0000","last_clean_scrub_stamp":"2024-09-21T00:27:33.485503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:07:36.731179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T00:27:37.409033+0000","last_change":"2024-09-21T00:27:35.518517+0000","last_active":"2024-09-21T00:27:37.409033+0000","last_peered":"2024-09-21T00:27:37.409033+0000","last_clean":"2024-09-21T00:27:37.409033+0000","last_became_active":"2024-09-21T00:27:35.518195+0000","last_became_peered":"2024-09-21T00:27:35.518195+0000","last_unstale":"2024-09-21T00:27:37.409033+0000","last_undegraded":"2024-09-21T00:27:37.409033+0000","last_fullsized":"2024-09-21T00:27:37.409033+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_clean_scrub_stamp":"2024-09-21T00:27:34.492209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:51:42.474636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.523101+0000","last_change":"2024-09-21T00:27:34.530611+0000","last_active":"2024-09-21T00:27:35.523101+0000","last_peered":"2024-09-21T00:27:35.523101+0000","last_clean":"2024-09-21T00:27:35.523101+0000","last_became_active":"2024-09-21T00:27:34.530240+0000","last_became_peered":"2024-09-21T00:27:34.530240+0000","last_unstale":"2024-09-21T00:27:35.523101+0000","last_undegraded":"2024-09-21T00:27:35.523101+0000","last_fullsized":"2024-09-21T00:27:35.523101+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:33.485503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:33.485503+0000","last_clean_scrub_stamp":"2024-09-21T00:27:33.485503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:41:31.568549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T00:27:36.515070+0000","last_change":"2024-09-21T00:27:35.528157+0000","last_active":"2024-09-21T00:27:36.515070+0000","last_peered":"2024-09-21T00:27:36.515070+0000","last_clean":"2024-09-21T00:27:36.515070+0000","last_became_active":"2024-09-21T00:27:35.527781+0000","last_became_peered":"2024-09-21T00:27:35.527781+0000","last_unstale":"2024-09-21T00:27:36.515070+0000","last_undegraded":"2024-09-21T00:27:36.515070+0000","last_fullsized":"2024-09-21T00:27:36.515070+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_clean_scrub_stamp":"2024-09-21T00:27:34.492209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:44:30.436245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.508350+0000","last_change":"2024-09-21T00:27:34.516100+0000","last_active":"2024-09-21T00:27:35.508350+0000","last_peered":"2024-09-21T00:27:35.508350+0000","last_clean":"2024-09-21T00:27:35.508350+0000","last_became_active":"2024-09-21T00:27:34.515766+0000","last_became_peered":"2024-09-21T00:27:34.515766+0000","last_unstale":"2024-09-21T00:27:35.508350+0000","last_undegraded":"2024-09-21T00:27:35.508350+0000","last_fullsized":"2024-09-21T00:27:35.508350+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:33.485503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:33.485503+0000","last_clean_scrub_stamp":"2024-09-21T00:27:33.485503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:34:12.486121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.520855+0000","last_change":"2024-09-21T00:27:35.517823+0000","last_active":"2024-09-21T00:27:35.520855+0000","last_peered":"2024-09-21T00:27:35.520855+0000","last_clean":"2024-09-21T00:27:35.520855+0000","last_became_active":"2024-09-21T00:27:35.517502+0000","last_became_peered":"2024-09-21T00:27:35.517502+0000","last_unstale":"2024-09-21T00:27:35.520855+0000","last_undegraded":"2024-09-21T00:27:35.520855+0000","last_fullsized":"2024-09-21T00:27:35.520855+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_clean_scrub_stamp":"2024-09-21T00:27:34.492209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:15:10.884195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"59'3","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.537845+0000","last_change":"2024-09-21T00:27:34.531404+0000","last_active":"2024-09-21T00:27:35.537845+0000","last_peered":"2024-09-21T00:27:35.537845+0000","last_clean":"2024-09-21T00:27:35.537845+0000","last_became_active":"2024-09-21T00:27:34.531057+0000","last_became_peered":"2024-09-21T00:27:34.531057+0000","last_unstale":"2024-09-21T00:27:35.537845+0000","last_undegraded":"2024-09-21T00:27:35.537845+0000","last_fullsized":"2024-09-21T00:27:35.537845+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:33.485503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:33.485503+0000","last_clean_scrub_stamp":"2024-09-21T00:27:33.485503+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:56:25.415201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.527059+0000","last_change":"2024-09-21T00:27:35.520181+0000","last_active":"2024-09-21T00:27:35.527059+0000","last_peered":"2024-09-21T00:27:35.527059+0000","last_clean":"2024-09-21T00:27:35.527059+0000","last_became_active":"2024-09-21T00:27:35.519872+0000","last_became_peered":"2024-09-21T00:27:35.519872+0000","last_unstale":"2024-09-21T00:27:35.527059+0000","last_undegraded":"2024-09-21T00:27:35.527059+0000","last_fullsized":"2024-09-21T00:27:35.527059+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_clean_scrub_stamp":"2024-09-21T00:27:34.492209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:45:05.229795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"59'6","reported_seq":21,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T00:27:37.409079+0000","last_change":"2024-09-21T00:27:34.522068+0000","last_active":"2024-09-21T00:27:37.409079+0000","last_peered":"2024-09-21T00:27:37.409079+0000","last_clean":"2024-09-21T00:27:37.409079+0000","last_became_active":"2024-09-21T00:27:34.521740+0000","last_became_peered":"2024-09-21T00:27:34.521740+0000","last_unstale":"2024-09-21T00:27:37.409079+0000","last_undegraded":"2024-09-21T00:27:37.409079+0000","last_fullsized":"2024-09-21T00:27:37.409079+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:33.485503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:33.485503+0000","last_clean_scrub_stamp":"2024-09-21T00:27:33.485503+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:23:25.141322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.520727+0000","last_change":"2024-09-21T00:27:35.518937+0000","last_active":"2024-09-21T00:27:35.520727+0000","last_peered":"2024-09-21T00:27:35.520727+0000","last_clean":"2024-09-21T00:27:35.520727+0000","last_became_active":"2024-09-21T00:27:35.518430+0000","last_became_peered":"2024-09-21T00:27:35.518430+0000","last_unstale":"2024-09-21T00:27:35.520727+0000","last_undegraded":"2024-09-21T00:27:35.520727+0000","last_fullsized":"2024-09-21T00:27:35.520727+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_clean_scrub_stamp":"2024-09-21T00:27:34.492209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:43:57.028000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"59'2","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.537916+0000","last_change":"2024-09-21T00:27:34.529294+0000","last_active":"2024-09-21T00:27:35.537916+0000","last_peered":"2024-09-21T00:27:35.537916+0000","last_clean":"2024-09-21T00:27:35.537916+0000","last_became_active":"2024-09-21T00:27:34.528987+0000","last_became_peered":"2024-09-21T00:27:34.528987+0000","last_unstale":"2024-09-21T00:27:35.537916+0000","last_undegraded":"2024-09-21T00:27:35.537916+0000","last_fullsized":"2024-09-21T00:27:35.537916+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:33.485503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:33.485503+0000","last_clean_scrub_stamp":"2024-09-21T00:27:33.485503+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:36:30.230893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.517413+0000","last_change":"2024-09-21T00:27:35.515129+0000","last_active":"2024-09-21T00:27:35.517413+0000","last_peered":"2024-09-21T00:27:35.517413+0000","last_clean":"2024-09-21T00:27:35.517413+0000","last_became_active":"2024-09-21T00:27:35.513951+0000","last_became_peered":"2024-09-21T00:27:35.513951+0000","last_unstale":"2024-09-21T00:27:35.517413+0000","last_undegraded":"2024-09-21T00:27:35.517413+0000","last_fullsized":"2024-09-21T00:27:35.517413+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_clean_scrub_stamp":"2024-09-21T00:27:34.492209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:41:42.243083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.506519+0000","last_change":"2024-09-21T00:27:34.522796+0000","last_active":"2024-09-21T00:27:35.506519+0000","last_peered":"2024-09-21T00:27:35.506519+0000","last_clean":"2024-09-21T00:27:35.506519+0000","last_became_active":"2024-09-21T00:27:34.522486+0000","last_became_peered":"2024-09-21T00:27:34.522486+0000","last_unstale":"2024-09-21T00:27:35.506519+0000","last_undegraded":"2024-09-21T00:27:35.506519+0000","last_fullsized":"2024-09-21T00:27:35.506519+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:33.485503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:33.485503+0000","last_clean_scrub_stamp":"2024-09-21T00:27:33.485503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:42:41.914717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.529896+0000","last_change":"2024-09-21T00:27:35.524809+0000","last_active":"2024-09-21T00:27:35.529896+0000","last_peered":"2024-09-21T00:27:35.529896+0000","last_clean":"2024-09-21T00:27:35.529896+0000","last_became_active":"2024-09-21T00:27:35.524466+0000","last_became_peered":"2024-09-21T00:27:35.524466+0000","last_unstale":"2024-09-21T00:27:35.529896+0000","last_undegraded":"2024-09-21T00:27:35.529896+0000","last_fullsized":"2024-09-21T00:27:35.529896+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_clean_scrub_stamp":"2024-09-21T00:27:34.492209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:18:07.594471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.517922+0000","last_change":"2024-09-21T00:27:34.529049+0000","last_active":"2024-09-21T00:27:35.517922+0000","last_peered":"2024-09-21T00:27:35.517922+0000","last_clean":"2024-09-21T00:27:35.517922+0000","last_became_active":"2024-09-21T00:27:34.528479+0000","last_became_peered":"2024-09-21T00:27:34.528479+0000","last_unstale":"2024-09-21T00:27:35.517922+0000","last_undegraded":"2024-09-21T00:27:35.517922+0000","last_fullsized":"2024-09-21T00:27:35.517922+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:33.485503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:33.485503+0000","last_clean_scrub_stamp":"2024-09-21T00:27:33.485503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:02:32.240791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T00:27:37.409039+0000","last_change":"2024-09-21T00:27:35.521805+0000","last_active":"2024-09-21T00:27:37.409039+0000","last_peered":"2024-09-21T00:27:37.409039+0000","last_clean":"2024-09-21T00:27:37.409039+0000","last_became_active":"2024-09-21T00:27:35.521486+0000","last_became_peered":"2024-09-21T00:27:35.521486+0000","last_unstale":"2024-09-21T00:27:37.409039+0000","last_undegraded":"2024-09-21T00:27:37.409039+0000","last_fullsized":"2024-09-21T00:27:37.409039+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_clean_scrub_stamp":"2024-09-21T00:27:34.492209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:54:33.269043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"59'8","reported_seq":21,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.542733+0000","last_change":"2024-09-21T00:27:34.521390+0000","last_active":"2024-09-21T00:27:35.542733+0000","last_peered":"2024-09-21T00:27:35.542733+0000","last_clean":"2024-09-21T00:27:35.542733+0000","last_became_active":"2024-09-21T00:27:34.521057+0000","last_became_peered":"2024-09-21T00:27:34.521057+0000","last_unstale":"2024-09-21T00:27:35.542733+0000","last_undegraded":"2024-09-21T00:27:35.542733+0000","last_fullsized":"2024-09-21T00:27:35.542733+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:33.485503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:33.485503+0000","last_clean_scrub_stamp":"2024-09-21T00:27:33.485503+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:41:29.148475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.532713+0000","last_change":"2024-09-21T00:27:35.530833+0000","last_active":"2024-09-21T00:27:35.532713+0000","last_peered":"2024-09-21T00:27:35.532713+0000","last_clean":"2024-09-21T00:27:35.532713+0000","last_became_active":"2024-09-21T00:27:35.530490+0000","last_became_peered":"2024-09-21T00:27:35.530490+0000","last_unstale":"2024-09-21T00:27:35.532713+0000","last_undegraded":"2024-09-21T00:27:35.532713+0000","last_fullsized":"2024-09-21T00:27:35.532713+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_clean_scrub_stamp":"2024-09-21T00:27:34.492209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:25:12.925267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"59'1","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.537392+0000","last_change":"2024-09-21T00:27:34.525466+0000","last_active":"2024-09-21T00:27:35.537392+0000","last_peered":"2024-09-21T00:27:35.537392+0000","last_clean":"2024-09-21T00:27:35.537392+0000","last_became_active":"2024-09-21T00:27:34.525237+0000","last_became_peered":"2024-09-21T00:27:34.525237+0000","last_unstale":"2024-09-21T00:27:35.537392+0000","last_undegraded":"2024-09-21T00:27:35.537392+0000","last_fullsized":"2024-09-21T00:27:35.537392+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:33.485503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:33.485503+0000","last_clean_scrub_stamp":"2024-09-21T00:27:33.485503+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:26:09.992862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.528346+0000","last_change":"2024-09-21T00:27:35.522238+0000","last_active":"2024-09-21T00:27:35.528346+0000","last_peered":"2024-09-21T00:27:35.528346+0000","last_clean":"2024-09-21T00:27:35.528346+0000","last_became_active":"2024-09-21T00:27:35.521864+0000","last_became_peered":"2024-09-21T00:27:35.521864+0000","last_unstale":"2024-09-21T00:27:35.528346+0000","last_undegraded":"2024-09-21T00:27:35.528346+0000","last_fullsized":"2024-09-21T00:27:35.528346+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_clean_scrub_stamp":"2024-09-21T00:27:34.492209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:57:44.119937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"59'2","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.536402+0000","last_change":"2024-09-21T00:27:34.529105+0000","last_active":"2024-09-21T00:27:35.536402+0000","last_peered":"2024-09-21T00:27:35.536402+0000","last_clean":"2024-09-21T00:27:35.536402+0000","last_became_active":"2024-09-21T00:27:34.528755+0000","last_became_peered":"2024-09-21T00:27:34.528755+0000","last_unstale":"2024-09-21T00:27:35.536402+0000","last_undegraded":"2024-09-21T00:27:35.536402+0000","last_fullsized":"2024-09-21T00:27:35.536402+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:33.485503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:33.485503+0000","last_clean_scrub_stamp":"2024-09-21T00:27:33.485503+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:30:54.154685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.530219+0000","last_change":"2024-09-21T00:27:35.527735+0000","last_active":"2024-09-21T00:27:35.530219+0000","last_peered":"2024-09-21T00:27:35.530219+0000","last_clean":"2024-09-21T00:27:35.530219+0000","last_became_active":"2024-09-21T00:27:35.527422+0000","last_became_peered":"2024-09-21T00:27:35.527422+0000","last_unstale":"2024-09-21T00:27:35.530219+0000","last_undegraded":"2024-09-21T00:27:35.530219+0000","last_fullsized":"2024-09-21T00:27:35.530219+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_clean_scrub_stamp":"2024-09-21T00:27:34.492209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:47:24.818304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"59'1","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.537936+0000","last_change":"2024-09-21T00:27:34.530636+0000","last_active":"2024-09-21T00:27:35.537936+0000","last_peered":"2024-09-21T00:27:35.537936+0000","last_clean":"2024-09-21T00:27:35.537936+0000","last_became_active":"2024-09-21T00:27:34.530331+0000","last_became_peered":"2024-09-21T00:27:34.530331+0000","last_unstale":"2024-09-21T00:27:35.537936+0000","last_undegraded":"2024-09-21T00:27:35.537936+0000","last_fullsized":"2024-09-21T00:27:35.537936+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:33.485503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:33.485503+0000","last_clean_scrub_stamp":"2024-09-21T00:27:33.485503+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:28:19.136531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.528950+0000","last_change":"2024-09-21T00:27:35.522279+0000","last_active":"2024-09-21T00:27:35.528950+0000","last_peered":"2024-09-21T00:27:35.528950+0000","last_clean":"2024-09-21T00:27:35.528950+0000","last_became_active":"2024-09-21T00:27:35.521995+0000","last_became_peered":"2024-09-21T00:27:35.521995+0000","last_unstale":"2024-09-21T00:27:35.528950+0000","last_undegraded":"2024-09-21T00:27:35.528950+0000","last_fullsized":"2024-09-21T00:27:35.528950+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_clean_scrub_stamp":"2024-09-21T00:27:34.492209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:52:28.779174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T00:27:36.515016+0000","last_change":"2024-09-21T00:27:34.521958+0000","last_active":"2024-09-21T00:27:36.515016+0000","last_peered":"2024-09-21T00:27:36.515016+0000","last_clean":"2024-09-21T00:27:36.515016+0000","last_became_active":"2024-09-21T00:27:34.521684+0000","last_became_peered":"2024-09-21T00:27:34.521684+0000","last_unstale":"2024-09-21T00:27:36.515016+0000","last_undegraded":"2024-09-21T00:27:36.515016+0000","last_fullsized":"2024-09-21T00:27:36.515016+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:33.485503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:33.485503+0000","last_clean_scrub_stamp":"2024-09-21T00:27:33.485503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:55:01.415137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.526575+0000","last_change":"2024-09-21T00:27:35.522994+0000","last_active":"2024-09-21T00:27:35.526575+0000","last_peered":"2024-09-21T00:27:35.526575+0000","last_clean":"2024-09-21T00:27:35.526575+0000","last_became_active":"2024-09-21T00:27:35.522685+0000","last_became_peered":"2024-09-21T00:27:35.522685+0000","last_unstale":"2024-09-21T00:27:35.526575+0000","last_undegraded":"2024-09-21T00:27:35.526575+0000","last_fullsized":"2024-09-21T00:27:35.526575+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_clean_scrub_stamp":"2024-09-21T00:27:34.492209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:10:47.234078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"59'2","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.535763+0000","last_change":"2024-09-21T00:27:34.517888+0000","last_active":"2024-09-21T00:27:35.535763+0000","last_peered":"2024-09-21T00:27:35.535763+0000","last_clean":"2024-09-21T00:27:35.535763+0000","last_became_active":"2024-09-21T00:27:34.517609+0000","last_became_peered":"2024-09-21T00:27:34.517609+0000","last_unstale":"2024-09-21T00:27:35.535763+0000","last_undegraded":"2024-09-21T00:27:35.535763+0000","last_fullsized":"2024-09-21T00:27:35.535763+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:33.485503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:33.485503+0000","last_clean_scrub_stamp":"2024-09-21T00:27:33.485503+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:54:19.403918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.526843+0000","last_change":"2024-09-21T00:27:35.522553+0000","last_active":"2024-09-21T00:27:35.526843+0000","last_peered":"2024-09-21T00:27:35.526843+0000","last_clean":"2024-09-21T00:27:35.526843+0000","last_became_active":"2024-09-21T00:27:35.522202+0000","last_became_peered":"2024-09-21T00:27:35.522202+0000","last_unstale":"2024-09-21T00:27:35.526843+0000","last_undegraded":"2024-09-21T00:27:35.526843+0000","last_fullsized":"2024-09-21T00:27:35.526843+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_clean_scrub_stamp":"2024-09-21T00:27:34.492209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:35:43.755267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.506675+0000","last_change":"2024-09-21T00:27:34.522341+0000","last_active":"2024-09-21T00:27:35.506675+0000","last_peered":"2024-09-21T00:27:35.506675+0000","last_clean":"2024-09-21T00:27:35.506675+0000","last_became_active":"2024-09-21T00:27:34.521969+0000","last_became_peered":"2024-09-21T00:27:34.521969+0000","last_unstale":"2024-09-21T00:27:35.506675+0000","last_undegraded":"2024-09-21T00:27:35.506675+0000","last_fullsized":"2024-09-21T00:27:35.506675+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:33.485503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:33.485503+0000","last_clean_scrub_stamp":"2024-09-21T00:27:33.485503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:07:02.103399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.521457+0000","last_change":"2024-09-21T00:27:35.516840+0000","last_active":"2024-09-21T00:27:35.521457+0000","last_peered":"2024-09-21T00:27:35.521457+0000","last_clean":"2024-09-21T00:27:35.521457+0000","last_became_active":"2024-09-21T00:27:35.516527+0000","last_became_peered":"2024-09-21T00:27:35.516527+0000","last_unstale":"2024-09-21T00:27:35.521457+0000","last_undegraded":"2024-09-21T00:27:35.521457+0000","last_fullsized":"2024-09-21T00:27:35.521457+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_clean_scrub_stamp":"2024-09-21T00:27:34.492209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:21:34.578319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.504059+0000","last_change":"2024-09-21T00:27:34.524960+0000","last_active":"2024-09-21T00:27:35.504059+0000","last_peered":"2024-09-21T00:27:35.504059+0000","last_clean":"2024-09-21T00:27:35.504059+0000","last_became_active":"2024-09-21T00:27:34.524684+0000","last_became_peered":"2024-09-21T00:27:34.524684+0000","last_unstale":"2024-09-21T00:27:35.504059+0000","last_undegraded":"2024-09-21T00:27:35.504059+0000","last_fullsized":"2024-09-21T00:27:35.504059+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:33.485503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:33.485503+0000","last_clean_scrub_stamp":"2024-09-21T00:27:33.485503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:51:49.528077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.522135+0000","last_change":"2024-09-21T00:27:35.514020+0000","last_active":"2024-09-21T00:27:35.522135+0000","last_peered":"2024-09-21T00:27:35.522135+0000","last_clean":"2024-09-21T00:27:35.522135+0000","last_became_active":"2024-09-21T00:27:35.513692+0000","last_became_peered":"2024-09-21T00:27:35.513692+0000","last_unstale":"2024-09-21T00:27:35.522135+0000","last_undegraded":"2024-09-21T00:27:35.522135+0000","last_fullsized":"2024-09-21T00:27:35.522135+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_clean_scrub_stamp":"2024-09-21T00:27:34.492209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:52:41.255394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"59'2","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.538104+0000","last_change":"2024-09-21T00:27:34.529099+0000","last_active":"2024-09-21T00:27:35.538104+0000","last_peered":"2024-09-21T00:27:35.538104+0000","last_clean":"2024-09-21T00:27:35.538104+0000","last_became_active":"2024-09-21T00:27:34.528758+0000","last_became_peered":"2024-09-21T00:27:34.528758+0000","last_unstale":"2024-09-21T00:27:35.538104+0000","last_undegraded":"2024-09-21T00:27:35.538104+0000","last_fullsized":"2024-09-21T00:27:35.538104+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:33.485503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:33.485503+0000","last_clean_scrub_stamp":"2024-09-21T00:27:33.485503+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:06:32.789690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.519739+0000","last_change":"2024-09-21T00:27:35.517401+0000","last_active":"2024-09-21T00:27:35.519739+0000","last_peered":"2024-09-21T00:27:35.519739+0000","last_clean":"2024-09-21T00:27:35.519739+0000","last_became_active":"2024-09-21T00:27:35.517053+0000","last_became_peered":"2024-09-21T00:27:35.517053+0000","last_unstale":"2024-09-21T00:27:35.519739+0000","last_undegraded":"2024-09-21T00:27:35.519739+0000","last_fullsized":"2024-09-21T00:27:35.519739+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_clean_scrub_stamp":"2024-09-21T00:27:34.492209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:21:59.104831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.506873+0000","last_change":"2024-09-21T00:27:34.527355+0000","last_active":"2024-09-21T00:27:35.506873+0000","last_peered":"2024-09-21T00:27:35.506873+0000","last_clean":"2024-09-21T00:27:35.506873+0000","last_became_active":"2024-09-21T00:27:34.527075+0000","last_became_peered":"2024-09-21T00:27:34.527075+0000","last_unstale":"2024-09-21T00:27:35.506873+0000","last_undegraded":"2024-09-21T00:27:35.506873+0000","last_fullsized":"2024-09-21T00:27:35.506873+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:33.485503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:33.485503+0000","last_clean_scrub_stamp":"2024-09-21T00:27:33.485503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:55:17.563693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.517379+0000","last_change":"2024-09-21T00:27:35.514745+0000","last_active":"2024-09-21T00:27:35.517379+0000","last_peered":"2024-09-21T00:27:35.517379+0000","last_clean":"2024-09-21T00:27:35.517379+0000","last_became_active":"2024-09-21T00:27:35.514449+0000","last_became_peered":"2024-09-21T00:27:35.514449+0000","last_unstale":"2024-09-21T00:27:35.517379+0000","last_undegraded":"2024-09-21T00:27:35.517379+0000","last_fullsized":"2024-09-21T00:27:35.517379+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_clean_scrub_stamp":"2024-09-21T00:27:34.492209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:16:59.907598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.504508+0000","last_change":"2024-09-21T00:27:34.525921+0000","last_active":"2024-09-21T00:27:35.504508+0000","last_peered":"2024-09-21T00:27:35.504508+0000","last_clean":"2024-09-21T00:27:35.504508+0000","last_became_active":"2024-09-21T00:27:34.525674+0000","last_became_peered":"2024-09-21T00:27:34.525674+0000","last_unstale":"2024-09-21T00:27:35.504508+0000","last_undegraded":"2024-09-21T00:27:35.504508+0000","last_fullsized":"2024-09-21T00:27:35.504508+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:33.485503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:33.485503+0000","last_clean_scrub_stamp":"2024-09-21T00:27:33.485503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:40:55.902137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.528921+0000","last_change":"2024-09-21T00:27:35.521339+0000","last_active":"2024-09-21T00:27:35.528921+0000","last_peered":"2024-09-21T00:27:35.528921+0000","last_clean":"2024-09-21T00:27:35.528921+0000","last_became_active":"2024-09-21T00:27:35.520986+0000","last_became_peered":"2024-09-21T00:27:35.520986+0000","last_unstale":"2024-09-21T00:27:35.528921+0000","last_undegraded":"2024-09-21T00:27:35.528921+0000","last_fullsized":"2024-09-21T00:27:35.528921+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_clean_scrub_stamp":"2024-09-21T00:27:34.492209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:33:38.410530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T00:27:37.409104+0000","last_change":"2024-09-21T00:27:34.521876+0000","last_active":"2024-09-21T00:27:37.409104+0000","last_peered":"2024-09-21T00:27:37.409104+0000","last_clean":"2024-09-21T00:27:37.409104+0000","last_became_active":"2024-09-21T00:27:34.521322+0000","last_became_peered":"2024-09-21T00:27:34.521322+0000","last_unstale":"2024-09-21T00:27:37.409104+0000","last_undegraded":"2024-09-21T00:27:37.409104+0000","last_fullsized":"2024-09-21T00:27:37.409104+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:33.485503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:33.485503+0000","last_clean_scrub_stamp":"2024-09-21T00:27:33.485503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:32:25.779379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.523472+0000","last_change":"2024-09-21T00:27:35.518016+0000","last_active":"2024-09-21T00:27:35.523472+0000","last_peered":"2024-09-21T00:27:35.523472+0000","last_clean":"2024-09-21T00:27:35.523472+0000","last_became_active":"2024-09-21T00:27:35.517718+0000","last_became_peered":"2024-09-21T00:27:35.517718+0000","last_unstale":"2024-09-21T00:27:35.523472+0000","last_undegraded":"2024-09-21T00:27:35.523472+0000","last_fullsized":"2024-09-21T00:27:35.523472+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_clean_scrub_stamp":"2024-09-21T00:27:34.492209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:17:05.651995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.526622+0000","last_change":"2024-09-21T00:27:34.528175+0000","last_active":"2024-09-21T00:27:35.526622+0000","last_peered":"2024-09-21T00:27:35.526622+0000","last_clean":"2024-09-21T00:27:35.526622+0000","last_became_active":"2024-09-21T00:27:34.527859+0000","last_became_peered":"2024-09-21T00:27:34.527859+0000","last_unstale":"2024-09-21T00:27:35.526622+0000","last_undegraded":"2024-09-21T00:27:35.526622+0000","last_fullsized":"2024-09-21T00:27:35.526622+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:33.485503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:33.485503+0000","last_clean_scrub_stamp":"2024-09-21T00:27:33.485503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:25:39.405975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.520718+0000","last_change":"2024-09-21T00:27:35.519359+0000","last_active":"2024-09-21T00:27:35.520718+0000","last_peered":"2024-09-21T00:27:35.520718+0000","last_clean":"2024-09-21T00:27:35.520718+0000","last_became_active":"2024-09-21T00:27:35.519043+0000","last_became_peered":"2024-09-21T00:27:35.519043+0000","last_unstale":"2024-09-21T00:27:35.520718+0000","last_undegraded":"2024-09-21T00:27:35.520718+0000","last_fullsized":"2024-09-21T00:27:35.520718+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_clean_scrub_stamp":"2024-09-21T00:27:34.492209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:02:54.748305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T00:27:36.515219+0000","last_change":"2024-09-21T00:27:34.519817+0000","last_active":"2024-09-21T00:27:36.515219+0000","last_peered":"2024-09-21T00:27:36.515219+0000","last_clean":"2024-09-21T00:27:36.515219+0000","last_became_active":"2024-09-21T00:27:34.519494+0000","last_became_peered":"2024-09-21T00:27:34.519494+0000","last_unstale":"2024-09-21T00:27:36.515219+0000","last_undegraded":"2024-09-21T00:27:36.515219+0000","last_fullsized":"2024-09-21T00:27:36.515219+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:33.485503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:33.485503+0000","last_clean_scrub_stamp":"2024-09-21T00:27:33.485503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:38:52.061271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T00:27:37.408913+0000","last_change":"2024-09-21T00:27:35.520761+0000","last_active":"2024-09-21T00:27:37.408913+0000","last_peered":"2024-09-21T00:27:37.408913+0000","last_clean":"2024-09-21T00:27:37.408913+0000","last_became_active":"2024-09-21T00:27:35.520408+0000","last_became_peered":"2024-09-21T00:27:35.520408+0000","last_unstale":"2024-09-21T00:27:37.408913+0000","last_undegraded":"2024-09-21T00:27:37.408913+0000","last_fullsized":"2024-09-21T00:27:37.408913+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_clean_scrub_stamp":"2024-09-21T00:27:34.492209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:07:30.325168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T00:27:36.514940+0000","last_change":"2024-09-21T00:27:34.517259+0000","last_active":"2024-09-21T00:27:36.514940+0000","last_peered":"2024-09-21T00:27:36.514940+0000","last_clean":"2024-09-21T00:27:36.514940+0000","last_became_active":"2024-09-21T00:27:34.517040+0000","last_became_peered":"2024-09-21T00:27:34.517040+0000","last_unstale":"2024-09-21T00:27:36.514940+0000","last_undegraded":"2024-09-21T00:27:36.514940+0000","last_fullsized":"2024-09-21T00:27:36.514940+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:33.485503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:33.485503+0000","last_clean_scrub_stamp":"2024-09-21T00:27:33.485503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:10:46.227372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.530737+0000","last_change":"2024-09-21T00:27:35.522666+0000","last_active":"2024-09-21T00:27:35.530737+0000","last_peered":"2024-09-21T00:27:35.530737+0000","last_clean":"2024-09-21T00:27:35.530737+0000","last_became_active":"2024-09-21T00:27:35.522354+0000","last_became_peered":"2024-09-21T00:27:35.522354+0000","last_unstale":"2024-09-21T00:27:35.530737+0000","last_undegraded":"2024-09-21T00:27:35.530737+0000","last_fullsized":"2024-09-21T00:27:35.530737+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_clean_scrub_stamp":"2024-09-21T00:27:34.492209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:33:26.078455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T00:27:37.408637+0000","last_change":"2024-09-21T00:27:34.522486+0000","last_active":"2024-09-21T00:27:37.408637+0000","last_peered":"2024-09-21T00:27:37.408637+0000","last_clean":"2024-09-21T00:27:37.408637+0000","last_became_active":"2024-09-21T00:27:34.522150+0000","last_became_peered":"2024-09-21T00:27:34.522150+0000","last_unstale":"2024-09-21T00:27:37.408637+0000","last_undegraded":"2024-09-21T00:27:37.408637+0000","last_fullsized":"2024-09-21T00:27:37.408637+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:33.485503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:33.485503+0000","last_clean_scrub_stamp":"2024-09-21T00:27:33.485503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:37:20.763443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.524141+0000","last_change":"2024-09-21T00:27:35.519635+0000","last_active":"2024-09-21T00:27:35.524141+0000","last_peered":"2024-09-21T00:27:35.524141+0000","last_clean":"2024-09-21T00:27:35.524141+0000","last_became_active":"2024-09-21T00:27:35.519334+0000","last_became_peered":"2024-09-21T00:27:35.519334+0000","last_unstale":"2024-09-21T00:27:35.524141+0000","last_undegraded":"2024-09-21T00:27:35.524141+0000","last_fullsized":"2024-09-21T00:27:35.524141+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_clean_scrub_stamp":"2024-09-21T00:27:34.492209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:35:21.403953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T00:27:36.514745+0000","last_change":"2024-09-21T00:27:34.520004+0000","last_active":"2024-09-21T00:27:36.514745+0000","last_peered":"2024-09-21T00:27:36.514745+0000","last_clean":"2024-09-21T00:27:36.514745+0000","last_became_active":"2024-09-21T00:27:34.519708+0000","last_became_peered":"2024-09-21T00:27:34.519708+0000","last_unstale":"2024-09-21T00:27:36.514745+0000","last_undegraded":"2024-09-21T00:27:36.514745+0000","last_fullsized":"2024-09-21T00:27:36.514745+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:33.485503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:33.485503+0000","last_clean_scrub_stamp":"2024-09-21T00:27:33.485503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:52:40.248688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":177,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.514107+0000","last_change":"2024-09-21T00:25:38.725792+0000","last_active":"2024-09-21T00:27:35.514107+0000","last_peered":"2024-09-21T00:27:35.514107+0000","last_clean":"2024-09-21T00:27:35.514107+0000","last_became_active":"2024-09-21T00:25:38.724668+0000","last_became_peered":"2024-09-21T00:25:38.724668+0000","last_unstale":"2024-09-21T00:27:35.514107+0000","last_undegraded":"2024-09-21T00:27:35.514107+0000","last_fullsized":"2024-09-21T00:27:35.514107+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:37.163216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:37.163216+0000","last_clean_scrub_stamp":"2024-09-21T00:25:37.163216+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:14:36.588660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509467,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28312,"kb_used_data":1284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343528,"statfs":{"total":96636764160,"available":96607772672,"internally_reserved":0,"allocated":1314816,"data_stored":901355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509468,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27720,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344120,"statfs":{"total":96636764160,"available":96608378880,"internally_reserved":0,"allocated":708608,"data_stored":310245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509467,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27724,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344116,"statfs":{"total":96636764160,"available":96608374784,"internally_reserved":0,"allocated":708608,"data_stored":310233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509466,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27728,"kb_used_data":700,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344112,"statfs":{"total":96636764160,"available":96608370688,"internally_reserved":0,"allocated":716800,"data_stored":311823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509468,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27720,"kb_used_data":688,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344120,"statfs":{"total":96636764160,"available":96608378880,"internally_reserved":0,"allocated":704512,"data_stored":310211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509468,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27716,"kb_used_data":688,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344124,"statfs":{"total":96636764160,"available":96608382976,"internally_reserved":0,"allocated":704512,"data_stored":310211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509468,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28300,"kb_used_data":1268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343540,"statfs":{"total":96636764160,"available":96607784960,"internally_reserved":0,"allocated":1298432,"data_stored":900579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574876,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31916,"kb_used_data":728,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339924,"statfs":{"total":96636764160,"available":96604082176,"internally_reserved":0,"allocated":745472,"data_stored":319279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T00:27:39.529 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T00:27:39.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.678+0000 7f1afeab6640 1 -- 172.21.15.112:0/2485225509 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1af815cb10 msgr2=0x7f1af815cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:39.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.678+0000 7f1afeab6640 1 --2- 172.21.15.112:0/2485225509 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1af815cb10 0x7f1af815cf20 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f1ae8009830 tx=0x7f1ae802a510 comp rx=0 tx=0).stop 2024-09-21T00:27:39.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.678+0000 7f1afeab6640 1 -- 172.21.15.112:0/2485225509 shutdown_connections 2024-09-21T00:27:39.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.678+0000 7f1afeab6640 1 --2- 172.21.15.112:0/2485225509 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1af815cb10 0x7f1af815cf20 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:39.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.678+0000 7f1afeab6640 1 -- 172.21.15.112:0/2485225509 >> 172.21.15.112:0/2485225509 conn(0x7f1af80d2350 msgr2=0x7f1af80d2760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:39.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.678+0000 7f1afeab6640 1 -- 172.21.15.112:0/2485225509 shutdown_connections 2024-09-21T00:27:39.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.678+0000 7f1afeab6640 1 -- 172.21.15.112:0/2485225509 wait complete. 2024-09-21T00:27:39.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.679+0000 7f1afeab6640 1 Processor -- start 2024-09-21T00:27:39.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.679+0000 7f1afeab6640 1 -- start start 2024-09-21T00:27:39.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.681+0000 7f1afeab6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1af80b6250 0x7f1af815b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:39.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.681+0000 7f1afeab6640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1af80d12e0 0x7f1af81f38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:39.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.681+0000 7f1afeab6640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1af815cb10 0x7f1af81f3e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:39.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.681+0000 7f1afeab6640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1af80b7100 con 0x7f1af80b6250 2024-09-21T00:27:39.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.681+0000 7f1afeab6640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f1af80b6f80 con 0x7f1af815cb10 2024-09-21T00:27:39.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.681+0000 7f1afeab6640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f1af80b7280 con 0x7f1af80d12e0 2024-09-21T00:27:39.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.681+0000 7f1afc852640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1af80b6250 0x7f1af815b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:39.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.681+0000 7f1afc852640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1af80b6250 0x7f1af815b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48276/0 (socket says 172.21.15.112:48276) 2024-09-21T00:27:39.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.681+0000 7f1afc852640 1 -- 172.21.15.112:0/356532107 learned_addr learned my addr 172.21.15.112:0/356532107 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:27:39.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.681+0000 7f1afc852640 1 -- 172.21.15.112:0/356532107 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1af80d12e0 msgr2=0x7f1af81f38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:39.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.681+0000 7f1afc852640 1 --2- 172.21.15.112:0/356532107 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1af80d12e0 0x7f1af81f38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:39.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.681+0000 7f1afc852640 1 -- 172.21.15.112:0/356532107 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1af815cb10 msgr2=0x7f1af81f3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:39.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.681+0000 7f1afc852640 1 --2- 172.21.15.112:0/356532107 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1af815cb10 0x7f1af81f3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:39.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.681+0000 7f1afc852640 1 -- 172.21.15.112:0/356532107 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1af80c7210 con 0x7f1af80b6250 2024-09-21T00:27:39.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.682+0000 7f1afc852640 1 --2- 172.21.15.112:0/356532107 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1af80b6250 0x7f1af815b8b0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f1ae802ab00 tx=0x7f1ae8033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:39.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.682+0000 7f1af57fa640 1 -- 172.21.15.112:0/356532107 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ae8031d60 con 0x7f1af80b6250 2024-09-21T00:27:39.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.682+0000 7f1af57fa640 1 -- 172.21.15.112:0/356532107 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1ae8033e10 con 0x7f1af80b6250 2024-09-21T00:27:39.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.682+0000 7f1afeab6640 1 -- 172.21.15.112:0/356532107 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1af80c74b0 con 0x7f1af80b6250 2024-09-21T00:27:39.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.682+0000 7f1af57fa640 1 -- 172.21.15.112:0/356532107 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ae8005280 con 0x7f1af80b6250 2024-09-21T00:27:39.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.682+0000 7f1afeab6640 1 -- 172.21.15.112:0/356532107 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1af80c7a70 con 0x7f1af80b6250 2024-09-21T00:27:39.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.683+0000 7f1afeab6640 1 -- 172.21.15.112:0/356532107 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1af815cf20 con 0x7f1af80b6250 2024-09-21T00:27:39.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.683+0000 7f1af57fa640 1 -- 172.21.15.112:0/356532107 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1ae8005420 con 0x7f1af80b6250 2024-09-21T00:27:39.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.684+0000 7f1af57fa640 1 --2- 172.21.15.112:0/356532107 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1ae4078b90 0x7f1ae407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:39.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.684+0000 7f1af57fa640 1 -- 172.21.15.112:0/356532107 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7f1ae80bc140 con 0x7f1af80b6250 2024-09-21T00:27:39.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.684+0000 7f1af7fff640 1 --2- 172.21.15.112:0/356532107 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1ae4078b90 0x7f1ae407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:39.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.685+0000 7f1af7fff640 1 --2- 172.21.15.112:0/356532107 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1ae4078b90 0x7f1ae407b0a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f1aec006fd0 tx=0x7f1aec006bc0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:39.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.688+0000 7f1af57fa640 1 -- 172.21.15.112:0/356532107 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1ae803f030 con 0x7f1af80b6250 2024-09-21T00:27:39.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.862+0000 7f1afeab6640 1 -- 172.21.15.112:0/356532107 --> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1af80b56b0 con 0x7f1ae4078b90 2024-09-21T00:27:39.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.867+0000 7f1af57fa640 1 -- 172.21.15.112:0/356532107 <== mgr.4103 v2:172.21.15.187:6800/2921522906 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252006 (secure 0 0 0) 0x7f1af80b56b0 con 0x7f1ae4078b90 2024-09-21T00:27:39.870 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:27:39.871 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2024-09-21T00:27:39.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.870+0000 7f1afeab6640 1 -- 172.21.15.112:0/356532107 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1ae4078b90 msgr2=0x7f1ae407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:39.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.870+0000 7f1afeab6640 1 --2- 172.21.15.112:0/356532107 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1ae4078b90 0x7f1ae407b0a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f1aec006fd0 tx=0x7f1aec006bc0 comp rx=0 tx=0).stop 2024-09-21T00:27:39.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.870+0000 7f1afeab6640 1 -- 172.21.15.112:0/356532107 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1af80b6250 msgr2=0x7f1af815b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:39.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.870+0000 7f1afeab6640 1 --2- 172.21.15.112:0/356532107 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1af80b6250 0x7f1af815b8b0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f1ae802ab00 tx=0x7f1ae8033c20 comp rx=0 tx=0).stop 2024-09-21T00:27:39.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.870+0000 7f1afeab6640 1 -- 172.21.15.112:0/356532107 shutdown_connections 2024-09-21T00:27:39.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.870+0000 7f1afeab6640 1 --2- 172.21.15.112:0/356532107 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1af815cb10 0x7f1af81f3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:39.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.870+0000 7f1afeab6640 1 --2- 172.21.15.112:0/356532107 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1af80d12e0 0x7f1af81f38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:39.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.870+0000 7f1afeab6640 1 --2- 172.21.15.112:0/356532107 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1ae4078b90 0x7f1ae407b0a0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:39.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.870+0000 7f1afeab6640 1 --2- 172.21.15.112:0/356532107 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1af80b6250 0x7f1af815b8b0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:39.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.870+0000 7f1afeab6640 1 -- 172.21.15.112:0/356532107 >> 172.21.15.112:0/356532107 conn(0x7f1af80d2350 msgr2=0x7f1af8160090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:39.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.870+0000 7f1afeab6640 1 -- 172.21.15.112:0/356532107 shutdown_connections 2024-09-21T00:27:39.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:39.870+0000 7f1afeab6640 1 -- 172.21.15.112:0/356532107 wait complete. 2024-09-21T00:27:39.882 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":116,"stamp":"2024-09-21T00:27:39.116995+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":152,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":227136,"kb_used_data":6740,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754747584,"statfs":{"total":773094113280,"available":772861526016,"internally_reserved":0,"allocated":6901760,"data_stored":3673936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":225496625},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.619749"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T00:27:37.409263+0000","last_change":"2024-09-21T00:27:35.533183+0000","last_active":"2024-09-21T00:27:37.409263+0000","last_peered":"2024-09-21T00:27:37.409263+0000","last_clean":"2024-09-21T00:27:37.409263+0000","last_became_active":"2024-09-21T00:27:35.532857+0000","last_became_peered":"2024-09-21T00:27:35.532857+0000","last_unstale":"2024-09-21T00:27:37.409263+0000","last_undegraded":"2024-09-21T00:27:37.409263+0000","last_fullsized":"2024-09-21T00:27:37.409263+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_clean_scrub_stamp":"2024-09-21T00:27:34.492209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:56:34.768513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T00:27:37.409269+0000","last_change":"2024-09-21T00:27:35.517476+0000","last_active":"2024-09-21T00:27:37.409269+0000","last_peered":"2024-09-21T00:27:37.409269+0000","last_clean":"2024-09-21T00:27:37.409269+0000","last_became_active":"2024-09-21T00:27:35.517143+0000","last_became_peered":"2024-09-21T00:27:35.517143+0000","last_unstale":"2024-09-21T00:27:37.409269+0000","last_undegraded":"2024-09-21T00:27:37.409269+0000","last_fullsized":"2024-09-21T00:27:37.409269+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_clean_scrub_stamp":"2024-09-21T00:27:34.492209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:16:00.924147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.537995+0000","last_change":"2024-09-21T00:27:35.535773+0000","last_active":"2024-09-21T00:27:35.537995+0000","last_peered":"2024-09-21T00:27:35.537995+0000","last_clean":"2024-09-21T00:27:35.537995+0000","last_became_active":"2024-09-21T00:27:35.535423+0000","last_became_peered":"2024-09-21T00:27:35.535423+0000","last_unstale":"2024-09-21T00:27:35.537995+0000","last_undegraded":"2024-09-21T00:27:35.537995+0000","last_fullsized":"2024-09-21T00:27:35.537995+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_clean_scrub_stamp":"2024-09-21T00:27:34.492209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:08:04.400599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.515905+0000","last_change":"2024-09-21T00:27:35.514118+0000","last_active":"2024-09-21T00:27:35.515905+0000","last_peered":"2024-09-21T00:27:35.515905+0000","last_clean":"2024-09-21T00:27:35.515905+0000","last_became_active":"2024-09-21T00:27:35.513735+0000","last_became_peered":"2024-09-21T00:27:35.513735+0000","last_unstale":"2024-09-21T00:27:35.515905+0000","last_undegraded":"2024-09-21T00:27:35.515905+0000","last_fullsized":"2024-09-21T00:27:35.515905+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_clean_scrub_stamp":"2024-09-21T00:27:34.492209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:23:58.019025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.528293+0000","last_change":"2024-09-21T00:27:35.520986+0000","last_active":"2024-09-21T00:27:35.528293+0000","last_peered":"2024-09-21T00:27:35.528293+0000","last_clean":"2024-09-21T00:27:35.528293+0000","last_became_active":"2024-09-21T00:27:35.520665+0000","last_became_peered":"2024-09-21T00:27:35.520665+0000","last_unstale":"2024-09-21T00:27:35.528293+0000","last_undegraded":"2024-09-21T00:27:35.528293+0000","last_fullsized":"2024-09-21T00:27:35.528293+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_clean_scrub_stamp":"2024-09-21T00:27:34.492209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:54:50.289245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.521363+0000","last_change":"2024-09-21T00:27:35.514410+0000","last_active":"2024-09-21T00:27:35.521363+0000","last_peered":"2024-09-21T00:27:35.521363+0000","last_clean":"2024-09-21T00:27:35.521363+0000","last_became_active":"2024-09-21T00:27:35.514093+0000","last_became_peered":"2024-09-21T00:27:35.514093+0000","last_unstale":"2024-09-21T00:27:35.521363+0000","last_undegraded":"2024-09-21T00:27:35.521363+0000","last_fullsized":"2024-09-21T00:27:35.521363+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_clean_scrub_stamp":"2024-09-21T00:27:34.492209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:35:10.379534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.535764+0000","last_change":"2024-09-21T00:27:35.530803+0000","last_active":"2024-09-21T00:27:35.535764+0000","last_peered":"2024-09-21T00:27:35.535764+0000","last_clean":"2024-09-21T00:27:35.535764+0000","last_became_active":"2024-09-21T00:27:35.530477+0000","last_became_peered":"2024-09-21T00:27:35.530477+0000","last_unstale":"2024-09-21T00:27:35.535764+0000","last_undegraded":"2024-09-21T00:27:35.535764+0000","last_fullsized":"2024-09-21T00:27:35.535764+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_clean_scrub_stamp":"2024-09-21T00:27:34.492209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:27:34.663733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.521171+0000","last_change":"2024-09-21T00:27:35.519053+0000","last_active":"2024-09-21T00:27:35.521171+0000","last_peered":"2024-09-21T00:27:35.521171+0000","last_clean":"2024-09-21T00:27:35.521171+0000","last_became_active":"2024-09-21T00:27:35.518683+0000","last_became_peered":"2024-09-21T00:27:35.518683+0000","last_unstale":"2024-09-21T00:27:35.521171+0000","last_undegraded":"2024-09-21T00:27:35.521171+0000","last_fullsized":"2024-09-21T00:27:35.521171+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_clean_scrub_stamp":"2024-09-21T00:27:34.492209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:13:07.679402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.531775+0000","last_change":"2024-09-21T00:27:35.529654+0000","last_active":"2024-09-21T00:27:35.531775+0000","last_peered":"2024-09-21T00:27:35.531775+0000","last_clean":"2024-09-21T00:27:35.531775+0000","last_became_active":"2024-09-21T00:27:35.529378+0000","last_became_peered":"2024-09-21T00:27:35.529378+0000","last_unstale":"2024-09-21T00:27:35.531775+0000","last_undegraded":"2024-09-21T00:27:35.531775+0000","last_fullsized":"2024-09-21T00:27:35.531775+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_clean_scrub_stamp":"2024-09-21T00:27:34.492209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:40:00.426293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.538402+0000","last_change":"2024-09-21T00:27:35.536010+0000","last_active":"2024-09-21T00:27:35.538402+0000","last_peered":"2024-09-21T00:27:35.538402+0000","last_clean":"2024-09-21T00:27:35.538402+0000","last_became_active":"2024-09-21T00:27:35.535536+0000","last_became_peered":"2024-09-21T00:27:35.535536+0000","last_unstale":"2024-09-21T00:27:35.538402+0000","last_undegraded":"2024-09-21T00:27:35.538402+0000","last_fullsized":"2024-09-21T00:27:35.538402+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_clean_scrub_stamp":"2024-09-21T00:27:34.492209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:52:41.255394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.521696+0000","last_change":"2024-09-21T00:27:35.518652+0000","last_active":"2024-09-21T00:27:35.521696+0000","last_peered":"2024-09-21T00:27:35.521696+0000","last_clean":"2024-09-21T00:27:35.521696+0000","last_became_active":"2024-09-21T00:27:35.518328+0000","last_became_peered":"2024-09-21T00:27:35.518328+0000","last_unstale":"2024-09-21T00:27:35.521696+0000","last_undegraded":"2024-09-21T00:27:35.521696+0000","last_fullsized":"2024-09-21T00:27:35.521696+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_clean_scrub_stamp":"2024-09-21T00:27:34.492209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:46:33.919951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.523776+0000","last_change":"2024-09-21T00:27:35.519935+0000","last_active":"2024-09-21T00:27:35.523776+0000","last_peered":"2024-09-21T00:27:35.523776+0000","last_clean":"2024-09-21T00:27:35.523776+0000","last_became_active":"2024-09-21T00:27:35.519554+0000","last_became_peered":"2024-09-21T00:27:35.519554+0000","last_unstale":"2024-09-21T00:27:35.523776+0000","last_undegraded":"2024-09-21T00:27:35.523776+0000","last_fullsized":"2024-09-21T00:27:35.523776+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_clean_scrub_stamp":"2024-09-21T00:27:34.492209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:30:44.444600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.531949+0000","last_change":"2024-09-21T00:27:35.525423+0000","last_active":"2024-09-21T00:27:35.531949+0000","last_peered":"2024-09-21T00:27:35.531949+0000","last_clean":"2024-09-21T00:27:35.531949+0000","last_became_active":"2024-09-21T00:27:35.524893+0000","last_became_peered":"2024-09-21T00:27:35.524893+0000","last_unstale":"2024-09-21T00:27:35.531949+0000","last_undegraded":"2024-09-21T00:27:35.531949+0000","last_fullsized":"2024-09-21T00:27:35.531949+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_clean_scrub_stamp":"2024-09-21T00:27:34.492209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T00:52:55.971384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.537832+0000","last_change":"2024-09-21T00:27:35.535033+0000","last_active":"2024-09-21T00:27:35.537832+0000","last_peered":"2024-09-21T00:27:35.537832+0000","last_clean":"2024-09-21T00:27:35.537832+0000","last_became_active":"2024-09-21T00:27:35.534707+0000","last_became_peered":"2024-09-21T00:27:35.534707+0000","last_unstale":"2024-09-21T00:27:35.537832+0000","last_undegraded":"2024-09-21T00:27:35.537832+0000","last_fullsized":"2024-09-21T00:27:35.537832+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_clean_scrub_stamp":"2024-09-21T00:27:34.492209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:40:56.908844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.536232+0000","last_change":"2024-09-21T00:27:35.532937+0000","last_active":"2024-09-21T00:27:35.536232+0000","last_peered":"2024-09-21T00:27:35.536232+0000","last_clean":"2024-09-21T00:27:35.536232+0000","last_became_active":"2024-09-21T00:27:35.532608+0000","last_became_peered":"2024-09-21T00:27:35.532608+0000","last_unstale":"2024-09-21T00:27:35.536232+0000","last_undegraded":"2024-09-21T00:27:35.536232+0000","last_fullsized":"2024-09-21T00:27:35.536232+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_clean_scrub_stamp":"2024-09-21T00:27:34.492209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:51:50.534783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.530851+0000","last_change":"2024-09-21T00:27:35.523073+0000","last_active":"2024-09-21T00:27:35.530851+0000","last_peered":"2024-09-21T00:27:35.530851+0000","last_clean":"2024-09-21T00:27:35.530851+0000","last_became_active":"2024-09-21T00:27:35.522764+0000","last_became_peered":"2024-09-21T00:27:35.522764+0000","last_unstale":"2024-09-21T00:27:35.530851+0000","last_undegraded":"2024-09-21T00:27:35.530851+0000","last_fullsized":"2024-09-21T00:27:35.530851+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_clean_scrub_stamp":"2024-09-21T00:27:34.492209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:50:56.828657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T00:27:37.409336+0000","last_change":"2024-09-21T00:27:35.520120+0000","last_active":"2024-09-21T00:27:37.409336+0000","last_peered":"2024-09-21T00:27:37.409336+0000","last_clean":"2024-09-21T00:27:37.409336+0000","last_became_active":"2024-09-21T00:27:35.519768+0000","last_became_peered":"2024-09-21T00:27:35.519768+0000","last_unstale":"2024-09-21T00:27:37.409336+0000","last_undegraded":"2024-09-21T00:27:37.409336+0000","last_fullsized":"2024-09-21T00:27:37.409336+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_clean_scrub_stamp":"2024-09-21T00:27:34.492209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:31:16.795347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.515946+0000","last_change":"2024-09-21T00:27:35.514337+0000","last_active":"2024-09-21T00:27:35.515946+0000","last_peered":"2024-09-21T00:27:35.515946+0000","last_clean":"2024-09-21T00:27:35.515946+0000","last_became_active":"2024-09-21T00:27:35.513984+0000","last_became_peered":"2024-09-21T00:27:35.513984+0000","last_unstale":"2024-09-21T00:27:35.515946+0000","last_undegraded":"2024-09-21T00:27:35.515946+0000","last_fullsized":"2024-09-21T00:27:35.515946+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_clean_scrub_stamp":"2024-09-21T00:27:34.492209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:41:31.678499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.529944+0000","last_change":"2024-09-21T00:27:35.527039+0000","last_active":"2024-09-21T00:27:35.529944+0000","last_peered":"2024-09-21T00:27:35.529944+0000","last_clean":"2024-09-21T00:27:35.529944+0000","last_became_active":"2024-09-21T00:27:35.526629+0000","last_became_peered":"2024-09-21T00:27:35.526629+0000","last_unstale":"2024-09-21T00:27:35.529944+0000","last_undegraded":"2024-09-21T00:27:35.529944+0000","last_fullsized":"2024-09-21T00:27:35.529944+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_clean_scrub_stamp":"2024-09-21T00:27:34.492209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:37:21.770149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T00:27:36.515157+0000","last_change":"2024-09-21T00:27:35.524293+0000","last_active":"2024-09-21T00:27:36.515157+0000","last_peered":"2024-09-21T00:27:36.515157+0000","last_clean":"2024-09-21T00:27:36.515157+0000","last_became_active":"2024-09-21T00:27:35.523950+0000","last_became_peered":"2024-09-21T00:27:35.523950+0000","last_unstale":"2024-09-21T00:27:36.515157+0000","last_undegraded":"2024-09-21T00:27:36.515157+0000","last_fullsized":"2024-09-21T00:27:36.515157+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_clean_scrub_stamp":"2024-09-21T00:27:34.492209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:12:37.705673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.531915+0000","last_change":"2024-09-21T00:27:35.523872+0000","last_active":"2024-09-21T00:27:35.531915+0000","last_peered":"2024-09-21T00:27:35.531915+0000","last_clean":"2024-09-21T00:27:35.531915+0000","last_became_active":"2024-09-21T00:27:35.523547+0000","last_became_peered":"2024-09-21T00:27:35.523547+0000","last_unstale":"2024-09-21T00:27:35.531915+0000","last_undegraded":"2024-09-21T00:27:35.531915+0000","last_fullsized":"2024-09-21T00:27:35.531915+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_clean_scrub_stamp":"2024-09-21T00:27:34.492209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:48:17.322079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T00:27:36.515069+0000","last_change":"2024-09-21T00:27:35.528987+0000","last_active":"2024-09-21T00:27:36.515069+0000","last_peered":"2024-09-21T00:27:36.515069+0000","last_clean":"2024-09-21T00:27:36.515069+0000","last_became_active":"2024-09-21T00:27:35.528649+0000","last_became_peered":"2024-09-21T00:27:35.528649+0000","last_unstale":"2024-09-21T00:27:36.515069+0000","last_undegraded":"2024-09-21T00:27:36.515069+0000","last_fullsized":"2024-09-21T00:27:36.515069+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_clean_scrub_stamp":"2024-09-21T00:27:34.492209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:19:00.754529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T00:27:36.514888+0000","last_change":"2024-09-21T00:27:35.527021+0000","last_active":"2024-09-21T00:27:36.514888+0000","last_peered":"2024-09-21T00:27:36.514888+0000","last_clean":"2024-09-21T00:27:36.514888+0000","last_became_active":"2024-09-21T00:27:35.526717+0000","last_became_peered":"2024-09-21T00:27:35.526717+0000","last_unstale":"2024-09-21T00:27:36.514888+0000","last_undegraded":"2024-09-21T00:27:36.514888+0000","last_fullsized":"2024-09-21T00:27:36.514888+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_clean_scrub_stamp":"2024-09-21T00:27:34.492209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:31:08.019610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.528674+0000","last_change":"2024-09-21T00:27:35.526864+0000","last_active":"2024-09-21T00:27:35.528674+0000","last_peered":"2024-09-21T00:27:35.528674+0000","last_clean":"2024-09-21T00:27:35.528674+0000","last_became_active":"2024-09-21T00:27:35.526200+0000","last_became_peered":"2024-09-21T00:27:35.526200+0000","last_unstale":"2024-09-21T00:27:35.528674+0000","last_undegraded":"2024-09-21T00:27:35.528674+0000","last_fullsized":"2024-09-21T00:27:35.528674+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_clean_scrub_stamp":"2024-09-21T00:27:34.492209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:08:02.078672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.526882+0000","last_change":"2024-09-21T00:27:35.523337+0000","last_active":"2024-09-21T00:27:35.526882+0000","last_peered":"2024-09-21T00:27:35.526882+0000","last_clean":"2024-09-21T00:27:35.526882+0000","last_became_active":"2024-09-21T00:27:35.522210+0000","last_became_peered":"2024-09-21T00:27:35.522210+0000","last_unstale":"2024-09-21T00:27:35.526882+0000","last_undegraded":"2024-09-21T00:27:35.526882+0000","last_fullsized":"2024-09-21T00:27:35.526882+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_clean_scrub_stamp":"2024-09-21T00:27:34.492209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:32:43.120336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.526677+0000","last_change":"2024-09-21T00:27:35.522817+0000","last_active":"2024-09-21T00:27:35.526677+0000","last_peered":"2024-09-21T00:27:35.526677+0000","last_clean":"2024-09-21T00:27:35.526677+0000","last_became_active":"2024-09-21T00:27:35.522477+0000","last_became_peered":"2024-09-21T00:27:35.522477+0000","last_unstale":"2024-09-21T00:27:35.526677+0000","last_undegraded":"2024-09-21T00:27:35.526677+0000","last_fullsized":"2024-09-21T00:27:35.526677+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_clean_scrub_stamp":"2024-09-21T00:27:34.492209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:08:15.835768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.527928+0000","last_change":"2024-09-21T00:27:35.521160+0000","last_active":"2024-09-21T00:27:35.527928+0000","last_peered":"2024-09-21T00:27:35.527928+0000","last_clean":"2024-09-21T00:27:35.527928+0000","last_became_active":"2024-09-21T00:27:35.520844+0000","last_became_peered":"2024-09-21T00:27:35.520844+0000","last_unstale":"2024-09-21T00:27:35.527928+0000","last_undegraded":"2024-09-21T00:27:35.527928+0000","last_fullsized":"2024-09-21T00:27:35.527928+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_clean_scrub_stamp":"2024-09-21T00:27:34.492209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:04:15.011318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T00:27:37.409002+0000","last_change":"2024-09-21T00:27:35.519886+0000","last_active":"2024-09-21T00:27:37.409002+0000","last_peered":"2024-09-21T00:27:37.409002+0000","last_clean":"2024-09-21T00:27:37.409002+0000","last_became_active":"2024-09-21T00:27:35.519582+0000","last_became_peered":"2024-09-21T00:27:35.519582+0000","last_unstale":"2024-09-21T00:27:37.409002+0000","last_undegraded":"2024-09-21T00:27:37.409002+0000","last_fullsized":"2024-09-21T00:27:37.409002+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_clean_scrub_stamp":"2024-09-21T00:27:34.492209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:37:10.390524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.535208+0000","last_change":"2024-09-21T00:27:35.531245+0000","last_active":"2024-09-21T00:27:35.535208+0000","last_peered":"2024-09-21T00:27:35.535208+0000","last_clean":"2024-09-21T00:27:35.535208+0000","last_became_active":"2024-09-21T00:27:35.530833+0000","last_became_peered":"2024-09-21T00:27:35.530833+0000","last_unstale":"2024-09-21T00:27:35.535208+0000","last_undegraded":"2024-09-21T00:27:35.535208+0000","last_fullsized":"2024-09-21T00:27:35.535208+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_clean_scrub_stamp":"2024-09-21T00:27:34.492209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:54:20.410625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.526621+0000","last_change":"2024-09-21T00:27:35.521518+0000","last_active":"2024-09-21T00:27:35.526621+0000","last_peered":"2024-09-21T00:27:35.526621+0000","last_clean":"2024-09-21T00:27:35.526621+0000","last_became_active":"2024-09-21T00:27:35.520434+0000","last_became_peered":"2024-09-21T00:27:35.520434+0000","last_unstale":"2024-09-21T00:27:35.526621+0000","last_undegraded":"2024-09-21T00:27:35.526621+0000","last_fullsized":"2024-09-21T00:27:35.526621+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_clean_scrub_stamp":"2024-09-21T00:27:34.492209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T00:34:05.265950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.528366+0000","last_change":"2024-09-21T00:27:35.523445+0000","last_active":"2024-09-21T00:27:35.528366+0000","last_peered":"2024-09-21T00:27:35.528366+0000","last_clean":"2024-09-21T00:27:35.528366+0000","last_became_active":"2024-09-21T00:27:35.523102+0000","last_became_peered":"2024-09-21T00:27:35.523102+0000","last_unstale":"2024-09-21T00:27:35.528366+0000","last_undegraded":"2024-09-21T00:27:35.528366+0000","last_fullsized":"2024-09-21T00:27:35.528366+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_clean_scrub_stamp":"2024-09-21T00:27:34.492209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:49:39.758258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.527233+0000","last_change":"2024-09-21T00:27:35.523218+0000","last_active":"2024-09-21T00:27:35.527233+0000","last_peered":"2024-09-21T00:27:35.527233+0000","last_clean":"2024-09-21T00:27:35.527233+0000","last_became_active":"2024-09-21T00:27:35.522901+0000","last_became_peered":"2024-09-21T00:27:35.522901+0000","last_unstale":"2024-09-21T00:27:35.527233+0000","last_undegraded":"2024-09-21T00:27:35.527233+0000","last_fullsized":"2024-09-21T00:27:35.527233+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_clean_scrub_stamp":"2024-09-21T00:27:34.492209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:25:21.445951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.524186+0000","last_change":"2024-09-21T00:27:35.518906+0000","last_active":"2024-09-21T00:27:35.524186+0000","last_peered":"2024-09-21T00:27:35.524186+0000","last_clean":"2024-09-21T00:27:35.524186+0000","last_became_active":"2024-09-21T00:27:35.518612+0000","last_became_peered":"2024-09-21T00:27:35.518612+0000","last_unstale":"2024-09-21T00:27:35.524186+0000","last_undegraded":"2024-09-21T00:27:35.524186+0000","last_fullsized":"2024-09-21T00:27:35.524186+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_clean_scrub_stamp":"2024-09-21T00:27:34.492209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:39:24.102737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"59'4","reported_seq":19,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T00:27:37.409328+0000","last_change":"2024-09-21T00:27:34.520898+0000","last_active":"2024-09-21T00:27:37.409328+0000","last_peered":"2024-09-21T00:27:37.409328+0000","last_clean":"2024-09-21T00:27:37.409328+0000","last_became_active":"2024-09-21T00:27:34.520547+0000","last_became_peered":"2024-09-21T00:27:34.520547+0000","last_unstale":"2024-09-21T00:27:37.409328+0000","last_undegraded":"2024-09-21T00:27:37.409328+0000","last_fullsized":"2024-09-21T00:27:37.409328+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:33.485503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:33.485503+0000","last_clean_scrub_stamp":"2024-09-21T00:27:33.485503+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:15:18.184601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.520164+0000","last_change":"2024-09-21T00:27:35.515491+0000","last_active":"2024-09-21T00:27:35.520164+0000","last_peered":"2024-09-21T00:27:35.520164+0000","last_clean":"2024-09-21T00:27:35.520164+0000","last_became_active":"2024-09-21T00:27:35.515183+0000","last_became_peered":"2024-09-21T00:27:35.515183+0000","last_unstale":"2024-09-21T00:27:35.520164+0000","last_undegraded":"2024-09-21T00:27:35.520164+0000","last_fullsized":"2024-09-21T00:27:35.520164+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_clean_scrub_stamp":"2024-09-21T00:27:34.492209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:00:21.769573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"59'1","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.538187+0000","last_change":"2024-09-21T00:27:34.522863+0000","last_active":"2024-09-21T00:27:35.538187+0000","last_peered":"2024-09-21T00:27:35.538187+0000","last_clean":"2024-09-21T00:27:35.538187+0000","last_became_active":"2024-09-21T00:27:34.522533+0000","last_became_peered":"2024-09-21T00:27:34.522533+0000","last_unstale":"2024-09-21T00:27:35.538187+0000","last_undegraded":"2024-09-21T00:27:35.538187+0000","last_fullsized":"2024-09-21T00:27:35.538187+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:33.485503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:33.485503+0000","last_clean_scrub_stamp":"2024-09-21T00:27:33.485503+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:08:03.393892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.526523+0000","last_change":"2024-09-21T00:27:35.523056+0000","last_active":"2024-09-21T00:27:35.526523+0000","last_peered":"2024-09-21T00:27:35.526523+0000","last_clean":"2024-09-21T00:27:35.526523+0000","last_became_active":"2024-09-21T00:27:35.522711+0000","last_became_peered":"2024-09-21T00:27:35.522711+0000","last_unstale":"2024-09-21T00:27:35.526523+0000","last_undegraded":"2024-09-21T00:27:35.526523+0000","last_fullsized":"2024-09-21T00:27:35.526523+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_clean_scrub_stamp":"2024-09-21T00:27:34.492209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:00:32.489280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"59'2","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.535586+0000","last_change":"2024-09-21T00:27:34.525915+0000","last_active":"2024-09-21T00:27:35.535586+0000","last_peered":"2024-09-21T00:27:35.535586+0000","last_clean":"2024-09-21T00:27:35.535586+0000","last_became_active":"2024-09-21T00:27:34.525637+0000","last_became_peered":"2024-09-21T00:27:34.525637+0000","last_unstale":"2024-09-21T00:27:35.535586+0000","last_undegraded":"2024-09-21T00:27:35.535586+0000","last_fullsized":"2024-09-21T00:27:35.535586+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:33.485503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:33.485503+0000","last_clean_scrub_stamp":"2024-09-21T00:27:33.485503+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:25:11.918561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.537705+0000","last_change":"2024-09-21T00:27:35.535006+0000","last_active":"2024-09-21T00:27:35.537705+0000","last_peered":"2024-09-21T00:27:35.537705+0000","last_clean":"2024-09-21T00:27:35.537705+0000","last_became_active":"2024-09-21T00:27:35.534291+0000","last_became_peered":"2024-09-21T00:27:35.534291+0000","last_unstale":"2024-09-21T00:27:35.537705+0000","last_undegraded":"2024-09-21T00:27:35.537705+0000","last_fullsized":"2024-09-21T00:27:35.537705+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_clean_scrub_stamp":"2024-09-21T00:27:34.492209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:55:03.674651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"59'2","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.537543+0000","last_change":"2024-09-21T00:27:34.520042+0000","last_active":"2024-09-21T00:27:35.537543+0000","last_peered":"2024-09-21T00:27:35.537543+0000","last_clean":"2024-09-21T00:27:35.537543+0000","last_became_active":"2024-09-21T00:27:34.519732+0000","last_became_peered":"2024-09-21T00:27:34.519732+0000","last_unstale":"2024-09-21T00:27:35.537543+0000","last_undegraded":"2024-09-21T00:27:35.537543+0000","last_fullsized":"2024-09-21T00:27:35.537543+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:33.485503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:33.485503+0000","last_clean_scrub_stamp":"2024-09-21T00:27:33.485503+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:55:02.667945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.526280+0000","last_change":"2024-09-21T00:27:35.520574+0000","last_active":"2024-09-21T00:27:35.526280+0000","last_peered":"2024-09-21T00:27:35.526280+0000","last_clean":"2024-09-21T00:27:35.526280+0000","last_became_active":"2024-09-21T00:27:35.520070+0000","last_became_peered":"2024-09-21T00:27:35.520070+0000","last_unstale":"2024-09-21T00:27:35.526280+0000","last_undegraded":"2024-09-21T00:27:35.526280+0000","last_fullsized":"2024-09-21T00:27:35.526280+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_clean_scrub_stamp":"2024-09-21T00:27:34.492209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:53:52.068121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"59'1","reported_seq":16,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T00:27:36.515061+0000","last_change":"2024-09-21T00:27:34.520070+0000","last_active":"2024-09-21T00:27:36.515061+0000","last_peered":"2024-09-21T00:27:36.515061+0000","last_clean":"2024-09-21T00:27:36.515061+0000","last_became_active":"2024-09-21T00:27:34.519759+0000","last_became_peered":"2024-09-21T00:27:34.519759+0000","last_unstale":"2024-09-21T00:27:36.515061+0000","last_undegraded":"2024-09-21T00:27:36.515061+0000","last_fullsized":"2024-09-21T00:27:36.515061+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:33.485503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:33.485503+0000","last_clean_scrub_stamp":"2024-09-21T00:27:33.485503+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:35:09.372828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T00:27:37.408748+0000","last_change":"2024-09-21T00:27:35.519269+0000","last_active":"2024-09-21T00:27:37.408748+0000","last_peered":"2024-09-21T00:27:37.408748+0000","last_clean":"2024-09-21T00:27:37.408748+0000","last_became_active":"2024-09-21T00:27:35.518967+0000","last_became_peered":"2024-09-21T00:27:35.518967+0000","last_unstale":"2024-09-21T00:27:37.408748+0000","last_undegraded":"2024-09-21T00:27:37.408748+0000","last_fullsized":"2024-09-21T00:27:37.408748+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_clean_scrub_stamp":"2024-09-21T00:27:34.492209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:26:53.920532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"59'1","reported_seq":16,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T00:27:36.515164+0000","last_change":"2024-09-21T00:27:34.516561+0000","last_active":"2024-09-21T00:27:36.515164+0000","last_peered":"2024-09-21T00:27:36.515164+0000","last_clean":"2024-09-21T00:27:36.515164+0000","last_became_active":"2024-09-21T00:27:34.516207+0000","last_became_peered":"2024-09-21T00:27:34.516207+0000","last_unstale":"2024-09-21T00:27:36.515164+0000","last_undegraded":"2024-09-21T00:27:36.515164+0000","last_fullsized":"2024-09-21T00:27:36.515164+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:33.485503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:33.485503+0000","last_clean_scrub_stamp":"2024-09-21T00:27:33.485503+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:25:20.439245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.521482+0000","last_change":"2024-09-21T00:27:35.517687+0000","last_active":"2024-09-21T00:27:35.521482+0000","last_peered":"2024-09-21T00:27:35.521482+0000","last_clean":"2024-09-21T00:27:35.521482+0000","last_became_active":"2024-09-21T00:27:35.517360+0000","last_became_peered":"2024-09-21T00:27:35.517360+0000","last_unstale":"2024-09-21T00:27:35.521482+0000","last_undegraded":"2024-09-21T00:27:35.521482+0000","last_fullsized":"2024-09-21T00:27:35.521482+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_clean_scrub_stamp":"2024-09-21T00:27:34.492209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:22:37.851253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T00:27:36.514818+0000","last_change":"2024-09-21T00:27:34.517502+0000","last_active":"2024-09-21T00:27:36.514818+0000","last_peered":"2024-09-21T00:27:36.514818+0000","last_clean":"2024-09-21T00:27:36.514818+0000","last_became_active":"2024-09-21T00:27:34.517161+0000","last_became_peered":"2024-09-21T00:27:34.517161+0000","last_unstale":"2024-09-21T00:27:36.514818+0000","last_undegraded":"2024-09-21T00:27:36.514818+0000","last_fullsized":"2024-09-21T00:27:36.514818+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:33.485503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:33.485503+0000","last_clean_scrub_stamp":"2024-09-21T00:27:33.485503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:00:31.482573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T00:27:36.515230+0000","last_change":"2024-09-21T00:27:35.516946+0000","last_active":"2024-09-21T00:27:36.515230+0000","last_peered":"2024-09-21T00:27:36.515230+0000","last_clean":"2024-09-21T00:27:36.515230+0000","last_became_active":"2024-09-21T00:27:35.516613+0000","last_became_peered":"2024-09-21T00:27:35.516613+0000","last_unstale":"2024-09-21T00:27:36.515230+0000","last_undegraded":"2024-09-21T00:27:36.515230+0000","last_fullsized":"2024-09-21T00:27:36.515230+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_clean_scrub_stamp":"2024-09-21T00:27:34.492209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:05:20.690235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.512761+0000","last_change":"2024-09-21T00:27:34.524357+0000","last_active":"2024-09-21T00:27:35.512761+0000","last_peered":"2024-09-21T00:27:35.512761+0000","last_clean":"2024-09-21T00:27:35.512761+0000","last_became_active":"2024-09-21T00:27:34.523998+0000","last_became_peered":"2024-09-21T00:27:34.523998+0000","last_unstale":"2024-09-21T00:27:35.512761+0000","last_undegraded":"2024-09-21T00:27:35.512761+0000","last_fullsized":"2024-09-21T00:27:35.512761+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:33.485503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:33.485503+0000","last_clean_scrub_stamp":"2024-09-21T00:27:33.485503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:07:36.731179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T00:27:37.409033+0000","last_change":"2024-09-21T00:27:35.518517+0000","last_active":"2024-09-21T00:27:37.409033+0000","last_peered":"2024-09-21T00:27:37.409033+0000","last_clean":"2024-09-21T00:27:37.409033+0000","last_became_active":"2024-09-21T00:27:35.518195+0000","last_became_peered":"2024-09-21T00:27:35.518195+0000","last_unstale":"2024-09-21T00:27:37.409033+0000","last_undegraded":"2024-09-21T00:27:37.409033+0000","last_fullsized":"2024-09-21T00:27:37.409033+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_clean_scrub_stamp":"2024-09-21T00:27:34.492209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:51:42.474636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.523101+0000","last_change":"2024-09-21T00:27:34.530611+0000","last_active":"2024-09-21T00:27:35.523101+0000","last_peered":"2024-09-21T00:27:35.523101+0000","last_clean":"2024-09-21T00:27:35.523101+0000","last_became_active":"2024-09-21T00:27:34.530240+0000","last_became_peered":"2024-09-21T00:27:34.530240+0000","last_unstale":"2024-09-21T00:27:35.523101+0000","last_undegraded":"2024-09-21T00:27:35.523101+0000","last_fullsized":"2024-09-21T00:27:35.523101+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:33.485503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:33.485503+0000","last_clean_scrub_stamp":"2024-09-21T00:27:33.485503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:41:31.568549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T00:27:36.515070+0000","last_change":"2024-09-21T00:27:35.528157+0000","last_active":"2024-09-21T00:27:36.515070+0000","last_peered":"2024-09-21T00:27:36.515070+0000","last_clean":"2024-09-21T00:27:36.515070+0000","last_became_active":"2024-09-21T00:27:35.527781+0000","last_became_peered":"2024-09-21T00:27:35.527781+0000","last_unstale":"2024-09-21T00:27:36.515070+0000","last_undegraded":"2024-09-21T00:27:36.515070+0000","last_fullsized":"2024-09-21T00:27:36.515070+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_clean_scrub_stamp":"2024-09-21T00:27:34.492209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:44:30.436245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.508350+0000","last_change":"2024-09-21T00:27:34.516100+0000","last_active":"2024-09-21T00:27:35.508350+0000","last_peered":"2024-09-21T00:27:35.508350+0000","last_clean":"2024-09-21T00:27:35.508350+0000","last_became_active":"2024-09-21T00:27:34.515766+0000","last_became_peered":"2024-09-21T00:27:34.515766+0000","last_unstale":"2024-09-21T00:27:35.508350+0000","last_undegraded":"2024-09-21T00:27:35.508350+0000","last_fullsized":"2024-09-21T00:27:35.508350+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:33.485503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:33.485503+0000","last_clean_scrub_stamp":"2024-09-21T00:27:33.485503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:34:12.486121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.520855+0000","last_change":"2024-09-21T00:27:35.517823+0000","last_active":"2024-09-21T00:27:35.520855+0000","last_peered":"2024-09-21T00:27:35.520855+0000","last_clean":"2024-09-21T00:27:35.520855+0000","last_became_active":"2024-09-21T00:27:35.517502+0000","last_became_peered":"2024-09-21T00:27:35.517502+0000","last_unstale":"2024-09-21T00:27:35.520855+0000","last_undegraded":"2024-09-21T00:27:35.520855+0000","last_fullsized":"2024-09-21T00:27:35.520855+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_clean_scrub_stamp":"2024-09-21T00:27:34.492209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:15:10.884195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"59'3","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.537845+0000","last_change":"2024-09-21T00:27:34.531404+0000","last_active":"2024-09-21T00:27:35.537845+0000","last_peered":"2024-09-21T00:27:35.537845+0000","last_clean":"2024-09-21T00:27:35.537845+0000","last_became_active":"2024-09-21T00:27:34.531057+0000","last_became_peered":"2024-09-21T00:27:34.531057+0000","last_unstale":"2024-09-21T00:27:35.537845+0000","last_undegraded":"2024-09-21T00:27:35.537845+0000","last_fullsized":"2024-09-21T00:27:35.537845+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:33.485503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:33.485503+0000","last_clean_scrub_stamp":"2024-09-21T00:27:33.485503+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:56:25.415201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.527059+0000","last_change":"2024-09-21T00:27:35.520181+0000","last_active":"2024-09-21T00:27:35.527059+0000","last_peered":"2024-09-21T00:27:35.527059+0000","last_clean":"2024-09-21T00:27:35.527059+0000","last_became_active":"2024-09-21T00:27:35.519872+0000","last_became_peered":"2024-09-21T00:27:35.519872+0000","last_unstale":"2024-09-21T00:27:35.527059+0000","last_undegraded":"2024-09-21T00:27:35.527059+0000","last_fullsized":"2024-09-21T00:27:35.527059+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_clean_scrub_stamp":"2024-09-21T00:27:34.492209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:45:05.229795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"59'6","reported_seq":21,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T00:27:37.409079+0000","last_change":"2024-09-21T00:27:34.522068+0000","last_active":"2024-09-21T00:27:37.409079+0000","last_peered":"2024-09-21T00:27:37.409079+0000","last_clean":"2024-09-21T00:27:37.409079+0000","last_became_active":"2024-09-21T00:27:34.521740+0000","last_became_peered":"2024-09-21T00:27:34.521740+0000","last_unstale":"2024-09-21T00:27:37.409079+0000","last_undegraded":"2024-09-21T00:27:37.409079+0000","last_fullsized":"2024-09-21T00:27:37.409079+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:33.485503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:33.485503+0000","last_clean_scrub_stamp":"2024-09-21T00:27:33.485503+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:23:25.141322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.520727+0000","last_change":"2024-09-21T00:27:35.518937+0000","last_active":"2024-09-21T00:27:35.520727+0000","last_peered":"2024-09-21T00:27:35.520727+0000","last_clean":"2024-09-21T00:27:35.520727+0000","last_became_active":"2024-09-21T00:27:35.518430+0000","last_became_peered":"2024-09-21T00:27:35.518430+0000","last_unstale":"2024-09-21T00:27:35.520727+0000","last_undegraded":"2024-09-21T00:27:35.520727+0000","last_fullsized":"2024-09-21T00:27:35.520727+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_clean_scrub_stamp":"2024-09-21T00:27:34.492209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:43:57.028000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"59'2","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.537916+0000","last_change":"2024-09-21T00:27:34.529294+0000","last_active":"2024-09-21T00:27:35.537916+0000","last_peered":"2024-09-21T00:27:35.537916+0000","last_clean":"2024-09-21T00:27:35.537916+0000","last_became_active":"2024-09-21T00:27:34.528987+0000","last_became_peered":"2024-09-21T00:27:34.528987+0000","last_unstale":"2024-09-21T00:27:35.537916+0000","last_undegraded":"2024-09-21T00:27:35.537916+0000","last_fullsized":"2024-09-21T00:27:35.537916+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:33.485503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:33.485503+0000","last_clean_scrub_stamp":"2024-09-21T00:27:33.485503+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:36:30.230893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.517413+0000","last_change":"2024-09-21T00:27:35.515129+0000","last_active":"2024-09-21T00:27:35.517413+0000","last_peered":"2024-09-21T00:27:35.517413+0000","last_clean":"2024-09-21T00:27:35.517413+0000","last_became_active":"2024-09-21T00:27:35.513951+0000","last_became_peered":"2024-09-21T00:27:35.513951+0000","last_unstale":"2024-09-21T00:27:35.517413+0000","last_undegraded":"2024-09-21T00:27:35.517413+0000","last_fullsized":"2024-09-21T00:27:35.517413+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_clean_scrub_stamp":"2024-09-21T00:27:34.492209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:41:42.243083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.506519+0000","last_change":"2024-09-21T00:27:34.522796+0000","last_active":"2024-09-21T00:27:35.506519+0000","last_peered":"2024-09-21T00:27:35.506519+0000","last_clean":"2024-09-21T00:27:35.506519+0000","last_became_active":"2024-09-21T00:27:34.522486+0000","last_became_peered":"2024-09-21T00:27:34.522486+0000","last_unstale":"2024-09-21T00:27:35.506519+0000","last_undegraded":"2024-09-21T00:27:35.506519+0000","last_fullsized":"2024-09-21T00:27:35.506519+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:33.485503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:33.485503+0000","last_clean_scrub_stamp":"2024-09-21T00:27:33.485503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:42:41.914717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.529896+0000","last_change":"2024-09-21T00:27:35.524809+0000","last_active":"2024-09-21T00:27:35.529896+0000","last_peered":"2024-09-21T00:27:35.529896+0000","last_clean":"2024-09-21T00:27:35.529896+0000","last_became_active":"2024-09-21T00:27:35.524466+0000","last_became_peered":"2024-09-21T00:27:35.524466+0000","last_unstale":"2024-09-21T00:27:35.529896+0000","last_undegraded":"2024-09-21T00:27:35.529896+0000","last_fullsized":"2024-09-21T00:27:35.529896+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_clean_scrub_stamp":"2024-09-21T00:27:34.492209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:18:07.594471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.517922+0000","last_change":"2024-09-21T00:27:34.529049+0000","last_active":"2024-09-21T00:27:35.517922+0000","last_peered":"2024-09-21T00:27:35.517922+0000","last_clean":"2024-09-21T00:27:35.517922+0000","last_became_active":"2024-09-21T00:27:34.528479+0000","last_became_peered":"2024-09-21T00:27:34.528479+0000","last_unstale":"2024-09-21T00:27:35.517922+0000","last_undegraded":"2024-09-21T00:27:35.517922+0000","last_fullsized":"2024-09-21T00:27:35.517922+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:33.485503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:33.485503+0000","last_clean_scrub_stamp":"2024-09-21T00:27:33.485503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:02:32.240791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T00:27:37.409039+0000","last_change":"2024-09-21T00:27:35.521805+0000","last_active":"2024-09-21T00:27:37.409039+0000","last_peered":"2024-09-21T00:27:37.409039+0000","last_clean":"2024-09-21T00:27:37.409039+0000","last_became_active":"2024-09-21T00:27:35.521486+0000","last_became_peered":"2024-09-21T00:27:35.521486+0000","last_unstale":"2024-09-21T00:27:37.409039+0000","last_undegraded":"2024-09-21T00:27:37.409039+0000","last_fullsized":"2024-09-21T00:27:37.409039+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_clean_scrub_stamp":"2024-09-21T00:27:34.492209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:54:33.269043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"59'8","reported_seq":21,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.542733+0000","last_change":"2024-09-21T00:27:34.521390+0000","last_active":"2024-09-21T00:27:35.542733+0000","last_peered":"2024-09-21T00:27:35.542733+0000","last_clean":"2024-09-21T00:27:35.542733+0000","last_became_active":"2024-09-21T00:27:34.521057+0000","last_became_peered":"2024-09-21T00:27:34.521057+0000","last_unstale":"2024-09-21T00:27:35.542733+0000","last_undegraded":"2024-09-21T00:27:35.542733+0000","last_fullsized":"2024-09-21T00:27:35.542733+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:33.485503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:33.485503+0000","last_clean_scrub_stamp":"2024-09-21T00:27:33.485503+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:41:29.148475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.532713+0000","last_change":"2024-09-21T00:27:35.530833+0000","last_active":"2024-09-21T00:27:35.532713+0000","last_peered":"2024-09-21T00:27:35.532713+0000","last_clean":"2024-09-21T00:27:35.532713+0000","last_became_active":"2024-09-21T00:27:35.530490+0000","last_became_peered":"2024-09-21T00:27:35.530490+0000","last_unstale":"2024-09-21T00:27:35.532713+0000","last_undegraded":"2024-09-21T00:27:35.532713+0000","last_fullsized":"2024-09-21T00:27:35.532713+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_clean_scrub_stamp":"2024-09-21T00:27:34.492209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:25:12.925267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"59'1","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.537392+0000","last_change":"2024-09-21T00:27:34.525466+0000","last_active":"2024-09-21T00:27:35.537392+0000","last_peered":"2024-09-21T00:27:35.537392+0000","last_clean":"2024-09-21T00:27:35.537392+0000","last_became_active":"2024-09-21T00:27:34.525237+0000","last_became_peered":"2024-09-21T00:27:34.525237+0000","last_unstale":"2024-09-21T00:27:35.537392+0000","last_undegraded":"2024-09-21T00:27:35.537392+0000","last_fullsized":"2024-09-21T00:27:35.537392+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:33.485503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:33.485503+0000","last_clean_scrub_stamp":"2024-09-21T00:27:33.485503+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:26:09.992862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.528346+0000","last_change":"2024-09-21T00:27:35.522238+0000","last_active":"2024-09-21T00:27:35.528346+0000","last_peered":"2024-09-21T00:27:35.528346+0000","last_clean":"2024-09-21T00:27:35.528346+0000","last_became_active":"2024-09-21T00:27:35.521864+0000","last_became_peered":"2024-09-21T00:27:35.521864+0000","last_unstale":"2024-09-21T00:27:35.528346+0000","last_undegraded":"2024-09-21T00:27:35.528346+0000","last_fullsized":"2024-09-21T00:27:35.528346+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_clean_scrub_stamp":"2024-09-21T00:27:34.492209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:57:44.119937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"59'2","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.536402+0000","last_change":"2024-09-21T00:27:34.529105+0000","last_active":"2024-09-21T00:27:35.536402+0000","last_peered":"2024-09-21T00:27:35.536402+0000","last_clean":"2024-09-21T00:27:35.536402+0000","last_became_active":"2024-09-21T00:27:34.528755+0000","last_became_peered":"2024-09-21T00:27:34.528755+0000","last_unstale":"2024-09-21T00:27:35.536402+0000","last_undegraded":"2024-09-21T00:27:35.536402+0000","last_fullsized":"2024-09-21T00:27:35.536402+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:33.485503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:33.485503+0000","last_clean_scrub_stamp":"2024-09-21T00:27:33.485503+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:30:54.154685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.530219+0000","last_change":"2024-09-21T00:27:35.527735+0000","last_active":"2024-09-21T00:27:35.530219+0000","last_peered":"2024-09-21T00:27:35.530219+0000","last_clean":"2024-09-21T00:27:35.530219+0000","last_became_active":"2024-09-21T00:27:35.527422+0000","last_became_peered":"2024-09-21T00:27:35.527422+0000","last_unstale":"2024-09-21T00:27:35.530219+0000","last_undegraded":"2024-09-21T00:27:35.530219+0000","last_fullsized":"2024-09-21T00:27:35.530219+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_clean_scrub_stamp":"2024-09-21T00:27:34.492209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:47:24.818304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"59'1","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.537936+0000","last_change":"2024-09-21T00:27:34.530636+0000","last_active":"2024-09-21T00:27:35.537936+0000","last_peered":"2024-09-21T00:27:35.537936+0000","last_clean":"2024-09-21T00:27:35.537936+0000","last_became_active":"2024-09-21T00:27:34.530331+0000","last_became_peered":"2024-09-21T00:27:34.530331+0000","last_unstale":"2024-09-21T00:27:35.537936+0000","last_undegraded":"2024-09-21T00:27:35.537936+0000","last_fullsized":"2024-09-21T00:27:35.537936+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:33.485503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:33.485503+0000","last_clean_scrub_stamp":"2024-09-21T00:27:33.485503+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:28:19.136531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.528950+0000","last_change":"2024-09-21T00:27:35.522279+0000","last_active":"2024-09-21T00:27:35.528950+0000","last_peered":"2024-09-21T00:27:35.528950+0000","last_clean":"2024-09-21T00:27:35.528950+0000","last_became_active":"2024-09-21T00:27:35.521995+0000","last_became_peered":"2024-09-21T00:27:35.521995+0000","last_unstale":"2024-09-21T00:27:35.528950+0000","last_undegraded":"2024-09-21T00:27:35.528950+0000","last_fullsized":"2024-09-21T00:27:35.528950+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_clean_scrub_stamp":"2024-09-21T00:27:34.492209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:52:28.779174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T00:27:36.515016+0000","last_change":"2024-09-21T00:27:34.521958+0000","last_active":"2024-09-21T00:27:36.515016+0000","last_peered":"2024-09-21T00:27:36.515016+0000","last_clean":"2024-09-21T00:27:36.515016+0000","last_became_active":"2024-09-21T00:27:34.521684+0000","last_became_peered":"2024-09-21T00:27:34.521684+0000","last_unstale":"2024-09-21T00:27:36.515016+0000","last_undegraded":"2024-09-21T00:27:36.515016+0000","last_fullsized":"2024-09-21T00:27:36.515016+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:33.485503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:33.485503+0000","last_clean_scrub_stamp":"2024-09-21T00:27:33.485503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:55:01.415137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.526575+0000","last_change":"2024-09-21T00:27:35.522994+0000","last_active":"2024-09-21T00:27:35.526575+0000","last_peered":"2024-09-21T00:27:35.526575+0000","last_clean":"2024-09-21T00:27:35.526575+0000","last_became_active":"2024-09-21T00:27:35.522685+0000","last_became_peered":"2024-09-21T00:27:35.522685+0000","last_unstale":"2024-09-21T00:27:35.526575+0000","last_undegraded":"2024-09-21T00:27:35.526575+0000","last_fullsized":"2024-09-21T00:27:35.526575+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_clean_scrub_stamp":"2024-09-21T00:27:34.492209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:10:47.234078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"59'2","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.535763+0000","last_change":"2024-09-21T00:27:34.517888+0000","last_active":"2024-09-21T00:27:35.535763+0000","last_peered":"2024-09-21T00:27:35.535763+0000","last_clean":"2024-09-21T00:27:35.535763+0000","last_became_active":"2024-09-21T00:27:34.517609+0000","last_became_peered":"2024-09-21T00:27:34.517609+0000","last_unstale":"2024-09-21T00:27:35.535763+0000","last_undegraded":"2024-09-21T00:27:35.535763+0000","last_fullsized":"2024-09-21T00:27:35.535763+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:33.485503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:33.485503+0000","last_clean_scrub_stamp":"2024-09-21T00:27:33.485503+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:54:19.403918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.526843+0000","last_change":"2024-09-21T00:27:35.522553+0000","last_active":"2024-09-21T00:27:35.526843+0000","last_peered":"2024-09-21T00:27:35.526843+0000","last_clean":"2024-09-21T00:27:35.526843+0000","last_became_active":"2024-09-21T00:27:35.522202+0000","last_became_peered":"2024-09-21T00:27:35.522202+0000","last_unstale":"2024-09-21T00:27:35.526843+0000","last_undegraded":"2024-09-21T00:27:35.526843+0000","last_fullsized":"2024-09-21T00:27:35.526843+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_clean_scrub_stamp":"2024-09-21T00:27:34.492209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:35:43.755267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.506675+0000","last_change":"2024-09-21T00:27:34.522341+0000","last_active":"2024-09-21T00:27:35.506675+0000","last_peered":"2024-09-21T00:27:35.506675+0000","last_clean":"2024-09-21T00:27:35.506675+0000","last_became_active":"2024-09-21T00:27:34.521969+0000","last_became_peered":"2024-09-21T00:27:34.521969+0000","last_unstale":"2024-09-21T00:27:35.506675+0000","last_undegraded":"2024-09-21T00:27:35.506675+0000","last_fullsized":"2024-09-21T00:27:35.506675+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:33.485503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:33.485503+0000","last_clean_scrub_stamp":"2024-09-21T00:27:33.485503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:07:02.103399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.521457+0000","last_change":"2024-09-21T00:27:35.516840+0000","last_active":"2024-09-21T00:27:35.521457+0000","last_peered":"2024-09-21T00:27:35.521457+0000","last_clean":"2024-09-21T00:27:35.521457+0000","last_became_active":"2024-09-21T00:27:35.516527+0000","last_became_peered":"2024-09-21T00:27:35.516527+0000","last_unstale":"2024-09-21T00:27:35.521457+0000","last_undegraded":"2024-09-21T00:27:35.521457+0000","last_fullsized":"2024-09-21T00:27:35.521457+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_clean_scrub_stamp":"2024-09-21T00:27:34.492209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:21:34.578319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.504059+0000","last_change":"2024-09-21T00:27:34.524960+0000","last_active":"2024-09-21T00:27:35.504059+0000","last_peered":"2024-09-21T00:27:35.504059+0000","last_clean":"2024-09-21T00:27:35.504059+0000","last_became_active":"2024-09-21T00:27:34.524684+0000","last_became_peered":"2024-09-21T00:27:34.524684+0000","last_unstale":"2024-09-21T00:27:35.504059+0000","last_undegraded":"2024-09-21T00:27:35.504059+0000","last_fullsized":"2024-09-21T00:27:35.504059+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:33.485503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:33.485503+0000","last_clean_scrub_stamp":"2024-09-21T00:27:33.485503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:51:49.528077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.522135+0000","last_change":"2024-09-21T00:27:35.514020+0000","last_active":"2024-09-21T00:27:35.522135+0000","last_peered":"2024-09-21T00:27:35.522135+0000","last_clean":"2024-09-21T00:27:35.522135+0000","last_became_active":"2024-09-21T00:27:35.513692+0000","last_became_peered":"2024-09-21T00:27:35.513692+0000","last_unstale":"2024-09-21T00:27:35.522135+0000","last_undegraded":"2024-09-21T00:27:35.522135+0000","last_fullsized":"2024-09-21T00:27:35.522135+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_clean_scrub_stamp":"2024-09-21T00:27:34.492209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:52:41.255394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"59'2","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.538104+0000","last_change":"2024-09-21T00:27:34.529099+0000","last_active":"2024-09-21T00:27:35.538104+0000","last_peered":"2024-09-21T00:27:35.538104+0000","last_clean":"2024-09-21T00:27:35.538104+0000","last_became_active":"2024-09-21T00:27:34.528758+0000","last_became_peered":"2024-09-21T00:27:34.528758+0000","last_unstale":"2024-09-21T00:27:35.538104+0000","last_undegraded":"2024-09-21T00:27:35.538104+0000","last_fullsized":"2024-09-21T00:27:35.538104+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:33.485503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:33.485503+0000","last_clean_scrub_stamp":"2024-09-21T00:27:33.485503+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:06:32.789690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.519739+0000","last_change":"2024-09-21T00:27:35.517401+0000","last_active":"2024-09-21T00:27:35.519739+0000","last_peered":"2024-09-21T00:27:35.519739+0000","last_clean":"2024-09-21T00:27:35.519739+0000","last_became_active":"2024-09-21T00:27:35.517053+0000","last_became_peered":"2024-09-21T00:27:35.517053+0000","last_unstale":"2024-09-21T00:27:35.519739+0000","last_undegraded":"2024-09-21T00:27:35.519739+0000","last_fullsized":"2024-09-21T00:27:35.519739+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_clean_scrub_stamp":"2024-09-21T00:27:34.492209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:21:59.104831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.506873+0000","last_change":"2024-09-21T00:27:34.527355+0000","last_active":"2024-09-21T00:27:35.506873+0000","last_peered":"2024-09-21T00:27:35.506873+0000","last_clean":"2024-09-21T00:27:35.506873+0000","last_became_active":"2024-09-21T00:27:34.527075+0000","last_became_peered":"2024-09-21T00:27:34.527075+0000","last_unstale":"2024-09-21T00:27:35.506873+0000","last_undegraded":"2024-09-21T00:27:35.506873+0000","last_fullsized":"2024-09-21T00:27:35.506873+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:33.485503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:33.485503+0000","last_clean_scrub_stamp":"2024-09-21T00:27:33.485503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:55:17.563693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.517379+0000","last_change":"2024-09-21T00:27:35.514745+0000","last_active":"2024-09-21T00:27:35.517379+0000","last_peered":"2024-09-21T00:27:35.517379+0000","last_clean":"2024-09-21T00:27:35.517379+0000","last_became_active":"2024-09-21T00:27:35.514449+0000","last_became_peered":"2024-09-21T00:27:35.514449+0000","last_unstale":"2024-09-21T00:27:35.517379+0000","last_undegraded":"2024-09-21T00:27:35.517379+0000","last_fullsized":"2024-09-21T00:27:35.517379+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_clean_scrub_stamp":"2024-09-21T00:27:34.492209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:16:59.907598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.504508+0000","last_change":"2024-09-21T00:27:34.525921+0000","last_active":"2024-09-21T00:27:35.504508+0000","last_peered":"2024-09-21T00:27:35.504508+0000","last_clean":"2024-09-21T00:27:35.504508+0000","last_became_active":"2024-09-21T00:27:34.525674+0000","last_became_peered":"2024-09-21T00:27:34.525674+0000","last_unstale":"2024-09-21T00:27:35.504508+0000","last_undegraded":"2024-09-21T00:27:35.504508+0000","last_fullsized":"2024-09-21T00:27:35.504508+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:33.485503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:33.485503+0000","last_clean_scrub_stamp":"2024-09-21T00:27:33.485503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:40:55.902137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.528921+0000","last_change":"2024-09-21T00:27:35.521339+0000","last_active":"2024-09-21T00:27:35.528921+0000","last_peered":"2024-09-21T00:27:35.528921+0000","last_clean":"2024-09-21T00:27:35.528921+0000","last_became_active":"2024-09-21T00:27:35.520986+0000","last_became_peered":"2024-09-21T00:27:35.520986+0000","last_unstale":"2024-09-21T00:27:35.528921+0000","last_undegraded":"2024-09-21T00:27:35.528921+0000","last_fullsized":"2024-09-21T00:27:35.528921+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_clean_scrub_stamp":"2024-09-21T00:27:34.492209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:33:38.410530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T00:27:37.409104+0000","last_change":"2024-09-21T00:27:34.521876+0000","last_active":"2024-09-21T00:27:37.409104+0000","last_peered":"2024-09-21T00:27:37.409104+0000","last_clean":"2024-09-21T00:27:37.409104+0000","last_became_active":"2024-09-21T00:27:34.521322+0000","last_became_peered":"2024-09-21T00:27:34.521322+0000","last_unstale":"2024-09-21T00:27:37.409104+0000","last_undegraded":"2024-09-21T00:27:37.409104+0000","last_fullsized":"2024-09-21T00:27:37.409104+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:33.485503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:33.485503+0000","last_clean_scrub_stamp":"2024-09-21T00:27:33.485503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:32:25.779379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.523472+0000","last_change":"2024-09-21T00:27:35.518016+0000","last_active":"2024-09-21T00:27:35.523472+0000","last_peered":"2024-09-21T00:27:35.523472+0000","last_clean":"2024-09-21T00:27:35.523472+0000","last_became_active":"2024-09-21T00:27:35.517718+0000","last_became_peered":"2024-09-21T00:27:35.517718+0000","last_unstale":"2024-09-21T00:27:35.523472+0000","last_undegraded":"2024-09-21T00:27:35.523472+0000","last_fullsized":"2024-09-21T00:27:35.523472+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_clean_scrub_stamp":"2024-09-21T00:27:34.492209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:17:05.651995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.526622+0000","last_change":"2024-09-21T00:27:34.528175+0000","last_active":"2024-09-21T00:27:35.526622+0000","last_peered":"2024-09-21T00:27:35.526622+0000","last_clean":"2024-09-21T00:27:35.526622+0000","last_became_active":"2024-09-21T00:27:34.527859+0000","last_became_peered":"2024-09-21T00:27:34.527859+0000","last_unstale":"2024-09-21T00:27:35.526622+0000","last_undegraded":"2024-09-21T00:27:35.526622+0000","last_fullsized":"2024-09-21T00:27:35.526622+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:33.485503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:33.485503+0000","last_clean_scrub_stamp":"2024-09-21T00:27:33.485503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:25:39.405975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.520718+0000","last_change":"2024-09-21T00:27:35.519359+0000","last_active":"2024-09-21T00:27:35.520718+0000","last_peered":"2024-09-21T00:27:35.520718+0000","last_clean":"2024-09-21T00:27:35.520718+0000","last_became_active":"2024-09-21T00:27:35.519043+0000","last_became_peered":"2024-09-21T00:27:35.519043+0000","last_unstale":"2024-09-21T00:27:35.520718+0000","last_undegraded":"2024-09-21T00:27:35.520718+0000","last_fullsized":"2024-09-21T00:27:35.520718+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_clean_scrub_stamp":"2024-09-21T00:27:34.492209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:02:54.748305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T00:27:36.515219+0000","last_change":"2024-09-21T00:27:34.519817+0000","last_active":"2024-09-21T00:27:36.515219+0000","last_peered":"2024-09-21T00:27:36.515219+0000","last_clean":"2024-09-21T00:27:36.515219+0000","last_became_active":"2024-09-21T00:27:34.519494+0000","last_became_peered":"2024-09-21T00:27:34.519494+0000","last_unstale":"2024-09-21T00:27:36.515219+0000","last_undegraded":"2024-09-21T00:27:36.515219+0000","last_fullsized":"2024-09-21T00:27:36.515219+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:33.485503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:33.485503+0000","last_clean_scrub_stamp":"2024-09-21T00:27:33.485503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:38:52.061271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T00:27:37.408913+0000","last_change":"2024-09-21T00:27:35.520761+0000","last_active":"2024-09-21T00:27:37.408913+0000","last_peered":"2024-09-21T00:27:37.408913+0000","last_clean":"2024-09-21T00:27:37.408913+0000","last_became_active":"2024-09-21T00:27:35.520408+0000","last_became_peered":"2024-09-21T00:27:35.520408+0000","last_unstale":"2024-09-21T00:27:37.408913+0000","last_undegraded":"2024-09-21T00:27:37.408913+0000","last_fullsized":"2024-09-21T00:27:37.408913+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_clean_scrub_stamp":"2024-09-21T00:27:34.492209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:07:30.325168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T00:27:36.514940+0000","last_change":"2024-09-21T00:27:34.517259+0000","last_active":"2024-09-21T00:27:36.514940+0000","last_peered":"2024-09-21T00:27:36.514940+0000","last_clean":"2024-09-21T00:27:36.514940+0000","last_became_active":"2024-09-21T00:27:34.517040+0000","last_became_peered":"2024-09-21T00:27:34.517040+0000","last_unstale":"2024-09-21T00:27:36.514940+0000","last_undegraded":"2024-09-21T00:27:36.514940+0000","last_fullsized":"2024-09-21T00:27:36.514940+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:33.485503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:33.485503+0000","last_clean_scrub_stamp":"2024-09-21T00:27:33.485503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:10:46.227372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.530737+0000","last_change":"2024-09-21T00:27:35.522666+0000","last_active":"2024-09-21T00:27:35.530737+0000","last_peered":"2024-09-21T00:27:35.530737+0000","last_clean":"2024-09-21T00:27:35.530737+0000","last_became_active":"2024-09-21T00:27:35.522354+0000","last_became_peered":"2024-09-21T00:27:35.522354+0000","last_unstale":"2024-09-21T00:27:35.530737+0000","last_undegraded":"2024-09-21T00:27:35.530737+0000","last_fullsized":"2024-09-21T00:27:35.530737+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_clean_scrub_stamp":"2024-09-21T00:27:34.492209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:33:26.078455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T00:27:37.408637+0000","last_change":"2024-09-21T00:27:34.522486+0000","last_active":"2024-09-21T00:27:37.408637+0000","last_peered":"2024-09-21T00:27:37.408637+0000","last_clean":"2024-09-21T00:27:37.408637+0000","last_became_active":"2024-09-21T00:27:34.522150+0000","last_became_peered":"2024-09-21T00:27:34.522150+0000","last_unstale":"2024-09-21T00:27:37.408637+0000","last_undegraded":"2024-09-21T00:27:37.408637+0000","last_fullsized":"2024-09-21T00:27:37.408637+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:33.485503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:33.485503+0000","last_clean_scrub_stamp":"2024-09-21T00:27:33.485503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:37:20.763443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.524141+0000","last_change":"2024-09-21T00:27:35.519635+0000","last_active":"2024-09-21T00:27:35.524141+0000","last_peered":"2024-09-21T00:27:35.524141+0000","last_clean":"2024-09-21T00:27:35.524141+0000","last_became_active":"2024-09-21T00:27:35.519334+0000","last_became_peered":"2024-09-21T00:27:35.519334+0000","last_unstale":"2024-09-21T00:27:35.524141+0000","last_undegraded":"2024-09-21T00:27:35.524141+0000","last_fullsized":"2024-09-21T00:27:35.524141+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:34.492209+0000","last_clean_scrub_stamp":"2024-09-21T00:27:34.492209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:35:21.403953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T00:27:36.514745+0000","last_change":"2024-09-21T00:27:34.520004+0000","last_active":"2024-09-21T00:27:36.514745+0000","last_peered":"2024-09-21T00:27:36.514745+0000","last_clean":"2024-09-21T00:27:36.514745+0000","last_became_active":"2024-09-21T00:27:34.519708+0000","last_became_peered":"2024-09-21T00:27:34.519708+0000","last_unstale":"2024-09-21T00:27:36.514745+0000","last_undegraded":"2024-09-21T00:27:36.514745+0000","last_fullsized":"2024-09-21T00:27:36.514745+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:27:33.485503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:27:33.485503+0000","last_clean_scrub_stamp":"2024-09-21T00:27:33.485503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:52:40.248688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":177,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T00:27:35.514107+0000","last_change":"2024-09-21T00:25:38.725792+0000","last_active":"2024-09-21T00:27:35.514107+0000","last_peered":"2024-09-21T00:27:35.514107+0000","last_clean":"2024-09-21T00:27:35.514107+0000","last_became_active":"2024-09-21T00:25:38.724668+0000","last_became_peered":"2024-09-21T00:25:38.724668+0000","last_unstale":"2024-09-21T00:27:35.514107+0000","last_undegraded":"2024-09-21T00:27:35.514107+0000","last_fullsized":"2024-09-21T00:27:35.514107+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:37.163216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:37.163216+0000","last_clean_scrub_stamp":"2024-09-21T00:25:37.163216+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:14:36.588660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509467,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28312,"kb_used_data":1284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343528,"statfs":{"total":96636764160,"available":96607772672,"internally_reserved":0,"allocated":1314816,"data_stored":901355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509468,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27720,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344120,"statfs":{"total":96636764160,"available":96608378880,"internally_reserved":0,"allocated":708608,"data_stored":310245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509467,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27724,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344116,"statfs":{"total":96636764160,"available":96608374784,"internally_reserved":0,"allocated":708608,"data_stored":310233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509466,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27728,"kb_used_data":700,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344112,"statfs":{"total":96636764160,"available":96608370688,"internally_reserved":0,"allocated":716800,"data_stored":311823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509468,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27720,"kb_used_data":688,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344120,"statfs":{"total":96636764160,"available":96608378880,"internally_reserved":0,"allocated":704512,"data_stored":310211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509468,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27716,"kb_used_data":688,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344124,"statfs":{"total":96636764160,"available":96608382976,"internally_reserved":0,"allocated":704512,"data_stored":310211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509468,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28300,"kb_used_data":1268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343540,"statfs":{"total":96636764160,"available":96607784960,"internally_reserved":0,"allocated":1298432,"data_stored":900579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574876,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31916,"kb_used_data":728,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339924,"statfs":{"total":96636764160,"available":96604082176,"internally_reserved":0,"allocated":745472,"data_stored":319279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T00:27:39.886 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T00:27:39.886 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-21T00:27:40.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:40.043+0000 7f17d350b640 1 -- 172.21.15.112:0/372911992 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17b8003a50 msgr2=0x7f17b8003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:40.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:40.043+0000 7f17d350b640 1 --2- 172.21.15.112:0/372911992 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17b8003a50 0x7f17b8003ec0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f17bc009b50 tx=0x7f17b8005830 comp rx=0 tx=0).stop 2024-09-21T00:27:40.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:40.043+0000 7f17d350b640 1 -- 172.21.15.112:0/372911992 shutdown_connections 2024-09-21T00:27:40.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:40.043+0000 7f17d350b640 1 --2- 172.21.15.112:0/372911992 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17b8003a50 0x7f17b8003ec0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:40.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:40.043+0000 7f17d350b640 1 -- 172.21.15.112:0/372911992 >> 172.21.15.112:0/372911992 conn(0x7f17cc0d4440 msgr2=0x7f17cc0d4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:40.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:40.043+0000 7f17d350b640 1 -- 172.21.15.112:0/372911992 shutdown_connections 2024-09-21T00:27:40.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:40.043+0000 7f17d350b640 1 -- 172.21.15.112:0/372911992 wait complete. 2024-09-21T00:27:40.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:40.044+0000 7f17d350b640 1 Processor -- start 2024-09-21T00:27:40.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:40.045+0000 7f17d350b640 1 -- start start 2024-09-21T00:27:40.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:40.046+0000 7f17d350b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17b8003a50 0x7f17cc1538d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:40.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:40.046+0000 7f17d350b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f17cc0b4ba0 0x7f17cc1f85b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:40.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:40.046+0000 7f17d350b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f17cc1f8af0 0x7f17cc1f5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:40.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:40.046+0000 7f17d350b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f17cc162e90 con 0x7f17b8003a50 2024-09-21T00:27:40.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:40.046+0000 7f17d350b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f17cc162d10 con 0x7f17cc0b4ba0 2024-09-21T00:27:40.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:40.046+0000 7f17d350b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f17cc0d2a20 con 0x7f17cc1f8af0 2024-09-21T00:27:40.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:40.046+0000 7f17d12a7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17b8003a50 0x7f17cc1538d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:40.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:40.046+0000 7f17d12a7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17b8003a50 0x7f17cc1538d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48288/0 (socket says 172.21.15.112:48288) 2024-09-21T00:27:40.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:40.046+0000 7f17d12a7640 1 -- 172.21.15.112:0/2287292908 learned_addr learned my addr 172.21.15.112:0/2287292908 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:27:40.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:40.046+0000 7f17d12a7640 1 -- 172.21.15.112:0/2287292908 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f17cc1f8af0 msgr2=0x7f17cc1f5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:40.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:40.046+0000 7f17d12a7640 1 --2- 172.21.15.112:0/2287292908 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f17cc1f8af0 0x7f17cc1f5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:40.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:40.046+0000 7f17d12a7640 1 -- 172.21.15.112:0/2287292908 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f17cc0b4ba0 msgr2=0x7f17cc1f85b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:40.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:40.046+0000 7f17d12a7640 1 --2- 172.21.15.112:0/2287292908 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f17cc0b4ba0 0x7f17cc1f85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:40.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:40.046+0000 7f17d12a7640 1 -- 172.21.15.112:0/2287292908 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17cc1f9430 con 0x7f17b8003a50 2024-09-21T00:27:40.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:40.046+0000 7f17d12a7640 1 --2- 172.21.15.112:0/2287292908 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17b8003a50 0x7f17cc1538d0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f17bc036040 tx=0x7f17bc0336f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:40.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:40.046+0000 7f17c1ffb640 1 -- 172.21.15.112:0/2287292908 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17bc038040 con 0x7f17b8003a50 2024-09-21T00:27:40.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:40.046+0000 7f17c1ffb640 1 -- 172.21.15.112:0/2287292908 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f17bc004230 con 0x7f17b8003a50 2024-09-21T00:27:40.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:40.046+0000 7f17d350b640 1 -- 172.21.15.112:0/2287292908 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17cc1f8f00 con 0x7f17b8003a50 2024-09-21T00:27:40.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:40.047+0000 7f17c1ffb640 1 -- 172.21.15.112:0/2287292908 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17bc0338e0 con 0x7f17b8003a50 2024-09-21T00:27:40.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:40.047+0000 7f17d350b640 1 -- 172.21.15.112:0/2287292908 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17cc15d5c0 con 0x7f17b8003a50 2024-09-21T00:27:40.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:40.048+0000 7f17d350b640 1 -- 172.21.15.112:0/2287292908 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17cc0d2f80 con 0x7f17b8003a50 2024-09-21T00:27:40.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:40.049+0000 7f17c1ffb640 1 -- 172.21.15.112:0/2287292908 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f17bc02b8a0 con 0x7f17b8003a50 2024-09-21T00:27:40.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:40.049+0000 7f17c1ffb640 1 --2- 172.21.15.112:0/2287292908 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f17b0078bc0 0x7f17b007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:40.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:40.049+0000 7f17c1ffb640 1 -- 172.21.15.112:0/2287292908 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7f17bc0bc050 con 0x7f17b8003a50 2024-09-21T00:27:40.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:40.049+0000 7f17d0aa6640 1 --2- 172.21.15.112:0/2287292908 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f17b0078bc0 0x7f17b007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:40.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:40.050+0000 7f17d0aa6640 1 --2- 172.21.15.112:0/2287292908 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f17b0078bc0 0x7f17b007b0d0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f17c8001860 tx=0x7f17c8000ed0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:40.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:40.053+0000 7f17c1ffb640 1 -- 172.21.15.112:0/2287292908 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f17bc084470 con 0x7f17b8003a50 2024-09-21T00:27:40.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:40.299+0000 7f17d350b640 1 -- 172.21.15.112:0/2287292908 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f17cc153e40 con 0x7f17b8003a50 2024-09-21T00:27:40.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:40.302+0000 7f17c1ffb640 1 -- 172.21.15.112:0/2287292908 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v17) ==== 190+0+147 (secure 0 0 0) 0x7f17bc083d70 con 0x7f17b8003a50 2024-09-21T00:27:40.303 INFO:teuthology.orchestra.run.smithi112.stdout:[client.0] 2024-09-21T00:27:40.303 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQBPEu5m49NPHBAAl/rcjiE8xYI+XdWwvZ6FOw== 2024-09-21T00:27:40.303 INFO:teuthology.orchestra.run.smithi112.stdout: caps mds = "allow" 2024-09-21T00:27:40.303 INFO:teuthology.orchestra.run.smithi112.stdout: caps mon = "allow r" 2024-09-21T00:27:40.303 INFO:teuthology.orchestra.run.smithi112.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-21T00:27:40.303 INFO:teuthology.orchestra.run.smithi112.stderr:updated caps for client.0 2024-09-21T00:27:40.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:40.304+0000 7f17d350b640 1 -- 172.21.15.112:0/2287292908 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f17b0078bc0 msgr2=0x7f17b007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:40.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:40.304+0000 7f17d350b640 1 --2- 172.21.15.112:0/2287292908 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f17b0078bc0 0x7f17b007b0d0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f17c8001860 tx=0x7f17c8000ed0 comp rx=0 tx=0).stop 2024-09-21T00:27:40.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:40.304+0000 7f17d350b640 1 -- 172.21.15.112:0/2287292908 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17b8003a50 msgr2=0x7f17cc1538d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:40.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:40.304+0000 7f17d350b640 1 --2- 172.21.15.112:0/2287292908 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17b8003a50 0x7f17cc1538d0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f17bc036040 tx=0x7f17bc0336f0 comp rx=0 tx=0).stop 2024-09-21T00:27:40.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:40.304+0000 7f17d350b640 1 -- 172.21.15.112:0/2287292908 shutdown_connections 2024-09-21T00:27:40.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:40.304+0000 7f17d350b640 1 --2- 172.21.15.112:0/2287292908 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f17cc1f8af0 0x7f17cc1f5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:40.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:40.304+0000 7f17d350b640 1 --2- 172.21.15.112:0/2287292908 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f17cc0b4ba0 0x7f17cc1f85b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:40.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:40.304+0000 7f17d350b640 1 --2- 172.21.15.112:0/2287292908 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f17b0078bc0 0x7f17b007b0d0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:40.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:40.304+0000 7f17d350b640 1 --2- 172.21.15.112:0/2287292908 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17b8003a50 0x7f17cc1538d0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:40.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:40.304+0000 7f17d350b640 1 -- 172.21.15.112:0/2287292908 >> 172.21.15.112:0/2287292908 conn(0x7f17cc0d4440 msgr2=0x7f17cc1609e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:40.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:40.305+0000 7f17d350b640 1 -- 172.21.15.112:0/2287292908 shutdown_connections 2024-09-21T00:27:40.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:40.305+0000 7f17d350b640 1 -- 172.21.15.112:0/2287292908 wait complete. 2024-09-21T00:27:40.316 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T00:27:40.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:40.428+0000 7fec43f89640 1 -- 172.21.15.112:0/1829612570 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec24003a50 msgr2=0x7fec24003ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:40.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:40.428+0000 7fec43f89640 1 --2- 172.21.15.112:0/1829612570 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec24003a50 0x7fec24003ee0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fec2c009ed0 tx=0x7fec24005830 comp rx=0 tx=0).stop 2024-09-21T00:27:40.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:40.428+0000 7fec43f89640 1 -- 172.21.15.112:0/1829612570 shutdown_connections 2024-09-21T00:27:40.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:40.428+0000 7fec43f89640 1 --2- 172.21.15.112:0/1829612570 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec24003a50 0x7fec24003ee0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:40.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:40.428+0000 7fec43f89640 1 -- 172.21.15.112:0/1829612570 >> 172.21.15.112:0/1829612570 conn(0x7fec3c0b2cc0 msgr2=0x7fec3c0b30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:40.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:40.428+0000 7fec43f89640 1 -- 172.21.15.112:0/1829612570 shutdown_connections 2024-09-21T00:27:40.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:40.428+0000 7fec43f89640 1 -- 172.21.15.112:0/1829612570 wait complete. 2024-09-21T00:27:40.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:40.430+0000 7fec43f89640 1 Processor -- start 2024-09-21T00:27:40.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:40.430+0000 7fec43f89640 1 -- start start 2024-09-21T00:27:40.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:40.431+0000 7fec43f89640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fec24003a50 0x7fec3c14ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:40.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:40.431+0000 7fec43f89640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fec3c15d330 0x7fec3c15d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:40.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:40.431+0000 7fec43f89640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec3c0cc520 0x7fec3c0cc930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:40.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:40.431+0000 7fec43f89640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fec3c0742e0 con 0x7fec3c0cc520 2024-09-21T00:27:40.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:40.431+0000 7fec43f89640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fec3c074160 con 0x7fec3c15d330 2024-09-21T00:27:40.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:40.431+0000 7fec43f89640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fec3c074460 con 0x7fec24003a50 2024-09-21T00:27:40.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:40.431+0000 7fec42526640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec3c0cc520 0x7fec3c0cc930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:40.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:40.431+0000 7fec42526640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec3c0cc520 0x7fec3c0cc930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48308/0 (socket says 172.21.15.112:48308) 2024-09-21T00:27:40.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:40.431+0000 7fec42526640 1 -- 172.21.15.112:0/386797927 learned_addr learned my addr 172.21.15.112:0/386797927 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:27:40.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:40.431+0000 7fec41524640 1 --2- 172.21.15.112:0/386797927 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fec3c15d330 0x7fec3c15d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:40.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:40.431+0000 7fec41d25640 1 --2- 172.21.15.112:0/386797927 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fec24003a50 0x7fec3c14ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:40.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:40.431+0000 7fec42526640 1 -- 172.21.15.112:0/386797927 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fec24003a50 msgr2=0x7fec3c14ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:40.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:40.431+0000 7fec42526640 1 --2- 172.21.15.112:0/386797927 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fec24003a50 0x7fec3c14ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:40.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:40.431+0000 7fec42526640 1 -- 172.21.15.112:0/386797927 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fec3c15d330 msgr2=0x7fec3c15d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:40.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:40.431+0000 7fec42526640 1 --2- 172.21.15.112:0/386797927 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fec3c15d330 0x7fec3c15d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:40.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:40.431+0000 7fec42526640 1 -- 172.21.15.112:0/386797927 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec3c0cce70 con 0x7fec3c0cc520 2024-09-21T00:27:40.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:40.431+0000 7fec41d25640 1 --2- 172.21.15.112:0/386797927 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fec24003a50 0x7fec3c14ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:27:40.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:40.432+0000 7fec42526640 1 --2- 172.21.15.112:0/386797927 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec3c0cc520 0x7fec3c0cc930 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fec3c075390 tx=0x7fec34001630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:40.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:40.432+0000 7fec3a7fc640 1 -- 172.21.15.112:0/386797927 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec34013860 con 0x7fec3c0cc520 2024-09-21T00:27:40.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:40.432+0000 7fec3a7fc640 1 -- 172.21.15.112:0/386797927 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fec34013a00 con 0x7fec3c0cc520 2024-09-21T00:27:40.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:40.432+0000 7fec43f89640 1 -- 172.21.15.112:0/386797927 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec3c1f5aa0 con 0x7fec3c0cc520 2024-09-21T00:27:40.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:40.432+0000 7fec3a7fc640 1 -- 172.21.15.112:0/386797927 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec34013e50 con 0x7fec3c0cc520 2024-09-21T00:27:40.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:40.432+0000 7fec43f89640 1 -- 172.21.15.112:0/386797927 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec3c1f6060 con 0x7fec3c0cc520 2024-09-21T00:27:40.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:40.433+0000 7fec43f89640 1 -- 172.21.15.112:0/386797927 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec3c0b4010 con 0x7fec3c0cc520 2024-09-21T00:27:40.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:40.434+0000 7fec3a7fc640 1 -- 172.21.15.112:0/386797927 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fec3401f470 con 0x7fec3c0cc520 2024-09-21T00:27:40.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:40.434+0000 7fec3a7fc640 1 --2- 172.21.15.112:0/386797927 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fec28078d30 0x7fec2807b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:40.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:40.434+0000 7fec3a7fc640 1 -- 172.21.15.112:0/386797927 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7fec3409da10 con 0x7fec3c0cc520 2024-09-21T00:27:40.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:40.434+0000 7fec41d25640 1 --2- 172.21.15.112:0/386797927 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fec28078d30 0x7fec2807b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:40.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:40.435+0000 7fec41d25640 1 --2- 172.21.15.112:0/386797927 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fec28078d30 0x7fec2807b240 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fec2c035000 tx=0x7fec2c004a40 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:40.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:40.439+0000 7fec3a7fc640 1 -- 172.21.15.112:0/386797927 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fec34065d50 con 0x7fec3c0cc520 2024-09-21T00:27:40.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:40.651+0000 7fec43f89640 1 -- 172.21.15.112:0/386797927 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fec3c0b0dd0 con 0x7fec3c0cc520 2024-09-21T00:27:40.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:40.652+0000 7fec3a7fc640 1 -- 172.21.15.112:0/386797927 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+5233 (secure 0 0 0) 0x7fec34065650 con 0x7fec3c0cc520 2024-09-21T00:27:40.653 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:27:40.653 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":35,"btime":"2024-09-21T00:27:38:527458+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5389,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6836/3984762342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":3984762342},{"type":"v1","addr":"172.21.15.187:6836","nonce":3984762342}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5392,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6837/763698806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6835","nonce":763698806},{"type":"v1","addr":"172.21.15.187:6837","nonce":763698806}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5736,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6834/1432588455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1432588455},{"type":"v1","addr":"172.21.15.112:6834","nonce":1432588455}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T00:27:34.871721+0000","modified":"2024-09-21T00:27:37.890630+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5739},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5739":{"gid":5739,"name":"a","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6835/3610025869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":3610025869},{"type":"v1","addr":"172.21.15.112:6835","nonce":3610025869}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5739,"qdb_cluster":[5739]},"id":4}]} 2024-09-21T00:27:40.653 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 35 2024-09-21T00:27:40.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:40.654+0000 7fec43f89640 1 -- 172.21.15.112:0/386797927 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fec28078d30 msgr2=0x7fec2807b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:40.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:40.654+0000 7fec43f89640 1 --2- 172.21.15.112:0/386797927 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fec28078d30 0x7fec2807b240 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fec2c035000 tx=0x7fec2c004a40 comp rx=0 tx=0).stop 2024-09-21T00:27:40.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:40.654+0000 7fec43f89640 1 -- 172.21.15.112:0/386797927 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec3c0cc520 msgr2=0x7fec3c0cc930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:40.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:40.654+0000 7fec43f89640 1 --2- 172.21.15.112:0/386797927 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec3c0cc520 0x7fec3c0cc930 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fec3c075390 tx=0x7fec34001630 comp rx=0 tx=0).stop 2024-09-21T00:27:40.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:40.654+0000 7fec43f89640 1 -- 172.21.15.112:0/386797927 shutdown_connections 2024-09-21T00:27:40.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:40.654+0000 7fec43f89640 1 --2- 172.21.15.112:0/386797927 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fec28078d30 0x7fec2807b240 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:40.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:40.654+0000 7fec43f89640 1 --2- 172.21.15.112:0/386797927 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec3c0cc520 0x7fec3c0cc930 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:40.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:40.654+0000 7fec43f89640 1 --2- 172.21.15.112:0/386797927 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fec3c15d330 0x7fec3c15d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:40.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:40.654+0000 7fec43f89640 1 --2- 172.21.15.112:0/386797927 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fec24003a50 0x7fec3c14ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:40.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:40.654+0000 7fec43f89640 1 -- 172.21.15.112:0/386797927 >> 172.21.15.112:0/386797927 conn(0x7fec3c0b2cc0 msgr2=0x7fec3c0cd2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:40.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:40.654+0000 7fec43f89640 1 -- 172.21.15.112:0/386797927 shutdown_connections 2024-09-21T00:27:40.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:40.654+0000 7fec43f89640 1 -- 172.21.15.112:0/386797927 wait complete. 2024-09-21T00:27:41.069 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 35, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T00:27:34.871721+0000', 'modified': '2024-09-21T00:27:37.890630+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5739}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5739': {'gid': 5739, 'name': 'a', 'rank': 0, 'incarnation': 31, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.112:6835/3610025869', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6833', 'nonce': 3610025869}, {'type': 'v1', 'addr': '172.21.15.112:6835', 'nonce': 3610025869}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5739, 'qdb_cluster': [5739]} 2024-09-21T00:27:41.069 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-21T00:27:41.069 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-21T00:27:41.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.191+0000 7f56c9121640 1 -- 172.21.15.112:0/926208885 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56c406def0 msgr2=0x7f56c406f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:41.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.191+0000 7f56c9121640 1 --2- 172.21.15.112:0/926208885 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56c406def0 0x7f56c406f410 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f56c40ced50 tx=0x7f56b402a560 comp rx=0 tx=0).stop 2024-09-21T00:27:41.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.191+0000 7f56c9121640 1 -- 172.21.15.112:0/926208885 shutdown_connections 2024-09-21T00:27:41.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.191+0000 7f56c9121640 1 --2- 172.21.15.112:0/926208885 >> v2:172.21.15.187:3300/0 conn(0x7f56c40b5670 0x7f56c415f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:41.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.191+0000 7f56c9121640 1 --2- 172.21.15.112:0/926208885 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56c406def0 0x7f56c406f410 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:41.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.191+0000 7f56c9121640 1 -- 172.21.15.112:0/926208885 >> 172.21.15.112:0/926208885 conn(0x7f56c40cdca0 msgr2=0x7f56c40ce0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:41.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.191+0000 7f56c9121640 1 -- 172.21.15.112:0/926208885 shutdown_connections 2024-09-21T00:27:41.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.191+0000 7f56c9121640 1 -- 172.21.15.112:0/926208885 wait complete. 2024-09-21T00:27:41.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.193+0000 7f56c9121640 1 Processor -- start 2024-09-21T00:27:41.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.193+0000 7f56c9121640 1 -- start start 2024-09-21T00:27:41.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.194+0000 7f56c9121640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f56c406def0 0x7f56c40ca2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:41.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.194+0000 7f56c9121640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56c40b5260 0x7f56c415cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:41.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.194+0000 7f56c9121640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f56c40b5670 0x7f56c415d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:41.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.194+0000 7f56c9121640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f56c415ff00 con 0x7f56c40b5260 2024-09-21T00:27:41.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.194+0000 7f56c9121640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f56c415fd80 con 0x7f56c40b5670 2024-09-21T00:27:41.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.194+0000 7f56c9121640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f56c4071300 con 0x7f56c406def0 2024-09-21T00:27:41.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.195+0000 7f56bbfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56c40b5260 0x7f56c415cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:41.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.195+0000 7f56bbfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56c40b5260 0x7f56c415cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48318/0 (socket says 172.21.15.112:48318) 2024-09-21T00:27:41.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.195+0000 7f56bbfff640 1 -- 172.21.15.112:0/3085280203 learned_addr learned my addr 172.21.15.112:0/3085280203 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:27:41.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.195+0000 7f56c359e640 1 --2- 172.21.15.112:0/3085280203 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f56c40b5670 0x7f56c415d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:41.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.195+0000 7f56c2d9d640 1 --2- 172.21.15.112:0/3085280203 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f56c406def0 0x7f56c40ca2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:41.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.195+0000 7f56bbfff640 1 -- 172.21.15.112:0/3085280203 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f56c406def0 msgr2=0x7f56c40ca2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:41.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.195+0000 7f56bbfff640 1 --2- 172.21.15.112:0/3085280203 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f56c406def0 0x7f56c40ca2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:41.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.195+0000 7f56bbfff640 1 -- 172.21.15.112:0/3085280203 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f56c40b5670 msgr2=0x7f56c415d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:41.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.195+0000 7f56bbfff640 1 --2- 172.21.15.112:0/3085280203 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f56c40b5670 0x7f56c415d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:41.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.195+0000 7f56bbfff640 1 -- 172.21.15.112:0/3085280203 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56c409bee0 con 0x7f56c40b5260 2024-09-21T00:27:41.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.195+0000 7f56bbfff640 1 --2- 172.21.15.112:0/3085280203 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56c40b5260 0x7f56c415cf60 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f56ac00b3c0 tx=0x7f56ac00b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:41.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.195+0000 7f56bb7fe640 1 -- 172.21.15.112:0/3085280203 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56ac011830 con 0x7f56c40b5260 2024-09-21T00:27:41.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.195+0000 7f56bb7fe640 1 -- 172.21.15.112:0/3085280203 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f56ac016070 con 0x7f56c40b5260 2024-09-21T00:27:41.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.195+0000 7f56bb7fe640 1 -- 172.21.15.112:0/3085280203 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56ac011da0 con 0x7f56c40b5260 2024-09-21T00:27:41.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.195+0000 7f56c9121640 1 -- 172.21.15.112:0/3085280203 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56c409c1c0 con 0x7f56c40b5260 2024-09-21T00:27:41.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.195+0000 7f56c9121640 1 -- 172.21.15.112:0/3085280203 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56c409c780 con 0x7f56c40b5260 2024-09-21T00:27:41.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.197+0000 7f56bb7fe640 1 -- 172.21.15.112:0/3085280203 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f56ac01b460 con 0x7f56c40b5260 2024-09-21T00:27:41.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.198+0000 7f56bb7fe640 1 --2- 172.21.15.112:0/3085280203 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f56a4078b90 0x7f56a407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:41.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.198+0000 7f56bb7fe640 1 -- 172.21.15.112:0/3085280203 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7f56ac09a7b0 con 0x7f56c40b5260 2024-09-21T00:27:41.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.198+0000 7f56c2d9d640 1 --2- 172.21.15.112:0/3085280203 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f56a4078b90 0x7f56a407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:41.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.198+0000 7f56c2d9d640 1 --2- 172.21.15.112:0/3085280203 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f56a4078b90 0x7f56a407b0a0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f56b4033000 tx=0x7f56b4004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:41.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.199+0000 7f56d0089740 1 Processor -- start 2024-09-21T00:27:41.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.199+0000 7f56d0089740 1 -- start start 2024-09-21T00:27:41.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.200+0000 7f56d0089740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ef30044270 0x55ef30404970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:41.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.200+0000 7f56d0089740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55ef3001b3b0 0x55ef30412e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:41.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.200+0000 7f56d0089740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ef30023590 0x55ef304133b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:41.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.200+0000 7f56d0089740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55ef3035e820 con 0x55ef30023590 2024-09-21T00:27:41.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.200+0000 7f56d0089740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55ef3034dc50 con 0x55ef30044270 2024-09-21T00:27:41.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.200+0000 7f56d0089740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x55ef30351a60 con 0x55ef3001b3b0 2024-09-21T00:27:41.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.200+0000 7f56c359e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ef30023590 0x55ef304133b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:41.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.200+0000 7f56c359e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ef30023590 0x55ef304133b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48326/0 (socket says 172.21.15.112:48326) 2024-09-21T00:27:41.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.200+0000 7f56c359e640 1 -- 172.21.15.112:0/1788674325 learned_addr learned my addr 172.21.15.112:0/1788674325 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:27:41.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.200+0000 7f56c2d9d640 1 --2- 172.21.15.112:0/1788674325 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ef30044270 0x55ef30404970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:41.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.200+0000 7f56bbfff640 1 --2- 172.21.15.112:0/1788674325 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55ef3001b3b0 0x55ef30412e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:41.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.201+0000 7f56c359e640 1 -- 172.21.15.112:0/1788674325 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55ef3001b3b0 msgr2=0x55ef30412e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:27:41.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.201+0000 7f56c359e640 1 --2- 172.21.15.112:0/1788674325 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55ef3001b3b0 0x55ef30412e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:41.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.201+0000 7f56c359e640 1 -- 172.21.15.112:0/1788674325 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ef30044270 msgr2=0x55ef30404970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:27:41.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.201+0000 7f56c359e640 1 --2- 172.21.15.112:0/1788674325 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ef30044270 0x55ef30404970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:41.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.201+0000 7f56c359e640 1 -- 172.21.15.112:0/1788674325 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef300af850 con 0x55ef30023590 2024-09-21T00:27:41.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.201+0000 7f56c2d9d640 1 --2- 172.21.15.112:0/1788674325 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ef30044270 0x55ef30404970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:27:41.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.201+0000 7f56c359e640 1 --2- 172.21.15.112:0/1788674325 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ef30023590 0x55ef304133b0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f56bc009f50 tx=0x7f56bc010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3495a0a3c47514a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:41.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.201+0000 7f56b8ff9640 1 -- 172.21.15.112:0/1788674325 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56bc016070 con 0x55ef30023590 2024-09-21T00:27:41.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.201+0000 7f56b8ff9640 1 -- 172.21.15.112:0/1788674325 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f56bc004e10 con 0x55ef30023590 2024-09-21T00:27:41.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.201+0000 7f56b8ff9640 1 -- 172.21.15.112:0/1788674325 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56bc005220 con 0x55ef30023590 2024-09-21T00:27:41.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.202+0000 7f56d0089740 1 -- 172.21.15.112:0/1788674325 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ef30023590 msgr2=0x55ef304133b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:41.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.202+0000 7f56d0089740 1 --2- 172.21.15.112:0/1788674325 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ef30023590 0x55ef304133b0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f56bc009f50 tx=0x7f56bc010800 comp rx=0 tx=0).stop 2024-09-21T00:27:41.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.202+0000 7f56d0089740 1 -- 172.21.15.112:0/1788674325 shutdown_connections 2024-09-21T00:27:41.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.202+0000 7f56d0089740 1 --2- 172.21.15.112:0/1788674325 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ef30023590 0x55ef304133b0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:41.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.202+0000 7f56d0089740 1 --2- 172.21.15.112:0/1788674325 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55ef3001b3b0 0x55ef30412e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:41.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.202+0000 7f56d0089740 1 --2- 172.21.15.112:0/1788674325 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ef30044270 0x55ef30404970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:41.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.202+0000 7f56d0089740 1 -- 172.21.15.112:0/1788674325 >> 172.21.15.112:0/1788674325 conn(0x55ef300383c0 msgr2=0x55ef303ec260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:41.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.202+0000 7f56d0089740 1 -- 172.21.15.112:0/1788674325 shutdown_connections 2024-09-21T00:27:41.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.202+0000 7f56d0089740 1 -- 172.21.15.112:0/1788674325 wait complete. 2024-09-21T00:27:41.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.203+0000 7f56d0089740 1 -- start start 2024-09-21T00:27:41.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.203+0000 7f56d0089740 1 Processor -- start 2024-09-21T00:27:41.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.204+0000 7f56d0089740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55ef3001b3b0 0x55ef30444550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:41.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.204+0000 7f56d0089740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ef30023590 0x55ef3042b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:41.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.204+0000 7f56d0089740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ef30044270 0x55ef3042b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:41.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.204+0000 7f56d0089740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55ef300be620 con 0x55ef30023590 2024-09-21T00:27:41.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.204+0000 7f56d0089740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55ef302f35f0 con 0x55ef30044270 2024-09-21T00:27:41.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.204+0000 7f56d0089740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x55ef303f71b0 con 0x55ef3001b3b0 2024-09-21T00:27:41.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.204+0000 7f56d0089740 10 client.0 _ll_register_callbacks cb 0x55ef302f70a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:27:41.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.204+0000 7f56bbfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ef30023590 0x55ef3042b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:41.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.204+0000 7f56bbfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ef30023590 0x55ef3042b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48342/0 (socket says 172.21.15.112:48342) 2024-09-21T00:27:41.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.204+0000 7f56bbfff640 1 -- 172.21.15.112:0/2383656703 learned_addr learned my addr 172.21.15.112:0/2383656703 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:27:41.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.204+0000 7f56c2d9d640 1 --2- 172.21.15.112:0/2383656703 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55ef3001b3b0 0x55ef30444550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:41.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.204+0000 7f56c359e640 1 --2- 172.21.15.112:0/2383656703 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ef30044270 0x55ef3042b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:41.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.205+0000 7f56bbfff640 1 -- 172.21.15.112:0/2383656703 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55ef3001b3b0 msgr2=0x55ef30444550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:27:41.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.205+0000 7f56bbfff640 1 --2- 172.21.15.112:0/2383656703 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55ef3001b3b0 0x55ef30444550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:41.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.205+0000 7f56bbfff640 1 -- 172.21.15.112:0/2383656703 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ef30044270 msgr2=0x55ef3042b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:27:41.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.205+0000 7f56bbfff640 1 --2- 172.21.15.112:0/2383656703 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ef30044270 0x55ef3042b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:41.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.205+0000 7f56bbfff640 1 -- 172.21.15.112:0/2383656703 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef301266b0 con 0x55ef30023590 2024-09-21T00:27:41.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.205+0000 7f56c2d9d640 1 --2- 172.21.15.112:0/2383656703 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55ef3001b3b0 0x55ef30444550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:27:41.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.205+0000 7f56bbfff640 1 --2- 172.21.15.112:0/2383656703 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ef30023590 0x55ef3042b370 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f56ac098b50 tx=0x7f56ac061f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ab58e24203927b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:41.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.205+0000 7f56a1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T00:27:41.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.205+0000 7f56a1ffb640 1 -- 172.21.15.112:0/2383656703 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56ac066ed0 con 0x55ef30023590 2024-09-21T00:27:41.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.205+0000 7f56a1ffb640 1 -- 172.21.15.112:0/2383656703 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7f56ac079d50 con 0x55ef30023590 2024-09-21T00:27:41.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.205+0000 7f56d0089740 1 -- 172.21.15.112:0/2383656703 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f56bc004e10 con 0x55ef30023590 2024-09-21T00:27:41.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.205+0000 7f56a1ffb640 1 -- 172.21.15.112:0/2383656703 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f56ac06bd80 con 0x55ef30023590 2024-09-21T00:27:41.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.205+0000 7f56d0089740 10 client.5877 fetch_fsmap learned FSMap version 35 2024-09-21T00:27:41.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.205+0000 7f56d0089740 1 -- 172.21.15.112:0/2383656703 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) -- 0x55ef30444a90 con 0x55ef30023590 2024-09-21T00:27:41.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.205+0000 7f56a1ffb640 1 -- 172.21.15.112:0/2383656703 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f56bc004e10 con 0x55ef30023590 2024-09-21T00:27:41.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.205+0000 7f56a1ffb640 1 -- 172.21.15.112:0/2383656703 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 35) ==== 3502+0+0 (secure 0 0 0) 0x7f56ac084b60 con 0x55ef30023590 2024-09-21T00:27:41.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.205+0000 7f56d0089740 10 client.5877 fetch_fsmap finished waiting for FSMap version 35 2024-09-21T00:27:41.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.205+0000 7f56d0089740 10 client.5877 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-21T00:27:41.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.205+0000 7f56d0089740 20 client.5877 populate_metadata read hostname 'smithi112' 2024-09-21T00:27:41.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.205+0000 7f56d0089740 1 --2- 172.21.15.112:0/2383656703 >> [v2:172.21.15.112:6833/3610025869,v1:172.21.15.112:6835/3610025869] conn(0x55ef30002e40 0x55ef303ec0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:41.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.205+0000 7f56d0089740 4 client.5877 mds_command: new command op to 5739 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T00:27:41.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.205+0000 7f56d0089740 1 -- 172.21.15.112:0/2383656703 --> [v2:172.21.15.112:6833/3610025869,v1:172.21.15.112:6835/3610025869] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ef30360770 con 0x55ef30002e40 2024-09-21T00:27:41.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.206+0000 7f56c2d9d640 1 --2- 172.21.15.112:0/2383656703 >> [v2:172.21.15.112:6833/3610025869,v1:172.21.15.112:6835/3610025869] conn(0x55ef30002e40 0x55ef303ec0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:41.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.206+0000 7f56a1ffb640 10 client.5877 ms_handle_connect on v2:172.21.15.112:6833/3610025869 2024-09-21T00:27:41.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.206+0000 7f56c2d9d640 1 --2- 172.21.15.112:0/2383656703 >> [v2:172.21.15.112:6833/3610025869,v1:172.21.15.112:6835/3610025869] conn(0x55ef30002e40 0x55ef303ec0a0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36f15d72e49797e6 server_cookie=7e0d54c2423bd144 in_seq=0 out_seq=0 2024-09-21T00:27:41.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.206+0000 7f56a1ffb640 1 -- 172.21.15.112:0/2383656703 <== mds.0 v2:172.21.15.112:6833/3610025869 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ef30360770 con 0x55ef30002e40 2024-09-21T00:27:41.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.206+0000 7f56a1ffb640 10 client.5877 handle_command_reply: tid=0 2024-09-21T00:27:41.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.207+0000 7f56d0089740 1 client.5877 shutdown 2024-09-21T00:27:41.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.207+0000 7f56d0089740 1 -- 172.21.15.112:0/2383656703 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ef30023590 msgr2=0x55ef3042b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:41.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.207+0000 7f56d0089740 1 --2- 172.21.15.112:0/2383656703 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ef30023590 0x55ef3042b370 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f56ac098b50 tx=0x7f56ac061f10 comp rx=0 tx=0).stop 2024-09-21T00:27:41.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.207+0000 7f56d0089740 1 -- 172.21.15.112:0/2383656703 shutdown_connections 2024-09-21T00:27:41.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.207+0000 7f56d0089740 1 --2- 172.21.15.112:0/2383656703 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ef30044270 0x55ef3042b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:41.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.207+0000 7f56d0089740 1 --2- 172.21.15.112:0/2383656703 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ef30023590 0x55ef3042b370 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:41.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.207+0000 7f56d0089740 1 --2- 172.21.15.112:0/2383656703 >> [v2:172.21.15.112:6833/3610025869,v1:172.21.15.112:6835/3610025869] conn(0x55ef30002e40 0x55ef303ec0a0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:41.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.207+0000 7f56d0089740 1 --2- 172.21.15.112:0/2383656703 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55ef3001b3b0 0x55ef30444550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:41.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.207+0000 7f56a1ffb640 0 client.5877 ms_handle_reset on v2:172.21.15.112:6833/3610025869 2024-09-21T00:27:41.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.207+0000 7f56d0089740 1 -- 172.21.15.112:0/2383656703 >> 172.21.15.112:0/2383656703 conn(0x55ef300383c0 msgr2=0x55ef304061b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:41.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.207+0000 7f56d0089740 1 -- 172.21.15.112:0/2383656703 shutdown_connections 2024-09-21T00:27:41.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.207+0000 7f56d0089740 1 -- 172.21.15.112:0/2383656703 wait complete. 2024-09-21T00:27:41.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.207+0000 7f56d0089740 20 client.5877 trim_cache size 0 max 16384 2024-09-21T00:27:41.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.225+0000 7f56d0089740 1 Processor -- start 2024-09-21T00:27:41.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.225+0000 7f56d0089740 1 -- start start 2024-09-21T00:27:41.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.226+0000 7f56d0089740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ef3001b3b0 0x55ef3042a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:41.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.226+0000 7f56d0089740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ef30023590 0x55ef304580c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:41.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.226+0000 7f56d0089740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55ef30044270 0x55ef30458600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:41.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.226+0000 7f56d0089740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f56c41fe210 con 0x55ef3001b3b0 2024-09-21T00:27:41.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.226+0000 7f56d0089740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f56c41fd270 con 0x55ef30023590 2024-09-21T00:27:41.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.226+0000 7f56d0089740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f56c406d740 con 0x55ef30044270 2024-09-21T00:27:41.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.226+0000 7f56c2d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ef3001b3b0 0x55ef3042a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:41.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.226+0000 7f56c2d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ef3001b3b0 0x55ef3042a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48356/0 (socket says 172.21.15.112:48356) 2024-09-21T00:27:41.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.226+0000 7f56c2d9d640 1 -- 172.21.15.112:0/2407539012 learned_addr learned my addr 172.21.15.112:0/2407539012 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:27:41.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.227+0000 7f56c2d9d640 1 -- 172.21.15.112:0/2407539012 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55ef30044270 msgr2=0x55ef30458600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:27:41.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.227+0000 7f56c2d9d640 1 --2- 172.21.15.112:0/2407539012 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55ef30044270 0x55ef30458600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:41.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.227+0000 7f56c2d9d640 1 -- 172.21.15.112:0/2407539012 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ef30023590 msgr2=0x55ef304580c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:27:41.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.227+0000 7f56c2d9d640 1 --2- 172.21.15.112:0/2407539012 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ef30023590 0x55ef304580c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:41.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.227+0000 7f56c2d9d640 1 -- 172.21.15.112:0/2407539012 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef30447c90 con 0x55ef3001b3b0 2024-09-21T00:27:41.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.227+0000 7f56c2d9d640 1 --2- 172.21.15.112:0/2407539012 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ef3001b3b0 0x55ef3042a900 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f56b4034040 tx=0x7f56b403d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22e588498b17a8d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:41.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.227+0000 7f56a3fff640 1 -- 172.21.15.112:0/2407539012 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56b4035040 con 0x55ef3001b3b0 2024-09-21T00:27:41.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.227+0000 7f56a3fff640 1 -- 172.21.15.112:0/2407539012 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f56b4057070 con 0x55ef3001b3b0 2024-09-21T00:27:41.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.227+0000 7f56a3fff640 1 -- 172.21.15.112:0/2407539012 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56b40049a0 con 0x55ef3001b3b0 2024-09-21T00:27:41.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.228+0000 7f56d0089740 1 -- 172.21.15.112:0/2407539012 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ef3001b3b0 msgr2=0x55ef3042a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:41.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.228+0000 7f56d0089740 1 --2- 172.21.15.112:0/2407539012 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ef3001b3b0 0x55ef3042a900 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f56b4034040 tx=0x7f56b403d000 comp rx=0 tx=0).stop 2024-09-21T00:27:41.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.228+0000 7f56d0089740 1 -- 172.21.15.112:0/2407539012 shutdown_connections 2024-09-21T00:27:41.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.228+0000 7f56d0089740 1 --2- 172.21.15.112:0/2407539012 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55ef30044270 0x55ef30458600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:41.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.228+0000 7f56d0089740 1 --2- 172.21.15.112:0/2407539012 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ef30023590 0x55ef304580c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:41.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.228+0000 7f56d0089740 1 --2- 172.21.15.112:0/2407539012 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ef3001b3b0 0x55ef3042a900 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:41.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.228+0000 7f56d0089740 1 -- 172.21.15.112:0/2407539012 >> 172.21.15.112:0/2407539012 conn(0x55ef300383c0 msgr2=0x55ef30431e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:41.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.228+0000 7f56d0089740 1 -- 172.21.15.112:0/2407539012 shutdown_connections 2024-09-21T00:27:41.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.228+0000 7f56d0089740 1 -- 172.21.15.112:0/2407539012 wait complete. 2024-09-21T00:27:41.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.228+0000 7f56d0089740 1 -- start start 2024-09-21T00:27:41.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.229+0000 7f56d0089740 1 Processor -- start 2024-09-21T00:27:41.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.230+0000 7f56d0089740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55ef3001b3b0 0x55ef304b8120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:41.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.230+0000 7f56d0089740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ef30023590 0x55ef303e7f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:41.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.230+0000 7f56d0089740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ef30044270 0x55ef30404bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:41.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.230+0000 7f56d0089740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55ef30448810 con 0x55ef30023590 2024-09-21T00:27:41.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.230+0000 7f56d0089740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55ef30448340 con 0x55ef30044270 2024-09-21T00:27:41.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.230+0000 7f56d0089740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x55ef303f94d0 con 0x55ef3001b3b0 2024-09-21T00:27:41.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.230+0000 7f56d0089740 10 client.0 _ll_register_callbacks cb 0x7f56ac061fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:27:41.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.230+0000 7f56bbfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ef30023590 0x55ef303e7f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:41.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.230+0000 7f56bbfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ef30023590 0x55ef303e7f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48366/0 (socket says 172.21.15.112:48366) 2024-09-21T00:27:41.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.230+0000 7f56bbfff640 1 -- 172.21.15.112:0/1224354985 learned_addr learned my addr 172.21.15.112:0/1224354985 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:27:41.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.230+0000 7f56c359e640 1 --2- 172.21.15.112:0/1224354985 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ef30044270 0x55ef30404bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:41.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.230+0000 7f56bbfff640 1 -- 172.21.15.112:0/1224354985 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55ef3001b3b0 msgr2=0x55ef304b8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:27:41.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.230+0000 7f56c2d9d640 1 --2- 172.21.15.112:0/1224354985 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55ef3001b3b0 0x55ef304b8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:41.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.230+0000 7f56bbfff640 1 --2- 172.21.15.112:0/1224354985 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55ef3001b3b0 0x55ef304b8120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:41.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.230+0000 7f56bbfff640 1 -- 172.21.15.112:0/1224354985 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ef30044270 msgr2=0x55ef30404bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:27:41.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.230+0000 7f56bbfff640 1 --2- 172.21.15.112:0/1224354985 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ef30044270 0x55ef30404bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:41.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.230+0000 7f56bbfff640 1 -- 172.21.15.112:0/1224354985 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef30414ce0 con 0x55ef30023590 2024-09-21T00:27:41.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.230+0000 7f56c359e640 1 --2- 172.21.15.112:0/1224354985 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ef30044270 0x55ef30404bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:27:41.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.230+0000 7f56bbfff640 1 --2- 172.21.15.112:0/1224354985 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ef30023590 0x55ef303e7f00 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f56ac098ba0 tx=0x7f56ac05e450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7273012fef16e25 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:41.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.230+0000 7f56a1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T00:27:41.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.230+0000 7f56a1ffb640 1 -- 172.21.15.112:0/1224354985 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56ac066ed0 con 0x55ef30023590 2024-09-21T00:27:41.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.231+0000 7f56a1ffb640 1 -- 172.21.15.112:0/1224354985 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7f56ac079d50 con 0x55ef30023590 2024-09-21T00:27:41.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.231+0000 7f56d0089740 1 -- 172.21.15.112:0/1224354985 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f56b4057070 con 0x55ef30023590 2024-09-21T00:27:41.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.231+0000 7f56a1ffb640 1 -- 172.21.15.112:0/1224354985 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f56ac061170 con 0x55ef30023590 2024-09-21T00:27:41.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.231+0000 7f56d0089740 10 client.5883 fetch_fsmap learned FSMap version 35 2024-09-21T00:27:41.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.231+0000 7f56d0089740 1 -- 172.21.15.112:0/1224354985 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) -- 0x55ef303e85a0 con 0x55ef30023590 2024-09-21T00:27:41.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.231+0000 7f56a1ffb640 1 -- 172.21.15.112:0/1224354985 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f56b4057070 con 0x55ef30023590 2024-09-21T00:27:41.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.231+0000 7f56a1ffb640 1 -- 172.21.15.112:0/1224354985 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 35) ==== 3502+0+0 (secure 0 0 0) 0x7f56ac083070 con 0x55ef30023590 2024-09-21T00:27:41.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.231+0000 7f56d0089740 10 client.5883 fetch_fsmap finished waiting for FSMap version 35 2024-09-21T00:27:41.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.231+0000 7f56d0089740 10 client.5883 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-21T00:27:41.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.231+0000 7f56d0089740 20 client.5883 populate_metadata read hostname 'smithi112' 2024-09-21T00:27:41.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.231+0000 7f56d0089740 1 --2- 172.21.15.112:0/1224354985 >> [v2:172.21.15.112:6833/3610025869,v1:172.21.15.112:6835/3610025869] conn(0x55ef3044f0d0 0x55ef303f7340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:41.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.231+0000 7f56d0089740 4 client.5883 mds_command: new command op to 5739 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T00:27:41.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.231+0000 7f56d0089740 1 -- 172.21.15.112:0/1224354985 --> [v2:172.21.15.112:6833/3610025869,v1:172.21.15.112:6835/3610025869] -- command(tid 0: {"prefix": "status"}) -- 0x55ef3035c220 con 0x55ef3044f0d0 2024-09-21T00:27:41.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.231+0000 7f56c2d9d640 1 --2- 172.21.15.112:0/1224354985 >> [v2:172.21.15.112:6833/3610025869,v1:172.21.15.112:6835/3610025869] conn(0x55ef3044f0d0 0x55ef303f7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:41.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.231+0000 7f56a1ffb640 10 client.5883 ms_handle_connect on v2:172.21.15.112:6833/3610025869 2024-09-21T00:27:41.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.231+0000 7f56c2d9d640 1 --2- 172.21.15.112:0/1224354985 >> [v2:172.21.15.112:6833/3610025869,v1:172.21.15.112:6835/3610025869] conn(0x55ef3044f0d0 0x55ef303f7340 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3fd675a0b2302852 server_cookie=e65317d1193893ce in_seq=0 out_seq=0 2024-09-21T00:27:41.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.232+0000 7f56a1ffb640 1 -- 172.21.15.112:0/1224354985 <== mds.0 v2:172.21.15.112:6833/3610025869 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55ef3035c220 con 0x55ef3044f0d0 2024-09-21T00:27:41.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.232+0000 7f56a1ffb640 10 client.5883 handle_command_reply: tid=0 2024-09-21T00:27:41.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.232+0000 7f56d0089740 1 client.5883 shutdown 2024-09-21T00:27:41.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.232+0000 7f56d0089740 1 -- 172.21.15.112:0/1224354985 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ef30023590 msgr2=0x55ef303e7f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:41.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.232+0000 7f56d0089740 1 --2- 172.21.15.112:0/1224354985 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ef30023590 0x55ef303e7f00 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f56ac098ba0 tx=0x7f56ac05e450 comp rx=0 tx=0).stop 2024-09-21T00:27:41.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.232+0000 7f56d0089740 1 -- 172.21.15.112:0/1224354985 shutdown_connections 2024-09-21T00:27:41.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.232+0000 7f56d0089740 1 --2- 172.21.15.112:0/1224354985 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ef30044270 0x55ef30404bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:41.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.232+0000 7f56d0089740 1 --2- 172.21.15.112:0/1224354985 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ef30023590 0x55ef303e7f00 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:41.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.232+0000 7f56d0089740 1 --2- 172.21.15.112:0/1224354985 >> [v2:172.21.15.112:6833/3610025869,v1:172.21.15.112:6835/3610025869] conn(0x55ef3044f0d0 0x55ef303f7340 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:41.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.232+0000 7f56d0089740 1 --2- 172.21.15.112:0/1224354985 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55ef3001b3b0 0x55ef304b8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:41.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.232+0000 7f56a1ffb640 0 client.5883 ms_handle_reset on v2:172.21.15.112:6833/3610025869 2024-09-21T00:27:41.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.232+0000 7f56d0089740 1 -- 172.21.15.112:0/1224354985 >> 172.21.15.112:0/1224354985 conn(0x55ef300383c0 msgr2=0x55ef303e1920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:41.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.232+0000 7f56d0089740 1 -- 172.21.15.112:0/1224354985 shutdown_connections 2024-09-21T00:27:41.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.232+0000 7f56d0089740 1 -- 172.21.15.112:0/1224354985 wait complete. 2024-09-21T00:27:41.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.232+0000 7f56d0089740 20 client.5883 trim_cache size 0 max 16384 2024-09-21T00:27:41.247 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-21T00:27:41.247 INFO:teuthology.orchestra.run.smithi112.stdout: "cluster_fsid": "8b34692b-dc65-49b2-a08f-26f707b96ac5", 2024-09-21T00:27:41.247 INFO:teuthology.orchestra.run.smithi112.stdout: "whoami": 0, 2024-09-21T00:27:41.248 INFO:teuthology.orchestra.run.smithi112.stdout: "id": 5739, 2024-09-21T00:27:41.248 INFO:teuthology.orchestra.run.smithi112.stdout: "want_state": "up:active", 2024-09-21T00:27:41.248 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "up:active", 2024-09-21T00:27:41.248 INFO:teuthology.orchestra.run.smithi112.stdout: "fs_name": "cephfs", 2024-09-21T00:27:41.248 INFO:teuthology.orchestra.run.smithi112.stdout: "rank_uptime": 5.7193431849999996, 2024-09-21T00:27:41.248 INFO:teuthology.orchestra.run.smithi112.stdout: "mdsmap_epoch": 35, 2024-09-21T00:27:41.248 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch": 60, 2024-09-21T00:27:41.248 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch_barrier": 60, 2024-09-21T00:27:41.248 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 10.403259004000001 2024-09-21T00:27:41.248 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-21T00:27:41.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.232+0000 7f56c9121640 1 -- 172.21.15.112:0/3085280203 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f56a4078b90 msgr2=0x7f56a407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:41.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.232+0000 7f56c9121640 1 --2- 172.21.15.112:0/3085280203 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f56a4078b90 0x7f56a407b0a0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f56b4033000 tx=0x7f56b4004730 comp rx=0 tx=0).stop 2024-09-21T00:27:41.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.232+0000 7f56c9121640 1 -- 172.21.15.112:0/3085280203 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56c40b5260 msgr2=0x7f56c415cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:41.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.232+0000 7f56c9121640 1 --2- 172.21.15.112:0/3085280203 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56c40b5260 0x7f56c415cf60 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f56ac00b3c0 tx=0x7f56ac00b880 comp rx=0 tx=0).stop 2024-09-21T00:27:41.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.232+0000 7f56c9121640 1 -- 172.21.15.112:0/3085280203 shutdown_connections 2024-09-21T00:27:41.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.232+0000 7f56c9121640 1 --2- 172.21.15.112:0/3085280203 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f56c40b5670 0x7f56c415d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:41.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.232+0000 7f56c9121640 1 --2- 172.21.15.112:0/3085280203 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f56a4078b90 0x7f56a407b0a0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:41.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.232+0000 7f56c9121640 1 --2- 172.21.15.112:0/3085280203 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56c40b5260 0x7f56c415cf60 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:41.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.233+0000 7f56c9121640 1 --2- 172.21.15.112:0/3085280203 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f56c406def0 0x7f56c40ca2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:41.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.233+0000 7f56c9121640 1 -- 172.21.15.112:0/3085280203 >> 172.21.15.112:0/3085280203 conn(0x7f56c40cdca0 msgr2=0x7f56c4071820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:41.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.233+0000 7f56c9121640 1 -- 172.21.15.112:0/3085280203 shutdown_connections 2024-09-21T00:27:41.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.233+0000 7f56c9121640 1 -- 172.21.15.112:0/3085280203 wait complete. 2024-09-21T00:27:41.253 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T00:27:41.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.369+0000 7fb637b9b640 1 -- 172.21.15.112:0/4143332215 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb63015a6f0 msgr2=0x7fb63015e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:41.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.369+0000 7fb637b9b640 1 --2- 172.21.15.112:0/4143332215 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb63015a6f0 0x7fb63015e230 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fb630073e10 tx=0x7fb62402aa30 comp rx=0 tx=0).stop 2024-09-21T00:27:41.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.369+0000 7fb637b9b640 1 -- 172.21.15.112:0/4143332215 shutdown_connections 2024-09-21T00:27:41.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.369+0000 7fb637b9b640 1 --2- 172.21.15.112:0/4143332215 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb63015a6f0 0x7fb63015e230 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:41.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.369+0000 7fb637b9b640 1 --2- 172.21.15.112:0/4143332215 >> v2:172.21.15.187:3300/0 conn(0x7fb6300d01f0 0x7fb63015daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:41.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.369+0000 7fb637b9b640 1 -- 172.21.15.112:0/4143332215 >> 172.21.15.112:0/4143332215 conn(0x7fb630072990 msgr2=0x7fb630072da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:41.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.369+0000 7fb637b9b640 1 -- 172.21.15.112:0/4143332215 shutdown_connections 2024-09-21T00:27:41.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.369+0000 7fb637b9b640 1 -- 172.21.15.112:0/4143332215 wait complete. 2024-09-21T00:27:41.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.370+0000 7fb637b9b640 1 Processor -- start 2024-09-21T00:27:41.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.371+0000 7fb637b9b640 1 -- start start 2024-09-21T00:27:41.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.372+0000 7fb637b9b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb6300d01f0 0x7fb630157910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:41.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.372+0000 7fb637b9b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb6300d0980 0x7fb6301f3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:41.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.372+0000 7fb637b9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb63015a6f0 0x7fb6301488c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:41.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.372+0000 7fb637b9b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb6300cd130 con 0x7fb63015a6f0 2024-09-21T00:27:41.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.372+0000 7fb637b9b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb6300ccfb0 con 0x7fb6300d01f0 2024-09-21T00:27:41.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.372+0000 7fb637b9b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fb6300cd2b0 con 0x7fb6300d0980 2024-09-21T00:27:41.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.372+0000 7fb636138640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb63015a6f0 0x7fb6301488c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:41.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.372+0000 7fb636138640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb63015a6f0 0x7fb6301488c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48390/0 (socket says 172.21.15.112:48390) 2024-09-21T00:27:41.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.372+0000 7fb636138640 1 -- 172.21.15.112:0/448548813 learned_addr learned my addr 172.21.15.112:0/448548813 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:27:41.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.372+0000 7fb635136640 1 --2- 172.21.15.112:0/448548813 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb6300d0980 0x7fb6301f3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:41.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.372+0000 7fb635937640 1 --2- 172.21.15.112:0/448548813 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb6300d01f0 0x7fb630157910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:41.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.372+0000 7fb636138640 1 -- 172.21.15.112:0/448548813 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb6300d0980 msgr2=0x7fb6301f3060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:41.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.372+0000 7fb636138640 1 --2- 172.21.15.112:0/448548813 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb6300d0980 0x7fb6301f3060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:41.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.372+0000 7fb636138640 1 -- 172.21.15.112:0/448548813 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb6300d01f0 msgr2=0x7fb630157910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:41.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.372+0000 7fb636138640 1 --2- 172.21.15.112:0/448548813 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb6300d01f0 0x7fb630157910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:41.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.372+0000 7fb636138640 1 -- 172.21.15.112:0/448548813 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6300cc1d0 con 0x7fb63015a6f0 2024-09-21T00:27:41.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.372+0000 7fb635937640 1 --2- 172.21.15.112:0/448548813 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb6300d01f0 0x7fb630157910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:27:41.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.372+0000 7fb636138640 1 --2- 172.21.15.112:0/448548813 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb63015a6f0 0x7fb6301488c0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fb62800b410 tx=0x7fb62800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:41.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.372+0000 7fb62e7fc640 1 -- 172.21.15.112:0/448548813 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb628015030 con 0x7fb63015a6f0 2024-09-21T00:27:41.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.372+0000 7fb62e7fc640 1 -- 172.21.15.112:0/448548813 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb628004020 con 0x7fb63015a6f0 2024-09-21T00:27:41.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.372+0000 7fb62e7fc640 1 -- 172.21.15.112:0/448548813 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb628011630 con 0x7fb63015a6f0 2024-09-21T00:27:41.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.373+0000 7fb637b9b640 1 -- 172.21.15.112:0/448548813 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6301f35a0 con 0x7fb63015a6f0 2024-09-21T00:27:41.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.373+0000 7fb637b9b640 1 -- 172.21.15.112:0/448548813 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb630146a80 con 0x7fb63015a6f0 2024-09-21T00:27:41.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.373+0000 7fb637b9b640 1 -- 172.21.15.112:0/448548813 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6300cd7e0 con 0x7fb63015a6f0 2024-09-21T00:27:41.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.374+0000 7fb62e7fc640 1 -- 172.21.15.112:0/448548813 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb628018070 con 0x7fb63015a6f0 2024-09-21T00:27:41.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.375+0000 7fb62e7fc640 1 --2- 172.21.15.112:0/448548813 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fb61c078bc0 0x7fb61c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:41.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.375+0000 7fb62e7fc640 1 -- 172.21.15.112:0/448548813 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7fb62809a8b0 con 0x7fb63015a6f0 2024-09-21T00:27:41.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.375+0000 7fb635937640 1 --2- 172.21.15.112:0/448548813 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fb61c078bc0 0x7fb61c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:41.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.376+0000 7fb635937640 1 --2- 172.21.15.112:0/448548813 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fb61c078bc0 0x7fb61c07b0d0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fb620005fd0 tx=0x7fb62000f360 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:41.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.380+0000 7fb62e7fc640 1 -- 172.21.15.112:0/448548813 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb628062bf0 con 0x7fb63015a6f0 2024-09-21T00:27:41.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.595+0000 7fb637b9b640 1 -- 172.21.15.112:0/448548813 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fb630146830 con 0x7fb63015a6f0 2024-09-21T00:27:41.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.596+0000 7fb62e7fc640 1 -- 172.21.15.112:0/448548813 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v35) ==== 53+0+73 (secure 0 0 0) 0x7fb6280624f0 con 0x7fb63015a6f0 2024-09-21T00:27:41.597 INFO:teuthology.orchestra.run.smithi112.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T00:27:41.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.598+0000 7fb637b9b640 1 -- 172.21.15.112:0/448548813 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fb61c078bc0 msgr2=0x7fb61c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:41.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.598+0000 7fb637b9b640 1 --2- 172.21.15.112:0/448548813 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fb61c078bc0 0x7fb61c07b0d0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fb620005fd0 tx=0x7fb62000f360 comp rx=0 tx=0).stop 2024-09-21T00:27:41.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.598+0000 7fb637b9b640 1 -- 172.21.15.112:0/448548813 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb63015a6f0 msgr2=0x7fb6301488c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:41.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.598+0000 7fb637b9b640 1 --2- 172.21.15.112:0/448548813 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb63015a6f0 0x7fb6301488c0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fb62800b410 tx=0x7fb62800b8d0 comp rx=0 tx=0).stop 2024-09-21T00:27:41.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.598+0000 7fb637b9b640 1 -- 172.21.15.112:0/448548813 shutdown_connections 2024-09-21T00:27:41.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.598+0000 7fb637b9b640 1 --2- 172.21.15.112:0/448548813 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fb61c078bc0 0x7fb61c07b0d0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:41.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.598+0000 7fb637b9b640 1 --2- 172.21.15.112:0/448548813 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb63015a6f0 0x7fb6301488c0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:41.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.598+0000 7fb637b9b640 1 --2- 172.21.15.112:0/448548813 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb6300d0980 0x7fb6301f3060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:41.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.598+0000 7fb637b9b640 1 --2- 172.21.15.112:0/448548813 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb6300d01f0 0x7fb630157910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:41.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.598+0000 7fb637b9b640 1 -- 172.21.15.112:0/448548813 >> 172.21.15.112:0/448548813 conn(0x7fb630072990 msgr2=0x7fb63015f520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:41.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.598+0000 7fb637b9b640 1 -- 172.21.15.112:0/448548813 shutdown_connections 2024-09-21T00:27:41.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41.598+0000 7fb637b9b640 1 -- 172.21.15.112:0/448548813 wait complete. 2024-09-21T00:27:41.610 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T00:27:41.610 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T00:27:41.611 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi112.front.sepia.ceph.com 2024-09-21T00:27:41.611 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T00:27:41.611 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T00:27:41.611 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T00:27:41.611 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-21T00:27:41.611 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T00:27:41.611 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T00:27:41.611 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:27:41.611 DEBUG:teuthology.orchestra.run.smithi112:> ip addr 2024-09-21T00:27:41.630 INFO:teuthology.orchestra.run.smithi112.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T00:27:41.630 INFO:teuthology.orchestra.run.smithi112.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T00:27:41.630 INFO:teuthology.orchestra.run.smithi112.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T00:27:41.631 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:27:41.631 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 ::1/128 scope host 2024-09-21T00:27:41.631 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:27:41.631 INFO:teuthology.orchestra.run.smithi112.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T00:27:41.631 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:d8:a5:2c brd ff:ff:ff:ff:ff:ff 2024-09-21T00:27:41.631 INFO:teuthology.orchestra.run.smithi112.stdout: altname enp7s0f0 2024-09-21T00:27:41.631 INFO:teuthology.orchestra.run.smithi112.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T00:27:41.631 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:d8:a5:2d brd ff:ff:ff:ff:ff:ff 2024-09-21T00:27:41.631 INFO:teuthology.orchestra.run.smithi112.stdout: altname enp7s0f1 2024-09-21T00:27:41.631 INFO:teuthology.orchestra.run.smithi112.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T00:27:41.631 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:8f:cb:68 brd ff:ff:ff:ff:ff:ff 2024-09-21T00:27:41.631 INFO:teuthology.orchestra.run.smithi112.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T00:27:41.631 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:8f:cb:69 brd ff:ff:ff:ff:ff:ff 2024-09-21T00:27:41.631 INFO:teuthology.orchestra.run.smithi112.stdout: inet 172.21.15.112/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-21T00:27:41.632 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft 42661sec preferred_lft 42661sec 2024-09-21T00:27:41.632 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::ec4:7aff:fe8f:cb69/64 scope link noprefixroute 2024-09-21T00:27:41.632 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:27:41.632 INFO:teuthology.orchestra.run.smithi112.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T00:27:41.632 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 1a:39:64:e7:fe:d7 brd ff:ff:ff:ff:ff:ff 2024-09-21T00:27:41.632 INFO:teuthology.orchestra.run.smithi112.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T00:27:41.632 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:27:41.632 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::5858:3dff:feaf:1774/64 scope link 2024-09-21T00:27:41.632 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:27:41.632 INFO:teuthology.orchestra.run.smithi112.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T00:27:41.632 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 1a:39:64:e7:fe:d7 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T00:27:41.632 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::1839:64ff:fee7:fed7/64 scope link 2024-09-21T00:27:41.632 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:27:41.632 INFO:teuthology.orchestra.run.smithi112.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T00:27:41.633 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 86:c9:6c:ef:e4:e9 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T00:27:41.633 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::84c9:6cff:feef:e4e9/64 scope link 2024-09-21T00:27:41.633 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:27:41.633 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:27:41.633 DEBUG:teuthology.orchestra.run.smithi112:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T00:27:41.697 INFO:teuthology.orchestra.run.smithi112.stdout:1 2024-09-21T00:27:41.699 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:27:41.699 DEBUG:teuthology.orchestra.run.smithi112:> route 2024-09-21T00:27:41.756 INFO:teuthology.orchestra.run.smithi112.stdout:Kernel IP routing table 2024-09-21T00:27:41.757 INFO:teuthology.orchestra.run.smithi112.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T00:27:41.757 INFO:teuthology.orchestra.run.smithi112.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-21T00:27:41.757 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-21T00:27:41.757 INFO:teuthology.orchestra.run.smithi112.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T00:27:41.757 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T00:27:41.757 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2024-09-21T00:27:41.757 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-21T00:27:41.757 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-21T00:27:41.757 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-21T00:27:41.758 DEBUG:teuthology.orchestra.run.smithi112:> ') 2024-09-21T00:27:41.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T00:27:41.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T00:27:41.884 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:27:41.884 DEBUG:teuthology.orchestra.run.smithi112:> ip netns list 2024-09-21T00:27:41.899 INFO:teuthology.orchestra.run.smithi112.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-21T00:27:41.900 INFO:teuthology.orchestra.run.smithi112.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T00:27:41.900 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T00:27:41.900 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:27:41.900 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link set brx.0 up 2024-09-21T00:27:41.964 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T00:27:41.964 DEBUG:teuthology.orchestra.run.smithi112:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T00:27:42.020 INFO:teuthology.orchestra.run.smithi112.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T00:27:42.020 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T00:27:42.020 DEBUG:teuthology.orchestra.run.smithi112:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T00:27:42.074 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-21T00:27:42.075 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:27:42.075 DEBUG:teuthology.orchestra.run.smithi112:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T00:27:42.169 INFO:teuthology.orchestra.run.smithi112.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T00:27:42.169 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T00:27:42.169 INFO:teuthology.orchestra.run.smithi112.stdout:invalid new device string format 2024-09-21T00:27:42.180 INFO:teuthology.orchestra.run.smithi112.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T00:27:42.180 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T00:27:42.180 INFO:teuthology.orchestra.run.smithi112.stdout:invalid new device string format 2024-09-21T00:27:42.180 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: resolved to: "172.21.15.112:3300,172.21.15.187:3300,172.21.15.187:3301" 2024-09-21T00:27:42.180 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T00:27:42.186 INFO:teuthology.orchestra.run.smithi112.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T00:27:42.186 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T00:27:42.186 INFO:teuthology.orchestra.run.smithi112.stdout:invalid new device string format 2024-09-21T00:27:42.186 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: resolved to: "172.21.15.112:3300,172.21.15.187:3300,172.21.15.187:3301" 2024-09-21T00:27:42.186 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T00:27:42.187 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: resolved to: "172.21.15.112:6789,172.21.15.187:6789,172.21.15.187:6790" 2024-09-21T00:27:42.187 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.112:6789,172.21.15.187:6789,172.21.15.187:6790:/ 2024-09-21T00:27:42.187 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=8b34692b-dc65-49b2-a08f-26f707b96ac5" will pass to kernel 2024-09-21T00:27:42.188 INFO:teuthology.orchestra.run.smithi112.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-21T00:27:42.189 INFO:teuthology.orchestra.run.smithi112.stdout: You just mounted a file system that supports labels which does not 2024-09-21T00:27:42.189 INFO:teuthology.orchestra.run.smithi112.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T00:27:42.189 INFO:teuthology.orchestra.run.smithi112.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T00:27:42.189 INFO:teuthology.orchestra.run.smithi112.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T00:27:42.189 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T00:27:42.189 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:27:42.189 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T00:27:42.219 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T00:27:42.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:42 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T00:27:42.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T00:27:42.321 INFO:teuthology.orchestra.run.smithi112.stdout:client5491 2024-09-21T00:27:42.321 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T00:27:42.321 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:27:42.338 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T00:27:42.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:42.489+0000 7f706f2ec640 1 -- 172.21.15.112:0/2407043666 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7054004d90 msgr2=0x7f7054005220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:42.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:42.489+0000 7f706f2ec640 1 --2- 172.21.15.112:0/2407043666 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7054004d90 0x7f7054005220 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f7060009ed0 tx=0x7f7054005830 comp rx=0 tx=0).stop 2024-09-21T00:27:42.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:42.489+0000 7f706d889640 1 -- 172.21.15.112:0/2407043666 reap_dead start 2024-09-21T00:27:42.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:42.489+0000 7f706f2ec640 1 -- 172.21.15.112:0/2407043666 shutdown_connections 2024-09-21T00:27:42.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:42.489+0000 7f706f2ec640 1 -- 172.21.15.112:0/2407043666 >> 172.21.15.112:0/2407043666 conn(0x7f70680d4440 msgr2=0x7f70680d4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:42.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:42.489+0000 7f706f2ec640 1 -- 172.21.15.112:0/2407043666 shutdown_connections 2024-09-21T00:27:42.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:42.489+0000 7f706f2ec640 1 -- 172.21.15.112:0/2407043666 wait complete. 2024-09-21T00:27:42.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:42.491+0000 7f706f2ec640 1 Processor -- start 2024-09-21T00:27:42.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:42.491+0000 7f706f2ec640 1 -- start start 2024-09-21T00:27:42.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:42.492+0000 7f706f2ec640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7054004d90 0x7f706815bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:42.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:42.492+0000 7f706f2ec640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f70681f5380 0x7f70681f5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:42.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:42.492+0000 7f706f2ec640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7068157280 0x7f7068157690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:42.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:42.492+0000 7f706f2ec640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f70680b31f0 con 0x7f7054004d90 2024-09-21T00:27:42.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:42.492+0000 7f706f2ec640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f70680b3070 con 0x7f7068157280 2024-09-21T00:27:42.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:42.492+0000 7f706f2ec640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f70680b3370 con 0x7f70681f5380 2024-09-21T00:27:42.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:42.492+0000 7f706d088640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7054004d90 0x7f706815bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:42.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:42.492+0000 7f706d088640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7054004d90 0x7f706815bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48402/0 (socket says 172.21.15.112:48402) 2024-09-21T00:27:42.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:42.492+0000 7f706d088640 1 -- 172.21.15.112:0/3879512599 learned_addr learned my addr 172.21.15.112:0/3879512599 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:27:42.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:42.492+0000 7f706d889640 1 --2- 172.21.15.112:0/3879512599 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7068157280 0x7f7068157690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:42.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:42.492+0000 7f706c887640 1 --2- 172.21.15.112:0/3879512599 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f70681f5380 0x7f70681f5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:42.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:42.492+0000 7f706d088640 1 -- 172.21.15.112:0/3879512599 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f70681f5380 msgr2=0x7f70681f5790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:42.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:42.492+0000 7f706d088640 1 --2- 172.21.15.112:0/3879512599 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f70681f5380 0x7f70681f5790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:42.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:42.492+0000 7f706d088640 1 -- 172.21.15.112:0/3879512599 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7068157280 msgr2=0x7f7068157690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:42.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:42.492+0000 7f706d088640 1 --2- 172.21.15.112:0/3879512599 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7068157280 0x7f7068157690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:42.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:42.492+0000 7f706d088640 1 -- 172.21.15.112:0/3879512599 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70681f66f0 con 0x7f7054004d90 2024-09-21T00:27:42.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:42.493+0000 7f706d088640 1 --2- 172.21.15.112:0/3879512599 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7054004d90 0x7f706815bb90 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f7064003e40 tx=0x7f7064001800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:42.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:42.493+0000 7f705dffb640 1 -- 172.21.15.112:0/3879512599 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7064011700 con 0x7f7054004d90 2024-09-21T00:27:42.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:42.493+0000 7f706f2ec640 1 -- 172.21.15.112:0/3879512599 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7068157bd0 con 0x7f7054004d90 2024-09-21T00:27:42.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:42.493+0000 7f705dffb640 1 -- 172.21.15.112:0/3879512599 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7064016020 con 0x7f7054004d90 2024-09-21T00:27:42.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:42.493+0000 7f705dffb640 1 -- 172.21.15.112:0/3879512599 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7064011b60 con 0x7f7054004d90 2024-09-21T00:27:42.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:42.493+0000 7f706f2ec640 1 -- 172.21.15.112:0/3879512599 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70681f69b0 con 0x7f7054004d90 2024-09-21T00:27:42.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:42.494+0000 7f706f2ec640 1 -- 172.21.15.112:0/3879512599 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70680b3900 con 0x7f7054004d90 2024-09-21T00:27:42.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:42.495+0000 7f705dffb640 1 -- 172.21.15.112:0/3879512599 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f7064011d00 con 0x7f7054004d90 2024-09-21T00:27:42.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:42.495+0000 7f705dffb640 1 --2- 172.21.15.112:0/3879512599 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f704c078b90 0x7f704c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:42.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:42.495+0000 7f705dffb640 1 -- 172.21.15.112:0/3879512599 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7f706409bd20 con 0x7f7054004d90 2024-09-21T00:27:42.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:42.496+0000 7f706c887640 1 --2- 172.21.15.112:0/3879512599 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f704c078b90 0x7f704c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:42.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:42.496+0000 7f706c887640 1 --2- 172.21.15.112:0/3879512599 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f704c078b90 0x7f704c07b0a0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f7068069500 tx=0x7f7058001dd0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:42.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:42.500+0000 7f705dffb640 1 -- 172.21.15.112:0/3879512599 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f7064020020 con 0x7f7054004d90 2024-09-21T00:27:42.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:42.716+0000 7f706f2ec640 1 -- 172.21.15.112:0/3879512599 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f70680b56d0 con 0x7f7054004d90 2024-09-21T00:27:42.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:42.716+0000 7f705dffb640 1 -- 172.21.15.112:0/3879512599 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v35) ==== 53+0+73 (secure 0 0 0) 0x7f7064063f50 con 0x7f7054004d90 2024-09-21T00:27:42.717 INFO:teuthology.orchestra.run.smithi112.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T00:27:42.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:42.718+0000 7f706f2ec640 1 -- 172.21.15.112:0/3879512599 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f704c078b90 msgr2=0x7f704c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:42.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:42.718+0000 7f706f2ec640 1 --2- 172.21.15.112:0/3879512599 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f704c078b90 0x7f704c07b0a0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f7068069500 tx=0x7f7058001dd0 comp rx=0 tx=0).stop 2024-09-21T00:27:42.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:42.718+0000 7f706f2ec640 1 -- 172.21.15.112:0/3879512599 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7054004d90 msgr2=0x7f706815bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:42.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:42.718+0000 7f706f2ec640 1 --2- 172.21.15.112:0/3879512599 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7054004d90 0x7f706815bb90 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f7064003e40 tx=0x7f7064001800 comp rx=0 tx=0).stop 2024-09-21T00:27:42.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:42.718+0000 7f706f2ec640 1 -- 172.21.15.112:0/3879512599 shutdown_connections 2024-09-21T00:27:42.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:42.718+0000 7f706f2ec640 1 --2- 172.21.15.112:0/3879512599 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7068157280 0x7f7068157690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:42.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:42.718+0000 7f706f2ec640 1 --2- 172.21.15.112:0/3879512599 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f70681f5380 0x7f70681f5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:42.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:42.718+0000 7f706f2ec640 1 --2- 172.21.15.112:0/3879512599 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f704c078b90 0x7f704c07b0a0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:42.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:42.718+0000 7f706f2ec640 1 --2- 172.21.15.112:0/3879512599 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7054004d90 0x7f706815bb90 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:42.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:42.718+0000 7f706f2ec640 1 -- 172.21.15.112:0/3879512599 >> 172.21.15.112:0/3879512599 conn(0x7f70680d4440 msgr2=0x7f7068158cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:42.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:42.719+0000 7f706f2ec640 1 -- 172.21.15.112:0/3879512599 shutdown_connections 2024-09-21T00:27:42.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:42.719+0000 7f706f2ec640 1 -- 172.21.15.112:0/3879512599 wait complete. 2024-09-21T00:27:42.731 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T00:27:42.731 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T00:27:42.731 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi187.front.sepia.ceph.com 2024-09-21T00:27:42.731 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-21T00:27:42.731 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-21T00:27:42.731 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T00:27:42.731 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-21T00:27:42.731 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T00:27:42.731 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-21T00:27:42.732 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:27:42.732 DEBUG:teuthology.orchestra.run.smithi187:> ip addr 2024-09-21T00:27:42.751 INFO:teuthology.orchestra.run.smithi187.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T00:27:42.751 INFO:teuthology.orchestra.run.smithi187.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T00:27:42.751 INFO:teuthology.orchestra.run.smithi187.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T00:27:42.751 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:27:42.752 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 ::1/128 scope host 2024-09-21T00:27:42.752 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:27:42.752 INFO:teuthology.orchestra.run.smithi187.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T00:27:42.752 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 0c:c4:7a:d9:36:78 brd ff:ff:ff:ff:ff:ff 2024-09-21T00:27:42.752 INFO:teuthology.orchestra.run.smithi187.stdout: altname enp7s0f0 2024-09-21T00:27:42.752 INFO:teuthology.orchestra.run.smithi187.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T00:27:42.752 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 0c:c4:7a:d9:36:79 brd ff:ff:ff:ff:ff:ff 2024-09-21T00:27:42.752 INFO:teuthology.orchestra.run.smithi187.stdout: altname enp7s0f1 2024-09-21T00:27:42.752 INFO:teuthology.orchestra.run.smithi187.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T00:27:42.752 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 0c:c4:7a:88:77:b0 brd ff:ff:ff:ff:ff:ff 2024-09-21T00:27:42.752 INFO:teuthology.orchestra.run.smithi187.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T00:27:42.752 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 0c:c4:7a:88:77:b1 brd ff:ff:ff:ff:ff:ff 2024-09-21T00:27:42.752 INFO:teuthology.orchestra.run.smithi187.stdout: inet 172.21.15.187/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-21T00:27:42.753 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft 42648sec preferred_lft 42648sec 2024-09-21T00:27:42.753 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 fe80::ec4:7aff:fe88:77b1/64 scope link noprefixroute 2024-09-21T00:27:42.753 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:27:42.753 INFO:teuthology.orchestra.run.smithi187.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T00:27:42.753 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 86:3b:f6:5b:1b:32 brd ff:ff:ff:ff:ff:ff 2024-09-21T00:27:42.753 INFO:teuthology.orchestra.run.smithi187.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T00:27:42.753 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:27:42.753 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 fe80::ac6f:97ff:fe09:b82e/64 scope link 2024-09-21T00:27:42.753 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:27:42.753 INFO:teuthology.orchestra.run.smithi187.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T00:27:42.753 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether ca:0b:a0:f5:2b:b5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T00:27:42.753 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 fe80::c80b:a0ff:fef5:2bb5/64 scope link 2024-09-21T00:27:42.753 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:27:42.753 INFO:teuthology.orchestra.run.smithi187.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T00:27:42.754 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 86:3b:f6:5b:1b:32 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-21T00:27:42.754 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 fe80::843b:f6ff:fe5b:1b32/64 scope link 2024-09-21T00:27:42.754 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:27:42.754 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:27:42.754 DEBUG:teuthology.orchestra.run.smithi187:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T00:27:42.820 INFO:teuthology.orchestra.run.smithi187.stdout:1 2024-09-21T00:27:42.822 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:27:42.822 DEBUG:teuthology.orchestra.run.smithi187:> route 2024-09-21T00:27:42.880 INFO:teuthology.orchestra.run.smithi187.stdout:Kernel IP routing table 2024-09-21T00:27:42.880 INFO:teuthology.orchestra.run.smithi187.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T00:27:42.881 INFO:teuthology.orchestra.run.smithi187.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-21T00:27:42.881 INFO:teuthology.orchestra.run.smithi187.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-21T00:27:42.882 INFO:teuthology.orchestra.run.smithi187.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T00:27:42.883 DEBUG:teuthology.orchestra.run.smithi187:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T00:27:42.883 DEBUG:teuthology.orchestra.run.smithi187:> set -e 2024-09-21T00:27:42.883 DEBUG:teuthology.orchestra.run.smithi187:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-21T00:27:42.883 DEBUG:teuthology.orchestra.run.smithi187:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-21T00:27:42.883 DEBUG:teuthology.orchestra.run.smithi187:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-21T00:27:42.883 DEBUG:teuthology.orchestra.run.smithi187:> ') 2024-09-21T00:27:42.973 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:27:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T00:27:43.007 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:27:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T00:27:43.013 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:27:43.013 DEBUG:teuthology.orchestra.run.smithi187:> ip netns list 2024-09-21T00:27:43.029 INFO:teuthology.orchestra.run.smithi187.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-21T00:27:43.029 INFO:teuthology.orchestra.run.smithi187.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-21T00:27:43.029 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-21T00:27:43.029 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:27:43.029 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip link set brx.0 up 2024-09-21T00:27:43.094 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T00:27:43.094 DEBUG:teuthology.orchestra.run.smithi187:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-21T00:27:43.149 INFO:teuthology.orchestra.run.smithi187.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-21T00:27:43.149 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T00:27:43.149 DEBUG:teuthology.orchestra.run.smithi187:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-21T00:27:43.204 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-21T00:27:43.204 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:27:43.204 DEBUG:teuthology.orchestra.run.smithi187:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-21T00:27:43.298 INFO:teuthology.orchestra.run.smithi187.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-21T00:27:43.298 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-21T00:27:43.298 INFO:teuthology.orchestra.run.smithi187.stdout:invalid new device string format 2024-09-21T00:27:43.308 INFO:teuthology.orchestra.run.smithi187.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-21T00:27:43.308 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-21T00:27:43.309 INFO:teuthology.orchestra.run.smithi187.stdout:invalid new device string format 2024-09-21T00:27:43.309 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: resolved to: "172.21.15.112:3300,172.21.15.187:3300,172.21.15.187:3301" 2024-09-21T00:27:43.309 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T00:27:43.320 INFO:teuthology.orchestra.run.smithi187.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-21T00:27:43.320 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-21T00:27:43.320 INFO:teuthology.orchestra.run.smithi187.stdout:invalid new device string format 2024-09-21T00:27:43.320 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: resolved to: "172.21.15.112:3300,172.21.15.187:3300,172.21.15.187:3301" 2024-09-21T00:27:43.320 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T00:27:43.320 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: resolved to: "172.21.15.112:6789,172.21.15.187:6789,172.21.15.187:6790" 2024-09-21T00:27:43.320 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.112:6789,172.21.15.187:6789,172.21.15.187:6790:/ 2024-09-21T00:27:43.320 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=8b34692b-dc65-49b2-a08f-26f707b96ac5" will pass to kernel 2024-09-21T00:27:43.321 INFO:teuthology.orchestra.run.smithi187.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-09-21T00:27:43.321 INFO:teuthology.orchestra.run.smithi187.stdout: You just mounted a file system that supports labels which does not 2024-09-21T00:27:43.321 INFO:teuthology.orchestra.run.smithi187.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T00:27:43.321 INFO:teuthology.orchestra.run.smithi187.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T00:27:43.321 INFO:teuthology.orchestra.run.smithi187.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T00:27:43.322 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T00:27:43.323 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:27:43.323 DEBUG:teuthology.orchestra.run.smithi187:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-21T00:27:43.347 DEBUG:teuthology.orchestra.run.smithi187:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T00:27:43.437 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:27:43 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T00:27:43.438 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:27:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T00:27:43.444 INFO:teuthology.orchestra.run.smithi187.stdout:client5898 2024-09-21T00:27:43.444 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-09-21T00:27:43.444 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:27:43.460 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_session_blocklist_on_evict False 2024-09-21T00:27:43.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:43.578+0000 7fd34829b640 1 -- 172.21.15.112:0/914980058 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd32c004d40 msgr2=0x7fd32c005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:43.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:43.578+0000 7fd34829b640 1 --2- 172.21.15.112:0/914980058 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd32c004d40 0x7fd32c005190 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fd338009ed0 tx=0x7fd32c005830 comp rx=0 tx=0).stop 2024-09-21T00:27:43.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:43.578+0000 7fd346838640 1 -- 172.21.15.112:0/914980058 reap_dead start 2024-09-21T00:27:43.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:43.578+0000 7fd34829b640 1 -- 172.21.15.112:0/914980058 shutdown_connections 2024-09-21T00:27:43.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:43.578+0000 7fd34829b640 1 -- 172.21.15.112:0/914980058 >> 172.21.15.112:0/914980058 conn(0x7fd34009d830 msgr2=0x7fd3400b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:43.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:43.579+0000 7fd34829b640 1 -- 172.21.15.112:0/914980058 shutdown_connections 2024-09-21T00:27:43.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:43.579+0000 7fd34829b640 1 -- 172.21.15.112:0/914980058 wait complete. 2024-09-21T00:27:43.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:43.580+0000 7fd34829b640 1 Processor -- start 2024-09-21T00:27:43.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:43.580+0000 7fd34829b640 1 -- start start 2024-09-21T00:27:43.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:43.582+0000 7fd34829b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd32c004d40 0x7fd34006e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:43.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:43.582+0000 7fd34829b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd340156770 0x7fd34014e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:43.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:43.582+0000 7fd34829b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd3401f54e0 0x7fd3401f58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:43.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:43.582+0000 7fd34829b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd3400cf330 con 0x7fd340156770 2024-09-21T00:27:43.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:43.582+0000 7fd34829b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd3400cf1b0 con 0x7fd32c004d40 2024-09-21T00:27:43.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:43.582+0000 7fd34829b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd3400b6580 con 0x7fd3401f54e0 2024-09-21T00:27:43.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:43.582+0000 7fd345836640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd340156770 0x7fd34014e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:43.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:43.582+0000 7fd345836640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd340156770 0x7fd34014e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48422/0 (socket says 172.21.15.112:48422) 2024-09-21T00:27:43.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:43.582+0000 7fd345836640 1 -- 172.21.15.112:0/4184684732 learned_addr learned my addr 172.21.15.112:0/4184684732 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:27:43.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:43.582+0000 7fd346037640 1 --2- 172.21.15.112:0/4184684732 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd32c004d40 0x7fd34006e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:43.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:43.582+0000 7fd346838640 1 --2- 172.21.15.112:0/4184684732 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd3401f54e0 0x7fd3401f58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:43.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:43.582+0000 7fd345836640 1 -- 172.21.15.112:0/4184684732 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd3401f54e0 msgr2=0x7fd3401f58f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:43.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:43.582+0000 7fd345836640 1 --2- 172.21.15.112:0/4184684732 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd3401f54e0 0x7fd3401f58f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:43.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:43.582+0000 7fd345836640 1 -- 172.21.15.112:0/4184684732 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd32c004d40 msgr2=0x7fd34006e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:43.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:43.582+0000 7fd345836640 1 --2- 172.21.15.112:0/4184684732 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd32c004d40 0x7fd34006e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:43.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:43.582+0000 7fd345836640 1 -- 172.21.15.112:0/4184684732 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3401f5e30 con 0x7fd340156770 2024-09-21T00:27:43.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:43.582+0000 7fd345836640 1 --2- 172.21.15.112:0/4184684732 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd340156770 0x7fd34014e7c0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fd330006340 tx=0x7fd3300039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:43.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:43.582+0000 7fd336ffd640 1 -- 172.21.15.112:0/4184684732 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd330015070 con 0x7fd340156770 2024-09-21T00:27:43.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:43.582+0000 7fd336ffd640 1 -- 172.21.15.112:0/4184684732 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd330006510 con 0x7fd340156770 2024-09-21T00:27:43.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:43.582+0000 7fd34829b640 1 -- 172.21.15.112:0/4184684732 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd34014ed00 con 0x7fd340156770 2024-09-21T00:27:43.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:43.582+0000 7fd336ffd640 1 -- 172.21.15.112:0/4184684732 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd330011b00 con 0x7fd340156770 2024-09-21T00:27:43.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:43.582+0000 7fd34829b640 1 -- 172.21.15.112:0/4184684732 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd34006d6a0 con 0x7fd340156770 2024-09-21T00:27:43.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:43.583+0000 7fd34829b640 1 -- 172.21.15.112:0/4184684732 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3400b6ae0 con 0x7fd340156770 2024-09-21T00:27:43.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:43.584+0000 7fd336ffd640 1 -- 172.21.15.112:0/4184684732 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd330018070 con 0x7fd340156770 2024-09-21T00:27:43.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:43.584+0000 7fd336ffd640 1 --2- 172.21.15.112:0/4184684732 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fd324078bc0 0x7fd32407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:43.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:43.585+0000 7fd336ffd640 1 -- 172.21.15.112:0/4184684732 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7fd33009b9e0 con 0x7fd340156770 2024-09-21T00:27:43.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:43.585+0000 7fd346037640 1 --2- 172.21.15.112:0/4184684732 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fd324078bc0 0x7fd32407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:43.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:43.585+0000 7fd346037640 1 --2- 172.21.15.112:0/4184684732 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fd324078bc0 0x7fd32407b0d0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fd33c0065f0 tx=0x7fd33c001110 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:43.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:43.589+0000 7fd336ffd640 1 -- 172.21.15.112:0/4184684732 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd3401592d0 con 0x7fd340156770 2024-09-21T00:27:43.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:43.766+0000 7fd34829b640 1 -- 172.21.15.112:0/4184684732 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command([{prefix=config set, name=mds_session_blocklist_on_evict}] v 0) -- 0x7fd34006d150 con 0x7fd340156770 2024-09-21T00:27:43.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:43.769+0000 7fd336ffd640 1 -- 172.21.15.112:0/4184684732 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_session_blocklist_on_evict}]=0 v38)=0 v38) ==== 132+0+0 (secure 0 0 0) 0x7fd330063650 con 0x7fd340156770 2024-09-21T00:27:43.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:43.772+0000 7fd34829b640 1 -- 172.21.15.112:0/4184684732 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fd324078bc0 msgr2=0x7fd32407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:43.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:43.772+0000 7fd34829b640 1 --2- 172.21.15.112:0/4184684732 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fd324078bc0 0x7fd32407b0d0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fd33c0065f0 tx=0x7fd33c001110 comp rx=0 tx=0).stop 2024-09-21T00:27:43.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:43.772+0000 7fd34829b640 1 -- 172.21.15.112:0/4184684732 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd340156770 msgr2=0x7fd34014e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:43.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:43.772+0000 7fd34829b640 1 --2- 172.21.15.112:0/4184684732 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd340156770 0x7fd34014e7c0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fd330006340 tx=0x7fd3300039c0 comp rx=0 tx=0).stop 2024-09-21T00:27:43.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:43.772+0000 7fd34829b640 1 -- 172.21.15.112:0/4184684732 shutdown_connections 2024-09-21T00:27:43.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:43.772+0000 7fd34829b640 1 --2- 172.21.15.112:0/4184684732 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd3401f54e0 0x7fd3401f58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:43.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:43.772+0000 7fd34829b640 1 --2- 172.21.15.112:0/4184684732 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fd324078bc0 0x7fd32407b0d0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:43.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:43.772+0000 7fd34829b640 1 --2- 172.21.15.112:0/4184684732 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd340156770 0x7fd34014e7c0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:43.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:43.772+0000 7fd34829b640 1 --2- 172.21.15.112:0/4184684732 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd32c004d40 0x7fd34006e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:43.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:43.772+0000 7fd34829b640 1 -- 172.21.15.112:0/4184684732 >> 172.21.15.112:0/4184684732 conn(0x7fd34009d830 msgr2=0x7fd3400b5120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:43.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:43.772+0000 7fd34829b640 1 -- 172.21.15.112:0/4184684732 shutdown_connections 2024-09-21T00:27:43.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:43.772+0000 7fd34829b640 1 -- 172.21.15.112:0/4184684732 wait complete. 2024-09-21T00:27:43.785 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 2 2024-09-21T00:27:43.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:43.898+0000 7fef4efad640 1 -- 172.21.15.112:0/6422922 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fef480b9ad0 msgr2=0x7fef480d0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:43.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:43.898+0000 7fef4efad640 1 --2- 172.21.15.112:0/6422922 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fef480b9ad0 0x7fef480d0be0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fef380098e0 tx=0x7fef3802a530 comp rx=0 tx=0).stop 2024-09-21T00:27:43.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:43.899+0000 7fef4efad640 1 -- 172.21.15.112:0/6422922 shutdown_connections 2024-09-21T00:27:43.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:43.899+0000 7fef4efad640 1 --2- 172.21.15.112:0/6422922 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fef480b9ad0 0x7fef480d0be0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:43.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:43.899+0000 7fef4efad640 1 --2- 172.21.15.112:0/6422922 >> v2:172.21.15.187:3300/0 conn(0x7fef4815ee80 0x7fef480d0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:43.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:43.899+0000 7fef4efad640 1 -- 172.21.15.112:0/6422922 >> 172.21.15.112:0/6422922 conn(0x7fef4809d830 msgr2=0x7fef480b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:43.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:43.899+0000 7fef4efad640 1 -- 172.21.15.112:0/6422922 shutdown_connections 2024-09-21T00:27:43.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:43.899+0000 7fef4efad640 1 -- 172.21.15.112:0/6422922 wait complete. 2024-09-21T00:27:43.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:43.900+0000 7fef4efad640 1 Processor -- start 2024-09-21T00:27:43.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:43.900+0000 7fef4efad640 1 -- start start 2024-09-21T00:27:43.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:43.902+0000 7fef4efad640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fef480b9ad0 0x7fef4806f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:43.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:43.902+0000 7fef4efad640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef480d1120 0x7fef4814f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:43.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:43.902+0000 7fef4efad640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fef4815ee80 0x7fef480b2440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:43.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:43.902+0000 7fef4efad640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fef480b6700 con 0x7fef480d1120 2024-09-21T00:27:43.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:43.902+0000 7fef4efad640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fef480b6580 con 0x7fef480b9ad0 2024-09-21T00:27:43.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:43.902+0000 7fef4efad640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fef480b6880 con 0x7fef4815ee80 2024-09-21T00:27:43.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:43.902+0000 7fef3ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef480d1120 0x7fef4814f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:43.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:43.902+0000 7fef3ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef480d1120 0x7fef4814f990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48430/0 (socket says 172.21.15.112:48430) 2024-09-21T00:27:43.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:43.902+0000 7fef3ffff640 1 -- 172.21.15.112:0/626288365 learned_addr learned my addr 172.21.15.112:0/626288365 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:27:43.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:43.902+0000 7fef4cd49640 1 --2- 172.21.15.112:0/626288365 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fef480b9ad0 0x7fef4806f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:43.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:43.902+0000 7fef4d54a640 1 --2- 172.21.15.112:0/626288365 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fef4815ee80 0x7fef480b2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:43.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:43.902+0000 7fef3ffff640 1 -- 172.21.15.112:0/626288365 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fef4815ee80 msgr2=0x7fef480b2440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:43.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:43.902+0000 7fef3ffff640 1 --2- 172.21.15.112:0/626288365 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fef4815ee80 0x7fef480b2440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:43.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:43.902+0000 7fef3ffff640 1 -- 172.21.15.112:0/626288365 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fef480b9ad0 msgr2=0x7fef4806f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:43.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:43.902+0000 7fef3ffff640 1 --2- 172.21.15.112:0/626288365 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fef480b9ad0 0x7fef4806f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:43.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:43.902+0000 7fef3ffff640 1 -- 172.21.15.112:0/626288365 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef4806db60 con 0x7fef480d1120 2024-09-21T00:27:43.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:43.903+0000 7fef3ffff640 1 --2- 172.21.15.112:0/626288365 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef480d1120 0x7fef4814f990 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fef3802ab20 tx=0x7fef380052f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:43.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:43.903+0000 7fef3d7fa640 1 -- 172.21.15.112:0/626288365 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef3803f070 con 0x7fef480d1120 2024-09-21T00:27:43.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:43.903+0000 7fef4efad640 1 -- 172.21.15.112:0/626288365 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef4806e840 con 0x7fef480d1120 2024-09-21T00:27:43.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:43.903+0000 7fef3d7fa640 1 -- 172.21.15.112:0/626288365 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fef38004250 con 0x7fef480d1120 2024-09-21T00:27:43.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:43.903+0000 7fef3d7fa640 1 -- 172.21.15.112:0/626288365 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef38005cf0 con 0x7fef480d1120 2024-09-21T00:27:43.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:43.903+0000 7fef4efad640 1 -- 172.21.15.112:0/626288365 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef480cca00 con 0x7fef480d1120 2024-09-21T00:27:43.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:43.904+0000 7fef4efad640 1 -- 172.21.15.112:0/626288365 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef480b6d30 con 0x7fef480d1120 2024-09-21T00:27:43.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:43.905+0000 7fef3d7fa640 1 -- 172.21.15.112:0/626288365 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fef38042070 con 0x7fef480d1120 2024-09-21T00:27:43.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:43.905+0000 7fef3d7fa640 1 --2- 172.21.15.112:0/626288365 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fef34078b90 0x7fef3407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:43.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:43.905+0000 7fef3d7fa640 1 -- 172.21.15.112:0/626288365 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7fef380bbca0 con 0x7fef480d1120 2024-09-21T00:27:43.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:43.905+0000 7fef4cd49640 1 --2- 172.21.15.112:0/626288365 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fef34078b90 0x7fef3407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:43.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:43.906+0000 7fef4cd49640 1 --2- 172.21.15.112:0/626288365 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fef34078b90 0x7fef3407b0a0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fef440095f0 tx=0x7fef44005f90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:43.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:43.910+0000 7fef3d7fa640 1 -- 172.21.15.112:0/626288365 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fef38083910 con 0x7fef480d1120 2024-09-21T00:27:44.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:44.126+0000 7fef4efad640 1 -- 172.21.15.112:0/626288365 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"} v 0) -- 0x7fef48158f90 con 0x7fef480d1120 2024-09-21T00:27:44.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:44.773+0000 7fef3d7fa640 1 -- 172.21.15.112:0/626288365 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]=0 v36) ==== 105+0+0 (secure 0 0 0) 0x7fef38088340 con 0x7fef480d1120 2024-09-21T00:27:44.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:44.776+0000 7fef4efad640 1 -- 172.21.15.112:0/626288365 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fef34078b90 msgr2=0x7fef3407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:44.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:44.776+0000 7fef4efad640 1 --2- 172.21.15.112:0/626288365 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fef34078b90 0x7fef3407b0a0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fef440095f0 tx=0x7fef44005f90 comp rx=0 tx=0).stop 2024-09-21T00:27:44.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:44.776+0000 7fef4efad640 1 -- 172.21.15.112:0/626288365 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef480d1120 msgr2=0x7fef4814f990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:44.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:44.776+0000 7fef4efad640 1 --2- 172.21.15.112:0/626288365 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef480d1120 0x7fef4814f990 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fef3802ab20 tx=0x7fef380052f0 comp rx=0 tx=0).stop 2024-09-21T00:27:44.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:44.776+0000 7fef4efad640 1 -- 172.21.15.112:0/626288365 shutdown_connections 2024-09-21T00:27:44.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:44.776+0000 7fef4efad640 1 --2- 172.21.15.112:0/626288365 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fef4815ee80 0x7fef480b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:44.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:44.776+0000 7fef4efad640 1 --2- 172.21.15.112:0/626288365 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fef34078b90 0x7fef3407b0a0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:44.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:44.776+0000 7fef4efad640 1 --2- 172.21.15.112:0/626288365 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef480d1120 0x7fef4814f990 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:44.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:44.776+0000 7fef4efad640 1 --2- 172.21.15.112:0/626288365 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fef480b9ad0 0x7fef4806f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:44.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:44.776+0000 7fef4efad640 1 -- 172.21.15.112:0/626288365 >> 172.21.15.112:0/626288365 conn(0x7fef4809d830 msgr2=0x7fef480cae40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:44.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:44.776+0000 7fef4efad640 1 -- 172.21.15.112:0/626288365 shutdown_connections 2024-09-21T00:27:44.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:44.776+0000 7fef4efad640 1 -- 172.21.15.112:0/626288365 wait complete. 2024-09-21T00:27:44.795 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T00:27:44.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:44.908+0000 7f1ae8870640 1 -- 172.21.15.112:0/4218058230 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ae415b5f0 msgr2=0x7f1ae406ff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:44.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:44.908+0000 7f1ae8870640 1 --2- 172.21.15.112:0/4218058230 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ae415b5f0 0x7f1ae406ff00 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f1ad4005fd0 tx=0x7f1ad402a540 comp rx=0 tx=0).stop 2024-09-21T00:27:44.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:44.908+0000 7f1ae8870640 1 -- 172.21.15.112:0/4218058230 shutdown_connections 2024-09-21T00:27:44.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:44.908+0000 7f1ae8870640 1 --2- 172.21.15.112:0/4218058230 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1ae40745b0 0x7f1ae406e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:44.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:44.908+0000 7f1ae8870640 1 --2- 172.21.15.112:0/4218058230 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ae415b5f0 0x7f1ae406ff00 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:44.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:44.908+0000 7f1ae8870640 1 -- 172.21.15.112:0/4218058230 >> 172.21.15.112:0/4218058230 conn(0x7f1ae40d1140 msgr2=0x7f1ae40d1550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:44.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:44.908+0000 7f1ae8870640 1 -- 172.21.15.112:0/4218058230 shutdown_connections 2024-09-21T00:27:44.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:44.908+0000 7f1ae8870640 1 -- 172.21.15.112:0/4218058230 wait complete. 2024-09-21T00:27:44.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:44.910+0000 7f1ae8870640 1 Processor -- start 2024-09-21T00:27:44.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:44.910+0000 7f1ae8870640 1 -- start start 2024-09-21T00:27:44.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:44.911+0000 7f1ae8870640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1ae415b5f0 0x7f1ae40b6c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:44.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:44.911+0000 7f1ae8870640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1ae415efb0 0x7f1ae4153120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:44.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:44.911+0000 7f1ae8870640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ae40c6c60 0x7f1ae40c7070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:44.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:44.911+0000 7f1ae8870640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae406ee10 con 0x7f1ae40c6c60 2024-09-21T00:27:44.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:44.911+0000 7f1ae8870640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae406ec90 con 0x7f1ae415efb0 2024-09-21T00:27:44.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:44.911+0000 7f1ae8870640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f1ae406ef90 con 0x7f1ae415b5f0 2024-09-21T00:27:44.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:44.911+0000 7f1ae2d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ae40c6c60 0x7f1ae40c7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:44.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:44.911+0000 7f1ae2d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ae40c6c60 0x7f1ae40c7070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48458/0 (socket says 172.21.15.112:48458) 2024-09-21T00:27:44.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:44.911+0000 7f1ae2d9d640 1 -- 172.21.15.112:0/3398353654 learned_addr learned my addr 172.21.15.112:0/3398353654 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:27:44.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:44.911+0000 7f1ae259c640 1 --2- 172.21.15.112:0/3398353654 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1ae415b5f0 0x7f1ae40b6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:44.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:44.911+0000 7f1ae1d9b640 1 --2- 172.21.15.112:0/3398353654 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1ae415efb0 0x7f1ae4153120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:44.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:44.911+0000 7f1ae2d9d640 1 -- 172.21.15.112:0/3398353654 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1ae415b5f0 msgr2=0x7f1ae40b6c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:44.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:44.911+0000 7f1ae2d9d640 1 --2- 172.21.15.112:0/3398353654 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1ae415b5f0 0x7f1ae40b6c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:44.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:44.911+0000 7f1ae2d9d640 1 -- 172.21.15.112:0/3398353654 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1ae415efb0 msgr2=0x7f1ae4153120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:44.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:44.911+0000 7f1ae2d9d640 1 --2- 172.21.15.112:0/3398353654 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1ae415efb0 0x7f1ae4153120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:44.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:44.911+0000 7f1ae2d9d640 1 -- 172.21.15.112:0/3398353654 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ae40c75b0 con 0x7f1ae40c6c60 2024-09-21T00:27:44.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:44.912+0000 7f1ae2d9d640 1 --2- 172.21.15.112:0/3398353654 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ae40c6c60 0x7f1ae40c7070 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f1adc00e9c0 tx=0x7f1adc00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:44.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:44.912+0000 7f1ad2ffd640 1 -- 172.21.15.112:0/3398353654 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1adc004d20 con 0x7f1ae40c6c60 2024-09-21T00:27:44.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:44.912+0000 7f1ad2ffd640 1 -- 172.21.15.112:0/3398353654 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1adc00cac0 con 0x7f1ae40c6c60 2024-09-21T00:27:44.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:44.912+0000 7f1ae8870640 1 -- 172.21.15.112:0/3398353654 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ae415bb50 con 0x7f1ae40c6c60 2024-09-21T00:27:44.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:44.912+0000 7f1ad2ffd640 1 -- 172.21.15.112:0/3398353654 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1adc005020 con 0x7f1ae40c6c60 2024-09-21T00:27:44.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:44.912+0000 7f1ae8870640 1 -- 172.21.15.112:0/3398353654 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ae415be30 con 0x7f1ae40c6c60 2024-09-21T00:27:44.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:44.913+0000 7f1ae8870640 1 -- 172.21.15.112:0/3398353654 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ae4074f20 con 0x7f1ae40c6c60 2024-09-21T00:27:44.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:44.914+0000 7f1ad2ffd640 1 -- 172.21.15.112:0/3398353654 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1adc0051c0 con 0x7f1ae40c6c60 2024-09-21T00:27:44.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:44.914+0000 7f1ad2ffd640 1 --2- 172.21.15.112:0/3398353654 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1ac4078b90 0x7f1ac407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:44.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:44.914+0000 7f1ad2ffd640 1 -- 172.21.15.112:0/3398353654 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7f1adc09b7e0 con 0x7f1ae40c6c60 2024-09-21T00:27:44.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:44.914+0000 7f1ae259c640 1 --2- 172.21.15.112:0/3398353654 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1ac4078b90 0x7f1ac407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:44.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:44.915+0000 7f1ae259c640 1 --2- 172.21.15.112:0/3398353654 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1ac4078b90 0x7f1ac407b0a0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f1ad4033000 tx=0x7f1ad4004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:44.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:44.919+0000 7f1ad2ffd640 1 -- 172.21.15.112:0/3398353654 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1ae40c75b0 con 0x7f1ae40c6c60 2024-09-21T00:27:45.076 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T00:27:45.078 DEBUG:teuthology.orchestra.run.smithi187:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T00:27:45.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:45.135+0000 7f1ae8870640 1 -- 172.21.15.112:0/3398353654 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1ae406cff0 con 0x7f1ae40c6c60 2024-09-21T00:27:45.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:45.135+0000 7f1ad2ffd640 1 -- 172.21.15.112:0/3398353654 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+5249 (secure 0 0 0) 0x7f1adc016030 con 0x7f1ae40c6c60 2024-09-21T00:27:45.137 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:27:45.137 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":37,"btime":"2024-09-21T00:27:44:777481+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5389,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6836/3984762342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":3984762342},{"type":"v1","addr":"172.21.15.187:6836","nonce":3984762342}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5392,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6837/763698806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6835","nonce":763698806},{"type":"v1","addr":"172.21.15.187:6837","nonce":763698806}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T00:27:34.871721+0000","modified":"2024-09-21T00:27:44.777463+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5739,"mds_1":5736},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5736":{"gid":5736,"name":"c","rank":1,"incarnation":37,"state":"up:creating","state_seq":1,"addr":"172.21.15.112:6834/1432588455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1432588455},{"type":"v1","addr":"172.21.15.112:6834","nonce":1432588455}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5739":{"gid":5739,"name":"a","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6835/3610025869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":3610025869},{"type":"v1","addr":"172.21.15.112:6835","nonce":3610025869}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5739,"qdb_cluster":[5739]},"id":4}]} 2024-09-21T00:27:45.138 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 37 2024-09-21T00:27:45.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:45.138+0000 7f1ae8870640 1 -- 172.21.15.112:0/3398353654 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1ac4078b90 msgr2=0x7f1ac407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:45.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:45.138+0000 7f1ae8870640 1 --2- 172.21.15.112:0/3398353654 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1ac4078b90 0x7f1ac407b0a0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f1ad4033000 tx=0x7f1ad4004730 comp rx=0 tx=0).stop 2024-09-21T00:27:45.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:45.138+0000 7f1ae8870640 1 -- 172.21.15.112:0/3398353654 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ae40c6c60 msgr2=0x7f1ae40c7070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:45.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:45.138+0000 7f1ae8870640 1 --2- 172.21.15.112:0/3398353654 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ae40c6c60 0x7f1ae40c7070 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f1adc00e9c0 tx=0x7f1adc00ee80 comp rx=0 tx=0).stop 2024-09-21T00:27:45.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:45.138+0000 7f1ae8870640 1 -- 172.21.15.112:0/3398353654 shutdown_connections 2024-09-21T00:27:45.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:45.138+0000 7f1ae8870640 1 --2- 172.21.15.112:0/3398353654 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1ac4078b90 0x7f1ac407b0a0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:45.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:45.138+0000 7f1ae8870640 1 --2- 172.21.15.112:0/3398353654 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ae40c6c60 0x7f1ae40c7070 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:45.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:45.138+0000 7f1ae8870640 1 --2- 172.21.15.112:0/3398353654 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1ae415efb0 0x7f1ae4153120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:45.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:45.138+0000 7f1ae8870640 1 --2- 172.21.15.112:0/3398353654 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1ae415b5f0 0x7f1ae40b6c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:45.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:45.138+0000 7f1ae8870640 1 -- 172.21.15.112:0/3398353654 >> 172.21.15.112:0/3398353654 conn(0x7f1ae40d1140 msgr2=0x7f1ae415a820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:45.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:45.138+0000 7f1ae8870640 1 -- 172.21.15.112:0/3398353654 shutdown_connections 2024-09-21T00:27:45.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:45.138+0000 7f1ae8870640 1 -- 172.21.15.112:0/3398353654 wait complete. 2024-09-21T00:27:45.553 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 37, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T00:27:34.871721+0000', 'modified': '2024-09-21T00:27:44.777463+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 2, 'in': [0, 1], 'up': {'mds_0': 5739, 'mds_1': 5736}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5736': {'gid': 5736, 'name': 'c', 'rank': 1, 'incarnation': 37, 'state': 'up:creating', 'state_seq': 1, 'addr': '172.21.15.112:6834/1432588455', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6832', 'nonce': 1432588455}, {'type': 'v1', 'addr': '172.21.15.112:6834', 'nonce': 1432588455}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_5739': {'gid': 5739, 'name': 'a', 'rank': 0, 'incarnation': 31, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.112:6835/3610025869', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6833', 'nonce': 3610025869}, {'type': 'v1', 'addr': '172.21.15.112:6835', 'nonce': 3610025869}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5739, 'qdb_cluster': [5739]} 2024-09-21T00:27:45.554 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_5736:up:creating 2024-09-21T00:27:46.554 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T00:27:46.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:46.678+0000 7f1c19ef9640 1 -- 172.21.15.112:0/1944432951 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c140d19f0 msgr2=0x7f1c1415cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:46.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:46.678+0000 7f1c19ef9640 1 --2- 172.21.15.112:0/1944432951 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c140d19f0 0x7f1c1415cf20 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f1c08009830 tx=0x7f1c0802a510 comp rx=0 tx=0).stop 2024-09-21T00:27:46.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:46.678+0000 7f1c19ef9640 1 -- 172.21.15.112:0/1944432951 shutdown_connections 2024-09-21T00:27:46.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:46.678+0000 7f1c19ef9640 1 --2- 172.21.15.112:0/1944432951 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c140d19f0 0x7f1c1415cf20 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:46.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:46.678+0000 7f1c19ef9640 1 -- 172.21.15.112:0/1944432951 >> 172.21.15.112:0/1944432951 conn(0x7f1c140d2350 msgr2=0x7f1c140d2760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:46.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:46.678+0000 7f1c19ef9640 1 -- 172.21.15.112:0/1944432951 shutdown_connections 2024-09-21T00:27:46.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:46.678+0000 7f1c19ef9640 1 -- 172.21.15.112:0/1944432951 wait complete. 2024-09-21T00:27:46.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:46.680+0000 7f1c19ef9640 1 Processor -- start 2024-09-21T00:27:46.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:46.680+0000 7f1c19ef9640 1 -- start start 2024-09-21T00:27:46.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:46.681+0000 7f1c19ef9640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1c140b6250 0x7f1c1415b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:46.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:46.681+0000 7f1c19ef9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c140d19f0 0x7f1c1415bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:46.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:46.681+0000 7f1c19ef9640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1c1415cb10 0x7f1c141f8c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:46.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:46.681+0000 7f1c19ef9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1c140b7100 con 0x7f1c140d19f0 2024-09-21T00:27:46.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:46.681+0000 7f1c19ef9640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f1c140b6f80 con 0x7f1c1415cb10 2024-09-21T00:27:46.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:46.681+0000 7f1c19ef9640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f1c140b7280 con 0x7f1c140b6250 2024-09-21T00:27:46.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:46.682+0000 7f1c12ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c140d19f0 0x7f1c1415bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:46.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:46.682+0000 7f1c12ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c140d19f0 0x7f1c1415bb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57184/0 (socket says 172.21.15.112:57184) 2024-09-21T00:27:46.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:46.682+0000 7f1c12ffd640 1 -- 172.21.15.112:0/2491463910 learned_addr learned my addr 172.21.15.112:0/2491463910 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:27:46.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:46.682+0000 7f1c137fe640 1 --2- 172.21.15.112:0/2491463910 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1c140b6250 0x7f1c1415b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:46.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:46.682+0000 7f1c13fff640 1 --2- 172.21.15.112:0/2491463910 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1c1415cb10 0x7f1c141f8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:46.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:46.682+0000 7f1c12ffd640 1 -- 172.21.15.112:0/2491463910 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1c140b6250 msgr2=0x7f1c1415b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:46.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:46.682+0000 7f1c12ffd640 1 --2- 172.21.15.112:0/2491463910 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1c140b6250 0x7f1c1415b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:46.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:46.682+0000 7f1c12ffd640 1 -- 172.21.15.112:0/2491463910 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1c1415cb10 msgr2=0x7f1c141f8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:46.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:46.682+0000 7f1c12ffd640 1 --2- 172.21.15.112:0/2491463910 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1c1415cb10 0x7f1c141f8c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:46.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:46.682+0000 7f1c12ffd640 1 -- 172.21.15.112:0/2491463910 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c141f5660 con 0x7f1c140d19f0 2024-09-21T00:27:46.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:46.682+0000 7f1c12ffd640 1 --2- 172.21.15.112:0/2491463910 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c140d19f0 0x7f1c1415bb20 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f1c0802ab00 tx=0x7f1c08033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:46.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:46.682+0000 7f1bfbfff640 1 -- 172.21.15.112:0/2491463910 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c08031d60 con 0x7f1c140d19f0 2024-09-21T00:27:46.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:46.682+0000 7f1bfbfff640 1 -- 172.21.15.112:0/2491463910 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1c08033e10 con 0x7f1c140d19f0 2024-09-21T00:27:46.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:46.682+0000 7f1bfbfff640 1 -- 172.21.15.112:0/2491463910 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c08005280 con 0x7f1c140d19f0 2024-09-21T00:27:46.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:46.682+0000 7f1c19ef9640 1 -- 172.21.15.112:0/2491463910 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c141f5920 con 0x7f1c140d19f0 2024-09-21T00:27:46.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:46.683+0000 7f1c19ef9640 1 -- 172.21.15.112:0/2491463910 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c141f5ee0 con 0x7f1c140d19f0 2024-09-21T00:27:46.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:46.684+0000 7f1c19ef9640 1 -- 172.21.15.112:0/2491463910 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c1415cf20 con 0x7f1c140d19f0 2024-09-21T00:27:46.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:46.684+0000 7f1bfbfff640 1 -- 172.21.15.112:0/2491463910 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1c08005420 con 0x7f1c140d19f0 2024-09-21T00:27:46.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:46.685+0000 7f1bfbfff640 1 --2- 172.21.15.112:0/2491463910 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1c00078b90 0x7f1c0007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:46.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:46.685+0000 7f1bfbfff640 1 -- 172.21.15.112:0/2491463910 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7f1c080bc0d0 con 0x7f1c140d19f0 2024-09-21T00:27:46.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:46.685+0000 7f1c137fe640 1 --2- 172.21.15.112:0/2491463910 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1c00078b90 0x7f1c0007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:46.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:46.686+0000 7f1c137fe640 1 --2- 172.21.15.112:0/2491463910 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1c00078b90 0x7f1c0007b0a0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f1c04009f20 tx=0x7f1c040094d0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:46.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:46.689+0000 7f1bfbfff640 1 -- 172.21.15.112:0/2491463910 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1c0803f030 con 0x7f1c140d19f0 2024-09-21T00:27:46.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:46.906+0000 7f1c19ef9640 1 -- 172.21.15.112:0/2491463910 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1c140b56b0 con 0x7f1c140d19f0 2024-09-21T00:27:46.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:46.906+0000 7f1bfbfff640 1 -- 172.21.15.112:0/2491463910 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+5252 (secure 0 0 0) 0x7f1c080844a0 con 0x7f1c140d19f0 2024-09-21T00:27:46.908 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:27:46.908 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":38,"btime":"2024-09-21T00:27:45:778934+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5389,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6836/3984762342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":3984762342},{"type":"v1","addr":"172.21.15.187:6836","nonce":3984762342}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5392,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6837/763698806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6835","nonce":763698806},{"type":"v1","addr":"172.21.15.187:6837","nonce":763698806}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T00:27:34.871721+0000","modified":"2024-09-21T00:27:45.778927+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5739,"mds_1":5736},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5736":{"gid":5736,"name":"c","rank":1,"incarnation":37,"state":"up:active","state_seq":5,"addr":"172.21.15.112:6834/1432588455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1432588455},{"type":"v1","addr":"172.21.15.112:6834","nonce":1432588455}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5739":{"gid":5739,"name":"a","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6835/3610025869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":3610025869},{"type":"v1","addr":"172.21.15.112:6835","nonce":3610025869}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5739,"qdb_cluster":[5736,5739]},"id":4}]} 2024-09-21T00:27:46.908 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 38 2024-09-21T00:27:46.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:46.908+0000 7f1c19ef9640 1 -- 172.21.15.112:0/2491463910 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1c00078b90 msgr2=0x7f1c0007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:46.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:46.908+0000 7f1c19ef9640 1 --2- 172.21.15.112:0/2491463910 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1c00078b90 0x7f1c0007b0a0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f1c04009f20 tx=0x7f1c040094d0 comp rx=0 tx=0).stop 2024-09-21T00:27:46.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:46.908+0000 7f1c19ef9640 1 -- 172.21.15.112:0/2491463910 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c140d19f0 msgr2=0x7f1c1415bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:46.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:46.908+0000 7f1c19ef9640 1 --2- 172.21.15.112:0/2491463910 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c140d19f0 0x7f1c1415bb20 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f1c0802ab00 tx=0x7f1c08033c20 comp rx=0 tx=0).stop 2024-09-21T00:27:46.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:46.908+0000 7f1c19ef9640 1 -- 172.21.15.112:0/2491463910 shutdown_connections 2024-09-21T00:27:46.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:46.908+0000 7f1c19ef9640 1 --2- 172.21.15.112:0/2491463910 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1c1415cb10 0x7f1c141f8c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:46.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:46.908+0000 7f1c19ef9640 1 --2- 172.21.15.112:0/2491463910 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1c00078b90 0x7f1c0007b0a0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:46.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:46.908+0000 7f1c19ef9640 1 --2- 172.21.15.112:0/2491463910 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c140d19f0 0x7f1c1415bb20 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:46.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:46.908+0000 7f1c19ef9640 1 --2- 172.21.15.112:0/2491463910 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1c140b6250 0x7f1c1415b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:46.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:46.908+0000 7f1c19ef9640 1 -- 172.21.15.112:0/2491463910 >> 172.21.15.112:0/2491463910 conn(0x7f1c140d2350 msgr2=0x7f1c14160090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:46.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:46.909+0000 7f1c19ef9640 1 -- 172.21.15.112:0/2491463910 shutdown_connections 2024-09-21T00:27:46.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:46.909+0000 7f1c19ef9640 1 -- 172.21.15.112:0/2491463910 wait complete. 2024-09-21T00:27:47.324 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 38, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T00:27:34.871721+0000', 'modified': '2024-09-21T00:27:45.778927+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 2, 'in': [0, 1], 'up': {'mds_0': 5739, 'mds_1': 5736}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5736': {'gid': 5736, 'name': 'c', 'rank': 1, 'incarnation': 37, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.112:6834/1432588455', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6832', 'nonce': 1432588455}, {'type': 'v1', 'addr': '172.21.15.112:6834', 'nonce': 1432588455}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_5739': {'gid': 5739, 'name': 'a', 'rank': 0, 'incarnation': 31, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.112:6835/3610025869', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6833', 'nonce': 3610025869}, {'type': 'v1', 'addr': '172.21.15.112:6835', 'nonce': 3610025869}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5739, 'qdb_cluster': [5736, 5739]} 2024-09-21T00:27:47.325 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 2/2 2024-09-21T00:27:47.325 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-21T00:27:47.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.437+0000 7f61f4ef9640 1 -- 172.21.15.112:0/664210151 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f61f015ee80 msgr2=0x7f61f00d0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:47.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.437+0000 7f61f4ef9640 1 --2- 172.21.15.112:0/664210151 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f61f015ee80 0x7f61f00d0170 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f61e8005fd0 tx=0x7f61e802a820 comp rx=0 tx=0).stop 2024-09-21T00:27:47.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.437+0000 7f61f4ef9640 1 -- 172.21.15.112:0/664210151 shutdown_connections 2024-09-21T00:27:47.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.437+0000 7f61f4ef9640 1 --2- 172.21.15.112:0/664210151 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f61f00ceb90 0x7f61f00cefa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:47.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.437+0000 7f61f4ef9640 1 --2- 172.21.15.112:0/664210151 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f61f015ee80 0x7f61f00d0170 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:47.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.437+0000 7f61f4ef9640 1 -- 172.21.15.112:0/664210151 >> 172.21.15.112:0/664210151 conn(0x7f61f009d830 msgr2=0x7f61f00b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:47.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.437+0000 7f61f4ef9640 1 -- 172.21.15.112:0/664210151 shutdown_connections 2024-09-21T00:27:47.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.437+0000 7f61f4ef9640 1 -- 172.21.15.112:0/664210151 wait complete. 2024-09-21T00:27:47.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.439+0000 7f61f4ef9640 1 Processor -- start 2024-09-21T00:27:47.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.439+0000 7f61f4ef9640 1 -- start start 2024-09-21T00:27:47.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.440+0000 7f61f4ef9640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f61f00b9ad0 0x7f61f0158310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:47.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.440+0000 7f61f4ef9640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f61f00ceb90 0x7f61f006ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:47.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.440+0000 7f61f4ef9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f61f015ee80 0x7f61f00b2760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:47.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.440+0000 7f61f4ef9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f61f00b6700 con 0x7f61f015ee80 2024-09-21T00:27:47.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.440+0000 7f61f4ef9640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f61f00b6580 con 0x7f61f00ceb90 2024-09-21T00:27:47.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.440+0000 7f61f4ef9640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f61f00b6880 con 0x7f61f00b9ad0 2024-09-21T00:27:47.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.440+0000 7f61eed9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f61f015ee80 0x7f61f00b2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:47.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.440+0000 7f61eed9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f61f015ee80 0x7f61f00b2760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57198/0 (socket says 172.21.15.112:57198) 2024-09-21T00:27:47.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.440+0000 7f61eed9d640 1 -- 172.21.15.112:0/1612005258 learned_addr learned my addr 172.21.15.112:0/1612005258 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:27:47.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.440+0000 7f61ee59c640 1 --2- 172.21.15.112:0/1612005258 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f61f00b9ad0 0x7f61f0158310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:47.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.440+0000 7f61edd9b640 1 --2- 172.21.15.112:0/1612005258 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f61f00ceb90 0x7f61f006ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:47.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.441+0000 7f61eed9d640 1 -- 172.21.15.112:0/1612005258 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f61f00b9ad0 msgr2=0x7f61f0158310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:47.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.441+0000 7f61eed9d640 1 --2- 172.21.15.112:0/1612005258 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f61f00b9ad0 0x7f61f0158310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:47.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.441+0000 7f61eed9d640 1 -- 172.21.15.112:0/1612005258 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f61f00ceb90 msgr2=0x7f61f006ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:47.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.441+0000 7f61eed9d640 1 --2- 172.21.15.112:0/1612005258 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f61f00ceb90 0x7f61f006ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:47.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.441+0000 7f61eed9d640 1 -- 172.21.15.112:0/1612005258 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61f01f8ff0 con 0x7f61f015ee80 2024-09-21T00:27:47.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.441+0000 7f61eed9d640 1 --2- 172.21.15.112:0/1612005258 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f61f015ee80 0x7f61f00b2760 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f61e000d660 tx=0x7f61e000db20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:47.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.441+0000 7f61e6ffd640 1 -- 172.21.15.112:0/1612005258 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61e0017070 con 0x7f61f015ee80 2024-09-21T00:27:47.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.441+0000 7f61e6ffd640 1 -- 172.21.15.112:0/1612005258 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f61e0005310 con 0x7f61f015ee80 2024-09-21T00:27:47.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.441+0000 7f61f4ef9640 1 -- 172.21.15.112:0/1612005258 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61f01f92d0 con 0x7f61f015ee80 2024-09-21T00:27:47.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.441+0000 7f61e6ffd640 1 -- 172.21.15.112:0/1612005258 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61e0009820 con 0x7f61f015ee80 2024-09-21T00:27:47.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.441+0000 7f61f4ef9640 1 -- 172.21.15.112:0/1612005258 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61f006f190 con 0x7f61f015ee80 2024-09-21T00:27:47.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.443+0000 7f61e6ffd640 1 -- 172.21.15.112:0/1612005258 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f61e00099c0 con 0x7f61f015ee80 2024-09-21T00:27:47.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.443+0000 7f61e6ffd640 1 --2- 172.21.15.112:0/1612005258 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f61d0078b90 0x7f61d007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:47.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.444+0000 7f61e6ffd640 1 -- 172.21.15.112:0/1612005258 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7f61e009aed0 con 0x7f61f015ee80 2024-09-21T00:27:47.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.444+0000 7f61ee59c640 1 --2- 172.21.15.112:0/1612005258 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f61d0078b90 0x7f61d007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:47.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.444+0000 7f61fbe6c740 1 Processor -- start 2024-09-21T00:27:47.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.444+0000 7f61ee59c640 1 --2- 172.21.15.112:0/1612005258 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f61d0078b90 0x7f61d007b0a0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f61e8034000 tx=0x7f61e8004ab0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:47.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.444+0000 7f61fbe6c740 1 -- start start 2024-09-21T00:27:47.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.446+0000 7f61fbe6c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559099abf270 0x559099e90fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:47.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.446+0000 7f61fbe6c740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x559099a963b0 0x559099e91500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:47.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.446+0000 7f61fbe6c740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559099a9e590 0x559099eae400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:47.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.446+0000 7f61fbe6c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x559099dd9820 con 0x559099abf270 2024-09-21T00:27:47.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.446+0000 7f61fbe6c740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x559099dc8c50 con 0x559099a9e590 2024-09-21T00:27:47.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.446+0000 7f61fbe6c740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x559099dcca60 con 0x559099a963b0 2024-09-21T00:27:47.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.446+0000 7f61ee59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559099abf270 0x559099e90fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:47.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.446+0000 7f61ee59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559099abf270 0x559099e90fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57210/0 (socket says 172.21.15.112:57210) 2024-09-21T00:27:47.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.446+0000 7f61ee59c640 1 -- 172.21.15.112:0/1285724302 learned_addr learned my addr 172.21.15.112:0/1285724302 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:27:47.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.446+0000 7f61edd9b640 1 --2- 172.21.15.112:0/1285724302 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x559099a963b0 0x559099e91500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:47.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.446+0000 7f61eed9d640 1 --2- 172.21.15.112:0/1285724302 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559099a9e590 0x559099eae400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:47.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.446+0000 7f61ee59c640 1 -- 172.21.15.112:0/1285724302 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x559099a963b0 msgr2=0x559099e91500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:27:47.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.446+0000 7f61ee59c640 1 --2- 172.21.15.112:0/1285724302 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x559099a963b0 0x559099e91500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:47.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.446+0000 7f61ee59c640 1 -- 172.21.15.112:0/1285724302 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559099a9e590 msgr2=0x559099eae400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:27:47.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.446+0000 7f61ee59c640 1 --2- 172.21.15.112:0/1285724302 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559099a9e590 0x559099eae400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:47.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.446+0000 7f61ee59c640 1 -- 172.21.15.112:0/1285724302 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559099b2a850 con 0x559099abf270 2024-09-21T00:27:47.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.446+0000 7f61ee59c640 1 --2- 172.21.15.112:0/1285724302 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559099abf270 0x559099e90fc0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f61e8035040 tx=0x7f61e803c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9d8f7521f492f1d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:47.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.446+0000 7f61cffff640 1 -- 172.21.15.112:0/1285724302 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61e8037040 con 0x559099abf270 2024-09-21T00:27:47.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.446+0000 7f61cffff640 1 -- 172.21.15.112:0/1285724302 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f61e8043070 con 0x559099abf270 2024-09-21T00:27:47.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.446+0000 7f61cffff640 1 -- 172.21.15.112:0/1285724302 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61e8032870 con 0x559099abf270 2024-09-21T00:27:47.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.447+0000 7f61fbe6c740 1 -- 172.21.15.112:0/1285724302 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559099abf270 msgr2=0x559099e90fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:47.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.447+0000 7f61fbe6c740 1 --2- 172.21.15.112:0/1285724302 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559099abf270 0x559099e90fc0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f61e8035040 tx=0x7f61e803c000 comp rx=0 tx=0).stop 2024-09-21T00:27:47.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.447+0000 7f61fbe6c740 1 -- 172.21.15.112:0/1285724302 shutdown_connections 2024-09-21T00:27:47.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.447+0000 7f61fbe6c740 1 --2- 172.21.15.112:0/1285724302 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559099a9e590 0x559099eae400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:47.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.447+0000 7f61fbe6c740 1 --2- 172.21.15.112:0/1285724302 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x559099a963b0 0x559099e91500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:47.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.447+0000 7f61fbe6c740 1 --2- 172.21.15.112:0/1285724302 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559099abf270 0x559099e90fc0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:47.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.447+0000 7f61fbe6c740 1 -- 172.21.15.112:0/1285724302 >> 172.21.15.112:0/1285724302 conn(0x559099ab33c0 msgr2=0x559099e67280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:47.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.447+0000 7f61fbe6c740 1 -- 172.21.15.112:0/1285724302 shutdown_connections 2024-09-21T00:27:47.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.447+0000 7f61fbe6c740 1 -- 172.21.15.112:0/1285724302 wait complete. 2024-09-21T00:27:47.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.448+0000 7f61fbe6c740 1 -- start start 2024-09-21T00:27:47.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.448+0000 7f61fbe6c740 1 Processor -- start 2024-09-21T00:27:47.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.450+0000 7f61fbe6c740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x559099a963b0 0x559099ea57d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:47.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.450+0000 7f61fbe6c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559099a9e590 0x559099e7a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:47.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.450+0000 7f61fbe6c740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559099abf270 0x559099ebe550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:47.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.450+0000 7f61fbe6c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x559099b39620 con 0x559099a9e590 2024-09-21T00:27:47.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.450+0000 7f61fbe6c740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x559099d6e5f0 con 0x559099abf270 2024-09-21T00:27:47.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.450+0000 7f61fbe6c740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x559099e91a40 con 0x559099a963b0 2024-09-21T00:27:47.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.450+0000 7f61fbe6c740 10 client.0 _ll_register_callbacks cb 0x559099d720a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:27:47.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.450+0000 7f61edd9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559099a9e590 0x559099e7a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:47.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.450+0000 7f61edd9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559099a9e590 0x559099e7a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57220/0 (socket says 172.21.15.112:57220) 2024-09-21T00:27:47.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.450+0000 7f61edd9b640 1 -- 172.21.15.112:0/3626386057 learned_addr learned my addr 172.21.15.112:0/3626386057 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:27:47.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.450+0000 7f61ee59c640 1 --2- 172.21.15.112:0/3626386057 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x559099a963b0 0x559099ea57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:47.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.450+0000 7f61eed9d640 1 --2- 172.21.15.112:0/3626386057 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559099abf270 0x559099ebe550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:47.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.450+0000 7f61edd9b640 1 -- 172.21.15.112:0/3626386057 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x559099a963b0 msgr2=0x559099ea57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:27:47.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.450+0000 7f61edd9b640 1 --2- 172.21.15.112:0/3626386057 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x559099a963b0 0x559099ea57d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:47.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.450+0000 7f61edd9b640 1 -- 172.21.15.112:0/3626386057 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559099abf270 msgr2=0x559099ebe550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:27:47.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.450+0000 7f61edd9b640 1 --2- 172.21.15.112:0/3626386057 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559099abf270 0x559099ebe550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:47.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.450+0000 7f61edd9b640 1 -- 172.21.15.112:0/3626386057 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559099ba16b0 con 0x559099a9e590 2024-09-21T00:27:47.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.450+0000 7f61edd9b640 1 --2- 172.21.15.112:0/3626386057 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559099a9e590 0x559099e7a440 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f61f00d28f0 tx=0x7f61d8012980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c659a5f535f358a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:47.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.450+0000 7f61cd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T00:27:47.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.450+0000 7f61cd7fa640 1 -- 172.21.15.112:0/3626386057 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d8019020 con 0x559099a9e590 2024-09-21T00:27:47.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.450+0000 7f61cd7fa640 1 -- 172.21.15.112:0/3626386057 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7f61d80052e0 con 0x559099a9e590 2024-09-21T00:27:47.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.450+0000 7f61fbe6c740 1 -- 172.21.15.112:0/3626386057 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61e8043070 con 0x559099a9e590 2024-09-21T00:27:47.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.451+0000 7f61cd7fa640 1 -- 172.21.15.112:0/3626386057 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f61d8023050 con 0x559099a9e590 2024-09-21T00:27:47.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.451+0000 7f61fbe6c740 10 client.5940 fetch_fsmap learned FSMap version 38 2024-09-21T00:27:47.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.451+0000 7f61fbe6c740 1 -- 172.21.15.112:0/3626386057 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=2+}) -- 0x559099eaf8e0 con 0x559099a9e590 2024-09-21T00:27:47.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.451+0000 7f61cd7fa640 1 -- 172.21.15.112:0/3626386057 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f61e8043070 con 0x559099a9e590 2024-09-21T00:27:47.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.451+0000 7f61cd7fa640 1 -- 172.21.15.112:0/3626386057 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 38) ==== 3522+0+0 (secure 0 0 0) 0x7f61d8026020 con 0x559099a9e590 2024-09-21T00:27:47.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.451+0000 7f61fbe6c740 10 client.5940 fetch_fsmap finished waiting for FSMap version 38 2024-09-21T00:27:47.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.451+0000 7f61fbe6c740 10 client.5940 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T00:27:47.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.451+0000 7f61fbe6c740 20 client.5940 populate_metadata read hostname 'smithi112' 2024-09-21T00:27:47.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.451+0000 7f61fbe6c740 1 --2- 172.21.15.112:0/3626386057 >> [v2:172.21.15.112:6832/1432588455,v1:172.21.15.112:6834/1432588455] conn(0x559099a7de40 0x559099ebb030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:47.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.451+0000 7f61fbe6c740 4 client.5940 mds_command: new command op to 5736 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T00:27:47.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.451+0000 7f61fbe6c740 1 -- 172.21.15.112:0/3626386057 --> [v2:172.21.15.112:6832/1432588455,v1:172.21.15.112:6834/1432588455] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559099ddb770 con 0x559099a7de40 2024-09-21T00:27:47.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.451+0000 7f61ee59c640 1 --2- 172.21.15.112:0/3626386057 >> [v2:172.21.15.112:6832/1432588455,v1:172.21.15.112:6834/1432588455] conn(0x559099a7de40 0x559099ebb030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:47.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.451+0000 7f61cd7fa640 10 client.5940 ms_handle_connect on v2:172.21.15.112:6832/1432588455 2024-09-21T00:27:47.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.452+0000 7f61ee59c640 1 --2- 172.21.15.112:0/3626386057 >> [v2:172.21.15.112:6832/1432588455,v1:172.21.15.112:6834/1432588455] conn(0x559099a7de40 0x559099ebb030 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8d5505c0dc87c94e server_cookie=9d770eaa70ede9d9 in_seq=0 out_seq=0 2024-09-21T00:27:47.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.452+0000 7f61cd7fa640 1 -- 172.21.15.112:0/3626386057 <== mds.1 v2:172.21.15.112:6832/1432588455 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559099ddb770 con 0x559099a7de40 2024-09-21T00:27:47.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.452+0000 7f61cd7fa640 10 client.5940 handle_command_reply: tid=0 2024-09-21T00:27:47.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.452+0000 7f61fbe6c740 1 client.5940 shutdown 2024-09-21T00:27:47.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.453+0000 7f61fbe6c740 1 -- 172.21.15.112:0/3626386057 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559099a9e590 msgr2=0x559099e7a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:47.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.453+0000 7f61fbe6c740 1 --2- 172.21.15.112:0/3626386057 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559099a9e590 0x559099e7a440 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f61f00d28f0 tx=0x7f61d8012980 comp rx=0 tx=0).stop 2024-09-21T00:27:47.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.453+0000 7f61fbe6c740 1 -- 172.21.15.112:0/3626386057 shutdown_connections 2024-09-21T00:27:47.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.453+0000 7f61fbe6c740 1 --2- 172.21.15.112:0/3626386057 >> [v2:172.21.15.112:6832/1432588455,v1:172.21.15.112:6834/1432588455] conn(0x559099a7de40 0x559099ebb030 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:47.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.453+0000 7f61fbe6c740 1 --2- 172.21.15.112:0/3626386057 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559099abf270 0x559099ebe550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:47.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.453+0000 7f61fbe6c740 1 --2- 172.21.15.112:0/3626386057 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559099a9e590 0x559099e7a440 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:47.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.453+0000 7f61fbe6c740 1 --2- 172.21.15.112:0/3626386057 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x559099a963b0 0x559099ea57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:47.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.453+0000 7f61cd7fa640 0 client.5940 ms_handle_reset on v2:172.21.15.112:6832/1432588455 2024-09-21T00:27:47.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.453+0000 7f61fbe6c740 1 -- 172.21.15.112:0/3626386057 >> 172.21.15.112:0/3626386057 conn(0x559099ab33c0 msgr2=0x559099e67130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:47.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.453+0000 7f61fbe6c740 1 -- 172.21.15.112:0/3626386057 shutdown_connections 2024-09-21T00:27:47.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.453+0000 7f61fbe6c740 1 -- 172.21.15.112:0/3626386057 wait complete. 2024-09-21T00:27:47.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.453+0000 7f61fbe6c740 20 client.5940 trim_cache size 0 max 16384 2024-09-21T00:27:47.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.471+0000 7f61fbe6c740 1 Processor -- start 2024-09-21T00:27:47.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.471+0000 7f61fbe6c740 1 -- start start 2024-09-21T00:27:47.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.472+0000 7f61fbe6c740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x559099ab33c0 0x559099e7ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:47.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.472+0000 7f61fbe6c740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559099a963b0 0x559099ea3c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:47.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.472+0000 7f61fbe6c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559099a9e590 0x559099ea41a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:47.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.472+0000 7f61fbe6c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f61f0202610 con 0x559099a9e590 2024-09-21T00:27:47.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.472+0000 7f61fbe6c740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f61f006d9b0 con 0x559099a963b0 2024-09-21T00:27:47.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.472+0000 7f61fbe6c740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f61f0200fc0 con 0x559099ab33c0 2024-09-21T00:27:47.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.473+0000 7f61eed9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559099a9e590 0x559099ea41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:47.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.473+0000 7f61eed9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559099a9e590 0x559099ea41a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57224/0 (socket says 172.21.15.112:57224) 2024-09-21T00:27:47.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.473+0000 7f61eed9d640 1 -- 172.21.15.112:0/3980626001 learned_addr learned my addr 172.21.15.112:0/3980626001 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:27:47.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.473+0000 7f61edd9b640 1 --2- 172.21.15.112:0/3980626001 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559099a963b0 0x559099ea3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:47.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.473+0000 7f61ee59c640 1 --2- 172.21.15.112:0/3980626001 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x559099ab33c0 0x559099e7ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:47.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.473+0000 7f61eed9d640 1 -- 172.21.15.112:0/3980626001 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x559099ab33c0 msgr2=0x559099e7ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:27:47.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.473+0000 7f61eed9d640 1 --2- 172.21.15.112:0/3980626001 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x559099ab33c0 0x559099e7ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:47.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.473+0000 7f61eed9d640 1 -- 172.21.15.112:0/3980626001 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559099a963b0 msgr2=0x559099ea3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:27:47.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.473+0000 7f61eed9d640 1 --2- 172.21.15.112:0/3980626001 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559099a963b0 0x559099ea3c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:47.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.473+0000 7f61eed9d640 1 -- 172.21.15.112:0/3980626001 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559099ea4ed0 con 0x559099a9e590 2024-09-21T00:27:47.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.473+0000 7f61eed9d640 1 --2- 172.21.15.112:0/3980626001 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559099a9e590 0x559099ea41a0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f61e0062ac0 tx=0x7f61e001d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6b9d5f46a37c564 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:47.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.473+0000 7f61cf7fe640 1 -- 172.21.15.112:0/3980626001 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61e007c020 con 0x559099a9e590 2024-09-21T00:27:47.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.473+0000 7f61cf7fe640 1 -- 172.21.15.112:0/3980626001 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f61e006c4a0 con 0x559099a9e590 2024-09-21T00:27:47.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.473+0000 7f61cf7fe640 1 -- 172.21.15.112:0/3980626001 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61e0020040 con 0x559099a9e590 2024-09-21T00:27:47.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.474+0000 7f61fbe6c740 1 -- 172.21.15.112:0/3980626001 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559099a9e590 msgr2=0x559099ea41a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:47.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.474+0000 7f61fbe6c740 1 --2- 172.21.15.112:0/3980626001 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559099a9e590 0x559099ea41a0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f61e0062ac0 tx=0x7f61e001d040 comp rx=0 tx=0).stop 2024-09-21T00:27:47.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.474+0000 7f61fbe6c740 1 -- 172.21.15.112:0/3980626001 shutdown_connections 2024-09-21T00:27:47.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.474+0000 7f61fbe6c740 1 --2- 172.21.15.112:0/3980626001 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559099a9e590 0x559099ea41a0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:47.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.474+0000 7f61fbe6c740 1 --2- 172.21.15.112:0/3980626001 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559099a963b0 0x559099ea3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:47.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.474+0000 7f61fbe6c740 1 --2- 172.21.15.112:0/3980626001 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x559099ab33c0 0x559099e7ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:47.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.474+0000 7f61fbe6c740 1 -- 172.21.15.112:0/3980626001 >> 172.21.15.112:0/3980626001 conn(0x559099ea2120 msgr2=0x559099e62f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:47.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.474+0000 7f61fbe6c740 1 -- 172.21.15.112:0/3980626001 shutdown_connections 2024-09-21T00:27:47.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.474+0000 7f61fbe6c740 1 -- 172.21.15.112:0/3980626001 wait complete. 2024-09-21T00:27:47.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.475+0000 7f61fbe6c740 1 -- start start 2024-09-21T00:27:47.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.475+0000 7f61fbe6c740 1 Processor -- start 2024-09-21T00:27:47.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.476+0000 7f61fbe6c740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x559099a963b0 0x559099ea5830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:47.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.476+0000 7f61fbe6c740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559099a9e590 0x559099f2b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:47.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.476+0000 7f61fbe6c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559099ab33c0 0x559099ebd040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:47.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.476+0000 7f61fbe6c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x559099e7ab90 con 0x559099ab33c0 2024-09-21T00:27:47.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.476+0000 7f61fbe6c740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x559099ebf260 con 0x559099a9e590 2024-09-21T00:27:47.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.476+0000 7f61fbe6c740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x559099eaed30 con 0x559099a963b0 2024-09-21T00:27:47.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.476+0000 7f61fbe6c740 10 client.0 _ll_register_callbacks cb 0x7f61d8007b60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:27:47.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.476+0000 7f61eed9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559099ab33c0 0x559099ebd040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:47.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.476+0000 7f61eed9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559099ab33c0 0x559099ebd040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57234/0 (socket says 172.21.15.112:57234) 2024-09-21T00:27:47.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.476+0000 7f61eed9d640 1 -- 172.21.15.112:0/4249202456 learned_addr learned my addr 172.21.15.112:0/4249202456 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:27:47.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.476+0000 7f61ee59c640 1 --2- 172.21.15.112:0/4249202456 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x559099a963b0 0x559099ea5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:47.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.476+0000 7f61edd9b640 1 --2- 172.21.15.112:0/4249202456 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559099a9e590 0x559099f2b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:47.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.477+0000 7f61eed9d640 1 -- 172.21.15.112:0/4249202456 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x559099a963b0 msgr2=0x559099ea5830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:27:47.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.477+0000 7f61eed9d640 1 --2- 172.21.15.112:0/4249202456 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x559099a963b0 0x559099ea5830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:47.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.477+0000 7f61eed9d640 1 -- 172.21.15.112:0/4249202456 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559099a9e590 msgr2=0x559099f2b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:27:47.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.477+0000 7f61eed9d640 1 --2- 172.21.15.112:0/4249202456 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559099a9e590 0x559099f2b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:47.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.477+0000 7f61eed9d640 1 -- 172.21.15.112:0/4249202456 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559099ebdbe0 con 0x559099ab33c0 2024-09-21T00:27:47.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.477+0000 7f61eed9d640 1 --2- 172.21.15.112:0/4249202456 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559099ab33c0 0x559099ebd040 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f61e00625f0 tx=0x7f61e006c240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61086f89dd854605 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:47.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.477+0000 7f61cd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T00:27:47.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.477+0000 7f61cd7fa640 1 -- 172.21.15.112:0/4249202456 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61e0020070 con 0x559099ab33c0 2024-09-21T00:27:47.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.477+0000 7f61cd7fa640 1 -- 172.21.15.112:0/4249202456 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7f61e0061fa0 con 0x559099ab33c0 2024-09-21T00:27:47.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.477+0000 7f61fbe6c740 1 -- 172.21.15.112:0/4249202456 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61e006c4a0 con 0x559099ab33c0 2024-09-21T00:27:47.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.477+0000 7f61cd7fa640 1 -- 172.21.15.112:0/4249202456 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f61e007c070 con 0x559099ab33c0 2024-09-21T00:27:47.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.477+0000 7f61fbe6c740 10 client.5946 fetch_fsmap learned FSMap version 38 2024-09-21T00:27:47.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.477+0000 7f61fbe6c740 1 -- 172.21.15.112:0/4249202456 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=2+}) -- 0x559099f2b860 con 0x559099ab33c0 2024-09-21T00:27:47.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.477+0000 7f61cd7fa640 1 -- 172.21.15.112:0/4249202456 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f61e006c4a0 con 0x559099ab33c0 2024-09-21T00:27:47.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.477+0000 7f61cd7fa640 1 -- 172.21.15.112:0/4249202456 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 38) ==== 3522+0+0 (secure 0 0 0) 0x7f61e0087040 con 0x559099ab33c0 2024-09-21T00:27:47.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.477+0000 7f61fbe6c740 10 client.5946 fetch_fsmap finished waiting for FSMap version 38 2024-09-21T00:27:47.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.477+0000 7f61fbe6c740 10 client.5946 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T00:27:47.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.477+0000 7f61fbe6c740 20 client.5946 populate_metadata read hostname 'smithi112' 2024-09-21T00:27:47.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.477+0000 7f61fbe6c740 1 --2- 172.21.15.112:0/4249202456 >> [v2:172.21.15.112:6832/1432588455,v1:172.21.15.112:6834/1432588455] conn(0x559099a53280 0x559099ea3e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:47.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.477+0000 7f61fbe6c740 4 client.5946 mds_command: new command op to 5736 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T00:27:47.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.477+0000 7f61fbe6c740 1 -- 172.21.15.112:0/4249202456 --> [v2:172.21.15.112:6832/1432588455,v1:172.21.15.112:6834/1432588455] -- command(tid 0: {"prefix": "status"}) -- 0x559099ea3960 con 0x559099a53280 2024-09-21T00:27:47.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.478+0000 7f61ee59c640 1 --2- 172.21.15.112:0/4249202456 >> [v2:172.21.15.112:6832/1432588455,v1:172.21.15.112:6834/1432588455] conn(0x559099a53280 0x559099ea3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:47.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.478+0000 7f61cd7fa640 10 client.5946 ms_handle_connect on v2:172.21.15.112:6832/1432588455 2024-09-21T00:27:47.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.478+0000 7f61ee59c640 1 --2- 172.21.15.112:0/4249202456 >> [v2:172.21.15.112:6832/1432588455,v1:172.21.15.112:6834/1432588455] conn(0x559099a53280 0x559099ea3e90 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c34826765e8cd8e8 server_cookie=ed2599544a5813ab in_seq=0 out_seq=0 2024-09-21T00:27:47.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.478+0000 7f61cd7fa640 1 -- 172.21.15.112:0/4249202456 <== mds.1 v2:172.21.15.112:6832/1432588455 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x559099ea3960 con 0x559099a53280 2024-09-21T00:27:47.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.478+0000 7f61cd7fa640 10 client.5946 handle_command_reply: tid=0 2024-09-21T00:27:47.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.478+0000 7f61fbe6c740 1 client.5946 shutdown 2024-09-21T00:27:47.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.478+0000 7f61fbe6c740 1 -- 172.21.15.112:0/4249202456 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559099ab33c0 msgr2=0x559099ebd040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:47.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.478+0000 7f61fbe6c740 1 --2- 172.21.15.112:0/4249202456 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559099ab33c0 0x559099ebd040 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f61e00625f0 tx=0x7f61e006c240 comp rx=0 tx=0).stop 2024-09-21T00:27:47.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.478+0000 7f61fbe6c740 1 -- 172.21.15.112:0/4249202456 shutdown_connections 2024-09-21T00:27:47.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.478+0000 7f61fbe6c740 1 --2- 172.21.15.112:0/4249202456 >> [v2:172.21.15.112:6832/1432588455,v1:172.21.15.112:6834/1432588455] conn(0x559099a53280 0x559099ea3e90 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:47.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.478+0000 7f61fbe6c740 1 --2- 172.21.15.112:0/4249202456 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559099ab33c0 0x559099ebd040 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:47.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.478+0000 7f61cd7fa640 0 client.5946 ms_handle_reset on v2:172.21.15.112:6832/1432588455 2024-09-21T00:27:47.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.478+0000 7f61fbe6c740 1 --2- 172.21.15.112:0/4249202456 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559099a9e590 0x559099f2b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:47.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.478+0000 7f61fbe6c740 1 --2- 172.21.15.112:0/4249202456 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x559099a963b0 0x559099ea5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:47.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.478+0000 7f61fbe6c740 1 -- 172.21.15.112:0/4249202456 >> 172.21.15.112:0/4249202456 conn(0x559099ea2120 msgr2=0x559099e7f640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:47.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.478+0000 7f61fbe6c740 1 -- 172.21.15.112:0/4249202456 shutdown_connections 2024-09-21T00:27:47.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.479+0000 7f61fbe6c740 1 -- 172.21.15.112:0/4249202456 wait complete. 2024-09-21T00:27:47.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.479+0000 7f61fbe6c740 20 client.5946 trim_cache size 0 max 16384 2024-09-21T00:27:47.486 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-21T00:27:47.486 INFO:teuthology.orchestra.run.smithi112.stdout: "cluster_fsid": "8b34692b-dc65-49b2-a08f-26f707b96ac5", 2024-09-21T00:27:47.486 INFO:teuthology.orchestra.run.smithi112.stdout: "whoami": 1, 2024-09-21T00:27:47.486 INFO:teuthology.orchestra.run.smithi112.stdout: "id": 5736, 2024-09-21T00:27:47.486 INFO:teuthology.orchestra.run.smithi112.stdout: "want_state": "up:active", 2024-09-21T00:27:47.486 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "up:active", 2024-09-21T00:27:47.486 INFO:teuthology.orchestra.run.smithi112.stdout: "fs_name": "cephfs", 2024-09-21T00:27:47.486 INFO:teuthology.orchestra.run.smithi112.stdout: "rank_uptime": 2.7007835949999999, 2024-09-21T00:27:47.486 INFO:teuthology.orchestra.run.smithi112.stdout: "mdsmap_epoch": 38, 2024-09-21T00:27:47.486 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch": 60, 2024-09-21T00:27:47.486 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch_barrier": 60, 2024-09-21T00:27:47.487 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 16.651225404000002 2024-09-21T00:27:47.487 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-21T00:27:47.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.479+0000 7f61f4ef9640 1 -- 172.21.15.112:0/1612005258 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f61d0078b90 msgr2=0x7f61d007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:47.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.479+0000 7f61f4ef9640 1 --2- 172.21.15.112:0/1612005258 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f61d0078b90 0x7f61d007b0a0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f61e8034000 tx=0x7f61e8004ab0 comp rx=0 tx=0).stop 2024-09-21T00:27:47.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.479+0000 7f61f4ef9640 1 -- 172.21.15.112:0/1612005258 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f61f015ee80 msgr2=0x7f61f00b2760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:47.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.479+0000 7f61f4ef9640 1 --2- 172.21.15.112:0/1612005258 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f61f015ee80 0x7f61f00b2760 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f61e000d660 tx=0x7f61e000db20 comp rx=0 tx=0).stop 2024-09-21T00:27:47.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.479+0000 7f61f4ef9640 1 -- 172.21.15.112:0/1612005258 shutdown_connections 2024-09-21T00:27:47.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.479+0000 7f61f4ef9640 1 --2- 172.21.15.112:0/1612005258 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f61d0078b90 0x7f61d007b0a0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:47.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.479+0000 7f61f4ef9640 1 --2- 172.21.15.112:0/1612005258 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f61f015ee80 0x7f61f00b2760 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:47.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.479+0000 7f61f4ef9640 1 --2- 172.21.15.112:0/1612005258 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f61f00ceb90 0x7f61f006ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:47.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.479+0000 7f61f4ef9640 1 --2- 172.21.15.112:0/1612005258 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f61f00b9ad0 0x7f61f0158310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:47.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.479+0000 7f61f4ef9640 1 -- 172.21.15.112:0/1612005258 >> 172.21.15.112:0/1612005258 conn(0x7f61f009d830 msgr2=0x7f61f00b3fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:47.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.479+0000 7f61f4ef9640 1 -- 172.21.15.112:0/1612005258 shutdown_connections 2024-09-21T00:27:47.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.479+0000 7f61f4ef9640 1 -- 172.21.15.112:0/1612005258 wait complete. 2024-09-21T00:27:47.492 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-21T00:27:47.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.603+0000 7f5dfba69640 1 -- 172.21.15.112:0/1105432899 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5df40b5bd0 msgr2=0x7f5df415ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:47.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.603+0000 7f5dfba69640 1 --2- 172.21.15.112:0/1105432899 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5df40b5bd0 0x7f5df415ead0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f5df0005fd0 tx=0x7f5df002a560 comp rx=0 tx=0).stop 2024-09-21T00:27:47.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.604+0000 7f5dfba69640 1 -- 172.21.15.112:0/1105432899 shutdown_connections 2024-09-21T00:27:47.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.604+0000 7f5dfba69640 1 --2- 172.21.15.112:0/1105432899 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5df40b5bd0 0x7f5df415ead0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:47.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.604+0000 7f5dfba69640 1 -- 172.21.15.112:0/1105432899 >> 172.21.15.112:0/1105432899 conn(0x7f5df40b46e0 msgr2=0x7f5df40d23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:47.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.604+0000 7f5dfba69640 1 -- 172.21.15.112:0/1105432899 shutdown_connections 2024-09-21T00:27:47.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.604+0000 7f5dfba69640 1 -- 172.21.15.112:0/1105432899 wait complete. 2024-09-21T00:27:47.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.605+0000 7f5dfba69640 1 Processor -- start 2024-09-21T00:27:47.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.606+0000 7f5dfba69640 1 -- start start 2024-09-21T00:27:47.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.607+0000 7f5dfba69640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5df40b5bd0 0x7f5df40cbda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:47.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.607+0000 7f5dfba69640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5df4162710 0x7f5df415a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:47.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.607+0000 7f5dfba69640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5df4163230 0x7f5df415a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:47.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.607+0000 7f5dfba69640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5df40ce0e0 con 0x7f5df4163230 2024-09-21T00:27:47.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.607+0000 7f5dfba69640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5df40cdf60 con 0x7f5df40b5bd0 2024-09-21T00:27:47.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.607+0000 7f5dfba69640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f5df40ce260 con 0x7f5df4162710 2024-09-21T00:27:47.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.607+0000 7f5dfa006640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5df4163230 0x7f5df415a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:47.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.607+0000 7f5dfa006640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5df4163230 0x7f5df415a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57248/0 (socket says 172.21.15.112:57248) 2024-09-21T00:27:47.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.607+0000 7f5dfa006640 1 -- 172.21.15.112:0/3322304653 learned_addr learned my addr 172.21.15.112:0/3322304653 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:27:47.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.607+0000 7f5df9004640 1 --2- 172.21.15.112:0/3322304653 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5df4162710 0x7f5df415a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:47.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.607+0000 7f5df9805640 1 --2- 172.21.15.112:0/3322304653 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5df40b5bd0 0x7f5df40cbda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:47.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.607+0000 7f5dfa006640 1 -- 172.21.15.112:0/3322304653 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5df4162710 msgr2=0x7f5df415a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:47.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.607+0000 7f5dfa006640 1 --2- 172.21.15.112:0/3322304653 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5df4162710 0x7f5df415a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:47.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.607+0000 7f5dfa006640 1 -- 172.21.15.112:0/3322304653 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5df40b5bd0 msgr2=0x7f5df40cbda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:47.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.607+0000 7f5dfa006640 1 --2- 172.21.15.112:0/3322304653 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5df40b5bd0 0x7f5df40cbda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:47.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.607+0000 7f5dfa006640 1 -- 172.21.15.112:0/3322304653 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5df40af500 con 0x7f5df4163230 2024-09-21T00:27:47.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.607+0000 7f5df9004640 1 --2- 172.21.15.112:0/3322304653 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5df4162710 0x7f5df415a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:27:47.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.607+0000 7f5dfa006640 1 --2- 172.21.15.112:0/3322304653 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5df4163230 0x7f5df415a7f0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f5dec00c730 tx=0x7f5dec00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:47.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.608+0000 7f5dea7fc640 1 -- 172.21.15.112:0/3322304653 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dec003b70 con 0x7f5df4163230 2024-09-21T00:27:47.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.608+0000 7f5dea7fc640 1 -- 172.21.15.112:0/3322304653 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5dec003d10 con 0x7f5df4163230 2024-09-21T00:27:47.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.608+0000 7f5dfba69640 1 -- 172.21.15.112:0/3322304653 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5df40b0c70 con 0x7f5df4163230 2024-09-21T00:27:47.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.608+0000 7f5dea7fc640 1 -- 172.21.15.112:0/3322304653 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dec00aab0 con 0x7f5df4163230 2024-09-21T00:27:47.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.608+0000 7f5dfba69640 1 -- 172.21.15.112:0/3322304653 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5df40adc80 con 0x7f5df4163230 2024-09-21T00:27:47.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.610+0000 7f5dea7fc640 1 -- 172.21.15.112:0/3322304653 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5dec00ac50 con 0x7f5df4163230 2024-09-21T00:27:47.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.610+0000 7f5dea7fc640 1 --2- 172.21.15.112:0/3322304653 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f5dd8078b90 0x7f5dd807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:47.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.610+0000 7f5dea7fc640 1 -- 172.21.15.112:0/3322304653 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7f5dec09ac50 con 0x7f5df4163230 2024-09-21T00:27:47.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.610+0000 7f5df9805640 1 --2- 172.21.15.112:0/3322304653 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f5dd8078b90 0x7f5dd807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:47.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.611+0000 7f5df9805640 1 --2- 172.21.15.112:0/3322304653 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f5dd8078b90 0x7f5dd807b0a0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f5df0033000 tx=0x7f5df0004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:47.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.611+0000 7f5e029d8740 1 Processor -- start 2024-09-21T00:27:47.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.611+0000 7f5e029d8740 1 -- start start 2024-09-21T00:27:47.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.612+0000 7f5e029d8740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564c17561270 0x564c17932fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:47.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.612+0000 7f5e029d8740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x564c175383b0 0x564c179334e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:47.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.612+0000 7f5e029d8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c17540590 0x564c179503e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:47.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.612+0000 7f5e029d8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x564c1787b820 con 0x564c17540590 2024-09-21T00:27:47.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.612+0000 7f5e029d8740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x564c1786ac50 con 0x564c17561270 2024-09-21T00:27:47.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.612+0000 7f5e029d8740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x564c1786ea60 con 0x564c175383b0 2024-09-21T00:27:47.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.612+0000 7f5dfa006640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c17540590 0x564c179503e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:47.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.612+0000 7f5df9805640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564c17561270 0x564c17932fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:47.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.612+0000 7f5dfa006640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c17540590 0x564c179503e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57250/0 (socket says 172.21.15.112:57250) 2024-09-21T00:27:47.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.612+0000 7f5dfa006640 1 -- 172.21.15.112:0/1642675808 learned_addr learned my addr 172.21.15.112:0/1642675808 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:27:47.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.612+0000 7f5df9004640 1 --2- 172.21.15.112:0/1642675808 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x564c175383b0 0x564c179334e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:47.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.613+0000 7f5dfa006640 1 -- 172.21.15.112:0/1642675808 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x564c175383b0 msgr2=0x564c179334e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:27:47.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.613+0000 7f5dfa006640 1 --2- 172.21.15.112:0/1642675808 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x564c175383b0 0x564c179334e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:47.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.613+0000 7f5dfa006640 1 -- 172.21.15.112:0/1642675808 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564c17561270 msgr2=0x564c17932fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:27:47.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.613+0000 7f5dfa006640 1 --2- 172.21.15.112:0/1642675808 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564c17561270 0x564c17932fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:47.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.613+0000 7f5dfa006640 1 -- 172.21.15.112:0/1642675808 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c175cc850 con 0x564c17540590 2024-09-21T00:27:47.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.613+0000 7f5dfa006640 1 --2- 172.21.15.112:0/1642675808 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c17540590 0x564c179503e0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f5dec05fb00 tx=0x7f5dec062000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7656f7b7b5e884d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:47.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.613+0000 7f5dd77fe640 1 -- 172.21.15.112:0/1642675808 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dec062f90 con 0x564c17540590 2024-09-21T00:27:47.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.613+0000 7f5dd77fe640 1 -- 172.21.15.112:0/1642675808 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5dec067030 con 0x564c17540590 2024-09-21T00:27:47.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.613+0000 7f5dd77fe640 1 -- 172.21.15.112:0/1642675808 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dec063e20 con 0x564c17540590 2024-09-21T00:27:47.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.614+0000 7f5e029d8740 1 -- 172.21.15.112:0/1642675808 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c17540590 msgr2=0x564c179503e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:47.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.614+0000 7f5e029d8740 1 --2- 172.21.15.112:0/1642675808 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c17540590 0x564c179503e0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f5dec05fb00 tx=0x7f5dec062000 comp rx=0 tx=0).stop 2024-09-21T00:27:47.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.614+0000 7f5e029d8740 1 -- 172.21.15.112:0/1642675808 shutdown_connections 2024-09-21T00:27:47.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.614+0000 7f5e029d8740 1 --2- 172.21.15.112:0/1642675808 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c17540590 0x564c179503e0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:47.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.614+0000 7f5e029d8740 1 --2- 172.21.15.112:0/1642675808 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x564c175383b0 0x564c179334e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:47.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.614+0000 7f5e029d8740 1 --2- 172.21.15.112:0/1642675808 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564c17561270 0x564c17932fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:47.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.614+0000 7f5e029d8740 1 -- 172.21.15.112:0/1642675808 >> 172.21.15.112:0/1642675808 conn(0x564c175553c0 msgr2=0x564c17909260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:47.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.614+0000 7f5e029d8740 1 -- 172.21.15.112:0/1642675808 shutdown_connections 2024-09-21T00:27:47.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.614+0000 7f5e029d8740 1 -- 172.21.15.112:0/1642675808 wait complete. 2024-09-21T00:27:47.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.615+0000 7f5e029d8740 1 -- start start 2024-09-21T00:27:47.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.615+0000 7f5e029d8740 1 Processor -- start 2024-09-21T00:27:47.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.616+0000 7f5e029d8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c175383b0 0x564c17945b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:47.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.616+0000 7f5e029d8740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x564c17540590 0x564c179602e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:47.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.616+0000 7f5e029d8740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564c17561270 0x564c17960820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:47.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.616+0000 7f5e029d8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x564c175db620 con 0x564c175383b0 2024-09-21T00:27:47.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.616+0000 7f5e029d8740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x564c178105f0 con 0x564c17561270 2024-09-21T00:27:47.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.616+0000 7f5e029d8740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x564c17933a20 con 0x564c17540590 2024-09-21T00:27:47.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.616+0000 7f5e029d8740 10 client.0 _ll_register_callbacks cb 0x564c178140a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:27:47.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.616+0000 7f5df9805640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c175383b0 0x564c17945b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:47.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.617+0000 7f5df9805640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c175383b0 0x564c17945b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57258/0 (socket says 172.21.15.112:57258) 2024-09-21T00:27:47.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.617+0000 7f5df9805640 1 -- 172.21.15.112:0/1834311805 learned_addr learned my addr 172.21.15.112:0/1834311805 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:27:47.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.617+0000 7f5df9004640 1 --2- 172.21.15.112:0/1834311805 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x564c17540590 0x564c179602e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:47.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.617+0000 7f5dfa006640 1 --2- 172.21.15.112:0/1834311805 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564c17561270 0x564c17960820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:47.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.617+0000 7f5df9805640 1 -- 172.21.15.112:0/1834311805 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x564c17540590 msgr2=0x564c179602e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:27:47.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.617+0000 7f5df9805640 1 --2- 172.21.15.112:0/1834311805 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x564c17540590 0x564c179602e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:47.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.617+0000 7f5df9805640 1 -- 172.21.15.112:0/1834311805 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564c17561270 msgr2=0x564c17960820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:27:47.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.617+0000 7f5df9805640 1 --2- 172.21.15.112:0/1834311805 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564c17561270 0x564c17960820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:47.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.617+0000 7f5df9805640 1 -- 172.21.15.112:0/1834311805 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c176436b0 con 0x564c175383b0 2024-09-21T00:27:47.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.617+0000 7f5dfa006640 1 --2- 172.21.15.112:0/1834311805 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564c17561270 0x564c17960820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:27:47.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.617+0000 7f5df9805640 1 --2- 172.21.15.112:0/1834311805 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c175383b0 0x564c17945b80 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f5df00099f0 tx=0x7f5df0009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1132b5b36ba3b3ae server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:47.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.617+0000 7f5dd4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T00:27:47.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.617+0000 7f5dd4ff9640 1 -- 172.21.15.112:0/1834311805 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df004e070 con 0x564c175383b0 2024-09-21T00:27:47.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.617+0000 7f5dd4ff9640 1 -- 172.21.15.112:0/1834311805 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7f5df0004bb0 con 0x564c175383b0 2024-09-21T00:27:47.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.617+0000 7f5e029d8740 1 -- 172.21.15.112:0/1834311805 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5dec067030 con 0x564c175383b0 2024-09-21T00:27:47.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.617+0000 7f5dd4ff9640 1 -- 172.21.15.112:0/1834311805 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f5df0059050 con 0x564c175383b0 2024-09-21T00:27:47.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.617+0000 7f5e029d8740 10 client.5961 fetch_fsmap learned FSMap version 38 2024-09-21T00:27:47.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.617+0000 7f5e029d8740 1 -- 172.21.15.112:0/1834311805 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=2+}) -- 0x564c179472e0 con 0x564c175383b0 2024-09-21T00:27:47.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.617+0000 7f5dd4ff9640 1 -- 172.21.15.112:0/1834311805 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5dec067030 con 0x564c175383b0 2024-09-21T00:27:47.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.618+0000 7f5dd4ff9640 1 -- 172.21.15.112:0/1834311805 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 38) ==== 3522+0+0 (secure 0 0 0) 0x7f5df005c020 con 0x564c175383b0 2024-09-21T00:27:47.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.618+0000 7f5e029d8740 10 client.5961 fetch_fsmap finished waiting for FSMap version 38 2024-09-21T00:27:47.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.618+0000 7f5e029d8740 10 client.5961 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-21T00:27:47.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.618+0000 7f5e029d8740 20 client.5961 populate_metadata read hostname 'smithi112' 2024-09-21T00:27:47.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.618+0000 7f5e029d8740 1 --2- 172.21.15.112:0/1834311805 >> [v2:172.21.15.112:6833/3610025869,v1:172.21.15.112:6835/3610025869] conn(0x564c1751fe40 0x564c1795ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:47.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.618+0000 7f5e029d8740 4 client.5961 mds_command: new command op to 5739 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T00:27:47.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.618+0000 7f5e029d8740 1 -- 172.21.15.112:0/1834311805 --> [v2:172.21.15.112:6833/3610025869,v1:172.21.15.112:6835/3610025869] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564c1787d770 con 0x564c1751fe40 2024-09-21T00:27:47.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.618+0000 7f5df9004640 1 --2- 172.21.15.112:0/1834311805 >> [v2:172.21.15.112:6833/3610025869,v1:172.21.15.112:6835/3610025869] conn(0x564c1751fe40 0x564c1795ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:47.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.618+0000 7f5dd4ff9640 10 client.5961 ms_handle_connect on v2:172.21.15.112:6833/3610025869 2024-09-21T00:27:47.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.618+0000 7f5df9004640 1 --2- 172.21.15.112:0/1834311805 >> [v2:172.21.15.112:6833/3610025869,v1:172.21.15.112:6835/3610025869] conn(0x564c1751fe40 0x564c1795ced0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=606eebeea2cce99a server_cookie=b8d1a68134d130b4 in_seq=0 out_seq=0 2024-09-21T00:27:47.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.619+0000 7f5dd4ff9640 1 -- 172.21.15.112:0/1834311805 <== mds.0 v2:172.21.15.112:6833/3610025869 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564c1787d770 con 0x564c1751fe40 2024-09-21T00:27:47.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.619+0000 7f5dd4ff9640 10 client.5961 handle_command_reply: tid=0 2024-09-21T00:27:47.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.619+0000 7f5e029d8740 1 client.5961 shutdown 2024-09-21T00:27:47.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.619+0000 7f5e029d8740 1 -- 172.21.15.112:0/1834311805 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c175383b0 msgr2=0x564c17945b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:47.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.619+0000 7f5e029d8740 1 --2- 172.21.15.112:0/1834311805 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c175383b0 0x564c17945b80 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f5df00099f0 tx=0x7f5df0009260 comp rx=0 tx=0).stop 2024-09-21T00:27:47.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.619+0000 7f5e029d8740 1 -- 172.21.15.112:0/1834311805 shutdown_connections 2024-09-21T00:27:47.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.619+0000 7f5e029d8740 1 --2- 172.21.15.112:0/1834311805 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564c17561270 0x564c17960820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:47.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.619+0000 7f5e029d8740 1 --2- 172.21.15.112:0/1834311805 >> [v2:172.21.15.112:6833/3610025869,v1:172.21.15.112:6835/3610025869] conn(0x564c1751fe40 0x564c1795ced0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:47.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.619+0000 7f5e029d8740 1 --2- 172.21.15.112:0/1834311805 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x564c17540590 0x564c179602e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:47.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.619+0000 7f5e029d8740 1 --2- 172.21.15.112:0/1834311805 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c175383b0 0x564c17945b80 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:47.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.619+0000 7f5dd4ff9640 0 client.5961 ms_handle_reset on v2:172.21.15.112:6833/3610025869 2024-09-21T00:27:47.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.619+0000 7f5e029d8740 1 -- 172.21.15.112:0/1834311805 >> 172.21.15.112:0/1834311805 conn(0x564c175553c0 msgr2=0x564c17909110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:47.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.619+0000 7f5e029d8740 1 -- 172.21.15.112:0/1834311805 shutdown_connections 2024-09-21T00:27:47.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.619+0000 7f5e029d8740 1 -- 172.21.15.112:0/1834311805 wait complete. 2024-09-21T00:27:47.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.619+0000 7f5e029d8740 20 client.5961 trim_cache size 0 max 16384 2024-09-21T00:27:47.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.637+0000 7f5e029d8740 1 Processor -- start 2024-09-21T00:27:47.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.637+0000 7f5e029d8740 1 -- start start 2024-09-21T00:27:47.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.639+0000 7f5e029d8740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x564c175383b0 0x564c179450e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:47.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.639+0000 7f5e029d8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c17540590 0x564c179092e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:47.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.639+0000 7f5e029d8740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564c17561270 0x564c1795b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:47.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.639+0000 7f5e029d8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5df4202020 con 0x564c17540590 2024-09-21T00:27:47.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.639+0000 7f5e029d8740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5df4202510 con 0x564c17561270 2024-09-21T00:27:47.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.639+0000 7f5e029d8740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f5df41f5fa0 con 0x564c175383b0 2024-09-21T00:27:47.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.639+0000 7f5df9004640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c17540590 0x564c179092e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:47.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.639+0000 7f5df9004640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c17540590 0x564c179092e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57270/0 (socket says 172.21.15.112:57270) 2024-09-21T00:27:47.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.639+0000 7f5df9004640 1 -- 172.21.15.112:0/1958434901 learned_addr learned my addr 172.21.15.112:0/1958434901 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:27:47.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.639+0000 7f5df9805640 1 --2- 172.21.15.112:0/1958434901 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x564c175383b0 0x564c179450e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:47.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.639+0000 7f5df9004640 1 -- 172.21.15.112:0/1958434901 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x564c175383b0 msgr2=0x564c179450e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:27:47.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.639+0000 7f5df9004640 1 --2- 172.21.15.112:0/1958434901 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x564c175383b0 0x564c179450e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:47.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.639+0000 7f5df9004640 1 -- 172.21.15.112:0/1958434901 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564c17561270 msgr2=0x564c1795b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:27:47.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.639+0000 7f5df9004640 1 --2- 172.21.15.112:0/1958434901 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564c17561270 0x564c1795b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:47.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.639+0000 7f5df9004640 1 -- 172.21.15.112:0/1958434901 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c177ed1e0 con 0x564c17540590 2024-09-21T00:27:47.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.639+0000 7f5df9004640 1 --2- 172.21.15.112:0/1958434901 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c17540590 0x564c179092e0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f5df40b5420 tx=0x7f5de4007b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8456fbe7440e5910 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:47.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.639+0000 7f5dd6ffd640 1 -- 172.21.15.112:0/1958434901 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de40051d0 con 0x564c17540590 2024-09-21T00:27:47.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.639+0000 7f5dd6ffd640 1 -- 172.21.15.112:0/1958434901 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5de4022020 con 0x564c17540590 2024-09-21T00:27:47.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.639+0000 7f5dd6ffd640 1 -- 172.21.15.112:0/1958434901 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de4005430 con 0x564c17540590 2024-09-21T00:27:47.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.640+0000 7f5e029d8740 1 -- 172.21.15.112:0/1958434901 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c17540590 msgr2=0x564c179092e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:47.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.640+0000 7f5e029d8740 1 --2- 172.21.15.112:0/1958434901 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c17540590 0x564c179092e0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f5df40b5420 tx=0x7f5de4007b30 comp rx=0 tx=0).stop 2024-09-21T00:27:47.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.640+0000 7f5e029d8740 1 -- 172.21.15.112:0/1958434901 shutdown_connections 2024-09-21T00:27:47.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.640+0000 7f5e029d8740 1 --2- 172.21.15.112:0/1958434901 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564c17561270 0x564c1795b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:47.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.640+0000 7f5e029d8740 1 --2- 172.21.15.112:0/1958434901 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c17540590 0x564c179092e0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:47.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.640+0000 7f5e029d8740 1 --2- 172.21.15.112:0/1958434901 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x564c175383b0 0x564c179450e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:47.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.640+0000 7f5e029d8740 1 -- 172.21.15.112:0/1958434901 >> 172.21.15.112:0/1958434901 conn(0x564c175553c0 msgr2=0x564c17945b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:47.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.640+0000 7f5e029d8740 1 -- 172.21.15.112:0/1958434901 shutdown_connections 2024-09-21T00:27:47.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.640+0000 7f5e029d8740 1 -- 172.21.15.112:0/1958434901 wait complete. 2024-09-21T00:27:47.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.641+0000 7f5e029d8740 1 -- start start 2024-09-21T00:27:47.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.641+0000 7f5e029d8740 1 Processor -- start 2024-09-21T00:27:47.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.642+0000 7f5e029d8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c175383b0 0x564c179b32a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:47.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.642+0000 7f5e029d8740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x564c17540590 0x564c1795b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:47.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.642+0000 7f5e029d8740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564c17561270 0x564c179092e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:47.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.642+0000 7f5e029d8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x564c1794d0f0 con 0x564c175383b0 2024-09-21T00:27:47.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.642+0000 7f5e029d8740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5df40ae460 con 0x564c17561270 2024-09-21T00:27:47.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.642+0000 7f5e029d8740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x564c1794d6e0 con 0x564c17540590 2024-09-21T00:27:47.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.643+0000 7f5e029d8740 10 client.0 _ll_register_callbacks cb 0x7f5df0005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:27:47.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.643+0000 7f5df9805640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c175383b0 0x564c179b32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:47.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.643+0000 7f5df9805640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c175383b0 0x564c179b32a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57280/0 (socket says 172.21.15.112:57280) 2024-09-21T00:27:47.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.643+0000 7f5df9805640 1 -- 172.21.15.112:0/3987645985 learned_addr learned my addr 172.21.15.112:0/3987645985 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:27:47.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.643+0000 7f5df9805640 1 -- 172.21.15.112:0/3987645985 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x564c17540590 msgr2=0x564c1795b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:27:47.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.643+0000 7f5df9805640 1 --2- 172.21.15.112:0/3987645985 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x564c17540590 0x564c1795b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:47.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.643+0000 7f5df9805640 1 -- 172.21.15.112:0/3987645985 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564c17561270 msgr2=0x564c179092e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:27:47.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.643+0000 7f5dfa006640 1 --2- 172.21.15.112:0/3987645985 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564c17561270 0x564c179092e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:47.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.643+0000 7f5df9805640 1 --2- 172.21.15.112:0/3987645985 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564c17561270 0x564c179092e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:47.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.643+0000 7f5df9805640 1 -- 172.21.15.112:0/3987645985 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c17946290 con 0x564c175383b0 2024-09-21T00:27:47.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.643+0000 7f5df9805640 1 --2- 172.21.15.112:0/3987645985 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c175383b0 0x564c179b32a0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f5df00099c0 tx=0x7f5df00044b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9455402080778e97 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:47.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.643+0000 7f5dd4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T00:27:47.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.643+0000 7f5dd4ff9640 1 -- 172.21.15.112:0/3987645985 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df0062060 con 0x564c175383b0 2024-09-21T00:27:47.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.643+0000 7f5dd4ff9640 1 -- 172.21.15.112:0/3987645985 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7f5df003c950 con 0x564c175383b0 2024-09-21T00:27:47.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.643+0000 7f5e029d8740 1 -- 172.21.15.112:0/3987645985 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5de4022020 con 0x564c175383b0 2024-09-21T00:27:47.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.643+0000 7f5dd4ff9640 1 -- 172.21.15.112:0/3987645985 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f5df0068050 con 0x564c175383b0 2024-09-21T00:27:47.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.643+0000 7f5e029d8740 10 client.5967 fetch_fsmap learned FSMap version 38 2024-09-21T00:27:47.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.643+0000 7f5e029d8740 1 -- 172.21.15.112:0/3987645985 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=2+}) -- 0x564c179209f0 con 0x564c175383b0 2024-09-21T00:27:47.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.644+0000 7f5dd4ff9640 1 -- 172.21.15.112:0/3987645985 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5de4022020 con 0x564c175383b0 2024-09-21T00:27:47.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.644+0000 7f5dd4ff9640 1 -- 172.21.15.112:0/3987645985 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 38) ==== 3522+0+0 (secure 0 0 0) 0x7f5df006b020 con 0x564c175383b0 2024-09-21T00:27:47.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.644+0000 7f5e029d8740 10 client.5967 fetch_fsmap finished waiting for FSMap version 38 2024-09-21T00:27:47.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.644+0000 7f5e029d8740 10 client.5967 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-21T00:27:47.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.644+0000 7f5e029d8740 20 client.5967 populate_metadata read hostname 'smithi112' 2024-09-21T00:27:47.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.644+0000 7f5e029d8740 1 --2- 172.21.15.112:0/3987645985 >> [v2:172.21.15.112:6833/3610025869,v1:172.21.15.112:6835/3610025869] conn(0x564c17922160 0x564c1794c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:47.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.644+0000 7f5e029d8740 4 client.5967 mds_command: new command op to 5739 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T00:27:47.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.644+0000 7f5e029d8740 1 -- 172.21.15.112:0/3987645985 --> [v2:172.21.15.112:6833/3610025869,v1:172.21.15.112:6835/3610025869] -- command(tid 0: {"prefix": "status"}) -- 0x564c17879220 con 0x564c17922160 2024-09-21T00:27:47.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.644+0000 7f5df9004640 1 --2- 172.21.15.112:0/3987645985 >> [v2:172.21.15.112:6833/3610025869,v1:172.21.15.112:6835/3610025869] conn(0x564c17922160 0x564c1794c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:47.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.644+0000 7f5dd4ff9640 10 client.5967 ms_handle_connect on v2:172.21.15.112:6833/3610025869 2024-09-21T00:27:47.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.644+0000 7f5df9004640 1 --2- 172.21.15.112:0/3987645985 >> [v2:172.21.15.112:6833/3610025869,v1:172.21.15.112:6835/3610025869] conn(0x564c17922160 0x564c1794c2f0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=879dbbc3eaafe94f server_cookie=aaaccb58c203cd57 in_seq=0 out_seq=0 2024-09-21T00:27:47.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.645+0000 7f5dd4ff9640 1 -- 172.21.15.112:0/3987645985 <== mds.0 v2:172.21.15.112:6833/3610025869 1 ==== command_reply(tid 0: 0 ) ==== 8+0+319 (crc 0 0 0) 0x564c17879220 con 0x564c17922160 2024-09-21T00:27:47.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.645+0000 7f5dd4ff9640 10 client.5967 handle_command_reply: tid=0 2024-09-21T00:27:47.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.645+0000 7f5e029d8740 1 client.5967 shutdown 2024-09-21T00:27:47.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.645+0000 7f5e029d8740 1 -- 172.21.15.112:0/3987645985 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c175383b0 msgr2=0x564c179b32a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:47.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.645+0000 7f5e029d8740 1 --2- 172.21.15.112:0/3987645985 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c175383b0 0x564c179b32a0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f5df00099c0 tx=0x7f5df00044b0 comp rx=0 tx=0).stop 2024-09-21T00:27:47.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.645+0000 7f5e029d8740 1 -- 172.21.15.112:0/3987645985 shutdown_connections 2024-09-21T00:27:47.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.645+0000 7f5e029d8740 1 --2- 172.21.15.112:0/3987645985 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564c17561270 0x564c179092e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:47.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.645+0000 7f5e029d8740 1 --2- 172.21.15.112:0/3987645985 >> [v2:172.21.15.112:6833/3610025869,v1:172.21.15.112:6835/3610025869] conn(0x564c17922160 0x564c1794c2f0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:47.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.645+0000 7f5e029d8740 1 --2- 172.21.15.112:0/3987645985 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x564c17540590 0x564c1795b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:47.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.645+0000 7f5e029d8740 1 --2- 172.21.15.112:0/3987645985 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c175383b0 0x564c179b32a0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:47.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.645+0000 7f5dd4ff9640 0 client.5967 ms_handle_reset on v2:172.21.15.112:6833/3610025869 2024-09-21T00:27:47.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.645+0000 7f5e029d8740 1 -- 172.21.15.112:0/3987645985 >> 172.21.15.112:0/3987645985 conn(0x564c175553c0 msgr2=0x564c17945b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:47.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.645+0000 7f5e029d8740 1 -- 172.21.15.112:0/3987645985 shutdown_connections 2024-09-21T00:27:47.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.645+0000 7f5e029d8740 1 -- 172.21.15.112:0/3987645985 wait complete. 2024-09-21T00:27:47.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.645+0000 7f5e029d8740 20 client.5967 trim_cache size 0 max 16384 2024-09-21T00:27:47.652 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-21T00:27:47.652 INFO:teuthology.orchestra.run.smithi112.stdout: "cluster_fsid": "8b34692b-dc65-49b2-a08f-26f707b96ac5", 2024-09-21T00:27:47.652 INFO:teuthology.orchestra.run.smithi112.stdout: "whoami": 0, 2024-09-21T00:27:47.652 INFO:teuthology.orchestra.run.smithi112.stdout: "id": 5739, 2024-09-21T00:27:47.652 INFO:teuthology.orchestra.run.smithi112.stdout: "want_state": "up:active", 2024-09-21T00:27:47.652 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "up:active", 2024-09-21T00:27:47.652 INFO:teuthology.orchestra.run.smithi112.stdout: "fs_name": "cephfs", 2024-09-21T00:27:47.652 INFO:teuthology.orchestra.run.smithi112.stdout: "rank_uptime": 12.133304848, 2024-09-21T00:27:47.652 INFO:teuthology.orchestra.run.smithi112.stdout: "mdsmap_epoch": 38, 2024-09-21T00:27:47.652 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch": 60, 2024-09-21T00:27:47.653 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch_barrier": 60, 2024-09-21T00:27:47.653 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 16.817221822 2024-09-21T00:27:47.653 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-21T00:27:47.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.645+0000 7f5dfba69640 1 -- 172.21.15.112:0/3322304653 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f5dd8078b90 msgr2=0x7f5dd807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:47.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.645+0000 7f5dfba69640 1 --2- 172.21.15.112:0/3322304653 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f5dd8078b90 0x7f5dd807b0a0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f5df0033000 tx=0x7f5df0004730 comp rx=0 tx=0).stop 2024-09-21T00:27:47.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.646+0000 7f5dfba69640 1 -- 172.21.15.112:0/3322304653 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5df4163230 msgr2=0x7f5df415a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:47.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.646+0000 7f5dfba69640 1 --2- 172.21.15.112:0/3322304653 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5df4163230 0x7f5df415a7f0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f5dec00c730 tx=0x7f5dec00cbf0 comp rx=0 tx=0).stop 2024-09-21T00:27:47.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.646+0000 7f5dfba69640 1 -- 172.21.15.112:0/3322304653 shutdown_connections 2024-09-21T00:27:47.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.646+0000 7f5dfba69640 1 --2- 172.21.15.112:0/3322304653 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f5dd8078b90 0x7f5dd807b0a0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:47.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.646+0000 7f5dfba69640 1 --2- 172.21.15.112:0/3322304653 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5df4163230 0x7f5df415a7f0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:47.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.646+0000 7f5dfba69640 1 --2- 172.21.15.112:0/3322304653 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5df4162710 0x7f5df415a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:47.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.646+0000 7f5dfba69640 1 --2- 172.21.15.112:0/3322304653 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5df40b5bd0 0x7f5df40cbda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:47.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.646+0000 7f5dfba69640 1 -- 172.21.15.112:0/3322304653 >> 172.21.15.112:0/3322304653 conn(0x7f5df40b46e0 msgr2=0x7f5df4068d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:47.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.646+0000 7f5dfba69640 1 -- 172.21.15.112:0/3322304653 shutdown_connections 2024-09-21T00:27:47.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.646+0000 7f5dfba69640 1 -- 172.21.15.112:0/3322304653 wait complete. 2024-09-21T00:27:47.659 DEBUG:teuthology.orchestra.run.smithi112:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-21T00:27:47.662 DEBUG:teuthology.orchestra.run.smithi187:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-21T00:27:47.702 INFO:teuthology.orchestra.run.smithi112.stdout:[global] 2024-09-21T00:27:47.702 INFO:teuthology.orchestra.run.smithi112.stdout: chdir = "" 2024-09-21T00:27:47.702 INFO:teuthology.orchestra.run.smithi112.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-21T00:27:47.702 INFO:teuthology.orchestra.run.smithi112.stdout: auth supported = cephx 2024-09-21T00:27:47.702 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:27:47.702 INFO:teuthology.orchestra.run.smithi112.stdout: filestore xattr use omap = true 2024-09-21T00:27:47.703 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:27:47.703 INFO:teuthology.orchestra.run.smithi112.stdout: mon clock drift allowed = 1.000 2024-09-21T00:27:47.703 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:27:47.703 INFO:teuthology.orchestra.run.smithi112.stdout: osd crush chooseleaf type = 0 2024-09-21T00:27:47.703 INFO:teuthology.orchestra.run.smithi112.stdout: auth debug = true 2024-09-21T00:27:47.703 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:27:47.703 INFO:teuthology.orchestra.run.smithi112.stdout: ms die on old message = true 2024-09-21T00:27:47.703 INFO:teuthology.orchestra.run.smithi112.stdout: ms die on bug = true 2024-09-21T00:27:47.703 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:27:47.703 INFO:teuthology.orchestra.run.smithi112.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-21T00:27:47.703 INFO:teuthology.orchestra.run.smithi112.stdout: mon pg warn max object skew = 0 2024-09-21T00:27:47.703 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:27:47.703 INFO:teuthology.orchestra.run.smithi112.stdout: # disable pg_autoscaler by default for new pools 2024-09-21T00:27:47.703 INFO:teuthology.orchestra.run.smithi112.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-21T00:27:47.703 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:27:47.704 INFO:teuthology.orchestra.run.smithi112.stdout: osd pool default size = 2 2024-09-21T00:27:47.704 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:27:47.704 INFO:teuthology.orchestra.run.smithi112.stdout: mon osd allow primary affinity = true 2024-09-21T00:27:47.704 INFO:teuthology.orchestra.run.smithi112.stdout: mon osd allow pg remap = true 2024-09-21T00:27:47.704 INFO:teuthology.orchestra.run.smithi112.stdout: mon warn on legacy crush tunables = false 2024-09-21T00:27:47.704 INFO:teuthology.orchestra.run.smithi112.stdout: mon warn on crush straw calc version zero = false 2024-09-21T00:27:47.704 INFO:teuthology.orchestra.run.smithi112.stdout: mon warn on no sortbitwise = false 2024-09-21T00:27:47.704 INFO:teuthology.orchestra.run.smithi112.stdout: mon warn on osd down out interval zero = false 2024-09-21T00:27:47.704 INFO:teuthology.orchestra.run.smithi112.stdout: mon warn on too few osds = false 2024-09-21T00:27:47.704 INFO:teuthology.orchestra.run.smithi112.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-21T00:27:47.704 INFO:teuthology.orchestra.run.smithi112.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-21T00:27:47.704 INFO:teuthology.orchestra.run.smithi112.stdout: mon_allow_pool_size_one = true 2024-09-21T00:27:47.704 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:27:47.704 INFO:teuthology.orchestra.run.smithi112.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-21T00:27:47.704 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:27:47.705 INFO:teuthology.orchestra.run.smithi112.stdout: osd default data pool replay window = 5 2024-09-21T00:27:47.705 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:27:47.705 INFO:teuthology.orchestra.run.smithi112.stdout: mon allow pool delete = true 2024-09-21T00:27:47.705 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:27:47.705 INFO:teuthology.orchestra.run.smithi112.stdout: mon cluster log file level = debug 2024-09-21T00:27:47.705 INFO:teuthology.orchestra.run.smithi112.stdout: debug asserts on shutdown = true 2024-09-21T00:27:47.705 INFO:teuthology.orchestra.run.smithi112.stdout: mon health detail to clog = false 2024-09-21T00:27:47.705 INFO:teuthology.orchestra.run.smithi112.stdout: mon host = "172.21.15.112,172.21.15.187,[v2:172.21.15.187:3301,v1:172.21.15.187:6790]" 2024-09-21T00:27:47.705 INFO:teuthology.orchestra.run.smithi112.stdout: ms die on skipped message = False 2024-09-21T00:27:47.705 INFO:teuthology.orchestra.run.smithi112.stdout: fsid = 8b34692b-dc65-49b2-a08f-26f707b96ac5 2024-09-21T00:27:47.705 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:27:47.705 INFO:teuthology.orchestra.run.smithi112.stdout:[osd] 2024-09-21T00:27:47.705 INFO:teuthology.orchestra.run.smithi112.stdout: osd journal size = 100 2024-09-21T00:27:47.705 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:27:47.705 INFO:teuthology.orchestra.run.smithi112.stdout: osd scrub load threshold = 5.0 2024-09-21T00:27:47.706 INFO:teuthology.orchestra.run.smithi112.stdout: osd scrub max interval = 600 2024-09-21T00:27:47.706 INFO:teuthology.orchestra.run.smithi112.stdout: osd mclock profile = high_recovery_ops 2024-09-21T00:27:47.706 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:27:47.706 INFO:teuthology.orchestra.run.smithi112.stdout: osd recover clone overlap = true 2024-09-21T00:27:47.706 INFO:teuthology.orchestra.run.smithi112.stdout: osd recovery max chunk = 1048576 2024-09-21T00:27:47.706 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:27:47.706 INFO:teuthology.orchestra.run.smithi112.stdout: osd debug shutdown = true 2024-09-21T00:27:47.706 INFO:teuthology.orchestra.run.smithi112.stdout: osd debug op order = true 2024-09-21T00:27:47.706 INFO:teuthology.orchestra.run.smithi112.stdout: osd debug verify stray on activate = true 2024-09-21T00:27:47.706 INFO:teuthology.orchestra.run.smithi112.stdout: osd debug trim objects = true 2024-09-21T00:27:47.706 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:27:47.706 INFO:teuthology.orchestra.run.smithi112.stdout: osd open classes on start = true 2024-09-21T00:27:47.706 INFO:teuthology.orchestra.run.smithi112.stdout: osd debug pg log writeout = true 2024-09-21T00:27:47.706 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:27:47.706 INFO:teuthology.orchestra.run.smithi112.stdout: osd deep scrub update digest min age = 30 2024-09-21T00:27:47.707 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:27:47.707 INFO:teuthology.orchestra.run.smithi112.stdout: osd map max advance = 10 2024-09-21T00:27:47.707 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:27:47.707 INFO:teuthology.orchestra.run.smithi112.stdout: journal zero on create = true 2024-09-21T00:27:47.707 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:27:47.707 INFO:teuthology.orchestra.run.smithi112.stdout: filestore ondisk finisher threads = 3 2024-09-21T00:27:47.707 INFO:teuthology.orchestra.run.smithi112.stdout: filestore apply finisher threads = 3 2024-09-21T00:27:47.707 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:27:47.707 INFO:teuthology.orchestra.run.smithi112.stdout: bdev debug aio = true 2024-09-21T00:27:47.707 INFO:teuthology.orchestra.run.smithi112.stdout: osd debug misdirected ops = true 2024-09-21T00:27:47.707 INFO:teuthology.orchestra.run.smithi112.stdout: bluestore block size = 96636764160 2024-09-21T00:27:47.707 INFO:teuthology.orchestra.run.smithi112.stdout: bluestore fsck on mount = True 2024-09-21T00:27:47.707 INFO:teuthology.orchestra.run.smithi112.stdout: debug bluefs = 20 2024-09-21T00:27:47.707 INFO:teuthology.orchestra.run.smithi112.stdout: debug bluestore = 20 2024-09-21T00:27:47.708 INFO:teuthology.orchestra.run.smithi112.stdout: debug ms = 1 2024-09-21T00:27:47.708 INFO:teuthology.orchestra.run.smithi112.stdout: debug osd = 20 2024-09-21T00:27:47.708 INFO:teuthology.orchestra.run.smithi112.stdout: debug rocksdb = 10 2024-09-21T00:27:47.708 INFO:teuthology.orchestra.run.smithi112.stdout: mon osd backfillfull_ratio = 0.85 2024-09-21T00:27:47.708 INFO:teuthology.orchestra.run.smithi112.stdout: mon osd full ratio = 0.9 2024-09-21T00:27:47.708 INFO:teuthology.orchestra.run.smithi112.stdout: mon osd nearfull ratio = 0.8 2024-09-21T00:27:47.708 INFO:teuthology.orchestra.run.smithi112.stdout: osd failsafe full ratio = 0.95 2024-09-21T00:27:47.708 INFO:teuthology.orchestra.run.smithi112.stdout: osd objectstore = bluestore 2024-09-21T00:27:47.708 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:27:47.708 INFO:teuthology.orchestra.run.smithi112.stdout:[mgr] 2024-09-21T00:27:47.708 INFO:teuthology.orchestra.run.smithi112.stdout: debug ms = 1 2024-09-21T00:27:47.708 INFO:teuthology.orchestra.run.smithi112.stdout: debug mgr = 20 2024-09-21T00:27:47.708 INFO:teuthology.orchestra.run.smithi112.stdout: debug mon = 20 2024-09-21T00:27:47.708 INFO:teuthology.orchestra.run.smithi112.stdout: debug auth = 20 2024-09-21T00:27:47.708 INFO:teuthology.orchestra.run.smithi112.stdout: mon reweight min pgs per osd = 4 2024-09-21T00:27:47.708 INFO:teuthology.orchestra.run.smithi112.stdout: mon reweight min bytes per osd = 10 2024-09-21T00:27:47.709 INFO:teuthology.orchestra.run.smithi112.stdout: mgr/telemetry/nag = false 2024-09-21T00:27:47.709 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:27:47.709 INFO:teuthology.orchestra.run.smithi112.stdout:[mon] 2024-09-21T00:27:47.709 INFO:teuthology.orchestra.run.smithi112.stdout: debug ms = 1 2024-09-21T00:27:47.709 INFO:teuthology.orchestra.run.smithi112.stdout: debug mon = 20 2024-09-21T00:27:47.709 INFO:teuthology.orchestra.run.smithi112.stdout: debug paxos = 20 2024-09-21T00:27:47.709 INFO:teuthology.orchestra.run.smithi112.stdout: debug auth = 20 2024-09-21T00:27:47.709 INFO:teuthology.orchestra.run.smithi112.stdout: mon data avail warn = 5 2024-09-21T00:27:47.709 INFO:teuthology.orchestra.run.smithi112.stdout: mon mgr mkfs grace = 240 2024-09-21T00:27:47.709 INFO:teuthology.orchestra.run.smithi112.stdout: mon reweight min pgs per osd = 4 2024-09-21T00:27:47.709 INFO:teuthology.orchestra.run.smithi112.stdout: mon osd reporter subtree level = osd 2024-09-21T00:27:47.709 INFO:teuthology.orchestra.run.smithi112.stdout: mon osd prime pg temp = true 2024-09-21T00:27:47.709 INFO:teuthology.orchestra.run.smithi112.stdout: mon reweight min bytes per osd = 10 2024-09-21T00:27:47.709 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:27:47.709 INFO:teuthology.orchestra.run.smithi112.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-21T00:27:47.710 INFO:teuthology.orchestra.run.smithi112.stdout: auth mon ticket ttl = 660 # 11m 2024-09-21T00:27:47.710 INFO:teuthology.orchestra.run.smithi112.stdout: auth service ticket ttl = 240 # 4m 2024-09-21T00:27:47.710 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:27:47.710 INFO:teuthology.orchestra.run.smithi112.stdout: # don't complain about insecure global_id in the test suite 2024-09-21T00:27:47.710 INFO:teuthology.orchestra.run.smithi112.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-21T00:27:47.710 INFO:teuthology.orchestra.run.smithi112.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-21T00:27:47.710 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:27:47.710 INFO:teuthology.orchestra.run.smithi112.stdout: # 1m isn't quite enough 2024-09-21T00:27:47.710 INFO:teuthology.orchestra.run.smithi112.stdout: mon_down_mkfs_grace = 2m 2024-09-21T00:27:47.710 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:27:47.710 INFO:teuthology.orchestra.run.smithi112.stdout: mon_warn_on_filestore_osds = false 2024-09-21T00:27:47.710 INFO:teuthology.orchestra.run.smithi112.stdout: mon down mkfs grace = 300 2024-09-21T00:27:47.710 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:27:47.710 INFO:teuthology.orchestra.run.smithi112.stdout:[client] 2024-09-21T00:27:47.711 INFO:teuthology.orchestra.run.smithi112.stdout: rgw cache enabled = true 2024-09-21T00:27:47.711 INFO:teuthology.orchestra.run.smithi112.stdout: rgw enable ops log = true 2024-09-21T00:27:47.711 INFO:teuthology.orchestra.run.smithi112.stdout: rgw enable usage log = true 2024-09-21T00:27:47.711 INFO:teuthology.orchestra.run.smithi112.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-21T00:27:47.711 INFO:teuthology.orchestra.run.smithi112.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-21T00:27:47.711 INFO:teuthology.orchestra.run.smithi112.stdout: pid file = "" 2024-09-21T00:27:47.711 INFO:teuthology.orchestra.run.smithi112.stdout: client reconnect stale = True 2024-09-21T00:27:47.711 INFO:teuthology.orchestra.run.smithi112.stdout:[mon.a] 2024-09-21T00:27:47.711 INFO:teuthology.orchestra.run.smithi112.stdout:[mon.b] 2024-09-21T00:27:47.711 INFO:teuthology.orchestra.run.smithi112.stdout:[mon.c] 2024-09-21T00:27:47.711 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.a] 2024-09-21T00:27:47.711 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.c] 2024-09-21T00:27:47.711 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.b] 2024-09-21T00:27:47.711 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.d] 2024-09-21T00:27:47.712 INFO:teuthology.orchestra.run.smithi187.stdout:[global] 2024-09-21T00:27:47.712 INFO:teuthology.orchestra.run.smithi187.stdout: chdir = "" 2024-09-21T00:27:47.712 INFO:teuthology.orchestra.run.smithi187.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-21T00:27:47.712 INFO:teuthology.orchestra.run.smithi187.stdout: auth supported = cephx 2024-09-21T00:27:47.713 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:27:47.713 INFO:teuthology.orchestra.run.smithi187.stdout: filestore xattr use omap = true 2024-09-21T00:27:47.713 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:27:47.713 INFO:teuthology.orchestra.run.smithi187.stdout: mon clock drift allowed = 1.000 2024-09-21T00:27:47.713 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:27:47.713 INFO:teuthology.orchestra.run.smithi187.stdout: osd crush chooseleaf type = 0 2024-09-21T00:27:47.713 INFO:teuthology.orchestra.run.smithi187.stdout: auth debug = true 2024-09-21T00:27:47.713 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:27:47.713 INFO:teuthology.orchestra.run.smithi187.stdout: ms die on old message = true 2024-09-21T00:27:47.713 INFO:teuthology.orchestra.run.smithi187.stdout: ms die on bug = true 2024-09-21T00:27:47.713 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:27:47.713 INFO:teuthology.orchestra.run.smithi187.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-21T00:27:47.713 INFO:teuthology.orchestra.run.smithi187.stdout: mon pg warn max object skew = 0 2024-09-21T00:27:47.714 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:27:47.714 INFO:teuthology.orchestra.run.smithi187.stdout: # disable pg_autoscaler by default for new pools 2024-09-21T00:27:47.714 INFO:teuthology.orchestra.run.smithi187.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-21T00:27:47.714 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:27:47.714 INFO:teuthology.orchestra.run.smithi187.stdout: osd pool default size = 2 2024-09-21T00:27:47.714 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:27:47.714 INFO:teuthology.orchestra.run.smithi187.stdout: mon osd allow primary affinity = true 2024-09-21T00:27:47.714 INFO:teuthology.orchestra.run.smithi187.stdout: mon osd allow pg remap = true 2024-09-21T00:27:47.714 INFO:teuthology.orchestra.run.smithi187.stdout: mon warn on legacy crush tunables = false 2024-09-21T00:27:47.714 INFO:teuthology.orchestra.run.smithi187.stdout: mon warn on crush straw calc version zero = false 2024-09-21T00:27:47.714 INFO:teuthology.orchestra.run.smithi187.stdout: mon warn on no sortbitwise = false 2024-09-21T00:27:47.714 INFO:teuthology.orchestra.run.smithi187.stdout: mon warn on osd down out interval zero = false 2024-09-21T00:27:47.715 INFO:teuthology.orchestra.run.smithi187.stdout: mon warn on too few osds = false 2024-09-21T00:27:47.715 INFO:teuthology.orchestra.run.smithi187.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-21T00:27:47.715 INFO:teuthology.orchestra.run.smithi187.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-21T00:27:47.715 INFO:teuthology.orchestra.run.smithi187.stdout: mon_allow_pool_size_one = true 2024-09-21T00:27:47.715 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:27:47.715 INFO:teuthology.orchestra.run.smithi187.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-21T00:27:47.715 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:27:47.715 INFO:teuthology.orchestra.run.smithi187.stdout: osd default data pool replay window = 5 2024-09-21T00:27:47.715 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:27:47.715 INFO:teuthology.orchestra.run.smithi187.stdout: mon allow pool delete = true 2024-09-21T00:27:47.715 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:27:47.715 INFO:teuthology.orchestra.run.smithi187.stdout: mon cluster log file level = debug 2024-09-21T00:27:47.715 INFO:teuthology.orchestra.run.smithi187.stdout: debug asserts on shutdown = true 2024-09-21T00:27:47.716 INFO:teuthology.orchestra.run.smithi187.stdout: mon health detail to clog = false 2024-09-21T00:27:47.716 INFO:teuthology.orchestra.run.smithi187.stdout: mon host = "172.21.15.112,172.21.15.187,[v2:172.21.15.187:3301,v1:172.21.15.187:6790]" 2024-09-21T00:27:47.716 INFO:teuthology.orchestra.run.smithi187.stdout: ms die on skipped message = False 2024-09-21T00:27:47.716 INFO:teuthology.orchestra.run.smithi187.stdout: fsid = 8b34692b-dc65-49b2-a08f-26f707b96ac5 2024-09-21T00:27:47.716 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:27:47.716 INFO:teuthology.orchestra.run.smithi187.stdout:[osd] 2024-09-21T00:27:47.716 INFO:teuthology.orchestra.run.smithi187.stdout: osd journal size = 100 2024-09-21T00:27:47.716 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:27:47.716 INFO:teuthology.orchestra.run.smithi187.stdout: osd scrub load threshold = 5.0 2024-09-21T00:27:47.716 INFO:teuthology.orchestra.run.smithi187.stdout: osd scrub max interval = 600 2024-09-21T00:27:47.716 INFO:teuthology.orchestra.run.smithi187.stdout: osd mclock profile = high_recovery_ops 2024-09-21T00:27:47.716 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:27:47.717 INFO:teuthology.orchestra.run.smithi187.stdout: osd recover clone overlap = true 2024-09-21T00:27:47.717 INFO:teuthology.orchestra.run.smithi187.stdout: osd recovery max chunk = 1048576 2024-09-21T00:27:47.717 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:27:47.717 INFO:teuthology.orchestra.run.smithi187.stdout: osd debug shutdown = true 2024-09-21T00:27:47.717 INFO:teuthology.orchestra.run.smithi187.stdout: osd debug op order = true 2024-09-21T00:27:47.717 INFO:teuthology.orchestra.run.smithi187.stdout: osd debug verify stray on activate = true 2024-09-21T00:27:47.717 INFO:teuthology.orchestra.run.smithi187.stdout: osd debug trim objects = true 2024-09-21T00:27:47.717 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:27:47.717 INFO:teuthology.orchestra.run.smithi187.stdout: osd open classes on start = true 2024-09-21T00:27:47.717 INFO:teuthology.orchestra.run.smithi187.stdout: osd debug pg log writeout = true 2024-09-21T00:27:47.717 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:27:47.717 INFO:teuthology.orchestra.run.smithi187.stdout: osd deep scrub update digest min age = 30 2024-09-21T00:27:47.718 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:27:47.718 INFO:teuthology.orchestra.run.smithi187.stdout: osd map max advance = 10 2024-09-21T00:27:47.718 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:27:47.718 INFO:teuthology.orchestra.run.smithi187.stdout: journal zero on create = true 2024-09-21T00:27:47.718 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:27:47.718 INFO:teuthology.orchestra.run.smithi187.stdout: filestore ondisk finisher threads = 3 2024-09-21T00:27:47.718 INFO:teuthology.orchestra.run.smithi187.stdout: filestore apply finisher threads = 3 2024-09-21T00:27:47.718 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:27:47.718 INFO:teuthology.orchestra.run.smithi187.stdout: bdev debug aio = true 2024-09-21T00:27:47.718 INFO:teuthology.orchestra.run.smithi187.stdout: osd debug misdirected ops = true 2024-09-21T00:27:47.718 INFO:teuthology.orchestra.run.smithi187.stdout: bluestore block size = 96636764160 2024-09-21T00:27:47.718 INFO:teuthology.orchestra.run.smithi187.stdout: bluestore fsck on mount = True 2024-09-21T00:27:47.718 INFO:teuthology.orchestra.run.smithi187.stdout: debug bluefs = 20 2024-09-21T00:27:47.719 INFO:teuthology.orchestra.run.smithi187.stdout: debug bluestore = 20 2024-09-21T00:27:47.719 INFO:teuthology.orchestra.run.smithi187.stdout: debug ms = 1 2024-09-21T00:27:47.719 INFO:teuthology.orchestra.run.smithi187.stdout: debug osd = 20 2024-09-21T00:27:47.719 INFO:teuthology.orchestra.run.smithi187.stdout: debug rocksdb = 10 2024-09-21T00:27:47.719 INFO:teuthology.orchestra.run.smithi187.stdout: mon osd backfillfull_ratio = 0.85 2024-09-21T00:27:47.719 INFO:teuthology.orchestra.run.smithi187.stdout: mon osd full ratio = 0.9 2024-09-21T00:27:47.719 INFO:teuthology.orchestra.run.smithi187.stdout: mon osd nearfull ratio = 0.8 2024-09-21T00:27:47.719 INFO:teuthology.orchestra.run.smithi187.stdout: osd failsafe full ratio = 0.95 2024-09-21T00:27:47.719 INFO:teuthology.orchestra.run.smithi187.stdout: osd objectstore = bluestore 2024-09-21T00:27:47.719 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:27:47.719 INFO:teuthology.orchestra.run.smithi187.stdout:[mgr] 2024-09-21T00:27:47.719 INFO:teuthology.orchestra.run.smithi187.stdout: debug ms = 1 2024-09-21T00:27:47.719 INFO:teuthology.orchestra.run.smithi187.stdout: debug mgr = 20 2024-09-21T00:27:47.720 INFO:teuthology.orchestra.run.smithi187.stdout: debug mon = 20 2024-09-21T00:27:47.720 INFO:teuthology.orchestra.run.smithi187.stdout: debug auth = 20 2024-09-21T00:27:47.720 INFO:teuthology.orchestra.run.smithi187.stdout: mon reweight min pgs per osd = 4 2024-09-21T00:27:47.720 INFO:teuthology.orchestra.run.smithi187.stdout: mon reweight min bytes per osd = 10 2024-09-21T00:27:47.720 INFO:teuthology.orchestra.run.smithi187.stdout: mgr/telemetry/nag = false 2024-09-21T00:27:47.720 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:27:47.720 INFO:teuthology.orchestra.run.smithi187.stdout:[mon] 2024-09-21T00:27:47.720 INFO:teuthology.orchestra.run.smithi187.stdout: debug ms = 1 2024-09-21T00:27:47.720 INFO:teuthology.orchestra.run.smithi187.stdout: debug mon = 20 2024-09-21T00:27:47.720 INFO:teuthology.orchestra.run.smithi187.stdout: debug paxos = 20 2024-09-21T00:27:47.720 INFO:teuthology.orchestra.run.smithi187.stdout: debug auth = 20 2024-09-21T00:27:47.720 INFO:teuthology.orchestra.run.smithi187.stdout: mon data avail warn = 5 2024-09-21T00:27:47.721 INFO:teuthology.orchestra.run.smithi187.stdout: mon mgr mkfs grace = 240 2024-09-21T00:27:47.721 INFO:teuthology.orchestra.run.smithi187.stdout: mon reweight min pgs per osd = 4 2024-09-21T00:27:47.721 INFO:teuthology.orchestra.run.smithi187.stdout: mon osd reporter subtree level = osd 2024-09-21T00:27:47.721 INFO:teuthology.orchestra.run.smithi187.stdout: mon osd prime pg temp = true 2024-09-21T00:27:47.721 INFO:teuthology.orchestra.run.smithi187.stdout: mon reweight min bytes per osd = 10 2024-09-21T00:27:47.721 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:27:47.721 INFO:teuthology.orchestra.run.smithi187.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-21T00:27:47.721 INFO:teuthology.orchestra.run.smithi187.stdout: auth mon ticket ttl = 660 # 11m 2024-09-21T00:27:47.721 INFO:teuthology.orchestra.run.smithi187.stdout: auth service ticket ttl = 240 # 4m 2024-09-21T00:27:47.721 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:27:47.721 INFO:teuthology.orchestra.run.smithi187.stdout: # don't complain about insecure global_id in the test suite 2024-09-21T00:27:47.721 INFO:teuthology.orchestra.run.smithi187.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-21T00:27:47.721 INFO:teuthology.orchestra.run.smithi187.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-21T00:27:47.722 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:27:47.722 INFO:teuthology.orchestra.run.smithi187.stdout: # 1m isn't quite enough 2024-09-21T00:27:47.722 INFO:teuthology.orchestra.run.smithi187.stdout: mon_down_mkfs_grace = 2m 2024-09-21T00:27:47.722 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:27:47.722 INFO:teuthology.orchestra.run.smithi187.stdout: mon_warn_on_filestore_osds = false 2024-09-21T00:27:47.722 INFO:teuthology.orchestra.run.smithi187.stdout: mon down mkfs grace = 300 2024-09-21T00:27:47.722 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:27:47.722 INFO:teuthology.orchestra.run.smithi187.stdout:[client] 2024-09-21T00:27:47.722 INFO:teuthology.orchestra.run.smithi187.stdout: rgw cache enabled = true 2024-09-21T00:27:47.722 INFO:teuthology.orchestra.run.smithi187.stdout: rgw enable ops log = true 2024-09-21T00:27:47.722 INFO:teuthology.orchestra.run.smithi187.stdout: rgw enable usage log = true 2024-09-21T00:27:47.722 INFO:teuthology.orchestra.run.smithi187.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-21T00:27:47.723 INFO:teuthology.orchestra.run.smithi187.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-21T00:27:47.723 INFO:teuthology.orchestra.run.smithi187.stdout: pid file = "" 2024-09-21T00:27:47.723 INFO:teuthology.orchestra.run.smithi187.stdout: client reconnect stale = True 2024-09-21T00:27:47.723 INFO:teuthology.orchestra.run.smithi187.stdout:[mon.a] 2024-09-21T00:27:47.723 INFO:teuthology.orchestra.run.smithi187.stdout:[mon.b] 2024-09-21T00:27:47.723 INFO:teuthology.orchestra.run.smithi187.stdout:[mon.c] 2024-09-21T00:27:47.723 INFO:teuthology.orchestra.run.smithi187.stdout:[mds.a] 2024-09-21T00:27:47.723 INFO:teuthology.orchestra.run.smithi187.stdout:[mds.c] 2024-09-21T00:27:47.723 INFO:teuthology.orchestra.run.smithi187.stdout:[mds.b] 2024-09-21T00:27:47.723 INFO:teuthology.orchestra.run.smithi187.stdout:[mds.d] 2024-09-21T00:27:47.725 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T00:27:47.725 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:27:47.781 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T00:27:47.781 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:27:47.836 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T00:27:47.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.987+0000 7f4014119640 1 -- 172.21.15.112:0/2510728241 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f400c0ccce0 msgr2=0x7f400c0c65e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:47.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.987+0000 7f4014119640 1 --2- 172.21.15.112:0/2510728241 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f400c0ccce0 0x7f400c0c65e0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f4004005fd0 tx=0x7f400402a940 comp rx=0 tx=0).stop 2024-09-21T00:27:47.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.987+0000 7f4014119640 1 -- 172.21.15.112:0/2510728241 shutdown_connections 2024-09-21T00:27:47.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.987+0000 7f4014119640 1 --2- 172.21.15.112:0/2510728241 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f400c0ccce0 0x7f400c0c65e0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:47.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.987+0000 7f4014119640 1 -- 172.21.15.112:0/2510728241 >> 172.21.15.112:0/2510728241 conn(0x7f400c0cf220 msgr2=0x7f400c0cf630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:47.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.987+0000 7f4014119640 1 -- 172.21.15.112:0/2510728241 shutdown_connections 2024-09-21T00:27:47.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.987+0000 7f4014119640 1 -- 172.21.15.112:0/2510728241 wait complete. 2024-09-21T00:27:47.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.989+0000 7f4014119640 1 Processor -- start 2024-09-21T00:27:47.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.989+0000 7f4014119640 1 -- start start 2024-09-21T00:27:47.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.990+0000 7f4014119640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f400c0ccce0 0x7f400c15f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:47.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.990+0000 7f4014119640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f400c15ca50 0x7f400c0c9430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:47.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.990+0000 7f4014119640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f400c0b10a0 0x7f400c159180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:47.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.990+0000 7f4014119640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f400c0d2130 con 0x7f400c0b10a0 2024-09-21T00:27:47.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.990+0000 7f4014119640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f400c0d1fb0 con 0x7f400c0ccce0 2024-09-21T00:27:47.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.990+0000 7f4014119640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f400c0c6b20 con 0x7f400c15ca50 2024-09-21T00:27:47.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.990+0000 7f40126b6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f400c0b10a0 0x7f400c159180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:47.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.991+0000 7f40126b6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f400c0b10a0 0x7f400c159180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57298/0 (socket says 172.21.15.112:57298) 2024-09-21T00:27:47.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.991+0000 7f40126b6640 1 -- 172.21.15.112:0/1946692915 learned_addr learned my addr 172.21.15.112:0/1946692915 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:27:47.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.991+0000 7f40116b4640 1 --2- 172.21.15.112:0/1946692915 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f400c15ca50 0x7f400c0c9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:47.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.991+0000 7f4011eb5640 1 --2- 172.21.15.112:0/1946692915 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f400c0ccce0 0x7f400c15f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:47.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.991+0000 7f40126b6640 1 -- 172.21.15.112:0/1946692915 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f400c15ca50 msgr2=0x7f400c0c9430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:47.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.991+0000 7f40126b6640 1 --2- 172.21.15.112:0/1946692915 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f400c15ca50 0x7f400c0c9430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:47.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.991+0000 7f40126b6640 1 -- 172.21.15.112:0/1946692915 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f400c0ccce0 msgr2=0x7f400c15f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:47.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.991+0000 7f40126b6640 1 --2- 172.21.15.112:0/1946692915 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f400c0ccce0 0x7f400c15f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:47.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.991+0000 7f40126b6640 1 -- 172.21.15.112:0/1946692915 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f400c15d100 con 0x7f400c0b10a0 2024-09-21T00:27:47.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.991+0000 7f40126b6640 1 --2- 172.21.15.112:0/1946692915 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f400c0b10a0 0x7f400c159180 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f4004009cd0 tx=0x7f4004031ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:47.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.991+0000 7f40027fc640 1 -- 172.21.15.112:0/1946692915 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f400403e070 con 0x7f400c0b10a0 2024-09-21T00:27:47.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.991+0000 7f40027fc640 1 -- 172.21.15.112:0/1946692915 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4004004560 con 0x7f400c0b10a0 2024-09-21T00:27:47.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.991+0000 7f4014119640 1 -- 172.21.15.112:0/1946692915 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f400c15d3e0 con 0x7f400c0b10a0 2024-09-21T00:27:47.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.991+0000 7f40027fc640 1 -- 172.21.15.112:0/1946692915 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4004043460 con 0x7f400c0b10a0 2024-09-21T00:27:47.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.991+0000 7f4014119640 1 -- 172.21.15.112:0/1946692915 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f400c0c9970 con 0x7f400c0b10a0 2024-09-21T00:27:47.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.992+0000 7f4014119640 1 -- 172.21.15.112:0/1946692915 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f400c15ce90 con 0x7f400c0b10a0 2024-09-21T00:27:47.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.993+0000 7f40027fc640 1 -- 172.21.15.112:0/1946692915 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f400403a440 con 0x7f400c0b10a0 2024-09-21T00:27:47.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.994+0000 7f40027fc640 1 --2- 172.21.15.112:0/1946692915 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f3ff0078b90 0x7f3ff007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:47.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.994+0000 7f40027fc640 1 -- 172.21.15.112:0/1946692915 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7f40040bc9c0 con 0x7f400c0b10a0 2024-09-21T00:27:47.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.994+0000 7f4011eb5640 1 --2- 172.21.15.112:0/1946692915 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f3ff0078b90 0x7f3ff007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:47.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.994+0000 7f4011eb5640 1 --2- 172.21.15.112:0/1946692915 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f3ff0078b90 0x7f3ff007b0a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f4008001720 tx=0x7f4008009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:47.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:47.998+0000 7f40027fc640 1 -- 172.21.15.112:0/1946692915 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f40040bee20 con 0x7f400c0b10a0 2024-09-21T00:27:48.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:48.188+0000 7f4014119640 1 -- 172.21.15.112:0/1946692915 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f400c15fe00 con 0x7f400c0b10a0 2024-09-21T00:27:48.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:48.188+0000 7f40027fc640 1 -- 172.21.15.112:0/1946692915 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v60) ==== 80+0+0 (secure 0 0 0) 0x7f40040846b0 con 0x7f400c0b10a0 2024-09-21T00:27:48.190 INFO:teuthology.orchestra.run.smithi112.stderr:listed 0 entries 2024-09-21T00:27:48.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:48.191+0000 7f4014119640 1 -- 172.21.15.112:0/1946692915 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f3ff0078b90 msgr2=0x7f3ff007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:48.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:48.191+0000 7f4014119640 1 --2- 172.21.15.112:0/1946692915 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f3ff0078b90 0x7f3ff007b0a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f4008001720 tx=0x7f4008009040 comp rx=0 tx=0).stop 2024-09-21T00:27:48.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:48.191+0000 7f4014119640 1 -- 172.21.15.112:0/1946692915 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f400c0b10a0 msgr2=0x7f400c159180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:48.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:48.191+0000 7f4014119640 1 --2- 172.21.15.112:0/1946692915 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f400c0b10a0 0x7f400c159180 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f4004009cd0 tx=0x7f4004031ea0 comp rx=0 tx=0).stop 2024-09-21T00:27:48.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:48.191+0000 7f4014119640 1 -- 172.21.15.112:0/1946692915 shutdown_connections 2024-09-21T00:27:48.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:48.191+0000 7f4014119640 1 --2- 172.21.15.112:0/1946692915 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f3ff0078b90 0x7f3ff007b0a0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:48.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:48.191+0000 7f4014119640 1 --2- 172.21.15.112:0/1946692915 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f400c0b10a0 0x7f400c159180 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:48.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:48.191+0000 7f4014119640 1 --2- 172.21.15.112:0/1946692915 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f400c15ca50 0x7f400c0c9430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:48.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:48.191+0000 7f4014119640 1 --2- 172.21.15.112:0/1946692915 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f400c0ccce0 0x7f400c15f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:48.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:48.191+0000 7f4014119640 1 -- 172.21.15.112:0/1946692915 >> 172.21.15.112:0/1946692915 conn(0x7f400c0cf220 msgr2=0x7f400c0c73f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:48.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:48.191+0000 7f4014119640 1 -- 172.21.15.112:0/1946692915 shutdown_connections 2024-09-21T00:27:48.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:48.191+0000 7f4014119640 1 -- 172.21.15.112:0/1946692915 wait complete. 2024-09-21T00:27:48.203 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-21T00:27:48.204 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:27:48.204 DEBUG:teuthology.orchestra.run.smithi112:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-21T00:27:48.232 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi112.front.sepia.ceph.com 2024-09-21T00:27:48.232 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:27:48.232 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T00:27:48.288 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T00:27:48.288 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:27:48.344 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056010,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1439 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248716k,nr_inodes=812179,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T00:27:48.344 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T00:27:48.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:48.495+0000 7f840524d640 1 -- 172.21.15.112:0/1267975342 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83e8003ae0 msgr2=0x7f83e8003f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:48.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:48.495+0000 7f840524d640 1 --2- 172.21.15.112:0/1267975342 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83e8003ae0 0x7f83e8003f50 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f84000d44f0 tx=0x7f83e8005830 comp rx=0 tx=0).stop 2024-09-21T00:27:48.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:48.495+0000 7f83ff59e640 1 -- 172.21.15.112:0/1267975342 reap_dead start 2024-09-21T00:27:48.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:48.495+0000 7f840524d640 1 -- 172.21.15.112:0/1267975342 shutdown_connections 2024-09-21T00:27:48.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:48.495+0000 7f840524d640 1 -- 172.21.15.112:0/1267975342 >> 172.21.15.112:0/1267975342 conn(0x7f840008bfe0 msgr2=0x7f840009d890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:48.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:48.495+0000 7f840524d640 1 -- 172.21.15.112:0/1267975342 shutdown_connections 2024-09-21T00:27:48.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:48.495+0000 7f840524d640 1 -- 172.21.15.112:0/1267975342 wait complete. 2024-09-21T00:27:48.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:48.497+0000 7f840524d640 1 Processor -- start 2024-09-21T00:27:48.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:48.497+0000 7f840524d640 1 -- start start 2024-09-21T00:27:48.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:48.499+0000 7f840524d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f83e8003ae0 0x7f84000caa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:48.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:48.499+0000 7f840524d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8400162050 0x7f8400154a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:48.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:48.499+0000 7f840524d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f84000b2cb0 0x7f84000b30c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:48.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:48.499+0000 7f840524d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f84000d4150 con 0x7f8400162050 2024-09-21T00:27:48.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:48.499+0000 7f840524d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f84000d3fd0 con 0x7f83e8003ae0 2024-09-21T00:27:48.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:48.499+0000 7f840524d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f84000d20d0 con 0x7f84000b2cb0 2024-09-21T00:27:48.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:48.499+0000 7f83fe59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8400162050 0x7f8400154a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:48.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:48.499+0000 7f83fe59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8400162050 0x7f8400154a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57314/0 (socket says 172.21.15.112:57314) 2024-09-21T00:27:48.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:48.499+0000 7f83fe59c640 1 -- 172.21.15.112:0/1059112337 learned_addr learned my addr 172.21.15.112:0/1059112337 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:27:48.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:48.499+0000 7f83fed9d640 1 --2- 172.21.15.112:0/1059112337 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f83e8003ae0 0x7f84000caa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:48.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:48.499+0000 7f83ff59e640 1 --2- 172.21.15.112:0/1059112337 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f84000b2cb0 0x7f84000b30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:48.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:48.499+0000 7f83fe59c640 1 -- 172.21.15.112:0/1059112337 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f84000b2cb0 msgr2=0x7f84000b30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:48.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:48.499+0000 7f83fe59c640 1 --2- 172.21.15.112:0/1059112337 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f84000b2cb0 0x7f84000b30c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:48.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:48.499+0000 7f83fe59c640 1 -- 172.21.15.112:0/1059112337 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f83e8003ae0 msgr2=0x7f84000caa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:48.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:48.499+0000 7f83fe59c640 1 --2- 172.21.15.112:0/1059112337 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f83e8003ae0 0x7f84000caa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:48.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:48.499+0000 7f83fe59c640 1 -- 172.21.15.112:0/1059112337 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84000b3600 con 0x7f8400162050 2024-09-21T00:27:48.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:48.499+0000 7f83fe59c640 1 --2- 172.21.15.112:0/1059112337 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8400162050 0x7f8400154a70 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f83f4011ae0 tx=0x7f83f4011ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:48.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:48.499+0000 7f83e77fe640 1 -- 172.21.15.112:0/1059112337 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83f400fcb0 con 0x7f8400162050 2024-09-21T00:27:48.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:48.499+0000 7f83e77fe640 1 -- 172.21.15.112:0/1059112337 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f83f400fe50 con 0x7f8400162050 2024-09-21T00:27:48.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:48.499+0000 7f840524d640 1 -- 172.21.15.112:0/1059112337 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f840006fb60 con 0x7f8400162050 2024-09-21T00:27:48.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:48.499+0000 7f83e77fe640 1 -- 172.21.15.112:0/1059112337 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83f4005280 con 0x7f8400162050 2024-09-21T00:27:48.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:48.499+0000 7f840524d640 1 -- 172.21.15.112:0/1059112337 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84000c87d0 con 0x7f8400162050 2024-09-21T00:27:48.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:48.501+0000 7f840524d640 1 -- 172.21.15.112:0/1059112337 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84000c8a10 con 0x7f8400162050 2024-09-21T00:27:48.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:48.501+0000 7f83e77fe640 1 -- 172.21.15.112:0/1059112337 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f83f4020020 con 0x7f8400162050 2024-09-21T00:27:48.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:48.502+0000 7f83e77fe640 1 --2- 172.21.15.112:0/1059112337 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f83ec078b90 0x7f83ec07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:48.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:48.502+0000 7f83e77fe640 1 -- 172.21.15.112:0/1059112337 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7f83f409b5f0 con 0x7f8400162050 2024-09-21T00:27:48.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:48.502+0000 7f83fed9d640 1 --2- 172.21.15.112:0/1059112337 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f83ec078b90 0x7f83ec07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:48.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:48.503+0000 7f83fed9d640 1 --2- 172.21.15.112:0/1059112337 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f83ec078b90 0x7f83ec07b0a0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f83f0035000 tx=0x7f83f002d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:48.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:48.506+0000 7f83e77fe640 1 -- 172.21.15.112:0/1059112337 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f84000c8a10 con 0x7f8400162050 2024-09-21T00:27:48.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:48.718+0000 7f840524d640 1 -- 172.21.15.112:0/1059112337 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f8400163260 con 0x7f8400162050 2024-09-21T00:27:48.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:48.719+0000 7f83e77fe640 1 -- 172.21.15.112:0/1059112337 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v38) ==== 53+0+73 (secure 0 0 0) 0x7f84000b3600 con 0x7f8400162050 2024-09-21T00:27:48.721 INFO:teuthology.orchestra.run.smithi112.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T00:27:48.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:48.721+0000 7f840524d640 1 -- 172.21.15.112:0/1059112337 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f83ec078b90 msgr2=0x7f83ec07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:48.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:48.721+0000 7f840524d640 1 --2- 172.21.15.112:0/1059112337 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f83ec078b90 0x7f83ec07b0a0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f83f0035000 tx=0x7f83f002d040 comp rx=0 tx=0).stop 2024-09-21T00:27:48.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:48.721+0000 7f840524d640 1 -- 172.21.15.112:0/1059112337 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8400162050 msgr2=0x7f8400154a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:48.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:48.721+0000 7f840524d640 1 --2- 172.21.15.112:0/1059112337 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8400162050 0x7f8400154a70 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f83f4011ae0 tx=0x7f83f4011ee0 comp rx=0 tx=0).stop 2024-09-21T00:27:48.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:48.721+0000 7f840524d640 1 -- 172.21.15.112:0/1059112337 shutdown_connections 2024-09-21T00:27:48.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:48.721+0000 7f840524d640 1 --2- 172.21.15.112:0/1059112337 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f84000b2cb0 0x7f84000b30c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:48.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:48.721+0000 7f840524d640 1 --2- 172.21.15.112:0/1059112337 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f83ec078b90 0x7f83ec07b0a0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:48.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:48.721+0000 7f840524d640 1 --2- 172.21.15.112:0/1059112337 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8400162050 0x7f8400154a70 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:48.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:48.721+0000 7f840524d640 1 --2- 172.21.15.112:0/1059112337 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f83e8003ae0 0x7f84000caa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:48.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:48.721+0000 7f840524d640 1 -- 172.21.15.112:0/1059112337 >> 172.21.15.112:0/1059112337 conn(0x7f840008bfe0 msgr2=0x7f84000b5050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:48.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:48.721+0000 7f840524d640 1 -- 172.21.15.112:0/1059112337 shutdown_connections 2024-09-21T00:27:48.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:48.721+0000 7f840524d640 1 -- 172.21.15.112:0/1059112337 wait complete. 2024-09-21T00:27:48.734 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T00:27:48.734 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T00:27:48.734 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi112.front.sepia.ceph.com 2024-09-21T00:27:48.735 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T00:27:48.735 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T00:27:48.735 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T00:27:48.735 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-21T00:27:48.735 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T00:27:48.735 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T00:27:48.735 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:27:48.735 DEBUG:teuthology.orchestra.run.smithi112:> ip addr 2024-09-21T00:27:48.751 INFO:teuthology.orchestra.run.smithi112.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T00:27:48.751 INFO:teuthology.orchestra.run.smithi112.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T00:27:48.751 INFO:teuthology.orchestra.run.smithi112.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T00:27:48.751 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:27:48.751 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 ::1/128 scope host 2024-09-21T00:27:48.751 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:27:48.751 INFO:teuthology.orchestra.run.smithi112.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T00:27:48.751 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:d8:a5:2c brd ff:ff:ff:ff:ff:ff 2024-09-21T00:27:48.752 INFO:teuthology.orchestra.run.smithi112.stdout: altname enp7s0f0 2024-09-21T00:27:48.752 INFO:teuthology.orchestra.run.smithi112.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T00:27:48.752 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:d8:a5:2d brd ff:ff:ff:ff:ff:ff 2024-09-21T00:27:48.752 INFO:teuthology.orchestra.run.smithi112.stdout: altname enp7s0f1 2024-09-21T00:27:48.752 INFO:teuthology.orchestra.run.smithi112.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T00:27:48.752 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:8f:cb:68 brd ff:ff:ff:ff:ff:ff 2024-09-21T00:27:48.752 INFO:teuthology.orchestra.run.smithi112.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T00:27:48.752 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:8f:cb:69 brd ff:ff:ff:ff:ff:ff 2024-09-21T00:27:48.752 INFO:teuthology.orchestra.run.smithi112.stdout: inet 172.21.15.112/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-21T00:27:48.752 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft 42653sec preferred_lft 42653sec 2024-09-21T00:27:48.752 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::ec4:7aff:fe8f:cb69/64 scope link noprefixroute 2024-09-21T00:27:48.752 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:27:48.752 INFO:teuthology.orchestra.run.smithi112.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T00:27:48.752 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 1a:39:64:e7:fe:d7 brd ff:ff:ff:ff:ff:ff 2024-09-21T00:27:48.753 INFO:teuthology.orchestra.run.smithi112.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T00:27:48.753 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:27:48.753 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::5858:3dff:feaf:1774/64 scope link 2024-09-21T00:27:48.753 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:27:48.753 INFO:teuthology.orchestra.run.smithi112.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T00:27:48.753 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 1a:39:64:e7:fe:d7 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T00:27:48.753 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::1839:64ff:fee7:fed7/64 scope link 2024-09-21T00:27:48.753 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:27:48.753 INFO:teuthology.orchestra.run.smithi112.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T00:27:48.753 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 86:c9:6c:ef:e4:e9 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T00:27:48.753 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::84c9:6cff:feef:e4e9/64 scope link 2024-09-21T00:27:48.753 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:27:48.753 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:27:48.754 DEBUG:teuthology.orchestra.run.smithi112:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T00:27:48.816 INFO:teuthology.orchestra.run.smithi112.stdout:1 2024-09-21T00:27:48.818 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:27:48.818 DEBUG:teuthology.orchestra.run.smithi112:> route 2024-09-21T00:27:48.874 INFO:teuthology.orchestra.run.smithi112.stdout:Kernel IP routing table 2024-09-21T00:27:48.874 INFO:teuthology.orchestra.run.smithi112.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T00:27:48.874 INFO:teuthology.orchestra.run.smithi112.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-21T00:27:48.874 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-21T00:27:48.874 INFO:teuthology.orchestra.run.smithi112.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T00:27:48.874 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T00:27:48.874 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2024-09-21T00:27:48.875 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-21T00:27:48.875 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-21T00:27:48.875 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-21T00:27:48.875 DEBUG:teuthology.orchestra.run.smithi112:> ') 2024-09-21T00:27:48.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:48 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T00:27:48.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T00:27:49.001 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:27:49.002 DEBUG:teuthology.orchestra.run.smithi112:> ip netns list 2024-09-21T00:27:49.017 INFO:teuthology.orchestra.run.smithi112.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-21T00:27:49.017 INFO:teuthology.orchestra.run.smithi112.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T00:27:49.017 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T00:27:49.017 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:27:49.017 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link set brx.0 up 2024-09-21T00:27:49.082 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T00:27:49.082 DEBUG:teuthology.orchestra.run.smithi112:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T00:27:49.138 INFO:teuthology.orchestra.run.smithi112.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T00:27:49.138 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T00:27:49.138 DEBUG:teuthology.orchestra.run.smithi112:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T00:27:49.194 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-21T00:27:49.194 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:27:49.194 DEBUG:teuthology.orchestra.run.smithi112:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T00:27:49.288 INFO:teuthology.orchestra.run.smithi112.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T00:27:49.288 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T00:27:49.289 INFO:teuthology.orchestra.run.smithi112.stdout:invalid new device string format 2024-09-21T00:27:49.299 INFO:teuthology.orchestra.run.smithi112.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T00:27:49.299 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T00:27:49.299 INFO:teuthology.orchestra.run.smithi112.stdout:invalid new device string format 2024-09-21T00:27:49.299 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: resolved to: "172.21.15.112:3300,172.21.15.187:3300,172.21.15.187:3301" 2024-09-21T00:27:49.299 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T00:27:49.305 INFO:teuthology.orchestra.run.smithi112.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T00:27:49.305 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T00:27:49.305 INFO:teuthology.orchestra.run.smithi112.stdout:invalid new device string format 2024-09-21T00:27:49.305 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: resolved to: "172.21.15.112:3300,172.21.15.187:3300,172.21.15.187:3301" 2024-09-21T00:27:49.305 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T00:27:49.305 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: resolved to: "172.21.15.112:6789,172.21.15.187:6789,172.21.15.187:6790" 2024-09-21T00:27:49.306 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.112:6789,172.21.15.187:6789,172.21.15.187:6790:/ 2024-09-21T00:27:49.306 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=8b34692b-dc65-49b2-a08f-26f707b96ac5" will pass to kernel 2024-09-21T00:27:49.306 INFO:teuthology.orchestra.run.smithi112.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-21T00:27:49.306 INFO:teuthology.orchestra.run.smithi112.stdout: You just mounted a file system that supports labels which does not 2024-09-21T00:27:49.306 INFO:teuthology.orchestra.run.smithi112.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T00:27:49.306 INFO:teuthology.orchestra.run.smithi112.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T00:27:49.306 INFO:teuthology.orchestra.run.smithi112.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T00:27:49.308 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T00:27:49.308 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:27:49.308 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T00:27:49.332 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T00:27:49.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:49 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T00:27:49.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T00:27:49.428 INFO:teuthology.orchestra.run.smithi112.stdout:client5985 2024-09-21T00:27:49.428 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T00:27:49.428 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:27:49.444 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-09-21T00:27:49.445 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:27:49.464 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-09-21T00:27:49.464 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:27:49.522 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T00:27:49.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:49.633+0000 7f909b24d640 1 -- 172.21.15.112:0/2201112752 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90940d2030 msgr2=0x7f909415d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:49.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:49.633+0000 7f909b24d640 1 --2- 172.21.15.112:0/2201112752 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90940d2030 0x7f909415d870 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f9088009830 tx=0x7f908802a5c0 comp rx=0 tx=0).stop 2024-09-21T00:27:49.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:49.633+0000 7f909b24d640 1 -- 172.21.15.112:0/2201112752 shutdown_connections 2024-09-21T00:27:49.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:49.633+0000 7f909b24d640 1 --2- 172.21.15.112:0/2201112752 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90940d2030 0x7f909415d870 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:49.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:49.633+0000 7f909b24d640 1 -- 172.21.15.112:0/2201112752 >> 172.21.15.112:0/2201112752 conn(0x7f90940b43f0 msgr2=0x7f90940b4830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:49.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:49.633+0000 7f909b24d640 1 -- 172.21.15.112:0/2201112752 shutdown_connections 2024-09-21T00:27:49.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:49.633+0000 7f909b24d640 1 -- 172.21.15.112:0/2201112752 wait complete. 2024-09-21T00:27:49.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:49.635+0000 7f909b24d640 1 Processor -- start 2024-09-21T00:27:49.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:49.635+0000 7f909b24d640 1 -- start start 2024-09-21T00:27:49.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:49.636+0000 7f909b24d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f90940ccae0 0x7f90941570e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:49.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:49.637+0000 7f909b24d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90940d0860 0x7f90941590f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:49.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:49.637+0000 7f909b24d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f90940d2030 0x7f9094159630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:49.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:49.637+0000 7f909b24d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9094167c70 con 0x7f90940d0860 2024-09-21T00:27:49.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:49.637+0000 7f909b24d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9094167af0 con 0x7f90940d2030 2024-09-21T00:27:49.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:49.637+0000 7f909b24d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f9094167df0 con 0x7f90940ccae0 2024-09-21T00:27:49.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:49.637+0000 7f9093fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90940d0860 0x7f90941590f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:49.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:49.637+0000 7f9093fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90940d0860 0x7f90941590f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57326/0 (socket says 172.21.15.112:57326) 2024-09-21T00:27:49.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:49.637+0000 7f9093fff640 1 -- 172.21.15.112:0/3546237725 learned_addr learned my addr 172.21.15.112:0/3546237725 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:27:49.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:49.637+0000 7f9098fe9640 1 --2- 172.21.15.112:0/3546237725 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f90940ccae0 0x7f90941570e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:49.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:49.637+0000 7f90997ea640 1 --2- 172.21.15.112:0/3546237725 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f90940d2030 0x7f9094159630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:49.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:49.637+0000 7f9093fff640 1 -- 172.21.15.112:0/3546237725 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f90940ccae0 msgr2=0x7f90941570e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:49.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:49.637+0000 7f9093fff640 1 --2- 172.21.15.112:0/3546237725 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f90940ccae0 0x7f90941570e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:49.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:49.637+0000 7f9093fff640 1 -- 172.21.15.112:0/3546237725 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f90940d2030 msgr2=0x7f9094159630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:49.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:49.637+0000 7f9093fff640 1 --2- 172.21.15.112:0/3546237725 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f90940d2030 0x7f9094159630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:49.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:49.637+0000 7f9093fff640 1 -- 172.21.15.112:0/3546237725 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90940cafd0 con 0x7f90940d0860 2024-09-21T00:27:49.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:49.637+0000 7f9093fff640 1 --2- 172.21.15.112:0/3546237725 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90940d0860 0x7f90941590f0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f908000c340 tx=0x7f908000c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:49.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:49.637+0000 7f90917fa640 1 -- 172.21.15.112:0/3546237725 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9080017070 con 0x7f90940d0860 2024-09-21T00:27:49.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:49.637+0000 7f90917fa640 1 -- 172.21.15.112:0/3546237725 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f90800053c0 con 0x7f90940d0860 2024-09-21T00:27:49.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:49.637+0000 7f90917fa640 1 -- 172.21.15.112:0/3546237725 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9080003850 con 0x7f90940d0860 2024-09-21T00:27:49.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:49.637+0000 7f909b24d640 1 -- 172.21.15.112:0/3546237725 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90940cb2b0 con 0x7f90940d0860 2024-09-21T00:27:49.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:49.637+0000 7f909b24d640 1 -- 172.21.15.112:0/3546237725 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90940c9f80 con 0x7f90940d0860 2024-09-21T00:27:49.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:49.638+0000 7f909b24d640 1 -- 172.21.15.112:0/3546237725 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f909415d870 con 0x7f90940d0860 2024-09-21T00:27:49.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:49.639+0000 7f90917fa640 1 -- 172.21.15.112:0/3546237725 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f908001a070 con 0x7f90940d0860 2024-09-21T00:27:49.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:49.639+0000 7f90917fa640 1 --2- 172.21.15.112:0/3546237725 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f9078078bc0 0x7f907807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:49.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:49.640+0000 7f90917fa640 1 -- 172.21.15.112:0/3546237725 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7f908009a0e0 con 0x7f90940d0860 2024-09-21T00:27:49.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:49.640+0000 7f9098fe9640 1 --2- 172.21.15.112:0/3546237725 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f9078078bc0 0x7f907807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:49.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:49.640+0000 7f9098fe9640 1 --2- 172.21.15.112:0/3546237725 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f9078078bc0 0x7f907807b0d0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f9088034000 tx=0x7f9088004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:49.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:49.644+0000 7f90917fa640 1 -- 172.21.15.112:0/3546237725 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f90800624b0 con 0x7f90940d0860 2024-09-21T00:27:49.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:49.833+0000 7f909b24d640 1 -- 172.21.15.112:0/3546237725 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f90940718d0 con 0x7f90940d0860 2024-09-21T00:27:49.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:49.834+0000 7f90917fa640 1 -- 172.21.15.112:0/3546237725 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v60) ==== 80+0+0 (secure 0 0 0) 0x7f9080061db0 con 0x7f90940d0860 2024-09-21T00:27:49.835 INFO:teuthology.orchestra.run.smithi112.stderr:listed 0 entries 2024-09-21T00:27:49.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:49.836+0000 7f909b24d640 1 -- 172.21.15.112:0/3546237725 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f9078078bc0 msgr2=0x7f907807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:49.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:49.836+0000 7f909b24d640 1 --2- 172.21.15.112:0/3546237725 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f9078078bc0 0x7f907807b0d0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f9088034000 tx=0x7f9088004730 comp rx=0 tx=0).stop 2024-09-21T00:27:49.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:49.836+0000 7f909b24d640 1 -- 172.21.15.112:0/3546237725 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90940d0860 msgr2=0x7f90941590f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:49.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:49.836+0000 7f909b24d640 1 --2- 172.21.15.112:0/3546237725 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90940d0860 0x7f90941590f0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f908000c340 tx=0x7f908000c800 comp rx=0 tx=0).stop 2024-09-21T00:27:49.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:49.836+0000 7f909b24d640 1 -- 172.21.15.112:0/3546237725 shutdown_connections 2024-09-21T00:27:49.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:49.836+0000 7f909b24d640 1 --2- 172.21.15.112:0/3546237725 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f90940d2030 0x7f9094159630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:49.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:49.836+0000 7f909b24d640 1 --2- 172.21.15.112:0/3546237725 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f9078078bc0 0x7f907807b0d0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:49.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:49.836+0000 7f909b24d640 1 --2- 172.21.15.112:0/3546237725 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90940d0860 0x7f90941590f0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:49.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:49.836+0000 7f909b24d640 1 --2- 172.21.15.112:0/3546237725 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f90940ccae0 0x7f90941570e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:49.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:49.836+0000 7f909b24d640 1 -- 172.21.15.112:0/3546237725 >> 172.21.15.112:0/3546237725 conn(0x7f90940b43f0 msgr2=0x7f9094071df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:49.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:49.836+0000 7f909b24d640 1 -- 172.21.15.112:0/3546237725 shutdown_connections 2024-09-21T00:27:49.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:49.836+0000 7f909b24d640 1 -- 172.21.15.112:0/3546237725 wait complete. 2024-09-21T00:27:49.849 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-09-21T00:27:49.849 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:27:49.849 DEBUG:teuthology.orchestra.run.smithi187:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-09-21T00:27:49.882 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi187.front.sepia.ceph.com 2024-09-21T00:27:49.882 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:27:49.882 DEBUG:teuthology.orchestra.run.smithi187:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-21T00:27:49.940 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-09-21T00:27:49.940 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:27:49.998 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056013,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497440k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16835 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248720k,nr_inodes=812180,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497440k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T00:27:49.998 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T00:27:50.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:50.107+0000 7fbf9a757640 1 -- 172.21.15.112:0/3982697770 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf940ccae0 msgr2=0x7fbf940ccef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:50.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:50.107+0000 7fbf9a757640 1 --2- 172.21.15.112:0/3982697770 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf940ccae0 0x7fbf940ccef0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fbf8c005fd0 tx=0x7fbf8c02aa00 comp rx=0 tx=0).stop 2024-09-21T00:27:50.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:50.108+0000 7fbf9a757640 1 -- 172.21.15.112:0/3982697770 shutdown_connections 2024-09-21T00:27:50.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:50.108+0000 7fbf9a757640 1 --2- 172.21.15.112:0/3982697770 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf940ccae0 0x7fbf940ccef0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:50.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:50.108+0000 7fbf9a757640 1 --2- 172.21.15.112:0/3982697770 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbf940d2030 0x7fbf9415d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:50.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:50.108+0000 7fbf9a757640 1 -- 172.21.15.112:0/3982697770 >> 172.21.15.112:0/3982697770 conn(0x7fbf940b43f0 msgr2=0x7fbf940b4830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:50.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:50.108+0000 7fbf9a757640 1 -- 172.21.15.112:0/3982697770 shutdown_connections 2024-09-21T00:27:50.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:50.108+0000 7fbf9a757640 1 -- 172.21.15.112:0/3982697770 wait complete. 2024-09-21T00:27:50.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:50.109+0000 7fbf9a757640 1 Processor -- start 2024-09-21T00:27:50.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:50.110+0000 7fbf9a757640 1 -- start start 2024-09-21T00:27:50.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:50.111+0000 7fbf9a757640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf940ccae0 0x7fbf940cbc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:50.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:50.111+0000 7fbf9a757640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbf940d0860 0x7fbf94157160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:50.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:50.111+0000 7fbf9a757640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbf940d2030 0x7fbf941f71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:50.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:50.111+0000 7fbf9a757640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbf940b22d0 con 0x7fbf940ccae0 2024-09-21T00:27:50.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:50.111+0000 7fbf9a757640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fbf940b2150 con 0x7fbf940d2030 2024-09-21T00:27:50.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:50.111+0000 7fbf9a757640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fbf940b2450 con 0x7fbf940d0860 2024-09-21T00:27:50.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:50.111+0000 7fbf93fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf940ccae0 0x7fbf940cbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:50.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:50.111+0000 7fbf93fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf940ccae0 0x7fbf940cbc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57334/0 (socket says 172.21.15.112:57334) 2024-09-21T00:27:50.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:50.111+0000 7fbf93fff640 1 -- 172.21.15.112:0/3616081256 learned_addr learned my addr 172.21.15.112:0/3616081256 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:27:50.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:50.111+0000 7fbf937fe640 1 --2- 172.21.15.112:0/3616081256 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbf940d0860 0x7fbf94157160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:50.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:50.111+0000 7fbf98cf4640 1 --2- 172.21.15.112:0/3616081256 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbf940d2030 0x7fbf941f71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:50.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:50.111+0000 7fbf93fff640 1 -- 172.21.15.112:0/3616081256 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbf940d0860 msgr2=0x7fbf94157160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:50.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:50.111+0000 7fbf93fff640 1 --2- 172.21.15.112:0/3616081256 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbf940d0860 0x7fbf94157160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:50.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:50.111+0000 7fbf93fff640 1 -- 172.21.15.112:0/3616081256 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbf940d2030 msgr2=0x7fbf941f71b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:50.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:50.111+0000 7fbf93fff640 1 --2- 172.21.15.112:0/3616081256 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbf940d2030 0x7fbf941f71b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:50.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:50.111+0000 7fbf93fff640 1 -- 172.21.15.112:0/3616081256 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf94158bc0 con 0x7fbf940ccae0 2024-09-21T00:27:50.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:50.111+0000 7fbf98cf4640 1 --2- 172.21.15.112:0/3616081256 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbf940d2030 0x7fbf941f71b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:27:50.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:50.112+0000 7fbf93fff640 1 --2- 172.21.15.112:0/3616081256 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf940ccae0 0x7fbf940cbc50 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fbf8400c9a0 tx=0x7fbf8400ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:50.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:50.112+0000 7fbf90ff9640 1 -- 172.21.15.112:0/3616081256 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf84007bb0 con 0x7fbf940ccae0 2024-09-21T00:27:50.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:50.112+0000 7fbf90ff9640 1 -- 172.21.15.112:0/3616081256 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbf84007e40 con 0x7fbf940ccae0 2024-09-21T00:27:50.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:50.112+0000 7fbf9a757640 1 -- 172.21.15.112:0/3616081256 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf941576a0 con 0x7fbf940ccae0 2024-09-21T00:27:50.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:50.112+0000 7fbf90ff9640 1 -- 172.21.15.112:0/3616081256 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf8400f690 con 0x7fbf940ccae0 2024-09-21T00:27:50.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:50.112+0000 7fbf9a757640 1 -- 172.21.15.112:0/3616081256 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf94157e00 con 0x7fbf940ccae0 2024-09-21T00:27:50.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:50.113+0000 7fbf9a757640 1 -- 172.21.15.112:0/3616081256 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf940696c0 con 0x7fbf940ccae0 2024-09-21T00:27:50.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:50.114+0000 7fbf90ff9640 1 -- 172.21.15.112:0/3616081256 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fbf8400f830 con 0x7fbf940ccae0 2024-09-21T00:27:50.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:50.114+0000 7fbf90ff9640 1 --2- 172.21.15.112:0/3616081256 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fbf80078b90 0x7fbf8007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:27:50.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:50.114+0000 7fbf90ff9640 1 -- 172.21.15.112:0/3616081256 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7fbf8409b6f0 con 0x7fbf940ccae0 2024-09-21T00:27:50.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:50.114+0000 7fbf937fe640 1 --2- 172.21.15.112:0/3616081256 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fbf80078b90 0x7fbf8007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:27:50.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:50.115+0000 7fbf937fe640 1 --2- 172.21.15.112:0/3616081256 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fbf80078b90 0x7fbf8007b0a0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fbf88006fd0 tx=0x7fbf88008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:27:50.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:50.119+0000 7fbf90ff9640 1 -- 172.21.15.112:0/3616081256 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fbf940696c0 con 0x7fbf940ccae0 2024-09-21T00:27:50.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:50.331+0000 7fbf9a757640 1 -- 172.21.15.112:0/3616081256 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fbf9415f480 con 0x7fbf940ccae0 2024-09-21T00:27:50.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:50.331+0000 7fbf90ff9640 1 -- 172.21.15.112:0/3616081256 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v38) ==== 53+0+73 (secure 0 0 0) 0x7fbf9415f480 con 0x7fbf940ccae0 2024-09-21T00:27:50.333 INFO:teuthology.orchestra.run.smithi112.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T00:27:50.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:50.334+0000 7fbf9a757640 1 -- 172.21.15.112:0/3616081256 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fbf80078b90 msgr2=0x7fbf8007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:50.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:50.334+0000 7fbf9a757640 1 --2- 172.21.15.112:0/3616081256 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fbf80078b90 0x7fbf8007b0a0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fbf88006fd0 tx=0x7fbf88008040 comp rx=0 tx=0).stop 2024-09-21T00:27:50.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:50.334+0000 7fbf9a757640 1 -- 172.21.15.112:0/3616081256 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf940ccae0 msgr2=0x7fbf940cbc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:27:50.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:50.334+0000 7fbf9a757640 1 --2- 172.21.15.112:0/3616081256 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf940ccae0 0x7fbf940cbc50 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fbf8400c9a0 tx=0x7fbf8400ce60 comp rx=0 tx=0).stop 2024-09-21T00:27:50.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:50.334+0000 7fbf9a757640 1 -- 172.21.15.112:0/3616081256 shutdown_connections 2024-09-21T00:27:50.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:50.334+0000 7fbf9a757640 1 --2- 172.21.15.112:0/3616081256 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbf940d2030 0x7fbf941f71b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:50.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:50.334+0000 7fbf9a757640 1 --2- 172.21.15.112:0/3616081256 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbf940d0860 0x7fbf94157160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:50.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:50.334+0000 7fbf9a757640 1 --2- 172.21.15.112:0/3616081256 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fbf80078b90 0x7fbf8007b0a0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:50.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:50.334+0000 7fbf9a757640 1 --2- 172.21.15.112:0/3616081256 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf940ccae0 0x7fbf940cbc50 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:27:50.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:50.334+0000 7fbf9a757640 1 -- 172.21.15.112:0/3616081256 >> 172.21.15.112:0/3616081256 conn(0x7fbf940b43f0 msgr2=0x7fbf9415e870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:27:50.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:50.334+0000 7fbf9a757640 1 -- 172.21.15.112:0/3616081256 shutdown_connections 2024-09-21T00:27:50.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:50.334+0000 7fbf9a757640 1 -- 172.21.15.112:0/3616081256 wait complete. 2024-09-21T00:27:50.347 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T00:27:50.347 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T00:27:50.347 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi187.front.sepia.ceph.com 2024-09-21T00:27:50.347 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-21T00:27:50.347 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-21T00:27:50.347 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T00:27:50.347 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-21T00:27:50.347 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T00:27:50.347 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-21T00:27:50.347 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:27:50.348 DEBUG:teuthology.orchestra.run.smithi187:> ip addr 2024-09-21T00:27:50.367 INFO:teuthology.orchestra.run.smithi187.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T00:27:50.367 INFO:teuthology.orchestra.run.smithi187.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T00:27:50.367 INFO:teuthology.orchestra.run.smithi187.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T00:27:50.367 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:27:50.367 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 ::1/128 scope host 2024-09-21T00:27:50.367 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:27:50.367 INFO:teuthology.orchestra.run.smithi187.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T00:27:50.368 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 0c:c4:7a:d9:36:78 brd ff:ff:ff:ff:ff:ff 2024-09-21T00:27:50.368 INFO:teuthology.orchestra.run.smithi187.stdout: altname enp7s0f0 2024-09-21T00:27:50.368 INFO:teuthology.orchestra.run.smithi187.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T00:27:50.368 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 0c:c4:7a:d9:36:79 brd ff:ff:ff:ff:ff:ff 2024-09-21T00:27:50.368 INFO:teuthology.orchestra.run.smithi187.stdout: altname enp7s0f1 2024-09-21T00:27:50.368 INFO:teuthology.orchestra.run.smithi187.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T00:27:50.368 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 0c:c4:7a:88:77:b0 brd ff:ff:ff:ff:ff:ff 2024-09-21T00:27:50.368 INFO:teuthology.orchestra.run.smithi187.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T00:27:50.368 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 0c:c4:7a:88:77:b1 brd ff:ff:ff:ff:ff:ff 2024-09-21T00:27:50.368 INFO:teuthology.orchestra.run.smithi187.stdout: inet 172.21.15.187/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-21T00:27:50.368 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft 42640sec preferred_lft 42640sec 2024-09-21T00:27:50.368 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 fe80::ec4:7aff:fe88:77b1/64 scope link noprefixroute 2024-09-21T00:27:50.368 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:27:50.368 INFO:teuthology.orchestra.run.smithi187.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T00:27:50.369 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 86:3b:f6:5b:1b:32 brd ff:ff:ff:ff:ff:ff 2024-09-21T00:27:50.369 INFO:teuthology.orchestra.run.smithi187.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T00:27:50.369 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:27:50.369 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 fe80::ac6f:97ff:fe09:b82e/64 scope link 2024-09-21T00:27:50.369 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:27:50.369 INFO:teuthology.orchestra.run.smithi187.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T00:27:50.369 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether ca:0b:a0:f5:2b:b5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T00:27:50.369 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 fe80::c80b:a0ff:fef5:2bb5/64 scope link 2024-09-21T00:27:50.369 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:27:50.369 INFO:teuthology.orchestra.run.smithi187.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T00:27:50.369 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 86:3b:f6:5b:1b:32 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-21T00:27:50.369 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 fe80::843b:f6ff:fe5b:1b32/64 scope link 2024-09-21T00:27:50.369 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:27:50.370 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:27:50.370 DEBUG:teuthology.orchestra.run.smithi187:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T00:27:50.436 INFO:teuthology.orchestra.run.smithi187.stdout:1 2024-09-21T00:27:50.437 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:27:50.438 DEBUG:teuthology.orchestra.run.smithi187:> route 2024-09-21T00:27:50.497 INFO:teuthology.orchestra.run.smithi187.stdout:Kernel IP routing table 2024-09-21T00:27:50.497 INFO:teuthology.orchestra.run.smithi187.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T00:27:50.497 INFO:teuthology.orchestra.run.smithi187.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-21T00:27:50.497 INFO:teuthology.orchestra.run.smithi187.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-21T00:27:50.497 INFO:teuthology.orchestra.run.smithi187.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T00:27:50.498 DEBUG:teuthology.orchestra.run.smithi187:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T00:27:50.498 DEBUG:teuthology.orchestra.run.smithi187:> set -e 2024-09-21T00:27:50.498 DEBUG:teuthology.orchestra.run.smithi187:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-21T00:27:50.498 DEBUG:teuthology.orchestra.run.smithi187:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-21T00:27:50.498 DEBUG:teuthology.orchestra.run.smithi187:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-21T00:27:50.498 DEBUG:teuthology.orchestra.run.smithi187:> ') 2024-09-21T00:27:50.588 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:27:50 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T00:27:50.621 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:27:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T00:27:50.627 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:27:50.627 DEBUG:teuthology.orchestra.run.smithi187:> ip netns list 2024-09-21T00:27:50.643 INFO:teuthology.orchestra.run.smithi187.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-21T00:27:50.643 INFO:teuthology.orchestra.run.smithi187.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-21T00:27:50.643 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-21T00:27:50.643 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:27:50.643 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip link set brx.0 up 2024-09-21T00:27:50.708 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T00:27:50.708 DEBUG:teuthology.orchestra.run.smithi187:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-21T00:27:50.763 INFO:teuthology.orchestra.run.smithi187.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-21T00:27:50.764 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T00:27:50.764 DEBUG:teuthology.orchestra.run.smithi187:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-21T00:27:50.819 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-21T00:27:50.819 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:27:50.819 DEBUG:teuthology.orchestra.run.smithi187:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-21T00:27:50.913 INFO:teuthology.orchestra.run.smithi187.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-21T00:27:50.914 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-21T00:27:50.914 INFO:teuthology.orchestra.run.smithi187.stdout:invalid new device string format 2024-09-21T00:27:50.924 INFO:teuthology.orchestra.run.smithi187.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-21T00:27:50.924 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-21T00:27:50.925 INFO:teuthology.orchestra.run.smithi187.stdout:invalid new device string format 2024-09-21T00:27:50.925 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: resolved to: "172.21.15.112:3300,172.21.15.187:3300,172.21.15.187:3301" 2024-09-21T00:27:50.925 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T00:27:50.930 INFO:teuthology.orchestra.run.smithi187.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-21T00:27:50.930 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-21T00:27:50.931 INFO:teuthology.orchestra.run.smithi187.stdout:invalid new device string format 2024-09-21T00:27:50.931 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: resolved to: "172.21.15.112:3300,172.21.15.187:3300,172.21.15.187:3301" 2024-09-21T00:27:50.931 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T00:27:50.931 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: resolved to: "172.21.15.112:6789,172.21.15.187:6789,172.21.15.187:6790" 2024-09-21T00:27:50.931 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.112:6789,172.21.15.187:6789,172.21.15.187:6790:/ 2024-09-21T00:27:50.931 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=8b34692b-dc65-49b2-a08f-26f707b96ac5" will pass to kernel 2024-09-21T00:27:50.932 INFO:teuthology.orchestra.run.smithi187.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-09-21T00:27:50.932 INFO:teuthology.orchestra.run.smithi187.stdout: You just mounted a file system that supports labels which does not 2024-09-21T00:27:50.932 INFO:teuthology.orchestra.run.smithi187.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T00:27:50.932 INFO:teuthology.orchestra.run.smithi187.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T00:27:50.932 INFO:teuthology.orchestra.run.smithi187.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T00:27:50.933 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T00:27:50.933 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:27:50.933 DEBUG:teuthology.orchestra.run.smithi187:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-21T00:27:50.958 DEBUG:teuthology.orchestra.run.smithi187:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T00:27:51.046 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:27:51 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T00:27:51.048 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:27:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T00:27:51.054 INFO:teuthology.orchestra.run.smithi187.stdout:client5560 2024-09-21T00:27:51.054 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-09-21T00:27:51.054 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:27:51.070 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'mkdir {d0,d1} && touch {d0,d1}/file') 2024-09-21T00:27:51.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:51 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T00:27:51.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:27:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T00:27:51.141 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:27:51.141 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 0 d0) 2024-09-21T00:27:51.159 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:27:51.160 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 1 d1) 2024-09-21T00:27:51.216 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-09-21T00:27:51.379 INFO:teuthology.orchestra.run.smithi112.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"13","version":"9","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":2,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":1},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-09-21T00:27:51.380 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 2, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 1, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "13", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "9" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-21T00:27:51.380 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d0', 0), ('/d1', 1)] 2024-09-21T00:27:53.381 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-09-21T00:27:53.518 INFO:teuthology.orchestra.run.smithi112.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"13","version":"9","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":2,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":1},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-09-21T00:27:53.518 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 2, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 1, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "13", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "9" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-21T00:27:53.518 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d0', 0), ('/d1', 1)] 2024-09-21T00:27:55.520 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-09-21T00:27:55.656 INFO:teuthology.orchestra.run.smithi112.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"13","version":"9","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":2,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":1},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-09-21T00:27:55.657 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 2, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 1, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "13", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "9" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-21T00:27:55.657 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d0', 0), ('/d1', 1)] 2024-09-21T00:27:57.657 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-09-21T00:27:57.794 INFO:teuthology.orchestra.run.smithi112.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"13","version":"13","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-09-21T00:27:57.795 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "13", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-21T00:27:57.795 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d0', 0), ('/d1', 1)] 2024-09-21T00:27:59.796 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-09-21T00:27:59.934 INFO:teuthology.orchestra.run.smithi112.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"13","version":"13","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-09-21T00:27:59.934 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "13", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-21T00:27:59.935 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d0', 0), ('/d1', 1)] 2024-09-21T00:28:01.935 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-09-21T00:28:02.073 INFO:teuthology.orchestra.run.smithi112.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"14","version":"14","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":0,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/d0","dirfrag":"0x10000000000","snapid_first":2,"projected_version":"3","version":"3","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":1,"authpin":0},"nref":3}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1","dirfrag":"0x10000000001","snapid_first":2,"projected_version":"3","version":"3","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}}] 2024-09-21T00:28:02.073 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x10000000000", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "/d0", "pins": { "authpin": 0, "child": 1, "dirty": 1, "subtree": 1 }, "projected_version": "3", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "3" }, "distributed_ephemeral_pin": false, "export_pin": 0, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000001", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/d1", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "3", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "3" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false } ] 2024-09-21T00:28:02.073 INFO:tasks.cephfs.cephfs_test_case:[('/d0', 0), ('/d1', 1)] =?= [('/d0', 0), ('/d1', 1)] 2024-09-21T00:28:02.074 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:28:02.074 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch d0/f0) 2024-09-21T00:28:02.091 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:28:02.091 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch d1/f0) 2024-09-21T00:28:02.150 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:28:02.150 DEBUG:teuthology.orchestra.run.smithi187:> (cd /home/ubuntu/cephtest/mnt.1 && exec touch d0/f1) 2024-09-21T00:28:02.175 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:28:02.176 DEBUG:teuthology.orchestra.run.smithi187:> (cd /home/ubuntu/cephtest/mnt.1 && exec touch d1/f1) 2024-09-21T00:28:02.240 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2024-09-21T00:28:02.365 INFO:teuthology.orchestra.run.smithi112.stdout:[{"id":5560,"entity":{"name":{"type":"client","num":5560},"addr":{"type":"v1","addr":"192.168.0.1:0","nonce":4271479196}},"state":"open","num_leases":1,"num_caps":4,"request_load_avg":6,"uptime":11.428724545,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":7.9207440979069945,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":8,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.5560 v1:192.168.0.1:0/4271479196","completed_requests":[],"prealloc_inos":[{"start":"0x100000005e1","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x000000000003fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"1","hostname":"smithi187","kernel_version":"5.14.0-508.el9.x86_64","root":"/"}},{"id":5985,"entity":{"name":{"type":"client","num":5985},"addr":{"type":"v1","addr":"192.168.0.1:0","nonce":431973012}},"state":"open","num_leases":3,"num_caps":5,"request_load_avg":11,"uptime":13.053685379999999,"requests_in_flight":0,"num_completed_requests":1,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":15.661211813955617,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":13,"last_trim_completed_flushes_tid":0,"delegated_inos":[{"start":"0x10000000003","length":500}],"inst":"client.5985 v1:192.168.0.1:0/431973012","completed_requests":[{"tid":13,"created_ino":"0x100000001f8"}],"prealloc_inos":[{"start":"0x10000000003","length":500},{"start":"0x100000001f9","length":999}],"client_metadata":{"client_features":{"feature_bits":"0x000000000003fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"0","hostname":"smithi112","kernel_version":"5.14.0-508.el9.x86_64","root":"/"}}] 2024-09-21T00:28:02.366 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000003fbff" }, "entity_id": "1", "hostname": "smithi187", "kernel_version": "5.14.0-508.el9.x86_64", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 4271479196, "type": "v1" }, "name": { "num": 5560, "type": "client" } }, "id": 5560, "inst": "client.5560 v1:192.168.0.1:0/4271479196", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 8, "num_caps": 4, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 1, "prealloc_inos": [ { "length": 1000, "start": "0x100000005e1" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 6, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 7.9207440979069945 }, "state": "open", "uptime": 11.428724545 }, { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000003fbff" }, "entity_id": "0", "hostname": "smithi112", "kernel_version": "5.14.0-508.el9.x86_64", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [ { "created_ino": "0x100000001f8", "tid": 13 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000003" } ], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 431973012, "type": "v1" }, "name": { "num": 5985, "type": "client" } }, "id": 5985, "inst": "client.5985 v1:192.168.0.1:0/431973012", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 13, "num_caps": 5, "num_completed_flushes": 0, "num_completed_requests": 1, "num_leases": 3, "prealloc_inos": [ { "length": 500, "start": "0x10000000003" }, { "length": 999, "start": "0x100000001f9" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 11, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 15.661211813955617 }, "state": "open", "uptime": 13.05368538 } ] 2024-09-21T00:28:02.366 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2024-09-21T00:28:02.491 INFO:teuthology.orchestra.run.smithi112.stdout:[{"id":5560,"entity":{"name":{"type":"client","num":5560},"addr":{"type":"v1","addr":"192.168.0.1:0","nonce":4271479196}},"state":"open","num_leases":1,"num_caps":2,"request_load_avg":0,"uptime":0.24599407100000001,"requests_in_flight":0,"num_completed_requests":1,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":3.9977502692958784,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":8,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.5560 v1:192.168.0.1:0/4271479196","completed_requests":[{"tid":8,"created_ino":"0x200000003e9"}],"prealloc_inos":[{"start":"0x200000003ea","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x000000000003fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"1","hostname":"smithi187","kernel_version":"5.14.0-508.el9.x86_64","root":"/"}},{"id":5985,"entity":{"name":{"type":"client","num":5985},"addr":{"type":"v1","addr":"192.168.0.1:0","nonce":431973012}},"state":"open","num_leases":1,"num_caps":3,"request_load_avg":0,"uptime":0.66498397300000001,"requests_in_flight":0,"num_completed_requests":1,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":5.9938577755252211,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":13,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.5985 v1:192.168.0.1:0/431973012","completed_requests":[{"tid":14,"created_ino":"0x20000000000"}],"prealloc_inos":[{"start":"0x20000000001","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x000000000003fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"0","hostname":"smithi112","kernel_version":"5.14.0-508.el9.x86_64","root":"/"}}] 2024-09-21T00:28:02.492 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000003fbff" }, "entity_id": "1", "hostname": "smithi187", "kernel_version": "5.14.0-508.el9.x86_64", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [ { "created_ino": "0x200000003e9", "tid": 8 } ], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 4271479196, "type": "v1" }, "name": { "num": 5560, "type": "client" } }, "id": 5560, "inst": "client.5560 v1:192.168.0.1:0/4271479196", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 8, "num_caps": 2, "num_completed_flushes": 0, "num_completed_requests": 1, "num_leases": 1, "prealloc_inos": [ { "length": 1000, "start": "0x200000003ea" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 0, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 3.9977502692958784 }, "state": "open", "uptime": 0.245994071 }, { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000003fbff" }, "entity_id": "0", "hostname": "smithi112", "kernel_version": "5.14.0-508.el9.x86_64", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [ { "created_ino": "0x20000000000", "tid": 14 } ], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 431973012, "type": "v1" }, "name": { "num": 5985, "type": "client" } }, "id": 5985, "inst": "client.5985 v1:192.168.0.1:0/431973012", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 13, "num_caps": 3, "num_completed_flushes": 0, "num_completed_requests": 1, "num_leases": 1, "prealloc_inos": [ { "length": 1000, "start": "0x20000000001" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 0, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 5.993857775525221 }, "state": "open", "uptime": 0.664983973 } ] 2024-09-21T00:28:02.493 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T00:28:02.493 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:28:02.510 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T00:28:02.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:02 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T00:28:02.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T00:28:02.607 INFO:teuthology.orchestra.run.smithi112.stdout:client5985 2024-09-21T00:28:02.607 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session evict 5985 2024-09-21T00:28:02.756 DEBUG:tasks.cephfs.filesystem:_json_asok output empty 2024-09-21T00:28:02.756 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:28:02.756 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch d0/f00) 2024-09-21T00:28:03.044 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:28:03.044 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch d1/f00) 2024-09-21T00:28:03.062 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:28:03.063 DEBUG:teuthology.orchestra.run.smithi187:> (cd /home/ubuntu/cephtest/mnt.1 && exec touch d0/f10) 2024-09-21T00:28:03.082 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:28:03.082 DEBUG:teuthology.orchestra.run.smithi187:> (cd /home/ubuntu/cephtest/mnt.1 && exec touch d1/f10) 2024-09-21T00:28:13.142 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2024-09-21T00:28:13.283 INFO:teuthology.orchestra.run.smithi112.stdout:[{"id":5985,"entity":{"name":{"type":"client","num":5985},"addr":{"type":"v1","addr":"192.168.0.1:0","nonce":431973012}},"state":"open","num_leases":1,"num_caps":4,"request_load_avg":1,"uptime":10.237753250000001,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":5.8598934553894368,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":21,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.5985 v1:192.168.0.1:0/431973012","completed_requests":[],"prealloc_inos":[{"start":"0x10000000004","length":499},{"start":"0x100000001f9","length":501}],"client_metadata":{"client_features":{"feature_bits":"0x000000000003fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"0","hostname":"smithi112","kernel_version":"5.14.0-508.el9.x86_64","root":"/"}},{"id":5560,"entity":{"name":{"type":"client","num":5560},"addr":{"type":"v1","addr":"192.168.0.1:0","nonce":4271479196}},"state":"open","num_leases":2,"num_caps":5,"request_load_avg":6,"uptime":22.346461406,"requests_in_flight":0,"num_completed_requests":1,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":10.653614363499363,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":9,"last_trim_completed_flushes_tid":0,"delegated_inos":[{"start":"0x100000005e2","length":500}],"inst":"client.5560 v1:192.168.0.1:0/4271479196","completed_requests":[{"tid":9,"created_ino":"0x100000005e1"}],"prealloc_inos":[{"start":"0x100000005e2","length":999}],"client_metadata":{"client_features":{"feature_bits":"0x000000000003fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"1","hostname":"smithi187","kernel_version":"5.14.0-508.el9.x86_64","root":"/"}}] 2024-09-21T00:28:13.283 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000003fbff" }, "entity_id": "0", "hostname": "smithi112", "kernel_version": "5.14.0-508.el9.x86_64", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 431973012, "type": "v1" }, "name": { "num": 5985, "type": "client" } }, "id": 5985, "inst": "client.5985 v1:192.168.0.1:0/431973012", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 21, "num_caps": 4, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 1, "prealloc_inos": [ { "length": 499, "start": "0x10000000004" }, { "length": 501, "start": "0x100000001f9" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 1, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 5.859893455389437 }, "state": "open", "uptime": 10.23775325 }, { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000003fbff" }, "entity_id": "1", "hostname": "smithi187", "kernel_version": "5.14.0-508.el9.x86_64", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [ { "created_ino": "0x100000005e1", "tid": 9 } ], "delegated_inos": [ { "length": 500, "start": "0x100000005e2" } ], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 4271479196, "type": "v1" }, "name": { "num": 5560, "type": "client" } }, "id": 5560, "inst": "client.5560 v1:192.168.0.1:0/4271479196", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 9, "num_caps": 5, "num_completed_flushes": 0, "num_completed_requests": 1, "num_leases": 2, "prealloc_inos": [ { "length": 999, "start": "0x100000005e2" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 6, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 10.653614363499363 }, "state": "open", "uptime": 22.346461406 } ] 2024-09-21T00:28:13.284 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2024-09-21T00:28:13.408 INFO:teuthology.orchestra.run.smithi112.stdout:[{"id":5560,"entity":{"name":{"type":"client","num":5560},"addr":{"type":"v1","addr":"192.168.0.1:0","nonce":4271479196}},"state":"open","num_leases":2,"num_caps":3,"request_load_avg":1,"uptime":11.163730932,"requests_in_flight":0,"num_completed_requests":1,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":6.8278880839591505,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":9,"last_trim_completed_flushes_tid":0,"delegated_inos":[{"start":"0x200000003eb","length":500}],"inst":"client.5560 v1:192.168.0.1:0/4271479196","completed_requests":[{"tid":10,"created_ino":"0x200000003ea"}],"prealloc_inos":[{"start":"0x200000003eb","length":999}],"client_metadata":{"client_features":{"feature_bits":"0x000000000003fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"1","hostname":"smithi187","kernel_version":"5.14.0-508.el9.x86_64","root":"/"}},{"id":5985,"entity":{"name":{"type":"client","num":5985},"addr":{"type":"v1","addr":"192.168.0.1:0","nonce":431973012}},"state":"open","num_leases":2,"num_caps":5,"request_load_avg":2,"uptime":11.582720834,"requests_in_flight":0,"num_completed_requests":1,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0.91362232397261089,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":9.7501381737274553,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":21,"last_trim_completed_flushes_tid":0,"delegated_inos":[{"start":"0x20000000002","length":500}],"inst":"client.5985 v1:192.168.0.1:0/431973012","completed_requests":[{"tid":21,"created_ino":"0x20000000001"}],"prealloc_inos":[{"start":"0x20000000002","length":999}],"client_metadata":{"client_features":{"feature_bits":"0x000000000003fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"0","hostname":"smithi112","kernel_version":"5.14.0-508.el9.x86_64","root":"/"}}] 2024-09-21T00:28:13.409 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000003fbff" }, "entity_id": "1", "hostname": "smithi187", "kernel_version": "5.14.0-508.el9.x86_64", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [ { "created_ino": "0x200000003ea", "tid": 10 } ], "delegated_inos": [ { "length": 500, "start": "0x200000003eb" } ], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 4271479196, "type": "v1" }, "name": { "num": 5560, "type": "client" } }, "id": 5560, "inst": "client.5560 v1:192.168.0.1:0/4271479196", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 9, "num_caps": 3, "num_completed_flushes": 0, "num_completed_requests": 1, "num_leases": 2, "prealloc_inos": [ { "length": 999, "start": "0x200000003eb" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 1, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 6.8278880839591505 }, "state": "open", "uptime": 11.163730932 }, { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000003fbff" }, "entity_id": "0", "hostname": "smithi112", "kernel_version": "5.14.0-508.el9.x86_64", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [ { "created_ino": "0x20000000001", "tid": 21 } ], "delegated_inos": [ { "length": 500, "start": "0x20000000002" } ], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 431973012, "type": "v1" }, "name": { "num": 5985, "type": "client" } }, "id": 5985, "inst": "client.5985 v1:192.168.0.1:0/431973012", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 21, "num_caps": 5, "num_completed_flushes": 0, "num_completed_requests": 1, "num_leases": 2, "prealloc_inos": [ { "length": 999, "start": "0x20000000002" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0.9136223239726109 }, "request_load_avg": 2, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 9.750138173727455 }, "state": "open", "uptime": 11.582720834 } ] 2024-09-21T00:28:13.409 INFO:tasks.cephfs.mount:Cleaning up killed connection on ubuntu@smithi112.front.sepia.ceph.com 2024-09-21T00:28:13.409 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T00:28:13.409 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:28:13.425 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T00:28:13.425 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:28:13.481 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T00:28:13.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:13.633+0000 7f6af2550640 1 -- 172.21.15.112:0/4039636850 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6aec0d48a0 msgr2=0x7f6aec0d3dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:13.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:13.633+0000 7f6af2550640 1 --2- 172.21.15.112:0/4039636850 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6aec0d48a0 0x7f6aec0d3dc0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f6aec0b7490 tx=0x7f6adc02a560 comp rx=0 tx=0).stop 2024-09-21T00:28:13.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:13.633+0000 7f6af2550640 1 -- 172.21.15.112:0/4039636850 shutdown_connections 2024-09-21T00:28:13.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:13.633+0000 7f6af2550640 1 --2- 172.21.15.112:0/4039636850 >> v2:172.21.15.187:3300/0 conn(0x7f6aec0d01b0 0x7f6aec0d05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:13.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:13.633+0000 7f6af2550640 1 --2- 172.21.15.112:0/4039636850 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6aec0d3880 0x7f6aec163260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:13.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:13.633+0000 7f6af2550640 1 --2- 172.21.15.112:0/4039636850 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6aec0d48a0 0x7f6aec0d3dc0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:13.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:13.633+0000 7f6af2550640 1 -- 172.21.15.112:0/4039636850 >> 172.21.15.112:0/4039636850 conn(0x7f6aec08bfe0 msgr2=0x7f6aec09d890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:13.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:13.633+0000 7f6af2550640 1 -- 172.21.15.112:0/4039636850 shutdown_connections 2024-09-21T00:28:13.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:13.633+0000 7f6af2550640 1 -- 172.21.15.112:0/4039636850 wait complete. 2024-09-21T00:28:13.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:13.635+0000 7f6af2550640 1 Processor -- start 2024-09-21T00:28:13.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:13.635+0000 7f6af2550640 1 -- start start 2024-09-21T00:28:13.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:13.636+0000 7f6af2550640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6aec0d01b0 0x7f6aec1f59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:13.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:13.636+0000 7f6af2550640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6aec0d3880 0x7f6aec0ccbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:13.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:13.636+0000 7f6af2550640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6aec0d48a0 0x7f6aec0cd100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:13.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:13.636+0000 7f6af2550640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6aec0d2460 con 0x7f6aec0d48a0 2024-09-21T00:28:13.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:13.636+0000 7f6af2550640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f6aec0d22e0 con 0x7f6aec0d01b0 2024-09-21T00:28:13.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:13.636+0000 7f6af2550640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f6aec0d25e0 con 0x7f6aec0d3880 2024-09-21T00:28:13.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:13.636+0000 7f6af0aed640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6aec0d48a0 0x7f6aec0cd100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:13.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:13.636+0000 7f6af0aed640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6aec0d48a0 0x7f6aec0cd100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33796/0 (socket says 172.21.15.112:33796) 2024-09-21T00:28:13.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:13.636+0000 7f6af0aed640 1 -- 172.21.15.112:0/657005139 learned_addr learned my addr 172.21.15.112:0/657005139 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:28:13.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:13.636+0000 7f6aeb7fe640 1 --2- 172.21.15.112:0/657005139 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6aec0d3880 0x7f6aec0ccbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:13.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:13.636+0000 7f6aebfff640 1 --2- 172.21.15.112:0/657005139 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6aec0d01b0 0x7f6aec1f59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:13.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:13.637+0000 7f6af0aed640 1 -- 172.21.15.112:0/657005139 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6aec0d3880 msgr2=0x7f6aec0ccbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:13.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:13.637+0000 7f6af0aed640 1 --2- 172.21.15.112:0/657005139 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6aec0d3880 0x7f6aec0ccbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:13.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:13.637+0000 7f6af0aed640 1 -- 172.21.15.112:0/657005139 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6aec0d01b0 msgr2=0x7f6aec1f59c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:13.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:13.637+0000 7f6af0aed640 1 --2- 172.21.15.112:0/657005139 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6aec0d01b0 0x7f6aec1f59c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:13.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:13.637+0000 7f6af0aed640 1 -- 172.21.15.112:0/657005139 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6aec1f6210 con 0x7f6aec0d48a0 2024-09-21T00:28:13.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:13.637+0000 7f6af0aed640 1 --2- 172.21.15.112:0/657005139 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6aec0d48a0 0x7f6aec0cd100 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f6ae400e990 tx=0x7f6ae400ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:13.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:13.637+0000 7f6ae8ff9640 1 -- 172.21.15.112:0/657005139 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae4004d20 con 0x7f6aec0d48a0 2024-09-21T00:28:13.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:13.637+0000 7f6ae8ff9640 1 -- 172.21.15.112:0/657005139 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6ae400caa0 con 0x7f6aec0d48a0 2024-09-21T00:28:13.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:13.637+0000 7f6af2550640 1 -- 172.21.15.112:0/657005139 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6aec1f64f0 con 0x7f6aec0d48a0 2024-09-21T00:28:13.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:13.637+0000 7f6ae8ff9640 1 -- 172.21.15.112:0/657005139 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae4005020 con 0x7f6aec0d48a0 2024-09-21T00:28:13.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:13.637+0000 7f6af2550640 1 -- 172.21.15.112:0/657005139 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6aec1f8330 con 0x7f6aec0d48a0 2024-09-21T00:28:13.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:13.638+0000 7f6af2550640 1 -- 172.21.15.112:0/657005139 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6aec1f8590 con 0x7f6aec0d48a0 2024-09-21T00:28:13.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:13.639+0000 7f6ae8ff9640 1 -- 172.21.15.112:0/657005139 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6ae40051c0 con 0x7f6aec0d48a0 2024-09-21T00:28:13.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:13.639+0000 7f6ae8ff9640 1 --2- 172.21.15.112:0/657005139 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f6ad8078b90 0x7f6ad807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:13.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:13.639+0000 7f6ae8ff9640 1 -- 172.21.15.112:0/657005139 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7f6ae409b5c0 con 0x7f6aec0d48a0 2024-09-21T00:28:13.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:13.639+0000 7f6aebfff640 1 --2- 172.21.15.112:0/657005139 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f6ad8078b90 0x7f6ad807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:13.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:13.640+0000 7f6aebfff640 1 --2- 172.21.15.112:0/657005139 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f6ad8078b90 0x7f6ad807b0a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f6adc033000 tx=0x7f6adc004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:13.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:13.644+0000 7f6ae8ff9640 1 -- 172.21.15.112:0/657005139 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f6ae4016030 con 0x7f6aec0d48a0 2024-09-21T00:28:13.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:13.834+0000 7f6af2550640 1 -- 172.21.15.112:0/657005139 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f6aec1f5fa0 con 0x7f6aec0d48a0 2024-09-21T00:28:13.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:13.834+0000 7f6ae8ff9640 1 -- 172.21.15.112:0/657005139 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v60) ==== 80+0+0 (secure 0 0 0) 0x7f6ae4063340 con 0x7f6aec0d48a0 2024-09-21T00:28:13.835 INFO:teuthology.orchestra.run.smithi112.stderr:listed 0 entries 2024-09-21T00:28:13.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:13.836+0000 7f6af2550640 1 -- 172.21.15.112:0/657005139 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f6ad8078b90 msgr2=0x7f6ad807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:13.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:13.836+0000 7f6af2550640 1 --2- 172.21.15.112:0/657005139 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f6ad8078b90 0x7f6ad807b0a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f6adc033000 tx=0x7f6adc004730 comp rx=0 tx=0).stop 2024-09-21T00:28:13.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:13.836+0000 7f6af2550640 1 -- 172.21.15.112:0/657005139 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6aec0d48a0 msgr2=0x7f6aec0cd100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:13.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:13.836+0000 7f6af2550640 1 --2- 172.21.15.112:0/657005139 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6aec0d48a0 0x7f6aec0cd100 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f6ae400e990 tx=0x7f6ae400ee50 comp rx=0 tx=0).stop 2024-09-21T00:28:13.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:13.836+0000 7f6af2550640 1 -- 172.21.15.112:0/657005139 shutdown_connections 2024-09-21T00:28:13.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:13.836+0000 7f6af2550640 1 --2- 172.21.15.112:0/657005139 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f6ad8078b90 0x7f6ad807b0a0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:13.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:13.836+0000 7f6af2550640 1 --2- 172.21.15.112:0/657005139 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6aec0d48a0 0x7f6aec0cd100 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:13.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:13.836+0000 7f6af2550640 1 --2- 172.21.15.112:0/657005139 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6aec0d3880 0x7f6aec0ccbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:13.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:13.836+0000 7f6af2550640 1 --2- 172.21.15.112:0/657005139 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6aec0d01b0 0x7f6aec1f59c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:13.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:13.836+0000 7f6af2550640 1 -- 172.21.15.112:0/657005139 >> 172.21.15.112:0/657005139 conn(0x7f6aec08bfe0 msgr2=0x7f6aec0b3010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:13.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:13.837+0000 7f6af2550640 1 -- 172.21.15.112:0/657005139 shutdown_connections 2024-09-21T00:28:13.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:13.837+0000 7f6af2550640 1 -- 172.21.15.112:0/657005139 wait complete. 2024-09-21T00:28:13.849 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-21T00:28:13.849 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:28:13.849 DEBUG:teuthology.orchestra.run.smithi112:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-09-21T00:28:13.894 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi112.front.sepia.ceph.com 2024-09-21T00:28:13.894 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:28:13.894 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T00:28:13.923 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T00:28:14.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:14.076+0000 7f5c4680e640 1 -- 172.21.15.112:0/133767372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c400d4a90 msgr2=0x7f5c400d38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:14.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:14.076+0000 7f5c4680e640 1 --2- 172.21.15.112:0/133767372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c400d4a90 0x7f5c400d38c0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f5c4008dc00 tx=0x7f5c3402a4a0 comp rx=0 tx=0).stop 2024-09-21T00:28:14.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:14.076+0000 7f5c4680e640 1 -- 172.21.15.112:0/133767372 shutdown_connections 2024-09-21T00:28:14.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:14.076+0000 7f5c4680e640 1 --2- 172.21.15.112:0/133767372 >> v2:172.21.15.187:3300/0 conn(0x7f5c4015ea40 0x7f5c40070250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:14.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:14.076+0000 7f5c4680e640 1 --2- 172.21.15.112:0/133767372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c400d4a90 0x7f5c400d38c0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:14.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:14.076+0000 7f5c4680e640 1 -- 172.21.15.112:0/133767372 >> 172.21.15.112:0/133767372 conn(0x7f5c4009d8a0 msgr2=0x7f5c4008bfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:14.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:14.076+0000 7f5c4680e640 1 -- 172.21.15.112:0/133767372 shutdown_connections 2024-09-21T00:28:14.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:14.076+0000 7f5c4680e640 1 -- 172.21.15.112:0/133767372 wait complete. 2024-09-21T00:28:14.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:14.077+0000 7f5c4680e640 1 Processor -- start 2024-09-21T00:28:14.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:14.078+0000 7f5c4680e640 1 -- start start 2024-09-21T00:28:14.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:14.079+0000 7f5c4680e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5c400d0ae0 0x7f5c401f9480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:14.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:14.079+0000 7f5c4680e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c400d4a90 0x7f5c400b63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:14.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:14.079+0000 7f5c4680e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5c4015ea40 0x7f5c400d2b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:14.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:14.079+0000 7f5c4680e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5c40162b70 con 0x7f5c400d4a90 2024-09-21T00:28:14.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:14.079+0000 7f5c4680e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5c401629f0 con 0x7f5c4015ea40 2024-09-21T00:28:14.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:14.079+0000 7f5c4680e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f5c40162cf0 con 0x7f5c400d0ae0 2024-09-21T00:28:14.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:14.079+0000 7f5c3f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c400d4a90 0x7f5c400b63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:14.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:14.079+0000 7f5c3f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c400d4a90 0x7f5c400b63e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33810/0 (socket says 172.21.15.112:33810) 2024-09-21T00:28:14.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:14.079+0000 7f5c3f7fe640 1 -- 172.21.15.112:0/1088678217 learned_addr learned my addr 172.21.15.112:0/1088678217 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:28:14.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:14.079+0000 7f5c3ffff640 1 --2- 172.21.15.112:0/1088678217 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5c400d0ae0 0x7f5c401f9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:14.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:14.079+0000 7f5c44dab640 1 --2- 172.21.15.112:0/1088678217 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5c4015ea40 0x7f5c400d2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:14.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:14.079+0000 7f5c3f7fe640 1 -- 172.21.15.112:0/1088678217 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5c400d0ae0 msgr2=0x7f5c401f9480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:14.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:14.079+0000 7f5c3f7fe640 1 --2- 172.21.15.112:0/1088678217 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5c400d0ae0 0x7f5c401f9480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:14.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:14.079+0000 7f5c3f7fe640 1 -- 172.21.15.112:0/1088678217 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5c4015ea40 msgr2=0x7f5c400d2b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:14.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:14.079+0000 7f5c3f7fe640 1 --2- 172.21.15.112:0/1088678217 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5c4015ea40 0x7f5c400d2b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:14.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:14.079+0000 7f5c3f7fe640 1 -- 172.21.15.112:0/1088678217 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c401f5890 con 0x7f5c400d4a90 2024-09-21T00:28:14.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:14.079+0000 7f5c3ffff640 1 --2- 172.21.15.112:0/1088678217 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5c400d0ae0 0x7f5c401f9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:28:14.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:14.079+0000 7f5c44dab640 1 --2- 172.21.15.112:0/1088678217 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5c4015ea40 0x7f5c400d2b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:28:14.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:14.080+0000 7f5c3f7fe640 1 --2- 172.21.15.112:0/1088678217 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c400d4a90 0x7f5c400b63e0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f5c3402aa90 tx=0x7f5c34033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:14.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:14.080+0000 7f5c3cff9640 1 -- 172.21.15.112:0/1088678217 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c34031d60 con 0x7f5c400d4a90 2024-09-21T00:28:14.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:14.080+0000 7f5c3cff9640 1 -- 172.21.15.112:0/1088678217 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5c34033e10 con 0x7f5c400d4a90 2024-09-21T00:28:14.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:14.080+0000 7f5c4680e640 1 -- 172.21.15.112:0/1088678217 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c401f5b70 con 0x7f5c400d4a90 2024-09-21T00:28:14.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:14.080+0000 7f5c3cff9640 1 -- 172.21.15.112:0/1088678217 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c34005260 con 0x7f5c400d4a90 2024-09-21T00:28:14.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:14.080+0000 7f5c4680e640 1 -- 172.21.15.112:0/1088678217 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c401f6130 con 0x7f5c400d4a90 2024-09-21T00:28:14.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:14.081+0000 7f5c4680e640 1 -- 172.21.15.112:0/1088678217 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c40163210 con 0x7f5c400d4a90 2024-09-21T00:28:14.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:14.081+0000 7f5c3cff9640 1 -- 172.21.15.112:0/1088678217 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5c34005400 con 0x7f5c400d4a90 2024-09-21T00:28:14.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:14.082+0000 7f5c3cff9640 1 --2- 172.21.15.112:0/1088678217 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f5c2c078b90 0x7f5c2c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:14.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:14.082+0000 7f5c3cff9640 1 -- 172.21.15.112:0/1088678217 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7f5c340bc1b0 con 0x7f5c400d4a90 2024-09-21T00:28:14.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:14.082+0000 7f5c3ffff640 1 --2- 172.21.15.112:0/1088678217 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f5c2c078b90 0x7f5c2c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:14.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:14.083+0000 7f5c3ffff640 1 --2- 172.21.15.112:0/1088678217 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f5c2c078b90 0x7f5c2c07b0a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f5c30001090 tx=0x7f5c30009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:14.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:14.086+0000 7f5c3cff9640 1 -- 172.21.15.112:0/1088678217 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5c3403f030 con 0x7f5c400d4a90 2024-09-21T00:28:14.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:14.301+0000 7f5c4680e640 1 -- 172.21.15.112:0/1088678217 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f5c4015de40 con 0x7f5c400d4a90 2024-09-21T00:28:14.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:14.301+0000 7f5c3cff9640 1 -- 172.21.15.112:0/1088678217 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v39) ==== 53+0+73 (secure 0 0 0) 0x7f5c34005d00 con 0x7f5c400d4a90 2024-09-21T00:28:14.302 INFO:teuthology.orchestra.run.smithi112.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T00:28:14.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:14.303+0000 7f5c4680e640 1 -- 172.21.15.112:0/1088678217 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f5c2c078b90 msgr2=0x7f5c2c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:14.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:14.303+0000 7f5c4680e640 1 --2- 172.21.15.112:0/1088678217 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f5c2c078b90 0x7f5c2c07b0a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f5c30001090 tx=0x7f5c30009040 comp rx=0 tx=0).stop 2024-09-21T00:28:14.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:14.303+0000 7f5c4680e640 1 -- 172.21.15.112:0/1088678217 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c400d4a90 msgr2=0x7f5c400b63e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:14.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:14.303+0000 7f5c4680e640 1 --2- 172.21.15.112:0/1088678217 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c400d4a90 0x7f5c400b63e0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f5c3402aa90 tx=0x7f5c34033c20 comp rx=0 tx=0).stop 2024-09-21T00:28:14.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:14.303+0000 7f5c4680e640 1 -- 172.21.15.112:0/1088678217 shutdown_connections 2024-09-21T00:28:14.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:14.304+0000 7f5c4680e640 1 --2- 172.21.15.112:0/1088678217 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5c4015ea40 0x7f5c400d2b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:14.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:14.304+0000 7f5c4680e640 1 --2- 172.21.15.112:0/1088678217 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f5c2c078b90 0x7f5c2c07b0a0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:14.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:14.304+0000 7f5c4680e640 1 --2- 172.21.15.112:0/1088678217 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c400d4a90 0x7f5c400b63e0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:14.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:14.304+0000 7f5c4680e640 1 --2- 172.21.15.112:0/1088678217 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5c400d0ae0 0x7f5c401f9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:14.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:14.304+0000 7f5c4680e640 1 -- 172.21.15.112:0/1088678217 >> 172.21.15.112:0/1088678217 conn(0x7f5c4009d8a0 msgr2=0x7f5c4015a2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:14.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:14.304+0000 7f5c4680e640 1 -- 172.21.15.112:0/1088678217 shutdown_connections 2024-09-21T00:28:14.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:14.304+0000 7f5c4680e640 1 -- 172.21.15.112:0/1088678217 wait complete. 2024-09-21T00:28:14.315 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T00:28:14.315 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T00:28:14.315 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi112.front.sepia.ceph.com 2024-09-21T00:28:14.315 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T00:28:14.316 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T00:28:14.316 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T00:28:14.316 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-21T00:28:14.316 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T00:28:14.316 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T00:28:14.316 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:28:14.316 DEBUG:teuthology.orchestra.run.smithi112:> ip addr 2024-09-21T00:28:14.332 INFO:teuthology.orchestra.run.smithi112.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T00:28:14.332 INFO:teuthology.orchestra.run.smithi112.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T00:28:14.332 INFO:teuthology.orchestra.run.smithi112.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T00:28:14.332 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:28:14.332 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 ::1/128 scope host 2024-09-21T00:28:14.332 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:28:14.332 INFO:teuthology.orchestra.run.smithi112.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T00:28:14.332 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:d8:a5:2c brd ff:ff:ff:ff:ff:ff 2024-09-21T00:28:14.333 INFO:teuthology.orchestra.run.smithi112.stdout: altname enp7s0f0 2024-09-21T00:28:14.333 INFO:teuthology.orchestra.run.smithi112.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T00:28:14.333 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:d8:a5:2d brd ff:ff:ff:ff:ff:ff 2024-09-21T00:28:14.333 INFO:teuthology.orchestra.run.smithi112.stdout: altname enp7s0f1 2024-09-21T00:28:14.333 INFO:teuthology.orchestra.run.smithi112.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T00:28:14.333 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:8f:cb:68 brd ff:ff:ff:ff:ff:ff 2024-09-21T00:28:14.333 INFO:teuthology.orchestra.run.smithi112.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T00:28:14.333 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:8f:cb:69 brd ff:ff:ff:ff:ff:ff 2024-09-21T00:28:14.333 INFO:teuthology.orchestra.run.smithi112.stdout: inet 172.21.15.112/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-21T00:28:14.333 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft 42628sec preferred_lft 42628sec 2024-09-21T00:28:14.333 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::ec4:7aff:fe8f:cb69/64 scope link noprefixroute 2024-09-21T00:28:14.334 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:28:14.334 INFO:teuthology.orchestra.run.smithi112.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T00:28:14.334 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 1a:39:64:e7:fe:d7 brd ff:ff:ff:ff:ff:ff 2024-09-21T00:28:14.334 INFO:teuthology.orchestra.run.smithi112.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T00:28:14.334 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:28:14.334 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::5858:3dff:feaf:1774/64 scope link 2024-09-21T00:28:14.334 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:28:14.334 INFO:teuthology.orchestra.run.smithi112.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T00:28:14.334 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 1a:39:64:e7:fe:d7 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T00:28:14.334 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::1839:64ff:fee7:fed7/64 scope link 2024-09-21T00:28:14.334 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:28:14.334 INFO:teuthology.orchestra.run.smithi112.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T00:28:14.335 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 86:c9:6c:ef:e4:e9 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T00:28:14.335 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::84c9:6cff:feef:e4e9/64 scope link 2024-09-21T00:28:14.335 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:28:14.335 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:28:14.335 DEBUG:teuthology.orchestra.run.smithi112:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T00:28:14.396 INFO:teuthology.orchestra.run.smithi112.stdout:1 2024-09-21T00:28:14.397 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:28:14.398 DEBUG:teuthology.orchestra.run.smithi112:> route 2024-09-21T00:28:14.453 INFO:teuthology.orchestra.run.smithi112.stdout:Kernel IP routing table 2024-09-21T00:28:14.453 INFO:teuthology.orchestra.run.smithi112.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T00:28:14.453 INFO:teuthology.orchestra.run.smithi112.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-21T00:28:14.453 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-21T00:28:14.453 INFO:teuthology.orchestra.run.smithi112.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T00:28:14.454 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T00:28:14.454 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2024-09-21T00:28:14.454 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-21T00:28:14.454 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-21T00:28:14.454 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-21T00:28:14.454 DEBUG:teuthology.orchestra.run.smithi112:> ') 2024-09-21T00:28:14.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T00:28:14.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T00:28:14.580 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:28:14.580 DEBUG:teuthology.orchestra.run.smithi112:> ip netns list 2024-09-21T00:28:14.595 INFO:teuthology.orchestra.run.smithi112.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-21T00:28:14.595 INFO:teuthology.orchestra.run.smithi112.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T00:28:14.596 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T00:28:14.596 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:28:14.596 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link set brx.0 up 2024-09-21T00:28:14.660 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T00:28:14.660 DEBUG:teuthology.orchestra.run.smithi112:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T00:28:14.714 INFO:teuthology.orchestra.run.smithi112.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T00:28:14.715 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T00:28:14.715 DEBUG:teuthology.orchestra.run.smithi112:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T00:28:14.769 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-21T00:28:14.769 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:28:14.769 DEBUG:teuthology.orchestra.run.smithi112:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T00:28:14.863 INFO:teuthology.orchestra.run.smithi112.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T00:28:14.863 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T00:28:14.863 INFO:teuthology.orchestra.run.smithi112.stdout:invalid new device string format 2024-09-21T00:28:14.874 INFO:teuthology.orchestra.run.smithi112.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T00:28:14.874 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T00:28:14.874 INFO:teuthology.orchestra.run.smithi112.stdout:invalid new device string format 2024-09-21T00:28:14.874 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: resolved to: "172.21.15.112:3300,172.21.15.187:3300,172.21.15.187:3301" 2024-09-21T00:28:14.874 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T00:28:14.879 INFO:teuthology.orchestra.run.smithi112.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T00:28:14.880 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T00:28:14.880 INFO:teuthology.orchestra.run.smithi112.stdout:invalid new device string format 2024-09-21T00:28:14.880 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: resolved to: "172.21.15.112:3300,172.21.15.187:3300,172.21.15.187:3301" 2024-09-21T00:28:14.880 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T00:28:14.880 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: resolved to: "172.21.15.112:6789,172.21.15.187:6789,172.21.15.187:6790" 2024-09-21T00:28:14.880 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.112:6789,172.21.15.187:6789,172.21.15.187:6790:/ 2024-09-21T00:28:14.880 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=8b34692b-dc65-49b2-a08f-26f707b96ac5" will pass to kernel 2024-09-21T00:28:14.881 INFO:teuthology.orchestra.run.smithi112.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-21T00:28:14.881 INFO:teuthology.orchestra.run.smithi112.stdout: You just mounted a file system that supports labels which does not 2024-09-21T00:28:14.881 INFO:teuthology.orchestra.run.smithi112.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T00:28:14.881 INFO:teuthology.orchestra.run.smithi112.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T00:28:14.881 INFO:teuthology.orchestra.run.smithi112.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T00:28:14.882 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T00:28:14.882 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:28:14.882 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T00:28:14.912 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T00:28:15.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:15 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T00:28:15.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T00:28:15.008 INFO:teuthology.orchestra.run.smithi112.stdout:client5324 2024-09-21T00:28:15.008 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T00:28:15.008 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:28:15.023 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T00:28:15.024 DEBUG:teuthology.orchestra.run.smithi112:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T00:28:15.100 DEBUG:teuthology.orchestra.run.smithi187:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T00:28:15.145 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T00:28:15.148 DEBUG:teuthology.orchestra.run.smithi187:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T00:28:15.153 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T00:28:15.153 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T00:28:15.153 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T00:28:15.153 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:28:15.204 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T00:28:15.204 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:28:15.261 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T00:28:15.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:15.413+0000 7f23545eb640 1 -- 172.21.15.112:0/1347199167 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f234c0d2030 msgr2=0x7f234c15d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:15.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:15.413+0000 7f23545eb640 1 --2- 172.21.15.112:0/1347199167 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f234c0d2030 0x7f234c15d870 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f234c15cd80 tx=0x7f234802a530 comp rx=0 tx=0).stop 2024-09-21T00:28:15.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:15.413+0000 7f23545eb640 1 -- 172.21.15.112:0/1347199167 shutdown_connections 2024-09-21T00:28:15.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:15.413+0000 7f23545eb640 1 --2- 172.21.15.112:0/1347199167 >> v2:172.21.15.187:3300/0 conn(0x7f234c0d0860 0x7f234c15a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:15.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:15.413+0000 7f23545eb640 1 --2- 172.21.15.112:0/1347199167 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f234c0d2030 0x7f234c15d870 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:15.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:15.413+0000 7f23545eb640 1 -- 172.21.15.112:0/1347199167 >> 172.21.15.112:0/1347199167 conn(0x7f234c0b43f0 msgr2=0x7f234c0b4830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:15.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:15.413+0000 7f23545eb640 1 -- 172.21.15.112:0/1347199167 shutdown_connections 2024-09-21T00:28:15.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:15.413+0000 7f23545eb640 1 -- 172.21.15.112:0/1347199167 wait complete. 2024-09-21T00:28:15.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:15.414+0000 7f23545eb640 1 Processor -- start 2024-09-21T00:28:15.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:15.414+0000 7f23545eb640 1 -- start start 2024-09-21T00:28:15.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:15.416+0000 7f23545eb640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f234c0ccae0 0x7f234c1f6400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:15.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:15.416+0000 7f23545eb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f234c0d0860 0x7f234c0c7510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:15.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:15.416+0000 7f23545eb640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f234c0d2030 0x7f234c1f8e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:15.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:15.416+0000 7f23545eb640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f234c0b22d0 con 0x7f234c0d0860 2024-09-21T00:28:15.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:15.416+0000 7f23545eb640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f234c0b2150 con 0x7f234c0ccae0 2024-09-21T00:28:15.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:15.416+0000 7f23545eb640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f234c0b2450 con 0x7f234c0d2030 2024-09-21T00:28:15.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:15.416+0000 7f2351b86640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f234c0d0860 0x7f234c0c7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:15.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:15.416+0000 7f2351b86640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f234c0d0860 0x7f234c0c7510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33822/0 (socket says 172.21.15.112:33822) 2024-09-21T00:28:15.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:15.416+0000 7f2351b86640 1 -- 172.21.15.112:0/4002670649 learned_addr learned my addr 172.21.15.112:0/4002670649 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:28:15.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:15.416+0000 7f2352387640 1 --2- 172.21.15.112:0/4002670649 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f234c0ccae0 0x7f234c1f6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:15.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:15.416+0000 7f2352b88640 1 --2- 172.21.15.112:0/4002670649 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f234c0d2030 0x7f234c1f8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:15.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:15.416+0000 7f2351b86640 1 -- 172.21.15.112:0/4002670649 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f234c0d2030 msgr2=0x7f234c1f8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:15.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:15.416+0000 7f2351b86640 1 --2- 172.21.15.112:0/4002670649 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f234c0d2030 0x7f234c1f8e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:15.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:15.416+0000 7f2351b86640 1 -- 172.21.15.112:0/4002670649 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f234c0ccae0 msgr2=0x7f234c1f6400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:15.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:15.416+0000 7f2351b86640 1 --2- 172.21.15.112:0/4002670649 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f234c0ccae0 0x7f234c1f6400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:15.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:15.416+0000 7f2351b86640 1 -- 172.21.15.112:0/4002670649 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f234c1f99e0 con 0x7f234c0d0860 2024-09-21T00:28:15.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:15.416+0000 7f2352387640 1 --2- 172.21.15.112:0/4002670649 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f234c0ccae0 0x7f234c1f6400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:28:15.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:15.416+0000 7f2351b86640 1 --2- 172.21.15.112:0/4002670649 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f234c0d0860 0x7f234c0c7510 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f233c00d6a0 tx=0x7f233c00db60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:15.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:15.416+0000 7f2342ffd640 1 -- 172.21.15.112:0/4002670649 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f233c017070 con 0x7f234c0d0860 2024-09-21T00:28:15.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:15.416+0000 7f2342ffd640 1 -- 172.21.15.112:0/4002670649 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f233c004310 con 0x7f234c0d0860 2024-09-21T00:28:15.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:15.417+0000 7f2342ffd640 1 -- 172.21.15.112:0/4002670649 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f233c009a20 con 0x7f234c0d0860 2024-09-21T00:28:15.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:15.417+0000 7f23545eb640 1 -- 172.21.15.112:0/4002670649 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f234c1f9cc0 con 0x7f234c0d0860 2024-09-21T00:28:15.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:15.417+0000 7f23545eb640 1 -- 172.21.15.112:0/4002670649 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f234c1f5c00 con 0x7f234c0d0860 2024-09-21T00:28:15.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:15.418+0000 7f23545eb640 1 -- 172.21.15.112:0/4002670649 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f234c0b2900 con 0x7f234c0d0860 2024-09-21T00:28:15.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:15.418+0000 7f2342ffd640 1 -- 172.21.15.112:0/4002670649 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f233c009bc0 con 0x7f234c0d0860 2024-09-21T00:28:15.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:15.419+0000 7f2342ffd640 1 --2- 172.21.15.112:0/4002670649 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2330078b90 0x7f233007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:15.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:15.419+0000 7f2342ffd640 1 -- 172.21.15.112:0/4002670649 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7f233c09af20 con 0x7f234c0d0860 2024-09-21T00:28:15.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:15.419+0000 7f2352387640 1 --2- 172.21.15.112:0/4002670649 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2330078b90 0x7f233007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:15.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:15.420+0000 7f2352387640 1 --2- 172.21.15.112:0/4002670649 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2330078b90 0x7f233007b0a0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f2348034000 tx=0x7f2348004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:15.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:15.423+0000 7f2342ffd640 1 -- 172.21.15.112:0/4002670649 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f233c0631e0 con 0x7f234c0d0860 2024-09-21T00:28:15.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:15.614+0000 7f23545eb640 1 -- 172.21.15.112:0/4002670649 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f234c1f9730 con 0x7f234c0d0860 2024-09-21T00:28:15.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:15.614+0000 7f2342ffd640 1 -- 172.21.15.112:0/4002670649 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v60) ==== 80+0+0 (secure 0 0 0) 0x7f233c062ae0 con 0x7f234c0d0860 2024-09-21T00:28:15.615 INFO:teuthology.orchestra.run.smithi112.stderr:listed 0 entries 2024-09-21T00:28:15.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:15.616+0000 7f23545eb640 1 -- 172.21.15.112:0/4002670649 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2330078b90 msgr2=0x7f233007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:15.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:15.616+0000 7f23545eb640 1 --2- 172.21.15.112:0/4002670649 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2330078b90 0x7f233007b0a0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f2348034000 tx=0x7f2348004730 comp rx=0 tx=0).stop 2024-09-21T00:28:15.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:15.616+0000 7f23545eb640 1 -- 172.21.15.112:0/4002670649 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f234c0d0860 msgr2=0x7f234c0c7510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:15.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:15.616+0000 7f23545eb640 1 --2- 172.21.15.112:0/4002670649 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f234c0d0860 0x7f234c0c7510 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f233c00d6a0 tx=0x7f233c00db60 comp rx=0 tx=0).stop 2024-09-21T00:28:15.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:15.616+0000 7f23545eb640 1 -- 172.21.15.112:0/4002670649 shutdown_connections 2024-09-21T00:28:15.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:15.616+0000 7f23545eb640 1 --2- 172.21.15.112:0/4002670649 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f234c0d2030 0x7f234c1f8e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:15.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:15.616+0000 7f23545eb640 1 --2- 172.21.15.112:0/4002670649 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2330078b90 0x7f233007b0a0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:15.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:15.616+0000 7f23545eb640 1 --2- 172.21.15.112:0/4002670649 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f234c0d0860 0x7f234c0c7510 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:15.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:15.616+0000 7f23545eb640 1 --2- 172.21.15.112:0/4002670649 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f234c0ccae0 0x7f234c1f6400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:15.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:15.616+0000 7f23545eb640 1 -- 172.21.15.112:0/4002670649 >> 172.21.15.112:0/4002670649 conn(0x7f234c0b43f0 msgr2=0x7f234c15f150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:15.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:15.616+0000 7f23545eb640 1 -- 172.21.15.112:0/4002670649 shutdown_connections 2024-09-21T00:28:15.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:15.616+0000 7f23545eb640 1 -- 172.21.15.112:0/4002670649 wait complete. 2024-09-21T00:28:15.628 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-21T00:28:15.628 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:28:15.628 DEBUG:teuthology.orchestra.run.smithi112:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-21T00:28:15.656 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi112.front.sepia.ceph.com 2024-09-21T00:28:15.656 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:28:15.656 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T00:28:15.712 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T00:28:15.712 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-09-21T00:28:15.712 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:28:15.732 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-09-21T00:28:15.732 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:28:15.791 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T00:28:15.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:15.906+0000 7f7b99ef9640 1 -- 172.21.15.112:0/3303577552 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7b940b46f0 msgr2=0x7f7b940b4b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:15.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:15.906+0000 7f7b99ef9640 1 --2- 172.21.15.112:0/3303577552 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7b940b46f0 0x7f7b940b4b00 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c005fd0 tx=0x7f7b8c02a820 comp rx=0 tx=0).stop 2024-09-21T00:28:15.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:15.906+0000 7f7b99ef9640 1 -- 172.21.15.112:0/3303577552 shutdown_connections 2024-09-21T00:28:15.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:15.906+0000 7f7b99ef9640 1 --2- 172.21.15.112:0/3303577552 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7b940b46f0 0x7f7b940b4b00 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:15.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:15.906+0000 7f7b99ef9640 1 --2- 172.21.15.112:0/3303577552 >> v2:172.21.15.187:3300/0 conn(0x7f7b9415f0a0 0x7f7b9415f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:15.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:15.906+0000 7f7b99ef9640 1 -- 172.21.15.112:0/3303577552 >> 172.21.15.112:0/3303577552 conn(0x7f7b94071690 msgr2=0x7f7b94071aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:15.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:15.906+0000 7f7b99ef9640 1 -- 172.21.15.112:0/3303577552 shutdown_connections 2024-09-21T00:28:15.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:15.906+0000 7f7b99ef9640 1 -- 172.21.15.112:0/3303577552 wait complete. 2024-09-21T00:28:15.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:15.907+0000 7f7b99ef9640 1 Processor -- start 2024-09-21T00:28:15.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:15.908+0000 7f7b99ef9640 1 -- start start 2024-09-21T00:28:15.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:15.909+0000 7f7b99ef9640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7b94070540 0x7f7b940a94b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:15.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:15.909+0000 7f7b99ef9640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7b940b46f0 0x7f7b940b03e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:15.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:15.909+0000 7f7b99ef9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7b9415f0a0 0x7f7b941f6b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:15.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:15.909+0000 7f7b99ef9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7b940b53d0 con 0x7f7b9415f0a0 2024-09-21T00:28:15.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:15.909+0000 7f7b99ef9640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7b940b5250 con 0x7f7b940b46f0 2024-09-21T00:28:15.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:15.909+0000 7f7b99ef9640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f7b940b5550 con 0x7f7b94070540 2024-09-21T00:28:15.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:15.909+0000 7f7b93fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7b9415f0a0 0x7f7b941f6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:15.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:15.909+0000 7f7b93fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7b9415f0a0 0x7f7b941f6b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33832/0 (socket says 172.21.15.112:33832) 2024-09-21T00:28:15.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:15.909+0000 7f7b93fff640 1 -- 172.21.15.112:0/2665369312 learned_addr learned my addr 172.21.15.112:0/2665369312 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:28:15.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:15.909+0000 7f7b92ffd640 1 --2- 172.21.15.112:0/2665369312 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7b940b46f0 0x7f7b940b03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:15.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:15.909+0000 7f7b937fe640 1 --2- 172.21.15.112:0/2665369312 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7b94070540 0x7f7b940a94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:15.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:15.909+0000 7f7b93fff640 1 -- 172.21.15.112:0/2665369312 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7b94070540 msgr2=0x7f7b940a94b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:15.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:15.909+0000 7f7b93fff640 1 --2- 172.21.15.112:0/2665369312 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7b94070540 0x7f7b940a94b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:15.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:15.909+0000 7f7b93fff640 1 -- 172.21.15.112:0/2665369312 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7b940b46f0 msgr2=0x7f7b940b03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:15.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:15.909+0000 7f7b93fff640 1 --2- 172.21.15.112:0/2665369312 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7b940b46f0 0x7f7b940b03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:15.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:15.910+0000 7f7b93fff640 1 -- 172.21.15.112:0/2665369312 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b940a99f0 con 0x7f7b9415f0a0 2024-09-21T00:28:15.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:15.910+0000 7f7b93fff640 1 --2- 172.21.15.112:0/2665369312 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7b9415f0a0 0x7f7b941f6b70 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c009b30 tx=0x7f7b8c032be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:15.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:15.910+0000 7f7b7bfff640 1 -- 172.21.15.112:0/2665369312 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b8c037040 con 0x7f7b9415f0a0 2024-09-21T00:28:15.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:15.910+0000 7f7b7bfff640 1 -- 172.21.15.112:0/2665369312 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7b8c042070 con 0x7f7b9415f0a0 2024-09-21T00:28:15.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:15.910+0000 7f7b99ef9640 1 -- 172.21.15.112:0/2665369312 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b940c8de0 con 0x7f7b9415f0a0 2024-09-21T00:28:15.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:15.910+0000 7f7b7bfff640 1 -- 172.21.15.112:0/2665369312 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b8c004040 con 0x7f7b9415f0a0 2024-09-21T00:28:15.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:15.910+0000 7f7b99ef9640 1 -- 172.21.15.112:0/2665369312 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b940c93a0 con 0x7f7b9415f0a0 2024-09-21T00:28:15.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:15.911+0000 7f7b99ef9640 1 -- 172.21.15.112:0/2665369312 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b940b4b00 con 0x7f7b9415f0a0 2024-09-21T00:28:15.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:15.912+0000 7f7b7bfff640 1 -- 172.21.15.112:0/2665369312 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f7b8c0041e0 con 0x7f7b9415f0a0 2024-09-21T00:28:15.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:15.912+0000 7f7b7bfff640 1 --2- 172.21.15.112:0/2665369312 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f7b80078b90 0x7f7b8007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:15.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:15.912+0000 7f7b7bfff640 1 -- 172.21.15.112:0/2665369312 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7f7b8c0bbb30 con 0x7f7b9415f0a0 2024-09-21T00:28:15.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:15.912+0000 7f7b937fe640 1 --2- 172.21.15.112:0/2665369312 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f7b80078b90 0x7f7b8007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:15.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:15.913+0000 7f7b937fe640 1 --2- 172.21.15.112:0/2665369312 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f7b80078b90 0x7f7b8007b0a0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f7b84007fd0 tx=0x7f7b84001370 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:15.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:15.917+0000 7f7b7bfff640 1 -- 172.21.15.112:0/2665369312 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f7b8c083710 con 0x7f7b9415f0a0 2024-09-21T00:28:16.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:16.106+0000 7f7b99ef9640 1 -- 172.21.15.112:0/2665369312 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f7b9406ff10 con 0x7f7b9415f0a0 2024-09-21T00:28:16.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:16.106+0000 7f7b7bfff640 1 -- 172.21.15.112:0/2665369312 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v60) ==== 80+0+0 (secure 0 0 0) 0x7f7b8c088140 con 0x7f7b9415f0a0 2024-09-21T00:28:16.107 INFO:teuthology.orchestra.run.smithi112.stderr:listed 0 entries 2024-09-21T00:28:16.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:16.109+0000 7f7b99ef9640 1 -- 172.21.15.112:0/2665369312 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f7b80078b90 msgr2=0x7f7b8007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:16.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:16.109+0000 7f7b99ef9640 1 --2- 172.21.15.112:0/2665369312 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f7b80078b90 0x7f7b8007b0a0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f7b84007fd0 tx=0x7f7b84001370 comp rx=0 tx=0).stop 2024-09-21T00:28:16.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:16.109+0000 7f7b99ef9640 1 -- 172.21.15.112:0/2665369312 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7b9415f0a0 msgr2=0x7f7b941f6b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:16.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:16.109+0000 7f7b99ef9640 1 --2- 172.21.15.112:0/2665369312 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7b9415f0a0 0x7f7b941f6b70 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c009b30 tx=0x7f7b8c032be0 comp rx=0 tx=0).stop 2024-09-21T00:28:16.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:16.109+0000 7f7b99ef9640 1 -- 172.21.15.112:0/2665369312 shutdown_connections 2024-09-21T00:28:16.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:16.109+0000 7f7b99ef9640 1 --2- 172.21.15.112:0/2665369312 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f7b80078b90 0x7f7b8007b0a0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:16.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:16.109+0000 7f7b99ef9640 1 --2- 172.21.15.112:0/2665369312 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7b9415f0a0 0x7f7b941f6b70 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:16.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:16.109+0000 7f7b99ef9640 1 --2- 172.21.15.112:0/2665369312 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7b940b46f0 0x7f7b940b03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:16.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:16.109+0000 7f7b99ef9640 1 --2- 172.21.15.112:0/2665369312 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7b94070540 0x7f7b940a94b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:16.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:16.109+0000 7f7b99ef9640 1 -- 172.21.15.112:0/2665369312 >> 172.21.15.112:0/2665369312 conn(0x7f7b94071690 msgr2=0x7f7b94068fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:16.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:16.109+0000 7f7b99ef9640 1 -- 172.21.15.112:0/2665369312 shutdown_connections 2024-09-21T00:28:16.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:16.109+0000 7f7b99ef9640 1 -- 172.21.15.112:0/2665369312 wait complete. 2024-09-21T00:28:16.120 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-09-21T00:28:16.120 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:28:16.121 DEBUG:teuthology.orchestra.run.smithi187:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-09-21T00:28:16.185 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi187.front.sepia.ceph.com 2024-09-21T00:28:16.185 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:28:16.185 DEBUG:teuthology.orchestra.run.smithi187:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-21T00:28:16.221 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T00:28:16.221 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T00:28:16.221 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:28:16.240 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056010,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1439 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248716k,nr_inodes=812179,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T00:28:16.241 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T00:28:16.241 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-09-21T00:28:16.241 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:28:16.277 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056013,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497440k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16835 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248720k,nr_inodes=812180,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497440k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T00:28:16.278 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T00:28:16.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:16.393+0000 7f4a3b952640 1 -- 172.21.15.112:0/2215105839 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a20003b30 msgr2=0x7f4a20003fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:16.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:16.393+0000 7f4a3b952640 1 --2- 172.21.15.112:0/2215105839 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a20003b30 0x7f4a20003fa0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f4a30009ec0 tx=0x7f4a20005830 comp rx=0 tx=0).stop 2024-09-21T00:28:16.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:16.393+0000 7f4a3b952640 1 -- 172.21.15.112:0/2215105839 shutdown_connections 2024-09-21T00:28:16.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:16.393+0000 7f4a3b952640 1 --2- 172.21.15.112:0/2215105839 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a20003b30 0x7f4a20003fa0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:16.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:16.393+0000 7f4a3b952640 1 -- 172.21.15.112:0/2215105839 >> 172.21.15.112:0/2215105839 conn(0x7f4a3409d8a0 msgr2=0x7f4a3408bfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:16.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:16.393+0000 7f4a3b952640 1 -- 172.21.15.112:0/2215105839 shutdown_connections 2024-09-21T00:28:16.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:16.393+0000 7f4a3b952640 1 -- 172.21.15.112:0/2215105839 wait complete. 2024-09-21T00:28:16.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:16.395+0000 7f4a3b952640 1 Processor -- start 2024-09-21T00:28:16.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:16.395+0000 7f4a3b952640 1 -- start start 2024-09-21T00:28:16.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:16.396+0000 7f4a3b952640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4a20003b30 0x7f4a341f49f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:16.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:16.396+0000 7f4a3b952640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a3406efd0 0x7f4a3406f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:16.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:16.396+0000 7f4a3b952640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4a340b5060 0x7f4a340b5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:16.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:16.396+0000 7f4a3b952640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4a340705e0 con 0x7f4a3406efd0 2024-09-21T00:28:16.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:16.396+0000 7f4a3b952640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f4a34070460 con 0x7f4a340b5060 2024-09-21T00:28:16.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:16.396+0000 7f4a3b952640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f4a34070760 con 0x7f4a20003b30 2024-09-21T00:28:16.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:16.396+0000 7f4a38eed640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a3406efd0 0x7f4a3406f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:16.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:16.396+0000 7f4a38eed640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a3406efd0 0x7f4a3406f3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42276/0 (socket says 172.21.15.112:42276) 2024-09-21T00:28:16.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:16.396+0000 7f4a38eed640 1 -- 172.21.15.112:0/4094990260 learned_addr learned my addr 172.21.15.112:0/4094990260 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:28:16.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:16.396+0000 7f4a396ee640 1 --2- 172.21.15.112:0/4094990260 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4a20003b30 0x7f4a341f49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:16.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:16.396+0000 7f4a39eef640 1 --2- 172.21.15.112:0/4094990260 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4a340b5060 0x7f4a340b5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:16.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:16.396+0000 7f4a38eed640 1 -- 172.21.15.112:0/4094990260 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4a20003b30 msgr2=0x7f4a341f49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:16.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:16.396+0000 7f4a38eed640 1 --2- 172.21.15.112:0/4094990260 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4a20003b30 0x7f4a341f49f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:16.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:16.396+0000 7f4a38eed640 1 -- 172.21.15.112:0/4094990260 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4a340b5060 msgr2=0x7f4a340b5470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:16.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:16.396+0000 7f4a38eed640 1 --2- 172.21.15.112:0/4094990260 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4a340b5060 0x7f4a340b5470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:16.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:16.396+0000 7f4a38eed640 1 -- 172.21.15.112:0/4094990260 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a340b59b0 con 0x7f4a3406efd0 2024-09-21T00:28:16.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:16.396+0000 7f4a38eed640 1 --2- 172.21.15.112:0/4094990260 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a3406efd0 0x7f4a3406f3e0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f4a24003dd0 tx=0x7f4a24001630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:16.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:16.397+0000 7f4a29ffb640 1 -- 172.21.15.112:0/4094990260 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a24015070 con 0x7f4a3406efd0 2024-09-21T00:28:16.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:16.397+0000 7f4a29ffb640 1 -- 172.21.15.112:0/4094990260 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4a24011840 con 0x7f4a3406efd0 2024-09-21T00:28:16.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:16.397+0000 7f4a3b952640 1 -- 172.21.15.112:0/4094990260 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a3406eda0 con 0x7f4a3406efd0 2024-09-21T00:28:16.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:16.397+0000 7f4a29ffb640 1 -- 172.21.15.112:0/4094990260 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a24001d40 con 0x7f4a3406efd0 2024-09-21T00:28:16.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:16.397+0000 7f4a3b952640 1 -- 172.21.15.112:0/4094990260 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a341f8d00 con 0x7f4a3406efd0 2024-09-21T00:28:16.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:16.398+0000 7f4a3b952640 1 -- 172.21.15.112:0/4094990260 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a3406fa70 con 0x7f4a3406efd0 2024-09-21T00:28:16.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:16.398+0000 7f4a29ffb640 1 -- 172.21.15.112:0/4094990260 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f4a24018070 con 0x7f4a3406efd0 2024-09-21T00:28:16.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:16.399+0000 7f4a29ffb640 1 --2- 172.21.15.112:0/4094990260 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4a18078b90 0x7f4a1807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:16.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:16.399+0000 7f4a29ffb640 1 -- 172.21.15.112:0/4094990260 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7f4a2409b8e0 con 0x7f4a3406efd0 2024-09-21T00:28:16.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:16.399+0000 7f4a396ee640 1 --2- 172.21.15.112:0/4094990260 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4a18078b90 0x7f4a1807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:16.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:16.400+0000 7f4a396ee640 1 --2- 172.21.15.112:0/4094990260 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4a18078b90 0x7f4a1807b0a0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f4a30035000 tx=0x7f4a3002bf90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:16.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:16.403+0000 7f4a29ffb640 1 -- 172.21.15.112:0/4094990260 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f4a340b59b0 con 0x7f4a3406efd0 2024-09-21T00:28:16.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:16.617+0000 7f4a3b952640 1 -- 172.21.15.112:0/4094990260 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4a340d4620 con 0x7f4a3406efd0 2024-09-21T00:28:16.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:16.618+0000 7f4a29ffb640 1 -- 172.21.15.112:0/4094990260 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+5253 (secure 0 0 0) 0x7f4a240635d0 con 0x7f4a3406efd0 2024-09-21T00:28:16.618 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:28:16.619 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":39,"btime":"2024-09-21T00:28:01:807440+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5389,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6836/3984762342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":3984762342},{"type":"v1","addr":"172.21.15.187:6836","nonce":3984762342}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5392,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6837/763698806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6835","nonce":763698806},{"type":"v1","addr":"172.21.15.187:6837","nonce":763698806}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T00:27:34.871721+0000","modified":"2024-09-21T00:28:00.832760+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5739,"mds_1":5736},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5736":{"gid":5736,"name":"c","rank":1,"incarnation":37,"state":"up:active","state_seq":5,"addr":"172.21.15.112:6834/1432588455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1432588455},{"type":"v1","addr":"172.21.15.112:6834","nonce":1432588455}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5739":{"gid":5739,"name":"a","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6835/3610025869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":3610025869},{"type":"v1","addr":"172.21.15.112:6835","nonce":3610025869}]},"join_fscid":-1,"export_targets":[1],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5739,"qdb_cluster":[5739,5736]},"id":4}]} 2024-09-21T00:28:16.619 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 39 2024-09-21T00:28:16.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:16.620+0000 7f4a3b952640 1 -- 172.21.15.112:0/4094990260 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4a18078b90 msgr2=0x7f4a1807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:16.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:16.620+0000 7f4a3b952640 1 --2- 172.21.15.112:0/4094990260 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4a18078b90 0x7f4a1807b0a0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f4a30035000 tx=0x7f4a3002bf90 comp rx=0 tx=0).stop 2024-09-21T00:28:16.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:16.620+0000 7f4a3b952640 1 -- 172.21.15.112:0/4094990260 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a3406efd0 msgr2=0x7f4a3406f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:16.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:16.620+0000 7f4a3b952640 1 --2- 172.21.15.112:0/4094990260 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a3406efd0 0x7f4a3406f3e0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f4a24003dd0 tx=0x7f4a24001630 comp rx=0 tx=0).stop 2024-09-21T00:28:16.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:16.620+0000 7f4a3b952640 1 -- 172.21.15.112:0/4094990260 shutdown_connections 2024-09-21T00:28:16.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:16.620+0000 7f4a3b952640 1 --2- 172.21.15.112:0/4094990260 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4a340b5060 0x7f4a340b5470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:16.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:16.620+0000 7f4a3b952640 1 --2- 172.21.15.112:0/4094990260 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4a18078b90 0x7f4a1807b0a0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:16.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:16.620+0000 7f4a3b952640 1 --2- 172.21.15.112:0/4094990260 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a3406efd0 0x7f4a3406f3e0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:16.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:16.620+0000 7f4a3b952640 1 --2- 172.21.15.112:0/4094990260 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4a20003b30 0x7f4a341f49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:16.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:16.620+0000 7f4a3b952640 1 -- 172.21.15.112:0/4094990260 >> 172.21.15.112:0/4094990260 conn(0x7f4a3409d8a0 msgr2=0x7f4a34160d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:16.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:16.620+0000 7f4a3b952640 1 -- 172.21.15.112:0/4094990260 shutdown_connections 2024-09-21T00:28:16.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:16.620+0000 7f4a3b952640 1 -- 172.21.15.112:0/4094990260 wait complete. 2024-09-21T00:28:16.632 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T00:28:16.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:16.742+0000 7f1604238640 1 -- 172.21.15.112:0/428434970 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15fc15ca50 msgr2=0x7f15fc0b41e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:16.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:16.742+0000 7f1604238640 1 --2- 172.21.15.112:0/428434970 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15fc15ca50 0x7f15fc0b41e0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f15f8005fd0 tx=0x7f15f802a530 comp rx=0 tx=0).stop 2024-09-21T00:28:16.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:16.742+0000 7f1604238640 1 -- 172.21.15.112:0/428434970 shutdown_connections 2024-09-21T00:28:16.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:16.742+0000 7f1604238640 1 --2- 172.21.15.112:0/428434970 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f15fc0b4720 0x7f15fc0b22d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:16.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:16.742+0000 7f1604238640 1 --2- 172.21.15.112:0/428434970 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15fc15ca50 0x7f15fc0b41e0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:16.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:16.742+0000 7f1604238640 1 -- 172.21.15.112:0/428434970 >> 172.21.15.112:0/428434970 conn(0x7f15fc0cf220 msgr2=0x7f15fc0cf630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:16.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:16.743+0000 7f1604238640 1 -- 172.21.15.112:0/428434970 shutdown_connections 2024-09-21T00:28:16.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:16.743+0000 7f1604238640 1 -- 172.21.15.112:0/428434970 wait complete. 2024-09-21T00:28:16.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:16.744+0000 7f1604238640 1 Processor -- start 2024-09-21T00:28:16.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:16.744+0000 7f1604238640 1 -- start start 2024-09-21T00:28:16.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:16.745+0000 7f1604238640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f15fc0b4720 0x7f15fc0cb500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:16.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:16.745+0000 7f1604238640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f15fc0ccce0 0x7f15fc157a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:16.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:16.745+0000 7f1604238640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15fc15ca50 0x7f15fc157fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:16.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:16.745+0000 7f1604238640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f15fc0cd320 con 0x7f15fc15ca50 2024-09-21T00:28:16.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:16.745+0000 7f1604238640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f15fc0cd1a0 con 0x7f15fc0ccce0 2024-09-21T00:28:16.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:16.745+0000 7f1604238640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f15fc0b2810 con 0x7f15fc0b4720 2024-09-21T00:28:16.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:16.746+0000 7f16027d5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15fc15ca50 0x7f15fc157fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:16.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:16.746+0000 7f16027d5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15fc15ca50 0x7f15fc157fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42306/0 (socket says 172.21.15.112:42306) 2024-09-21T00:28:16.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:16.746+0000 7f16017d3640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f15fc0ccce0 0x7f15fc157a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:16.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:16.746+0000 7f16027d5640 1 -- 172.21.15.112:0/2223201239 learned_addr learned my addr 172.21.15.112:0/2223201239 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:28:16.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:16.746+0000 7f1601fd4640 1 --2- 172.21.15.112:0/2223201239 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f15fc0b4720 0x7f15fc0cb500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:16.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:16.746+0000 7f16027d5640 1 -- 172.21.15.112:0/2223201239 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f15fc0b4720 msgr2=0x7f15fc0cb500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:16.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:16.746+0000 7f16027d5640 1 --2- 172.21.15.112:0/2223201239 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f15fc0b4720 0x7f15fc0cb500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:16.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:16.746+0000 7f16027d5640 1 -- 172.21.15.112:0/2223201239 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f15fc0ccce0 msgr2=0x7f15fc157a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:16.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:16.746+0000 7f16027d5640 1 --2- 172.21.15.112:0/2223201239 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f15fc0ccce0 0x7f15fc157a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:16.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:16.746+0000 7f16027d5640 1 -- 172.21.15.112:0/2223201239 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15fc1586f0 con 0x7f15fc15ca50 2024-09-21T00:28:16.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:16.746+0000 7f16027d5640 1 --2- 172.21.15.112:0/2223201239 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15fc15ca50 0x7f15fc157fd0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f15f400e960 tx=0x7f15f400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:16.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:16.746+0000 7f15f27fc640 1 -- 172.21.15.112:0/2223201239 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15f4004d20 con 0x7f15fc15ca50 2024-09-21T00:28:16.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:16.746+0000 7f15f27fc640 1 -- 172.21.15.112:0/2223201239 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f15f400cac0 con 0x7f15fc15ca50 2024-09-21T00:28:16.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:16.746+0000 7f1604238640 1 -- 172.21.15.112:0/2223201239 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15fc1589d0 con 0x7f15fc15ca50 2024-09-21T00:28:16.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:16.746+0000 7f15f27fc640 1 -- 172.21.15.112:0/2223201239 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15f4005020 con 0x7f15fc15ca50 2024-09-21T00:28:16.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:16.746+0000 7f1604238640 1 -- 172.21.15.112:0/2223201239 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15fc158f90 con 0x7f15fc15ca50 2024-09-21T00:28:16.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:16.747+0000 7f1604238640 1 -- 172.21.15.112:0/2223201239 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15fc0d2140 con 0x7f15fc15ca50 2024-09-21T00:28:16.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:16.748+0000 7f15f27fc640 1 -- 172.21.15.112:0/2223201239 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f15f40051c0 con 0x7f15fc15ca50 2024-09-21T00:28:16.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:16.749+0000 7f15f27fc640 1 --2- 172.21.15.112:0/2223201239 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f15e0078b90 0x7f15e007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:16.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:16.749+0000 7f15f27fc640 1 -- 172.21.15.112:0/2223201239 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7f15f409b6b0 con 0x7f15fc15ca50 2024-09-21T00:28:16.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:16.749+0000 7f1601fd4640 1 --2- 172.21.15.112:0/2223201239 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f15e0078b90 0x7f15e007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:16.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:16.749+0000 7f1601fd4640 1 --2- 172.21.15.112:0/2223201239 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f15e0078b90 0x7f15e007b0a0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f15f8033000 tx=0x7f15f8004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:16.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:16.753+0000 7f15f27fc640 1 -- 172.21.15.112:0/2223201239 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f15f4016030 con 0x7f15fc15ca50 2024-09-21T00:28:16.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:16.966+0000 7f1604238640 1 -- 172.21.15.112:0/2223201239 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f15fc0b44a0 con 0x7f15fc15ca50 2024-09-21T00:28:16.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:16.966+0000 7f15f27fc640 1 -- 172.21.15.112:0/2223201239 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+5253 (secure 0 0 0) 0x7f15f40633a0 con 0x7f15fc15ca50 2024-09-21T00:28:16.967 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:28:16.967 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":39,"btime":"2024-09-21T00:28:01:807440+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5389,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6836/3984762342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":3984762342},{"type":"v1","addr":"172.21.15.187:6836","nonce":3984762342}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5392,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6837/763698806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6835","nonce":763698806},{"type":"v1","addr":"172.21.15.187:6837","nonce":763698806}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T00:27:34.871721+0000","modified":"2024-09-21T00:28:00.832760+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5739,"mds_1":5736},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5736":{"gid":5736,"name":"c","rank":1,"incarnation":37,"state":"up:active","state_seq":5,"addr":"172.21.15.112:6834/1432588455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1432588455},{"type":"v1","addr":"172.21.15.112:6834","nonce":1432588455}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5739":{"gid":5739,"name":"a","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6835/3610025869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":3610025869},{"type":"v1","addr":"172.21.15.112:6835","nonce":3610025869}]},"join_fscid":-1,"export_targets":[1],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5739,"qdb_cluster":[5739,5736]},"id":4}]} 2024-09-21T00:28:16.967 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 39 2024-09-21T00:28:16.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:16.968+0000 7f1604238640 1 -- 172.21.15.112:0/2223201239 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f15e0078b90 msgr2=0x7f15e007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:16.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:16.968+0000 7f1604238640 1 --2- 172.21.15.112:0/2223201239 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f15e0078b90 0x7f15e007b0a0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f15f8033000 tx=0x7f15f8004730 comp rx=0 tx=0).stop 2024-09-21T00:28:16.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:16.968+0000 7f1604238640 1 -- 172.21.15.112:0/2223201239 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15fc15ca50 msgr2=0x7f15fc157fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:16.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:16.968+0000 7f1604238640 1 --2- 172.21.15.112:0/2223201239 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15fc15ca50 0x7f15fc157fd0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f15f400e960 tx=0x7f15f400ee20 comp rx=0 tx=0).stop 2024-09-21T00:28:16.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:16.968+0000 7f1604238640 1 -- 172.21.15.112:0/2223201239 shutdown_connections 2024-09-21T00:28:16.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:16.968+0000 7f1604238640 1 --2- 172.21.15.112:0/2223201239 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f15e0078b90 0x7f15e007b0a0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:16.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:16.968+0000 7f1604238640 1 --2- 172.21.15.112:0/2223201239 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15fc15ca50 0x7f15fc157fd0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:16.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:16.968+0000 7f1604238640 1 --2- 172.21.15.112:0/2223201239 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f15fc0ccce0 0x7f15fc157a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:16.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:16.968+0000 7f1604238640 1 --2- 172.21.15.112:0/2223201239 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f15fc0b4720 0x7f15fc0cb500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:16.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:16.968+0000 7f1604238640 1 -- 172.21.15.112:0/2223201239 >> 172.21.15.112:0/2223201239 conn(0x7f15fc0cf220 msgr2=0x7f15fc0cf630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:16.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:16.968+0000 7f1604238640 1 -- 172.21.15.112:0/2223201239 shutdown_connections 2024-09-21T00:28:16.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:16.968+0000 7f1604238640 1 -- 172.21.15.112:0/2223201239 wait complete. 2024-09-21T00:28:16.980 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T00:28:17.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.093+0000 7f77d37e4640 1 -- 172.21.15.112:0/2088629902 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f77cc0d4a90 msgr2=0x7f77cc0d38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:17.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.093+0000 7f77d37e4640 1 --2- 172.21.15.112:0/2088629902 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f77cc0d4a90 0x7f77cc0d38c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f77bc005fd0 tx=0x7f77bc02a540 comp rx=0 tx=0).stop 2024-09-21T00:28:17.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.093+0000 7f77d37e4640 1 -- 172.21.15.112:0/2088629902 shutdown_connections 2024-09-21T00:28:17.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.093+0000 7f77d37e4640 1 --2- 172.21.15.112:0/2088629902 >> v2:172.21.15.187:3300/0 conn(0x7f77cc15ea40 0x7f77cc070250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:17.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.093+0000 7f77d37e4640 1 --2- 172.21.15.112:0/2088629902 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f77cc0d4a90 0x7f77cc0d38c0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:17.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.093+0000 7f77d37e4640 1 -- 172.21.15.112:0/2088629902 >> 172.21.15.112:0/2088629902 conn(0x7f77cc09d8a0 msgr2=0x7f77cc08bfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:17.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.093+0000 7f77d37e4640 1 -- 172.21.15.112:0/2088629902 shutdown_connections 2024-09-21T00:28:17.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.093+0000 7f77d37e4640 1 -- 172.21.15.112:0/2088629902 wait complete. 2024-09-21T00:28:17.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.095+0000 7f77d37e4640 1 Processor -- start 2024-09-21T00:28:17.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.095+0000 7f77d37e4640 1 -- start start 2024-09-21T00:28:17.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.096+0000 7f77d37e4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77cc0d0ae0 0x7f77cc06e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:17.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.096+0000 7f77d37e4640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f77cc0d4a90 0x7f77cc06dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:17.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.096+0000 7f77d37e4640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f77cc15ea40 0x7f77cc159fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:17.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.096+0000 7f77d37e4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f77cc162b50 con 0x7f77cc0d0ae0 2024-09-21T00:28:17.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.096+0000 7f77d37e4640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f77cc1629d0 con 0x7f77cc0d4a90 2024-09-21T00:28:17.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.096+0000 7f77d37e4640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f77cc162cd0 con 0x7f77cc15ea40 2024-09-21T00:28:17.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.096+0000 7f77d1580640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77cc0d0ae0 0x7f77cc06e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:17.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.096+0000 7f77d1580640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77cc0d0ae0 0x7f77cc06e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42310/0 (socket says 172.21.15.112:42310) 2024-09-21T00:28:17.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.096+0000 7f77d1580640 1 -- 172.21.15.112:0/3249698407 learned_addr learned my addr 172.21.15.112:0/3249698407 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:28:17.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.096+0000 7f77d0d7f640 1 --2- 172.21.15.112:0/3249698407 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f77cc0d4a90 0x7f77cc06dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:17.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.096+0000 7f77d1d81640 1 --2- 172.21.15.112:0/3249698407 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f77cc15ea40 0x7f77cc159fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:17.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.096+0000 7f77d1580640 1 -- 172.21.15.112:0/3249698407 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f77cc15ea40 msgr2=0x7f77cc159fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:17.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.096+0000 7f77d1580640 1 --2- 172.21.15.112:0/3249698407 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f77cc15ea40 0x7f77cc159fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:17.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.096+0000 7f77d1580640 1 -- 172.21.15.112:0/3249698407 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f77cc0d4a90 msgr2=0x7f77cc06dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:17.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.096+0000 7f77d1580640 1 --2- 172.21.15.112:0/3249698407 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f77cc0d4a90 0x7f77cc06dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:17.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.096+0000 7f77d1580640 1 -- 172.21.15.112:0/3249698407 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77cc0cbfc0 con 0x7f77cc0d0ae0 2024-09-21T00:28:17.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.096+0000 7f77d1d81640 1 --2- 172.21.15.112:0/3249698407 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f77cc15ea40 0x7f77cc159fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:28:17.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.097+0000 7f77d1580640 1 --2- 172.21.15.112:0/3249698407 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77cc0d0ae0 0x7f77cc06e4b0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f77c800c3f0 tx=0x7f77c800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:17.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.097+0000 7f77c1ffb640 1 -- 172.21.15.112:0/3249698407 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c80052d0 con 0x7f77cc0d0ae0 2024-09-21T00:28:17.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.097+0000 7f77c1ffb640 1 -- 172.21.15.112:0/3249698407 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f77c8006820 con 0x7f77cc0d0ae0 2024-09-21T00:28:17.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.097+0000 7f77d37e4640 1 -- 172.21.15.112:0/3249698407 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77cc06db40 con 0x7f77cc0d0ae0 2024-09-21T00:28:17.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.097+0000 7f77c1ffb640 1 -- 172.21.15.112:0/3249698407 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c8006c70 con 0x7f77cc0d0ae0 2024-09-21T00:28:17.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.097+0000 7f77d37e4640 1 -- 172.21.15.112:0/3249698407 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77cc1f9070 con 0x7f77cc0d0ae0 2024-09-21T00:28:17.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.098+0000 7f77d37e4640 1 -- 172.21.15.112:0/3249698407 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77cc1631f0 con 0x7f77cc0d0ae0 2024-09-21T00:28:17.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.099+0000 7f77c1ffb640 1 -- 172.21.15.112:0/3249698407 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f77c8007020 con 0x7f77cc0d0ae0 2024-09-21T00:28:17.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.099+0000 7f77c1ffb640 1 --2- 172.21.15.112:0/3249698407 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f77b0078bc0 0x7f77b007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:17.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.099+0000 7f77c1ffb640 1 -- 172.21.15.112:0/3249698407 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7f77c809a220 con 0x7f77cc0d0ae0 2024-09-21T00:28:17.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.099+0000 7f77d0d7f640 1 --2- 172.21.15.112:0/3249698407 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f77b0078bc0 0x7f77b007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:17.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.100+0000 7f77d0d7f640 1 --2- 172.21.15.112:0/3249698407 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f77b0078bc0 0x7f77b007b0d0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f77bc033000 tx=0x7f77bc004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:17.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.104+0000 7f77c1ffb640 1 -- 172.21.15.112:0/3249698407 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f77c8061f10 con 0x7f77cc0d0ae0 2024-09-21T00:28:17.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.281+0000 7f77d37e4640 1 -- 172.21.15.112:0/3249698407 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f77cc0b2140 con 0x7f77cc0d0ae0 2024-09-21T00:28:17.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.282+0000 7f77c1ffb640 1 -- 172.21.15.112:0/3249698407 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v60) ==== 74+0+16968 (secure 0 0 0) 0x7f77c8066940 con 0x7f77cc0d0ae0 2024-09-21T00:28:17.284 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:28:17.284 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":60,"fsid":"8b34692b-dc65-49b2-a08f-26f707b96ac5","created":"2024-09-21T00:25:22.788490+0000","modified":"2024-09-21T00:27:36.511536+0000","last_up_change":"2024-09-21T00:25:36.161835+0000","last_in_change":"2024-09-21T00:25:26.346500+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T00:25:37.105006+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-09-21T00:27:32.510522+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-09-21T00:27:33.828833+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9c575086-67d2-4de2-b01b-d44dcc21d604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6809","nonce":755760621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6811","nonce":755760621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6815","nonce":755760621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6813","nonce":755760621}]},"public_addr":"172.21.15.112:6809/755760621","cluster_addr":"172.21.15.112:6811/755760621","heartbeat_back_addr":"172.21.15.112:6815/755760621","heartbeat_front_addr":"172.21.15.112:6813/755760621","state":["exists","up"]},{"osd":1,"uuid":"a983c227-a3ce-4bdd-82f1-438ba7e3ecd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6801","nonce":344932592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6803","nonce":344932592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6807","nonce":344932592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6805","nonce":344932592}]},"public_addr":"172.21.15.112:6801/344932592","cluster_addr":"172.21.15.112:6803/344932592","heartbeat_back_addr":"172.21.15.112:6807/344932592","heartbeat_front_addr":"172.21.15.112:6805/344932592","state":["exists","up"]},{"osd":2,"uuid":"1193c1d8-bfff-4a57-8200-5f69bf1ea3af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6825","nonce":407933616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6827","nonce":407933616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6831","nonce":407933616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6829","nonce":407933616}]},"public_addr":"172.21.15.112:6825/407933616","cluster_addr":"172.21.15.112:6827/407933616","heartbeat_back_addr":"172.21.15.112:6831/407933616","heartbeat_front_addr":"172.21.15.112:6829/407933616","state":["exists","up"]},{"osd":3,"uuid":"af04330a-36ad-40cc-a8dd-fa259b221de5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6817","nonce":4120164806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6819","nonce":4120164806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6823","nonce":4120164806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6821","nonce":4120164806}]},"public_addr":"172.21.15.112:6817/4120164806","cluster_addr":"172.21.15.112:6819/4120164806","heartbeat_back_addr":"172.21.15.112:6823/4120164806","heartbeat_front_addr":"172.21.15.112:6821/4120164806","state":["exists","up"]},{"osd":4,"uuid":"5b893111-292c-4c5a-b184-53ad38c7199e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6803","nonce":522077560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6805","nonce":522077560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6809","nonce":522077560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6807","nonce":522077560}]},"public_addr":"172.21.15.187:6803/522077560","cluster_addr":"172.21.15.187:6805/522077560","heartbeat_back_addr":"172.21.15.187:6809/522077560","heartbeat_front_addr":"172.21.15.187:6807/522077560","state":["exists","up"]},{"osd":5,"uuid":"c0b8e011-45fe-4c5e-925e-6be2a7f1868a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6827","nonce":3593144721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6829","nonce":3593144721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6833","nonce":3593144721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6831","nonce":3593144721}]},"public_addr":"172.21.15.187:6827/3593144721","cluster_addr":"172.21.15.187:6829/3593144721","heartbeat_back_addr":"172.21.15.187:6833/3593144721","heartbeat_front_addr":"172.21.15.187:6831/3593144721","state":["exists","up"]},{"osd":6,"uuid":"87d2d650-7489-4593-98a3-a1f433ce8dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6811","nonce":1494387657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6813","nonce":1494387657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6817","nonce":1494387657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6815","nonce":1494387657}]},"public_addr":"172.21.15.187:6811/1494387657","cluster_addr":"172.21.15.187:6813/1494387657","heartbeat_back_addr":"172.21.15.187:6817/1494387657","heartbeat_front_addr":"172.21.15.187:6815/1494387657","state":["exists","up"]},{"osd":7,"uuid":"c6b4f8e2-a1ee-4438-aa37-67ff4b6ef1f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6819","nonce":1624699842}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6821","nonce":1624699842}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6825","nonce":1624699842}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6823","nonce":1624699842}]},"public_addr":"172.21.15.187:6819/1624699842","cluster_addr":"172.21.15.187:6821/1624699842","heartbeat_back_addr":"172.21.15.187:6825/1624699842","heartbeat_front_addr":"172.21.15.187:6823/1624699842","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.873162+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.400506+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.664317+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.159791+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.202630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.113212+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.384786+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.448441+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T00:28:17.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.285+0000 7f77d37e4640 1 -- 172.21.15.112:0/3249698407 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f77b0078bc0 msgr2=0x7f77b007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:17.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.285+0000 7f77d37e4640 1 --2- 172.21.15.112:0/3249698407 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f77b0078bc0 0x7f77b007b0d0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f77bc033000 tx=0x7f77bc004730 comp rx=0 tx=0).stop 2024-09-21T00:28:17.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.285+0000 7f77d37e4640 1 -- 172.21.15.112:0/3249698407 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77cc0d0ae0 msgr2=0x7f77cc06e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:17.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.285+0000 7f77d37e4640 1 --2- 172.21.15.112:0/3249698407 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77cc0d0ae0 0x7f77cc06e4b0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f77c800c3f0 tx=0x7f77c800c8b0 comp rx=0 tx=0).stop 2024-09-21T00:28:17.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.285+0000 7f77d37e4640 1 -- 172.21.15.112:0/3249698407 shutdown_connections 2024-09-21T00:28:17.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.285+0000 7f77d37e4640 1 --2- 172.21.15.112:0/3249698407 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f77cc15ea40 0x7f77cc159fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:17.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.285+0000 7f77d37e4640 1 --2- 172.21.15.112:0/3249698407 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f77cc0d4a90 0x7f77cc06dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:17.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.285+0000 7f77d37e4640 1 --2- 172.21.15.112:0/3249698407 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f77b0078bc0 0x7f77b007b0d0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:17.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.285+0000 7f77d37e4640 1 --2- 172.21.15.112:0/3249698407 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77cc0d0ae0 0x7f77cc06e4b0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:17.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.285+0000 7f77d37e4640 1 -- 172.21.15.112:0/3249698407 >> 172.21.15.112:0/3249698407 conn(0x7f77cc09d8a0 msgr2=0x7f77cc15f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:17.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.285+0000 7f77d37e4640 1 -- 172.21.15.112:0/3249698407 shutdown_connections 2024-09-21T00:28:17.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.285+0000 7f77d37e4640 1 -- 172.21.15.112:0/3249698407 wait complete. 2024-09-21T00:28:17.296 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-21T00:28:17.297 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T00:28:17.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.408+0000 7fcff05c6640 1 -- 172.21.15.112:0/2519370306 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcfe80d48a0 msgr2=0x7fcfe80d3dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:17.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.408+0000 7fcff05c6640 1 --2- 172.21.15.112:0/2519370306 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcfe80d48a0 0x7fcfe80d3dc0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fcfe80cea80 tx=0x7fcfd802aac0 comp rx=0 tx=0).stop 2024-09-21T00:28:17.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.408+0000 7fcff05c6640 1 -- 172.21.15.112:0/2519370306 shutdown_connections 2024-09-21T00:28:17.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.408+0000 7fcff05c6640 1 --2- 172.21.15.112:0/2519370306 >> v2:172.21.15.187:3300/0 conn(0x7fcfe80d3880 0x7fcfe8163260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:17.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.408+0000 7fcff05c6640 1 --2- 172.21.15.112:0/2519370306 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcfe80d48a0 0x7fcfe80d3dc0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:17.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.408+0000 7fcff05c6640 1 -- 172.21.15.112:0/2519370306 >> 172.21.15.112:0/2519370306 conn(0x7fcfe808bfe0 msgr2=0x7fcfe809d890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:17.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.408+0000 7fcff05c6640 1 -- 172.21.15.112:0/2519370306 shutdown_connections 2024-09-21T00:28:17.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.408+0000 7fcff05c6640 1 -- 172.21.15.112:0/2519370306 wait complete. 2024-09-21T00:28:17.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.410+0000 7fcff05c6640 1 Processor -- start 2024-09-21T00:28:17.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.410+0000 7fcff05c6640 1 -- start start 2024-09-21T00:28:17.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.411+0000 7fcff05c6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcfe80d01b0 0x7fcfe8160b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:17.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.411+0000 7fcff05c6640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcfe80d3880 0x7fcfe80ca800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:17.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.411+0000 7fcff05c6640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcfe80d48a0 0x7fcfe80c8530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:17.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.411+0000 7fcff05c6640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcfe80d2460 con 0x7fcfe80d01b0 2024-09-21T00:28:17.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.411+0000 7fcff05c6640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fcfe80d22e0 con 0x7fcfe80d48a0 2024-09-21T00:28:17.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.411+0000 7fcff05c6640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fcfe80d25e0 con 0x7fcfe80d3880 2024-09-21T00:28:17.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.411+0000 7fcfee362640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcfe80d01b0 0x7fcfe8160b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:17.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.411+0000 7fcfee362640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcfe80d01b0 0x7fcfe8160b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42320/0 (socket says 172.21.15.112:42320) 2024-09-21T00:28:17.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.411+0000 7fcfee362640 1 -- 172.21.15.112:0/606704536 learned_addr learned my addr 172.21.15.112:0/606704536 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:28:17.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.411+0000 7fcfedb61640 1 --2- 172.21.15.112:0/606704536 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcfe80d3880 0x7fcfe80ca800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:17.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.411+0000 7fcfeeb63640 1 --2- 172.21.15.112:0/606704536 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcfe80d48a0 0x7fcfe80c8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:17.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.412+0000 7fcfee362640 1 -- 172.21.15.112:0/606704536 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcfe80d3880 msgr2=0x7fcfe80ca800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:17.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.412+0000 7fcfee362640 1 --2- 172.21.15.112:0/606704536 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcfe80d3880 0x7fcfe80ca800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:17.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.412+0000 7fcfee362640 1 -- 172.21.15.112:0/606704536 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcfe80d48a0 msgr2=0x7fcfe80c8530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:17.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.412+0000 7fcfee362640 1 --2- 172.21.15.112:0/606704536 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcfe80d48a0 0x7fcfe80c8530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:17.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.412+0000 7fcfee362640 1 -- 172.21.15.112:0/606704536 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfe80cc790 con 0x7fcfe80d01b0 2024-09-21T00:28:17.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.412+0000 7fcfee362640 1 --2- 172.21.15.112:0/606704536 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcfe80d01b0 0x7fcfe8160b70 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fcfd8009bf0 tx=0x7fcfd8003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:17.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.412+0000 7fcfe6ffd640 1 -- 172.21.15.112:0/606704536 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfd8044070 con 0x7fcfe80d01b0 2024-09-21T00:28:17.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.412+0000 7fcfe6ffd640 1 -- 172.21.15.112:0/606704536 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcfd8004250 con 0x7fcfe80d01b0 2024-09-21T00:28:17.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.412+0000 7fcfe6ffd640 1 -- 172.21.15.112:0/606704536 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfd8004680 con 0x7fcfe80d01b0 2024-09-21T00:28:17.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.412+0000 7fcff05c6640 1 -- 172.21.15.112:0/606704536 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfe81f5e10 con 0x7fcfe80d01b0 2024-09-21T00:28:17.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.412+0000 7fcff05c6640 1 -- 172.21.15.112:0/606704536 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfe81f63d0 con 0x7fcfe80d01b0 2024-09-21T00:28:17.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.413+0000 7fcff05c6640 1 -- 172.21.15.112:0/606704536 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfe80d2ab0 con 0x7fcfe80d01b0 2024-09-21T00:28:17.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.414+0000 7fcfe6ffd640 1 -- 172.21.15.112:0/606704536 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fcfd8036400 con 0x7fcfe80d01b0 2024-09-21T00:28:17.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.414+0000 7fcfe6ffd640 1 --2- 172.21.15.112:0/606704536 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fcfd4078b90 0x7fcfd407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:17.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.415+0000 7fcfe6ffd640 1 -- 172.21.15.112:0/606704536 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7fcfd80bb570 con 0x7fcfe80d01b0 2024-09-21T00:28:17.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.415+0000 7fcfedb61640 1 --2- 172.21.15.112:0/606704536 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fcfd4078b90 0x7fcfd407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:17.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.415+0000 7fcfedb61640 1 --2- 172.21.15.112:0/606704536 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fcfd4078b90 0x7fcfd407b0a0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fcfe815e620 tx=0x7fcfdc005f90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:17.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.419+0000 7fcfe6ffd640 1 -- 172.21.15.112:0/606704536 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fcfd8032030 con 0x7fcfe80d01b0 2024-09-21T00:28:17.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.632+0000 7fcff05c6640 1 -- 172.21.15.112:0/606704536 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcfe80b6f90 con 0x7fcfe80d01b0 2024-09-21T00:28:17.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.633+0000 7fcfe6ffd640 1 -- 172.21.15.112:0/606704536 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+5253 (secure 0 0 0) 0x7fcfd8004820 con 0x7fcfe80d01b0 2024-09-21T00:28:17.633 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:28:17.634 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":39,"btime":"2024-09-21T00:28:01:807440+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5389,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6836/3984762342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":3984762342},{"type":"v1","addr":"172.21.15.187:6836","nonce":3984762342}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5392,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6837/763698806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6835","nonce":763698806},{"type":"v1","addr":"172.21.15.187:6837","nonce":763698806}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T00:27:34.871721+0000","modified":"2024-09-21T00:28:00.832760+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5739,"mds_1":5736},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5736":{"gid":5736,"name":"c","rank":1,"incarnation":37,"state":"up:active","state_seq":5,"addr":"172.21.15.112:6834/1432588455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1432588455},{"type":"v1","addr":"172.21.15.112:6834","nonce":1432588455}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5739":{"gid":5739,"name":"a","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6835/3610025869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":3610025869},{"type":"v1","addr":"172.21.15.112:6835","nonce":3610025869}]},"join_fscid":-1,"export_targets":[1],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5739,"qdb_cluster":[5739,5736]},"id":4}]} 2024-09-21T00:28:17.634 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 39 2024-09-21T00:28:17.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.635+0000 7fcff05c6640 1 -- 172.21.15.112:0/606704536 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fcfd4078b90 msgr2=0x7fcfd407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:17.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.635+0000 7fcff05c6640 1 --2- 172.21.15.112:0/606704536 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fcfd4078b90 0x7fcfd407b0a0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fcfe815e620 tx=0x7fcfdc005f90 comp rx=0 tx=0).stop 2024-09-21T00:28:17.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.635+0000 7fcff05c6640 1 -- 172.21.15.112:0/606704536 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcfe80d01b0 msgr2=0x7fcfe8160b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:17.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.635+0000 7fcff05c6640 1 --2- 172.21.15.112:0/606704536 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcfe80d01b0 0x7fcfe8160b70 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fcfd8009bf0 tx=0x7fcfd8003ce0 comp rx=0 tx=0).stop 2024-09-21T00:28:17.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.635+0000 7fcff05c6640 1 -- 172.21.15.112:0/606704536 shutdown_connections 2024-09-21T00:28:17.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.635+0000 7fcff05c6640 1 --2- 172.21.15.112:0/606704536 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcfe80d48a0 0x7fcfe80c8530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:17.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.635+0000 7fcff05c6640 1 --2- 172.21.15.112:0/606704536 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcfe80d3880 0x7fcfe80ca800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:17.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.635+0000 7fcff05c6640 1 --2- 172.21.15.112:0/606704536 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fcfd4078b90 0x7fcfd407b0a0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:17.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.635+0000 7fcff05c6640 1 --2- 172.21.15.112:0/606704536 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcfe80d01b0 0x7fcfe8160b70 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:17.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.635+0000 7fcff05c6640 1 -- 172.21.15.112:0/606704536 >> 172.21.15.112:0/606704536 conn(0x7fcfe808bfe0 msgr2=0x7fcfe80b6220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:17.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.635+0000 7fcff05c6640 1 -- 172.21.15.112:0/606704536 shutdown_connections 2024-09-21T00:28:17.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.635+0000 7fcff05c6640 1 -- 172.21.15.112:0/606704536 wait complete. 2024-09-21T00:28:17.647 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T00:28:17.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.759+0000 7f67a8c7f640 1 -- 172.21.15.112:0/1237938409 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6790004800 msgr2=0x7f6790004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:17.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.759+0000 7f67a8c7f640 1 --2- 172.21.15.112:0/1237938409 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6790004800 0x7f6790004c10 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f678c001630 tx=0x7f6790005830 comp rx=0 tx=0).stop 2024-09-21T00:28:17.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.759+0000 7f67a2d9d640 1 -- 172.21.15.112:0/1237938409 reap_dead start 2024-09-21T00:28:17.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.759+0000 7f67a8c7f640 1 -- 172.21.15.112:0/1237938409 shutdown_connections 2024-09-21T00:28:17.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.759+0000 7f67a8c7f640 1 -- 172.21.15.112:0/1237938409 >> 172.21.15.112:0/1237938409 conn(0x7f67a40cf220 msgr2=0x7f67a40cf630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:17.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.759+0000 7f67a8c7f640 1 -- 172.21.15.112:0/1237938409 shutdown_connections 2024-09-21T00:28:17.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.759+0000 7f67a8c7f640 1 -- 172.21.15.112:0/1237938409 wait complete. 2024-09-21T00:28:17.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.761+0000 7f67a8c7f640 1 Processor -- start 2024-09-21T00:28:17.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.761+0000 7f67a8c7f640 1 -- start start 2024-09-21T00:28:17.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.762+0000 7f67a8c7f640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6790004800 0x7f67a40b7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:17.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.762+0000 7f67a8c7f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f67a40cb210 0x7f67a40c9800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:17.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.762+0000 7f67a8c7f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f67a41f68e0 0x7f67a41f6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:17.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.762+0000 7f67a8c7f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f67a40d2130 con 0x7f67a41f68e0 2024-09-21T00:28:17.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.762+0000 7f67a8c7f640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f67a40d1fb0 con 0x7f67a40cb210 2024-09-21T00:28:17.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.762+0000 7f67a8c7f640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f67a40c65e0 con 0x7f6790004800 2024-09-21T00:28:17.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.762+0000 7f67a2d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f67a41f68e0 0x7f67a41f6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:17.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.762+0000 7f67a2d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f67a41f68e0 0x7f67a41f6cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42344/0 (socket says 172.21.15.112:42344) 2024-09-21T00:28:17.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.762+0000 7f67a2d9d640 1 -- 172.21.15.112:0/3890332652 learned_addr learned my addr 172.21.15.112:0/3890332652 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:28:17.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.762+0000 7f67a1d9b640 1 --2- 172.21.15.112:0/3890332652 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f67a40cb210 0x7f67a40c9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:17.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.762+0000 7f67a259c640 1 --2- 172.21.15.112:0/3890332652 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6790004800 0x7f67a40b7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:17.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.762+0000 7f67a2d9d640 1 -- 172.21.15.112:0/3890332652 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6790004800 msgr2=0x7f67a40b7ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:17.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.762+0000 7f67a2d9d640 1 --2- 172.21.15.112:0/3890332652 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6790004800 0x7f67a40b7ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:17.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.762+0000 7f67a2d9d640 1 -- 172.21.15.112:0/3890332652 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f67a40cb210 msgr2=0x7f67a40c9800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:17.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.762+0000 7f67a2d9d640 1 --2- 172.21.15.112:0/3890332652 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f67a40cb210 0x7f67a40c9800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:17.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.762+0000 7f67a2d9d640 1 -- 172.21.15.112:0/3890332652 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67a40cc2e0 con 0x7f67a41f68e0 2024-09-21T00:28:17.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.763+0000 7f67a1d9b640 1 --2- 172.21.15.112:0/3890332652 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f67a40cb210 0x7f67a40c9800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:28:17.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.763+0000 7f67a2d9d640 1 --2- 172.21.15.112:0/3890332652 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f67a41f68e0 0x7f67a41f6cf0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f67a40b3430 tx=0x7f6794011bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:17.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.763+0000 7f679affd640 1 -- 172.21.15.112:0/3890332652 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f679400f8f0 con 0x7f67a41f68e0 2024-09-21T00:28:17.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.763+0000 7f679affd640 1 -- 172.21.15.112:0/3890332652 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f679400fa90 con 0x7f67a41f68e0 2024-09-21T00:28:17.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.763+0000 7f67a8c7f640 1 -- 172.21.15.112:0/3890332652 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67a41f7230 con 0x7f67a41f68e0 2024-09-21T00:28:17.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.763+0000 7f679affd640 1 -- 172.21.15.112:0/3890332652 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6794005020 con 0x7f67a41f68e0 2024-09-21T00:28:17.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.763+0000 7f67a8c7f640 1 -- 172.21.15.112:0/3890332652 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67a40cbd20 con 0x7f67a41f68e0 2024-09-21T00:28:17.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.764+0000 7f67a8c7f640 1 -- 172.21.15.112:0/3890332652 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67a40c6b70 con 0x7f67a41f68e0 2024-09-21T00:28:17.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.765+0000 7f679affd640 1 -- 172.21.15.112:0/3890332652 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f67940051c0 con 0x7f67a41f68e0 2024-09-21T00:28:17.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.765+0000 7f679affd640 1 --2- 172.21.15.112:0/3890332652 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f6784078b90 0x7f678407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:17.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.765+0000 7f679affd640 1 -- 172.21.15.112:0/3890332652 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7f679409b860 con 0x7f67a41f68e0 2024-09-21T00:28:17.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.765+0000 7f67a259c640 1 --2- 172.21.15.112:0/3890332652 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f6784078b90 0x7f678407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:17.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.766+0000 7f67a259c640 1 --2- 172.21.15.112:0/3890332652 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f6784078b90 0x7f678407b0a0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f679c006fd0 tx=0x7f679c006aa0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:17.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.770+0000 7f679affd640 1 -- 172.21.15.112:0/3890332652 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f6794020020 con 0x7f67a41f68e0 2024-09-21T00:28:17.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.984+0000 7f67a8c7f640 1 -- 172.21.15.112:0/3890332652 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f67a40b2490 con 0x7f67a41f68e0 2024-09-21T00:28:17.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.984+0000 7f679affd640 1 -- 172.21.15.112:0/3890332652 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+5253 (secure 0 0 0) 0x7f6794011e20 con 0x7f67a41f68e0 2024-09-21T00:28:17.985 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:28:17.985 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":39,"btime":"2024-09-21T00:28:01:807440+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5389,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6836/3984762342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":3984762342},{"type":"v1","addr":"172.21.15.187:6836","nonce":3984762342}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5392,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6837/763698806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6835","nonce":763698806},{"type":"v1","addr":"172.21.15.187:6837","nonce":763698806}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T00:27:34.871721+0000","modified":"2024-09-21T00:28:00.832760+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5739,"mds_1":5736},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5736":{"gid":5736,"name":"c","rank":1,"incarnation":37,"state":"up:active","state_seq":5,"addr":"172.21.15.112:6834/1432588455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1432588455},{"type":"v1","addr":"172.21.15.112:6834","nonce":1432588455}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5739":{"gid":5739,"name":"a","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6835/3610025869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":3610025869},{"type":"v1","addr":"172.21.15.112:6835","nonce":3610025869}]},"join_fscid":-1,"export_targets":[1],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5739,"qdb_cluster":[5739,5736]},"id":4}]} 2024-09-21T00:28:17.985 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 39 2024-09-21T00:28:17.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.986+0000 7f67a8c7f640 1 -- 172.21.15.112:0/3890332652 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f6784078b90 msgr2=0x7f678407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:17.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.986+0000 7f67a8c7f640 1 --2- 172.21.15.112:0/3890332652 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f6784078b90 0x7f678407b0a0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f679c006fd0 tx=0x7f679c006aa0 comp rx=0 tx=0).stop 2024-09-21T00:28:17.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.986+0000 7f67a8c7f640 1 -- 172.21.15.112:0/3890332652 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f67a41f68e0 msgr2=0x7f67a41f6cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:17.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.986+0000 7f67a8c7f640 1 --2- 172.21.15.112:0/3890332652 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f67a41f68e0 0x7f67a41f6cf0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f67a40b3430 tx=0x7f6794011bd0 comp rx=0 tx=0).stop 2024-09-21T00:28:17.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.986+0000 7f67a8c7f640 1 -- 172.21.15.112:0/3890332652 shutdown_connections 2024-09-21T00:28:17.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.986+0000 7f67a8c7f640 1 --2- 172.21.15.112:0/3890332652 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f6784078b90 0x7f678407b0a0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:17.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.986+0000 7f67a8c7f640 1 --2- 172.21.15.112:0/3890332652 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f67a41f68e0 0x7f67a41f6cf0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:17.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.986+0000 7f67a8c7f640 1 --2- 172.21.15.112:0/3890332652 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f67a40cb210 0x7f67a40c9800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:17.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.986+0000 7f67a8c7f640 1 --2- 172.21.15.112:0/3890332652 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6790004800 0x7f67a40b7ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:17.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.987+0000 7f67a8c7f640 1 -- 172.21.15.112:0/3890332652 >> 172.21.15.112:0/3890332652 conn(0x7f67a40cf220 msgr2=0x7f67a41e8cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:17.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.987+0000 7f67a8c7f640 1 -- 172.21.15.112:0/3890332652 shutdown_connections 2024-09-21T00:28:17.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:17.987+0000 7f67a8c7f640 1 -- 172.21.15.112:0/3890332652 wait complete. 2024-09-21T00:28:17.998 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T00:28:18.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.109+0000 7ff3c9957640 1 -- 172.21.15.112:0/1544612138 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff3c40d0ae0 msgr2=0x7ff3c40d38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:18.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.109+0000 7ff3c9957640 1 --2- 172.21.15.112:0/1544612138 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff3c40d0ae0 0x7ff3c40d38c0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7ff3c40d5530 tx=0x7ff3b402a4b0 comp rx=0 tx=0).stop 2024-09-21T00:28:18.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.109+0000 7ff3c9957640 1 -- 172.21.15.112:0/1544612138 shutdown_connections 2024-09-21T00:28:18.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.109+0000 7ff3c9957640 1 --2- 172.21.15.112:0/1544612138 >> v2:172.21.15.187:3300/0 conn(0x7ff3c415ea40 0x7ff3c4070250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:18.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.109+0000 7ff3c9957640 1 --2- 172.21.15.112:0/1544612138 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff3c40d4380 0x7ff3c40d4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:18.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.109+0000 7ff3c9957640 1 --2- 172.21.15.112:0/1544612138 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff3c40d0ae0 0x7ff3c40d38c0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:18.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.109+0000 7ff3c9957640 1 -- 172.21.15.112:0/1544612138 >> 172.21.15.112:0/1544612138 conn(0x7ff3c409d8a0 msgr2=0x7ff3c408bfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:18.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.109+0000 7ff3c9957640 1 -- 172.21.15.112:0/1544612138 shutdown_connections 2024-09-21T00:28:18.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.109+0000 7ff3c9957640 1 -- 172.21.15.112:0/1544612138 wait complete. 2024-09-21T00:28:18.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.111+0000 7ff3c9957640 1 Processor -- start 2024-09-21T00:28:18.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.111+0000 7ff3c9957640 1 -- start start 2024-09-21T00:28:18.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.112+0000 7ff3c9957640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff3c40d0ae0 0x7ff3c40a1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:18.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.112+0000 7ff3c9957640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff3c40d4380 0x7ff3c40d2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:18.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.112+0000 7ff3c9957640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff3c415ea40 0x7ff3c40d3500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:18.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.112+0000 7ff3c9957640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff3c40b6490 con 0x7ff3c40d4380 2024-09-21T00:28:18.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.112+0000 7ff3c9957640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff3c40b6310 con 0x7ff3c415ea40 2024-09-21T00:28:18.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.112+0000 7ff3c9957640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff3c40b6610 con 0x7ff3c40d0ae0 2024-09-21T00:28:18.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.112+0000 7ff3ba7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff3c40d4380 0x7ff3c40d2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:18.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.113+0000 7ff3ba7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff3c40d4380 0x7ff3c40d2fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42372/0 (socket says 172.21.15.112:42372) 2024-09-21T00:28:18.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.113+0000 7ff3ba7fc640 1 -- 172.21.15.112:0/2040494263 learned_addr learned my addr 172.21.15.112:0/2040494263 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:28:18.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.113+0000 7ff3c2ffd640 1 --2- 172.21.15.112:0/2040494263 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff3c40d0ae0 0x7ff3c40a1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:18.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.113+0000 7ff3c37fe640 1 --2- 172.21.15.112:0/2040494263 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff3c415ea40 0x7ff3c40d3500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:18.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.113+0000 7ff3ba7fc640 1 -- 172.21.15.112:0/2040494263 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff3c40d0ae0 msgr2=0x7ff3c40a1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:18.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.113+0000 7ff3ba7fc640 1 --2- 172.21.15.112:0/2040494263 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff3c40d0ae0 0x7ff3c40a1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:18.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.113+0000 7ff3ba7fc640 1 -- 172.21.15.112:0/2040494263 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff3c415ea40 msgr2=0x7ff3c40d3500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:18.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.113+0000 7ff3ba7fc640 1 --2- 172.21.15.112:0/2040494263 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff3c415ea40 0x7ff3c40d3500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:18.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.113+0000 7ff3ba7fc640 1 -- 172.21.15.112:0/2040494263 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3c41fd230 con 0x7ff3c40d4380 2024-09-21T00:28:18.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.113+0000 7ff3ba7fc640 1 --2- 172.21.15.112:0/2040494263 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff3c40d4380 0x7ff3c40d2fc0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7ff3ac00c970 tx=0x7ff3ac00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:18.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.113+0000 7ff3c0ff9640 1 -- 172.21.15.112:0/2040494263 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3ac004220 con 0x7ff3c40d4380 2024-09-21T00:28:18.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.113+0000 7ff3c0ff9640 1 -- 172.21.15.112:0/2040494263 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff3ac0044b0 con 0x7ff3c40d4380 2024-09-21T00:28:18.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.113+0000 7ff3c9957640 1 -- 172.21.15.112:0/2040494263 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3c41fd4d0 con 0x7ff3c40d4380 2024-09-21T00:28:18.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.113+0000 7ff3c0ff9640 1 -- 172.21.15.112:0/2040494263 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3ac007c10 con 0x7ff3c40d4380 2024-09-21T00:28:18.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.113+0000 7ff3c9957640 1 -- 172.21.15.112:0/2040494263 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3c41fda90 con 0x7ff3c40d4380 2024-09-21T00:28:18.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.114+0000 7ff3c9957640 1 -- 172.21.15.112:0/2040494263 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3c40b6b30 con 0x7ff3c40d4380 2024-09-21T00:28:18.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.115+0000 7ff3c0ff9640 1 -- 172.21.15.112:0/2040494263 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff3ac003610 con 0x7ff3c40d4380 2024-09-21T00:28:18.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.116+0000 7ff3c0ff9640 1 --2- 172.21.15.112:0/2040494263 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff3a4078b90 0x7ff3a407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:18.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.116+0000 7ff3c0ff9640 1 -- 172.21.15.112:0/2040494263 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7ff3ac09b130 con 0x7ff3c40d4380 2024-09-21T00:28:18.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.116+0000 7ff3c2ffd640 1 --2- 172.21.15.112:0/2040494263 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff3a4078b90 0x7ff3a407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:18.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.117+0000 7ff3c2ffd640 1 --2- 172.21.15.112:0/2040494263 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff3a4078b90 0x7ff3a407b0a0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ff3b4034000 tx=0x7ff3b4004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:18.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.120+0000 7ff3c0ff9640 1 -- 172.21.15.112:0/2040494263 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ff3ac014030 con 0x7ff3c40d4380 2024-09-21T00:28:18.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.298+0000 7ff3c9957640 1 -- 172.21.15.112:0/2040494263 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff3c4069230 con 0x7ff3c40d4380 2024-09-21T00:28:18.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.299+0000 7ff3c0ff9640 1 -- 172.21.15.112:0/2040494263 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v60) ==== 74+0+16968 (secure 0 0 0) 0x7ff3ac062e20 con 0x7ff3c40d4380 2024-09-21T00:28:18.300 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:28:18.300 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":60,"fsid":"8b34692b-dc65-49b2-a08f-26f707b96ac5","created":"2024-09-21T00:25:22.788490+0000","modified":"2024-09-21T00:27:36.511536+0000","last_up_change":"2024-09-21T00:25:36.161835+0000","last_in_change":"2024-09-21T00:25:26.346500+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T00:25:37.105006+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-09-21T00:27:32.510522+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-09-21T00:27:33.828833+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9c575086-67d2-4de2-b01b-d44dcc21d604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6809","nonce":755760621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6811","nonce":755760621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6815","nonce":755760621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6813","nonce":755760621}]},"public_addr":"172.21.15.112:6809/755760621","cluster_addr":"172.21.15.112:6811/755760621","heartbeat_back_addr":"172.21.15.112:6815/755760621","heartbeat_front_addr":"172.21.15.112:6813/755760621","state":["exists","up"]},{"osd":1,"uuid":"a983c227-a3ce-4bdd-82f1-438ba7e3ecd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6801","nonce":344932592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6803","nonce":344932592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6807","nonce":344932592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6805","nonce":344932592}]},"public_addr":"172.21.15.112:6801/344932592","cluster_addr":"172.21.15.112:6803/344932592","heartbeat_back_addr":"172.21.15.112:6807/344932592","heartbeat_front_addr":"172.21.15.112:6805/344932592","state":["exists","up"]},{"osd":2,"uuid":"1193c1d8-bfff-4a57-8200-5f69bf1ea3af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6825","nonce":407933616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6827","nonce":407933616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6831","nonce":407933616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6829","nonce":407933616}]},"public_addr":"172.21.15.112:6825/407933616","cluster_addr":"172.21.15.112:6827/407933616","heartbeat_back_addr":"172.21.15.112:6831/407933616","heartbeat_front_addr":"172.21.15.112:6829/407933616","state":["exists","up"]},{"osd":3,"uuid":"af04330a-36ad-40cc-a8dd-fa259b221de5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6817","nonce":4120164806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6819","nonce":4120164806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6823","nonce":4120164806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6821","nonce":4120164806}]},"public_addr":"172.21.15.112:6817/4120164806","cluster_addr":"172.21.15.112:6819/4120164806","heartbeat_back_addr":"172.21.15.112:6823/4120164806","heartbeat_front_addr":"172.21.15.112:6821/4120164806","state":["exists","up"]},{"osd":4,"uuid":"5b893111-292c-4c5a-b184-53ad38c7199e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6803","nonce":522077560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6805","nonce":522077560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6809","nonce":522077560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6807","nonce":522077560}]},"public_addr":"172.21.15.187:6803/522077560","cluster_addr":"172.21.15.187:6805/522077560","heartbeat_back_addr":"172.21.15.187:6809/522077560","heartbeat_front_addr":"172.21.15.187:6807/522077560","state":["exists","up"]},{"osd":5,"uuid":"c0b8e011-45fe-4c5e-925e-6be2a7f1868a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6827","nonce":3593144721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6829","nonce":3593144721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6833","nonce":3593144721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6831","nonce":3593144721}]},"public_addr":"172.21.15.187:6827/3593144721","cluster_addr":"172.21.15.187:6829/3593144721","heartbeat_back_addr":"172.21.15.187:6833/3593144721","heartbeat_front_addr":"172.21.15.187:6831/3593144721","state":["exists","up"]},{"osd":6,"uuid":"87d2d650-7489-4593-98a3-a1f433ce8dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6811","nonce":1494387657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6813","nonce":1494387657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6817","nonce":1494387657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6815","nonce":1494387657}]},"public_addr":"172.21.15.187:6811/1494387657","cluster_addr":"172.21.15.187:6813/1494387657","heartbeat_back_addr":"172.21.15.187:6817/1494387657","heartbeat_front_addr":"172.21.15.187:6815/1494387657","state":["exists","up"]},{"osd":7,"uuid":"c6b4f8e2-a1ee-4438-aa37-67ff4b6ef1f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6819","nonce":1624699842}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6821","nonce":1624699842}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6825","nonce":1624699842}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6823","nonce":1624699842}]},"public_addr":"172.21.15.187:6819/1624699842","cluster_addr":"172.21.15.187:6821/1624699842","heartbeat_back_addr":"172.21.15.187:6825/1624699842","heartbeat_front_addr":"172.21.15.187:6823/1624699842","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.873162+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.400506+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.664317+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.159791+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.202630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.113212+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.384786+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.448441+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T00:28:18.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.301+0000 7ff3c9957640 1 -- 172.21.15.112:0/2040494263 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff3a4078b90 msgr2=0x7ff3a407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:18.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.301+0000 7ff3c9957640 1 --2- 172.21.15.112:0/2040494263 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff3a4078b90 0x7ff3a407b0a0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ff3b4034000 tx=0x7ff3b4004730 comp rx=0 tx=0).stop 2024-09-21T00:28:18.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.301+0000 7ff3c9957640 1 -- 172.21.15.112:0/2040494263 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff3c40d4380 msgr2=0x7ff3c40d2fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:18.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.301+0000 7ff3c9957640 1 --2- 172.21.15.112:0/2040494263 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff3c40d4380 0x7ff3c40d2fc0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7ff3ac00c970 tx=0x7ff3ac00ce30 comp rx=0 tx=0).stop 2024-09-21T00:28:18.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.301+0000 7ff3c9957640 1 -- 172.21.15.112:0/2040494263 shutdown_connections 2024-09-21T00:28:18.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.301+0000 7ff3c9957640 1 --2- 172.21.15.112:0/2040494263 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff3c415ea40 0x7ff3c40d3500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:18.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.301+0000 7ff3c9957640 1 --2- 172.21.15.112:0/2040494263 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff3a4078b90 0x7ff3a407b0a0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:18.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.301+0000 7ff3c9957640 1 --2- 172.21.15.112:0/2040494263 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff3c40d4380 0x7ff3c40d2fc0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:18.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.301+0000 7ff3c9957640 1 --2- 172.21.15.112:0/2040494263 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff3c40d0ae0 0x7ff3c40a1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:18.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.301+0000 7ff3c9957640 1 -- 172.21.15.112:0/2040494263 >> 172.21.15.112:0/2040494263 conn(0x7ff3c409d8a0 msgr2=0x7ff3c4160f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:18.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.301+0000 7ff3c9957640 1 -- 172.21.15.112:0/2040494263 shutdown_connections 2024-09-21T00:28:18.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.301+0000 7ff3c9957640 1 -- 172.21.15.112:0/2040494263 wait complete. 2024-09-21T00:28:18.313 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T00:28:18.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.426+0000 7fef2c58b640 1 -- 172.21.15.112:0/1906063908 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fef240ccbf0 msgr2=0x7fef240cd000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:18.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.426+0000 7fef2c58b640 1 --2- 172.21.15.112:0/1906063908 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fef240ccbf0 0x7fef240cd000 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fef20009830 tx=0x7fef2002a4a0 comp rx=0 tx=0).stop 2024-09-21T00:28:18.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.426+0000 7fef2c58b640 1 -- 172.21.15.112:0/1906063908 shutdown_connections 2024-09-21T00:28:18.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.426+0000 7fef2c58b640 1 --2- 172.21.15.112:0/1906063908 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fef240ccbf0 0x7fef240cd000 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:18.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.426+0000 7fef2c58b640 1 -- 172.21.15.112:0/1906063908 >> 172.21.15.112:0/1906063908 conn(0x7fef240b42e0 msgr2=0x7fef240b4720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:18.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.426+0000 7fef2c58b640 1 -- 172.21.15.112:0/1906063908 shutdown_connections 2024-09-21T00:28:18.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.426+0000 7fef2c58b640 1 -- 172.21.15.112:0/1906063908 wait complete. 2024-09-21T00:28:18.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.428+0000 7fef2c58b640 1 Processor -- start 2024-09-21T00:28:18.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.428+0000 7fef2c58b640 1 -- start start 2024-09-21T00:28:18.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.429+0000 7fef2c58b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef240c5ee0 0x7fef2414ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:18.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.429+0000 7fef2c58b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fef240c6a00 0x7fef240c9e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:18.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.429+0000 7fef2c58b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fef240ccbf0 0x7fef240ca390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:18.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.429+0000 7fef2c58b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fef240c6f60 con 0x7fef240c5ee0 2024-09-21T00:28:18.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.429+0000 7fef2c58b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fef240b3660 con 0x7fef240c6a00 2024-09-21T00:28:18.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.429+0000 7fef2c58b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fef24167ed0 con 0x7fef240ccbf0 2024-09-21T00:28:18.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.429+0000 7fef2a327640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef240c5ee0 0x7fef2414ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:18.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.429+0000 7fef2a327640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef240c5ee0 0x7fef2414ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42378/0 (socket says 172.21.15.112:42378) 2024-09-21T00:28:18.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.429+0000 7fef2a327640 1 -- 172.21.15.112:0/3806653029 learned_addr learned my addr 172.21.15.112:0/3806653029 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:28:18.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.430+0000 7fef2ab28640 1 --2- 172.21.15.112:0/3806653029 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fef240ccbf0 0x7fef240ca390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:18.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.430+0000 7fef29b26640 1 --2- 172.21.15.112:0/3806653029 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fef240c6a00 0x7fef240c9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:18.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.430+0000 7fef2a327640 1 -- 172.21.15.112:0/3806653029 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fef240ccbf0 msgr2=0x7fef240ca390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:18.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.430+0000 7fef2a327640 1 --2- 172.21.15.112:0/3806653029 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fef240ccbf0 0x7fef240ca390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:18.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.430+0000 7fef2a327640 1 -- 172.21.15.112:0/3806653029 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fef240c6a00 msgr2=0x7fef240c9e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:18.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.430+0000 7fef2a327640 1 --2- 172.21.15.112:0/3806653029 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fef240c6a00 0x7fef240c9e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:18.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.430+0000 7fef2a327640 1 -- 172.21.15.112:0/3806653029 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef24158540 con 0x7fef240c5ee0 2024-09-21T00:28:18.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.430+0000 7fef2ab28640 1 --2- 172.21.15.112:0/3806653029 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fef240ccbf0 0x7fef240ca390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:28:18.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.430+0000 7fef2a327640 1 --2- 172.21.15.112:0/3806653029 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef240c5ee0 0x7fef2414ce20 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fef2002aa90 tx=0x7fef20033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:18.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.430+0000 7fef1affd640 1 -- 172.21.15.112:0/3806653029 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef20005c90 con 0x7fef240c5ee0 2024-09-21T00:28:18.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.430+0000 7fef2c58b640 1 -- 172.21.15.112:0/3806653029 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef24158260 con 0x7fef240c5ee0 2024-09-21T00:28:18.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.430+0000 7fef1affd640 1 -- 172.21.15.112:0/3806653029 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fef20031e50 con 0x7fef240c5ee0 2024-09-21T00:28:18.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.430+0000 7fef1affd640 1 -- 172.21.15.112:0/3806653029 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef20005260 con 0x7fef240c5ee0 2024-09-21T00:28:18.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.430+0000 7fef2c58b640 1 -- 172.21.15.112:0/3806653029 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef24153ec0 con 0x7fef240c5ee0 2024-09-21T00:28:18.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.431+0000 7fef2c58b640 1 -- 172.21.15.112:0/3806653029 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef240cd000 con 0x7fef240c5ee0 2024-09-21T00:28:18.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.432+0000 7fef1affd640 1 -- 172.21.15.112:0/3806653029 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fef20005400 con 0x7fef240c5ee0 2024-09-21T00:28:18.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.432+0000 7fef1affd640 1 --2- 172.21.15.112:0/3806653029 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fef08078b90 0x7fef0807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:18.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.432+0000 7fef1affd640 1 -- 172.21.15.112:0/3806653029 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7fef200bbfc0 con 0x7fef240c5ee0 2024-09-21T00:28:18.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.432+0000 7fef29b26640 1 --2- 172.21.15.112:0/3806653029 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fef08078b90 0x7fef0807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:18.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.433+0000 7fef29b26640 1 --2- 172.21.15.112:0/3806653029 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fef08078b90 0x7fef0807b0a0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fef14001120 tx=0x7fef14009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:18.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.437+0000 7fef1affd640 1 -- 172.21.15.112:0/3806653029 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fef2003f030 con 0x7fef240c5ee0 2024-09-21T00:28:18.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.649+0000 7fef2c58b640 1 -- 172.21.15.112:0/3806653029 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fef240b18e0 con 0x7fef240c5ee0 2024-09-21T00:28:18.650 INFO:tasks.ceph.mon.a.smithi112.stderr:2024-09-21T00:28:18.650+0000 7f7656c13640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T00:28:18.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.651+0000 7fef1affd640 1 -- 172.21.15.112:0/3806653029 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v40) ==== 161+0+0 (secure 0 0 0) 0x7fef200841f0 con 0x7fef240c5ee0 2024-09-21T00:28:18.652 INFO:teuthology.orchestra.run.smithi112.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T00:28:18.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.655+0000 7fef2c58b640 1 -- 172.21.15.112:0/3806653029 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fef08078b90 msgr2=0x7fef0807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:18.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.655+0000 7fef2c58b640 1 --2- 172.21.15.112:0/3806653029 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fef08078b90 0x7fef0807b0a0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fef14001120 tx=0x7fef14009040 comp rx=0 tx=0).stop 2024-09-21T00:28:18.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.655+0000 7fef2c58b640 1 -- 172.21.15.112:0/3806653029 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef240c5ee0 msgr2=0x7fef2414ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:18.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.655+0000 7fef2c58b640 1 --2- 172.21.15.112:0/3806653029 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef240c5ee0 0x7fef2414ce20 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fef2002aa90 tx=0x7fef20033c20 comp rx=0 tx=0).stop 2024-09-21T00:28:18.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.655+0000 7fef2c58b640 1 -- 172.21.15.112:0/3806653029 shutdown_connections 2024-09-21T00:28:18.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.655+0000 7fef2c58b640 1 --2- 172.21.15.112:0/3806653029 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fef240ccbf0 0x7fef240ca390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:18.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.655+0000 7fef2c58b640 1 --2- 172.21.15.112:0/3806653029 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fef240c6a00 0x7fef240c9e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:18.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.655+0000 7fef2c58b640 1 --2- 172.21.15.112:0/3806653029 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fef08078b90 0x7fef0807b0a0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:18.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.655+0000 7fef2c58b640 1 --2- 172.21.15.112:0/3806653029 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef240c5ee0 0x7fef2414ce20 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:18.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.655+0000 7fef2c58b640 1 -- 172.21.15.112:0/3806653029 >> 172.21.15.112:0/3806653029 conn(0x7fef240b42e0 msgr2=0x7fef240cc0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:18.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.655+0000 7fef2c58b640 1 -- 172.21.15.112:0/3806653029 shutdown_connections 2024-09-21T00:28:18.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.655+0000 7fef2c58b640 1 -- 172.21.15.112:0/3806653029 wait complete. 2024-09-21T00:28:18.668 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-21T00:28:18.716 INFO:tasks.ceph.mds.c.smithi112.stdout:starting mds.c at 2024-09-21T00:28:18.716 INFO:tasks.ceph.mds.a.smithi112.stdout:starting mds.a at 2024-09-21T00:28:18.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.786+0000 7f2dac430640 1 -- 172.21.15.112:0/3121923376 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2da4070210 msgr2=0x7f2da4070620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:18.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.786+0000 7f2dac430640 1 --2- 172.21.15.112:0/3121923376 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2da4070210 0x7f2da4070620 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f2da406fd80 tx=0x7f2d9402a4d0 comp rx=0 tx=0).stop 2024-09-21T00:28:18.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.786+0000 7f2dac430640 1 -- 172.21.15.112:0/3121923376 shutdown_connections 2024-09-21T00:28:18.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.786+0000 7f2dac430640 1 --2- 172.21.15.112:0/3121923376 >> v2:172.21.15.187:3300/0 conn(0x7f2da40ccbb0 0x7f2da40ccfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:18.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.786+0000 7f2dac430640 1 --2- 172.21.15.112:0/3121923376 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2da4070210 0x7f2da4070620 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:18.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.786+0000 7f2dac430640 1 -- 172.21.15.112:0/3121923376 >> 172.21.15.112:0/3121923376 conn(0x7f2da4073fe0 msgr2=0x7f2da40743f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:18.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.786+0000 7f2dac430640 1 -- 172.21.15.112:0/3121923376 shutdown_connections 2024-09-21T00:28:18.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.786+0000 7f2dac430640 1 -- 172.21.15.112:0/3121923376 wait complete. 2024-09-21T00:28:18.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.788+0000 7f2dac430640 1 Processor -- start 2024-09-21T00:28:18.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.788+0000 7f2dac430640 1 -- start start 2024-09-21T00:28:18.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.789+0000 7f2dac430640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2da4070210 0x7f2da41f9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:18.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.789+0000 7f2dac430640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2da40ccbb0 0x7f2da41322e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:18.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.789+0000 7f2dac430640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2da415c5f0 0x7f2da4132820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:18.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.789+0000 7f2dac430640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2da415ca00 con 0x7f2da4070210 2024-09-21T00:28:18.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.789+0000 7f2dac430640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f2da4070b60 con 0x7f2da415c5f0 2024-09-21T00:28:18.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.789+0000 7f2dac430640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f2da4156890 con 0x7f2da40ccbb0 2024-09-21T00:28:18.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.789+0000 7f2daa1cc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2da4070210 0x7f2da41f9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:18.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.789+0000 7f2daa1cc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2da4070210 0x7f2da41f9ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42442/0 (socket says 172.21.15.112:42442) 2024-09-21T00:28:18.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.789+0000 7f2daa1cc640 1 -- 172.21.15.112:0/266008586 learned_addr learned my addr 172.21.15.112:0/266008586 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:28:18.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.790+0000 7f2da99cb640 1 --2- 172.21.15.112:0/266008586 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2da40ccbb0 0x7f2da41322e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:18.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.790+0000 7f2daa9cd640 1 --2- 172.21.15.112:0/266008586 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2da415c5f0 0x7f2da4132820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:18.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.790+0000 7f2daa1cc640 1 -- 172.21.15.112:0/266008586 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2da40ccbb0 msgr2=0x7f2da41322e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:18.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.790+0000 7f2daa1cc640 1 --2- 172.21.15.112:0/266008586 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2da40ccbb0 0x7f2da41322e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:18.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.790+0000 7f2daa1cc640 1 -- 172.21.15.112:0/266008586 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2da415c5f0 msgr2=0x7f2da4132820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:18.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.790+0000 7f2daa1cc640 1 --2- 172.21.15.112:0/266008586 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2da415c5f0 0x7f2da4132820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:18.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.790+0000 7f2daa1cc640 1 -- 172.21.15.112:0/266008586 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2da41f6030 con 0x7f2da4070210 2024-09-21T00:28:18.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.790+0000 7f2daa1cc640 1 --2- 172.21.15.112:0/266008586 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2da4070210 0x7f2da41f9ba0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f2da000c3a0 tx=0x7f2da000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:18.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.790+0000 7f2d9affd640 1 -- 172.21.15.112:0/266008586 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2da0017070 con 0x7f2da4070210 2024-09-21T00:28:18.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.790+0000 7f2d9affd640 1 -- 172.21.15.112:0/266008586 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2da00053c0 con 0x7f2da4070210 2024-09-21T00:28:18.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.790+0000 7f2d9affd640 1 -- 172.21.15.112:0/266008586 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2da0003850 con 0x7f2da4070210 2024-09-21T00:28:18.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.790+0000 7f2dac430640 1 -- 172.21.15.112:0/266008586 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2da41f6310 con 0x7f2da4070210 2024-09-21T00:28:18.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.790+0000 7f2dac430640 1 -- 172.21.15.112:0/266008586 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2da41f68d0 con 0x7f2da4070210 2024-09-21T00:28:18.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.791+0000 7f2dac430640 1 -- 172.21.15.112:0/266008586 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2da4070620 con 0x7f2da4070210 2024-09-21T00:28:18.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.792+0000 7f2d9affd640 1 -- 172.21.15.112:0/266008586 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2da001a070 con 0x7f2da4070210 2024-09-21T00:28:18.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.792+0000 7f2d9affd640 1 --2- 172.21.15.112:0/266008586 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2d88078b90 0x7f2d8807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:18.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.792+0000 7f2d9affd640 1 -- 172.21.15.112:0/266008586 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6273+0+0 (secure 0 0 0) 0x7f2da009a270 con 0x7f2da4070210 2024-09-21T00:28:18.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.793+0000 7f2da99cb640 1 --2- 172.21.15.112:0/266008586 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2d88078b90 0x7f2d8807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:18.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.793+0000 7f2da99cb640 1 --2- 172.21.15.112:0/266008586 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2d88078b90 0x7f2d8807b0a0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f2d94033000 tx=0x7f2d94004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:18.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:18.797+0000 7f2d9affd640 1 -- 172.21.15.112:0/266008586 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2da0062500 con 0x7f2da4070210 2024-09-21T00:28:19.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:19.010+0000 7f2dac430640 1 -- 172.21.15.112:0/266008586 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f2da4072330 con 0x7f2da4070210 2024-09-21T00:28:19.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:19.653+0000 7f2d9affd640 1 -- 172.21.15.112:0/266008586 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v41) ==== 104+0+0 (secure 0 0 0) 0x7f2da0061e00 con 0x7f2da4070210 2024-09-21T00:28:19.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:19.656+0000 7f2dac430640 1 -- 172.21.15.112:0/266008586 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2d88078b90 msgr2=0x7f2d8807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:19.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:19.656+0000 7f2dac430640 1 --2- 172.21.15.112:0/266008586 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2d88078b90 0x7f2d8807b0a0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f2d94033000 tx=0x7f2d94004730 comp rx=0 tx=0).stop 2024-09-21T00:28:19.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:19.656+0000 7f2dac430640 1 -- 172.21.15.112:0/266008586 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2da4070210 msgr2=0x7f2da41f9ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:19.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:19.656+0000 7f2dac430640 1 --2- 172.21.15.112:0/266008586 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2da4070210 0x7f2da41f9ba0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f2da000c3a0 tx=0x7f2da000c860 comp rx=0 tx=0).stop 2024-09-21T00:28:19.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:19.656+0000 7f2dac430640 1 -- 172.21.15.112:0/266008586 shutdown_connections 2024-09-21T00:28:19.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:19.656+0000 7f2dac430640 1 --2- 172.21.15.112:0/266008586 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2da415c5f0 0x7f2da4132820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:19.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:19.656+0000 7f2dac430640 1 --2- 172.21.15.112:0/266008586 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2da40ccbb0 0x7f2da41322e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:19.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:19.656+0000 7f2dac430640 1 --2- 172.21.15.112:0/266008586 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2d88078b90 0x7f2d8807b0a0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:19.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:19.656+0000 7f2dac430640 1 --2- 172.21.15.112:0/266008586 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2da4070210 0x7f2da41f9ba0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:19.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:19.656+0000 7f2dac430640 1 -- 172.21.15.112:0/266008586 >> 172.21.15.112:0/266008586 conn(0x7f2da4073fe0 msgr2=0x7f2da40cf820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:19.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:19.656+0000 7f2dac430640 1 -- 172.21.15.112:0/266008586 shutdown_connections 2024-09-21T00:28:19.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:19.656+0000 7f2dac430640 1 -- 172.21.15.112:0/266008586 wait complete. 2024-09-21T00:28:19.669 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-21T00:28:19.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:19.786+0000 7ff2805a1640 1 -- 172.21.15.112:0/2592630731 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2780cfc00 msgr2=0x7ff27815bba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:19.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:19.786+0000 7ff2805a1640 1 --2- 172.21.15.112:0/2592630731 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2780cfc00 0x7ff27815bba0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7ff268005fd0 tx=0x7ff26802a8b0 comp rx=0 tx=0).stop 2024-09-21T00:28:19.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:19.786+0000 7ff2805a1640 1 -- 172.21.15.112:0/2592630731 shutdown_connections 2024-09-21T00:28:19.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:19.786+0000 7ff2805a1640 1 --2- 172.21.15.112:0/2592630731 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff278152720 0x7ff27815c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:19.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:19.786+0000 7ff2805a1640 1 --2- 172.21.15.112:0/2592630731 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2780cfc00 0x7ff27815bba0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:19.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:19.786+0000 7ff2805a1640 1 -- 172.21.15.112:0/2592630731 >> 172.21.15.112:0/2592630731 conn(0x7ff278071790 msgr2=0x7ff278071ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:19.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:19.786+0000 7ff2805a1640 1 -- 172.21.15.112:0/2592630731 shutdown_connections 2024-09-21T00:28:19.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:19.786+0000 7ff2805a1640 1 -- 172.21.15.112:0/2592630731 wait complete. 2024-09-21T00:28:19.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:19.788+0000 7ff2805a1640 1 Processor -- start 2024-09-21T00:28:19.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:19.788+0000 7ff2805a1640 1 -- start start 2024-09-21T00:28:19.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:19.789+0000 7ff2805a1640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff2780cfc00 0x7ff2781f3290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:19.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:19.789+0000 7ff2805a1640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff278152720 0x7ff2781f6680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:19.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:19.789+0000 7ff2805a1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff27815c620 0x7ff2781f92d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:19.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:19.789+0000 7ff2805a1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff27815f8a0 con 0x7ff27815c620 2024-09-21T00:28:19.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:19.789+0000 7ff2805a1640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff278070e30 con 0x7ff278152720 2024-09-21T00:28:19.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:19.789+0000 7ff2805a1640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff27815fa20 con 0x7ff2780cfc00 2024-09-21T00:28:19.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:19.789+0000 7ff27eb3e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff27815c620 0x7ff2781f92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:19.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:19.789+0000 7ff27eb3e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff27815c620 0x7ff2781f92d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42464/0 (socket says 172.21.15.112:42464) 2024-09-21T00:28:19.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:19.789+0000 7ff27eb3e640 1 -- 172.21.15.112:0/415010401 learned_addr learned my addr 172.21.15.112:0/415010401 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:28:19.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:19.790+0000 7ff27e33d640 1 --2- 172.21.15.112:0/415010401 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff2780cfc00 0x7ff2781f3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:19.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:19.790+0000 7ff27db3c640 1 --2- 172.21.15.112:0/415010401 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff278152720 0x7ff2781f6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:19.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:19.790+0000 7ff27eb3e640 1 -- 172.21.15.112:0/415010401 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff2780cfc00 msgr2=0x7ff2781f3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:19.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:19.790+0000 7ff27eb3e640 1 --2- 172.21.15.112:0/415010401 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff2780cfc00 0x7ff2781f3290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:19.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:19.790+0000 7ff27eb3e640 1 -- 172.21.15.112:0/415010401 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff278152720 msgr2=0x7ff2781f6680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:19.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:19.790+0000 7ff27eb3e640 1 --2- 172.21.15.112:0/415010401 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff278152720 0x7ff2781f6680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:19.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:19.790+0000 7ff27eb3e640 1 -- 172.21.15.112:0/415010401 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff278155a60 con 0x7ff27815c620 2024-09-21T00:28:19.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:19.790+0000 7ff27db3c640 1 --2- 172.21.15.112:0/415010401 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff278152720 0x7ff2781f6680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:28:19.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:19.790+0000 7ff27eb3e640 1 --2- 172.21.15.112:0/415010401 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff27815c620 0x7ff2781f92d0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7ff27000c3c0 tx=0x7ff27000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:19.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:19.790+0000 7ff276ffd640 1 -- 172.21.15.112:0/415010401 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff270011790 con 0x7ff27815c620 2024-09-21T00:28:19.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:19.790+0000 7ff276ffd640 1 -- 172.21.15.112:0/415010401 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff270017070 con 0x7ff27815c620 2024-09-21T00:28:19.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:19.790+0000 7ff2805a1640 1 -- 172.21.15.112:0/415010401 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2780ad960 con 0x7ff27815c620 2024-09-21T00:28:19.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:19.790+0000 7ff276ffd640 1 -- 172.21.15.112:0/415010401 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff270011c40 con 0x7ff27815c620 2024-09-21T00:28:19.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:19.790+0000 7ff2805a1640 1 -- 172.21.15.112:0/415010401 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2780adf20 con 0x7ff27815c620 2024-09-21T00:28:19.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:19.791+0000 7ff2805a1640 1 -- 172.21.15.112:0/415010401 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2781fa0c0 con 0x7ff27815c620 2024-09-21T00:28:19.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:19.792+0000 7ff276ffd640 1 -- 172.21.15.112:0/415010401 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff27001c460 con 0x7ff27815c620 2024-09-21T00:28:19.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:19.792+0000 7ff276ffd640 1 --2- 172.21.15.112:0/415010401 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff264078b90 0x7ff26407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:19.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:19.792+0000 7ff276ffd640 1 -- 172.21.15.112:0/415010401 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6273+0+0 (secure 0 0 0) 0x7ff27009ac20 con 0x7ff27815c620 2024-09-21T00:28:19.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:19.793+0000 7ff27e33d640 1 --2- 172.21.15.112:0/415010401 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff264078b90 0x7ff26407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:19.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:19.793+0000 7ff27e33d640 1 --2- 172.21.15.112:0/415010401 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff264078b90 0x7ff26407b0a0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ff268034000 tx=0x7ff268004ab0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:19.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:19.797+0000 7ff276ffd640 1 -- 172.21.15.112:0/415010401 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ff270062e20 con 0x7ff27815c620 2024-09-21T00:28:19.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:19.977+0000 7ff2805a1640 1 -- 172.21.15.112:0/415010401 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7ff27815fe80 con 0x7ff27815c620 2024-09-21T00:28:20.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:20.867+0000 7ff276ffd640 1 -- 172.21.15.112:0/415010401 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v62) ==== 181+0+0 (secure 0 0 0) 0x7ff270062720 con 0x7ff27815c620 2024-09-21T00:28:20.868 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'cephfs_metadata' removed 2024-09-21T00:28:20.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:20.871+0000 7ff2805a1640 1 -- 172.21.15.112:0/415010401 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff264078b90 msgr2=0x7ff26407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:20.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:20.871+0000 7ff2805a1640 1 --2- 172.21.15.112:0/415010401 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff264078b90 0x7ff26407b0a0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ff268034000 tx=0x7ff268004ab0 comp rx=0 tx=0).stop 2024-09-21T00:28:20.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:20.871+0000 7ff2805a1640 1 -- 172.21.15.112:0/415010401 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff27815c620 msgr2=0x7ff2781f92d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:20.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:20.871+0000 7ff2805a1640 1 --2- 172.21.15.112:0/415010401 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff27815c620 0x7ff2781f92d0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7ff27000c3c0 tx=0x7ff27000c880 comp rx=0 tx=0).stop 2024-09-21T00:28:20.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:20.871+0000 7ff2805a1640 1 -- 172.21.15.112:0/415010401 shutdown_connections 2024-09-21T00:28:20.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:20.871+0000 7ff2805a1640 1 --2- 172.21.15.112:0/415010401 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff264078b90 0x7ff26407b0a0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:20.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:20.871+0000 7ff2805a1640 1 --2- 172.21.15.112:0/415010401 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff27815c620 0x7ff2781f92d0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:20.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:20.871+0000 7ff2805a1640 1 --2- 172.21.15.112:0/415010401 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff278152720 0x7ff2781f6680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:20.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:20.871+0000 7ff2805a1640 1 --2- 172.21.15.112:0/415010401 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff2780cfc00 0x7ff2781f3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:20.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:20.871+0000 7ff2805a1640 1 -- 172.21.15.112:0/415010401 >> 172.21.15.112:0/415010401 conn(0x7ff278071790 msgr2=0x7ff27815cdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:20.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:20.871+0000 7ff2805a1640 1 -- 172.21.15.112:0/415010401 shutdown_connections 2024-09-21T00:28:20.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:20.871+0000 7ff2805a1640 1 -- 172.21.15.112:0/415010401 wait complete. 2024-09-21T00:28:20.887 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-21T00:28:21.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:21.007+0000 7f1694558640 1 -- 172.21.15.112:0/3917264588 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f168c070a80 msgr2=0x7f168c0cd8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:21.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:21.007+0000 7f1694558640 1 --2- 172.21.15.112:0/3917264588 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f168c070a80 0x7f168c0cd8a0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f1688005fd0 tx=0x7f168802a530 comp rx=0 tx=0).stop 2024-09-21T00:28:21.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:21.007+0000 7f1694558640 1 -- 172.21.15.112:0/3917264588 shutdown_connections 2024-09-21T00:28:21.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:21.007+0000 7f1694558640 1 --2- 172.21.15.112:0/3917264588 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f168c06e3c0 0x7f168c0cca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:21.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:21.007+0000 7f1694558640 1 --2- 172.21.15.112:0/3917264588 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f168c070a80 0x7f168c0cd8a0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:21.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:21.007+0000 7f1694558640 1 -- 172.21.15.112:0/3917264588 >> 172.21.15.112:0/3917264588 conn(0x7f168c0d0c20 msgr2=0x7f168c0cd070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:21.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:21.007+0000 7f1694558640 1 -- 172.21.15.112:0/3917264588 shutdown_connections 2024-09-21T00:28:21.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:21.007+0000 7f1694558640 1 -- 172.21.15.112:0/3917264588 wait complete. 2024-09-21T00:28:21.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:21.009+0000 7f1694558640 1 Processor -- start 2024-09-21T00:28:21.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:21.009+0000 7f1694558640 1 -- start start 2024-09-21T00:28:21.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:21.010+0000 7f1694558640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f168c06e3c0 0x7f168c0c98c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:21.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:21.010+0000 7f1694558640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f168c070a80 0x7f168c1f7c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:21.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:21.010+0000 7f1694558640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f168c15f0f0 0x7f168c0c7c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:21.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:21.010+0000 7f1694558640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f168c0ce7f0 con 0x7f168c06e3c0 2024-09-21T00:28:21.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:21.010+0000 7f1694558640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f168c0ce670 con 0x7f168c15f0f0 2024-09-21T00:28:21.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:21.010+0000 7f1694558640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f168c0ce970 con 0x7f168c070a80 2024-09-21T00:28:21.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:21.010+0000 7f16922f4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f168c06e3c0 0x7f168c0c98c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:21.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:21.010+0000 7f16922f4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f168c06e3c0 0x7f168c0c98c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42482/0 (socket says 172.21.15.112:42482) 2024-09-21T00:28:21.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:21.010+0000 7f16922f4640 1 -- 172.21.15.112:0/1776560721 learned_addr learned my addr 172.21.15.112:0/1776560721 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:28:21.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:21.010+0000 7f1692af5640 1 --2- 172.21.15.112:0/1776560721 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f168c15f0f0 0x7f168c0c7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:21.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:21.010+0000 7f1691af3640 1 --2- 172.21.15.112:0/1776560721 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f168c070a80 0x7f168c1f7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:21.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:21.010+0000 7f16922f4640 1 -- 172.21.15.112:0/1776560721 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f168c070a80 msgr2=0x7f168c1f7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:21.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:21.010+0000 7f16922f4640 1 --2- 172.21.15.112:0/1776560721 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f168c070a80 0x7f168c1f7c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:21.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:21.010+0000 7f16922f4640 1 -- 172.21.15.112:0/1776560721 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f168c15f0f0 msgr2=0x7f168c0c7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:21.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:21.010+0000 7f16922f4640 1 --2- 172.21.15.112:0/1776560721 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f168c15f0f0 0x7f168c0c7c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:21.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:21.010+0000 7f16922f4640 1 -- 172.21.15.112:0/1776560721 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f168c0ca150 con 0x7f168c06e3c0 2024-09-21T00:28:21.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:21.010+0000 7f1691af3640 1 --2- 172.21.15.112:0/1776560721 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f168c070a80 0x7f168c1f7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:28:21.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:21.010+0000 7f16922f4640 1 --2- 172.21.15.112:0/1776560721 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f168c06e3c0 0x7f168c0c98c0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f1688034040 tx=0x7f16880039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:21.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:21.011+0000 7f1682ffd640 1 -- 172.21.15.112:0/1776560721 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f168803e030 con 0x7f168c06e3c0 2024-09-21T00:28:21.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:21.011+0000 7f1682ffd640 1 -- 172.21.15.112:0/1776560721 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f16880047a0 con 0x7f168c06e3c0 2024-09-21T00:28:21.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:21.011+0000 7f1694558640 1 -- 172.21.15.112:0/1776560721 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f168c1f7140 con 0x7f168c06e3c0 2024-09-21T00:28:21.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:21.011+0000 7f1682ffd640 1 -- 172.21.15.112:0/1776560721 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1688035040 con 0x7f168c06e3c0 2024-09-21T00:28:21.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:21.011+0000 7f1694558640 1 -- 172.21.15.112:0/1776560721 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f168c1f7700 con 0x7f168c06e3c0 2024-09-21T00:28:21.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:21.012+0000 7f1694558640 1 -- 172.21.15.112:0/1776560721 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f168c0b1ea0 con 0x7f168c06e3c0 2024-09-21T00:28:21.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:21.012+0000 7f1682ffd640 1 -- 172.21.15.112:0/1776560721 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1688041030 con 0x7f168c06e3c0 2024-09-21T00:28:21.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:21.013+0000 7f1682ffd640 1 --2- 172.21.15.112:0/1776560721 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1670078b90 0x7f167007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:21.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:21.013+0000 7f1682ffd640 1 -- 172.21.15.112:0/1776560721 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5835+0+0 (secure 0 0 0) 0x7f16880bb620 con 0x7f168c06e3c0 2024-09-21T00:28:21.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:21.013+0000 7f1691af3640 1 --2- 172.21.15.112:0/1776560721 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1670078b90 0x7f167007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:21.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:21.014+0000 7f1691af3640 1 --2- 172.21.15.112:0/1776560721 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1670078b90 0x7f167007b0a0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f167c005fd0 tx=0x7f167c005e00 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:21.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:21.017+0000 7f1682ffd640 1 -- 172.21.15.112:0/1776560721 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f168c0b1ea0 con 0x7f168c06e3c0 2024-09-21T00:28:21.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:21.195+0000 7f1694558640 1 -- 172.21.15.112:0/1776560721 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f168c0cb2e0 con 0x7f168c06e3c0 2024-09-21T00:28:21.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:21.875+0000 7f1682ffd640 1 -- 172.21.15.112:0/1776560721 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v63) ==== 169+0+0 (secure 0 0 0) 0x7f168c0cb2e0 con 0x7f168c06e3c0 2024-09-21T00:28:21.875 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'cephfs_data' removed 2024-09-21T00:28:21.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:21.877+0000 7f1694558640 1 -- 172.21.15.112:0/1776560721 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1670078b90 msgr2=0x7f167007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:21.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:21.877+0000 7f1694558640 1 --2- 172.21.15.112:0/1776560721 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1670078b90 0x7f167007b0a0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f167c005fd0 tx=0x7f167c005e00 comp rx=0 tx=0).stop 2024-09-21T00:28:21.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:21.877+0000 7f1694558640 1 -- 172.21.15.112:0/1776560721 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f168c06e3c0 msgr2=0x7f168c0c98c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:21.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:21.877+0000 7f1694558640 1 --2- 172.21.15.112:0/1776560721 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f168c06e3c0 0x7f168c0c98c0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f1688034040 tx=0x7f16880039e0 comp rx=0 tx=0).stop 2024-09-21T00:28:21.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:21.877+0000 7f1694558640 1 -- 172.21.15.112:0/1776560721 shutdown_connections 2024-09-21T00:28:21.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:21.877+0000 7f1694558640 1 --2- 172.21.15.112:0/1776560721 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f168c15f0f0 0x7f168c0c7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:21.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:21.877+0000 7f1694558640 1 --2- 172.21.15.112:0/1776560721 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f168c070a80 0x7f168c1f7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:21.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:21.877+0000 7f1694558640 1 --2- 172.21.15.112:0/1776560721 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1670078b90 0x7f167007b0a0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:21.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:21.877+0000 7f1694558640 1 --2- 172.21.15.112:0/1776560721 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f168c06e3c0 0x7f168c0c98c0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:21.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:21.877+0000 7f1694558640 1 -- 172.21.15.112:0/1776560721 >> 172.21.15.112:0/1776560721 conn(0x7f168c0d0c20 msgr2=0x7f168c0b1740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:21.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:21.877+0000 7f1694558640 1 -- 172.21.15.112:0/1776560721 shutdown_connections 2024-09-21T00:28:21.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:21.877+0000 7f1694558640 1 -- 172.21.15.112:0/1776560721 wait complete. 2024-09-21T00:28:21.893 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 31 2024-09-21T00:28:22.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:22.005+0000 7ff1f6640640 1 -- 172.21.15.112:0/580890750 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1f0162400 msgr2=0x7ff1f015e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:22.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:22.005+0000 7ff1f6640640 1 --2- 172.21.15.112:0/580890750 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1f0162400 0x7ff1f015e9b0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7ff1d8009860 tx=0x7ff1d802a540 comp rx=0 tx=0).stop 2024-09-21T00:28:22.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:22.005+0000 7ff1f6640640 1 -- 172.21.15.112:0/580890750 shutdown_connections 2024-09-21T00:28:22.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:22.005+0000 7ff1f6640640 1 --2- 172.21.15.112:0/580890750 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1f0162400 0x7ff1f015e9b0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:22.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:22.005+0000 7ff1f6640640 1 -- 172.21.15.112:0/580890750 >> 172.21.15.112:0/580890750 conn(0x7ff1f008bfe0 msgr2=0x7ff1f009d850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:22.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:22.005+0000 7ff1f6640640 1 -- 172.21.15.112:0/580890750 shutdown_connections 2024-09-21T00:28:22.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:22.005+0000 7ff1f6640640 1 -- 172.21.15.112:0/580890750 wait complete. 2024-09-21T00:28:22.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:22.007+0000 7ff1f6640640 1 Processor -- start 2024-09-21T00:28:22.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:22.007+0000 7ff1f6640640 1 -- start start 2024-09-21T00:28:22.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:22.008+0000 7ff1f6640640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1f00cfdd0 0x7ff1f0160ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:22.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:22.008+0000 7ff1f6640640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff1f00d4520 0x7ff1f0161430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:22.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:22.008+0000 7ff1f6640640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff1f0162400 0x7ff1f00b6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:22.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:22.008+0000 7ff1f6640640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f0169ec0 con 0x7ff1f00cfdd0 2024-09-21T00:28:22.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:22.008+0000 7ff1f6640640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f0169d40 con 0x7ff1f00d4520 2024-09-21T00:28:22.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:22.008+0000 7ff1f6640640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff1f016a040 con 0x7ff1f0162400 2024-09-21T00:28:22.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:22.008+0000 7ff1effff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1f00cfdd0 0x7ff1f0160ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:22.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:22.008+0000 7ff1effff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1f00cfdd0 0x7ff1f0160ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42514/0 (socket says 172.21.15.112:42514) 2024-09-21T00:28:22.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:22.008+0000 7ff1effff640 1 -- 172.21.15.112:0/1675536679 learned_addr learned my addr 172.21.15.112:0/1675536679 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:28:22.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:22.009+0000 7ff1ef7fe640 1 --2- 172.21.15.112:0/1675536679 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff1f00d4520 0x7ff1f0161430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:22.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:22.009+0000 7ff1f4bdd640 1 --2- 172.21.15.112:0/1675536679 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff1f0162400 0x7ff1f00b6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:22.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:22.009+0000 7ff1effff640 1 -- 172.21.15.112:0/1675536679 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff1f0162400 msgr2=0x7ff1f00b6180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:22.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:22.009+0000 7ff1effff640 1 --2- 172.21.15.112:0/1675536679 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff1f0162400 0x7ff1f00b6180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:22.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:22.009+0000 7ff1effff640 1 -- 172.21.15.112:0/1675536679 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff1f00d4520 msgr2=0x7ff1f0161430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:22.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:22.009+0000 7ff1effff640 1 --2- 172.21.15.112:0/1675536679 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff1f00d4520 0x7ff1f0161430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:22.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:22.009+0000 7ff1effff640 1 -- 172.21.15.112:0/1675536679 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1f0161970 con 0x7ff1f00cfdd0 2024-09-21T00:28:22.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:22.009+0000 7ff1ef7fe640 1 --2- 172.21.15.112:0/1675536679 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff1f00d4520 0x7ff1f0161430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:28:22.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:22.009+0000 7ff1effff640 1 --2- 172.21.15.112:0/1675536679 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1f00cfdd0 0x7ff1f0160ef0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7ff1e000c3a0 tx=0x7ff1e000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:22.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:22.009+0000 7ff1ecff9640 1 -- 172.21.15.112:0/1675536679 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e0011790 con 0x7ff1f00cfdd0 2024-09-21T00:28:22.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:22.009+0000 7ff1ecff9640 1 -- 172.21.15.112:0/1675536679 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff1e0017070 con 0x7ff1f00cfdd0 2024-09-21T00:28:22.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:22.009+0000 7ff1f6640640 1 -- 172.21.15.112:0/1675536679 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1f015dc20 con 0x7ff1f00cfdd0 2024-09-21T00:28:22.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:22.009+0000 7ff1ecff9640 1 -- 172.21.15.112:0/1675536679 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e0011ce0 con 0x7ff1f00cfdd0 2024-09-21T00:28:22.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:22.009+0000 7ff1f6640640 1 -- 172.21.15.112:0/1675536679 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1f00b56a0 con 0x7ff1f00cfdd0 2024-09-21T00:28:22.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:22.010+0000 7ff1f6640640 1 -- 172.21.15.112:0/1675536679 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1f015e9b0 con 0x7ff1f00cfdd0 2024-09-21T00:28:22.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:22.011+0000 7ff1ecff9640 1 -- 172.21.15.112:0/1675536679 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff1e001c460 con 0x7ff1f00cfdd0 2024-09-21T00:28:22.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:22.011+0000 7ff1ecff9640 1 --2- 172.21.15.112:0/1675536679 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff1d0078b90 0x7ff1d007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:22.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:22.012+0000 7ff1ecff9640 1 -- 172.21.15.112:0/1675536679 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5421+0+0 (secure 0 0 0) 0x7ff1e009a9a0 con 0x7ff1f00cfdd0 2024-09-21T00:28:22.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:22.012+0000 7ff1ef7fe640 1 --2- 172.21.15.112:0/1675536679 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff1d0078b90 0x7ff1d007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:22.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:22.012+0000 7ff1ef7fe640 1 --2- 172.21.15.112:0/1675536679 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff1d0078b90 0x7ff1d007b0a0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7ff1d8034000 tx=0x7ff1d8004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:22.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:22.016+0000 7ff1ecff9640 1 -- 172.21.15.112:0/1675536679 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ff1e0062f90 con 0x7ff1f00cfdd0 2024-09-21T00:28:22.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:22.194+0000 7ff1f6640640 1 -- 172.21.15.112:0/1675536679 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "config reset", "num": 31} v 0) -- 0x7ff1f00b1ed0 con 0x7ff1f00cfdd0 2024-09-21T00:28:22.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:22.198+0000 7ff1ecff9640 1 -- 172.21.15.112:0/1675536679 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 31}]=0 v39) ==== 71+0+0 (secure 0 0 0) 0x7ff1e0062890 con 0x7ff1f00cfdd0 2024-09-21T00:28:22.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:22.200+0000 7ff1f6640640 1 -- 172.21.15.112:0/1675536679 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff1d0078b90 msgr2=0x7ff1d007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:22.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:22.200+0000 7ff1f6640640 1 --2- 172.21.15.112:0/1675536679 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff1d0078b90 0x7ff1d007b0a0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7ff1d8034000 tx=0x7ff1d8004730 comp rx=0 tx=0).stop 2024-09-21T00:28:22.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:22.200+0000 7ff1f6640640 1 -- 172.21.15.112:0/1675536679 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1f00cfdd0 msgr2=0x7ff1f0160ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:22.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:22.200+0000 7ff1f6640640 1 --2- 172.21.15.112:0/1675536679 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1f00cfdd0 0x7ff1f0160ef0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7ff1e000c3a0 tx=0x7ff1e000c860 comp rx=0 tx=0).stop 2024-09-21T00:28:22.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:22.200+0000 7ff1f6640640 1 -- 172.21.15.112:0/1675536679 shutdown_connections 2024-09-21T00:28:22.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:22.200+0000 7ff1f6640640 1 --2- 172.21.15.112:0/1675536679 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff1f0162400 0x7ff1f00b6180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:22.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:22.200+0000 7ff1f6640640 1 --2- 172.21.15.112:0/1675536679 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff1f00d4520 0x7ff1f0161430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:22.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:22.200+0000 7ff1f6640640 1 --2- 172.21.15.112:0/1675536679 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff1d0078b90 0x7ff1d007b0a0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:22.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:22.200+0000 7ff1f6640640 1 --2- 172.21.15.112:0/1675536679 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1f00cfdd0 0x7ff1f0160ef0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:22.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:22.200+0000 7ff1f6640640 1 -- 172.21.15.112:0/1675536679 >> 172.21.15.112:0/1675536679 conn(0x7ff1f008bfe0 msgr2=0x7ff1f00b23f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:22.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:22.200+0000 7ff1f6640640 1 -- 172.21.15.112:0/1675536679 shutdown_connections 2024-09-21T00:28:22.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:22.200+0000 7ff1f6640640 1 -- 172.21.15.112:0/1675536679 wait complete. 2024-09-21T00:28:22.212 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_sessionmap.TestSessionMap.test_session_evict_non_blocklisted' 2024-09-21T00:28:22.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:22.331+0000 7f5d71fcc640 1 -- 172.21.15.112:0/3673482273 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d6c0d0ae0 msgr2=0x7f5d6c0d38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:22.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:22.331+0000 7f5d71fcc640 1 --2- 172.21.15.112:0/3673482273 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d6c0d0ae0 0x7f5d6c0d38c0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f5d5c0098e0 tx=0x7f5d5c02a530 comp rx=0 tx=0).stop 2024-09-21T00:28:22.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:22.331+0000 7f5d71fcc640 1 -- 172.21.15.112:0/3673482273 shutdown_connections 2024-09-21T00:28:22.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:22.331+0000 7f5d71fcc640 1 --2- 172.21.15.112:0/3673482273 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5d6c0d4a90 0x7f5d6c070250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:22.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:22.331+0000 7f5d71fcc640 1 --2- 172.21.15.112:0/3673482273 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d6c0d0ae0 0x7f5d6c0d38c0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:22.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:22.331+0000 7f5d71fcc640 1 -- 172.21.15.112:0/3673482273 >> 172.21.15.112:0/3673482273 conn(0x7f5d6c09d8a0 msgr2=0x7f5d6c08bfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:22.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:22.331+0000 7f5d71fcc640 1 -- 172.21.15.112:0/3673482273 shutdown_connections 2024-09-21T00:28:22.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:22.331+0000 7f5d71fcc640 1 -- 172.21.15.112:0/3673482273 wait complete. 2024-09-21T00:28:22.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:22.332+0000 7f5d71fcc640 1 Processor -- start 2024-09-21T00:28:22.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:22.333+0000 7f5d71fcc640 1 -- start start 2024-09-21T00:28:22.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:22.334+0000 7f5d71fcc640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5d6c0d0ae0 0x7f5d6c153360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:22.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:22.334+0000 7f5d71fcc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d6c0d4a90 0x7f5d6c06e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:22.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:22.334+0000 7f5d71fcc640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5d6c15ea40 0x7f5d6c06e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:22.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:22.334+0000 7f5d71fcc640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5d6c162b60 con 0x7f5d6c0d4a90 2024-09-21T00:28:22.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:22.334+0000 7f5d71fcc640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5d6c1629e0 con 0x7f5d6c15ea40 2024-09-21T00:28:22.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:22.334+0000 7f5d71fcc640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f5d6c162ce0 con 0x7f5d6c0d0ae0 2024-09-21T00:28:22.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:22.334+0000 7f5d6affd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d6c0d4a90 0x7f5d6c06e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:22.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:22.334+0000 7f5d6affd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d6c0d4a90 0x7f5d6c06e2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42532/0 (socket says 172.21.15.112:42532) 2024-09-21T00:28:22.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:22.334+0000 7f5d6affd640 1 -- 172.21.15.112:0/908526096 learned_addr learned my addr 172.21.15.112:0/908526096 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:28:22.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:22.334+0000 7f5d6b7fe640 1 --2- 172.21.15.112:0/908526096 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5d6c0d0ae0 0x7f5d6c153360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:22.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:22.334+0000 7f5d6bfff640 1 --2- 172.21.15.112:0/908526096 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5d6c15ea40 0x7f5d6c06e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:22.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:22.334+0000 7f5d6affd640 1 -- 172.21.15.112:0/908526096 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5d6c0d0ae0 msgr2=0x7f5d6c153360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:22.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:22.334+0000 7f5d6affd640 1 --2- 172.21.15.112:0/908526096 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5d6c0d0ae0 0x7f5d6c153360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:22.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:22.334+0000 7f5d6affd640 1 -- 172.21.15.112:0/908526096 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5d6c15ea40 msgr2=0x7f5d6c06e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:22.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:22.334+0000 7f5d6affd640 1 --2- 172.21.15.112:0/908526096 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5d6c15ea40 0x7f5d6c06e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:22.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:22.334+0000 7f5d6affd640 1 -- 172.21.15.112:0/908526096 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d6c0c9400 con 0x7f5d6c0d4a90 2024-09-21T00:28:22.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:22.334+0000 7f5d6b7fe640 1 --2- 172.21.15.112:0/908526096 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5d6c0d0ae0 0x7f5d6c153360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:28:22.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:22.334+0000 7f5d6affd640 1 --2- 172.21.15.112:0/908526096 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d6c0d4a90 0x7f5d6c06e2e0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f5d6000c3c0 tx=0x7f5d6000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:22.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:22.335+0000 7f5d53fff640 1 -- 172.21.15.112:0/908526096 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d60017070 con 0x7f5d6c0d4a90 2024-09-21T00:28:22.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:22.335+0000 7f5d53fff640 1 -- 172.21.15.112:0/908526096 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5d600053e0 con 0x7f5d6c0d4a90 2024-09-21T00:28:22.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:22.335+0000 7f5d53fff640 1 -- 172.21.15.112:0/908526096 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d60003870 con 0x7f5d6c0d4a90 2024-09-21T00:28:22.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:22.335+0000 7f5d71fcc640 1 -- 172.21.15.112:0/908526096 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d6c0c96e0 con 0x7f5d6c0d4a90 2024-09-21T00:28:22.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:22.335+0000 7f5d71fcc640 1 -- 172.21.15.112:0/908526096 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d6c15cc70 con 0x7f5d6c0d4a90 2024-09-21T00:28:22.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:22.336+0000 7f5d71fcc640 1 -- 172.21.15.112:0/908526096 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d6c1631b0 con 0x7f5d6c0d4a90 2024-09-21T00:28:22.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:22.337+0000 7f5d53fff640 1 -- 172.21.15.112:0/908526096 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5d6001a070 con 0x7f5d6c0d4a90 2024-09-21T00:28:22.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:22.337+0000 7f5d53fff640 1 --2- 172.21.15.112:0/908526096 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f5d58078b90 0x7f5d5807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:22.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:22.337+0000 7f5d53fff640 1 -- 172.21.15.112:0/908526096 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5421+0+0 (secure 0 0 0) 0x7f5d60099e50 con 0x7f5d6c0d4a90 2024-09-21T00:28:22.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:22.337+0000 7f5d6b7fe640 1 --2- 172.21.15.112:0/908526096 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f5d58078b90 0x7f5d5807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:22.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:22.338+0000 7f5d6b7fe640 1 --2- 172.21.15.112:0/908526096 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f5d58078b90 0x7f5d5807b0a0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f5d5c034000 tx=0x7f5d5c004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:22.342 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:22.341+0000 7f5d53fff640 1 -- 172.21.15.112:0/908526096 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5d60061f40 con 0x7f5d6c0d4a90 2024-09-21T00:28:22.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:22.586+0000 7f5d71fcc640 1 -- 172.21.15.112:0/908526096 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_sessionmap.TestSessionMap.test_session_evict_non_blocklisted"]} v 0) -- 0x7f5d6c15cee0 con 0x7f5d6c0d4a90 2024-09-21T00:28:22.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:22.876+0000 7f5d53fff640 1 -- 172.21.15.112:0/908526096 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_sessionmap.TestSessionMap.test_session_evict_non_blocklisted"]}]=0 v161) ==== 157+0+0 (secure 0 0 0) 0x7f5d60066970 con 0x7f5d6c0d4a90 2024-09-21T00:28:22.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:22.878+0000 7f5d71fcc640 1 -- 172.21.15.112:0/908526096 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f5d58078b90 msgr2=0x7f5d5807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:22.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:22.878+0000 7f5d71fcc640 1 --2- 172.21.15.112:0/908526096 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f5d58078b90 0x7f5d5807b0a0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f5d5c034000 tx=0x7f5d5c004730 comp rx=0 tx=0).stop 2024-09-21T00:28:22.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:22.878+0000 7f5d71fcc640 1 -- 172.21.15.112:0/908526096 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d6c0d4a90 msgr2=0x7f5d6c06e2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:22.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:22.878+0000 7f5d71fcc640 1 --2- 172.21.15.112:0/908526096 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d6c0d4a90 0x7f5d6c06e2e0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f5d6000c3c0 tx=0x7f5d6000c880 comp rx=0 tx=0).stop 2024-09-21T00:28:22.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:22.878+0000 7f5d71fcc640 1 -- 172.21.15.112:0/908526096 shutdown_connections 2024-09-21T00:28:22.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:22.878+0000 7f5d71fcc640 1 --2- 172.21.15.112:0/908526096 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5d6c15ea40 0x7f5d6c06e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:22.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:22.878+0000 7f5d71fcc640 1 --2- 172.21.15.112:0/908526096 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f5d58078b90 0x7f5d5807b0a0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:22.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:22.878+0000 7f5d71fcc640 1 --2- 172.21.15.112:0/908526096 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d6c0d4a90 0x7f5d6c06e2e0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:22.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:22.878+0000 7f5d71fcc640 1 --2- 172.21.15.112:0/908526096 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5d6c0d0ae0 0x7f5d6c153360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:22.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:22.878+0000 7f5d71fcc640 1 -- 172.21.15.112:0/908526096 >> 172.21.15.112:0/908526096 conn(0x7f5d6c09d8a0 msgr2=0x7f5d6c0d4350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:22.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:22.879+0000 7f5d71fcc640 1 -- 172.21.15.112:0/908526096 shutdown_connections 2024-09-21T00:28:22.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:22.879+0000 7f5d71fcc640 1 -- 172.21.15.112:0/908526096 wait complete. 2024-09-21T00:28:22.893 INFO:tasks.cephfs_test_runner:Check that mds evicts without blocklisting client ... ok 2024-09-21T00:28:22.893 INFO:tasks.cephfs_test_runner:Starting test: test_session_reject (tasks.cephfs.test_sessionmap.TestSessionMap) 2024-09-21T00:28:22.893 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T00:28:23.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.005+0000 7f543690b640 1 -- 172.21.15.112:0/1849476565 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5418004800 msgr2=0x7f5418004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:23.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.005+0000 7f543690b640 1 --2- 172.21.15.112:0/1849476565 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5418004800 0x7f5418004c10 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f54300ced50 tx=0x7f5418005830 comp rx=0 tx=0).stop 2024-09-21T00:28:23.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.005+0000 7f543690b640 1 -- 172.21.15.112:0/1849476565 shutdown_connections 2024-09-21T00:28:23.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.005+0000 7f543690b640 1 --2- 172.21.15.112:0/1849476565 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5418004800 0x7f5418004c10 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:23.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.005+0000 7f543690b640 1 -- 172.21.15.112:0/1849476565 >> 172.21.15.112:0/1849476565 conn(0x7f54300b42e0 msgr2=0x7f54300b4720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:23.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.005+0000 7f543690b640 1 -- 172.21.15.112:0/1849476565 shutdown_connections 2024-09-21T00:28:23.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.005+0000 7f543690b640 1 -- 172.21.15.112:0/1849476565 wait complete. 2024-09-21T00:28:23.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.007+0000 7f543690b640 1 Processor -- start 2024-09-21T00:28:23.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.007+0000 7f543690b640 1 -- start start 2024-09-21T00:28:23.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.009+0000 7f543690b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5418004800 0x7f5430153d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:23.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.009+0000 7f543690b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f54300b70f0 0x7f54300c76b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:23.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.009+0000 7f543690b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f54300c7bf0 0x7f5430156050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:23.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.009+0000 7f543690b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f54300cd390 con 0x7f54300c7bf0 2024-09-21T00:28:23.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.009+0000 7f543690b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f54300cd210 con 0x7f54300b70f0 2024-09-21T00:28:23.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.009+0000 7f543690b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f54300cd510 con 0x7f5418004800 2024-09-21T00:28:23.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.009+0000 7f5434ea8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f54300c7bf0 0x7f5430156050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:23.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.009+0000 7f5434ea8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f54300c7bf0 0x7f5430156050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42552/0 (socket says 172.21.15.112:42552) 2024-09-21T00:28:23.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.009+0000 7f5434ea8640 1 -- 172.21.15.112:0/1121356577 learned_addr learned my addr 172.21.15.112:0/1121356577 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:28:23.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.009+0000 7f542f7fe640 1 --2- 172.21.15.112:0/1121356577 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f54300b70f0 0x7f54300c76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:23.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.009+0000 7f542ffff640 1 --2- 172.21.15.112:0/1121356577 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5418004800 0x7f5430153d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:23.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.009+0000 7f5434ea8640 1 -- 172.21.15.112:0/1121356577 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5418004800 msgr2=0x7f5430153d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:23.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.009+0000 7f5434ea8640 1 --2- 172.21.15.112:0/1121356577 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5418004800 0x7f5430153d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:23.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.009+0000 7f5434ea8640 1 -- 172.21.15.112:0/1121356577 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f54300b70f0 msgr2=0x7f54300c76b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:23.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.009+0000 7f5434ea8640 1 --2- 172.21.15.112:0/1121356577 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f54300b70f0 0x7f54300c76b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:23.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.009+0000 7f5434ea8640 1 -- 172.21.15.112:0/1121356577 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54300c8000 con 0x7f54300c7bf0 2024-09-21T00:28:23.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.009+0000 7f542ffff640 1 --2- 172.21.15.112:0/1121356577 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5418004800 0x7f5430153d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:28:23.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.009+0000 7f5434ea8640 1 --2- 172.21.15.112:0/1121356577 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f54300c7bf0 0x7f5430156050 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f54300ce190 tx=0x7f5428010f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:23.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.009+0000 7f542cff9640 1 -- 172.21.15.112:0/1121356577 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5428003970 con 0x7f54300c7bf0 2024-09-21T00:28:23.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.009+0000 7f542cff9640 1 -- 172.21.15.112:0/1121356577 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5428015070 con 0x7f54300c7bf0 2024-09-21T00:28:23.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.009+0000 7f542cff9640 1 -- 172.21.15.112:0/1121356577 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5428004030 con 0x7f54300c7bf0 2024-09-21T00:28:23.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.009+0000 7f543690b640 1 -- 172.21.15.112:0/1121356577 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54300c9e70 con 0x7f54300c7bf0 2024-09-21T00:28:23.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.009+0000 7f543690b640 1 -- 172.21.15.112:0/1121356577 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54300c94e0 con 0x7f54300c7bf0 2024-09-21T00:28:23.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.010+0000 7f543690b640 1 -- 172.21.15.112:0/1121356577 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54300c6940 con 0x7f54300c7bf0 2024-09-21T00:28:23.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.011+0000 7f542cff9640 1 -- 172.21.15.112:0/1121356577 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f54280041d0 con 0x7f54300c7bf0 2024-09-21T00:28:23.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.012+0000 7f542cff9640 1 --2- 172.21.15.112:0/1121356577 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f541c078b90 0x7f541c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:23.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.012+0000 7f542cff9640 1 -- 172.21.15.112:0/1121356577 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5421+0+0 (secure 0 0 0) 0x7f542809b6d0 con 0x7f54300c7bf0 2024-09-21T00:28:23.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.012+0000 7f542ffff640 1 --2- 172.21.15.112:0/1121356577 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f541c078b90 0x7f541c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:23.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.012+0000 7f542ffff640 1 --2- 172.21.15.112:0/1121356577 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f541c078b90 0x7f541c07b0a0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f543006c310 tx=0x7f5420006ba0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:23.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.016+0000 7f542cff9640 1 -- 172.21.15.112:0/1121356577 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f54300c8000 con 0x7f54300c7bf0 2024-09-21T00:28:23.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.196+0000 7f543690b640 1 -- 172.21.15.112:0/1121356577 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5430068fe0 con 0x7f54300c7bf0 2024-09-21T00:28:23.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.197+0000 7f542cff9640 1 -- 172.21.15.112:0/1121356577 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) ==== 74+0+13464 (secure 0 0 0) 0x7f5428063700 con 0x7f54300c7bf0 2024-09-21T00:28:23.198 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:28:23.198 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":63,"fsid":"8b34692b-dc65-49b2-a08f-26f707b96ac5","created":"2024-09-21T00:25:22.788490+0000","modified":"2024-09-21T00:28:21.867999+0000","last_up_change":"2024-09-21T00:25:36.161835+0000","last_in_change":"2024-09-21T00:25:26.346500+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T00:25:37.105006+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9c575086-67d2-4de2-b01b-d44dcc21d604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6809","nonce":755760621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6811","nonce":755760621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6815","nonce":755760621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6813","nonce":755760621}]},"public_addr":"172.21.15.112:6809/755760621","cluster_addr":"172.21.15.112:6811/755760621","heartbeat_back_addr":"172.21.15.112:6815/755760621","heartbeat_front_addr":"172.21.15.112:6813/755760621","state":["exists","up"]},{"osd":1,"uuid":"a983c227-a3ce-4bdd-82f1-438ba7e3ecd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6801","nonce":344932592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6803","nonce":344932592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6807","nonce":344932592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6805","nonce":344932592}]},"public_addr":"172.21.15.112:6801/344932592","cluster_addr":"172.21.15.112:6803/344932592","heartbeat_back_addr":"172.21.15.112:6807/344932592","heartbeat_front_addr":"172.21.15.112:6805/344932592","state":["exists","up"]},{"osd":2,"uuid":"1193c1d8-bfff-4a57-8200-5f69bf1ea3af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6825","nonce":407933616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6827","nonce":407933616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6831","nonce":407933616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6829","nonce":407933616}]},"public_addr":"172.21.15.112:6825/407933616","cluster_addr":"172.21.15.112:6827/407933616","heartbeat_back_addr":"172.21.15.112:6831/407933616","heartbeat_front_addr":"172.21.15.112:6829/407933616","state":["exists","up"]},{"osd":3,"uuid":"af04330a-36ad-40cc-a8dd-fa259b221de5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6817","nonce":4120164806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6819","nonce":4120164806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6823","nonce":4120164806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6821","nonce":4120164806}]},"public_addr":"172.21.15.112:6817/4120164806","cluster_addr":"172.21.15.112:6819/4120164806","heartbeat_back_addr":"172.21.15.112:6823/4120164806","heartbeat_front_addr":"172.21.15.112:6821/4120164806","state":["exists","up"]},{"osd":4,"uuid":"5b893111-292c-4c5a-b184-53ad38c7199e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6803","nonce":522077560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6805","nonce":522077560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6809","nonce":522077560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6807","nonce":522077560}]},"public_addr":"172.21.15.187:6803/522077560","cluster_addr":"172.21.15.187:6805/522077560","heartbeat_back_addr":"172.21.15.187:6809/522077560","heartbeat_front_addr":"172.21.15.187:6807/522077560","state":["exists","up"]},{"osd":5,"uuid":"c0b8e011-45fe-4c5e-925e-6be2a7f1868a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6827","nonce":3593144721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6829","nonce":3593144721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6833","nonce":3593144721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6831","nonce":3593144721}]},"public_addr":"172.21.15.187:6827/3593144721","cluster_addr":"172.21.15.187:6829/3593144721","heartbeat_back_addr":"172.21.15.187:6833/3593144721","heartbeat_front_addr":"172.21.15.187:6831/3593144721","state":["exists","up"]},{"osd":6,"uuid":"87d2d650-7489-4593-98a3-a1f433ce8dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6811","nonce":1494387657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6813","nonce":1494387657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6817","nonce":1494387657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6815","nonce":1494387657}]},"public_addr":"172.21.15.187:6811/1494387657","cluster_addr":"172.21.15.187:6813/1494387657","heartbeat_back_addr":"172.21.15.187:6817/1494387657","heartbeat_front_addr":"172.21.15.187:6815/1494387657","state":["exists","up"]},{"osd":7,"uuid":"c6b4f8e2-a1ee-4438-aa37-67ff4b6ef1f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6819","nonce":1624699842}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6821","nonce":1624699842}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6825","nonce":1624699842}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6823","nonce":1624699842}]},"public_addr":"172.21.15.187:6819/1624699842","cluster_addr":"172.21.15.187:6821/1624699842","heartbeat_back_addr":"172.21.15.187:6825/1624699842","heartbeat_front_addr":"172.21.15.187:6823/1624699842","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.873162+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.400506+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.664317+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.159791+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.202630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.113212+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.384786+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.448441+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:6832/1432588455":"2024-09-22T00:28:18.650326+0000","172.21.15.112:6834/1432588455":"2024-09-22T00:28:18.650326+0000","172.21.15.112:6835/3610025869":"2024-09-22T00:28:18.650344+0000","172.21.15.112:6833/3610025869":"2024-09-22T00:28:18.650344+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T00:28:23.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.199+0000 7f543690b640 1 -- 172.21.15.112:0/1121356577 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f541c078b90 msgr2=0x7f541c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:23.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.199+0000 7f543690b640 1 --2- 172.21.15.112:0/1121356577 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f541c078b90 0x7f541c07b0a0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f543006c310 tx=0x7f5420006ba0 comp rx=0 tx=0).stop 2024-09-21T00:28:23.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.199+0000 7f543690b640 1 -- 172.21.15.112:0/1121356577 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f54300c7bf0 msgr2=0x7f5430156050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:23.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.199+0000 7f543690b640 1 --2- 172.21.15.112:0/1121356577 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f54300c7bf0 0x7f5430156050 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f54300ce190 tx=0x7f5428010f40 comp rx=0 tx=0).stop 2024-09-21T00:28:23.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.200+0000 7f543690b640 1 -- 172.21.15.112:0/1121356577 shutdown_connections 2024-09-21T00:28:23.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.200+0000 7f543690b640 1 --2- 172.21.15.112:0/1121356577 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f541c078b90 0x7f541c07b0a0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:23.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.200+0000 7f543690b640 1 --2- 172.21.15.112:0/1121356577 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f54300c7bf0 0x7f5430156050 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:23.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.200+0000 7f543690b640 1 --2- 172.21.15.112:0/1121356577 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f54300b70f0 0x7f54300c76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:23.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.200+0000 7f543690b640 1 --2- 172.21.15.112:0/1121356577 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5418004800 0x7f5430153d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:23.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.200+0000 7f543690b640 1 -- 172.21.15.112:0/1121356577 >> 172.21.15.112:0/1121356577 conn(0x7f54300b42e0 msgr2=0x7f54300b2f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:23.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.200+0000 7f543690b640 1 -- 172.21.15.112:0/1121356577 shutdown_connections 2024-09-21T00:28:23.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.200+0000 7f543690b640 1 -- 172.21.15.112:0/1121356577 wait complete. 2024-09-21T00:28:23.212 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T00:25:37.105006+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-21T00:28:23.212 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-21T00:28:23.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.375+0000 7f1399e8b640 1 -- 172.21.15.112:0/3382673198 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13940ccce0 msgr2=0x7f13940c65e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:23.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.375+0000 7f1399e8b640 1 --2- 172.21.15.112:0/3382673198 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13940ccce0 0x7f13940c65e0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f138c005fd0 tx=0x7f138c02a940 comp rx=0 tx=0).stop 2024-09-21T00:28:23.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.375+0000 7f1399e8b640 1 -- 172.21.15.112:0/3382673198 shutdown_connections 2024-09-21T00:28:23.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.375+0000 7f1399e8b640 1 --2- 172.21.15.112:0/3382673198 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13940ccce0 0x7f13940c65e0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:23.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.375+0000 7f1399e8b640 1 -- 172.21.15.112:0/3382673198 >> 172.21.15.112:0/3382673198 conn(0x7f13940cf220 msgr2=0x7f13940cf630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:23.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.375+0000 7f1399e8b640 1 -- 172.21.15.112:0/3382673198 shutdown_connections 2024-09-21T00:28:23.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.375+0000 7f1399e8b640 1 -- 172.21.15.112:0/3382673198 wait complete. 2024-09-21T00:28:23.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.377+0000 7f1399e8b640 1 Processor -- start 2024-09-21T00:28:23.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.377+0000 7f1399e8b640 1 -- start start 2024-09-21T00:28:23.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.378+0000 7f1399e8b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13940ccce0 0x7f139415f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:23.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.378+0000 7f1399e8b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f139415ca50 0x7f13940b6840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:23.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.378+0000 7f1399e8b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f139415f590 0x7f13940b6d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:23.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.378+0000 7f1399e8b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f13940d2130 con 0x7f13940ccce0 2024-09-21T00:28:23.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.378+0000 7f1399e8b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f13940d1fb0 con 0x7f139415f590 2024-09-21T00:28:23.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.378+0000 7f1399e8b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f13940c6b20 con 0x7f139415ca50 2024-09-21T00:28:23.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.378+0000 7f13937fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13940ccce0 0x7f139415f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:23.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.379+0000 7f13937fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13940ccce0 0x7f139415f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42584/0 (socket says 172.21.15.112:42584) 2024-09-21T00:28:23.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.379+0000 7f13937fe640 1 -- 172.21.15.112:0/2885433952 learned_addr learned my addr 172.21.15.112:0/2885433952 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:28:23.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.379+0000 7f1393fff640 1 --2- 172.21.15.112:0/2885433952 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f139415f590 0x7f13940b6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:23.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.379+0000 7f1392ffd640 1 --2- 172.21.15.112:0/2885433952 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f139415ca50 0x7f13940b6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:23.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.379+0000 7f13937fe640 1 -- 172.21.15.112:0/2885433952 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f139415ca50 msgr2=0x7f13940b6840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:23.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.379+0000 7f13937fe640 1 --2- 172.21.15.112:0/2885433952 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f139415ca50 0x7f13940b6840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:23.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.379+0000 7f13937fe640 1 -- 172.21.15.112:0/2885433952 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f139415f590 msgr2=0x7f13940b6d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:23.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.379+0000 7f13937fe640 1 --2- 172.21.15.112:0/2885433952 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f139415f590 0x7f13940b6d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:23.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.379+0000 7f13937fe640 1 -- 172.21.15.112:0/2885433952 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13940b74a0 con 0x7f13940ccce0 2024-09-21T00:28:23.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.379+0000 7f13937fe640 1 --2- 172.21.15.112:0/2885433952 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13940ccce0 0x7f139415f050 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f138400c370 tx=0x7f138400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:23.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.379+0000 7f137bfff640 1 -- 172.21.15.112:0/2885433952 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1384017070 con 0x7f13940ccce0 2024-09-21T00:28:23.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.379+0000 7f137bfff640 1 -- 172.21.15.112:0/2885433952 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f13840035f0 con 0x7f13940ccce0 2024-09-21T00:28:23.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.379+0000 7f1399e8b640 1 -- 172.21.15.112:0/2885433952 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13940b7780 con 0x7f13940ccce0 2024-09-21T00:28:23.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.379+0000 7f137bfff640 1 -- 172.21.15.112:0/2885433952 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1384003a40 con 0x7f13940ccce0 2024-09-21T00:28:23.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.379+0000 7f1399e8b640 1 -- 172.21.15.112:0/2885433952 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f139415e560 con 0x7f13940ccce0 2024-09-21T00:28:23.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.380+0000 7f1399e8b640 1 -- 172.21.15.112:0/2885433952 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f139415e7c0 con 0x7f13940ccce0 2024-09-21T00:28:23.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.381+0000 7f137bfff640 1 -- 172.21.15.112:0/2885433952 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f138401a070 con 0x7f13940ccce0 2024-09-21T00:28:23.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.381+0000 7f137bfff640 1 --2- 172.21.15.112:0/2885433952 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1380078b90 0x7f138007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:23.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.382+0000 7f137bfff640 1 -- 172.21.15.112:0/2885433952 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5421+0+0 (secure 0 0 0) 0x7f1384099e30 con 0x7f13940ccce0 2024-09-21T00:28:23.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.382+0000 7f1392ffd640 1 --2- 172.21.15.112:0/2885433952 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1380078b90 0x7f138007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:23.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.382+0000 7f1392ffd640 1 --2- 172.21.15.112:0/2885433952 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1380078b90 0x7f138007b0a0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f13941e6e30 tx=0x7f1388009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:23.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.386+0000 7f137bfff640 1 -- 172.21.15.112:0/2885433952 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1384062420 con 0x7f13940ccce0 2024-09-21T00:28:23.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.563+0000 7f1399e8b640 1 -- 172.21.15.112:0/2885433952 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f13940c5c50 con 0x7f13940ccce0 2024-09-21T00:28:23.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.563+0000 7f137bfff640 1 -- 172.21.15.112:0/2885433952 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v63) ==== 93+0+10 (secure 0 0 0) 0x7f1384061d20 con 0x7f13940ccce0 2024-09-21T00:28:23.564 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2024-09-21T00:28:23.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.565+0000 7f1399e8b640 1 -- 172.21.15.112:0/2885433952 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1380078b90 msgr2=0x7f138007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:23.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.565+0000 7f1399e8b640 1 --2- 172.21.15.112:0/2885433952 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1380078b90 0x7f138007b0a0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f13941e6e30 tx=0x7f1388009040 comp rx=0 tx=0).stop 2024-09-21T00:28:23.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.565+0000 7f1399e8b640 1 -- 172.21.15.112:0/2885433952 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13940ccce0 msgr2=0x7f139415f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:23.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.565+0000 7f1399e8b640 1 --2- 172.21.15.112:0/2885433952 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13940ccce0 0x7f139415f050 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f138400c370 tx=0x7f138400c830 comp rx=0 tx=0).stop 2024-09-21T00:28:23.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.565+0000 7f1399e8b640 1 -- 172.21.15.112:0/2885433952 shutdown_connections 2024-09-21T00:28:23.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.565+0000 7f1399e8b640 1 --2- 172.21.15.112:0/2885433952 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f139415f590 0x7f13940b6d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:23.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.565+0000 7f1399e8b640 1 --2- 172.21.15.112:0/2885433952 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f139415ca50 0x7f13940b6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:23.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.565+0000 7f1399e8b640 1 --2- 172.21.15.112:0/2885433952 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1380078b90 0x7f138007b0a0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:23.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.565+0000 7f1399e8b640 1 --2- 172.21.15.112:0/2885433952 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13940ccce0 0x7f139415f050 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:23.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.565+0000 7f1399e8b640 1 -- 172.21.15.112:0/2885433952 >> 172.21.15.112:0/2885433952 conn(0x7f13940cf220 msgr2=0x7f13940d0890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:23.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.565+0000 7f1399e8b640 1 -- 172.21.15.112:0/2885433952 shutdown_connections 2024-09-21T00:28:23.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.565+0000 7f1399e8b640 1 -- 172.21.15.112:0/2885433952 wait complete. 2024-09-21T00:28:23.578 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_sessionmap.TestSessionMap.test_session_reject' 2024-09-21T00:28:23.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.696+0000 7fe436d32640 1 -- 172.21.15.112:0/1972962739 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe430070440 msgr2=0x7fe430155dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:23.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.696+0000 7fe436d32640 1 --2- 172.21.15.112:0/1972962739 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe430070440 0x7fe430155dd0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe42c009830 tx=0x7fe42c02a510 comp rx=0 tx=0).stop 2024-09-21T00:28:23.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.696+0000 7fe436d32640 1 -- 172.21.15.112:0/1972962739 shutdown_connections 2024-09-21T00:28:23.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.696+0000 7fe436d32640 1 --2- 172.21.15.112:0/1972962739 >> v2:172.21.15.187:3300/0 conn(0x7fe43015bec0 0x7fe43015c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:23.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.696+0000 7fe436d32640 1 --2- 172.21.15.112:0/1972962739 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe430070440 0x7fe430155dd0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:23.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.696+0000 7fe436d32640 1 -- 172.21.15.112:0/1972962739 >> 172.21.15.112:0/1972962739 conn(0x7fe4300cd5c0 msgr2=0x7fe4300cd9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:23.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.696+0000 7fe436d32640 1 -- 172.21.15.112:0/1972962739 shutdown_connections 2024-09-21T00:28:23.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.696+0000 7fe436d32640 1 -- 172.21.15.112:0/1972962739 wait complete. 2024-09-21T00:28:23.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.697+0000 7fe436d32640 1 Processor -- start 2024-09-21T00:28:23.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.698+0000 7fe436d32640 1 -- start start 2024-09-21T00:28:23.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.699+0000 7fe436d32640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe430070440 0x7fe43013c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:23.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.699+0000 7fe436d32640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4300cea40 0x7fe43015e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:23.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.699+0000 7fe436d32640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe43015bec0 0x7fe43015d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:23.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.699+0000 7fe436d32640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe4300cf6e0 con 0x7fe4300cea40 2024-09-21T00:28:23.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.699+0000 7fe436d32640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe4300cf560 con 0x7fe43015bec0 2024-09-21T00:28:23.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.699+0000 7fe436d32640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe43015c810 con 0x7fe430070440 2024-09-21T00:28:23.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.699+0000 7fe427fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4300cea40 0x7fe43015e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:23.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.699+0000 7fe427fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4300cea40 0x7fe43015e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42600/0 (socket says 172.21.15.112:42600) 2024-09-21T00:28:23.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.699+0000 7fe427fff640 1 -- 172.21.15.112:0/1223377646 learned_addr learned my addr 172.21.15.112:0/1223377646 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:28:23.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.699+0000 7fe434ace640 1 --2- 172.21.15.112:0/1223377646 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe430070440 0x7fe43013c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:23.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.699+0000 7fe4352cf640 1 --2- 172.21.15.112:0/1223377646 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe43015bec0 0x7fe43015d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:23.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.699+0000 7fe427fff640 1 -- 172.21.15.112:0/1223377646 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe430070440 msgr2=0x7fe43013c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:23.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.699+0000 7fe427fff640 1 --2- 172.21.15.112:0/1223377646 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe430070440 0x7fe43013c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:23.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.699+0000 7fe427fff640 1 -- 172.21.15.112:0/1223377646 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe43015bec0 msgr2=0x7fe43015d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:23.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.699+0000 7fe427fff640 1 --2- 172.21.15.112:0/1223377646 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe43015bec0 0x7fe43015d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:23.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.699+0000 7fe427fff640 1 -- 172.21.15.112:0/1223377646 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe43013cc80 con 0x7fe4300cea40 2024-09-21T00:28:23.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.700+0000 7fe427fff640 1 --2- 172.21.15.112:0/1223377646 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4300cea40 0x7fe43015e720 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fe42000c3d0 tx=0x7fe42000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:23.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.700+0000 7fe4257fa640 1 -- 172.21.15.112:0/1223377646 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe420017070 con 0x7fe4300cea40 2024-09-21T00:28:23.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.700+0000 7fe4257fa640 1 -- 172.21.15.112:0/1223377646 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe4200053c0 con 0x7fe4300cea40 2024-09-21T00:28:23.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.700+0000 7fe436d32640 1 -- 172.21.15.112:0/1223377646 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe43013cf60 con 0x7fe4300cea40 2024-09-21T00:28:23.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.700+0000 7fe4257fa640 1 -- 172.21.15.112:0/1223377646 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe420003850 con 0x7fe4300cea40 2024-09-21T00:28:23.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.700+0000 7fe436d32640 1 -- 172.21.15.112:0/1223377646 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe43015cf50 con 0x7fe4300cea40 2024-09-21T00:28:23.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.701+0000 7fe436d32640 1 -- 172.21.15.112:0/1223377646 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe430166bc0 con 0x7fe4300cea40 2024-09-21T00:28:23.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.701+0000 7fe4257fa640 1 -- 172.21.15.112:0/1223377646 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe42001a070 con 0x7fe4300cea40 2024-09-21T00:28:23.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.702+0000 7fe4257fa640 1 --2- 172.21.15.112:0/1223377646 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fe41c078b90 0x7fe41c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:23.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.702+0000 7fe4257fa640 1 -- 172.21.15.112:0/1223377646 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5421+0+0 (secure 0 0 0) 0x7fe42009a060 con 0x7fe4300cea40 2024-09-21T00:28:23.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.702+0000 7fe434ace640 1 --2- 172.21.15.112:0/1223377646 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fe41c078b90 0x7fe41c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:23.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.703+0000 7fe434ace640 1 --2- 172.21.15.112:0/1223377646 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fe41c078b90 0x7fe41c07b0a0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fe42c034000 tx=0x7fe42c004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:23.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.706+0000 7fe4257fa640 1 -- 172.21.15.112:0/1223377646 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe420062540 con 0x7fe4300cea40 2024-09-21T00:28:23.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:23.952+0000 7fe436d32640 1 -- 172.21.15.112:0/1223377646 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_sessionmap.TestSessionMap.test_session_reject"]} v 0) -- 0x7fe430160110 con 0x7fe4300cea40 2024-09-21T00:28:24.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:24.883+0000 7fe4257fa640 1 -- 172.21.15.112:0/1223377646 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_sessionmap.TestSessionMap.test_session_reject"]}]=0 v163) ==== 145+0+0 (secure 0 0 0) 0x7fe420061e40 con 0x7fe4300cea40 2024-09-21T00:28:24.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:24.886+0000 7fe436d32640 1 -- 172.21.15.112:0/1223377646 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fe41c078b90 msgr2=0x7fe41c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:24.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:24.886+0000 7fe436d32640 1 --2- 172.21.15.112:0/1223377646 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fe41c078b90 0x7fe41c07b0a0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fe42c034000 tx=0x7fe42c004730 comp rx=0 tx=0).stop 2024-09-21T00:28:24.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:24.886+0000 7fe436d32640 1 -- 172.21.15.112:0/1223377646 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4300cea40 msgr2=0x7fe43015e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:24.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:24.886+0000 7fe436d32640 1 --2- 172.21.15.112:0/1223377646 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4300cea40 0x7fe43015e720 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fe42000c3d0 tx=0x7fe42000c890 comp rx=0 tx=0).stop 2024-09-21T00:28:24.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:24.886+0000 7fe436d32640 1 -- 172.21.15.112:0/1223377646 shutdown_connections 2024-09-21T00:28:24.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:24.886+0000 7fe436d32640 1 --2- 172.21.15.112:0/1223377646 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe43015bec0 0x7fe43015d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:24.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:24.886+0000 7fe436d32640 1 --2- 172.21.15.112:0/1223377646 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fe41c078b90 0x7fe41c07b0a0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:24.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:24.886+0000 7fe436d32640 1 --2- 172.21.15.112:0/1223377646 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4300cea40 0x7fe43015e720 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:24.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:24.886+0000 7fe436d32640 1 --2- 172.21.15.112:0/1223377646 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe430070440 0x7fe43013c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:24.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:24.886+0000 7fe436d32640 1 -- 172.21.15.112:0/1223377646 >> 172.21.15.112:0/1223377646 conn(0x7fe4300cd5c0 msgr2=0x7fe4300b1ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:24.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:24.886+0000 7fe436d32640 1 -- 172.21.15.112:0/1223377646 shutdown_connections 2024-09-21T00:28:24.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:24.886+0000 7fe436d32640 1 -- 172.21.15.112:0/1223377646 wait complete. 2024-09-21T00:28:24.901 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-21T00:28:25.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.022+0000 7faf454ce640 1 -- 172.21.15.112:0/3814210413 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf400ceb40 msgr2=0x7faf400d0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:25.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.022+0000 7faf454ce640 1 --2- 172.21.15.112:0/3814210413 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf400ceb40 0x7faf400d0f90 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7faf34009830 tx=0x7faf3402a480 comp rx=0 tx=0).stop 2024-09-21T00:28:25.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.022+0000 7faf454ce640 1 -- 172.21.15.112:0/3814210413 shutdown_connections 2024-09-21T00:28:25.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.022+0000 7faf454ce640 1 --2- 172.21.15.112:0/3814210413 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faf400d14d0 0x7faf4015fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:25.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.022+0000 7faf454ce640 1 --2- 172.21.15.112:0/3814210413 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf400ceb40 0x7faf400d0f90 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:25.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.022+0000 7faf454ce640 1 -- 172.21.15.112:0/3814210413 >> 172.21.15.112:0/3814210413 conn(0x7faf400b9820 msgr2=0x7faf4009d830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:25.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.022+0000 7faf454ce640 1 -- 172.21.15.112:0/3814210413 shutdown_connections 2024-09-21T00:28:25.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.022+0000 7faf454ce640 1 -- 172.21.15.112:0/3814210413 wait complete. 2024-09-21T00:28:25.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.024+0000 7faf454ce640 1 Processor -- start 2024-09-21T00:28:25.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.024+0000 7faf454ce640 1 -- start start 2024-09-21T00:28:25.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.025+0000 7faf454ce640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faf400ceb40 0x7faf401539e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:25.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.025+0000 7faf454ce640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faf400d14d0 0x7faf4006e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:25.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.025+0000 7faf454ce640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf400d3360 0x7faf4006edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:25.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.025+0000 7faf454ce640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7faf401600c0 con 0x7faf400d3360 2024-09-21T00:28:25.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.025+0000 7faf454ce640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7faf4015ff40 con 0x7faf400ceb40 2024-09-21T00:28:25.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.025+0000 7faf454ce640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7faf40160240 con 0x7faf400d14d0 2024-09-21T00:28:25.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.025+0000 7faf3f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf400d3360 0x7faf4006edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:25.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.025+0000 7faf3f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf400d3360 0x7faf4006edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42612/0 (socket says 172.21.15.112:42612) 2024-09-21T00:28:25.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.025+0000 7faf3f7fe640 1 -- 172.21.15.112:0/1504174696 learned_addr learned my addr 172.21.15.112:0/1504174696 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:28:25.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.025+0000 7faf3e7fc640 1 --2- 172.21.15.112:0/1504174696 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faf400d14d0 0x7faf4006e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:25.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.025+0000 7faf3effd640 1 --2- 172.21.15.112:0/1504174696 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faf400ceb40 0x7faf401539e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:25.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.026+0000 7faf3f7fe640 1 -- 172.21.15.112:0/1504174696 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faf400d14d0 msgr2=0x7faf4006e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:25.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.026+0000 7faf3f7fe640 1 --2- 172.21.15.112:0/1504174696 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faf400d14d0 0x7faf4006e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:25.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.026+0000 7faf3f7fe640 1 -- 172.21.15.112:0/1504174696 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faf400ceb40 msgr2=0x7faf401539e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:25.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.026+0000 7faf3f7fe640 1 --2- 172.21.15.112:0/1504174696 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faf400ceb40 0x7faf401539e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:25.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.026+0000 7faf3f7fe640 1 -- 172.21.15.112:0/1504174696 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf400cae50 con 0x7faf400d3360 2024-09-21T00:28:25.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.026+0000 7faf3f7fe640 1 --2- 172.21.15.112:0/1504174696 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf400d3360 0x7faf4006edd0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7faf3800e990 tx=0x7faf3800ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:25.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.026+0000 7faf277fe640 1 -- 172.21.15.112:0/1504174696 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf38004d20 con 0x7faf400d3360 2024-09-21T00:28:25.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.026+0000 7faf277fe640 1 -- 172.21.15.112:0/1504174696 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faf3800caa0 con 0x7faf400d3360 2024-09-21T00:28:25.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.026+0000 7faf454ce640 1 -- 172.21.15.112:0/1504174696 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf4006d890 con 0x7faf400d3360 2024-09-21T00:28:25.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.026+0000 7faf277fe640 1 -- 172.21.15.112:0/1504174696 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf38005020 con 0x7faf400d3360 2024-09-21T00:28:25.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.026+0000 7faf454ce640 1 -- 172.21.15.112:0/1504174696 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf4006de50 con 0x7faf400d3360 2024-09-21T00:28:25.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.027+0000 7faf454ce640 1 -- 172.21.15.112:0/1504174696 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf400caa20 con 0x7faf400d3360 2024-09-21T00:28:25.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.028+0000 7faf277fe640 1 -- 172.21.15.112:0/1504174696 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7faf380051c0 con 0x7faf400d3360 2024-09-21T00:28:25.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.028+0000 7faf277fe640 1 --2- 172.21.15.112:0/1504174696 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7faf2c078b90 0x7faf2c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:25.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.028+0000 7faf277fe640 1 -- 172.21.15.112:0/1504174696 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5421+0+0 (secure 0 0 0) 0x7faf3809b420 con 0x7faf400d3360 2024-09-21T00:28:25.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.028+0000 7faf3effd640 1 --2- 172.21.15.112:0/1504174696 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7faf2c078b90 0x7faf2c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:25.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.029+0000 7faf3effd640 1 --2- 172.21.15.112:0/1504174696 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7faf2c078b90 0x7faf2c07b0a0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7faf30007d60 tx=0x7faf30007cf0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:25.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.033+0000 7faf277fe640 1 -- 172.21.15.112:0/1504174696 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7faf38016030 con 0x7faf400d3360 2024-09-21T00:28:25.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.209+0000 7faf454ce640 1 -- 172.21.15.112:0/1504174696 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7faf4008cc70 con 0x7faf400d3360 2024-09-21T00:28:25.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.210+0000 7faf277fe640 1 -- 172.21.15.112:0/1504174696 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v39) ==== 59+0+2294 (secure 0 0 0) 0x7faf4008cc70 con 0x7faf400d3360 2024-09-21T00:28:25.212 INFO:teuthology.orchestra.run.smithi112.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-21T00:28:25.212 INFO:teuthology.orchestra.run.smithi112.stdout:mon advanced mon_op_complaint_time 120 2024-09-21T00:28:25.212 INFO:teuthology.orchestra.run.smithi112.stdout:mgr advanced client_mount_timeout 30 2024-09-21T00:28:25.212 INFO:teuthology.orchestra.run.smithi112.stdout:mgr advanced debug_client 20/20 2024-09-21T00:28:25.212 INFO:teuthology.orchestra.run.smithi112.stdout:mgr advanced debug_mgr 20/20 2024-09-21T00:28:25.212 INFO:teuthology.orchestra.run.smithi112.stdout:mgr advanced debug_ms 1/1 2024-09-21T00:28:25.212 INFO:teuthology.orchestra.run.smithi112.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-21T00:28:25.212 INFO:teuthology.orchestra.run.smithi112.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-21T00:28:25.212 INFO:teuthology.orchestra.run.smithi112.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 5090.401743 2024-09-21T00:28:25.212 INFO:teuthology.orchestra.run.smithi112.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 7876.743127 2024-09-21T00:28:25.213 INFO:teuthology.orchestra.run.smithi112.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 4865.073913 2024-09-21T00:28:25.213 INFO:teuthology.orchestra.run.smithi112.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 5117.206240 2024-09-21T00:28:25.213 INFO:teuthology.orchestra.run.smithi112.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 4329.674604 2024-09-21T00:28:25.213 INFO:teuthology.orchestra.run.smithi112.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4322.652151 2024-09-21T00:28:25.213 INFO:teuthology.orchestra.run.smithi112.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 4244.674337 2024-09-21T00:28:25.213 INFO:teuthology.orchestra.run.smithi112.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 4472.316624 2024-09-21T00:28:25.213 INFO:teuthology.orchestra.run.smithi112.stdout:mds advanced debug_mds 20/20 2024-09-21T00:28:25.213 INFO:teuthology.orchestra.run.smithi112.stdout:mds advanced debug_mds_balancer 20/20 2024-09-21T00:28:25.213 INFO:teuthology.orchestra.run.smithi112.stdout:mds advanced debug_ms 2/2 2024-09-21T00:28:25.213 INFO:teuthology.orchestra.run.smithi112.stdout:mds dev mds_debug_frag true 2024-09-21T00:28:25.213 INFO:teuthology.orchestra.run.smithi112.stdout:mds dev mds_debug_scatterstat true 2024-09-21T00:28:25.213 INFO:teuthology.orchestra.run.smithi112.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-21T00:28:25.213 INFO:teuthology.orchestra.run.smithi112.stdout:mds dev mds_verify_scatter true 2024-09-21T00:28:25.213 INFO:teuthology.orchestra.run.smithi112.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-21T00:28:25.213 INFO:teuthology.orchestra.run.smithi112.stdout:mds advanced rados_mon_op_timeout 900 2024-09-21T00:28:25.214 INFO:teuthology.orchestra.run.smithi112.stdout:mds advanced rados_osd_op_timeout 900 2024-09-21T00:28:25.214 INFO:teuthology.orchestra.run.smithi112.stdout:client advanced client_mount_timeout 600 2024-09-21T00:28:25.214 INFO:teuthology.orchestra.run.smithi112.stdout:client advanced debug_client 20/20 2024-09-21T00:28:25.214 INFO:teuthology.orchestra.run.smithi112.stdout:client advanced debug_ms 1/1 2024-09-21T00:28:25.214 INFO:teuthology.orchestra.run.smithi112.stdout:client advanced rados_mon_op_timeout 900 2024-09-21T00:28:25.214 INFO:teuthology.orchestra.run.smithi112.stdout:client advanced rados_osd_op_timeout 900 2024-09-21T00:28:25.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.212+0000 7faf454ce640 1 -- 172.21.15.112:0/1504174696 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7faf2c078b90 msgr2=0x7faf2c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:25.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.212+0000 7faf454ce640 1 --2- 172.21.15.112:0/1504174696 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7faf2c078b90 0x7faf2c07b0a0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7faf30007d60 tx=0x7faf30007cf0 comp rx=0 tx=0).stop 2024-09-21T00:28:25.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.212+0000 7faf454ce640 1 -- 172.21.15.112:0/1504174696 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf400d3360 msgr2=0x7faf4006edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:25.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.212+0000 7faf454ce640 1 --2- 172.21.15.112:0/1504174696 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf400d3360 0x7faf4006edd0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7faf3800e990 tx=0x7faf3800ee50 comp rx=0 tx=0).stop 2024-09-21T00:28:25.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.212+0000 7faf454ce640 1 -- 172.21.15.112:0/1504174696 shutdown_connections 2024-09-21T00:28:25.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.212+0000 7faf454ce640 1 --2- 172.21.15.112:0/1504174696 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7faf2c078b90 0x7faf2c07b0a0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:25.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.212+0000 7faf454ce640 1 --2- 172.21.15.112:0/1504174696 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf400d3360 0x7faf4006edd0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:25.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.212+0000 7faf454ce640 1 --2- 172.21.15.112:0/1504174696 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faf400d14d0 0x7faf4006e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:25.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.212+0000 7faf454ce640 1 --2- 172.21.15.112:0/1504174696 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faf400ceb40 0x7faf401539e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:25.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.212+0000 7faf454ce640 1 -- 172.21.15.112:0/1504174696 >> 172.21.15.112:0/1504174696 conn(0x7faf400b9820 msgr2=0x7faf400b58e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:25.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.212+0000 7faf454ce640 1 -- 172.21.15.112:0/1504174696 shutdown_connections 2024-09-21T00:28:25.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.212+0000 7faf454ce640 1 -- 172.21.15.112:0/1504174696 wait complete. 2024-09-21T00:28:25.225 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-21T00:28:25.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.339+0000 7f2204fcc640 1 -- 172.21.15.112:0/3163807812 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22000b5ab0 msgr2=0x7f22000b0730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:25.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.339+0000 7f2204fcc640 1 --2- 172.21.15.112:0/3163807812 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22000b5ab0 0x7f22000b0730 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f22000d0c20 tx=0x7f21f002aa30 comp rx=0 tx=0).stop 2024-09-21T00:28:25.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.339+0000 7f2204fcc640 1 -- 172.21.15.112:0/3163807812 shutdown_connections 2024-09-21T00:28:25.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.339+0000 7f2204fcc640 1 --2- 172.21.15.112:0/3163807812 >> v2:172.21.15.187:3300/0 conn(0x7f2200144dd0 0x7f220015f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:25.342 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.339+0000 7f2204fcc640 1 --2- 172.21.15.112:0/3163807812 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22000b5ab0 0x7f22000b0730 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:25.342 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.339+0000 7f2204fcc640 1 -- 172.21.15.112:0/3163807812 >> 172.21.15.112:0/3163807812 conn(0x7f2200073fe0 msgr2=0x7f22000743f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:25.342 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.339+0000 7f2204fcc640 1 -- 172.21.15.112:0/3163807812 shutdown_connections 2024-09-21T00:28:25.342 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.339+0000 7f2204fcc640 1 -- 172.21.15.112:0/3163807812 wait complete. 2024-09-21T00:28:25.343 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.341+0000 7f2204fcc640 1 Processor -- start 2024-09-21T00:28:25.343 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.341+0000 7f2204fcc640 1 -- start start 2024-09-21T00:28:25.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.342+0000 7f2204fcc640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2200144dd0 0x7f22000afef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:25.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.342+0000 7f2204fcc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f220015f620 0x7f2200133150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:25.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.342+0000 7f2204fcc640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f220015afa0 0x7f22001306f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:25.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.342+0000 7f2204fcc640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f220006ee90 con 0x7f220015f620 2024-09-21T00:28:25.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.342+0000 7f2204fcc640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f22000d04d0 con 0x7f2200144dd0 2024-09-21T00:28:25.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.342+0000 7f2204fcc640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f220006f010 con 0x7f220015afa0 2024-09-21T00:28:25.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.342+0000 7f21fdd9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f220015f620 0x7f2200133150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:25.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.342+0000 7f21fdd9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f220015f620 0x7f2200133150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42628/0 (socket says 172.21.15.112:42628) 2024-09-21T00:28:25.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.342+0000 7f21fdd9b640 1 -- 172.21.15.112:0/1750936359 learned_addr learned my addr 172.21.15.112:0/1750936359 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:28:25.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.342+0000 7f21fe59c640 1 --2- 172.21.15.112:0/1750936359 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2200144dd0 0x7f22000afef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:25.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.343+0000 7f21fed9d640 1 --2- 172.21.15.112:0/1750936359 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f220015afa0 0x7f22001306f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:25.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.343+0000 7f21fdd9b640 1 -- 172.21.15.112:0/1750936359 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f220015afa0 msgr2=0x7f22001306f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:25.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.343+0000 7f21fdd9b640 1 --2- 172.21.15.112:0/1750936359 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f220015afa0 0x7f22001306f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:25.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.343+0000 7f21fdd9b640 1 -- 172.21.15.112:0/1750936359 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2200144dd0 msgr2=0x7f22000afef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:25.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.343+0000 7f21fdd9b640 1 --2- 172.21.15.112:0/1750936359 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2200144dd0 0x7f22000afef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:25.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.343+0000 7f21fdd9b640 1 -- 172.21.15.112:0/1750936359 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22000ae950 con 0x7f220015f620 2024-09-21T00:28:25.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.343+0000 7f21fdd9b640 1 --2- 172.21.15.112:0/1750936359 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f220015f620 0x7f2200133150 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f21f400d720 tx=0x7f21f400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:25.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.343+0000 7f21eeffd640 1 -- 172.21.15.112:0/1750936359 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21f4017070 con 0x7f220015f620 2024-09-21T00:28:25.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.343+0000 7f21eeffd640 1 -- 172.21.15.112:0/1750936359 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f21f4009b10 con 0x7f220015f620 2024-09-21T00:28:25.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.343+0000 7f2204fcc640 1 -- 172.21.15.112:0/1750936359 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22000aee50 con 0x7f220015f620 2024-09-21T00:28:25.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.343+0000 7f21eeffd640 1 -- 172.21.15.112:0/1750936359 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21f4010460 con 0x7f220015f620 2024-09-21T00:28:25.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.343+0000 7f2204fcc640 1 -- 172.21.15.112:0/1750936359 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22000af410 con 0x7f220015f620 2024-09-21T00:28:25.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.344+0000 7f2204fcc640 1 -- 172.21.15.112:0/1750936359 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22000aebc0 con 0x7f220015f620 2024-09-21T00:28:25.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.345+0000 7f21eeffd640 1 -- 172.21.15.112:0/1750936359 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f21f401a070 con 0x7f220015f620 2024-09-21T00:28:25.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.345+0000 7f21eeffd640 1 --2- 172.21.15.112:0/1750936359 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f21e0078b90 0x7f21e007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:25.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.346+0000 7f21eeffd640 1 -- 172.21.15.112:0/1750936359 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5421+0+0 (secure 0 0 0) 0x7f21f409ad10 con 0x7f220015f620 2024-09-21T00:28:25.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.346+0000 7f21fe59c640 1 --2- 172.21.15.112:0/1750936359 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f21e0078b90 0x7f21e007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:25.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.346+0000 7f21fe59c640 1 --2- 172.21.15.112:0/1750936359 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f21e0078b90 0x7f21e007b0a0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f21f0035000 tx=0x7f21f003b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:25.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.350+0000 7f21eeffd640 1 -- 172.21.15.112:0/1750936359 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f21f4062c20 con 0x7f220015f620 2024-09-21T00:28:25.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.526+0000 7f2204fcc640 1 -- 172.21.15.112:0/1750936359 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f22000706d0 con 0x7f220015f620 2024-09-21T00:28:25.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.530+0000 7f21eeffd640 1 -- 172.21.15.112:0/1750936359 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v40)=0 v40) ==== 122+0+0 (secure 0 0 0) 0x7f21f4067650 con 0x7f220015f620 2024-09-21T00:28:25.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.532+0000 7f2204fcc640 1 -- 172.21.15.112:0/1750936359 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f21e0078b90 msgr2=0x7f21e007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:25.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.532+0000 7f2204fcc640 1 --2- 172.21.15.112:0/1750936359 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f21e0078b90 0x7f21e007b0a0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f21f0035000 tx=0x7f21f003b040 comp rx=0 tx=0).stop 2024-09-21T00:28:25.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.532+0000 7f2204fcc640 1 -- 172.21.15.112:0/1750936359 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f220015f620 msgr2=0x7f2200133150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:25.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.532+0000 7f2204fcc640 1 --2- 172.21.15.112:0/1750936359 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f220015f620 0x7f2200133150 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f21f400d720 tx=0x7f21f400dbe0 comp rx=0 tx=0).stop 2024-09-21T00:28:25.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.532+0000 7f2204fcc640 1 -- 172.21.15.112:0/1750936359 shutdown_connections 2024-09-21T00:28:25.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.532+0000 7f2204fcc640 1 --2- 172.21.15.112:0/1750936359 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f220015afa0 0x7f22001306f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:25.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.532+0000 7f2204fcc640 1 --2- 172.21.15.112:0/1750936359 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f21e0078b90 0x7f21e007b0a0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:25.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.532+0000 7f2204fcc640 1 --2- 172.21.15.112:0/1750936359 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f220015f620 0x7f2200133150 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:25.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.532+0000 7f2204fcc640 1 --2- 172.21.15.112:0/1750936359 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2200144dd0 0x7f22000afef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:25.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.532+0000 7f2204fcc640 1 -- 172.21.15.112:0/1750936359 >> 172.21.15.112:0/1750936359 conn(0x7f2200073fe0 msgr2=0x7f220006e780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:25.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.532+0000 7f2204fcc640 1 -- 172.21.15.112:0/1750936359 shutdown_connections 2024-09-21T00:28:25.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.532+0000 7f2204fcc640 1 -- 172.21.15.112:0/1750936359 wait complete. 2024-09-21T00:28:25.545 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T00:28:25.545 DEBUG:teuthology.orchestra.run.smithi112:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T00:28:25.585 DEBUG:teuthology.orchestra.run.smithi187:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T00:28:25.634 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T00:28:25.634 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T00:28:25.634 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:28:25.650 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056010,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1439 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248716k,nr_inodes=812179,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T00:28:25.650 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-09-21T00:28:25.650 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:28:25.668 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056013,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497440k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16835 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248720k,nr_inodes=812180,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497440k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T00:28:25.668 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T00:28:25.668 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:28:25.706 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056010,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1439 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248716k,nr_inodes=812179,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T00:28:25.706 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-09-21T00:28:25.706 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:28:25.728 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056013,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497440k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16835 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248720k,nr_inodes=812180,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497440k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T00:28:25.729 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-21T00:28:25.729 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T00:28:25.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.856+0000 7f47f594a640 1 -- 172.21.15.112:0/4124790333 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47f015c3c0 msgr2=0x7f47f00b3260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:25.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.856+0000 7f47f594a640 1 --2- 172.21.15.112:0/4124790333 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47f015c3c0 0x7f47f00b3260 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f47f00cb610 tx=0x7f47e002a4b0 comp rx=0 tx=0).stop 2024-09-21T00:28:25.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.856+0000 7f47f594a640 1 -- 172.21.15.112:0/4124790333 shutdown_connections 2024-09-21T00:28:25.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.856+0000 7f47f594a640 1 --2- 172.21.15.112:0/4124790333 >> v2:172.21.15.187:3300/0 conn(0x7f47f00b3bb0 0x7f47f00b6570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:25.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.856+0000 7f47f594a640 1 --2- 172.21.15.112:0/4124790333 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f47f00b37a0 0x7f47f00b6030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:25.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.856+0000 7f47f594a640 1 --2- 172.21.15.112:0/4124790333 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47f015c3c0 0x7f47f00b3260 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:25.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.857+0000 7f47f594a640 1 -- 172.21.15.112:0/4124790333 >> 172.21.15.112:0/4124790333 conn(0x7f47f008dcb0 msgr2=0x7f47f009d8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:25.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.857+0000 7f47f594a640 1 -- 172.21.15.112:0/4124790333 shutdown_connections 2024-09-21T00:28:25.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.857+0000 7f47f594a640 1 -- 172.21.15.112:0/4124790333 wait complete. 2024-09-21T00:28:25.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.858+0000 7f47f594a640 1 Processor -- start 2024-09-21T00:28:25.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.858+0000 7f47f594a640 1 -- start start 2024-09-21T00:28:25.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.860+0000 7f47f594a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f47f00b37a0 0x7f47f00a1db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:25.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.860+0000 7f47f594a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47f00b3bb0 0x7f47f01587b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:25.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.860+0000 7f47f594a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f47f015c3c0 0x7f47f0160ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:25.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.860+0000 7f47f594a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f47f00b6ab0 con 0x7f47f00b3bb0 2024-09-21T00:28:25.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.860+0000 7f47f594a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f47f0163390 con 0x7f47f015c3c0 2024-09-21T00:28:25.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.860+0000 7f47f594a640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f47f00b6c30 con 0x7f47f00b37a0 2024-09-21T00:28:25.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.860+0000 7f47ee7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47f00b3bb0 0x7f47f01587b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:25.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.860+0000 7f47ee7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47f00b3bb0 0x7f47f01587b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42646/0 (socket says 172.21.15.112:42646) 2024-09-21T00:28:25.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.860+0000 7f47ee7fc640 1 -- 172.21.15.112:0/2666669716 learned_addr learned my addr 172.21.15.112:0/2666669716 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:28:25.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.860+0000 7f47eeffd640 1 --2- 172.21.15.112:0/2666669716 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f47f00b37a0 0x7f47f00a1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:25.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.860+0000 7f47ee7fc640 1 -- 172.21.15.112:0/2666669716 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f47f00b37a0 msgr2=0x7f47f00a1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:25.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.860+0000 7f47ee7fc640 1 --2- 172.21.15.112:0/2666669716 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f47f00b37a0 0x7f47f00a1db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:25.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.860+0000 7f47ee7fc640 1 -- 172.21.15.112:0/2666669716 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f47f015c3c0 msgr2=0x7f47f0160ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:25.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.860+0000 7f47ee7fc640 1 --2- 172.21.15.112:0/2666669716 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f47f015c3c0 0x7f47f0160ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:25.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.860+0000 7f47ee7fc640 1 -- 172.21.15.112:0/2666669716 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47f006fa50 con 0x7f47f00b3bb0 2024-09-21T00:28:25.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.860+0000 7f47ee7fc640 1 --2- 172.21.15.112:0/2666669716 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47f00b3bb0 0x7f47f01587b0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f47e400d690 tx=0x7f47e400db50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:25.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.860+0000 7f47d7fff640 1 -- 172.21.15.112:0/2666669716 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e4017070 con 0x7f47f00b3bb0 2024-09-21T00:28:25.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.860+0000 7f47d7fff640 1 -- 172.21.15.112:0/2666669716 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f47e4005310 con 0x7f47f00b3bb0 2024-09-21T00:28:25.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.860+0000 7f47d7fff640 1 -- 172.21.15.112:0/2666669716 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e4009820 con 0x7f47f00b3bb0 2024-09-21T00:28:25.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.860+0000 7f47f594a640 1 -- 172.21.15.112:0/2666669716 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47f006d030 con 0x7f47f00b3bb0 2024-09-21T00:28:25.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.861+0000 7f47f594a640 1 -- 172.21.15.112:0/2666669716 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47f00a22f0 con 0x7f47f00b3bb0 2024-09-21T00:28:25.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.861+0000 7f47f594a640 1 -- 172.21.15.112:0/2666669716 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47f015b120 con 0x7f47f00b3bb0 2024-09-21T00:28:25.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.862+0000 7f47d7fff640 1 -- 172.21.15.112:0/2666669716 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f47e40099c0 con 0x7f47f00b3bb0 2024-09-21T00:28:25.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.863+0000 7f47d7fff640 1 --2- 172.21.15.112:0/2666669716 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f47dc078bc0 0x7f47dc07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:25.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.863+0000 7f47d7fff640 1 -- 172.21.15.112:0/2666669716 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5421+0+0 (secure 0 0 0) 0x7f47e409ac80 con 0x7f47f00b3bb0 2024-09-21T00:28:25.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.863+0000 7f47eeffd640 1 --2- 172.21.15.112:0/2666669716 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f47dc078bc0 0x7f47dc07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:25.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.864+0000 7f47eeffd640 1 --2- 172.21.15.112:0/2666669716 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f47dc078bc0 0x7f47dc07b0d0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f47e0034000 tx=0x7f47e0004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:25.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:25.867+0000 7f47d7fff640 1 -- 172.21.15.112:0/2666669716 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f47e40a0020 con 0x7f47f00b3bb0 2024-09-21T00:28:26.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:26.083+0000 7f47f594a640 1 -- 172.21.15.112:0/2666669716 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f47f006f7e0 con 0x7f47f00b3bb0 2024-09-21T00:28:26.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:26.083+0000 7f47d7fff640 1 -- 172.21.15.112:0/2666669716 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+3920 (secure 0 0 0) 0x7f47e40631d0 con 0x7f47f00b3bb0 2024-09-21T00:28:26.085 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:28:26.085 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":41,"btime":"2024-09-21T00:28:19:652236+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5389,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6836/3984762342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":3984762342},{"type":"v1","addr":"172.21.15.187:6836","nonce":3984762342}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5392,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6837/763698806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6835","nonce":763698806},{"type":"v1","addr":"172.21.15.187:6837","nonce":763698806}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":6075,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/2390783164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":2390783164},{"type":"v1","addr":"172.21.15.112:6833","nonce":2390783164}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6078,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/3099229453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3099229453},{"type":"v1","addr":"172.21.15.112:6835","nonce":3099229453}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41}],"filesystems":[]} 2024-09-21T00:28:26.085 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 41 2024-09-21T00:28:26.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:26.086+0000 7f47f594a640 1 -- 172.21.15.112:0/2666669716 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f47dc078bc0 msgr2=0x7f47dc07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:26.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:26.086+0000 7f47f594a640 1 --2- 172.21.15.112:0/2666669716 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f47dc078bc0 0x7f47dc07b0d0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f47e0034000 tx=0x7f47e0004730 comp rx=0 tx=0).stop 2024-09-21T00:28:26.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:26.086+0000 7f47f594a640 1 -- 172.21.15.112:0/2666669716 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47f00b3bb0 msgr2=0x7f47f01587b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:26.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:26.086+0000 7f47f594a640 1 --2- 172.21.15.112:0/2666669716 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47f00b3bb0 0x7f47f01587b0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f47e400d690 tx=0x7f47e400db50 comp rx=0 tx=0).stop 2024-09-21T00:28:26.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:26.086+0000 7f47f594a640 1 -- 172.21.15.112:0/2666669716 shutdown_connections 2024-09-21T00:28:26.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:26.086+0000 7f47f594a640 1 --2- 172.21.15.112:0/2666669716 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f47f015c3c0 0x7f47f0160ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:26.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:26.086+0000 7f47f594a640 1 --2- 172.21.15.112:0/2666669716 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f47dc078bc0 0x7f47dc07b0d0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:26.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:26.086+0000 7f47f594a640 1 --2- 172.21.15.112:0/2666669716 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47f00b3bb0 0x7f47f01587b0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:26.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:26.086+0000 7f47f594a640 1 --2- 172.21.15.112:0/2666669716 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f47f00b37a0 0x7f47f00a1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:26.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:26.086+0000 7f47f594a640 1 -- 172.21.15.112:0/2666669716 >> 172.21.15.112:0/2666669716 conn(0x7f47f008dcb0 msgr2=0x7f47f00b6020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:26.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:26.086+0000 7f47f594a640 1 -- 172.21.15.112:0/2666669716 shutdown_connections 2024-09-21T00:28:26.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:26.086+0000 7f47f594a640 1 -- 172.21.15.112:0/2666669716 wait complete. 2024-09-21T00:28:26.099 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-21T00:28:26.099 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-21T00:28:26.099 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-21T00:28:26.099 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T00:28:26.099 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-21T00:28:26.099 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-21T00:28:26.100 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-21T00:28:26.100 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T00:28:26.100 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-21T00:28:26.100 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-21T00:28:26.100 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-21T00:28:26.100 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T00:28:26.100 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-21T00:28:26.100 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-21T00:28:26.101 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-21T00:28:26.101 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T00:28:26.111 INFO:tasks.ceph.mds.a:Stopped 2024-09-21T00:28:26.111 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-21T00:28:26.112 INFO:tasks.ceph.mds.c:Stopped 2024-09-21T00:28:26.112 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-21T00:28:26.113 INFO:tasks.ceph.mds.d:Stopped 2024-09-21T00:28:26.114 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-21T00:28:26.114 INFO:tasks.ceph.mds.b:Stopped 2024-09-21T00:28:26.114 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-21T00:28:26.154 INFO:tasks.ceph.mds.a:Started 2024-09-21T00:28:26.154 DEBUG:teuthology.parallel:result is None 2024-09-21T00:28:26.155 INFO:tasks.ceph.mds.c:Started 2024-09-21T00:28:26.155 DEBUG:teuthology.parallel:result is None 2024-09-21T00:28:26.157 INFO:tasks.ceph.mds.d:Started 2024-09-21T00:28:26.157 DEBUG:teuthology.parallel:result is None 2024-09-21T00:28:26.158 INFO:tasks.ceph.mds.b:Started 2024-09-21T00:28:26.158 DEBUG:teuthology.parallel:result is None 2024-09-21T00:28:26.158 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-21T00:28:26.241 INFO:tasks.ceph.mds.c.smithi112.stdout:starting mds.c at 2024-09-21T00:28:26.241 INFO:tasks.ceph.mds.a.smithi112.stdout:starting mds.a at 2024-09-21T00:28:26.251 INFO:tasks.ceph.mds.d.smithi187.stdout:starting mds.d at 2024-09-21T00:28:26.251 INFO:tasks.ceph.mds.b.smithi187.stdout:starting mds.b at 2024-09-21T00:28:26.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:26.276+0000 7fe2ba29b640 1 -- 172.21.15.112:0/3812844025 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2b415c5f0 msgr2=0x7fe2b4155900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:26.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:26.276+0000 7fe2ba29b640 1 --2- 172.21.15.112:0/3812844025 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2b415c5f0 0x7fe2b4155900 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fe2a4009830 tx=0x7fe2a402a4a0 comp rx=0 tx=0).stop 2024-09-21T00:28:26.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:26.276+0000 7fe2ba29b640 1 -- 172.21.15.112:0/3812844025 shutdown_connections 2024-09-21T00:28:26.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:26.276+0000 7fe2ba29b640 1 --2- 172.21.15.112:0/3812844025 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2b415c5f0 0x7fe2b4155900 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:26.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:26.276+0000 7fe2ba29b640 1 -- 172.21.15.112:0/3812844025 >> 172.21.15.112:0/3812844025 conn(0x7fe2b4073fe0 msgr2=0x7fe2b40743f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:26.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:26.276+0000 7fe2ba29b640 1 -- 172.21.15.112:0/3812844025 shutdown_connections 2024-09-21T00:28:26.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:26.276+0000 7fe2ba29b640 1 -- 172.21.15.112:0/3812844025 wait complete. 2024-09-21T00:28:26.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:26.278+0000 7fe2ba29b640 1 Processor -- start 2024-09-21T00:28:26.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:26.278+0000 7fe2ba29b640 1 -- start start 2024-09-21T00:28:26.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:26.279+0000 7fe2ba29b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2b4070210 0x7fe2b40b5660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:26.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:26.279+0000 7fe2ba29b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe2b40ccbb0 0x7fe2b4156eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:26.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:26.279+0000 7fe2ba29b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2b415c5f0 0x7fe2b41573f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:26.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:26.279+0000 7fe2ba29b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b415ca00 con 0x7fe2b4070210 2024-09-21T00:28:26.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:26.280+0000 7fe2ba29b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b4070fd0 con 0x7fe2b40ccbb0 2024-09-21T00:28:26.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:26.280+0000 7fe2b3fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2b4070210 0x7fe2b40b5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:26.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:26.280+0000 7fe2b3fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2b4070210 0x7fe2b40b5660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42914/0 (socket says 172.21.15.112:42914) 2024-09-21T00:28:26.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:26.280+0000 7fe2b3fff640 1 -- 172.21.15.112:0/2028705797 learned_addr learned my addr 172.21.15.112:0/2028705797 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:28:26.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:26.280+0000 7fe2ba29b640 1 -- 172.21.15.112:0/2028705797 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe2b4157d60 con 0x7fe2b415c5f0 2024-09-21T00:28:26.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:26.280+0000 7fe2b37fe640 1 --2- 172.21.15.112:0/2028705797 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe2b40ccbb0 0x7fe2b4156eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:26.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:26.280+0000 7fe2b8838640 1 --2- 172.21.15.112:0/2028705797 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2b415c5f0 0x7fe2b41573f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:26.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:26.280+0000 7fe2b3fff640 1 -- 172.21.15.112:0/2028705797 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2b415c5f0 msgr2=0x7fe2b41573f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:26.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:26.280+0000 7fe2b3fff640 1 --2- 172.21.15.112:0/2028705797 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2b415c5f0 0x7fe2b41573f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:26.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:26.280+0000 7fe2b3fff640 1 -- 172.21.15.112:0/2028705797 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe2b40ccbb0 msgr2=0x7fe2b4156eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:26.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:26.280+0000 7fe2b3fff640 1 --2- 172.21.15.112:0/2028705797 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe2b40ccbb0 0x7fe2b4156eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:26.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:26.280+0000 7fe2b3fff640 1 -- 172.21.15.112:0/2028705797 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2b4141930 con 0x7fe2b4070210 2024-09-21T00:28:26.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:26.280+0000 7fe2b3fff640 1 --2- 172.21.15.112:0/2028705797 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2b4070210 0x7fe2b40b5660 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fe2a4000c00 tx=0x7fe2a4033d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:26.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:26.280+0000 7fe2b0ff9640 1 -- 172.21.15.112:0/2028705797 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a4031d60 con 0x7fe2b4070210 2024-09-21T00:28:26.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:26.280+0000 7fe2b0ff9640 1 -- 172.21.15.112:0/2028705797 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe2a4040030 con 0x7fe2b4070210 2024-09-21T00:28:26.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:26.280+0000 7fe2ba29b640 1 -- 172.21.15.112:0/2028705797 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2b4141c10 con 0x7fe2b4070210 2024-09-21T00:28:26.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:26.280+0000 7fe2b0ff9640 1 -- 172.21.15.112:0/2028705797 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a4033330 con 0x7fe2b4070210 2024-09-21T00:28:26.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:26.280+0000 7fe2ba29b640 1 -- 172.21.15.112:0/2028705797 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2b41421d0 con 0x7fe2b4070210 2024-09-21T00:28:26.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:26.281+0000 7fe2ba29b640 1 -- 172.21.15.112:0/2028705797 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2b41433d0 con 0x7fe2b4070210 2024-09-21T00:28:26.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:26.283+0000 7fe2b0ff9640 1 -- 172.21.15.112:0/2028705797 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe2a40334d0 con 0x7fe2b4070210 2024-09-21T00:28:26.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:26.283+0000 7fe2b0ff9640 1 --2- 172.21.15.112:0/2028705797 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fe2a0078b90 0x7fe2a007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:26.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:26.283+0000 7fe2b0ff9640 1 -- 172.21.15.112:0/2028705797 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5421+0+0 (secure 0 0 0) 0x7fe2a40bdf30 con 0x7fe2b4070210 2024-09-21T00:28:26.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:26.283+0000 7fe2b37fe640 1 --2- 172.21.15.112:0/2028705797 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fe2a0078b90 0x7fe2a007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:26.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:26.284+0000 7fe2b37fe640 1 --2- 172.21.15.112:0/2028705797 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fe2a0078b90 0x7fe2a007b0a0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fe2b4156e40 tx=0x7fe2a8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:26.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:26.286+0000 7fe2b0ff9640 1 -- 172.21.15.112:0/2028705797 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe2a4085fb0 con 0x7fe2b4070210 2024-09-21T00:28:26.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:26.475+0000 7fe2ba29b640 1 -- 172.21.15.112:0/2028705797 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fe2b40d0430 con 0x7fe2b4070210 2024-09-21T00:28:26.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:26.900+0000 7fe2b0ff9640 1 -- 172.21.15.112:0/2028705797 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v65) ==== 97+0+0 (secure 0 0 0) 0x7fe2a408a9e0 con 0x7fe2b4070210 2024-09-21T00:28:26.902 INFO:teuthology.orchestra.run.smithi112.stderr: removed all blocklist entries 2024-09-21T00:28:26.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:26.902+0000 7fe2ba29b640 1 -- 172.21.15.112:0/2028705797 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fe2a0078b90 msgr2=0x7fe2a007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:26.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:26.902+0000 7fe2ba29b640 1 --2- 172.21.15.112:0/2028705797 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fe2a0078b90 0x7fe2a007b0a0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fe2b4156e40 tx=0x7fe2a8009040 comp rx=0 tx=0).stop 2024-09-21T00:28:26.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:26.902+0000 7fe2ba29b640 1 -- 172.21.15.112:0/2028705797 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2b4070210 msgr2=0x7fe2b40b5660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:26.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:26.902+0000 7fe2ba29b640 1 --2- 172.21.15.112:0/2028705797 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2b4070210 0x7fe2b40b5660 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fe2a4000c00 tx=0x7fe2a4033d60 comp rx=0 tx=0).stop 2024-09-21T00:28:26.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:26.902+0000 7fe2ba29b640 1 -- 172.21.15.112:0/2028705797 shutdown_connections 2024-09-21T00:28:26.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:26.902+0000 7fe2ba29b640 1 --2- 172.21.15.112:0/2028705797 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2b415c5f0 0x7fe2b41573f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:26.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:26.902+0000 7fe2ba29b640 1 --2- 172.21.15.112:0/2028705797 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe2b40ccbb0 0x7fe2b4156eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:26.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:26.902+0000 7fe2ba29b640 1 --2- 172.21.15.112:0/2028705797 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fe2a0078b90 0x7fe2a007b0a0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:26.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:26.902+0000 7fe2ba29b640 1 --2- 172.21.15.112:0/2028705797 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2b4070210 0x7fe2b40b5660 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:26.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:26.902+0000 7fe2ba29b640 1 -- 172.21.15.112:0/2028705797 >> 172.21.15.112:0/2028705797 conn(0x7fe2b4073fe0 msgr2=0x7fe2b40743f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:26.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:26.902+0000 7fe2ba29b640 1 -- 172.21.15.112:0/2028705797 shutdown_connections 2024-09-21T00:28:26.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:26.902+0000 7fe2ba29b640 1 -- 172.21.15.112:0/2028705797 wait complete. 2024-09-21T00:28:26.917 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-21T00:28:27.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.027+0000 7f03259a1640 1 -- 172.21.15.112:0/3218854256 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f030c003b30 msgr2=0x7f030c003fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:27.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.027+0000 7f03259a1640 1 --2- 172.21.15.112:0/3218854256 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f030c003b30 0x7f030c003fa0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f0310009b50 tx=0x7f030c005830 comp rx=0 tx=0).stop 2024-09-21T00:28:27.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.027+0000 7f03259a1640 1 -- 172.21.15.112:0/3218854256 shutdown_connections 2024-09-21T00:28:27.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.027+0000 7f03259a1640 1 --2- 172.21.15.112:0/3218854256 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f030c003b30 0x7f030c003fa0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:27.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.027+0000 7f03259a1640 1 -- 172.21.15.112:0/3218854256 >> 172.21.15.112:0/3218854256 conn(0x7f032008bfe0 msgr2=0x7f032009d890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:27.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.028+0000 7f03259a1640 1 -- 172.21.15.112:0/3218854256 shutdown_connections 2024-09-21T00:28:27.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.028+0000 7f03259a1640 1 -- 172.21.15.112:0/3218854256 wait complete. 2024-09-21T00:28:27.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.029+0000 7f03259a1640 1 Processor -- start 2024-09-21T00:28:27.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.029+0000 7f03259a1640 1 -- start start 2024-09-21T00:28:27.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.031+0000 7f03259a1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f030c003b30 0x7f032006fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:27.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.031+0000 7f03259a1640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f032006cde0 0x7f03201603f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:27.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.031+0000 7f03259a1640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0320160930 0x7f03201f8e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:27.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.031+0000 7f03259a1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f03201633e0 con 0x7f030c003b30 2024-09-21T00:28:27.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.031+0000 7f03259a1640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0320163260 con 0x7f0320160930 2024-09-21T00:28:27.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.031+0000 7f03259a1640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f0320163560 con 0x7f032006cde0 2024-09-21T00:28:27.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.031+0000 7f031effd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f030c003b30 0x7f032006fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:27.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.031+0000 7f031effd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f030c003b30 0x7f032006fab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42936/0 (socket says 172.21.15.112:42936) 2024-09-21T00:28:27.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.031+0000 7f031effd640 1 -- 172.21.15.112:0/389366010 learned_addr learned my addr 172.21.15.112:0/389366010 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:28:27.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.031+0000 7f031effd640 1 -- 172.21.15.112:0/389366010 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f032006cde0 msgr2=0x7f03201603f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:27.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.031+0000 7f031effd640 1 --2- 172.21.15.112:0/389366010 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f032006cde0 0x7f03201603f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:27.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.031+0000 7f031effd640 1 -- 172.21.15.112:0/389366010 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0320160930 msgr2=0x7f03201f8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:27.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.031+0000 7f031effd640 1 --2- 172.21.15.112:0/389366010 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0320160930 0x7f03201f8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:27.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.031+0000 7f031effd640 1 -- 172.21.15.112:0/389366010 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f032006ecf0 con 0x7f030c003b30 2024-09-21T00:28:27.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.031+0000 7f031effd640 1 --2- 172.21.15.112:0/389366010 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f030c003b30 0x7f032006fab0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f0310009a70 tx=0x7f03100336f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:27.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.031+0000 7f031cff9640 1 -- 172.21.15.112:0/389366010 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0310038040 con 0x7f030c003b30 2024-09-21T00:28:27.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.031+0000 7f031cff9640 1 -- 172.21.15.112:0/389366010 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0310033950 con 0x7f030c003b30 2024-09-21T00:28:27.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.031+0000 7f03259a1640 1 -- 172.21.15.112:0/389366010 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f032006efd0 con 0x7f030c003b30 2024-09-21T00:28:27.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.031+0000 7f031cff9640 1 -- 172.21.15.112:0/389366010 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0310033da0 con 0x7f030c003b30 2024-09-21T00:28:27.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.032+0000 7f03259a1640 1 -- 172.21.15.112:0/389366010 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f032006f590 con 0x7f030c003b30 2024-09-21T00:28:27.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.032+0000 7f03259a1640 1 -- 172.21.15.112:0/389366010 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03200d0610 con 0x7f030c003b30 2024-09-21T00:28:27.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.033+0000 7f031cff9640 1 -- 172.21.15.112:0/389366010 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0310043020 con 0x7f030c003b30 2024-09-21T00:28:27.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.034+0000 7f031cff9640 1 --2- 172.21.15.112:0/389366010 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f0300078bc0 0x7f030007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:27.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.034+0000 7f031cff9640 1 -- 172.21.15.112:0/389366010 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5249+0+0 (secure 0 0 0) 0x7f03100bac10 con 0x7f030c003b30 2024-09-21T00:28:27.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.034+0000 7f03167fc640 1 --2- 172.21.15.112:0/389366010 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f0300078bc0 0x7f030007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:27.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.035+0000 7f03167fc640 1 --2- 172.21.15.112:0/389366010 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f0300078bc0 0x7f030007b0d0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f0308001920 tx=0x7f03080014f0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:27.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.038+0000 7f031cff9640 1 -- 172.21.15.112:0/389366010 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f031004a020 con 0x7f030c003b30 2024-09-21T00:28:27.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.280+0000 7f03259a1640 1 -- 172.21.15.112:0/389366010 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f03200b6f90 con 0x7f030c003b30 2024-09-21T00:28:27.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.281+0000 7f031cff9640 1 -- 172.21.15.112:0/389366010 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v17) ==== 80+0+5304 (secure 0 0 0) 0x7f03100832a0 con 0x7f030c003b30 2024-09-21T00:28:27.283 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:28:27.283 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-21T00:28:27.283 INFO:teuthology.orchestra.run.smithi112.stdout: "auth_dump": [ 2024-09-21T00:28:27.283 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:28:27.283 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "mds.a", 2024-09-21T00:28:27.283 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBPEu5m3capAhAAczRM4L8ldX985NW4ep3w5Q==", 2024-09-21T00:28:27.283 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:28:27.283 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2024-09-21T00:28:27.283 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow *", 2024-09-21T00:28:27.283 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow *", 2024-09-21T00:28:27.283 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2024-09-21T00:28:27.283 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:28:27.283 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:28:27.284 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:28:27.284 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "mds.b", 2024-09-21T00:28:27.284 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBPEu5mxlDlDxAAtqngbTOhPqXDjFhJpSCLHg==", 2024-09-21T00:28:27.284 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:28:27.284 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2024-09-21T00:28:27.284 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow *", 2024-09-21T00:28:27.284 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow *", 2024-09-21T00:28:27.284 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2024-09-21T00:28:27.284 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:28:27.284 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:28:27.284 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:28:27.284 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "mds.c", 2024-09-21T00:28:27.284 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBPEu5mXNVRChAA1qsPbEybmY16jw5Jwk/v1A==", 2024-09-21T00:28:27.284 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:28:27.285 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2024-09-21T00:28:27.285 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow *", 2024-09-21T00:28:27.285 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow *", 2024-09-21T00:28:27.285 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2024-09-21T00:28:27.285 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:28:27.285 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:28:27.285 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:28:27.285 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "mds.d", 2024-09-21T00:28:27.285 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBPEu5mnzySFxAAX3VfTgGYGcDweI7rPf3xvw==", 2024-09-21T00:28:27.285 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:28:27.285 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2024-09-21T00:28:27.285 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow *", 2024-09-21T00:28:27.285 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow *", 2024-09-21T00:28:27.285 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2024-09-21T00:28:27.285 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:28:27.286 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:28:27.286 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:28:27.286 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.0", 2024-09-21T00:28:27.286 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBSEu5moQg4OBAAi8mzLx7tWnORlrL64/MWlw==", 2024-09-21T00:28:27.286 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:28:27.286 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2024-09-21T00:28:27.286 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2024-09-21T00:28:27.286 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2024-09-21T00:28:27.286 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:28:27.286 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:28:27.286 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:28:27.286 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.1", 2024-09-21T00:28:27.286 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBWEu5mxfobDhAAi7I6LYq9udB0W8bnYaBY9Q==", 2024-09-21T00:28:27.286 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:28:27.287 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2024-09-21T00:28:27.287 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2024-09-21T00:28:27.287 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2024-09-21T00:28:27.287 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:28:27.287 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:28:27.287 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:28:27.287 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.2", 2024-09-21T00:28:27.287 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBZEu5mDqC/EhAAwzTbj22ocR4xJ8DMJvN7Lg==", 2024-09-21T00:28:27.287 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:28:27.287 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2024-09-21T00:28:27.287 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2024-09-21T00:28:27.287 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2024-09-21T00:28:27.287 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:28:27.287 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:28:27.287 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:28:27.288 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.3", 2024-09-21T00:28:27.288 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBcEu5mwweGJBAAT3hZn5pto0aWt6BwGuaqcw==", 2024-09-21T00:28:27.288 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:28:27.288 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2024-09-21T00:28:27.288 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2024-09-21T00:28:27.288 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2024-09-21T00:28:27.288 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:28:27.288 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:28:27.288 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:28:27.288 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.4", 2024-09-21T00:28:27.288 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBjEu5mIPd3ARAAkxTYphn38uKh48UUVEB+Fg==", 2024-09-21T00:28:27.288 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:28:27.288 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2024-09-21T00:28:27.288 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2024-09-21T00:28:27.288 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2024-09-21T00:28:27.289 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:28:27.289 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:28:27.289 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:28:27.289 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.5", 2024-09-21T00:28:27.289 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBmEu5mP9IVBBAAftfK5V1K4vYRyFyJS2UzKQ==", 2024-09-21T00:28:27.289 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:28:27.289 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2024-09-21T00:28:27.289 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2024-09-21T00:28:27.289 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2024-09-21T00:28:27.289 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:28:27.289 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:28:27.289 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:28:27.289 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.6", 2024-09-21T00:28:27.289 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBpEu5mmjiyFRAAbiW2M8JE0jJOaNg1u+1kgw==", 2024-09-21T00:28:27.290 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:28:27.290 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2024-09-21T00:28:27.290 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2024-09-21T00:28:27.290 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2024-09-21T00:28:27.290 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:28:27.290 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:28:27.290 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:28:27.290 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.7", 2024-09-21T00:28:27.290 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBsEu5moklPKBAA6hzVvgvpmZMkPaFFce4mqw==", 2024-09-21T00:28:27.290 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:28:27.290 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2024-09-21T00:28:27.290 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2024-09-21T00:28:27.290 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2024-09-21T00:28:27.290 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:28:27.290 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:28:27.291 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:28:27.291 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "client.0", 2024-09-21T00:28:27.291 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBPEu5m49NPHBAAl/rcjiE8xYI+XdWwvZ6FOw==", 2024-09-21T00:28:27.291 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:28:27.291 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2024-09-21T00:28:27.291 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow r", 2024-09-21T00:28:27.291 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-21T00:28:27.291 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:28:27.291 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:28:27.291 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:28:27.291 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "client.1", 2024-09-21T00:28:27.291 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBPEu5mG33aIxAAirQhTlVyF/pRJx93T/0bIQ==", 2024-09-21T00:28:27.291 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:28:27.291 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2024-09-21T00:28:27.291 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow r", 2024-09-21T00:28:27.292 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow rw", 2024-09-21T00:28:27.292 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow rwx" 2024-09-21T00:28:27.292 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:28:27.292 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:28:27.292 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:28:27.292 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "client.2", 2024-09-21T00:28:27.292 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBPEu5mreUUIBAAELeN1N58srw6GqBrj8UOoQ==", 2024-09-21T00:28:27.292 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:28:27.292 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2024-09-21T00:28:27.292 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow r", 2024-09-21T00:28:27.292 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow rw", 2024-09-21T00:28:27.292 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow rwx" 2024-09-21T00:28:27.292 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:28:27.292 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:28:27.293 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:28:27.293 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "client.3", 2024-09-21T00:28:27.293 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBPEu5mcTmiJxAAY6bmTtow0R0heJHjx0ahDQ==", 2024-09-21T00:28:27.293 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:28:27.293 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2024-09-21T00:28:27.293 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow r", 2024-09-21T00:28:27.293 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow rw", 2024-09-21T00:28:27.293 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow rwx" 2024-09-21T00:28:27.293 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:28:27.293 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:28:27.293 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:28:27.293 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "client.admin", 2024-09-21T00:28:27.293 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBOEu5mTxhNIBAAlStXGsGd/okXhakY6WI/4w==", 2024-09-21T00:28:27.293 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:28:27.294 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow *", 2024-09-21T00:28:27.294 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow *", 2024-09-21T00:28:27.294 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow *", 2024-09-21T00:28:27.294 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2024-09-21T00:28:27.294 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:28:27.294 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:28:27.294 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:28:27.294 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "mgr.x", 2024-09-21T00:28:27.294 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBOEu5mjGpDOhAAJFudb2wZlOBjo8Ss65Q+eg==", 2024-09-21T00:28:27.294 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:28:27.294 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow *", 2024-09-21T00:28:27.294 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile mgr", 2024-09-21T00:28:27.294 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2024-09-21T00:28:27.294 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:28:27.294 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:28:27.295 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:28:27.295 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "mgr.y", 2024-09-21T00:28:27.295 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBOEu5myGjKNRAA9cMDObZYYOXMGdv2mOXkJA==", 2024-09-21T00:28:27.295 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:28:27.295 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow *", 2024-09-21T00:28:27.295 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile mgr", 2024-09-21T00:28:27.295 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2024-09-21T00:28:27.295 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:28:27.295 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:28:27.295 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-09-21T00:28:27.295 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-21T00:28:27.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.283+0000 7f03259a1640 1 -- 172.21.15.112:0/389366010 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f0300078bc0 msgr2=0x7f030007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:27.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.283+0000 7f03259a1640 1 --2- 172.21.15.112:0/389366010 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f0300078bc0 0x7f030007b0d0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f0308001920 tx=0x7f03080014f0 comp rx=0 tx=0).stop 2024-09-21T00:28:27.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.283+0000 7f03259a1640 1 -- 172.21.15.112:0/389366010 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f030c003b30 msgr2=0x7f032006fab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:27.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.283+0000 7f03259a1640 1 --2- 172.21.15.112:0/389366010 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f030c003b30 0x7f032006fab0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f0310009a70 tx=0x7f03100336f0 comp rx=0 tx=0).stop 2024-09-21T00:28:27.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.283+0000 7f03259a1640 1 -- 172.21.15.112:0/389366010 shutdown_connections 2024-09-21T00:28:27.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.283+0000 7f03259a1640 1 --2- 172.21.15.112:0/389366010 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0320160930 0x7f03201f8e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:27.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.283+0000 7f03259a1640 1 --2- 172.21.15.112:0/389366010 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f032006cde0 0x7f03201603f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:27.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.283+0000 7f03259a1640 1 --2- 172.21.15.112:0/389366010 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f0300078bc0 0x7f030007b0d0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:27.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.283+0000 7f03259a1640 1 --2- 172.21.15.112:0/389366010 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f030c003b30 0x7f032006fab0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:27.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.283+0000 7f03259a1640 1 -- 172.21.15.112:0/389366010 >> 172.21.15.112:0/389366010 conn(0x7f032008bfe0 msgr2=0x7f03200b74c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:27.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.283+0000 7f03259a1640 1 -- 172.21.15.112:0/389366010 shutdown_connections 2024-09-21T00:28:27.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.283+0000 7f03259a1640 1 -- 172.21.15.112:0/389366010 wait complete. 2024-09-21T00:28:27.298 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-21T00:28:27.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.408+0000 7fbd64630640 1 -- 172.21.15.112:0/79525785 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbd48005270 msgr2=0x7fbd4800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:27.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.408+0000 7fbd64630640 1 --2- 172.21.15.112:0/79525785 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbd48005270 0x7fbd4800b9c0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fbd5c0713c0 tx=0x7fbd48005830 comp rx=0 tx=0).stop 2024-09-21T00:28:27.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.408+0000 7fbd62bcd640 1 -- 172.21.15.112:0/79525785 reap_dead start 2024-09-21T00:28:27.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.408+0000 7fbd64630640 1 -- 172.21.15.112:0/79525785 shutdown_connections 2024-09-21T00:28:27.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.408+0000 7fbd64630640 1 -- 172.21.15.112:0/79525785 >> 172.21.15.112:0/79525785 conn(0x7fbd5c073fe0 msgr2=0x7fbd5c0743f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:27.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.408+0000 7fbd64630640 1 -- 172.21.15.112:0/79525785 shutdown_connections 2024-09-21T00:28:27.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.408+0000 7fbd64630640 1 -- 172.21.15.112:0/79525785 wait complete. 2024-09-21T00:28:27.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.410+0000 7fbd64630640 1 Processor -- start 2024-09-21T00:28:27.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.410+0000 7fbd64630640 1 -- start start 2024-09-21T00:28:27.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.411+0000 7fbd64630640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbd48005270 0x7fbd5c15af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:27.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.411+0000 7fbd64630640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbd5c1f3340 0x7fbd5c1f3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:27.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.411+0000 7fbd64630640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbd5c0aa2f0 0x7fbd5c0aa700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:27.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.411+0000 7fbd64630640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbd5c15d480 con 0x7fbd5c1f3340 2024-09-21T00:28:27.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.412+0000 7fbd64630640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fbd5c15d300 con 0x7fbd48005270 2024-09-21T00:28:27.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.412+0000 7fbd64630640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fbd5c0d0f50 con 0x7fbd5c0aa2f0 2024-09-21T00:28:27.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.412+0000 7fbd61bcb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbd5c1f3340 0x7fbd5c1f3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:27.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.412+0000 7fbd61bcb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbd5c1f3340 0x7fbd5c1f3ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42950/0 (socket says 172.21.15.112:42950) 2024-09-21T00:28:27.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.412+0000 7fbd61bcb640 1 -- 172.21.15.112:0/1807504179 learned_addr learned my addr 172.21.15.112:0/1807504179 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:28:27.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.412+0000 7fbd623cc640 1 --2- 172.21.15.112:0/1807504179 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbd48005270 0x7fbd5c15af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:27.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.412+0000 7fbd62bcd640 1 --2- 172.21.15.112:0/1807504179 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbd5c0aa2f0 0x7fbd5c0aa700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:27.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.412+0000 7fbd61bcb640 1 -- 172.21.15.112:0/1807504179 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbd5c0aa2f0 msgr2=0x7fbd5c0aa700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:27.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.412+0000 7fbd61bcb640 1 --2- 172.21.15.112:0/1807504179 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbd5c0aa2f0 0x7fbd5c0aa700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:27.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.412+0000 7fbd61bcb640 1 -- 172.21.15.112:0/1807504179 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbd48005270 msgr2=0x7fbd5c15af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:27.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.412+0000 7fbd61bcb640 1 --2- 172.21.15.112:0/1807504179 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbd48005270 0x7fbd5c15af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:27.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.412+0000 7fbd61bcb640 1 -- 172.21.15.112:0/1807504179 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd5c15c870 con 0x7fbd5c1f3340 2024-09-21T00:28:27.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.412+0000 7fbd623cc640 1 --2- 172.21.15.112:0/1807504179 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbd48005270 0x7fbd5c15af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:28:27.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.412+0000 7fbd61bcb640 1 --2- 172.21.15.112:0/1807504179 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbd5c1f3340 0x7fbd5c1f3ca0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c005fd0 tx=0x7fbd4c00eda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:27.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.412+0000 7fbd52ffd640 1 -- 172.21.15.112:0/1807504179 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd4c00cc30 con 0x7fbd5c1f3340 2024-09-21T00:28:27.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.412+0000 7fbd52ffd640 1 -- 172.21.15.112:0/1807504179 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbd4c005020 con 0x7fbd5c1f3340 2024-09-21T00:28:27.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.412+0000 7fbd64630640 1 -- 172.21.15.112:0/1807504179 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd5c15b7d0 con 0x7fbd5c1f3340 2024-09-21T00:28:27.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.412+0000 7fbd52ffd640 1 -- 172.21.15.112:0/1807504179 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd4c005470 con 0x7fbd5c1f3340 2024-09-21T00:28:27.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.412+0000 7fbd64630640 1 -- 172.21.15.112:0/1807504179 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd5c15bd90 con 0x7fbd5c1f3340 2024-09-21T00:28:27.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.413+0000 7fbd64630640 1 -- 172.21.15.112:0/1807504179 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd5c15b540 con 0x7fbd5c1f3340 2024-09-21T00:28:27.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.414+0000 7fbd52ffd640 1 -- 172.21.15.112:0/1807504179 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fbd4c0047a0 con 0x7fbd5c1f3340 2024-09-21T00:28:27.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.415+0000 7fbd52ffd640 1 --2- 172.21.15.112:0/1807504179 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fbd40078b90 0x7fbd4007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:27.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.415+0000 7fbd52ffd640 1 -- 172.21.15.112:0/1807504179 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5249+0+0 (secure 0 0 0) 0x7fbd4c09b410 con 0x7fbd5c1f3340 2024-09-21T00:28:27.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.415+0000 7fbd623cc640 1 --2- 172.21.15.112:0/1807504179 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fbd40078b90 0x7fbd4007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:27.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.415+0000 7fbd623cc640 1 --2- 172.21.15.112:0/1807504179 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fbd40078b90 0x7fbd4007b0a0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fbd58005d00 tx=0x7fbd58009730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:27.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.419+0000 7fbd52ffd640 1 -- 172.21.15.112:0/1807504179 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fbd4c016030 con 0x7fbd5c1f3340 2024-09-21T00:28:27.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.602+0000 7fbd64630640 1 -- 172.21.15.112:0/1807504179 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fbd5c0d07f0 con 0x7fbd5c1f3340 2024-09-21T00:28:27.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.602+0000 7fbd52ffd640 1 -- 172.21.15.112:0/1807504179 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v67) ==== 84+0+65 (secure 0 0 0) 0x7fbd4c063aa0 con 0x7fbd5c1f3340 2024-09-21T00:28:27.604 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:28:27.604 INFO:teuthology.orchestra.run.smithi112.stdout:[ 2024-09-21T00:28:27.605 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:28:27.605 INFO:teuthology.orchestra.run.smithi112.stdout: "poolnum": 1, 2024-09-21T00:28:27.605 INFO:teuthology.orchestra.run.smithi112.stdout: "poolname": ".mgr" 2024-09-21T00:28:27.605 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:28:27.605 INFO:teuthology.orchestra.run.smithi112.stdout:] 2024-09-21T00:28:27.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.605+0000 7fbd64630640 1 -- 172.21.15.112:0/1807504179 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fbd40078b90 msgr2=0x7fbd4007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:27.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.605+0000 7fbd64630640 1 --2- 172.21.15.112:0/1807504179 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fbd40078b90 0x7fbd4007b0a0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fbd58005d00 tx=0x7fbd58009730 comp rx=0 tx=0).stop 2024-09-21T00:28:27.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.605+0000 7fbd64630640 1 -- 172.21.15.112:0/1807504179 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbd5c1f3340 msgr2=0x7fbd5c1f3ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:27.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.605+0000 7fbd64630640 1 --2- 172.21.15.112:0/1807504179 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbd5c1f3340 0x7fbd5c1f3ca0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c005fd0 tx=0x7fbd4c00eda0 comp rx=0 tx=0).stop 2024-09-21T00:28:27.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.605+0000 7fbd64630640 1 -- 172.21.15.112:0/1807504179 shutdown_connections 2024-09-21T00:28:27.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.605+0000 7fbd64630640 1 --2- 172.21.15.112:0/1807504179 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbd5c0aa2f0 0x7fbd5c0aa700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:27.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.605+0000 7fbd64630640 1 --2- 172.21.15.112:0/1807504179 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fbd40078b90 0x7fbd4007b0a0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:27.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.605+0000 7fbd64630640 1 --2- 172.21.15.112:0/1807504179 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbd5c1f3340 0x7fbd5c1f3ca0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:27.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.605+0000 7fbd64630640 1 --2- 172.21.15.112:0/1807504179 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbd48005270 0x7fbd5c15af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:27.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.605+0000 7fbd64630640 1 -- 172.21.15.112:0/1807504179 >> 172.21.15.112:0/1807504179 conn(0x7fbd5c073fe0 msgr2=0x7fbd5c0b1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:27.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.605+0000 7fbd64630640 1 -- 172.21.15.112:0/1807504179 shutdown_connections 2024-09-21T00:28:27.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.605+0000 7fbd64630640 1 -- 172.21.15.112:0/1807504179 wait complete. 2024-09-21T00:28:27.618 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-21T00:28:27.618 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-21T00:28:27.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.734+0000 7fcede9eb640 1 -- 172.21.15.112:0/324394172 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fced815f4d0 msgr2=0x7fced815f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:27.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.734+0000 7fcede9eb640 1 --2- 172.21.15.112:0/324394172 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fced815f4d0 0x7fced815f8e0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fced808d020 tx=0x7fced002a980 comp rx=0 tx=0).stop 2024-09-21T00:28:27.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.734+0000 7fcede9eb640 1 -- 172.21.15.112:0/324394172 shutdown_connections 2024-09-21T00:28:27.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.734+0000 7fcede9eb640 1 --2- 172.21.15.112:0/324394172 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fced815f4d0 0x7fced815f8e0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:27.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.734+0000 7fcede9eb640 1 --2- 172.21.15.112:0/324394172 >> v2:172.21.15.187:3300/0 conn(0x7fced80d4d20 0x7fced80d02d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:27.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.734+0000 7fcede9eb640 1 -- 172.21.15.112:0/324394172 >> 172.21.15.112:0/324394172 conn(0x7fced809d990 msgr2=0x7fced808bfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:27.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.734+0000 7fcede9eb640 1 -- 172.21.15.112:0/324394172 shutdown_connections 2024-09-21T00:28:27.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.734+0000 7fcede9eb640 1 -- 172.21.15.112:0/324394172 wait complete. 2024-09-21T00:28:27.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.736+0000 7fcede9eb640 1 Processor -- start 2024-09-21T00:28:27.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.736+0000 7fcede9eb640 1 -- start start 2024-09-21T00:28:27.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.737+0000 7fcede9eb640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fced80cfec0 0x7fced81f53a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:27.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.737+0000 7fcede9eb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fced815f4d0 0x7fced80cb070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:27.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.737+0000 7fcede9eb640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fced80b5530 0x7fced815beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:27.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.737+0000 7fcede9eb640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fced815ead0 con 0x7fced815f4d0 2024-09-21T00:28:27.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.737+0000 7fcede9eb640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fced815e950 con 0x7fced80cfec0 2024-09-21T00:28:27.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.737+0000 7fcede9eb640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fced815ec50 con 0x7fced80b5530 2024-09-21T00:28:27.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.737+0000 7fced77fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fced815f4d0 0x7fced80cb070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:27.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.737+0000 7fced77fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fced815f4d0 0x7fced80cb070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42960/0 (socket says 172.21.15.112:42960) 2024-09-21T00:28:27.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.738+0000 7fced77fe640 1 -- 172.21.15.112:0/2831712946 learned_addr learned my addr 172.21.15.112:0/2831712946 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:28:27.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.738+0000 7fced7fff640 1 --2- 172.21.15.112:0/2831712946 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fced80cfec0 0x7fced81f53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:27.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.738+0000 7fcedcf88640 1 --2- 172.21.15.112:0/2831712946 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fced80b5530 0x7fced815beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:27.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.738+0000 7fced77fe640 1 -- 172.21.15.112:0/2831712946 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fced80b5530 msgr2=0x7fced815beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:27.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.738+0000 7fced77fe640 1 --2- 172.21.15.112:0/2831712946 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fced80b5530 0x7fced815beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:27.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.738+0000 7fced77fe640 1 -- 172.21.15.112:0/2831712946 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fced80cfec0 msgr2=0x7fced81f53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:27.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.738+0000 7fced77fe640 1 --2- 172.21.15.112:0/2831712946 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fced80cfec0 0x7fced81f53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:27.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.738+0000 7fced77fe640 1 -- 172.21.15.112:0/2831712946 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fced81f5b60 con 0x7fced815f4d0 2024-09-21T00:28:27.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.738+0000 7fcedcf88640 1 --2- 172.21.15.112:0/2831712946 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fced80b5530 0x7fced815beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:28:27.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.738+0000 7fced7fff640 1 --2- 172.21.15.112:0/2831712946 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fced80cfec0 0x7fced81f53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:28:27.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.738+0000 7fced77fe640 1 --2- 172.21.15.112:0/2831712946 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fced815f4d0 0x7fced80cb070 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fcecc00d700 tx=0x7fcecc00dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:27.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.738+0000 7fced4ff9640 1 -- 172.21.15.112:0/2831712946 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcecc017070 con 0x7fced815f4d0 2024-09-21T00:28:27.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.738+0000 7fced4ff9640 1 -- 172.21.15.112:0/2831712946 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcecc005310 con 0x7fced815f4d0 2024-09-21T00:28:27.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.738+0000 7fced4ff9640 1 -- 172.21.15.112:0/2831712946 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcecc009820 con 0x7fced815f4d0 2024-09-21T00:28:27.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.738+0000 7fcede9eb640 1 -- 172.21.15.112:0/2831712946 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fced81f5e40 con 0x7fced815f4d0 2024-09-21T00:28:27.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.738+0000 7fcede9eb640 1 -- 172.21.15.112:0/2831712946 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fced815c3f0 con 0x7fced815f4d0 2024-09-21T00:28:27.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.739+0000 7fcede9eb640 1 -- 172.21.15.112:0/2831712946 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fced80b3d90 con 0x7fced815f4d0 2024-09-21T00:28:27.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.740+0000 7fced4ff9640 1 -- 172.21.15.112:0/2831712946 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fcecc0099c0 con 0x7fced815f4d0 2024-09-21T00:28:27.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.740+0000 7fced4ff9640 1 --2- 172.21.15.112:0/2831712946 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fcec4078b90 0x7fcec407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:27.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.741+0000 7fced4ff9640 1 -- 172.21.15.112:0/2831712946 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5249+0+0 (secure 0 0 0) 0x7fcecc09acc0 con 0x7fced815f4d0 2024-09-21T00:28:27.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.741+0000 7fced7fff640 1 --2- 172.21.15.112:0/2831712946 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fcec4078b90 0x7fcec407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:27.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.741+0000 7fced7fff640 1 --2- 172.21.15.112:0/2831712946 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fcec4078b90 0x7fcec407b0a0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fcec800a830 tx=0x7fcec8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:27.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.745+0000 7fced4ff9640 1 -- 172.21.15.112:0/2831712946 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fcecc0632c0 con 0x7fced815f4d0 2024-09-21T00:28:27.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:27.925+0000 7fcede9eb640 1 -- 172.21.15.112:0/2831712946 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fced81f58f0 con 0x7fced815f4d0 2024-09-21T00:28:28.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:28.916+0000 7fced4ff9640 1 -- 172.21.15.112:0/2831712946 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v68) ==== 138+0+0 (secure 0 0 0) 0x7fcecc062bc0 con 0x7fced815f4d0 2024-09-21T00:28:28.918 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'cephfs_metadata' created 2024-09-21T00:28:28.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:28.919+0000 7fcede9eb640 1 -- 172.21.15.112:0/2831712946 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fcec4078b90 msgr2=0x7fcec407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:28.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:28.919+0000 7fcede9eb640 1 --2- 172.21.15.112:0/2831712946 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fcec4078b90 0x7fcec407b0a0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fcec800a830 tx=0x7fcec8008040 comp rx=0 tx=0).stop 2024-09-21T00:28:28.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:28.919+0000 7fcede9eb640 1 -- 172.21.15.112:0/2831712946 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fced815f4d0 msgr2=0x7fced80cb070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:28.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:28.919+0000 7fcede9eb640 1 --2- 172.21.15.112:0/2831712946 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fced815f4d0 0x7fced80cb070 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fcecc00d700 tx=0x7fcecc00dbc0 comp rx=0 tx=0).stop 2024-09-21T00:28:28.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:28.919+0000 7fcede9eb640 1 -- 172.21.15.112:0/2831712946 shutdown_connections 2024-09-21T00:28:28.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:28.919+0000 7fcede9eb640 1 --2- 172.21.15.112:0/2831712946 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fced80b5530 0x7fced815beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:28.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:28.919+0000 7fcede9eb640 1 --2- 172.21.15.112:0/2831712946 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fcec4078b90 0x7fcec407b0a0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:28.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:28.919+0000 7fcede9eb640 1 --2- 172.21.15.112:0/2831712946 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fced815f4d0 0x7fced80cb070 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:28.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:28.919+0000 7fcede9eb640 1 --2- 172.21.15.112:0/2831712946 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fced80cfec0 0x7fced81f53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:28.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:28.919+0000 7fcede9eb640 1 -- 172.21.15.112:0/2831712946 >> 172.21.15.112:0/2831712946 conn(0x7fced809d990 msgr2=0x7fced80b12f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:28.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:28.920+0000 7fcede9eb640 1 -- 172.21.15.112:0/2831712946 shutdown_connections 2024-09-21T00:28:28.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:28.920+0000 7fcede9eb640 1 -- 172.21.15.112:0/2831712946 wait complete. 2024-09-21T00:28:28.939 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-21T00:28:29.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:29.062+0000 7f1e51ec4640 1 -- 172.21.15.112:0/3350595367 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e34003ae0 msgr2=0x7f1e34003f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:29.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:29.062+0000 7f1e51ec4640 1 --2- 172.21.15.112:0/3350595367 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e34003ae0 0x7f1e34003f30 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c00c310 tx=0x7f1e34005830 comp rx=0 tx=0).stop 2024-09-21T00:28:29.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:29.062+0000 7f1e51ec4640 1 -- 172.21.15.112:0/3350595367 shutdown_connections 2024-09-21T00:28:29.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:29.062+0000 7f1e51ec4640 1 --2- 172.21.15.112:0/3350595367 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e34003ae0 0x7f1e34003f30 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:29.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:29.062+0000 7f1e51ec4640 1 -- 172.21.15.112:0/3350595367 >> 172.21.15.112:0/3350595367 conn(0x7f1e4c0b42e0 msgr2=0x7f1e4c0b4720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:29.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:29.062+0000 7f1e51ec4640 1 -- 172.21.15.112:0/3350595367 shutdown_connections 2024-09-21T00:28:29.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:29.062+0000 7f1e51ec4640 1 -- 172.21.15.112:0/3350595367 wait complete. 2024-09-21T00:28:29.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:29.064+0000 7f1e51ec4640 1 Processor -- start 2024-09-21T00:28:29.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:29.064+0000 7f1e51ec4640 1 -- start start 2024-09-21T00:28:29.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:29.065+0000 7f1e51ec4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e34003ae0 0x7f1e4c0b6900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:29.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:29.065+0000 7f1e51ec4640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1e4c155640 0x7f1e4c161d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:29.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:29.065+0000 7f1e51ec4640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1e4c162280 0x7f1e4c156570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:29.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:29.065+0000 7f1e51ec4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1e4c0cd390 con 0x7f1e34003ae0 2024-09-21T00:28:29.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:29.065+0000 7f1e51ec4640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f1e4c0cd210 con 0x7f1e4c162280 2024-09-21T00:28:29.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:29.065+0000 7f1e51ec4640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f1e4c0cd510 con 0x7f1e4c155640 2024-09-21T00:28:29.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:29.066+0000 7f1e4b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e34003ae0 0x7f1e4c0b6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:29.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:29.066+0000 7f1e4b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e34003ae0 0x7f1e4c0b6900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42972/0 (socket says 172.21.15.112:42972) 2024-09-21T00:28:29.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:29.066+0000 7f1e4b7fe640 1 -- 172.21.15.112:0/3827627662 learned_addr learned my addr 172.21.15.112:0/3827627662 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:28:29.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:29.066+0000 7f1e4affd640 1 --2- 172.21.15.112:0/3827627662 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1e4c155640 0x7f1e4c161d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:29.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:29.066+0000 7f1e4bfff640 1 --2- 172.21.15.112:0/3827627662 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1e4c162280 0x7f1e4c156570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:29.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:29.066+0000 7f1e4b7fe640 1 -- 172.21.15.112:0/3827627662 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1e4c155640 msgr2=0x7f1e4c161d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:29.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:29.066+0000 7f1e4b7fe640 1 --2- 172.21.15.112:0/3827627662 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1e4c155640 0x7f1e4c161d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:29.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:29.066+0000 7f1e4b7fe640 1 -- 172.21.15.112:0/3827627662 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1e4c162280 msgr2=0x7f1e4c156570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:29.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:29.066+0000 7f1e4b7fe640 1 --2- 172.21.15.112:0/3827627662 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1e4c162280 0x7f1e4c156570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:29.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:29.066+0000 7f1e4b7fe640 1 -- 172.21.15.112:0/3827627662 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e4c1553f0 con 0x7f1e34003ae0 2024-09-21T00:28:29.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:29.066+0000 7f1e4b7fe640 1 --2- 172.21.15.112:0/3827627662 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e34003ae0 0x7f1e4c0b6900 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c00c370 tx=0x7f1e3c001dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:29.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:29.066+0000 7f1e33fff640 1 -- 172.21.15.112:0/3827627662 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e3c0035f0 con 0x7f1e34003ae0 2024-09-21T00:28:29.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:29.066+0000 7f1e33fff640 1 -- 172.21.15.112:0/3827627662 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1e3c003880 con 0x7f1e34003ae0 2024-09-21T00:28:29.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:29.066+0000 7f1e51ec4640 1 -- 172.21.15.112:0/3827627662 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e4c06f8b0 con 0x7f1e34003ae0 2024-09-21T00:28:29.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:29.066+0000 7f1e33fff640 1 -- 172.21.15.112:0/3827627662 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e3c003cd0 con 0x7f1e34003ae0 2024-09-21T00:28:29.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:29.066+0000 7f1e51ec4640 1 -- 172.21.15.112:0/3827627662 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e4c0b6100 con 0x7f1e34003ae0 2024-09-21T00:28:29.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:29.067+0000 7f1e51ec4640 1 -- 172.21.15.112:0/3827627662 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e4c0c6180 con 0x7f1e34003ae0 2024-09-21T00:28:29.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:29.068+0000 7f1e33fff640 1 -- 172.21.15.112:0/3827627662 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1e3c035020 con 0x7f1e34003ae0 2024-09-21T00:28:29.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:29.069+0000 7f1e33fff640 1 --2- 172.21.15.112:0/3827627662 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1e38078bc0 0x7f1e3807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:29.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:29.069+0000 7f1e33fff640 1 -- 172.21.15.112:0/3827627662 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 5619+0+0 (secure 0 0 0) 0x7f1e3c0bb7e0 con 0x7f1e34003ae0 2024-09-21T00:28:29.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:29.069+0000 7f1e4affd640 1 --2- 172.21.15.112:0/3827627662 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1e38078bc0 0x7f1e3807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:29.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:29.070+0000 7f1e4affd640 1 --2- 172.21.15.112:0/3827627662 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1e38078bc0 0x7f1e3807b0d0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c1642a0 tx=0x7f1e40006bc0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:29.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:29.073+0000 7f1e33fff640 1 -- 172.21.15.112:0/3827627662 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1e3c047060 con 0x7f1e34003ae0 2024-09-21T00:28:29.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:29.250+0000 7f1e51ec4640 1 -- 172.21.15.112:0/3827627662 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f1e4c164900 con 0x7f1e34003ae0 2024-09-21T00:28:29.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:29.922+0000 7f1e33fff640 1 -- 172.21.15.112:0/3827627662 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v69) ==== 170+0+0 (secure 0 0 0) 0x7f1e3c083d50 con 0x7f1e34003ae0 2024-09-21T00:28:29.924 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'cephfs_data' created 2024-09-21T00:28:29.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:29.925+0000 7f1e51ec4640 1 -- 172.21.15.112:0/3827627662 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1e38078bc0 msgr2=0x7f1e3807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:29.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:29.925+0000 7f1e51ec4640 1 --2- 172.21.15.112:0/3827627662 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1e38078bc0 0x7f1e3807b0d0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c1642a0 tx=0x7f1e40006bc0 comp rx=0 tx=0).stop 2024-09-21T00:28:29.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:29.925+0000 7f1e51ec4640 1 -- 172.21.15.112:0/3827627662 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e34003ae0 msgr2=0x7f1e4c0b6900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:29.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:29.925+0000 7f1e51ec4640 1 --2- 172.21.15.112:0/3827627662 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e34003ae0 0x7f1e4c0b6900 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c00c370 tx=0x7f1e3c001dc0 comp rx=0 tx=0).stop 2024-09-21T00:28:29.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:29.925+0000 7f1e51ec4640 1 -- 172.21.15.112:0/3827627662 shutdown_connections 2024-09-21T00:28:29.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:29.925+0000 7f1e51ec4640 1 --2- 172.21.15.112:0/3827627662 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1e4c162280 0x7f1e4c156570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:29.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:29.925+0000 7f1e51ec4640 1 --2- 172.21.15.112:0/3827627662 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1e4c155640 0x7f1e4c161d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:29.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:29.925+0000 7f1e51ec4640 1 --2- 172.21.15.112:0/3827627662 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1e38078bc0 0x7f1e3807b0d0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:29.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:29.925+0000 7f1e51ec4640 1 --2- 172.21.15.112:0/3827627662 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e34003ae0 0x7f1e4c0b6900 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:29.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:29.925+0000 7f1e51ec4640 1 -- 172.21.15.112:0/3827627662 >> 172.21.15.112:0/3827627662 conn(0x7f1e4c0b42e0 msgr2=0x7f1e4c0b7960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:29.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:29.925+0000 7f1e51ec4640 1 -- 172.21.15.112:0/3827627662 shutdown_connections 2024-09-21T00:28:29.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:29.925+0000 7f1e51ec4640 1 -- 172.21.15.112:0/3827627662 wait complete. 2024-09-21T00:28:29.954 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-21T00:28:30.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:30.074+0000 7fdbf8f20640 1 -- 172.21.15.112:0/2998871694 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdbf415f910 msgr2=0x7fdbf415fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:30.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:30.074+0000 7fdbf8f20640 1 --2- 172.21.15.112:0/2998871694 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdbf415f910 0x7fdbf415fd20 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc005fd0 tx=0x7fdbdc02a540 comp rx=0 tx=0).stop 2024-09-21T00:28:30.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:30.074+0000 7fdbf8f20640 1 -- 172.21.15.112:0/2998871694 shutdown_connections 2024-09-21T00:28:30.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:30.074+0000 7fdbf8f20640 1 --2- 172.21.15.112:0/2998871694 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdbf415b7d0 0x7fdbf415bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:30.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:30.074+0000 7fdbf8f20640 1 --2- 172.21.15.112:0/2998871694 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdbf415f910 0x7fdbf415fd20 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:30.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:30.074+0000 7fdbf8f20640 1 -- 172.21.15.112:0/2998871694 >> 172.21.15.112:0/2998871694 conn(0x7fdbf40cd470 msgr2=0x7fdbf40cd880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:30.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:30.074+0000 7fdbf8f20640 1 -- 172.21.15.112:0/2998871694 shutdown_connections 2024-09-21T00:28:30.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:30.074+0000 7fdbf8f20640 1 -- 172.21.15.112:0/2998871694 wait complete. 2024-09-21T00:28:30.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:30.076+0000 7fdbf8f20640 1 Processor -- start 2024-09-21T00:28:30.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:30.076+0000 7fdbf8f20640 1 -- start start 2024-09-21T00:28:30.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:30.077+0000 7fdbf8f20640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdbf40cf2f0 0x7fdbf40c3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:30.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:30.077+0000 7fdbf8f20640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdbf415b7d0 0x7fdbf41e4f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:30.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:30.077+0000 7fdbf8f20640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdbf415f910 0x7fdbf41594a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:30.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:30.077+0000 7fdbf8f20640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf415c4b0 con 0x7fdbf415f910 2024-09-21T00:28:30.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:30.077+0000 7fdbf8f20640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf415c330 con 0x7fdbf40cf2f0 2024-09-21T00:28:30.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:30.077+0000 7fdbf8f20640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fdbf415c630 con 0x7fdbf415b7d0 2024-09-21T00:28:30.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:30.077+0000 7fdbf2d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdbf415f910 0x7fdbf41594a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:30.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:30.077+0000 7fdbf2d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdbf415f910 0x7fdbf41594a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42986/0 (socket says 172.21.15.112:42986) 2024-09-21T00:28:30.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:30.077+0000 7fdbf2d9d640 1 -- 172.21.15.112:0/78515063 learned_addr learned my addr 172.21.15.112:0/78515063 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:28:30.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:30.077+0000 7fdbf259c640 1 --2- 172.21.15.112:0/78515063 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdbf40cf2f0 0x7fdbf40c3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:30.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:30.077+0000 7fdbf1d9b640 1 --2- 172.21.15.112:0/78515063 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdbf415b7d0 0x7fdbf41e4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:30.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:30.077+0000 7fdbf2d9d640 1 -- 172.21.15.112:0/78515063 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdbf415b7d0 msgr2=0x7fdbf41e4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:30.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:30.077+0000 7fdbf2d9d640 1 --2- 172.21.15.112:0/78515063 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdbf415b7d0 0x7fdbf41e4f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:30.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:30.077+0000 7fdbf2d9d640 1 -- 172.21.15.112:0/78515063 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdbf40cf2f0 msgr2=0x7fdbf40c3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:30.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:30.077+0000 7fdbf2d9d640 1 --2- 172.21.15.112:0/78515063 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdbf40cf2f0 0x7fdbf40c3800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:30.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:30.077+0000 7fdbf2d9d640 1 -- 172.21.15.112:0/78515063 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbf41e5b50 con 0x7fdbf415f910 2024-09-21T00:28:30.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:30.078+0000 7fdbf2d9d640 1 --2- 172.21.15.112:0/78515063 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdbf415f910 0x7fdbf41594a0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fdbec00e8d0 tx=0x7fdbec00ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:30.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:30.078+0000 7fdbeaffd640 1 -- 172.21.15.112:0/78515063 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbec00ccc0 con 0x7fdbf415f910 2024-09-21T00:28:30.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:30.078+0000 7fdbeaffd640 1 -- 172.21.15.112:0/78515063 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdbec00ce60 con 0x7fdbf415f910 2024-09-21T00:28:30.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:30.078+0000 7fdbf8f20640 1 -- 172.21.15.112:0/78515063 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbf4141880 con 0x7fdbf415f910 2024-09-21T00:28:30.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:30.078+0000 7fdbeaffd640 1 -- 172.21.15.112:0/78515063 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbec004280 con 0x7fdbf415f910 2024-09-21T00:28:30.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:30.078+0000 7fdbf8f20640 1 -- 172.21.15.112:0/78515063 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbf4141e40 con 0x7fdbf415f910 2024-09-21T00:28:30.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:30.079+0000 7fdbf8f20640 1 -- 172.21.15.112:0/78515063 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbf406cf90 con 0x7fdbf415f910 2024-09-21T00:28:30.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:30.080+0000 7fdbeaffd640 1 -- 172.21.15.112:0/78515063 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fdbec0035f0 con 0x7fdbf415f910 2024-09-21T00:28:30.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:30.080+0000 7fdbeaffd640 1 --2- 172.21.15.112:0/78515063 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fdbd4078b90 0x7fdbd407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:30.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:30.080+0000 7fdbeaffd640 1 -- 172.21.15.112:0/78515063 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6001+0+0 (secure 0 0 0) 0x7fdbec09b7c0 con 0x7fdbf415f910 2024-09-21T00:28:30.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:30.080+0000 7fdbf259c640 1 --2- 172.21.15.112:0/78515063 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fdbd4078b90 0x7fdbd407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:30.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:30.081+0000 7fdbf259c640 1 --2- 172.21.15.112:0/78515063 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fdbd4078b90 0x7fdbd407b0a0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fdbe4001e20 tx=0x7fdbe4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:30.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:30.084+0000 7fdbeaffd640 1 -- 172.21.15.112:0/78515063 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fdbf406cf90 con 0x7fdbf415f910 2024-09-21T00:28:30.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:30.296+0000 7fdbf8f20640 1 -- 172.21.15.112:0/78515063 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fdbf415bbe0 con 0x7fdbf415f910 2024-09-21T00:28:30.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:30.923+0000 7fdbeaffd640 1 -- 172.21.15.112:0/78515063 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 11 and data pool 12 v44) ==== 174+0+0 (secure 0 0 0) 0x7fdbec0635a0 con 0x7fdbf415f910 2024-09-21T00:28:30.925 INFO:teuthology.orchestra.run.smithi112.stderr:new fs with metadata pool 11 and data pool 12 2024-09-21T00:28:30.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:30.929+0000 7fdbf8f20640 1 -- 172.21.15.112:0/78515063 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fdbd4078b90 msgr2=0x7fdbd407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:30.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:30.929+0000 7fdbf8f20640 1 --2- 172.21.15.112:0/78515063 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fdbd4078b90 0x7fdbd407b0a0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fdbe4001e20 tx=0x7fdbe4009040 comp rx=0 tx=0).stop 2024-09-21T00:28:30.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:30.929+0000 7fdbf8f20640 1 -- 172.21.15.112:0/78515063 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdbf415f910 msgr2=0x7fdbf41594a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:30.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:30.929+0000 7fdbf8f20640 1 --2- 172.21.15.112:0/78515063 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdbf415f910 0x7fdbf41594a0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fdbec00e8d0 tx=0x7fdbec00ed90 comp rx=0 tx=0).stop 2024-09-21T00:28:30.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:30.929+0000 7fdbf8f20640 1 -- 172.21.15.112:0/78515063 shutdown_connections 2024-09-21T00:28:30.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:30.929+0000 7fdbf8f20640 1 --2- 172.21.15.112:0/78515063 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fdbd4078b90 0x7fdbd407b0a0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:30.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:30.929+0000 7fdbf8f20640 1 --2- 172.21.15.112:0/78515063 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdbf415f910 0x7fdbf41594a0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:30.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:30.929+0000 7fdbf8f20640 1 --2- 172.21.15.112:0/78515063 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdbf415b7d0 0x7fdbf41e4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:30.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:30.929+0000 7fdbf8f20640 1 --2- 172.21.15.112:0/78515063 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdbf40cf2f0 0x7fdbf40c3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:30.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:30.930+0000 7fdbf8f20640 1 -- 172.21.15.112:0/78515063 >> 172.21.15.112:0/78515063 conn(0x7fdbf40cd470 msgr2=0x7fdbf406d670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:30.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:30.930+0000 7fdbf8f20640 1 -- 172.21.15.112:0/78515063 shutdown_connections 2024-09-21T00:28:30.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:30.930+0000 7fdbf8f20640 1 -- 172.21.15.112:0/78515063 wait complete. 2024-09-21T00:28:30.961 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T00:28:31.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.076+0000 7f96e1339640 1 -- 172.21.15.112:0/2541447175 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96dc15e8a0 msgr2=0x7f96dc15ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:31.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.076+0000 7f96e1339640 1 --2- 172.21.15.112:0/2541447175 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96dc15e8a0 0x7f96dc15ecb0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f96dc0caff0 tx=0x7f96d002a560 comp rx=0 tx=0).stop 2024-09-21T00:28:31.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.076+0000 7f96e1339640 1 -- 172.21.15.112:0/2541447175 shutdown_connections 2024-09-21T00:28:31.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.076+0000 7f96e1339640 1 --2- 172.21.15.112:0/2541447175 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96dc15e8a0 0x7f96dc15ecb0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:31.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.076+0000 7f96e1339640 1 --2- 172.21.15.112:0/2541447175 >> v2:172.21.15.187:3300/0 conn(0x7f96dc0afdf0 0x7f96dc0cf500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:31.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.076+0000 7f96e1339640 1 -- 172.21.15.112:0/2541447175 >> 172.21.15.112:0/2541447175 conn(0x7f96dc0cd470 msgr2=0x7f96dc0cd880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:31.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.076+0000 7f96e1339640 1 -- 172.21.15.112:0/2541447175 shutdown_connections 2024-09-21T00:28:31.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.076+0000 7f96e1339640 1 -- 172.21.15.112:0/2541447175 wait complete. 2024-09-21T00:28:31.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.078+0000 7f96e1339640 1 Processor -- start 2024-09-21T00:28:31.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.078+0000 7f96e1339640 1 -- start start 2024-09-21T00:28:31.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.079+0000 7f96e1339640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96dc0afdf0 0x7f96dc1f7930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:31.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.079+0000 7f96e1339640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f96dc15e8a0 0x7f96dc1e80b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:31.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.079+0000 7f96e1339640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f96dc15f1f0 0x7f96dc0c9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:31.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.079+0000 7f96e1339640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f96dc0cfbc0 con 0x7f96dc0afdf0 2024-09-21T00:28:31.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.079+0000 7f96e1339640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f96dc0cfa40 con 0x7f96dc15f1f0 2024-09-21T00:28:31.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.079+0000 7f96e1339640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f96dc15b650 con 0x7f96dc15e8a0 2024-09-21T00:28:31.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.079+0000 7f96daffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96dc0afdf0 0x7f96dc1f7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:31.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.079+0000 7f96daffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96dc0afdf0 0x7f96dc1f7930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42998/0 (socket says 172.21.15.112:42998) 2024-09-21T00:28:31.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.079+0000 7f96daffd640 1 -- 172.21.15.112:0/1995135763 learned_addr learned my addr 172.21.15.112:0/1995135763 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:28:31.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.079+0000 7f96db7fe640 1 --2- 172.21.15.112:0/1995135763 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f96dc15f1f0 0x7f96dc0c9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:31.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.079+0000 7f96da7fc640 1 --2- 172.21.15.112:0/1995135763 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f96dc15e8a0 0x7f96dc1e80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:31.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.080+0000 7f96daffd640 1 -- 172.21.15.112:0/1995135763 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f96dc15e8a0 msgr2=0x7f96dc1e80b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:31.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.080+0000 7f96daffd640 1 --2- 172.21.15.112:0/1995135763 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f96dc15e8a0 0x7f96dc1e80b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:31.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.080+0000 7f96daffd640 1 -- 172.21.15.112:0/1995135763 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f96dc15f1f0 msgr2=0x7f96dc0c9fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:31.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.080+0000 7f96daffd640 1 --2- 172.21.15.112:0/1995135763 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f96dc15f1f0 0x7f96dc0c9fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:31.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.080+0000 7f96daffd640 1 -- 172.21.15.112:0/1995135763 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96dc1e85f0 con 0x7f96dc0afdf0 2024-09-21T00:28:31.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.080+0000 7f96daffd640 1 --2- 172.21.15.112:0/1995135763 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96dc0afdf0 0x7f96dc1f7930 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f96cc00ba80 tx=0x7f96cc00bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:31.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.080+0000 7f96c37fe640 1 -- 172.21.15.112:0/1995135763 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96cc019070 con 0x7f96dc0afdf0 2024-09-21T00:28:31.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.080+0000 7f96c37fe640 1 -- 172.21.15.112:0/1995135763 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f96cc00ccb0 con 0x7f96dc0afdf0 2024-09-21T00:28:31.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.080+0000 7f96e1339640 1 -- 172.21.15.112:0/1995135763 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96dc1f3880 con 0x7f96dc0afdf0 2024-09-21T00:28:31.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.080+0000 7f96c37fe640 1 -- 172.21.15.112:0/1995135763 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96cc0146a0 con 0x7f96dc0afdf0 2024-09-21T00:28:31.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.080+0000 7f96e1339640 1 -- 172.21.15.112:0/1995135763 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96dc1f3e40 con 0x7f96dc0afdf0 2024-09-21T00:28:31.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.081+0000 7f96e1339640 1 -- 172.21.15.112:0/1995135763 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96dc15bb10 con 0x7f96dc0afdf0 2024-09-21T00:28:31.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.082+0000 7f96c37fe640 1 -- 172.21.15.112:0/1995135763 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f96cc01c070 con 0x7f96dc0afdf0 2024-09-21T00:28:31.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.083+0000 7f96c37fe640 1 --2- 172.21.15.112:0/1995135763 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f96c8078b90 0x7f96c807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:31.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.083+0000 7f96c37fe640 1 -- 172.21.15.112:0/1995135763 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6101+0+0 (secure 0 0 0) 0x7f96cc09aaf0 con 0x7f96dc0afdf0 2024-09-21T00:28:31.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.083+0000 7f96da7fc640 1 --2- 172.21.15.112:0/1995135763 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f96c8078b90 0x7f96c807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:31.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.083+0000 7f96da7fc640 1 --2- 172.21.15.112:0/1995135763 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f96c8078b90 0x7f96c807b0a0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f96d0034000 tx=0x7f96d0004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:31.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.087+0000 7f96c37fe640 1 -- 172.21.15.112:0/1995135763 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f96cc0627e0 con 0x7f96dc0afdf0 2024-09-21T00:28:31.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.268+0000 7f96e1339640 1 -- 172.21.15.112:0/1995135763 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f96dc06fe80 con 0x7f96dc0afdf0 2024-09-21T00:28:31.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.269+0000 7f96c37fe640 1 -- 172.21.15.112:0/1995135763 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) ==== 74+0+17042 (secure 0 0 0) 0x7f96cc067210 con 0x7f96dc0afdf0 2024-09-21T00:28:31.272 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:28:31.272 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":70,"fsid":"8b34692b-dc65-49b2-a08f-26f707b96ac5","created":"2024-09-21T00:25:22.788490+0000","modified":"2024-09-21T00:28:30.298583+0000","last_up_change":"2024-09-21T00:25:36.161835+0000","last_in_change":"2024-09-21T00:25:26.346500+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T00:25:37.105006+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-09-21T00:28:27.930288+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-09-21T00:28:29.255565+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9c575086-67d2-4de2-b01b-d44dcc21d604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6809","nonce":755760621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6811","nonce":755760621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6815","nonce":755760621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6813","nonce":755760621}]},"public_addr":"172.21.15.112:6809/755760621","cluster_addr":"172.21.15.112:6811/755760621","heartbeat_back_addr":"172.21.15.112:6815/755760621","heartbeat_front_addr":"172.21.15.112:6813/755760621","state":["exists","up"]},{"osd":1,"uuid":"a983c227-a3ce-4bdd-82f1-438ba7e3ecd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6801","nonce":344932592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6803","nonce":344932592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6807","nonce":344932592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6805","nonce":344932592}]},"public_addr":"172.21.15.112:6801/344932592","cluster_addr":"172.21.15.112:6803/344932592","heartbeat_back_addr":"172.21.15.112:6807/344932592","heartbeat_front_addr":"172.21.15.112:6805/344932592","state":["exists","up"]},{"osd":2,"uuid":"1193c1d8-bfff-4a57-8200-5f69bf1ea3af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6825","nonce":407933616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6827","nonce":407933616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6831","nonce":407933616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6829","nonce":407933616}]},"public_addr":"172.21.15.112:6825/407933616","cluster_addr":"172.21.15.112:6827/407933616","heartbeat_back_addr":"172.21.15.112:6831/407933616","heartbeat_front_addr":"172.21.15.112:6829/407933616","state":["exists","up"]},{"osd":3,"uuid":"af04330a-36ad-40cc-a8dd-fa259b221de5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6817","nonce":4120164806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6819","nonce":4120164806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6823","nonce":4120164806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6821","nonce":4120164806}]},"public_addr":"172.21.15.112:6817/4120164806","cluster_addr":"172.21.15.112:6819/4120164806","heartbeat_back_addr":"172.21.15.112:6823/4120164806","heartbeat_front_addr":"172.21.15.112:6821/4120164806","state":["exists","up"]},{"osd":4,"uuid":"5b893111-292c-4c5a-b184-53ad38c7199e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6803","nonce":522077560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6805","nonce":522077560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6809","nonce":522077560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6807","nonce":522077560}]},"public_addr":"172.21.15.187:6803/522077560","cluster_addr":"172.21.15.187:6805/522077560","heartbeat_back_addr":"172.21.15.187:6809/522077560","heartbeat_front_addr":"172.21.15.187:6807/522077560","state":["exists","up"]},{"osd":5,"uuid":"c0b8e011-45fe-4c5e-925e-6be2a7f1868a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6827","nonce":3593144721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6829","nonce":3593144721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6833","nonce":3593144721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6831","nonce":3593144721}]},"public_addr":"172.21.15.187:6827/3593144721","cluster_addr":"172.21.15.187:6829/3593144721","heartbeat_back_addr":"172.21.15.187:6833/3593144721","heartbeat_front_addr":"172.21.15.187:6831/3593144721","state":["exists","up"]},{"osd":6,"uuid":"87d2d650-7489-4593-98a3-a1f433ce8dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6811","nonce":1494387657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6813","nonce":1494387657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6817","nonce":1494387657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6815","nonce":1494387657}]},"public_addr":"172.21.15.187:6811/1494387657","cluster_addr":"172.21.15.187:6813/1494387657","heartbeat_back_addr":"172.21.15.187:6817/1494387657","heartbeat_front_addr":"172.21.15.187:6815/1494387657","state":["exists","up"]},{"osd":7,"uuid":"c6b4f8e2-a1ee-4438-aa37-67ff4b6ef1f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6819","nonce":1624699842}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6821","nonce":1624699842}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6825","nonce":1624699842}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6823","nonce":1624699842}]},"public_addr":"172.21.15.187:6819/1624699842","cluster_addr":"172.21.15.187:6821/1624699842","heartbeat_back_addr":"172.21.15.187:6825/1624699842","heartbeat_front_addr":"172.21.15.187:6823/1624699842","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.873162+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.400506+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.664317+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.159791+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.202630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.113212+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.384786+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.448441+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T00:28:31.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.272+0000 7f96e1339640 1 -- 172.21.15.112:0/1995135763 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f96c8078b90 msgr2=0x7f96c807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:31.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.272+0000 7f96e1339640 1 --2- 172.21.15.112:0/1995135763 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f96c8078b90 0x7f96c807b0a0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f96d0034000 tx=0x7f96d0004730 comp rx=0 tx=0).stop 2024-09-21T00:28:31.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.272+0000 7f96e1339640 1 -- 172.21.15.112:0/1995135763 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96dc0afdf0 msgr2=0x7f96dc1f7930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:31.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.272+0000 7f96e1339640 1 --2- 172.21.15.112:0/1995135763 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96dc0afdf0 0x7f96dc1f7930 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f96cc00ba80 tx=0x7f96cc00bf40 comp rx=0 tx=0).stop 2024-09-21T00:28:31.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.272+0000 7f96e1339640 1 -- 172.21.15.112:0/1995135763 shutdown_connections 2024-09-21T00:28:31.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.272+0000 7f96e1339640 1 --2- 172.21.15.112:0/1995135763 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f96dc15f1f0 0x7f96dc0c9fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:31.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.272+0000 7f96e1339640 1 --2- 172.21.15.112:0/1995135763 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f96dc15e8a0 0x7f96dc1e80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:31.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.272+0000 7f96e1339640 1 --2- 172.21.15.112:0/1995135763 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f96c8078b90 0x7f96c807b0a0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:31.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.272+0000 7f96e1339640 1 --2- 172.21.15.112:0/1995135763 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96dc0afdf0 0x7f96dc1f7930 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:31.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.272+0000 7f96e1339640 1 -- 172.21.15.112:0/1995135763 >> 172.21.15.112:0/1995135763 conn(0x7f96dc0cd470 msgr2=0x7f96dc15ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:31.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.272+0000 7f96e1339640 1 -- 172.21.15.112:0/1995135763 shutdown_connections 2024-09-21T00:28:31.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.272+0000 7f96e1339640 1 -- 172.21.15.112:0/1995135763 wait complete. 2024-09-21T00:28:31.286 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T00:28:31.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.395+0000 7f9dac827640 1 -- 172.21.15.112:0/544011731 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9da80cd1f0 msgr2=0x7f9da80d0860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:31.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.395+0000 7f9dac827640 1 --2- 172.21.15.112:0/544011731 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9da80cd1f0 0x7f9da80d0860 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f9da0005fd0 tx=0x7f9da002a8b0 comp rx=0 tx=0).stop 2024-09-21T00:28:31.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.395+0000 7f9dac827640 1 -- 172.21.15.112:0/544011731 shutdown_connections 2024-09-21T00:28:31.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.395+0000 7f9dac827640 1 --2- 172.21.15.112:0/544011731 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9da80cd1f0 0x7f9da80d0860 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:31.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.395+0000 7f9dac827640 1 -- 172.21.15.112:0/544011731 >> 172.21.15.112:0/544011731 conn(0x7f9da80b43f0 msgr2=0x7f9da80b4830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:31.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.395+0000 7f9dac827640 1 -- 172.21.15.112:0/544011731 shutdown_connections 2024-09-21T00:28:31.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.395+0000 7f9dac827640 1 -- 172.21.15.112:0/544011731 wait complete. 2024-09-21T00:28:31.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.397+0000 7f9dac827640 1 Processor -- start 2024-09-21T00:28:31.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.397+0000 7f9dac827640 1 -- start start 2024-09-21T00:28:31.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.398+0000 7f9dac827640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9da80d2030 0x7f9da8153cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:31.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.398+0000 7f9dac827640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9da815d870 0x7f9da81545e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:31.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.398+0000 7f9dac827640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9da80cab90 0x7f9da8154b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:31.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.398+0000 7f9dac827640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9da8167c20 con 0x7f9da80cab90 2024-09-21T00:28:31.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.398+0000 7f9dac827640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9da8167aa0 con 0x7f9da815d870 2024-09-21T00:28:31.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.398+0000 7f9dac827640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f9da8167da0 con 0x7f9da80d2030 2024-09-21T00:28:31.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.398+0000 7f9da6d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9da80cab90 0x7f9da8154b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:31.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.398+0000 7f9da6d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9da80cab90 0x7f9da8154b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:43020/0 (socket says 172.21.15.112:43020) 2024-09-21T00:28:31.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.398+0000 7f9da6d9d640 1 -- 172.21.15.112:0/3750581949 learned_addr learned my addr 172.21.15.112:0/3750581949 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:28:31.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.398+0000 7f9d9dd9b640 1 --2- 172.21.15.112:0/3750581949 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9da815d870 0x7f9da81545e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:31.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.398+0000 7f9da659c640 1 --2- 172.21.15.112:0/3750581949 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9da80d2030 0x7f9da8153cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:31.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.399+0000 7f9da6d9d640 1 -- 172.21.15.112:0/3750581949 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9da80d2030 msgr2=0x7f9da8153cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:31.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.399+0000 7f9da6d9d640 1 --2- 172.21.15.112:0/3750581949 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9da80d2030 0x7f9da8153cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:31.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.399+0000 7f9da6d9d640 1 -- 172.21.15.112:0/3750581949 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9da815d870 msgr2=0x7f9da81545e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:31.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.399+0000 7f9da6d9d640 1 --2- 172.21.15.112:0/3750581949 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9da815d870 0x7f9da81545e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:31.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.399+0000 7f9da6d9d640 1 -- 172.21.15.112:0/3750581949 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9da80ca0f0 con 0x7f9da80cab90 2024-09-21T00:28:31.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.399+0000 7f9da6d9d640 1 --2- 172.21.15.112:0/3750581949 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9da80cab90 0x7f9da8154b20 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f9da0006010 tx=0x7f9da0003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:31.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.399+0000 7f9d9f7fe640 1 -- 172.21.15.112:0/3750581949 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da0031b90 con 0x7f9da80cab90 2024-09-21T00:28:31.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.399+0000 7f9d9f7fe640 1 -- 172.21.15.112:0/3750581949 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9da0031e20 con 0x7f9da80cab90 2024-09-21T00:28:31.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.399+0000 7f9d9f7fe640 1 -- 172.21.15.112:0/3750581949 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da00332f0 con 0x7f9da80cab90 2024-09-21T00:28:31.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.399+0000 7f9dac827640 1 -- 172.21.15.112:0/3750581949 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9da80cd880 con 0x7f9da80cab90 2024-09-21T00:28:31.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.399+0000 7f9dac827640 1 -- 172.21.15.112:0/3750581949 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9da80ca670 con 0x7f9da80cab90 2024-09-21T00:28:31.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.400+0000 7f9dac827640 1 -- 172.21.15.112:0/3750581949 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9da80d0860 con 0x7f9da80cab90 2024-09-21T00:28:31.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.401+0000 7f9d9f7fe640 1 -- 172.21.15.112:0/3750581949 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9da0031660 con 0x7f9da80cab90 2024-09-21T00:28:31.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.401+0000 7f9d9f7fe640 1 --2- 172.21.15.112:0/3750581949 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f9d88078b90 0x7f9d8807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:31.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.401+0000 7f9d9f7fe640 1 -- 172.21.15.112:0/3750581949 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6101+0+0 (secure 0 0 0) 0x7f9da00bc230 con 0x7f9da80cab90 2024-09-21T00:28:31.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.402+0000 7f9da659c640 1 --2- 172.21.15.112:0/3750581949 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f9d88078b90 0x7f9d8807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:31.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.402+0000 7f9da659c640 1 --2- 172.21.15.112:0/3750581949 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f9d88078b90 0x7f9d8807b0a0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f9d98000f60 tx=0x7f9d98009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:31.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.406+0000 7f9d9f7fe640 1 -- 172.21.15.112:0/3750581949 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f9da004a030 con 0x7f9da80cab90 2024-09-21T00:28:31.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.585+0000 7f9dac827640 1 -- 172.21.15.112:0/3750581949 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9da815c370 con 0x7f9da80cab90 2024-09-21T00:28:31.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.586+0000 7f9d9f7fe640 1 -- 172.21.15.112:0/3750581949 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) ==== 74+0+17042 (secure 0 0 0) 0x7f9da003f030 con 0x7f9da80cab90 2024-09-21T00:28:31.588 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:28:31.588 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":70,"fsid":"8b34692b-dc65-49b2-a08f-26f707b96ac5","created":"2024-09-21T00:25:22.788490+0000","modified":"2024-09-21T00:28:30.298583+0000","last_up_change":"2024-09-21T00:25:36.161835+0000","last_in_change":"2024-09-21T00:25:26.346500+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T00:25:37.105006+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-09-21T00:28:27.930288+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-09-21T00:28:29.255565+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9c575086-67d2-4de2-b01b-d44dcc21d604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6809","nonce":755760621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6811","nonce":755760621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6815","nonce":755760621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6813","nonce":755760621}]},"public_addr":"172.21.15.112:6809/755760621","cluster_addr":"172.21.15.112:6811/755760621","heartbeat_back_addr":"172.21.15.112:6815/755760621","heartbeat_front_addr":"172.21.15.112:6813/755760621","state":["exists","up"]},{"osd":1,"uuid":"a983c227-a3ce-4bdd-82f1-438ba7e3ecd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6801","nonce":344932592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6803","nonce":344932592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6807","nonce":344932592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6805","nonce":344932592}]},"public_addr":"172.21.15.112:6801/344932592","cluster_addr":"172.21.15.112:6803/344932592","heartbeat_back_addr":"172.21.15.112:6807/344932592","heartbeat_front_addr":"172.21.15.112:6805/344932592","state":["exists","up"]},{"osd":2,"uuid":"1193c1d8-bfff-4a57-8200-5f69bf1ea3af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6825","nonce":407933616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6827","nonce":407933616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6831","nonce":407933616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6829","nonce":407933616}]},"public_addr":"172.21.15.112:6825/407933616","cluster_addr":"172.21.15.112:6827/407933616","heartbeat_back_addr":"172.21.15.112:6831/407933616","heartbeat_front_addr":"172.21.15.112:6829/407933616","state":["exists","up"]},{"osd":3,"uuid":"af04330a-36ad-40cc-a8dd-fa259b221de5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6817","nonce":4120164806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6819","nonce":4120164806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6823","nonce":4120164806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6821","nonce":4120164806}]},"public_addr":"172.21.15.112:6817/4120164806","cluster_addr":"172.21.15.112:6819/4120164806","heartbeat_back_addr":"172.21.15.112:6823/4120164806","heartbeat_front_addr":"172.21.15.112:6821/4120164806","state":["exists","up"]},{"osd":4,"uuid":"5b893111-292c-4c5a-b184-53ad38c7199e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6803","nonce":522077560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6805","nonce":522077560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6809","nonce":522077560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6807","nonce":522077560}]},"public_addr":"172.21.15.187:6803/522077560","cluster_addr":"172.21.15.187:6805/522077560","heartbeat_back_addr":"172.21.15.187:6809/522077560","heartbeat_front_addr":"172.21.15.187:6807/522077560","state":["exists","up"]},{"osd":5,"uuid":"c0b8e011-45fe-4c5e-925e-6be2a7f1868a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6827","nonce":3593144721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6829","nonce":3593144721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6833","nonce":3593144721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6831","nonce":3593144721}]},"public_addr":"172.21.15.187:6827/3593144721","cluster_addr":"172.21.15.187:6829/3593144721","heartbeat_back_addr":"172.21.15.187:6833/3593144721","heartbeat_front_addr":"172.21.15.187:6831/3593144721","state":["exists","up"]},{"osd":6,"uuid":"87d2d650-7489-4593-98a3-a1f433ce8dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6811","nonce":1494387657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6813","nonce":1494387657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6817","nonce":1494387657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6815","nonce":1494387657}]},"public_addr":"172.21.15.187:6811/1494387657","cluster_addr":"172.21.15.187:6813/1494387657","heartbeat_back_addr":"172.21.15.187:6817/1494387657","heartbeat_front_addr":"172.21.15.187:6815/1494387657","state":["exists","up"]},{"osd":7,"uuid":"c6b4f8e2-a1ee-4438-aa37-67ff4b6ef1f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6819","nonce":1624699842}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6821","nonce":1624699842}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6825","nonce":1624699842}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6823","nonce":1624699842}]},"public_addr":"172.21.15.187:6819/1624699842","cluster_addr":"172.21.15.187:6821/1624699842","heartbeat_back_addr":"172.21.15.187:6825/1624699842","heartbeat_front_addr":"172.21.15.187:6823/1624699842","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.873162+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.400506+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.664317+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.159791+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.202630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.113212+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.384786+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.448441+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T00:28:31.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.588+0000 7f9dac827640 1 -- 172.21.15.112:0/3750581949 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f9d88078b90 msgr2=0x7f9d8807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:31.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.588+0000 7f9dac827640 1 --2- 172.21.15.112:0/3750581949 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f9d88078b90 0x7f9d8807b0a0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f9d98000f60 tx=0x7f9d98009040 comp rx=0 tx=0).stop 2024-09-21T00:28:31.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.588+0000 7f9dac827640 1 -- 172.21.15.112:0/3750581949 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9da80cab90 msgr2=0x7f9da8154b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:31.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.588+0000 7f9dac827640 1 --2- 172.21.15.112:0/3750581949 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9da80cab90 0x7f9da8154b20 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f9da0006010 tx=0x7f9da0003ba0 comp rx=0 tx=0).stop 2024-09-21T00:28:31.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.588+0000 7f9dac827640 1 -- 172.21.15.112:0/3750581949 shutdown_connections 2024-09-21T00:28:31.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.588+0000 7f9dac827640 1 --2- 172.21.15.112:0/3750581949 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f9d88078b90 0x7f9d8807b0a0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:31.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.588+0000 7f9dac827640 1 --2- 172.21.15.112:0/3750581949 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9da80cab90 0x7f9da8154b20 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:31.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.588+0000 7f9dac827640 1 --2- 172.21.15.112:0/3750581949 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9da815d870 0x7f9da81545e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:31.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.588+0000 7f9dac827640 1 --2- 172.21.15.112:0/3750581949 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9da80d2030 0x7f9da8153cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:31.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.588+0000 7f9dac827640 1 -- 172.21.15.112:0/3750581949 >> 172.21.15.112:0/3750581949 conn(0x7f9da80b43f0 msgr2=0x7f9da80b4800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:31.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.588+0000 7f9dac827640 1 -- 172.21.15.112:0/3750581949 shutdown_connections 2024-09-21T00:28:31.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.588+0000 7f9dac827640 1 -- 172.21.15.112:0/3750581949 wait complete. 2024-09-21T00:28:31.601 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-21T00:28:31.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.714+0000 7f9dadbc3640 1 -- 172.21.15.112:0/669495164 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9da8164eb0 msgr2=0x7f9da81672d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:31.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.714+0000 7f9dadbc3640 1 --2- 172.21.15.112:0/669495164 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9da8164eb0 0x7f9da81672d0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f9da0005fd0 tx=0x7f9da002a780 comp rx=0 tx=0).stop 2024-09-21T00:28:31.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.714+0000 7f9dadbc3640 1 -- 172.21.15.112:0/669495164 shutdown_connections 2024-09-21T00:28:31.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.714+0000 7f9dadbc3640 1 --2- 172.21.15.112:0/669495164 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9da8164eb0 0x7f9da81672d0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:31.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.714+0000 7f9dadbc3640 1 -- 172.21.15.112:0/669495164 >> 172.21.15.112:0/669495164 conn(0x7f9da80b42e0 msgr2=0x7f9da80b4720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:31.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.714+0000 7f9dadbc3640 1 -- 172.21.15.112:0/669495164 shutdown_connections 2024-09-21T00:28:31.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.714+0000 7f9dadbc3640 1 -- 172.21.15.112:0/669495164 wait complete. 2024-09-21T00:28:31.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.716+0000 7f9dadbc3640 1 Processor -- start 2024-09-21T00:28:31.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.716+0000 7f9dadbc3640 1 -- start start 2024-09-21T00:28:31.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.718+0000 7f9dadbc3640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9da80b1d30 0x7f9da8156750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:31.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.718+0000 7f9dadbc3640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9da80ccbf0 0x7f9da81576e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:31.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.718+0000 7f9dadbc3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9da8164eb0 0x7f9da8157c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:31.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.718+0000 7f9dadbc3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9da8167bd0 con 0x7f9da8164eb0 2024-09-21T00:28:31.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.718+0000 7f9dadbc3640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9da8167a50 con 0x7f9da80ccbf0 2024-09-21T00:28:31.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.718+0000 7f9dadbc3640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f9da8167d50 con 0x7f9da80b1d30 2024-09-21T00:28:31.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.718+0000 7f9da7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9da8164eb0 0x7f9da8157c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:31.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.718+0000 7f9da7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9da8164eb0 0x7f9da8157c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:43036/0 (socket says 172.21.15.112:43036) 2024-09-21T00:28:31.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.718+0000 7f9da7fff640 1 -- 172.21.15.112:0/2747420632 learned_addr learned my addr 172.21.15.112:0/2747420632 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:28:31.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.718+0000 7f9da6ffd640 1 --2- 172.21.15.112:0/2747420632 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9da80ccbf0 0x7f9da81576e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:31.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.718+0000 7f9da77fe640 1 --2- 172.21.15.112:0/2747420632 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9da80b1d30 0x7f9da8156750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:31.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.718+0000 7f9da7fff640 1 -- 172.21.15.112:0/2747420632 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9da80b1d30 msgr2=0x7f9da8156750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:31.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.718+0000 7f9da7fff640 1 --2- 172.21.15.112:0/2747420632 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9da80b1d30 0x7f9da8156750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:31.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.718+0000 7f9da7fff640 1 -- 172.21.15.112:0/2747420632 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9da80ccbf0 msgr2=0x7f9da81576e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:31.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.718+0000 7f9da7fff640 1 --2- 172.21.15.112:0/2747420632 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9da80ccbf0 0x7f9da81576e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:31.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.718+0000 7f9da7fff640 1 -- 172.21.15.112:0/2747420632 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9da81584b0 con 0x7f9da8164eb0 2024-09-21T00:28:31.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.718+0000 7f9da7fff640 1 --2- 172.21.15.112:0/2747420632 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9da8164eb0 0x7f9da8157c20 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f9da0005fd0 tx=0x7f9da0003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:31.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.718+0000 7f9d9ffff640 1 -- 172.21.15.112:0/2747420632 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da0040070 con 0x7f9da8164eb0 2024-09-21T00:28:31.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.718+0000 7f9d9ffff640 1 -- 172.21.15.112:0/2747420632 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9da0004540 con 0x7f9da8164eb0 2024-09-21T00:28:31.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.719+0000 7f9dadbc3640 1 -- 172.21.15.112:0/2747420632 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9da80ca3e0 con 0x7f9da8164eb0 2024-09-21T00:28:31.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.719+0000 7f9d9ffff640 1 -- 172.21.15.112:0/2747420632 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da0003e10 con 0x7f9da8164eb0 2024-09-21T00:28:31.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.719+0000 7f9dadbc3640 1 -- 172.21.15.112:0/2747420632 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9da80ca9a0 con 0x7f9da8164eb0 2024-09-21T00:28:31.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.720+0000 7f9dadbc3640 1 -- 172.21.15.112:0/2747420632 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9da80cd000 con 0x7f9da8164eb0 2024-09-21T00:28:31.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.720+0000 7f9d9ffff640 1 -- 172.21.15.112:0/2747420632 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9da0043070 con 0x7f9da8164eb0 2024-09-21T00:28:31.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.721+0000 7f9d9ffff640 1 --2- 172.21.15.112:0/2747420632 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f9d88078b90 0x7f9d8807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:31.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.721+0000 7f9d9ffff640 1 -- 172.21.15.112:0/2747420632 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6101+0+0 (secure 0 0 0) 0x7f9da00bc150 con 0x7f9da8164eb0 2024-09-21T00:28:31.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.721+0000 7f9da77fe640 1 --2- 172.21.15.112:0/2747420632 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f9d88078b90 0x7f9d8807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:31.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.721+0000 7f9da77fe640 1 --2- 172.21.15.112:0/2747420632 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f9d88078b90 0x7f9d8807b0a0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f9d98007fd0 tx=0x7f9d98001370 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:31.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.725+0000 7f9d9ffff640 1 -- 172.21.15.112:0/2747420632 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f9da0083db0 con 0x7f9da8164eb0 2024-09-21T00:28:31.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:31.940+0000 7f9dadbc3640 1 -- 172.21.15.112:0/2747420632 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f9da8075bc0 con 0x7f9da8164eb0 2024-09-21T00:28:32.904 INFO:tasks.ceph.mds.a.smithi112.stderr:2024-09-21T00:28:32.903+0000 7fcd4ff42640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T00:28:32.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:32.945+0000 7f9d9ffff640 1 -- 172.21.15.112:0/2747420632 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v46) ==== 118+0+0 (secure 0 0 0) 0x7f9da00887e0 con 0x7f9da8164eb0 2024-09-21T00:28:32.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:32.948+0000 7f9dadbc3640 1 -- 172.21.15.112:0/2747420632 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f9d88078b90 msgr2=0x7f9d8807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:32.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:32.948+0000 7f9dadbc3640 1 --2- 172.21.15.112:0/2747420632 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f9d88078b90 0x7f9d8807b0a0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f9d98007fd0 tx=0x7f9d98001370 comp rx=0 tx=0).stop 2024-09-21T00:28:32.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:32.948+0000 7f9dadbc3640 1 -- 172.21.15.112:0/2747420632 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9da8164eb0 msgr2=0x7f9da8157c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:32.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:32.948+0000 7f9dadbc3640 1 --2- 172.21.15.112:0/2747420632 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9da8164eb0 0x7f9da8157c20 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f9da0005fd0 tx=0x7f9da0003c20 comp rx=0 tx=0).stop 2024-09-21T00:28:32.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:32.949+0000 7f9dadbc3640 1 -- 172.21.15.112:0/2747420632 shutdown_connections 2024-09-21T00:28:32.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:32.949+0000 7f9dadbc3640 1 --2- 172.21.15.112:0/2747420632 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f9d88078b90 0x7f9d8807b0a0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:32.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:32.949+0000 7f9dadbc3640 1 --2- 172.21.15.112:0/2747420632 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9da8164eb0 0x7f9da8157c20 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:32.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:32.949+0000 7f9dadbc3640 1 --2- 172.21.15.112:0/2747420632 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9da80ccbf0 0x7f9da81576e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:32.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:32.949+0000 7f9dadbc3640 1 --2- 172.21.15.112:0/2747420632 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9da80b1d30 0x7f9da8156750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:32.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:32.949+0000 7f9dadbc3640 1 -- 172.21.15.112:0/2747420632 >> 172.21.15.112:0/2747420632 conn(0x7f9da80b42e0 msgr2=0x7f9da80b2140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:32.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:32.949+0000 7f9dadbc3640 1 -- 172.21.15.112:0/2747420632 shutdown_connections 2024-09-21T00:28:32.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:32.949+0000 7f9dadbc3640 1 -- 172.21.15.112:0/2747420632 wait complete. 2024-09-21T00:28:32.964 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-21T00:28:32.965 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-21T00:28:33.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:33.085+0000 7fcce497b640 1 -- 172.21.15.112:0/3389094008 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcce00d0ae0 msgr2=0x7fcce00d38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:33.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:33.085+0000 7fcce497b640 1 --2- 172.21.15.112:0/3389094008 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcce00d0ae0 0x7fcce00d38c0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fccd00098e0 tx=0x7fccd002a530 comp rx=0 tx=0).stop 2024-09-21T00:28:33.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:33.085+0000 7fcce497b640 1 -- 172.21.15.112:0/3389094008 shutdown_connections 2024-09-21T00:28:33.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:33.085+0000 7fcce497b640 1 --2- 172.21.15.112:0/3389094008 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcce015ea40 0x7fcce0070250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:33.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:33.085+0000 7fcce497b640 1 --2- 172.21.15.112:0/3389094008 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcce00d0ae0 0x7fcce00d38c0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:33.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:33.085+0000 7fcce497b640 1 -- 172.21.15.112:0/3389094008 >> 172.21.15.112:0/3389094008 conn(0x7fcce009d8a0 msgr2=0x7fcce008bfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:33.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:33.085+0000 7fcce497b640 1 -- 172.21.15.112:0/3389094008 shutdown_connections 2024-09-21T00:28:33.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:33.085+0000 7fcce497b640 1 -- 172.21.15.112:0/3389094008 wait complete. 2024-09-21T00:28:33.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:33.087+0000 7fcce497b640 1 Processor -- start 2024-09-21T00:28:33.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:33.087+0000 7fcce497b640 1 -- start start 2024-09-21T00:28:33.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:33.088+0000 7fcce497b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcce00d0ae0 0x7fcce015a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:33.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:33.088+0000 7fcce497b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcce00d4380 0x7fcce00a1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:33.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:33.088+0000 7fcce497b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcce015ea40 0x7fcce00a2030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:33.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:33.088+0000 7fcce497b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcce0162b50 con 0x7fcce00d4380 2024-09-21T00:28:33.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:33.088+0000 7fcce497b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fcce01629d0 con 0x7fcce00d0ae0 2024-09-21T00:28:33.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:33.088+0000 7fcce497b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fcce0162cd0 con 0x7fcce015ea40 2024-09-21T00:28:33.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:33.088+0000 7fccddd9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcce00d4380 0x7fcce00a1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:33.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:33.089+0000 7fccddd9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcce00d4380 0x7fcce00a1af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:43054/0 (socket says 172.21.15.112:43054) 2024-09-21T00:28:33.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:33.089+0000 7fccddd9b640 1 -- 172.21.15.112:0/3954967453 learned_addr learned my addr 172.21.15.112:0/3954967453 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:28:33.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:33.089+0000 7fccde59c640 1 --2- 172.21.15.112:0/3954967453 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcce00d0ae0 0x7fcce015a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:33.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:33.089+0000 7fccded9d640 1 --2- 172.21.15.112:0/3954967453 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcce015ea40 0x7fcce00a2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:33.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:33.089+0000 7fccddd9b640 1 -- 172.21.15.112:0/3954967453 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcce015ea40 msgr2=0x7fcce00a2030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:33.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:33.089+0000 7fccddd9b640 1 --2- 172.21.15.112:0/3954967453 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcce015ea40 0x7fcce00a2030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:33.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:33.089+0000 7fccddd9b640 1 -- 172.21.15.112:0/3954967453 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcce00d0ae0 msgr2=0x7fcce015a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:33.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:33.089+0000 7fccddd9b640 1 --2- 172.21.15.112:0/3954967453 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcce00d0ae0 0x7fcce015a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:33.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:33.089+0000 7fccddd9b640 1 -- 172.21.15.112:0/3954967453 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcce015ac10 con 0x7fcce00d4380 2024-09-21T00:28:33.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:33.089+0000 7fccded9d640 1 --2- 172.21.15.112:0/3954967453 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcce015ea40 0x7fcce00a2030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:28:33.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:33.089+0000 7fccddd9b640 1 --2- 172.21.15.112:0/3954967453 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcce00d4380 0x7fcce00a1af0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fccd400d7a0 tx=0x7fccd400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:33.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:33.089+0000 7fccceffd640 1 -- 172.21.15.112:0/3954967453 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccd4017070 con 0x7fcce00d4380 2024-09-21T00:28:33.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:33.089+0000 7fccceffd640 1 -- 172.21.15.112:0/3954967453 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fccd4009d20 con 0x7fcce00d4380 2024-09-21T00:28:33.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:33.089+0000 7fcce497b640 1 -- 172.21.15.112:0/3954967453 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcce00cda80 con 0x7fcce00d4380 2024-09-21T00:28:33.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:33.089+0000 7fccceffd640 1 -- 172.21.15.112:0/3954967453 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccd4004280 con 0x7fcce00d4380 2024-09-21T00:28:33.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:33.089+0000 7fcce497b640 1 -- 172.21.15.112:0/3954967453 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcce00ce040 con 0x7fcce00d4380 2024-09-21T00:28:33.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:33.090+0000 7fcce497b640 1 -- 172.21.15.112:0/3954967453 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcce01631f0 con 0x7fcce00d4380 2024-09-21T00:28:33.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:33.091+0000 7fccceffd640 1 -- 172.21.15.112:0/3954967453 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fccd4005020 con 0x7fcce00d4380 2024-09-21T00:28:33.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:33.091+0000 7fccceffd640 1 --2- 172.21.15.112:0/3954967453 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fccc0078b90 0x7fccc007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:33.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:33.091+0000 7fccceffd640 1 -- 172.21.15.112:0/3954967453 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6101+0+0 (secure 0 0 0) 0x7fccd409b260 con 0x7fcce00d4380 2024-09-21T00:28:33.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:33.092+0000 7fccde59c640 1 --2- 172.21.15.112:0/3954967453 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fccc0078b90 0x7fccc007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:33.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:33.092+0000 7fccde59c640 1 --2- 172.21.15.112:0/3954967453 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fccc0078b90 0x7fccc007b0a0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fccd0034000 tx=0x7fccd0004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:33.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:33.096+0000 7fccceffd640 1 -- 172.21.15.112:0/3954967453 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fccd4020020 con 0x7fcce00d4380 2024-09-21T00:28:33.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:33.312+0000 7fcce497b640 1 -- 172.21.15.112:0/3954967453 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fcce015d1e0 con 0x7fcce00d4380 2024-09-21T00:28:33.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:33.955+0000 7fccceffd640 1 -- 172.21.15.112:0/3954967453 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v47) ==== 122+0+0 (secure 0 0 0) 0x7fccd4063510 con 0x7fcce00d4380 2024-09-21T00:28:33.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:33.957+0000 7fcce497b640 1 -- 172.21.15.112:0/3954967453 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fccc0078b90 msgr2=0x7fccc007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:33.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:33.957+0000 7fcce497b640 1 --2- 172.21.15.112:0/3954967453 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fccc0078b90 0x7fccc007b0a0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fccd0034000 tx=0x7fccd0004730 comp rx=0 tx=0).stop 2024-09-21T00:28:33.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:33.957+0000 7fcce497b640 1 -- 172.21.15.112:0/3954967453 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcce00d4380 msgr2=0x7fcce00a1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:33.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:33.957+0000 7fcce497b640 1 --2- 172.21.15.112:0/3954967453 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcce00d4380 0x7fcce00a1af0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fccd400d7a0 tx=0x7fccd400dc60 comp rx=0 tx=0).stop 2024-09-21T00:28:33.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:33.957+0000 7fcce497b640 1 -- 172.21.15.112:0/3954967453 shutdown_connections 2024-09-21T00:28:33.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:33.957+0000 7fcce497b640 1 --2- 172.21.15.112:0/3954967453 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcce015ea40 0x7fcce00a2030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:33.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:33.957+0000 7fcce497b640 1 --2- 172.21.15.112:0/3954967453 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fccc0078b90 0x7fccc007b0a0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:33.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:33.957+0000 7fcce497b640 1 --2- 172.21.15.112:0/3954967453 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcce00d4380 0x7fcce00a1af0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:33.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:33.957+0000 7fcce497b640 1 --2- 172.21.15.112:0/3954967453 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcce00d0ae0 0x7fcce015a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:33.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:33.957+0000 7fcce497b640 1 -- 172.21.15.112:0/3954967453 >> 172.21.15.112:0/3954967453 conn(0x7fcce009d8a0 msgr2=0x7fcce00d4790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:33.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:33.957+0000 7fcce497b640 1 -- 172.21.15.112:0/3954967453 shutdown_connections 2024-09-21T00:28:33.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:33.957+0000 7fcce497b640 1 -- 172.21.15.112:0/3954967453 wait complete. 2024-09-21T00:28:33.971 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-21T00:28:33.971 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T00:28:34.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.082+0000 7fd6ffe69640 1 -- 172.21.15.112:0/4009238935 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6f80b41c0 msgr2=0x7fd6f80b45d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:34.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.082+0000 7fd6ffe69640 1 --2- 172.21.15.112:0/4009238935 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6f80b41c0 0x7fd6f80b45d0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fd6f0005fd0 tx=0x7fd6f002aa90 comp rx=0 tx=0).stop 2024-09-21T00:28:34.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.082+0000 7fd6ffe69640 1 -- 172.21.15.112:0/4009238935 shutdown_connections 2024-09-21T00:28:34.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.082+0000 7fd6ffe69640 1 --2- 172.21.15.112:0/4009238935 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6f80b41c0 0x7fd6f80b45d0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:34.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.082+0000 7fd6ffe69640 1 --2- 172.21.15.112:0/4009238935 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd6f80d1a80 0x7fd6f815c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:34.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.082+0000 7fd6ffe69640 1 -- 172.21.15.112:0/4009238935 >> 172.21.15.112:0/4009238935 conn(0x7fd6f80cf200 msgr2=0x7fd6f80cf610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:34.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.082+0000 7fd6ffe69640 1 -- 172.21.15.112:0/4009238935 shutdown_connections 2024-09-21T00:28:34.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.082+0000 7fd6ffe69640 1 -- 172.21.15.112:0/4009238935 wait complete. 2024-09-21T00:28:34.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.084+0000 7fd6ffe69640 1 Processor -- start 2024-09-21T00:28:34.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.084+0000 7fd6ffe69640 1 -- start start 2024-09-21T00:28:34.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.086+0000 7fd6ffe69640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd6f80b1c10 0x7fd6f806e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:34.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.086+0000 7fd6ffe69640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6f80b41c0 0x7fd6f8152a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:34.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.086+0000 7fd6ffe69640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd6f80d1a80 0x7fd6f80caf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:34.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.086+0000 7fd6ffe69640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f80d0d00 con 0x7fd6f80b41c0 2024-09-21T00:28:34.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.086+0000 7fd6ffe69640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f80d0b80 con 0x7fd6f80d1a80 2024-09-21T00:28:34.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.086+0000 7fd6ffe69640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd6f80d0e80 con 0x7fd6f80b1c10 2024-09-21T00:28:34.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.086+0000 7fd6fd404640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6f80b41c0 0x7fd6f8152a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:34.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.086+0000 7fd6fd404640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6f80b41c0 0x7fd6f8152a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:43072/0 (socket says 172.21.15.112:43072) 2024-09-21T00:28:34.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.086+0000 7fd6fd404640 1 -- 172.21.15.112:0/1559795199 learned_addr learned my addr 172.21.15.112:0/1559795199 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:28:34.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.086+0000 7fd6fdc05640 1 --2- 172.21.15.112:0/1559795199 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd6f80b1c10 0x7fd6f806e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:34.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.086+0000 7fd6fe406640 1 --2- 172.21.15.112:0/1559795199 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd6f80d1a80 0x7fd6f80caf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:34.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.086+0000 7fd6fd404640 1 -- 172.21.15.112:0/1559795199 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd6f80b1c10 msgr2=0x7fd6f806e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:34.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.086+0000 7fd6fd404640 1 --2- 172.21.15.112:0/1559795199 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd6f80b1c10 0x7fd6f806e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:34.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.086+0000 7fd6fd404640 1 -- 172.21.15.112:0/1559795199 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd6f80d1a80 msgr2=0x7fd6f80caf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:34.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.086+0000 7fd6fd404640 1 --2- 172.21.15.112:0/1559795199 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd6f80d1a80 0x7fd6f80caf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:34.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.086+0000 7fd6fd404640 1 -- 172.21.15.112:0/1559795199 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6f806ecc0 con 0x7fd6f80b41c0 2024-09-21T00:28:34.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.086+0000 7fd6fdc05640 1 --2- 172.21.15.112:0/1559795199 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd6f80b1c10 0x7fd6f806e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:28:34.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.086+0000 7fd6fd404640 1 --2- 172.21.15.112:0/1559795199 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6f80b41c0 0x7fd6f8152a00 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec00c3f0 tx=0x7fd6ec00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:34.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.086+0000 7fd6f67fc640 1 -- 172.21.15.112:0/1559795199 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6ec017070 con 0x7fd6f80b41c0 2024-09-21T00:28:34.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.086+0000 7fd6f67fc640 1 -- 172.21.15.112:0/1559795199 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd6ec0035f0 con 0x7fd6f80b41c0 2024-09-21T00:28:34.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.086+0000 7fd6ffe69640 1 -- 172.21.15.112:0/1559795199 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6f8158c00 con 0x7fd6f80b41c0 2024-09-21T00:28:34.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.086+0000 7fd6f67fc640 1 -- 172.21.15.112:0/1559795199 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6ec003a40 con 0x7fd6f80b41c0 2024-09-21T00:28:34.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.086+0000 7fd6ffe69640 1 -- 172.21.15.112:0/1559795199 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6f80ca420 con 0x7fd6f80b41c0 2024-09-21T00:28:34.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.087+0000 7fd6ffe69640 1 -- 172.21.15.112:0/1559795199 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6f80d1390 con 0x7fd6f80b41c0 2024-09-21T00:28:34.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.088+0000 7fd6f67fc640 1 -- 172.21.15.112:0/1559795199 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd6ec01a070 con 0x7fd6f80b41c0 2024-09-21T00:28:34.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.089+0000 7fd6f67fc640 1 --2- 172.21.15.112:0/1559795199 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fd6e4078b90 0x7fd6e407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:34.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.089+0000 7fd6f67fc640 1 -- 172.21.15.112:0/1559795199 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6101+0+0 (secure 0 0 0) 0x7fd6ec09a0a0 con 0x7fd6f80b41c0 2024-09-21T00:28:34.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.089+0000 7fd6fdc05640 1 --2- 172.21.15.112:0/1559795199 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fd6e4078b90 0x7fd6e407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:34.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.090+0000 7fd6fdc05640 1 --2- 172.21.15.112:0/1559795199 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fd6e4078b90 0x7fd6e407b0a0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fd6f80750a0 tx=0x7fd6e8005f90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:34.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.093+0000 7fd6f67fc640 1 -- 172.21.15.112:0/1559795199 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd6f80b4b10 con 0x7fd6f80b41c0 2024-09-21T00:28:34.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.305+0000 7fd6ffe69640 1 -- 172.21.15.112:0/1559795199 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd6f80c8bf0 con 0x7fd6f80b41c0 2024-09-21T00:28:34.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.306+0000 7fd6f67fc640 1 -- 172.21.15.112:0/1559795199 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+5237 (secure 0 0 0) 0x7fd6ec061d00 con 0x7fd6f80b41c0 2024-09-21T00:28:34.307 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:28:34.308 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":47,"btime":"2024-09-21T00:28:33:951960+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5435,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6837/3039805497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6835","nonce":3039805497},{"type":"v1","addr":"172.21.15.187:6837","nonce":3039805497}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43},{"gid":5719,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6836/2736940636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":2736940636},{"type":"v1","addr":"172.21.15.187:6836","nonce":2736940636}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43},{"gid":6162,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6834/2295407428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":2295407428},{"type":"v1","addr":"172.21.15.112:6834","nonce":2295407428}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T00:28:30.298540+0000","modified":"2024-09-21T00:28:33.314195+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6165},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6165":{"gid":6165,"name":"a","rank":0,"incarnation":44,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6835/1624734538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":1624734538},{"type":"v1","addr":"172.21.15.112:6835","nonce":1624734538}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6165,"qdb_cluster":[6165]},"id":5}]} 2024-09-21T00:28:34.308 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 47 2024-09-21T00:28:34.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.308+0000 7fd6ffe69640 1 -- 172.21.15.112:0/1559795199 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fd6e4078b90 msgr2=0x7fd6e407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:34.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.308+0000 7fd6ffe69640 1 --2- 172.21.15.112:0/1559795199 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fd6e4078b90 0x7fd6e407b0a0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fd6f80750a0 tx=0x7fd6e8005f90 comp rx=0 tx=0).stop 2024-09-21T00:28:34.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.308+0000 7fd6ffe69640 1 -- 172.21.15.112:0/1559795199 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6f80b41c0 msgr2=0x7fd6f8152a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:34.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.308+0000 7fd6ffe69640 1 --2- 172.21.15.112:0/1559795199 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6f80b41c0 0x7fd6f8152a00 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec00c3f0 tx=0x7fd6ec00c8b0 comp rx=0 tx=0).stop 2024-09-21T00:28:34.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.308+0000 7fd6ffe69640 1 -- 172.21.15.112:0/1559795199 shutdown_connections 2024-09-21T00:28:34.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.308+0000 7fd6ffe69640 1 --2- 172.21.15.112:0/1559795199 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd6f80d1a80 0x7fd6f80caf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:34.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.308+0000 7fd6ffe69640 1 --2- 172.21.15.112:0/1559795199 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fd6e4078b90 0x7fd6e407b0a0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:34.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.308+0000 7fd6ffe69640 1 --2- 172.21.15.112:0/1559795199 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6f80b41c0 0x7fd6f8152a00 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:34.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.308+0000 7fd6ffe69640 1 --2- 172.21.15.112:0/1559795199 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd6f80b1c10 0x7fd6f806e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:34.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.308+0000 7fd6ffe69640 1 -- 172.21.15.112:0/1559795199 >> 172.21.15.112:0/1559795199 conn(0x7fd6f80cf200 msgr2=0x7fd6f80b5190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:34.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.308+0000 7fd6ffe69640 1 -- 172.21.15.112:0/1559795199 shutdown_connections 2024-09-21T00:28:34.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.308+0000 7fd6ffe69640 1 -- 172.21.15.112:0/1559795199 wait complete. 2024-09-21T00:28:34.321 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T00:28:34.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.432+0000 7f3e0b20e640 1 -- 172.21.15.112:0/2496316652 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e040d01b0 msgr2=0x7f3e040d05c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:34.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.432+0000 7f3e0b20e640 1 --2- 172.21.15.112:0/2496316652 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e040d01b0 0x7f3e040d05c0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f3e040cea80 tx=0x7f3dfc02aa30 comp rx=0 tx=0).stop 2024-09-21T00:28:34.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.432+0000 7f3e0b20e640 1 -- 172.21.15.112:0/2496316652 shutdown_connections 2024-09-21T00:28:34.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.432+0000 7f3e0b20e640 1 --2- 172.21.15.112:0/2496316652 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e040d01b0 0x7f3e040d05c0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:34.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.432+0000 7f3e0b20e640 1 --2- 172.21.15.112:0/2496316652 >> v2:172.21.15.187:3300/0 conn(0x7f3e040d3880 0x7f3e04163260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:34.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.432+0000 7f3e0b20e640 1 -- 172.21.15.112:0/2496316652 >> 172.21.15.112:0/2496316652 conn(0x7f3e0408bfe0 msgr2=0x7f3e0409d890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:34.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.432+0000 7f3e0b20e640 1 -- 172.21.15.112:0/2496316652 shutdown_connections 2024-09-21T00:28:34.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.432+0000 7f3e0b20e640 1 -- 172.21.15.112:0/2496316652 wait complete. 2024-09-21T00:28:34.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.433+0000 7f3e0b20e640 1 Processor -- start 2024-09-21T00:28:34.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.433+0000 7f3e0b20e640 1 -- start start 2024-09-21T00:28:34.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.435+0000 7f3e0b20e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3e040d01b0 0x7f3e040ce1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:34.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.435+0000 7f3e0b20e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3e040d3880 0x7f3e040c9da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:34.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.435+0000 7f3e0b20e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e040d48a0 0x7f3e040cbb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:34.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.435+0000 7f3e0b20e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3e040d2460 con 0x7f3e040d48a0 2024-09-21T00:28:34.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.435+0000 7f3e0b20e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3e040d22e0 con 0x7f3e040d3880 2024-09-21T00:28:34.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.435+0000 7f3e0b20e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3e040d25e0 con 0x7f3e040d01b0 2024-09-21T00:28:34.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.435+0000 7f3e097ab640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e040d48a0 0x7f3e040cbb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:34.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.435+0000 7f3e097ab640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e040d48a0 0x7f3e040cbb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:43088/0 (socket says 172.21.15.112:43088) 2024-09-21T00:28:34.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.435+0000 7f3e097ab640 1 -- 172.21.15.112:0/1865026568 learned_addr learned my addr 172.21.15.112:0/1865026568 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:28:34.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.435+0000 7f3e08faa640 1 --2- 172.21.15.112:0/1865026568 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3e040d01b0 0x7f3e040ce1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:34.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.435+0000 7f3dfbfff640 1 --2- 172.21.15.112:0/1865026568 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3e040d3880 0x7f3e040c9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:34.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.435+0000 7f3e097ab640 1 -- 172.21.15.112:0/1865026568 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3e040d01b0 msgr2=0x7f3e040ce1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:34.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.435+0000 7f3e097ab640 1 --2- 172.21.15.112:0/1865026568 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3e040d01b0 0x7f3e040ce1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:34.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.435+0000 7f3e097ab640 1 -- 172.21.15.112:0/1865026568 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3e040d3880 msgr2=0x7f3e040c9da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:34.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.435+0000 7f3e097ab640 1 --2- 172.21.15.112:0/1865026568 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3e040d3880 0x7f3e040c9da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:34.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.435+0000 7f3e097ab640 1 -- 172.21.15.112:0/1865026568 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e041602e0 con 0x7f3e040d48a0 2024-09-21T00:28:34.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.435+0000 7f3e08faa640 1 --2- 172.21.15.112:0/1865026568 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3e040d01b0 0x7f3e040ce1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:28:34.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.435+0000 7f3e097ab640 1 --2- 172.21.15.112:0/1865026568 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e040d48a0 0x7f3e040cbb30 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f3dfc005fd0 tx=0x7f3dfc003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:34.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.436+0000 7f3df97fa640 1 -- 172.21.15.112:0/1865026568 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dfc033050 con 0x7f3e040d48a0 2024-09-21T00:28:34.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.436+0000 7f3df97fa640 1 -- 172.21.15.112:0/1865026568 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3dfc0045f0 con 0x7f3e040d48a0 2024-09-21T00:28:34.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.436+0000 7f3e0b20e640 1 -- 172.21.15.112:0/1865026568 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e0415f6d0 con 0x7f3e040d48a0 2024-09-21T00:28:34.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.436+0000 7f3df97fa640 1 -- 172.21.15.112:0/1865026568 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dfc038450 con 0x7f3e040d48a0 2024-09-21T00:28:34.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.436+0000 7f3e0b20e640 1 -- 172.21.15.112:0/1865026568 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e0415fc90 con 0x7f3e040d48a0 2024-09-21T00:28:34.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.437+0000 7f3e0b20e640 1 -- 172.21.15.112:0/1865026568 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e040d2a70 con 0x7f3e040d48a0 2024-09-21T00:28:34.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.438+0000 7f3df97fa640 1 -- 172.21.15.112:0/1865026568 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3dfc036070 con 0x7f3e040d48a0 2024-09-21T00:28:34.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.438+0000 7f3df97fa640 1 --2- 172.21.15.112:0/1865026568 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f3df0078b90 0x7f3df007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:34.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.438+0000 7f3df97fa640 1 -- 172.21.15.112:0/1865026568 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6101+0+0 (secure 0 0 0) 0x7f3dfc0bc4b0 con 0x7f3e040d48a0 2024-09-21T00:28:34.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.438+0000 7f3e08faa640 1 --2- 172.21.15.112:0/1865026568 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f3df0078b90 0x7f3df007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:34.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.439+0000 7f3e08faa640 1 --2- 172.21.15.112:0/1865026568 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f3df0078b90 0x7f3df007b0a0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f3e00006fd0 tx=0x7f3e00008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:34.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.442+0000 7f3df97fa640 1 -- 172.21.15.112:0/1865026568 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3dfc084740 con 0x7f3e040d48a0 2024-09-21T00:28:34.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.617+0000 7f3e0b20e640 1 -- 172.21.15.112:0/1865026568 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3e04163580 con 0x7f3e040d48a0 2024-09-21T00:28:34.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.618+0000 7f3df97fa640 1 -- 172.21.15.112:0/1865026568 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) ==== 74+0+17029 (secure 0 0 0) 0x7f3dfc084040 con 0x7f3e040d48a0 2024-09-21T00:28:34.620 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:28:34.621 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":71,"fsid":"8b34692b-dc65-49b2-a08f-26f707b96ac5","created":"2024-09-21T00:25:22.788490+0000","modified":"2024-09-21T00:28:31.940059+0000","last_up_change":"2024-09-21T00:25:36.161835+0000","last_in_change":"2024-09-21T00:25:26.346500+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T00:25:37.105006+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-09-21T00:28:27.930288+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-09-21T00:28:29.255565+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9c575086-67d2-4de2-b01b-d44dcc21d604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6809","nonce":755760621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6811","nonce":755760621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6815","nonce":755760621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6813","nonce":755760621}]},"public_addr":"172.21.15.112:6809/755760621","cluster_addr":"172.21.15.112:6811/755760621","heartbeat_back_addr":"172.21.15.112:6815/755760621","heartbeat_front_addr":"172.21.15.112:6813/755760621","state":["exists","up"]},{"osd":1,"uuid":"a983c227-a3ce-4bdd-82f1-438ba7e3ecd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6801","nonce":344932592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6803","nonce":344932592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6807","nonce":344932592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6805","nonce":344932592}]},"public_addr":"172.21.15.112:6801/344932592","cluster_addr":"172.21.15.112:6803/344932592","heartbeat_back_addr":"172.21.15.112:6807/344932592","heartbeat_front_addr":"172.21.15.112:6805/344932592","state":["exists","up"]},{"osd":2,"uuid":"1193c1d8-bfff-4a57-8200-5f69bf1ea3af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6825","nonce":407933616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6827","nonce":407933616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6831","nonce":407933616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6829","nonce":407933616}]},"public_addr":"172.21.15.112:6825/407933616","cluster_addr":"172.21.15.112:6827/407933616","heartbeat_back_addr":"172.21.15.112:6831/407933616","heartbeat_front_addr":"172.21.15.112:6829/407933616","state":["exists","up"]},{"osd":3,"uuid":"af04330a-36ad-40cc-a8dd-fa259b221de5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6817","nonce":4120164806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6819","nonce":4120164806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6823","nonce":4120164806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6821","nonce":4120164806}]},"public_addr":"172.21.15.112:6817/4120164806","cluster_addr":"172.21.15.112:6819/4120164806","heartbeat_back_addr":"172.21.15.112:6823/4120164806","heartbeat_front_addr":"172.21.15.112:6821/4120164806","state":["exists","up"]},{"osd":4,"uuid":"5b893111-292c-4c5a-b184-53ad38c7199e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6803","nonce":522077560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6805","nonce":522077560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6809","nonce":522077560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6807","nonce":522077560}]},"public_addr":"172.21.15.187:6803/522077560","cluster_addr":"172.21.15.187:6805/522077560","heartbeat_back_addr":"172.21.15.187:6809/522077560","heartbeat_front_addr":"172.21.15.187:6807/522077560","state":["exists","up"]},{"osd":5,"uuid":"c0b8e011-45fe-4c5e-925e-6be2a7f1868a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6827","nonce":3593144721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6829","nonce":3593144721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6833","nonce":3593144721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6831","nonce":3593144721}]},"public_addr":"172.21.15.187:6827/3593144721","cluster_addr":"172.21.15.187:6829/3593144721","heartbeat_back_addr":"172.21.15.187:6833/3593144721","heartbeat_front_addr":"172.21.15.187:6831/3593144721","state":["exists","up"]},{"osd":6,"uuid":"87d2d650-7489-4593-98a3-a1f433ce8dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6811","nonce":1494387657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6813","nonce":1494387657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6817","nonce":1494387657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6815","nonce":1494387657}]},"public_addr":"172.21.15.187:6811/1494387657","cluster_addr":"172.21.15.187:6813/1494387657","heartbeat_back_addr":"172.21.15.187:6817/1494387657","heartbeat_front_addr":"172.21.15.187:6815/1494387657","state":["exists","up"]},{"osd":7,"uuid":"c6b4f8e2-a1ee-4438-aa37-67ff4b6ef1f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6819","nonce":1624699842}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6821","nonce":1624699842}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6825","nonce":1624699842}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6823","nonce":1624699842}]},"public_addr":"172.21.15.187:6819/1624699842","cluster_addr":"172.21.15.187:6821/1624699842","heartbeat_back_addr":"172.21.15.187:6825/1624699842","heartbeat_front_addr":"172.21.15.187:6823/1624699842","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.873162+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.400506+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.664317+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.159791+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.202630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.113212+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.384786+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.448441+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T00:28:34.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.621+0000 7f3e0b20e640 1 -- 172.21.15.112:0/1865026568 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f3df0078b90 msgr2=0x7f3df007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:34.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.621+0000 7f3e0b20e640 1 --2- 172.21.15.112:0/1865026568 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f3df0078b90 0x7f3df007b0a0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f3e00006fd0 tx=0x7f3e00008040 comp rx=0 tx=0).stop 2024-09-21T00:28:34.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.621+0000 7f3e0b20e640 1 -- 172.21.15.112:0/1865026568 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e040d48a0 msgr2=0x7f3e040cbb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:34.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.621+0000 7f3e0b20e640 1 --2- 172.21.15.112:0/1865026568 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e040d48a0 0x7f3e040cbb30 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f3dfc005fd0 tx=0x7f3dfc003f90 comp rx=0 tx=0).stop 2024-09-21T00:28:34.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.621+0000 7f3e0b20e640 1 -- 172.21.15.112:0/1865026568 shutdown_connections 2024-09-21T00:28:34.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.621+0000 7f3e0b20e640 1 --2- 172.21.15.112:0/1865026568 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f3df0078b90 0x7f3df007b0a0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:34.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.621+0000 7f3e0b20e640 1 --2- 172.21.15.112:0/1865026568 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e040d48a0 0x7f3e040cbb30 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:34.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.621+0000 7f3e0b20e640 1 --2- 172.21.15.112:0/1865026568 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3e040d3880 0x7f3e040c9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:34.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.621+0000 7f3e0b20e640 1 --2- 172.21.15.112:0/1865026568 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3e040d01b0 0x7f3e040ce1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:34.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.621+0000 7f3e0b20e640 1 -- 172.21.15.112:0/1865026568 >> 172.21.15.112:0/1865026568 conn(0x7f3e0408bfe0 msgr2=0x7f3e040b15f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:34.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.621+0000 7f3e0b20e640 1 -- 172.21.15.112:0/1865026568 shutdown_connections 2024-09-21T00:28:34.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.621+0000 7f3e0b20e640 1 -- 172.21.15.112:0/1865026568 wait complete. 2024-09-21T00:28:34.634 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T00:28:34.634 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T00:28:34.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.743+0000 7f37a1101640 1 -- 172.21.15.112:0/2283255450 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f379c0ce510 msgr2=0x7f379c153920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:34.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.743+0000 7f37a1101640 1 --2- 172.21.15.112:0/2283255450 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f379c0ce510 0x7f379c153920 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f3790005fd0 tx=0x7f379002a5a0 comp rx=0 tx=0).stop 2024-09-21T00:28:34.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.744+0000 7f37a1101640 1 -- 172.21.15.112:0/2283255450 shutdown_connections 2024-09-21T00:28:34.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.744+0000 7f37a1101640 1 --2- 172.21.15.112:0/2283255450 >> v2:172.21.15.187:3300/0 conn(0x7f379c0b3b30 0x7f379c0b5290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:34.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.744+0000 7f37a1101640 1 --2- 172.21.15.112:0/2283255450 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f379c0ce510 0x7f379c153920 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:34.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.744+0000 7f37a1101640 1 -- 172.21.15.112:0/2283255450 >> 172.21.15.112:0/2283255450 conn(0x7f379c0cd5c0 msgr2=0x7f379c0cd9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:34.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.744+0000 7f37a1101640 1 -- 172.21.15.112:0/2283255450 shutdown_connections 2024-09-21T00:28:34.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.744+0000 7f37a1101640 1 -- 172.21.15.112:0/2283255450 wait complete. 2024-09-21T00:28:34.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.746+0000 7f37a1101640 1 Processor -- start 2024-09-21T00:28:34.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.746+0000 7f37a1101640 1 -- start start 2024-09-21T00:28:34.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.747+0000 7f37a1101640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f379c0b3b30 0x7f379c155130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:34.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.747+0000 7f37a1101640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f379c0ce510 0x7f379c155670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:34.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.747+0000 7f37a1101640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f379c15c330 0x7f379c15c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:34.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.747+0000 7f37a1101640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f379c15d6d0 con 0x7f379c15c330 2024-09-21T00:28:34.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.747+0000 7f37a1101640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f379c15d550 con 0x7f379c0ce510 2024-09-21T00:28:34.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.747+0000 7f37a1101640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f379c15d850 con 0x7f379c0b3b30 2024-09-21T00:28:34.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.747+0000 7f379b59e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f379c15c330 0x7f379c15c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:34.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.747+0000 7f379b59e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f379c15c330 0x7f379c15c740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:43106/0 (socket says 172.21.15.112:43106) 2024-09-21T00:28:34.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.747+0000 7f379b59e640 1 -- 172.21.15.112:0/1468877984 learned_addr learned my addr 172.21.15.112:0/1468877984 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:28:34.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.747+0000 7f379ad9d640 1 --2- 172.21.15.112:0/1468877984 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f379c0b3b30 0x7f379c155130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:34.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.747+0000 7f379a59c640 1 --2- 172.21.15.112:0/1468877984 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f379c0ce510 0x7f379c155670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:34.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.747+0000 7f379b59e640 1 -- 172.21.15.112:0/1468877984 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f379c0b3b30 msgr2=0x7f379c155130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:34.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.747+0000 7f379b59e640 1 --2- 172.21.15.112:0/1468877984 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f379c0b3b30 0x7f379c155130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:34.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.747+0000 7f379b59e640 1 -- 172.21.15.112:0/1468877984 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f379c0ce510 msgr2=0x7f379c155670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:34.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.747+0000 7f379b59e640 1 --2- 172.21.15.112:0/1468877984 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f379c0ce510 0x7f379c155670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:34.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.747+0000 7f379b59e640 1 -- 172.21.15.112:0/1468877984 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f379c155ed0 con 0x7f379c15c330 2024-09-21T00:28:34.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.748+0000 7f379b59e640 1 --2- 172.21.15.112:0/1468877984 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f379c15c330 0x7f379c15c740 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f379400d690 tx=0x7f379400db50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:34.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.748+0000 7f37837fe640 1 -- 172.21.15.112:0/1468877984 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3794017070 con 0x7f379c15c330 2024-09-21T00:28:34.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.748+0000 7f37837fe640 1 -- 172.21.15.112:0/1468877984 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3794005310 con 0x7f379c15c330 2024-09-21T00:28:34.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.748+0000 7f37837fe640 1 -- 172.21.15.112:0/1468877984 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3794009820 con 0x7f379c15c330 2024-09-21T00:28:34.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.748+0000 7f37a1101640 1 -- 172.21.15.112:0/1468877984 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f379c1561b0 con 0x7f379c15c330 2024-09-21T00:28:34.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.748+0000 7f37a1101640 1 -- 172.21.15.112:0/1468877984 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f379c156770 con 0x7f379c15c330 2024-09-21T00:28:34.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.749+0000 7f37a1101640 1 -- 172.21.15.112:0/1468877984 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f379c15dd40 con 0x7f379c15c330 2024-09-21T00:28:34.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.750+0000 7f37837fe640 1 -- 172.21.15.112:0/1468877984 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f37940099c0 con 0x7f379c15c330 2024-09-21T00:28:34.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.750+0000 7f37837fe640 1 --2- 172.21.15.112:0/1468877984 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f3788078b90 0x7f378807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:34.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.750+0000 7f37837fe640 1 -- 172.21.15.112:0/1468877984 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6101+0+0 (secure 0 0 0) 0x7f379409aed0 con 0x7f379c15c330 2024-09-21T00:28:34.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.750+0000 7f379ad9d640 1 --2- 172.21.15.112:0/1468877984 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f3788078b90 0x7f378807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:34.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.751+0000 7f379ad9d640 1 --2- 172.21.15.112:0/1468877984 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f3788078b90 0x7f378807b0a0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f378c001e20 tx=0x7f378c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:34.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.755+0000 7f37837fe640 1 -- 172.21.15.112:0/1468877984 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3794062b30 con 0x7f379c15c330 2024-09-21T00:28:34.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.929+0000 7f37a1101640 1 -- 172.21.15.112:0/1468877984 --> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f379c155c60 con 0x7f3788078b90 2024-09-21T00:28:34.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.934+0000 7f37837fe640 1 -- 172.21.15.112:0/1468877984 <== mgr.4103 v2:172.21.15.187:6800/2921522906 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+250563 (secure 0 0 0) 0x7f379c155c60 con 0x7f3788078b90 2024-09-21T00:28:34.938 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:28:34.939 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2024-09-21T00:28:34.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.937+0000 7f37a1101640 1 -- 172.21.15.112:0/1468877984 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f3788078b90 msgr2=0x7f378807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:34.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.937+0000 7f37a1101640 1 --2- 172.21.15.112:0/1468877984 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f3788078b90 0x7f378807b0a0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f378c001e20 tx=0x7f378c009040 comp rx=0 tx=0).stop 2024-09-21T00:28:34.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.937+0000 7f37a1101640 1 -- 172.21.15.112:0/1468877984 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f379c15c330 msgr2=0x7f379c15c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:34.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.937+0000 7f37a1101640 1 --2- 172.21.15.112:0/1468877984 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f379c15c330 0x7f379c15c740 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f379400d690 tx=0x7f379400db50 comp rx=0 tx=0).stop 2024-09-21T00:28:34.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.937+0000 7f37a1101640 1 -- 172.21.15.112:0/1468877984 shutdown_connections 2024-09-21T00:28:34.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.937+0000 7f37a1101640 1 --2- 172.21.15.112:0/1468877984 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f3788078b90 0x7f378807b0a0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:34.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.937+0000 7f37a1101640 1 --2- 172.21.15.112:0/1468877984 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f379c15c330 0x7f379c15c740 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:34.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.937+0000 7f37a1101640 1 --2- 172.21.15.112:0/1468877984 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f379c0ce510 0x7f379c155670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:34.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.937+0000 7f37a1101640 1 --2- 172.21.15.112:0/1468877984 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f379c0b3b30 0x7f379c155130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:34.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.937+0000 7f37a1101640 1 -- 172.21.15.112:0/1468877984 >> 172.21.15.112:0/1468877984 conn(0x7f379c0cd5c0 msgr2=0x7f379c0ccd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:34.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.937+0000 7f37a1101640 1 -- 172.21.15.112:0/1468877984 shutdown_connections 2024-09-21T00:28:34.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:34.937+0000 7f37a1101640 1 -- 172.21.15.112:0/1468877984 wait complete. 2024-09-21T00:28:34.950 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":154,"stamp":"2024-09-21T00:28:33.131866+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":176,"acting":176,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":244804,"kb_used_data":7744,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754729916,"statfs":{"total":773094113280,"available":772843433984,"internally_reserved":0,"allocated":7929856,"data_stored":4167330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":242535985},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.212734"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.958377+0000","last_change":"2024-09-21T00:28:30.954953+0000","last_active":"2024-09-21T00:28:30.958377+0000","last_peered":"2024-09-21T00:28:30.958377+0000","last_clean":"2024-09-21T00:28:30.958377+0000","last_became_active":"2024-09-21T00:28:30.954631+0000","last_became_peered":"2024-09-21T00:28:30.954631+0000","last_unstale":"2024-09-21T00:28:30.958377+0000","last_undegraded":"2024-09-21T00:28:30.958377+0000","last_fullsized":"2024-09-21T00:28:30.958377+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_clean_scrub_stamp":"2024-09-21T00:28:29.918310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:07:18.418422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.957593+0000","last_change":"2024-09-21T00:28:30.951440+0000","last_active":"2024-09-21T00:28:30.957593+0000","last_peered":"2024-09-21T00:28:30.957593+0000","last_clean":"2024-09-21T00:28:30.957593+0000","last_became_active":"2024-09-21T00:28:30.951133+0000","last_became_peered":"2024-09-21T00:28:30.951133+0000","last_unstale":"2024-09-21T00:28:30.957593+0000","last_undegraded":"2024-09-21T00:28:30.957593+0000","last_fullsized":"2024-09-21T00:28:30.957593+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_clean_scrub_stamp":"2024-09-21T00:28:29.918310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:52:55.340063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:28:29.918310+0000","last_change":"2024-09-21T00:28:29.918310+0000","last_active":"2024-09-21T00:28:29.918310+0000","last_peered":"2024-09-21T00:28:29.918310+0000","last_clean":"2024-09-21T00:28:29.918310+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:28:29.918310+0000","last_undegraded":"2024-09-21T00:28:29.918310+0000","last_fullsized":"2024-09-21T00:28:29.918310+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_clean_scrub_stamp":"2024-09-21T00:28:29.918310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.953198+0000","last_change":"2024-09-21T00:28:30.950561+0000","last_active":"2024-09-21T00:28:30.953198+0000","last_peered":"2024-09-21T00:28:30.953198+0000","last_clean":"2024-09-21T00:28:30.953198+0000","last_became_active":"2024-09-21T00:28:30.949672+0000","last_became_peered":"2024-09-21T00:28:30.949672+0000","last_unstale":"2024-09-21T00:28:30.953198+0000","last_undegraded":"2024-09-21T00:28:30.953198+0000","last_fullsized":"2024-09-21T00:28:30.953198+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_clean_scrub_stamp":"2024-09-21T00:28:29.918310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:25:55.834586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.960469+0000","last_change":"2024-09-21T00:28:30.953634+0000","last_active":"2024-09-21T00:28:30.960469+0000","last_peered":"2024-09-21T00:28:30.960469+0000","last_clean":"2024-09-21T00:28:30.960469+0000","last_became_active":"2024-09-21T00:28:30.953327+0000","last_became_peered":"2024-09-21T00:28:30.953327+0000","last_unstale":"2024-09-21T00:28:30.960469+0000","last_undegraded":"2024-09-21T00:28:30.960469+0000","last_fullsized":"2024-09-21T00:28:30.960469+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_clean_scrub_stamp":"2024-09-21T00:28:29.918310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:58:27.359494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.961560+0000","last_change":"2024-09-21T00:28:30.958525+0000","last_active":"2024-09-21T00:28:30.961560+0000","last_peered":"2024-09-21T00:28:30.961560+0000","last_clean":"2024-09-21T00:28:30.961560+0000","last_became_active":"2024-09-21T00:28:30.958195+0000","last_became_peered":"2024-09-21T00:28:30.958195+0000","last_unstale":"2024-09-21T00:28:30.961560+0000","last_undegraded":"2024-09-21T00:28:30.961560+0000","last_fullsized":"2024-09-21T00:28:30.961560+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_clean_scrub_stamp":"2024-09-21T00:28:29.918310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:41:07.332812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.965339+0000","last_change":"2024-09-21T00:28:30.961682+0000","last_active":"2024-09-21T00:28:30.965339+0000","last_peered":"2024-09-21T00:28:30.965339+0000","last_clean":"2024-09-21T00:28:30.965339+0000","last_became_active":"2024-09-21T00:28:30.961374+0000","last_became_peered":"2024-09-21T00:28:30.961374+0000","last_unstale":"2024-09-21T00:28:30.965339+0000","last_undegraded":"2024-09-21T00:28:30.965339+0000","last_fullsized":"2024-09-21T00:28:30.965339+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_clean_scrub_stamp":"2024-09-21T00:28:29.918310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:31:52.439535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.960759+0000","last_change":"2024-09-21T00:28:30.952916+0000","last_active":"2024-09-21T00:28:30.960759+0000","last_peered":"2024-09-21T00:28:30.960759+0000","last_clean":"2024-09-21T00:28:30.960759+0000","last_became_active":"2024-09-21T00:28:30.952630+0000","last_became_peered":"2024-09-21T00:28:30.952630+0000","last_unstale":"2024-09-21T00:28:30.960759+0000","last_undegraded":"2024-09-21T00:28:30.960759+0000","last_fullsized":"2024-09-21T00:28:30.960759+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_clean_scrub_stamp":"2024-09-21T00:28:29.918310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:14:43.923166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.956690+0000","last_change":"2024-09-21T00:28:30.947905+0000","last_active":"2024-09-21T00:28:30.956690+0000","last_peered":"2024-09-21T00:28:30.956690+0000","last_clean":"2024-09-21T00:28:30.956690+0000","last_became_active":"2024-09-21T00:28:30.947489+0000","last_became_peered":"2024-09-21T00:28:30.947489+0000","last_unstale":"2024-09-21T00:28:30.956690+0000","last_undegraded":"2024-09-21T00:28:30.956690+0000","last_fullsized":"2024-09-21T00:28:30.956690+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_clean_scrub_stamp":"2024-09-21T00:28:29.918310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:55:59.100753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.953224+0000","last_change":"2024-09-21T00:28:30.948702+0000","last_active":"2024-09-21T00:28:30.953224+0000","last_peered":"2024-09-21T00:28:30.953224+0000","last_clean":"2024-09-21T00:28:30.953224+0000","last_became_active":"2024-09-21T00:28:30.948385+0000","last_became_peered":"2024-09-21T00:28:30.948385+0000","last_unstale":"2024-09-21T00:28:30.953224+0000","last_undegraded":"2024-09-21T00:28:30.953224+0000","last_fullsized":"2024-09-21T00:28:30.953224+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_clean_scrub_stamp":"2024-09-21T00:28:29.918310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:58:17.081795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.964635+0000","last_change":"2024-09-21T00:28:30.961901+0000","last_active":"2024-09-21T00:28:30.964635+0000","last_peered":"2024-09-21T00:28:30.964635+0000","last_clean":"2024-09-21T00:28:30.964635+0000","last_became_active":"2024-09-21T00:28:30.961589+0000","last_became_peered":"2024-09-21T00:28:30.961589+0000","last_unstale":"2024-09-21T00:28:30.964635+0000","last_undegraded":"2024-09-21T00:28:30.964635+0000","last_fullsized":"2024-09-21T00:28:30.964635+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_clean_scrub_stamp":"2024-09-21T00:28:29.918310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:32:46.627248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.947262+0000","last_change":"2024-09-21T00:28:30.944871+0000","last_active":"2024-09-21T00:28:30.947262+0000","last_peered":"2024-09-21T00:28:30.947262+0000","last_clean":"2024-09-21T00:28:30.947262+0000","last_became_active":"2024-09-21T00:28:30.944587+0000","last_became_peered":"2024-09-21T00:28:30.944587+0000","last_unstale":"2024-09-21T00:28:30.947262+0000","last_undegraded":"2024-09-21T00:28:30.947262+0000","last_fullsized":"2024-09-21T00:28:30.947262+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_clean_scrub_stamp":"2024-09-21T00:28:29.918310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:25:52.076873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.960190+0000","last_change":"2024-09-21T00:28:30.952079+0000","last_active":"2024-09-21T00:28:30.960190+0000","last_peered":"2024-09-21T00:28:30.960190+0000","last_clean":"2024-09-21T00:28:30.960190+0000","last_became_active":"2024-09-21T00:28:30.951792+0000","last_became_peered":"2024-09-21T00:28:30.951792+0000","last_unstale":"2024-09-21T00:28:30.960190+0000","last_undegraded":"2024-09-21T00:28:30.960190+0000","last_fullsized":"2024-09-21T00:28:30.960190+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_clean_scrub_stamp":"2024-09-21T00:28:29.918310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:58:45.340669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.960540+0000","last_change":"2024-09-21T00:28:30.955689+0000","last_active":"2024-09-21T00:28:30.960540+0000","last_peered":"2024-09-21T00:28:30.960540+0000","last_clean":"2024-09-21T00:28:30.960540+0000","last_became_active":"2024-09-21T00:28:30.955385+0000","last_became_peered":"2024-09-21T00:28:30.955385+0000","last_unstale":"2024-09-21T00:28:30.960540+0000","last_undegraded":"2024-09-21T00:28:30.960540+0000","last_fullsized":"2024-09-21T00:28:30.960540+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_clean_scrub_stamp":"2024-09-21T00:28:29.918310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:52:43.572069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.947068+0000","last_change":"2024-09-21T00:28:30.944302+0000","last_active":"2024-09-21T00:28:30.947068+0000","last_peered":"2024-09-21T00:28:30.947068+0000","last_clean":"2024-09-21T00:28:30.947068+0000","last_became_active":"2024-09-21T00:28:30.943996+0000","last_became_peered":"2024-09-21T00:28:30.943996+0000","last_unstale":"2024-09-21T00:28:30.947068+0000","last_undegraded":"2024-09-21T00:28:30.947068+0000","last_fullsized":"2024-09-21T00:28:30.947068+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_clean_scrub_stamp":"2024-09-21T00:28:29.918310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T00:42:18.223655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.954559+0000","last_change":"2024-09-21T00:28:30.948888+0000","last_active":"2024-09-21T00:28:30.954559+0000","last_peered":"2024-09-21T00:28:30.954559+0000","last_clean":"2024-09-21T00:28:30.954559+0000","last_became_active":"2024-09-21T00:28:30.948412+0000","last_became_peered":"2024-09-21T00:28:30.948412+0000","last_unstale":"2024-09-21T00:28:30.954559+0000","last_undegraded":"2024-09-21T00:28:30.954559+0000","last_fullsized":"2024-09-21T00:28:30.954559+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_clean_scrub_stamp":"2024-09-21T00:28:29.918310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:52:55.340063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.954481+0000","last_change":"2024-09-21T00:28:30.950609+0000","last_active":"2024-09-21T00:28:30.954481+0000","last_peered":"2024-09-21T00:28:30.954481+0000","last_clean":"2024-09-21T00:28:30.954481+0000","last_became_active":"2024-09-21T00:28:30.950260+0000","last_became_peered":"2024-09-21T00:28:30.950260+0000","last_unstale":"2024-09-21T00:28:30.954481+0000","last_undegraded":"2024-09-21T00:28:30.954481+0000","last_fullsized":"2024-09-21T00:28:30.954481+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_clean_scrub_stamp":"2024-09-21T00:28:29.918310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:46:00.655896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:28:29.918310+0000","last_change":"2024-09-21T00:28:29.918310+0000","last_active":"2024-09-21T00:28:29.918310+0000","last_peered":"2024-09-21T00:28:29.918310+0000","last_clean":"2024-09-21T00:28:29.918310+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:28:29.918310+0000","last_undegraded":"2024-09-21T00:28:29.918310+0000","last_fullsized":"2024-09-21T00:28:29.918310+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_clean_scrub_stamp":"2024-09-21T00:28:29.918310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.960578+0000","last_change":"2024-09-21T00:28:30.957260+0000","last_active":"2024-09-21T00:28:30.960578+0000","last_peered":"2024-09-21T00:28:30.960578+0000","last_clean":"2024-09-21T00:28:30.960578+0000","last_became_active":"2024-09-21T00:28:30.956956+0000","last_became_peered":"2024-09-21T00:28:30.956956+0000","last_unstale":"2024-09-21T00:28:30.960578+0000","last_undegraded":"2024-09-21T00:28:30.960578+0000","last_fullsized":"2024-09-21T00:28:30.960578+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_clean_scrub_stamp":"2024-09-21T00:28:29.918310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:07:18.418422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.960233+0000","last_change":"2024-09-21T00:28:30.952315+0000","last_active":"2024-09-21T00:28:30.960233+0000","last_peered":"2024-09-21T00:28:30.960233+0000","last_clean":"2024-09-21T00:28:30.960233+0000","last_became_active":"2024-09-21T00:28:30.952007+0000","last_became_peered":"2024-09-21T00:28:30.952007+0000","last_unstale":"2024-09-21T00:28:30.960233+0000","last_undegraded":"2024-09-21T00:28:30.960233+0000","last_fullsized":"2024-09-21T00:28:30.960233+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_clean_scrub_stamp":"2024-09-21T00:28:29.918310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:49:46.951442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.958307+0000","last_change":"2024-09-21T00:28:30.947677+0000","last_active":"2024-09-21T00:28:30.958307+0000","last_peered":"2024-09-21T00:28:30.958307+0000","last_clean":"2024-09-21T00:28:30.958307+0000","last_became_active":"2024-09-21T00:28:30.946020+0000","last_became_peered":"2024-09-21T00:28:30.946020+0000","last_unstale":"2024-09-21T00:28:30.958307+0000","last_undegraded":"2024-09-21T00:28:30.958307+0000","last_fullsized":"2024-09-21T00:28:30.958307+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_clean_scrub_stamp":"2024-09-21T00:28:29.918310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:52:45.960884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.960172+0000","last_change":"2024-09-21T00:28:30.951954+0000","last_active":"2024-09-21T00:28:30.960172+0000","last_peered":"2024-09-21T00:28:30.960172+0000","last_clean":"2024-09-21T00:28:30.960172+0000","last_became_active":"2024-09-21T00:28:30.951492+0000","last_became_peered":"2024-09-21T00:28:30.951492+0000","last_unstale":"2024-09-21T00:28:30.960172+0000","last_undegraded":"2024-09-21T00:28:30.960172+0000","last_fullsized":"2024-09-21T00:28:30.960172+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_clean_scrub_stamp":"2024-09-21T00:28:29.918310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:48:20.244405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.957004+0000","last_change":"2024-09-21T00:28:30.945517+0000","last_active":"2024-09-21T00:28:30.957004+0000","last_peered":"2024-09-21T00:28:30.957004+0000","last_clean":"2024-09-21T00:28:30.957004+0000","last_became_active":"2024-09-21T00:28:30.945209+0000","last_became_peered":"2024-09-21T00:28:30.945209+0000","last_unstale":"2024-09-21T00:28:30.957004+0000","last_undegraded":"2024-09-21T00:28:30.957004+0000","last_fullsized":"2024-09-21T00:28:30.957004+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_clean_scrub_stamp":"2024-09-21T00:28:29.918310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:26:08.351369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.954477+0000","last_change":"2024-09-21T00:28:30.947336+0000","last_active":"2024-09-21T00:28:30.954477+0000","last_peered":"2024-09-21T00:28:30.954477+0000","last_clean":"2024-09-21T00:28:30.954477+0000","last_became_active":"2024-09-21T00:28:30.946783+0000","last_became_peered":"2024-09-21T00:28:30.946783+0000","last_unstale":"2024-09-21T00:28:30.954477+0000","last_undegraded":"2024-09-21T00:28:30.954477+0000","last_fullsized":"2024-09-21T00:28:30.954477+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_clean_scrub_stamp":"2024-09-21T00:28:29.918310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:46:23.384000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.954438+0000","last_change":"2024-09-21T00:28:30.947086+0000","last_active":"2024-09-21T00:28:30.954438+0000","last_peered":"2024-09-21T00:28:30.954438+0000","last_clean":"2024-09-21T00:28:30.954438+0000","last_became_active":"2024-09-21T00:28:30.946737+0000","last_became_peered":"2024-09-21T00:28:30.946737+0000","last_unstale":"2024-09-21T00:28:30.954438+0000","last_undegraded":"2024-09-21T00:28:30.954438+0000","last_fullsized":"2024-09-21T00:28:30.954438+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_clean_scrub_stamp":"2024-09-21T00:28:29.918310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:15:13.025831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.947053+0000","last_change":"2024-09-21T00:28:30.944085+0000","last_active":"2024-09-21T00:28:30.947053+0000","last_peered":"2024-09-21T00:28:30.947053+0000","last_clean":"2024-09-21T00:28:30.947053+0000","last_became_active":"2024-09-21T00:28:30.943774+0000","last_became_peered":"2024-09-21T00:28:30.943774+0000","last_unstale":"2024-09-21T00:28:30.947053+0000","last_undegraded":"2024-09-21T00:28:30.947053+0000","last_fullsized":"2024-09-21T00:28:30.947053+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_clean_scrub_stamp":"2024-09-21T00:28:29.918310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:11:52.856844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.954406+0000","last_change":"2024-09-21T00:28:30.948912+0000","last_active":"2024-09-21T00:28:30.954406+0000","last_peered":"2024-09-21T00:28:30.954406+0000","last_clean":"2024-09-21T00:28:30.954406+0000","last_became_active":"2024-09-21T00:28:30.948572+0000","last_became_peered":"2024-09-21T00:28:30.948572+0000","last_unstale":"2024-09-21T00:28:30.954406+0000","last_undegraded":"2024-09-21T00:28:30.954406+0000","last_fullsized":"2024-09-21T00:28:30.954406+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_clean_scrub_stamp":"2024-09-21T00:28:29.918310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:45:08.521920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.958867+0000","last_change":"2024-09-21T00:28:30.951013+0000","last_active":"2024-09-21T00:28:30.958867+0000","last_peered":"2024-09-21T00:28:30.958867+0000","last_clean":"2024-09-21T00:28:30.958867+0000","last_became_active":"2024-09-21T00:28:30.950631+0000","last_became_peered":"2024-09-21T00:28:30.950631+0000","last_unstale":"2024-09-21T00:28:30.958867+0000","last_undegraded":"2024-09-21T00:28:30.958867+0000","last_fullsized":"2024-09-21T00:28:30.958867+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_clean_scrub_stamp":"2024-09-21T00:28:29.918310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:50:35.184359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.954066+0000","last_change":"2024-09-21T00:28:30.948178+0000","last_active":"2024-09-21T00:28:30.954066+0000","last_peered":"2024-09-21T00:28:30.954066+0000","last_clean":"2024-09-21T00:28:30.954066+0000","last_became_active":"2024-09-21T00:28:30.947853+0000","last_became_peered":"2024-09-21T00:28:30.947853+0000","last_unstale":"2024-09-21T00:28:30.954066+0000","last_undegraded":"2024-09-21T00:28:30.954066+0000","last_fullsized":"2024-09-21T00:28:30.954066+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_clean_scrub_stamp":"2024-09-21T00:28:29.918310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:08:59.115916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.956455+0000","last_change":"2024-09-21T00:28:30.953941+0000","last_active":"2024-09-21T00:28:30.956455+0000","last_peered":"2024-09-21T00:28:30.956455+0000","last_clean":"2024-09-21T00:28:30.956455+0000","last_became_active":"2024-09-21T00:28:30.953576+0000","last_became_peered":"2024-09-21T00:28:30.953576+0000","last_unstale":"2024-09-21T00:28:30.956455+0000","last_undegraded":"2024-09-21T00:28:30.956455+0000","last_fullsized":"2024-09-21T00:28:30.956455+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_clean_scrub_stamp":"2024-09-21T00:28:29.918310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:32:46.627248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:28:29.918310+0000","last_change":"2024-09-21T00:28:29.918310+0000","last_active":"2024-09-21T00:28:29.918310+0000","last_peered":"2024-09-21T00:28:29.918310+0000","last_clean":"2024-09-21T00:28:29.918310+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:28:29.918310+0000","last_undegraded":"2024-09-21T00:28:29.918310+0000","last_fullsized":"2024-09-21T00:28:29.918310+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_clean_scrub_stamp":"2024-09-21T00:28:29.918310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.954618+0000","last_change":"2024-09-21T00:28:30.949895+0000","last_active":"2024-09-21T00:28:30.954618+0000","last_peered":"2024-09-21T00:28:30.954618+0000","last_clean":"2024-09-21T00:28:30.954618+0000","last_became_active":"2024-09-21T00:28:30.949595+0000","last_became_peered":"2024-09-21T00:28:30.949595+0000","last_unstale":"2024-09-21T00:28:30.954618+0000","last_undegraded":"2024-09-21T00:28:30.954618+0000","last_fullsized":"2024-09-21T00:28:30.954618+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_clean_scrub_stamp":"2024-09-21T00:28:29.918310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:56:01.061375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:28:29.918310+0000","last_change":"2024-09-21T00:28:29.918310+0000","last_active":"2024-09-21T00:28:29.918310+0000","last_peered":"2024-09-21T00:28:29.918310+0000","last_clean":"2024-09-21T00:28:29.918310+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:28:29.918310+0000","last_undegraded":"2024-09-21T00:28:29.918310+0000","last_fullsized":"2024-09-21T00:28:29.918310+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_clean_scrub_stamp":"2024-09-21T00:28:29.918310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1f","version":"70'4","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.966527+0000","last_change":"2024-09-21T00:28:29.943891+0000","last_active":"2024-09-21T00:28:30.966527+0000","last_peered":"2024-09-21T00:28:30.966527+0000","last_clean":"2024-09-21T00:28:30.966527+0000","last_became_active":"2024-09-21T00:28:29.943594+0000","last_became_peered":"2024-09-21T00:28:29.943594+0000","last_unstale":"2024-09-21T00:28:30.966527+0000","last_undegraded":"2024-09-21T00:28:30.966527+0000","last_fullsized":"2024-09-21T00:28:30.966527+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:28.914450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:28.914450+0000","last_clean_scrub_stamp":"2024-09-21T00:28:28.914450+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:35:07.915069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.943167+0000","last_change":"2024-09-21T00:28:30.939893+0000","last_active":"2024-09-21T00:28:30.943167+0000","last_peered":"2024-09-21T00:28:30.943167+0000","last_clean":"2024-09-21T00:28:30.943167+0000","last_became_active":"2024-09-21T00:28:30.939601+0000","last_became_peered":"2024-09-21T00:28:30.939601+0000","last_unstale":"2024-09-21T00:28:30.943167+0000","last_undegraded":"2024-09-21T00:28:30.943167+0000","last_fullsized":"2024-09-21T00:28:30.943167+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_clean_scrub_stamp":"2024-09-21T00:28:29.918310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:33:41.972645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"70'1","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.959405+0000","last_change":"2024-09-21T00:28:29.948845+0000","last_active":"2024-09-21T00:28:30.959405+0000","last_peered":"2024-09-21T00:28:30.959405+0000","last_clean":"2024-09-21T00:28:30.959405+0000","last_became_active":"2024-09-21T00:28:29.948328+0000","last_became_peered":"2024-09-21T00:28:29.948328+0000","last_unstale":"2024-09-21T00:28:30.959405+0000","last_undegraded":"2024-09-21T00:28:30.959405+0000","last_fullsized":"2024-09-21T00:28:30.959405+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:28.914450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:28.914450+0000","last_clean_scrub_stamp":"2024-09-21T00:28:28.914450+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T00:54:42.550514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.956826+0000","last_change":"2024-09-21T00:28:30.952712+0000","last_active":"2024-09-21T00:28:30.956826+0000","last_peered":"2024-09-21T00:28:30.956826+0000","last_clean":"2024-09-21T00:28:30.956826+0000","last_became_active":"2024-09-21T00:28:30.952085+0000","last_became_peered":"2024-09-21T00:28:30.952085+0000","last_unstale":"2024-09-21T00:28:30.956826+0000","last_undegraded":"2024-09-21T00:28:30.956826+0000","last_fullsized":"2024-09-21T00:28:30.956826+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_clean_scrub_stamp":"2024-09-21T00:28:29.918310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:08:59.826700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"70'2","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.958755+0000","last_change":"2024-09-21T00:28:29.946568+0000","last_active":"2024-09-21T00:28:30.958755+0000","last_peered":"2024-09-21T00:28:30.958755+0000","last_clean":"2024-09-21T00:28:30.958755+0000","last_became_active":"2024-09-21T00:28:29.945758+0000","last_became_peered":"2024-09-21T00:28:29.945758+0000","last_unstale":"2024-09-21T00:28:30.958755+0000","last_undegraded":"2024-09-21T00:28:30.958755+0000","last_fullsized":"2024-09-21T00:28:30.958755+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:28.914450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:28.914450+0000","last_clean_scrub_stamp":"2024-09-21T00:28:28.914450+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:07:58.312580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.955625+0000","last_change":"2024-09-21T00:28:30.944552+0000","last_active":"2024-09-21T00:28:30.955625+0000","last_peered":"2024-09-21T00:28:30.955625+0000","last_clean":"2024-09-21T00:28:30.955625+0000","last_became_active":"2024-09-21T00:28:30.944241+0000","last_became_peered":"2024-09-21T00:28:30.944241+0000","last_unstale":"2024-09-21T00:28:30.955625+0000","last_undegraded":"2024-09-21T00:28:30.955625+0000","last_fullsized":"2024-09-21T00:28:30.955625+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_clean_scrub_stamp":"2024-09-21T00:28:29.918310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:40:55.852394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"70'2","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.960450+0000","last_change":"2024-09-21T00:28:29.945862+0000","last_active":"2024-09-21T00:28:30.960450+0000","last_peered":"2024-09-21T00:28:30.960450+0000","last_clean":"2024-09-21T00:28:30.960450+0000","last_became_active":"2024-09-21T00:28:29.945557+0000","last_became_peered":"2024-09-21T00:28:29.945557+0000","last_unstale":"2024-09-21T00:28:30.960450+0000","last_undegraded":"2024-09-21T00:28:30.960450+0000","last_fullsized":"2024-09-21T00:28:30.960450+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:28.914450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:28.914450+0000","last_clean_scrub_stamp":"2024-09-21T00:28:28.914450+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:37:13.568386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.957513+0000","last_change":"2024-09-21T00:28:30.953402+0000","last_active":"2024-09-21T00:28:30.957513+0000","last_peered":"2024-09-21T00:28:30.957513+0000","last_clean":"2024-09-21T00:28:30.957513+0000","last_became_active":"2024-09-21T00:28:30.953047+0000","last_became_peered":"2024-09-21T00:28:30.953047+0000","last_unstale":"2024-09-21T00:28:30.957513+0000","last_undegraded":"2024-09-21T00:28:30.957513+0000","last_fullsized":"2024-09-21T00:28:30.957513+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_clean_scrub_stamp":"2024-09-21T00:28:29.918310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:45:08.521920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"70'1","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.962274+0000","last_change":"2024-09-21T00:28:29.938518+0000","last_active":"2024-09-21T00:28:30.962274+0000","last_peered":"2024-09-21T00:28:30.962274+0000","last_clean":"2024-09-21T00:28:30.962274+0000","last_became_active":"2024-09-21T00:28:29.938155+0000","last_became_peered":"2024-09-21T00:28:29.938155+0000","last_unstale":"2024-09-21T00:28:30.962274+0000","last_undegraded":"2024-09-21T00:28:30.962274+0000","last_fullsized":"2024-09-21T00:28:30.962274+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:28.914450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:28.914450+0000","last_clean_scrub_stamp":"2024-09-21T00:28:28.914450+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:08:56.500914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.964599+0000","last_change":"2024-09-21T00:28:30.961806+0000","last_active":"2024-09-21T00:28:30.964599+0000","last_peered":"2024-09-21T00:28:30.964599+0000","last_clean":"2024-09-21T00:28:30.964599+0000","last_became_active":"2024-09-21T00:28:30.961157+0000","last_became_peered":"2024-09-21T00:28:30.961157+0000","last_unstale":"2024-09-21T00:28:30.964599+0000","last_undegraded":"2024-09-21T00:28:30.964599+0000","last_fullsized":"2024-09-21T00:28:30.964599+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_clean_scrub_stamp":"2024-09-21T00:28:29.918310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:52:43.572069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"70'1","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.958773+0000","last_change":"2024-09-21T00:28:29.944301+0000","last_active":"2024-09-21T00:28:30.958773+0000","last_peered":"2024-09-21T00:28:30.958773+0000","last_clean":"2024-09-21T00:28:30.958773+0000","last_became_active":"2024-09-21T00:28:29.943978+0000","last_became_peered":"2024-09-21T00:28:29.943978+0000","last_unstale":"2024-09-21T00:28:30.958773+0000","last_undegraded":"2024-09-21T00:28:30.958773+0000","last_fullsized":"2024-09-21T00:28:30.958773+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:28.914450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:28.914450+0000","last_clean_scrub_stamp":"2024-09-21T00:28:28.914450+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:47:51.557791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.956979+0000","last_change":"2024-09-21T00:28:30.947405+0000","last_active":"2024-09-21T00:28:30.956979+0000","last_peered":"2024-09-21T00:28:30.956979+0000","last_clean":"2024-09-21T00:28:30.956979+0000","last_became_active":"2024-09-21T00:28:30.947013+0000","last_became_peered":"2024-09-21T00:28:30.947013+0000","last_unstale":"2024-09-21T00:28:30.956979+0000","last_undegraded":"2024-09-21T00:28:30.956979+0000","last_fullsized":"2024-09-21T00:28:30.956979+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_clean_scrub_stamp":"2024-09-21T00:28:29.918310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:55:15.836726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.928104+0000","last_change":"2024-09-21T00:28:29.946314+0000","last_active":"2024-09-21T00:28:30.928104+0000","last_peered":"2024-09-21T00:28:30.928104+0000","last_clean":"2024-09-21T00:28:30.928104+0000","last_became_active":"2024-09-21T00:28:29.945966+0000","last_became_peered":"2024-09-21T00:28:29.945966+0000","last_unstale":"2024-09-21T00:28:30.928104+0000","last_undegraded":"2024-09-21T00:28:30.928104+0000","last_fullsized":"2024-09-21T00:28:30.928104+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:28.914450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:28.914450+0000","last_clean_scrub_stamp":"2024-09-21T00:28:28.914450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:23:09.381529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.963465+0000","last_change":"2024-09-21T00:28:30.957296+0000","last_active":"2024-09-21T00:28:30.963465+0000","last_peered":"2024-09-21T00:28:30.963465+0000","last_clean":"2024-09-21T00:28:30.963465+0000","last_became_active":"2024-09-21T00:28:30.956981+0000","last_became_peered":"2024-09-21T00:28:30.956981+0000","last_unstale":"2024-09-21T00:28:30.963465+0000","last_undegraded":"2024-09-21T00:28:30.963465+0000","last_fullsized":"2024-09-21T00:28:30.963465+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_clean_scrub_stamp":"2024-09-21T00:28:29.918310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:14:48.640699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.936129+0000","last_change":"2024-09-21T00:28:29.941559+0000","last_active":"2024-09-21T00:28:30.936129+0000","last_peered":"2024-09-21T00:28:30.936129+0000","last_clean":"2024-09-21T00:28:30.936129+0000","last_became_active":"2024-09-21T00:28:29.941178+0000","last_became_peered":"2024-09-21T00:28:29.941178+0000","last_unstale":"2024-09-21T00:28:30.936129+0000","last_undegraded":"2024-09-21T00:28:30.936129+0000","last_fullsized":"2024-09-21T00:28:30.936129+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:28.914450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:28.914450+0000","last_clean_scrub_stamp":"2024-09-21T00:28:28.914450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:01:16.191815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.944629+0000","last_change":"2024-09-21T00:28:30.942634+0000","last_active":"2024-09-21T00:28:30.944629+0000","last_peered":"2024-09-21T00:28:30.944629+0000","last_clean":"2024-09-21T00:28:30.944629+0000","last_became_active":"2024-09-21T00:28:30.942316+0000","last_became_peered":"2024-09-21T00:28:30.942316+0000","last_unstale":"2024-09-21T00:28:30.944629+0000","last_undegraded":"2024-09-21T00:28:30.944629+0000","last_fullsized":"2024-09-21T00:28:30.944629+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_clean_scrub_stamp":"2024-09-21T00:28:29.918310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:52:35.974242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.941799+0000","last_change":"2024-09-21T00:28:29.948717+0000","last_active":"2024-09-21T00:28:30.941799+0000","last_peered":"2024-09-21T00:28:30.941799+0000","last_clean":"2024-09-21T00:28:30.941799+0000","last_became_active":"2024-09-21T00:28:29.948358+0000","last_became_peered":"2024-09-21T00:28:29.948358+0000","last_unstale":"2024-09-21T00:28:30.941799+0000","last_undegraded":"2024-09-21T00:28:30.941799+0000","last_fullsized":"2024-09-21T00:28:30.941799+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:28.914450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:28.914450+0000","last_clean_scrub_stamp":"2024-09-21T00:28:28.914450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:46:22.380140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.953284+0000","last_change":"2024-09-21T00:28:30.950082+0000","last_active":"2024-09-21T00:28:30.953284+0000","last_peered":"2024-09-21T00:28:30.953284+0000","last_clean":"2024-09-21T00:28:30.953284+0000","last_became_active":"2024-09-21T00:28:30.949686+0000","last_became_peered":"2024-09-21T00:28:30.949686+0000","last_unstale":"2024-09-21T00:28:30.953284+0000","last_undegraded":"2024-09-21T00:28:30.953284+0000","last_fullsized":"2024-09-21T00:28:30.953284+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_clean_scrub_stamp":"2024-09-21T00:28:29.918310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:24:05.849633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.934710+0000","last_change":"2024-09-21T00:28:29.946140+0000","last_active":"2024-09-21T00:28:30.934710+0000","last_peered":"2024-09-21T00:28:30.934710+0000","last_clean":"2024-09-21T00:28:30.934710+0000","last_became_active":"2024-09-21T00:28:29.945761+0000","last_became_peered":"2024-09-21T00:28:29.945761+0000","last_unstale":"2024-09-21T00:28:30.934710+0000","last_undegraded":"2024-09-21T00:28:30.934710+0000","last_fullsized":"2024-09-21T00:28:30.934710+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:28.914450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:28.914450+0000","last_clean_scrub_stamp":"2024-09-21T00:28:28.914450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:22:12.208115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.954771+0000","last_change":"2024-09-21T00:28:30.946678+0000","last_active":"2024-09-21T00:28:30.954771+0000","last_peered":"2024-09-21T00:28:30.954771+0000","last_clean":"2024-09-21T00:28:30.954771+0000","last_became_active":"2024-09-21T00:28:30.946226+0000","last_became_peered":"2024-09-21T00:28:30.946226+0000","last_unstale":"2024-09-21T00:28:30.954771+0000","last_undegraded":"2024-09-21T00:28:30.954771+0000","last_fullsized":"2024-09-21T00:28:30.954771+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_clean_scrub_stamp":"2024-09-21T00:28:29.918310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:52:55.340063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"70'3","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.966600+0000","last_change":"2024-09-21T00:28:29.940783+0000","last_active":"2024-09-21T00:28:30.966600+0000","last_peered":"2024-09-21T00:28:30.966600+0000","last_clean":"2024-09-21T00:28:30.966600+0000","last_became_active":"2024-09-21T00:28:29.940449+0000","last_became_peered":"2024-09-21T00:28:29.940449+0000","last_unstale":"2024-09-21T00:28:30.966600+0000","last_undegraded":"2024-09-21T00:28:30.966600+0000","last_fullsized":"2024-09-21T00:28:30.966600+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:28.914450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:28.914450+0000","last_clean_scrub_stamp":"2024-09-21T00:28:28.914450+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:47:28.342192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:28:29.918310+0000","last_change":"2024-09-21T00:28:29.918310+0000","last_active":"2024-09-21T00:28:29.918310+0000","last_peered":"2024-09-21T00:28:29.918310+0000","last_clean":"2024-09-21T00:28:29.918310+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:28:29.918310+0000","last_undegraded":"2024-09-21T00:28:29.918310+0000","last_fullsized":"2024-09-21T00:28:29.918310+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_clean_scrub_stamp":"2024-09-21T00:28:29.918310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.14","version":"70'6","reported_seq":19,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.970142+0000","last_change":"2024-09-21T00:28:29.942822+0000","last_active":"2024-09-21T00:28:30.970142+0000","last_peered":"2024-09-21T00:28:30.970142+0000","last_clean":"2024-09-21T00:28:30.970142+0000","last_became_active":"2024-09-21T00:28:29.942389+0000","last_became_peered":"2024-09-21T00:28:29.942389+0000","last_unstale":"2024-09-21T00:28:30.970142+0000","last_undegraded":"2024-09-21T00:28:30.970142+0000","last_fullsized":"2024-09-21T00:28:30.970142+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:28.914450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:28.914450+0000","last_clean_scrub_stamp":"2024-09-21T00:28:28.914450+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:10:24.411387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.952392+0000","last_change":"2024-09-21T00:28:30.947393+0000","last_active":"2024-09-21T00:28:30.952392+0000","last_peered":"2024-09-21T00:28:30.952392+0000","last_clean":"2024-09-21T00:28:30.952392+0000","last_became_active":"2024-09-21T00:28:30.946923+0000","last_became_peered":"2024-09-21T00:28:30.946923+0000","last_unstale":"2024-09-21T00:28:30.952392+0000","last_undegraded":"2024-09-21T00:28:30.952392+0000","last_fullsized":"2024-09-21T00:28:30.952392+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_clean_scrub_stamp":"2024-09-21T00:28:29.918310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:16:33.054120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"70'2","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.962769+0000","last_change":"2024-09-21T00:28:29.934927+0000","last_active":"2024-09-21T00:28:30.962769+0000","last_peered":"2024-09-21T00:28:30.962769+0000","last_clean":"2024-09-21T00:28:30.962769+0000","last_became_active":"2024-09-21T00:28:29.934601+0000","last_became_peered":"2024-09-21T00:28:29.934601+0000","last_unstale":"2024-09-21T00:28:30.962769+0000","last_undegraded":"2024-09-21T00:28:30.962769+0000","last_fullsized":"2024-09-21T00:28:30.962769+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:28.914450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:28.914450+0000","last_clean_scrub_stamp":"2024-09-21T00:28:28.914450+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:33:46.518632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.965265+0000","last_change":"2024-09-21T00:28:30.960338+0000","last_active":"2024-09-21T00:28:30.965265+0000","last_peered":"2024-09-21T00:28:30.965265+0000","last_clean":"2024-09-21T00:28:30.965265+0000","last_became_active":"2024-09-21T00:28:30.960006+0000","last_became_peered":"2024-09-21T00:28:30.960006+0000","last_unstale":"2024-09-21T00:28:30.965265+0000","last_undegraded":"2024-09-21T00:28:30.965265+0000","last_fullsized":"2024-09-21T00:28:30.965265+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_clean_scrub_stamp":"2024-09-21T00:28:29.918310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:13:49.866866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.938388+0000","last_change":"2024-09-21T00:28:29.949796+0000","last_active":"2024-09-21T00:28:30.938388+0000","last_peered":"2024-09-21T00:28:30.938388+0000","last_clean":"2024-09-21T00:28:30.938388+0000","last_became_active":"2024-09-21T00:28:29.949527+0000","last_became_peered":"2024-09-21T00:28:29.949527+0000","last_unstale":"2024-09-21T00:28:30.938388+0000","last_undegraded":"2024-09-21T00:28:30.938388+0000","last_fullsized":"2024-09-21T00:28:30.938388+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:28.914450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:28.914450+0000","last_clean_scrub_stamp":"2024-09-21T00:28:28.914450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:52:34.970382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.965221+0000","last_change":"2024-09-21T00:28:30.959923+0000","last_active":"2024-09-21T00:28:30.965221+0000","last_peered":"2024-09-21T00:28:30.965221+0000","last_clean":"2024-09-21T00:28:30.965221+0000","last_became_active":"2024-09-21T00:28:30.959157+0000","last_became_peered":"2024-09-21T00:28:30.959157+0000","last_unstale":"2024-09-21T00:28:30.965221+0000","last_undegraded":"2024-09-21T00:28:30.965221+0000","last_fullsized":"2024-09-21T00:28:30.965221+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_clean_scrub_stamp":"2024-09-21T00:28:29.918310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:29:22.486616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.936033+0000","last_change":"2024-09-21T00:28:29.944758+0000","last_active":"2024-09-21T00:28:30.936033+0000","last_peered":"2024-09-21T00:28:30.936033+0000","last_clean":"2024-09-21T00:28:30.936033+0000","last_became_active":"2024-09-21T00:28:29.944330+0000","last_became_peered":"2024-09-21T00:28:29.944330+0000","last_unstale":"2024-09-21T00:28:30.936033+0000","last_undegraded":"2024-09-21T00:28:30.936033+0000","last_fullsized":"2024-09-21T00:28:30.936033+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:28.914450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:28.914450+0000","last_clean_scrub_stamp":"2024-09-21T00:28:28.914450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:42:26.100740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.959653+0000","last_change":"2024-09-21T00:28:30.954447+0000","last_active":"2024-09-21T00:28:30.959653+0000","last_peered":"2024-09-21T00:28:30.959653+0000","last_clean":"2024-09-21T00:28:30.959653+0000","last_became_active":"2024-09-21T00:28:30.954135+0000","last_became_peered":"2024-09-21T00:28:30.954135+0000","last_unstale":"2024-09-21T00:28:30.959653+0000","last_undegraded":"2024-09-21T00:28:30.959653+0000","last_fullsized":"2024-09-21T00:28:30.959653+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_clean_scrub_stamp":"2024-09-21T00:28:29.918310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:26:56.342443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"70'8","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.966633+0000","last_change":"2024-09-21T00:28:29.942001+0000","last_active":"2024-09-21T00:28:30.966633+0000","last_peered":"2024-09-21T00:28:30.966633+0000","last_clean":"2024-09-21T00:28:30.966633+0000","last_became_active":"2024-09-21T00:28:29.941566+0000","last_became_peered":"2024-09-21T00:28:29.941566+0000","last_unstale":"2024-09-21T00:28:30.966633+0000","last_undegraded":"2024-09-21T00:28:30.966633+0000","last_fullsized":"2024-09-21T00:28:30.966633+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:28.914450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:28.914450+0000","last_clean_scrub_stamp":"2024-09-21T00:28:28.914450+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:55:47.073868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.960623+0000","last_change":"2024-09-21T00:28:30.956886+0000","last_active":"2024-09-21T00:28:30.960623+0000","last_peered":"2024-09-21T00:28:30.960623+0000","last_clean":"2024-09-21T00:28:30.960623+0000","last_became_active":"2024-09-21T00:28:30.956468+0000","last_became_peered":"2024-09-21T00:28:30.956468+0000","last_unstale":"2024-09-21T00:28:30.960623+0000","last_undegraded":"2024-09-21T00:28:30.960623+0000","last_fullsized":"2024-09-21T00:28:30.960623+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_clean_scrub_stamp":"2024-09-21T00:28:29.918310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:32:46.627248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"70'1","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.959291+0000","last_change":"2024-09-21T00:28:29.947764+0000","last_active":"2024-09-21T00:28:30.959291+0000","last_peered":"2024-09-21T00:28:30.959291+0000","last_clean":"2024-09-21T00:28:30.959291+0000","last_became_active":"2024-09-21T00:28:29.947398+0000","last_became_peered":"2024-09-21T00:28:29.947398+0000","last_unstale":"2024-09-21T00:28:30.959291+0000","last_undegraded":"2024-09-21T00:28:30.959291+0000","last_fullsized":"2024-09-21T00:28:30.959291+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:28.914450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:28.914450+0000","last_clean_scrub_stamp":"2024-09-21T00:28:28.914450+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:08:58.112056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.958949+0000","last_change":"2024-09-21T00:28:30.950953+0000","last_active":"2024-09-21T00:28:30.958949+0000","last_peered":"2024-09-21T00:28:30.958949+0000","last_clean":"2024-09-21T00:28:30.958949+0000","last_became_active":"2024-09-21T00:28:30.950588+0000","last_became_peered":"2024-09-21T00:28:30.950588+0000","last_unstale":"2024-09-21T00:28:30.958949+0000","last_undegraded":"2024-09-21T00:28:30.958949+0000","last_fullsized":"2024-09-21T00:28:30.958949+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_clean_scrub_stamp":"2024-09-21T00:28:29.918310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:05:10.437419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"70'2","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.963623+0000","last_change":"2024-09-21T00:28:29.941321+0000","last_active":"2024-09-21T00:28:30.963623+0000","last_peered":"2024-09-21T00:28:30.963623+0000","last_clean":"2024-09-21T00:28:30.963623+0000","last_became_active":"2024-09-21T00:28:29.940975+0000","last_became_peered":"2024-09-21T00:28:29.940975+0000","last_unstale":"2024-09-21T00:28:30.963623+0000","last_undegraded":"2024-09-21T00:28:30.963623+0000","last_fullsized":"2024-09-21T00:28:30.963623+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:28.914450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:28.914450+0000","last_clean_scrub_stamp":"2024-09-21T00:28:28.914450+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:31:49.583633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.950572+0000","last_change":"2024-09-21T00:28:30.947463+0000","last_active":"2024-09-21T00:28:30.950572+0000","last_peered":"2024-09-21T00:28:30.950572+0000","last_clean":"2024-09-21T00:28:30.950572+0000","last_became_active":"2024-09-21T00:28:30.947118+0000","last_became_peered":"2024-09-21T00:28:30.947118+0000","last_unstale":"2024-09-21T00:28:30.950572+0000","last_undegraded":"2024-09-21T00:28:30.950572+0000","last_fullsized":"2024-09-21T00:28:30.950572+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_clean_scrub_stamp":"2024-09-21T00:28:29.918310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:03:50.174406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"70'1","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.966506+0000","last_change":"2024-09-21T00:28:29.940181+0000","last_active":"2024-09-21T00:28:30.966506+0000","last_peered":"2024-09-21T00:28:30.966506+0000","last_clean":"2024-09-21T00:28:30.966506+0000","last_became_active":"2024-09-21T00:28:29.939780+0000","last_became_peered":"2024-09-21T00:28:29.939780+0000","last_unstale":"2024-09-21T00:28:30.966506+0000","last_undegraded":"2024-09-21T00:28:30.966506+0000","last_fullsized":"2024-09-21T00:28:30.966506+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:28.914450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:28.914450+0000","last_clean_scrub_stamp":"2024-09-21T00:28:28.914450+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:44:51.450242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.959165+0000","last_change":"2024-09-21T00:28:30.954744+0000","last_active":"2024-09-21T00:28:30.959165+0000","last_peered":"2024-09-21T00:28:30.959165+0000","last_clean":"2024-09-21T00:28:30.959165+0000","last_became_active":"2024-09-21T00:28:30.954447+0000","last_became_peered":"2024-09-21T00:28:30.954447+0000","last_unstale":"2024-09-21T00:28:30.959165+0000","last_undegraded":"2024-09-21T00:28:30.959165+0000","last_fullsized":"2024-09-21T00:28:30.959165+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_clean_scrub_stamp":"2024-09-21T00:28:29.918310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:56:01.061375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:28:28.914450+0000","last_change":"2024-09-21T00:28:28.914450+0000","last_active":"2024-09-21T00:28:28.914450+0000","last_peered":"2024-09-21T00:28:28.914450+0000","last_clean":"2024-09-21T00:28:28.914450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:28:28.914450+0000","last_undegraded":"2024-09-21T00:28:28.914450+0000","last_fullsized":"2024-09-21T00:28:28.914450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:28.914450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:28.914450+0000","last_clean_scrub_stamp":"2024-09-21T00:28:28.914450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.947585+0000","last_change":"2024-09-21T00:28:30.944110+0000","last_active":"2024-09-21T00:28:30.947585+0000","last_peered":"2024-09-21T00:28:30.947585+0000","last_clean":"2024-09-21T00:28:30.947585+0000","last_became_active":"2024-09-21T00:28:30.943750+0000","last_became_peered":"2024-09-21T00:28:30.943750+0000","last_unstale":"2024-09-21T00:28:30.947585+0000","last_undegraded":"2024-09-21T00:28:30.947585+0000","last_fullsized":"2024-09-21T00:28:30.947585+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_clean_scrub_stamp":"2024-09-21T00:28:29.918310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T00:54:43.554374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"70'2","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.963044+0000","last_change":"2024-09-21T00:28:29.946672+0000","last_active":"2024-09-21T00:28:30.963044+0000","last_peered":"2024-09-21T00:28:30.963044+0000","last_clean":"2024-09-21T00:28:30.963044+0000","last_became_active":"2024-09-21T00:28:29.945563+0000","last_became_peered":"2024-09-21T00:28:29.945563+0000","last_unstale":"2024-09-21T00:28:30.963044+0000","last_undegraded":"2024-09-21T00:28:30.963044+0000","last_fullsized":"2024-09-21T00:28:30.963044+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:28.914450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:28.914450+0000","last_clean_scrub_stamp":"2024-09-21T00:28:28.914450+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:45:07.518060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.949592+0000","last_change":"2024-09-21T00:28:30.945284+0000","last_active":"2024-09-21T00:28:30.949592+0000","last_peered":"2024-09-21T00:28:30.949592+0000","last_clean":"2024-09-21T00:28:30.949592+0000","last_became_active":"2024-09-21T00:28:30.944965+0000","last_became_peered":"2024-09-21T00:28:30.944965+0000","last_unstale":"2024-09-21T00:28:30.949592+0000","last_undegraded":"2024-09-21T00:28:30.949592+0000","last_fullsized":"2024-09-21T00:28:30.949592+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_clean_scrub_stamp":"2024-09-21T00:28:29.918310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:07:59.316440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.939986+0000","last_change":"2024-09-21T00:28:29.945512+0000","last_active":"2024-09-21T00:28:30.939986+0000","last_peered":"2024-09-21T00:28:30.939986+0000","last_clean":"2024-09-21T00:28:30.939986+0000","last_became_active":"2024-09-21T00:28:29.945083+0000","last_became_peered":"2024-09-21T00:28:29.945083+0000","last_unstale":"2024-09-21T00:28:30.939986+0000","last_undegraded":"2024-09-21T00:28:30.939986+0000","last_fullsized":"2024-09-21T00:28:30.939986+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:28.914450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:28.914450+0000","last_clean_scrub_stamp":"2024-09-21T00:28:28.914450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:15:12.021971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.965165+0000","last_change":"2024-09-21T00:28:30.959001+0000","last_active":"2024-09-21T00:28:30.965165+0000","last_peered":"2024-09-21T00:28:30.965165+0000","last_clean":"2024-09-21T00:28:30.965165+0000","last_became_active":"2024-09-21T00:28:30.957614+0000","last_became_peered":"2024-09-21T00:28:30.957614+0000","last_unstale":"2024-09-21T00:28:30.965165+0000","last_undegraded":"2024-09-21T00:28:30.965165+0000","last_fullsized":"2024-09-21T00:28:30.965165+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_clean_scrub_stamp":"2024-09-21T00:28:29.918310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:14:08.603016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.940247+0000","last_change":"2024-09-21T00:28:29.945343+0000","last_active":"2024-09-21T00:28:30.940247+0000","last_peered":"2024-09-21T00:28:30.940247+0000","last_clean":"2024-09-21T00:28:30.940247+0000","last_became_active":"2024-09-21T00:28:29.945024+0000","last_became_peered":"2024-09-21T00:28:29.945024+0000","last_unstale":"2024-09-21T00:28:30.940247+0000","last_undegraded":"2024-09-21T00:28:30.940247+0000","last_fullsized":"2024-09-21T00:28:30.940247+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:28.914450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:28.914450+0000","last_clean_scrub_stamp":"2024-09-21T00:28:28.914450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:46:22.380140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.958863+0000","last_change":"2024-09-21T00:28:30.955844+0000","last_active":"2024-09-21T00:28:30.958863+0000","last_peered":"2024-09-21T00:28:30.958863+0000","last_clean":"2024-09-21T00:28:30.958863+0000","last_became_active":"2024-09-21T00:28:30.955212+0000","last_became_peered":"2024-09-21T00:28:30.955212+0000","last_unstale":"2024-09-21T00:28:30.958863+0000","last_undegraded":"2024-09-21T00:28:30.958863+0000","last_fullsized":"2024-09-21T00:28:30.958863+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_clean_scrub_stamp":"2024-09-21T00:28:29.918310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:53:36.681496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"70'2","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.956268+0000","last_change":"2024-09-21T00:28:29.939267+0000","last_active":"2024-09-21T00:28:30.956268+0000","last_peered":"2024-09-21T00:28:30.956268+0000","last_clean":"2024-09-21T00:28:30.956268+0000","last_became_active":"2024-09-21T00:28:29.938961+0000","last_became_peered":"2024-09-21T00:28:29.938961+0000","last_unstale":"2024-09-21T00:28:30.956268+0000","last_undegraded":"2024-09-21T00:28:30.956268+0000","last_fullsized":"2024-09-21T00:28:30.956268+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:28.914450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:28.914450+0000","last_clean_scrub_stamp":"2024-09-21T00:28:28.914450+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:57:57.255009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.954644+0000","last_change":"2024-09-21T00:28:30.947697+0000","last_active":"2024-09-21T00:28:30.954644+0000","last_peered":"2024-09-21T00:28:30.954644+0000","last_clean":"2024-09-21T00:28:30.954644+0000","last_became_active":"2024-09-21T00:28:30.947352+0000","last_became_peered":"2024-09-21T00:28:30.947352+0000","last_unstale":"2024-09-21T00:28:30.954644+0000","last_undegraded":"2024-09-21T00:28:30.954644+0000","last_fullsized":"2024-09-21T00:28:30.954644+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_clean_scrub_stamp":"2024-09-21T00:28:29.918310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:12:42.513714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.928179+0000","last_change":"2024-09-21T00:28:29.943061+0000","last_active":"2024-09-21T00:28:30.928179+0000","last_peered":"2024-09-21T00:28:30.928179+0000","last_clean":"2024-09-21T00:28:30.928179+0000","last_became_active":"2024-09-21T00:28:29.942693+0000","last_became_peered":"2024-09-21T00:28:29.942693+0000","last_unstale":"2024-09-21T00:28:30.928179+0000","last_undegraded":"2024-09-21T00:28:30.928179+0000","last_fullsized":"2024-09-21T00:28:30.928179+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:28.914450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:28.914450+0000","last_clean_scrub_stamp":"2024-09-21T00:28:28.914450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:12:36.499940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.956794+0000","last_change":"2024-09-21T00:28:30.952387+0000","last_active":"2024-09-21T00:28:30.956794+0000","last_peered":"2024-09-21T00:28:30.956794+0000","last_clean":"2024-09-21T00:28:30.956794+0000","last_became_active":"2024-09-21T00:28:30.951670+0000","last_became_peered":"2024-09-21T00:28:30.951670+0000","last_unstale":"2024-09-21T00:28:30.956794+0000","last_undegraded":"2024-09-21T00:28:30.956794+0000","last_fullsized":"2024-09-21T00:28:30.956794+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_clean_scrub_stamp":"2024-09-21T00:28:29.918310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:41:52.334945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.940997+0000","last_change":"2024-09-21T00:28:29.946233+0000","last_active":"2024-09-21T00:28:30.940997+0000","last_peered":"2024-09-21T00:28:30.940997+0000","last_clean":"2024-09-21T00:28:30.940997+0000","last_became_active":"2024-09-21T00:28:29.945680+0000","last_became_peered":"2024-09-21T00:28:29.945680+0000","last_unstale":"2024-09-21T00:28:30.940997+0000","last_undegraded":"2024-09-21T00:28:30.940997+0000","last_fullsized":"2024-09-21T00:28:30.940997+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:28.914450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:28.914450+0000","last_clean_scrub_stamp":"2024-09-21T00:28:28.914450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:15:12.021971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.957558+0000","last_change":"2024-09-21T00:28:30.953554+0000","last_active":"2024-09-21T00:28:30.957558+0000","last_peered":"2024-09-21T00:28:30.957558+0000","last_clean":"2024-09-21T00:28:30.957558+0000","last_became_active":"2024-09-21T00:28:30.953191+0000","last_became_peered":"2024-09-21T00:28:30.953191+0000","last_unstale":"2024-09-21T00:28:30.957558+0000","last_undegraded":"2024-09-21T00:28:30.957558+0000","last_fullsized":"2024-09-21T00:28:30.957558+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_clean_scrub_stamp":"2024-09-21T00:28:29.918310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:56:01.061375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.938834+0000","last_change":"2024-09-21T00:28:29.948601+0000","last_active":"2024-09-21T00:28:30.938834+0000","last_peered":"2024-09-21T00:28:30.938834+0000","last_clean":"2024-09-21T00:28:30.938834+0000","last_became_active":"2024-09-21T00:28:29.948279+0000","last_became_peered":"2024-09-21T00:28:29.948279+0000","last_unstale":"2024-09-21T00:28:30.938834+0000","last_undegraded":"2024-09-21T00:28:30.938834+0000","last_fullsized":"2024-09-21T00:28:30.938834+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:28.914450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:28.914450+0000","last_clean_scrub_stamp":"2024-09-21T00:28:28.914450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T00:42:17.219795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.963437+0000","last_change":"2024-09-21T00:28:30.955305+0000","last_active":"2024-09-21T00:28:30.963437+0000","last_peered":"2024-09-21T00:28:30.963437+0000","last_clean":"2024-09-21T00:28:30.963437+0000","last_became_active":"2024-09-21T00:28:30.955008+0000","last_became_peered":"2024-09-21T00:28:30.955008+0000","last_unstale":"2024-09-21T00:28:30.963437+0000","last_undegraded":"2024-09-21T00:28:30.963437+0000","last_fullsized":"2024-09-21T00:28:30.963437+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_clean_scrub_stamp":"2024-09-21T00:28:29.918310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:11:36.689705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.943541+0000","last_change":"2024-09-21T00:28:29.942909+0000","last_active":"2024-09-21T00:28:30.943541+0000","last_peered":"2024-09-21T00:28:30.943541+0000","last_clean":"2024-09-21T00:28:30.943541+0000","last_became_active":"2024-09-21T00:28:29.942571+0000","last_became_peered":"2024-09-21T00:28:29.942571+0000","last_unstale":"2024-09-21T00:28:30.943541+0000","last_undegraded":"2024-09-21T00:28:30.943541+0000","last_fullsized":"2024-09-21T00:28:30.943541+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:28.914450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:28.914450+0000","last_clean_scrub_stamp":"2024-09-21T00:28:28.914450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:13:26.027352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.963427+0000","last_change":"2024-09-21T00:28:30.957257+0000","last_active":"2024-09-21T00:28:30.963427+0000","last_peered":"2024-09-21T00:28:30.963427+0000","last_clean":"2024-09-21T00:28:30.963427+0000","last_became_active":"2024-09-21T00:28:30.956927+0000","last_became_peered":"2024-09-21T00:28:30.956927+0000","last_unstale":"2024-09-21T00:28:30.963427+0000","last_undegraded":"2024-09-21T00:28:30.963427+0000","last_fullsized":"2024-09-21T00:28:30.963427+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_clean_scrub_stamp":"2024-09-21T00:28:29.918310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:46:26.394456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.935874+0000","last_change":"2024-09-21T00:28:29.944482+0000","last_active":"2024-09-21T00:28:30.935874+0000","last_peered":"2024-09-21T00:28:30.935874+0000","last_clean":"2024-09-21T00:28:30.935874+0000","last_became_active":"2024-09-21T00:28:29.944168+0000","last_became_peered":"2024-09-21T00:28:29.944168+0000","last_unstale":"2024-09-21T00:28:30.935874+0000","last_undegraded":"2024-09-21T00:28:30.935874+0000","last_fullsized":"2024-09-21T00:28:30.935874+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:28.914450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:28.914450+0000","last_clean_scrub_stamp":"2024-09-21T00:28:28.914450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:24:52.441266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:28:29.918310+0000","last_change":"2024-09-21T00:28:29.918310+0000","last_active":"2024-09-21T00:28:29.918310+0000","last_peered":"2024-09-21T00:28:29.918310+0000","last_clean":"2024-09-21T00:28:29.918310+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:28:29.918310+0000","last_undegraded":"2024-09-21T00:28:29.918310+0000","last_fullsized":"2024-09-21T00:28:29.918310+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_clean_scrub_stamp":"2024-09-21T00:28:29.918310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.941468+0000","last_change":"2024-09-21T00:28:29.945265+0000","last_active":"2024-09-21T00:28:30.941468+0000","last_peered":"2024-09-21T00:28:30.941468+0000","last_clean":"2024-09-21T00:28:30.941468+0000","last_became_active":"2024-09-21T00:28:29.944907+0000","last_became_peered":"2024-09-21T00:28:29.944907+0000","last_unstale":"2024-09-21T00:28:30.941468+0000","last_undegraded":"2024-09-21T00:28:30.941468+0000","last_fullsized":"2024-09-21T00:28:30.941468+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:28.914450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:28.914450+0000","last_clean_scrub_stamp":"2024-09-21T00:28:28.914450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:23:25.884700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:28:29.918310+0000","last_change":"2024-09-21T00:28:29.918310+0000","last_active":"2024-09-21T00:28:29.918310+0000","last_peered":"2024-09-21T00:28:29.918310+0000","last_clean":"2024-09-21T00:28:29.918310+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:28:29.918310+0000","last_undegraded":"2024-09-21T00:28:29.918310+0000","last_fullsized":"2024-09-21T00:28:29.918310+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_clean_scrub_stamp":"2024-09-21T00:28:29.918310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:28:28.914450+0000","last_change":"2024-09-21T00:28:28.914450+0000","last_active":"2024-09-21T00:28:28.914450+0000","last_peered":"2024-09-21T00:28:28.914450+0000","last_clean":"2024-09-21T00:28:28.914450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:28:28.914450+0000","last_undegraded":"2024-09-21T00:28:28.914450+0000","last_fullsized":"2024-09-21T00:28:28.914450+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:28.914450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:28.914450+0000","last_clean_scrub_stamp":"2024-09-21T00:28:28.914450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.954438+0000","last_change":"2024-09-21T00:28:30.950650+0000","last_active":"2024-09-21T00:28:30.954438+0000","last_peered":"2024-09-21T00:28:30.954438+0000","last_clean":"2024-09-21T00:28:30.954438+0000","last_became_active":"2024-09-21T00:28:30.950135+0000","last_became_peered":"2024-09-21T00:28:30.950135+0000","last_unstale":"2024-09-21T00:28:30.954438+0000","last_undegraded":"2024-09-21T00:28:30.954438+0000","last_fullsized":"2024-09-21T00:28:30.954438+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_clean_scrub_stamp":"2024-09-21T00:28:29.918310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:52:43.572069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.941835+0000","last_change":"2024-09-21T00:28:29.946124+0000","last_active":"2024-09-21T00:28:30.941835+0000","last_peered":"2024-09-21T00:28:30.941835+0000","last_clean":"2024-09-21T00:28:30.941835+0000","last_became_active":"2024-09-21T00:28:29.945131+0000","last_became_peered":"2024-09-21T00:28:29.945131+0000","last_unstale":"2024-09-21T00:28:30.941835+0000","last_undegraded":"2024-09-21T00:28:30.941835+0000","last_fullsized":"2024-09-21T00:28:30.941835+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:28.914450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:28.914450+0000","last_clean_scrub_stamp":"2024-09-21T00:28:28.914450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:08:58.112056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":199,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.940646+0000","last_change":"2024-09-21T00:25:38.725792+0000","last_active":"2024-09-21T00:28:30.940646+0000","last_peered":"2024-09-21T00:28:30.940646+0000","last_clean":"2024-09-21T00:28:30.940646+0000","last_became_active":"2024-09-21T00:25:38.724668+0000","last_became_peered":"2024-09-21T00:25:38.724668+0000","last_unstale":"2024-09-21T00:28:30.940646+0000","last_undegraded":"2024-09-21T00:28:30.940646+0000","last_fullsized":"2024-09-21T00:28:30.940646+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:37.163216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:37.163216+0000","last_clean_scrub_stamp":"2024-09-21T00:25:37.163216+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:14:36.588660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":114,"acting":114,"num_store_stats":7},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":4708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":60,"acting":60,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509478,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32608,"kb_used_data":1412,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339232,"statfs":{"total":96636764160,"available":96603373568,"internally_reserved":0,"allocated":1445888,"data_stored":968613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509479,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27844,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343996,"statfs":{"total":96636764160,"available":96608251904,"internally_reserved":0,"allocated":835584,"data_stored":371056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509478,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27868,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343972,"statfs":{"total":96636764160,"available":96608227328,"internally_reserved":0,"allocated":851968,"data_stored":378257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509477,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32044,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339796,"statfs":{"total":96636764160,"available":96603951104,"internally_reserved":0,"allocated":868352,"data_stored":379939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509479,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32028,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339812,"statfs":{"total":96636764160,"available":96603967488,"internally_reserved":0,"allocated":851968,"data_stored":378805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509479,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27844,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343996,"statfs":{"total":96636764160,"available":96608251904,"internally_reserved":0,"allocated":835584,"data_stored":371056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509479,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32588,"kb_used_data":1396,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339252,"statfs":{"total":96636764160,"available":96603394048,"internally_reserved":0,"allocated":1429504,"data_stored":961424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574886,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31980,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339860,"statfs":{"total":96636764160,"available":96604016640,"internally_reserved":0,"allocated":811008,"data_stored":358180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T00:28:34.954 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T00:28:35.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.099+0000 7f1701c93640 1 -- 172.21.15.112:0/3477638596 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16fc0d0860 msgr2=0x7f16fc15a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:35.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.099+0000 7f1701c93640 1 --2- 172.21.15.112:0/3477638596 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16fc0d0860 0x7f16fc15a0c0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f16fc0b4f40 tx=0x7f16f002a560 comp rx=0 tx=0).stop 2024-09-21T00:28:35.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.099+0000 7f1701c93640 1 -- 172.21.15.112:0/3477638596 shutdown_connections 2024-09-21T00:28:35.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.099+0000 7f1701c93640 1 --2- 172.21.15.112:0/3477638596 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f16fc0ccae0 0x7f16fc0ccef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:35.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.099+0000 7f1701c93640 1 --2- 172.21.15.112:0/3477638596 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16fc0d0860 0x7f16fc15a0c0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:35.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.099+0000 7f1701c93640 1 --2- 172.21.15.112:0/3477638596 >> v2:172.21.15.187:3300/0 conn(0x7f16fc0d2030 0x7f16fc15d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:35.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.099+0000 7f1701c93640 1 -- 172.21.15.112:0/3477638596 >> 172.21.15.112:0/3477638596 conn(0x7f16fc0b43f0 msgr2=0x7f16fc0b4830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:35.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.099+0000 7f1701c93640 1 -- 172.21.15.112:0/3477638596 shutdown_connections 2024-09-21T00:28:35.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.099+0000 7f1701c93640 1 -- 172.21.15.112:0/3477638596 wait complete. 2024-09-21T00:28:35.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.101+0000 7f1701c93640 1 Processor -- start 2024-09-21T00:28:35.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.101+0000 7f1701c93640 1 -- start start 2024-09-21T00:28:35.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.102+0000 7f1701c93640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16fc0ccae0 0x7f16fc0cb8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:35.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.102+0000 7f1701c93640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f16fc0d0860 0x7f16fc1594c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:35.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.102+0000 7f1701c93640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f16fc0d2030 0x7f16fc1e6be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:35.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.102+0000 7f1701c93640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f16fc160700 con 0x7f16fc0ccae0 2024-09-21T00:28:35.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.102+0000 7f1701c93640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f16fc160580 con 0x7f16fc0d0860 2024-09-21T00:28:35.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.102+0000 7f1701c93640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f16fc160880 con 0x7f16fc0d2030 2024-09-21T00:28:35.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.102+0000 7f16fb7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16fc0ccae0 0x7f16fc0cb8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:35.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.102+0000 7f16fb7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16fc0ccae0 0x7f16fc0cb8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:43116/0 (socket says 172.21.15.112:43116) 2024-09-21T00:28:35.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.102+0000 7f16fb7fe640 1 -- 172.21.15.112:0/952023244 learned_addr learned my addr 172.21.15.112:0/952023244 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:28:35.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.102+0000 7f16faffd640 1 --2- 172.21.15.112:0/952023244 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f16fc0d0860 0x7f16fc1594c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:35.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.102+0000 7f16fb7fe640 1 -- 172.21.15.112:0/952023244 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f16fc0d2030 msgr2=0x7f16fc1e6be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:35.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.102+0000 7f16fbfff640 1 --2- 172.21.15.112:0/952023244 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f16fc0d2030 0x7f16fc1e6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:35.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.102+0000 7f16fb7fe640 1 --2- 172.21.15.112:0/952023244 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f16fc0d2030 0x7f16fc1e6be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:35.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.102+0000 7f16fb7fe640 1 -- 172.21.15.112:0/952023244 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f16fc0d0860 msgr2=0x7f16fc1594c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:35.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.102+0000 7f16fb7fe640 1 --2- 172.21.15.112:0/952023244 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f16fc0d0860 0x7f16fc1594c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:35.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.102+0000 7f16fb7fe640 1 -- 172.21.15.112:0/952023244 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16fc0ca3d0 con 0x7f16fc0ccae0 2024-09-21T00:28:35.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.102+0000 7f16fbfff640 1 --2- 172.21.15.112:0/952023244 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f16fc0d2030 0x7f16fc1e6be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:28:35.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.102+0000 7f16faffd640 1 --2- 172.21.15.112:0/952023244 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f16fc0d0860 0x7f16fc1594c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:28:35.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.103+0000 7f16fb7fe640 1 --2- 172.21.15.112:0/952023244 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16fc0ccae0 0x7f16fc0cb8e0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f16ec00d770 tx=0x7f16ec00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:35.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.103+0000 7f16e3fff640 1 -- 172.21.15.112:0/952023244 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16ec017070 con 0x7f16fc0ccae0 2024-09-21T00:28:35.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.103+0000 7f16e3fff640 1 -- 172.21.15.112:0/952023244 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f16ec009d20 con 0x7f16fc0ccae0 2024-09-21T00:28:35.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.103+0000 7f1701c93640 1 -- 172.21.15.112:0/952023244 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16fc0ca840 con 0x7f16fc0ccae0 2024-09-21T00:28:35.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.103+0000 7f16e3fff640 1 -- 172.21.15.112:0/952023244 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16ec0106c0 con 0x7f16fc0ccae0 2024-09-21T00:28:35.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.103+0000 7f1701c93640 1 -- 172.21.15.112:0/952023244 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16fc0cae00 con 0x7f16fc0ccae0 2024-09-21T00:28:35.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.104+0000 7f1701c93640 1 -- 172.21.15.112:0/952023244 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16fc15a0c0 con 0x7f16fc0ccae0 2024-09-21T00:28:35.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.105+0000 7f16e3fff640 1 -- 172.21.15.112:0/952023244 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f16ec01a070 con 0x7f16fc0ccae0 2024-09-21T00:28:35.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.105+0000 7f16e3fff640 1 --2- 172.21.15.112:0/952023244 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f16e8078b90 0x7f16e807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:35.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.105+0000 7f16e3fff640 1 -- 172.21.15.112:0/952023244 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6101+0+0 (secure 0 0 0) 0x7f16ec09aff0 con 0x7f16fc0ccae0 2024-09-21T00:28:35.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.105+0000 7f16faffd640 1 --2- 172.21.15.112:0/952023244 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f16e8078b90 0x7f16e807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:35.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.106+0000 7f16faffd640 1 --2- 172.21.15.112:0/952023244 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f16e8078b90 0x7f16e807b0a0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f16f0033000 tx=0x7f16f0004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:35.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.109+0000 7f16e3fff640 1 -- 172.21.15.112:0/952023244 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f16ec063340 con 0x7f16fc0ccae0 2024-09-21T00:28:35.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.283+0000 7f1701c93640 1 -- 172.21.15.112:0/952023244 --> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f16fc071ed0 con 0x7f16e8078b90 2024-09-21T00:28:35.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.289+0000 7f16e3fff640 1 -- 172.21.15.112:0/952023244 <== mgr.4103 v2:172.21.15.187:6800/2921522906 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252053 (secure 0 0 0) 0x7f16fc071ed0 con 0x7f16e8078b90 2024-09-21T00:28:35.294 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:28:35.295 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2024-09-21T00:28:35.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.293+0000 7f1701c93640 1 -- 172.21.15.112:0/952023244 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f16e8078b90 msgr2=0x7f16e807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:35.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.293+0000 7f1701c93640 1 --2- 172.21.15.112:0/952023244 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f16e8078b90 0x7f16e807b0a0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f16f0033000 tx=0x7f16f0004730 comp rx=0 tx=0).stop 2024-09-21T00:28:35.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.293+0000 7f1701c93640 1 -- 172.21.15.112:0/952023244 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16fc0ccae0 msgr2=0x7f16fc0cb8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:35.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.293+0000 7f1701c93640 1 --2- 172.21.15.112:0/952023244 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16fc0ccae0 0x7f16fc0cb8e0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f16ec00d770 tx=0x7f16ec00dc30 comp rx=0 tx=0).stop 2024-09-21T00:28:35.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.293+0000 7f1701c93640 1 -- 172.21.15.112:0/952023244 shutdown_connections 2024-09-21T00:28:35.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.293+0000 7f1701c93640 1 --2- 172.21.15.112:0/952023244 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f16fc0d2030 0x7f16fc1e6be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:35.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.293+0000 7f1701c93640 1 --2- 172.21.15.112:0/952023244 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f16fc0d0860 0x7f16fc1594c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:35.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.293+0000 7f1701c93640 1 --2- 172.21.15.112:0/952023244 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f16e8078b90 0x7f16e807b0a0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:35.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.293+0000 7f1701c93640 1 --2- 172.21.15.112:0/952023244 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16fc0ccae0 0x7f16fc0cb8e0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:35.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.293+0000 7f1701c93640 1 -- 172.21.15.112:0/952023244 >> 172.21.15.112:0/952023244 conn(0x7f16fc0b43f0 msgr2=0x7f16fc15a910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:35.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.293+0000 7f1701c93640 1 -- 172.21.15.112:0/952023244 shutdown_connections 2024-09-21T00:28:35.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.293+0000 7f1701c93640 1 -- 172.21.15.112:0/952023244 wait complete. 2024-09-21T00:28:35.306 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":155,"stamp":"2024-09-21T00:28:35.132415+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":244864,"kb_used_data":7800,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754729856,"statfs":{"total":773094113280,"available":772843372544,"internally_reserved":0,"allocated":7987200,"data_stored":4194177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":242535985},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001507"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.958377+0000","last_change":"2024-09-21T00:28:30.954953+0000","last_active":"2024-09-21T00:28:30.958377+0000","last_peered":"2024-09-21T00:28:30.958377+0000","last_clean":"2024-09-21T00:28:30.958377+0000","last_became_active":"2024-09-21T00:28:30.954631+0000","last_became_peered":"2024-09-21T00:28:30.954631+0000","last_unstale":"2024-09-21T00:28:30.958377+0000","last_undegraded":"2024-09-21T00:28:30.958377+0000","last_fullsized":"2024-09-21T00:28:30.958377+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_clean_scrub_stamp":"2024-09-21T00:28:29.918310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:07:18.418422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.957593+0000","last_change":"2024-09-21T00:28:30.951440+0000","last_active":"2024-09-21T00:28:30.957593+0000","last_peered":"2024-09-21T00:28:30.957593+0000","last_clean":"2024-09-21T00:28:30.957593+0000","last_became_active":"2024-09-21T00:28:30.951133+0000","last_became_peered":"2024-09-21T00:28:30.951133+0000","last_unstale":"2024-09-21T00:28:30.957593+0000","last_undegraded":"2024-09-21T00:28:30.957593+0000","last_fullsized":"2024-09-21T00:28:30.957593+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_clean_scrub_stamp":"2024-09-21T00:28:29.918310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:52:55.340063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T00:28:31.947012+0000","last_change":"2024-09-21T00:28:30.945761+0000","last_active":"2024-09-21T00:28:31.947012+0000","last_peered":"2024-09-21T00:28:31.947012+0000","last_clean":"2024-09-21T00:28:31.947012+0000","last_became_active":"2024-09-21T00:28:30.945383+0000","last_became_peered":"2024-09-21T00:28:30.945383+0000","last_unstale":"2024-09-21T00:28:31.947012+0000","last_undegraded":"2024-09-21T00:28:31.947012+0000","last_fullsized":"2024-09-21T00:28:31.947012+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_clean_scrub_stamp":"2024-09-21T00:28:29.918310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:31:15.492495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.953198+0000","last_change":"2024-09-21T00:28:30.950561+0000","last_active":"2024-09-21T00:28:30.953198+0000","last_peered":"2024-09-21T00:28:30.953198+0000","last_clean":"2024-09-21T00:28:30.953198+0000","last_became_active":"2024-09-21T00:28:30.949672+0000","last_became_peered":"2024-09-21T00:28:30.949672+0000","last_unstale":"2024-09-21T00:28:30.953198+0000","last_undegraded":"2024-09-21T00:28:30.953198+0000","last_fullsized":"2024-09-21T00:28:30.953198+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_clean_scrub_stamp":"2024-09-21T00:28:29.918310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:25:55.834586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.960469+0000","last_change":"2024-09-21T00:28:30.953634+0000","last_active":"2024-09-21T00:28:30.960469+0000","last_peered":"2024-09-21T00:28:30.960469+0000","last_clean":"2024-09-21T00:28:30.960469+0000","last_became_active":"2024-09-21T00:28:30.953327+0000","last_became_peered":"2024-09-21T00:28:30.953327+0000","last_unstale":"2024-09-21T00:28:30.960469+0000","last_undegraded":"2024-09-21T00:28:30.960469+0000","last_fullsized":"2024-09-21T00:28:30.960469+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_clean_scrub_stamp":"2024-09-21T00:28:29.918310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:58:27.359494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.961560+0000","last_change":"2024-09-21T00:28:30.958525+0000","last_active":"2024-09-21T00:28:30.961560+0000","last_peered":"2024-09-21T00:28:30.961560+0000","last_clean":"2024-09-21T00:28:30.961560+0000","last_became_active":"2024-09-21T00:28:30.958195+0000","last_became_peered":"2024-09-21T00:28:30.958195+0000","last_unstale":"2024-09-21T00:28:30.961560+0000","last_undegraded":"2024-09-21T00:28:30.961560+0000","last_fullsized":"2024-09-21T00:28:30.961560+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_clean_scrub_stamp":"2024-09-21T00:28:29.918310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:41:07.332812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.965339+0000","last_change":"2024-09-21T00:28:30.961682+0000","last_active":"2024-09-21T00:28:30.965339+0000","last_peered":"2024-09-21T00:28:30.965339+0000","last_clean":"2024-09-21T00:28:30.965339+0000","last_became_active":"2024-09-21T00:28:30.961374+0000","last_became_peered":"2024-09-21T00:28:30.961374+0000","last_unstale":"2024-09-21T00:28:30.965339+0000","last_undegraded":"2024-09-21T00:28:30.965339+0000","last_fullsized":"2024-09-21T00:28:30.965339+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_clean_scrub_stamp":"2024-09-21T00:28:29.918310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:31:52.439535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.960759+0000","last_change":"2024-09-21T00:28:30.952916+0000","last_active":"2024-09-21T00:28:30.960759+0000","last_peered":"2024-09-21T00:28:30.960759+0000","last_clean":"2024-09-21T00:28:30.960759+0000","last_became_active":"2024-09-21T00:28:30.952630+0000","last_became_peered":"2024-09-21T00:28:30.952630+0000","last_unstale":"2024-09-21T00:28:30.960759+0000","last_undegraded":"2024-09-21T00:28:30.960759+0000","last_fullsized":"2024-09-21T00:28:30.960759+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_clean_scrub_stamp":"2024-09-21T00:28:29.918310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:14:43.923166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.956690+0000","last_change":"2024-09-21T00:28:30.947905+0000","last_active":"2024-09-21T00:28:30.956690+0000","last_peered":"2024-09-21T00:28:30.956690+0000","last_clean":"2024-09-21T00:28:30.956690+0000","last_became_active":"2024-09-21T00:28:30.947489+0000","last_became_peered":"2024-09-21T00:28:30.947489+0000","last_unstale":"2024-09-21T00:28:30.956690+0000","last_undegraded":"2024-09-21T00:28:30.956690+0000","last_fullsized":"2024-09-21T00:28:30.956690+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_clean_scrub_stamp":"2024-09-21T00:28:29.918310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:55:59.100753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.953224+0000","last_change":"2024-09-21T00:28:30.948702+0000","last_active":"2024-09-21T00:28:30.953224+0000","last_peered":"2024-09-21T00:28:30.953224+0000","last_clean":"2024-09-21T00:28:30.953224+0000","last_became_active":"2024-09-21T00:28:30.948385+0000","last_became_peered":"2024-09-21T00:28:30.948385+0000","last_unstale":"2024-09-21T00:28:30.953224+0000","last_undegraded":"2024-09-21T00:28:30.953224+0000","last_fullsized":"2024-09-21T00:28:30.953224+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_clean_scrub_stamp":"2024-09-21T00:28:29.918310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:58:17.081795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.964635+0000","last_change":"2024-09-21T00:28:30.961901+0000","last_active":"2024-09-21T00:28:30.964635+0000","last_peered":"2024-09-21T00:28:30.964635+0000","last_clean":"2024-09-21T00:28:30.964635+0000","last_became_active":"2024-09-21T00:28:30.961589+0000","last_became_peered":"2024-09-21T00:28:30.961589+0000","last_unstale":"2024-09-21T00:28:30.964635+0000","last_undegraded":"2024-09-21T00:28:30.964635+0000","last_fullsized":"2024-09-21T00:28:30.964635+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_clean_scrub_stamp":"2024-09-21T00:28:29.918310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:32:46.627248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.947262+0000","last_change":"2024-09-21T00:28:30.944871+0000","last_active":"2024-09-21T00:28:30.947262+0000","last_peered":"2024-09-21T00:28:30.947262+0000","last_clean":"2024-09-21T00:28:30.947262+0000","last_became_active":"2024-09-21T00:28:30.944587+0000","last_became_peered":"2024-09-21T00:28:30.944587+0000","last_unstale":"2024-09-21T00:28:30.947262+0000","last_undegraded":"2024-09-21T00:28:30.947262+0000","last_fullsized":"2024-09-21T00:28:30.947262+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_clean_scrub_stamp":"2024-09-21T00:28:29.918310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:25:52.076873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.960190+0000","last_change":"2024-09-21T00:28:30.952079+0000","last_active":"2024-09-21T00:28:30.960190+0000","last_peered":"2024-09-21T00:28:30.960190+0000","last_clean":"2024-09-21T00:28:30.960190+0000","last_became_active":"2024-09-21T00:28:30.951792+0000","last_became_peered":"2024-09-21T00:28:30.951792+0000","last_unstale":"2024-09-21T00:28:30.960190+0000","last_undegraded":"2024-09-21T00:28:30.960190+0000","last_fullsized":"2024-09-21T00:28:30.960190+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_clean_scrub_stamp":"2024-09-21T00:28:29.918310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:58:45.340669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.960540+0000","last_change":"2024-09-21T00:28:30.955689+0000","last_active":"2024-09-21T00:28:30.960540+0000","last_peered":"2024-09-21T00:28:30.960540+0000","last_clean":"2024-09-21T00:28:30.960540+0000","last_became_active":"2024-09-21T00:28:30.955385+0000","last_became_peered":"2024-09-21T00:28:30.955385+0000","last_unstale":"2024-09-21T00:28:30.960540+0000","last_undegraded":"2024-09-21T00:28:30.960540+0000","last_fullsized":"2024-09-21T00:28:30.960540+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_clean_scrub_stamp":"2024-09-21T00:28:29.918310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:52:43.572069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.947068+0000","last_change":"2024-09-21T00:28:30.944302+0000","last_active":"2024-09-21T00:28:30.947068+0000","last_peered":"2024-09-21T00:28:30.947068+0000","last_clean":"2024-09-21T00:28:30.947068+0000","last_became_active":"2024-09-21T00:28:30.943996+0000","last_became_peered":"2024-09-21T00:28:30.943996+0000","last_unstale":"2024-09-21T00:28:30.947068+0000","last_undegraded":"2024-09-21T00:28:30.947068+0000","last_fullsized":"2024-09-21T00:28:30.947068+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_clean_scrub_stamp":"2024-09-21T00:28:29.918310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T00:42:18.223655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.954559+0000","last_change":"2024-09-21T00:28:30.948888+0000","last_active":"2024-09-21T00:28:30.954559+0000","last_peered":"2024-09-21T00:28:30.954559+0000","last_clean":"2024-09-21T00:28:30.954559+0000","last_became_active":"2024-09-21T00:28:30.948412+0000","last_became_peered":"2024-09-21T00:28:30.948412+0000","last_unstale":"2024-09-21T00:28:30.954559+0000","last_undegraded":"2024-09-21T00:28:30.954559+0000","last_fullsized":"2024-09-21T00:28:30.954559+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_clean_scrub_stamp":"2024-09-21T00:28:29.918310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:52:55.340063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.954481+0000","last_change":"2024-09-21T00:28:30.950609+0000","last_active":"2024-09-21T00:28:30.954481+0000","last_peered":"2024-09-21T00:28:30.954481+0000","last_clean":"2024-09-21T00:28:30.954481+0000","last_became_active":"2024-09-21T00:28:30.950260+0000","last_became_peered":"2024-09-21T00:28:30.950260+0000","last_unstale":"2024-09-21T00:28:30.954481+0000","last_undegraded":"2024-09-21T00:28:30.954481+0000","last_fullsized":"2024-09-21T00:28:30.954481+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_clean_scrub_stamp":"2024-09-21T00:28:29.918310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:46:00.655896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T00:28:31.946991+0000","last_change":"2024-09-21T00:28:30.950423+0000","last_active":"2024-09-21T00:28:31.946991+0000","last_peered":"2024-09-21T00:28:31.946991+0000","last_clean":"2024-09-21T00:28:31.946991+0000","last_became_active":"2024-09-21T00:28:30.950106+0000","last_became_peered":"2024-09-21T00:28:30.950106+0000","last_unstale":"2024-09-21T00:28:31.946991+0000","last_undegraded":"2024-09-21T00:28:31.946991+0000","last_fullsized":"2024-09-21T00:28:31.946991+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_clean_scrub_stamp":"2024-09-21T00:28:29.918310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:04:27.994479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.960578+0000","last_change":"2024-09-21T00:28:30.957260+0000","last_active":"2024-09-21T00:28:30.960578+0000","last_peered":"2024-09-21T00:28:30.960578+0000","last_clean":"2024-09-21T00:28:30.960578+0000","last_became_active":"2024-09-21T00:28:30.956956+0000","last_became_peered":"2024-09-21T00:28:30.956956+0000","last_unstale":"2024-09-21T00:28:30.960578+0000","last_undegraded":"2024-09-21T00:28:30.960578+0000","last_fullsized":"2024-09-21T00:28:30.960578+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_clean_scrub_stamp":"2024-09-21T00:28:29.918310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:07:18.418422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.960233+0000","last_change":"2024-09-21T00:28:30.952315+0000","last_active":"2024-09-21T00:28:30.960233+0000","last_peered":"2024-09-21T00:28:30.960233+0000","last_clean":"2024-09-21T00:28:30.960233+0000","last_became_active":"2024-09-21T00:28:30.952007+0000","last_became_peered":"2024-09-21T00:28:30.952007+0000","last_unstale":"2024-09-21T00:28:30.960233+0000","last_undegraded":"2024-09-21T00:28:30.960233+0000","last_fullsized":"2024-09-21T00:28:30.960233+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_clean_scrub_stamp":"2024-09-21T00:28:29.918310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:49:46.951442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.958307+0000","last_change":"2024-09-21T00:28:30.947677+0000","last_active":"2024-09-21T00:28:30.958307+0000","last_peered":"2024-09-21T00:28:30.958307+0000","last_clean":"2024-09-21T00:28:30.958307+0000","last_became_active":"2024-09-21T00:28:30.946020+0000","last_became_peered":"2024-09-21T00:28:30.946020+0000","last_unstale":"2024-09-21T00:28:30.958307+0000","last_undegraded":"2024-09-21T00:28:30.958307+0000","last_fullsized":"2024-09-21T00:28:30.958307+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_clean_scrub_stamp":"2024-09-21T00:28:29.918310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:52:45.960884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.960172+0000","last_change":"2024-09-21T00:28:30.951954+0000","last_active":"2024-09-21T00:28:30.960172+0000","last_peered":"2024-09-21T00:28:30.960172+0000","last_clean":"2024-09-21T00:28:30.960172+0000","last_became_active":"2024-09-21T00:28:30.951492+0000","last_became_peered":"2024-09-21T00:28:30.951492+0000","last_unstale":"2024-09-21T00:28:30.960172+0000","last_undegraded":"2024-09-21T00:28:30.960172+0000","last_fullsized":"2024-09-21T00:28:30.960172+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_clean_scrub_stamp":"2024-09-21T00:28:29.918310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:48:20.244405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.957004+0000","last_change":"2024-09-21T00:28:30.945517+0000","last_active":"2024-09-21T00:28:30.957004+0000","last_peered":"2024-09-21T00:28:30.957004+0000","last_clean":"2024-09-21T00:28:30.957004+0000","last_became_active":"2024-09-21T00:28:30.945209+0000","last_became_peered":"2024-09-21T00:28:30.945209+0000","last_unstale":"2024-09-21T00:28:30.957004+0000","last_undegraded":"2024-09-21T00:28:30.957004+0000","last_fullsized":"2024-09-21T00:28:30.957004+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_clean_scrub_stamp":"2024-09-21T00:28:29.918310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:26:08.351369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.954477+0000","last_change":"2024-09-21T00:28:30.947336+0000","last_active":"2024-09-21T00:28:30.954477+0000","last_peered":"2024-09-21T00:28:30.954477+0000","last_clean":"2024-09-21T00:28:30.954477+0000","last_became_active":"2024-09-21T00:28:30.946783+0000","last_became_peered":"2024-09-21T00:28:30.946783+0000","last_unstale":"2024-09-21T00:28:30.954477+0000","last_undegraded":"2024-09-21T00:28:30.954477+0000","last_fullsized":"2024-09-21T00:28:30.954477+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_clean_scrub_stamp":"2024-09-21T00:28:29.918310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:46:23.384000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.954438+0000","last_change":"2024-09-21T00:28:30.947086+0000","last_active":"2024-09-21T00:28:30.954438+0000","last_peered":"2024-09-21T00:28:30.954438+0000","last_clean":"2024-09-21T00:28:30.954438+0000","last_became_active":"2024-09-21T00:28:30.946737+0000","last_became_peered":"2024-09-21T00:28:30.946737+0000","last_unstale":"2024-09-21T00:28:30.954438+0000","last_undegraded":"2024-09-21T00:28:30.954438+0000","last_fullsized":"2024-09-21T00:28:30.954438+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_clean_scrub_stamp":"2024-09-21T00:28:29.918310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:15:13.025831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.947053+0000","last_change":"2024-09-21T00:28:30.944085+0000","last_active":"2024-09-21T00:28:30.947053+0000","last_peered":"2024-09-21T00:28:30.947053+0000","last_clean":"2024-09-21T00:28:30.947053+0000","last_became_active":"2024-09-21T00:28:30.943774+0000","last_became_peered":"2024-09-21T00:28:30.943774+0000","last_unstale":"2024-09-21T00:28:30.947053+0000","last_undegraded":"2024-09-21T00:28:30.947053+0000","last_fullsized":"2024-09-21T00:28:30.947053+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_clean_scrub_stamp":"2024-09-21T00:28:29.918310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:11:52.856844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.954406+0000","last_change":"2024-09-21T00:28:30.948912+0000","last_active":"2024-09-21T00:28:30.954406+0000","last_peered":"2024-09-21T00:28:30.954406+0000","last_clean":"2024-09-21T00:28:30.954406+0000","last_became_active":"2024-09-21T00:28:30.948572+0000","last_became_peered":"2024-09-21T00:28:30.948572+0000","last_unstale":"2024-09-21T00:28:30.954406+0000","last_undegraded":"2024-09-21T00:28:30.954406+0000","last_fullsized":"2024-09-21T00:28:30.954406+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_clean_scrub_stamp":"2024-09-21T00:28:29.918310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:45:08.521920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.958867+0000","last_change":"2024-09-21T00:28:30.951013+0000","last_active":"2024-09-21T00:28:30.958867+0000","last_peered":"2024-09-21T00:28:30.958867+0000","last_clean":"2024-09-21T00:28:30.958867+0000","last_became_active":"2024-09-21T00:28:30.950631+0000","last_became_peered":"2024-09-21T00:28:30.950631+0000","last_unstale":"2024-09-21T00:28:30.958867+0000","last_undegraded":"2024-09-21T00:28:30.958867+0000","last_fullsized":"2024-09-21T00:28:30.958867+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_clean_scrub_stamp":"2024-09-21T00:28:29.918310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:50:35.184359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.954066+0000","last_change":"2024-09-21T00:28:30.948178+0000","last_active":"2024-09-21T00:28:30.954066+0000","last_peered":"2024-09-21T00:28:30.954066+0000","last_clean":"2024-09-21T00:28:30.954066+0000","last_became_active":"2024-09-21T00:28:30.947853+0000","last_became_peered":"2024-09-21T00:28:30.947853+0000","last_unstale":"2024-09-21T00:28:30.954066+0000","last_undegraded":"2024-09-21T00:28:30.954066+0000","last_fullsized":"2024-09-21T00:28:30.954066+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_clean_scrub_stamp":"2024-09-21T00:28:29.918310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:08:59.115916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.956455+0000","last_change":"2024-09-21T00:28:30.953941+0000","last_active":"2024-09-21T00:28:30.956455+0000","last_peered":"2024-09-21T00:28:30.956455+0000","last_clean":"2024-09-21T00:28:30.956455+0000","last_became_active":"2024-09-21T00:28:30.953576+0000","last_became_peered":"2024-09-21T00:28:30.953576+0000","last_unstale":"2024-09-21T00:28:30.956455+0000","last_undegraded":"2024-09-21T00:28:30.956455+0000","last_fullsized":"2024-09-21T00:28:30.956455+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_clean_scrub_stamp":"2024-09-21T00:28:29.918310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:32:46.627248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T00:28:31.946762+0000","last_change":"2024-09-21T00:28:30.959027+0000","last_active":"2024-09-21T00:28:31.946762+0000","last_peered":"2024-09-21T00:28:31.946762+0000","last_clean":"2024-09-21T00:28:31.946762+0000","last_became_active":"2024-09-21T00:28:30.958691+0000","last_became_peered":"2024-09-21T00:28:30.958691+0000","last_unstale":"2024-09-21T00:28:31.946762+0000","last_undegraded":"2024-09-21T00:28:31.946762+0000","last_fullsized":"2024-09-21T00:28:31.946762+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_clean_scrub_stamp":"2024-09-21T00:28:29.918310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:08:05.070338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.954618+0000","last_change":"2024-09-21T00:28:30.949895+0000","last_active":"2024-09-21T00:28:30.954618+0000","last_peered":"2024-09-21T00:28:30.954618+0000","last_clean":"2024-09-21T00:28:30.954618+0000","last_became_active":"2024-09-21T00:28:30.949595+0000","last_became_peered":"2024-09-21T00:28:30.949595+0000","last_unstale":"2024-09-21T00:28:30.954618+0000","last_undegraded":"2024-09-21T00:28:30.954618+0000","last_fullsized":"2024-09-21T00:28:30.954618+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_clean_scrub_stamp":"2024-09-21T00:28:29.918310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:56:01.061375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T00:28:31.946736+0000","last_change":"2024-09-21T00:28:30.949421+0000","last_active":"2024-09-21T00:28:31.946736+0000","last_peered":"2024-09-21T00:28:31.946736+0000","last_clean":"2024-09-21T00:28:31.946736+0000","last_became_active":"2024-09-21T00:28:30.949106+0000","last_became_peered":"2024-09-21T00:28:30.949106+0000","last_unstale":"2024-09-21T00:28:31.946736+0000","last_undegraded":"2024-09-21T00:28:31.946736+0000","last_fullsized":"2024-09-21T00:28:31.946736+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_clean_scrub_stamp":"2024-09-21T00:28:29.918310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:30:33.541213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"70'4","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.966527+0000","last_change":"2024-09-21T00:28:29.943891+0000","last_active":"2024-09-21T00:28:30.966527+0000","last_peered":"2024-09-21T00:28:30.966527+0000","last_clean":"2024-09-21T00:28:30.966527+0000","last_became_active":"2024-09-21T00:28:29.943594+0000","last_became_peered":"2024-09-21T00:28:29.943594+0000","last_unstale":"2024-09-21T00:28:30.966527+0000","last_undegraded":"2024-09-21T00:28:30.966527+0000","last_fullsized":"2024-09-21T00:28:30.966527+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:28.914450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:28.914450+0000","last_clean_scrub_stamp":"2024-09-21T00:28:28.914450+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:35:07.915069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.943167+0000","last_change":"2024-09-21T00:28:30.939893+0000","last_active":"2024-09-21T00:28:30.943167+0000","last_peered":"2024-09-21T00:28:30.943167+0000","last_clean":"2024-09-21T00:28:30.943167+0000","last_became_active":"2024-09-21T00:28:30.939601+0000","last_became_peered":"2024-09-21T00:28:30.939601+0000","last_unstale":"2024-09-21T00:28:30.943167+0000","last_undegraded":"2024-09-21T00:28:30.943167+0000","last_fullsized":"2024-09-21T00:28:30.943167+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_clean_scrub_stamp":"2024-09-21T00:28:29.918310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:33:41.972645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"70'1","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.959405+0000","last_change":"2024-09-21T00:28:29.948845+0000","last_active":"2024-09-21T00:28:30.959405+0000","last_peered":"2024-09-21T00:28:30.959405+0000","last_clean":"2024-09-21T00:28:30.959405+0000","last_became_active":"2024-09-21T00:28:29.948328+0000","last_became_peered":"2024-09-21T00:28:29.948328+0000","last_unstale":"2024-09-21T00:28:30.959405+0000","last_undegraded":"2024-09-21T00:28:30.959405+0000","last_fullsized":"2024-09-21T00:28:30.959405+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:28.914450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:28.914450+0000","last_clean_scrub_stamp":"2024-09-21T00:28:28.914450+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T00:54:42.550514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.956826+0000","last_change":"2024-09-21T00:28:30.952712+0000","last_active":"2024-09-21T00:28:30.956826+0000","last_peered":"2024-09-21T00:28:30.956826+0000","last_clean":"2024-09-21T00:28:30.956826+0000","last_became_active":"2024-09-21T00:28:30.952085+0000","last_became_peered":"2024-09-21T00:28:30.952085+0000","last_unstale":"2024-09-21T00:28:30.956826+0000","last_undegraded":"2024-09-21T00:28:30.956826+0000","last_fullsized":"2024-09-21T00:28:30.956826+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_clean_scrub_stamp":"2024-09-21T00:28:29.918310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:08:59.826700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"70'2","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.958755+0000","last_change":"2024-09-21T00:28:29.946568+0000","last_active":"2024-09-21T00:28:30.958755+0000","last_peered":"2024-09-21T00:28:30.958755+0000","last_clean":"2024-09-21T00:28:30.958755+0000","last_became_active":"2024-09-21T00:28:29.945758+0000","last_became_peered":"2024-09-21T00:28:29.945758+0000","last_unstale":"2024-09-21T00:28:30.958755+0000","last_undegraded":"2024-09-21T00:28:30.958755+0000","last_fullsized":"2024-09-21T00:28:30.958755+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:28.914450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:28.914450+0000","last_clean_scrub_stamp":"2024-09-21T00:28:28.914450+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:07:58.312580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.955625+0000","last_change":"2024-09-21T00:28:30.944552+0000","last_active":"2024-09-21T00:28:30.955625+0000","last_peered":"2024-09-21T00:28:30.955625+0000","last_clean":"2024-09-21T00:28:30.955625+0000","last_became_active":"2024-09-21T00:28:30.944241+0000","last_became_peered":"2024-09-21T00:28:30.944241+0000","last_unstale":"2024-09-21T00:28:30.955625+0000","last_undegraded":"2024-09-21T00:28:30.955625+0000","last_fullsized":"2024-09-21T00:28:30.955625+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_clean_scrub_stamp":"2024-09-21T00:28:29.918310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:40:55.852394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"70'2","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.960450+0000","last_change":"2024-09-21T00:28:29.945862+0000","last_active":"2024-09-21T00:28:30.960450+0000","last_peered":"2024-09-21T00:28:30.960450+0000","last_clean":"2024-09-21T00:28:30.960450+0000","last_became_active":"2024-09-21T00:28:29.945557+0000","last_became_peered":"2024-09-21T00:28:29.945557+0000","last_unstale":"2024-09-21T00:28:30.960450+0000","last_undegraded":"2024-09-21T00:28:30.960450+0000","last_fullsized":"2024-09-21T00:28:30.960450+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:28.914450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:28.914450+0000","last_clean_scrub_stamp":"2024-09-21T00:28:28.914450+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:37:13.568386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.957513+0000","last_change":"2024-09-21T00:28:30.953402+0000","last_active":"2024-09-21T00:28:30.957513+0000","last_peered":"2024-09-21T00:28:30.957513+0000","last_clean":"2024-09-21T00:28:30.957513+0000","last_became_active":"2024-09-21T00:28:30.953047+0000","last_became_peered":"2024-09-21T00:28:30.953047+0000","last_unstale":"2024-09-21T00:28:30.957513+0000","last_undegraded":"2024-09-21T00:28:30.957513+0000","last_fullsized":"2024-09-21T00:28:30.957513+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_clean_scrub_stamp":"2024-09-21T00:28:29.918310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:45:08.521920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"70'1","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.962274+0000","last_change":"2024-09-21T00:28:29.938518+0000","last_active":"2024-09-21T00:28:30.962274+0000","last_peered":"2024-09-21T00:28:30.962274+0000","last_clean":"2024-09-21T00:28:30.962274+0000","last_became_active":"2024-09-21T00:28:29.938155+0000","last_became_peered":"2024-09-21T00:28:29.938155+0000","last_unstale":"2024-09-21T00:28:30.962274+0000","last_undegraded":"2024-09-21T00:28:30.962274+0000","last_fullsized":"2024-09-21T00:28:30.962274+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:28.914450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:28.914450+0000","last_clean_scrub_stamp":"2024-09-21T00:28:28.914450+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:08:56.500914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.964599+0000","last_change":"2024-09-21T00:28:30.961806+0000","last_active":"2024-09-21T00:28:30.964599+0000","last_peered":"2024-09-21T00:28:30.964599+0000","last_clean":"2024-09-21T00:28:30.964599+0000","last_became_active":"2024-09-21T00:28:30.961157+0000","last_became_peered":"2024-09-21T00:28:30.961157+0000","last_unstale":"2024-09-21T00:28:30.964599+0000","last_undegraded":"2024-09-21T00:28:30.964599+0000","last_fullsized":"2024-09-21T00:28:30.964599+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_clean_scrub_stamp":"2024-09-21T00:28:29.918310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:52:43.572069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"70'1","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.958773+0000","last_change":"2024-09-21T00:28:29.944301+0000","last_active":"2024-09-21T00:28:30.958773+0000","last_peered":"2024-09-21T00:28:30.958773+0000","last_clean":"2024-09-21T00:28:30.958773+0000","last_became_active":"2024-09-21T00:28:29.943978+0000","last_became_peered":"2024-09-21T00:28:29.943978+0000","last_unstale":"2024-09-21T00:28:30.958773+0000","last_undegraded":"2024-09-21T00:28:30.958773+0000","last_fullsized":"2024-09-21T00:28:30.958773+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:28.914450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:28.914450+0000","last_clean_scrub_stamp":"2024-09-21T00:28:28.914450+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:47:51.557791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.956979+0000","last_change":"2024-09-21T00:28:30.947405+0000","last_active":"2024-09-21T00:28:30.956979+0000","last_peered":"2024-09-21T00:28:30.956979+0000","last_clean":"2024-09-21T00:28:30.956979+0000","last_became_active":"2024-09-21T00:28:30.947013+0000","last_became_peered":"2024-09-21T00:28:30.947013+0000","last_unstale":"2024-09-21T00:28:30.956979+0000","last_undegraded":"2024-09-21T00:28:30.956979+0000","last_fullsized":"2024-09-21T00:28:30.956979+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_clean_scrub_stamp":"2024-09-21T00:28:29.918310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:55:15.836726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.928104+0000","last_change":"2024-09-21T00:28:29.946314+0000","last_active":"2024-09-21T00:28:30.928104+0000","last_peered":"2024-09-21T00:28:30.928104+0000","last_clean":"2024-09-21T00:28:30.928104+0000","last_became_active":"2024-09-21T00:28:29.945966+0000","last_became_peered":"2024-09-21T00:28:29.945966+0000","last_unstale":"2024-09-21T00:28:30.928104+0000","last_undegraded":"2024-09-21T00:28:30.928104+0000","last_fullsized":"2024-09-21T00:28:30.928104+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:28.914450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:28.914450+0000","last_clean_scrub_stamp":"2024-09-21T00:28:28.914450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:23:09.381529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.963465+0000","last_change":"2024-09-21T00:28:30.957296+0000","last_active":"2024-09-21T00:28:30.963465+0000","last_peered":"2024-09-21T00:28:30.963465+0000","last_clean":"2024-09-21T00:28:30.963465+0000","last_became_active":"2024-09-21T00:28:30.956981+0000","last_became_peered":"2024-09-21T00:28:30.956981+0000","last_unstale":"2024-09-21T00:28:30.963465+0000","last_undegraded":"2024-09-21T00:28:30.963465+0000","last_fullsized":"2024-09-21T00:28:30.963465+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_clean_scrub_stamp":"2024-09-21T00:28:29.918310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:14:48.640699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.936129+0000","last_change":"2024-09-21T00:28:29.941559+0000","last_active":"2024-09-21T00:28:30.936129+0000","last_peered":"2024-09-21T00:28:30.936129+0000","last_clean":"2024-09-21T00:28:30.936129+0000","last_became_active":"2024-09-21T00:28:29.941178+0000","last_became_peered":"2024-09-21T00:28:29.941178+0000","last_unstale":"2024-09-21T00:28:30.936129+0000","last_undegraded":"2024-09-21T00:28:30.936129+0000","last_fullsized":"2024-09-21T00:28:30.936129+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:28.914450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:28.914450+0000","last_clean_scrub_stamp":"2024-09-21T00:28:28.914450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:01:16.191815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.944629+0000","last_change":"2024-09-21T00:28:30.942634+0000","last_active":"2024-09-21T00:28:30.944629+0000","last_peered":"2024-09-21T00:28:30.944629+0000","last_clean":"2024-09-21T00:28:30.944629+0000","last_became_active":"2024-09-21T00:28:30.942316+0000","last_became_peered":"2024-09-21T00:28:30.942316+0000","last_unstale":"2024-09-21T00:28:30.944629+0000","last_undegraded":"2024-09-21T00:28:30.944629+0000","last_fullsized":"2024-09-21T00:28:30.944629+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_clean_scrub_stamp":"2024-09-21T00:28:29.918310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:52:35.974242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.941799+0000","last_change":"2024-09-21T00:28:29.948717+0000","last_active":"2024-09-21T00:28:30.941799+0000","last_peered":"2024-09-21T00:28:30.941799+0000","last_clean":"2024-09-21T00:28:30.941799+0000","last_became_active":"2024-09-21T00:28:29.948358+0000","last_became_peered":"2024-09-21T00:28:29.948358+0000","last_unstale":"2024-09-21T00:28:30.941799+0000","last_undegraded":"2024-09-21T00:28:30.941799+0000","last_fullsized":"2024-09-21T00:28:30.941799+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:28.914450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:28.914450+0000","last_clean_scrub_stamp":"2024-09-21T00:28:28.914450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:46:22.380140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.953284+0000","last_change":"2024-09-21T00:28:30.950082+0000","last_active":"2024-09-21T00:28:30.953284+0000","last_peered":"2024-09-21T00:28:30.953284+0000","last_clean":"2024-09-21T00:28:30.953284+0000","last_became_active":"2024-09-21T00:28:30.949686+0000","last_became_peered":"2024-09-21T00:28:30.949686+0000","last_unstale":"2024-09-21T00:28:30.953284+0000","last_undegraded":"2024-09-21T00:28:30.953284+0000","last_fullsized":"2024-09-21T00:28:30.953284+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_clean_scrub_stamp":"2024-09-21T00:28:29.918310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:24:05.849633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.934710+0000","last_change":"2024-09-21T00:28:29.946140+0000","last_active":"2024-09-21T00:28:30.934710+0000","last_peered":"2024-09-21T00:28:30.934710+0000","last_clean":"2024-09-21T00:28:30.934710+0000","last_became_active":"2024-09-21T00:28:29.945761+0000","last_became_peered":"2024-09-21T00:28:29.945761+0000","last_unstale":"2024-09-21T00:28:30.934710+0000","last_undegraded":"2024-09-21T00:28:30.934710+0000","last_fullsized":"2024-09-21T00:28:30.934710+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:28.914450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:28.914450+0000","last_clean_scrub_stamp":"2024-09-21T00:28:28.914450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:22:12.208115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.954771+0000","last_change":"2024-09-21T00:28:30.946678+0000","last_active":"2024-09-21T00:28:30.954771+0000","last_peered":"2024-09-21T00:28:30.954771+0000","last_clean":"2024-09-21T00:28:30.954771+0000","last_became_active":"2024-09-21T00:28:30.946226+0000","last_became_peered":"2024-09-21T00:28:30.946226+0000","last_unstale":"2024-09-21T00:28:30.954771+0000","last_undegraded":"2024-09-21T00:28:30.954771+0000","last_fullsized":"2024-09-21T00:28:30.954771+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_clean_scrub_stamp":"2024-09-21T00:28:29.918310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:52:55.340063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"70'3","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.966600+0000","last_change":"2024-09-21T00:28:29.940783+0000","last_active":"2024-09-21T00:28:30.966600+0000","last_peered":"2024-09-21T00:28:30.966600+0000","last_clean":"2024-09-21T00:28:30.966600+0000","last_became_active":"2024-09-21T00:28:29.940449+0000","last_became_peered":"2024-09-21T00:28:29.940449+0000","last_unstale":"2024-09-21T00:28:30.966600+0000","last_undegraded":"2024-09-21T00:28:30.966600+0000","last_fullsized":"2024-09-21T00:28:30.966600+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:28.914450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:28.914450+0000","last_clean_scrub_stamp":"2024-09-21T00:28:28.914450+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:47:28.342192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T00:28:31.946792+0000","last_change":"2024-09-21T00:28:30.946138+0000","last_active":"2024-09-21T00:28:31.946792+0000","last_peered":"2024-09-21T00:28:31.946792+0000","last_clean":"2024-09-21T00:28:31.946792+0000","last_became_active":"2024-09-21T00:28:30.945800+0000","last_became_peered":"2024-09-21T00:28:30.945800+0000","last_unstale":"2024-09-21T00:28:31.946792+0000","last_undegraded":"2024-09-21T00:28:31.946792+0000","last_fullsized":"2024-09-21T00:28:31.946792+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_clean_scrub_stamp":"2024-09-21T00:28:29.918310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:19:59.582905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"70'6","reported_seq":19,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.970142+0000","last_change":"2024-09-21T00:28:29.942822+0000","last_active":"2024-09-21T00:28:30.970142+0000","last_peered":"2024-09-21T00:28:30.970142+0000","last_clean":"2024-09-21T00:28:30.970142+0000","last_became_active":"2024-09-21T00:28:29.942389+0000","last_became_peered":"2024-09-21T00:28:29.942389+0000","last_unstale":"2024-09-21T00:28:30.970142+0000","last_undegraded":"2024-09-21T00:28:30.970142+0000","last_fullsized":"2024-09-21T00:28:30.970142+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:28.914450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:28.914450+0000","last_clean_scrub_stamp":"2024-09-21T00:28:28.914450+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:10:24.411387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.952392+0000","last_change":"2024-09-21T00:28:30.947393+0000","last_active":"2024-09-21T00:28:30.952392+0000","last_peered":"2024-09-21T00:28:30.952392+0000","last_clean":"2024-09-21T00:28:30.952392+0000","last_became_active":"2024-09-21T00:28:30.946923+0000","last_became_peered":"2024-09-21T00:28:30.946923+0000","last_unstale":"2024-09-21T00:28:30.952392+0000","last_undegraded":"2024-09-21T00:28:30.952392+0000","last_fullsized":"2024-09-21T00:28:30.952392+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_clean_scrub_stamp":"2024-09-21T00:28:29.918310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:16:33.054120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"70'2","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.962769+0000","last_change":"2024-09-21T00:28:29.934927+0000","last_active":"2024-09-21T00:28:30.962769+0000","last_peered":"2024-09-21T00:28:30.962769+0000","last_clean":"2024-09-21T00:28:30.962769+0000","last_became_active":"2024-09-21T00:28:29.934601+0000","last_became_peered":"2024-09-21T00:28:29.934601+0000","last_unstale":"2024-09-21T00:28:30.962769+0000","last_undegraded":"2024-09-21T00:28:30.962769+0000","last_fullsized":"2024-09-21T00:28:30.962769+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:28.914450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:28.914450+0000","last_clean_scrub_stamp":"2024-09-21T00:28:28.914450+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:33:46.518632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.965265+0000","last_change":"2024-09-21T00:28:30.960338+0000","last_active":"2024-09-21T00:28:30.965265+0000","last_peered":"2024-09-21T00:28:30.965265+0000","last_clean":"2024-09-21T00:28:30.965265+0000","last_became_active":"2024-09-21T00:28:30.960006+0000","last_became_peered":"2024-09-21T00:28:30.960006+0000","last_unstale":"2024-09-21T00:28:30.965265+0000","last_undegraded":"2024-09-21T00:28:30.965265+0000","last_fullsized":"2024-09-21T00:28:30.965265+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_clean_scrub_stamp":"2024-09-21T00:28:29.918310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:13:49.866866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.938388+0000","last_change":"2024-09-21T00:28:29.949796+0000","last_active":"2024-09-21T00:28:30.938388+0000","last_peered":"2024-09-21T00:28:30.938388+0000","last_clean":"2024-09-21T00:28:30.938388+0000","last_became_active":"2024-09-21T00:28:29.949527+0000","last_became_peered":"2024-09-21T00:28:29.949527+0000","last_unstale":"2024-09-21T00:28:30.938388+0000","last_undegraded":"2024-09-21T00:28:30.938388+0000","last_fullsized":"2024-09-21T00:28:30.938388+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:28.914450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:28.914450+0000","last_clean_scrub_stamp":"2024-09-21T00:28:28.914450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:52:34.970382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.965221+0000","last_change":"2024-09-21T00:28:30.959923+0000","last_active":"2024-09-21T00:28:30.965221+0000","last_peered":"2024-09-21T00:28:30.965221+0000","last_clean":"2024-09-21T00:28:30.965221+0000","last_became_active":"2024-09-21T00:28:30.959157+0000","last_became_peered":"2024-09-21T00:28:30.959157+0000","last_unstale":"2024-09-21T00:28:30.965221+0000","last_undegraded":"2024-09-21T00:28:30.965221+0000","last_fullsized":"2024-09-21T00:28:30.965221+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_clean_scrub_stamp":"2024-09-21T00:28:29.918310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:29:22.486616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.936033+0000","last_change":"2024-09-21T00:28:29.944758+0000","last_active":"2024-09-21T00:28:30.936033+0000","last_peered":"2024-09-21T00:28:30.936033+0000","last_clean":"2024-09-21T00:28:30.936033+0000","last_became_active":"2024-09-21T00:28:29.944330+0000","last_became_peered":"2024-09-21T00:28:29.944330+0000","last_unstale":"2024-09-21T00:28:30.936033+0000","last_undegraded":"2024-09-21T00:28:30.936033+0000","last_fullsized":"2024-09-21T00:28:30.936033+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:28.914450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:28.914450+0000","last_clean_scrub_stamp":"2024-09-21T00:28:28.914450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:42:26.100740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.959653+0000","last_change":"2024-09-21T00:28:30.954447+0000","last_active":"2024-09-21T00:28:30.959653+0000","last_peered":"2024-09-21T00:28:30.959653+0000","last_clean":"2024-09-21T00:28:30.959653+0000","last_became_active":"2024-09-21T00:28:30.954135+0000","last_became_peered":"2024-09-21T00:28:30.954135+0000","last_unstale":"2024-09-21T00:28:30.959653+0000","last_undegraded":"2024-09-21T00:28:30.959653+0000","last_fullsized":"2024-09-21T00:28:30.959653+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_clean_scrub_stamp":"2024-09-21T00:28:29.918310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:26:56.342443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"70'8","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.966633+0000","last_change":"2024-09-21T00:28:29.942001+0000","last_active":"2024-09-21T00:28:30.966633+0000","last_peered":"2024-09-21T00:28:30.966633+0000","last_clean":"2024-09-21T00:28:30.966633+0000","last_became_active":"2024-09-21T00:28:29.941566+0000","last_became_peered":"2024-09-21T00:28:29.941566+0000","last_unstale":"2024-09-21T00:28:30.966633+0000","last_undegraded":"2024-09-21T00:28:30.966633+0000","last_fullsized":"2024-09-21T00:28:30.966633+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:28.914450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:28.914450+0000","last_clean_scrub_stamp":"2024-09-21T00:28:28.914450+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:55:47.073868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.960623+0000","last_change":"2024-09-21T00:28:30.956886+0000","last_active":"2024-09-21T00:28:30.960623+0000","last_peered":"2024-09-21T00:28:30.960623+0000","last_clean":"2024-09-21T00:28:30.960623+0000","last_became_active":"2024-09-21T00:28:30.956468+0000","last_became_peered":"2024-09-21T00:28:30.956468+0000","last_unstale":"2024-09-21T00:28:30.960623+0000","last_undegraded":"2024-09-21T00:28:30.960623+0000","last_fullsized":"2024-09-21T00:28:30.960623+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_clean_scrub_stamp":"2024-09-21T00:28:29.918310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:32:46.627248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"70'1","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.959291+0000","last_change":"2024-09-21T00:28:29.947764+0000","last_active":"2024-09-21T00:28:30.959291+0000","last_peered":"2024-09-21T00:28:30.959291+0000","last_clean":"2024-09-21T00:28:30.959291+0000","last_became_active":"2024-09-21T00:28:29.947398+0000","last_became_peered":"2024-09-21T00:28:29.947398+0000","last_unstale":"2024-09-21T00:28:30.959291+0000","last_undegraded":"2024-09-21T00:28:30.959291+0000","last_fullsized":"2024-09-21T00:28:30.959291+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:28.914450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:28.914450+0000","last_clean_scrub_stamp":"2024-09-21T00:28:28.914450+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:08:58.112056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.958949+0000","last_change":"2024-09-21T00:28:30.950953+0000","last_active":"2024-09-21T00:28:30.958949+0000","last_peered":"2024-09-21T00:28:30.958949+0000","last_clean":"2024-09-21T00:28:30.958949+0000","last_became_active":"2024-09-21T00:28:30.950588+0000","last_became_peered":"2024-09-21T00:28:30.950588+0000","last_unstale":"2024-09-21T00:28:30.958949+0000","last_undegraded":"2024-09-21T00:28:30.958949+0000","last_fullsized":"2024-09-21T00:28:30.958949+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_clean_scrub_stamp":"2024-09-21T00:28:29.918310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:05:10.437419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"70'2","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.963623+0000","last_change":"2024-09-21T00:28:29.941321+0000","last_active":"2024-09-21T00:28:30.963623+0000","last_peered":"2024-09-21T00:28:30.963623+0000","last_clean":"2024-09-21T00:28:30.963623+0000","last_became_active":"2024-09-21T00:28:29.940975+0000","last_became_peered":"2024-09-21T00:28:29.940975+0000","last_unstale":"2024-09-21T00:28:30.963623+0000","last_undegraded":"2024-09-21T00:28:30.963623+0000","last_fullsized":"2024-09-21T00:28:30.963623+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:28.914450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:28.914450+0000","last_clean_scrub_stamp":"2024-09-21T00:28:28.914450+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:31:49.583633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.950572+0000","last_change":"2024-09-21T00:28:30.947463+0000","last_active":"2024-09-21T00:28:30.950572+0000","last_peered":"2024-09-21T00:28:30.950572+0000","last_clean":"2024-09-21T00:28:30.950572+0000","last_became_active":"2024-09-21T00:28:30.947118+0000","last_became_peered":"2024-09-21T00:28:30.947118+0000","last_unstale":"2024-09-21T00:28:30.950572+0000","last_undegraded":"2024-09-21T00:28:30.950572+0000","last_fullsized":"2024-09-21T00:28:30.950572+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_clean_scrub_stamp":"2024-09-21T00:28:29.918310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:03:50.174406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"70'1","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.966506+0000","last_change":"2024-09-21T00:28:29.940181+0000","last_active":"2024-09-21T00:28:30.966506+0000","last_peered":"2024-09-21T00:28:30.966506+0000","last_clean":"2024-09-21T00:28:30.966506+0000","last_became_active":"2024-09-21T00:28:29.939780+0000","last_became_peered":"2024-09-21T00:28:29.939780+0000","last_unstale":"2024-09-21T00:28:30.966506+0000","last_undegraded":"2024-09-21T00:28:30.966506+0000","last_fullsized":"2024-09-21T00:28:30.966506+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:28.914450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:28.914450+0000","last_clean_scrub_stamp":"2024-09-21T00:28:28.914450+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:44:51.450242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.959165+0000","last_change":"2024-09-21T00:28:30.954744+0000","last_active":"2024-09-21T00:28:30.959165+0000","last_peered":"2024-09-21T00:28:30.959165+0000","last_clean":"2024-09-21T00:28:30.959165+0000","last_became_active":"2024-09-21T00:28:30.954447+0000","last_became_peered":"2024-09-21T00:28:30.954447+0000","last_unstale":"2024-09-21T00:28:30.959165+0000","last_undegraded":"2024-09-21T00:28:30.959165+0000","last_fullsized":"2024-09-21T00:28:30.959165+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_clean_scrub_stamp":"2024-09-21T00:28:29.918310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:56:01.061375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T00:28:31.946799+0000","last_change":"2024-09-21T00:28:29.936072+0000","last_active":"2024-09-21T00:28:31.946799+0000","last_peered":"2024-09-21T00:28:31.946799+0000","last_clean":"2024-09-21T00:28:31.946799+0000","last_became_active":"2024-09-21T00:28:29.935715+0000","last_became_peered":"2024-09-21T00:28:29.935715+0000","last_unstale":"2024-09-21T00:28:31.946799+0000","last_undegraded":"2024-09-21T00:28:31.946799+0000","last_fullsized":"2024-09-21T00:28:31.946799+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:28.914450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:28.914450+0000","last_clean_scrub_stamp":"2024-09-21T00:28:28.914450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:46:25.390596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.947585+0000","last_change":"2024-09-21T00:28:30.944110+0000","last_active":"2024-09-21T00:28:30.947585+0000","last_peered":"2024-09-21T00:28:30.947585+0000","last_clean":"2024-09-21T00:28:30.947585+0000","last_became_active":"2024-09-21T00:28:30.943750+0000","last_became_peered":"2024-09-21T00:28:30.943750+0000","last_unstale":"2024-09-21T00:28:30.947585+0000","last_undegraded":"2024-09-21T00:28:30.947585+0000","last_fullsized":"2024-09-21T00:28:30.947585+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_clean_scrub_stamp":"2024-09-21T00:28:29.918310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T00:54:43.554374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"70'2","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.963044+0000","last_change":"2024-09-21T00:28:29.946672+0000","last_active":"2024-09-21T00:28:30.963044+0000","last_peered":"2024-09-21T00:28:30.963044+0000","last_clean":"2024-09-21T00:28:30.963044+0000","last_became_active":"2024-09-21T00:28:29.945563+0000","last_became_peered":"2024-09-21T00:28:29.945563+0000","last_unstale":"2024-09-21T00:28:30.963044+0000","last_undegraded":"2024-09-21T00:28:30.963044+0000","last_fullsized":"2024-09-21T00:28:30.963044+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:28.914450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:28.914450+0000","last_clean_scrub_stamp":"2024-09-21T00:28:28.914450+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:45:07.518060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.949592+0000","last_change":"2024-09-21T00:28:30.945284+0000","last_active":"2024-09-21T00:28:30.949592+0000","last_peered":"2024-09-21T00:28:30.949592+0000","last_clean":"2024-09-21T00:28:30.949592+0000","last_became_active":"2024-09-21T00:28:30.944965+0000","last_became_peered":"2024-09-21T00:28:30.944965+0000","last_unstale":"2024-09-21T00:28:30.949592+0000","last_undegraded":"2024-09-21T00:28:30.949592+0000","last_fullsized":"2024-09-21T00:28:30.949592+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_clean_scrub_stamp":"2024-09-21T00:28:29.918310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:07:59.316440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.939986+0000","last_change":"2024-09-21T00:28:29.945512+0000","last_active":"2024-09-21T00:28:30.939986+0000","last_peered":"2024-09-21T00:28:30.939986+0000","last_clean":"2024-09-21T00:28:30.939986+0000","last_became_active":"2024-09-21T00:28:29.945083+0000","last_became_peered":"2024-09-21T00:28:29.945083+0000","last_unstale":"2024-09-21T00:28:30.939986+0000","last_undegraded":"2024-09-21T00:28:30.939986+0000","last_fullsized":"2024-09-21T00:28:30.939986+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:28.914450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:28.914450+0000","last_clean_scrub_stamp":"2024-09-21T00:28:28.914450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:15:12.021971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.965165+0000","last_change":"2024-09-21T00:28:30.959001+0000","last_active":"2024-09-21T00:28:30.965165+0000","last_peered":"2024-09-21T00:28:30.965165+0000","last_clean":"2024-09-21T00:28:30.965165+0000","last_became_active":"2024-09-21T00:28:30.957614+0000","last_became_peered":"2024-09-21T00:28:30.957614+0000","last_unstale":"2024-09-21T00:28:30.965165+0000","last_undegraded":"2024-09-21T00:28:30.965165+0000","last_fullsized":"2024-09-21T00:28:30.965165+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_clean_scrub_stamp":"2024-09-21T00:28:29.918310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:14:08.603016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.940247+0000","last_change":"2024-09-21T00:28:29.945343+0000","last_active":"2024-09-21T00:28:30.940247+0000","last_peered":"2024-09-21T00:28:30.940247+0000","last_clean":"2024-09-21T00:28:30.940247+0000","last_became_active":"2024-09-21T00:28:29.945024+0000","last_became_peered":"2024-09-21T00:28:29.945024+0000","last_unstale":"2024-09-21T00:28:30.940247+0000","last_undegraded":"2024-09-21T00:28:30.940247+0000","last_fullsized":"2024-09-21T00:28:30.940247+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:28.914450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:28.914450+0000","last_clean_scrub_stamp":"2024-09-21T00:28:28.914450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:46:22.380140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.958863+0000","last_change":"2024-09-21T00:28:30.955844+0000","last_active":"2024-09-21T00:28:30.958863+0000","last_peered":"2024-09-21T00:28:30.958863+0000","last_clean":"2024-09-21T00:28:30.958863+0000","last_became_active":"2024-09-21T00:28:30.955212+0000","last_became_peered":"2024-09-21T00:28:30.955212+0000","last_unstale":"2024-09-21T00:28:30.958863+0000","last_undegraded":"2024-09-21T00:28:30.958863+0000","last_fullsized":"2024-09-21T00:28:30.958863+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_clean_scrub_stamp":"2024-09-21T00:28:29.918310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:53:36.681496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"70'2","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.956268+0000","last_change":"2024-09-21T00:28:29.939267+0000","last_active":"2024-09-21T00:28:30.956268+0000","last_peered":"2024-09-21T00:28:30.956268+0000","last_clean":"2024-09-21T00:28:30.956268+0000","last_became_active":"2024-09-21T00:28:29.938961+0000","last_became_peered":"2024-09-21T00:28:29.938961+0000","last_unstale":"2024-09-21T00:28:30.956268+0000","last_undegraded":"2024-09-21T00:28:30.956268+0000","last_fullsized":"2024-09-21T00:28:30.956268+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:28.914450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:28.914450+0000","last_clean_scrub_stamp":"2024-09-21T00:28:28.914450+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:57:57.255009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.954644+0000","last_change":"2024-09-21T00:28:30.947697+0000","last_active":"2024-09-21T00:28:30.954644+0000","last_peered":"2024-09-21T00:28:30.954644+0000","last_clean":"2024-09-21T00:28:30.954644+0000","last_became_active":"2024-09-21T00:28:30.947352+0000","last_became_peered":"2024-09-21T00:28:30.947352+0000","last_unstale":"2024-09-21T00:28:30.954644+0000","last_undegraded":"2024-09-21T00:28:30.954644+0000","last_fullsized":"2024-09-21T00:28:30.954644+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_clean_scrub_stamp":"2024-09-21T00:28:29.918310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:12:42.513714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.928179+0000","last_change":"2024-09-21T00:28:29.943061+0000","last_active":"2024-09-21T00:28:30.928179+0000","last_peered":"2024-09-21T00:28:30.928179+0000","last_clean":"2024-09-21T00:28:30.928179+0000","last_became_active":"2024-09-21T00:28:29.942693+0000","last_became_peered":"2024-09-21T00:28:29.942693+0000","last_unstale":"2024-09-21T00:28:30.928179+0000","last_undegraded":"2024-09-21T00:28:30.928179+0000","last_fullsized":"2024-09-21T00:28:30.928179+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:28.914450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:28.914450+0000","last_clean_scrub_stamp":"2024-09-21T00:28:28.914450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:12:36.499940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.956794+0000","last_change":"2024-09-21T00:28:30.952387+0000","last_active":"2024-09-21T00:28:30.956794+0000","last_peered":"2024-09-21T00:28:30.956794+0000","last_clean":"2024-09-21T00:28:30.956794+0000","last_became_active":"2024-09-21T00:28:30.951670+0000","last_became_peered":"2024-09-21T00:28:30.951670+0000","last_unstale":"2024-09-21T00:28:30.956794+0000","last_undegraded":"2024-09-21T00:28:30.956794+0000","last_fullsized":"2024-09-21T00:28:30.956794+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_clean_scrub_stamp":"2024-09-21T00:28:29.918310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:41:52.334945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.940997+0000","last_change":"2024-09-21T00:28:29.946233+0000","last_active":"2024-09-21T00:28:30.940997+0000","last_peered":"2024-09-21T00:28:30.940997+0000","last_clean":"2024-09-21T00:28:30.940997+0000","last_became_active":"2024-09-21T00:28:29.945680+0000","last_became_peered":"2024-09-21T00:28:29.945680+0000","last_unstale":"2024-09-21T00:28:30.940997+0000","last_undegraded":"2024-09-21T00:28:30.940997+0000","last_fullsized":"2024-09-21T00:28:30.940997+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:28.914450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:28.914450+0000","last_clean_scrub_stamp":"2024-09-21T00:28:28.914450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:15:12.021971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.957558+0000","last_change":"2024-09-21T00:28:30.953554+0000","last_active":"2024-09-21T00:28:30.957558+0000","last_peered":"2024-09-21T00:28:30.957558+0000","last_clean":"2024-09-21T00:28:30.957558+0000","last_became_active":"2024-09-21T00:28:30.953191+0000","last_became_peered":"2024-09-21T00:28:30.953191+0000","last_unstale":"2024-09-21T00:28:30.957558+0000","last_undegraded":"2024-09-21T00:28:30.957558+0000","last_fullsized":"2024-09-21T00:28:30.957558+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_clean_scrub_stamp":"2024-09-21T00:28:29.918310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:56:01.061375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.938834+0000","last_change":"2024-09-21T00:28:29.948601+0000","last_active":"2024-09-21T00:28:30.938834+0000","last_peered":"2024-09-21T00:28:30.938834+0000","last_clean":"2024-09-21T00:28:30.938834+0000","last_became_active":"2024-09-21T00:28:29.948279+0000","last_became_peered":"2024-09-21T00:28:29.948279+0000","last_unstale":"2024-09-21T00:28:30.938834+0000","last_undegraded":"2024-09-21T00:28:30.938834+0000","last_fullsized":"2024-09-21T00:28:30.938834+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:28.914450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:28.914450+0000","last_clean_scrub_stamp":"2024-09-21T00:28:28.914450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T00:42:17.219795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.963437+0000","last_change":"2024-09-21T00:28:30.955305+0000","last_active":"2024-09-21T00:28:30.963437+0000","last_peered":"2024-09-21T00:28:30.963437+0000","last_clean":"2024-09-21T00:28:30.963437+0000","last_became_active":"2024-09-21T00:28:30.955008+0000","last_became_peered":"2024-09-21T00:28:30.955008+0000","last_unstale":"2024-09-21T00:28:30.963437+0000","last_undegraded":"2024-09-21T00:28:30.963437+0000","last_fullsized":"2024-09-21T00:28:30.963437+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_clean_scrub_stamp":"2024-09-21T00:28:29.918310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:11:36.689705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.943541+0000","last_change":"2024-09-21T00:28:29.942909+0000","last_active":"2024-09-21T00:28:30.943541+0000","last_peered":"2024-09-21T00:28:30.943541+0000","last_clean":"2024-09-21T00:28:30.943541+0000","last_became_active":"2024-09-21T00:28:29.942571+0000","last_became_peered":"2024-09-21T00:28:29.942571+0000","last_unstale":"2024-09-21T00:28:30.943541+0000","last_undegraded":"2024-09-21T00:28:30.943541+0000","last_fullsized":"2024-09-21T00:28:30.943541+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:28.914450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:28.914450+0000","last_clean_scrub_stamp":"2024-09-21T00:28:28.914450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:13:26.027352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.963427+0000","last_change":"2024-09-21T00:28:30.957257+0000","last_active":"2024-09-21T00:28:30.963427+0000","last_peered":"2024-09-21T00:28:30.963427+0000","last_clean":"2024-09-21T00:28:30.963427+0000","last_became_active":"2024-09-21T00:28:30.956927+0000","last_became_peered":"2024-09-21T00:28:30.956927+0000","last_unstale":"2024-09-21T00:28:30.963427+0000","last_undegraded":"2024-09-21T00:28:30.963427+0000","last_fullsized":"2024-09-21T00:28:30.963427+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_clean_scrub_stamp":"2024-09-21T00:28:29.918310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:46:26.394456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.935874+0000","last_change":"2024-09-21T00:28:29.944482+0000","last_active":"2024-09-21T00:28:30.935874+0000","last_peered":"2024-09-21T00:28:30.935874+0000","last_clean":"2024-09-21T00:28:30.935874+0000","last_became_active":"2024-09-21T00:28:29.944168+0000","last_became_peered":"2024-09-21T00:28:29.944168+0000","last_unstale":"2024-09-21T00:28:30.935874+0000","last_undegraded":"2024-09-21T00:28:30.935874+0000","last_fullsized":"2024-09-21T00:28:30.935874+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:28.914450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:28.914450+0000","last_clean_scrub_stamp":"2024-09-21T00:28:28.914450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:24:52.441266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T00:28:31.947028+0000","last_change":"2024-09-21T00:28:30.946464+0000","last_active":"2024-09-21T00:28:31.947028+0000","last_peered":"2024-09-21T00:28:31.947028+0000","last_clean":"2024-09-21T00:28:31.947028+0000","last_became_active":"2024-09-21T00:28:30.946145+0000","last_became_peered":"2024-09-21T00:28:30.946145+0000","last_unstale":"2024-09-21T00:28:31.947028+0000","last_undegraded":"2024-09-21T00:28:31.947028+0000","last_fullsized":"2024-09-21T00:28:31.947028+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_clean_scrub_stamp":"2024-09-21T00:28:29.918310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:29:49.641648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.941468+0000","last_change":"2024-09-21T00:28:29.945265+0000","last_active":"2024-09-21T00:28:30.941468+0000","last_peered":"2024-09-21T00:28:30.941468+0000","last_clean":"2024-09-21T00:28:30.941468+0000","last_became_active":"2024-09-21T00:28:29.944907+0000","last_became_peered":"2024-09-21T00:28:29.944907+0000","last_unstale":"2024-09-21T00:28:30.941468+0000","last_undegraded":"2024-09-21T00:28:30.941468+0000","last_fullsized":"2024-09-21T00:28:30.941468+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:28.914450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:28.914450+0000","last_clean_scrub_stamp":"2024-09-21T00:28:28.914450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:23:25.884700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T00:28:31.947219+0000","last_change":"2024-09-21T00:28:30.956345+0000","last_active":"2024-09-21T00:28:31.947219+0000","last_peered":"2024-09-21T00:28:31.947219+0000","last_clean":"2024-09-21T00:28:31.947219+0000","last_became_active":"2024-09-21T00:28:30.955999+0000","last_became_peered":"2024-09-21T00:28:30.955999+0000","last_unstale":"2024-09-21T00:28:31.947219+0000","last_undegraded":"2024-09-21T00:28:31.947219+0000","last_fullsized":"2024-09-21T00:28:31.947219+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_clean_scrub_stamp":"2024-09-21T00:28:29.918310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:42:29.083581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-21T00:28:31.946650+0000","last_change":"2024-09-21T00:28:29.945623+0000","last_active":"2024-09-21T00:28:31.946650+0000","last_peered":"2024-09-21T00:28:31.946650+0000","last_clean":"2024-09-21T00:28:31.946650+0000","last_became_active":"2024-09-21T00:28:29.945299+0000","last_became_peered":"2024-09-21T00:28:29.945299+0000","last_unstale":"2024-09-21T00:28:31.946650+0000","last_undegraded":"2024-09-21T00:28:31.946650+0000","last_fullsized":"2024-09-21T00:28:31.946650+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:28.914450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:28.914450+0000","last_clean_scrub_stamp":"2024-09-21T00:28:28.914450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:11:35.685845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.954438+0000","last_change":"2024-09-21T00:28:30.950650+0000","last_active":"2024-09-21T00:28:30.954438+0000","last_peered":"2024-09-21T00:28:30.954438+0000","last_clean":"2024-09-21T00:28:30.954438+0000","last_became_active":"2024-09-21T00:28:30.950135+0000","last_became_peered":"2024-09-21T00:28:30.950135+0000","last_unstale":"2024-09-21T00:28:30.954438+0000","last_undegraded":"2024-09-21T00:28:30.954438+0000","last_fullsized":"2024-09-21T00:28:30.954438+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:29.918310+0000","last_clean_scrub_stamp":"2024-09-21T00:28:29.918310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:52:43.572069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.941835+0000","last_change":"2024-09-21T00:28:29.946124+0000","last_active":"2024-09-21T00:28:30.941835+0000","last_peered":"2024-09-21T00:28:30.941835+0000","last_clean":"2024-09-21T00:28:30.941835+0000","last_became_active":"2024-09-21T00:28:29.945131+0000","last_became_peered":"2024-09-21T00:28:29.945131+0000","last_unstale":"2024-09-21T00:28:30.941835+0000","last_undegraded":"2024-09-21T00:28:30.941835+0000","last_fullsized":"2024-09-21T00:28:30.941835+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:28.914450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:28.914450+0000","last_clean_scrub_stamp":"2024-09-21T00:28:28.914450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:08:58.112056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":199,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T00:28:30.940646+0000","last_change":"2024-09-21T00:25:38.725792+0000","last_active":"2024-09-21T00:28:30.940646+0000","last_peered":"2024-09-21T00:28:30.940646+0000","last_clean":"2024-09-21T00:28:30.940646+0000","last_became_active":"2024-09-21T00:25:38.724668+0000","last_became_peered":"2024-09-21T00:25:38.724668+0000","last_unstale":"2024-09-21T00:28:30.940646+0000","last_undegraded":"2024-09-21T00:28:30.940646+0000","last_fullsized":"2024-09-21T00:28:30.940646+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:37.163216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:37.163216+0000","last_clean_scrub_stamp":"2024-09-21T00:25:37.163216+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:14:36.588660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509478,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32608,"kb_used_data":1412,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339232,"statfs":{"total":96636764160,"available":96603373568,"internally_reserved":0,"allocated":1445888,"data_stored":968613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509479,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27844,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343996,"statfs":{"total":96636764160,"available":96608251904,"internally_reserved":0,"allocated":835584,"data_stored":371056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509478,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27868,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343972,"statfs":{"total":96636764160,"available":96608227328,"internally_reserved":0,"allocated":851968,"data_stored":378257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509477,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32044,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339796,"statfs":{"total":96636764160,"available":96603951104,"internally_reserved":0,"allocated":868352,"data_stored":379939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509479,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32028,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339812,"statfs":{"total":96636764160,"available":96603967488,"internally_reserved":0,"allocated":851968,"data_stored":378805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509479,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27844,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343996,"statfs":{"total":96636764160,"available":96608251904,"internally_reserved":0,"allocated":835584,"data_stored":371056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509479,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32588,"kb_used_data":1396,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339252,"statfs":{"total":96636764160,"available":96603394048,"internally_reserved":0,"allocated":1429504,"data_stored":961424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574887,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32040,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339800,"statfs":{"total":96636764160,"available":96603955200,"internally_reserved":0,"allocated":868352,"data_stored":385027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T00:28:35.310 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T00:28:35.310 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-21T00:28:35.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.464+0000 7f66eb0b6640 1 -- 172.21.15.112:0/4256169259 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66e40d2030 msgr2=0x7f66e415d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:35.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.464+0000 7f66eb0b6640 1 --2- 172.21.15.112:0/4256169259 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66e40d2030 0x7f66e415d870 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f66e0009830 tx=0x7f66e002a510 comp rx=0 tx=0).stop 2024-09-21T00:28:35.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.464+0000 7f66eb0b6640 1 -- 172.21.15.112:0/4256169259 shutdown_connections 2024-09-21T00:28:35.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.464+0000 7f66eb0b6640 1 --2- 172.21.15.112:0/4256169259 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f66e40d0860 0x7f66e415a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:35.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.464+0000 7f66eb0b6640 1 --2- 172.21.15.112:0/4256169259 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66e40d2030 0x7f66e415d870 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:35.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.464+0000 7f66eb0b6640 1 -- 172.21.15.112:0/4256169259 >> 172.21.15.112:0/4256169259 conn(0x7f66e40b43f0 msgr2=0x7f66e40b4830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:35.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.464+0000 7f66eb0b6640 1 -- 172.21.15.112:0/4256169259 shutdown_connections 2024-09-21T00:28:35.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.464+0000 7f66eb0b6640 1 -- 172.21.15.112:0/4256169259 wait complete. 2024-09-21T00:28:35.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.466+0000 7f66eb0b6640 1 Processor -- start 2024-09-21T00:28:35.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.466+0000 7f66eb0b6640 1 -- start start 2024-09-21T00:28:35.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.467+0000 7f66eb0b6640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f66e40ccae0 0x7f66e41508f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:35.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.467+0000 7f66eb0b6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66e40d0860 0x7f66e40b65e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:35.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.467+0000 7f66eb0b6640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f66e40d2030 0x7f66e40cb350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:35.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.467+0000 7f66eb0b6640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f66e40b22d0 con 0x7f66e40d0860 2024-09-21T00:28:35.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.467+0000 7f66eb0b6640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f66e40b2150 con 0x7f66e40d2030 2024-09-21T00:28:35.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.467+0000 7f66eb0b6640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f66e40b2450 con 0x7f66e40ccae0 2024-09-21T00:28:35.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.467+0000 7f66dbfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66e40d0860 0x7f66e40b65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:35.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.467+0000 7f66dbfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66e40d0860 0x7f66e40b65e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:43130/0 (socket says 172.21.15.112:43130) 2024-09-21T00:28:35.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.467+0000 7f66dbfff640 1 -- 172.21.15.112:0/4176770659 learned_addr learned my addr 172.21.15.112:0/4176770659 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:28:35.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.468+0000 7f66e8e52640 1 --2- 172.21.15.112:0/4176770659 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f66e40ccae0 0x7f66e41508f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:35.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.468+0000 7f66e9653640 1 --2- 172.21.15.112:0/4176770659 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f66e40d2030 0x7f66e40cb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:35.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.468+0000 7f66dbfff640 1 -- 172.21.15.112:0/4176770659 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f66e40ccae0 msgr2=0x7f66e41508f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:35.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.468+0000 7f66dbfff640 1 --2- 172.21.15.112:0/4176770659 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f66e40ccae0 0x7f66e41508f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:35.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.468+0000 7f66dbfff640 1 -- 172.21.15.112:0/4176770659 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f66e40d2030 msgr2=0x7f66e40cb350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:35.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.468+0000 7f66dbfff640 1 --2- 172.21.15.112:0/4176770659 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f66e40d2030 0x7f66e40cb350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:35.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.468+0000 7f66dbfff640 1 -- 172.21.15.112:0/4176770659 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66e40b81e0 con 0x7f66e40d0860 2024-09-21T00:28:35.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.468+0000 7f66dbfff640 1 --2- 172.21.15.112:0/4176770659 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66e40d0860 0x7f66e40b65e0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f66d400c9a0 tx=0x7f66d400ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:35.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.468+0000 7f66d97fa640 1 -- 172.21.15.112:0/4176770659 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d4007bb0 con 0x7f66e40d0860 2024-09-21T00:28:35.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.468+0000 7f66d97fa640 1 -- 172.21.15.112:0/4176770659 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f66d4007e40 con 0x7f66e40d0860 2024-09-21T00:28:35.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.468+0000 7f66eb0b6640 1 -- 172.21.15.112:0/4176770659 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66e40b7c20 con 0x7f66e40d0860 2024-09-21T00:28:35.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.468+0000 7f66d97fa640 1 -- 172.21.15.112:0/4176770659 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d400f690 con 0x7f66e40d0860 2024-09-21T00:28:35.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.468+0000 7f66eb0b6640 1 -- 172.21.15.112:0/4176770659 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66e40b5de0 con 0x7f66e40d0860 2024-09-21T00:28:35.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.469+0000 7f66eb0b6640 1 -- 172.21.15.112:0/4176770659 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66e40b2900 con 0x7f66e40d0860 2024-09-21T00:28:35.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.470+0000 7f66d97fa640 1 -- 172.21.15.112:0/4176770659 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f66d400f830 con 0x7f66e40d0860 2024-09-21T00:28:35.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.470+0000 7f66d97fa640 1 --2- 172.21.15.112:0/4176770659 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f66d0078bc0 0x7f66d007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:35.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.471+0000 7f66d97fa640 1 -- 172.21.15.112:0/4176770659 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6101+0+0 (secure 0 0 0) 0x7f66d409b620 con 0x7f66e40d0860 2024-09-21T00:28:35.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.471+0000 7f66e8e52640 1 --2- 172.21.15.112:0/4176770659 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f66d0078bc0 0x7f66d007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:35.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.471+0000 7f66e8e52640 1 --2- 172.21.15.112:0/4176770659 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f66d0078bc0 0x7f66d007b0d0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f66e0034000 tx=0x7f66e0004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:35.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.475+0000 7f66d97fa640 1 -- 172.21.15.112:0/4176770659 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f66d4014030 con 0x7f66e40d0860 2024-09-21T00:28:35.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.723+0000 7f66eb0b6640 1 -- 172.21.15.112:0/4176770659 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f66e4071df0 con 0x7f66e40d0860 2024-09-21T00:28:35.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.725+0000 7f66d97fa640 1 -- 172.21.15.112:0/4176770659 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v18) ==== 190+0+147 (secure 0 0 0) 0x7f66d4063960 con 0x7f66e40d0860 2024-09-21T00:28:35.727 INFO:teuthology.orchestra.run.smithi112.stdout:[client.0] 2024-09-21T00:28:35.727 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQBPEu5m49NPHBAAl/rcjiE8xYI+XdWwvZ6FOw== 2024-09-21T00:28:35.727 INFO:teuthology.orchestra.run.smithi112.stdout: caps mds = "allow" 2024-09-21T00:28:35.727 INFO:teuthology.orchestra.run.smithi112.stdout: caps mon = "allow r" 2024-09-21T00:28:35.727 INFO:teuthology.orchestra.run.smithi112.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-21T00:28:35.728 INFO:teuthology.orchestra.run.smithi112.stderr:updated caps for client.0 2024-09-21T00:28:35.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.728+0000 7f66eb0b6640 1 -- 172.21.15.112:0/4176770659 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f66d0078bc0 msgr2=0x7f66d007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:35.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.728+0000 7f66eb0b6640 1 --2- 172.21.15.112:0/4176770659 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f66d0078bc0 0x7f66d007b0d0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f66e0034000 tx=0x7f66e0004730 comp rx=0 tx=0).stop 2024-09-21T00:28:35.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.728+0000 7f66eb0b6640 1 -- 172.21.15.112:0/4176770659 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66e40d0860 msgr2=0x7f66e40b65e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:35.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.728+0000 7f66eb0b6640 1 --2- 172.21.15.112:0/4176770659 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66e40d0860 0x7f66e40b65e0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f66d400c9a0 tx=0x7f66d400ce60 comp rx=0 tx=0).stop 2024-09-21T00:28:35.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.728+0000 7f66eb0b6640 1 -- 172.21.15.112:0/4176770659 shutdown_connections 2024-09-21T00:28:35.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.728+0000 7f66eb0b6640 1 --2- 172.21.15.112:0/4176770659 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f66e40d2030 0x7f66e40cb350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:35.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.728+0000 7f66eb0b6640 1 --2- 172.21.15.112:0/4176770659 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f66d0078bc0 0x7f66d007b0d0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:35.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.728+0000 7f66eb0b6640 1 --2- 172.21.15.112:0/4176770659 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66e40d0860 0x7f66e40b65e0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:35.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.728+0000 7f66eb0b6640 1 --2- 172.21.15.112:0/4176770659 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f66e40ccae0 0x7f66e41508f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:35.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.728+0000 7f66eb0b6640 1 -- 172.21.15.112:0/4176770659 >> 172.21.15.112:0/4176770659 conn(0x7f66e40b43f0 msgr2=0x7f66e415d5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:35.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.728+0000 7f66eb0b6640 1 -- 172.21.15.112:0/4176770659 shutdown_connections 2024-09-21T00:28:35.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.728+0000 7f66eb0b6640 1 -- 172.21.15.112:0/4176770659 wait complete. 2024-09-21T00:28:35.741 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T00:28:35.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.851+0000 7f4228c1d640 1 -- 172.21.15.112:0/3193082799 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f42240d3880 msgr2=0x7f4224163260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:35.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.851+0000 7f4228c1d640 1 --2- 172.21.15.112:0/3193082799 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f42240d3880 0x7f4224163260 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f42240d2050 tx=0x7f421802a560 comp rx=0 tx=0).stop 2024-09-21T00:28:35.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.851+0000 7f4228c1d640 1 -- 172.21.15.112:0/3193082799 shutdown_connections 2024-09-21T00:28:35.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.851+0000 7f4228c1d640 1 --2- 172.21.15.112:0/3193082799 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f42240d01b0 0x7f42240d05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:35.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.851+0000 7f4228c1d640 1 --2- 172.21.15.112:0/3193082799 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f42240d3880 0x7f4224163260 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:35.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.851+0000 7f4228c1d640 1 --2- 172.21.15.112:0/3193082799 >> v2:172.21.15.187:3300/0 conn(0x7f42240d48a0 0x7f42240d3dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:35.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.851+0000 7f4228c1d640 1 -- 172.21.15.112:0/3193082799 >> 172.21.15.112:0/3193082799 conn(0x7f422408bfe0 msgr2=0x7f422409d890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:35.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.851+0000 7f4228c1d640 1 -- 172.21.15.112:0/3193082799 shutdown_connections 2024-09-21T00:28:35.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.851+0000 7f4228c1d640 1 -- 172.21.15.112:0/3193082799 wait complete. 2024-09-21T00:28:35.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.853+0000 7f4228c1d640 1 Processor -- start 2024-09-21T00:28:35.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.853+0000 7f4228c1d640 1 -- start start 2024-09-21T00:28:35.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.854+0000 7f4228c1d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f42240d01b0 0x7f42240cd790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:35.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.854+0000 7f4228c1d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f42240d3880 0x7f42240cdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:35.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.854+0000 7f4228c1d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f42240d48a0 0x7f42241f5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:35.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.854+0000 7f4228c1d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f42240d2250 con 0x7f42240d48a0 2024-09-21T00:28:35.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.854+0000 7f4228c1d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f42240d20d0 con 0x7f42240d3880 2024-09-21T00:28:35.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.854+0000 7f4228c1d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f42240d23d0 con 0x7f42240d01b0 2024-09-21T00:28:35.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.855+0000 7f4222d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f42240d48a0 0x7f42241f5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:35.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.855+0000 7f4222d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f42240d48a0 0x7f42241f5ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:43134/0 (socket says 172.21.15.112:43134) 2024-09-21T00:28:35.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.855+0000 7f4222d9d640 1 -- 172.21.15.112:0/3913048021 learned_addr learned my addr 172.21.15.112:0/3913048021 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:28:35.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.855+0000 7f422259c640 1 --2- 172.21.15.112:0/3913048021 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f42240d01b0 0x7f42240cd790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:35.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.855+0000 7f4221d9b640 1 --2- 172.21.15.112:0/3913048021 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f42240d3880 0x7f42240cdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:35.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.855+0000 7f4222d9d640 1 -- 172.21.15.112:0/3913048021 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f42240d01b0 msgr2=0x7f42240cd790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:35.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.855+0000 7f4222d9d640 1 --2- 172.21.15.112:0/3913048021 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f42240d01b0 0x7f42240cd790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:35.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.855+0000 7f4222d9d640 1 -- 172.21.15.112:0/3913048021 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f42240d3880 msgr2=0x7f42240cdcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:35.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.855+0000 7f4222d9d640 1 --2- 172.21.15.112:0/3913048021 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f42240d3880 0x7f42240cdcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:35.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.855+0000 7f4222d9d640 1 -- 172.21.15.112:0/3913048021 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f422406d920 con 0x7f42240d48a0 2024-09-21T00:28:35.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.855+0000 7f4222d9d640 1 --2- 172.21.15.112:0/3913048021 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f42240d48a0 0x7f42241f5ac0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f421c00e960 tx=0x7f421c00ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:35.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.855+0000 7f4212ffd640 1 -- 172.21.15.112:0/3913048021 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f421c004d20 con 0x7f42240d48a0 2024-09-21T00:28:35.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.855+0000 7f4212ffd640 1 -- 172.21.15.112:0/3913048021 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f421c00ca60 con 0x7f42240d48a0 2024-09-21T00:28:35.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.855+0000 7f4212ffd640 1 -- 172.21.15.112:0/3913048021 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f421c005020 con 0x7f42240d48a0 2024-09-21T00:28:35.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.855+0000 7f4228c1d640 1 -- 172.21.15.112:0/3913048021 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42240cbdf0 con 0x7f42240d48a0 2024-09-21T00:28:35.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.855+0000 7f4228c1d640 1 -- 172.21.15.112:0/3913048021 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4224160640 con 0x7f42240d48a0 2024-09-21T00:28:35.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.856+0000 7f4228c1d640 1 -- 172.21.15.112:0/3913048021 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f422406daf0 con 0x7f42240d48a0 2024-09-21T00:28:35.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.857+0000 7f4212ffd640 1 -- 172.21.15.112:0/3913048021 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f421c0051c0 con 0x7f42240d48a0 2024-09-21T00:28:35.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.857+0000 7f4212ffd640 1 --2- 172.21.15.112:0/3913048021 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4204078b90 0x7f420407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:35.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.858+0000 7f4212ffd640 1 -- 172.21.15.112:0/3913048021 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6101+0+0 (secure 0 0 0) 0x7f421c09b900 con 0x7f42240d48a0 2024-09-21T00:28:35.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.858+0000 7f422259c640 1 --2- 172.21.15.112:0/3913048021 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4204078b90 0x7f420407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:35.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.858+0000 7f422259c640 1 --2- 172.21.15.112:0/3913048021 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4204078b90 0x7f420407b0a0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f421400bb50 tx=0x7f421400b2e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:35.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:35.862+0000 7f4212ffd640 1 -- 172.21.15.112:0/3913048021 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f421c016030 con 0x7f42240d48a0 2024-09-21T00:28:36.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.074+0000 7f4228c1d640 1 -- 172.21.15.112:0/3913048021 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f42240d28f0 con 0x7f42240d48a0 2024-09-21T00:28:36.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.075+0000 7f4212ffd640 1 -- 172.21.15.112:0/3913048021 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+5237 (secure 0 0 0) 0x7f421c06a030 con 0x7f42240d48a0 2024-09-21T00:28:36.077 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:28:36.077 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":47,"btime":"2024-09-21T00:28:33:951960+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5435,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6837/3039805497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6835","nonce":3039805497},{"type":"v1","addr":"172.21.15.187:6837","nonce":3039805497}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43},{"gid":5719,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6836/2736940636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":2736940636},{"type":"v1","addr":"172.21.15.187:6836","nonce":2736940636}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43},{"gid":6162,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6834/2295407428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":2295407428},{"type":"v1","addr":"172.21.15.112:6834","nonce":2295407428}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T00:28:30.298540+0000","modified":"2024-09-21T00:28:33.314195+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6165},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6165":{"gid":6165,"name":"a","rank":0,"incarnation":44,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6835/1624734538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":1624734538},{"type":"v1","addr":"172.21.15.112:6835","nonce":1624734538}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6165,"qdb_cluster":[6165]},"id":5}]} 2024-09-21T00:28:36.077 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 47 2024-09-21T00:28:36.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.077+0000 7f4228c1d640 1 -- 172.21.15.112:0/3913048021 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4204078b90 msgr2=0x7f420407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:36.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.077+0000 7f4228c1d640 1 --2- 172.21.15.112:0/3913048021 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4204078b90 0x7f420407b0a0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f421400bb50 tx=0x7f421400b2e0 comp rx=0 tx=0).stop 2024-09-21T00:28:36.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.077+0000 7f4228c1d640 1 -- 172.21.15.112:0/3913048021 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f42240d48a0 msgr2=0x7f42241f5ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:36.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.077+0000 7f4228c1d640 1 --2- 172.21.15.112:0/3913048021 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f42240d48a0 0x7f42241f5ac0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f421c00e960 tx=0x7f421c00ee20 comp rx=0 tx=0).stop 2024-09-21T00:28:36.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.077+0000 7f4228c1d640 1 -- 172.21.15.112:0/3913048021 shutdown_connections 2024-09-21T00:28:36.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.077+0000 7f4228c1d640 1 --2- 172.21.15.112:0/3913048021 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4204078b90 0x7f420407b0a0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:36.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.077+0000 7f4228c1d640 1 --2- 172.21.15.112:0/3913048021 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f42240d48a0 0x7f42241f5ac0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:36.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.077+0000 7f4228c1d640 1 --2- 172.21.15.112:0/3913048021 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f42240d3880 0x7f42240cdcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:36.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.077+0000 7f4228c1d640 1 --2- 172.21.15.112:0/3913048021 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f42240d01b0 0x7f42240cd790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:36.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.077+0000 7f4228c1d640 1 -- 172.21.15.112:0/3913048021 >> 172.21.15.112:0/3913048021 conn(0x7f422408bfe0 msgr2=0x7f42240b7a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:36.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.077+0000 7f4228c1d640 1 -- 172.21.15.112:0/3913048021 shutdown_connections 2024-09-21T00:28:36.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.077+0000 7f4228c1d640 1 -- 172.21.15.112:0/3913048021 wait complete. 2024-09-21T00:28:36.492 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 47, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T00:28:30.298540+0000', 'modified': '2024-09-21T00:28:33.314195+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6165}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6165': {'gid': 6165, 'name': 'a', 'rank': 0, 'incarnation': 44, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.112:6835/1624734538', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6833', 'nonce': 1624734538}, {'type': 'v1', 'addr': '172.21.15.112:6835', 'nonce': 1624734538}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6165, 'qdb_cluster': [6165]} 2024-09-21T00:28:36.493 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-21T00:28:36.493 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-21T00:28:36.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.604+0000 7ff11d7f1640 1 -- 172.21.15.112:0/675778208 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1180b3bb0 msgr2=0x7ff1180b6740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:36.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.604+0000 7ff11d7f1640 1 --2- 172.21.15.112:0/675778208 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1180b3bb0 0x7ff1180b6740 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7ff11808d530 tx=0x7ff11002a820 comp rx=0 tx=0).stop 2024-09-21T00:28:36.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.604+0000 7ff11d7f1640 1 -- 172.21.15.112:0/675778208 shutdown_connections 2024-09-21T00:28:36.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.604+0000 7ff11d7f1640 1 --2- 172.21.15.112:0/675778208 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1180b3bb0 0x7ff1180b6740 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:36.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.604+0000 7ff11d7f1640 1 --2- 172.21.15.112:0/675778208 >> v2:172.21.15.187:3300/0 conn(0x7ff11815c3c0 0x7ff1180b3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:36.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.604+0000 7ff11d7f1640 1 -- 172.21.15.112:0/675778208 >> 172.21.15.112:0/675778208 conn(0x7ff11808dcb0 msgr2=0x7ff11809d8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:36.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.604+0000 7ff11d7f1640 1 -- 172.21.15.112:0/675778208 shutdown_connections 2024-09-21T00:28:36.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.604+0000 7ff11d7f1640 1 -- 172.21.15.112:0/675778208 wait complete. 2024-09-21T00:28:36.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.606+0000 7ff11d7f1640 1 Processor -- start 2024-09-21T00:28:36.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.606+0000 7ff11d7f1640 1 -- start start 2024-09-21T00:28:36.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.608+0000 7ff11d7f1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1180b37a0 0x7ff1181f5a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:36.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.608+0000 7ff11d7f1640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff1180b3bb0 0x7ff11806d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:36.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.608+0000 7ff11d7f1640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff11815c3c0 0x7ff11814dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:36.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.608+0000 7ff11d7f1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff118163180 con 0x7ff1180b37a0 2024-09-21T00:28:36.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.608+0000 7ff11d7f1640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff1180d0d30 con 0x7ff11815c3c0 2024-09-21T00:28:36.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.608+0000 7ff11d7f1640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff118163300 con 0x7ff1180b3bb0 2024-09-21T00:28:36.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.608+0000 7ff116ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1180b37a0 0x7ff1181f5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:36.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.608+0000 7ff116ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1180b37a0 0x7ff1181f5a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33188/0 (socket says 172.21.15.112:33188) 2024-09-21T00:28:36.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.608+0000 7ff116ffd640 1 -- 172.21.15.112:0/2789548892 learned_addr learned my addr 172.21.15.112:0/2789548892 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:28:36.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.608+0000 7ff1177fe640 1 --2- 172.21.15.112:0/2789548892 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff11815c3c0 0x7ff11814dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:36.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.608+0000 7ff10e7fc640 1 --2- 172.21.15.112:0/2789548892 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff1180b3bb0 0x7ff11806d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:36.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.608+0000 7ff116ffd640 1 -- 172.21.15.112:0/2789548892 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff1180b3bb0 msgr2=0x7ff11806d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:36.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.608+0000 7ff116ffd640 1 --2- 172.21.15.112:0/2789548892 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff1180b3bb0 0x7ff11806d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:36.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.608+0000 7ff116ffd640 1 -- 172.21.15.112:0/2789548892 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff11815c3c0 msgr2=0x7ff11814dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:36.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.608+0000 7ff116ffd640 1 --2- 172.21.15.112:0/2789548892 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff11815c3c0 0x7ff11814dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:36.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.608+0000 7ff116ffd640 1 -- 172.21.15.112:0/2789548892 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff118070420 con 0x7ff1180b37a0 2024-09-21T00:28:36.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.608+0000 7ff1177fe640 1 --2- 172.21.15.112:0/2789548892 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff11815c3c0 0x7ff11814dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:28:36.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.608+0000 7ff116ffd640 1 --2- 172.21.15.112:0/2789548892 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1180b37a0 0x7ff1181f5a00 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7ff10800b420 tx=0x7ff10800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:36.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.608+0000 7ff10ffff640 1 -- 172.21.15.112:0/2789548892 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff108005970 con 0x7ff1180b37a0 2024-09-21T00:28:36.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.608+0000 7ff11d7f1640 1 -- 172.21.15.112:0/2789548892 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff11806ec00 con 0x7ff1180b37a0 2024-09-21T00:28:36.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.608+0000 7ff10ffff640 1 -- 172.21.15.112:0/2789548892 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff108005c00 con 0x7ff1180b37a0 2024-09-21T00:28:36.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.608+0000 7ff10ffff640 1 -- 172.21.15.112:0/2789548892 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff108006020 con 0x7ff1180b37a0 2024-09-21T00:28:36.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.608+0000 7ff11d7f1640 1 -- 172.21.15.112:0/2789548892 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1181f64a0 con 0x7ff1180b37a0 2024-09-21T00:28:36.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.610+0000 7ff10ffff640 1 -- 172.21.15.112:0/2789548892 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff108018020 con 0x7ff1180b37a0 2024-09-21T00:28:36.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.611+0000 7ff10ffff640 1 --2- 172.21.15.112:0/2789548892 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff0f8078b90 0x7ff0f807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:36.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.611+0000 7ff10ffff640 1 -- 172.21.15.112:0/2789548892 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6101+0+0 (secure 0 0 0) 0x7ff10809a540 con 0x7ff1180b37a0 2024-09-21T00:28:36.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.611+0000 7ff10e7fc640 1 --2- 172.21.15.112:0/2789548892 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff0f8078b90 0x7ff0f807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:36.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.612+0000 7ff10e7fc640 1 --2- 172.21.15.112:0/2789548892 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff0f8078b90 0x7ff0f807b0a0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7ff11815bb00 tx=0x7ff100009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:36.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.612+0000 7ff12483c740 1 Processor -- start 2024-09-21T00:28:36.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.612+0000 7ff12483c740 1 -- start start 2024-09-21T00:28:36.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.613+0000 7ff12483c740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55ccaccae270 0x55ccad07ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:36.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.613+0000 7ff12483c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ccacc853b0 0x55ccad0804e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:36.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.613+0000 7ff12483c740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ccacc8d590 0x55ccad09d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:36.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.613+0000 7ff12483c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55ccacfc8820 con 0x55ccacc853b0 2024-09-21T00:28:36.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.613+0000 7ff12483c740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55ccacfb7c50 con 0x55ccacc8d590 2024-09-21T00:28:36.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.613+0000 7ff12483c740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x55ccacfbba60 con 0x55ccaccae270 2024-09-21T00:28:36.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.613+0000 7ff10e7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ccacc853b0 0x55ccad0804e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:36.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.613+0000 7ff10e7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ccacc853b0 0x55ccad0804e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33192/0 (socket says 172.21.15.112:33192) 2024-09-21T00:28:36.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.613+0000 7ff10e7fc640 1 -- 172.21.15.112:0/3571092180 learned_addr learned my addr 172.21.15.112:0/3571092180 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:28:36.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.614+0000 7ff116ffd640 1 --2- 172.21.15.112:0/3571092180 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55ccaccae270 0x55ccad07ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:36.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.614+0000 7ff1177fe640 1 --2- 172.21.15.112:0/3571092180 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ccacc8d590 0x55ccad09d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:36.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.614+0000 7ff10e7fc640 1 -- 172.21.15.112:0/3571092180 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55ccaccae270 msgr2=0x55ccad07ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:28:36.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.614+0000 7ff10e7fc640 1 --2- 172.21.15.112:0/3571092180 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55ccaccae270 0x55ccad07ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:36.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.614+0000 7ff10e7fc640 1 -- 172.21.15.112:0/3571092180 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ccacc8d590 msgr2=0x55ccad09d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:28:36.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.614+0000 7ff10e7fc640 1 --2- 172.21.15.112:0/3571092180 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ccacc8d590 0x55ccad09d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:36.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.614+0000 7ff10e7fc640 1 -- 172.21.15.112:0/3571092180 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ccacd19850 con 0x55ccacc853b0 2024-09-21T00:28:36.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.614+0000 7ff10e7fc640 1 --2- 172.21.15.112:0/3571092180 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ccacc853b0 0x55ccad0804e0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7ff100005210 tx=0x7ff100011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bede2726da43173f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:36.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.614+0000 7ff10cff9640 1 -- 172.21.15.112:0/3571092180 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff10000e4b0 con 0x55ccacc853b0 2024-09-21T00:28:36.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.614+0000 7ff10cff9640 1 -- 172.21.15.112:0/3571092180 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff10000e650 con 0x55ccacc853b0 2024-09-21T00:28:36.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.614+0000 7ff10cff9640 1 -- 172.21.15.112:0/3571092180 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff10000ea20 con 0x55ccacc853b0 2024-09-21T00:28:36.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.615+0000 7ff12483c740 1 -- 172.21.15.112:0/3571092180 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ccacc853b0 msgr2=0x55ccad0804e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:36.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.615+0000 7ff12483c740 1 --2- 172.21.15.112:0/3571092180 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ccacc853b0 0x55ccad0804e0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7ff100005210 tx=0x7ff100011000 comp rx=0 tx=0).stop 2024-09-21T00:28:36.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.615+0000 7ff12483c740 1 -- 172.21.15.112:0/3571092180 shutdown_connections 2024-09-21T00:28:36.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.615+0000 7ff12483c740 1 --2- 172.21.15.112:0/3571092180 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ccacc8d590 0x55ccad09d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:36.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.615+0000 7ff12483c740 1 --2- 172.21.15.112:0/3571092180 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ccacc853b0 0x55ccad0804e0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:36.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.615+0000 7ff12483c740 1 --2- 172.21.15.112:0/3571092180 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55ccaccae270 0x55ccad07ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:36.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.615+0000 7ff12483c740 1 -- 172.21.15.112:0/3571092180 >> 172.21.15.112:0/3571092180 conn(0x55ccacca23c0 msgr2=0x55ccad056260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:36.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.615+0000 7ff12483c740 1 -- 172.21.15.112:0/3571092180 shutdown_connections 2024-09-21T00:28:36.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.615+0000 7ff12483c740 1 -- 172.21.15.112:0/3571092180 wait complete. 2024-09-21T00:28:36.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.616+0000 7ff12483c740 1 -- start start 2024-09-21T00:28:36.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.616+0000 7ff12483c740 1 Processor -- start 2024-09-21T00:28:36.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.617+0000 7ff12483c740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ccacc853b0 0x55ccad0782d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:36.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.617+0000 7ff12483c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ccacc8d590 0x55ccad0ad520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:36.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.617+0000 7ff12483c740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55ccaccae270 0x55ccad0aa700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:36.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.617+0000 7ff12483c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55ccacd28620 con 0x55ccacc8d590 2024-09-21T00:28:36.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.617+0000 7ff12483c740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55ccacf5d5f0 con 0x55ccacc853b0 2024-09-21T00:28:36.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.617+0000 7ff12483c740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x55ccad080a20 con 0x55ccaccae270 2024-09-21T00:28:36.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.617+0000 7ff12483c740 10 client.0 _ll_register_callbacks cb 0x55ccacf610a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:28:36.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.617+0000 7ff10e7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ccacc8d590 0x55ccad0ad520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:36.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.617+0000 7ff10e7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ccacc8d590 0x55ccad0ad520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33200/0 (socket says 172.21.15.112:33200) 2024-09-21T00:28:36.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.617+0000 7ff10e7fc640 1 -- 172.21.15.112:0/907014049 learned_addr learned my addr 172.21.15.112:0/907014049 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:28:36.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.618+0000 7ff116ffd640 1 --2- 172.21.15.112:0/907014049 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ccacc853b0 0x55ccad0782d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:36.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.618+0000 7ff1177fe640 1 --2- 172.21.15.112:0/907014049 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55ccaccae270 0x55ccad0aa700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:36.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.618+0000 7ff10e7fc640 1 -- 172.21.15.112:0/907014049 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55ccaccae270 msgr2=0x55ccad0aa700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:28:36.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.618+0000 7ff10e7fc640 1 --2- 172.21.15.112:0/907014049 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55ccaccae270 0x55ccad0aa700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:36.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.618+0000 7ff10e7fc640 1 -- 172.21.15.112:0/907014049 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ccacc853b0 msgr2=0x55ccad0782d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:28:36.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.618+0000 7ff10e7fc640 1 --2- 172.21.15.112:0/907014049 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ccacc853b0 0x55ccad0782d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:36.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.618+0000 7ff10e7fc640 1 -- 172.21.15.112:0/907014049 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ccacd906b0 con 0x55ccacc8d590 2024-09-21T00:28:36.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.618+0000 7ff10e7fc640 1 --2- 172.21.15.112:0/907014049 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ccacc8d590 0x55ccad0ad520 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7ff100001eb0 tx=0x7ff10000eab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13b608cf5921105a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:36.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.618+0000 7ff0f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T00:28:36.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.618+0000 7ff0f5ffb640 1 -- 172.21.15.112:0/907014049 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff100007500 con 0x55ccacc8d590 2024-09-21T00:28:36.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.618+0000 7ff0f5ffb640 1 -- 172.21.15.112:0/907014049 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(71..71 src has 1..71) ==== 6101+0+0 (secure 0 0 0) 0x7ff1000067b0 con 0x55ccacc8d590 2024-09-21T00:28:36.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.618+0000 7ff12483c740 1 -- 172.21.15.112:0/907014049 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff10000e650 con 0x55ccacc8d590 2024-09-21T00:28:36.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.618+0000 7ff0f5ffb640 1 -- 172.21.15.112:0/907014049 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7ff10002e050 con 0x55ccacc8d590 2024-09-21T00:28:36.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.618+0000 7ff12483c740 10 client.6300 fetch_fsmap learned FSMap version 47 2024-09-21T00:28:36.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.618+0000 7ff12483c740 1 -- 172.21.15.112:0/907014049 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=2+}) -- 0x55ccad0ada60 con 0x55ccacc8d590 2024-09-21T00:28:36.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.618+0000 7ff0f5ffb640 1 -- 172.21.15.112:0/907014049 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff10000e650 con 0x55ccacc8d590 2024-09-21T00:28:36.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.619+0000 7ff0f5ffb640 1 -- 172.21.15.112:0/907014049 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 47) ==== 3502+0+0 (secure 0 0 0) 0x7ff100031020 con 0x55ccacc8d590 2024-09-21T00:28:36.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.619+0000 7ff12483c740 10 client.6300 fetch_fsmap finished waiting for FSMap version 47 2024-09-21T00:28:36.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.619+0000 7ff12483c740 10 client.6300 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-21T00:28:36.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.619+0000 7ff12483c740 20 client.6300 populate_metadata read hostname 'smithi112' 2024-09-21T00:28:36.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.619+0000 7ff12483c740 1 --2- 172.21.15.112:0/907014049 >> [v2:172.21.15.112:6833/1624734538,v1:172.21.15.112:6835/1624734538] conn(0x55ccacc6ce40 0x55ccad091b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:36.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.619+0000 7ff12483c740 4 client.6300 mds_command: new command op to 6165 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T00:28:36.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.619+0000 7ff12483c740 1 -- 172.21.15.112:0/907014049 --> [v2:172.21.15.112:6833/1624734538,v1:172.21.15.112:6835/1624734538] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ccacfca770 con 0x55ccacc6ce40 2024-09-21T00:28:36.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.619+0000 7ff116ffd640 1 --2- 172.21.15.112:0/907014049 >> [v2:172.21.15.112:6833/1624734538,v1:172.21.15.112:6835/1624734538] conn(0x55ccacc6ce40 0x55ccad091b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:36.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.619+0000 7ff0f5ffb640 10 client.6300 ms_handle_connect on v2:172.21.15.112:6833/1624734538 2024-09-21T00:28:36.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.619+0000 7ff116ffd640 1 --2- 172.21.15.112:0/907014049 >> [v2:172.21.15.112:6833/1624734538,v1:172.21.15.112:6835/1624734538] conn(0x55ccacc6ce40 0x55ccad091b90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=519c979ab8db081 server_cookie=b0173c9d4abedfd8 in_seq=0 out_seq=0 2024-09-21T00:28:36.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.620+0000 7ff0f5ffb640 1 -- 172.21.15.112:0/907014049 <== mds.0 v2:172.21.15.112:6833/1624734538 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ccacfca770 con 0x55ccacc6ce40 2024-09-21T00:28:36.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.620+0000 7ff0f5ffb640 10 client.6300 handle_command_reply: tid=0 2024-09-21T00:28:36.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.620+0000 7ff12483c740 1 client.6300 shutdown 2024-09-21T00:28:36.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.620+0000 7ff12483c740 1 -- 172.21.15.112:0/907014049 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ccacc8d590 msgr2=0x55ccad0ad520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:36.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.620+0000 7ff12483c740 1 --2- 172.21.15.112:0/907014049 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ccacc8d590 0x55ccad0ad520 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7ff100001eb0 tx=0x7ff10000eab0 comp rx=0 tx=0).stop 2024-09-21T00:28:36.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.620+0000 7ff12483c740 1 -- 172.21.15.112:0/907014049 shutdown_connections 2024-09-21T00:28:36.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.620+0000 7ff12483c740 1 --2- 172.21.15.112:0/907014049 >> [v2:172.21.15.112:6833/1624734538,v1:172.21.15.112:6835/1624734538] conn(0x55ccacc6ce40 0x55ccad091b90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:36.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.620+0000 7ff12483c740 1 --2- 172.21.15.112:0/907014049 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55ccaccae270 0x55ccad0aa700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:36.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.620+0000 7ff12483c740 1 --2- 172.21.15.112:0/907014049 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ccacc8d590 0x55ccad0ad520 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:36.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.620+0000 7ff0f5ffb640 0 client.6300 ms_handle_reset on v2:172.21.15.112:6833/1624734538 2024-09-21T00:28:36.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.620+0000 7ff12483c740 1 --2- 172.21.15.112:0/907014049 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ccacc853b0 0x55ccad0782d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:36.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.620+0000 7ff12483c740 1 -- 172.21.15.112:0/907014049 >> 172.21.15.112:0/907014049 conn(0x55ccacca23c0 msgr2=0x55ccad04f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:36.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.620+0000 7ff12483c740 1 -- 172.21.15.112:0/907014049 shutdown_connections 2024-09-21T00:28:36.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.620+0000 7ff12483c740 1 -- 172.21.15.112:0/907014049 wait complete. 2024-09-21T00:28:36.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.620+0000 7ff12483c740 20 client.6300 trim_cache size 0 max 16384 2024-09-21T00:28:36.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.638+0000 7ff12483c740 1 Processor -- start 2024-09-21T00:28:36.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.638+0000 7ff12483c740 1 -- start start 2024-09-21T00:28:36.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.640+0000 7ff12483c740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55ccacc853b0 0x55ccad0803a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:36.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.640+0000 7ff12483c740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ccacc8d590 0x55ccad06e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:36.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.640+0000 7ff12483c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ccaccae270 0x55ccad0aa410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:36.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.640+0000 7ff12483c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff118201610 con 0x55ccaccae270 2024-09-21T00:28:36.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.640+0000 7ff12483c740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff118070170 con 0x55ccacc8d590 2024-09-21T00:28:36.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.640+0000 7ff12483c740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff1181f9cb0 con 0x55ccacc853b0 2024-09-21T00:28:36.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.640+0000 7ff1177fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ccaccae270 0x55ccad0aa410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:36.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.640+0000 7ff1177fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ccaccae270 0x55ccad0aa410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33212/0 (socket says 172.21.15.112:33212) 2024-09-21T00:28:36.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.640+0000 7ff1177fe640 1 -- 172.21.15.112:0/2617473362 learned_addr learned my addr 172.21.15.112:0/2617473362 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:28:36.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.640+0000 7ff1177fe640 1 -- 172.21.15.112:0/2617473362 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55ccacc853b0 msgr2=0x55ccad0803a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:28:36.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.640+0000 7ff1177fe640 1 --2- 172.21.15.112:0/2617473362 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55ccacc853b0 0x55ccad0803a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:36.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.640+0000 7ff1177fe640 1 -- 172.21.15.112:0/2617473362 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ccacc8d590 msgr2=0x55ccad06e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:28:36.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.640+0000 7ff10e7fc640 1 --2- 172.21.15.112:0/2617473362 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ccacc8d590 0x55ccad06e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:36.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.640+0000 7ff1177fe640 1 --2- 172.21.15.112:0/2617473362 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ccacc8d590 0x55ccad06e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:36.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.640+0000 7ff1177fe640 1 -- 172.21.15.112:0/2617473362 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ccad07a780 con 0x55ccaccae270 2024-09-21T00:28:36.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.640+0000 7ff10e7fc640 1 --2- 172.21.15.112:0/2617473362 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ccacc8d590 0x55ccad06e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:28:36.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.640+0000 7ff1177fe640 1 --2- 172.21.15.112:0/2617473362 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ccaccae270 0x55ccad0aa410 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7ff11002ae10 tx=0x7ff110004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23e8a07feb604b7c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:36.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.640+0000 7ff0f7fff640 1 -- 172.21.15.112:0/2617473362 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff110037040 con 0x55ccaccae270 2024-09-21T00:28:36.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.640+0000 7ff0f7fff640 1 -- 172.21.15.112:0/2617473362 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff110003ba0 con 0x55ccaccae270 2024-09-21T00:28:36.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.640+0000 7ff0f7fff640 1 -- 172.21.15.112:0/2617473362 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff110004040 con 0x55ccaccae270 2024-09-21T00:28:36.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.641+0000 7ff12483c740 1 -- 172.21.15.112:0/2617473362 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ccaccae270 msgr2=0x55ccad0aa410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:36.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.641+0000 7ff12483c740 1 --2- 172.21.15.112:0/2617473362 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ccaccae270 0x55ccad0aa410 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7ff11002ae10 tx=0x7ff110004a40 comp rx=0 tx=0).stop 2024-09-21T00:28:36.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.641+0000 7ff12483c740 1 -- 172.21.15.112:0/2617473362 shutdown_connections 2024-09-21T00:28:36.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.641+0000 7ff12483c740 1 --2- 172.21.15.112:0/2617473362 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ccaccae270 0x55ccad0aa410 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:36.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.641+0000 7ff12483c740 1 --2- 172.21.15.112:0/2617473362 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ccacc8d590 0x55ccad06e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:36.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.641+0000 7ff12483c740 1 --2- 172.21.15.112:0/2617473362 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55ccacc853b0 0x55ccad0803a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:36.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.641+0000 7ff12483c740 1 -- 172.21.15.112:0/2617473362 >> 172.21.15.112:0/2617473362 conn(0x55ccacca23c0 msgr2=0x55ccad04b920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:36.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.641+0000 7ff12483c740 1 -- 172.21.15.112:0/2617473362 shutdown_connections 2024-09-21T00:28:36.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.641+0000 7ff12483c740 1 -- 172.21.15.112:0/2617473362 wait complete. 2024-09-21T00:28:36.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.642+0000 7ff12483c740 1 -- start start 2024-09-21T00:28:36.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.642+0000 7ff12483c740 1 Processor -- start 2024-09-21T00:28:36.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.643+0000 7ff12483c740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55ccacc853b0 0x55ccad0bd3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:36.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.643+0000 7ff12483c740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ccacc8d590 0x55ccad11d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:36.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.643+0000 7ff12483c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ccaccae270 0x55ccad0a8380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:36.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.643+0000 7ff12483c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55ccad055610 con 0x55ccaccae270 2024-09-21T00:28:36.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.643+0000 7ff12483c740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55ccad0aac40 con 0x55ccacc8d590 2024-09-21T00:28:36.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.643+0000 7ff12483c740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x55ccad094500 con 0x55ccacc853b0 2024-09-21T00:28:36.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.643+0000 7ff12483c740 10 client.0 _ll_register_callbacks cb 0x7ff100005110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:28:36.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.644+0000 7ff1177fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ccaccae270 0x55ccad0a8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:36.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.644+0000 7ff1177fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ccaccae270 0x55ccad0a8380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33222/0 (socket says 172.21.15.112:33222) 2024-09-21T00:28:36.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.644+0000 7ff1177fe640 1 -- 172.21.15.112:0/1544616920 learned_addr learned my addr 172.21.15.112:0/1544616920 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:28:36.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.644+0000 7ff116ffd640 1 --2- 172.21.15.112:0/1544616920 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55ccacc853b0 0x55ccad0bd3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:36.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.644+0000 7ff10e7fc640 1 --2- 172.21.15.112:0/1544616920 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ccacc8d590 0x55ccad11d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:36.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.644+0000 7ff1177fe640 1 -- 172.21.15.112:0/1544616920 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55ccacc853b0 msgr2=0x55ccad0bd3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:28:36.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.644+0000 7ff1177fe640 1 --2- 172.21.15.112:0/1544616920 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55ccacc853b0 0x55ccad0bd3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:36.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.644+0000 7ff1177fe640 1 -- 172.21.15.112:0/1544616920 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ccacc8d590 msgr2=0x55ccad11d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:28:36.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.644+0000 7ff1177fe640 1 --2- 172.21.15.112:0/1544616920 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ccacc8d590 0x55ccad11d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:36.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.644+0000 7ff1177fe640 1 -- 172.21.15.112:0/1544616920 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ccad0bd920 con 0x55ccaccae270 2024-09-21T00:28:36.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.644+0000 7ff10e7fc640 1 --2- 172.21.15.112:0/1544616920 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ccacc8d590 0x55ccad11d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:28:36.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.644+0000 7ff1177fe640 1 --2- 172.21.15.112:0/1544616920 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ccaccae270 0x55ccad0a8380 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7ff110044000 tx=0x7ff110033780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3c035b0310b4a08 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:36.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.644+0000 7ff0f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T00:28:36.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.644+0000 7ff0f5ffb640 1 -- 172.21.15.112:0/1544616920 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff11004d070 con 0x55ccaccae270 2024-09-21T00:28:36.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.644+0000 7ff0f5ffb640 1 -- 172.21.15.112:0/1544616920 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(71..71 src has 1..71) ==== 6101+0+0 (secure 0 0 0) 0x7ff1100339b0 con 0x55ccaccae270 2024-09-21T00:28:36.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.644+0000 7ff12483c740 1 -- 172.21.15.112:0/1544616920 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff110003ba0 con 0x55ccaccae270 2024-09-21T00:28:36.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.644+0000 7ff0f5ffb640 1 -- 172.21.15.112:0/1544616920 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7ff110037050 con 0x55ccaccae270 2024-09-21T00:28:36.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.644+0000 7ff12483c740 10 client.6306 fetch_fsmap learned FSMap version 47 2024-09-21T00:28:36.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.644+0000 7ff12483c740 1 -- 172.21.15.112:0/1544616920 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=2+}) -- 0x55ccad11dee0 con 0x55ccaccae270 2024-09-21T00:28:36.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.645+0000 7ff0f5ffb640 1 -- 172.21.15.112:0/1544616920 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff110003ba0 con 0x55ccaccae270 2024-09-21T00:28:36.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.645+0000 7ff0f5ffb640 1 -- 172.21.15.112:0/1544616920 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 47) ==== 3502+0+0 (secure 0 0 0) 0x7ff110033bb0 con 0x55ccaccae270 2024-09-21T00:28:36.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.645+0000 7ff12483c740 10 client.6306 fetch_fsmap finished waiting for FSMap version 47 2024-09-21T00:28:36.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.645+0000 7ff12483c740 10 client.6306 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-21T00:28:36.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.645+0000 7ff12483c740 20 client.6306 populate_metadata read hostname 'smithi112' 2024-09-21T00:28:36.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.645+0000 7ff12483c740 1 --2- 172.21.15.112:0/1544616920 >> [v2:172.21.15.112:6833/1624734538,v1:172.21.15.112:6835/1624734538] conn(0x55ccad0a88c0 0x55ccad053de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:36.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.645+0000 7ff12483c740 4 client.6306 mds_command: new command op to 6165 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T00:28:36.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.645+0000 7ff12483c740 1 -- 172.21.15.112:0/1544616920 --> [v2:172.21.15.112:6833/1624734538,v1:172.21.15.112:6835/1624734538] -- command(tid 0: {"prefix": "status"}) -- 0x55ccacfc6220 con 0x55ccad0a88c0 2024-09-21T00:28:36.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.645+0000 7ff116ffd640 1 --2- 172.21.15.112:0/1544616920 >> [v2:172.21.15.112:6833/1624734538,v1:172.21.15.112:6835/1624734538] conn(0x55ccad0a88c0 0x55ccad053de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:36.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.645+0000 7ff0f5ffb640 10 client.6306 ms_handle_connect on v2:172.21.15.112:6833/1624734538 2024-09-21T00:28:36.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.645+0000 7ff116ffd640 1 --2- 172.21.15.112:0/1544616920 >> [v2:172.21.15.112:6833/1624734538,v1:172.21.15.112:6835/1624734538] conn(0x55ccad0a88c0 0x55ccad053de0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f04a693f8961b378 server_cookie=ccef835b4d772da7 in_seq=0 out_seq=0 2024-09-21T00:28:36.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.645+0000 7ff0f5ffb640 1 -- 172.21.15.112:0/1544616920 <== mds.0 v2:172.21.15.112:6833/1624734538 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55ccacfc6220 con 0x55ccad0a88c0 2024-09-21T00:28:36.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.645+0000 7ff0f5ffb640 10 client.6306 handle_command_reply: tid=0 2024-09-21T00:28:36.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.645+0000 7ff12483c740 1 client.6306 shutdown 2024-09-21T00:28:36.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.646+0000 7ff12483c740 1 -- 172.21.15.112:0/1544616920 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ccaccae270 msgr2=0x55ccad0a8380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:36.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.646+0000 7ff12483c740 1 --2- 172.21.15.112:0/1544616920 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ccaccae270 0x55ccad0a8380 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7ff110044000 tx=0x7ff110033780 comp rx=0 tx=0).stop 2024-09-21T00:28:36.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.646+0000 7ff12483c740 1 -- 172.21.15.112:0/1544616920 shutdown_connections 2024-09-21T00:28:36.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.646+0000 7ff12483c740 1 --2- 172.21.15.112:0/1544616920 >> [v2:172.21.15.112:6833/1624734538,v1:172.21.15.112:6835/1624734538] conn(0x55ccad0a88c0 0x55ccad053de0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:36.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.646+0000 7ff12483c740 1 --2- 172.21.15.112:0/1544616920 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ccaccae270 0x55ccad0a8380 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:36.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.646+0000 7ff0f5ffb640 0 client.6306 ms_handle_reset on v2:172.21.15.112:6833/1624734538 2024-09-21T00:28:36.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.646+0000 7ff12483c740 1 --2- 172.21.15.112:0/1544616920 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ccacc8d590 0x55ccad11d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:36.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.646+0000 7ff12483c740 1 --2- 172.21.15.112:0/1544616920 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55ccacc853b0 0x55ccad0bd3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:36.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.646+0000 7ff12483c740 1 -- 172.21.15.112:0/1544616920 >> 172.21.15.112:0/1544616920 conn(0x55ccacca23c0 msgr2=0x55ccad077840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:36.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.646+0000 7ff12483c740 1 -- 172.21.15.112:0/1544616920 shutdown_connections 2024-09-21T00:28:36.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.646+0000 7ff12483c740 1 -- 172.21.15.112:0/1544616920 wait complete. 2024-09-21T00:28:36.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.646+0000 7ff12483c740 20 client.6306 trim_cache size 0 max 16384 2024-09-21T00:28:36.653 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-21T00:28:36.653 INFO:teuthology.orchestra.run.smithi112.stdout: "cluster_fsid": "8b34692b-dc65-49b2-a08f-26f707b96ac5", 2024-09-21T00:28:36.653 INFO:teuthology.orchestra.run.smithi112.stdout: "whoami": 0, 2024-09-21T00:28:36.653 INFO:teuthology.orchestra.run.smithi112.stdout: "id": 6165, 2024-09-21T00:28:36.653 INFO:teuthology.orchestra.run.smithi112.stdout: "want_state": "up:active", 2024-09-21T00:28:36.653 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "up:active", 2024-09-21T00:28:36.653 INFO:teuthology.orchestra.run.smithi112.stdout: "fs_name": "cephfs", 2024-09-21T00:28:36.653 INFO:teuthology.orchestra.run.smithi112.stdout: "rank_uptime": 5.7214963870000002, 2024-09-21T00:28:36.653 INFO:teuthology.orchestra.run.smithi112.stdout: "mdsmap_epoch": 47, 2024-09-21T00:28:36.654 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch": 71, 2024-09-21T00:28:36.654 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch_barrier": 71, 2024-09-21T00:28:36.654 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 10.404450457999999 2024-09-21T00:28:36.654 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-21T00:28:36.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.646+0000 7ff11d7f1640 1 -- 172.21.15.112:0/2789548892 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff0f8078b90 msgr2=0x7ff0f807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:36.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.646+0000 7ff11d7f1640 1 --2- 172.21.15.112:0/2789548892 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff0f8078b90 0x7ff0f807b0a0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7ff11815bb00 tx=0x7ff100009040 comp rx=0 tx=0).stop 2024-09-21T00:28:36.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.646+0000 7ff11d7f1640 1 -- 172.21.15.112:0/2789548892 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1180b37a0 msgr2=0x7ff1181f5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:36.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.646+0000 7ff11d7f1640 1 --2- 172.21.15.112:0/2789548892 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1180b37a0 0x7ff1181f5a00 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7ff10800b420 tx=0x7ff10800b8e0 comp rx=0 tx=0).stop 2024-09-21T00:28:36.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.646+0000 7ff11d7f1640 1 -- 172.21.15.112:0/2789548892 shutdown_connections 2024-09-21T00:28:36.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.646+0000 7ff11d7f1640 1 --2- 172.21.15.112:0/2789548892 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff11815c3c0 0x7ff11814dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:36.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.646+0000 7ff11d7f1640 1 --2- 172.21.15.112:0/2789548892 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff1180b3bb0 0x7ff11806d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:36.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.646+0000 7ff11d7f1640 1 --2- 172.21.15.112:0/2789548892 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff0f8078b90 0x7ff0f807b0a0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:36.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.646+0000 7ff11d7f1640 1 --2- 172.21.15.112:0/2789548892 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1180b37a0 0x7ff1181f5a00 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:36.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.646+0000 7ff11d7f1640 1 -- 172.21.15.112:0/2789548892 >> 172.21.15.112:0/2789548892 conn(0x7ff11808dcb0 msgr2=0x7ff118160450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:36.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.646+0000 7ff11d7f1640 1 -- 172.21.15.112:0/2789548892 shutdown_connections 2024-09-21T00:28:36.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.646+0000 7ff11d7f1640 1 -- 172.21.15.112:0/2789548892 wait complete. 2024-09-21T00:28:36.659 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T00:28:36.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.769+0000 7f9f964d5640 1 -- 172.21.15.112:0/3921686234 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f900d3880 msgr2=0x7f9f90163260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:36.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.769+0000 7f9f964d5640 1 --2- 172.21.15.112:0/3921686234 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f900d3880 0x7f9f90163260 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f9f84009830 tx=0x7f9f8402a510 comp rx=0 tx=0).stop 2024-09-21T00:28:36.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.769+0000 7f9f964d5640 1 -- 172.21.15.112:0/3921686234 shutdown_connections 2024-09-21T00:28:36.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.769+0000 7f9f964d5640 1 --2- 172.21.15.112:0/3921686234 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f900d3880 0x7f9f90163260 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:36.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.769+0000 7f9f964d5640 1 --2- 172.21.15.112:0/3921686234 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9f900d48a0 0x7f9f900d3dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:36.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.769+0000 7f9f964d5640 1 -- 172.21.15.112:0/3921686234 >> 172.21.15.112:0/3921686234 conn(0x7f9f9008bfe0 msgr2=0x7f9f9009d890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:36.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.769+0000 7f9f964d5640 1 -- 172.21.15.112:0/3921686234 shutdown_connections 2024-09-21T00:28:36.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.769+0000 7f9f964d5640 1 -- 172.21.15.112:0/3921686234 wait complete. 2024-09-21T00:28:36.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.771+0000 7f9f964d5640 1 Processor -- start 2024-09-21T00:28:36.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.771+0000 7f9f964d5640 1 -- start start 2024-09-21T00:28:36.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.772+0000 7f9f964d5640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9f900d01b0 0x7f9f900c98f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:36.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.772+0000 7f9f964d5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f900d3880 0x7f9f900c9e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:36.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.772+0000 7f9f964d5640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9f900d48a0 0x7f9f901f67c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:36.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.772+0000 7f9f964d5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9f900d2490 con 0x7f9f900d3880 2024-09-21T00:28:36.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.772+0000 7f9f964d5640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9f900d2310 con 0x7f9f900d01b0 2024-09-21T00:28:36.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.772+0000 7f9f964d5640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f9f900d2610 con 0x7f9f900d48a0 2024-09-21T00:28:36.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.772+0000 7f9f8f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f900d3880 0x7f9f900c9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:36.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.772+0000 7f9f8f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f900d3880 0x7f9f900c9e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33242/0 (socket says 172.21.15.112:33242) 2024-09-21T00:28:36.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.772+0000 7f9f8f7fe640 1 -- 172.21.15.112:0/1741441658 learned_addr learned my addr 172.21.15.112:0/1741441658 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:28:36.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.773+0000 7f9f8ffff640 1 --2- 172.21.15.112:0/1741441658 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9f900d01b0 0x7f9f900c98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:36.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.773+0000 7f9f94a72640 1 --2- 172.21.15.112:0/1741441658 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9f900d48a0 0x7f9f901f67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:36.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.773+0000 7f9f8f7fe640 1 -- 172.21.15.112:0/1741441658 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9f900d48a0 msgr2=0x7f9f901f67c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:36.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.773+0000 7f9f8f7fe640 1 --2- 172.21.15.112:0/1741441658 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9f900d48a0 0x7f9f901f67c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:36.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.773+0000 7f9f8f7fe640 1 -- 172.21.15.112:0/1741441658 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9f900d01b0 msgr2=0x7f9f900c98f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:36.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.773+0000 7f9f8f7fe640 1 --2- 172.21.15.112:0/1741441658 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9f900d01b0 0x7f9f900c98f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:36.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.773+0000 7f9f8f7fe640 1 -- 172.21.15.112:0/1741441658 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f9015ffe0 con 0x7f9f900d3880 2024-09-21T00:28:36.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.773+0000 7f9f8ffff640 1 --2- 172.21.15.112:0/1741441658 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9f900d01b0 0x7f9f900c98f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:28:36.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.773+0000 7f9f8f7fe640 1 --2- 172.21.15.112:0/1741441658 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f900d3880 0x7f9f900c9e30 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f9f84009950 tx=0x7f9f84033c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:36.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.773+0000 7f9f8cff9640 1 -- 172.21.15.112:0/1741441658 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f84005c90 con 0x7f9f900d3880 2024-09-21T00:28:36.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.773+0000 7f9f8cff9640 1 -- 172.21.15.112:0/1741441658 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9f84031e50 con 0x7f9f900d3880 2024-09-21T00:28:36.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.773+0000 7f9f964d5640 1 -- 172.21.15.112:0/1741441658 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f901f58c0 con 0x7f9f900d3880 2024-09-21T00:28:36.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.773+0000 7f9f8cff9640 1 -- 172.21.15.112:0/1741441658 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f84005260 con 0x7f9f900d3880 2024-09-21T00:28:36.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.773+0000 7f9f964d5640 1 -- 172.21.15.112:0/1741441658 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f901f5e80 con 0x7f9f900d3880 2024-09-21T00:28:36.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.774+0000 7f9f964d5640 1 -- 172.21.15.112:0/1741441658 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f900d2b60 con 0x7f9f900d3880 2024-09-21T00:28:36.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.775+0000 7f9f8cff9640 1 -- 172.21.15.112:0/1741441658 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9f84005400 con 0x7f9f900d3880 2024-09-21T00:28:36.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.775+0000 7f9f8cff9640 1 --2- 172.21.15.112:0/1741441658 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f9f7c078b90 0x7f9f7c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:36.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.776+0000 7f9f8cff9640 1 -- 172.21.15.112:0/1741441658 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6101+0+0 (secure 0 0 0) 0x7f9f840bc100 con 0x7f9f900d3880 2024-09-21T00:28:36.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.776+0000 7f9f8ffff640 1 --2- 172.21.15.112:0/1741441658 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f9f7c078b90 0x7f9f7c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:36.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.776+0000 7f9f8ffff640 1 --2- 172.21.15.112:0/1741441658 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f9f7c078b90 0x7f9f7c07b0a0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f9f80005fd0 tx=0x7f9f8000f360 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:36.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.780+0000 7f9f8cff9640 1 -- 172.21.15.112:0/1741441658 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f9f8403f030 con 0x7f9f900d3880 2024-09-21T00:28:36.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.993+0000 7f9f964d5640 1 -- 172.21.15.112:0/1741441658 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f9f9015fd70 con 0x7f9f900d3880 2024-09-21T00:28:36.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.994+0000 7f9f8cff9640 1 -- 172.21.15.112:0/1741441658 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v47) ==== 53+0+73 (secure 0 0 0) 0x7f9f840843b0 con 0x7f9f900d3880 2024-09-21T00:28:36.996 INFO:teuthology.orchestra.run.smithi112.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T00:28:36.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.996+0000 7f9f964d5640 1 -- 172.21.15.112:0/1741441658 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f9f7c078b90 msgr2=0x7f9f7c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:36.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.996+0000 7f9f964d5640 1 --2- 172.21.15.112:0/1741441658 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f9f7c078b90 0x7f9f7c07b0a0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f9f80005fd0 tx=0x7f9f8000f360 comp rx=0 tx=0).stop 2024-09-21T00:28:36.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.996+0000 7f9f964d5640 1 -- 172.21.15.112:0/1741441658 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f900d3880 msgr2=0x7f9f900c9e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:36.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.996+0000 7f9f964d5640 1 --2- 172.21.15.112:0/1741441658 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f900d3880 0x7f9f900c9e30 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f9f84009950 tx=0x7f9f84033c90 comp rx=0 tx=0).stop 2024-09-21T00:28:36.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.996+0000 7f9f964d5640 1 -- 172.21.15.112:0/1741441658 shutdown_connections 2024-09-21T00:28:36.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.996+0000 7f9f964d5640 1 --2- 172.21.15.112:0/1741441658 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9f900d48a0 0x7f9f901f67c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:36.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.996+0000 7f9f964d5640 1 --2- 172.21.15.112:0/1741441658 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f9f7c078b90 0x7f9f7c07b0a0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:37.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.996+0000 7f9f964d5640 1 --2- 172.21.15.112:0/1741441658 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f900d3880 0x7f9f900c9e30 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:37.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.996+0000 7f9f964d5640 1 --2- 172.21.15.112:0/1741441658 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9f900d01b0 0x7f9f900c98f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:37.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.996+0000 7f9f964d5640 1 -- 172.21.15.112:0/1741441658 >> 172.21.15.112:0/1741441658 conn(0x7f9f9008bfe0 msgr2=0x7f9f900b2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:37.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.996+0000 7f9f964d5640 1 -- 172.21.15.112:0/1741441658 shutdown_connections 2024-09-21T00:28:37.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:36.996+0000 7f9f964d5640 1 -- 172.21.15.112:0/1741441658 wait complete. 2024-09-21T00:28:37.009 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T00:28:37.009 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T00:28:37.010 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi112.front.sepia.ceph.com 2024-09-21T00:28:37.010 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T00:28:37.010 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T00:28:37.010 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T00:28:37.010 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-21T00:28:37.010 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T00:28:37.011 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T00:28:37.011 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:28:37.011 DEBUG:teuthology.orchestra.run.smithi112:> ip addr 2024-09-21T00:28:37.027 INFO:teuthology.orchestra.run.smithi112.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T00:28:37.027 INFO:teuthology.orchestra.run.smithi112.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T00:28:37.028 INFO:teuthology.orchestra.run.smithi112.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T00:28:37.028 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:28:37.028 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 ::1/128 scope host 2024-09-21T00:28:37.028 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:28:37.028 INFO:teuthology.orchestra.run.smithi112.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T00:28:37.028 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:d8:a5:2c brd ff:ff:ff:ff:ff:ff 2024-09-21T00:28:37.028 INFO:teuthology.orchestra.run.smithi112.stdout: altname enp7s0f0 2024-09-21T00:28:37.028 INFO:teuthology.orchestra.run.smithi112.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T00:28:37.028 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:d8:a5:2d brd ff:ff:ff:ff:ff:ff 2024-09-21T00:28:37.028 INFO:teuthology.orchestra.run.smithi112.stdout: altname enp7s0f1 2024-09-21T00:28:37.028 INFO:teuthology.orchestra.run.smithi112.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T00:28:37.028 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:8f:cb:68 brd ff:ff:ff:ff:ff:ff 2024-09-21T00:28:37.028 INFO:teuthology.orchestra.run.smithi112.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T00:28:37.028 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:8f:cb:69 brd ff:ff:ff:ff:ff:ff 2024-09-21T00:28:37.028 INFO:teuthology.orchestra.run.smithi112.stdout: inet 172.21.15.112/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-21T00:28:37.029 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft 42605sec preferred_lft 42605sec 2024-09-21T00:28:37.029 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::ec4:7aff:fe8f:cb69/64 scope link noprefixroute 2024-09-21T00:28:37.029 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:28:37.029 INFO:teuthology.orchestra.run.smithi112.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T00:28:37.029 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 1a:39:64:e7:fe:d7 brd ff:ff:ff:ff:ff:ff 2024-09-21T00:28:37.029 INFO:teuthology.orchestra.run.smithi112.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T00:28:37.029 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:28:37.029 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::5858:3dff:feaf:1774/64 scope link 2024-09-21T00:28:37.029 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:28:37.029 INFO:teuthology.orchestra.run.smithi112.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T00:28:37.029 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 1a:39:64:e7:fe:d7 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T00:28:37.029 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::1839:64ff:fee7:fed7/64 scope link 2024-09-21T00:28:37.029 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:28:37.030 INFO:teuthology.orchestra.run.smithi112.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T00:28:37.030 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 86:c9:6c:ef:e4:e9 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T00:28:37.030 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::84c9:6cff:feef:e4e9/64 scope link 2024-09-21T00:28:37.030 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:28:37.030 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:28:37.030 DEBUG:teuthology.orchestra.run.smithi112:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T00:28:37.091 INFO:teuthology.orchestra.run.smithi112.stdout:1 2024-09-21T00:28:37.092 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:28:37.093 DEBUG:teuthology.orchestra.run.smithi112:> route 2024-09-21T00:28:37.148 INFO:teuthology.orchestra.run.smithi112.stdout:Kernel IP routing table 2024-09-21T00:28:37.148 INFO:teuthology.orchestra.run.smithi112.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T00:28:37.148 INFO:teuthology.orchestra.run.smithi112.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-21T00:28:37.148 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-21T00:28:37.149 INFO:teuthology.orchestra.run.smithi112.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T00:28:37.149 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T00:28:37.149 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2024-09-21T00:28:37.149 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-21T00:28:37.149 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-21T00:28:37.149 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-21T00:28:37.149 DEBUG:teuthology.orchestra.run.smithi112:> ') 2024-09-21T00:28:37.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:37 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T00:28:37.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T00:28:37.274 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:28:37.274 DEBUG:teuthology.orchestra.run.smithi112:> ip netns list 2024-09-21T00:28:37.290 INFO:teuthology.orchestra.run.smithi112.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-21T00:28:37.290 INFO:teuthology.orchestra.run.smithi112.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T00:28:37.290 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T00:28:37.290 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:28:37.290 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link set brx.0 up 2024-09-21T00:28:37.355 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T00:28:37.356 DEBUG:teuthology.orchestra.run.smithi112:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T00:28:37.411 INFO:teuthology.orchestra.run.smithi112.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T00:28:37.411 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T00:28:37.411 DEBUG:teuthology.orchestra.run.smithi112:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T00:28:37.466 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-21T00:28:37.467 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:28:37.467 DEBUG:teuthology.orchestra.run.smithi112:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T00:28:37.560 INFO:teuthology.orchestra.run.smithi112.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T00:28:37.561 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T00:28:37.561 INFO:teuthology.orchestra.run.smithi112.stdout:invalid new device string format 2024-09-21T00:28:37.571 INFO:teuthology.orchestra.run.smithi112.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T00:28:37.571 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T00:28:37.571 INFO:teuthology.orchestra.run.smithi112.stdout:invalid new device string format 2024-09-21T00:28:37.571 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: resolved to: "172.21.15.112:3300,172.21.15.187:3300,172.21.15.187:3301" 2024-09-21T00:28:37.571 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T00:28:37.577 INFO:teuthology.orchestra.run.smithi112.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T00:28:37.577 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T00:28:37.577 INFO:teuthology.orchestra.run.smithi112.stdout:invalid new device string format 2024-09-21T00:28:37.578 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: resolved to: "172.21.15.112:3300,172.21.15.187:3300,172.21.15.187:3301" 2024-09-21T00:28:37.578 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T00:28:37.578 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: resolved to: "172.21.15.112:6789,172.21.15.187:6789,172.21.15.187:6790" 2024-09-21T00:28:37.578 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.112:6789,172.21.15.187:6789,172.21.15.187:6790:/ 2024-09-21T00:28:37.578 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=8b34692b-dc65-49b2-a08f-26f707b96ac5" will pass to kernel 2024-09-21T00:28:37.579 INFO:teuthology.orchestra.run.smithi112.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-21T00:28:37.579 INFO:teuthology.orchestra.run.smithi112.stdout: You just mounted a file system that supports labels which does not 2024-09-21T00:28:37.579 INFO:teuthology.orchestra.run.smithi112.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T00:28:37.579 INFO:teuthology.orchestra.run.smithi112.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T00:28:37.579 INFO:teuthology.orchestra.run.smithi112.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T00:28:37.580 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T00:28:37.580 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:28:37.580 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T00:28:37.609 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T00:28:37.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:37 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T00:28:37.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T00:28:37.713 INFO:teuthology.orchestra.run.smithi112.stdout:client6318 2024-09-21T00:28:37.713 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T00:28:37.713 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:28:37.729 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T00:28:37.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:37.882+0000 7f6fa655e640 1 -- 172.21.15.112:0/2767735432 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6f8c004800 msgr2=0x7f6f8c004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:37.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:37.882+0000 7f6fa655e640 1 --2- 172.21.15.112:0/2767735432 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6f8c004800 0x7f6f8c004c10 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f6fa00b7490 tx=0x7f6f8c005830 comp rx=0 tx=0).stop 2024-09-21T00:28:37.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:37.882+0000 7f6fa655e640 1 -- 172.21.15.112:0/2767735432 shutdown_connections 2024-09-21T00:28:37.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:37.882+0000 7f6fa655e640 1 --2- 172.21.15.112:0/2767735432 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6f8c004800 0x7f6f8c004c10 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:37.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:37.882+0000 7f6fa655e640 1 -- 172.21.15.112:0/2767735432 >> 172.21.15.112:0/2767735432 conn(0x7f6fa009d8a0 msgr2=0x7f6fa008bfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:37.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:37.882+0000 7f6fa655e640 1 -- 172.21.15.112:0/2767735432 shutdown_connections 2024-09-21T00:28:37.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:37.882+0000 7f6fa655e640 1 -- 172.21.15.112:0/2767735432 wait complete. 2024-09-21T00:28:37.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:37.883+0000 7f6fa655e640 1 Processor -- start 2024-09-21T00:28:37.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:37.884+0000 7f6fa655e640 1 -- start start 2024-09-21T00:28:37.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:37.885+0000 7f6fa655e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6f8c004800 0x7f6fa00b3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:37.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:37.885+0000 7f6fa655e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6fa01f5460 0x7f6fa00b5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:37.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:37.885+0000 7f6fa655e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6fa01f2840 0x7f6fa01f90f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:37.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:37.885+0000 7f6fa655e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6fa00705e0 con 0x7f6fa01f2840 2024-09-21T00:28:37.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:37.885+0000 7f6fa655e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f6fa0070460 con 0x7f6fa01f5460 2024-09-21T00:28:37.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:37.885+0000 7f6fa655e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f6fa0070760 con 0x7f6f8c004800 2024-09-21T00:28:37.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:37.885+0000 7f6fa4afb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6fa01f2840 0x7f6fa01f90f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:37.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:37.885+0000 7f6fa4afb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6fa01f2840 0x7f6fa01f90f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33260/0 (socket says 172.21.15.112:33260) 2024-09-21T00:28:37.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:37.885+0000 7f6fa4afb640 1 -- 172.21.15.112:0/2820370914 learned_addr learned my addr 172.21.15.112:0/2820370914 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:28:37.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:37.885+0000 7f6f9ffff640 1 --2- 172.21.15.112:0/2820370914 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6f8c004800 0x7f6fa00b3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:37.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:37.885+0000 7f6fa4afb640 1 -- 172.21.15.112:0/2820370914 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6f8c004800 msgr2=0x7f6fa00b3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:37.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:37.885+0000 7f6f9f7fe640 1 --2- 172.21.15.112:0/2820370914 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6fa01f5460 0x7f6fa00b5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:37.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:37.885+0000 7f6fa4afb640 1 --2- 172.21.15.112:0/2820370914 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6f8c004800 0x7f6fa00b3ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:37.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:37.885+0000 7f6fa4afb640 1 -- 172.21.15.112:0/2820370914 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6fa01f5460 msgr2=0x7f6fa00b5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:37.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:37.885+0000 7f6fa4afb640 1 --2- 172.21.15.112:0/2820370914 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6fa01f5460 0x7f6fa00b5990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:37.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:37.885+0000 7f6fa4afb640 1 -- 172.21.15.112:0/2820370914 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fa01f63c0 con 0x7f6fa01f2840 2024-09-21T00:28:37.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:37.885+0000 7f6f9ffff640 1 --2- 172.21.15.112:0/2820370914 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6f8c004800 0x7f6fa00b3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:28:37.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:37.885+0000 7f6f9f7fe640 1 --2- 172.21.15.112:0/2820370914 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6fa01f5460 0x7f6fa00b5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:28:37.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:37.885+0000 7f6fa4afb640 1 --2- 172.21.15.112:0/2820370914 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6fa01f2840 0x7f6fa01f90f0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f6f98005fd0 tx=0x7f6f98010f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:37.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:37.886+0000 7f6f9cff9640 1 -- 172.21.15.112:0/2820370914 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f98003970 con 0x7f6fa01f2840 2024-09-21T00:28:37.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:37.886+0000 7f6f9cff9640 1 -- 172.21.15.112:0/2820370914 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6f98015070 con 0x7f6fa01f2840 2024-09-21T00:28:37.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:37.886+0000 7f6fa655e640 1 -- 172.21.15.112:0/2820370914 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fa00b5ed0 con 0x7f6fa01f2840 2024-09-21T00:28:37.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:37.886+0000 7f6f9cff9640 1 -- 172.21.15.112:0/2820370914 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f98004030 con 0x7f6fa01f2840 2024-09-21T00:28:37.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:37.886+0000 7f6fa655e640 1 -- 172.21.15.112:0/2820370914 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fa01f5e00 con 0x7f6fa01f2840 2024-09-21T00:28:37.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:37.887+0000 7f6fa655e640 1 -- 172.21.15.112:0/2820370914 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fa00b3210 con 0x7f6fa01f2840 2024-09-21T00:28:37.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:37.887+0000 7f6f9cff9640 1 -- 172.21.15.112:0/2820370914 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6f980041d0 con 0x7f6fa01f2840 2024-09-21T00:28:37.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:37.888+0000 7f6f9cff9640 1 --2- 172.21.15.112:0/2820370914 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f6f80078bc0 0x7f6f8007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:37.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:37.888+0000 7f6f9cff9640 1 -- 172.21.15.112:0/2820370914 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6101+0+0 (secure 0 0 0) 0x7f6f9809ba00 con 0x7f6fa01f2840 2024-09-21T00:28:37.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:37.888+0000 7f6f9ffff640 1 --2- 172.21.15.112:0/2820370914 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f6f80078bc0 0x7f6f8007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:37.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:37.889+0000 7f6f9ffff640 1 --2- 172.21.15.112:0/2820370914 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f6f80078bc0 0x7f6f8007b0d0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f6fa006c310 tx=0x7f6f88006b50 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:37.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:37.893+0000 7f6f9cff9640 1 -- 172.21.15.112:0/2820370914 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f6fa00b3210 con 0x7f6fa01f2840 2024-09-21T00:28:38.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:38.112+0000 7f6fa655e640 1 -- 172.21.15.112:0/2820370914 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f6fa01f58b0 con 0x7f6fa01f2840 2024-09-21T00:28:38.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:38.112+0000 7f6f9cff9640 1 -- 172.21.15.112:0/2820370914 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v47) ==== 53+0+73 (secure 0 0 0) 0x7f6f98063d20 con 0x7f6fa01f2840 2024-09-21T00:28:38.114 INFO:teuthology.orchestra.run.smithi112.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T00:28:38.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:38.114+0000 7f6fa655e640 1 -- 172.21.15.112:0/2820370914 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f6f80078bc0 msgr2=0x7f6f8007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:38.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:38.114+0000 7f6fa655e640 1 --2- 172.21.15.112:0/2820370914 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f6f80078bc0 0x7f6f8007b0d0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f6fa006c310 tx=0x7f6f88006b50 comp rx=0 tx=0).stop 2024-09-21T00:28:38.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:38.114+0000 7f6fa655e640 1 -- 172.21.15.112:0/2820370914 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6fa01f2840 msgr2=0x7f6fa01f90f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:38.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:38.114+0000 7f6fa655e640 1 --2- 172.21.15.112:0/2820370914 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6fa01f2840 0x7f6fa01f90f0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f6f98005fd0 tx=0x7f6f98010f40 comp rx=0 tx=0).stop 2024-09-21T00:28:38.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:38.114+0000 7f6fa655e640 1 -- 172.21.15.112:0/2820370914 shutdown_connections 2024-09-21T00:28:38.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:38.114+0000 7f6fa655e640 1 --2- 172.21.15.112:0/2820370914 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f6f80078bc0 0x7f6f8007b0d0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:38.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:38.114+0000 7f6fa655e640 1 --2- 172.21.15.112:0/2820370914 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6fa01f2840 0x7f6fa01f90f0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:38.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:38.114+0000 7f6fa655e640 1 --2- 172.21.15.112:0/2820370914 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6fa01f5460 0x7f6fa00b5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:38.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:38.114+0000 7f6fa655e640 1 --2- 172.21.15.112:0/2820370914 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6f8c004800 0x7f6fa00b3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:38.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:38.114+0000 7f6fa655e640 1 -- 172.21.15.112:0/2820370914 >> 172.21.15.112:0/2820370914 conn(0x7f6fa009d8a0 msgr2=0x7f6fa015ef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:38.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:38.114+0000 7f6fa655e640 1 -- 172.21.15.112:0/2820370914 shutdown_connections 2024-09-21T00:28:38.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:38.115+0000 7f6fa655e640 1 -- 172.21.15.112:0/2820370914 wait complete. 2024-09-21T00:28:38.127 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T00:28:38.128 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T00:28:38.128 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi187.front.sepia.ceph.com 2024-09-21T00:28:38.128 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-21T00:28:38.128 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-21T00:28:38.128 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T00:28:38.128 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-21T00:28:38.128 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T00:28:38.128 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-21T00:28:38.128 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:28:38.128 DEBUG:teuthology.orchestra.run.smithi187:> ip addr 2024-09-21T00:28:38.149 INFO:teuthology.orchestra.run.smithi187.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T00:28:38.149 INFO:teuthology.orchestra.run.smithi187.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T00:28:38.149 INFO:teuthology.orchestra.run.smithi187.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T00:28:38.149 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:28:38.149 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 ::1/128 scope host 2024-09-21T00:28:38.149 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:28:38.149 INFO:teuthology.orchestra.run.smithi187.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T00:28:38.149 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 0c:c4:7a:d9:36:78 brd ff:ff:ff:ff:ff:ff 2024-09-21T00:28:38.149 INFO:teuthology.orchestra.run.smithi187.stdout: altname enp7s0f0 2024-09-21T00:28:38.150 INFO:teuthology.orchestra.run.smithi187.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T00:28:38.150 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 0c:c4:7a:d9:36:79 brd ff:ff:ff:ff:ff:ff 2024-09-21T00:28:38.150 INFO:teuthology.orchestra.run.smithi187.stdout: altname enp7s0f1 2024-09-21T00:28:38.150 INFO:teuthology.orchestra.run.smithi187.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T00:28:38.150 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 0c:c4:7a:88:77:b0 brd ff:ff:ff:ff:ff:ff 2024-09-21T00:28:38.150 INFO:teuthology.orchestra.run.smithi187.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T00:28:38.150 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 0c:c4:7a:88:77:b1 brd ff:ff:ff:ff:ff:ff 2024-09-21T00:28:38.150 INFO:teuthology.orchestra.run.smithi187.stdout: inet 172.21.15.187/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-21T00:28:38.151 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft 42593sec preferred_lft 42593sec 2024-09-21T00:28:38.151 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 fe80::ec4:7aff:fe88:77b1/64 scope link noprefixroute 2024-09-21T00:28:38.151 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:28:38.151 INFO:teuthology.orchestra.run.smithi187.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T00:28:38.151 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 86:3b:f6:5b:1b:32 brd ff:ff:ff:ff:ff:ff 2024-09-21T00:28:38.151 INFO:teuthology.orchestra.run.smithi187.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T00:28:38.151 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:28:38.151 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 fe80::ac6f:97ff:fe09:b82e/64 scope link 2024-09-21T00:28:38.151 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:28:38.151 INFO:teuthology.orchestra.run.smithi187.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T00:28:38.151 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether ca:0b:a0:f5:2b:b5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T00:28:38.152 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 fe80::c80b:a0ff:fef5:2bb5/64 scope link 2024-09-21T00:28:38.152 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:28:38.152 INFO:teuthology.orchestra.run.smithi187.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T00:28:38.152 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 86:3b:f6:5b:1b:32 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-21T00:28:38.152 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 fe80::843b:f6ff:fe5b:1b32/64 scope link 2024-09-21T00:28:38.152 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:28:38.152 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:28:38.152 DEBUG:teuthology.orchestra.run.smithi187:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T00:28:38.217 INFO:teuthology.orchestra.run.smithi187.stdout:1 2024-09-21T00:28:38.218 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:28:38.219 DEBUG:teuthology.orchestra.run.smithi187:> route 2024-09-21T00:28:38.277 INFO:teuthology.orchestra.run.smithi187.stdout:Kernel IP routing table 2024-09-21T00:28:38.277 INFO:teuthology.orchestra.run.smithi187.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T00:28:38.277 INFO:teuthology.orchestra.run.smithi187.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-21T00:28:38.277 INFO:teuthology.orchestra.run.smithi187.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-21T00:28:38.277 INFO:teuthology.orchestra.run.smithi187.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T00:28:38.277 DEBUG:teuthology.orchestra.run.smithi187:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T00:28:38.277 DEBUG:teuthology.orchestra.run.smithi187:> set -e 2024-09-21T00:28:38.278 DEBUG:teuthology.orchestra.run.smithi187:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-21T00:28:38.278 DEBUG:teuthology.orchestra.run.smithi187:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-21T00:28:38.278 DEBUG:teuthology.orchestra.run.smithi187:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-21T00:28:38.278 DEBUG:teuthology.orchestra.run.smithi187:> ') 2024-09-21T00:28:38.369 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:28:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T00:28:38.402 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:28:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T00:28:38.408 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:28:38.408 DEBUG:teuthology.orchestra.run.smithi187:> ip netns list 2024-09-21T00:28:38.424 INFO:teuthology.orchestra.run.smithi187.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-21T00:28:38.424 INFO:teuthology.orchestra.run.smithi187.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-21T00:28:38.424 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-21T00:28:38.425 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:28:38.425 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip link set brx.0 up 2024-09-21T00:28:38.489 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T00:28:38.489 DEBUG:teuthology.orchestra.run.smithi187:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-21T00:28:38.544 INFO:teuthology.orchestra.run.smithi187.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-21T00:28:38.545 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T00:28:38.545 DEBUG:teuthology.orchestra.run.smithi187:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-21T00:28:38.600 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-21T00:28:38.600 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:28:38.600 DEBUG:teuthology.orchestra.run.smithi187:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-21T00:28:38.694 INFO:teuthology.orchestra.run.smithi187.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-21T00:28:38.694 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-21T00:28:38.694 INFO:teuthology.orchestra.run.smithi187.stdout:invalid new device string format 2024-09-21T00:28:38.704 INFO:teuthology.orchestra.run.smithi187.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-21T00:28:38.704 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-21T00:28:38.705 INFO:teuthology.orchestra.run.smithi187.stdout:invalid new device string format 2024-09-21T00:28:38.705 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: resolved to: "172.21.15.112:3300,172.21.15.187:3300,172.21.15.187:3301" 2024-09-21T00:28:38.705 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T00:28:38.710 INFO:teuthology.orchestra.run.smithi187.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-21T00:28:38.711 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-21T00:28:38.711 INFO:teuthology.orchestra.run.smithi187.stdout:invalid new device string format 2024-09-21T00:28:38.711 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: resolved to: "172.21.15.112:3300,172.21.15.187:3300,172.21.15.187:3301" 2024-09-21T00:28:38.711 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T00:28:38.711 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: resolved to: "172.21.15.112:6789,172.21.15.187:6789,172.21.15.187:6790" 2024-09-21T00:28:38.711 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.112:6789,172.21.15.187:6789,172.21.15.187:6790:/ 2024-09-21T00:28:38.711 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=8b34692b-dc65-49b2-a08f-26f707b96ac5" will pass to kernel 2024-09-21T00:28:38.711 INFO:teuthology.orchestra.run.smithi187.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-09-21T00:28:38.712 INFO:teuthology.orchestra.run.smithi187.stdout: You just mounted a file system that supports labels which does not 2024-09-21T00:28:38.712 INFO:teuthology.orchestra.run.smithi187.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T00:28:38.712 INFO:teuthology.orchestra.run.smithi187.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T00:28:38.712 INFO:teuthology.orchestra.run.smithi187.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T00:28:38.713 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T00:28:38.713 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:28:38.713 DEBUG:teuthology.orchestra.run.smithi187:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-21T00:28:38.737 DEBUG:teuthology.orchestra.run.smithi187:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T00:28:38.827 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:28:38 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T00:28:38.828 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:28:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T00:28:38.834 INFO:teuthology.orchestra.run.smithi187.stdout:client5516 2024-09-21T00:28:38.835 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-09-21T00:28:38.835 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:28:38.850 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T00:28:38.850 DEBUG:teuthology.orchestra.run.smithi112:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T00:28:38.892 DEBUG:teuthology.orchestra.run.smithi187:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T00:28:38.924 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T00:28:38.924 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T00:28:38.925 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T00:28:38.925 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:28:38.950 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T00:28:38.951 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:28:39.008 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T00:28:39.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:39.164+0000 7fa9a2c7e640 1 -- 172.21.15.112:0/2664624752 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa99c164eb0 msgr2=0x7fa99c1672d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:39.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:39.164+0000 7fa9a2c7e640 1 --2- 172.21.15.112:0/2664624752 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa99c164eb0 0x7fa99c1672d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa984009860 tx=0x7fa98402a480 comp rx=0 tx=0).stop 2024-09-21T00:28:39.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:39.164+0000 7fa9a2c7e640 1 -- 172.21.15.112:0/2664624752 shutdown_connections 2024-09-21T00:28:39.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:39.164+0000 7fa9a2c7e640 1 --2- 172.21.15.112:0/2664624752 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa99c164eb0 0x7fa99c1672d0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:39.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:39.164+0000 7fa9a2c7e640 1 -- 172.21.15.112:0/2664624752 >> 172.21.15.112:0/2664624752 conn(0x7fa99c0b42e0 msgr2=0x7fa99c0b4720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:39.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:39.164+0000 7fa9a2c7e640 1 -- 172.21.15.112:0/2664624752 shutdown_connections 2024-09-21T00:28:39.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:39.164+0000 7fa9a2c7e640 1 -- 172.21.15.112:0/2664624752 wait complete. 2024-09-21T00:28:39.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:39.165+0000 7fa9a2c7e640 1 Processor -- start 2024-09-21T00:28:39.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:39.165+0000 7fa9a2c7e640 1 -- start start 2024-09-21T00:28:39.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:39.167+0000 7fa9a2c7e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa99c0b1d30 0x7fa99c163030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:39.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:39.167+0000 7fa9a2c7e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa99c0ccbf0 0x7fa99c1f6b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:39.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:39.167+0000 7fa9a2c7e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa99c164eb0 0x7fa99c1f7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:39.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:39.167+0000 7fa9a2c7e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa99c167c00 con 0x7fa99c0b1d30 2024-09-21T00:28:39.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:39.167+0000 7fa9a2c7e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa99c167a80 con 0x7fa99c164eb0 2024-09-21T00:28:39.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:39.167+0000 7fa9a2c7e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa99c167d80 con 0x7fa99c0ccbf0 2024-09-21T00:28:39.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:39.167+0000 7fa9a0a1a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa99c0b1d30 0x7fa99c163030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:39.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:39.167+0000 7fa9a0a1a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa99c0b1d30 0x7fa99c163030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33266/0 (socket says 172.21.15.112:33266) 2024-09-21T00:28:39.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:39.167+0000 7fa9a0a1a640 1 -- 172.21.15.112:0/3619252978 learned_addr learned my addr 172.21.15.112:0/3619252978 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:28:39.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:39.167+0000 7fa9a121b640 1 --2- 172.21.15.112:0/3619252978 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa99c164eb0 0x7fa99c1f7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:39.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:39.167+0000 7fa99bfff640 1 --2- 172.21.15.112:0/3619252978 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa99c0ccbf0 0x7fa99c1f6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:39.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:39.167+0000 7fa9a0a1a640 1 -- 172.21.15.112:0/3619252978 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa99c0ccbf0 msgr2=0x7fa99c1f6b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:39.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:39.167+0000 7fa9a0a1a640 1 --2- 172.21.15.112:0/3619252978 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa99c0ccbf0 0x7fa99c1f6b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:39.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:39.167+0000 7fa9a0a1a640 1 -- 172.21.15.112:0/3619252978 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa99c164eb0 msgr2=0x7fa99c1f7090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:39.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:39.167+0000 7fa9a0a1a640 1 --2- 172.21.15.112:0/3619252978 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa99c164eb0 0x7fa99c1f7090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:39.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:39.167+0000 7fa9a0a1a640 1 -- 172.21.15.112:0/3619252978 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa99c15ba60 con 0x7fa99c0b1d30 2024-09-21T00:28:39.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:39.168+0000 7fa9a0a1a640 1 --2- 172.21.15.112:0/3619252978 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa99c0b1d30 0x7fa99c163030 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fa99400c770 tx=0x7fa99400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:39.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:39.168+0000 7fa9997fa640 1 -- 172.21.15.112:0/3619252978 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa994003b70 con 0x7fa99c0b1d30 2024-09-21T00:28:39.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:39.168+0000 7fa9997fa640 1 -- 172.21.15.112:0/3619252978 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa994003d10 con 0x7fa99c0b1d30 2024-09-21T00:28:39.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:39.168+0000 7fa9a2c7e640 1 -- 172.21.15.112:0/3619252978 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa99c156bf0 con 0x7fa99c0b1d30 2024-09-21T00:28:39.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:39.168+0000 7fa9997fa640 1 -- 172.21.15.112:0/3619252978 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa99400aab0 con 0x7fa99c0b1d30 2024-09-21T00:28:39.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:39.168+0000 7fa9a2c7e640 1 -- 172.21.15.112:0/3619252978 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa99c1571b0 con 0x7fa99c0b1d30 2024-09-21T00:28:39.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:39.169+0000 7fa9a2c7e640 1 -- 172.21.15.112:0/3619252978 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa99c0cf600 con 0x7fa99c0b1d30 2024-09-21T00:28:39.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:39.170+0000 7fa9997fa640 1 -- 172.21.15.112:0/3619252978 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa99400ac50 con 0x7fa99c0b1d30 2024-09-21T00:28:39.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:39.170+0000 7fa9997fa640 1 --2- 172.21.15.112:0/3619252978 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fa97c078b90 0x7fa97c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:39.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:39.170+0000 7fa9997fa640 1 -- 172.21.15.112:0/3619252978 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6101+0+0 (secure 0 0 0) 0x7fa99409ace0 con 0x7fa99c0b1d30 2024-09-21T00:28:39.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:39.170+0000 7fa99bfff640 1 --2- 172.21.15.112:0/3619252978 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fa97c078b90 0x7fa97c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:39.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:39.171+0000 7fa99bfff640 1 --2- 172.21.15.112:0/3619252978 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fa97c078b90 0x7fa97c07b0a0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fa98c007fd0 tx=0x7fa98c007eb0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:39.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:39.175+0000 7fa9997fa640 1 -- 172.21.15.112:0/3619252978 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa99c0cf600 con 0x7fa99c0b1d30 2024-09-21T00:28:39.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:39.362+0000 7fa9a2c7e640 1 -- 172.21.15.112:0/3619252978 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fa99c0cd000 con 0x7fa99c0b1d30 2024-09-21T00:28:39.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:39.362+0000 7fa9997fa640 1 -- 172.21.15.112:0/3619252978 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v71) ==== 80+0+0 (secure 0 0 0) 0x7fa9940628c0 con 0x7fa99c0b1d30 2024-09-21T00:28:39.364 INFO:teuthology.orchestra.run.smithi112.stderr:listed 0 entries 2024-09-21T00:28:39.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:39.365+0000 7fa9a2c7e640 1 -- 172.21.15.112:0/3619252978 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fa97c078b90 msgr2=0x7fa97c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:39.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:39.365+0000 7fa9a2c7e640 1 --2- 172.21.15.112:0/3619252978 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fa97c078b90 0x7fa97c07b0a0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fa98c007fd0 tx=0x7fa98c007eb0 comp rx=0 tx=0).stop 2024-09-21T00:28:39.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:39.365+0000 7fa9a2c7e640 1 -- 172.21.15.112:0/3619252978 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa99c0b1d30 msgr2=0x7fa99c163030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:39.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:39.365+0000 7fa9a2c7e640 1 --2- 172.21.15.112:0/3619252978 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa99c0b1d30 0x7fa99c163030 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fa99400c770 tx=0x7fa99400cc30 comp rx=0 tx=0).stop 2024-09-21T00:28:39.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:39.365+0000 7fa9a2c7e640 1 -- 172.21.15.112:0/3619252978 shutdown_connections 2024-09-21T00:28:39.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:39.365+0000 7fa9a2c7e640 1 --2- 172.21.15.112:0/3619252978 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa99c164eb0 0x7fa99c1f7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:39.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:39.365+0000 7fa9a2c7e640 1 --2- 172.21.15.112:0/3619252978 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa99c0ccbf0 0x7fa99c1f6b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:39.370 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:39.365+0000 7fa9a2c7e640 1 --2- 172.21.15.112:0/3619252978 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fa97c078b90 0x7fa97c07b0a0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:39.370 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:39.365+0000 7fa9a2c7e640 1 --2- 172.21.15.112:0/3619252978 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa99c0b1d30 0x7fa99c163030 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:39.370 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:39.365+0000 7fa9a2c7e640 1 -- 172.21.15.112:0/3619252978 >> 172.21.15.112:0/3619252978 conn(0x7fa99c0b42e0 msgr2=0x7fa99c0b71d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:39.370 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:39.365+0000 7fa9a2c7e640 1 -- 172.21.15.112:0/3619252978 shutdown_connections 2024-09-21T00:28:39.370 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:39.365+0000 7fa9a2c7e640 1 -- 172.21.15.112:0/3619252978 wait complete. 2024-09-21T00:28:39.378 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-21T00:28:39.378 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:28:39.378 DEBUG:teuthology.orchestra.run.smithi112:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-21T00:28:39.406 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi112.front.sepia.ceph.com 2024-09-21T00:28:39.406 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:28:39.407 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T00:28:39.462 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T00:28:39.462 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-09-21T00:28:39.462 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:28:39.480 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-09-21T00:28:39.480 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:28:39.537 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T00:28:39.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:39.648+0000 7fa484fe6640 1 -- 172.21.15.112:0/1696104396 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa4800b2680 msgr2=0x7fa4800b2a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:39.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:39.648+0000 7fa484fe6640 1 --2- 172.21.15.112:0/1696104396 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa4800b2680 0x7fa4800b2a90 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fa4800cd880 tx=0x7fa47802a8f0 comp rx=0 tx=0).stop 2024-09-21T00:28:39.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:39.648+0000 7fa484fe6640 1 -- 172.21.15.112:0/1696104396 shutdown_connections 2024-09-21T00:28:39.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:39.648+0000 7fa484fe6640 1 --2- 172.21.15.112:0/1696104396 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa4800b2680 0x7fa4800b2a90 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:39.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:39.648+0000 7fa484fe6640 1 --2- 172.21.15.112:0/1696104396 >> v2:172.21.15.187:3300/0 conn(0x7fa4800b1d30 0x7fa4800b2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:39.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:39.648+0000 7fa484fe6640 1 -- 172.21.15.112:0/1696104396 >> 172.21.15.112:0/1696104396 conn(0x7fa4800b42e0 msgr2=0x7fa4800b4720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:39.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:39.648+0000 7fa484fe6640 1 -- 172.21.15.112:0/1696104396 shutdown_connections 2024-09-21T00:28:39.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:39.648+0000 7fa484fe6640 1 -- 172.21.15.112:0/1696104396 wait complete. 2024-09-21T00:28:39.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:39.650+0000 7fa484fe6640 1 Processor -- start 2024-09-21T00:28:39.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:39.650+0000 7fa484fe6640 1 -- start start 2024-09-21T00:28:39.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:39.651+0000 7fa484fe6640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa4800b1d30 0x7fa4800caa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:39.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:39.651+0000 7fa484fe6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa4800b2680 0x7fa480155740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:39.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:39.651+0000 7fa484fe6640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa4800ccbf0 0x7fa480156870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:39.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:39.651+0000 7fa484fe6640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa4800b34a0 con 0x7fa4800b2680 2024-09-21T00:28:39.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:39.651+0000 7fa484fe6640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa4800b3320 con 0x7fa4800b1d30 2024-09-21T00:28:39.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:39.651+0000 7fa484fe6640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa4800b3620 con 0x7fa4800ccbf0 2024-09-21T00:28:39.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:39.651+0000 7fa475d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa4800b2680 0x7fa480155740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:39.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:39.651+0000 7fa475d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa4800b2680 0x7fa480155740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33282/0 (socket says 172.21.15.112:33282) 2024-09-21T00:28:39.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:39.651+0000 7fa475d9b640 1 -- 172.21.15.112:0/1703218759 learned_addr learned my addr 172.21.15.112:0/1703218759 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:28:39.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:39.652+0000 7fa47e59c640 1 --2- 172.21.15.112:0/1703218759 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa4800b1d30 0x7fa4800caa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:39.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:39.652+0000 7fa47ed9d640 1 --2- 172.21.15.112:0/1703218759 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa4800ccbf0 0x7fa480156870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:39.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:39.652+0000 7fa475d9b640 1 -- 172.21.15.112:0/1703218759 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa4800ccbf0 msgr2=0x7fa480156870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:39.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:39.652+0000 7fa475d9b640 1 --2- 172.21.15.112:0/1703218759 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa4800ccbf0 0x7fa480156870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:39.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:39.652+0000 7fa475d9b640 1 -- 172.21.15.112:0/1703218759 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa4800b1d30 msgr2=0x7fa4800caa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:39.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:39.652+0000 7fa475d9b640 1 --2- 172.21.15.112:0/1703218759 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa4800b1d30 0x7fa4800caa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:39.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:39.652+0000 7fa475d9b640 1 -- 172.21.15.112:0/1703218759 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa480156db0 con 0x7fa4800b2680 2024-09-21T00:28:39.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:39.652+0000 7fa47e59c640 1 --2- 172.21.15.112:0/1703218759 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa4800b1d30 0x7fa4800caa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:28:39.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:39.652+0000 7fa475d9b640 1 --2- 172.21.15.112:0/1703218759 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa4800b2680 0x7fa480155740 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fa46800b440 tx=0x7fa46800b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:39.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:39.652+0000 7fa4777fe640 1 -- 172.21.15.112:0/1703218759 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa468015020 con 0x7fa4800b2680 2024-09-21T00:28:39.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:39.652+0000 7fa4777fe640 1 -- 172.21.15.112:0/1703218759 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa468011980 con 0x7fa4800b2680 2024-09-21T00:28:39.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:39.652+0000 7fa484fe6640 1 -- 172.21.15.112:0/1703218759 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4801582e0 con 0x7fa4800b2680 2024-09-21T00:28:39.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:39.652+0000 7fa4777fe640 1 -- 172.21.15.112:0/1703218759 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa468011dd0 con 0x7fa4800b2680 2024-09-21T00:28:39.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:39.652+0000 7fa484fe6640 1 -- 172.21.15.112:0/1703218759 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4801588a0 con 0x7fa4800b2680 2024-09-21T00:28:39.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:39.653+0000 7fa484fe6640 1 -- 172.21.15.112:0/1703218759 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4800b2140 con 0x7fa4800b2680 2024-09-21T00:28:39.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:39.654+0000 7fa4777fe640 1 -- 172.21.15.112:0/1703218759 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa468018020 con 0x7fa4800b2680 2024-09-21T00:28:39.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:39.654+0000 7fa4777fe640 1 --2- 172.21.15.112:0/1703218759 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fa460078b90 0x7fa46007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:39.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:39.655+0000 7fa4777fe640 1 -- 172.21.15.112:0/1703218759 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6101+0+0 (secure 0 0 0) 0x7fa46809af90 con 0x7fa4800b2680 2024-09-21T00:28:39.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:39.655+0000 7fa47e59c640 1 --2- 172.21.15.112:0/1703218759 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fa460078b90 0x7fa46007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:39.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:39.655+0000 7fa47e59c640 1 --2- 172.21.15.112:0/1703218759 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fa460078b90 0x7fa46007b0a0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fa470006fd0 tx=0x7fa470008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:39.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:39.659+0000 7fa4777fe640 1 -- 172.21.15.112:0/1703218759 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa46801f020 con 0x7fa4800b2680 2024-09-21T00:28:39.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:39.845+0000 7fa484fe6640 1 -- 172.21.15.112:0/1703218759 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fa480157050 con 0x7fa4800b2680 2024-09-21T00:28:39.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:39.845+0000 7fa4777fe640 1 -- 172.21.15.112:0/1703218759 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v71) ==== 80+0+0 (secure 0 0 0) 0x7fa46800d040 con 0x7fa4800b2680 2024-09-21T00:28:39.847 INFO:teuthology.orchestra.run.smithi112.stderr:listed 0 entries 2024-09-21T00:28:39.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:39.847+0000 7fa484fe6640 1 -- 172.21.15.112:0/1703218759 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fa460078b90 msgr2=0x7fa46007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:39.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:39.847+0000 7fa484fe6640 1 --2- 172.21.15.112:0/1703218759 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fa460078b90 0x7fa46007b0a0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fa470006fd0 tx=0x7fa470008040 comp rx=0 tx=0).stop 2024-09-21T00:28:39.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:39.847+0000 7fa484fe6640 1 -- 172.21.15.112:0/1703218759 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa4800b2680 msgr2=0x7fa480155740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:39.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:39.847+0000 7fa484fe6640 1 --2- 172.21.15.112:0/1703218759 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa4800b2680 0x7fa480155740 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fa46800b440 tx=0x7fa46800b900 comp rx=0 tx=0).stop 2024-09-21T00:28:39.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:39.847+0000 7fa484fe6640 1 -- 172.21.15.112:0/1703218759 shutdown_connections 2024-09-21T00:28:39.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:39.847+0000 7fa484fe6640 1 --2- 172.21.15.112:0/1703218759 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa4800ccbf0 0x7fa480156870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:39.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:39.847+0000 7fa484fe6640 1 --2- 172.21.15.112:0/1703218759 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fa460078b90 0x7fa46007b0a0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:39.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:39.847+0000 7fa484fe6640 1 --2- 172.21.15.112:0/1703218759 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa4800b2680 0x7fa480155740 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:39.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:39.847+0000 7fa484fe6640 1 --2- 172.21.15.112:0/1703218759 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa4800b1d30 0x7fa4800caa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:39.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:39.847+0000 7fa484fe6640 1 -- 172.21.15.112:0/1703218759 >> 172.21.15.112:0/1703218759 conn(0x7fa4800b42e0 msgr2=0x7fa480071cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:39.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:39.847+0000 7fa484fe6640 1 -- 172.21.15.112:0/1703218759 shutdown_connections 2024-09-21T00:28:39.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:39.847+0000 7fa484fe6640 1 -- 172.21.15.112:0/1703218759 wait complete. 2024-09-21T00:28:39.860 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-09-21T00:28:39.860 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:28:39.860 DEBUG:teuthology.orchestra.run.smithi187:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-09-21T00:28:39.895 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi187.front.sepia.ceph.com 2024-09-21T00:28:39.895 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:28:39.895 DEBUG:teuthology.orchestra.run.smithi187:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-21T00:28:39.953 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T00:28:39.953 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T00:28:39.954 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:28:39.969 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056010,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1439 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248716k,nr_inodes=812179,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T00:28:39.969 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T00:28:39.970 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-09-21T00:28:39.970 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:28:40.011 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056013,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497440k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16835 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248720k,nr_inodes=812180,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497440k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T00:28:40.011 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T00:28:40.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.122+0000 7f7f587b0640 1 -- 172.21.15.112:0/60718946 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f500b48f0 msgr2=0x7f7f500c5ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:40.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.122+0000 7f7f587b0640 1 --2- 172.21.15.112:0/60718946 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f500b48f0 0x7f7f500c5ed0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f7f40009830 tx=0x7f7f4002a510 comp rx=0 tx=0).stop 2024-09-21T00:28:40.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.122+0000 7f7f587b0640 1 -- 172.21.15.112:0/60718946 shutdown_connections 2024-09-21T00:28:40.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.122+0000 7f7f587b0640 1 --2- 172.21.15.112:0/60718946 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f500b48f0 0x7f7f500c5ed0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:40.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.122+0000 7f7f587b0640 1 -- 172.21.15.112:0/60718946 >> 172.21.15.112:0/60718946 conn(0x7f7f500cf220 msgr2=0x7f7f500cf630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:40.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.122+0000 7f7f587b0640 1 -- 172.21.15.112:0/60718946 shutdown_connections 2024-09-21T00:28:40.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.122+0000 7f7f587b0640 1 -- 172.21.15.112:0/60718946 wait complete. 2024-09-21T00:28:40.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.124+0000 7f7f587b0640 1 Processor -- start 2024-09-21T00:28:40.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.124+0000 7f7f587b0640 1 -- start start 2024-09-21T00:28:40.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.125+0000 7f7f587b0640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7f500ccce0 0x7f7f50157390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:40.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.125+0000 7f7f587b0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f5015ca50 0x7f7f501598a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:40.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.125+0000 7f7f587b0640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7f50159de0 0x7f7f500b6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:40.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.125+0000 7f7f587b0640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7f500d2130 con 0x7f7f5015ca50 2024-09-21T00:28:40.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.125+0000 7f7f587b0640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7f500d1fb0 con 0x7f7f500ccce0 2024-09-21T00:28:40.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.125+0000 7f7f587b0640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f7f500c6b20 con 0x7f7f50159de0 2024-09-21T00:28:40.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.125+0000 7f7f55d4b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f5015ca50 0x7f7f501598a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:40.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.125+0000 7f7f55d4b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f5015ca50 0x7f7f501598a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33300/0 (socket says 172.21.15.112:33300) 2024-09-21T00:28:40.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.125+0000 7f7f55d4b640 1 -- 172.21.15.112:0/3565375543 learned_addr learned my addr 172.21.15.112:0/3565375543 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:28:40.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.125+0000 7f7f5654c640 1 --2- 172.21.15.112:0/3565375543 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7f500ccce0 0x7f7f50157390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:40.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.125+0000 7f7f56d4d640 1 --2- 172.21.15.112:0/3565375543 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7f50159de0 0x7f7f500b6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:40.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.126+0000 7f7f55d4b640 1 -- 172.21.15.112:0/3565375543 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7f50159de0 msgr2=0x7f7f500b6fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:40.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.126+0000 7f7f55d4b640 1 --2- 172.21.15.112:0/3565375543 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7f50159de0 0x7f7f500b6fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:40.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.126+0000 7f7f55d4b640 1 -- 172.21.15.112:0/3565375543 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7f500ccce0 msgr2=0x7f7f50157390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:40.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.126+0000 7f7f55d4b640 1 --2- 172.21.15.112:0/3565375543 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7f500ccce0 0x7f7f50157390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:40.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.126+0000 7f7f55d4b640 1 -- 172.21.15.112:0/3565375543 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f501f3050 con 0x7f7f5015ca50 2024-09-21T00:28:40.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.126+0000 7f7f55d4b640 1 --2- 172.21.15.112:0/3565375543 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f5015ca50 0x7f7f501598a0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f7f40009950 tx=0x7f7f40033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:40.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.126+0000 7f7f46ffd640 1 -- 172.21.15.112:0/3565375543 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f40031d60 con 0x7f7f5015ca50 2024-09-21T00:28:40.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.126+0000 7f7f46ffd640 1 -- 172.21.15.112:0/3565375543 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7f40033e10 con 0x7f7f5015ca50 2024-09-21T00:28:40.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.126+0000 7f7f46ffd640 1 -- 172.21.15.112:0/3565375543 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f40005280 con 0x7f7f5015ca50 2024-09-21T00:28:40.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.126+0000 7f7f587b0640 1 -- 172.21.15.112:0/3565375543 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f501f32c0 con 0x7f7f5015ca50 2024-09-21T00:28:40.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.126+0000 7f7f587b0640 1 -- 172.21.15.112:0/3565375543 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f501f3880 con 0x7f7f5015ca50 2024-09-21T00:28:40.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.127+0000 7f7f587b0640 1 -- 172.21.15.112:0/3565375543 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f5015a260 con 0x7f7f5015ca50 2024-09-21T00:28:40.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.128+0000 7f7f46ffd640 1 -- 172.21.15.112:0/3565375543 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f7f40005420 con 0x7f7f5015ca50 2024-09-21T00:28:40.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.128+0000 7f7f46ffd640 1 --2- 172.21.15.112:0/3565375543 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f7f34078b90 0x7f7f3407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:40.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.128+0000 7f7f46ffd640 1 -- 172.21.15.112:0/3565375543 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6101+0+0 (secure 0 0 0) 0x7f7f400bc0c0 con 0x7f7f5015ca50 2024-09-21T00:28:40.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.129+0000 7f7f5654c640 1 --2- 172.21.15.112:0/3565375543 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f7f34078b90 0x7f7f3407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:40.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.129+0000 7f7f5654c640 1 --2- 172.21.15.112:0/3565375543 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f7f34078b90 0x7f7f3407b0a0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c000f60 tx=0x7f7f4c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:40.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.133+0000 7f7f46ffd640 1 -- 172.21.15.112:0/3565375543 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f7f4003f030 con 0x7f7f5015ca50 2024-09-21T00:28:40.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.345+0000 7f7f587b0640 1 -- 172.21.15.112:0/3565375543 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7f500b41e0 con 0x7f7f5015ca50 2024-09-21T00:28:40.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.346+0000 7f7f46ffd640 1 -- 172.21.15.112:0/3565375543 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+5237 (secure 0 0 0) 0x7f7f40083db0 con 0x7f7f5015ca50 2024-09-21T00:28:40.347 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:28:40.348 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":47,"btime":"2024-09-21T00:28:33:951960+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5435,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6837/3039805497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6835","nonce":3039805497},{"type":"v1","addr":"172.21.15.187:6837","nonce":3039805497}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43},{"gid":5719,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6836/2736940636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":2736940636},{"type":"v1","addr":"172.21.15.187:6836","nonce":2736940636}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43},{"gid":6162,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6834/2295407428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":2295407428},{"type":"v1","addr":"172.21.15.112:6834","nonce":2295407428}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T00:28:30.298540+0000","modified":"2024-09-21T00:28:33.314195+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6165},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6165":{"gid":6165,"name":"a","rank":0,"incarnation":44,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6835/1624734538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":1624734538},{"type":"v1","addr":"172.21.15.112:6835","nonce":1624734538}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6165,"qdb_cluster":[6165]},"id":5}]} 2024-09-21T00:28:40.348 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 47 2024-09-21T00:28:40.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.348+0000 7f7f587b0640 1 -- 172.21.15.112:0/3565375543 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f7f34078b90 msgr2=0x7f7f3407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:40.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.348+0000 7f7f587b0640 1 --2- 172.21.15.112:0/3565375543 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f7f34078b90 0x7f7f3407b0a0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c000f60 tx=0x7f7f4c009040 comp rx=0 tx=0).stop 2024-09-21T00:28:40.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.348+0000 7f7f587b0640 1 -- 172.21.15.112:0/3565375543 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f5015ca50 msgr2=0x7f7f501598a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:40.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.348+0000 7f7f587b0640 1 --2- 172.21.15.112:0/3565375543 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f5015ca50 0x7f7f501598a0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f7f40009950 tx=0x7f7f40033c20 comp rx=0 tx=0).stop 2024-09-21T00:28:40.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.348+0000 7f7f587b0640 1 -- 172.21.15.112:0/3565375543 shutdown_connections 2024-09-21T00:28:40.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.348+0000 7f7f587b0640 1 --2- 172.21.15.112:0/3565375543 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7f50159de0 0x7f7f500b6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:40.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.348+0000 7f7f587b0640 1 --2- 172.21.15.112:0/3565375543 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f7f34078b90 0x7f7f3407b0a0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:40.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.348+0000 7f7f587b0640 1 --2- 172.21.15.112:0/3565375543 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f5015ca50 0x7f7f501598a0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:40.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.348+0000 7f7f587b0640 1 --2- 172.21.15.112:0/3565375543 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7f500ccce0 0x7f7f50157390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:40.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.348+0000 7f7f587b0640 1 -- 172.21.15.112:0/3565375543 >> 172.21.15.112:0/3565375543 conn(0x7f7f500cf220 msgr2=0x7f7f5015a820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:40.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.348+0000 7f7f587b0640 1 -- 172.21.15.112:0/3565375543 shutdown_connections 2024-09-21T00:28:40.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.348+0000 7f7f587b0640 1 -- 172.21.15.112:0/3565375543 wait complete. 2024-09-21T00:28:40.361 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T00:28:40.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.471+0000 7fb80619b640 1 -- 172.21.15.112:0/1910991961 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8000d3880 msgr2=0x7fb800163260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:40.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.471+0000 7fb80619b640 1 --2- 172.21.15.112:0/1910991961 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8000d3880 0x7fb800163260 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fb8000b7490 tx=0x7fb7f402a580 comp rx=0 tx=0).stop 2024-09-21T00:28:40.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.471+0000 7fb80619b640 1 -- 172.21.15.112:0/1910991961 shutdown_connections 2024-09-21T00:28:40.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.471+0000 7fb80619b640 1 --2- 172.21.15.112:0/1910991961 >> v2:172.21.15.187:3300/0 conn(0x7fb8000d01b0 0x7fb8000d05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:40.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.471+0000 7fb80619b640 1 --2- 172.21.15.112:0/1910991961 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8000d3880 0x7fb800163260 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:40.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.471+0000 7fb80619b640 1 --2- 172.21.15.112:0/1910991961 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb8000d48a0 0x7fb8000d3dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:40.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.471+0000 7fb80619b640 1 -- 172.21.15.112:0/1910991961 >> 172.21.15.112:0/1910991961 conn(0x7fb80008bfe0 msgr2=0x7fb80009d890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:40.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.471+0000 7fb80619b640 1 -- 172.21.15.112:0/1910991961 shutdown_connections 2024-09-21T00:28:40.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.471+0000 7fb80619b640 1 -- 172.21.15.112:0/1910991961 wait complete. 2024-09-21T00:28:40.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.473+0000 7fb80619b640 1 Processor -- start 2024-09-21T00:28:40.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.473+0000 7fb80619b640 1 -- start start 2024-09-21T00:28:40.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.474+0000 7fb80619b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb8000d01b0 0x7fb800160ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:40.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.474+0000 7fb80619b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb8000d3880 0x7fb800161400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:40.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.474+0000 7fb80619b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8000d48a0 0x7fb800150840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:40.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.474+0000 7fb80619b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb8000d2460 con 0x7fb8000d48a0 2024-09-21T00:28:40.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.474+0000 7fb80619b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb8000d22e0 con 0x7fb8000d01b0 2024-09-21T00:28:40.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.474+0000 7fb80619b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fb8000d25e0 con 0x7fb8000d3880 2024-09-21T00:28:40.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.474+0000 7fb7fffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8000d48a0 0x7fb800150840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:40.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.474+0000 7fb7fffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8000d48a0 0x7fb800150840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33316/0 (socket says 172.21.15.112:33316) 2024-09-21T00:28:40.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.474+0000 7fb7fffff640 1 -- 172.21.15.112:0/3967411937 learned_addr learned my addr 172.21.15.112:0/3967411937 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:28:40.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.474+0000 7fb7feffd640 1 --2- 172.21.15.112:0/3967411937 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb8000d3880 0x7fb800161400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:40.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.474+0000 7fb7ff7fe640 1 --2- 172.21.15.112:0/3967411937 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb8000d01b0 0x7fb800160ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:40.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.475+0000 7fb7fffff640 1 -- 172.21.15.112:0/3967411937 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb8000d3880 msgr2=0x7fb800161400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:40.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.475+0000 7fb7fffff640 1 --2- 172.21.15.112:0/3967411937 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb8000d3880 0x7fb800161400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:40.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.475+0000 7fb7fffff640 1 -- 172.21.15.112:0/3967411937 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb8000d01b0 msgr2=0x7fb800160ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:40.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.475+0000 7fb7fffff640 1 --2- 172.21.15.112:0/3967411937 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb8000d01b0 0x7fb800160ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:40.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.475+0000 7fb7fffff640 1 -- 172.21.15.112:0/3967411937 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb800160100 con 0x7fb8000d48a0 2024-09-21T00:28:40.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.475+0000 7fb7fffff640 1 --2- 172.21.15.112:0/3967411937 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8000d48a0 0x7fb800150840 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fb7f800ba10 tx=0x7fb7f800bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:40.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.475+0000 7fb7e7fff640 1 -- 172.21.15.112:0/3967411937 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7f8019070 con 0x7fb8000d48a0 2024-09-21T00:28:40.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.475+0000 7fb7e7fff640 1 -- 172.21.15.112:0/3967411937 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb7f800cc30 con 0x7fb8000d48a0 2024-09-21T00:28:40.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.475+0000 7fb80619b640 1 -- 172.21.15.112:0/3967411937 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8001603e0 con 0x7fb8000d48a0 2024-09-21T00:28:40.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.475+0000 7fb7e7fff640 1 -- 172.21.15.112:0/3967411937 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7f80146a0 con 0x7fb8000d48a0 2024-09-21T00:28:40.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.475+0000 7fb80619b640 1 -- 172.21.15.112:0/3967411937 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8000c9aa0 con 0x7fb8000d48a0 2024-09-21T00:28:40.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.476+0000 7fb80619b640 1 -- 172.21.15.112:0/3967411937 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8000c9d00 con 0x7fb8000d48a0 2024-09-21T00:28:40.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.477+0000 7fb7e7fff640 1 -- 172.21.15.112:0/3967411937 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb7f801c070 con 0x7fb8000d48a0 2024-09-21T00:28:40.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.477+0000 7fb7e7fff640 1 --2- 172.21.15.112:0/3967411937 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fb7ec078b90 0x7fb7ec07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:40.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.477+0000 7fb7e7fff640 1 -- 172.21.15.112:0/3967411937 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6101+0+0 (secure 0 0 0) 0x7fb7f809ab40 con 0x7fb8000d48a0 2024-09-21T00:28:40.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.477+0000 7fb7ff7fe640 1 --2- 172.21.15.112:0/3967411937 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fb7ec078b90 0x7fb7ec07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:40.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.478+0000 7fb7ff7fe640 1 --2- 172.21.15.112:0/3967411937 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fb7ec078b90 0x7fb7ec07b0a0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fb7f000b810 tx=0x7fb7f0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:40.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.482+0000 7fb7e7fff640 1 -- 172.21.15.112:0/3967411937 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb7f800ce20 con 0x7fb8000d48a0 2024-09-21T00:28:40.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.694+0000 7fb80619b640 1 -- 172.21.15.112:0/3967411937 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb8000d2b60 con 0x7fb8000d48a0 2024-09-21T00:28:40.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.695+0000 7fb7e7fff640 1 -- 172.21.15.112:0/3967411937 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+5237 (secure 0 0 0) 0x7fb7f8062e80 con 0x7fb8000d48a0 2024-09-21T00:28:40.697 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:28:40.697 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":47,"btime":"2024-09-21T00:28:33:951960+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5435,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6837/3039805497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6835","nonce":3039805497},{"type":"v1","addr":"172.21.15.187:6837","nonce":3039805497}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43},{"gid":5719,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6836/2736940636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":2736940636},{"type":"v1","addr":"172.21.15.187:6836","nonce":2736940636}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43},{"gid":6162,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6834/2295407428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":2295407428},{"type":"v1","addr":"172.21.15.112:6834","nonce":2295407428}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T00:28:30.298540+0000","modified":"2024-09-21T00:28:33.314195+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6165},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6165":{"gid":6165,"name":"a","rank":0,"incarnation":44,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6835/1624734538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":1624734538},{"type":"v1","addr":"172.21.15.112:6835","nonce":1624734538}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6165,"qdb_cluster":[6165]},"id":5}]} 2024-09-21T00:28:40.697 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 47 2024-09-21T00:28:40.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.697+0000 7fb80619b640 1 -- 172.21.15.112:0/3967411937 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fb7ec078b90 msgr2=0x7fb7ec07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:40.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.697+0000 7fb80619b640 1 --2- 172.21.15.112:0/3967411937 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fb7ec078b90 0x7fb7ec07b0a0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fb7f000b810 tx=0x7fb7f0008040 comp rx=0 tx=0).stop 2024-09-21T00:28:40.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.697+0000 7fb80619b640 1 -- 172.21.15.112:0/3967411937 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8000d48a0 msgr2=0x7fb800150840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:40.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.697+0000 7fb80619b640 1 --2- 172.21.15.112:0/3967411937 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8000d48a0 0x7fb800150840 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fb7f800ba10 tx=0x7fb7f800bed0 comp rx=0 tx=0).stop 2024-09-21T00:28:40.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.697+0000 7fb80619b640 1 -- 172.21.15.112:0/3967411937 shutdown_connections 2024-09-21T00:28:40.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.697+0000 7fb80619b640 1 --2- 172.21.15.112:0/3967411937 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fb7ec078b90 0x7fb7ec07b0a0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:40.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.697+0000 7fb80619b640 1 --2- 172.21.15.112:0/3967411937 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8000d48a0 0x7fb800150840 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:40.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.697+0000 7fb80619b640 1 --2- 172.21.15.112:0/3967411937 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb8000d3880 0x7fb800161400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:40.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.697+0000 7fb80619b640 1 --2- 172.21.15.112:0/3967411937 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb8000d01b0 0x7fb800160ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:40.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.697+0000 7fb80619b640 1 -- 172.21.15.112:0/3967411937 >> 172.21.15.112:0/3967411937 conn(0x7fb80008bfe0 msgr2=0x7fb8000b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:40.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.697+0000 7fb80619b640 1 -- 172.21.15.112:0/3967411937 shutdown_connections 2024-09-21T00:28:40.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.697+0000 7fb80619b640 1 -- 172.21.15.112:0/3967411937 wait complete. 2024-09-21T00:28:40.710 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T00:28:40.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.821+0000 7f38e8470640 1 -- 172.21.15.112:0/187198291 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38e0074da0 msgr2=0x7f38e015f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:40.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.821+0000 7f38e8470640 1 --2- 172.21.15.112:0/187198291 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38e0074da0 0x7f38e015f150 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f38e0074cc0 tx=0x7f38d002a4d0 comp rx=0 tx=0).stop 2024-09-21T00:28:40.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.821+0000 7f38e8470640 1 -- 172.21.15.112:0/187198291 shutdown_connections 2024-09-21T00:28:40.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.821+0000 7f38e8470640 1 --2- 172.21.15.112:0/187198291 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f38e015f690 0x7f38e015bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:40.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.821+0000 7f38e8470640 1 --2- 172.21.15.112:0/187198291 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38e0074da0 0x7f38e015f150 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:40.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.821+0000 7f38e8470640 1 --2- 172.21.15.112:0/187198291 >> v2:172.21.15.187:3300/0 conn(0x7f38e01526e0 0x7f38e00b4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:40.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.821+0000 7f38e8470640 1 -- 172.21.15.112:0/187198291 >> 172.21.15.112:0/187198291 conn(0x7f38e0073f00 msgr2=0x7f38e0074310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:40.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.821+0000 7f38e8470640 1 -- 172.21.15.112:0/187198291 shutdown_connections 2024-09-21T00:28:40.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.821+0000 7f38e8470640 1 -- 172.21.15.112:0/187198291 wait complete. 2024-09-21T00:28:40.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.823+0000 7f38e8470640 1 Processor -- start 2024-09-21T00:28:40.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.823+0000 7f38e8470640 1 -- start start 2024-09-21T00:28:40.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.824+0000 7f38e8470640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f38e0074da0 0x7f38e0150280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:40.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.824+0000 7f38e8470640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38e01526e0 0x7f38e0154960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:40.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.824+0000 7f38e8470640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f38e015f690 0x7f38e0157cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:40.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.824+0000 7f38e8470640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f38e015c3d0 con 0x7f38e01526e0 2024-09-21T00:28:40.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.824+0000 7f38e8470640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f38e015c250 con 0x7f38e015f690 2024-09-21T00:28:40.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.824+0000 7f38e8470640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f38e015c550 con 0x7f38e0074da0 2024-09-21T00:28:40.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.824+0000 7f38e5a0b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38e01526e0 0x7f38e0154960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:40.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.824+0000 7f38e5a0b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38e01526e0 0x7f38e0154960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33324/0 (socket says 172.21.15.112:33324) 2024-09-21T00:28:40.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.824+0000 7f38e5a0b640 1 -- 172.21.15.112:0/2592845949 learned_addr learned my addr 172.21.15.112:0/2592845949 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:28:40.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.824+0000 7f38e620c640 1 --2- 172.21.15.112:0/2592845949 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f38e0074da0 0x7f38e0150280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:40.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.824+0000 7f38e6a0d640 1 --2- 172.21.15.112:0/2592845949 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f38e015f690 0x7f38e0157cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:40.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.824+0000 7f38e5a0b640 1 -- 172.21.15.112:0/2592845949 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f38e0074da0 msgr2=0x7f38e0150280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:40.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.824+0000 7f38e5a0b640 1 --2- 172.21.15.112:0/2592845949 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f38e0074da0 0x7f38e0150280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:40.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.824+0000 7f38e5a0b640 1 -- 172.21.15.112:0/2592845949 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f38e015f690 msgr2=0x7f38e0157cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:40.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.824+0000 7f38e5a0b640 1 --2- 172.21.15.112:0/2592845949 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f38e015f690 0x7f38e0157cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:40.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.824+0000 7f38e5a0b640 1 -- 172.21.15.112:0/2592845949 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38e0156c30 con 0x7f38e01526e0 2024-09-21T00:28:40.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.824+0000 7f38e5a0b640 1 --2- 172.21.15.112:0/2592845949 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38e01526e0 0x7f38e0154960 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f38d0034040 tx=0x7f38d0003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:40.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.825+0000 7f38d6ffd640 1 -- 172.21.15.112:0/2592845949 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38d0035040 con 0x7f38e01526e0 2024-09-21T00:28:40.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.825+0000 7f38d6ffd640 1 -- 172.21.15.112:0/2592845949 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f38d00047a0 con 0x7f38e01526e0 2024-09-21T00:28:40.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.825+0000 7f38d6ffd640 1 -- 172.21.15.112:0/2592845949 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38d0004b70 con 0x7f38e01526e0 2024-09-21T00:28:40.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.825+0000 7f38e8470640 1 -- 172.21.15.112:0/2592845949 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38e0156f10 con 0x7f38e01526e0 2024-09-21T00:28:40.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.825+0000 7f38e8470640 1 -- 172.21.15.112:0/2592845949 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38e01574d0 con 0x7f38e01526e0 2024-09-21T00:28:40.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.826+0000 7f38e8470640 1 -- 172.21.15.112:0/2592845949 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38e0154f20 con 0x7f38e01526e0 2024-09-21T00:28:40.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.826+0000 7f38d6ffd640 1 -- 172.21.15.112:0/2592845949 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f38d0004d10 con 0x7f38e01526e0 2024-09-21T00:28:40.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.827+0000 7f38d6ffd640 1 --2- 172.21.15.112:0/2592845949 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f38c4078bc0 0x7f38c407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:40.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.827+0000 7f38d6ffd640 1 -- 172.21.15.112:0/2592845949 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6101+0+0 (secure 0 0 0) 0x7f38d00bd6e0 con 0x7f38e01526e0 2024-09-21T00:28:40.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.827+0000 7f38e620c640 1 --2- 172.21.15.112:0/2592845949 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f38c4078bc0 0x7f38c407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:40.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.828+0000 7f38e620c640 1 --2- 172.21.15.112:0/2592845949 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f38c4078bc0 0x7f38c407b0d0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f38e00712a0 tx=0x7f38dc009550 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:40.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:40.831+0000 7f38d6ffd640 1 -- 172.21.15.112:0/2592845949 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f38d0085340 con 0x7f38e01526e0 2024-09-21T00:28:41.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.010+0000 7f38e8470640 1 -- 172.21.15.112:0/2592845949 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f38e00b4c50 con 0x7f38e01526e0 2024-09-21T00:28:41.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.011+0000 7f38d6ffd640 1 -- 172.21.15.112:0/2592845949 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) ==== 74+0+17029 (secure 0 0 0) 0x7f38d0089d70 con 0x7f38e01526e0 2024-09-21T00:28:41.013 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:28:41.013 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":71,"fsid":"8b34692b-dc65-49b2-a08f-26f707b96ac5","created":"2024-09-21T00:25:22.788490+0000","modified":"2024-09-21T00:28:31.940059+0000","last_up_change":"2024-09-21T00:25:36.161835+0000","last_in_change":"2024-09-21T00:25:26.346500+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T00:25:37.105006+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-09-21T00:28:27.930288+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-09-21T00:28:29.255565+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9c575086-67d2-4de2-b01b-d44dcc21d604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6809","nonce":755760621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6811","nonce":755760621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6815","nonce":755760621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6813","nonce":755760621}]},"public_addr":"172.21.15.112:6809/755760621","cluster_addr":"172.21.15.112:6811/755760621","heartbeat_back_addr":"172.21.15.112:6815/755760621","heartbeat_front_addr":"172.21.15.112:6813/755760621","state":["exists","up"]},{"osd":1,"uuid":"a983c227-a3ce-4bdd-82f1-438ba7e3ecd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6801","nonce":344932592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6803","nonce":344932592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6807","nonce":344932592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6805","nonce":344932592}]},"public_addr":"172.21.15.112:6801/344932592","cluster_addr":"172.21.15.112:6803/344932592","heartbeat_back_addr":"172.21.15.112:6807/344932592","heartbeat_front_addr":"172.21.15.112:6805/344932592","state":["exists","up"]},{"osd":2,"uuid":"1193c1d8-bfff-4a57-8200-5f69bf1ea3af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6825","nonce":407933616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6827","nonce":407933616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6831","nonce":407933616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6829","nonce":407933616}]},"public_addr":"172.21.15.112:6825/407933616","cluster_addr":"172.21.15.112:6827/407933616","heartbeat_back_addr":"172.21.15.112:6831/407933616","heartbeat_front_addr":"172.21.15.112:6829/407933616","state":["exists","up"]},{"osd":3,"uuid":"af04330a-36ad-40cc-a8dd-fa259b221de5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6817","nonce":4120164806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6819","nonce":4120164806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6823","nonce":4120164806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6821","nonce":4120164806}]},"public_addr":"172.21.15.112:6817/4120164806","cluster_addr":"172.21.15.112:6819/4120164806","heartbeat_back_addr":"172.21.15.112:6823/4120164806","heartbeat_front_addr":"172.21.15.112:6821/4120164806","state":["exists","up"]},{"osd":4,"uuid":"5b893111-292c-4c5a-b184-53ad38c7199e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6803","nonce":522077560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6805","nonce":522077560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6809","nonce":522077560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6807","nonce":522077560}]},"public_addr":"172.21.15.187:6803/522077560","cluster_addr":"172.21.15.187:6805/522077560","heartbeat_back_addr":"172.21.15.187:6809/522077560","heartbeat_front_addr":"172.21.15.187:6807/522077560","state":["exists","up"]},{"osd":5,"uuid":"c0b8e011-45fe-4c5e-925e-6be2a7f1868a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6827","nonce":3593144721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6829","nonce":3593144721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6833","nonce":3593144721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6831","nonce":3593144721}]},"public_addr":"172.21.15.187:6827/3593144721","cluster_addr":"172.21.15.187:6829/3593144721","heartbeat_back_addr":"172.21.15.187:6833/3593144721","heartbeat_front_addr":"172.21.15.187:6831/3593144721","state":["exists","up"]},{"osd":6,"uuid":"87d2d650-7489-4593-98a3-a1f433ce8dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6811","nonce":1494387657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6813","nonce":1494387657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6817","nonce":1494387657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6815","nonce":1494387657}]},"public_addr":"172.21.15.187:6811/1494387657","cluster_addr":"172.21.15.187:6813/1494387657","heartbeat_back_addr":"172.21.15.187:6817/1494387657","heartbeat_front_addr":"172.21.15.187:6815/1494387657","state":["exists","up"]},{"osd":7,"uuid":"c6b4f8e2-a1ee-4438-aa37-67ff4b6ef1f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6819","nonce":1624699842}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6821","nonce":1624699842}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6825","nonce":1624699842}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6823","nonce":1624699842}]},"public_addr":"172.21.15.187:6819/1624699842","cluster_addr":"172.21.15.187:6821/1624699842","heartbeat_back_addr":"172.21.15.187:6825/1624699842","heartbeat_front_addr":"172.21.15.187:6823/1624699842","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.873162+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.400506+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.664317+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.159791+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.202630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.113212+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.384786+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.448441+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T00:28:41.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.013+0000 7f38e8470640 1 -- 172.21.15.112:0/2592845949 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f38c4078bc0 msgr2=0x7f38c407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:41.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.013+0000 7f38e8470640 1 --2- 172.21.15.112:0/2592845949 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f38c4078bc0 0x7f38c407b0d0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f38e00712a0 tx=0x7f38dc009550 comp rx=0 tx=0).stop 2024-09-21T00:28:41.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.013+0000 7f38e8470640 1 -- 172.21.15.112:0/2592845949 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38e01526e0 msgr2=0x7f38e0154960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:41.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.013+0000 7f38e8470640 1 --2- 172.21.15.112:0/2592845949 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38e01526e0 0x7f38e0154960 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f38d0034040 tx=0x7f38d0003910 comp rx=0 tx=0).stop 2024-09-21T00:28:41.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.013+0000 7f38e8470640 1 -- 172.21.15.112:0/2592845949 shutdown_connections 2024-09-21T00:28:41.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.013+0000 7f38e8470640 1 --2- 172.21.15.112:0/2592845949 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f38e015f690 0x7f38e0157cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:41.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.013+0000 7f38e8470640 1 --2- 172.21.15.112:0/2592845949 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f38c4078bc0 0x7f38c407b0d0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:41.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.013+0000 7f38e8470640 1 --2- 172.21.15.112:0/2592845949 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38e01526e0 0x7f38e0154960 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:41.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.013+0000 7f38e8470640 1 --2- 172.21.15.112:0/2592845949 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f38e0074da0 0x7f38e0150280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:41.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.013+0000 7f38e8470640 1 -- 172.21.15.112:0/2592845949 >> 172.21.15.112:0/2592845949 conn(0x7f38e0073f00 msgr2=0x7f38e0068d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:41.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.013+0000 7f38e8470640 1 -- 172.21.15.112:0/2592845949 shutdown_connections 2024-09-21T00:28:41.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.013+0000 7f38e8470640 1 -- 172.21.15.112:0/2592845949 wait complete. 2024-09-21T00:28:41.027 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-21T00:28:41.027 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T00:28:41.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.137+0000 7f8ebd094640 1 -- 172.21.15.112:0/3122105461 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8eb815d2e0 msgr2=0x7f8eb80714e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:41.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.137+0000 7f8ebd094640 1 --2- 172.21.15.112:0/3122105461 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8eb815d2e0 0x7f8eb80714e0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f8eb8073e90 tx=0x7f8eb002a440 comp rx=0 tx=0).stop 2024-09-21T00:28:41.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.137+0000 7f8ebd094640 1 -- 172.21.15.112:0/3122105461 shutdown_connections 2024-09-21T00:28:41.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.137+0000 7f8ebd094640 1 --2- 172.21.15.112:0/3122105461 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8eb815d2e0 0x7f8eb80714e0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:41.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.137+0000 7f8ebd094640 1 --2- 172.21.15.112:0/3122105461 >> v2:172.21.15.187:3300/0 conn(0x7f8eb815ced0 0x7f8eb806d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:41.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.137+0000 7f8ebd094640 1 -- 172.21.15.112:0/3122105461 >> 172.21.15.112:0/3122105461 conn(0x7f8eb80cdd30 msgr2=0x7f8eb80ce140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:41.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.137+0000 7f8ebd094640 1 -- 172.21.15.112:0/3122105461 shutdown_connections 2024-09-21T00:28:41.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.137+0000 7f8ebd094640 1 -- 172.21.15.112:0/3122105461 wait complete. 2024-09-21T00:28:41.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.139+0000 7f8ebd094640 1 Processor -- start 2024-09-21T00:28:41.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.139+0000 7f8ebd094640 1 -- start start 2024-09-21T00:28:41.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.140+0000 7f8ebd094640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8eb80721a0 0x7f8eb8141890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:41.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.140+0000 7f8ebd094640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8eb815ced0 0x7f8eb8157650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:41.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.140+0000 7f8ebd094640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8eb815d2e0 0x7f8eb81551f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:41.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.140+0000 7f8ebd094640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb80cfbd0 con 0x7f8eb80721a0 2024-09-21T00:28:41.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.140+0000 7f8ebd094640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb80cfa50 con 0x7f8eb815ced0 2024-09-21T00:28:41.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.140+0000 7f8ebd094640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f8eb80cfd50 con 0x7f8eb815d2e0 2024-09-21T00:28:41.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.141+0000 7f8eb6d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8eb80721a0 0x7f8eb8141890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:41.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.141+0000 7f8eb6d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8eb80721a0 0x7f8eb8141890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33330/0 (socket says 172.21.15.112:33330) 2024-09-21T00:28:41.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.141+0000 7f8eb6d9d640 1 -- 172.21.15.112:0/3117497539 learned_addr learned my addr 172.21.15.112:0/3117497539 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:28:41.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.141+0000 7f8eb659c640 1 --2- 172.21.15.112:0/3117497539 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8eb815ced0 0x7f8eb8157650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:41.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.141+0000 7f8eb6d9d640 1 -- 172.21.15.112:0/3117497539 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8eb815d2e0 msgr2=0x7f8eb81551f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:41.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.141+0000 7f8eb759e640 1 --2- 172.21.15.112:0/3117497539 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8eb815d2e0 0x7f8eb81551f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:41.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.141+0000 7f8eb6d9d640 1 --2- 172.21.15.112:0/3117497539 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8eb815d2e0 0x7f8eb81551f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:41.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.141+0000 7f8eb6d9d640 1 -- 172.21.15.112:0/3117497539 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8eb815ced0 msgr2=0x7f8eb8157650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:41.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.141+0000 7f8eb6d9d640 1 --2- 172.21.15.112:0/3117497539 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8eb815ced0 0x7f8eb8157650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:41.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.141+0000 7f8eb6d9d640 1 -- 172.21.15.112:0/3117497539 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8eb8158380 con 0x7f8eb80721a0 2024-09-21T00:28:41.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.141+0000 7f8eb6d9d640 1 --2- 172.21.15.112:0/3117497539 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8eb80721a0 0x7f8eb8141890 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f8ea800ba80 tx=0x7f8ea800bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:41.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.141+0000 7f8e9f7fe640 1 -- 172.21.15.112:0/3117497539 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ea800cc00 con 0x7f8eb80721a0 2024-09-21T00:28:41.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.141+0000 7f8e9f7fe640 1 -- 172.21.15.112:0/3117497539 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8ea8003970 con 0x7f8eb80721a0 2024-09-21T00:28:41.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.141+0000 7f8ebd094640 1 -- 172.21.15.112:0/3117497539 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8eb8142190 con 0x7f8eb80721a0 2024-09-21T00:28:41.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.141+0000 7f8e9f7fe640 1 -- 172.21.15.112:0/3117497539 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ea8003e00 con 0x7f8eb80721a0 2024-09-21T00:28:41.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.141+0000 7f8ebd094640 1 -- 172.21.15.112:0/3117497539 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8eb8142750 con 0x7f8eb80721a0 2024-09-21T00:28:41.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.142+0000 7f8ebd094640 1 -- 172.21.15.112:0/3117497539 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8eb80d0270 con 0x7f8eb80721a0 2024-09-21T00:28:41.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.143+0000 7f8e9f7fe640 1 -- 172.21.15.112:0/3117497539 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8ea8004000 con 0x7f8eb80721a0 2024-09-21T00:28:41.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.143+0000 7f8e9f7fe640 1 --2- 172.21.15.112:0/3117497539 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f8ea4078b90 0x7f8ea407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:41.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.144+0000 7f8e9f7fe640 1 -- 172.21.15.112:0/3117497539 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6101+0+0 (secure 0 0 0) 0x7f8ea809adf0 con 0x7f8eb80721a0 2024-09-21T00:28:41.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.144+0000 7f8eb659c640 1 --2- 172.21.15.112:0/3117497539 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f8ea4078b90 0x7f8ea407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:41.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.144+0000 7f8eb659c640 1 --2- 172.21.15.112:0/3117497539 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f8ea4078b90 0x7f8ea407b0a0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f8eac005fd0 tx=0x7f8eac005850 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:41.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.148+0000 7f8e9f7fe640 1 -- 172.21.15.112:0/3117497539 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f8ea8063130 con 0x7f8eb80721a0 2024-09-21T00:28:41.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.363+0000 7f8ebd094640 1 -- 172.21.15.112:0/3117497539 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8eb8156670 con 0x7f8eb80721a0 2024-09-21T00:28:41.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.363+0000 7f8e9f7fe640 1 -- 172.21.15.112:0/3117497539 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+5237 (secure 0 0 0) 0x7f8ea8062a30 con 0x7f8eb80721a0 2024-09-21T00:28:41.365 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:28:41.365 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":47,"btime":"2024-09-21T00:28:33:951960+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5435,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6837/3039805497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6835","nonce":3039805497},{"type":"v1","addr":"172.21.15.187:6837","nonce":3039805497}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43},{"gid":5719,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6836/2736940636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":2736940636},{"type":"v1","addr":"172.21.15.187:6836","nonce":2736940636}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43},{"gid":6162,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6834/2295407428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":2295407428},{"type":"v1","addr":"172.21.15.112:6834","nonce":2295407428}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T00:28:30.298540+0000","modified":"2024-09-21T00:28:33.314195+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6165},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6165":{"gid":6165,"name":"a","rank":0,"incarnation":44,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6835/1624734538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":1624734538},{"type":"v1","addr":"172.21.15.112:6835","nonce":1624734538}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6165,"qdb_cluster":[6165]},"id":5}]} 2024-09-21T00:28:41.365 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 47 2024-09-21T00:28:41.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.366+0000 7f8ebd094640 1 -- 172.21.15.112:0/3117497539 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f8ea4078b90 msgr2=0x7f8ea407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:41.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.366+0000 7f8ebd094640 1 --2- 172.21.15.112:0/3117497539 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f8ea4078b90 0x7f8ea407b0a0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f8eac005fd0 tx=0x7f8eac005850 comp rx=0 tx=0).stop 2024-09-21T00:28:41.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.366+0000 7f8ebd094640 1 -- 172.21.15.112:0/3117497539 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8eb80721a0 msgr2=0x7f8eb8141890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:41.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.366+0000 7f8ebd094640 1 --2- 172.21.15.112:0/3117497539 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8eb80721a0 0x7f8eb8141890 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f8ea800ba80 tx=0x7f8ea800bf40 comp rx=0 tx=0).stop 2024-09-21T00:28:41.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.366+0000 7f8ebd094640 1 -- 172.21.15.112:0/3117497539 shutdown_connections 2024-09-21T00:28:41.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.366+0000 7f8ebd094640 1 --2- 172.21.15.112:0/3117497539 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8eb815d2e0 0x7f8eb81551f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:41.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.366+0000 7f8ebd094640 1 --2- 172.21.15.112:0/3117497539 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8eb815ced0 0x7f8eb8157650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:41.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.366+0000 7f8ebd094640 1 --2- 172.21.15.112:0/3117497539 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f8ea4078b90 0x7f8ea407b0a0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:41.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.366+0000 7f8ebd094640 1 --2- 172.21.15.112:0/3117497539 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8eb80721a0 0x7f8eb8141890 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:41.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.366+0000 7f8ebd094640 1 -- 172.21.15.112:0/3117497539 >> 172.21.15.112:0/3117497539 conn(0x7f8eb80cdd30 msgr2=0x7f8eb80d0480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:41.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.366+0000 7f8ebd094640 1 -- 172.21.15.112:0/3117497539 shutdown_connections 2024-09-21T00:28:41.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.366+0000 7f8ebd094640 1 -- 172.21.15.112:0/3117497539 wait complete. 2024-09-21T00:28:41.379 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T00:28:41.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.489+0000 7f2c0a320640 1 -- 172.21.15.112:0/1355206131 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2c040ccbf0 msgr2=0x7f2c040cd000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:41.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.489+0000 7f2c0a320640 1 --2- 172.21.15.112:0/1355206131 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2c040ccbf0 0x7f2c040cd000 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f2bf4005fd0 tx=0x7f2bf402a410 comp rx=0 tx=0).stop 2024-09-21T00:28:41.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.489+0000 7f2c0a320640 1 -- 172.21.15.112:0/1355206131 shutdown_connections 2024-09-21T00:28:41.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.489+0000 7f2c0a320640 1 --2- 172.21.15.112:0/1355206131 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2c040c6a00 0x7f2c040b2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:41.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.489+0000 7f2c0a320640 1 --2- 172.21.15.112:0/1355206131 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2c040ccbf0 0x7f2c040cd000 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:41.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.489+0000 7f2c0a320640 1 -- 172.21.15.112:0/1355206131 >> 172.21.15.112:0/1355206131 conn(0x7f2c040b42e0 msgr2=0x7f2c040b4720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:41.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.489+0000 7f2c0a320640 1 -- 172.21.15.112:0/1355206131 shutdown_connections 2024-09-21T00:28:41.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.489+0000 7f2c0a320640 1 -- 172.21.15.112:0/1355206131 wait complete. 2024-09-21T00:28:41.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.491+0000 7f2c0a320640 1 Processor -- start 2024-09-21T00:28:41.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.491+0000 7f2c0a320640 1 -- start start 2024-09-21T00:28:41.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.492+0000 7f2c0a320640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2c040c5ee0 0x7f2c04147310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:41.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.492+0000 7f2c0a320640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2c040c6a00 0x7f2c04147850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:41.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.492+0000 7f2c0a320640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2c040ccbf0 0x7f2c04145980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:41.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.492+0000 7f2c0a320640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2c040b3610 con 0x7f2c040c6a00 2024-09-21T00:28:41.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.492+0000 7f2c0a320640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f2c040b3490 con 0x7f2c040ccbf0 2024-09-21T00:28:41.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.492+0000 7f2c0a320640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f2c040b3790 con 0x7f2c040c5ee0 2024-09-21T00:28:41.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.492+0000 7f2c037fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2c040c6a00 0x7f2c04147850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:41.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.492+0000 7f2c037fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2c040c6a00 0x7f2c04147850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33348/0 (socket says 172.21.15.112:33348) 2024-09-21T00:28:41.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.492+0000 7f2c037fe640 1 -- 172.21.15.112:0/3075347383 learned_addr learned my addr 172.21.15.112:0/3075347383 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:28:41.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.492+0000 7f2c03fff640 1 --2- 172.21.15.112:0/3075347383 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2c040c5ee0 0x7f2c04147310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:41.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.492+0000 7f2c088bd640 1 --2- 172.21.15.112:0/3075347383 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2c040ccbf0 0x7f2c04145980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:41.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.493+0000 7f2c037fe640 1 -- 172.21.15.112:0/3075347383 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2c040c5ee0 msgr2=0x7f2c04147310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:41.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.493+0000 7f2c037fe640 1 --2- 172.21.15.112:0/3075347383 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2c040c5ee0 0x7f2c04147310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:41.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.493+0000 7f2c037fe640 1 -- 172.21.15.112:0/3075347383 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2c040ccbf0 msgr2=0x7f2c04145980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:41.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.493+0000 7f2c037fe640 1 --2- 172.21.15.112:0/3075347383 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2c040ccbf0 0x7f2c04145980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:41.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.493+0000 7f2c037fe640 1 -- 172.21.15.112:0/3075347383 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c04159c50 con 0x7f2c040c6a00 2024-09-21T00:28:41.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.493+0000 7f2c037fe640 1 --2- 172.21.15.112:0/3075347383 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2c040c6a00 0x7f2c04147850 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f2bf800b3f0 tx=0x7f2bf800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:41.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.493+0000 7f2c00ff9640 1 -- 172.21.15.112:0/3075347383 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bf8015020 con 0x7f2c040c6a00 2024-09-21T00:28:41.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.493+0000 7f2c00ff9640 1 -- 172.21.15.112:0/3075347383 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2bf8011920 con 0x7f2c040c6a00 2024-09-21T00:28:41.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.493+0000 7f2c00ff9640 1 -- 172.21.15.112:0/3075347383 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bf8011d70 con 0x7f2c040c6a00 2024-09-21T00:28:41.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.493+0000 7f2c0a320640 1 -- 172.21.15.112:0/3075347383 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c040b6a00 con 0x7f2c040c6a00 2024-09-21T00:28:41.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.493+0000 7f2c0a320640 1 -- 172.21.15.112:0/3075347383 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c040b6fc0 con 0x7f2c040c6a00 2024-09-21T00:28:41.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.494+0000 7f2c0a320640 1 -- 172.21.15.112:0/3075347383 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c040cd8b0 con 0x7f2c040c6a00 2024-09-21T00:28:41.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.495+0000 7f2c00ff9640 1 -- 172.21.15.112:0/3075347383 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2bf8018020 con 0x7f2c040c6a00 2024-09-21T00:28:41.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.495+0000 7f2c00ff9640 1 --2- 172.21.15.112:0/3075347383 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2bf0078b90 0x7f2bf007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:41.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.496+0000 7f2c00ff9640 1 -- 172.21.15.112:0/3075347383 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6101+0+0 (secure 0 0 0) 0x7f2bf809a990 con 0x7f2c040c6a00 2024-09-21T00:28:41.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.496+0000 7f2c03fff640 1 --2- 172.21.15.112:0/3075347383 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2bf0078b90 0x7f2bf007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:41.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.496+0000 7f2c03fff640 1 --2- 172.21.15.112:0/3075347383 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2bf0078b90 0x7f2bf007b0a0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f2bf4033000 tx=0x7f2bf4004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:41.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.500+0000 7f2c00ff9640 1 -- 172.21.15.112:0/3075347383 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2c040cd8b0 con 0x7f2c040c6a00 2024-09-21T00:28:41.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.712+0000 7f2c0a320640 1 -- 172.21.15.112:0/3075347383 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2c04071ba0 con 0x7f2c040c6a00 2024-09-21T00:28:41.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.712+0000 7f2c00ff9640 1 -- 172.21.15.112:0/3075347383 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+5237 (secure 0 0 0) 0x7f2c04071ba0 con 0x7f2c040c6a00 2024-09-21T00:28:41.714 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:28:41.714 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":47,"btime":"2024-09-21T00:28:33:951960+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5435,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6837/3039805497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6835","nonce":3039805497},{"type":"v1","addr":"172.21.15.187:6837","nonce":3039805497}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43},{"gid":5719,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6836/2736940636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":2736940636},{"type":"v1","addr":"172.21.15.187:6836","nonce":2736940636}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43},{"gid":6162,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6834/2295407428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":2295407428},{"type":"v1","addr":"172.21.15.112:6834","nonce":2295407428}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T00:28:30.298540+0000","modified":"2024-09-21T00:28:33.314195+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6165},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6165":{"gid":6165,"name":"a","rank":0,"incarnation":44,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6835/1624734538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":1624734538},{"type":"v1","addr":"172.21.15.112:6835","nonce":1624734538}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6165,"qdb_cluster":[6165]},"id":5}]} 2024-09-21T00:28:41.715 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 47 2024-09-21T00:28:41.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.714+0000 7f2c0a320640 1 -- 172.21.15.112:0/3075347383 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2bf0078b90 msgr2=0x7f2bf007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:41.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.714+0000 7f2c0a320640 1 --2- 172.21.15.112:0/3075347383 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2bf0078b90 0x7f2bf007b0a0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f2bf4033000 tx=0x7f2bf4004730 comp rx=0 tx=0).stop 2024-09-21T00:28:41.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.715+0000 7f2c0a320640 1 -- 172.21.15.112:0/3075347383 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2c040c6a00 msgr2=0x7f2c04147850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:41.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.715+0000 7f2c0a320640 1 --2- 172.21.15.112:0/3075347383 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2c040c6a00 0x7f2c04147850 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f2bf800b3f0 tx=0x7f2bf800b8b0 comp rx=0 tx=0).stop 2024-09-21T00:28:41.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.715+0000 7f2c0a320640 1 -- 172.21.15.112:0/3075347383 shutdown_connections 2024-09-21T00:28:41.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.715+0000 7f2c0a320640 1 --2- 172.21.15.112:0/3075347383 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2c040ccbf0 0x7f2c04145980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:41.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.715+0000 7f2c0a320640 1 --2- 172.21.15.112:0/3075347383 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2bf0078b90 0x7f2bf007b0a0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:41.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.715+0000 7f2c0a320640 1 --2- 172.21.15.112:0/3075347383 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2c040c6a00 0x7f2c04147850 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:41.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.715+0000 7f2c0a320640 1 --2- 172.21.15.112:0/3075347383 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2c040c5ee0 0x7f2c04147310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:41.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.715+0000 7f2c0a320640 1 -- 172.21.15.112:0/3075347383 >> 172.21.15.112:0/3075347383 conn(0x7f2c040b42e0 msgr2=0x7f2c0415b490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:41.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.715+0000 7f2c0a320640 1 -- 172.21.15.112:0/3075347383 shutdown_connections 2024-09-21T00:28:41.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.715+0000 7f2c0a320640 1 -- 172.21.15.112:0/3075347383 wait complete. 2024-09-21T00:28:41.728 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T00:28:41.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.838+0000 7fb7f3dc6640 1 -- 172.21.15.112:0/4206097244 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7ec164620 msgr2=0x7fb7ec166a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:41.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.838+0000 7fb7f3dc6640 1 --2- 172.21.15.112:0/4206097244 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7ec164620 0x7fb7ec166a40 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc009be0 tx=0x7fb7dc02a8d0 comp rx=0 tx=0).stop 2024-09-21T00:28:41.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.838+0000 7fb7f3dc6640 1 -- 172.21.15.112:0/4206097244 shutdown_connections 2024-09-21T00:28:41.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.838+0000 7fb7f3dc6640 1 --2- 172.21.15.112:0/4206097244 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7ec164620 0x7fb7ec166a40 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:41.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.838+0000 7fb7f3dc6640 1 -- 172.21.15.112:0/4206097244 >> 172.21.15.112:0/4206097244 conn(0x7fb7ec0cd5c0 msgr2=0x7fb7ec0cd9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:41.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.838+0000 7fb7f3dc6640 1 -- 172.21.15.112:0/4206097244 shutdown_connections 2024-09-21T00:28:41.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.838+0000 7fb7f3dc6640 1 -- 172.21.15.112:0/4206097244 wait complete. 2024-09-21T00:28:41.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.840+0000 7fb7f3dc6640 1 Processor -- start 2024-09-21T00:28:41.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.840+0000 7fb7f3dc6640 1 -- start start 2024-09-21T00:28:41.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.841+0000 7fb7f3dc6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7ec070440 0x7fb7ec148680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:41.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.841+0000 7fb7f3dc6640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb7ec0cea40 0x7fb7ec1f36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:41.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.841+0000 7fb7f3dc6640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb7ec164620 0x7fb7ec1665f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:41.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.841+0000 7fb7f3dc6640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb7ec0cf6e0 con 0x7fb7ec070440 2024-09-21T00:28:41.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.841+0000 7fb7f3dc6640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb7ec0cf560 con 0x7fb7ec0cea40 2024-09-21T00:28:41.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.841+0000 7fb7f3dc6640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fb7ec166f80 con 0x7fb7ec164620 2024-09-21T00:28:41.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.841+0000 7fb7f1b62640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7ec070440 0x7fb7ec148680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:41.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.841+0000 7fb7f1b62640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7ec070440 0x7fb7ec148680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33370/0 (socket says 172.21.15.112:33370) 2024-09-21T00:28:41.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.841+0000 7fb7f1b62640 1 -- 172.21.15.112:0/4181202273 learned_addr learned my addr 172.21.15.112:0/4181202273 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:28:41.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.841+0000 7fb7f1b62640 1 -- 172.21.15.112:0/4181202273 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb7ec164620 msgr2=0x7fb7ec1665f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:41.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.841+0000 7fb7f1b62640 1 --2- 172.21.15.112:0/4181202273 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb7ec164620 0x7fb7ec1665f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:41.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.841+0000 7fb7f1b62640 1 -- 172.21.15.112:0/4181202273 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb7ec0cea40 msgr2=0x7fb7ec1f36c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:41.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.841+0000 7fb7f1b62640 1 --2- 172.21.15.112:0/4181202273 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb7ec0cea40 0x7fb7ec1f36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:41.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.841+0000 7fb7f1b62640 1 -- 172.21.15.112:0/4181202273 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7ec1f3c00 con 0x7fb7ec070440 2024-09-21T00:28:41.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.841+0000 7fb7f1b62640 1 --2- 172.21.15.112:0/4181202273 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7ec070440 0x7fb7ec148680 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fb7e400d9c0 tx=0x7fb7e400de80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:41.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.842+0000 7fb7ea7fc640 1 -- 172.21.15.112:0/4181202273 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7e400ed40 con 0x7fb7ec070440 2024-09-21T00:28:41.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.842+0000 7fb7ea7fc640 1 -- 172.21.15.112:0/4181202273 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb7e4004020 con 0x7fb7ec070440 2024-09-21T00:28:41.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.842+0000 7fb7ea7fc640 1 -- 172.21.15.112:0/4181202273 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7e4004470 con 0x7fb7ec070440 2024-09-21T00:28:41.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.842+0000 7fb7f3dc6640 1 -- 172.21.15.112:0/4181202273 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7ec165550 con 0x7fb7ec070440 2024-09-21T00:28:41.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.842+0000 7fb7f3dc6640 1 -- 172.21.15.112:0/4181202273 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7ec165b10 con 0x7fb7ec070440 2024-09-21T00:28:41.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.843+0000 7fb7f3dc6640 1 -- 172.21.15.112:0/4181202273 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7ec155dd0 con 0x7fb7ec070440 2024-09-21T00:28:41.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.843+0000 7fb7ea7fc640 1 -- 172.21.15.112:0/4181202273 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb7e40035f0 con 0x7fb7ec070440 2024-09-21T00:28:41.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.844+0000 7fb7ea7fc640 1 --2- 172.21.15.112:0/4181202273 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fb7d8078b90 0x7fb7d807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:41.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.844+0000 7fb7ea7fc640 1 -- 172.21.15.112:0/4181202273 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6101+0+0 (secure 0 0 0) 0x7fb7e409ce30 con 0x7fb7ec070440 2024-09-21T00:28:41.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.844+0000 7fb7f1361640 1 --2- 172.21.15.112:0/4181202273 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fb7d8078b90 0x7fb7d807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:41.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.845+0000 7fb7f1361640 1 --2- 172.21.15.112:0/4181202273 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fb7d8078b90 0x7fb7d807b0a0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec0b1b60 tx=0x7fb7e0007a30 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:41.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:41.849+0000 7fb7ea7fc640 1 -- 172.21.15.112:0/4181202273 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb7e40650e0 con 0x7fb7ec070440 2024-09-21T00:28:42.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:42.028+0000 7fb7f3dc6640 1 -- 172.21.15.112:0/4181202273 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb7ec0761c0 con 0x7fb7ec070440 2024-09-21T00:28:42.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:42.029+0000 7fb7ea7fc640 1 -- 172.21.15.112:0/4181202273 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) ==== 74+0+17029 (secure 0 0 0) 0x7fb7e40649e0 con 0x7fb7ec070440 2024-09-21T00:28:42.031 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:28:42.031 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":71,"fsid":"8b34692b-dc65-49b2-a08f-26f707b96ac5","created":"2024-09-21T00:25:22.788490+0000","modified":"2024-09-21T00:28:31.940059+0000","last_up_change":"2024-09-21T00:25:36.161835+0000","last_in_change":"2024-09-21T00:25:26.346500+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T00:25:37.105006+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-09-21T00:28:27.930288+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-09-21T00:28:29.255565+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9c575086-67d2-4de2-b01b-d44dcc21d604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6809","nonce":755760621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6811","nonce":755760621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6815","nonce":755760621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6813","nonce":755760621}]},"public_addr":"172.21.15.112:6809/755760621","cluster_addr":"172.21.15.112:6811/755760621","heartbeat_back_addr":"172.21.15.112:6815/755760621","heartbeat_front_addr":"172.21.15.112:6813/755760621","state":["exists","up"]},{"osd":1,"uuid":"a983c227-a3ce-4bdd-82f1-438ba7e3ecd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6801","nonce":344932592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6803","nonce":344932592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6807","nonce":344932592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6805","nonce":344932592}]},"public_addr":"172.21.15.112:6801/344932592","cluster_addr":"172.21.15.112:6803/344932592","heartbeat_back_addr":"172.21.15.112:6807/344932592","heartbeat_front_addr":"172.21.15.112:6805/344932592","state":["exists","up"]},{"osd":2,"uuid":"1193c1d8-bfff-4a57-8200-5f69bf1ea3af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6825","nonce":407933616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6827","nonce":407933616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6831","nonce":407933616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6829","nonce":407933616}]},"public_addr":"172.21.15.112:6825/407933616","cluster_addr":"172.21.15.112:6827/407933616","heartbeat_back_addr":"172.21.15.112:6831/407933616","heartbeat_front_addr":"172.21.15.112:6829/407933616","state":["exists","up"]},{"osd":3,"uuid":"af04330a-36ad-40cc-a8dd-fa259b221de5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6817","nonce":4120164806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6819","nonce":4120164806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6823","nonce":4120164806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6821","nonce":4120164806}]},"public_addr":"172.21.15.112:6817/4120164806","cluster_addr":"172.21.15.112:6819/4120164806","heartbeat_back_addr":"172.21.15.112:6823/4120164806","heartbeat_front_addr":"172.21.15.112:6821/4120164806","state":["exists","up"]},{"osd":4,"uuid":"5b893111-292c-4c5a-b184-53ad38c7199e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6803","nonce":522077560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6805","nonce":522077560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6809","nonce":522077560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6807","nonce":522077560}]},"public_addr":"172.21.15.187:6803/522077560","cluster_addr":"172.21.15.187:6805/522077560","heartbeat_back_addr":"172.21.15.187:6809/522077560","heartbeat_front_addr":"172.21.15.187:6807/522077560","state":["exists","up"]},{"osd":5,"uuid":"c0b8e011-45fe-4c5e-925e-6be2a7f1868a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6827","nonce":3593144721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6829","nonce":3593144721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6833","nonce":3593144721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6831","nonce":3593144721}]},"public_addr":"172.21.15.187:6827/3593144721","cluster_addr":"172.21.15.187:6829/3593144721","heartbeat_back_addr":"172.21.15.187:6833/3593144721","heartbeat_front_addr":"172.21.15.187:6831/3593144721","state":["exists","up"]},{"osd":6,"uuid":"87d2d650-7489-4593-98a3-a1f433ce8dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6811","nonce":1494387657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6813","nonce":1494387657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6817","nonce":1494387657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6815","nonce":1494387657}]},"public_addr":"172.21.15.187:6811/1494387657","cluster_addr":"172.21.15.187:6813/1494387657","heartbeat_back_addr":"172.21.15.187:6817/1494387657","heartbeat_front_addr":"172.21.15.187:6815/1494387657","state":["exists","up"]},{"osd":7,"uuid":"c6b4f8e2-a1ee-4438-aa37-67ff4b6ef1f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6819","nonce":1624699842}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6821","nonce":1624699842}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6825","nonce":1624699842}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6823","nonce":1624699842}]},"public_addr":"172.21.15.187:6819/1624699842","cluster_addr":"172.21.15.187:6821/1624699842","heartbeat_back_addr":"172.21.15.187:6825/1624699842","heartbeat_front_addr":"172.21.15.187:6823/1624699842","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.873162+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.400506+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.664317+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.159791+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.202630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.113212+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.384786+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.448441+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T00:28:42.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:42.031+0000 7fb7f3dc6640 1 -- 172.21.15.112:0/4181202273 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fb7d8078b90 msgr2=0x7fb7d807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:42.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:42.031+0000 7fb7f3dc6640 1 --2- 172.21.15.112:0/4181202273 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fb7d8078b90 0x7fb7d807b0a0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec0b1b60 tx=0x7fb7e0007a30 comp rx=0 tx=0).stop 2024-09-21T00:28:42.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:42.031+0000 7fb7f3dc6640 1 -- 172.21.15.112:0/4181202273 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7ec070440 msgr2=0x7fb7ec148680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:42.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:42.031+0000 7fb7f3dc6640 1 --2- 172.21.15.112:0/4181202273 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7ec070440 0x7fb7ec148680 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fb7e400d9c0 tx=0x7fb7e400de80 comp rx=0 tx=0).stop 2024-09-21T00:28:42.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:42.031+0000 7fb7f3dc6640 1 -- 172.21.15.112:0/4181202273 shutdown_connections 2024-09-21T00:28:42.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:42.031+0000 7fb7f3dc6640 1 --2- 172.21.15.112:0/4181202273 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb7ec164620 0x7fb7ec1665f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:42.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:42.031+0000 7fb7f3dc6640 1 --2- 172.21.15.112:0/4181202273 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb7ec0cea40 0x7fb7ec1f36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:42.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:42.031+0000 7fb7f3dc6640 1 --2- 172.21.15.112:0/4181202273 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fb7d8078b90 0x7fb7d807b0a0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:42.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:42.031+0000 7fb7f3dc6640 1 --2- 172.21.15.112:0/4181202273 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7ec070440 0x7fb7ec148680 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:42.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:42.031+0000 7fb7f3dc6640 1 -- 172.21.15.112:0/4181202273 >> 172.21.15.112:0/4181202273 conn(0x7fb7ec0cd5c0 msgr2=0x7fb7ec0cd9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:42.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:42.032+0000 7fb7f3dc6640 1 -- 172.21.15.112:0/4181202273 shutdown_connections 2024-09-21T00:28:42.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:42.032+0000 7fb7f3dc6640 1 -- 172.21.15.112:0/4181202273 wait complete. 2024-09-21T00:28:42.045 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T00:28:42.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:42.156+0000 7f8d8187e640 1 -- 172.21.15.112:0/2520468566 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8d7c0b3970 msgr2=0x7f8d7c15ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:42.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:42.156+0000 7f8d8187e640 1 --2- 172.21.15.112:0/2520468566 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8d7c0b3970 0x7f8d7c15ee60 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f8d7c1625d0 tx=0x7f8d7002a480 comp rx=0 tx=0).stop 2024-09-21T00:28:42.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:42.156+0000 7f8d8187e640 1 -- 172.21.15.112:0/2520468566 shutdown_connections 2024-09-21T00:28:42.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:42.156+0000 7f8d8187e640 1 --2- 172.21.15.112:0/2520468566 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8d7c0b3970 0x7f8d7c15ee60 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:42.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:42.156+0000 7f8d8187e640 1 -- 172.21.15.112:0/2520468566 >> 172.21.15.112:0/2520468566 conn(0x7f8d7c08dcb0 msgr2=0x7f8d7c09d8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:42.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:42.156+0000 7f8d8187e640 1 -- 172.21.15.112:0/2520468566 shutdown_connections 2024-09-21T00:28:42.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:42.156+0000 7f8d8187e640 1 -- 172.21.15.112:0/2520468566 wait complete. 2024-09-21T00:28:42.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:42.157+0000 7f8d8187e640 1 Processor -- start 2024-09-21T00:28:42.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:42.158+0000 7f8d8187e640 1 -- start start 2024-09-21T00:28:42.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:42.159+0000 7f8d8187e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8d7c0b3970 0x7f8d7c0b5680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:42.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:42.159+0000 7f8d8187e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8d7c15c3c0 0x7f8d7c0ce4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:42.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:42.159+0000 7f8d8187e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8d7c15f3a0 0x7f8d7c15b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:42.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:42.159+0000 7f8d8187e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8d7c163300 con 0x7f8d7c15c3c0 2024-09-21T00:28:42.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:42.159+0000 7f8d8187e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8d7c163180 con 0x7f8d7c15f3a0 2024-09-21T00:28:42.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:42.159+0000 7f8d8187e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f8d7c0d4870 con 0x7f8d7c0b3970 2024-09-21T00:28:42.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:42.159+0000 7f8d7a7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8d7c15c3c0 0x7f8d7c0ce4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:42.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:42.159+0000 7f8d7a7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8d7c15c3c0 0x7f8d7c0ce4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33398/0 (socket says 172.21.15.112:33398) 2024-09-21T00:28:42.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:42.159+0000 7f8d7a7fc640 1 -- 172.21.15.112:0/2875115864 learned_addr learned my addr 172.21.15.112:0/2875115864 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:28:42.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:42.159+0000 7f8d7affd640 1 --2- 172.21.15.112:0/2875115864 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8d7c0b3970 0x7f8d7c0b5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:42.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:42.159+0000 7f8d7b7fe640 1 --2- 172.21.15.112:0/2875115864 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8d7c15f3a0 0x7f8d7c15b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:42.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:42.159+0000 7f8d7a7fc640 1 -- 172.21.15.112:0/2875115864 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8d7c0b3970 msgr2=0x7f8d7c0b5680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:42.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:42.159+0000 7f8d7a7fc640 1 --2- 172.21.15.112:0/2875115864 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8d7c0b3970 0x7f8d7c0b5680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:42.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:42.159+0000 7f8d7a7fc640 1 -- 172.21.15.112:0/2875115864 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8d7c15f3a0 msgr2=0x7f8d7c15b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:42.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:42.159+0000 7f8d7a7fc640 1 --2- 172.21.15.112:0/2875115864 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8d7c15f3a0 0x7f8d7c15b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:42.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:42.159+0000 7f8d7a7fc640 1 -- 172.21.15.112:0/2875115864 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d7c0cbc00 con 0x7f8d7c15c3c0 2024-09-21T00:28:42.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:42.159+0000 7f8d7b7fe640 1 --2- 172.21.15.112:0/2875115864 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8d7c15f3a0 0x7f8d7c15b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:28:42.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:42.160+0000 7f8d7a7fc640 1 --2- 172.21.15.112:0/2875115864 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8d7c15c3c0 0x7f8d7c0ce4e0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f8d7002aa70 tx=0x7f8d70033d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:42.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:42.160+0000 7f8d63fff640 1 -- 172.21.15.112:0/2875115864 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d70031e20 con 0x7f8d7c15c3c0 2024-09-21T00:28:42.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:42.160+0000 7f8d63fff640 1 -- 172.21.15.112:0/2875115864 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8d70005110 con 0x7f8d7c15c3c0 2024-09-21T00:28:42.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:42.160+0000 7f8d8187e640 1 -- 172.21.15.112:0/2875115864 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d7c0cbee0 con 0x7f8d7c15c3c0 2024-09-21T00:28:42.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:42.160+0000 7f8d63fff640 1 -- 172.21.15.112:0/2875115864 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d70005560 con 0x7f8d7c15c3c0 2024-09-21T00:28:42.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:42.160+0000 7f8d8187e640 1 -- 172.21.15.112:0/2875115864 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d7c0b5c50 con 0x7f8d7c15c3c0 2024-09-21T00:28:42.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:42.161+0000 7f8d8187e640 1 -- 172.21.15.112:0/2875115864 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d7c0cc400 con 0x7f8d7c15c3c0 2024-09-21T00:28:42.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:42.162+0000 7f8d63fff640 1 -- 172.21.15.112:0/2875115864 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8d700047a0 con 0x7f8d7c15c3c0 2024-09-21T00:28:42.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:42.162+0000 7f8d63fff640 1 --2- 172.21.15.112:0/2875115864 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f8d68078b90 0x7f8d6807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:42.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:42.162+0000 7f8d63fff640 1 -- 172.21.15.112:0/2875115864 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6101+0+0 (secure 0 0 0) 0x7f8d700bc100 con 0x7f8d7c15c3c0 2024-09-21T00:28:42.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:42.162+0000 7f8d7affd640 1 --2- 172.21.15.112:0/2875115864 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f8d68078b90 0x7f8d6807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:42.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:42.163+0000 7f8d7affd640 1 --2- 172.21.15.112:0/2875115864 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f8d68078b90 0x7f8d6807b0a0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c001710 tx=0x7f8d6c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:42.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:42.167+0000 7f8d63fff640 1 -- 172.21.15.112:0/2875115864 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f8d7003f030 con 0x7f8d7c15c3c0 2024-09-21T00:28:42.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:42.379+0000 7f8d8187e640 1 -- 172.21.15.112:0/2875115864 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f8d7c15ee60 con 0x7f8d7c15c3c0 2024-09-21T00:28:42.381 INFO:tasks.ceph.mon.a.smithi112.stderr:2024-09-21T00:28:42.380+0000 7f7656c13640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T00:28:42.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:42.381+0000 7f8d63fff640 1 -- 172.21.15.112:0/2875115864 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v48) ==== 161+0+0 (secure 0 0 0) 0x7f8d70083df0 con 0x7f8d7c15c3c0 2024-09-21T00:28:42.383 INFO:teuthology.orchestra.run.smithi112.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T00:28:42.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:42.384+0000 7f8d8187e640 1 -- 172.21.15.112:0/2875115864 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f8d68078b90 msgr2=0x7f8d6807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:42.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:42.384+0000 7f8d8187e640 1 --2- 172.21.15.112:0/2875115864 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f8d68078b90 0x7f8d6807b0a0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c001710 tx=0x7f8d6c009040 comp rx=0 tx=0).stop 2024-09-21T00:28:42.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:42.384+0000 7f8d8187e640 1 -- 172.21.15.112:0/2875115864 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8d7c15c3c0 msgr2=0x7f8d7c0ce4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:42.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:42.384+0000 7f8d8187e640 1 --2- 172.21.15.112:0/2875115864 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8d7c15c3c0 0x7f8d7c0ce4e0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f8d7002aa70 tx=0x7f8d70033d60 comp rx=0 tx=0).stop 2024-09-21T00:28:42.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:42.384+0000 7f8d8187e640 1 -- 172.21.15.112:0/2875115864 shutdown_connections 2024-09-21T00:28:42.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:42.384+0000 7f8d8187e640 1 --2- 172.21.15.112:0/2875115864 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8d7c15f3a0 0x7f8d7c15b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:42.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:42.384+0000 7f8d8187e640 1 --2- 172.21.15.112:0/2875115864 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f8d68078b90 0x7f8d6807b0a0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:42.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:42.384+0000 7f8d8187e640 1 --2- 172.21.15.112:0/2875115864 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8d7c15c3c0 0x7f8d7c0ce4e0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:42.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:42.384+0000 7f8d8187e640 1 --2- 172.21.15.112:0/2875115864 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8d7c0b3970 0x7f8d7c0b5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:42.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:42.384+0000 7f8d8187e640 1 -- 172.21.15.112:0/2875115864 >> 172.21.15.112:0/2875115864 conn(0x7f8d7c08dcb0 msgr2=0x7f8d7c15d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:42.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:42.384+0000 7f8d8187e640 1 -- 172.21.15.112:0/2875115864 shutdown_connections 2024-09-21T00:28:42.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:42.384+0000 7f8d8187e640 1 -- 172.21.15.112:0/2875115864 wait complete. 2024-09-21T00:28:42.387 INFO:tasks.ceph.mds.a.smithi112.stderr: -2138> 2024-09-21T00:28:32.903+0000 7fcd4ff42640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T00:28:42.398 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-21T00:28:42.441 INFO:tasks.ceph.mds.a.smithi112.stdout:starting mds.a at 2024-09-21T00:28:42.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:42.514+0000 7f30f6cfa640 1 -- 172.21.15.112:0/2410964666 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f30f00ccce0 msgr2=0x7f30f00b4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:42.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:42.514+0000 7f30f6cfa640 1 --2- 172.21.15.112:0/2410964666 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f30f00ccce0 0x7f30f00b4720 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f30e4005fd0 tx=0x7f30e402a5d0 comp rx=0 tx=0).stop 2024-09-21T00:28:42.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:42.514+0000 7f30f6cfa640 1 -- 172.21.15.112:0/2410964666 shutdown_connections 2024-09-21T00:28:42.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:42.514+0000 7f30f6cfa640 1 --2- 172.21.15.112:0/2410964666 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f30f00ccce0 0x7f30f00b4720 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:42.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:42.514+0000 7f30f6cfa640 1 --2- 172.21.15.112:0/2410964666 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f30f015ca50 0x7f30f00b41e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:42.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:42.514+0000 7f30f6cfa640 1 -- 172.21.15.112:0/2410964666 >> 172.21.15.112:0/2410964666 conn(0x7f30f00cf220 msgr2=0x7f30f00cf630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:42.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:42.514+0000 7f30f6cfa640 1 -- 172.21.15.112:0/2410964666 shutdown_connections 2024-09-21T00:28:42.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:42.514+0000 7f30f6cfa640 1 -- 172.21.15.112:0/2410964666 wait complete. 2024-09-21T00:28:42.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:42.516+0000 7f30f6cfa640 1 Processor -- start 2024-09-21T00:28:42.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:42.516+0000 7f30f6cfa640 1 -- start start 2024-09-21T00:28:42.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:42.518+0000 7f30f6cfa640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f30f00b1bc0 0x7f30f00ca2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:42.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:42.518+0000 7f30f6cfa640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f30f00ccce0 0x7f30f014e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:42.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:42.518+0000 7f30f6cfa640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f30f015ca50 0x7f30f015f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:42.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:42.518+0000 7f30f6cfa640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f30f00d2130 con 0x7f30f00b1bc0 2024-09-21T00:28:42.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:42.518+0000 7f30f6cfa640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f30f00d1fb0 con 0x7f30f00ccce0 2024-09-21T00:28:42.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:42.518+0000 7f30f6cfa640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f30f00b26e0 con 0x7f30f015ca50 2024-09-21T00:28:42.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:42.518+0000 7f30f4a96640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f30f00b1bc0 0x7f30f00ca2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:42.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:42.518+0000 7f30f4a96640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f30f00b1bc0 0x7f30f00ca2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33438/0 (socket says 172.21.15.112:33438) 2024-09-21T00:28:42.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:42.518+0000 7f30f4a96640 1 -- 172.21.15.112:0/3461023608 learned_addr learned my addr 172.21.15.112:0/3461023608 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:28:42.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:42.518+0000 7f30effff640 1 --2- 172.21.15.112:0/3461023608 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f30f00ccce0 0x7f30f014e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:42.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:42.518+0000 7f30f5297640 1 --2- 172.21.15.112:0/3461023608 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f30f015ca50 0x7f30f015f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:42.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:42.518+0000 7f30f4a96640 1 -- 172.21.15.112:0/3461023608 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f30f015ca50 msgr2=0x7f30f015f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:42.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:42.518+0000 7f30f4a96640 1 --2- 172.21.15.112:0/3461023608 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f30f015ca50 0x7f30f015f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:42.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:42.518+0000 7f30f4a96640 1 -- 172.21.15.112:0/3461023608 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f30f00ccce0 msgr2=0x7f30f014e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:42.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:42.518+0000 7f30f4a96640 1 --2- 172.21.15.112:0/3461023608 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f30f00ccce0 0x7f30f014e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:42.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:42.518+0000 7f30f4a96640 1 -- 172.21.15.112:0/3461023608 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30f00c94e0 con 0x7f30f00b1bc0 2024-09-21T00:28:42.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:42.518+0000 7f30effff640 1 --2- 172.21.15.112:0/3461023608 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f30f00ccce0 0x7f30f014e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:28:42.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:42.518+0000 7f30f5297640 1 --2- 172.21.15.112:0/3461023608 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f30f015ca50 0x7f30f015f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:28:42.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:42.518+0000 7f30f4a96640 1 --2- 172.21.15.112:0/3461023608 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f30f00b1bc0 0x7f30f00ca2a0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f30e000e9f0 tx=0x7f30e000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:42.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:42.518+0000 7f30ed7fa640 1 -- 172.21.15.112:0/3461023608 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e000cd80 con 0x7f30f00b1bc0 2024-09-21T00:28:42.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:42.518+0000 7f30ed7fa640 1 -- 172.21.15.112:0/3461023608 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f30e0004c10 con 0x7f30f00b1bc0 2024-09-21T00:28:42.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:42.518+0000 7f30f6cfa640 1 -- 172.21.15.112:0/3461023608 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30f00c97c0 con 0x7f30f00b1bc0 2024-09-21T00:28:42.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:42.518+0000 7f30ed7fa640 1 -- 172.21.15.112:0/3461023608 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e0005020 con 0x7f30f00b1bc0 2024-09-21T00:28:42.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:42.518+0000 7f30f6cfa640 1 -- 172.21.15.112:0/3461023608 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30f00c9d80 con 0x7f30f00b1bc0 2024-09-21T00:28:42.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:42.519+0000 7f30f6cfa640 1 -- 172.21.15.112:0/3461023608 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30f00b1fd0 con 0x7f30f00b1bc0 2024-09-21T00:28:42.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:42.520+0000 7f30ed7fa640 1 -- 172.21.15.112:0/3461023608 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f30e00051c0 con 0x7f30f00b1bc0 2024-09-21T00:28:42.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:42.520+0000 7f30ed7fa640 1 --2- 172.21.15.112:0/3461023608 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f30dc078b90 0x7f30dc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:42.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:42.521+0000 7f30ed7fa640 1 -- 172.21.15.112:0/3461023608 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 6187+0+0 (secure 0 0 0) 0x7f30e009b8f0 con 0x7f30f00b1bc0 2024-09-21T00:28:42.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:42.521+0000 7f30effff640 1 --2- 172.21.15.112:0/3461023608 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f30dc078b90 0x7f30dc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:42.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:42.521+0000 7f30effff640 1 --2- 172.21.15.112:0/3461023608 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f30dc078b90 0x7f30dc07b0a0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f30e4033000 tx=0x7f30e4004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:42.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:42.525+0000 7f30ed7fa640 1 -- 172.21.15.112:0/3461023608 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f30e0016030 con 0x7f30f00b1bc0 2024-09-21T00:28:42.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:42.741+0000 7f30f6cfa640 1 -- 172.21.15.112:0/3461023608 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f30f0072030 con 0x7f30f00b1bc0 2024-09-21T00:28:43.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:43.383+0000 7f30ed7fa640 1 -- 172.21.15.112:0/3461023608 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v49) ==== 104+0+0 (secure 0 0 0) 0x7f30e0063b50 con 0x7f30f00b1bc0 2024-09-21T00:28:43.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:43.385+0000 7f30f6cfa640 1 -- 172.21.15.112:0/3461023608 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f30dc078b90 msgr2=0x7f30dc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:43.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:43.385+0000 7f30f6cfa640 1 --2- 172.21.15.112:0/3461023608 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f30dc078b90 0x7f30dc07b0a0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f30e4033000 tx=0x7f30e4004730 comp rx=0 tx=0).stop 2024-09-21T00:28:43.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:43.385+0000 7f30f6cfa640 1 -- 172.21.15.112:0/3461023608 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f30f00b1bc0 msgr2=0x7f30f00ca2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:43.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:43.385+0000 7f30f6cfa640 1 --2- 172.21.15.112:0/3461023608 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f30f00b1bc0 0x7f30f00ca2a0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f30e000e9f0 tx=0x7f30e000eeb0 comp rx=0 tx=0).stop 2024-09-21T00:28:43.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:43.385+0000 7f30f6cfa640 1 -- 172.21.15.112:0/3461023608 shutdown_connections 2024-09-21T00:28:43.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:43.385+0000 7f30f6cfa640 1 --2- 172.21.15.112:0/3461023608 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f30f015ca50 0x7f30f015f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:43.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:43.385+0000 7f30f6cfa640 1 --2- 172.21.15.112:0/3461023608 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f30f00ccce0 0x7f30f014e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:43.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:43.385+0000 7f30f6cfa640 1 --2- 172.21.15.112:0/3461023608 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f30dc078b90 0x7f30dc07b0a0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:43.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:43.385+0000 7f30f6cfa640 1 --2- 172.21.15.112:0/3461023608 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f30f00b1bc0 0x7f30f00ca2a0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:43.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:43.385+0000 7f30f6cfa640 1 -- 172.21.15.112:0/3461023608 >> 172.21.15.112:0/3461023608 conn(0x7f30f00cf220 msgr2=0x7f30f00d08e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:43.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:43.386+0000 7f30f6cfa640 1 -- 172.21.15.112:0/3461023608 shutdown_connections 2024-09-21T00:28:43.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:43.386+0000 7f30f6cfa640 1 -- 172.21.15.112:0/3461023608 wait complete. 2024-09-21T00:28:43.400 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-21T00:28:43.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:43.516+0000 7f2facb8b640 1 -- 172.21.15.112:0/562379925 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2fa815f4d0 msgr2=0x7f2fa815f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:43.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:43.516+0000 7f2facb8b640 1 --2- 172.21.15.112:0/562379925 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2fa815f4d0 0x7f2fa815f8e0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f2fa808c720 tx=0x7f2fa002a980 comp rx=0 tx=0).stop 2024-09-21T00:28:43.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:43.516+0000 7f2facb8b640 1 -- 172.21.15.112:0/562379925 shutdown_connections 2024-09-21T00:28:43.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:43.516+0000 7f2facb8b640 1 --2- 172.21.15.112:0/562379925 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2fa815f4d0 0x7f2fa815f8e0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:43.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:43.516+0000 7f2facb8b640 1 --2- 172.21.15.112:0/562379925 >> v2:172.21.15.187:3300/0 conn(0x7f2fa80cfec0 0x7f2fa80d02d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:43.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:43.516+0000 7f2facb8b640 1 -- 172.21.15.112:0/562379925 >> 172.21.15.112:0/562379925 conn(0x7f2fa809d990 msgr2=0x7f2fa808bfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:43.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:43.516+0000 7f2facb8b640 1 -- 172.21.15.112:0/562379925 shutdown_connections 2024-09-21T00:28:43.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:43.516+0000 7f2facb8b640 1 -- 172.21.15.112:0/562379925 wait complete. 2024-09-21T00:28:43.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:43.518+0000 7f2facb8b640 1 Processor -- start 2024-09-21T00:28:43.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:43.518+0000 7f2facb8b640 1 -- start start 2024-09-21T00:28:43.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:43.519+0000 7f2facb8b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2fa80cfec0 0x7f2fa814e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:43.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:43.519+0000 7f2facb8b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2fa80d4610 0x7f2fa8150c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:43.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:43.519+0000 7f2facb8b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2fa815f4d0 0x7f2fa8151140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:43.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:43.519+0000 7f2facb8b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2fa815ead0 con 0x7f2fa80cfec0 2024-09-21T00:28:43.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:43.519+0000 7f2facb8b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f2fa815e950 con 0x7f2fa80d4610 2024-09-21T00:28:43.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:43.519+0000 7f2facb8b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f2fa815ec50 con 0x7f2fa815f4d0 2024-09-21T00:28:43.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:43.519+0000 7f2fa659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2fa80cfec0 0x7f2fa814e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:43.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:43.520+0000 7f2fa659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2fa80cfec0 0x7f2fa814e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33464/0 (socket says 172.21.15.112:33464) 2024-09-21T00:28:43.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:43.520+0000 7f2fa659c640 1 -- 172.21.15.112:0/2488563449 learned_addr learned my addr 172.21.15.112:0/2488563449 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:28:43.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:43.520+0000 7f2fa6d9d640 1 --2- 172.21.15.112:0/2488563449 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2fa815f4d0 0x7f2fa8151140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:43.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:43.520+0000 7f2fa5d9b640 1 --2- 172.21.15.112:0/2488563449 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2fa80d4610 0x7f2fa8150c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:43.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:43.520+0000 7f2fa659c640 1 -- 172.21.15.112:0/2488563449 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2fa815f4d0 msgr2=0x7f2fa8151140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:43.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:43.520+0000 7f2fa659c640 1 --2- 172.21.15.112:0/2488563449 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2fa815f4d0 0x7f2fa8151140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:43.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:43.520+0000 7f2fa659c640 1 -- 172.21.15.112:0/2488563449 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2fa80d4610 msgr2=0x7f2fa8150c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:43.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:43.520+0000 7f2fa659c640 1 --2- 172.21.15.112:0/2488563449 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2fa80d4610 0x7f2fa8150c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:43.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:43.520+0000 7f2fa659c640 1 -- 172.21.15.112:0/2488563449 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fa80cb940 con 0x7f2fa80cfec0 2024-09-21T00:28:43.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:43.520+0000 7f2fa659c640 1 --2- 172.21.15.112:0/2488563449 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2fa80cfec0 0x7f2fa814e3b0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f2f9800c970 tx=0x7f2f9800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:43.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:43.520+0000 7f2f96ffd640 1 -- 172.21.15.112:0/2488563449 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f98007bb0 con 0x7f2fa80cfec0 2024-09-21T00:28:43.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:43.520+0000 7f2facb8b640 1 -- 172.21.15.112:0/2488563449 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fa80cbc20 con 0x7f2fa80cfec0 2024-09-21T00:28:43.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:43.520+0000 7f2f96ffd640 1 -- 172.21.15.112:0/2488563449 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2f98007d50 con 0x7f2fa80cfec0 2024-09-21T00:28:43.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:43.520+0000 7f2f96ffd640 1 -- 172.21.15.112:0/2488563449 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f98005290 con 0x7f2fa80cfec0 2024-09-21T00:28:43.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:43.520+0000 7f2facb8b640 1 -- 172.21.15.112:0/2488563449 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fa80cebd0 con 0x7f2fa80cfec0 2024-09-21T00:28:43.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:43.521+0000 7f2facb8b640 1 -- 172.21.15.112:0/2488563449 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fa80cd450 con 0x7f2fa80cfec0 2024-09-21T00:28:43.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:43.522+0000 7f2f96ffd640 1 -- 172.21.15.112:0/2488563449 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2f98005430 con 0x7f2fa80cfec0 2024-09-21T00:28:43.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:43.523+0000 7f2f96ffd640 1 --2- 172.21.15.112:0/2488563449 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2f88078b90 0x7f2f8807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:43.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:43.523+0000 7f2f96ffd640 1 -- 172.21.15.112:0/2488563449 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 6187+0+0 (secure 0 0 0) 0x7f2f9809b3e0 con 0x7f2fa80cfec0 2024-09-21T00:28:43.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:43.523+0000 7f2fa5d9b640 1 --2- 172.21.15.112:0/2488563449 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2f88078b90 0x7f2f8807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:43.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:43.524+0000 7f2fa5d9b640 1 --2- 172.21.15.112:0/2488563449 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2f88078b90 0x7f2f8807b0a0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f2fa815d5c0 tx=0x7f2f9c009060 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:43.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:43.527+0000 7f2f96ffd640 1 -- 172.21.15.112:0/2488563449 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2fa80cd450 con 0x7f2fa80cfec0 2024-09-21T00:28:43.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:43.705+0000 7f2facb8b640 1 -- 172.21.15.112:0/2488563449 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f2fa815f8e0 con 0x7f2fa80cfec0 2024-09-21T00:28:44.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:44.384+0000 7f2f96ffd640 1 -- 172.21.15.112:0/2488563449 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v73) ==== 181+0+0 (secure 0 0 0) 0x7f2f98014070 con 0x7f2fa80cfec0 2024-09-21T00:28:44.386 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'cephfs_metadata' removed 2024-09-21T00:28:44.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:44.388+0000 7f2facb8b640 1 -- 172.21.15.112:0/2488563449 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2f88078b90 msgr2=0x7f2f8807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:44.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:44.388+0000 7f2facb8b640 1 --2- 172.21.15.112:0/2488563449 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2f88078b90 0x7f2f8807b0a0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f2fa815d5c0 tx=0x7f2f9c009060 comp rx=0 tx=0).stop 2024-09-21T00:28:44.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:44.388+0000 7f2facb8b640 1 -- 172.21.15.112:0/2488563449 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2fa80cfec0 msgr2=0x7f2fa814e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:44.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:44.388+0000 7f2facb8b640 1 --2- 172.21.15.112:0/2488563449 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2fa80cfec0 0x7f2fa814e3b0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f2f9800c970 tx=0x7f2f9800ce30 comp rx=0 tx=0).stop 2024-09-21T00:28:44.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:44.388+0000 7f2facb8b640 1 -- 172.21.15.112:0/2488563449 shutdown_connections 2024-09-21T00:28:44.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:44.388+0000 7f2facb8b640 1 --2- 172.21.15.112:0/2488563449 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2fa815f4d0 0x7f2fa8151140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:44.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:44.388+0000 7f2facb8b640 1 --2- 172.21.15.112:0/2488563449 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2fa80d4610 0x7f2fa8150c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:44.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:44.388+0000 7f2facb8b640 1 --2- 172.21.15.112:0/2488563449 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2f88078b90 0x7f2f8807b0a0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:44.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:44.388+0000 7f2facb8b640 1 --2- 172.21.15.112:0/2488563449 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2fa80cfec0 0x7f2fa814e3b0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:44.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:44.388+0000 7f2facb8b640 1 -- 172.21.15.112:0/2488563449 >> 172.21.15.112:0/2488563449 conn(0x7f2fa809d990 msgr2=0x7f2fa8069180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:44.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:44.388+0000 7f2facb8b640 1 -- 172.21.15.112:0/2488563449 shutdown_connections 2024-09-21T00:28:44.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:44.388+0000 7f2facb8b640 1 -- 172.21.15.112:0/2488563449 wait complete. 2024-09-21T00:28:44.403 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-21T00:28:44.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:44.520+0000 7f8ca87ba640 1 -- 172.21.15.112:0/4083002596 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8ca00b3bf0 msgr2=0x7f8ca00d2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:44.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:44.520+0000 7f8ca87ba640 1 --2- 172.21.15.112:0/4083002596 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8ca00b3bf0 0x7f8ca00d2cf0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8c90005fd0 tx=0x7f8c9002aa10 comp rx=0 tx=0).stop 2024-09-21T00:28:44.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:44.520+0000 7f8ca87ba640 1 -- 172.21.15.112:0/4083002596 shutdown_connections 2024-09-21T00:28:44.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:44.520+0000 7f8ca87ba640 1 --2- 172.21.15.112:0/4083002596 >> v2:172.21.15.187:3300/0 conn(0x7f8ca0070470 0x7f8ca00b6780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:44.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:44.520+0000 7f8ca87ba640 1 --2- 172.21.15.112:0/4083002596 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8ca00b3bf0 0x7f8ca00d2cf0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:44.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:44.520+0000 7f8ca87ba640 1 -- 172.21.15.112:0/4083002596 >> 172.21.15.112:0/4083002596 conn(0x7f8ca008c1c0 msgr2=0x7f8ca009d930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:44.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:44.520+0000 7f8ca87ba640 1 -- 172.21.15.112:0/4083002596 shutdown_connections 2024-09-21T00:28:44.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:44.520+0000 7f8ca87ba640 1 -- 172.21.15.112:0/4083002596 wait complete. 2024-09-21T00:28:44.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:44.522+0000 7f8ca87ba640 1 Processor -- start 2024-09-21T00:28:44.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:44.522+0000 7f8ca87ba640 1 -- start start 2024-09-21T00:28:44.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:44.523+0000 7f8ca87ba640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ca0070470 0x7f8ca00ce040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:44.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:44.523+0000 7f8ca87ba640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8ca00b3bf0 0x7f8ca006e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:44.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:44.523+0000 7f8ca87ba640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8ca00d3360 0x7f8ca006e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:44.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:44.523+0000 7f8ca87ba640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8ca00d0880 con 0x7f8ca0070470 2024-09-21T00:28:44.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:44.523+0000 7f8ca87ba640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8ca00d0700 con 0x7f8ca00b3bf0 2024-09-21T00:28:44.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:44.523+0000 7f8ca87ba640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f8ca00d0a00 con 0x7f8ca00d3360 2024-09-21T00:28:44.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:44.524+0000 7f8ca6556640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ca0070470 0x7f8ca00ce040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:44.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:44.524+0000 7f8ca6556640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ca0070470 0x7f8ca00ce040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33470/0 (socket says 172.21.15.112:33470) 2024-09-21T00:28:44.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:44.524+0000 7f8ca6556640 1 -- 172.21.15.112:0/62050492 learned_addr learned my addr 172.21.15.112:0/62050492 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:28:44.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:44.524+0000 7f8ca6d57640 1 --2- 172.21.15.112:0/62050492 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8ca00d3360 0x7f8ca006e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:44.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:44.524+0000 7f8ca5d55640 1 --2- 172.21.15.112:0/62050492 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8ca00b3bf0 0x7f8ca006e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:44.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:44.524+0000 7f8ca6556640 1 -- 172.21.15.112:0/62050492 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8ca00d3360 msgr2=0x7f8ca006e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:44.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:44.524+0000 7f8ca6556640 1 --2- 172.21.15.112:0/62050492 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8ca00d3360 0x7f8ca006e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:44.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:44.524+0000 7f8ca6556640 1 -- 172.21.15.112:0/62050492 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8ca00b3bf0 msgr2=0x7f8ca006e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:44.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:44.524+0000 7f8ca6556640 1 --2- 172.21.15.112:0/62050492 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8ca00b3bf0 0x7f8ca006e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:44.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:44.524+0000 7f8ca6556640 1 -- 172.21.15.112:0/62050492 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ca00c9890 con 0x7f8ca0070470 2024-09-21T00:28:44.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:44.524+0000 7f8ca5d55640 1 --2- 172.21.15.112:0/62050492 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8ca00b3bf0 0x7f8ca006e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:28:44.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:44.524+0000 7f8ca6556640 1 --2- 172.21.15.112:0/62050492 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ca0070470 0x7f8ca00ce040 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f8c90006010 tx=0x7f8c90003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:44.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:44.524+0000 7f8c9effd640 1 -- 172.21.15.112:0/62050492 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c90046070 con 0x7f8ca0070470 2024-09-21T00:28:44.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:44.524+0000 7f8c9effd640 1 -- 172.21.15.112:0/62050492 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8c90004560 con 0x7f8ca0070470 2024-09-21T00:28:44.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:44.524+0000 7f8c9effd640 1 -- 172.21.15.112:0/62050492 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c90033070 con 0x7f8ca0070470 2024-09-21T00:28:44.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:44.524+0000 7f8ca87ba640 1 -- 172.21.15.112:0/62050492 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ca00c9b70 con 0x7f8ca0070470 2024-09-21T00:28:44.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:44.524+0000 7f8ca87ba640 1 -- 172.21.15.112:0/62050492 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ca00ca130 con 0x7f8ca0070470 2024-09-21T00:28:44.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:44.525+0000 7f8ca87ba640 1 -- 172.21.15.112:0/62050492 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ca00d0eb0 con 0x7f8ca0070470 2024-09-21T00:28:44.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:44.526+0000 7f8c9effd640 1 -- 172.21.15.112:0/62050492 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8c90049070 con 0x7f8ca0070470 2024-09-21T00:28:44.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:44.527+0000 7f8c9effd640 1 --2- 172.21.15.112:0/62050492 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f8c8c078b90 0x7f8c8c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:44.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:44.527+0000 7f8c9effd640 1 -- 172.21.15.112:0/62050492 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5749+0+0 (secure 0 0 0) 0x7f8c900bbed0 con 0x7f8ca0070470 2024-09-21T00:28:44.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:44.527+0000 7f8ca5d55640 1 --2- 172.21.15.112:0/62050492 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f8c8c078b90 0x7f8c8c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:44.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:44.528+0000 7f8ca5d55640 1 --2- 172.21.15.112:0/62050492 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f8c8c078b90 0x7f8c8c07b0a0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f8ca006f970 tx=0x7f8c94009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:44.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:44.531+0000 7f8c9effd640 1 -- 172.21.15.112:0/62050492 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f8c9003e070 con 0x7f8ca0070470 2024-09-21T00:28:44.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:44.708+0000 7f8ca87ba640 1 -- 172.21.15.112:0/62050492 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f8ca00d4c70 con 0x7f8ca0070470 2024-09-21T00:28:45.208 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T00:28:45.211 DEBUG:teuthology.orchestra.run.smithi187:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T00:28:45.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:45.391+0000 7f8c9effd640 1 -- 172.21.15.112:0/62050492 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v74) ==== 169+0+0 (secure 0 0 0) 0x7f8c90084240 con 0x7f8ca0070470 2024-09-21T00:28:45.393 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'cephfs_data' removed 2024-09-21T00:28:45.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:45.393+0000 7f8ca87ba640 1 -- 172.21.15.112:0/62050492 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f8c8c078b90 msgr2=0x7f8c8c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:45.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:45.393+0000 7f8ca87ba640 1 --2- 172.21.15.112:0/62050492 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f8c8c078b90 0x7f8c8c07b0a0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f8ca006f970 tx=0x7f8c94009040 comp rx=0 tx=0).stop 2024-09-21T00:28:45.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:45.394+0000 7f8ca87ba640 1 -- 172.21.15.112:0/62050492 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ca0070470 msgr2=0x7f8ca00ce040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:45.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:45.394+0000 7f8ca87ba640 1 --2- 172.21.15.112:0/62050492 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ca0070470 0x7f8ca00ce040 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f8c90006010 tx=0x7f8c90003f90 comp rx=0 tx=0).stop 2024-09-21T00:28:45.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:45.394+0000 7f8ca87ba640 1 -- 172.21.15.112:0/62050492 shutdown_connections 2024-09-21T00:28:45.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:45.394+0000 7f8ca87ba640 1 --2- 172.21.15.112:0/62050492 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8ca00d3360 0x7f8ca006e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:45.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:45.394+0000 7f8ca87ba640 1 --2- 172.21.15.112:0/62050492 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8ca00b3bf0 0x7f8ca006e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:45.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:45.394+0000 7f8ca87ba640 1 --2- 172.21.15.112:0/62050492 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f8c8c078b90 0x7f8c8c07b0a0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:45.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:45.394+0000 7f8ca87ba640 1 --2- 172.21.15.112:0/62050492 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ca0070470 0x7f8ca00ce040 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:45.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:45.394+0000 7f8ca87ba640 1 -- 172.21.15.112:0/62050492 >> 172.21.15.112:0/62050492 conn(0x7f8ca008c1c0 msgr2=0x7f8ca00d10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:45.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:45.396+0000 7f8ca87ba640 1 -- 172.21.15.112:0/62050492 shutdown_connections 2024-09-21T00:28:45.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:45.396+0000 7f8ca87ba640 1 -- 172.21.15.112:0/62050492 wait complete. 2024-09-21T00:28:45.412 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 31 2024-09-21T00:28:45.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:45.564+0000 7f1c4cd21640 1 -- 172.21.15.112:0/805275766 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c480b5ab0 msgr2=0x7f1c480b0730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:45.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:45.564+0000 7f1c4cd21640 1 --2- 172.21.15.112:0/805275766 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c480b5ab0 0x7f1c480b0730 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f1c480d09e0 tx=0x7f1c3802a560 comp rx=0 tx=0).stop 2024-09-21T00:28:45.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:45.564+0000 7f1c4cd21640 1 -- 172.21.15.112:0/805275766 shutdown_connections 2024-09-21T00:28:45.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:45.565+0000 7f1c4cd21640 1 --2- 172.21.15.112:0/805275766 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1c48144dd0 0x7f1c4815f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:45.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:45.565+0000 7f1c4cd21640 1 --2- 172.21.15.112:0/805275766 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c480b5ab0 0x7f1c480b0730 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:45.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:45.565+0000 7f1c4cd21640 1 -- 172.21.15.112:0/805275766 >> 172.21.15.112:0/805275766 conn(0x7f1c48073fe0 msgr2=0x7f1c480743f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:45.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:45.565+0000 7f1c4cd21640 1 -- 172.21.15.112:0/805275766 shutdown_connections 2024-09-21T00:28:45.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:45.565+0000 7f1c4cd21640 1 -- 172.21.15.112:0/805275766 wait complete. 2024-09-21T00:28:45.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:45.567+0000 7f1c4cd21640 1 Processor -- start 2024-09-21T00:28:45.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:45.567+0000 7f1c4cd21640 1 -- start start 2024-09-21T00:28:45.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:45.568+0000 7f1c4cd21640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c48144dd0 0x7f1c480ad560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:45.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:45.568+0000 7f1c4cd21640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1c4815f620 0x7f1c4815d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:45.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:45.568+0000 7f1c4cd21640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1c4815d960 0x7f1c481f0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:45.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:45.568+0000 7f1c4cd21640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1c4806ee90 con 0x7f1c48144dd0 2024-09-21T00:28:45.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:45.568+0000 7f1c4cd21640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f1c480d05c0 con 0x7f1c4815f620 2024-09-21T00:28:45.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:45.568+0000 7f1c4cd21640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f1c4806f010 con 0x7f1c4815d960 2024-09-21T00:28:45.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:45.568+0000 7f1c4659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c48144dd0 0x7f1c480ad560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:45.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:45.568+0000 7f1c4659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c48144dd0 0x7f1c480ad560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33490/0 (socket says 172.21.15.112:33490) 2024-09-21T00:28:45.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:45.568+0000 7f1c4659c640 1 -- 172.21.15.112:0/2342153579 learned_addr learned my addr 172.21.15.112:0/2342153579 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:28:45.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:45.568+0000 7f1c46d9d640 1 --2- 172.21.15.112:0/2342153579 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1c4815d960 0x7f1c481f0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:45.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:45.568+0000 7f1c3dd9b640 1 --2- 172.21.15.112:0/2342153579 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1c4815f620 0x7f1c4815d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:45.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:45.568+0000 7f1c4659c640 1 -- 172.21.15.112:0/2342153579 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1c4815d960 msgr2=0x7f1c481f0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:45.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:45.568+0000 7f1c4659c640 1 --2- 172.21.15.112:0/2342153579 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1c4815d960 0x7f1c481f0fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:45.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:45.568+0000 7f1c4659c640 1 -- 172.21.15.112:0/2342153579 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1c4815f620 msgr2=0x7f1c4815d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:45.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:45.568+0000 7f1c4659c640 1 --2- 172.21.15.112:0/2342153579 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1c4815f620 0x7f1c4815d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:45.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:45.568+0000 7f1c4659c640 1 -- 172.21.15.112:0/2342153579 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c480a94b0 con 0x7f1c48144dd0 2024-09-21T00:28:45.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:45.568+0000 7f1c3dd9b640 1 --2- 172.21.15.112:0/2342153579 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1c4815f620 0x7f1c4815d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:28:45.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:45.568+0000 7f1c46d9d640 1 --2- 172.21.15.112:0/2342153579 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1c4815d960 0x7f1c481f0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:28:45.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:45.569+0000 7f1c4659c640 1 --2- 172.21.15.112:0/2342153579 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c48144dd0 0x7f1c480ad560 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f1c380099c0 tx=0x7f1c380039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:45.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:45.569+0000 7f1c3f7fe640 1 -- 172.21.15.112:0/2342153579 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c38035040 con 0x7f1c48144dd0 2024-09-21T00:28:45.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:45.569+0000 7f1c3f7fe640 1 -- 172.21.15.112:0/2342153579 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1c380047a0 con 0x7f1c48144dd0 2024-09-21T00:28:45.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:45.569+0000 7f1c3f7fe640 1 -- 172.21.15.112:0/2342153579 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c38004bf0 con 0x7f1c48144dd0 2024-09-21T00:28:45.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:45.569+0000 7f1c4cd21640 1 -- 172.21.15.112:0/2342153579 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c480a9790 con 0x7f1c48144dd0 2024-09-21T00:28:45.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:45.569+0000 7f1c4cd21640 1 -- 172.21.15.112:0/2342153579 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c480a9d50 con 0x7f1c48144dd0 2024-09-21T00:28:45.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:45.570+0000 7f1c4cd21640 1 -- 172.21.15.112:0/2342153579 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c481f1ce0 con 0x7f1c48144dd0 2024-09-21T00:28:45.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:45.571+0000 7f1c3f7fe640 1 -- 172.21.15.112:0/2342153579 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1c38004d90 con 0x7f1c48144dd0 2024-09-21T00:28:45.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:45.571+0000 7f1c3f7fe640 1 --2- 172.21.15.112:0/2342153579 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1c28078bc0 0x7f1c2807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:45.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:45.571+0000 7f1c3f7fe640 1 -- 172.21.15.112:0/2342153579 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5335+0+0 (secure 0 0 0) 0x7f1c380bd550 con 0x7f1c48144dd0 2024-09-21T00:28:45.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:45.571+0000 7f1c3dd9b640 1 --2- 172.21.15.112:0/2342153579 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1c28078bc0 0x7f1c2807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:45.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:45.572+0000 7f1c3dd9b640 1 --2- 172.21.15.112:0/2342153579 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1c28078bc0 0x7f1c2807b0d0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f1c48136ab0 tx=0x7f1c30008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:45.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:45.575+0000 7f1c3f7fe640 1 -- 172.21.15.112:0/2342153579 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1c4815fa30 con 0x7f1c48144dd0 2024-09-21T00:28:45.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:45.758+0000 7f1c4cd21640 1 -- 172.21.15.112:0/2342153579 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "config reset", "num": 31} v 0) -- 0x7f1c4815eb70 con 0x7f1c48144dd0 2024-09-21T00:28:45.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:45.761+0000 7f1c3f7fe640 1 -- 172.21.15.112:0/2342153579 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 31}]=0 v41) ==== 71+0+0 (secure 0 0 0) 0x7f1c38085b90 con 0x7f1c48144dd0 2024-09-21T00:28:45.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:45.764+0000 7f1c4cd21640 1 -- 172.21.15.112:0/2342153579 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1c28078bc0 msgr2=0x7f1c2807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:45.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:45.764+0000 7f1c4cd21640 1 --2- 172.21.15.112:0/2342153579 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1c28078bc0 0x7f1c2807b0d0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f1c48136ab0 tx=0x7f1c30008040 comp rx=0 tx=0).stop 2024-09-21T00:28:45.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:45.764+0000 7f1c4cd21640 1 -- 172.21.15.112:0/2342153579 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c48144dd0 msgr2=0x7f1c480ad560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:45.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:45.764+0000 7f1c4cd21640 1 --2- 172.21.15.112:0/2342153579 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c48144dd0 0x7f1c480ad560 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f1c380099c0 tx=0x7f1c380039e0 comp rx=0 tx=0).stop 2024-09-21T00:28:45.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:45.764+0000 7f1c4cd21640 1 -- 172.21.15.112:0/2342153579 shutdown_connections 2024-09-21T00:28:45.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:45.764+0000 7f1c4cd21640 1 --2- 172.21.15.112:0/2342153579 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1c4815d960 0x7f1c481f0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:45.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:45.764+0000 7f1c4cd21640 1 --2- 172.21.15.112:0/2342153579 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1c4815f620 0x7f1c4815d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:45.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:45.764+0000 7f1c4cd21640 1 --2- 172.21.15.112:0/2342153579 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1c28078bc0 0x7f1c2807b0d0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:45.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:45.764+0000 7f1c4cd21640 1 --2- 172.21.15.112:0/2342153579 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c48144dd0 0x7f1c480ad560 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:45.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:45.764+0000 7f1c4cd21640 1 -- 172.21.15.112:0/2342153579 >> 172.21.15.112:0/2342153579 conn(0x7f1c48073fe0 msgr2=0x7f1c480b2140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:45.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:45.764+0000 7f1c4cd21640 1 -- 172.21.15.112:0/2342153579 shutdown_connections 2024-09-21T00:28:45.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:45.764+0000 7f1c4cd21640 1 -- 172.21.15.112:0/2342153579 wait complete. 2024-09-21T00:28:45.777 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_sessionmap.TestSessionMap.test_session_reject' 2024-09-21T00:28:45.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:45.888+0000 7f121ad8e640 1 -- 172.21.15.112:0/2006077229 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12140d0860 msgr2=0x7f121415a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:45.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:45.888+0000 7f121ad8e640 1 --2- 172.21.15.112:0/2006077229 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12140d0860 0x7f121415a0c0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f1204005fd0 tx=0x7f120402a530 comp rx=0 tx=0).stop 2024-09-21T00:28:45.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:45.888+0000 7f121ad8e640 1 -- 172.21.15.112:0/2006077229 shutdown_connections 2024-09-21T00:28:45.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:45.888+0000 7f121ad8e640 1 --2- 172.21.15.112:0/2006077229 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12140d0860 0x7f121415a0c0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:45.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:45.888+0000 7f121ad8e640 1 --2- 172.21.15.112:0/2006077229 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f12140d2030 0x7f121415d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:45.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:45.888+0000 7f121ad8e640 1 -- 172.21.15.112:0/2006077229 >> 172.21.15.112:0/2006077229 conn(0x7f12140b43f0 msgr2=0x7f12140b4830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:45.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:45.888+0000 7f121ad8e640 1 -- 172.21.15.112:0/2006077229 shutdown_connections 2024-09-21T00:28:45.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:45.888+0000 7f121ad8e640 1 -- 172.21.15.112:0/2006077229 wait complete. 2024-09-21T00:28:45.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:45.890+0000 7f121ad8e640 1 Processor -- start 2024-09-21T00:28:45.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:45.890+0000 7f121ad8e640 1 -- start start 2024-09-21T00:28:45.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:45.891+0000 7f121ad8e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f12140ccae0 0x7f12141574a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:45.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:45.891+0000 7f121ad8e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12140d0860 0x7f12140c7a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:45.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:45.891+0000 7f121ad8e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f12140d2030 0x7f12140c7f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:45.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:45.891+0000 7f121ad8e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f12140b22d0 con 0x7f12140d0860 2024-09-21T00:28:45.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:45.891+0000 7f121ad8e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f12140b2150 con 0x7f12140ccae0 2024-09-21T00:28:45.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:45.891+0000 7f121ad8e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f12140b2450 con 0x7f12140d2030 2024-09-21T00:28:45.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:45.891+0000 7f120bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12140d0860 0x7f12140c7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:45.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:45.891+0000 7f120bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12140d0860 0x7f12140c7a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33510/0 (socket says 172.21.15.112:33510) 2024-09-21T00:28:45.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:45.891+0000 7f120bfff640 1 -- 172.21.15.112:0/1358465142 learned_addr learned my addr 172.21.15.112:0/1358465142 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:28:45.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:45.891+0000 7f1218b2a640 1 --2- 172.21.15.112:0/1358465142 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f12140ccae0 0x7f12141574a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:45.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:45.891+0000 7f121932b640 1 --2- 172.21.15.112:0/1358465142 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f12140d2030 0x7f12140c7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:45.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:45.892+0000 7f120bfff640 1 -- 172.21.15.112:0/1358465142 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f12140d2030 msgr2=0x7f12140c7f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:45.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:45.892+0000 7f120bfff640 1 --2- 172.21.15.112:0/1358465142 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f12140d2030 0x7f12140c7f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:45.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:45.892+0000 7f120bfff640 1 -- 172.21.15.112:0/1358465142 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f12140ccae0 msgr2=0x7f12141574a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:45.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:45.892+0000 7f120bfff640 1 --2- 172.21.15.112:0/1358465142 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f12140ccae0 0x7f12141574a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:45.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:45.892+0000 7f120bfff640 1 -- 172.21.15.112:0/1358465142 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12140b8260 con 0x7f12140d0860 2024-09-21T00:28:45.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:45.892+0000 7f120bfff640 1 --2- 172.21.15.112:0/1358465142 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12140d0860 0x7f12140c7a50 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f1204034040 tx=0x7f12040039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:45.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:45.892+0000 7f12097fa640 1 -- 172.21.15.112:0/1358465142 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f120403e030 con 0x7f12140d0860 2024-09-21T00:28:45.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:45.892+0000 7f12097fa640 1 -- 172.21.15.112:0/1358465142 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f12040047a0 con 0x7f12140d0860 2024-09-21T00:28:45.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:45.892+0000 7f121ad8e640 1 -- 172.21.15.112:0/1358465142 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12141f4840 con 0x7f12140d0860 2024-09-21T00:28:45.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:45.892+0000 7f12097fa640 1 -- 172.21.15.112:0/1358465142 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1204035040 con 0x7f12140d0860 2024-09-21T00:28:45.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:45.892+0000 7f121ad8e640 1 -- 172.21.15.112:0/1358465142 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12141f3cc0 con 0x7f12140d0860 2024-09-21T00:28:45.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:45.893+0000 7f121ad8e640 1 -- 172.21.15.112:0/1358465142 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12140b2900 con 0x7f12140d0860 2024-09-21T00:28:45.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:45.894+0000 7f12097fa640 1 -- 172.21.15.112:0/1358465142 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1204041030 con 0x7f12140d0860 2024-09-21T00:28:45.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:45.894+0000 7f12097fa640 1 --2- 172.21.15.112:0/1358465142 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1200078b90 0x7f120007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:45.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:45.894+0000 7f12097fa640 1 -- 172.21.15.112:0/1358465142 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5335+0+0 (secure 0 0 0) 0x7f12040bb380 con 0x7f12140d0860 2024-09-21T00:28:45.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:45.895+0000 7f1218b2a640 1 --2- 172.21.15.112:0/1358465142 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1200078b90 0x7f120007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:45.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:45.895+0000 7f1218b2a640 1 --2- 172.21.15.112:0/1358465142 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1200078b90 0x7f120007b0a0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f121000a490 tx=0x7f121000a420 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:45.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:45.899+0000 7f12097fa640 1 -- 172.21.15.112:0/1358465142 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1204083400 con 0x7f12140d0860 2024-09-21T00:28:46.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:46.147+0000 7f121ad8e640 1 -- 172.21.15.112:0/1358465142 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_sessionmap.TestSessionMap.test_session_reject"]} v 0) -- 0x7f121406ad70 con 0x7f12140d0860 2024-09-21T00:28:46.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:46.392+0000 7f12097fa640 1 -- 172.21.15.112:0/1358465142 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_sessionmap.TestSessionMap.test_session_reject"]}]=0 v184) ==== 142+0+0 (secure 0 0 0) 0x7f1204087e30 con 0x7f12140d0860 2024-09-21T00:28:46.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:46.395+0000 7f121ad8e640 1 -- 172.21.15.112:0/1358465142 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1200078b90 msgr2=0x7f120007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:46.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:46.395+0000 7f121ad8e640 1 --2- 172.21.15.112:0/1358465142 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1200078b90 0x7f120007b0a0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f121000a490 tx=0x7f121000a420 comp rx=0 tx=0).stop 2024-09-21T00:28:46.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:46.395+0000 7f121ad8e640 1 -- 172.21.15.112:0/1358465142 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12140d0860 msgr2=0x7f12140c7a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:46.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:46.395+0000 7f121ad8e640 1 --2- 172.21.15.112:0/1358465142 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12140d0860 0x7f12140c7a50 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f1204034040 tx=0x7f12040039e0 comp rx=0 tx=0).stop 2024-09-21T00:28:46.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:46.395+0000 7f121ad8e640 1 -- 172.21.15.112:0/1358465142 shutdown_connections 2024-09-21T00:28:46.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:46.395+0000 7f121ad8e640 1 --2- 172.21.15.112:0/1358465142 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f12140d2030 0x7f12140c7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:46.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:46.395+0000 7f121ad8e640 1 --2- 172.21.15.112:0/1358465142 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1200078b90 0x7f120007b0a0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:46.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:46.395+0000 7f121ad8e640 1 --2- 172.21.15.112:0/1358465142 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12140d0860 0x7f12140c7a50 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:46.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:46.395+0000 7f121ad8e640 1 --2- 172.21.15.112:0/1358465142 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f12140ccae0 0x7f12141574a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:46.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:46.395+0000 7f121ad8e640 1 -- 172.21.15.112:0/1358465142 >> 172.21.15.112:0/1358465142 conn(0x7f12140b43f0 msgr2=0x7f1214070d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:46.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:46.395+0000 7f121ad8e640 1 -- 172.21.15.112:0/1358465142 shutdown_connections 2024-09-21T00:28:46.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:46.395+0000 7f121ad8e640 1 -- 172.21.15.112:0/1358465142 wait complete. 2024-09-21T00:28:46.408 INFO:tasks.cephfs_test_runner:test_session_reject (tasks.cephfs.test_sessionmap.TestSessionMap) ... skipped 'Requires FUSE client to inject client metadata' 2024-09-21T00:28:46.408 INFO:tasks.cephfs_test_runner:Starting test: test_tell_conn_close (tasks.cephfs.test_sessionmap.TestSessionMap) That when a `tell` command is sent using the python CLI, 2024-09-21T00:28:46.408 INFO:tasks.cephfs_test_runner:test_tell_conn_close (tasks.cephfs.test_sessionmap.TestSessionMap) 2024-09-21T00:28:46.408 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T00:28:46.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:46.518+0000 7f52f900e640 1 -- 172.21.15.112:0/3523676364 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52f40ccbf0 msgr2=0x7f52f40cd000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:46.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:46.518+0000 7f52f900e640 1 --2- 172.21.15.112:0/3523676364 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52f40ccbf0 0x7f52f40cd000 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f52f40cde90 tx=0x7f52e402a410 comp rx=0 tx=0).stop 2024-09-21T00:28:46.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:46.518+0000 7f52f900e640 1 -- 172.21.15.112:0/3523676364 shutdown_connections 2024-09-21T00:28:46.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:46.518+0000 7f52f900e640 1 --2- 172.21.15.112:0/3523676364 >> v2:172.21.15.187:3300/0 conn(0x7f52f40c6a00 0x7f52f40b2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:46.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:46.518+0000 7f52f900e640 1 --2- 172.21.15.112:0/3523676364 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52f40ccbf0 0x7f52f40cd000 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:46.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:46.518+0000 7f52f900e640 1 -- 172.21.15.112:0/3523676364 >> 172.21.15.112:0/3523676364 conn(0x7f52f40b42e0 msgr2=0x7f52f40b4720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:46.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:46.518+0000 7f52f900e640 1 -- 172.21.15.112:0/3523676364 shutdown_connections 2024-09-21T00:28:46.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:46.518+0000 7f52f900e640 1 -- 172.21.15.112:0/3523676364 wait complete. 2024-09-21T00:28:46.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:46.520+0000 7f52f900e640 1 Processor -- start 2024-09-21T00:28:46.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:46.520+0000 7f52f900e640 1 -- start start 2024-09-21T00:28:46.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:46.522+0000 7f52f900e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52f40c5ee0 0x7f52f4158a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:46.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:46.522+0000 7f52f900e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f52f40c6a00 0x7f52f40ab260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:46.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:46.522+0000 7f52f900e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f52f40ccbf0 0x7f52f40ab7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:46.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:46.522+0000 7f52f900e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f52f40b3610 con 0x7f52f40c5ee0 2024-09-21T00:28:46.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:46.522+0000 7f52f900e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f52f40b3490 con 0x7f52f40ccbf0 2024-09-21T00:28:46.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:46.522+0000 7f52f900e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f52f40b3790 con 0x7f52f40c6a00 2024-09-21T00:28:46.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:46.522+0000 7f52f2d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52f40c5ee0 0x7f52f4158a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:46.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:46.522+0000 7f52f2d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52f40c5ee0 0x7f52f4158a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60262/0 (socket says 172.21.15.112:60262) 2024-09-21T00:28:46.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:46.522+0000 7f52f2d9d640 1 -- 172.21.15.112:0/2666225571 learned_addr learned my addr 172.21.15.112:0/2666225571 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:28:46.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:46.522+0000 7f52ebfff640 1 --2- 172.21.15.112:0/2666225571 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f52f40c6a00 0x7f52f40ab260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:46.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:46.522+0000 7f52f359e640 1 --2- 172.21.15.112:0/2666225571 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f52f40ccbf0 0x7f52f40ab7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:46.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:46.522+0000 7f52f2d9d640 1 -- 172.21.15.112:0/2666225571 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f52f40c6a00 msgr2=0x7f52f40ab260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:46.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:46.522+0000 7f52f2d9d640 1 --2- 172.21.15.112:0/2666225571 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f52f40c6a00 0x7f52f40ab260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:46.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:46.522+0000 7f52f2d9d640 1 -- 172.21.15.112:0/2666225571 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f52f40ccbf0 msgr2=0x7f52f40ab7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:46.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:46.522+0000 7f52f2d9d640 1 --2- 172.21.15.112:0/2666225571 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f52f40ccbf0 0x7f52f40ab7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:46.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:46.522+0000 7f52f2d9d640 1 -- 172.21.15.112:0/2666225571 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52f40cc500 con 0x7f52f40c5ee0 2024-09-21T00:28:46.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:46.522+0000 7f52f2d9d640 1 --2- 172.21.15.112:0/2666225571 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52f40c5ee0 0x7f52f4158a90 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f52e4009780 tx=0x7f52e4033d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:46.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:46.522+0000 7f52eb7fe640 1 -- 172.21.15.112:0/2666225571 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52e4031d60 con 0x7f52f40c5ee0 2024-09-21T00:28:46.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:46.522+0000 7f52eb7fe640 1 -- 172.21.15.112:0/2666225571 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f52e4005020 con 0x7f52f40c5ee0 2024-09-21T00:28:46.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:46.522+0000 7f52f900e640 1 -- 172.21.15.112:0/2666225571 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52f40cc7e0 con 0x7f52f40c5ee0 2024-09-21T00:28:46.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:46.522+0000 7f52eb7fe640 1 -- 172.21.15.112:0/2666225571 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52e4005470 con 0x7f52f40c5ee0 2024-09-21T00:28:46.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:46.523+0000 7f52f900e640 1 -- 172.21.15.112:0/2666225571 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52f40ada70 con 0x7f52f40c5ee0 2024-09-21T00:28:46.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:46.524+0000 7f52f900e640 1 -- 172.21.15.112:0/2666225571 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52f40adcd0 con 0x7f52f40c5ee0 2024-09-21T00:28:46.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:46.524+0000 7f52eb7fe640 1 -- 172.21.15.112:0/2666225571 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f52e40047a0 con 0x7f52f40c5ee0 2024-09-21T00:28:46.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:46.525+0000 7f52eb7fe640 1 --2- 172.21.15.112:0/2666225571 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f52d4078b90 0x7f52d407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:46.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:46.525+0000 7f52eb7fe640 1 -- 172.21.15.112:0/2666225571 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5335+0+0 (secure 0 0 0) 0x7f52e40bbe50 con 0x7f52f40c5ee0 2024-09-21T00:28:46.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:46.525+0000 7f52ebfff640 1 --2- 172.21.15.112:0/2666225571 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f52d4078b90 0x7f52d407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:46.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:46.525+0000 7f52ebfff640 1 --2- 172.21.15.112:0/2666225571 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f52d4078b90 0x7f52d407b0a0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f52dc000f60 tx=0x7f52dc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:46.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:46.529+0000 7f52eb7fe640 1 -- 172.21.15.112:0/2666225571 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f52e403f030 con 0x7f52f40c5ee0 2024-09-21T00:28:46.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:46.709+0000 7f52f900e640 1 -- 172.21.15.112:0/2666225571 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f52f40cd000 con 0x7f52f40c5ee0 2024-09-21T00:28:46.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:46.710+0000 7f52eb7fe640 1 -- 172.21.15.112:0/2666225571 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) ==== 74+0+13332 (secure 0 0 0) 0x7f52e4084490 con 0x7f52f40c5ee0 2024-09-21T00:28:46.712 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:28:46.712 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":74,"fsid":"8b34692b-dc65-49b2-a08f-26f707b96ac5","created":"2024-09-21T00:25:22.788490+0000","modified":"2024-09-21T00:28:45.386366+0000","last_up_change":"2024-09-21T00:25:36.161835+0000","last_in_change":"2024-09-21T00:25:26.346500+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T00:25:37.105006+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9c575086-67d2-4de2-b01b-d44dcc21d604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6809","nonce":755760621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6811","nonce":755760621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6815","nonce":755760621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6813","nonce":755760621}]},"public_addr":"172.21.15.112:6809/755760621","cluster_addr":"172.21.15.112:6811/755760621","heartbeat_back_addr":"172.21.15.112:6815/755760621","heartbeat_front_addr":"172.21.15.112:6813/755760621","state":["exists","up"]},{"osd":1,"uuid":"a983c227-a3ce-4bdd-82f1-438ba7e3ecd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6801","nonce":344932592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6803","nonce":344932592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6807","nonce":344932592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6805","nonce":344932592}]},"public_addr":"172.21.15.112:6801/344932592","cluster_addr":"172.21.15.112:6803/344932592","heartbeat_back_addr":"172.21.15.112:6807/344932592","heartbeat_front_addr":"172.21.15.112:6805/344932592","state":["exists","up"]},{"osd":2,"uuid":"1193c1d8-bfff-4a57-8200-5f69bf1ea3af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6825","nonce":407933616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6827","nonce":407933616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6831","nonce":407933616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6829","nonce":407933616}]},"public_addr":"172.21.15.112:6825/407933616","cluster_addr":"172.21.15.112:6827/407933616","heartbeat_back_addr":"172.21.15.112:6831/407933616","heartbeat_front_addr":"172.21.15.112:6829/407933616","state":["exists","up"]},{"osd":3,"uuid":"af04330a-36ad-40cc-a8dd-fa259b221de5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6817","nonce":4120164806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6819","nonce":4120164806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6823","nonce":4120164806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6821","nonce":4120164806}]},"public_addr":"172.21.15.112:6817/4120164806","cluster_addr":"172.21.15.112:6819/4120164806","heartbeat_back_addr":"172.21.15.112:6823/4120164806","heartbeat_front_addr":"172.21.15.112:6821/4120164806","state":["exists","up"]},{"osd":4,"uuid":"5b893111-292c-4c5a-b184-53ad38c7199e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6803","nonce":522077560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6805","nonce":522077560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6809","nonce":522077560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6807","nonce":522077560}]},"public_addr":"172.21.15.187:6803/522077560","cluster_addr":"172.21.15.187:6805/522077560","heartbeat_back_addr":"172.21.15.187:6809/522077560","heartbeat_front_addr":"172.21.15.187:6807/522077560","state":["exists","up"]},{"osd":5,"uuid":"c0b8e011-45fe-4c5e-925e-6be2a7f1868a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6827","nonce":3593144721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6829","nonce":3593144721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6833","nonce":3593144721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6831","nonce":3593144721}]},"public_addr":"172.21.15.187:6827/3593144721","cluster_addr":"172.21.15.187:6829/3593144721","heartbeat_back_addr":"172.21.15.187:6833/3593144721","heartbeat_front_addr":"172.21.15.187:6831/3593144721","state":["exists","up"]},{"osd":6,"uuid":"87d2d650-7489-4593-98a3-a1f433ce8dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6811","nonce":1494387657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6813","nonce":1494387657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6817","nonce":1494387657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6815","nonce":1494387657}]},"public_addr":"172.21.15.187:6811/1494387657","cluster_addr":"172.21.15.187:6813/1494387657","heartbeat_back_addr":"172.21.15.187:6817/1494387657","heartbeat_front_addr":"172.21.15.187:6815/1494387657","state":["exists","up"]},{"osd":7,"uuid":"c6b4f8e2-a1ee-4438-aa37-67ff4b6ef1f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6819","nonce":1624699842}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6821","nonce":1624699842}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6825","nonce":1624699842}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6823","nonce":1624699842}]},"public_addr":"172.21.15.187:6819/1624699842","cluster_addr":"172.21.15.187:6821/1624699842","heartbeat_back_addr":"172.21.15.187:6825/1624699842","heartbeat_front_addr":"172.21.15.187:6823/1624699842","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.873162+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.400506+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.664317+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.159791+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.202630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.113212+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.384786+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.448441+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:6835/1624734538":"2024-09-22T00:28:42.381228+0000","172.21.15.112:6833/1624734538":"2024-09-22T00:28:42.381228+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T00:28:46.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:46.712+0000 7f52f900e640 1 -- 172.21.15.112:0/2666225571 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f52d4078b90 msgr2=0x7f52d407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:46.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:46.712+0000 7f52f900e640 1 --2- 172.21.15.112:0/2666225571 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f52d4078b90 0x7f52d407b0a0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f52dc000f60 tx=0x7f52dc009040 comp rx=0 tx=0).stop 2024-09-21T00:28:46.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:46.712+0000 7f52f900e640 1 -- 172.21.15.112:0/2666225571 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52f40c5ee0 msgr2=0x7f52f4158a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:46.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:46.712+0000 7f52f900e640 1 --2- 172.21.15.112:0/2666225571 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52f40c5ee0 0x7f52f4158a90 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f52e4009780 tx=0x7f52e4033d60 comp rx=0 tx=0).stop 2024-09-21T00:28:46.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:46.712+0000 7f52f900e640 1 -- 172.21.15.112:0/2666225571 shutdown_connections 2024-09-21T00:28:46.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:46.712+0000 7f52f900e640 1 --2- 172.21.15.112:0/2666225571 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f52f40ccbf0 0x7f52f40ab7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:46.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:46.712+0000 7f52f900e640 1 --2- 172.21.15.112:0/2666225571 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f52f40c6a00 0x7f52f40ab260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:46.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:46.712+0000 7f52f900e640 1 --2- 172.21.15.112:0/2666225571 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f52d4078b90 0x7f52d407b0a0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:46.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:46.712+0000 7f52f900e640 1 --2- 172.21.15.112:0/2666225571 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52f40c5ee0 0x7f52f4158a90 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:46.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:46.712+0000 7f52f900e640 1 -- 172.21.15.112:0/2666225571 >> 172.21.15.112:0/2666225571 conn(0x7f52f40b42e0 msgr2=0x7f52f4068d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:46.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:46.712+0000 7f52f900e640 1 -- 172.21.15.112:0/2666225571 shutdown_connections 2024-09-21T00:28:46.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:46.712+0000 7f52f900e640 1 -- 172.21.15.112:0/2666225571 wait complete. 2024-09-21T00:28:46.725 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T00:25:37.105006+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-21T00:28:46.726 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-21T00:28:46.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:46.840+0000 7f8cbcd62640 1 -- 172.21.15.112:0/893785026 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8cb80b0730 msgr2=0x7f8cb8144dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:46.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:46.840+0000 7f8cbcd62640 1 --2- 172.21.15.112:0/893785026 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8cb80b0730 0x7f8cb8144dd0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f8cb0009830 tx=0x7f8cb002a5c0 comp rx=0 tx=0).stop 2024-09-21T00:28:46.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:46.840+0000 7f8cbcd62640 1 -- 172.21.15.112:0/893785026 shutdown_connections 2024-09-21T00:28:46.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:46.840+0000 7f8cbcd62640 1 --2- 172.21.15.112:0/893785026 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8cb80b0730 0x7f8cb8144dd0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:46.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:46.840+0000 7f8cbcd62640 1 -- 172.21.15.112:0/893785026 >> 172.21.15.112:0/893785026 conn(0x7f8cb8073fe0 msgr2=0x7f8cb80743f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:46.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:46.840+0000 7f8cbcd62640 1 -- 172.21.15.112:0/893785026 shutdown_connections 2024-09-21T00:28:46.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:46.840+0000 7f8cbcd62640 1 -- 172.21.15.112:0/893785026 wait complete. 2024-09-21T00:28:46.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:46.842+0000 7f8cbcd62640 1 Processor -- start 2024-09-21T00:28:46.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:46.842+0000 7f8cbcd62640 1 -- start start 2024-09-21T00:28:46.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:46.843+0000 7f8cbcd62640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8cb80b0730 0x7f8cb815dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:46.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:46.843+0000 7f8cbcd62640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8cb815aac0 0x7f8cb81f6f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:46.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:46.843+0000 7f8cbcd62640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8cb806d670 0x7f8cb806da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:46.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:46.843+0000 7f8cbcd62640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8cb80cffd0 con 0x7f8cb806d670 2024-09-21T00:28:46.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:46.843+0000 7f8cbcd62640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8cb80cfe50 con 0x7f8cb80b0730 2024-09-21T00:28:46.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:46.843+0000 7f8cbcd62640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f8cb80d0150 con 0x7f8cb815aac0 2024-09-21T00:28:46.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:46.844+0000 7f8cb6d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8cb806d670 0x7f8cb806da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:46.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:46.844+0000 7f8cb6d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8cb806d670 0x7f8cb806da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60280/0 (socket says 172.21.15.112:60280) 2024-09-21T00:28:46.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:46.844+0000 7f8cb6d9d640 1 -- 172.21.15.112:0/4220560518 learned_addr learned my addr 172.21.15.112:0/4220560518 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:28:46.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:46.844+0000 7f8cb5d9b640 1 --2- 172.21.15.112:0/4220560518 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8cb815aac0 0x7f8cb81f6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:46.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:46.844+0000 7f8cb659c640 1 --2- 172.21.15.112:0/4220560518 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8cb80b0730 0x7f8cb815dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:46.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:46.844+0000 7f8cb6d9d640 1 -- 172.21.15.112:0/4220560518 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8cb815aac0 msgr2=0x7f8cb81f6f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:46.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:46.844+0000 7f8cb6d9d640 1 --2- 172.21.15.112:0/4220560518 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8cb815aac0 0x7f8cb81f6f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:46.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:46.844+0000 7f8cb6d9d640 1 -- 172.21.15.112:0/4220560518 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8cb80b0730 msgr2=0x7f8cb815dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:46.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:46.844+0000 7f8cb6d9d640 1 --2- 172.21.15.112:0/4220560518 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8cb80b0730 0x7f8cb815dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:46.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:46.844+0000 7f8cb6d9d640 1 -- 172.21.15.112:0/4220560518 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cb806dfc0 con 0x7f8cb806d670 2024-09-21T00:28:46.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:46.844+0000 7f8cb659c640 1 --2- 172.21.15.112:0/4220560518 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8cb80b0730 0x7f8cb815dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:28:46.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:46.844+0000 7f8cb6d9d640 1 --2- 172.21.15.112:0/4220560518 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8cb806d670 0x7f8cb806da80 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f8cb0009920 tx=0x7f8cb0033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:46.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:46.844+0000 7f8ca6ffd640 1 -- 172.21.15.112:0/4220560518 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb0031d60 con 0x7f8cb806d670 2024-09-21T00:28:46.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:46.844+0000 7f8ca6ffd640 1 -- 172.21.15.112:0/4220560518 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8cb0033e10 con 0x7f8cb806d670 2024-09-21T00:28:46.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:46.844+0000 7f8cbcd62640 1 -- 172.21.15.112:0/4220560518 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cb81f74d0 con 0x7f8cb806d670 2024-09-21T00:28:46.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:46.844+0000 7f8ca6ffd640 1 -- 172.21.15.112:0/4220560518 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb0005260 con 0x7f8cb806d670 2024-09-21T00:28:46.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:46.844+0000 7f8cbcd62640 1 -- 172.21.15.112:0/4220560518 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cb815e7b0 con 0x7f8cb806d670 2024-09-21T00:28:46.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:46.845+0000 7f8cbcd62640 1 -- 172.21.15.112:0/4220560518 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8cb815e240 con 0x7f8cb806d670 2024-09-21T00:28:46.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:46.846+0000 7f8ca6ffd640 1 -- 172.21.15.112:0/4220560518 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8cb0005400 con 0x7f8cb806d670 2024-09-21T00:28:46.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:46.847+0000 7f8ca6ffd640 1 --2- 172.21.15.112:0/4220560518 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f8c98078bc0 0x7f8c9807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:46.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:46.847+0000 7f8ca6ffd640 1 -- 172.21.15.112:0/4220560518 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5335+0+0 (secure 0 0 0) 0x7f8cb00bbdf0 con 0x7f8cb806d670 2024-09-21T00:28:46.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:46.847+0000 7f8cb659c640 1 --2- 172.21.15.112:0/4220560518 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f8c98078bc0 0x7f8c9807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:46.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:46.847+0000 7f8cb659c640 1 --2- 172.21.15.112:0/4220560518 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f8c98078bc0 0x7f8c9807b0d0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f8ca8007fd0 tx=0x7f8ca8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:46.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:46.851+0000 7f8ca6ffd640 1 -- 172.21.15.112:0/4220560518 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f8cb806dfc0 con 0x7f8cb806d670 2024-09-21T00:28:47.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:47.029+0000 7f8cbcd62640 1 -- 172.21.15.112:0/4220560518 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f8cb80b5ab0 con 0x7f8cb806d670 2024-09-21T00:28:47.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:47.029+0000 7f8ca6ffd640 1 -- 172.21.15.112:0/4220560518 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v74) ==== 93+0+10 (secure 0 0 0) 0x7f8cb003f030 con 0x7f8cb806d670 2024-09-21T00:28:47.031 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2024-09-21T00:28:47.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:47.031+0000 7f8cbcd62640 1 -- 172.21.15.112:0/4220560518 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f8c98078bc0 msgr2=0x7f8c9807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:47.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:47.031+0000 7f8cbcd62640 1 --2- 172.21.15.112:0/4220560518 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f8c98078bc0 0x7f8c9807b0d0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f8ca8007fd0 tx=0x7f8ca8009040 comp rx=0 tx=0).stop 2024-09-21T00:28:47.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:47.031+0000 7f8cbcd62640 1 -- 172.21.15.112:0/4220560518 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8cb806d670 msgr2=0x7f8cb806da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:47.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:47.031+0000 7f8cbcd62640 1 --2- 172.21.15.112:0/4220560518 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8cb806d670 0x7f8cb806da80 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f8cb0009920 tx=0x7f8cb0033c20 comp rx=0 tx=0).stop 2024-09-21T00:28:47.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:47.032+0000 7f8cbcd62640 1 -- 172.21.15.112:0/4220560518 shutdown_connections 2024-09-21T00:28:47.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:47.032+0000 7f8cbcd62640 1 --2- 172.21.15.112:0/4220560518 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f8c98078bc0 0x7f8c9807b0d0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:47.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:47.032+0000 7f8cbcd62640 1 --2- 172.21.15.112:0/4220560518 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8cb806d670 0x7f8cb806da80 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:47.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:47.032+0000 7f8cbcd62640 1 --2- 172.21.15.112:0/4220560518 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8cb815aac0 0x7f8cb81f6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:47.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:47.032+0000 7f8cbcd62640 1 --2- 172.21.15.112:0/4220560518 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8cb80b0730 0x7f8cb815dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:47.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:47.032+0000 7f8cbcd62640 1 -- 172.21.15.112:0/4220560518 >> 172.21.15.112:0/4220560518 conn(0x7f8cb8073fe0 msgr2=0x7f8cb8143570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:47.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:47.032+0000 7f8cbcd62640 1 -- 172.21.15.112:0/4220560518 shutdown_connections 2024-09-21T00:28:47.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:47.032+0000 7f8cbcd62640 1 -- 172.21.15.112:0/4220560518 wait complete. 2024-09-21T00:28:47.044 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_sessionmap.TestSessionMap.test_tell_conn_close' 2024-09-21T00:28:47.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:47.155+0000 7fe28356e640 1 -- 172.21.15.112:0/440390622 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe27c0d02d0 msgr2=0x7fe27c15e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:47.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:47.155+0000 7fe28356e640 1 --2- 172.21.15.112:0/440390622 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe27c0d02d0 0x7fe27c15e780 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fe27c074ec0 tx=0x7fe27802a4d0 comp rx=0 tx=0).stop 2024-09-21T00:28:47.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:47.155+0000 7fe28356e640 1 -- 172.21.15.112:0/440390622 shutdown_connections 2024-09-21T00:28:47.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:47.155+0000 7fe28356e640 1 --2- 172.21.15.112:0/440390622 >> v2:172.21.15.187:3300/0 conn(0x7fe27c0d0920 0x7fe27c0b4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:47.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:47.155+0000 7fe28356e640 1 --2- 172.21.15.112:0/440390622 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe27c0d02d0 0x7fe27c15e780 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:47.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:47.155+0000 7fe28356e640 1 -- 172.21.15.112:0/440390622 >> 172.21.15.112:0/440390622 conn(0x7fe27c0cf200 msgr2=0x7fe27c0cf610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:47.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:47.155+0000 7fe28356e640 1 -- 172.21.15.112:0/440390622 shutdown_connections 2024-09-21T00:28:47.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:47.155+0000 7fe28356e640 1 -- 172.21.15.112:0/440390622 wait complete. 2024-09-21T00:28:47.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:47.157+0000 7fe28356e640 1 Processor -- start 2024-09-21T00:28:47.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:47.157+0000 7fe28356e640 1 -- start start 2024-09-21T00:28:47.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:47.158+0000 7fe28356e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe27c0b4690 0x7fe27c0c6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:47.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:47.158+0000 7fe28356e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe27c0d02d0 0x7fe27c1f5e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:47.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:47.158+0000 7fe28356e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe27c0d0920 0x7fe27c0b0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:47.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:47.158+0000 7fe28356e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe27c0b2550 con 0x7fe27c0d02d0 2024-09-21T00:28:47.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:47.158+0000 7fe28356e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe27c0b23d0 con 0x7fe27c0b4690 2024-09-21T00:28:47.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:47.158+0000 7fe28356e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe27c0b26d0 con 0x7fe27c0d0920 2024-09-21T00:28:47.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:47.159+0000 7fe280b09640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe27c0d02d0 0x7fe27c1f5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:47.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:47.159+0000 7fe280b09640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe27c0d02d0 0x7fe27c1f5e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60304/0 (socket says 172.21.15.112:60304) 2024-09-21T00:28:47.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:47.159+0000 7fe280b09640 1 -- 172.21.15.112:0/2158461374 learned_addr learned my addr 172.21.15.112:0/2158461374 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:28:47.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:47.159+0000 7fe28130a640 1 --2- 172.21.15.112:0/2158461374 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe27c0b4690 0x7fe27c0c6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:47.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:47.159+0000 7fe281b0b640 1 --2- 172.21.15.112:0/2158461374 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe27c0d0920 0x7fe27c0b0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:47.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:47.159+0000 7fe280b09640 1 -- 172.21.15.112:0/2158461374 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe27c0d0920 msgr2=0x7fe27c0b0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:47.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:47.159+0000 7fe280b09640 1 --2- 172.21.15.112:0/2158461374 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe27c0d0920 0x7fe27c0b0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:47.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:47.159+0000 7fe280b09640 1 -- 172.21.15.112:0/2158461374 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe27c0b4690 msgr2=0x7fe27c0c6130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:47.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:47.159+0000 7fe280b09640 1 --2- 172.21.15.112:0/2158461374 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe27c0b4690 0x7fe27c0c6130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:47.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:47.159+0000 7fe280b09640 1 -- 172.21.15.112:0/2158461374 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe27c0c8140 con 0x7fe27c0d02d0 2024-09-21T00:28:47.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:47.159+0000 7fe280b09640 1 --2- 172.21.15.112:0/2158461374 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe27c0d02d0 0x7fe27c1f5e90 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fe26c00d750 tx=0x7fe26c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:47.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:47.159+0000 7fe271ffb640 1 -- 172.21.15.112:0/2158461374 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe26c017070 con 0x7fe27c0d02d0 2024-09-21T00:28:47.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:47.159+0000 7fe271ffb640 1 -- 172.21.15.112:0/2158461374 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe26c004310 con 0x7fe27c0d02d0 2024-09-21T00:28:47.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:47.159+0000 7fe28356e640 1 -- 172.21.15.112:0/2158461374 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe27c0c8420 con 0x7fe27c0d02d0 2024-09-21T00:28:47.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:47.159+0000 7fe271ffb640 1 -- 172.21.15.112:0/2158461374 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe26c00de60 con 0x7fe27c0d02d0 2024-09-21T00:28:47.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:47.159+0000 7fe28356e640 1 -- 172.21.15.112:0/2158461374 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe27c0c89e0 con 0x7fe27c0d02d0 2024-09-21T00:28:47.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:47.160+0000 7fe28356e640 1 -- 172.21.15.112:0/2158461374 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe27c15e780 con 0x7fe27c0d02d0 2024-09-21T00:28:47.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:47.161+0000 7fe271ffb640 1 -- 172.21.15.112:0/2158461374 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe26c009c30 con 0x7fe27c0d02d0 2024-09-21T00:28:47.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:47.162+0000 7fe271ffb640 1 --2- 172.21.15.112:0/2158461374 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fe260078b90 0x7fe26007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:47.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:47.162+0000 7fe271ffb640 1 -- 172.21.15.112:0/2158461374 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5335+0+0 (secure 0 0 0) 0x7fe26c09ae30 con 0x7fe27c0d02d0 2024-09-21T00:28:47.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:47.162+0000 7fe28130a640 1 --2- 172.21.15.112:0/2158461374 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fe260078b90 0x7fe26007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:47.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:47.162+0000 7fe28130a640 1 --2- 172.21.15.112:0/2158461374 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fe260078b90 0x7fe26007b0a0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fe278033000 tx=0x7fe278004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:47.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:47.166+0000 7fe271ffb640 1 -- 172.21.15.112:0/2158461374 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe26c020020 con 0x7fe27c0d02d0 2024-09-21T00:28:47.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:47.415+0000 7fe28356e640 1 -- 172.21.15.112:0/2158461374 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_sessionmap.TestSessionMap.test_tell_conn_close"]} v 0) -- 0x7fe27c15ddb0 con 0x7fe27c0d02d0 2024-09-21T00:28:48.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:48.402+0000 7fe271ffb640 1 -- 172.21.15.112:0/2158461374 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_sessionmap.TestSessionMap.test_tell_conn_close"]}]=0 v186) ==== 146+0+0 (secure 0 0 0) 0x7fe26c062e20 con 0x7fe27c0d02d0 2024-09-21T00:28:48.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:48.405+0000 7fe28356e640 1 -- 172.21.15.112:0/2158461374 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fe260078b90 msgr2=0x7fe26007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:48.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:48.405+0000 7fe28356e640 1 --2- 172.21.15.112:0/2158461374 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fe260078b90 0x7fe26007b0a0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fe278033000 tx=0x7fe278004730 comp rx=0 tx=0).stop 2024-09-21T00:28:48.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:48.405+0000 7fe28356e640 1 -- 172.21.15.112:0/2158461374 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe27c0d02d0 msgr2=0x7fe27c1f5e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:48.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:48.405+0000 7fe28356e640 1 --2- 172.21.15.112:0/2158461374 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe27c0d02d0 0x7fe27c1f5e90 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fe26c00d750 tx=0x7fe26c00dc10 comp rx=0 tx=0).stop 2024-09-21T00:28:48.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:48.405+0000 7fe28356e640 1 -- 172.21.15.112:0/2158461374 shutdown_connections 2024-09-21T00:28:48.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:48.405+0000 7fe28356e640 1 --2- 172.21.15.112:0/2158461374 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe27c0d0920 0x7fe27c0b0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:48.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:48.405+0000 7fe28356e640 1 --2- 172.21.15.112:0/2158461374 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fe260078b90 0x7fe26007b0a0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:48.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:48.405+0000 7fe28356e640 1 --2- 172.21.15.112:0/2158461374 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe27c0d02d0 0x7fe27c1f5e90 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:48.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:48.405+0000 7fe28356e640 1 --2- 172.21.15.112:0/2158461374 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe27c0b4690 0x7fe27c0c6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:48.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:48.405+0000 7fe28356e640 1 -- 172.21.15.112:0/2158461374 >> 172.21.15.112:0/2158461374 conn(0x7fe27c0cf200 msgr2=0x7fe27c071070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:48.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:48.405+0000 7fe28356e640 1 -- 172.21.15.112:0/2158461374 shutdown_connections 2024-09-21T00:28:48.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:48.405+0000 7fe28356e640 1 -- 172.21.15.112:0/2158461374 wait complete. 2024-09-21T00:28:48.420 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-21T00:28:48.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:48.538+0000 7f041ee7e640 1 -- 172.21.15.112:0/905859832 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04000042a0 msgr2=0x7f0400004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:48.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:48.538+0000 7f041ee7e640 1 --2- 172.21.15.112:0/905859832 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04000042a0 0x7f0400004800 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f040c006fc0 tx=0x7f0400005830 comp rx=0 tx=0).stop 2024-09-21T00:28:48.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:48.538+0000 7f041d41b640 1 -- 172.21.15.112:0/905859832 reap_dead start 2024-09-21T00:28:48.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:48.538+0000 7f041ee7e640 1 -- 172.21.15.112:0/905859832 shutdown_connections 2024-09-21T00:28:48.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:48.538+0000 7f041ee7e640 1 -- 172.21.15.112:0/905859832 >> 172.21.15.112:0/905859832 conn(0x7f041808bfe0 msgr2=0x7f041809d890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:48.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:48.539+0000 7f041ee7e640 1 -- 172.21.15.112:0/905859832 shutdown_connections 2024-09-21T00:28:48.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:48.539+0000 7f041ee7e640 1 -- 172.21.15.112:0/905859832 wait complete. 2024-09-21T00:28:48.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:48.540+0000 7f041ee7e640 1 Processor -- start 2024-09-21T00:28:48.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:48.540+0000 7f041ee7e640 1 -- start start 2024-09-21T00:28:48.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:48.542+0000 7f041ee7e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04000042a0 0x7f04181f4f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:48.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:48.542+0000 7f041ee7e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0418153d90 0x7f0418150290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:48.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:48.542+0000 7f041ee7e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04181507d0 0x7f0418150be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:48.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:48.542+0000 7f041ee7e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f04180d20d0 con 0x7f0418153d90 2024-09-21T00:28:48.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:48.542+0000 7f041ee7e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f04180d0c00 con 0x7f04181507d0 2024-09-21T00:28:48.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:48.542+0000 7f041ee7e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f04180d2250 con 0x7f04000042a0 2024-09-21T00:28:48.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:48.542+0000 7f0417fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0418153d90 0x7f0418150290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:48.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:48.542+0000 7f0417fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0418153d90 0x7f0418150290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60322/0 (socket says 172.21.15.112:60322) 2024-09-21T00:28:48.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:48.542+0000 7f0417fff640 1 -- 172.21.15.112:0/2247276461 learned_addr learned my addr 172.21.15.112:0/2247276461 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:28:48.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:48.542+0000 7f041cc1a640 1 --2- 172.21.15.112:0/2247276461 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04000042a0 0x7f04181f4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:48.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:48.542+0000 7f041d41b640 1 --2- 172.21.15.112:0/2247276461 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04181507d0 0x7f0418150be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:48.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:48.542+0000 7f0417fff640 1 -- 172.21.15.112:0/2247276461 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04000042a0 msgr2=0x7f04181f4f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:48.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:48.542+0000 7f0417fff640 1 --2- 172.21.15.112:0/2247276461 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04000042a0 0x7f04181f4f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:48.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:48.542+0000 7f0417fff640 1 -- 172.21.15.112:0/2247276461 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04181507d0 msgr2=0x7f0418150be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:48.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:48.542+0000 7f0417fff640 1 --2- 172.21.15.112:0/2247276461 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04181507d0 0x7f0418150be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:48.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:48.542+0000 7f0417fff640 1 -- 172.21.15.112:0/2247276461 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04180cd160 con 0x7f0418153d90 2024-09-21T00:28:48.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:48.542+0000 7f0417fff640 1 --2- 172.21.15.112:0/2247276461 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0418153d90 0x7f0418150290 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f040c006f60 tx=0x7f040c006c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:48.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:48.543+0000 7f04157fa640 1 -- 172.21.15.112:0/2247276461 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f040c032030 con 0x7f0418153d90 2024-09-21T00:28:48.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:48.543+0000 7f04157fa640 1 -- 172.21.15.112:0/2247276461 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f040c0038e0 con 0x7f0418153d90 2024-09-21T00:28:48.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:48.543+0000 7f041ee7e640 1 -- 172.21.15.112:0/2247276461 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04180cd440 con 0x7f0418153d90 2024-09-21T00:28:48.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:48.543+0000 7f04157fa640 1 -- 172.21.15.112:0/2247276461 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f040c003d10 con 0x7f0418153d90 2024-09-21T00:28:48.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:48.543+0000 7f041ee7e640 1 -- 172.21.15.112:0/2247276461 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04180cda00 con 0x7f0418153d90 2024-09-21T00:28:48.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:48.544+0000 7f041ee7e640 1 -- 172.21.15.112:0/2247276461 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04180d2750 con 0x7f0418153d90 2024-09-21T00:28:48.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:48.544+0000 7f04157fa640 1 -- 172.21.15.112:0/2247276461 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f040c035020 con 0x7f0418153d90 2024-09-21T00:28:48.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:48.545+0000 7f04157fa640 1 --2- 172.21.15.112:0/2247276461 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f0404078bc0 0x7f040407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:48.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:48.545+0000 7f04157fa640 1 -- 172.21.15.112:0/2247276461 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5335+0+0 (secure 0 0 0) 0x7f040c0bb0d0 con 0x7f0418153d90 2024-09-21T00:28:48.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:48.545+0000 7f041cc1a640 1 --2- 172.21.15.112:0/2247276461 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f0404078bc0 0x7f040407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:48.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:48.546+0000 7f041cc1a640 1 --2- 172.21.15.112:0/2247276461 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f0404078bc0 0x7f040407b0d0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f04080064e0 tx=0x7f0408006b10 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:48.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:48.549+0000 7f04157fa640 1 -- 172.21.15.112:0/2247276461 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f040c047030 con 0x7f0418153d90 2024-09-21T00:28:48.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:48.726+0000 7f041ee7e640 1 -- 172.21.15.112:0/2247276461 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f04180b6f90 con 0x7f0418153d90 2024-09-21T00:28:48.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:48.726+0000 7f04157fa640 1 -- 172.21.15.112:0/2247276461 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v41) ==== 59+0+2294 (secure 0 0 0) 0x7f040c083680 con 0x7f0418153d90 2024-09-21T00:28:48.729 INFO:teuthology.orchestra.run.smithi112.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-21T00:28:48.729 INFO:teuthology.orchestra.run.smithi112.stdout:mon advanced mon_op_complaint_time 120 2024-09-21T00:28:48.729 INFO:teuthology.orchestra.run.smithi112.stdout:mgr advanced client_mount_timeout 30 2024-09-21T00:28:48.729 INFO:teuthology.orchestra.run.smithi112.stdout:mgr advanced debug_client 20/20 2024-09-21T00:28:48.729 INFO:teuthology.orchestra.run.smithi112.stdout:mgr advanced debug_mgr 20/20 2024-09-21T00:28:48.729 INFO:teuthology.orchestra.run.smithi112.stdout:mgr advanced debug_ms 1/1 2024-09-21T00:28:48.730 INFO:teuthology.orchestra.run.smithi112.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-21T00:28:48.730 INFO:teuthology.orchestra.run.smithi112.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-21T00:28:48.730 INFO:teuthology.orchestra.run.smithi112.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 5090.401743 2024-09-21T00:28:48.730 INFO:teuthology.orchestra.run.smithi112.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 7876.743127 2024-09-21T00:28:48.730 INFO:teuthology.orchestra.run.smithi112.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 4865.073913 2024-09-21T00:28:48.731 INFO:teuthology.orchestra.run.smithi112.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 5117.206240 2024-09-21T00:28:48.731 INFO:teuthology.orchestra.run.smithi112.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 4329.674604 2024-09-21T00:28:48.731 INFO:teuthology.orchestra.run.smithi112.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4322.652151 2024-09-21T00:28:48.731 INFO:teuthology.orchestra.run.smithi112.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 4244.674337 2024-09-21T00:28:48.731 INFO:teuthology.orchestra.run.smithi112.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 4472.316624 2024-09-21T00:28:48.731 INFO:teuthology.orchestra.run.smithi112.stdout:mds advanced debug_mds 20/20 2024-09-21T00:28:48.731 INFO:teuthology.orchestra.run.smithi112.stdout:mds advanced debug_mds_balancer 20/20 2024-09-21T00:28:48.732 INFO:teuthology.orchestra.run.smithi112.stdout:mds advanced debug_ms 2/2 2024-09-21T00:28:48.732 INFO:teuthology.orchestra.run.smithi112.stdout:mds dev mds_debug_frag true 2024-09-21T00:28:48.732 INFO:teuthology.orchestra.run.smithi112.stdout:mds dev mds_debug_scatterstat true 2024-09-21T00:28:48.732 INFO:teuthology.orchestra.run.smithi112.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-21T00:28:48.732 INFO:teuthology.orchestra.run.smithi112.stdout:mds dev mds_verify_scatter true 2024-09-21T00:28:48.732 INFO:teuthology.orchestra.run.smithi112.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-21T00:28:48.732 INFO:teuthology.orchestra.run.smithi112.stdout:mds advanced rados_mon_op_timeout 900 2024-09-21T00:28:48.733 INFO:teuthology.orchestra.run.smithi112.stdout:mds advanced rados_osd_op_timeout 900 2024-09-21T00:28:48.733 INFO:teuthology.orchestra.run.smithi112.stdout:client advanced client_mount_timeout 600 2024-09-21T00:28:48.733 INFO:teuthology.orchestra.run.smithi112.stdout:client advanced debug_client 20/20 2024-09-21T00:28:48.733 INFO:teuthology.orchestra.run.smithi112.stdout:client advanced debug_ms 1/1 2024-09-21T00:28:48.733 INFO:teuthology.orchestra.run.smithi112.stdout:client advanced rados_mon_op_timeout 900 2024-09-21T00:28:48.733 INFO:teuthology.orchestra.run.smithi112.stdout:client advanced rados_osd_op_timeout 900 2024-09-21T00:28:48.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:48.729+0000 7f041ee7e640 1 -- 172.21.15.112:0/2247276461 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f0404078bc0 msgr2=0x7f040407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:48.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:48.729+0000 7f041ee7e640 1 --2- 172.21.15.112:0/2247276461 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f0404078bc0 0x7f040407b0d0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f04080064e0 tx=0x7f0408006b10 comp rx=0 tx=0).stop 2024-09-21T00:28:48.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:48.729+0000 7f041ee7e640 1 -- 172.21.15.112:0/2247276461 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0418153d90 msgr2=0x7f0418150290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:48.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:48.729+0000 7f041ee7e640 1 --2- 172.21.15.112:0/2247276461 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0418153d90 0x7f0418150290 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f040c006f60 tx=0x7f040c006c20 comp rx=0 tx=0).stop 2024-09-21T00:28:48.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:48.729+0000 7f041ee7e640 1 -- 172.21.15.112:0/2247276461 shutdown_connections 2024-09-21T00:28:48.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:48.729+0000 7f041ee7e640 1 --2- 172.21.15.112:0/2247276461 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04181507d0 0x7f0418150be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:48.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:48.729+0000 7f041ee7e640 1 --2- 172.21.15.112:0/2247276461 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f0404078bc0 0x7f040407b0d0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:48.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:48.729+0000 7f041ee7e640 1 --2- 172.21.15.112:0/2247276461 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0418153d90 0x7f0418150290 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:48.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:48.729+0000 7f041ee7e640 1 --2- 172.21.15.112:0/2247276461 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04000042a0 0x7f04181f4f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:48.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:48.729+0000 7f041ee7e640 1 -- 172.21.15.112:0/2247276461 >> 172.21.15.112:0/2247276461 conn(0x7f041808bfe0 msgr2=0x7f04180b1310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:48.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:48.729+0000 7f041ee7e640 1 -- 172.21.15.112:0/2247276461 shutdown_connections 2024-09-21T00:28:48.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:48.729+0000 7f041ee7e640 1 -- 172.21.15.112:0/2247276461 wait complete. 2024-09-21T00:28:48.742 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-21T00:28:48.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:48.856+0000 7f556733a640 1 -- 172.21.15.112:0/3105284248 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f556015d870 msgr2=0x7f55600d0860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:48.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:48.856+0000 7f556733a640 1 --2- 172.21.15.112:0/3105284248 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f556015d870 0x7f55600d0860 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f5550009830 tx=0x7f555002a5c0 comp rx=0 tx=0).stop 2024-09-21T00:28:48.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:48.856+0000 7f556733a640 1 -- 172.21.15.112:0/3105284248 shutdown_connections 2024-09-21T00:28:48.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:48.856+0000 7f556733a640 1 --2- 172.21.15.112:0/3105284248 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f556015d870 0x7f55600d0860 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:48.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:48.856+0000 7f556733a640 1 -- 172.21.15.112:0/3105284248 >> 172.21.15.112:0/3105284248 conn(0x7f55600b43f0 msgr2=0x7f55600b4830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:48.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:48.856+0000 7f556733a640 1 -- 172.21.15.112:0/3105284248 shutdown_connections 2024-09-21T00:28:48.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:48.856+0000 7f556733a640 1 -- 172.21.15.112:0/3105284248 wait complete. 2024-09-21T00:28:48.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:48.857+0000 7f556733a640 1 Processor -- start 2024-09-21T00:28:48.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:48.858+0000 7f556733a640 1 -- start start 2024-09-21T00:28:48.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:48.859+0000 7f556733a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f55600ccae0 0x7f5560157300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:48.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:48.859+0000 7f556733a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55600d2030 0x7f5560071c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:48.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:48.859+0000 7f556733a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f556015d870 0x7f55600721c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:48.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:48.859+0000 7f556733a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5560167c20 con 0x7f55600d2030 2024-09-21T00:28:48.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:48.859+0000 7f556733a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5560167aa0 con 0x7f556015d870 2024-09-21T00:28:48.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:48.859+0000 7f556733a640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f5560167da0 con 0x7f55600ccae0 2024-09-21T00:28:48.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:48.859+0000 7f55648d5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55600d2030 0x7f5560071c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:48.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:48.859+0000 7f55648d5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55600d2030 0x7f5560071c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60332/0 (socket says 172.21.15.112:60332) 2024-09-21T00:28:48.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:48.859+0000 7f55648d5640 1 -- 172.21.15.112:0/2589845571 learned_addr learned my addr 172.21.15.112:0/2589845571 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:28:48.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:48.859+0000 7f55650d6640 1 --2- 172.21.15.112:0/2589845571 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f55600ccae0 0x7f5560157300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:48.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:48.859+0000 7f55658d7640 1 --2- 172.21.15.112:0/2589845571 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f556015d870 0x7f55600721c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:48.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:48.859+0000 7f55648d5640 1 -- 172.21.15.112:0/2589845571 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f55600ccae0 msgr2=0x7f5560157300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:48.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:48.859+0000 7f55648d5640 1 --2- 172.21.15.112:0/2589845571 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f55600ccae0 0x7f5560157300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:48.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:48.859+0000 7f55648d5640 1 -- 172.21.15.112:0/2589845571 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f556015d870 msgr2=0x7f55600721c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:48.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:48.859+0000 7f55648d5640 1 --2- 172.21.15.112:0/2589845571 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f556015d870 0x7f55600721c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:48.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:48.859+0000 7f55648d5640 1 -- 172.21.15.112:0/2589845571 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55600c54b0 con 0x7f55600d2030 2024-09-21T00:28:48.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:48.859+0000 7f55650d6640 1 --2- 172.21.15.112:0/2589845571 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f55600ccae0 0x7f5560157300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:28:48.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:48.859+0000 7f55648d5640 1 --2- 172.21.15.112:0/2589845571 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55600d2030 0x7f5560071c80 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f5550009950 tx=0x7f5550033ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:48.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:48.860+0000 7f5555ffb640 1 -- 172.21.15.112:0/2589845571 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5550031d60 con 0x7f55600d2030 2024-09-21T00:28:48.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:48.860+0000 7f5555ffb640 1 -- 172.21.15.112:0/2589845571 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5550005020 con 0x7f55600d2030 2024-09-21T00:28:48.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:48.860+0000 7f556733a640 1 -- 172.21.15.112:0/2589845571 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55600b5750 con 0x7f55600d2030 2024-09-21T00:28:48.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:48.860+0000 7f5555ffb640 1 -- 172.21.15.112:0/2589845571 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5550005470 con 0x7f55600d2030 2024-09-21T00:28:48.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:48.860+0000 7f556733a640 1 -- 172.21.15.112:0/2589845571 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f556014bbf0 con 0x7f55600d2030 2024-09-21T00:28:48.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:48.861+0000 7f556733a640 1 -- 172.21.15.112:0/2589845571 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55600b5c70 con 0x7f55600d2030 2024-09-21T00:28:48.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:48.862+0000 7f5555ffb640 1 -- 172.21.15.112:0/2589845571 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f55500047a0 con 0x7f55600d2030 2024-09-21T00:28:48.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:48.862+0000 7f5555ffb640 1 --2- 172.21.15.112:0/2589845571 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f5544078bc0 0x7f554407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:48.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:48.862+0000 7f5555ffb640 1 -- 172.21.15.112:0/2589845571 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5335+0+0 (secure 0 0 0) 0x7f55500bbeb0 con 0x7f55600d2030 2024-09-21T00:28:48.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:48.862+0000 7f55650d6640 1 --2- 172.21.15.112:0/2589845571 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f5544078bc0 0x7f554407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:48.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:48.863+0000 7f55650d6640 1 --2- 172.21.15.112:0/2589845571 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f5544078bc0 0x7f554407b0d0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f555c0016a0 tx=0x7f555c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:48.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:48.867+0000 7f5555ffb640 1 -- 172.21.15.112:0/2589845571 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f55600b5c70 con 0x7f55600d2030 2024-09-21T00:28:49.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:49.041+0000 7f556733a640 1 -- 172.21.15.112:0/2589845571 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f55600d0860 con 0x7f55600d2030 2024-09-21T00:28:49.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:49.046+0000 7f5555ffb640 1 -- 172.21.15.112:0/2589845571 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v42)=0 v42) ==== 122+0+0 (secure 0 0 0) 0x7f555003f030 con 0x7f55600d2030 2024-09-21T00:28:49.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:49.048+0000 7f556733a640 1 -- 172.21.15.112:0/2589845571 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f5544078bc0 msgr2=0x7f554407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:49.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:49.048+0000 7f556733a640 1 --2- 172.21.15.112:0/2589845571 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f5544078bc0 0x7f554407b0d0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f555c0016a0 tx=0x7f555c009040 comp rx=0 tx=0).stop 2024-09-21T00:28:49.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:49.048+0000 7f556733a640 1 -- 172.21.15.112:0/2589845571 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55600d2030 msgr2=0x7f5560071c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:49.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:49.048+0000 7f556733a640 1 --2- 172.21.15.112:0/2589845571 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55600d2030 0x7f5560071c80 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f5550009950 tx=0x7f5550033ce0 comp rx=0 tx=0).stop 2024-09-21T00:28:49.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:49.048+0000 7f556733a640 1 -- 172.21.15.112:0/2589845571 shutdown_connections 2024-09-21T00:28:49.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:49.048+0000 7f556733a640 1 --2- 172.21.15.112:0/2589845571 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f556015d870 0x7f55600721c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:49.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:49.048+0000 7f556733a640 1 --2- 172.21.15.112:0/2589845571 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f5544078bc0 0x7f554407b0d0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:49.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:49.048+0000 7f556733a640 1 --2- 172.21.15.112:0/2589845571 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55600d2030 0x7f5560071c80 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:49.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:49.048+0000 7f556733a640 1 --2- 172.21.15.112:0/2589845571 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f55600ccae0 0x7f5560157300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:49.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:49.048+0000 7f556733a640 1 -- 172.21.15.112:0/2589845571 >> 172.21.15.112:0/2589845571 conn(0x7f55600b43f0 msgr2=0x7f5560152c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:49.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:49.048+0000 7f556733a640 1 -- 172.21.15.112:0/2589845571 shutdown_connections 2024-09-21T00:28:49.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:49.048+0000 7f556733a640 1 -- 172.21.15.112:0/2589845571 wait complete. 2024-09-21T00:28:49.062 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T00:28:49.062 DEBUG:teuthology.orchestra.run.smithi112:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T00:28:49.106 DEBUG:teuthology.orchestra.run.smithi187:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T00:28:49.148 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T00:28:49.148 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T00:28:49.148 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:28:49.164 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056010,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1439 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248716k,nr_inodes=812179,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T00:28:49.164 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-09-21T00:28:49.164 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:28:49.205 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056013,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497440k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16835 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248720k,nr_inodes=812180,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497440k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T00:28:49.205 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T00:28:49.205 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:28:49.220 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056010,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1439 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248716k,nr_inodes=812179,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T00:28:49.221 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-09-21T00:28:49.221 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:28:49.262 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056013,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497440k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16835 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248720k,nr_inodes=812180,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497440k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T00:28:49.262 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-21T00:28:49.262 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T00:28:49.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:49.372+0000 7ff47ccd3640 1 -- 172.21.15.112:0/1359719724 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff4780d0860 msgr2=0x7ff47815a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:49.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:49.372+0000 7ff47ccd3640 1 --2- 172.21.15.112:0/1359719724 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff4780d0860 0x7ff47815a0c0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7ff46c009830 tx=0x7ff46c02a510 comp rx=0 tx=0).stop 2024-09-21T00:28:49.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:49.372+0000 7ff47ccd3640 1 -- 172.21.15.112:0/1359719724 shutdown_connections 2024-09-21T00:28:49.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:49.372+0000 7ff47ccd3640 1 --2- 172.21.15.112:0/1359719724 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff4780d0860 0x7ff47815a0c0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:49.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:49.372+0000 7ff47ccd3640 1 --2- 172.21.15.112:0/1359719724 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff4780d2030 0x7ff47815d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:49.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:49.372+0000 7ff47ccd3640 1 -- 172.21.15.112:0/1359719724 >> 172.21.15.112:0/1359719724 conn(0x7ff4780b43f0 msgr2=0x7ff4780b4830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:49.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:49.372+0000 7ff47ccd3640 1 -- 172.21.15.112:0/1359719724 shutdown_connections 2024-09-21T00:28:49.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:49.372+0000 7ff47ccd3640 1 -- 172.21.15.112:0/1359719724 wait complete. 2024-09-21T00:28:49.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:49.374+0000 7ff47ccd3640 1 Processor -- start 2024-09-21T00:28:49.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:49.374+0000 7ff47ccd3640 1 -- start start 2024-09-21T00:28:49.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:49.376+0000 7ff47ccd3640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff4780ccae0 0x7ff4780b0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:49.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:49.376+0000 7ff47ccd3640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff4780d0860 0x7ff4780b1110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:49.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:49.376+0000 7ff47ccd3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4780d2030 0x7ff47814d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:49.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:49.376+0000 7ff47ccd3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff4780b22d0 con 0x7ff4780d2030 2024-09-21T00:28:49.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:49.376+0000 7ff47ccd3640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff4780b2150 con 0x7ff4780ccae0 2024-09-21T00:28:49.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:49.376+0000 7ff47ccd3640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff4780b2450 con 0x7ff4780d0860 2024-09-21T00:28:49.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:49.376+0000 7ff476d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4780d2030 0x7ff47814d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:49.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:49.376+0000 7ff476d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4780d2030 0x7ff47814d400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60348/0 (socket says 172.21.15.112:60348) 2024-09-21T00:28:49.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:49.376+0000 7ff476d9d640 1 -- 172.21.15.112:0/2028905470 learned_addr learned my addr 172.21.15.112:0/2028905470 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:28:49.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:49.376+0000 7ff475d9b640 1 --2- 172.21.15.112:0/2028905470 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff4780d0860 0x7ff4780b1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:49.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:49.376+0000 7ff47659c640 1 --2- 172.21.15.112:0/2028905470 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff4780ccae0 0x7ff4780b0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:49.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:49.376+0000 7ff476d9d640 1 -- 172.21.15.112:0/2028905470 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff4780d0860 msgr2=0x7ff4780b1110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:49.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:49.376+0000 7ff476d9d640 1 --2- 172.21.15.112:0/2028905470 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff4780d0860 0x7ff4780b1110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:49.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:49.376+0000 7ff476d9d640 1 -- 172.21.15.112:0/2028905470 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff4780ccae0 msgr2=0x7ff4780b0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:49.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:49.376+0000 7ff476d9d640 1 --2- 172.21.15.112:0/2028905470 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff4780ccae0 0x7ff4780b0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:49.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:49.376+0000 7ff476d9d640 1 -- 172.21.15.112:0/2028905470 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4781f4860 con 0x7ff4780d2030 2024-09-21T00:28:49.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:49.376+0000 7ff475d9b640 1 --2- 172.21.15.112:0/2028905470 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff4780d0860 0x7ff4780b1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:28:49.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:49.376+0000 7ff476d9d640 1 --2- 172.21.15.112:0/2028905470 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4780d2030 0x7ff47814d400 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7ff47000c6c0 tx=0x7ff47000cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:49.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:49.376+0000 7ff466ffd640 1 -- 172.21.15.112:0/2028905470 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff470017070 con 0x7ff4780d2030 2024-09-21T00:28:49.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:49.376+0000 7ff466ffd640 1 -- 172.21.15.112:0/2028905470 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff470003c60 con 0x7ff4780d2030 2024-09-21T00:28:49.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:49.376+0000 7ff47ccd3640 1 -- 172.21.15.112:0/2028905470 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4781f4b40 con 0x7ff4780d2030 2024-09-21T00:28:49.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:49.377+0000 7ff466ffd640 1 -- 172.21.15.112:0/2028905470 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4700052d0 con 0x7ff4780d2030 2024-09-21T00:28:49.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:49.377+0000 7ff47ccd3640 1 -- 172.21.15.112:0/2028905470 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4780c72f0 con 0x7ff4780d2030 2024-09-21T00:28:49.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:49.377+0000 7ff47ccd3640 1 -- 172.21.15.112:0/2028905470 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4780b2900 con 0x7ff4780d2030 2024-09-21T00:28:49.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:49.378+0000 7ff466ffd640 1 -- 172.21.15.112:0/2028905470 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff47001a070 con 0x7ff4780d2030 2024-09-21T00:28:49.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:49.379+0000 7ff466ffd640 1 --2- 172.21.15.112:0/2028905470 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff458078bc0 0x7ff45807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:49.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:49.379+0000 7ff466ffd640 1 -- 172.21.15.112:0/2028905470 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5335+0+0 (secure 0 0 0) 0x7ff47009a1e0 con 0x7ff4780d2030 2024-09-21T00:28:49.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:49.379+0000 7ff47659c640 1 --2- 172.21.15.112:0/2028905470 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff458078bc0 0x7ff45807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:49.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:49.379+0000 7ff47659c640 1 --2- 172.21.15.112:0/2028905470 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff458078bc0 0x7ff45807b0d0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7ff468009ac0 tx=0x7ff468005f90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:49.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:49.383+0000 7ff466ffd640 1 -- 172.21.15.112:0/2028905470 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ff4700627a0 con 0x7ff4780d2030 2024-09-21T00:28:49.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:49.597+0000 7ff47ccd3640 1 -- 172.21.15.112:0/2028905470 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff47815eb70 con 0x7ff4780d2030 2024-09-21T00:28:49.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:49.598+0000 7ff466ffd640 1 -- 172.21.15.112:0/2028905470 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) ==== 94+0+3923 (secure 0 0 0) 0x7ff4700620a0 con 0x7ff4780d2030 2024-09-21T00:28:49.600 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:28:49.600 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":49,"btime":"2024-09-21T00:28:43:382916+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5435,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6837/3039805497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6835","nonce":3039805497},{"type":"v1","addr":"172.21.15.187:6837","nonce":3039805497}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43},{"gid":5719,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6836/2736940636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":2736940636},{"type":"v1","addr":"172.21.15.187:6836","nonce":2736940636}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43},{"gid":6162,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6834/2295407428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":2295407428},{"type":"v1","addr":"172.21.15.112:6834","nonce":2295407428}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":6393,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/1123517411","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":1123517411},{"type":"v1","addr":"172.21.15.112:6835","nonce":1123517411}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49}],"filesystems":[]} 2024-09-21T00:28:49.600 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 49 2024-09-21T00:28:49.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:49.600+0000 7ff47ccd3640 1 -- 172.21.15.112:0/2028905470 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff458078bc0 msgr2=0x7ff45807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:49.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:49.600+0000 7ff47ccd3640 1 --2- 172.21.15.112:0/2028905470 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff458078bc0 0x7ff45807b0d0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7ff468009ac0 tx=0x7ff468005f90 comp rx=0 tx=0).stop 2024-09-21T00:28:49.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:49.600+0000 7ff47ccd3640 1 -- 172.21.15.112:0/2028905470 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4780d2030 msgr2=0x7ff47814d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:49.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:49.600+0000 7ff47ccd3640 1 --2- 172.21.15.112:0/2028905470 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4780d2030 0x7ff47814d400 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7ff47000c6c0 tx=0x7ff47000cb80 comp rx=0 tx=0).stop 2024-09-21T00:28:49.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:49.600+0000 7ff47ccd3640 1 -- 172.21.15.112:0/2028905470 shutdown_connections 2024-09-21T00:28:49.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:49.600+0000 7ff47ccd3640 1 --2- 172.21.15.112:0/2028905470 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff458078bc0 0x7ff45807b0d0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:49.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:49.600+0000 7ff47ccd3640 1 --2- 172.21.15.112:0/2028905470 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4780d2030 0x7ff47814d400 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:49.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:49.600+0000 7ff47ccd3640 1 --2- 172.21.15.112:0/2028905470 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff4780d0860 0x7ff4780b1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:49.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:49.600+0000 7ff47ccd3640 1 --2- 172.21.15.112:0/2028905470 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff4780ccae0 0x7ff4780b0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:49.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:49.600+0000 7ff47ccd3640 1 -- 172.21.15.112:0/2028905470 >> 172.21.15.112:0/2028905470 conn(0x7ff4780b43f0 msgr2=0x7ff4780ccef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:49.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:49.600+0000 7ff47ccd3640 1 -- 172.21.15.112:0/2028905470 shutdown_connections 2024-09-21T00:28:49.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:49.600+0000 7ff47ccd3640 1 -- 172.21.15.112:0/2028905470 wait complete. 2024-09-21T00:28:49.613 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-21T00:28:49.614 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-21T00:28:49.614 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-21T00:28:49.614 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T00:28:49.614 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-21T00:28:49.614 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-21T00:28:49.614 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-21T00:28:49.614 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T00:28:49.615 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-21T00:28:49.615 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-21T00:28:49.615 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-21T00:28:49.615 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T00:28:49.615 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-21T00:28:49.615 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-21T00:28:49.615 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-21T00:28:49.616 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T00:28:49.624 INFO:tasks.ceph.mds.c:Stopped 2024-09-21T00:28:49.624 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-21T00:28:49.625 INFO:tasks.ceph.mds.a:Stopped 2024-09-21T00:28:49.625 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-21T00:28:49.628 INFO:tasks.ceph.mds.b:Stopped 2024-09-21T00:28:49.628 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-21T00:28:49.628 INFO:tasks.ceph.mds.d:Stopped 2024-09-21T00:28:49.629 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-21T00:28:49.667 INFO:tasks.ceph.mds.c:Started 2024-09-21T00:28:49.668 DEBUG:teuthology.parallel:result is None 2024-09-21T00:28:49.669 INFO:tasks.ceph.mds.a:Started 2024-09-21T00:28:49.669 DEBUG:teuthology.parallel:result is None 2024-09-21T00:28:49.671 INFO:tasks.ceph.mds.b:Started 2024-09-21T00:28:49.671 DEBUG:teuthology.parallel:result is None 2024-09-21T00:28:49.672 INFO:tasks.ceph.mds.d:Started 2024-09-21T00:28:49.672 DEBUG:teuthology.parallel:result is None 2024-09-21T00:28:49.672 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-21T00:28:49.755 INFO:tasks.ceph.mds.a.smithi112.stdout:starting mds.a at 2024-09-21T00:28:49.755 INFO:tasks.ceph.mds.c.smithi112.stdout:starting mds.c at 2024-09-21T00:28:49.769 INFO:tasks.ceph.mds.b.smithi187.stdout:starting mds.b at 2024-09-21T00:28:49.769 INFO:tasks.ceph.mds.d.smithi187.stdout:starting mds.d at 2024-09-21T00:28:49.769 INFO:tasks.ceph.mds.d.smithi187.stderr:2024-09-21T00:28:49.768+0000 7f54a6488640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2024-09-21T00:28:49.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:49.790+0000 7f18076d5640 1 -- 172.21.15.112:0/3901416580 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f180015c3c0 msgr2=0x7f18000b3260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:49.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:49.790+0000 7f18076d5640 1 --2- 172.21.15.112:0/3901416580 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f180015c3c0 0x7f18000b3260 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f18000cb610 tx=0x7f17fc02a4b0 comp rx=0 tx=0).stop 2024-09-21T00:28:49.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:49.790+0000 7f18076d5640 1 -- 172.21.15.112:0/3901416580 shutdown_connections 2024-09-21T00:28:49.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:49.790+0000 7f18076d5640 1 --2- 172.21.15.112:0/3901416580 >> v2:172.21.15.187:3300/0 conn(0x7f18000b3bb0 0x7f18000b6570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:49.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:49.790+0000 7f18076d5640 1 --2- 172.21.15.112:0/3901416580 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f18000b37a0 0x7f18000b6030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:49.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:49.790+0000 7f18076d5640 1 --2- 172.21.15.112:0/3901416580 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f180015c3c0 0x7f18000b3260 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:49.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:49.790+0000 7f18076d5640 1 -- 172.21.15.112:0/3901416580 >> 172.21.15.112:0/3901416580 conn(0x7f180008dcb0 msgr2=0x7f180009d8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:49.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:49.790+0000 7f18076d5640 1 -- 172.21.15.112:0/3901416580 shutdown_connections 2024-09-21T00:28:49.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:49.790+0000 7f18076d5640 1 -- 172.21.15.112:0/3901416580 wait complete. 2024-09-21T00:28:49.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:49.792+0000 7f18076d5640 1 Processor -- start 2024-09-21T00:28:49.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:49.792+0000 7f18076d5640 1 -- start start 2024-09-21T00:28:49.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:49.793+0000 7f18076d5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f18000b37a0 0x7f1800151a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:49.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:49.793+0000 7f18076d5640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f18000b3bb0 0x7f18000b5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:49.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:49.793+0000 7f18076d5640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f180015c3c0 0x7f18000b55a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:49.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:49.793+0000 7f18076d5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f18000b6ab0 con 0x7f18000b37a0 2024-09-21T00:28:49.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:49.793+0000 7f18076d5640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f1800163390 con 0x7f180015c3c0 2024-09-21T00:28:49.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:49.793+0000 7f18076d5640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f18000b6c30 con 0x7f18000b3bb0 2024-09-21T00:28:49.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:49.793+0000 7f1805471640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f18000b37a0 0x7f1800151a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:49.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:49.793+0000 7f1805471640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f18000b37a0 0x7f1800151a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60416/0 (socket says 172.21.15.112:60416) 2024-09-21T00:28:49.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:49.793+0000 7f1805471640 1 -- 172.21.15.112:0/3990581842 learned_addr learned my addr 172.21.15.112:0/3990581842 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:28:49.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:49.793+0000 7f1804c70640 1 --2- 172.21.15.112:0/3990581842 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f18000b3bb0 0x7f18000b5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:49.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:49.793+0000 7f1805c72640 1 --2- 172.21.15.112:0/3990581842 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f180015c3c0 0x7f18000b55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:49.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:49.793+0000 7f1805471640 1 -- 172.21.15.112:0/3990581842 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f18000b3bb0 msgr2=0x7f18000b5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:49.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:49.793+0000 7f1805471640 1 --2- 172.21.15.112:0/3990581842 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f18000b3bb0 0x7f18000b5060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:49.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:49.793+0000 7f1805471640 1 -- 172.21.15.112:0/3990581842 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f180015c3c0 msgr2=0x7f18000b55a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:49.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:49.793+0000 7f1805471640 1 --2- 172.21.15.112:0/3990581842 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f180015c3c0 0x7f18000b55a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:49.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:49.793+0000 7f1805471640 1 -- 172.21.15.112:0/3990581842 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1800151fd0 con 0x7f18000b37a0 2024-09-21T00:28:49.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:49.793+0000 7f1804c70640 1 --2- 172.21.15.112:0/3990581842 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f18000b3bb0 0x7f18000b5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:28:49.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:49.793+0000 7f1805c72640 1 --2- 172.21.15.112:0/3990581842 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f180015c3c0 0x7f18000b55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:28:49.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:49.794+0000 7f1805471640 1 --2- 172.21.15.112:0/3990581842 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f18000b37a0 0x7f1800151a90 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f17fc0099d0 tx=0x7f17fc0052f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:49.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:49.794+0000 7f17f5ffb640 1 -- 172.21.15.112:0/3990581842 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17fc005cf0 con 0x7f18000b37a0 2024-09-21T00:28:49.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:49.794+0000 7f17f5ffb640 1 -- 172.21.15.112:0/3990581842 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f17fc031c40 con 0x7f18000b37a0 2024-09-21T00:28:49.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:49.794+0000 7f18076d5640 1 -- 172.21.15.112:0/3990581842 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f180006d5c0 con 0x7f18000b37a0 2024-09-21T00:28:49.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:49.794+0000 7f17f5ffb640 1 -- 172.21.15.112:0/3990581842 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17fc033330 con 0x7f18000b37a0 2024-09-21T00:28:49.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:49.794+0000 7f18076d5640 1 -- 172.21.15.112:0/3990581842 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f180006db80 con 0x7f18000b37a0 2024-09-21T00:28:49.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:49.795+0000 7f18076d5640 1 -- 172.21.15.112:0/3990581842 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18000b3260 con 0x7f18000b37a0 2024-09-21T00:28:49.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:49.796+0000 7f17f5ffb640 1 -- 172.21.15.112:0/3990581842 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f17fc0334d0 con 0x7f18000b37a0 2024-09-21T00:28:49.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:49.796+0000 7f17f5ffb640 1 --2- 172.21.15.112:0/3990581842 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f17e4078bc0 0x7f17e407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:49.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:49.797+0000 7f17f5ffb640 1 -- 172.21.15.112:0/3990581842 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5335+0+0 (secure 0 0 0) 0x7f17fc0bbee0 con 0x7f18000b37a0 2024-09-21T00:28:49.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:49.797+0000 7f1804c70640 1 --2- 172.21.15.112:0/3990581842 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f17e4078bc0 0x7f17e407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:49.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:49.797+0000 7f1804c70640 1 --2- 172.21.15.112:0/3990581842 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f17e4078bc0 0x7f17e407b0d0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f180015ab00 tx=0x7f17f0005e50 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:49.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:49.801+0000 7f17f5ffb640 1 -- 172.21.15.112:0/3990581842 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f17fc03f030 con 0x7f18000b37a0 2024-09-21T00:28:49.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:49.991+0000 7f18076d5640 1 -- 172.21.15.112:0/3990581842 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f180015eb00 con 0x7f18000b37a0 2024-09-21T00:28:51.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.068+0000 7f17f5ffb640 1 -- 172.21.15.112:0/3990581842 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v79) ==== 97+0+0 (secure 0 0 0) 0x7f17fc0845b0 con 0x7f18000b37a0 2024-09-21T00:28:51.069 INFO:teuthology.orchestra.run.smithi112.stderr: removed all blocklist entries 2024-09-21T00:28:51.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.072+0000 7f18076d5640 1 -- 172.21.15.112:0/3990581842 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f17e4078bc0 msgr2=0x7f17e407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:51.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.072+0000 7f18076d5640 1 --2- 172.21.15.112:0/3990581842 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f17e4078bc0 0x7f17e407b0d0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f180015ab00 tx=0x7f17f0005e50 comp rx=0 tx=0).stop 2024-09-21T00:28:51.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.072+0000 7f18076d5640 1 -- 172.21.15.112:0/3990581842 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f18000b37a0 msgr2=0x7f1800151a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:51.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.072+0000 7f18076d5640 1 --2- 172.21.15.112:0/3990581842 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f18000b37a0 0x7f1800151a90 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f17fc0099d0 tx=0x7f17fc0052f0 comp rx=0 tx=0).stop 2024-09-21T00:28:51.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.072+0000 7f18076d5640 1 -- 172.21.15.112:0/3990581842 shutdown_connections 2024-09-21T00:28:51.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.072+0000 7f18076d5640 1 --2- 172.21.15.112:0/3990581842 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f180015c3c0 0x7f18000b55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:51.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.072+0000 7f18076d5640 1 --2- 172.21.15.112:0/3990581842 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f18000b3bb0 0x7f18000b5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:51.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.072+0000 7f18076d5640 1 --2- 172.21.15.112:0/3990581842 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f17e4078bc0 0x7f17e407b0d0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:51.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.072+0000 7f18076d5640 1 --2- 172.21.15.112:0/3990581842 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f18000b37a0 0x7f1800151a90 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:51.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.072+0000 7f18076d5640 1 -- 172.21.15.112:0/3990581842 >> 172.21.15.112:0/3990581842 conn(0x7f180008dcb0 msgr2=0x7f18000d2b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:51.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.072+0000 7f18076d5640 1 -- 172.21.15.112:0/3990581842 shutdown_connections 2024-09-21T00:28:51.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.072+0000 7f18076d5640 1 -- 172.21.15.112:0/3990581842 wait complete. 2024-09-21T00:28:51.087 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-21T00:28:51.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.205+0000 7fe109cec640 1 -- 172.21.15.112:0/3613612117 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe1040cc270 msgr2=0x7fe1040d0700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:51.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.205+0000 7fe109cec640 1 --2- 172.21.15.112:0/3613612117 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe1040cc270 0x7fe1040d0700 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fe0f4009830 tx=0x7fe0f402a5a0 comp rx=0 tx=0).stop 2024-09-21T00:28:51.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.205+0000 7fe109cec640 1 -- 172.21.15.112:0/3613612117 shutdown_connections 2024-09-21T00:28:51.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.205+0000 7fe109cec640 1 --2- 172.21.15.112:0/3613612117 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe1040ce9e0 0x7fe1040d0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:51.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.205+0000 7fe109cec640 1 --2- 172.21.15.112:0/3613612117 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe1040cc270 0x7fe1040d0700 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:51.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.205+0000 7fe109cec640 1 -- 172.21.15.112:0/3613612117 >> 172.21.15.112:0/3613612117 conn(0x7fe1040b3580 msgr2=0x7fe1040b3990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:51.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.205+0000 7fe109cec640 1 -- 172.21.15.112:0/3613612117 shutdown_connections 2024-09-21T00:28:51.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.205+0000 7fe109cec640 1 -- 172.21.15.112:0/3613612117 wait complete. 2024-09-21T00:28:51.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.207+0000 7fe109cec640 1 Processor -- start 2024-09-21T00:28:51.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.207+0000 7fe109cec640 1 -- start start 2024-09-21T00:28:51.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.208+0000 7fe109cec640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe10406cde0 0x7fe10409a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:51.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.208+0000 7fe109cec640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe1040cc270 0x7fe1041f6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:51.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.208+0000 7fe109cec640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe1040ce9e0 0x7fe1041f5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:51.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.208+0000 7fe109cec640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe1040afad0 con 0x7fe1040ce9e0 2024-09-21T00:28:51.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.208+0000 7fe109cec640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe1040af950 con 0x7fe10406cde0 2024-09-21T00:28:51.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.208+0000 7fe109cec640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe10406d900 con 0x7fe1040cc270 2024-09-21T00:28:51.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.208+0000 7fe103fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe1040ce9e0 0x7fe1041f5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:51.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.208+0000 7fe103fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe1040ce9e0 0x7fe1041f5150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60430/0 (socket says 172.21.15.112:60430) 2024-09-21T00:28:51.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.208+0000 7fe103fff640 1 -- 172.21.15.112:0/4099156580 learned_addr learned my addr 172.21.15.112:0/4099156580 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:28:51.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.208+0000 7fe102ffd640 1 --2- 172.21.15.112:0/4099156580 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe1040cc270 0x7fe1041f6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:51.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.208+0000 7fe1037fe640 1 --2- 172.21.15.112:0/4099156580 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe10406cde0 0x7fe10409a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:51.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.209+0000 7fe103fff640 1 -- 172.21.15.112:0/4099156580 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe1040cc270 msgr2=0x7fe1041f6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:51.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.209+0000 7fe103fff640 1 --2- 172.21.15.112:0/4099156580 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe1040cc270 0x7fe1041f6390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:51.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.209+0000 7fe103fff640 1 -- 172.21.15.112:0/4099156580 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe10406cde0 msgr2=0x7fe10409a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:51.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.209+0000 7fe103fff640 1 --2- 172.21.15.112:0/4099156580 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe10406cde0 0x7fe10409a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:51.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.209+0000 7fe103fff640 1 -- 172.21.15.112:0/4099156580 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe10409aab0 con 0x7fe1040ce9e0 2024-09-21T00:28:51.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.209+0000 7fe103fff640 1 --2- 172.21.15.112:0/4099156580 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe1040ce9e0 0x7fe1041f5150 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc00c730 tx=0x7fe0fc00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:51.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.209+0000 7fe0ebfff640 1 -- 172.21.15.112:0/4099156580 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0fc003b70 con 0x7fe1040ce9e0 2024-09-21T00:28:51.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.209+0000 7fe0ebfff640 1 -- 172.21.15.112:0/4099156580 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe0fc003d30 con 0x7fe1040ce9e0 2024-09-21T00:28:51.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.209+0000 7fe109cec640 1 -- 172.21.15.112:0/4099156580 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe10409a7d0 con 0x7fe1040ce9e0 2024-09-21T00:28:51.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.209+0000 7fe0ebfff640 1 -- 172.21.15.112:0/4099156580 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0fc0052d0 con 0x7fe1040ce9e0 2024-09-21T00:28:51.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.209+0000 7fe109cec640 1 -- 172.21.15.112:0/4099156580 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1041f9360 con 0x7fe1040ce9e0 2024-09-21T00:28:51.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.210+0000 7fe109cec640 1 -- 172.21.15.112:0/4099156580 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1040ae420 con 0x7fe1040ce9e0 2024-09-21T00:28:51.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.211+0000 7fe0ebfff640 1 -- 172.21.15.112:0/4099156580 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe0fc005680 con 0x7fe1040ce9e0 2024-09-21T00:28:51.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.211+0000 7fe0ebfff640 1 --2- 172.21.15.112:0/4099156580 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fe0f0078b90 0x7fe0f007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:51.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.211+0000 7fe0ebfff640 1 -- 172.21.15.112:0/4099156580 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 5249+0+0 (secure 0 0 0) 0x7fe0fc09a920 con 0x7fe1040ce9e0 2024-09-21T00:28:51.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.212+0000 7fe1037fe640 1 --2- 172.21.15.112:0/4099156580 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fe0f0078b90 0x7fe0f007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:51.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.212+0000 7fe1037fe640 1 --2- 172.21.15.112:0/4099156580 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fe0f0078b90 0x7fe0f007b0a0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fe0f4034000 tx=0x7fe0f4004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:51.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.216+0000 7fe0ebfff640 1 -- 172.21.15.112:0/4099156580 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe10409aab0 con 0x7fe1040ce9e0 2024-09-21T00:28:51.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.464+0000 7fe109cec640 1 -- 172.21.15.112:0/4099156580 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fe1040ae140 con 0x7fe1040ce9e0 2024-09-21T00:28:51.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.464+0000 7fe0ebfff640 1 -- 172.21.15.112:0/4099156580 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v18) ==== 80+0+5304 (secure 0 0 0) 0x7fe0fc062960 con 0x7fe1040ce9e0 2024-09-21T00:28:51.465 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:28:51.465 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-21T00:28:51.465 INFO:teuthology.orchestra.run.smithi112.stdout: "auth_dump": [ 2024-09-21T00:28:51.466 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:28:51.466 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "mds.a", 2024-09-21T00:28:51.466 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBPEu5m3capAhAAczRM4L8ldX985NW4ep3w5Q==", 2024-09-21T00:28:51.466 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:28:51.466 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2024-09-21T00:28:51.466 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow *", 2024-09-21T00:28:51.466 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow *", 2024-09-21T00:28:51.466 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2024-09-21T00:28:51.466 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:28:51.466 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:28:51.467 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:28:51.467 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "mds.b", 2024-09-21T00:28:51.467 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBPEu5mxlDlDxAAtqngbTOhPqXDjFhJpSCLHg==", 2024-09-21T00:28:51.467 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:28:51.467 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2024-09-21T00:28:51.467 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow *", 2024-09-21T00:28:51.467 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow *", 2024-09-21T00:28:51.467 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2024-09-21T00:28:51.467 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:28:51.467 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:28:51.467 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:28:51.467 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "mds.c", 2024-09-21T00:28:51.468 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBPEu5mXNVRChAA1qsPbEybmY16jw5Jwk/v1A==", 2024-09-21T00:28:51.468 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:28:51.468 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2024-09-21T00:28:51.468 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow *", 2024-09-21T00:28:51.468 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow *", 2024-09-21T00:28:51.468 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2024-09-21T00:28:51.468 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:28:51.468 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:28:51.468 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:28:51.468 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "mds.d", 2024-09-21T00:28:51.468 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBPEu5mnzySFxAAX3VfTgGYGcDweI7rPf3xvw==", 2024-09-21T00:28:51.468 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:28:51.468 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2024-09-21T00:28:51.468 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow *", 2024-09-21T00:28:51.469 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow *", 2024-09-21T00:28:51.469 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2024-09-21T00:28:51.469 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:28:51.469 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:28:51.469 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:28:51.469 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.0", 2024-09-21T00:28:51.469 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBSEu5moQg4OBAAi8mzLx7tWnORlrL64/MWlw==", 2024-09-21T00:28:51.469 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:28:51.469 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2024-09-21T00:28:51.469 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2024-09-21T00:28:51.469 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2024-09-21T00:28:51.469 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:28:51.469 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:28:51.469 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:28:51.470 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.1", 2024-09-21T00:28:51.470 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBWEu5mxfobDhAAi7I6LYq9udB0W8bnYaBY9Q==", 2024-09-21T00:28:51.470 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:28:51.470 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2024-09-21T00:28:51.470 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2024-09-21T00:28:51.470 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2024-09-21T00:28:51.470 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:28:51.470 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:28:51.470 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:28:51.470 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.2", 2024-09-21T00:28:51.470 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBZEu5mDqC/EhAAwzTbj22ocR4xJ8DMJvN7Lg==", 2024-09-21T00:28:51.470 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:28:51.471 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2024-09-21T00:28:51.471 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2024-09-21T00:28:51.471 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2024-09-21T00:28:51.471 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:28:51.471 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:28:51.471 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:28:51.471 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.3", 2024-09-21T00:28:51.471 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBcEu5mwweGJBAAT3hZn5pto0aWt6BwGuaqcw==", 2024-09-21T00:28:51.472 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:28:51.472 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2024-09-21T00:28:51.472 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2024-09-21T00:28:51.472 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2024-09-21T00:28:51.472 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:28:51.472 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:28:51.472 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:28:51.472 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.4", 2024-09-21T00:28:51.472 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBjEu5mIPd3ARAAkxTYphn38uKh48UUVEB+Fg==", 2024-09-21T00:28:51.472 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:28:51.472 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2024-09-21T00:28:51.472 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2024-09-21T00:28:51.472 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2024-09-21T00:28:51.472 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:28:51.473 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:28:51.473 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:28:51.473 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.5", 2024-09-21T00:28:51.473 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBmEu5mP9IVBBAAftfK5V1K4vYRyFyJS2UzKQ==", 2024-09-21T00:28:51.473 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:28:51.473 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2024-09-21T00:28:51.473 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2024-09-21T00:28:51.473 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2024-09-21T00:28:51.473 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:28:51.473 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:28:51.473 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:28:51.473 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.6", 2024-09-21T00:28:51.473 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBpEu5mmjiyFRAAbiW2M8JE0jJOaNg1u+1kgw==", 2024-09-21T00:28:51.473 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:28:51.473 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2024-09-21T00:28:51.474 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2024-09-21T00:28:51.474 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2024-09-21T00:28:51.474 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:28:51.474 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:28:51.474 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:28:51.474 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.7", 2024-09-21T00:28:51.474 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBsEu5moklPKBAA6hzVvgvpmZMkPaFFce4mqw==", 2024-09-21T00:28:51.474 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:28:51.474 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2024-09-21T00:28:51.474 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2024-09-21T00:28:51.474 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2024-09-21T00:28:51.474 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:28:51.474 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:28:51.475 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:28:51.475 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "client.0", 2024-09-21T00:28:51.475 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBPEu5m49NPHBAAl/rcjiE8xYI+XdWwvZ6FOw==", 2024-09-21T00:28:51.475 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:28:51.475 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2024-09-21T00:28:51.475 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow r", 2024-09-21T00:28:51.475 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-21T00:28:51.475 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:28:51.475 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:28:51.475 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:28:51.475 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "client.1", 2024-09-21T00:28:51.475 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBPEu5mG33aIxAAirQhTlVyF/pRJx93T/0bIQ==", 2024-09-21T00:28:51.475 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:28:51.475 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2024-09-21T00:28:51.475 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow r", 2024-09-21T00:28:51.476 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow rw", 2024-09-21T00:28:51.476 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow rwx" 2024-09-21T00:28:51.476 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:28:51.476 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:28:51.476 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:28:51.476 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "client.2", 2024-09-21T00:28:51.476 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBPEu5mreUUIBAAELeN1N58srw6GqBrj8UOoQ==", 2024-09-21T00:28:51.476 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:28:51.476 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2024-09-21T00:28:51.476 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow r", 2024-09-21T00:28:51.476 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow rw", 2024-09-21T00:28:51.476 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow rwx" 2024-09-21T00:28:51.476 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:28:51.476 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:28:51.476 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:28:51.477 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "client.3", 2024-09-21T00:28:51.477 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBPEu5mcTmiJxAAY6bmTtow0R0heJHjx0ahDQ==", 2024-09-21T00:28:51.477 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:28:51.477 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2024-09-21T00:28:51.477 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow r", 2024-09-21T00:28:51.477 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow rw", 2024-09-21T00:28:51.477 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow rwx" 2024-09-21T00:28:51.477 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:28:51.477 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:28:51.477 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:28:51.477 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "client.admin", 2024-09-21T00:28:51.477 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBOEu5mTxhNIBAAlStXGsGd/okXhakY6WI/4w==", 2024-09-21T00:28:51.477 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:28:51.477 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow *", 2024-09-21T00:28:51.478 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow *", 2024-09-21T00:28:51.478 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow *", 2024-09-21T00:28:51.478 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2024-09-21T00:28:51.478 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:28:51.478 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:28:51.478 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:28:51.478 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "mgr.x", 2024-09-21T00:28:51.478 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBOEu5mjGpDOhAAJFudb2wZlOBjo8Ss65Q+eg==", 2024-09-21T00:28:51.478 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:28:51.478 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow *", 2024-09-21T00:28:51.478 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile mgr", 2024-09-21T00:28:51.478 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2024-09-21T00:28:51.478 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:28:51.478 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:28:51.479 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:28:51.479 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "mgr.y", 2024-09-21T00:28:51.479 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBOEu5myGjKNRAA9cMDObZYYOXMGdv2mOXkJA==", 2024-09-21T00:28:51.479 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:28:51.479 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow *", 2024-09-21T00:28:51.479 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile mgr", 2024-09-21T00:28:51.479 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2024-09-21T00:28:51.479 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:28:51.479 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:28:51.479 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-09-21T00:28:51.479 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-21T00:28:51.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.466+0000 7fe109cec640 1 -- 172.21.15.112:0/4099156580 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fe0f0078b90 msgr2=0x7fe0f007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:51.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.466+0000 7fe109cec640 1 --2- 172.21.15.112:0/4099156580 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fe0f0078b90 0x7fe0f007b0a0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fe0f4034000 tx=0x7fe0f4004730 comp rx=0 tx=0).stop 2024-09-21T00:28:51.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.466+0000 7fe109cec640 1 -- 172.21.15.112:0/4099156580 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe1040ce9e0 msgr2=0x7fe1041f5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:51.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.466+0000 7fe109cec640 1 --2- 172.21.15.112:0/4099156580 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe1040ce9e0 0x7fe1041f5150 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc00c730 tx=0x7fe0fc00cbf0 comp rx=0 tx=0).stop 2024-09-21T00:28:51.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.467+0000 7fe109cec640 1 -- 172.21.15.112:0/4099156580 shutdown_connections 2024-09-21T00:28:51.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.467+0000 7fe109cec640 1 --2- 172.21.15.112:0/4099156580 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fe0f0078b90 0x7fe0f007b0a0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:51.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.467+0000 7fe109cec640 1 --2- 172.21.15.112:0/4099156580 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe1040ce9e0 0x7fe1041f5150 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:51.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.467+0000 7fe109cec640 1 --2- 172.21.15.112:0/4099156580 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe1040cc270 0x7fe1041f6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:51.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.467+0000 7fe109cec640 1 --2- 172.21.15.112:0/4099156580 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe10406cde0 0x7fe10409a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:51.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.467+0000 7fe109cec640 1 -- 172.21.15.112:0/4099156580 >> 172.21.15.112:0/4099156580 conn(0x7fe1040b3580 msgr2=0x7fe1040ae940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:51.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.467+0000 7fe109cec640 1 -- 172.21.15.112:0/4099156580 shutdown_connections 2024-09-21T00:28:51.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.467+0000 7fe109cec640 1 -- 172.21.15.112:0/4099156580 wait complete. 2024-09-21T00:28:51.482 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-21T00:28:51.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.594+0000 7f4e4da94640 1 -- 172.21.15.112:0/3740514326 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4e48070210 msgr2=0x7f4e48070620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:51.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.594+0000 7f4e4da94640 1 --2- 172.21.15.112:0/3740514326 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4e48070210 0x7f4e48070620 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c009830 tx=0x7f4e3c02a480 comp rx=0 tx=0).stop 2024-09-21T00:28:51.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.594+0000 7f4e4da94640 1 -- 172.21.15.112:0/3740514326 shutdown_connections 2024-09-21T00:28:51.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.594+0000 7f4e4da94640 1 --2- 172.21.15.112:0/3740514326 >> v2:172.21.15.187:3300/0 conn(0x7f4e480ccbb0 0x7f4e480ccfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:51.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.594+0000 7f4e4da94640 1 --2- 172.21.15.112:0/3740514326 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4e48070210 0x7f4e48070620 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:51.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.594+0000 7f4e4da94640 1 -- 172.21.15.112:0/3740514326 >> 172.21.15.112:0/3740514326 conn(0x7f4e48073fe0 msgr2=0x7f4e480743f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:51.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.594+0000 7f4e4da94640 1 -- 172.21.15.112:0/3740514326 shutdown_connections 2024-09-21T00:28:51.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.594+0000 7f4e4da94640 1 -- 172.21.15.112:0/3740514326 wait complete. 2024-09-21T00:28:51.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.596+0000 7f4e4da94640 1 Processor -- start 2024-09-21T00:28:51.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.596+0000 7f4e4da94640 1 -- start start 2024-09-21T00:28:51.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.597+0000 7f4e4da94640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4e48070210 0x7f4e4815ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:51.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.597+0000 7f4e4da94640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e480ccbb0 0x7f4e481f9510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:51.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.597+0000 7f4e4da94640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4e4815c5f0 0x7f4e481f9a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:51.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.597+0000 7f4e4da94640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4e4815ca00 con 0x7f4e480ccbb0 2024-09-21T00:28:51.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.597+0000 7f4e4da94640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f4e48070b60 con 0x7f4e4815c5f0 2024-09-21T00:28:51.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.597+0000 7f4e4da94640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f4e4813c730 con 0x7f4e48070210 2024-09-21T00:28:51.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.597+0000 7f4e46ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e480ccbb0 0x7f4e481f9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:51.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.597+0000 7f4e46ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e480ccbb0 0x7f4e481f9510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60434/0 (socket says 172.21.15.112:60434) 2024-09-21T00:28:51.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.597+0000 7f4e46ffd640 1 -- 172.21.15.112:0/943574988 learned_addr learned my addr 172.21.15.112:0/943574988 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:28:51.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.597+0000 7f4e47fff640 1 --2- 172.21.15.112:0/943574988 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4e4815c5f0 0x7f4e481f9a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:51.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.597+0000 7f4e477fe640 1 --2- 172.21.15.112:0/943574988 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4e48070210 0x7f4e4815ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:51.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.597+0000 7f4e46ffd640 1 -- 172.21.15.112:0/943574988 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4e48070210 msgr2=0x7f4e4815ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:51.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.597+0000 7f4e46ffd640 1 --2- 172.21.15.112:0/943574988 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4e48070210 0x7f4e4815ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:51.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.597+0000 7f4e46ffd640 1 -- 172.21.15.112:0/943574988 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4e4815c5f0 msgr2=0x7f4e481f9a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:51.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.597+0000 7f4e46ffd640 1 --2- 172.21.15.112:0/943574988 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4e4815c5f0 0x7f4e481f9a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:51.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.597+0000 7f4e46ffd640 1 -- 172.21.15.112:0/943574988 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e481f5c20 con 0x7f4e480ccbb0 2024-09-21T00:28:51.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.597+0000 7f4e477fe640 1 --2- 172.21.15.112:0/943574988 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4e48070210 0x7f4e4815ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:28:51.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.598+0000 7f4e46ffd640 1 --2- 172.21.15.112:0/943574988 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e480ccbb0 0x7f4e481f9510 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c02aa70 tx=0x7f4e3c033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:51.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.598+0000 7f4e2ffff640 1 -- 172.21.15.112:0/943574988 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e3c031d60 con 0x7f4e480ccbb0 2024-09-21T00:28:51.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.598+0000 7f4e2ffff640 1 -- 172.21.15.112:0/943574988 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4e3c033e10 con 0x7f4e480ccbb0 2024-09-21T00:28:51.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.598+0000 7f4e4da94640 1 -- 172.21.15.112:0/943574988 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e481f5f00 con 0x7f4e480ccbb0 2024-09-21T00:28:51.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.598+0000 7f4e2ffff640 1 -- 172.21.15.112:0/943574988 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e3c005260 con 0x7f4e480ccbb0 2024-09-21T00:28:51.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.598+0000 7f4e4da94640 1 -- 172.21.15.112:0/943574988 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e481f64c0 con 0x7f4e480ccbb0 2024-09-21T00:28:51.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.599+0000 7f4e4da94640 1 -- 172.21.15.112:0/943574988 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e48070620 con 0x7f4e480ccbb0 2024-09-21T00:28:51.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.600+0000 7f4e2ffff640 1 -- 172.21.15.112:0/943574988 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f4e3c005400 con 0x7f4e480ccbb0 2024-09-21T00:28:51.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.600+0000 7f4e2ffff640 1 --2- 172.21.15.112:0/943574988 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4e34078b90 0x7f4e3407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:51.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.600+0000 7f4e2ffff640 1 -- 172.21.15.112:0/943574988 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 5249+0+0 (secure 0 0 0) 0x7f4e3c0bbc90 con 0x7f4e480ccbb0 2024-09-21T00:28:51.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.600+0000 7f4e477fe640 1 --2- 172.21.15.112:0/943574988 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4e34078b90 0x7f4e3407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:51.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.601+0000 7f4e477fe640 1 --2- 172.21.15.112:0/943574988 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4e34078b90 0x7f4e3407b0a0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f4e380020f0 tx=0x7f4e38009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:51.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.605+0000 7f4e2ffff640 1 -- 172.21.15.112:0/943574988 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f4e3c03f030 con 0x7f4e480ccbb0 2024-09-21T00:28:51.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.785+0000 7f4e4da94640 1 -- 172.21.15.112:0/943574988 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f4e48075bc0 con 0x7f4e480ccbb0 2024-09-21T00:28:51.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.785+0000 7f4e2ffff640 1 -- 172.21.15.112:0/943574988 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v79) ==== 84+0+65 (secure 0 0 0) 0x7f4e3c084290 con 0x7f4e480ccbb0 2024-09-21T00:28:51.786 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:28:51.786 INFO:teuthology.orchestra.run.smithi112.stdout:[ 2024-09-21T00:28:51.786 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:28:51.787 INFO:teuthology.orchestra.run.smithi112.stdout: "poolnum": 1, 2024-09-21T00:28:51.787 INFO:teuthology.orchestra.run.smithi112.stdout: "poolname": ".mgr" 2024-09-21T00:28:51.787 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:28:51.787 INFO:teuthology.orchestra.run.smithi112.stdout:] 2024-09-21T00:28:51.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.788+0000 7f4e4da94640 1 -- 172.21.15.112:0/943574988 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4e34078b90 msgr2=0x7f4e3407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:51.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.788+0000 7f4e4da94640 1 --2- 172.21.15.112:0/943574988 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4e34078b90 0x7f4e3407b0a0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f4e380020f0 tx=0x7f4e38009040 comp rx=0 tx=0).stop 2024-09-21T00:28:51.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.788+0000 7f4e4da94640 1 -- 172.21.15.112:0/943574988 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e480ccbb0 msgr2=0x7f4e481f9510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:51.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.788+0000 7f4e4da94640 1 --2- 172.21.15.112:0/943574988 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e480ccbb0 0x7f4e481f9510 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c02aa70 tx=0x7f4e3c033c20 comp rx=0 tx=0).stop 2024-09-21T00:28:51.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.788+0000 7f4e4da94640 1 -- 172.21.15.112:0/943574988 shutdown_connections 2024-09-21T00:28:51.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.788+0000 7f4e4da94640 1 --2- 172.21.15.112:0/943574988 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4e4815c5f0 0x7f4e481f9a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:51.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.788+0000 7f4e4da94640 1 --2- 172.21.15.112:0/943574988 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4e34078b90 0x7f4e3407b0a0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:51.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.788+0000 7f4e4da94640 1 --2- 172.21.15.112:0/943574988 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e480ccbb0 0x7f4e481f9510 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:51.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.788+0000 7f4e4da94640 1 --2- 172.21.15.112:0/943574988 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4e48070210 0x7f4e4815ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:51.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.788+0000 7f4e4da94640 1 -- 172.21.15.112:0/943574988 >> 172.21.15.112:0/943574988 conn(0x7f4e48073fe0 msgr2=0x7f4e4813b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:51.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.788+0000 7f4e4da94640 1 -- 172.21.15.112:0/943574988 shutdown_connections 2024-09-21T00:28:51.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.788+0000 7f4e4da94640 1 -- 172.21.15.112:0/943574988 wait complete. 2024-09-21T00:28:51.800 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-21T00:28:51.800 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-21T00:28:51.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.915+0000 7f79f4a4d640 1 -- 172.21.15.112:0/885210895 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f79f00cfc00 msgr2=0x7f79f015bba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:51.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.915+0000 7f79f4a4d640 1 --2- 172.21.15.112:0/885210895 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f79f00cfc00 0x7f79f015bba0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f79e0005fd0 tx=0x7f79e002a5d0 comp rx=0 tx=0).stop 2024-09-21T00:28:51.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.915+0000 7f79f4a4d640 1 -- 172.21.15.112:0/885210895 shutdown_connections 2024-09-21T00:28:51.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.915+0000 7f79f4a4d640 1 --2- 172.21.15.112:0/885210895 >> v2:172.21.15.187:3300/0 conn(0x7f79f0152720 0x7f79f015c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:51.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.915+0000 7f79f4a4d640 1 --2- 172.21.15.112:0/885210895 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f79f00cfc00 0x7f79f015bba0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:51.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.915+0000 7f79f4a4d640 1 -- 172.21.15.112:0/885210895 >> 172.21.15.112:0/885210895 conn(0x7f79f0071790 msgr2=0x7f79f0071ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:51.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.915+0000 7f79f4a4d640 1 -- 172.21.15.112:0/885210895 shutdown_connections 2024-09-21T00:28:51.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.915+0000 7f79f4a4d640 1 -- 172.21.15.112:0/885210895 wait complete. 2024-09-21T00:28:51.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.917+0000 7f79f4a4d640 1 Processor -- start 2024-09-21T00:28:51.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.918+0000 7f79f4a4d640 1 -- start start 2024-09-21T00:28:51.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.919+0000 7f79f4a4d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f79f00cfc00 0x7f79f0145650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:51.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.919+0000 7f79f4a4d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f79f0152720 0x7f79f0074230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:51.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.919+0000 7f79f4a4d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f79f015c620 0x7f79f0074770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:51.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.919+0000 7f79f4a4d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f79f015f8a0 con 0x7f79f00cfc00 2024-09-21T00:28:51.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.919+0000 7f79f4a4d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f79f0070e30 con 0x7f79f015c620 2024-09-21T00:28:51.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.919+0000 7f79f4a4d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f79f015fa20 con 0x7f79f0152720 2024-09-21T00:28:51.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.919+0000 7f79ee59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f79f00cfc00 0x7f79f0145650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:51.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.919+0000 7f79ee59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f79f00cfc00 0x7f79f0145650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60436/0 (socket says 172.21.15.112:60436) 2024-09-21T00:28:51.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.919+0000 7f79ee59c640 1 -- 172.21.15.112:0/2181452724 learned_addr learned my addr 172.21.15.112:0/2181452724 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:28:51.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.919+0000 7f79eed9d640 1 --2- 172.21.15.112:0/2181452724 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f79f015c620 0x7f79f0074770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:51.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.919+0000 7f79e5d9b640 1 --2- 172.21.15.112:0/2181452724 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f79f0152720 0x7f79f0074230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:51.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.919+0000 7f79ee59c640 1 -- 172.21.15.112:0/2181452724 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f79f0152720 msgr2=0x7f79f0074230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:51.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.919+0000 7f79ee59c640 1 --2- 172.21.15.112:0/2181452724 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f79f0152720 0x7f79f0074230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:51.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.919+0000 7f79ee59c640 1 -- 172.21.15.112:0/2181452724 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f79f015c620 msgr2=0x7f79f0074770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:51.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.919+0000 7f79ee59c640 1 --2- 172.21.15.112:0/2181452724 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f79f015c620 0x7f79f0074770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:51.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.919+0000 7f79ee59c640 1 -- 172.21.15.112:0/2181452724 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79f0145b90 con 0x7f79f00cfc00 2024-09-21T00:28:51.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.919+0000 7f79ee59c640 1 --2- 172.21.15.112:0/2181452724 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f79f00cfc00 0x7f79f0145650 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f79e0006010 tx=0x7f79e00035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:51.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.920+0000 7f79e77fe640 1 -- 172.21.15.112:0/2181452724 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e003e030 con 0x7f79f00cfc00 2024-09-21T00:28:51.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.920+0000 7f79e77fe640 1 -- 172.21.15.112:0/2181452724 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f79e0003ae0 con 0x7f79f00cfc00 2024-09-21T00:28:51.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.920+0000 7f79e77fe640 1 -- 172.21.15.112:0/2181452724 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e0035040 con 0x7f79f00cfc00 2024-09-21T00:28:51.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.920+0000 7f79f4a4d640 1 -- 172.21.15.112:0/2181452724 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79f01522e0 con 0x7f79f00cfc00 2024-09-21T00:28:51.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.920+0000 7f79f4a4d640 1 -- 172.21.15.112:0/2181452724 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79f01f6f40 con 0x7f79f00cfc00 2024-09-21T00:28:51.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.921+0000 7f79f4a4d640 1 -- 172.21.15.112:0/2181452724 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79f015fe80 con 0x7f79f00cfc00 2024-09-21T00:28:51.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.922+0000 7f79e77fe640 1 -- 172.21.15.112:0/2181452724 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f79e0041030 con 0x7f79f00cfc00 2024-09-21T00:28:51.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.922+0000 7f79e77fe640 1 --2- 172.21.15.112:0/2181452724 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f79d0078b90 0x7f79d007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:51.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.922+0000 7f79e77fe640 1 -- 172.21.15.112:0/2181452724 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 5249+0+0 (secure 0 0 0) 0x7f79e00bb0c0 con 0x7f79f00cfc00 2024-09-21T00:28:51.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.922+0000 7f79e5d9b640 1 --2- 172.21.15.112:0/2181452724 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f79d0078b90 0x7f79d007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:51.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.923+0000 7f79e5d9b640 1 --2- 172.21.15.112:0/2181452724 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f79d0078b90 0x7f79d007b0a0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f79f00b1d50 tx=0x7f79d8005f90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:51.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:51.926+0000 7f79e77fe640 1 -- 172.21.15.112:0/2181452724 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f79e0083070 con 0x7f79f00cfc00 2024-09-21T00:28:52.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:52.103+0000 7f79f4a4d640 1 -- 172.21.15.112:0/2181452724 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f79f0074ec0 con 0x7f79f00cfc00 2024-09-21T00:28:53.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:53.073+0000 7f79e77fe640 1 -- 172.21.15.112:0/2181452724 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v80) ==== 138+0+0 (secure 0 0 0) 0x7f79e008ae20 con 0x7f79f00cfc00 2024-09-21T00:28:53.074 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'cephfs_metadata' created 2024-09-21T00:28:53.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:53.076+0000 7f79f4a4d640 1 -- 172.21.15.112:0/2181452724 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f79d0078b90 msgr2=0x7f79d007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:53.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:53.076+0000 7f79f4a4d640 1 --2- 172.21.15.112:0/2181452724 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f79d0078b90 0x7f79d007b0a0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f79f00b1d50 tx=0x7f79d8005f90 comp rx=0 tx=0).stop 2024-09-21T00:28:53.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:53.076+0000 7f79f4a4d640 1 -- 172.21.15.112:0/2181452724 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f79f00cfc00 msgr2=0x7f79f0145650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:53.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:53.076+0000 7f79f4a4d640 1 --2- 172.21.15.112:0/2181452724 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f79f00cfc00 0x7f79f0145650 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f79e0006010 tx=0x7f79e00035f0 comp rx=0 tx=0).stop 2024-09-21T00:28:53.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:53.076+0000 7f79f4a4d640 1 -- 172.21.15.112:0/2181452724 shutdown_connections 2024-09-21T00:28:53.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:53.076+0000 7f79f4a4d640 1 --2- 172.21.15.112:0/2181452724 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f79f015c620 0x7f79f0074770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:53.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:53.076+0000 7f79f4a4d640 1 --2- 172.21.15.112:0/2181452724 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f79f0152720 0x7f79f0074230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:53.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:53.076+0000 7f79f4a4d640 1 --2- 172.21.15.112:0/2181452724 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f79d0078b90 0x7f79d007b0a0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:53.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:53.076+0000 7f79f4a4d640 1 --2- 172.21.15.112:0/2181452724 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f79f00cfc00 0x7f79f0145650 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:53.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:53.076+0000 7f79f4a4d640 1 -- 172.21.15.112:0/2181452724 >> 172.21.15.112:0/2181452724 conn(0x7f79f0071790 msgr2=0x7f79f00b2260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:53.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:53.076+0000 7f79f4a4d640 1 -- 172.21.15.112:0/2181452724 shutdown_connections 2024-09-21T00:28:53.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:53.076+0000 7f79f4a4d640 1 -- 172.21.15.112:0/2181452724 wait complete. 2024-09-21T00:28:53.094 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-21T00:28:53.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:53.219+0000 7f646fef9640 1 -- 172.21.15.112:0/3316697962 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6454004800 msgr2=0x7f6454004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:53.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:53.219+0000 7f646fef9640 1 --2- 172.21.15.112:0/3316697962 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6454004800 0x7f6454004c10 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f6458009b50 tx=0x7f6454005830 comp rx=0 tx=0).stop 2024-09-21T00:28:53.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:53.219+0000 7f646fef9640 1 -- 172.21.15.112:0/3316697962 shutdown_connections 2024-09-21T00:28:53.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:53.219+0000 7f646fef9640 1 --2- 172.21.15.112:0/3316697962 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6454004800 0x7f6454004c10 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:53.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:53.219+0000 7f646fef9640 1 -- 172.21.15.112:0/3316697962 >> 172.21.15.112:0/3316697962 conn(0x7f646808bfe0 msgr2=0x7f646809d890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:53.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:53.219+0000 7f646fef9640 1 -- 172.21.15.112:0/3316697962 shutdown_connections 2024-09-21T00:28:53.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:53.219+0000 7f646fef9640 1 -- 172.21.15.112:0/3316697962 wait complete. 2024-09-21T00:28:53.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:53.221+0000 7f646fef9640 1 Processor -- start 2024-09-21T00:28:53.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:53.221+0000 7f646fef9640 1 -- start start 2024-09-21T00:28:53.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:53.222+0000 7f646fef9640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6454004800 0x7f64681f6430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:53.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:53.222+0000 7f646fef9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64680cbfe0 0x7f64681f2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:53.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:53.222+0000 7f646fef9640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f646806dc00 0x7f64681616f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:53.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:53.222+0000 7f646fef9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f64681633e0 con 0x7f64680cbfe0 2024-09-21T00:28:53.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:53.222+0000 7f646fef9640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f6468163260 con 0x7f646806dc00 2024-09-21T00:28:53.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:53.222+0000 7f646fef9640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f6468163560 con 0x7f6454004800 2024-09-21T00:28:53.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:53.222+0000 7f646d494640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64680cbfe0 0x7f64681f2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:53.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:53.222+0000 7f646d494640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64680cbfe0 0x7f64681f2dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60460/0 (socket says 172.21.15.112:60460) 2024-09-21T00:28:53.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:53.222+0000 7f646d494640 1 -- 172.21.15.112:0/4104663352 learned_addr learned my addr 172.21.15.112:0/4104663352 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:28:53.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:53.222+0000 7f646dc95640 1 --2- 172.21.15.112:0/4104663352 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6454004800 0x7f64681f6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:53.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:53.222+0000 7f646e496640 1 --2- 172.21.15.112:0/4104663352 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f646806dc00 0x7f64681616f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:53.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:53.222+0000 7f646d494640 1 -- 172.21.15.112:0/4104663352 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6454004800 msgr2=0x7f64681f6430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:53.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:53.222+0000 7f646d494640 1 --2- 172.21.15.112:0/4104663352 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6454004800 0x7f64681f6430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:53.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:53.222+0000 7f646d494640 1 -- 172.21.15.112:0/4104663352 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f646806dc00 msgr2=0x7f64681616f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:53.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:53.222+0000 7f646d494640 1 --2- 172.21.15.112:0/4104663352 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f646806dc00 0x7f64681616f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:53.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:53.222+0000 7f646d494640 1 -- 172.21.15.112:0/4104663352 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6468160930 con 0x7f64680cbfe0 2024-09-21T00:28:53.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:53.223+0000 7f646d494640 1 --2- 172.21.15.112:0/4104663352 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64680cbfe0 0x7f64681f2dc0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f6458009a70 tx=0x7f64580336f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:53.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:53.223+0000 7f645e7fc640 1 -- 172.21.15.112:0/4104663352 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6458038040 con 0x7f64680cbfe0 2024-09-21T00:28:53.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:53.223+0000 7f646fef9640 1 -- 172.21.15.112:0/4104663352 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6468160c10 con 0x7f64680cbfe0 2024-09-21T00:28:53.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:53.223+0000 7f645e7fc640 1 -- 172.21.15.112:0/4104663352 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f64580339d0 con 0x7f64680cbfe0 2024-09-21T00:28:53.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:53.223+0000 7f645e7fc640 1 -- 172.21.15.112:0/4104663352 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6458033e20 con 0x7f64680cbfe0 2024-09-21T00:28:53.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:53.223+0000 7f646fef9640 1 -- 172.21.15.112:0/4104663352 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64681611d0 con 0x7f64680cbfe0 2024-09-21T00:28:53.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:53.224+0000 7f646fef9640 1 -- 172.21.15.112:0/4104663352 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64680d05c0 con 0x7f64680cbfe0 2024-09-21T00:28:53.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:53.225+0000 7f645e7fc640 1 -- 172.21.15.112:0/4104663352 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6458043020 con 0x7f64680cbfe0 2024-09-21T00:28:53.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:53.225+0000 7f645e7fc640 1 --2- 172.21.15.112:0/4104663352 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f644c078b90 0x7f644c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:53.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:53.226+0000 7f645e7fc640 1 -- 172.21.15.112:0/4104663352 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 5619+0+0 (secure 0 0 0) 0x7f64580bade0 con 0x7f64680cbfe0 2024-09-21T00:28:53.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:53.226+0000 7f646dc95640 1 --2- 172.21.15.112:0/4104663352 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f644c078b90 0x7f644c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:53.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:53.226+0000 7f646dc95640 1 --2- 172.21.15.112:0/4104663352 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f644c078b90 0x7f644c07b0a0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f6464001220 tx=0x7f64640067a0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:53.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:53.230+0000 7f645e7fc640 1 -- 172.21.15.112:0/4104663352 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f645804a020 con 0x7f64680cbfe0 2024-09-21T00:28:53.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:53.406+0000 7f646fef9640 1 -- 172.21.15.112:0/4104663352 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f64680cc450 con 0x7f64680cbfe0 2024-09-21T00:28:54.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:54.078+0000 7f645e7fc640 1 -- 172.21.15.112:0/4104663352 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v81) ==== 170+0+0 (secure 0 0 0) 0x7f6458082cc0 con 0x7f64680cbfe0 2024-09-21T00:28:54.079 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'cephfs_data' created 2024-09-21T00:28:54.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:54.082+0000 7f646fef9640 1 -- 172.21.15.112:0/4104663352 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f644c078b90 msgr2=0x7f644c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:54.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:54.082+0000 7f646fef9640 1 --2- 172.21.15.112:0/4104663352 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f644c078b90 0x7f644c07b0a0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f6464001220 tx=0x7f64640067a0 comp rx=0 tx=0).stop 2024-09-21T00:28:54.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:54.082+0000 7f646fef9640 1 -- 172.21.15.112:0/4104663352 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64680cbfe0 msgr2=0x7f64681f2dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:54.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:54.082+0000 7f646fef9640 1 --2- 172.21.15.112:0/4104663352 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64680cbfe0 0x7f64681f2dc0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f6458009a70 tx=0x7f64580336f0 comp rx=0 tx=0).stop 2024-09-21T00:28:54.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:54.082+0000 7f646fef9640 1 -- 172.21.15.112:0/4104663352 shutdown_connections 2024-09-21T00:28:54.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:54.082+0000 7f646fef9640 1 --2- 172.21.15.112:0/4104663352 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f646806dc00 0x7f64681616f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:54.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:54.082+0000 7f646fef9640 1 --2- 172.21.15.112:0/4104663352 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f644c078b90 0x7f644c07b0a0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:54.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:54.082+0000 7f646fef9640 1 --2- 172.21.15.112:0/4104663352 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64680cbfe0 0x7f64681f2dc0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:54.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:54.082+0000 7f646fef9640 1 --2- 172.21.15.112:0/4104663352 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6454004800 0x7f64681f6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:54.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:54.082+0000 7f646fef9640 1 -- 172.21.15.112:0/4104663352 >> 172.21.15.112:0/4104663352 conn(0x7f646808bfe0 msgr2=0x7f64680b7910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:54.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:54.082+0000 7f646fef9640 1 -- 172.21.15.112:0/4104663352 shutdown_connections 2024-09-21T00:28:54.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:54.082+0000 7f646fef9640 1 -- 172.21.15.112:0/4104663352 wait complete. 2024-09-21T00:28:54.109 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-21T00:28:54.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:54.226+0000 7f8a562de640 1 -- 172.21.15.112:0/3739501450 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a500d3880 msgr2=0x7f8a50163260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:54.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:54.226+0000 7f8a562de640 1 --2- 172.21.15.112:0/3739501450 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a500d3880 0x7f8a50163260 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f8a44009830 tx=0x7f8a4402a510 comp rx=0 tx=0).stop 2024-09-21T00:28:54.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:54.226+0000 7f8a562de640 1 -- 172.21.15.112:0/3739501450 shutdown_connections 2024-09-21T00:28:54.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:54.226+0000 7f8a562de640 1 --2- 172.21.15.112:0/3739501450 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8a500d01b0 0x7f8a500d05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:54.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:54.226+0000 7f8a562de640 1 --2- 172.21.15.112:0/3739501450 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a500d3880 0x7f8a50163260 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:54.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:54.226+0000 7f8a562de640 1 -- 172.21.15.112:0/3739501450 >> 172.21.15.112:0/3739501450 conn(0x7f8a5008bfe0 msgr2=0x7f8a5009d890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:54.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:54.226+0000 7f8a562de640 1 -- 172.21.15.112:0/3739501450 shutdown_connections 2024-09-21T00:28:54.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:54.226+0000 7f8a562de640 1 -- 172.21.15.112:0/3739501450 wait complete. 2024-09-21T00:28:54.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:54.228+0000 7f8a562de640 1 Processor -- start 2024-09-21T00:28:54.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:54.228+0000 7f8a562de640 1 -- start start 2024-09-21T00:28:54.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:54.229+0000 7f8a562de640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8a500d01b0 0x7f8a501f4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:54.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:54.229+0000 7f8a562de640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a500d3880 0x7f8a50161b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:54.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:54.229+0000 7f8a562de640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8a500d48a0 0x7f8a500b2c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:54.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:54.229+0000 7f8a562de640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8a500d2460 con 0x7f8a500d3880 2024-09-21T00:28:54.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:54.229+0000 7f8a562de640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8a500d22e0 con 0x7f8a500d01b0 2024-09-21T00:28:54.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:54.229+0000 7f8a562de640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f8a500d25e0 con 0x7f8a500d48a0 2024-09-21T00:28:54.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:54.229+0000 7f8a4f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a500d3880 0x7f8a50161b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:54.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:54.229+0000 7f8a4f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a500d3880 0x7f8a50161b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60472/0 (socket says 172.21.15.112:60472) 2024-09-21T00:28:54.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:54.229+0000 7f8a4f7fe640 1 -- 172.21.15.112:0/1523055472 learned_addr learned my addr 172.21.15.112:0/1523055472 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:28:54.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:54.229+0000 7f8a5487b640 1 --2- 172.21.15.112:0/1523055472 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8a500d48a0 0x7f8a500b2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:54.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:54.229+0000 7f8a4ffff640 1 --2- 172.21.15.112:0/1523055472 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8a500d01b0 0x7f8a501f4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:54.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:54.229+0000 7f8a4f7fe640 1 -- 172.21.15.112:0/1523055472 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8a500d48a0 msgr2=0x7f8a500b2c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:54.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:54.229+0000 7f8a4f7fe640 1 --2- 172.21.15.112:0/1523055472 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8a500d48a0 0x7f8a500b2c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:54.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:54.229+0000 7f8a4f7fe640 1 -- 172.21.15.112:0/1523055472 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8a500d01b0 msgr2=0x7f8a501f4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:54.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:54.229+0000 7f8a4f7fe640 1 --2- 172.21.15.112:0/1523055472 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8a500d01b0 0x7f8a501f4e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:54.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:54.229+0000 7f8a4f7fe640 1 -- 172.21.15.112:0/1523055472 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a501f5840 con 0x7f8a500d3880 2024-09-21T00:28:54.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:54.229+0000 7f8a5487b640 1 --2- 172.21.15.112:0/1523055472 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8a500d48a0 0x7f8a500b2c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:28:54.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:54.230+0000 7f8a4ffff640 1 --2- 172.21.15.112:0/1523055472 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8a500d01b0 0x7f8a501f4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:28:54.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:54.230+0000 7f8a4f7fe640 1 --2- 172.21.15.112:0/1523055472 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a500d3880 0x7f8a50161b60 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f8a44000c00 tx=0x7f8a44033d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:54.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:54.230+0000 7f8a4cff9640 1 -- 172.21.15.112:0/1523055472 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a44031d60 con 0x7f8a500d3880 2024-09-21T00:28:54.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:54.230+0000 7f8a4cff9640 1 -- 172.21.15.112:0/1523055472 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8a440330d0 con 0x7f8a500d3880 2024-09-21T00:28:54.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:54.230+0000 7f8a4cff9640 1 -- 172.21.15.112:0/1523055472 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a44033520 con 0x7f8a500d3880 2024-09-21T00:28:54.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:54.230+0000 7f8a562de640 1 -- 172.21.15.112:0/1523055472 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a501f5b20 con 0x7f8a500d3880 2024-09-21T00:28:54.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:54.230+0000 7f8a562de640 1 -- 172.21.15.112:0/1523055472 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a501f60e0 con 0x7f8a500d3880 2024-09-21T00:28:54.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:54.231+0000 7f8a562de640 1 -- 172.21.15.112:0/1523055472 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a500cb150 con 0x7f8a500d3880 2024-09-21T00:28:54.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:54.232+0000 7f8a4cff9640 1 -- 172.21.15.112:0/1523055472 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8a440035f0 con 0x7f8a500d3880 2024-09-21T00:28:54.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:54.232+0000 7f8a4cff9640 1 --2- 172.21.15.112:0/1523055472 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f8a3c078bc0 0x7f8a3c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:54.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:54.232+0000 7f8a4cff9640 1 -- 172.21.15.112:0/1523055472 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 6001+0+0 (secure 0 0 0) 0x7f8a440bc440 con 0x7f8a500d3880 2024-09-21T00:28:54.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:54.233+0000 7f8a4ffff640 1 --2- 172.21.15.112:0/1523055472 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f8a3c078bc0 0x7f8a3c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:54.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:54.233+0000 7f8a4ffff640 1 --2- 172.21.15.112:0/1523055472 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f8a3c078bc0 0x7f8a3c07b0d0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f8a40000f60 tx=0x7f8a40009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:54.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:54.237+0000 7f8a4cff9640 1 -- 172.21.15.112:0/1523055472 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f8a500cb150 con 0x7f8a500d3880 2024-09-21T00:28:54.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:54.452+0000 7f8a562de640 1 -- 172.21.15.112:0/1523055472 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f8a500d2b00 con 0x7f8a500d3880 2024-09-21T00:28:55.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.080+0000 7f8a4cff9640 1 -- 172.21.15.112:0/1523055472 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 13 and data pool 14 v51) ==== 174+0+0 (secure 0 0 0) 0x7f8a4403f030 con 0x7f8a500d3880 2024-09-21T00:28:55.081 INFO:teuthology.orchestra.run.smithi112.stderr:new fs with metadata pool 13 and data pool 14 2024-09-21T00:28:55.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.085+0000 7f8a562de640 1 -- 172.21.15.112:0/1523055472 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f8a3c078bc0 msgr2=0x7f8a3c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:55.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.085+0000 7f8a562de640 1 --2- 172.21.15.112:0/1523055472 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f8a3c078bc0 0x7f8a3c07b0d0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f8a40000f60 tx=0x7f8a40009040 comp rx=0 tx=0).stop 2024-09-21T00:28:55.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.086+0000 7f8a562de640 1 -- 172.21.15.112:0/1523055472 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a500d3880 msgr2=0x7f8a50161b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:55.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.086+0000 7f8a562de640 1 --2- 172.21.15.112:0/1523055472 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a500d3880 0x7f8a50161b60 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f8a44000c00 tx=0x7f8a44033d60 comp rx=0 tx=0).stop 2024-09-21T00:28:55.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.086+0000 7f8a562de640 1 -- 172.21.15.112:0/1523055472 shutdown_connections 2024-09-21T00:28:55.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.086+0000 7f8a562de640 1 --2- 172.21.15.112:0/1523055472 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8a500d48a0 0x7f8a500b2c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:55.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.086+0000 7f8a562de640 1 --2- 172.21.15.112:0/1523055472 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f8a3c078bc0 0x7f8a3c07b0d0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:55.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.086+0000 7f8a562de640 1 --2- 172.21.15.112:0/1523055472 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a500d3880 0x7f8a50161b60 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:55.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.086+0000 7f8a562de640 1 --2- 172.21.15.112:0/1523055472 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8a500d01b0 0x7f8a501f4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:55.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.086+0000 7f8a562de640 1 -- 172.21.15.112:0/1523055472 >> 172.21.15.112:0/1523055472 conn(0x7f8a5008bfe0 msgr2=0x7f8a50068d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:55.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.087+0000 7f8a562de640 1 -- 172.21.15.112:0/1523055472 shutdown_connections 2024-09-21T00:28:55.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.095+0000 7f8a562de640 1 -- 172.21.15.112:0/1523055472 wait complete. 2024-09-21T00:28:55.122 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T00:28:55.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.235+0000 7f20b4ce5640 1 -- 172.21.15.112:0/1184513754 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f209c003b30 msgr2=0x7f209c003fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:55.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.235+0000 7f20b4ce5640 1 --2- 172.21.15.112:0/1184513754 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f209c003b30 0x7f209c003fa0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f20a0006fd0 tx=0x7f209c005830 comp rx=0 tx=0).stop 2024-09-21T00:28:55.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.235+0000 7f20b4ce5640 1 -- 172.21.15.112:0/1184513754 shutdown_connections 2024-09-21T00:28:55.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.235+0000 7f20b4ce5640 1 --2- 172.21.15.112:0/1184513754 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f209c003b30 0x7f209c003fa0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:55.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.235+0000 7f20b4ce5640 1 -- 172.21.15.112:0/1184513754 >> 172.21.15.112:0/1184513754 conn(0x7f20b00cf220 msgr2=0x7f20b00cf630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:55.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.235+0000 7f20b4ce5640 1 -- 172.21.15.112:0/1184513754 shutdown_connections 2024-09-21T00:28:55.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.235+0000 7f20b4ce5640 1 -- 172.21.15.112:0/1184513754 wait complete. 2024-09-21T00:28:55.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.237+0000 7f20b4ce5640 1 Processor -- start 2024-09-21T00:28:55.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.237+0000 7f20b4ce5640 1 -- start start 2024-09-21T00:28:55.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.238+0000 7f20b4ce5640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f209c003b30 0x7f20b01f3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:55.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.238+0000 7f20b4ce5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20b00c8840 0x7f20b00b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:55.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.238+0000 7f20b4ce5640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f20b01f2f80 0x7f20b00b56f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:55.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.238+0000 7f20b4ce5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f20b00d2130 con 0x7f20b00c8840 2024-09-21T00:28:55.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.238+0000 7f20b4ce5640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f20b00d1fb0 con 0x7f209c003b30 2024-09-21T00:28:55.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.238+0000 7f20b4ce5640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f20b00b41e0 con 0x7f20b01f2f80 2024-09-21T00:28:55.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.238+0000 7f20a7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20b00c8840 0x7f20b00b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:55.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.238+0000 7f20a7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20b00c8840 0x7f20b00b51b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60492/0 (socket says 172.21.15.112:60492) 2024-09-21T00:28:55.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.238+0000 7f20a7fff640 1 -- 172.21.15.112:0/910446708 learned_addr learned my addr 172.21.15.112:0/910446708 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:28:55.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.238+0000 7f20ae59c640 1 --2- 172.21.15.112:0/910446708 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f209c003b30 0x7f20b01f3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:55.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.238+0000 7f20aed9d640 1 --2- 172.21.15.112:0/910446708 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f20b01f2f80 0x7f20b00b56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:55.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.239+0000 7f20a7fff640 1 -- 172.21.15.112:0/910446708 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f20b01f2f80 msgr2=0x7f20b00b56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:55.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.239+0000 7f20a7fff640 1 --2- 172.21.15.112:0/910446708 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f20b01f2f80 0x7f20b00b56f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:55.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.239+0000 7f20a7fff640 1 -- 172.21.15.112:0/910446708 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f209c003b30 msgr2=0x7f20b01f3910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:55.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.239+0000 7f20a7fff640 1 --2- 172.21.15.112:0/910446708 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f209c003b30 0x7f20b01f3910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:55.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.239+0000 7f20a7fff640 1 -- 172.21.15.112:0/910446708 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20b01f67a0 con 0x7f20b00c8840 2024-09-21T00:28:55.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.239+0000 7f20a7fff640 1 --2- 172.21.15.112:0/910446708 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20b00c8840 0x7f20b00b51b0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f2098011ae0 tx=0x7f2098009eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:55.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.239+0000 7f20a6ffd640 1 -- 172.21.15.112:0/910446708 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f209801e070 con 0x7f20b00c8840 2024-09-21T00:28:55.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.239+0000 7f20a6ffd640 1 -- 172.21.15.112:0/910446708 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f209800fe70 con 0x7f20b00c8840 2024-09-21T00:28:55.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.239+0000 7f20a6ffd640 1 -- 172.21.15.112:0/910446708 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2098004290 con 0x7f20b00c8840 2024-09-21T00:28:55.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.239+0000 7f20b4ce5640 1 -- 172.21.15.112:0/910446708 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20b01f3e50 con 0x7f20b00c8840 2024-09-21T00:28:55.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.239+0000 7f20b4ce5640 1 -- 172.21.15.112:0/910446708 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20b01f6d60 con 0x7f20b00c8840 2024-09-21T00:28:55.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.240+0000 7f20b4ce5640 1 -- 172.21.15.112:0/910446708 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20b00b4740 con 0x7f20b00c8840 2024-09-21T00:28:55.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.241+0000 7f20a6ffd640 1 -- 172.21.15.112:0/910446708 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2098021070 con 0x7f20b00c8840 2024-09-21T00:28:55.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.241+0000 7f20a6ffd640 1 --2- 172.21.15.112:0/910446708 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2090078b90 0x7f209007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:55.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.241+0000 7f20a6ffd640 1 -- 172.21.15.112:0/910446708 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6101+0+0 (secure 0 0 0) 0x7f209809b910 con 0x7f20b00c8840 2024-09-21T00:28:55.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.241+0000 7f20ae59c640 1 --2- 172.21.15.112:0/910446708 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2090078b90 0x7f209007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:55.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.242+0000 7f20ae59c640 1 --2- 172.21.15.112:0/910446708 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2090078b90 0x7f209007b0a0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f20a0033000 tx=0x7f20a0038040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:55.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.246+0000 7f20a6ffd640 1 -- 172.21.15.112:0/910446708 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2098063570 con 0x7f20b00c8840 2024-09-21T00:28:55.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.423+0000 7f20b4ce5640 1 -- 172.21.15.112:0/910446708 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f20b00710f0 con 0x7f20b00c8840 2024-09-21T00:28:55.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.424+0000 7f20a6ffd640 1 -- 172.21.15.112:0/910446708 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+16982 (secure 0 0 0) 0x7f2098067fa0 con 0x7f20b00c8840 2024-09-21T00:28:55.425 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:28:55.425 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":82,"fsid":"8b34692b-dc65-49b2-a08f-26f707b96ac5","created":"2024-09-21T00:25:22.788490+0000","modified":"2024-09-21T00:28:54.453271+0000","last_up_change":"2024-09-21T00:25:36.161835+0000","last_in_change":"2024-09-21T00:25:26.346500+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T00:25:37.105006+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-09-21T00:28:52.107455+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-09-21T00:28:53.410975+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9c575086-67d2-4de2-b01b-d44dcc21d604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6809","nonce":755760621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6811","nonce":755760621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6815","nonce":755760621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6813","nonce":755760621}]},"public_addr":"172.21.15.112:6809/755760621","cluster_addr":"172.21.15.112:6811/755760621","heartbeat_back_addr":"172.21.15.112:6815/755760621","heartbeat_front_addr":"172.21.15.112:6813/755760621","state":["exists","up"]},{"osd":1,"uuid":"a983c227-a3ce-4bdd-82f1-438ba7e3ecd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6801","nonce":344932592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6803","nonce":344932592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6807","nonce":344932592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6805","nonce":344932592}]},"public_addr":"172.21.15.112:6801/344932592","cluster_addr":"172.21.15.112:6803/344932592","heartbeat_back_addr":"172.21.15.112:6807/344932592","heartbeat_front_addr":"172.21.15.112:6805/344932592","state":["exists","up"]},{"osd":2,"uuid":"1193c1d8-bfff-4a57-8200-5f69bf1ea3af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6825","nonce":407933616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6827","nonce":407933616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6831","nonce":407933616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6829","nonce":407933616}]},"public_addr":"172.21.15.112:6825/407933616","cluster_addr":"172.21.15.112:6827/407933616","heartbeat_back_addr":"172.21.15.112:6831/407933616","heartbeat_front_addr":"172.21.15.112:6829/407933616","state":["exists","up"]},{"osd":3,"uuid":"af04330a-36ad-40cc-a8dd-fa259b221de5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6817","nonce":4120164806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6819","nonce":4120164806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6823","nonce":4120164806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6821","nonce":4120164806}]},"public_addr":"172.21.15.112:6817/4120164806","cluster_addr":"172.21.15.112:6819/4120164806","heartbeat_back_addr":"172.21.15.112:6823/4120164806","heartbeat_front_addr":"172.21.15.112:6821/4120164806","state":["exists","up"]},{"osd":4,"uuid":"5b893111-292c-4c5a-b184-53ad38c7199e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6803","nonce":522077560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6805","nonce":522077560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6809","nonce":522077560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6807","nonce":522077560}]},"public_addr":"172.21.15.187:6803/522077560","cluster_addr":"172.21.15.187:6805/522077560","heartbeat_back_addr":"172.21.15.187:6809/522077560","heartbeat_front_addr":"172.21.15.187:6807/522077560","state":["exists","up"]},{"osd":5,"uuid":"c0b8e011-45fe-4c5e-925e-6be2a7f1868a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6827","nonce":3593144721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6829","nonce":3593144721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6833","nonce":3593144721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6831","nonce":3593144721}]},"public_addr":"172.21.15.187:6827/3593144721","cluster_addr":"172.21.15.187:6829/3593144721","heartbeat_back_addr":"172.21.15.187:6833/3593144721","heartbeat_front_addr":"172.21.15.187:6831/3593144721","state":["exists","up"]},{"osd":6,"uuid":"87d2d650-7489-4593-98a3-a1f433ce8dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6811","nonce":1494387657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6813","nonce":1494387657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6817","nonce":1494387657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6815","nonce":1494387657}]},"public_addr":"172.21.15.187:6811/1494387657","cluster_addr":"172.21.15.187:6813/1494387657","heartbeat_back_addr":"172.21.15.187:6817/1494387657","heartbeat_front_addr":"172.21.15.187:6815/1494387657","state":["exists","up"]},{"osd":7,"uuid":"c6b4f8e2-a1ee-4438-aa37-67ff4b6ef1f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6819","nonce":1624699842}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6821","nonce":1624699842}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6825","nonce":1624699842}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6823","nonce":1624699842}]},"public_addr":"172.21.15.187:6819/1624699842","cluster_addr":"172.21.15.187:6821/1624699842","heartbeat_back_addr":"172.21.15.187:6825/1624699842","heartbeat_front_addr":"172.21.15.187:6823/1624699842","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.873162+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.400506+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.664317+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.159791+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.202630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.113212+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.384786+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.448441+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T00:28:55.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.426+0000 7f20b4ce5640 1 -- 172.21.15.112:0/910446708 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2090078b90 msgr2=0x7f209007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:55.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.426+0000 7f20b4ce5640 1 --2- 172.21.15.112:0/910446708 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2090078b90 0x7f209007b0a0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f20a0033000 tx=0x7f20a0038040 comp rx=0 tx=0).stop 2024-09-21T00:28:55.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.426+0000 7f20b4ce5640 1 -- 172.21.15.112:0/910446708 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20b00c8840 msgr2=0x7f20b00b51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:55.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.426+0000 7f20b4ce5640 1 --2- 172.21.15.112:0/910446708 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20b00c8840 0x7f20b00b51b0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f2098011ae0 tx=0x7f2098009eb0 comp rx=0 tx=0).stop 2024-09-21T00:28:55.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.426+0000 7f20b4ce5640 1 -- 172.21.15.112:0/910446708 shutdown_connections 2024-09-21T00:28:55.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.426+0000 7f20b4ce5640 1 --2- 172.21.15.112:0/910446708 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f20b01f2f80 0x7f20b00b56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:55.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.426+0000 7f20b4ce5640 1 --2- 172.21.15.112:0/910446708 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2090078b90 0x7f209007b0a0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:55.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.426+0000 7f20b4ce5640 1 --2- 172.21.15.112:0/910446708 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20b00c8840 0x7f20b00b51b0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:55.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.426+0000 7f20b4ce5640 1 --2- 172.21.15.112:0/910446708 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f209c003b30 0x7f20b01f3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:55.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.426+0000 7f20b4ce5640 1 -- 172.21.15.112:0/910446708 >> 172.21.15.112:0/910446708 conn(0x7f20b00cf220 msgr2=0x7f20b015f170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:55.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.426+0000 7f20b4ce5640 1 -- 172.21.15.112:0/910446708 shutdown_connections 2024-09-21T00:28:55.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.426+0000 7f20b4ce5640 1 -- 172.21.15.112:0/910446708 wait complete. 2024-09-21T00:28:55.439 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T00:28:55.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.551+0000 7f43b3e0e640 1 -- 172.21.15.112:0/285604807 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f43ac0c6830 msgr2=0x7f43ac0b1d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:55.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.551+0000 7f43b3e0e640 1 --2- 172.21.15.112:0/285604807 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f43ac0c6830 0x7f43ac0b1d30 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f43ac0b39c0 tx=0x7f43a402a980 comp rx=0 tx=0).stop 2024-09-21T00:28:55.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.551+0000 7f43b3e0e640 1 -- 172.21.15.112:0/285604807 shutdown_connections 2024-09-21T00:28:55.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.551+0000 7f43b3e0e640 1 --2- 172.21.15.112:0/285604807 >> v2:172.21.15.187:3300/0 conn(0x7f43ac0c5ee0 0x7f43ac0c62f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:55.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.551+0000 7f43b3e0e640 1 --2- 172.21.15.112:0/285604807 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f43ac0c6830 0x7f43ac0b1d30 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:55.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.551+0000 7f43b3e0e640 1 --2- 172.21.15.112:0/285604807 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f43ac0ccbf0 0x7f43ac0cd000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:55.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.551+0000 7f43b3e0e640 1 -- 172.21.15.112:0/285604807 >> 172.21.15.112:0/285604807 conn(0x7f43ac0b42e0 msgr2=0x7f43ac0b4720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:55.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.551+0000 7f43b3e0e640 1 -- 172.21.15.112:0/285604807 shutdown_connections 2024-09-21T00:28:55.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.551+0000 7f43b3e0e640 1 -- 172.21.15.112:0/285604807 wait complete. 2024-09-21T00:28:55.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.553+0000 7f43b3e0e640 1 Processor -- start 2024-09-21T00:28:55.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.553+0000 7f43b3e0e640 1 -- start start 2024-09-21T00:28:55.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.554+0000 7f43b3e0e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f43ac0c5ee0 0x7f43ac0ab4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:55.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.554+0000 7f43b3e0e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f43ac0c6830 0x7f43ac0b0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:55.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.554+0000 7f43b3e0e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f43ac0ccbf0 0x7f43ac1f2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:55.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.554+0000 7f43b3e0e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f43ac0b2270 con 0x7f43ac0ccbf0 2024-09-21T00:28:55.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.554+0000 7f43b3e0e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f43ac0c6e50 con 0x7f43ac0c5ee0 2024-09-21T00:28:55.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.554+0000 7f43b3e0e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f43ac0b23f0 con 0x7f43ac0c6830 2024-09-21T00:28:55.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.555+0000 7f43b23ab640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f43ac0ccbf0 0x7f43ac1f2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:55.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.555+0000 7f43b23ab640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f43ac0ccbf0 0x7f43ac1f2ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60518/0 (socket says 172.21.15.112:60518) 2024-09-21T00:28:55.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.555+0000 7f43b23ab640 1 -- 172.21.15.112:0/195532006 learned_addr learned my addr 172.21.15.112:0/195532006 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:28:55.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.555+0000 7f43b1baa640 1 --2- 172.21.15.112:0/195532006 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f43ac0c5ee0 0x7f43ac0ab4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:55.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.555+0000 7f43b13a9640 1 --2- 172.21.15.112:0/195532006 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f43ac0c6830 0x7f43ac0b0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:55.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.555+0000 7f43b23ab640 1 -- 172.21.15.112:0/195532006 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f43ac0c6830 msgr2=0x7f43ac0b0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:55.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.555+0000 7f43b23ab640 1 --2- 172.21.15.112:0/195532006 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f43ac0c6830 0x7f43ac0b0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:55.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.555+0000 7f43b23ab640 1 -- 172.21.15.112:0/195532006 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f43ac0c5ee0 msgr2=0x7f43ac0ab4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:55.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.555+0000 7f43b23ab640 1 --2- 172.21.15.112:0/195532006 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f43ac0c5ee0 0x7f43ac0ab4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:55.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.555+0000 7f43b23ab640 1 -- 172.21.15.112:0/195532006 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43ac1f3db0 con 0x7f43ac0ccbf0 2024-09-21T00:28:55.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.555+0000 7f43b23ab640 1 --2- 172.21.15.112:0/195532006 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f43ac0ccbf0 0x7f43ac1f2ef0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f43a4033040 tx=0x7f43a4003ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:55.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.555+0000 7f43a27fc640 1 -- 172.21.15.112:0/195532006 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43a4034040 con 0x7f43ac0ccbf0 2024-09-21T00:28:55.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.555+0000 7f43a27fc640 1 -- 172.21.15.112:0/195532006 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f43a4004640 con 0x7f43ac0ccbf0 2024-09-21T00:28:55.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.555+0000 7f43b3e0e640 1 -- 172.21.15.112:0/195532006 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43ac1f3ad0 con 0x7f43ac0ccbf0 2024-09-21T00:28:55.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.555+0000 7f43a27fc640 1 -- 172.21.15.112:0/195532006 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43a4045480 con 0x7f43ac0ccbf0 2024-09-21T00:28:55.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.555+0000 7f43b3e0e640 1 -- 172.21.15.112:0/195532006 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43ac0aacf0 con 0x7f43ac0ccbf0 2024-09-21T00:28:55.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.556+0000 7f43b3e0e640 1 -- 172.21.15.112:0/195532006 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43ac0ab130 con 0x7f43ac0ccbf0 2024-09-21T00:28:55.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.557+0000 7f43a27fc640 1 -- 172.21.15.112:0/195532006 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f43a4037020 con 0x7f43ac0ccbf0 2024-09-21T00:28:55.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.558+0000 7f43a27fc640 1 --2- 172.21.15.112:0/195532006 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4390078b90 0x7f439007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:55.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.558+0000 7f43a27fc640 1 -- 172.21.15.112:0/195532006 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6101+0+0 (secure 0 0 0) 0x7f43a40bc5f0 con 0x7f43ac0ccbf0 2024-09-21T00:28:55.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.558+0000 7f43b1baa640 1 --2- 172.21.15.112:0/195532006 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4390078b90 0x7f439007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:55.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.558+0000 7f43b1baa640 1 --2- 172.21.15.112:0/195532006 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4390078b90 0x7f439007b0a0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f43a8005fd0 tx=0x7f43a800a250 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:55.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.562+0000 7f43a27fc640 1 -- 172.21.15.112:0/195532006 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f43a4084370 con 0x7f43ac0ccbf0 2024-09-21T00:28:55.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.739+0000 7f43b3e0e640 1 -- 172.21.15.112:0/195532006 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f43ac0cc770 con 0x7f43ac0ccbf0 2024-09-21T00:28:55.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.740+0000 7f43a27fc640 1 -- 172.21.15.112:0/195532006 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+16982 (secure 0 0 0) 0x7f43a4088da0 con 0x7f43ac0ccbf0 2024-09-21T00:28:55.741 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:28:55.741 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":82,"fsid":"8b34692b-dc65-49b2-a08f-26f707b96ac5","created":"2024-09-21T00:25:22.788490+0000","modified":"2024-09-21T00:28:54.453271+0000","last_up_change":"2024-09-21T00:25:36.161835+0000","last_in_change":"2024-09-21T00:25:26.346500+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T00:25:37.105006+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-09-21T00:28:52.107455+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-09-21T00:28:53.410975+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9c575086-67d2-4de2-b01b-d44dcc21d604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6809","nonce":755760621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6811","nonce":755760621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6815","nonce":755760621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6813","nonce":755760621}]},"public_addr":"172.21.15.112:6809/755760621","cluster_addr":"172.21.15.112:6811/755760621","heartbeat_back_addr":"172.21.15.112:6815/755760621","heartbeat_front_addr":"172.21.15.112:6813/755760621","state":["exists","up"]},{"osd":1,"uuid":"a983c227-a3ce-4bdd-82f1-438ba7e3ecd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6801","nonce":344932592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6803","nonce":344932592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6807","nonce":344932592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6805","nonce":344932592}]},"public_addr":"172.21.15.112:6801/344932592","cluster_addr":"172.21.15.112:6803/344932592","heartbeat_back_addr":"172.21.15.112:6807/344932592","heartbeat_front_addr":"172.21.15.112:6805/344932592","state":["exists","up"]},{"osd":2,"uuid":"1193c1d8-bfff-4a57-8200-5f69bf1ea3af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6825","nonce":407933616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6827","nonce":407933616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6831","nonce":407933616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6829","nonce":407933616}]},"public_addr":"172.21.15.112:6825/407933616","cluster_addr":"172.21.15.112:6827/407933616","heartbeat_back_addr":"172.21.15.112:6831/407933616","heartbeat_front_addr":"172.21.15.112:6829/407933616","state":["exists","up"]},{"osd":3,"uuid":"af04330a-36ad-40cc-a8dd-fa259b221de5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6817","nonce":4120164806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6819","nonce":4120164806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6823","nonce":4120164806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6821","nonce":4120164806}]},"public_addr":"172.21.15.112:6817/4120164806","cluster_addr":"172.21.15.112:6819/4120164806","heartbeat_back_addr":"172.21.15.112:6823/4120164806","heartbeat_front_addr":"172.21.15.112:6821/4120164806","state":["exists","up"]},{"osd":4,"uuid":"5b893111-292c-4c5a-b184-53ad38c7199e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6803","nonce":522077560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6805","nonce":522077560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6809","nonce":522077560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6807","nonce":522077560}]},"public_addr":"172.21.15.187:6803/522077560","cluster_addr":"172.21.15.187:6805/522077560","heartbeat_back_addr":"172.21.15.187:6809/522077560","heartbeat_front_addr":"172.21.15.187:6807/522077560","state":["exists","up"]},{"osd":5,"uuid":"c0b8e011-45fe-4c5e-925e-6be2a7f1868a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6827","nonce":3593144721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6829","nonce":3593144721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6833","nonce":3593144721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6831","nonce":3593144721}]},"public_addr":"172.21.15.187:6827/3593144721","cluster_addr":"172.21.15.187:6829/3593144721","heartbeat_back_addr":"172.21.15.187:6833/3593144721","heartbeat_front_addr":"172.21.15.187:6831/3593144721","state":["exists","up"]},{"osd":6,"uuid":"87d2d650-7489-4593-98a3-a1f433ce8dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6811","nonce":1494387657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6813","nonce":1494387657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6817","nonce":1494387657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6815","nonce":1494387657}]},"public_addr":"172.21.15.187:6811/1494387657","cluster_addr":"172.21.15.187:6813/1494387657","heartbeat_back_addr":"172.21.15.187:6817/1494387657","heartbeat_front_addr":"172.21.15.187:6815/1494387657","state":["exists","up"]},{"osd":7,"uuid":"c6b4f8e2-a1ee-4438-aa37-67ff4b6ef1f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6819","nonce":1624699842}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6821","nonce":1624699842}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6825","nonce":1624699842}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6823","nonce":1624699842}]},"public_addr":"172.21.15.187:6819/1624699842","cluster_addr":"172.21.15.187:6821/1624699842","heartbeat_back_addr":"172.21.15.187:6825/1624699842","heartbeat_front_addr":"172.21.15.187:6823/1624699842","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.873162+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.400506+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.664317+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.159791+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.202630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.113212+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.384786+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.448441+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T00:28:55.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.742+0000 7f43b3e0e640 1 -- 172.21.15.112:0/195532006 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4390078b90 msgr2=0x7f439007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:55.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.742+0000 7f43b3e0e640 1 --2- 172.21.15.112:0/195532006 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4390078b90 0x7f439007b0a0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f43a8005fd0 tx=0x7f43a800a250 comp rx=0 tx=0).stop 2024-09-21T00:28:55.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.742+0000 7f43b3e0e640 1 -- 172.21.15.112:0/195532006 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f43ac0ccbf0 msgr2=0x7f43ac1f2ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:55.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.742+0000 7f43b3e0e640 1 --2- 172.21.15.112:0/195532006 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f43ac0ccbf0 0x7f43ac1f2ef0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f43a4033040 tx=0x7f43a4003ef0 comp rx=0 tx=0).stop 2024-09-21T00:28:55.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.743+0000 7f43b3e0e640 1 -- 172.21.15.112:0/195532006 shutdown_connections 2024-09-21T00:28:55.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.743+0000 7f43b3e0e640 1 --2- 172.21.15.112:0/195532006 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4390078b90 0x7f439007b0a0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:55.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.743+0000 7f43b3e0e640 1 --2- 172.21.15.112:0/195532006 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f43ac0ccbf0 0x7f43ac1f2ef0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:55.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.743+0000 7f43b3e0e640 1 --2- 172.21.15.112:0/195532006 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f43ac0c6830 0x7f43ac0b0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:55.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.743+0000 7f43b3e0e640 1 --2- 172.21.15.112:0/195532006 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f43ac0c5ee0 0x7f43ac0ab4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:55.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.743+0000 7f43b3e0e640 1 -- 172.21.15.112:0/195532006 >> 172.21.15.112:0/195532006 conn(0x7f43ac0b42e0 msgr2=0x7f43ac072160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:55.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.743+0000 7f43b3e0e640 1 -- 172.21.15.112:0/195532006 shutdown_connections 2024-09-21T00:28:55.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.743+0000 7f43b3e0e640 1 -- 172.21.15.112:0/195532006 wait complete. 2024-09-21T00:28:55.755 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-21T00:28:55.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.871+0000 7f31631ad640 1 -- 172.21.15.112:0/315886364 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f315c0c6830 msgr2=0x7f315c0b1d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:55.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.871+0000 7f31631ad640 1 --2- 172.21.15.112:0/315886364 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f315c0c6830 0x7f315c0b1d30 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f315c0b39c0 tx=0x7f315402a980 comp rx=0 tx=0).stop 2024-09-21T00:28:55.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.871+0000 7f31631ad640 1 -- 172.21.15.112:0/315886364 shutdown_connections 2024-09-21T00:28:55.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.871+0000 7f31631ad640 1 --2- 172.21.15.112:0/315886364 >> v2:172.21.15.187:3300/0 conn(0x7f315c0c5ee0 0x7f315c0c62f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:55.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.871+0000 7f31631ad640 1 --2- 172.21.15.112:0/315886364 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f315c0c6830 0x7f315c0b1d30 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:55.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.871+0000 7f31631ad640 1 --2- 172.21.15.112:0/315886364 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f315c0ccbf0 0x7f315c0cd000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:55.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.871+0000 7f31631ad640 1 -- 172.21.15.112:0/315886364 >> 172.21.15.112:0/315886364 conn(0x7f315c0b42e0 msgr2=0x7f315c0b4720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:55.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.871+0000 7f31631ad640 1 -- 172.21.15.112:0/315886364 shutdown_connections 2024-09-21T00:28:55.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.871+0000 7f31631ad640 1 -- 172.21.15.112:0/315886364 wait complete. 2024-09-21T00:28:55.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.873+0000 7f31631ad640 1 Processor -- start 2024-09-21T00:28:55.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.873+0000 7f31631ad640 1 -- start start 2024-09-21T00:28:55.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.874+0000 7f31631ad640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f315c0c5ee0 0x7f315c0cbf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:55.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.874+0000 7f31631ad640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f315c0c6830 0x7f315c0d25c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:55.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.874+0000 7f31631ad640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f315c0ccbf0 0x7f315c1e1b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:55.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.874+0000 7f31631ad640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f315c0b2270 con 0x7f315c0c5ee0 2024-09-21T00:28:55.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.874+0000 7f31631ad640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f315c0c6e50 con 0x7f315c0c6830 2024-09-21T00:28:55.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.874+0000 7f31631ad640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f315c0b23f0 con 0x7f315c0ccbf0 2024-09-21T00:28:55.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.874+0000 7f3160f49640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f315c0c5ee0 0x7f315c0cbf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:55.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.874+0000 7f3160f49640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f315c0c5ee0 0x7f315c0cbf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60534/0 (socket says 172.21.15.112:60534) 2024-09-21T00:28:55.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.874+0000 7f3160f49640 1 -- 172.21.15.112:0/83106317 learned_addr learned my addr 172.21.15.112:0/83106317 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:28:55.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.874+0000 7f316174a640 1 --2- 172.21.15.112:0/83106317 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f315c0ccbf0 0x7f315c1e1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:55.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.874+0000 7f3153fff640 1 --2- 172.21.15.112:0/83106317 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f315c0c6830 0x7f315c0d25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:55.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.874+0000 7f3160f49640 1 -- 172.21.15.112:0/83106317 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f315c0ccbf0 msgr2=0x7f315c1e1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:55.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.874+0000 7f3160f49640 1 --2- 172.21.15.112:0/83106317 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f315c0ccbf0 0x7f315c1e1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:55.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.874+0000 7f3160f49640 1 -- 172.21.15.112:0/83106317 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f315c0c6830 msgr2=0x7f315c0d25c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:55.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.874+0000 7f3160f49640 1 --2- 172.21.15.112:0/83106317 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f315c0c6830 0x7f315c0d25c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:55.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.874+0000 7f3160f49640 1 -- 172.21.15.112:0/83106317 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f315c0708c0 con 0x7f315c0c5ee0 2024-09-21T00:28:55.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.874+0000 7f3153fff640 1 --2- 172.21.15.112:0/83106317 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f315c0c6830 0x7f315c0d25c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:28:55.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.874+0000 7f316174a640 1 --2- 172.21.15.112:0/83106317 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f315c0ccbf0 0x7f315c1e1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:28:55.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.875+0000 7f3160f49640 1 --2- 172.21.15.112:0/83106317 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f315c0c5ee0 0x7f315c0cbf60 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f315800e7f0 tx=0x7f315800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:55.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.875+0000 7f31517fa640 1 -- 172.21.15.112:0/83106317 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f315800ccc0 con 0x7f315c0c5ee0 2024-09-21T00:28:55.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.875+0000 7f31517fa640 1 -- 172.21.15.112:0/83106317 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f315800ce80 con 0x7f315c0c5ee0 2024-09-21T00:28:55.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.875+0000 7f31631ad640 1 -- 172.21.15.112:0/83106317 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f315c070ba0 con 0x7f315c0c5ee0 2024-09-21T00:28:55.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.875+0000 7f31517fa640 1 -- 172.21.15.112:0/83106317 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3158005280 con 0x7f315c0c5ee0 2024-09-21T00:28:55.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.875+0000 7f31631ad640 1 -- 172.21.15.112:0/83106317 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f315c1e2b20 con 0x7f315c0c5ee0 2024-09-21T00:28:55.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.876+0000 7f31631ad640 1 -- 172.21.15.112:0/83106317 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f315c0008d0 con 0x7f315c0c5ee0 2024-09-21T00:28:55.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.877+0000 7f31517fa640 1 -- 172.21.15.112:0/83106317 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3158005420 con 0x7f315c0c5ee0 2024-09-21T00:28:55.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.877+0000 7f31517fa640 1 --2- 172.21.15.112:0/83106317 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f3148078b90 0x7f314807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:55.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.877+0000 7f31517fa640 1 -- 172.21.15.112:0/83106317 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6101+0+0 (secure 0 0 0) 0x7f315809b900 con 0x7f315c0c5ee0 2024-09-21T00:28:55.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.877+0000 7f3153fff640 1 --2- 172.21.15.112:0/83106317 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f3148078b90 0x7f314807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:55.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.878+0000 7f3153fff640 1 --2- 172.21.15.112:0/83106317 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f3148078b90 0x7f314807b0a0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f315c06f7c0 tx=0x7f314c005e70 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:55.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:55.882+0000 7f31517fa640 1 -- 172.21.15.112:0/83106317 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3158063bb0 con 0x7f315c0c5ee0 2024-09-21T00:28:56.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:56.098+0000 7f31631ad640 1 -- 172.21.15.112:0/83106317 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f315c075bc0 con 0x7f315c0c5ee0 2024-09-21T00:28:57.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:57.105+0000 7f31517fa640 1 -- 172.21.15.112:0/83106317 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v54) ==== 118+0+0 (secure 0 0 0) 0x7f31580634b0 con 0x7f315c0c5ee0 2024-09-21T00:28:57.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:57.108+0000 7f31631ad640 1 -- 172.21.15.112:0/83106317 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f3148078b90 msgr2=0x7f314807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:57.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:57.108+0000 7f31631ad640 1 --2- 172.21.15.112:0/83106317 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f3148078b90 0x7f314807b0a0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f315c06f7c0 tx=0x7f314c005e70 comp rx=0 tx=0).stop 2024-09-21T00:28:57.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:57.108+0000 7f31631ad640 1 -- 172.21.15.112:0/83106317 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f315c0c5ee0 msgr2=0x7f315c0cbf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:57.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:57.108+0000 7f31631ad640 1 --2- 172.21.15.112:0/83106317 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f315c0c5ee0 0x7f315c0cbf60 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f315800e7f0 tx=0x7f315800ecb0 comp rx=0 tx=0).stop 2024-09-21T00:28:57.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:57.108+0000 7f31631ad640 1 -- 172.21.15.112:0/83106317 shutdown_connections 2024-09-21T00:28:57.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:57.108+0000 7f31631ad640 1 --2- 172.21.15.112:0/83106317 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f315c0ccbf0 0x7f315c1e1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:57.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:57.108+0000 7f31631ad640 1 --2- 172.21.15.112:0/83106317 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f315c0c6830 0x7f315c0d25c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:57.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:57.108+0000 7f31631ad640 1 --2- 172.21.15.112:0/83106317 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f3148078b90 0x7f314807b0a0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:57.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:57.108+0000 7f31631ad640 1 --2- 172.21.15.112:0/83106317 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f315c0c5ee0 0x7f315c0cbf60 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:57.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:57.108+0000 7f31631ad640 1 -- 172.21.15.112:0/83106317 >> 172.21.15.112:0/83106317 conn(0x7f315c0b42e0 msgr2=0x7f315c0721f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:57.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:57.108+0000 7f31631ad640 1 -- 172.21.15.112:0/83106317 shutdown_connections 2024-09-21T00:28:57.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:57.108+0000 7f31631ad640 1 -- 172.21.15.112:0/83106317 wait complete. 2024-09-21T00:28:57.123 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-21T00:28:57.123 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-21T00:28:57.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:57.244+0000 7feb56939640 1 -- 172.21.15.112:0/311221445 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb38004800 msgr2=0x7feb38004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:57.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:57.244+0000 7feb56939640 1 --2- 172.21.15.112:0/311221445 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb38004800 0x7feb38004c10 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7feb4400baf0 tx=0x7feb38005830 comp rx=0 tx=0).stop 2024-09-21T00:28:57.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:57.244+0000 7feb54ed6640 1 -- 172.21.15.112:0/311221445 reap_dead start 2024-09-21T00:28:57.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:57.245+0000 7feb56939640 1 -- 172.21.15.112:0/311221445 shutdown_connections 2024-09-21T00:28:57.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:57.245+0000 7feb56939640 1 -- 172.21.15.112:0/311221445 >> 172.21.15.112:0/311221445 conn(0x7feb5008bfe0 msgr2=0x7feb5009d890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:57.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:57.245+0000 7feb56939640 1 -- 172.21.15.112:0/311221445 shutdown_connections 2024-09-21T00:28:57.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:57.245+0000 7feb56939640 1 -- 172.21.15.112:0/311221445 wait complete. 2024-09-21T00:28:57.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:57.246+0000 7feb56939640 1 Processor -- start 2024-09-21T00:28:57.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:57.247+0000 7feb56939640 1 -- start start 2024-09-21T00:28:57.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:57.248+0000 7feb56939640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb38004800 0x7feb500cdf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:57.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:57.248+0000 7feb56939640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feb501f69a0 0x7feb500ca0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:57.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:57.248+0000 7feb56939640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feb500ce490 0x7feb500ca5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:57.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:57.248+0000 7feb56939640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7feb50163860 con 0x7feb38004800 2024-09-21T00:28:57.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:57.248+0000 7feb56939640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7feb501636e0 con 0x7feb501f69a0 2024-09-21T00:28:57.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:57.248+0000 7feb56939640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7feb500d20d0 con 0x7feb500ce490 2024-09-21T00:28:57.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:57.248+0000 7feb4ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb38004800 0x7feb500cdf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:57.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:57.248+0000 7feb4ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb38004800 0x7feb500cdf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38094/0 (socket says 172.21.15.112:38094) 2024-09-21T00:28:57.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:57.248+0000 7feb4ffff640 1 -- 172.21.15.112:0/37220282 learned_addr learned my addr 172.21.15.112:0/37220282 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:28:57.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:57.248+0000 7feb54ed6640 1 --2- 172.21.15.112:0/37220282 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feb500ce490 0x7feb500ca5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:57.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:57.248+0000 7feb4f7fe640 1 --2- 172.21.15.112:0/37220282 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feb501f69a0 0x7feb500ca0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:57.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:57.248+0000 7feb4ffff640 1 -- 172.21.15.112:0/37220282 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feb500ce490 msgr2=0x7feb500ca5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:57.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:57.248+0000 7feb4ffff640 1 --2- 172.21.15.112:0/37220282 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feb500ce490 0x7feb500ca5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:57.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:57.248+0000 7feb4ffff640 1 -- 172.21.15.112:0/37220282 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feb501f69a0 msgr2=0x7feb500ca0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:57.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:57.248+0000 7feb4ffff640 1 --2- 172.21.15.112:0/37220282 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feb501f69a0 0x7feb500ca0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:57.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:57.248+0000 7feb4ffff640 1 -- 172.21.15.112:0/37220282 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb5014e190 con 0x7feb38004800 2024-09-21T00:28:57.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:57.248+0000 7feb54ed6640 1 --2- 172.21.15.112:0/37220282 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feb500ce490 0x7feb500ca5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:28:57.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:57.248+0000 7feb4f7fe640 1 --2- 172.21.15.112:0/37220282 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feb501f69a0 0x7feb500ca0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:28:57.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:57.248+0000 7feb4ffff640 1 --2- 172.21.15.112:0/37220282 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb38004800 0x7feb500cdf50 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7feb40006630 tx=0x7feb400039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:57.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:57.249+0000 7feb4cff9640 1 -- 172.21.15.112:0/37220282 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb40015070 con 0x7feb38004800 2024-09-21T00:28:57.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:57.249+0000 7feb4cff9640 1 -- 172.21.15.112:0/37220282 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feb40011900 con 0x7feb38004800 2024-09-21T00:28:57.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:57.249+0000 7feb56939640 1 -- 172.21.15.112:0/37220282 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb500c9000 con 0x7feb38004800 2024-09-21T00:28:57.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:57.249+0000 7feb4cff9640 1 -- 172.21.15.112:0/37220282 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb40011d30 con 0x7feb38004800 2024-09-21T00:28:57.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:57.249+0000 7feb56939640 1 -- 172.21.15.112:0/37220282 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb500c95c0 con 0x7feb38004800 2024-09-21T00:28:57.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:57.250+0000 7feb56939640 1 -- 172.21.15.112:0/37220282 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb500d05c0 con 0x7feb38004800 2024-09-21T00:28:57.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:57.250+0000 7feb4cff9640 1 -- 172.21.15.112:0/37220282 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7feb40018070 con 0x7feb38004800 2024-09-21T00:28:57.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:57.251+0000 7feb4cff9640 1 --2- 172.21.15.112:0/37220282 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7feb3c078b90 0x7feb3c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:57.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:57.251+0000 7feb4cff9640 1 -- 172.21.15.112:0/37220282 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 6101+0+0 (secure 0 0 0) 0x7feb4009bd50 con 0x7feb38004800 2024-09-21T00:28:57.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:57.251+0000 7feb4f7fe640 1 --2- 172.21.15.112:0/37220282 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7feb3c078b90 0x7feb3c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:57.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:57.252+0000 7feb4f7fe640 1 --2- 172.21.15.112:0/37220282 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7feb3c078b90 0x7feb3c07b0a0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7feb44036000 tx=0x7feb440046c0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:57.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:57.255+0000 7feb4cff9640 1 -- 172.21.15.112:0/37220282 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7feb4009f030 con 0x7feb38004800 2024-09-21T00:28:57.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:57.468+0000 7feb56939640 1 -- 172.21.15.112:0/37220282 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7feb500b6f90 con 0x7feb38004800 2024-09-21T00:28:58.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.109+0000 7feb4cff9640 1 -- 172.21.15.112:0/37220282 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v55) ==== 122+0+0 (secure 0 0 0) 0x7feb5014e190 con 0x7feb38004800 2024-09-21T00:28:58.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.111+0000 7feb56939640 1 -- 172.21.15.112:0/37220282 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7feb3c078b90 msgr2=0x7feb3c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:58.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.111+0000 7feb56939640 1 --2- 172.21.15.112:0/37220282 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7feb3c078b90 0x7feb3c07b0a0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7feb44036000 tx=0x7feb440046c0 comp rx=0 tx=0).stop 2024-09-21T00:28:58.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.111+0000 7feb56939640 1 -- 172.21.15.112:0/37220282 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb38004800 msgr2=0x7feb500cdf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:58.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.111+0000 7feb56939640 1 --2- 172.21.15.112:0/37220282 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb38004800 0x7feb500cdf50 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7feb40006630 tx=0x7feb400039c0 comp rx=0 tx=0).stop 2024-09-21T00:28:58.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.111+0000 7feb56939640 1 -- 172.21.15.112:0/37220282 shutdown_connections 2024-09-21T00:28:58.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.111+0000 7feb56939640 1 --2- 172.21.15.112:0/37220282 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feb500ce490 0x7feb500ca5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:58.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.111+0000 7feb56939640 1 --2- 172.21.15.112:0/37220282 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feb501f69a0 0x7feb500ca0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:58.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.111+0000 7feb56939640 1 --2- 172.21.15.112:0/37220282 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7feb3c078b90 0x7feb3c07b0a0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:58.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.111+0000 7feb56939640 1 --2- 172.21.15.112:0/37220282 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb38004800 0x7feb500cdf50 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:58.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.111+0000 7feb56939640 1 -- 172.21.15.112:0/37220282 >> 172.21.15.112:0/37220282 conn(0x7feb5008bfe0 msgr2=0x7feb500b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:58.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.111+0000 7feb56939640 1 -- 172.21.15.112:0/37220282 shutdown_connections 2024-09-21T00:28:58.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.111+0000 7feb56939640 1 -- 172.21.15.112:0/37220282 wait complete. 2024-09-21T00:28:58.124 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-21T00:28:58.124 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T00:28:58.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.236+0000 7f46e284d640 1 -- 172.21.15.112:0/1702008688 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f46dc0d2030 msgr2=0x7f46dc15d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:58.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.236+0000 7f46e284d640 1 --2- 172.21.15.112:0/1702008688 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f46dc0d2030 0x7f46dc15d870 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f46cc009830 tx=0x7f46cc02a510 comp rx=0 tx=0).stop 2024-09-21T00:28:58.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.236+0000 7f46e284d640 1 -- 172.21.15.112:0/1702008688 shutdown_connections 2024-09-21T00:28:58.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.236+0000 7f46e284d640 1 --2- 172.21.15.112:0/1702008688 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f46dc0d0860 0x7f46dc15a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:58.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.236+0000 7f46e284d640 1 --2- 172.21.15.112:0/1702008688 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f46dc0d2030 0x7f46dc15d870 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:58.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.236+0000 7f46e284d640 1 -- 172.21.15.112:0/1702008688 >> 172.21.15.112:0/1702008688 conn(0x7f46dc0b43f0 msgr2=0x7f46dc0b4830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:58.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.236+0000 7f46e284d640 1 -- 172.21.15.112:0/1702008688 shutdown_connections 2024-09-21T00:28:58.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.236+0000 7f46e284d640 1 -- 172.21.15.112:0/1702008688 wait complete. 2024-09-21T00:28:58.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.238+0000 7f46e284d640 1 Processor -- start 2024-09-21T00:28:58.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.238+0000 7f46e284d640 1 -- start start 2024-09-21T00:28:58.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.239+0000 7f46e284d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f46dc0ccae0 0x7f46dc0cae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:58.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.239+0000 7f46e284d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46dc0d0860 0x7f46dc14b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:58.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.239+0000 7f46e284d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f46dc0d2030 0x7f46dc15ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:58.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.239+0000 7f46e284d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f46dc0b22d0 con 0x7f46dc0d0860 2024-09-21T00:28:58.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.239+0000 7f46e284d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f46dc0b2150 con 0x7f46dc0ccae0 2024-09-21T00:28:58.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.239+0000 7f46e284d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f46dc0b2450 con 0x7f46dc0d2030 2024-09-21T00:28:58.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.240+0000 7f46db7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46dc0d0860 0x7f46dc14b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:58.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.240+0000 7f46db7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46dc0d0860 0x7f46dc14b020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38108/0 (socket says 172.21.15.112:38108) 2024-09-21T00:28:58.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.240+0000 7f46db7fe640 1 -- 172.21.15.112:0/1210697893 learned_addr learned my addr 172.21.15.112:0/1210697893 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:28:58.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.240+0000 7f46e0dea640 1 --2- 172.21.15.112:0/1210697893 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f46dc0d2030 0x7f46dc15ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:58.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.240+0000 7f46dbfff640 1 --2- 172.21.15.112:0/1210697893 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f46dc0ccae0 0x7f46dc0cae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:58.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.240+0000 7f46db7fe640 1 -- 172.21.15.112:0/1210697893 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f46dc0d2030 msgr2=0x7f46dc15ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:58.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.240+0000 7f46db7fe640 1 --2- 172.21.15.112:0/1210697893 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f46dc0d2030 0x7f46dc15ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:58.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.240+0000 7f46db7fe640 1 -- 172.21.15.112:0/1210697893 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f46dc0ccae0 msgr2=0x7f46dc0cae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:58.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.240+0000 7f46db7fe640 1 --2- 172.21.15.112:0/1210697893 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f46dc0ccae0 0x7f46dc0cae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:58.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.240+0000 7f46db7fe640 1 -- 172.21.15.112:0/1210697893 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46dc14b560 con 0x7f46dc0d0860 2024-09-21T00:28:58.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.240+0000 7f46dbfff640 1 --2- 172.21.15.112:0/1210697893 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f46dc0ccae0 0x7f46dc0cae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:28:58.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.240+0000 7f46db7fe640 1 --2- 172.21.15.112:0/1210697893 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46dc0d0860 0x7f46dc14b020 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f46c400e7c0 tx=0x7f46c400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:58.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.240+0000 7f46d8ff9640 1 -- 172.21.15.112:0/1210697893 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c400ccc0 con 0x7f46dc0d0860 2024-09-21T00:28:58.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.240+0000 7f46d8ff9640 1 -- 172.21.15.112:0/1210697893 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f46c400ce60 con 0x7f46dc0d0860 2024-09-21T00:28:58.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.240+0000 7f46e284d640 1 -- 172.21.15.112:0/1210697893 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46dc1f3400 con 0x7f46dc0d0860 2024-09-21T00:28:58.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.240+0000 7f46d8ff9640 1 -- 172.21.15.112:0/1210697893 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c4005280 con 0x7f46dc0d0860 2024-09-21T00:28:58.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.240+0000 7f46e284d640 1 -- 172.21.15.112:0/1210697893 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46dc0b5820 con 0x7f46dc0d0860 2024-09-21T00:28:58.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.241+0000 7f46e284d640 1 -- 172.21.15.112:0/1210697893 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46dc0b2900 con 0x7f46dc0d0860 2024-09-21T00:28:58.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.242+0000 7f46d8ff9640 1 -- 172.21.15.112:0/1210697893 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f46c4005420 con 0x7f46dc0d0860 2024-09-21T00:28:58.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.242+0000 7f46d8ff9640 1 --2- 172.21.15.112:0/1210697893 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f46bc078b90 0x7f46bc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:58.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.243+0000 7f46d8ff9640 1 -- 172.21.15.112:0/1210697893 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 6101+0+0 (secure 0 0 0) 0x7f46c409b8a0 con 0x7f46dc0d0860 2024-09-21T00:28:58.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.243+0000 7f46dbfff640 1 --2- 172.21.15.112:0/1210697893 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f46bc078b90 0x7f46bc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:58.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.243+0000 7f46dbfff640 1 --2- 172.21.15.112:0/1210697893 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f46bc078b90 0x7f46bc07b0a0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f46cc034000 tx=0x7f46cc004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:58.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.247+0000 7f46d8ff9640 1 -- 172.21.15.112:0/1210697893 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f46dc14b560 con 0x7f46dc0d0860 2024-09-21T00:28:58.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.460+0000 7f46e284d640 1 -- 172.21.15.112:0/1210697893 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f46dc0b52c0 con 0x7f46dc0d0860 2024-09-21T00:28:58.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.461+0000 7f46d8ff9640 1 -- 172.21.15.112:0/1210697893 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) ==== 94+0+5231 (secure 0 0 0) 0x7f46dc0b52c0 con 0x7f46dc0d0860 2024-09-21T00:28:58.461 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:28:58.462 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":55,"btime":"2024-09-21T00:28:58:105880+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5600,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6836/2708152171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":2708152171},{"type":"v1","addr":"172.21.15.187:6836","nonce":2708152171}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50},{"gid":6471,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6834/3529228075","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":3529228075},{"type":"v1","addr":"172.21.15.112:6834","nonce":3529228075}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50},{"gid":6474,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/101289292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":101289292},{"type":"v1","addr":"172.21.15.112:6835","nonce":101289292}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T00:28:54.453199+0000","modified":"2024-09-21T00:28:57.469605+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6486},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6486":{"gid":6486,"name":"d","rank":0,"incarnation":51,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6837/932692802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6835","nonce":932692802},{"type":"v1","addr":"172.21.15.187:6837","nonce":932692802}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6486,"qdb_cluster":[6486]},"id":6}]} 2024-09-21T00:28:58.462 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 55 2024-09-21T00:28:58.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.463+0000 7f46e284d640 1 -- 172.21.15.112:0/1210697893 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f46bc078b90 msgr2=0x7f46bc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:58.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.463+0000 7f46e284d640 1 --2- 172.21.15.112:0/1210697893 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f46bc078b90 0x7f46bc07b0a0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f46cc034000 tx=0x7f46cc004730 comp rx=0 tx=0).stop 2024-09-21T00:28:58.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.463+0000 7f46e284d640 1 -- 172.21.15.112:0/1210697893 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46dc0d0860 msgr2=0x7f46dc14b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:58.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.463+0000 7f46e284d640 1 --2- 172.21.15.112:0/1210697893 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46dc0d0860 0x7f46dc14b020 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f46c400e7c0 tx=0x7f46c400ec80 comp rx=0 tx=0).stop 2024-09-21T00:28:58.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.463+0000 7f46e284d640 1 -- 172.21.15.112:0/1210697893 shutdown_connections 2024-09-21T00:28:58.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.463+0000 7f46e284d640 1 --2- 172.21.15.112:0/1210697893 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f46dc0d2030 0x7f46dc15ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:58.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.463+0000 7f46e284d640 1 --2- 172.21.15.112:0/1210697893 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f46bc078b90 0x7f46bc07b0a0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:58.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.463+0000 7f46e284d640 1 --2- 172.21.15.112:0/1210697893 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46dc0d0860 0x7f46dc14b020 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:58.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.463+0000 7f46e284d640 1 --2- 172.21.15.112:0/1210697893 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f46dc0ccae0 0x7f46dc0cae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:58.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.463+0000 7f46e284d640 1 -- 172.21.15.112:0/1210697893 >> 172.21.15.112:0/1210697893 conn(0x7f46dc0b43f0 msgr2=0x7f46dc15deb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:58.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.463+0000 7f46e284d640 1 -- 172.21.15.112:0/1210697893 shutdown_connections 2024-09-21T00:28:58.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.463+0000 7f46e284d640 1 -- 172.21.15.112:0/1210697893 wait complete. 2024-09-21T00:28:58.476 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T00:28:58.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.587+0000 7fa068a5c640 1 -- 172.21.15.112:0/519397645 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa06415ca50 msgr2=0x7fa0640b41e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:58.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.587+0000 7fa068a5c640 1 --2- 172.21.15.112:0/519397645 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa06415ca50 0x7fa0640b41e0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fa054005fd0 tx=0x7fa05402aa90 comp rx=0 tx=0).stop 2024-09-21T00:28:58.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.587+0000 7fa068a5c640 1 -- 172.21.15.112:0/519397645 shutdown_connections 2024-09-21T00:28:58.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.587+0000 7fa068a5c640 1 --2- 172.21.15.112:0/519397645 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa06415ca50 0x7fa0640b41e0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:58.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.587+0000 7fa068a5c640 1 -- 172.21.15.112:0/519397645 >> 172.21.15.112:0/519397645 conn(0x7fa0640cf220 msgr2=0x7fa0640cf630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:58.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.587+0000 7fa068a5c640 1 -- 172.21.15.112:0/519397645 shutdown_connections 2024-09-21T00:28:58.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.587+0000 7fa068a5c640 1 -- 172.21.15.112:0/519397645 wait complete. 2024-09-21T00:28:58.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.589+0000 7fa068a5c640 1 Processor -- start 2024-09-21T00:28:58.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.589+0000 7fa068a5c640 1 -- start start 2024-09-21T00:28:58.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.591+0000 7fa068a5c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0640b4720 0x7fa0640c9400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:58.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.591+0000 7fa068a5c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa0640ccce0 0x7fa064158230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:58.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.591+0000 7fa068a5c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa06415ca50 0x7fa06415f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:58.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.591+0000 7fa068a5c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa0640cd320 con 0x7fa0640b4720 2024-09-21T00:28:58.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.591+0000 7fa068a5c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa0640cd1a0 con 0x7fa0640ccce0 2024-09-21T00:28:58.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.591+0000 7fa068a5c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa0640d1fb0 con 0x7fa06415ca50 2024-09-21T00:28:58.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.591+0000 7fa06259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0640b4720 0x7fa0640c9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:58.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.591+0000 7fa06259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0640b4720 0x7fa0640c9400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38124/0 (socket says 172.21.15.112:38124) 2024-09-21T00:28:58.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.591+0000 7fa06259c640 1 -- 172.21.15.112:0/1677615091 learned_addr learned my addr 172.21.15.112:0/1677615091 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:28:58.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.591+0000 7fa061d9b640 1 --2- 172.21.15.112:0/1677615091 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa0640ccce0 0x7fa064158230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:58.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.591+0000 7fa062d9d640 1 --2- 172.21.15.112:0/1677615091 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa06415ca50 0x7fa06415f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:58.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.591+0000 7fa06259c640 1 -- 172.21.15.112:0/1677615091 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa06415ca50 msgr2=0x7fa06415f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:58.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.591+0000 7fa06259c640 1 --2- 172.21.15.112:0/1677615091 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa06415ca50 0x7fa06415f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:58.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.591+0000 7fa06259c640 1 -- 172.21.15.112:0/1677615091 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa0640ccce0 msgr2=0x7fa064158230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:58.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.591+0000 7fa06259c640 1 --2- 172.21.15.112:0/1677615091 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa0640ccce0 0x7fa064158230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:58.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.591+0000 7fa06259c640 1 -- 172.21.15.112:0/1677615091 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa064158770 con 0x7fa0640b4720 2024-09-21T00:28:58.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.591+0000 7fa06259c640 1 --2- 172.21.15.112:0/1677615091 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0640b4720 0x7fa0640c9400 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fa054006010 tx=0x7fa054003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:58.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.591+0000 7fa05affd640 1 -- 172.21.15.112:0/1677615091 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa054044070 con 0x7fa0640b4720 2024-09-21T00:28:58.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.591+0000 7fa05affd640 1 -- 172.21.15.112:0/1677615091 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa054004250 con 0x7fa0640b4720 2024-09-21T00:28:58.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.591+0000 7fa068a5c640 1 -- 172.21.15.112:0/1677615091 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa06415f7f0 con 0x7fa0640b4720 2024-09-21T00:28:58.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.591+0000 7fa05affd640 1 -- 172.21.15.112:0/1677615091 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa054004640 con 0x7fa0640b4720 2024-09-21T00:28:58.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.591+0000 7fa068a5c640 1 -- 172.21.15.112:0/1677615091 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0641f7300 con 0x7fa0640b4720 2024-09-21T00:28:58.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.593+0000 7fa068a5c640 1 -- 172.21.15.112:0/1677615091 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0640b41e0 con 0x7fa0640b4720 2024-09-21T00:28:58.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.593+0000 7fa05affd640 1 -- 172.21.15.112:0/1677615091 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa054047070 con 0x7fa0640b4720 2024-09-21T00:28:58.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.594+0000 7fa05affd640 1 --2- 172.21.15.112:0/1677615091 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fa044078b90 0x7fa04407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:58.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.594+0000 7fa05affd640 1 -- 172.21.15.112:0/1677615091 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 6101+0+0 (secure 0 0 0) 0x7fa0540bbec0 con 0x7fa0640b4720 2024-09-21T00:28:58.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.594+0000 7fa061d9b640 1 --2- 172.21.15.112:0/1677615091 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fa044078b90 0x7fa04407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:58.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.594+0000 7fa061d9b640 1 --2- 172.21.15.112:0/1677615091 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fa044078b90 0x7fa04407b0a0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fa04c006fd0 tx=0x7fa04c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:58.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.598+0000 7fa05affd640 1 -- 172.21.15.112:0/1677615091 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa054032030 con 0x7fa0640b4720 2024-09-21T00:28:58.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.775+0000 7fa068a5c640 1 -- 172.21.15.112:0/1677615091 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa0640b3530 con 0x7fa0640b4720 2024-09-21T00:28:58.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.776+0000 7fa05affd640 1 -- 172.21.15.112:0/1677615091 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+16969 (secure 0 0 0) 0x7fa054084200 con 0x7fa0640b4720 2024-09-21T00:28:58.777 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:28:58.777 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":83,"fsid":"8b34692b-dc65-49b2-a08f-26f707b96ac5","created":"2024-09-21T00:25:22.788490+0000","modified":"2024-09-21T00:28:56.093597+0000","last_up_change":"2024-09-21T00:25:36.161835+0000","last_in_change":"2024-09-21T00:25:26.346500+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T00:25:37.105006+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-09-21T00:28:52.107455+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-09-21T00:28:53.410975+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9c575086-67d2-4de2-b01b-d44dcc21d604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6809","nonce":755760621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6811","nonce":755760621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6815","nonce":755760621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6813","nonce":755760621}]},"public_addr":"172.21.15.112:6809/755760621","cluster_addr":"172.21.15.112:6811/755760621","heartbeat_back_addr":"172.21.15.112:6815/755760621","heartbeat_front_addr":"172.21.15.112:6813/755760621","state":["exists","up"]},{"osd":1,"uuid":"a983c227-a3ce-4bdd-82f1-438ba7e3ecd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6801","nonce":344932592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6803","nonce":344932592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6807","nonce":344932592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6805","nonce":344932592}]},"public_addr":"172.21.15.112:6801/344932592","cluster_addr":"172.21.15.112:6803/344932592","heartbeat_back_addr":"172.21.15.112:6807/344932592","heartbeat_front_addr":"172.21.15.112:6805/344932592","state":["exists","up"]},{"osd":2,"uuid":"1193c1d8-bfff-4a57-8200-5f69bf1ea3af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6825","nonce":407933616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6827","nonce":407933616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6831","nonce":407933616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6829","nonce":407933616}]},"public_addr":"172.21.15.112:6825/407933616","cluster_addr":"172.21.15.112:6827/407933616","heartbeat_back_addr":"172.21.15.112:6831/407933616","heartbeat_front_addr":"172.21.15.112:6829/407933616","state":["exists","up"]},{"osd":3,"uuid":"af04330a-36ad-40cc-a8dd-fa259b221de5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6817","nonce":4120164806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6819","nonce":4120164806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6823","nonce":4120164806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6821","nonce":4120164806}]},"public_addr":"172.21.15.112:6817/4120164806","cluster_addr":"172.21.15.112:6819/4120164806","heartbeat_back_addr":"172.21.15.112:6823/4120164806","heartbeat_front_addr":"172.21.15.112:6821/4120164806","state":["exists","up"]},{"osd":4,"uuid":"5b893111-292c-4c5a-b184-53ad38c7199e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6803","nonce":522077560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6805","nonce":522077560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6809","nonce":522077560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6807","nonce":522077560}]},"public_addr":"172.21.15.187:6803/522077560","cluster_addr":"172.21.15.187:6805/522077560","heartbeat_back_addr":"172.21.15.187:6809/522077560","heartbeat_front_addr":"172.21.15.187:6807/522077560","state":["exists","up"]},{"osd":5,"uuid":"c0b8e011-45fe-4c5e-925e-6be2a7f1868a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6827","nonce":3593144721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6829","nonce":3593144721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6833","nonce":3593144721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6831","nonce":3593144721}]},"public_addr":"172.21.15.187:6827/3593144721","cluster_addr":"172.21.15.187:6829/3593144721","heartbeat_back_addr":"172.21.15.187:6833/3593144721","heartbeat_front_addr":"172.21.15.187:6831/3593144721","state":["exists","up"]},{"osd":6,"uuid":"87d2d650-7489-4593-98a3-a1f433ce8dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6811","nonce":1494387657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6813","nonce":1494387657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6817","nonce":1494387657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6815","nonce":1494387657}]},"public_addr":"172.21.15.187:6811/1494387657","cluster_addr":"172.21.15.187:6813/1494387657","heartbeat_back_addr":"172.21.15.187:6817/1494387657","heartbeat_front_addr":"172.21.15.187:6815/1494387657","state":["exists","up"]},{"osd":7,"uuid":"c6b4f8e2-a1ee-4438-aa37-67ff4b6ef1f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6819","nonce":1624699842}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6821","nonce":1624699842}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6825","nonce":1624699842}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6823","nonce":1624699842}]},"public_addr":"172.21.15.187:6819/1624699842","cluster_addr":"172.21.15.187:6821/1624699842","heartbeat_back_addr":"172.21.15.187:6825/1624699842","heartbeat_front_addr":"172.21.15.187:6823/1624699842","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.873162+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.400506+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.664317+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.159791+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.202630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.113212+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.384786+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.448441+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T00:28:58.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.778+0000 7fa068a5c640 1 -- 172.21.15.112:0/1677615091 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fa044078b90 msgr2=0x7fa04407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:58.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.778+0000 7fa068a5c640 1 --2- 172.21.15.112:0/1677615091 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fa044078b90 0x7fa04407b0a0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fa04c006fd0 tx=0x7fa04c008040 comp rx=0 tx=0).stop 2024-09-21T00:28:58.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.778+0000 7fa068a5c640 1 -- 172.21.15.112:0/1677615091 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0640b4720 msgr2=0x7fa0640c9400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:58.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.778+0000 7fa068a5c640 1 --2- 172.21.15.112:0/1677615091 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0640b4720 0x7fa0640c9400 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fa054006010 tx=0x7fa054003f10 comp rx=0 tx=0).stop 2024-09-21T00:28:58.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.778+0000 7fa068a5c640 1 -- 172.21.15.112:0/1677615091 shutdown_connections 2024-09-21T00:28:58.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.778+0000 7fa068a5c640 1 --2- 172.21.15.112:0/1677615091 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa06415ca50 0x7fa06415f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:58.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.778+0000 7fa068a5c640 1 --2- 172.21.15.112:0/1677615091 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa0640ccce0 0x7fa064158230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:58.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.778+0000 7fa068a5c640 1 --2- 172.21.15.112:0/1677615091 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fa044078b90 0x7fa04407b0a0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:58.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.778+0000 7fa068a5c640 1 --2- 172.21.15.112:0/1677615091 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0640b4720 0x7fa0640c9400 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:58.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.778+0000 7fa068a5c640 1 -- 172.21.15.112:0/1677615091 >> 172.21.15.112:0/1677615091 conn(0x7fa0640cf220 msgr2=0x7fa064159d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:58.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.778+0000 7fa068a5c640 1 -- 172.21.15.112:0/1677615091 shutdown_connections 2024-09-21T00:28:58.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.778+0000 7fa068a5c640 1 -- 172.21.15.112:0/1677615091 wait complete. 2024-09-21T00:28:58.791 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T00:28:58.791 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T00:28:58.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.903+0000 7ff47b550640 1 -- 172.21.15.112:0/2133267747 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4740c5ee0 msgr2=0x7ff4740c62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:58.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.903+0000 7ff47b550640 1 --2- 172.21.15.112:0/2133267747 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4740c5ee0 0x7ff4740c62f0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7ff474073f90 tx=0x7ff46402a440 comp rx=0 tx=0).stop 2024-09-21T00:28:58.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.903+0000 7ff47b550640 1 -- 172.21.15.112:0/2133267747 shutdown_connections 2024-09-21T00:28:58.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.903+0000 7ff47b550640 1 --2- 172.21.15.112:0/2133267747 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4740c5ee0 0x7ff4740c62f0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:58.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.903+0000 7ff47b550640 1 --2- 172.21.15.112:0/2133267747 >> v2:172.21.15.187:3300/0 conn(0x7ff4740ccbf0 0x7ff4740cd000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:58.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.903+0000 7ff47b550640 1 -- 172.21.15.112:0/2133267747 >> 172.21.15.112:0/2133267747 conn(0x7ff4740b42e0 msgr2=0x7ff4740b4720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:58.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.903+0000 7ff47b550640 1 -- 172.21.15.112:0/2133267747 shutdown_connections 2024-09-21T00:28:58.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.903+0000 7ff47b550640 1 -- 172.21.15.112:0/2133267747 wait complete. 2024-09-21T00:28:58.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.904+0000 7ff47b550640 1 Processor -- start 2024-09-21T00:28:58.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.904+0000 7ff47b550640 1 -- start start 2024-09-21T00:28:58.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.906+0000 7ff47b550640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff4740c5ee0 0x7ff47414ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:58.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.906+0000 7ff47b550640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4740c6830 0x7ff474144e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:58.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.906+0000 7ff47b550640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff4740ccbf0 0x7ff47414aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:58.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.906+0000 7ff47b550640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff4740b3450 con 0x7ff4740c6830 2024-09-21T00:28:58.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.906+0000 7ff47b550640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff4740c6f90 con 0x7ff4740c5ee0 2024-09-21T00:28:58.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.906+0000 7ff47b550640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff4740b35d0 con 0x7ff4740ccbf0 2024-09-21T00:28:58.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.906+0000 7ff478aeb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4740c6830 0x7ff474144e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:58.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.906+0000 7ff478aeb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4740c6830 0x7ff474144e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38146/0 (socket says 172.21.15.112:38146) 2024-09-21T00:28:58.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.906+0000 7ff478aeb640 1 -- 172.21.15.112:0/4270376653 learned_addr learned my addr 172.21.15.112:0/4270376653 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:28:58.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.906+0000 7ff4792ec640 1 --2- 172.21.15.112:0/4270376653 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff4740c5ee0 0x7ff47414ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:58.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.906+0000 7ff479aed640 1 --2- 172.21.15.112:0/4270376653 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff4740ccbf0 0x7ff47414aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:58.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.906+0000 7ff478aeb640 1 -- 172.21.15.112:0/4270376653 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff4740ccbf0 msgr2=0x7ff47414aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:58.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.906+0000 7ff478aeb640 1 --2- 172.21.15.112:0/4270376653 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff4740ccbf0 0x7ff47414aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:58.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.906+0000 7ff478aeb640 1 -- 172.21.15.112:0/4270376653 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff4740c5ee0 msgr2=0x7ff47414ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:58.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.906+0000 7ff478aeb640 1 --2- 172.21.15.112:0/4270376653 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff4740c5ee0 0x7ff47414ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:58.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.906+0000 7ff478aeb640 1 -- 172.21.15.112:0/4270376653 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4740cc9d0 con 0x7ff4740c6830 2024-09-21T00:28:58.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.906+0000 7ff479aed640 1 --2- 172.21.15.112:0/4270376653 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff4740ccbf0 0x7ff47414aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:28:58.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.906+0000 7ff478aeb640 1 --2- 172.21.15.112:0/4270376653 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4740c6830 0x7ff474144e70 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7ff464034040 tx=0x7ff4640039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:58.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.906+0000 7ff469ffb640 1 -- 172.21.15.112:0/4270376653 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff46403e030 con 0x7ff4740c6830 2024-09-21T00:28:58.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.906+0000 7ff469ffb640 1 -- 172.21.15.112:0/4270376653 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff4640047a0 con 0x7ff4740c6830 2024-09-21T00:28:58.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.906+0000 7ff47b550640 1 -- 172.21.15.112:0/4270376653 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4741f9a10 con 0x7ff4740c6830 2024-09-21T00:28:58.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.906+0000 7ff469ffb640 1 -- 172.21.15.112:0/4270376653 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff464035040 con 0x7ff4740c6830 2024-09-21T00:28:58.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.907+0000 7ff47b550640 1 -- 172.21.15.112:0/4270376653 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4740cc6f0 con 0x7ff4740c6830 2024-09-21T00:28:58.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.908+0000 7ff47b550640 1 -- 172.21.15.112:0/4270376653 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4740c62f0 con 0x7ff4740c6830 2024-09-21T00:28:58.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.908+0000 7ff469ffb640 1 -- 172.21.15.112:0/4270376653 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff464041030 con 0x7ff4740c6830 2024-09-21T00:28:58.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.909+0000 7ff469ffb640 1 --2- 172.21.15.112:0/4270376653 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff458078b90 0x7ff45807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:58.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.909+0000 7ff469ffb640 1 -- 172.21.15.112:0/4270376653 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 6101+0+0 (secure 0 0 0) 0x7ff4640bb5a0 con 0x7ff4740c6830 2024-09-21T00:28:58.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.909+0000 7ff4792ec640 1 --2- 172.21.15.112:0/4270376653 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff458078b90 0x7ff45807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:58.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.910+0000 7ff4792ec640 1 --2- 172.21.15.112:0/4270376653 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff458078b90 0x7ff45807b0a0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7ff470004630 tx=0x7ff47000b2e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:58.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:58.913+0000 7ff469ffb640 1 -- 172.21.15.112:0/4270376653 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ff4740cd540 con 0x7ff4740c6830 2024-09-21T00:28:59.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.086+0000 7ff47b550640 1 -- 172.21.15.112:0/4270376653 --> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff4740d06b0 con 0x7ff458078b90 2024-09-21T00:28:59.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.091+0000 7ff469ffb640 1 -- 172.21.15.112:0/4270376653 <== mgr.4103 v2:172.21.15.187:6800/2921522906 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+250338 (secure 0 0 0) 0x7ff4740d06b0 con 0x7ff458078b90 2024-09-21T00:28:59.095 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:28:59.112 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2024-09-21T00:28:59.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.094+0000 7ff47b550640 1 -- 172.21.15.112:0/4270376653 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff458078b90 msgr2=0x7ff45807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:59.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.095+0000 7ff47b550640 1 --2- 172.21.15.112:0/4270376653 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff458078b90 0x7ff45807b0a0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7ff470004630 tx=0x7ff47000b2e0 comp rx=0 tx=0).stop 2024-09-21T00:28:59.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.095+0000 7ff47b550640 1 -- 172.21.15.112:0/4270376653 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4740c6830 msgr2=0x7ff474144e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:59.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.095+0000 7ff47b550640 1 --2- 172.21.15.112:0/4270376653 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4740c6830 0x7ff474144e70 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7ff464034040 tx=0x7ff4640039e0 comp rx=0 tx=0).stop 2024-09-21T00:28:59.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.095+0000 7ff47b550640 1 -- 172.21.15.112:0/4270376653 shutdown_connections 2024-09-21T00:28:59.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.095+0000 7ff47b550640 1 --2- 172.21.15.112:0/4270376653 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff4740ccbf0 0x7ff47414aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:59.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.095+0000 7ff47b550640 1 --2- 172.21.15.112:0/4270376653 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff458078b90 0x7ff45807b0a0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:59.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.095+0000 7ff47b550640 1 --2- 172.21.15.112:0/4270376653 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4740c6830 0x7ff474144e70 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:59.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.095+0000 7ff47b550640 1 --2- 172.21.15.112:0/4270376653 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff4740c5ee0 0x7ff47414ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:59.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.095+0000 7ff47b550640 1 -- 172.21.15.112:0/4270376653 >> 172.21.15.112:0/4270376653 conn(0x7ff4740b42e0 msgr2=0x7ff4740c7ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:59.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.095+0000 7ff47b550640 1 -- 172.21.15.112:0/4270376653 shutdown_connections 2024-09-21T00:28:59.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.095+0000 7ff47b550640 1 -- 172.21.15.112:0/4270376653 wait complete. 2024-09-21T00:28:59.116 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":178,"stamp":"2024-09-21T00:28:57.137611+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":133,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":101,"ondisk_log_size":101,"up":174,"acting":174,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":98,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":254432,"kb_used_data":9048,"kb_used_omap":12,"kb_used_meta":245171,"kb_avail":754720288,"statfs":{"total":773094113280,"available":772833574912,"internally_reserved":0,"allocated":9265152,"data_stored":4790363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":251055665},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.062461"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.100503+0000","last_change":"2024-09-21T00:28:55.095212+0000","last_active":"2024-09-21T00:28:56.100503+0000","last_peered":"2024-09-21T00:28:56.100503+0000","last_clean":"2024-09-21T00:28:56.100503+0000","last_became_active":"2024-09-21T00:28:55.094938+0000","last_became_peered":"2024-09-21T00:28:55.094938+0000","last_unstale":"2024-09-21T00:28:56.100503+0000","last_undegraded":"2024-09-21T00:28:56.100503+0000","last_fullsized":"2024-09-21T00:28:56.100503+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_clean_scrub_stamp":"2024-09-21T00:28:54.073989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:30:46.125212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.117980+0000","last_change":"2024-09-21T00:28:55.096829+0000","last_active":"2024-09-21T00:28:56.117980+0000","last_peered":"2024-09-21T00:28:56.117980+0000","last_clean":"2024-09-21T00:28:56.117980+0000","last_became_active":"2024-09-21T00:28:55.096518+0000","last_became_peered":"2024-09-21T00:28:55.096518+0000","last_unstale":"2024-09-21T00:28:56.117980+0000","last_undegraded":"2024-09-21T00:28:56.117980+0000","last_fullsized":"2024-09-21T00:28:56.117980+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_clean_scrub_stamp":"2024-09-21T00:28:54.073989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:45:07.464813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.100223+0000","last_change":"2024-09-21T00:28:55.112040+0000","last_active":"2024-09-21T00:28:56.100223+0000","last_peered":"2024-09-21T00:28:56.100223+0000","last_clean":"2024-09-21T00:28:56.100223+0000","last_became_active":"2024-09-21T00:28:55.111715+0000","last_became_peered":"2024-09-21T00:28:55.111715+0000","last_unstale":"2024-09-21T00:28:56.100223+0000","last_undegraded":"2024-09-21T00:28:56.100223+0000","last_fullsized":"2024-09-21T00:28:56.100223+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_clean_scrub_stamp":"2024-09-21T00:28:54.073989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:08:29.568599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:28:54.073989+0000","last_change":"2024-09-21T00:28:54.073989+0000","last_active":"2024-09-21T00:28:54.073989+0000","last_peered":"2024-09-21T00:28:54.073989+0000","last_clean":"2024-09-21T00:28:54.073989+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:28:54.073989+0000","last_undegraded":"2024-09-21T00:28:54.073989+0000","last_fullsized":"2024-09-21T00:28:54.073989+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_clean_scrub_stamp":"2024-09-21T00:28:54.073989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.101835+0000","last_change":"2024-09-21T00:28:55.102995+0000","last_active":"2024-09-21T00:28:56.101835+0000","last_peered":"2024-09-21T00:28:56.101835+0000","last_clean":"2024-09-21T00:28:56.101835+0000","last_became_active":"2024-09-21T00:28:55.102678+0000","last_became_peered":"2024-09-21T00:28:55.102678+0000","last_unstale":"2024-09-21T00:28:56.101835+0000","last_undegraded":"2024-09-21T00:28:56.101835+0000","last_fullsized":"2024-09-21T00:28:56.101835+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_clean_scrub_stamp":"2024-09-21T00:28:54.073989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:57:54.006732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.117044+0000","last_change":"2024-09-21T00:28:55.104634+0000","last_active":"2024-09-21T00:28:56.117044+0000","last_peered":"2024-09-21T00:28:56.117044+0000","last_clean":"2024-09-21T00:28:56.117044+0000","last_became_active":"2024-09-21T00:28:55.104303+0000","last_became_peered":"2024-09-21T00:28:55.104303+0000","last_unstale":"2024-09-21T00:28:56.117044+0000","last_undegraded":"2024-09-21T00:28:56.117044+0000","last_fullsized":"2024-09-21T00:28:56.117044+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_clean_scrub_stamp":"2024-09-21T00:28:54.073989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:53:21.147188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.103023+0000","last_change":"2024-09-21T00:28:55.105218+0000","last_active":"2024-09-21T00:28:56.103023+0000","last_peered":"2024-09-21T00:28:56.103023+0000","last_clean":"2024-09-21T00:28:56.103023+0000","last_became_active":"2024-09-21T00:28:55.104883+0000","last_became_peered":"2024-09-21T00:28:55.104883+0000","last_unstale":"2024-09-21T00:28:56.103023+0000","last_undegraded":"2024-09-21T00:28:56.103023+0000","last_fullsized":"2024-09-21T00:28:56.103023+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_clean_scrub_stamp":"2024-09-21T00:28:54.073989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:30:46.125212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:28:54.073989+0000","last_change":"2024-09-21T00:28:54.073989+0000","last_active":"2024-09-21T00:28:54.073989+0000","last_peered":"2024-09-21T00:28:54.073989+0000","last_clean":"2024-09-21T00:28:54.073989+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:28:54.073989+0000","last_undegraded":"2024-09-21T00:28:54.073989+0000","last_fullsized":"2024-09-21T00:28:54.073989+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_clean_scrub_stamp":"2024-09-21T00:28:54.073989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.101639+0000","last_change":"2024-09-21T00:28:55.097933+0000","last_active":"2024-09-21T00:28:56.101639+0000","last_peered":"2024-09-21T00:28:56.101639+0000","last_clean":"2024-09-21T00:28:56.101639+0000","last_became_active":"2024-09-21T00:28:55.097556+0000","last_became_peered":"2024-09-21T00:28:55.097556+0000","last_unstale":"2024-09-21T00:28:56.101639+0000","last_undegraded":"2024-09-21T00:28:56.101639+0000","last_fullsized":"2024-09-21T00:28:56.101639+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_clean_scrub_stamp":"2024-09-21T00:28:54.073989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:46:42.028531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.118526+0000","last_change":"2024-09-21T00:28:55.104491+0000","last_active":"2024-09-21T00:28:56.118526+0000","last_peered":"2024-09-21T00:28:56.118526+0000","last_clean":"2024-09-21T00:28:56.118526+0000","last_became_active":"2024-09-21T00:28:55.104121+0000","last_became_peered":"2024-09-21T00:28:55.104121+0000","last_unstale":"2024-09-21T00:28:56.118526+0000","last_undegraded":"2024-09-21T00:28:56.118526+0000","last_fullsized":"2024-09-21T00:28:56.118526+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_clean_scrub_stamp":"2024-09-21T00:28:54.073989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:37:15.612342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.102169+0000","last_change":"2024-09-21T00:28:55.101986+0000","last_active":"2024-09-21T00:28:56.102169+0000","last_peered":"2024-09-21T00:28:56.102169+0000","last_clean":"2024-09-21T00:28:56.102169+0000","last_became_active":"2024-09-21T00:28:55.099761+0000","last_became_peered":"2024-09-21T00:28:55.099761+0000","last_unstale":"2024-09-21T00:28:56.102169+0000","last_undegraded":"2024-09-21T00:28:56.102169+0000","last_fullsized":"2024-09-21T00:28:56.102169+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_clean_scrub_stamp":"2024-09-21T00:28:54.073989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:11:40.478725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.116954+0000","last_change":"2024-09-21T00:28:55.097224+0000","last_active":"2024-09-21T00:28:56.116954+0000","last_peered":"2024-09-21T00:28:56.116954+0000","last_clean":"2024-09-21T00:28:56.116954+0000","last_became_active":"2024-09-21T00:28:55.096876+0000","last_became_peered":"2024-09-21T00:28:55.096876+0000","last_unstale":"2024-09-21T00:28:56.116954+0000","last_undegraded":"2024-09-21T00:28:56.116954+0000","last_fullsized":"2024-09-21T00:28:56.116954+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_clean_scrub_stamp":"2024-09-21T00:28:54.073989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:55:24.329815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:28:54.073989+0000","last_change":"2024-09-21T00:28:54.073989+0000","last_active":"2024-09-21T00:28:54.073989+0000","last_peered":"2024-09-21T00:28:54.073989+0000","last_clean":"2024-09-21T00:28:54.073989+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:28:54.073989+0000","last_undegraded":"2024-09-21T00:28:54.073989+0000","last_fullsized":"2024-09-21T00:28:54.073989+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_clean_scrub_stamp":"2024-09-21T00:28:54.073989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.101317+0000","last_change":"2024-09-21T00:28:55.110401+0000","last_active":"2024-09-21T00:28:56.101317+0000","last_peered":"2024-09-21T00:28:56.101317+0000","last_clean":"2024-09-21T00:28:56.101317+0000","last_became_active":"2024-09-21T00:28:55.109618+0000","last_became_peered":"2024-09-21T00:28:55.109618+0000","last_unstale":"2024-09-21T00:28:56.101317+0000","last_undegraded":"2024-09-21T00:28:56.101317+0000","last_fullsized":"2024-09-21T00:28:56.101317+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_clean_scrub_stamp":"2024-09-21T00:28:54.073989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:07:42.574101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:28:54.073989+0000","last_change":"2024-09-21T00:28:54.073989+0000","last_active":"2024-09-21T00:28:54.073989+0000","last_peered":"2024-09-21T00:28:54.073989+0000","last_clean":"2024-09-21T00:28:54.073989+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:28:54.073989+0000","last_undegraded":"2024-09-21T00:28:54.073989+0000","last_fullsized":"2024-09-21T00:28:54.073989+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_clean_scrub_stamp":"2024-09-21T00:28:54.073989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.117411+0000","last_change":"2024-09-21T00:28:55.102635+0000","last_active":"2024-09-21T00:28:56.117411+0000","last_peered":"2024-09-21T00:28:56.117411+0000","last_clean":"2024-09-21T00:28:56.117411+0000","last_became_active":"2024-09-21T00:28:55.098570+0000","last_became_peered":"2024-09-21T00:28:55.098570+0000","last_unstale":"2024-09-21T00:28:56.117411+0000","last_undegraded":"2024-09-21T00:28:56.117411+0000","last_fullsized":"2024-09-21T00:28:56.117411+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_clean_scrub_stamp":"2024-09-21T00:28:54.073989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:28:55.581773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.118625+0000","last_change":"2024-09-21T00:28:55.098790+0000","last_active":"2024-09-21T00:28:56.118625+0000","last_peered":"2024-09-21T00:28:56.118625+0000","last_clean":"2024-09-21T00:28:56.118625+0000","last_became_active":"2024-09-21T00:28:55.098436+0000","last_became_peered":"2024-09-21T00:28:55.098436+0000","last_unstale":"2024-09-21T00:28:56.118625+0000","last_undegraded":"2024-09-21T00:28:56.118625+0000","last_fullsized":"2024-09-21T00:28:56.118625+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_clean_scrub_stamp":"2024-09-21T00:28:54.073989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:21:44.839641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.100423+0000","last_change":"2024-09-21T00:28:55.096957+0000","last_active":"2024-09-21T00:28:56.100423+0000","last_peered":"2024-09-21T00:28:56.100423+0000","last_clean":"2024-09-21T00:28:56.100423+0000","last_became_active":"2024-09-21T00:28:55.096670+0000","last_became_peered":"2024-09-21T00:28:55.096670+0000","last_unstale":"2024-09-21T00:28:56.100423+0000","last_undegraded":"2024-09-21T00:28:56.100423+0000","last_fullsized":"2024-09-21T00:28:56.100423+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_clean_scrub_stamp":"2024-09-21T00:28:54.073989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:05:45.788017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.102307+0000","last_change":"2024-09-21T00:28:55.111532+0000","last_active":"2024-09-21T00:28:56.102307+0000","last_peered":"2024-09-21T00:28:56.102307+0000","last_clean":"2024-09-21T00:28:56.102307+0000","last_became_active":"2024-09-21T00:28:55.106336+0000","last_became_peered":"2024-09-21T00:28:55.106336+0000","last_unstale":"2024-09-21T00:28:56.102307+0000","last_undegraded":"2024-09-21T00:28:56.102307+0000","last_fullsized":"2024-09-21T00:28:56.102307+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_clean_scrub_stamp":"2024-09-21T00:28:54.073989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:57:19.321859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:28:54.073989+0000","last_change":"2024-09-21T00:28:54.073989+0000","last_active":"2024-09-21T00:28:54.073989+0000","last_peered":"2024-09-21T00:28:54.073989+0000","last_clean":"2024-09-21T00:28:54.073989+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:28:54.073989+0000","last_undegraded":"2024-09-21T00:28:54.073989+0000","last_fullsized":"2024-09-21T00:28:54.073989+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_clean_scrub_stamp":"2024-09-21T00:28:54.073989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.101448+0000","last_change":"2024-09-21T00:28:55.108612+0000","last_active":"2024-09-21T00:28:56.101448+0000","last_peered":"2024-09-21T00:28:56.101448+0000","last_clean":"2024-09-21T00:28:56.101448+0000","last_became_active":"2024-09-21T00:28:55.108272+0000","last_became_peered":"2024-09-21T00:28:55.108272+0000","last_unstale":"2024-09-21T00:28:56.101448+0000","last_undegraded":"2024-09-21T00:28:56.101448+0000","last_fullsized":"2024-09-21T00:28:56.101448+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_clean_scrub_stamp":"2024-09-21T00:28:54.073989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:35:00.733519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.101524+0000","last_change":"2024-09-21T00:28:55.109107+0000","last_active":"2024-09-21T00:28:56.101524+0000","last_peered":"2024-09-21T00:28:56.101524+0000","last_clean":"2024-09-21T00:28:56.101524+0000","last_became_active":"2024-09-21T00:28:55.107380+0000","last_became_peered":"2024-09-21T00:28:55.107380+0000","last_unstale":"2024-09-21T00:28:56.101524+0000","last_undegraded":"2024-09-21T00:28:56.101524+0000","last_fullsized":"2024-09-21T00:28:56.101524+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_clean_scrub_stamp":"2024-09-21T00:28:54.073989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:46:43.542825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.118127+0000","last_change":"2024-09-21T00:28:55.108284+0000","last_active":"2024-09-21T00:28:56.118127+0000","last_peered":"2024-09-21T00:28:56.118127+0000","last_clean":"2024-09-21T00:28:56.118127+0000","last_became_active":"2024-09-21T00:28:55.107930+0000","last_became_peered":"2024-09-21T00:28:55.107930+0000","last_unstale":"2024-09-21T00:28:56.118127+0000","last_undegraded":"2024-09-21T00:28:56.118127+0000","last_fullsized":"2024-09-21T00:28:56.118127+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_clean_scrub_stamp":"2024-09-21T00:28:54.073989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:12:40.407615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.100202+0000","last_change":"2024-09-21T00:28:55.105271+0000","last_active":"2024-09-21T00:28:56.100202+0000","last_peered":"2024-09-21T00:28:56.100202+0000","last_clean":"2024-09-21T00:28:56.100202+0000","last_became_active":"2024-09-21T00:28:55.104870+0000","last_became_peered":"2024-09-21T00:28:55.104870+0000","last_unstale":"2024-09-21T00:28:56.100202+0000","last_undegraded":"2024-09-21T00:28:56.100202+0000","last_fullsized":"2024-09-21T00:28:56.100202+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_clean_scrub_stamp":"2024-09-21T00:28:54.073989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:36:38.879430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.117745+0000","last_change":"2024-09-21T00:28:55.108523+0000","last_active":"2024-09-21T00:28:56.117745+0000","last_peered":"2024-09-21T00:28:56.117745+0000","last_clean":"2024-09-21T00:28:56.117745+0000","last_became_active":"2024-09-21T00:28:55.108179+0000","last_became_peered":"2024-09-21T00:28:55.108179+0000","last_unstale":"2024-09-21T00:28:56.117745+0000","last_undegraded":"2024-09-21T00:28:56.117745+0000","last_fullsized":"2024-09-21T00:28:56.117745+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_clean_scrub_stamp":"2024-09-21T00:28:54.073989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:32:16.237390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.102149+0000","last_change":"2024-09-21T00:28:55.106799+0000","last_active":"2024-09-21T00:28:56.102149+0000","last_peered":"2024-09-21T00:28:56.102149+0000","last_clean":"2024-09-21T00:28:56.102149+0000","last_became_active":"2024-09-21T00:28:55.106328+0000","last_became_peered":"2024-09-21T00:28:55.106328+0000","last_unstale":"2024-09-21T00:28:56.102149+0000","last_undegraded":"2024-09-21T00:28:56.102149+0000","last_fullsized":"2024-09-21T00:28:56.102149+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_clean_scrub_stamp":"2024-09-21T00:28:54.073989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:01:07.579102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.117158+0000","last_change":"2024-09-21T00:28:55.104075+0000","last_active":"2024-09-21T00:28:56.117158+0000","last_peered":"2024-09-21T00:28:56.117158+0000","last_clean":"2024-09-21T00:28:56.117158+0000","last_became_active":"2024-09-21T00:28:55.103545+0000","last_became_peered":"2024-09-21T00:28:55.103545+0000","last_unstale":"2024-09-21T00:28:56.117158+0000","last_undegraded":"2024-09-21T00:28:56.117158+0000","last_fullsized":"2024-09-21T00:28:56.117158+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_clean_scrub_stamp":"2024-09-21T00:28:54.073989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:01.343438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:28:54.073989+0000","last_change":"2024-09-21T00:28:54.073989+0000","last_active":"2024-09-21T00:28:54.073989+0000","last_peered":"2024-09-21T00:28:54.073989+0000","last_clean":"2024-09-21T00:28:54.073989+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:28:54.073989+0000","last_undegraded":"2024-09-21T00:28:54.073989+0000","last_fullsized":"2024-09-21T00:28:54.073989+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_clean_scrub_stamp":"2024-09-21T00:28:54.073989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.103442+0000","last_change":"2024-09-21T00:28:55.107136+0000","last_active":"2024-09-21T00:28:56.103442+0000","last_peered":"2024-09-21T00:28:56.103442+0000","last_clean":"2024-09-21T00:28:56.103442+0000","last_became_active":"2024-09-21T00:28:55.106829+0000","last_became_peered":"2024-09-21T00:28:55.106829+0000","last_unstale":"2024-09-21T00:28:56.103442+0000","last_undegraded":"2024-09-21T00:28:56.103442+0000","last_fullsized":"2024-09-21T00:28:56.103442+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_clean_scrub_stamp":"2024-09-21T00:28:54.073989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:26:18.993673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.101693+0000","last_change":"2024-09-21T00:28:55.099077+0000","last_active":"2024-09-21T00:28:56.101693+0000","last_peered":"2024-09-21T00:28:56.101693+0000","last_clean":"2024-09-21T00:28:56.101693+0000","last_became_active":"2024-09-21T00:28:55.098377+0000","last_became_peered":"2024-09-21T00:28:55.098377+0000","last_unstale":"2024-09-21T00:28:56.101693+0000","last_undegraded":"2024-09-21T00:28:56.101693+0000","last_fullsized":"2024-09-21T00:28:56.101693+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_clean_scrub_stamp":"2024-09-21T00:28:54.073989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:44:37.308136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.117119+0000","last_change":"2024-09-21T00:28:55.109292+0000","last_active":"2024-09-21T00:28:56.117119+0000","last_peered":"2024-09-21T00:28:56.117119+0000","last_clean":"2024-09-21T00:28:56.117119+0000","last_became_active":"2024-09-21T00:28:55.108969+0000","last_became_peered":"2024-09-21T00:28:55.108969+0000","last_unstale":"2024-09-21T00:28:56.117119+0000","last_undegraded":"2024-09-21T00:28:56.117119+0000","last_fullsized":"2024-09-21T00:28:56.117119+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_clean_scrub_stamp":"2024-09-21T00:28:54.073989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:31:02.605652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.116824+0000","last_change":"2024-09-21T00:28:55.099447+0000","last_active":"2024-09-21T00:28:56.116824+0000","last_peered":"2024-09-21T00:28:56.116824+0000","last_clean":"2024-09-21T00:28:56.116824+0000","last_became_active":"2024-09-21T00:28:55.097567+0000","last_became_peered":"2024-09-21T00:28:55.097567+0000","last_unstale":"2024-09-21T00:28:56.116824+0000","last_undegraded":"2024-09-21T00:28:56.116824+0000","last_fullsized":"2024-09-21T00:28:56.116824+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_clean_scrub_stamp":"2024-09-21T00:28:54.073989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:59:16.615558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.117471+0000","last_change":"2024-09-21T00:28:55.101000+0000","last_active":"2024-09-21T00:28:56.117471+0000","last_peered":"2024-09-21T00:28:56.117471+0000","last_clean":"2024-09-21T00:28:56.117471+0000","last_became_active":"2024-09-21T00:28:55.100265+0000","last_became_peered":"2024-09-21T00:28:55.100265+0000","last_unstale":"2024-09-21T00:28:56.117471+0000","last_undegraded":"2024-09-21T00:28:56.117471+0000","last_fullsized":"2024-09-21T00:28:56.117471+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_clean_scrub_stamp":"2024-09-21T00:28:54.073989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:24:20.572343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"82'4","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.117415+0000","last_change":"2024-09-21T00:28:54.103623+0000","last_active":"2024-09-21T00:28:56.117415+0000","last_peered":"2024-09-21T00:28:56.117415+0000","last_clean":"2024-09-21T00:28:56.117415+0000","last_became_active":"2024-09-21T00:28:54.103177+0000","last_became_peered":"2024-09-21T00:28:54.103177+0000","last_unstale":"2024-09-21T00:28:56.117415+0000","last_undegraded":"2024-09-21T00:28:56.117415+0000","last_fullsized":"2024-09-21T00:28:56.117415+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:53.070133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:53.070133+0000","last_clean_scrub_stamp":"2024-09-21T00:28:53.070133+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:57:41.206766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.117770+0000","last_change":"2024-09-21T00:28:55.099060+0000","last_active":"2024-09-21T00:28:56.117770+0000","last_peered":"2024-09-21T00:28:56.117770+0000","last_clean":"2024-09-21T00:28:56.117770+0000","last_became_active":"2024-09-21T00:28:55.096215+0000","last_became_peered":"2024-09-21T00:28:55.096215+0000","last_unstale":"2024-09-21T00:28:56.117770+0000","last_undegraded":"2024-09-21T00:28:56.117770+0000","last_fullsized":"2024-09-21T00:28:56.117770+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_clean_scrub_stamp":"2024-09-21T00:28:54.073989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:41:13.683153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"82'1","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.102341+0000","last_change":"2024-09-21T00:28:54.094593+0000","last_active":"2024-09-21T00:28:56.102341+0000","last_peered":"2024-09-21T00:28:56.102341+0000","last_clean":"2024-09-21T00:28:56.102341+0000","last_became_active":"2024-09-21T00:28:54.094275+0000","last_became_peered":"2024-09-21T00:28:54.094275+0000","last_unstale":"2024-09-21T00:28:56.102341+0000","last_undegraded":"2024-09-21T00:28:56.102341+0000","last_fullsized":"2024-09-21T00:28:56.102341+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:53.070133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:53.070133+0000","last_clean_scrub_stamp":"2024-09-21T00:28:53.070133+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:38:53.041398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.103172+0000","last_change":"2024-09-21T00:28:55.105392+0000","last_active":"2024-09-21T00:28:56.103172+0000","last_peered":"2024-09-21T00:28:56.103172+0000","last_clean":"2024-09-21T00:28:56.103172+0000","last_became_active":"2024-09-21T00:28:55.104858+0000","last_became_peered":"2024-09-21T00:28:55.104858+0000","last_unstale":"2024-09-21T00:28:56.103172+0000","last_undegraded":"2024-09-21T00:28:56.103172+0000","last_fullsized":"2024-09-21T00:28:56.103172+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_clean_scrub_stamp":"2024-09-21T00:28:54.073989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T00:48:51.087715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"82'2","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.103357+0000","last_change":"2024-09-21T00:28:54.102165+0000","last_active":"2024-09-21T00:28:56.103357+0000","last_peered":"2024-09-21T00:28:56.103357+0000","last_clean":"2024-09-21T00:28:56.103357+0000","last_became_active":"2024-09-21T00:28:54.101820+0000","last_became_peered":"2024-09-21T00:28:54.101820+0000","last_unstale":"2024-09-21T00:28:56.103357+0000","last_undegraded":"2024-09-21T00:28:56.103357+0000","last_fullsized":"2024-09-21T00:28:56.103357+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:53.070133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:53.070133+0000","last_clean_scrub_stamp":"2024-09-21T00:28:53.070133+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:51:30.664168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.118626+0000","last_change":"2024-09-21T00:28:55.113067+0000","last_active":"2024-09-21T00:28:56.118626+0000","last_peered":"2024-09-21T00:28:56.118626+0000","last_clean":"2024-09-21T00:28:56.118626+0000","last_became_active":"2024-09-21T00:28:55.109815+0000","last_became_peered":"2024-09-21T00:28:55.109815+0000","last_unstale":"2024-09-21T00:28:56.118626+0000","last_undegraded":"2024-09-21T00:28:56.118626+0000","last_fullsized":"2024-09-21T00:28:56.118626+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_clean_scrub_stamp":"2024-09-21T00:28:54.073989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:29:37.203382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"82'2","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.117648+0000","last_change":"2024-09-21T00:28:54.098815+0000","last_active":"2024-09-21T00:28:56.117648+0000","last_peered":"2024-09-21T00:28:56.117648+0000","last_clean":"2024-09-21T00:28:56.117648+0000","last_became_active":"2024-09-21T00:28:54.098487+0000","last_became_peered":"2024-09-21T00:28:54.098487+0000","last_unstale":"2024-09-21T00:28:56.117648+0000","last_undegraded":"2024-09-21T00:28:56.117648+0000","last_fullsized":"2024-09-21T00:28:56.117648+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:53.070133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:53.070133+0000","last_clean_scrub_stamp":"2024-09-21T00:28:53.070133+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:41:09.729837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:28:54.073989+0000","last_change":"2024-09-21T00:28:54.073989+0000","last_active":"2024-09-21T00:28:54.073989+0000","last_peered":"2024-09-21T00:28:54.073989+0000","last_clean":"2024-09-21T00:28:54.073989+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:28:54.073989+0000","last_undegraded":"2024-09-21T00:28:54.073989+0000","last_fullsized":"2024-09-21T00:28:54.073989+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_clean_scrub_stamp":"2024-09-21T00:28:54.073989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1b","version":"82'1","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.101829+0000","last_change":"2024-09-21T00:28:54.099669+0000","last_active":"2024-09-21T00:28:56.101829+0000","last_peered":"2024-09-21T00:28:56.101829+0000","last_clean":"2024-09-21T00:28:56.101829+0000","last_became_active":"2024-09-21T00:28:54.099356+0000","last_became_peered":"2024-09-21T00:28:54.099356+0000","last_unstale":"2024-09-21T00:28:56.101829+0000","last_undegraded":"2024-09-21T00:28:56.101829+0000","last_fullsized":"2024-09-21T00:28:56.101829+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:53.070133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:53.070133+0000","last_clean_scrub_stamp":"2024-09-21T00:28:53.070133+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:48:15.713474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.117768+0000","last_change":"2024-09-21T00:28:55.101974+0000","last_active":"2024-09-21T00:28:56.117768+0000","last_peered":"2024-09-21T00:28:56.117768+0000","last_clean":"2024-09-21T00:28:56.117768+0000","last_became_active":"2024-09-21T00:28:55.101451+0000","last_became_peered":"2024-09-21T00:28:55.101451+0000","last_unstale":"2024-09-21T00:28:56.117768+0000","last_undegraded":"2024-09-21T00:28:56.117768+0000","last_fullsized":"2024-09-21T00:28:56.117768+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_clean_scrub_stamp":"2024-09-21T00:28:54.073989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:12:08.777614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-09-21T00:28:53.081139+0000","last_change":"2024-09-21T00:28:53.081139+0000","last_active":"2024-09-21T00:28:53.070133+0000","last_peered":"2024-09-21T00:28:53.070133+0000","last_clean":"2024-09-21T00:28:53.070133+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:28:53.081139+0000","last_undegraded":"2024-09-21T00:28:53.081139+0000","last_fullsized":"2024-09-21T00:28:53.081139+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:53.070133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:53.070133+0000","last_clean_scrub_stamp":"2024-09-21T00:28:53.070133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.117305+0000","last_change":"2024-09-21T00:28:55.102571+0000","last_active":"2024-09-21T00:28:56.117305+0000","last_peered":"2024-09-21T00:28:56.117305+0000","last_clean":"2024-09-21T00:28:56.117305+0000","last_became_active":"2024-09-21T00:28:55.102270+0000","last_became_peered":"2024-09-21T00:28:55.102270+0000","last_unstale":"2024-09-21T00:28:56.117305+0000","last_undegraded":"2024-09-21T00:28:56.117305+0000","last_fullsized":"2024-09-21T00:28:56.117305+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_clean_scrub_stamp":"2024-09-21T00:28:54.073989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:14:27.778681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.100134+0000","last_change":"2024-09-21T00:28:54.096435+0000","last_active":"2024-09-21T00:28:56.100134+0000","last_peered":"2024-09-21T00:28:56.100134+0000","last_clean":"2024-09-21T00:28:56.100134+0000","last_became_active":"2024-09-21T00:28:54.096080+0000","last_became_peered":"2024-09-21T00:28:54.096080+0000","last_unstale":"2024-09-21T00:28:56.100134+0000","last_undegraded":"2024-09-21T00:28:56.100134+0000","last_fullsized":"2024-09-21T00:28:56.100134+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:53.070133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:53.070133+0000","last_clean_scrub_stamp":"2024-09-21T00:28:53.070133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T00:56:36.844091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:28:54.073989+0000","last_change":"2024-09-21T00:28:54.073989+0000","last_active":"2024-09-21T00:28:54.073989+0000","last_peered":"2024-09-21T00:28:54.073989+0000","last_clean":"2024-09-21T00:28:54.073989+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:28:54.073989+0000","last_undegraded":"2024-09-21T00:28:54.073989+0000","last_fullsized":"2024-09-21T00:28:54.073989+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_clean_scrub_stamp":"2024-09-21T00:28:54.073989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.102943+0000","last_change":"2024-09-21T00:28:54.102451+0000","last_active":"2024-09-21T00:28:56.102943+0000","last_peered":"2024-09-21T00:28:56.102943+0000","last_clean":"2024-09-21T00:28:56.102943+0000","last_became_active":"2024-09-21T00:28:54.102143+0000","last_became_peered":"2024-09-21T00:28:54.102143+0000","last_unstale":"2024-09-21T00:28:56.102943+0000","last_undegraded":"2024-09-21T00:28:56.102943+0000","last_fullsized":"2024-09-21T00:28:56.102943+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:53.070133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:53.070133+0000","last_clean_scrub_stamp":"2024-09-21T00:28:53.070133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:18:29.567994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.102206+0000","last_change":"2024-09-21T00:28:55.098462+0000","last_active":"2024-09-21T00:28:56.102206+0000","last_peered":"2024-09-21T00:28:56.102206+0000","last_clean":"2024-09-21T00:28:56.102206+0000","last_became_active":"2024-09-21T00:28:55.097998+0000","last_became_peered":"2024-09-21T00:28:55.097998+0000","last_unstale":"2024-09-21T00:28:56.102206+0000","last_undegraded":"2024-09-21T00:28:56.102206+0000","last_fullsized":"2024-09-21T00:28:56.102206+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_clean_scrub_stamp":"2024-09-21T00:28:54.073989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:52:09.676331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.118115+0000","last_change":"2024-09-21T00:28:54.099224+0000","last_active":"2024-09-21T00:28:56.118115+0000","last_peered":"2024-09-21T00:28:56.118115+0000","last_clean":"2024-09-21T00:28:56.118115+0000","last_became_active":"2024-09-21T00:28:54.098825+0000","last_became_peered":"2024-09-21T00:28:54.098825+0000","last_unstale":"2024-09-21T00:28:56.118115+0000","last_undegraded":"2024-09-21T00:28:56.118115+0000","last_fullsized":"2024-09-21T00:28:56.118115+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:53.070133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:53.070133+0000","last_clean_scrub_stamp":"2024-09-21T00:28:53.070133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:32:52.896230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.103170+0000","last_change":"2024-09-21T00:28:55.113005+0000","last_active":"2024-09-21T00:28:56.103170+0000","last_peered":"2024-09-21T00:28:56.103170+0000","last_clean":"2024-09-21T00:28:56.103170+0000","last_became_active":"2024-09-21T00:28:55.112590+0000","last_became_peered":"2024-09-21T00:28:55.112590+0000","last_unstale":"2024-09-21T00:28:56.103170+0000","last_undegraded":"2024-09-21T00:28:56.103170+0000","last_fullsized":"2024-09-21T00:28:56.103170+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_clean_scrub_stamp":"2024-09-21T00:28:54.073989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:30:09.350514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.100523+0000","last_change":"2024-09-21T00:28:54.092302+0000","last_active":"2024-09-21T00:28:56.100523+0000","last_peered":"2024-09-21T00:28:56.100523+0000","last_clean":"2024-09-21T00:28:56.100523+0000","last_became_active":"2024-09-21T00:28:54.091991+0000","last_became_peered":"2024-09-21T00:28:54.091991+0000","last_unstale":"2024-09-21T00:28:56.100523+0000","last_undegraded":"2024-09-21T00:28:56.100523+0000","last_fullsized":"2024-09-21T00:28:56.100523+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:53.070133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:53.070133+0000","last_clean_scrub_stamp":"2024-09-21T00:28:53.070133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:05:52.153493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.100471+0000","last_change":"2024-09-21T00:28:55.095188+0000","last_active":"2024-09-21T00:28:56.100471+0000","last_peered":"2024-09-21T00:28:56.100471+0000","last_clean":"2024-09-21T00:28:56.100471+0000","last_became_active":"2024-09-21T00:28:55.094877+0000","last_became_peered":"2024-09-21T00:28:55.094877+0000","last_unstale":"2024-09-21T00:28:56.100471+0000","last_undegraded":"2024-09-21T00:28:56.100471+0000","last_fullsized":"2024-09-21T00:28:56.100471+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_clean_scrub_stamp":"2024-09-21T00:28:54.073989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:40:41.255893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"82'3","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.102407+0000","last_change":"2024-09-21T00:28:54.099841+0000","last_active":"2024-09-21T00:28:56.102407+0000","last_peered":"2024-09-21T00:28:56.102407+0000","last_clean":"2024-09-21T00:28:56.102407+0000","last_became_active":"2024-09-21T00:28:54.099522+0000","last_became_peered":"2024-09-21T00:28:54.099522+0000","last_unstale":"2024-09-21T00:28:56.102407+0000","last_undegraded":"2024-09-21T00:28:56.102407+0000","last_fullsized":"2024-09-21T00:28:56.102407+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:53.070133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:53.070133+0000","last_clean_scrub_stamp":"2024-09-21T00:28:53.070133+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:27:40.663017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:28:54.073989+0000","last_change":"2024-09-21T00:28:54.073989+0000","last_active":"2024-09-21T00:28:54.073989+0000","last_peered":"2024-09-21T00:28:54.073989+0000","last_clean":"2024-09-21T00:28:54.073989+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:28:54.073989+0000","last_undegraded":"2024-09-21T00:28:54.073989+0000","last_fullsized":"2024-09-21T00:28:54.073989+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_clean_scrub_stamp":"2024-09-21T00:28:54.073989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.14","version":"82'6","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.117603+0000","last_change":"2024-09-21T00:28:54.100386+0000","last_active":"2024-09-21T00:28:56.117603+0000","last_peered":"2024-09-21T00:28:56.117603+0000","last_clean":"2024-09-21T00:28:56.117603+0000","last_became_active":"2024-09-21T00:28:54.100051+0000","last_became_peered":"2024-09-21T00:28:54.100051+0000","last_unstale":"2024-09-21T00:28:56.117603+0000","last_undegraded":"2024-09-21T00:28:56.117603+0000","last_fullsized":"2024-09-21T00:28:56.117603+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:53.070133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:53.070133+0000","last_clean_scrub_stamp":"2024-09-21T00:28:53.070133+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T00:53:51.470817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:28:54.073989+0000","last_change":"2024-09-21T00:28:54.073989+0000","last_active":"2024-09-21T00:28:54.073989+0000","last_peered":"2024-09-21T00:28:54.073989+0000","last_clean":"2024-09-21T00:28:54.073989+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:28:54.073989+0000","last_undegraded":"2024-09-21T00:28:54.073989+0000","last_fullsized":"2024-09-21T00:28:54.073989+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_clean_scrub_stamp":"2024-09-21T00:28:54.073989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.13","version":"82'2","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.103382+0000","last_change":"2024-09-21T00:28:54.098296+0000","last_active":"2024-09-21T00:28:56.103382+0000","last_peered":"2024-09-21T00:28:56.103382+0000","last_clean":"2024-09-21T00:28:56.103382+0000","last_became_active":"2024-09-21T00:28:54.097982+0000","last_became_peered":"2024-09-21T00:28:54.097982+0000","last_unstale":"2024-09-21T00:28:56.103382+0000","last_undegraded":"2024-09-21T00:28:56.103382+0000","last_fullsized":"2024-09-21T00:28:56.103382+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:53.070133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:53.070133+0000","last_clean_scrub_stamp":"2024-09-21T00:28:53.070133+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:32:15.233534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.101320+0000","last_change":"2024-09-21T00:28:55.104064+0000","last_active":"2024-09-21T00:28:56.101320+0000","last_peered":"2024-09-21T00:28:56.101320+0000","last_clean":"2024-09-21T00:28:56.101320+0000","last_became_active":"2024-09-21T00:28:55.103734+0000","last_became_peered":"2024-09-21T00:28:55.103734+0000","last_unstale":"2024-09-21T00:28:56.101320+0000","last_undegraded":"2024-09-21T00:28:56.101320+0000","last_fullsized":"2024-09-21T00:28:56.101320+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_clean_scrub_stamp":"2024-09-21T00:28:54.073989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:47:48.194847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.117170+0000","last_change":"2024-09-21T00:28:54.098640+0000","last_active":"2024-09-21T00:28:56.117170+0000","last_peered":"2024-09-21T00:28:56.117170+0000","last_clean":"2024-09-21T00:28:56.117170+0000","last_became_active":"2024-09-21T00:28:54.097610+0000","last_became_peered":"2024-09-21T00:28:54.097610+0000","last_unstale":"2024-09-21T00:28:56.117170+0000","last_undegraded":"2024-09-21T00:28:56.117170+0000","last_fullsized":"2024-09-21T00:28:56.117170+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:53.070133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:53.070133+0000","last_clean_scrub_stamp":"2024-09-21T00:28:53.070133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:46:44.835922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.117223+0000","last_change":"2024-09-21T00:28:55.106651+0000","last_active":"2024-09-21T00:28:56.117223+0000","last_peered":"2024-09-21T00:28:56.117223+0000","last_clean":"2024-09-21T00:28:56.117223+0000","last_became_active":"2024-09-21T00:28:55.106188+0000","last_became_peered":"2024-09-21T00:28:55.106188+0000","last_unstale":"2024-09-21T00:28:56.117223+0000","last_undegraded":"2024-09-21T00:28:56.117223+0000","last_fullsized":"2024-09-21T00:28:56.117223+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_clean_scrub_stamp":"2024-09-21T00:28:54.073989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:22:52.766853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-09-21T00:28:53.081094+0000","last_change":"2024-09-21T00:28:53.081094+0000","last_active":"2024-09-21T00:28:53.070133+0000","last_peered":"2024-09-21T00:28:53.070133+0000","last_clean":"2024-09-21T00:28:53.070133+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:28:53.081094+0000","last_undegraded":"2024-09-21T00:28:53.081094+0000","last_fullsized":"2024-09-21T00:28:53.081094+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:53.070133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:53.070133+0000","last_clean_scrub_stamp":"2024-09-21T00:28:53.070133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.117174+0000","last_change":"2024-09-21T00:28:55.105495+0000","last_active":"2024-09-21T00:28:56.117174+0000","last_peered":"2024-09-21T00:28:56.117174+0000","last_clean":"2024-09-21T00:28:56.117174+0000","last_became_active":"2024-09-21T00:28:55.105157+0000","last_became_peered":"2024-09-21T00:28:55.105157+0000","last_unstale":"2024-09-21T00:28:56.117174+0000","last_undegraded":"2024-09-21T00:28:56.117174+0000","last_fullsized":"2024-09-21T00:28:56.117174+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_clean_scrub_stamp":"2024-09-21T00:28:54.073989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:44:22.737668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-09-21T00:28:53.080727+0000","last_change":"2024-09-21T00:28:53.080727+0000","last_active":"2024-09-21T00:28:53.070133+0000","last_peered":"2024-09-21T00:28:53.070133+0000","last_clean":"2024-09-21T00:28:53.070133+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:28:53.080727+0000","last_undegraded":"2024-09-21T00:28:53.080727+0000","last_fullsized":"2024-09-21T00:28:53.080727+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:53.070133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:53.070133+0000","last_clean_scrub_stamp":"2024-09-21T00:28:53.070133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.117511+0000","last_change":"2024-09-21T00:28:55.100643+0000","last_active":"2024-09-21T00:28:56.117511+0000","last_peered":"2024-09-21T00:28:56.117511+0000","last_clean":"2024-09-21T00:28:56.117511+0000","last_became_active":"2024-09-21T00:28:55.099768+0000","last_became_peered":"2024-09-21T00:28:55.099768+0000","last_unstale":"2024-09-21T00:28:56.117511+0000","last_undegraded":"2024-09-21T00:28:56.117511+0000","last_fullsized":"2024-09-21T00:28:56.117511+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_clean_scrub_stamp":"2024-09-21T00:28:54.073989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:01:50.169364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"82'1","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.117647+0000","last_change":"2024-09-21T00:28:54.096470+0000","last_active":"2024-09-21T00:28:56.117647+0000","last_peered":"2024-09-21T00:28:56.117647+0000","last_clean":"2024-09-21T00:28:56.117647+0000","last_became_active":"2024-09-21T00:28:54.096025+0000","last_became_peered":"2024-09-21T00:28:54.096025+0000","last_unstale":"2024-09-21T00:28:56.117647+0000","last_undegraded":"2024-09-21T00:28:56.117647+0000","last_fullsized":"2024-09-21T00:28:56.117647+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:53.070133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:53.070133+0000","last_clean_scrub_stamp":"2024-09-21T00:28:53.070133+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:18:56.230631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.117432+0000","last_change":"2024-09-21T00:28:55.101246+0000","last_active":"2024-09-21T00:28:56.117432+0000","last_peered":"2024-09-21T00:28:56.117432+0000","last_clean":"2024-09-21T00:28:56.117432+0000","last_became_active":"2024-09-21T00:28:55.100937+0000","last_became_peered":"2024-09-21T00:28:55.100937+0000","last_unstale":"2024-09-21T00:28:56.117432+0000","last_undegraded":"2024-09-21T00:28:56.117432+0000","last_fullsized":"2024-09-21T00:28:56.117432+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_clean_scrub_stamp":"2024-09-21T00:28:54.073989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:28:58.329009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"82'2","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.103322+0000","last_change":"2024-09-21T00:28:54.098747+0000","last_active":"2024-09-21T00:28:56.103322+0000","last_peered":"2024-09-21T00:28:56.103322+0000","last_clean":"2024-09-21T00:28:56.103322+0000","last_became_active":"2024-09-21T00:28:54.098387+0000","last_became_peered":"2024-09-21T00:28:54.098387+0000","last_unstale":"2024-09-21T00:28:56.103322+0000","last_undegraded":"2024-09-21T00:28:56.103322+0000","last_fullsized":"2024-09-21T00:28:56.103322+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:53.070133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:53.070133+0000","last_clean_scrub_stamp":"2024-09-21T00:28:53.070133+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:31:01.601795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.102143+0000","last_change":"2024-09-21T00:28:55.097662+0000","last_active":"2024-09-21T00:28:56.102143+0000","last_peered":"2024-09-21T00:28:56.102143+0000","last_clean":"2024-09-21T00:28:56.102143+0000","last_became_active":"2024-09-21T00:28:55.097364+0000","last_became_peered":"2024-09-21T00:28:55.097364+0000","last_unstale":"2024-09-21T00:28:56.102143+0000","last_undegraded":"2024-09-21T00:28:56.102143+0000","last_fullsized":"2024-09-21T00:28:56.102143+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_clean_scrub_stamp":"2024-09-21T00:28:54.073989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:14:14.562450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"82'1","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.102426+0000","last_change":"2024-09-21T00:28:54.092612+0000","last_active":"2024-09-21T00:28:56.102426+0000","last_peered":"2024-09-21T00:28:56.102426+0000","last_clean":"2024-09-21T00:28:56.102426+0000","last_became_active":"2024-09-21T00:28:54.092276+0000","last_became_peered":"2024-09-21T00:28:54.092276+0000","last_unstale":"2024-09-21T00:28:56.102426+0000","last_undegraded":"2024-09-21T00:28:56.102426+0000","last_fullsized":"2024-09-21T00:28:56.102426+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:53.070133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:53.070133+0000","last_clean_scrub_stamp":"2024-09-21T00:28:53.070133+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:22:36.363798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.117050+0000","last_change":"2024-09-21T00:28:55.107909+0000","last_active":"2024-09-21T00:28:56.117050+0000","last_peered":"2024-09-21T00:28:56.117050+0000","last_clean":"2024-09-21T00:28:56.117050+0000","last_became_active":"2024-09-21T00:28:55.107607+0000","last_became_peered":"2024-09-21T00:28:55.107607+0000","last_unstale":"2024-09-21T00:28:56.117050+0000","last_undegraded":"2024-09-21T00:28:56.117050+0000","last_fullsized":"2024-09-21T00:28:56.117050+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_clean_scrub_stamp":"2024-09-21T00:28:54.073989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:34:52.284071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"82'2","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.117330+0000","last_change":"2024-09-21T00:28:54.087322+0000","last_active":"2024-09-21T00:28:56.117330+0000","last_peered":"2024-09-21T00:28:56.117330+0000","last_clean":"2024-09-21T00:28:56.117330+0000","last_became_active":"2024-09-21T00:28:54.086695+0000","last_became_peered":"2024-09-21T00:28:54.086695+0000","last_unstale":"2024-09-21T00:28:56.117330+0000","last_undegraded":"2024-09-21T00:28:56.117330+0000","last_fullsized":"2024-09-21T00:28:56.117330+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:53.070133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:53.070133+0000","last_clean_scrub_stamp":"2024-09-21T00:28:53.070133+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:11:39.474869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.117113+0000","last_change":"2024-09-21T00:28:55.103948+0000","last_active":"2024-09-21T00:28:56.117113+0000","last_peered":"2024-09-21T00:28:56.117113+0000","last_clean":"2024-09-21T00:28:56.117113+0000","last_became_active":"2024-09-21T00:28:55.103270+0000","last_became_peered":"2024-09-21T00:28:55.103270+0000","last_unstale":"2024-09-21T00:28:56.117113+0000","last_undegraded":"2024-09-21T00:28:56.117113+0000","last_fullsized":"2024-09-21T00:28:56.117113+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_clean_scrub_stamp":"2024-09-21T00:28:54.073989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:02:40.895903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.117437+0000","last_change":"2024-09-21T00:28:54.102703+0000","last_active":"2024-09-21T00:28:56.117437+0000","last_peered":"2024-09-21T00:28:56.117437+0000","last_clean":"2024-09-21T00:28:56.117437+0000","last_became_active":"2024-09-21T00:28:54.102324+0000","last_became_peered":"2024-09-21T00:28:54.102324+0000","last_unstale":"2024-09-21T00:28:56.117437+0000","last_undegraded":"2024-09-21T00:28:56.117437+0000","last_fullsized":"2024-09-21T00:28:56.117437+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:53.070133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:53.070133+0000","last_clean_scrub_stamp":"2024-09-21T00:28:53.070133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:26:58.990605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.117611+0000","last_change":"2024-09-21T00:28:55.108422+0000","last_active":"2024-09-21T00:28:56.117611+0000","last_peered":"2024-09-21T00:28:56.117611+0000","last_clean":"2024-09-21T00:28:56.117611+0000","last_became_active":"2024-09-21T00:28:55.108083+0000","last_became_peered":"2024-09-21T00:28:55.108083+0000","last_unstale":"2024-09-21T00:28:56.117611+0000","last_undegraded":"2024-09-21T00:28:56.117611+0000","last_fullsized":"2024-09-21T00:28:56.117611+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_clean_scrub_stamp":"2024-09-21T00:28:54.073989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:08:31.986233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.117254+0000","last_change":"2024-09-21T00:28:54.091594+0000","last_active":"2024-09-21T00:28:56.117254+0000","last_peered":"2024-09-21T00:28:56.117254+0000","last_clean":"2024-09-21T00:28:56.117254+0000","last_became_active":"2024-09-21T00:28:54.091281+0000","last_became_peered":"2024-09-21T00:28:54.091281+0000","last_unstale":"2024-09-21T00:28:56.117254+0000","last_undegraded":"2024-09-21T00:28:56.117254+0000","last_fullsized":"2024-09-21T00:28:56.117254+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:53.070133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:53.070133+0000","last_clean_scrub_stamp":"2024-09-21T00:28:53.070133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:07:02.063603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.117438+0000","last_change":"2024-09-21T00:28:55.096461+0000","last_active":"2024-09-21T00:28:56.117438+0000","last_peered":"2024-09-21T00:28:56.117438+0000","last_clean":"2024-09-21T00:28:56.117438+0000","last_became_active":"2024-09-21T00:28:55.096141+0000","last_became_peered":"2024-09-21T00:28:55.096141+0000","last_unstale":"2024-09-21T00:28:56.117438+0000","last_undegraded":"2024-09-21T00:28:56.117438+0000","last_fullsized":"2024-09-21T00:28:56.117438+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_clean_scrub_stamp":"2024-09-21T00:28:54.073989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:11:13.390972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.116888+0000","last_change":"2024-09-21T00:28:54.104692+0000","last_active":"2024-09-21T00:28:56.116888+0000","last_peered":"2024-09-21T00:28:56.116888+0000","last_clean":"2024-09-21T00:28:56.116888+0000","last_became_active":"2024-09-21T00:28:54.104402+0000","last_became_peered":"2024-09-21T00:28:54.104402+0000","last_unstale":"2024-09-21T00:28:56.116888+0000","last_undegraded":"2024-09-21T00:28:56.116888+0000","last_fullsized":"2024-09-21T00:28:56.116888+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:53.070133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:53.070133+0000","last_clean_scrub_stamp":"2024-09-21T00:28:53.070133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:37:49.815524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.117536+0000","last_change":"2024-09-21T00:28:55.111820+0000","last_active":"2024-09-21T00:28:56.117536+0000","last_peered":"2024-09-21T00:28:56.117536+0000","last_clean":"2024-09-21T00:28:56.117536+0000","last_became_active":"2024-09-21T00:28:55.110917+0000","last_became_peered":"2024-09-21T00:28:55.110917+0000","last_unstale":"2024-09-21T00:28:56.117536+0000","last_undegraded":"2024-09-21T00:28:56.117536+0000","last_fullsized":"2024-09-21T00:28:56.117536+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_clean_scrub_stamp":"2024-09-21T00:28:54.073989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:17:29.709648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"82'2","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.103597+0000","last_change":"2024-09-21T00:28:54.097697+0000","last_active":"2024-09-21T00:28:56.103597+0000","last_peered":"2024-09-21T00:28:56.103597+0000","last_clean":"2024-09-21T00:28:56.103597+0000","last_became_active":"2024-09-21T00:28:54.097295+0000","last_became_peered":"2024-09-21T00:28:54.097295+0000","last_unstale":"2024-09-21T00:28:56.103597+0000","last_undegraded":"2024-09-21T00:28:56.103597+0000","last_fullsized":"2024-09-21T00:28:56.103597+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:53.070133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:53.070133+0000","last_clean_scrub_stamp":"2024-09-21T00:28:53.070133+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:08:30.982377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.118267+0000","last_change":"2024-09-21T00:28:55.100919+0000","last_active":"2024-09-21T00:28:56.118267+0000","last_peered":"2024-09-21T00:28:56.118267+0000","last_clean":"2024-09-21T00:28:56.118267+0000","last_became_active":"2024-09-21T00:28:55.100512+0000","last_became_peered":"2024-09-21T00:28:55.100512+0000","last_unstale":"2024-09-21T00:28:56.118267+0000","last_undegraded":"2024-09-21T00:28:56.118267+0000","last_fullsized":"2024-09-21T00:28:56.118267+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_clean_scrub_stamp":"2024-09-21T00:28:54.073989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:20:28.590984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.102308+0000","last_change":"2024-09-21T00:28:54.092917+0000","last_active":"2024-09-21T00:28:56.102308+0000","last_peered":"2024-09-21T00:28:56.102308+0000","last_clean":"2024-09-21T00:28:56.102308+0000","last_became_active":"2024-09-21T00:28:54.091521+0000","last_became_peered":"2024-09-21T00:28:54.091521+0000","last_unstale":"2024-09-21T00:28:56.102308+0000","last_undegraded":"2024-09-21T00:28:56.102308+0000","last_fullsized":"2024-09-21T00:28:56.102308+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:53.070133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:53.070133+0000","last_clean_scrub_stamp":"2024-09-21T00:28:53.070133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:58:21.410692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.117845+0000","last_change":"2024-09-21T00:28:55.097489+0000","last_active":"2024-09-21T00:28:56.117845+0000","last_peered":"2024-09-21T00:28:56.117845+0000","last_clean":"2024-09-21T00:28:56.117845+0000","last_became_active":"2024-09-21T00:28:55.097153+0000","last_became_peered":"2024-09-21T00:28:55.097153+0000","last_unstale":"2024-09-21T00:28:56.117845+0000","last_undegraded":"2024-09-21T00:28:56.117845+0000","last_fullsized":"2024-09-21T00:28:56.117845+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_clean_scrub_stamp":"2024-09-21T00:28:54.073989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:34:23.039928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.102498+0000","last_change":"2024-09-21T00:28:54.099349+0000","last_active":"2024-09-21T00:28:56.102498+0000","last_peered":"2024-09-21T00:28:56.102498+0000","last_clean":"2024-09-21T00:28:56.102498+0000","last_became_active":"2024-09-21T00:28:54.098827+0000","last_became_peered":"2024-09-21T00:28:54.098827+0000","last_unstale":"2024-09-21T00:28:56.102498+0000","last_undegraded":"2024-09-21T00:28:56.102498+0000","last_fullsized":"2024-09-21T00:28:56.102498+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:53.070133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:53.070133+0000","last_clean_scrub_stamp":"2024-09-21T00:28:53.070133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:34:10.674315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.117992+0000","last_change":"2024-09-21T00:28:55.098865+0000","last_active":"2024-09-21T00:28:56.117992+0000","last_peered":"2024-09-21T00:28:56.117992+0000","last_clean":"2024-09-21T00:28:56.117992+0000","last_became_active":"2024-09-21T00:28:55.098542+0000","last_became_peered":"2024-09-21T00:28:55.098542+0000","last_unstale":"2024-09-21T00:28:56.117992+0000","last_undegraded":"2024-09-21T00:28:56.117992+0000","last_fullsized":"2024-09-21T00:28:56.117992+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_clean_scrub_stamp":"2024-09-21T00:28:54.073989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:36:38.879430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.117678+0000","last_change":"2024-09-21T00:28:54.100179+0000","last_active":"2024-09-21T00:28:56.117678+0000","last_peered":"2024-09-21T00:28:56.117678+0000","last_clean":"2024-09-21T00:28:56.117678+0000","last_became_active":"2024-09-21T00:28:54.099825+0000","last_became_peered":"2024-09-21T00:28:54.099825+0000","last_unstale":"2024-09-21T00:28:56.117678+0000","last_undegraded":"2024-09-21T00:28:56.117678+0000","last_fullsized":"2024-09-21T00:28:56.117678+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:53.070133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:53.070133+0000","last_clean_scrub_stamp":"2024-09-21T00:28:53.070133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:41:24.183356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.116768+0000","last_change":"2024-09-21T00:28:55.110728+0000","last_active":"2024-09-21T00:28:56.116768+0000","last_peered":"2024-09-21T00:28:56.116768+0000","last_clean":"2024-09-21T00:28:56.116768+0000","last_became_active":"2024-09-21T00:28:55.110402+0000","last_became_peered":"2024-09-21T00:28:55.110402+0000","last_unstale":"2024-09-21T00:28:56.116768+0000","last_undegraded":"2024-09-21T00:28:56.116768+0000","last_fullsized":"2024-09-21T00:28:56.116768+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_clean_scrub_stamp":"2024-09-21T00:28:54.073989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T00:54:15.553165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.117553+0000","last_change":"2024-09-21T00:28:54.098315+0000","last_active":"2024-09-21T00:28:56.117553+0000","last_peered":"2024-09-21T00:28:56.117553+0000","last_clean":"2024-09-21T00:28:56.117553+0000","last_became_active":"2024-09-21T00:28:54.098019+0000","last_became_peered":"2024-09-21T00:28:54.098019+0000","last_unstale":"2024-09-21T00:28:56.117553+0000","last_undegraded":"2024-09-21T00:28:56.117553+0000","last_fullsized":"2024-09-21T00:28:56.117553+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:53.070133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:53.070133+0000","last_clean_scrub_stamp":"2024-09-21T00:28:53.070133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:20:30.596891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.100181+0000","last_change":"2024-09-21T00:28:55.103561+0000","last_active":"2024-09-21T00:28:56.100181+0000","last_peered":"2024-09-21T00:28:56.100181+0000","last_clean":"2024-09-21T00:28:56.100181+0000","last_became_active":"2024-09-21T00:28:55.102654+0000","last_became_peered":"2024-09-21T00:28:55.102654+0000","last_unstale":"2024-09-21T00:28:56.100181+0000","last_undegraded":"2024-09-21T00:28:56.100181+0000","last_fullsized":"2024-09-21T00:28:56.100181+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_clean_scrub_stamp":"2024-09-21T00:28:54.073989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:45:07.464813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.117472+0000","last_change":"2024-09-21T00:28:54.086754+0000","last_active":"2024-09-21T00:28:56.117472+0000","last_peered":"2024-09-21T00:28:56.117472+0000","last_clean":"2024-09-21T00:28:56.117472+0000","last_became_active":"2024-09-21T00:28:54.086402+0000","last_became_peered":"2024-09-21T00:28:54.086402+0000","last_unstale":"2024-09-21T00:28:56.117472+0000","last_undegraded":"2024-09-21T00:28:56.117472+0000","last_fullsized":"2024-09-21T00:28:56.117472+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:53.070133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:53.070133+0000","last_clean_scrub_stamp":"2024-09-21T00:28:53.070133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:09:39.844275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.118130+0000","last_change":"2024-09-21T00:28:55.108305+0000","last_active":"2024-09-21T00:28:56.118130+0000","last_peered":"2024-09-21T00:28:56.118130+0000","last_clean":"2024-09-21T00:28:56.118130+0000","last_became_active":"2024-09-21T00:28:55.107988+0000","last_became_peered":"2024-09-21T00:28:55.107988+0000","last_unstale":"2024-09-21T00:28:56.118130+0000","last_undegraded":"2024-09-21T00:28:56.118130+0000","last_fullsized":"2024-09-21T00:28:56.118130+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_clean_scrub_stamp":"2024-09-21T00:28:54.073989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:11:21.675017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.118008+0000","last_change":"2024-09-21T00:28:54.100020+0000","last_active":"2024-09-21T00:28:56.118008+0000","last_peered":"2024-09-21T00:28:56.118008+0000","last_clean":"2024-09-21T00:28:56.118008+0000","last_became_active":"2024-09-21T00:28:54.099695+0000","last_became_peered":"2024-09-21T00:28:54.099695+0000","last_unstale":"2024-09-21T00:28:56.118008+0000","last_undegraded":"2024-09-21T00:28:56.118008+0000","last_fullsized":"2024-09-21T00:28:56.118008+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:53.070133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:53.070133+0000","last_clean_scrub_stamp":"2024-09-21T00:28:53.070133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:33:24.983637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.117677+0000","last_change":"2024-09-21T00:28:55.098909+0000","last_active":"2024-09-21T00:28:56.117677+0000","last_peered":"2024-09-21T00:28:56.117677+0000","last_clean":"2024-09-21T00:28:56.117677+0000","last_became_active":"2024-09-21T00:28:55.098525+0000","last_became_peered":"2024-09-21T00:28:55.098525+0000","last_unstale":"2024-09-21T00:28:56.117677+0000","last_undegraded":"2024-09-21T00:28:56.117677+0000","last_fullsized":"2024-09-21T00:28:56.117677+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_clean_scrub_stamp":"2024-09-21T00:28:54.073989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:19:07.363130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.103244+0000","last_change":"2024-09-21T00:28:54.102564+0000","last_active":"2024-09-21T00:28:56.103244+0000","last_peered":"2024-09-21T00:28:56.103244+0000","last_clean":"2024-09-21T00:28:56.103244+0000","last_became_active":"2024-09-21T00:28:54.102263+0000","last_became_peered":"2024-09-21T00:28:54.102263+0000","last_unstale":"2024-09-21T00:28:56.103244+0000","last_undegraded":"2024-09-21T00:28:56.103244+0000","last_fullsized":"2024-09-21T00:28:56.103244+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:53.070133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:53.070133+0000","last_clean_scrub_stamp":"2024-09-21T00:28:53.070133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:44:51.937525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.103416+0000","last_change":"2024-09-21T00:28:55.106196+0000","last_active":"2024-09-21T00:28:56.103416+0000","last_peered":"2024-09-21T00:28:56.103416+0000","last_clean":"2024-09-21T00:28:56.103416+0000","last_became_active":"2024-09-21T00:28:55.105879+0000","last_became_peered":"2024-09-21T00:28:55.105879+0000","last_unstale":"2024-09-21T00:28:56.103416+0000","last_undegraded":"2024-09-21T00:28:56.103416+0000","last_fullsized":"2024-09-21T00:28:56.103416+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_clean_scrub_stamp":"2024-09-21T00:28:54.073989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:25:50.171074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-09-21T00:28:53.081742+0000","last_change":"2024-09-21T00:28:53.081742+0000","last_active":"2024-09-21T00:28:53.070133+0000","last_peered":"2024-09-21T00:28:53.070133+0000","last_clean":"2024-09-21T00:28:53.070133+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:28:53.081742+0000","last_undegraded":"2024-09-21T00:28:53.081742+0000","last_fullsized":"2024-09-21T00:28:53.081742+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:53.070133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:53.070133+0000","last_clean_scrub_stamp":"2024-09-21T00:28:53.070133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":223,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.103093+0000","last_change":"2024-09-21T00:25:38.725792+0000","last_active":"2024-09-21T00:28:56.103093+0000","last_peered":"2024-09-21T00:28:56.103093+0000","last_clean":"2024-09-21T00:28:56.103093+0000","last_became_active":"2024-09-21T00:25:38.724668+0000","last_became_peered":"2024-09-21T00:25:38.724668+0000","last_unstale":"2024-09-21T00:28:56.103093+0000","last_undegraded":"2024-09-21T00:28:56.103093+0000","last_fullsized":"2024-09-21T00:28:56.103093+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:37.163216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:37.163216+0000","last_clean_scrub_stamp":"2024-09-21T00:25:37.163216+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:14:36.588660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":29,"ondisk_log_size":29,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":108,"acting":108,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509483,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32768,"kb_used_data":1580,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339072,"statfs":{"total":96636764160,"available":96603209728,"internally_reserved":0,"allocated":1617920,"data_stored":1048558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509484,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32196,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339644,"statfs":{"total":96636764160,"available":96603795456,"internally_reserved":0,"allocated":1024000,"data_stored":453658,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509483,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32176,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339664,"statfs":{"total":96636764160,"available":96603815936,"internally_reserved":0,"allocated":1003520,"data_stored":451442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509482,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32188,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339652,"statfs":{"total":96636764160,"available":96603803648,"internally_reserved":0,"allocated":1024000,"data_stored":458294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509484,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32176,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339664,"statfs":{"total":96636764160,"available":96603815936,"internally_reserved":0,"allocated":1003520,"data_stored":451442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509484,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28012,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343828,"statfs":{"total":96636764160,"available":96608079872,"internally_reserved":0,"allocated":1003520,"data_stored":451442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509484,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32780,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339060,"statfs":{"total":96636764160,"available":96603197440,"internally_reserved":0,"allocated":1622016,"data_stored":1044128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574891,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32136,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339704,"statfs":{"total":96636764160,"available":96603856896,"internally_reserved":0,"allocated":966656,"data_stored":431399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T00:28:59.120 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T00:28:59.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.236+0000 7ff932d19640 1 -- 172.21.15.112:0/3379229839 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff92c0ccbf0 msgr2=0x7ff92c0cd000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:59.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.236+0000 7ff932d19640 1 --2- 172.21.15.112:0/3379229839 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff92c0ccbf0 0x7ff92c0cd000 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7ff91c009830 tx=0x7ff91c02a480 comp rx=0 tx=0).stop 2024-09-21T00:28:59.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.236+0000 7ff932d19640 1 -- 172.21.15.112:0/3379229839 shutdown_connections 2024-09-21T00:28:59.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.236+0000 7ff932d19640 1 --2- 172.21.15.112:0/3379229839 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff92c0c5ee0 0x7ff92c0c62f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:59.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.236+0000 7ff932d19640 1 --2- 172.21.15.112:0/3379229839 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff92c0ccbf0 0x7ff92c0cd000 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:59.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.236+0000 7ff932d19640 1 -- 172.21.15.112:0/3379229839 >> 172.21.15.112:0/3379229839 conn(0x7ff92c0b42e0 msgr2=0x7ff92c0b4720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:59.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.236+0000 7ff932d19640 1 -- 172.21.15.112:0/3379229839 shutdown_connections 2024-09-21T00:28:59.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.236+0000 7ff932d19640 1 -- 172.21.15.112:0/3379229839 wait complete. 2024-09-21T00:28:59.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.238+0000 7ff932d19640 1 Processor -- start 2024-09-21T00:28:59.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.238+0000 7ff932d19640 1 -- start start 2024-09-21T00:28:59.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.239+0000 7ff932d19640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff92c0c5ee0 0x7ff92c141620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:59.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.239+0000 7ff932d19640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff92c0c6830 0x7ff92c155910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:59.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.239+0000 7ff932d19640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff92c0ccbf0 0x7ff92c155e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:59.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.239+0000 7ff932d19640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff92c0b3450 con 0x7ff92c0c6830 2024-09-21T00:28:59.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.239+0000 7ff932d19640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff92c0c6f10 con 0x7ff92c0ccbf0 2024-09-21T00:28:59.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.239+0000 7ff932d19640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff92c0b35d0 con 0x7ff92c0c5ee0 2024-09-21T00:28:59.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.239+0000 7ff92bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff92c0c6830 0x7ff92c155910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:59.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.239+0000 7ff92bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff92c0c6830 0x7ff92c155910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38158/0 (socket says 172.21.15.112:38158) 2024-09-21T00:28:59.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.239+0000 7ff92bfff640 1 -- 172.21.15.112:0/3527588665 learned_addr learned my addr 172.21.15.112:0/3527588665 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:28:59.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.240+0000 7ff930ab5640 1 --2- 172.21.15.112:0/3527588665 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff92c0c5ee0 0x7ff92c141620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:59.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.240+0000 7ff9312b6640 1 --2- 172.21.15.112:0/3527588665 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff92c0ccbf0 0x7ff92c155e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:59.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.240+0000 7ff92bfff640 1 -- 172.21.15.112:0/3527588665 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff92c0c5ee0 msgr2=0x7ff92c141620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:59.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.240+0000 7ff92bfff640 1 --2- 172.21.15.112:0/3527588665 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff92c0c5ee0 0x7ff92c141620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:59.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.240+0000 7ff92bfff640 1 -- 172.21.15.112:0/3527588665 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff92c0ccbf0 msgr2=0x7ff92c155e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:59.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.240+0000 7ff92bfff640 1 --2- 172.21.15.112:0/3527588665 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff92c0ccbf0 0x7ff92c155e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:59.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.240+0000 7ff92bfff640 1 -- 172.21.15.112:0/3527588665 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff92c150a30 con 0x7ff92c0c6830 2024-09-21T00:28:59.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.240+0000 7ff92bfff640 1 --2- 172.21.15.112:0/3527588665 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff92c0c6830 0x7ff92c155910 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7ff92000ecb0 tx=0x7ff92000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:59.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.240+0000 7ff9297fa640 1 -- 172.21.15.112:0/3527588665 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff920003b70 con 0x7ff92c0c6830 2024-09-21T00:28:59.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.240+0000 7ff9297fa640 1 -- 172.21.15.112:0/3527588665 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff920003d10 con 0x7ff92c0c6830 2024-09-21T00:28:59.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.240+0000 7ff932d19640 1 -- 172.21.15.112:0/3527588665 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff92c150d10 con 0x7ff92c0c6830 2024-09-21T00:28:59.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.240+0000 7ff9297fa640 1 -- 172.21.15.112:0/3527588665 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff920004280 con 0x7ff92c0c6830 2024-09-21T00:28:59.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.240+0000 7ff932d19640 1 -- 172.21.15.112:0/3527588665 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff92c1420f0 con 0x7ff92c0c6830 2024-09-21T00:28:59.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.241+0000 7ff932d19640 1 -- 172.21.15.112:0/3527588665 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff92c0c62f0 con 0x7ff92c0c6830 2024-09-21T00:28:59.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.242+0000 7ff9297fa640 1 -- 172.21.15.112:0/3527588665 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff920004420 con 0x7ff92c0c6830 2024-09-21T00:28:59.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.243+0000 7ff9297fa640 1 --2- 172.21.15.112:0/3527588665 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff918078b90 0x7ff91807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:59.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.243+0000 7ff9297fa640 1 -- 172.21.15.112:0/3527588665 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 6101+0+0 (secure 0 0 0) 0x7ff92009ba10 con 0x7ff92c0c6830 2024-09-21T00:28:59.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.243+0000 7ff930ab5640 1 --2- 172.21.15.112:0/3527588665 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff918078b90 0x7ff91807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:59.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.244+0000 7ff930ab5640 1 --2- 172.21.15.112:0/3527588665 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff918078b90 0x7ff91807b0a0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7ff91c034000 tx=0x7ff91c004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:59.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.247+0000 7ff9297fa640 1 -- 172.21.15.112:0/3527588665 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ff920063790 con 0x7ff92c0c6830 2024-09-21T00:28:59.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.422+0000 7ff932d19640 1 -- 172.21.15.112:0/3527588665 --> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff92c151860 con 0x7ff918078b90 2024-09-21T00:28:59.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.427+0000 7ff9297fa640 1 -- 172.21.15.112:0/3527588665 <== mgr.4103 v2:172.21.15.187:6800/2921522906 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252055 (secure 0 0 0) 0x7ff92c151860 con 0x7ff918078b90 2024-09-21T00:28:59.431 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:28:59.433 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2024-09-21T00:28:59.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.430+0000 7ff932d19640 1 -- 172.21.15.112:0/3527588665 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff918078b90 msgr2=0x7ff91807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:59.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.430+0000 7ff932d19640 1 --2- 172.21.15.112:0/3527588665 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff918078b90 0x7ff91807b0a0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7ff91c034000 tx=0x7ff91c004730 comp rx=0 tx=0).stop 2024-09-21T00:28:59.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.430+0000 7ff932d19640 1 -- 172.21.15.112:0/3527588665 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff92c0c6830 msgr2=0x7ff92c155910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:59.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.430+0000 7ff932d19640 1 --2- 172.21.15.112:0/3527588665 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff92c0c6830 0x7ff92c155910 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7ff92000ecb0 tx=0x7ff92000c6a0 comp rx=0 tx=0).stop 2024-09-21T00:28:59.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.430+0000 7ff932d19640 1 -- 172.21.15.112:0/3527588665 shutdown_connections 2024-09-21T00:28:59.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.430+0000 7ff932d19640 1 --2- 172.21.15.112:0/3527588665 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff92c0ccbf0 0x7ff92c155e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:59.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.430+0000 7ff932d19640 1 --2- 172.21.15.112:0/3527588665 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff918078b90 0x7ff91807b0a0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:59.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.430+0000 7ff932d19640 1 --2- 172.21.15.112:0/3527588665 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff92c0c6830 0x7ff92c155910 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:59.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.430+0000 7ff932d19640 1 --2- 172.21.15.112:0/3527588665 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff92c0c5ee0 0x7ff92c141620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:59.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.430+0000 7ff932d19640 1 -- 172.21.15.112:0/3527588665 >> 172.21.15.112:0/3527588665 conn(0x7ff92c0b42e0 msgr2=0x7ff92c0710b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:59.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.430+0000 7ff932d19640 1 -- 172.21.15.112:0/3527588665 shutdown_connections 2024-09-21T00:28:59.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.430+0000 7ff932d19640 1 -- 172.21.15.112:0/3527588665 wait complete. 2024-09-21T00:28:59.442 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":179,"stamp":"2024-09-21T00:28:59.138260+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":132,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":254484,"kb_used_data":9100,"kb_used_omap":12,"kb_used_meta":245171,"kb_avail":754720236,"statfs":{"total":773094113280,"available":772833521664,"internally_reserved":0,"allocated":9318400,"data_stored":4817168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":251055665},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001590"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.100503+0000","last_change":"2024-09-21T00:28:55.095212+0000","last_active":"2024-09-21T00:28:56.100503+0000","last_peered":"2024-09-21T00:28:56.100503+0000","last_clean":"2024-09-21T00:28:56.100503+0000","last_became_active":"2024-09-21T00:28:55.094938+0000","last_became_peered":"2024-09-21T00:28:55.094938+0000","last_unstale":"2024-09-21T00:28:56.100503+0000","last_undegraded":"2024-09-21T00:28:56.100503+0000","last_fullsized":"2024-09-21T00:28:56.100503+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_clean_scrub_stamp":"2024-09-21T00:28:54.073989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:30:46.125212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.117980+0000","last_change":"2024-09-21T00:28:55.096829+0000","last_active":"2024-09-21T00:28:56.117980+0000","last_peered":"2024-09-21T00:28:56.117980+0000","last_clean":"2024-09-21T00:28:56.117980+0000","last_became_active":"2024-09-21T00:28:55.096518+0000","last_became_peered":"2024-09-21T00:28:55.096518+0000","last_unstale":"2024-09-21T00:28:56.117980+0000","last_undegraded":"2024-09-21T00:28:56.117980+0000","last_fullsized":"2024-09-21T00:28:56.117980+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_clean_scrub_stamp":"2024-09-21T00:28:54.073989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:45:07.464813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.100223+0000","last_change":"2024-09-21T00:28:55.112040+0000","last_active":"2024-09-21T00:28:56.100223+0000","last_peered":"2024-09-21T00:28:56.100223+0000","last_clean":"2024-09-21T00:28:56.100223+0000","last_became_active":"2024-09-21T00:28:55.111715+0000","last_became_peered":"2024-09-21T00:28:55.111715+0000","last_unstale":"2024-09-21T00:28:56.100223+0000","last_undegraded":"2024-09-21T00:28:56.100223+0000","last_fullsized":"2024-09-21T00:28:56.100223+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_clean_scrub_stamp":"2024-09-21T00:28:54.073989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:08:29.568599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.118165+0000","last_change":"2024-09-21T00:28:55.114133+0000","last_active":"2024-09-21T00:28:56.118165+0000","last_peered":"2024-09-21T00:28:56.118165+0000","last_clean":"2024-09-21T00:28:56.118165+0000","last_became_active":"2024-09-21T00:28:55.113867+0000","last_became_peered":"2024-09-21T00:28:55.113867+0000","last_unstale":"2024-09-21T00:28:56.118165+0000","last_undegraded":"2024-09-21T00:28:56.118165+0000","last_fullsized":"2024-09-21T00:28:56.118165+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_clean_scrub_stamp":"2024-09-21T00:28:54.073989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:38:25.167831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.101835+0000","last_change":"2024-09-21T00:28:55.102995+0000","last_active":"2024-09-21T00:28:56.101835+0000","last_peered":"2024-09-21T00:28:56.101835+0000","last_clean":"2024-09-21T00:28:56.101835+0000","last_became_active":"2024-09-21T00:28:55.102678+0000","last_became_peered":"2024-09-21T00:28:55.102678+0000","last_unstale":"2024-09-21T00:28:56.101835+0000","last_undegraded":"2024-09-21T00:28:56.101835+0000","last_fullsized":"2024-09-21T00:28:56.101835+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_clean_scrub_stamp":"2024-09-21T00:28:54.073989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:57:54.006732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.117044+0000","last_change":"2024-09-21T00:28:55.104634+0000","last_active":"2024-09-21T00:28:56.117044+0000","last_peered":"2024-09-21T00:28:56.117044+0000","last_clean":"2024-09-21T00:28:56.117044+0000","last_became_active":"2024-09-21T00:28:55.104303+0000","last_became_peered":"2024-09-21T00:28:55.104303+0000","last_unstale":"2024-09-21T00:28:56.117044+0000","last_undegraded":"2024-09-21T00:28:56.117044+0000","last_fullsized":"2024-09-21T00:28:56.117044+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_clean_scrub_stamp":"2024-09-21T00:28:54.073989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:53:21.147188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.103023+0000","last_change":"2024-09-21T00:28:55.105218+0000","last_active":"2024-09-21T00:28:56.103023+0000","last_peered":"2024-09-21T00:28:56.103023+0000","last_clean":"2024-09-21T00:28:56.103023+0000","last_became_active":"2024-09-21T00:28:55.104883+0000","last_became_peered":"2024-09-21T00:28:55.104883+0000","last_unstale":"2024-09-21T00:28:56.103023+0000","last_undegraded":"2024-09-21T00:28:56.103023+0000","last_fullsized":"2024-09-21T00:28:56.103023+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_clean_scrub_stamp":"2024-09-21T00:28:54.073989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:30:46.125212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.118053+0000","last_change":"2024-09-21T00:28:55.110061+0000","last_active":"2024-09-21T00:28:56.118053+0000","last_peered":"2024-09-21T00:28:56.118053+0000","last_clean":"2024-09-21T00:28:56.118053+0000","last_became_active":"2024-09-21T00:28:55.109052+0000","last_became_peered":"2024-09-21T00:28:55.109052+0000","last_unstale":"2024-09-21T00:28:56.118053+0000","last_undegraded":"2024-09-21T00:28:56.118053+0000","last_fullsized":"2024-09-21T00:28:56.118053+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_clean_scrub_stamp":"2024-09-21T00:28:54.073989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:15:37.015438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.101639+0000","last_change":"2024-09-21T00:28:55.097933+0000","last_active":"2024-09-21T00:28:56.101639+0000","last_peered":"2024-09-21T00:28:56.101639+0000","last_clean":"2024-09-21T00:28:56.101639+0000","last_became_active":"2024-09-21T00:28:55.097556+0000","last_became_peered":"2024-09-21T00:28:55.097556+0000","last_unstale":"2024-09-21T00:28:56.101639+0000","last_undegraded":"2024-09-21T00:28:56.101639+0000","last_fullsized":"2024-09-21T00:28:56.101639+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_clean_scrub_stamp":"2024-09-21T00:28:54.073989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:46:42.028531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.118526+0000","last_change":"2024-09-21T00:28:55.104491+0000","last_active":"2024-09-21T00:28:56.118526+0000","last_peered":"2024-09-21T00:28:56.118526+0000","last_clean":"2024-09-21T00:28:56.118526+0000","last_became_active":"2024-09-21T00:28:55.104121+0000","last_became_peered":"2024-09-21T00:28:55.104121+0000","last_unstale":"2024-09-21T00:28:56.118526+0000","last_undegraded":"2024-09-21T00:28:56.118526+0000","last_fullsized":"2024-09-21T00:28:56.118526+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_clean_scrub_stamp":"2024-09-21T00:28:54.073989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:37:15.612342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.102169+0000","last_change":"2024-09-21T00:28:55.101986+0000","last_active":"2024-09-21T00:28:56.102169+0000","last_peered":"2024-09-21T00:28:56.102169+0000","last_clean":"2024-09-21T00:28:56.102169+0000","last_became_active":"2024-09-21T00:28:55.099761+0000","last_became_peered":"2024-09-21T00:28:55.099761+0000","last_unstale":"2024-09-21T00:28:56.102169+0000","last_undegraded":"2024-09-21T00:28:56.102169+0000","last_fullsized":"2024-09-21T00:28:56.102169+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_clean_scrub_stamp":"2024-09-21T00:28:54.073989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:11:40.478725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.116954+0000","last_change":"2024-09-21T00:28:55.097224+0000","last_active":"2024-09-21T00:28:56.116954+0000","last_peered":"2024-09-21T00:28:56.116954+0000","last_clean":"2024-09-21T00:28:56.116954+0000","last_became_active":"2024-09-21T00:28:55.096876+0000","last_became_peered":"2024-09-21T00:28:55.096876+0000","last_unstale":"2024-09-21T00:28:56.116954+0000","last_undegraded":"2024-09-21T00:28:56.116954+0000","last_fullsized":"2024-09-21T00:28:56.116954+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_clean_scrub_stamp":"2024-09-21T00:28:54.073989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:55:24.329815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.118097+0000","last_change":"2024-09-21T00:28:55.111365+0000","last_active":"2024-09-21T00:28:56.118097+0000","last_peered":"2024-09-21T00:28:56.118097+0000","last_clean":"2024-09-21T00:28:56.118097+0000","last_became_active":"2024-09-21T00:28:55.111059+0000","last_became_peered":"2024-09-21T00:28:55.111059+0000","last_unstale":"2024-09-21T00:28:56.118097+0000","last_undegraded":"2024-09-21T00:28:56.118097+0000","last_fullsized":"2024-09-21T00:28:56.118097+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_clean_scrub_stamp":"2024-09-21T00:28:54.073989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:11:21.675017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.101317+0000","last_change":"2024-09-21T00:28:55.110401+0000","last_active":"2024-09-21T00:28:56.101317+0000","last_peered":"2024-09-21T00:28:56.101317+0000","last_clean":"2024-09-21T00:28:56.101317+0000","last_became_active":"2024-09-21T00:28:55.109618+0000","last_became_peered":"2024-09-21T00:28:55.109618+0000","last_unstale":"2024-09-21T00:28:56.101317+0000","last_undegraded":"2024-09-21T00:28:56.101317+0000","last_fullsized":"2024-09-21T00:28:56.101317+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_clean_scrub_stamp":"2024-09-21T00:28:54.073989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:07:42.574101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.118604+0000","last_change":"2024-09-21T00:28:55.101903+0000","last_active":"2024-09-21T00:28:56.118604+0000","last_peered":"2024-09-21T00:28:56.118604+0000","last_clean":"2024-09-21T00:28:56.118604+0000","last_became_active":"2024-09-21T00:28:55.101528+0000","last_became_peered":"2024-09-21T00:28:55.101528+0000","last_unstale":"2024-09-21T00:28:56.118604+0000","last_undegraded":"2024-09-21T00:28:56.118604+0000","last_fullsized":"2024-09-21T00:28:56.118604+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_clean_scrub_stamp":"2024-09-21T00:28:54.073989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:27:29.318109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.117411+0000","last_change":"2024-09-21T00:28:55.102635+0000","last_active":"2024-09-21T00:28:56.117411+0000","last_peered":"2024-09-21T00:28:56.117411+0000","last_clean":"2024-09-21T00:28:56.117411+0000","last_became_active":"2024-09-21T00:28:55.098570+0000","last_became_peered":"2024-09-21T00:28:55.098570+0000","last_unstale":"2024-09-21T00:28:56.117411+0000","last_undegraded":"2024-09-21T00:28:56.117411+0000","last_fullsized":"2024-09-21T00:28:56.117411+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_clean_scrub_stamp":"2024-09-21T00:28:54.073989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:28:55.581773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.118625+0000","last_change":"2024-09-21T00:28:55.098790+0000","last_active":"2024-09-21T00:28:56.118625+0000","last_peered":"2024-09-21T00:28:56.118625+0000","last_clean":"2024-09-21T00:28:56.118625+0000","last_became_active":"2024-09-21T00:28:55.098436+0000","last_became_peered":"2024-09-21T00:28:55.098436+0000","last_unstale":"2024-09-21T00:28:56.118625+0000","last_undegraded":"2024-09-21T00:28:56.118625+0000","last_fullsized":"2024-09-21T00:28:56.118625+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_clean_scrub_stamp":"2024-09-21T00:28:54.073989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:21:44.839641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.100423+0000","last_change":"2024-09-21T00:28:55.096957+0000","last_active":"2024-09-21T00:28:56.100423+0000","last_peered":"2024-09-21T00:28:56.100423+0000","last_clean":"2024-09-21T00:28:56.100423+0000","last_became_active":"2024-09-21T00:28:55.096670+0000","last_became_peered":"2024-09-21T00:28:55.096670+0000","last_unstale":"2024-09-21T00:28:56.100423+0000","last_undegraded":"2024-09-21T00:28:56.100423+0000","last_fullsized":"2024-09-21T00:28:56.100423+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_clean_scrub_stamp":"2024-09-21T00:28:54.073989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:05:45.788017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.102307+0000","last_change":"2024-09-21T00:28:55.111532+0000","last_active":"2024-09-21T00:28:56.102307+0000","last_peered":"2024-09-21T00:28:56.102307+0000","last_clean":"2024-09-21T00:28:56.102307+0000","last_became_active":"2024-09-21T00:28:55.106336+0000","last_became_peered":"2024-09-21T00:28:55.106336+0000","last_unstale":"2024-09-21T00:28:56.102307+0000","last_undegraded":"2024-09-21T00:28:56.102307+0000","last_fullsized":"2024-09-21T00:28:56.102307+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_clean_scrub_stamp":"2024-09-21T00:28:54.073989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:57:19.321859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.118415+0000","last_change":"2024-09-21T00:28:55.101847+0000","last_active":"2024-09-21T00:28:56.118415+0000","last_peered":"2024-09-21T00:28:56.118415+0000","last_clean":"2024-09-21T00:28:56.118415+0000","last_became_active":"2024-09-21T00:28:55.101474+0000","last_became_peered":"2024-09-21T00:28:55.101474+0000","last_unstale":"2024-09-21T00:28:56.118415+0000","last_undegraded":"2024-09-21T00:28:56.118415+0000","last_fullsized":"2024-09-21T00:28:56.118415+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_clean_scrub_stamp":"2024-09-21T00:28:54.073989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:44:25.329810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.101448+0000","last_change":"2024-09-21T00:28:55.108612+0000","last_active":"2024-09-21T00:28:56.101448+0000","last_peered":"2024-09-21T00:28:56.101448+0000","last_clean":"2024-09-21T00:28:56.101448+0000","last_became_active":"2024-09-21T00:28:55.108272+0000","last_became_peered":"2024-09-21T00:28:55.108272+0000","last_unstale":"2024-09-21T00:28:56.101448+0000","last_undegraded":"2024-09-21T00:28:56.101448+0000","last_fullsized":"2024-09-21T00:28:56.101448+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_clean_scrub_stamp":"2024-09-21T00:28:54.073989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:35:00.733519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.101524+0000","last_change":"2024-09-21T00:28:55.109107+0000","last_active":"2024-09-21T00:28:56.101524+0000","last_peered":"2024-09-21T00:28:56.101524+0000","last_clean":"2024-09-21T00:28:56.101524+0000","last_became_active":"2024-09-21T00:28:55.107380+0000","last_became_peered":"2024-09-21T00:28:55.107380+0000","last_unstale":"2024-09-21T00:28:56.101524+0000","last_undegraded":"2024-09-21T00:28:56.101524+0000","last_fullsized":"2024-09-21T00:28:56.101524+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_clean_scrub_stamp":"2024-09-21T00:28:54.073989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:46:43.542825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.118127+0000","last_change":"2024-09-21T00:28:55.108284+0000","last_active":"2024-09-21T00:28:56.118127+0000","last_peered":"2024-09-21T00:28:56.118127+0000","last_clean":"2024-09-21T00:28:56.118127+0000","last_became_active":"2024-09-21T00:28:55.107930+0000","last_became_peered":"2024-09-21T00:28:55.107930+0000","last_unstale":"2024-09-21T00:28:56.118127+0000","last_undegraded":"2024-09-21T00:28:56.118127+0000","last_fullsized":"2024-09-21T00:28:56.118127+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_clean_scrub_stamp":"2024-09-21T00:28:54.073989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:12:40.407615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.100202+0000","last_change":"2024-09-21T00:28:55.105271+0000","last_active":"2024-09-21T00:28:56.100202+0000","last_peered":"2024-09-21T00:28:56.100202+0000","last_clean":"2024-09-21T00:28:56.100202+0000","last_became_active":"2024-09-21T00:28:55.104870+0000","last_became_peered":"2024-09-21T00:28:55.104870+0000","last_unstale":"2024-09-21T00:28:56.100202+0000","last_undegraded":"2024-09-21T00:28:56.100202+0000","last_fullsized":"2024-09-21T00:28:56.100202+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_clean_scrub_stamp":"2024-09-21T00:28:54.073989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:36:38.879430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.117745+0000","last_change":"2024-09-21T00:28:55.108523+0000","last_active":"2024-09-21T00:28:56.117745+0000","last_peered":"2024-09-21T00:28:56.117745+0000","last_clean":"2024-09-21T00:28:56.117745+0000","last_became_active":"2024-09-21T00:28:55.108179+0000","last_became_peered":"2024-09-21T00:28:55.108179+0000","last_unstale":"2024-09-21T00:28:56.117745+0000","last_undegraded":"2024-09-21T00:28:56.117745+0000","last_fullsized":"2024-09-21T00:28:56.117745+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_clean_scrub_stamp":"2024-09-21T00:28:54.073989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:32:16.237390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.102149+0000","last_change":"2024-09-21T00:28:55.106799+0000","last_active":"2024-09-21T00:28:56.102149+0000","last_peered":"2024-09-21T00:28:56.102149+0000","last_clean":"2024-09-21T00:28:56.102149+0000","last_became_active":"2024-09-21T00:28:55.106328+0000","last_became_peered":"2024-09-21T00:28:55.106328+0000","last_unstale":"2024-09-21T00:28:56.102149+0000","last_undegraded":"2024-09-21T00:28:56.102149+0000","last_fullsized":"2024-09-21T00:28:56.102149+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_clean_scrub_stamp":"2024-09-21T00:28:54.073989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:01:07.579102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.117158+0000","last_change":"2024-09-21T00:28:55.104075+0000","last_active":"2024-09-21T00:28:56.117158+0000","last_peered":"2024-09-21T00:28:56.117158+0000","last_clean":"2024-09-21T00:28:56.117158+0000","last_became_active":"2024-09-21T00:28:55.103545+0000","last_became_peered":"2024-09-21T00:28:55.103545+0000","last_unstale":"2024-09-21T00:28:56.117158+0000","last_undegraded":"2024-09-21T00:28:56.117158+0000","last_fullsized":"2024-09-21T00:28:56.117158+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_clean_scrub_stamp":"2024-09-21T00:28:54.073989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:01.343438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.118732+0000","last_change":"2024-09-21T00:28:55.113285+0000","last_active":"2024-09-21T00:28:56.118732+0000","last_peered":"2024-09-21T00:28:56.118732+0000","last_clean":"2024-09-21T00:28:56.118732+0000","last_became_active":"2024-09-21T00:28:55.112929+0000","last_became_peered":"2024-09-21T00:28:55.112929+0000","last_unstale":"2024-09-21T00:28:56.118732+0000","last_undegraded":"2024-09-21T00:28:56.118732+0000","last_fullsized":"2024-09-21T00:28:56.118732+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_clean_scrub_stamp":"2024-09-21T00:28:54.073989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:23:11.927488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.103442+0000","last_change":"2024-09-21T00:28:55.107136+0000","last_active":"2024-09-21T00:28:56.103442+0000","last_peered":"2024-09-21T00:28:56.103442+0000","last_clean":"2024-09-21T00:28:56.103442+0000","last_became_active":"2024-09-21T00:28:55.106829+0000","last_became_peered":"2024-09-21T00:28:55.106829+0000","last_unstale":"2024-09-21T00:28:56.103442+0000","last_undegraded":"2024-09-21T00:28:56.103442+0000","last_fullsized":"2024-09-21T00:28:56.103442+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_clean_scrub_stamp":"2024-09-21T00:28:54.073989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:26:18.993673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.101693+0000","last_change":"2024-09-21T00:28:55.099077+0000","last_active":"2024-09-21T00:28:56.101693+0000","last_peered":"2024-09-21T00:28:56.101693+0000","last_clean":"2024-09-21T00:28:56.101693+0000","last_became_active":"2024-09-21T00:28:55.098377+0000","last_became_peered":"2024-09-21T00:28:55.098377+0000","last_unstale":"2024-09-21T00:28:56.101693+0000","last_undegraded":"2024-09-21T00:28:56.101693+0000","last_fullsized":"2024-09-21T00:28:56.101693+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_clean_scrub_stamp":"2024-09-21T00:28:54.073989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:44:37.308136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.117119+0000","last_change":"2024-09-21T00:28:55.109292+0000","last_active":"2024-09-21T00:28:56.117119+0000","last_peered":"2024-09-21T00:28:56.117119+0000","last_clean":"2024-09-21T00:28:56.117119+0000","last_became_active":"2024-09-21T00:28:55.108969+0000","last_became_peered":"2024-09-21T00:28:55.108969+0000","last_unstale":"2024-09-21T00:28:56.117119+0000","last_undegraded":"2024-09-21T00:28:56.117119+0000","last_fullsized":"2024-09-21T00:28:56.117119+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_clean_scrub_stamp":"2024-09-21T00:28:54.073989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:31:02.605652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.116824+0000","last_change":"2024-09-21T00:28:55.099447+0000","last_active":"2024-09-21T00:28:56.116824+0000","last_peered":"2024-09-21T00:28:56.116824+0000","last_clean":"2024-09-21T00:28:56.116824+0000","last_became_active":"2024-09-21T00:28:55.097567+0000","last_became_peered":"2024-09-21T00:28:55.097567+0000","last_unstale":"2024-09-21T00:28:56.116824+0000","last_undegraded":"2024-09-21T00:28:56.116824+0000","last_fullsized":"2024-09-21T00:28:56.116824+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_clean_scrub_stamp":"2024-09-21T00:28:54.073989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:59:16.615558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.117471+0000","last_change":"2024-09-21T00:28:55.101000+0000","last_active":"2024-09-21T00:28:56.117471+0000","last_peered":"2024-09-21T00:28:56.117471+0000","last_clean":"2024-09-21T00:28:56.117471+0000","last_became_active":"2024-09-21T00:28:55.100265+0000","last_became_peered":"2024-09-21T00:28:55.100265+0000","last_unstale":"2024-09-21T00:28:56.117471+0000","last_undegraded":"2024-09-21T00:28:56.117471+0000","last_fullsized":"2024-09-21T00:28:56.117471+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_clean_scrub_stamp":"2024-09-21T00:28:54.073989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:24:20.572343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"82'4","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.117415+0000","last_change":"2024-09-21T00:28:54.103623+0000","last_active":"2024-09-21T00:28:56.117415+0000","last_peered":"2024-09-21T00:28:56.117415+0000","last_clean":"2024-09-21T00:28:56.117415+0000","last_became_active":"2024-09-21T00:28:54.103177+0000","last_became_peered":"2024-09-21T00:28:54.103177+0000","last_unstale":"2024-09-21T00:28:56.117415+0000","last_undegraded":"2024-09-21T00:28:56.117415+0000","last_fullsized":"2024-09-21T00:28:56.117415+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:53.070133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:53.070133+0000","last_clean_scrub_stamp":"2024-09-21T00:28:53.070133+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:57:41.206766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.117770+0000","last_change":"2024-09-21T00:28:55.099060+0000","last_active":"2024-09-21T00:28:56.117770+0000","last_peered":"2024-09-21T00:28:56.117770+0000","last_clean":"2024-09-21T00:28:56.117770+0000","last_became_active":"2024-09-21T00:28:55.096215+0000","last_became_peered":"2024-09-21T00:28:55.096215+0000","last_unstale":"2024-09-21T00:28:56.117770+0000","last_undegraded":"2024-09-21T00:28:56.117770+0000","last_fullsized":"2024-09-21T00:28:56.117770+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_clean_scrub_stamp":"2024-09-21T00:28:54.073989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:41:13.683153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"82'1","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.102341+0000","last_change":"2024-09-21T00:28:54.094593+0000","last_active":"2024-09-21T00:28:56.102341+0000","last_peered":"2024-09-21T00:28:56.102341+0000","last_clean":"2024-09-21T00:28:56.102341+0000","last_became_active":"2024-09-21T00:28:54.094275+0000","last_became_peered":"2024-09-21T00:28:54.094275+0000","last_unstale":"2024-09-21T00:28:56.102341+0000","last_undegraded":"2024-09-21T00:28:56.102341+0000","last_fullsized":"2024-09-21T00:28:56.102341+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:53.070133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:53.070133+0000","last_clean_scrub_stamp":"2024-09-21T00:28:53.070133+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:38:53.041398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.103172+0000","last_change":"2024-09-21T00:28:55.105392+0000","last_active":"2024-09-21T00:28:56.103172+0000","last_peered":"2024-09-21T00:28:56.103172+0000","last_clean":"2024-09-21T00:28:56.103172+0000","last_became_active":"2024-09-21T00:28:55.104858+0000","last_became_peered":"2024-09-21T00:28:55.104858+0000","last_unstale":"2024-09-21T00:28:56.103172+0000","last_undegraded":"2024-09-21T00:28:56.103172+0000","last_fullsized":"2024-09-21T00:28:56.103172+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_clean_scrub_stamp":"2024-09-21T00:28:54.073989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T00:48:51.087715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"82'2","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.103357+0000","last_change":"2024-09-21T00:28:54.102165+0000","last_active":"2024-09-21T00:28:56.103357+0000","last_peered":"2024-09-21T00:28:56.103357+0000","last_clean":"2024-09-21T00:28:56.103357+0000","last_became_active":"2024-09-21T00:28:54.101820+0000","last_became_peered":"2024-09-21T00:28:54.101820+0000","last_unstale":"2024-09-21T00:28:56.103357+0000","last_undegraded":"2024-09-21T00:28:56.103357+0000","last_fullsized":"2024-09-21T00:28:56.103357+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:53.070133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:53.070133+0000","last_clean_scrub_stamp":"2024-09-21T00:28:53.070133+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:51:30.664168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.118626+0000","last_change":"2024-09-21T00:28:55.113067+0000","last_active":"2024-09-21T00:28:56.118626+0000","last_peered":"2024-09-21T00:28:56.118626+0000","last_clean":"2024-09-21T00:28:56.118626+0000","last_became_active":"2024-09-21T00:28:55.109815+0000","last_became_peered":"2024-09-21T00:28:55.109815+0000","last_unstale":"2024-09-21T00:28:56.118626+0000","last_undegraded":"2024-09-21T00:28:56.118626+0000","last_fullsized":"2024-09-21T00:28:56.118626+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_clean_scrub_stamp":"2024-09-21T00:28:54.073989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:29:37.203382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"82'2","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.117648+0000","last_change":"2024-09-21T00:28:54.098815+0000","last_active":"2024-09-21T00:28:56.117648+0000","last_peered":"2024-09-21T00:28:56.117648+0000","last_clean":"2024-09-21T00:28:56.117648+0000","last_became_active":"2024-09-21T00:28:54.098487+0000","last_became_peered":"2024-09-21T00:28:54.098487+0000","last_unstale":"2024-09-21T00:28:56.117648+0000","last_undegraded":"2024-09-21T00:28:56.117648+0000","last_fullsized":"2024-09-21T00:28:56.117648+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:53.070133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:53.070133+0000","last_clean_scrub_stamp":"2024-09-21T00:28:53.070133+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:41:09.729837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.118780+0000","last_change":"2024-09-21T00:28:55.108449+0000","last_active":"2024-09-21T00:28:56.118780+0000","last_peered":"2024-09-21T00:28:56.118780+0000","last_clean":"2024-09-21T00:28:56.118780+0000","last_became_active":"2024-09-21T00:28:55.105775+0000","last_became_peered":"2024-09-21T00:28:55.105775+0000","last_unstale":"2024-09-21T00:28:56.118780+0000","last_undegraded":"2024-09-21T00:28:56.118780+0000","last_fullsized":"2024-09-21T00:28:56.118780+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_clean_scrub_stamp":"2024-09-21T00:28:54.073989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:46:52.052930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"82'1","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.101829+0000","last_change":"2024-09-21T00:28:54.099669+0000","last_active":"2024-09-21T00:28:56.101829+0000","last_peered":"2024-09-21T00:28:56.101829+0000","last_clean":"2024-09-21T00:28:56.101829+0000","last_became_active":"2024-09-21T00:28:54.099356+0000","last_became_peered":"2024-09-21T00:28:54.099356+0000","last_unstale":"2024-09-21T00:28:56.101829+0000","last_undegraded":"2024-09-21T00:28:56.101829+0000","last_fullsized":"2024-09-21T00:28:56.101829+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:53.070133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:53.070133+0000","last_clean_scrub_stamp":"2024-09-21T00:28:53.070133+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:48:15.713474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.117768+0000","last_change":"2024-09-21T00:28:55.101974+0000","last_active":"2024-09-21T00:28:56.117768+0000","last_peered":"2024-09-21T00:28:56.117768+0000","last_clean":"2024-09-21T00:28:56.117768+0000","last_became_active":"2024-09-21T00:28:55.101451+0000","last_became_peered":"2024-09-21T00:28:55.101451+0000","last_unstale":"2024-09-21T00:28:56.117768+0000","last_undegraded":"2024-09-21T00:28:56.117768+0000","last_fullsized":"2024-09-21T00:28:56.117768+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_clean_scrub_stamp":"2024-09-21T00:28:54.073989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:12:08.777614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"82'1","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.118406+0000","last_change":"2024-09-21T00:28:54.098875+0000","last_active":"2024-09-21T00:28:56.118406+0000","last_peered":"2024-09-21T00:28:56.118406+0000","last_clean":"2024-09-21T00:28:56.118406+0000","last_became_active":"2024-09-21T00:28:54.098578+0000","last_became_peered":"2024-09-21T00:28:54.098578+0000","last_unstale":"2024-09-21T00:28:56.118406+0000","last_undegraded":"2024-09-21T00:28:56.118406+0000","last_fullsized":"2024-09-21T00:28:56.118406+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:53.070133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:53.070133+0000","last_clean_scrub_stamp":"2024-09-21T00:28:53.070133+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:05:26.107188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.117305+0000","last_change":"2024-09-21T00:28:55.102571+0000","last_active":"2024-09-21T00:28:56.117305+0000","last_peered":"2024-09-21T00:28:56.117305+0000","last_clean":"2024-09-21T00:28:56.117305+0000","last_became_active":"2024-09-21T00:28:55.102270+0000","last_became_peered":"2024-09-21T00:28:55.102270+0000","last_unstale":"2024-09-21T00:28:56.117305+0000","last_undegraded":"2024-09-21T00:28:56.117305+0000","last_fullsized":"2024-09-21T00:28:56.117305+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_clean_scrub_stamp":"2024-09-21T00:28:54.073989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:14:27.778681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.100134+0000","last_change":"2024-09-21T00:28:54.096435+0000","last_active":"2024-09-21T00:28:56.100134+0000","last_peered":"2024-09-21T00:28:56.100134+0000","last_clean":"2024-09-21T00:28:56.100134+0000","last_became_active":"2024-09-21T00:28:54.096080+0000","last_became_peered":"2024-09-21T00:28:54.096080+0000","last_unstale":"2024-09-21T00:28:56.100134+0000","last_undegraded":"2024-09-21T00:28:56.100134+0000","last_fullsized":"2024-09-21T00:28:56.100134+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:53.070133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:53.070133+0000","last_clean_scrub_stamp":"2024-09-21T00:28:53.070133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T00:56:36.844091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.118653+0000","last_change":"2024-09-21T00:28:55.113870+0000","last_active":"2024-09-21T00:28:56.118653+0000","last_peered":"2024-09-21T00:28:56.118653+0000","last_clean":"2024-09-21T00:28:56.118653+0000","last_became_active":"2024-09-21T00:28:55.113578+0000","last_became_peered":"2024-09-21T00:28:55.113578+0000","last_unstale":"2024-09-21T00:28:56.118653+0000","last_undegraded":"2024-09-21T00:28:56.118653+0000","last_fullsized":"2024-09-21T00:28:56.118653+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_clean_scrub_stamp":"2024-09-21T00:28:54.073989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:39:56.389478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.102943+0000","last_change":"2024-09-21T00:28:54.102451+0000","last_active":"2024-09-21T00:28:56.102943+0000","last_peered":"2024-09-21T00:28:56.102943+0000","last_clean":"2024-09-21T00:28:56.102943+0000","last_became_active":"2024-09-21T00:28:54.102143+0000","last_became_peered":"2024-09-21T00:28:54.102143+0000","last_unstale":"2024-09-21T00:28:56.102943+0000","last_undegraded":"2024-09-21T00:28:56.102943+0000","last_fullsized":"2024-09-21T00:28:56.102943+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:53.070133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:53.070133+0000","last_clean_scrub_stamp":"2024-09-21T00:28:53.070133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:18:29.567994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.102206+0000","last_change":"2024-09-21T00:28:55.098462+0000","last_active":"2024-09-21T00:28:56.102206+0000","last_peered":"2024-09-21T00:28:56.102206+0000","last_clean":"2024-09-21T00:28:56.102206+0000","last_became_active":"2024-09-21T00:28:55.097998+0000","last_became_peered":"2024-09-21T00:28:55.097998+0000","last_unstale":"2024-09-21T00:28:56.102206+0000","last_undegraded":"2024-09-21T00:28:56.102206+0000","last_fullsized":"2024-09-21T00:28:56.102206+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_clean_scrub_stamp":"2024-09-21T00:28:54.073989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:52:09.676331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.118115+0000","last_change":"2024-09-21T00:28:54.099224+0000","last_active":"2024-09-21T00:28:56.118115+0000","last_peered":"2024-09-21T00:28:56.118115+0000","last_clean":"2024-09-21T00:28:56.118115+0000","last_became_active":"2024-09-21T00:28:54.098825+0000","last_became_peered":"2024-09-21T00:28:54.098825+0000","last_unstale":"2024-09-21T00:28:56.118115+0000","last_undegraded":"2024-09-21T00:28:56.118115+0000","last_fullsized":"2024-09-21T00:28:56.118115+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:53.070133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:53.070133+0000","last_clean_scrub_stamp":"2024-09-21T00:28:53.070133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:32:52.896230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.103170+0000","last_change":"2024-09-21T00:28:55.113005+0000","last_active":"2024-09-21T00:28:56.103170+0000","last_peered":"2024-09-21T00:28:56.103170+0000","last_clean":"2024-09-21T00:28:56.103170+0000","last_became_active":"2024-09-21T00:28:55.112590+0000","last_became_peered":"2024-09-21T00:28:55.112590+0000","last_unstale":"2024-09-21T00:28:56.103170+0000","last_undegraded":"2024-09-21T00:28:56.103170+0000","last_fullsized":"2024-09-21T00:28:56.103170+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_clean_scrub_stamp":"2024-09-21T00:28:54.073989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:30:09.350514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.100523+0000","last_change":"2024-09-21T00:28:54.092302+0000","last_active":"2024-09-21T00:28:56.100523+0000","last_peered":"2024-09-21T00:28:56.100523+0000","last_clean":"2024-09-21T00:28:56.100523+0000","last_became_active":"2024-09-21T00:28:54.091991+0000","last_became_peered":"2024-09-21T00:28:54.091991+0000","last_unstale":"2024-09-21T00:28:56.100523+0000","last_undegraded":"2024-09-21T00:28:56.100523+0000","last_fullsized":"2024-09-21T00:28:56.100523+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:53.070133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:53.070133+0000","last_clean_scrub_stamp":"2024-09-21T00:28:53.070133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:05:52.153493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.100471+0000","last_change":"2024-09-21T00:28:55.095188+0000","last_active":"2024-09-21T00:28:56.100471+0000","last_peered":"2024-09-21T00:28:56.100471+0000","last_clean":"2024-09-21T00:28:56.100471+0000","last_became_active":"2024-09-21T00:28:55.094877+0000","last_became_peered":"2024-09-21T00:28:55.094877+0000","last_unstale":"2024-09-21T00:28:56.100471+0000","last_undegraded":"2024-09-21T00:28:56.100471+0000","last_fullsized":"2024-09-21T00:28:56.100471+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_clean_scrub_stamp":"2024-09-21T00:28:54.073989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:40:41.255893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"82'3","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.102407+0000","last_change":"2024-09-21T00:28:54.099841+0000","last_active":"2024-09-21T00:28:56.102407+0000","last_peered":"2024-09-21T00:28:56.102407+0000","last_clean":"2024-09-21T00:28:56.102407+0000","last_became_active":"2024-09-21T00:28:54.099522+0000","last_became_peered":"2024-09-21T00:28:54.099522+0000","last_unstale":"2024-09-21T00:28:56.102407+0000","last_undegraded":"2024-09-21T00:28:56.102407+0000","last_fullsized":"2024-09-21T00:28:56.102407+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:53.070133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:53.070133+0000","last_clean_scrub_stamp":"2024-09-21T00:28:53.070133+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:27:40.663017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.118363+0000","last_change":"2024-09-21T00:28:55.106382+0000","last_active":"2024-09-21T00:28:56.118363+0000","last_peered":"2024-09-21T00:28:56.118363+0000","last_clean":"2024-09-21T00:28:56.118363+0000","last_became_active":"2024-09-21T00:28:55.106075+0000","last_became_peered":"2024-09-21T00:28:55.106075+0000","last_unstale":"2024-09-21T00:28:56.118363+0000","last_undegraded":"2024-09-21T00:28:56.118363+0000","last_fullsized":"2024-09-21T00:28:56.118363+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_clean_scrub_stamp":"2024-09-21T00:28:54.073989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:45:05.690417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"82'6","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.117603+0000","last_change":"2024-09-21T00:28:54.100386+0000","last_active":"2024-09-21T00:28:56.117603+0000","last_peered":"2024-09-21T00:28:56.117603+0000","last_clean":"2024-09-21T00:28:56.117603+0000","last_became_active":"2024-09-21T00:28:54.100051+0000","last_became_peered":"2024-09-21T00:28:54.100051+0000","last_unstale":"2024-09-21T00:28:56.117603+0000","last_undegraded":"2024-09-21T00:28:56.117603+0000","last_fullsized":"2024-09-21T00:28:56.117603+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:53.070133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:53.070133+0000","last_clean_scrub_stamp":"2024-09-21T00:28:53.070133+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T00:53:51.470817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.118804+0000","last_change":"2024-09-21T00:28:55.110227+0000","last_active":"2024-09-21T00:28:56.118804+0000","last_peered":"2024-09-21T00:28:56.118804+0000","last_clean":"2024-09-21T00:28:56.118804+0000","last_became_active":"2024-09-21T00:28:55.108876+0000","last_became_peered":"2024-09-21T00:28:55.108876+0000","last_unstale":"2024-09-21T00:28:56.118804+0000","last_undegraded":"2024-09-21T00:28:56.118804+0000","last_fullsized":"2024-09-21T00:28:56.118804+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_clean_scrub_stamp":"2024-09-21T00:28:54.073989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:06:33.858660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"82'2","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.103382+0000","last_change":"2024-09-21T00:28:54.098296+0000","last_active":"2024-09-21T00:28:56.103382+0000","last_peered":"2024-09-21T00:28:56.103382+0000","last_clean":"2024-09-21T00:28:56.103382+0000","last_became_active":"2024-09-21T00:28:54.097982+0000","last_became_peered":"2024-09-21T00:28:54.097982+0000","last_unstale":"2024-09-21T00:28:56.103382+0000","last_undegraded":"2024-09-21T00:28:56.103382+0000","last_fullsized":"2024-09-21T00:28:56.103382+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:53.070133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:53.070133+0000","last_clean_scrub_stamp":"2024-09-21T00:28:53.070133+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:32:15.233534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.101320+0000","last_change":"2024-09-21T00:28:55.104064+0000","last_active":"2024-09-21T00:28:56.101320+0000","last_peered":"2024-09-21T00:28:56.101320+0000","last_clean":"2024-09-21T00:28:56.101320+0000","last_became_active":"2024-09-21T00:28:55.103734+0000","last_became_peered":"2024-09-21T00:28:55.103734+0000","last_unstale":"2024-09-21T00:28:56.101320+0000","last_undegraded":"2024-09-21T00:28:56.101320+0000","last_fullsized":"2024-09-21T00:28:56.101320+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_clean_scrub_stamp":"2024-09-21T00:28:54.073989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:47:48.194847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.117170+0000","last_change":"2024-09-21T00:28:54.098640+0000","last_active":"2024-09-21T00:28:56.117170+0000","last_peered":"2024-09-21T00:28:56.117170+0000","last_clean":"2024-09-21T00:28:56.117170+0000","last_became_active":"2024-09-21T00:28:54.097610+0000","last_became_peered":"2024-09-21T00:28:54.097610+0000","last_unstale":"2024-09-21T00:28:56.117170+0000","last_undegraded":"2024-09-21T00:28:56.117170+0000","last_fullsized":"2024-09-21T00:28:56.117170+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:53.070133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:53.070133+0000","last_clean_scrub_stamp":"2024-09-21T00:28:53.070133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:46:44.835922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.117223+0000","last_change":"2024-09-21T00:28:55.106651+0000","last_active":"2024-09-21T00:28:56.117223+0000","last_peered":"2024-09-21T00:28:56.117223+0000","last_clean":"2024-09-21T00:28:56.117223+0000","last_became_active":"2024-09-21T00:28:55.106188+0000","last_became_peered":"2024-09-21T00:28:55.106188+0000","last_unstale":"2024-09-21T00:28:56.117223+0000","last_undegraded":"2024-09-21T00:28:56.117223+0000","last_fullsized":"2024-09-21T00:28:56.117223+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_clean_scrub_stamp":"2024-09-21T00:28:54.073989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:22:52.766853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.118703+0000","last_change":"2024-09-21T00:28:54.091919+0000","last_active":"2024-09-21T00:28:56.118703+0000","last_peered":"2024-09-21T00:28:56.118703+0000","last_clean":"2024-09-21T00:28:56.118703+0000","last_became_active":"2024-09-21T00:28:54.091604+0000","last_became_peered":"2024-09-21T00:28:54.091604+0000","last_unstale":"2024-09-21T00:28:56.118703+0000","last_undegraded":"2024-09-21T00:28:56.118703+0000","last_fullsized":"2024-09-21T00:28:56.118703+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:53.070133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:53.070133+0000","last_clean_scrub_stamp":"2024-09-21T00:28:53.070133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:53:49.831609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.117174+0000","last_change":"2024-09-21T00:28:55.105495+0000","last_active":"2024-09-21T00:28:56.117174+0000","last_peered":"2024-09-21T00:28:56.117174+0000","last_clean":"2024-09-21T00:28:56.117174+0000","last_became_active":"2024-09-21T00:28:55.105157+0000","last_became_peered":"2024-09-21T00:28:55.105157+0000","last_unstale":"2024-09-21T00:28:56.117174+0000","last_undegraded":"2024-09-21T00:28:56.117174+0000","last_fullsized":"2024-09-21T00:28:56.117174+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_clean_scrub_stamp":"2024-09-21T00:28:54.073989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:44:22.737668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"82'8","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.118854+0000","last_change":"2024-09-21T00:28:54.098865+0000","last_active":"2024-09-21T00:28:56.118854+0000","last_peered":"2024-09-21T00:28:56.118854+0000","last_clean":"2024-09-21T00:28:56.118854+0000","last_became_active":"2024-09-21T00:28:54.098566+0000","last_became_peered":"2024-09-21T00:28:54.098566+0000","last_unstale":"2024-09-21T00:28:56.118854+0000","last_undegraded":"2024-09-21T00:28:56.118854+0000","last_fullsized":"2024-09-21T00:28:56.118854+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:53.070133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:53.070133+0000","last_clean_scrub_stamp":"2024-09-21T00:28:53.070133+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:54:55.632581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.117511+0000","last_change":"2024-09-21T00:28:55.100643+0000","last_active":"2024-09-21T00:28:56.117511+0000","last_peered":"2024-09-21T00:28:56.117511+0000","last_clean":"2024-09-21T00:28:56.117511+0000","last_became_active":"2024-09-21T00:28:55.099768+0000","last_became_peered":"2024-09-21T00:28:55.099768+0000","last_unstale":"2024-09-21T00:28:56.117511+0000","last_undegraded":"2024-09-21T00:28:56.117511+0000","last_fullsized":"2024-09-21T00:28:56.117511+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_clean_scrub_stamp":"2024-09-21T00:28:54.073989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:01:50.169364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"82'1","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.117647+0000","last_change":"2024-09-21T00:28:54.096470+0000","last_active":"2024-09-21T00:28:56.117647+0000","last_peered":"2024-09-21T00:28:56.117647+0000","last_clean":"2024-09-21T00:28:56.117647+0000","last_became_active":"2024-09-21T00:28:54.096025+0000","last_became_peered":"2024-09-21T00:28:54.096025+0000","last_unstale":"2024-09-21T00:28:56.117647+0000","last_undegraded":"2024-09-21T00:28:56.117647+0000","last_fullsized":"2024-09-21T00:28:56.117647+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:53.070133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:53.070133+0000","last_clean_scrub_stamp":"2024-09-21T00:28:53.070133+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:18:56.230631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.117432+0000","last_change":"2024-09-21T00:28:55.101246+0000","last_active":"2024-09-21T00:28:56.117432+0000","last_peered":"2024-09-21T00:28:56.117432+0000","last_clean":"2024-09-21T00:28:56.117432+0000","last_became_active":"2024-09-21T00:28:55.100937+0000","last_became_peered":"2024-09-21T00:28:55.100937+0000","last_unstale":"2024-09-21T00:28:56.117432+0000","last_undegraded":"2024-09-21T00:28:56.117432+0000","last_fullsized":"2024-09-21T00:28:56.117432+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_clean_scrub_stamp":"2024-09-21T00:28:54.073989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:28:58.329009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"82'2","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.103322+0000","last_change":"2024-09-21T00:28:54.098747+0000","last_active":"2024-09-21T00:28:56.103322+0000","last_peered":"2024-09-21T00:28:56.103322+0000","last_clean":"2024-09-21T00:28:56.103322+0000","last_became_active":"2024-09-21T00:28:54.098387+0000","last_became_peered":"2024-09-21T00:28:54.098387+0000","last_unstale":"2024-09-21T00:28:56.103322+0000","last_undegraded":"2024-09-21T00:28:56.103322+0000","last_fullsized":"2024-09-21T00:28:56.103322+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:53.070133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:53.070133+0000","last_clean_scrub_stamp":"2024-09-21T00:28:53.070133+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:31:01.601795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.102143+0000","last_change":"2024-09-21T00:28:55.097662+0000","last_active":"2024-09-21T00:28:56.102143+0000","last_peered":"2024-09-21T00:28:56.102143+0000","last_clean":"2024-09-21T00:28:56.102143+0000","last_became_active":"2024-09-21T00:28:55.097364+0000","last_became_peered":"2024-09-21T00:28:55.097364+0000","last_unstale":"2024-09-21T00:28:56.102143+0000","last_undegraded":"2024-09-21T00:28:56.102143+0000","last_fullsized":"2024-09-21T00:28:56.102143+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_clean_scrub_stamp":"2024-09-21T00:28:54.073989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:14:14.562450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"82'1","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.102426+0000","last_change":"2024-09-21T00:28:54.092612+0000","last_active":"2024-09-21T00:28:56.102426+0000","last_peered":"2024-09-21T00:28:56.102426+0000","last_clean":"2024-09-21T00:28:56.102426+0000","last_became_active":"2024-09-21T00:28:54.092276+0000","last_became_peered":"2024-09-21T00:28:54.092276+0000","last_unstale":"2024-09-21T00:28:56.102426+0000","last_undegraded":"2024-09-21T00:28:56.102426+0000","last_fullsized":"2024-09-21T00:28:56.102426+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:53.070133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:53.070133+0000","last_clean_scrub_stamp":"2024-09-21T00:28:53.070133+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:22:36.363798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.117050+0000","last_change":"2024-09-21T00:28:55.107909+0000","last_active":"2024-09-21T00:28:56.117050+0000","last_peered":"2024-09-21T00:28:56.117050+0000","last_clean":"2024-09-21T00:28:56.117050+0000","last_became_active":"2024-09-21T00:28:55.107607+0000","last_became_peered":"2024-09-21T00:28:55.107607+0000","last_unstale":"2024-09-21T00:28:56.117050+0000","last_undegraded":"2024-09-21T00:28:56.117050+0000","last_fullsized":"2024-09-21T00:28:56.117050+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_clean_scrub_stamp":"2024-09-21T00:28:54.073989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:34:52.284071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"82'2","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.117330+0000","last_change":"2024-09-21T00:28:54.087322+0000","last_active":"2024-09-21T00:28:56.117330+0000","last_peered":"2024-09-21T00:28:56.117330+0000","last_clean":"2024-09-21T00:28:56.117330+0000","last_became_active":"2024-09-21T00:28:54.086695+0000","last_became_peered":"2024-09-21T00:28:54.086695+0000","last_unstale":"2024-09-21T00:28:56.117330+0000","last_undegraded":"2024-09-21T00:28:56.117330+0000","last_fullsized":"2024-09-21T00:28:56.117330+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:53.070133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:53.070133+0000","last_clean_scrub_stamp":"2024-09-21T00:28:53.070133+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:11:39.474869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.117113+0000","last_change":"2024-09-21T00:28:55.103948+0000","last_active":"2024-09-21T00:28:56.117113+0000","last_peered":"2024-09-21T00:28:56.117113+0000","last_clean":"2024-09-21T00:28:56.117113+0000","last_became_active":"2024-09-21T00:28:55.103270+0000","last_became_peered":"2024-09-21T00:28:55.103270+0000","last_unstale":"2024-09-21T00:28:56.117113+0000","last_undegraded":"2024-09-21T00:28:56.117113+0000","last_fullsized":"2024-09-21T00:28:56.117113+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_clean_scrub_stamp":"2024-09-21T00:28:54.073989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:02:40.895903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.117437+0000","last_change":"2024-09-21T00:28:54.102703+0000","last_active":"2024-09-21T00:28:56.117437+0000","last_peered":"2024-09-21T00:28:56.117437+0000","last_clean":"2024-09-21T00:28:56.117437+0000","last_became_active":"2024-09-21T00:28:54.102324+0000","last_became_peered":"2024-09-21T00:28:54.102324+0000","last_unstale":"2024-09-21T00:28:56.117437+0000","last_undegraded":"2024-09-21T00:28:56.117437+0000","last_fullsized":"2024-09-21T00:28:56.117437+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:53.070133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:53.070133+0000","last_clean_scrub_stamp":"2024-09-21T00:28:53.070133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:26:58.990605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.117611+0000","last_change":"2024-09-21T00:28:55.108422+0000","last_active":"2024-09-21T00:28:56.117611+0000","last_peered":"2024-09-21T00:28:56.117611+0000","last_clean":"2024-09-21T00:28:56.117611+0000","last_became_active":"2024-09-21T00:28:55.108083+0000","last_became_peered":"2024-09-21T00:28:55.108083+0000","last_unstale":"2024-09-21T00:28:56.117611+0000","last_undegraded":"2024-09-21T00:28:56.117611+0000","last_fullsized":"2024-09-21T00:28:56.117611+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_clean_scrub_stamp":"2024-09-21T00:28:54.073989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:08:31.986233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.117254+0000","last_change":"2024-09-21T00:28:54.091594+0000","last_active":"2024-09-21T00:28:56.117254+0000","last_peered":"2024-09-21T00:28:56.117254+0000","last_clean":"2024-09-21T00:28:56.117254+0000","last_became_active":"2024-09-21T00:28:54.091281+0000","last_became_peered":"2024-09-21T00:28:54.091281+0000","last_unstale":"2024-09-21T00:28:56.117254+0000","last_undegraded":"2024-09-21T00:28:56.117254+0000","last_fullsized":"2024-09-21T00:28:56.117254+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:53.070133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:53.070133+0000","last_clean_scrub_stamp":"2024-09-21T00:28:53.070133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:07:02.063603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.117438+0000","last_change":"2024-09-21T00:28:55.096461+0000","last_active":"2024-09-21T00:28:56.117438+0000","last_peered":"2024-09-21T00:28:56.117438+0000","last_clean":"2024-09-21T00:28:56.117438+0000","last_became_active":"2024-09-21T00:28:55.096141+0000","last_became_peered":"2024-09-21T00:28:55.096141+0000","last_unstale":"2024-09-21T00:28:56.117438+0000","last_undegraded":"2024-09-21T00:28:56.117438+0000","last_fullsized":"2024-09-21T00:28:56.117438+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_clean_scrub_stamp":"2024-09-21T00:28:54.073989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:11:13.390972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.116888+0000","last_change":"2024-09-21T00:28:54.104692+0000","last_active":"2024-09-21T00:28:56.116888+0000","last_peered":"2024-09-21T00:28:56.116888+0000","last_clean":"2024-09-21T00:28:56.116888+0000","last_became_active":"2024-09-21T00:28:54.104402+0000","last_became_peered":"2024-09-21T00:28:54.104402+0000","last_unstale":"2024-09-21T00:28:56.116888+0000","last_undegraded":"2024-09-21T00:28:56.116888+0000","last_fullsized":"2024-09-21T00:28:56.116888+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:53.070133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:53.070133+0000","last_clean_scrub_stamp":"2024-09-21T00:28:53.070133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:37:49.815524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.117536+0000","last_change":"2024-09-21T00:28:55.111820+0000","last_active":"2024-09-21T00:28:56.117536+0000","last_peered":"2024-09-21T00:28:56.117536+0000","last_clean":"2024-09-21T00:28:56.117536+0000","last_became_active":"2024-09-21T00:28:55.110917+0000","last_became_peered":"2024-09-21T00:28:55.110917+0000","last_unstale":"2024-09-21T00:28:56.117536+0000","last_undegraded":"2024-09-21T00:28:56.117536+0000","last_fullsized":"2024-09-21T00:28:56.117536+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_clean_scrub_stamp":"2024-09-21T00:28:54.073989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:17:29.709648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"82'2","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.103597+0000","last_change":"2024-09-21T00:28:54.097697+0000","last_active":"2024-09-21T00:28:56.103597+0000","last_peered":"2024-09-21T00:28:56.103597+0000","last_clean":"2024-09-21T00:28:56.103597+0000","last_became_active":"2024-09-21T00:28:54.097295+0000","last_became_peered":"2024-09-21T00:28:54.097295+0000","last_unstale":"2024-09-21T00:28:56.103597+0000","last_undegraded":"2024-09-21T00:28:56.103597+0000","last_fullsized":"2024-09-21T00:28:56.103597+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:53.070133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:53.070133+0000","last_clean_scrub_stamp":"2024-09-21T00:28:53.070133+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:08:30.982377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.118267+0000","last_change":"2024-09-21T00:28:55.100919+0000","last_active":"2024-09-21T00:28:56.118267+0000","last_peered":"2024-09-21T00:28:56.118267+0000","last_clean":"2024-09-21T00:28:56.118267+0000","last_became_active":"2024-09-21T00:28:55.100512+0000","last_became_peered":"2024-09-21T00:28:55.100512+0000","last_unstale":"2024-09-21T00:28:56.118267+0000","last_undegraded":"2024-09-21T00:28:56.118267+0000","last_fullsized":"2024-09-21T00:28:56.118267+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_clean_scrub_stamp":"2024-09-21T00:28:54.073989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:20:28.590984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.102308+0000","last_change":"2024-09-21T00:28:54.092917+0000","last_active":"2024-09-21T00:28:56.102308+0000","last_peered":"2024-09-21T00:28:56.102308+0000","last_clean":"2024-09-21T00:28:56.102308+0000","last_became_active":"2024-09-21T00:28:54.091521+0000","last_became_peered":"2024-09-21T00:28:54.091521+0000","last_unstale":"2024-09-21T00:28:56.102308+0000","last_undegraded":"2024-09-21T00:28:56.102308+0000","last_fullsized":"2024-09-21T00:28:56.102308+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:53.070133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:53.070133+0000","last_clean_scrub_stamp":"2024-09-21T00:28:53.070133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:58:21.410692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.117845+0000","last_change":"2024-09-21T00:28:55.097489+0000","last_active":"2024-09-21T00:28:56.117845+0000","last_peered":"2024-09-21T00:28:56.117845+0000","last_clean":"2024-09-21T00:28:56.117845+0000","last_became_active":"2024-09-21T00:28:55.097153+0000","last_became_peered":"2024-09-21T00:28:55.097153+0000","last_unstale":"2024-09-21T00:28:56.117845+0000","last_undegraded":"2024-09-21T00:28:56.117845+0000","last_fullsized":"2024-09-21T00:28:56.117845+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_clean_scrub_stamp":"2024-09-21T00:28:54.073989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:34:23.039928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.102498+0000","last_change":"2024-09-21T00:28:54.099349+0000","last_active":"2024-09-21T00:28:56.102498+0000","last_peered":"2024-09-21T00:28:56.102498+0000","last_clean":"2024-09-21T00:28:56.102498+0000","last_became_active":"2024-09-21T00:28:54.098827+0000","last_became_peered":"2024-09-21T00:28:54.098827+0000","last_unstale":"2024-09-21T00:28:56.102498+0000","last_undegraded":"2024-09-21T00:28:56.102498+0000","last_fullsized":"2024-09-21T00:28:56.102498+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:53.070133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:53.070133+0000","last_clean_scrub_stamp":"2024-09-21T00:28:53.070133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:34:10.674315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.117992+0000","last_change":"2024-09-21T00:28:55.098865+0000","last_active":"2024-09-21T00:28:56.117992+0000","last_peered":"2024-09-21T00:28:56.117992+0000","last_clean":"2024-09-21T00:28:56.117992+0000","last_became_active":"2024-09-21T00:28:55.098542+0000","last_became_peered":"2024-09-21T00:28:55.098542+0000","last_unstale":"2024-09-21T00:28:56.117992+0000","last_undegraded":"2024-09-21T00:28:56.117992+0000","last_fullsized":"2024-09-21T00:28:56.117992+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_clean_scrub_stamp":"2024-09-21T00:28:54.073989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:36:38.879430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.117678+0000","last_change":"2024-09-21T00:28:54.100179+0000","last_active":"2024-09-21T00:28:56.117678+0000","last_peered":"2024-09-21T00:28:56.117678+0000","last_clean":"2024-09-21T00:28:56.117678+0000","last_became_active":"2024-09-21T00:28:54.099825+0000","last_became_peered":"2024-09-21T00:28:54.099825+0000","last_unstale":"2024-09-21T00:28:56.117678+0000","last_undegraded":"2024-09-21T00:28:56.117678+0000","last_fullsized":"2024-09-21T00:28:56.117678+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:53.070133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:53.070133+0000","last_clean_scrub_stamp":"2024-09-21T00:28:53.070133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:41:24.183356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.116768+0000","last_change":"2024-09-21T00:28:55.110728+0000","last_active":"2024-09-21T00:28:56.116768+0000","last_peered":"2024-09-21T00:28:56.116768+0000","last_clean":"2024-09-21T00:28:56.116768+0000","last_became_active":"2024-09-21T00:28:55.110402+0000","last_became_peered":"2024-09-21T00:28:55.110402+0000","last_unstale":"2024-09-21T00:28:56.116768+0000","last_undegraded":"2024-09-21T00:28:56.116768+0000","last_fullsized":"2024-09-21T00:28:56.116768+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_clean_scrub_stamp":"2024-09-21T00:28:54.073989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T00:54:15.553165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.117553+0000","last_change":"2024-09-21T00:28:54.098315+0000","last_active":"2024-09-21T00:28:56.117553+0000","last_peered":"2024-09-21T00:28:56.117553+0000","last_clean":"2024-09-21T00:28:56.117553+0000","last_became_active":"2024-09-21T00:28:54.098019+0000","last_became_peered":"2024-09-21T00:28:54.098019+0000","last_unstale":"2024-09-21T00:28:56.117553+0000","last_undegraded":"2024-09-21T00:28:56.117553+0000","last_fullsized":"2024-09-21T00:28:56.117553+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:53.070133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:53.070133+0000","last_clean_scrub_stamp":"2024-09-21T00:28:53.070133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:20:30.596891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.100181+0000","last_change":"2024-09-21T00:28:55.103561+0000","last_active":"2024-09-21T00:28:56.100181+0000","last_peered":"2024-09-21T00:28:56.100181+0000","last_clean":"2024-09-21T00:28:56.100181+0000","last_became_active":"2024-09-21T00:28:55.102654+0000","last_became_peered":"2024-09-21T00:28:55.102654+0000","last_unstale":"2024-09-21T00:28:56.100181+0000","last_undegraded":"2024-09-21T00:28:56.100181+0000","last_fullsized":"2024-09-21T00:28:56.100181+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_clean_scrub_stamp":"2024-09-21T00:28:54.073989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:45:07.464813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.117472+0000","last_change":"2024-09-21T00:28:54.086754+0000","last_active":"2024-09-21T00:28:56.117472+0000","last_peered":"2024-09-21T00:28:56.117472+0000","last_clean":"2024-09-21T00:28:56.117472+0000","last_became_active":"2024-09-21T00:28:54.086402+0000","last_became_peered":"2024-09-21T00:28:54.086402+0000","last_unstale":"2024-09-21T00:28:56.117472+0000","last_undegraded":"2024-09-21T00:28:56.117472+0000","last_fullsized":"2024-09-21T00:28:56.117472+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:53.070133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:53.070133+0000","last_clean_scrub_stamp":"2024-09-21T00:28:53.070133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:09:39.844275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.118130+0000","last_change":"2024-09-21T00:28:55.108305+0000","last_active":"2024-09-21T00:28:56.118130+0000","last_peered":"2024-09-21T00:28:56.118130+0000","last_clean":"2024-09-21T00:28:56.118130+0000","last_became_active":"2024-09-21T00:28:55.107988+0000","last_became_peered":"2024-09-21T00:28:55.107988+0000","last_unstale":"2024-09-21T00:28:56.118130+0000","last_undegraded":"2024-09-21T00:28:56.118130+0000","last_fullsized":"2024-09-21T00:28:56.118130+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_clean_scrub_stamp":"2024-09-21T00:28:54.073989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:11:21.675017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.118008+0000","last_change":"2024-09-21T00:28:54.100020+0000","last_active":"2024-09-21T00:28:56.118008+0000","last_peered":"2024-09-21T00:28:56.118008+0000","last_clean":"2024-09-21T00:28:56.118008+0000","last_became_active":"2024-09-21T00:28:54.099695+0000","last_became_peered":"2024-09-21T00:28:54.099695+0000","last_unstale":"2024-09-21T00:28:56.118008+0000","last_undegraded":"2024-09-21T00:28:56.118008+0000","last_fullsized":"2024-09-21T00:28:56.118008+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:53.070133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:53.070133+0000","last_clean_scrub_stamp":"2024-09-21T00:28:53.070133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:33:24.983637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.117677+0000","last_change":"2024-09-21T00:28:55.098909+0000","last_active":"2024-09-21T00:28:56.117677+0000","last_peered":"2024-09-21T00:28:56.117677+0000","last_clean":"2024-09-21T00:28:56.117677+0000","last_became_active":"2024-09-21T00:28:55.098525+0000","last_became_peered":"2024-09-21T00:28:55.098525+0000","last_unstale":"2024-09-21T00:28:56.117677+0000","last_undegraded":"2024-09-21T00:28:56.117677+0000","last_fullsized":"2024-09-21T00:28:56.117677+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_clean_scrub_stamp":"2024-09-21T00:28:54.073989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:19:07.363130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.103244+0000","last_change":"2024-09-21T00:28:54.102564+0000","last_active":"2024-09-21T00:28:56.103244+0000","last_peered":"2024-09-21T00:28:56.103244+0000","last_clean":"2024-09-21T00:28:56.103244+0000","last_became_active":"2024-09-21T00:28:54.102263+0000","last_became_peered":"2024-09-21T00:28:54.102263+0000","last_unstale":"2024-09-21T00:28:56.103244+0000","last_undegraded":"2024-09-21T00:28:56.103244+0000","last_fullsized":"2024-09-21T00:28:56.103244+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:53.070133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:53.070133+0000","last_clean_scrub_stamp":"2024-09-21T00:28:53.070133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:44:51.937525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.103416+0000","last_change":"2024-09-21T00:28:55.106196+0000","last_active":"2024-09-21T00:28:56.103416+0000","last_peered":"2024-09-21T00:28:56.103416+0000","last_clean":"2024-09-21T00:28:56.103416+0000","last_became_active":"2024-09-21T00:28:55.105879+0000","last_became_peered":"2024-09-21T00:28:55.105879+0000","last_unstale":"2024-09-21T00:28:56.103416+0000","last_undegraded":"2024-09-21T00:28:56.103416+0000","last_fullsized":"2024-09-21T00:28:56.103416+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:54.073989+0000","last_clean_scrub_stamp":"2024-09-21T00:28:54.073989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:25:50.171074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.118812+0000","last_change":"2024-09-21T00:28:54.089350+0000","last_active":"2024-09-21T00:28:56.118812+0000","last_peered":"2024-09-21T00:28:56.118812+0000","last_clean":"2024-09-21T00:28:56.118812+0000","last_became_active":"2024-09-21T00:28:54.088494+0000","last_became_peered":"2024-09-21T00:28:54.088494+0000","last_unstale":"2024-09-21T00:28:56.118812+0000","last_undegraded":"2024-09-21T00:28:56.118812+0000","last_fullsized":"2024-09-21T00:28:56.118812+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:28:53.070133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:28:53.070133+0000","last_clean_scrub_stamp":"2024-09-21T00:28:53.070133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:49:40.065242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":223,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T00:28:56.103093+0000","last_change":"2024-09-21T00:25:38.725792+0000","last_active":"2024-09-21T00:28:56.103093+0000","last_peered":"2024-09-21T00:28:56.103093+0000","last_clean":"2024-09-21T00:28:56.103093+0000","last_became_active":"2024-09-21T00:25:38.724668+0000","last_became_peered":"2024-09-21T00:25:38.724668+0000","last_unstale":"2024-09-21T00:28:56.103093+0000","last_undegraded":"2024-09-21T00:28:56.103093+0000","last_fullsized":"2024-09-21T00:28:56.103093+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:37.163216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:37.163216+0000","last_clean_scrub_stamp":"2024-09-21T00:25:37.163216+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:14:36.588660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509483,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32768,"kb_used_data":1580,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339072,"statfs":{"total":96636764160,"available":96603209728,"internally_reserved":0,"allocated":1617920,"data_stored":1048558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509484,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32196,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339644,"statfs":{"total":96636764160,"available":96603795456,"internally_reserved":0,"allocated":1024000,"data_stored":453658,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509483,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32176,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339664,"statfs":{"total":96636764160,"available":96603815936,"internally_reserved":0,"allocated":1003520,"data_stored":451442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509482,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32188,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339652,"statfs":{"total":96636764160,"available":96603803648,"internally_reserved":0,"allocated":1024000,"data_stored":458294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509484,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32176,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339664,"statfs":{"total":96636764160,"available":96603815936,"internally_reserved":0,"allocated":1003520,"data_stored":451442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509484,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28012,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343828,"statfs":{"total":96636764160,"available":96608079872,"internally_reserved":0,"allocated":1003520,"data_stored":451442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509484,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32780,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339060,"statfs":{"total":96636764160,"available":96603197440,"internally_reserved":0,"allocated":1622016,"data_stored":1044128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574892,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32188,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339652,"statfs":{"total":96636764160,"available":96603803648,"internally_reserved":0,"allocated":1019904,"data_stored":458204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T00:28:59.446 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T00:28:59.446 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-21T00:28:59.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.602+0000 7fb457bc6640 1 -- 172.21.15.112:0/2043784180 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb4500ccbf0 msgr2=0x7fb4500cd000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:59.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.602+0000 7fb457bc6640 1 --2- 172.21.15.112:0/2043784180 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb4500ccbf0 0x7fb4500cd000 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fb44c009830 tx=0x7fb44c02a4a0 comp rx=0 tx=0).stop 2024-09-21T00:28:59.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.602+0000 7fb457bc6640 1 -- 172.21.15.112:0/2043784180 shutdown_connections 2024-09-21T00:28:59.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.602+0000 7fb457bc6640 1 --2- 172.21.15.112:0/2043784180 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb4500ccbf0 0x7fb4500cd000 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:59.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.602+0000 7fb457bc6640 1 -- 172.21.15.112:0/2043784180 >> 172.21.15.112:0/2043784180 conn(0x7fb4500b42e0 msgr2=0x7fb4500b4720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:59.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.602+0000 7fb457bc6640 1 -- 172.21.15.112:0/2043784180 shutdown_connections 2024-09-21T00:28:59.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.602+0000 7fb457bc6640 1 -- 172.21.15.112:0/2043784180 wait complete. 2024-09-21T00:28:59.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.604+0000 7fb457bc6640 1 Processor -- start 2024-09-21T00:28:59.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.604+0000 7fb457bc6640 1 -- start start 2024-09-21T00:28:59.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.605+0000 7fb457bc6640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb4500c5ee0 0x7fb4500b0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:59.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.605+0000 7fb457bc6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb4500c6a00 0x7fb4500b6990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:59.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.605+0000 7fb457bc6640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb4500ccbf0 0x7fb4501f95a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:59.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.605+0000 7fb457bc6640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb4500c6f60 con 0x7fb4500c6a00 2024-09-21T00:28:59.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.605+0000 7fb457bc6640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb4500b3660 con 0x7fb4500ccbf0 2024-09-21T00:28:59.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.605+0000 7fb457bc6640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fb450167ed0 con 0x7fb4500c5ee0 2024-09-21T00:28:59.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.605+0000 7fb455161640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb4500c6a00 0x7fb4500b6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:59.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.605+0000 7fb455161640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb4500c6a00 0x7fb4500b6990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38182/0 (socket says 172.21.15.112:38182) 2024-09-21T00:28:59.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.605+0000 7fb455161640 1 -- 172.21.15.112:0/3121983883 learned_addr learned my addr 172.21.15.112:0/3121983883 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:28:59.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.605+0000 7fb456163640 1 --2- 172.21.15.112:0/3121983883 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb4500ccbf0 0x7fb4501f95a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:59.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.605+0000 7fb455962640 1 --2- 172.21.15.112:0/3121983883 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb4500c5ee0 0x7fb4500b0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:59.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.606+0000 7fb455161640 1 -- 172.21.15.112:0/3121983883 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb4500c5ee0 msgr2=0x7fb4500b0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:59.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.606+0000 7fb455161640 1 --2- 172.21.15.112:0/3121983883 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb4500c5ee0 0x7fb4500b0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:59.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.606+0000 7fb455161640 1 -- 172.21.15.112:0/3121983883 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb4500ccbf0 msgr2=0x7fb4501f95a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:59.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.606+0000 7fb455161640 1 --2- 172.21.15.112:0/3121983883 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb4500ccbf0 0x7fb4501f95a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:59.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.606+0000 7fb455161640 1 -- 172.21.15.112:0/3121983883 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4500b6ed0 con 0x7fb4500c6a00 2024-09-21T00:28:59.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.606+0000 7fb455161640 1 --2- 172.21.15.112:0/3121983883 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb4500c6a00 0x7fb4500b6990 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fb440001eb0 tx=0x7fb44000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:59.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.606+0000 7fb4467fc640 1 -- 172.21.15.112:0/3121983883 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb440017070 con 0x7fb4500c6a00 2024-09-21T00:28:59.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.606+0000 7fb4467fc640 1 -- 172.21.15.112:0/3121983883 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb4400053c0 con 0x7fb4500c6a00 2024-09-21T00:28:59.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.606+0000 7fb457bc6640 1 -- 172.21.15.112:0/3121983883 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4501f9ae0 con 0x7fb4500c6a00 2024-09-21T00:28:59.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.606+0000 7fb4467fc640 1 -- 172.21.15.112:0/3121983883 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb440003830 con 0x7fb4500c6a00 2024-09-21T00:28:59.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.606+0000 7fb457bc6640 1 -- 172.21.15.112:0/3121983883 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb450070040 con 0x7fb4500c6a00 2024-09-21T00:28:59.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.607+0000 7fb457bc6640 1 -- 172.21.15.112:0/3121983883 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4500cd000 con 0x7fb4500c6a00 2024-09-21T00:28:59.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.608+0000 7fb4467fc640 1 -- 172.21.15.112:0/3121983883 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb44001a070 con 0x7fb4500c6a00 2024-09-21T00:28:59.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.608+0000 7fb4467fc640 1 --2- 172.21.15.112:0/3121983883 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fb434078b90 0x7fb43407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:59.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.608+0000 7fb4467fc640 1 -- 172.21.15.112:0/3121983883 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 6101+0+0 (secure 0 0 0) 0x7fb44009a1a0 con 0x7fb4500c6a00 2024-09-21T00:28:59.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.609+0000 7fb455962640 1 --2- 172.21.15.112:0/3121983883 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fb434078b90 0x7fb43407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:59.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.609+0000 7fb455962640 1 --2- 172.21.15.112:0/3121983883 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fb434078b90 0x7fb43407b0a0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fb44c034000 tx=0x7fb44c004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:59.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.613+0000 7fb4467fc640 1 -- 172.21.15.112:0/3121983883 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb440061e90 con 0x7fb4500c6a00 2024-09-21T00:28:59.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.860+0000 7fb457bc6640 1 -- 172.21.15.112:0/3121983883 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fb4500b2f50 con 0x7fb4500c6a00 2024-09-21T00:28:59.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.863+0000 7fb4467fc640 1 -- 172.21.15.112:0/3121983883 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v19) ==== 190+0+147 (secure 0 0 0) 0x7fb4400668c0 con 0x7fb4500c6a00 2024-09-21T00:28:59.864 INFO:teuthology.orchestra.run.smithi112.stdout:[client.0] 2024-09-21T00:28:59.864 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQBPEu5m49NPHBAAl/rcjiE8xYI+XdWwvZ6FOw== 2024-09-21T00:28:59.864 INFO:teuthology.orchestra.run.smithi112.stdout: caps mds = "allow" 2024-09-21T00:28:59.864 INFO:teuthology.orchestra.run.smithi112.stdout: caps mon = "allow r" 2024-09-21T00:28:59.864 INFO:teuthology.orchestra.run.smithi112.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-21T00:28:59.864 INFO:teuthology.orchestra.run.smithi112.stderr:updated caps for client.0 2024-09-21T00:28:59.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.865+0000 7fb457bc6640 1 -- 172.21.15.112:0/3121983883 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fb434078b90 msgr2=0x7fb43407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:59.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.866+0000 7fb457bc6640 1 --2- 172.21.15.112:0/3121983883 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fb434078b90 0x7fb43407b0a0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fb44c034000 tx=0x7fb44c004730 comp rx=0 tx=0).stop 2024-09-21T00:28:59.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.866+0000 7fb457bc6640 1 -- 172.21.15.112:0/3121983883 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb4500c6a00 msgr2=0x7fb4500b6990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:59.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.866+0000 7fb457bc6640 1 --2- 172.21.15.112:0/3121983883 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb4500c6a00 0x7fb4500b6990 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fb440001eb0 tx=0x7fb44000c890 comp rx=0 tx=0).stop 2024-09-21T00:28:59.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.866+0000 7fb457bc6640 1 -- 172.21.15.112:0/3121983883 shutdown_connections 2024-09-21T00:28:59.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.866+0000 7fb457bc6640 1 --2- 172.21.15.112:0/3121983883 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb4500ccbf0 0x7fb4501f95a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:59.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.866+0000 7fb457bc6640 1 --2- 172.21.15.112:0/3121983883 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fb434078b90 0x7fb43407b0a0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:59.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.866+0000 7fb457bc6640 1 --2- 172.21.15.112:0/3121983883 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb4500c6a00 0x7fb4500b6990 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:59.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.866+0000 7fb457bc6640 1 --2- 172.21.15.112:0/3121983883 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb4500c5ee0 0x7fb4500b0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:59.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.866+0000 7fb457bc6640 1 -- 172.21.15.112:0/3121983883 >> 172.21.15.112:0/3121983883 conn(0x7fb4500b42e0 msgr2=0x7fb4500c7f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:59.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.866+0000 7fb457bc6640 1 -- 172.21.15.112:0/3121983883 shutdown_connections 2024-09-21T00:28:59.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.866+0000 7fb457bc6640 1 -- 172.21.15.112:0/3121983883 wait complete. 2024-09-21T00:28:59.878 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T00:28:59.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.989+0000 7f8016139640 1 -- 172.21.15.112:0/3837340820 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80100d0860 msgr2=0x7f801015a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:59.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.989+0000 7f8016139640 1 --2- 172.21.15.112:0/3837340820 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80100d0860 0x7f801015a0c0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f80100b4f40 tx=0x7f800402a530 comp rx=0 tx=0).stop 2024-09-21T00:28:59.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.989+0000 7f8016139640 1 -- 172.21.15.112:0/3837340820 shutdown_connections 2024-09-21T00:28:59.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.989+0000 7f8016139640 1 --2- 172.21.15.112:0/3837340820 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80100d0860 0x7f801015a0c0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:59.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.989+0000 7f8016139640 1 --2- 172.21.15.112:0/3837340820 >> v2:172.21.15.187:3300/0 conn(0x7f80100d2030 0x7f801015d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:59.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.989+0000 7f8016139640 1 -- 172.21.15.112:0/3837340820 >> 172.21.15.112:0/3837340820 conn(0x7f80100b43f0 msgr2=0x7f80100b4830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:28:59.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.989+0000 7f8016139640 1 -- 172.21.15.112:0/3837340820 shutdown_connections 2024-09-21T00:28:59.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.989+0000 7f8016139640 1 -- 172.21.15.112:0/3837340820 wait complete. 2024-09-21T00:28:59.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.991+0000 7f8016139640 1 Processor -- start 2024-09-21T00:28:59.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.991+0000 7f8016139640 1 -- start start 2024-09-21T00:28:59.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.992+0000 7f8016139640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80100ccae0 0x7f8010159290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:59.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.992+0000 7f8016139640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f80100d0860 0x7f80100aac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:59.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.992+0000 7f8016139640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f80100d2030 0x7f80100ca890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:59.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.992+0000 7f8016139640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f80100b22d0 con 0x7f80100ccae0 2024-09-21T00:28:59.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.992+0000 7f8016139640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f80100b2150 con 0x7f80100d2030 2024-09-21T00:28:59.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.992+0000 7f8016139640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f80100b2450 con 0x7f80100d0860 2024-09-21T00:28:59.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.992+0000 7f800f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80100ccae0 0x7f8010159290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:59.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.992+0000 7f800f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80100ccae0 0x7f8010159290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38196/0 (socket says 172.21.15.112:38196) 2024-09-21T00:28:59.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.992+0000 7f800f7fe640 1 -- 172.21.15.112:0/3534396710 learned_addr learned my addr 172.21.15.112:0/3534396710 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:28:59.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.992+0000 7f800effd640 1 --2- 172.21.15.112:0/3534396710 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f80100d0860 0x7f80100aac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:59.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.992+0000 7f800ffff640 1 --2- 172.21.15.112:0/3534396710 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f80100d2030 0x7f80100ca890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:59.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.993+0000 7f800f7fe640 1 -- 172.21.15.112:0/3534396710 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f80100d0860 msgr2=0x7f80100aac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:59.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.993+0000 7f800f7fe640 1 --2- 172.21.15.112:0/3534396710 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f80100d0860 0x7f80100aac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:59.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.993+0000 7f800f7fe640 1 -- 172.21.15.112:0/3534396710 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f80100d2030 msgr2=0x7f80100ca890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:28:59.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.993+0000 7f800f7fe640 1 --2- 172.21.15.112:0/3534396710 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f80100d2030 0x7f80100ca890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:28:59.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.993+0000 7f800f7fe640 1 -- 172.21.15.112:0/3534396710 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80101f8ee0 con 0x7f80100ccae0 2024-09-21T00:28:59.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.993+0000 7f800f7fe640 1 --2- 172.21.15.112:0/3534396710 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80100ccae0 0x7f8010159290 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f800000d740 tx=0x7f800000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:28:59.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.993+0000 7f7ff7fff640 1 -- 172.21.15.112:0/3534396710 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8000017070 con 0x7f80100ccae0 2024-09-21T00:28:59.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.993+0000 7f7ff7fff640 1 -- 172.21.15.112:0/3534396710 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8000009d20 con 0x7f80100ccae0 2024-09-21T00:28:59.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.993+0000 7f8016139640 1 -- 172.21.15.112:0/3534396710 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80101f91c0 con 0x7f80100ccae0 2024-09-21T00:28:59.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.993+0000 7f7ff7fff640 1 -- 172.21.15.112:0/3534396710 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80000106c0 con 0x7f80100ccae0 2024-09-21T00:28:59.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.993+0000 7f8016139640 1 -- 172.21.15.112:0/3534396710 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80101f9780 con 0x7f80100ccae0 2024-09-21T00:28:59.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.994+0000 7f8016139640 1 -- 172.21.15.112:0/3534396710 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8010068d00 con 0x7f80100ccae0 2024-09-21T00:28:59.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.995+0000 7f7ff7fff640 1 -- 172.21.15.112:0/3534396710 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f800001a070 con 0x7f80100ccae0 2024-09-21T00:28:59.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.995+0000 7f7ff7fff640 1 --2- 172.21.15.112:0/3534396710 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f7ffc078b90 0x7f7ffc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:28:59.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.995+0000 7f7ff7fff640 1 -- 172.21.15.112:0/3534396710 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 6101+0+0 (secure 0 0 0) 0x7f800009afe0 con 0x7f80100ccae0 2024-09-21T00:28:59.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.995+0000 7f800effd640 1 --2- 172.21.15.112:0/3534396710 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f7ffc078b90 0x7f7ffc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:28:59.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:28:59.996+0000 7f800effd640 1 --2- 172.21.15.112:0/3534396710 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f7ffc078b90 0x7f7ffc07b0a0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f8004034000 tx=0x7f8004004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:00.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.000+0000 7f7ff7fff640 1 -- 172.21.15.112:0/3534396710 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f8000062c40 con 0x7f80100ccae0 2024-09-21T00:29:00.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.217+0000 7f8016139640 1 -- 172.21.15.112:0/3534396710 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f80100b2900 con 0x7f80100ccae0 2024-09-21T00:29:00.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.218+0000 7f7ff7fff640 1 -- 172.21.15.112:0/3534396710 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) ==== 94+0+5231 (secure 0 0 0) 0x7f8000067670 con 0x7f80100ccae0 2024-09-21T00:29:00.219 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:29:00.219 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":55,"btime":"2024-09-21T00:28:58:105880+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5600,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6836/2708152171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":2708152171},{"type":"v1","addr":"172.21.15.187:6836","nonce":2708152171}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50},{"gid":6471,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6834/3529228075","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":3529228075},{"type":"v1","addr":"172.21.15.112:6834","nonce":3529228075}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50},{"gid":6474,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/101289292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":101289292},{"type":"v1","addr":"172.21.15.112:6835","nonce":101289292}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T00:28:54.453199+0000","modified":"2024-09-21T00:28:57.469605+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6486},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6486":{"gid":6486,"name":"d","rank":0,"incarnation":51,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6837/932692802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6835","nonce":932692802},{"type":"v1","addr":"172.21.15.187:6837","nonce":932692802}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6486,"qdb_cluster":[6486]},"id":6}]} 2024-09-21T00:29:00.219 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 55 2024-09-21T00:29:00.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.220+0000 7f8016139640 1 -- 172.21.15.112:0/3534396710 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f7ffc078b90 msgr2=0x7f7ffc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:00.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.220+0000 7f8016139640 1 --2- 172.21.15.112:0/3534396710 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f7ffc078b90 0x7f7ffc07b0a0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f8004034000 tx=0x7f8004004730 comp rx=0 tx=0).stop 2024-09-21T00:29:00.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.220+0000 7f8016139640 1 -- 172.21.15.112:0/3534396710 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80100ccae0 msgr2=0x7f8010159290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:00.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.220+0000 7f8016139640 1 --2- 172.21.15.112:0/3534396710 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80100ccae0 0x7f8010159290 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f800000d740 tx=0x7f800000dc00 comp rx=0 tx=0).stop 2024-09-21T00:29:00.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.220+0000 7f8016139640 1 -- 172.21.15.112:0/3534396710 shutdown_connections 2024-09-21T00:29:00.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.220+0000 7f8016139640 1 --2- 172.21.15.112:0/3534396710 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f80100d2030 0x7f80100ca890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:00.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.220+0000 7f8016139640 1 --2- 172.21.15.112:0/3534396710 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f80100d0860 0x7f80100aac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:00.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.220+0000 7f8016139640 1 --2- 172.21.15.112:0/3534396710 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f7ffc078b90 0x7f7ffc07b0a0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:00.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.220+0000 7f8016139640 1 --2- 172.21.15.112:0/3534396710 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80100ccae0 0x7f8010159290 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:00.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.220+0000 7f8016139640 1 -- 172.21.15.112:0/3534396710 >> 172.21.15.112:0/3534396710 conn(0x7f80100b43f0 msgr2=0x7f801015f0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:00.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.220+0000 7f8016139640 1 -- 172.21.15.112:0/3534396710 shutdown_connections 2024-09-21T00:29:00.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.220+0000 7f8016139640 1 -- 172.21.15.112:0/3534396710 wait complete. 2024-09-21T00:29:00.416 INFO:tasks.ceph.mds.d.smithi187.stderr:2024-09-21T00:29:00.415+0000 7f54a047c640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T00:29:00.635 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 55, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T00:28:54.453199+0000', 'modified': '2024-09-21T00:28:57.469605+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6486}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6486': {'gid': 6486, 'name': 'd', 'rank': 0, 'incarnation': 51, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.187:6837/932692802', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.187:6835', 'nonce': 932692802}, {'type': 'v1', 'addr': '172.21.15.187:6837', 'nonce': 932692802}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6486, 'qdb_cluster': [6486]} 2024-09-21T00:29:00.635 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-21T00:29:00.635 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-09-21T00:29:00.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.750+0000 7ff2678d3640 1 -- 172.21.15.112:0/2147512080 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2600cab00 msgr2=0x7ff26015f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:00.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.750+0000 7ff2678d3640 1 --2- 172.21.15.112:0/2147512080 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2600cab00 0x7ff26015f040 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7ff258005fd0 tx=0x7ff25802a520 comp rx=0 tx=0).stop 2024-09-21T00:29:00.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.750+0000 7ff2678d3640 1 -- 172.21.15.112:0/2147512080 shutdown_connections 2024-09-21T00:29:00.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.750+0000 7ff2678d3640 1 --2- 172.21.15.112:0/2147512080 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2600cab00 0x7ff26015f040 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:00.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.750+0000 7ff2678d3640 1 --2- 172.21.15.112:0/2147512080 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff260158fc0 0x7ff2600ca5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:00.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.750+0000 7ff2678d3640 1 -- 172.21.15.112:0/2147512080 >> 172.21.15.112:0/2147512080 conn(0x7ff260073fe0 msgr2=0x7ff2600743f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:00.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.750+0000 7ff2678d3640 1 -- 172.21.15.112:0/2147512080 shutdown_connections 2024-09-21T00:29:00.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.750+0000 7ff2678d3640 1 -- 172.21.15.112:0/2147512080 wait complete. 2024-09-21T00:29:00.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.752+0000 7ff2678d3640 1 Processor -- start 2024-09-21T00:29:00.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.752+0000 7ff2678d3640 1 -- start start 2024-09-21T00:29:00.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.753+0000 7ff2678d3640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff2600708b0 0x7ff2601f5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:00.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.753+0000 7ff2678d3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2600cab00 0x7ff2600cef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:00.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.753+0000 7ff2678d3640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff260158fc0 0x7ff2600cf4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:00.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.753+0000 7ff2678d3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff26015f700 con 0x7ff2600cab00 2024-09-21T00:29:00.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.753+0000 7ff2678d3640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff26015f580 con 0x7ff2600708b0 2024-09-21T00:29:00.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.753+0000 7ff2678d3640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff26015f880 con 0x7ff260158fc0 2024-09-21T00:29:00.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.753+0000 7ff264e6e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2600cab00 0x7ff2600cef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:00.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.754+0000 7ff264e6e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2600cab00 0x7ff2600cef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38222/0 (socket says 172.21.15.112:38222) 2024-09-21T00:29:00.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.754+0000 7ff264e6e640 1 -- 172.21.15.112:0/3896539132 learned_addr learned my addr 172.21.15.112:0/3896539132 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:29:00.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.754+0000 7ff26566f640 1 --2- 172.21.15.112:0/3896539132 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff2600708b0 0x7ff2601f5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:00.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.754+0000 7ff265e70640 1 --2- 172.21.15.112:0/3896539132 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff260158fc0 0x7ff2600cf4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:00.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.754+0000 7ff264e6e640 1 -- 172.21.15.112:0/3896539132 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff260158fc0 msgr2=0x7ff2600cf4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:00.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.754+0000 7ff264e6e640 1 --2- 172.21.15.112:0/3896539132 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff260158fc0 0x7ff2600cf4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:00.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.754+0000 7ff264e6e640 1 -- 172.21.15.112:0/3896539132 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff2600708b0 msgr2=0x7ff2601f5250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:00.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.754+0000 7ff264e6e640 1 --2- 172.21.15.112:0/3896539132 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff2600708b0 0x7ff2601f5250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:00.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.754+0000 7ff264e6e640 1 -- 172.21.15.112:0/3896539132 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff26006f7d0 con 0x7ff2600cab00 2024-09-21T00:29:00.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.754+0000 7ff264e6e640 1 --2- 172.21.15.112:0/3896539132 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2600cab00 0x7ff2600cef80 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7ff25400c740 tx=0x7ff25400cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:00.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.754+0000 7ff25dffb640 1 -- 172.21.15.112:0/3896539132 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff254007bb0 con 0x7ff2600cab00 2024-09-21T00:29:00.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.754+0000 7ff25dffb640 1 -- 172.21.15.112:0/3896539132 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff25400ce50 con 0x7ff2600cab00 2024-09-21T00:29:00.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.754+0000 7ff2678d3640 1 -- 172.21.15.112:0/3896539132 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff26006fab0 con 0x7ff2600cab00 2024-09-21T00:29:00.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.754+0000 7ff25dffb640 1 -- 172.21.15.112:0/3896539132 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff254005280 con 0x7ff2600cab00 2024-09-21T00:29:00.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.754+0000 7ff2678d3640 1 -- 172.21.15.112:0/3896539132 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff260070070 con 0x7ff2600cab00 2024-09-21T00:29:00.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.756+0000 7ff25dffb640 1 -- 172.21.15.112:0/3896539132 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff254005420 con 0x7ff2600cab00 2024-09-21T00:29:00.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.756+0000 7ff25dffb640 1 --2- 172.21.15.112:0/3896539132 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff24c078bc0 0x7ff24c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:00.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.757+0000 7ff25dffb640 1 -- 172.21.15.112:0/3896539132 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 6101+0+0 (secure 0 0 0) 0x7ff25409b1e0 con 0x7ff2600cab00 2024-09-21T00:29:00.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.757+0000 7ff26566f640 1 --2- 172.21.15.112:0/3896539132 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff24c078bc0 0x7ff24c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:00.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.757+0000 7ff26566f640 1 --2- 172.21.15.112:0/3896539132 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff24c078bc0 0x7ff24c07b0d0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7ff2600b74e0 tx=0x7ff25000d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:00.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.758+0000 7ff26e843740 1 Processor -- start 2024-09-21T00:29:00.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.758+0000 7ff26e843740 1 -- start start 2024-09-21T00:29:00.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.759+0000 7ff26e843740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x555bca2eb270 0x555bca6bcfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:00.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.759+0000 7ff26e843740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555bca2c23b0 0x555bca6bd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:00.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.759+0000 7ff26e843740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555bca2ca590 0x555bca6da400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:00.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.759+0000 7ff26e843740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x555bca605820 con 0x555bca2c23b0 2024-09-21T00:29:00.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.759+0000 7ff26e843740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x555bca5f4c50 con 0x555bca2ca590 2024-09-21T00:29:00.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.759+0000 7ff26e843740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x555bca5f8a60 con 0x555bca2eb270 2024-09-21T00:29:00.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.759+0000 7ff264e6e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555bca2c23b0 0x555bca6bd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:00.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.759+0000 7ff264e6e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555bca2c23b0 0x555bca6bd500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38238/0 (socket says 172.21.15.112:38238) 2024-09-21T00:29:00.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.759+0000 7ff264e6e640 1 -- 172.21.15.112:0/3529275208 learned_addr learned my addr 172.21.15.112:0/3529275208 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:29:00.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.759+0000 7ff26566f640 1 --2- 172.21.15.112:0/3529275208 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x555bca2eb270 0x555bca6bcfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:00.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.759+0000 7ff265e70640 1 --2- 172.21.15.112:0/3529275208 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555bca2ca590 0x555bca6da400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:00.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.759+0000 7ff264e6e640 1 -- 172.21.15.112:0/3529275208 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x555bca2eb270 msgr2=0x555bca6bcfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:29:00.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.759+0000 7ff264e6e640 1 --2- 172.21.15.112:0/3529275208 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x555bca2eb270 0x555bca6bcfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:00.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.759+0000 7ff264e6e640 1 -- 172.21.15.112:0/3529275208 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555bca2ca590 msgr2=0x555bca6da400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:29:00.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.759+0000 7ff264e6e640 1 --2- 172.21.15.112:0/3529275208 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555bca2ca590 0x555bca6da400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:00.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.760+0000 7ff264e6e640 1 -- 172.21.15.112:0/3529275208 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555bca356850 con 0x555bca2c23b0 2024-09-21T00:29:00.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.760+0000 7ff265e70640 1 --2- 172.21.15.112:0/3529275208 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555bca2ca590 0x555bca6da400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:29:00.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.760+0000 7ff264e6e640 1 --2- 172.21.15.112:0/3529275208 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555bca2c23b0 0x555bca6bd500 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7ff254099520 tx=0x7ff254062640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e463a671034c67 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:00.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.760+0000 7ff246ffd640 1 -- 172.21.15.112:0/3529275208 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff25401d030 con 0x555bca2c23b0 2024-09-21T00:29:00.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.760+0000 7ff246ffd640 1 -- 172.21.15.112:0/3529275208 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff2540891c0 con 0x555bca2c23b0 2024-09-21T00:29:00.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.760+0000 7ff246ffd640 1 -- 172.21.15.112:0/3529275208 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff254062390 con 0x555bca2c23b0 2024-09-21T00:29:00.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.761+0000 7ff26e843740 1 -- 172.21.15.112:0/3529275208 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555bca2c23b0 msgr2=0x555bca6bd500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:00.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.761+0000 7ff26e843740 1 --2- 172.21.15.112:0/3529275208 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555bca2c23b0 0x555bca6bd500 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7ff254099520 tx=0x7ff254062640 comp rx=0 tx=0).stop 2024-09-21T00:29:00.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.761+0000 7ff26e843740 1 -- 172.21.15.112:0/3529275208 shutdown_connections 2024-09-21T00:29:00.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.761+0000 7ff26e843740 1 --2- 172.21.15.112:0/3529275208 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555bca2ca590 0x555bca6da400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:00.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.761+0000 7ff26e843740 1 --2- 172.21.15.112:0/3529275208 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555bca2c23b0 0x555bca6bd500 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:00.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.761+0000 7ff26e843740 1 --2- 172.21.15.112:0/3529275208 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x555bca2eb270 0x555bca6bcfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:00.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.761+0000 7ff26e843740 1 -- 172.21.15.112:0/3529275208 >> 172.21.15.112:0/3529275208 conn(0x555bca2df3c0 msgr2=0x555bca693280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:00.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.761+0000 7ff26e843740 1 -- 172.21.15.112:0/3529275208 shutdown_connections 2024-09-21T00:29:00.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.761+0000 7ff26e843740 1 -- 172.21.15.112:0/3529275208 wait complete. 2024-09-21T00:29:00.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.761+0000 7ff26e843740 1 -- start start 2024-09-21T00:29:00.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.762+0000 7ff26e843740 1 Processor -- start 2024-09-21T00:29:00.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.763+0000 7ff26e843740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x555bca2c23b0 0x555bca6b6720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:00.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.763+0000 7ff26e843740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555bca2ca590 0x555bca6cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:00.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.763+0000 7ff26e843740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555bca2eb270 0x555bca6cf0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:00.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.763+0000 7ff26e843740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x555bca365620 con 0x555bca2ca590 2024-09-21T00:29:00.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.763+0000 7ff26e843740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x555bca59a5f0 con 0x555bca2eb270 2024-09-21T00:29:00.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.763+0000 7ff26e843740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x555bca6bda40 con 0x555bca2c23b0 2024-09-21T00:29:00.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.763+0000 7ff26e843740 10 client.0 _ll_register_callbacks cb 0x555bca59e0a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:29:00.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.763+0000 7ff264e6e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555bca2ca590 0x555bca6cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:00.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.763+0000 7ff264e6e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555bca2ca590 0x555bca6cebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38254/0 (socket says 172.21.15.112:38254) 2024-09-21T00:29:00.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.763+0000 7ff264e6e640 1 -- 172.21.15.112:0/2886485102 learned_addr learned my addr 172.21.15.112:0/2886485102 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:29:00.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.763+0000 7ff264e6e640 1 -- 172.21.15.112:0/2886485102 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x555bca2c23b0 msgr2=0x555bca6b6720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:29:00.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.763+0000 7ff264e6e640 1 --2- 172.21.15.112:0/2886485102 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x555bca2c23b0 0x555bca6b6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:00.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.763+0000 7ff265e70640 1 --2- 172.21.15.112:0/2886485102 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555bca2eb270 0x555bca6cf0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:00.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.763+0000 7ff264e6e640 1 -- 172.21.15.112:0/2886485102 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555bca2eb270 msgr2=0x555bca6cf0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:29:00.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.763+0000 7ff264e6e640 1 --2- 172.21.15.112:0/2886485102 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555bca2eb270 0x555bca6cf0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:00.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.763+0000 7ff264e6e640 1 -- 172.21.15.112:0/2886485102 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555bca3cd6b0 con 0x555bca2ca590 2024-09-21T00:29:00.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.764+0000 7ff264e6e640 1 --2- 172.21.15.112:0/2886485102 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555bca2ca590 0x555bca6cebb0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7ff25409b500 tx=0x7ff254062340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e881af17921bf47 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:00.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.764+0000 7ff23ffff640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T00:29:00.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.764+0000 7ff23ffff640 1 -- 172.21.15.112:0/2886485102 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff25407d060 con 0x555bca2ca590 2024-09-21T00:29:00.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.764+0000 7ff23ffff640 1 -- 172.21.15.112:0/2886485102 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 6101+0+0 (secure 0 0 0) 0x7ff25404e1e0 con 0x555bca2ca590 2024-09-21T00:29:00.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.764+0000 7ff26e843740 1 -- 172.21.15.112:0/2886485102 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555bca607770 con 0x555bca2ca590 2024-09-21T00:29:00.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.764+0000 7ff23ffff640 1 -- 172.21.15.112:0/2886485102 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7ff254083050 con 0x555bca2ca590 2024-09-21T00:29:00.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.764+0000 7ff26e843740 10 client.6597 fetch_fsmap learned FSMap version 55 2024-09-21T00:29:00.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.764+0000 7ff26e843740 1 -- 172.21.15.112:0/2886485102 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=2+}) -- 0x555bca6cf8d0 con 0x555bca2ca590 2024-09-21T00:29:00.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.764+0000 7ff23ffff640 1 -- 172.21.15.112:0/2886485102 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x555bca607770 con 0x555bca2ca590 2024-09-21T00:29:00.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.764+0000 7ff23ffff640 1 -- 172.21.15.112:0/2886485102 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 55) ==== 3502+0+0 (secure 0 0 0) 0x7ff254061ba0 con 0x555bca2ca590 2024-09-21T00:29:00.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.764+0000 7ff26e843740 10 client.6597 fetch_fsmap finished waiting for FSMap version 55 2024-09-21T00:29:00.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.764+0000 7ff26e843740 10 client.6597 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-21T00:29:00.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.764+0000 7ff26e843740 20 client.6597 populate_metadata read hostname 'smithi112' 2024-09-21T00:29:00.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.764+0000 7ff26e843740 1 --2- 172.21.15.112:0/2886485102 >> [v2:172.21.15.187:6835/932692802,v1:172.21.15.187:6837/932692802] conn(0x555bca2a9e40 0x555bca6eb100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:00.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.764+0000 7ff26e843740 4 client.6597 mds_command: new command op to 6486 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T00:29:00.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.764+0000 7ff26e843740 1 -- 172.21.15.112:0/2886485102 --> [v2:172.21.15.187:6835/932692802,v1:172.21.15.187:6837/932692802] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555bca603220 con 0x555bca2a9e40 2024-09-21T00:29:00.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.764+0000 7ff26566f640 1 --2- 172.21.15.112:0/2886485102 >> [v2:172.21.15.187:6835/932692802,v1:172.21.15.187:6837/932692802] conn(0x555bca2a9e40 0x555bca6eb100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:00.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.765+0000 7ff23ffff640 10 client.6597 ms_handle_connect on v2:172.21.15.187:6835/932692802 2024-09-21T00:29:00.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.765+0000 7ff26566f640 1 --2- 172.21.15.112:0/2886485102 >> [v2:172.21.15.187:6835/932692802,v1:172.21.15.187:6837/932692802] conn(0x555bca2a9e40 0x555bca6eb100 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a13680f17dd7663 server_cookie=167cd7706258c7e in_seq=0 out_seq=0 2024-09-21T00:29:00.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.766+0000 7ff23ffff640 1 -- 172.21.15.112:0/2886485102 <== mds.0 v2:172.21.15.187:6835/932692802 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x555bca603220 con 0x555bca2a9e40 2024-09-21T00:29:00.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.766+0000 7ff23ffff640 10 client.6597 handle_command_reply: tid=0 2024-09-21T00:29:00.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.766+0000 7ff26e843740 1 client.6597 shutdown 2024-09-21T00:29:00.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.766+0000 7ff26e843740 1 -- 172.21.15.112:0/2886485102 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555bca2ca590 msgr2=0x555bca6cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:00.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.766+0000 7ff26e843740 1 --2- 172.21.15.112:0/2886485102 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555bca2ca590 0x555bca6cebb0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7ff25409b500 tx=0x7ff254062340 comp rx=0 tx=0).stop 2024-09-21T00:29:00.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.766+0000 7ff26e843740 1 -- 172.21.15.112:0/2886485102 shutdown_connections 2024-09-21T00:29:00.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.766+0000 7ff26e843740 1 --2- 172.21.15.112:0/2886485102 >> [v2:172.21.15.187:6835/932692802,v1:172.21.15.187:6837/932692802] conn(0x555bca2a9e40 0x555bca6eb100 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:00.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.766+0000 7ff26e843740 1 --2- 172.21.15.112:0/2886485102 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555bca2eb270 0x555bca6cf0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:00.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.766+0000 7ff26e843740 1 --2- 172.21.15.112:0/2886485102 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555bca2ca590 0x555bca6cebb0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:00.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.766+0000 7ff26e843740 1 --2- 172.21.15.112:0/2886485102 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x555bca2c23b0 0x555bca6b6720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:00.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.766+0000 7ff23ffff640 0 client.6597 ms_handle_reset on v2:172.21.15.187:6835/932692802 2024-09-21T00:29:00.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.766+0000 7ff26e843740 1 -- 172.21.15.112:0/2886485102 >> 172.21.15.112:0/2886485102 conn(0x555bca2df3c0 msgr2=0x555bca693560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:00.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.766+0000 7ff26e843740 1 -- 172.21.15.112:0/2886485102 shutdown_connections 2024-09-21T00:29:00.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.766+0000 7ff26e843740 1 -- 172.21.15.112:0/2886485102 wait complete. 2024-09-21T00:29:00.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.766+0000 7ff26e843740 20 client.6597 trim_cache size 0 max 16384 2024-09-21T00:29:00.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.784+0000 7ff26e843740 1 Processor -- start 2024-09-21T00:29:00.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.784+0000 7ff26e843740 1 -- start start 2024-09-21T00:29:00.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.786+0000 7ff26e843740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x555bca2c23b0 0x555bca6e9d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:00.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.786+0000 7ff26e843740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555bca2ca590 0x555bca6ceda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:00.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.786+0000 7ff26e843740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555bca2eb270 0x555bca6faaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:00.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.786+0000 7ff26e843740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff260201860 con 0x555bca2eb270 2024-09-21T00:29:00.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.786+0000 7ff26e843740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff260152e40 con 0x555bca2ca590 2024-09-21T00:29:00.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.786+0000 7ff26e843740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff260201340 con 0x555bca2c23b0 2024-09-21T00:29:00.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.786+0000 7ff265e70640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555bca2eb270 0x555bca6faaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:00.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.786+0000 7ff265e70640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555bca2eb270 0x555bca6faaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38258/0 (socket says 172.21.15.112:38258) 2024-09-21T00:29:00.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.786+0000 7ff265e70640 1 -- 172.21.15.112:0/3141834670 learned_addr learned my addr 172.21.15.112:0/3141834670 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:29:00.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.786+0000 7ff264e6e640 1 --2- 172.21.15.112:0/3141834670 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555bca2ca590 0x555bca6ceda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:00.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.786+0000 7ff26566f640 1 --2- 172.21.15.112:0/3141834670 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x555bca2c23b0 0x555bca6e9d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:00.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.786+0000 7ff265e70640 1 -- 172.21.15.112:0/3141834670 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x555bca2c23b0 msgr2=0x555bca6e9d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:29:00.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.786+0000 7ff265e70640 1 --2- 172.21.15.112:0/3141834670 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x555bca2c23b0 0x555bca6e9d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:00.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.786+0000 7ff265e70640 1 -- 172.21.15.112:0/3141834670 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555bca2ca590 msgr2=0x555bca6ceda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:29:00.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.786+0000 7ff265e70640 1 --2- 172.21.15.112:0/3141834670 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555bca2ca590 0x555bca6ceda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:00.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.786+0000 7ff265e70640 1 -- 172.21.15.112:0/3141834670 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555bca6d8a20 con 0x555bca2eb270 2024-09-21T00:29:00.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.786+0000 7ff264e6e640 1 --2- 172.21.15.112:0/3141834670 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555bca2ca590 0x555bca6ceda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:29:00.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.786+0000 7ff265e70640 1 --2- 172.21.15.112:0/3141834670 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555bca2eb270 0x555bca6faaf0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7ff258034040 tx=0x7ff2580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc0deec61d3c499 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:00.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.786+0000 7ff2467fc640 1 -- 172.21.15.112:0/3141834670 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff258035040 con 0x555bca2eb270 2024-09-21T00:29:00.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.786+0000 7ff2467fc640 1 -- 172.21.15.112:0/3141834670 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff258004320 con 0x555bca2eb270 2024-09-21T00:29:00.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.786+0000 7ff2467fc640 1 -- 172.21.15.112:0/3141834670 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff258003850 con 0x555bca2eb270 2024-09-21T00:29:00.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.787+0000 7ff26e843740 1 -- 172.21.15.112:0/3141834670 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555bca2eb270 msgr2=0x555bca6faaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:00.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.787+0000 7ff26e843740 1 --2- 172.21.15.112:0/3141834670 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555bca2eb270 0x555bca6faaf0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7ff258034040 tx=0x7ff2580046c0 comp rx=0 tx=0).stop 2024-09-21T00:29:00.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.787+0000 7ff26e843740 1 -- 172.21.15.112:0/3141834670 shutdown_connections 2024-09-21T00:29:00.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.787+0000 7ff26e843740 1 --2- 172.21.15.112:0/3141834670 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555bca2eb270 0x555bca6faaf0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:00.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.787+0000 7ff26e843740 1 --2- 172.21.15.112:0/3141834670 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555bca2ca590 0x555bca6ceda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:00.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.787+0000 7ff26e843740 1 --2- 172.21.15.112:0/3141834670 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x555bca2c23b0 0x555bca6e9d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:00.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.787+0000 7ff26e843740 1 -- 172.21.15.112:0/3141834670 >> 172.21.15.112:0/3141834670 conn(0x555bca2df3c0 msgr2=0x555bca6941a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:00.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.787+0000 7ff26e843740 1 -- 172.21.15.112:0/3141834670 shutdown_connections 2024-09-21T00:29:00.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.787+0000 7ff26e843740 1 -- 172.21.15.112:0/3141834670 wait complete. 2024-09-21T00:29:00.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.788+0000 7ff26e843740 1 -- start start 2024-09-21T00:29:00.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.788+0000 7ff26e843740 1 Processor -- start 2024-09-21T00:29:00.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.789+0000 7ff26e843740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x555bca2c23b0 0x555bca7549f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:00.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.789+0000 7ff26e843740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555bca2ca590 0x555bca6e9620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:00.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.789+0000 7ff26e843740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555bca2eb270 0x555bca6e9b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:00.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.789+0000 7ff26e843740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x555bca365620 con 0x555bca2ca590 2024-09-21T00:29:00.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.789+0000 7ff26e843740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x555bca59a5f0 con 0x555bca2eb270 2024-09-21T00:29:00.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.789+0000 7ff26e843740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x555bca6b9f70 con 0x555bca2c23b0 2024-09-21T00:29:00.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.789+0000 7ff26e843740 10 client.0 _ll_register_callbacks cb 0x7ff250009c30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:29:00.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.790+0000 7ff264e6e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555bca2ca590 0x555bca6e9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:00.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.790+0000 7ff264e6e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555bca2ca590 0x555bca6e9620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38260/0 (socket says 172.21.15.112:38260) 2024-09-21T00:29:00.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.790+0000 7ff264e6e640 1 -- 172.21.15.112:0/3286511949 learned_addr learned my addr 172.21.15.112:0/3286511949 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:29:00.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.790+0000 7ff265e70640 1 --2- 172.21.15.112:0/3286511949 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555bca2eb270 0x555bca6e9b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:00.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.790+0000 7ff26566f640 1 --2- 172.21.15.112:0/3286511949 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x555bca2c23b0 0x555bca7549f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:00.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.790+0000 7ff264e6e640 1 -- 172.21.15.112:0/3286511949 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x555bca2c23b0 msgr2=0x555bca7549f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:29:00.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.790+0000 7ff264e6e640 1 --2- 172.21.15.112:0/3286511949 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x555bca2c23b0 0x555bca7549f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:00.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.790+0000 7ff264e6e640 1 -- 172.21.15.112:0/3286511949 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555bca2eb270 msgr2=0x555bca6e9b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:29:00.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.790+0000 7ff264e6e640 1 --2- 172.21.15.112:0/3286511949 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555bca2eb270 0x555bca6e9b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:00.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.790+0000 7ff264e6e640 1 -- 172.21.15.112:0/3286511949 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555bca6fb080 con 0x555bca2ca590 2024-09-21T00:29:00.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.790+0000 7ff26566f640 1 --2- 172.21.15.112:0/3286511949 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x555bca2c23b0 0x555bca7549f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:29:00.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.790+0000 7ff265e70640 1 --2- 172.21.15.112:0/3286511949 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555bca2eb270 0x555bca6e9b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:29:00.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.790+0000 7ff264e6e640 1 --2- 172.21.15.112:0/3286511949 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555bca2ca590 0x555bca6e9620 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7ff25400a2a0 tx=0x7ff254062820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eced8b390aec2ea4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:00.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.790+0000 7ff23ffff640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T00:29:00.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.790+0000 7ff23ffff640 1 -- 172.21.15.112:0/3286511949 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff25401d070 con 0x555bca2ca590 2024-09-21T00:29:00.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.790+0000 7ff23ffff640 1 -- 172.21.15.112:0/3286511949 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 6101+0+0 (secure 0 0 0) 0x7ff254019c40 con 0x555bca2ca590 2024-09-21T00:29:00.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.790+0000 7ff26e843740 1 -- 172.21.15.112:0/3286511949 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff258004320 con 0x555bca2ca590 2024-09-21T00:29:00.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.790+0000 7ff23ffff640 1 -- 172.21.15.112:0/3286511949 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7ff254067950 con 0x555bca2ca590 2024-09-21T00:29:00.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.790+0000 7ff26e843740 10 client.6603 fetch_fsmap learned FSMap version 55 2024-09-21T00:29:00.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.790+0000 7ff26e843740 1 -- 172.21.15.112:0/3286511949 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=2+}) -- 0x555bca75ba50 con 0x555bca2ca590 2024-09-21T00:29:00.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.790+0000 7ff23ffff640 1 -- 172.21.15.112:0/3286511949 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff258004320 con 0x555bca2ca590 2024-09-21T00:29:00.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.791+0000 7ff23ffff640 1 -- 172.21.15.112:0/3286511949 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 55) ==== 3502+0+0 (secure 0 0 0) 0x7ff25404e1e0 con 0x555bca2ca590 2024-09-21T00:29:00.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.791+0000 7ff26e843740 10 client.6603 fetch_fsmap finished waiting for FSMap version 55 2024-09-21T00:29:00.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.791+0000 7ff26e843740 10 client.6603 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-21T00:29:00.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.791+0000 7ff26e843740 20 client.6603 populate_metadata read hostname 'smithi112' 2024-09-21T00:29:00.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.791+0000 7ff26e843740 1 --2- 172.21.15.112:0/3286511949 >> [v2:172.21.15.187:6835/932692802,v1:172.21.15.187:6837/932692802] conn(0x555bca6914e0 0x555bca6bc320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:00.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.791+0000 7ff26e843740 4 client.6603 mds_command: new command op to 6486 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T00:29:00.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.791+0000 7ff26e843740 1 -- 172.21.15.112:0/3286511949 --> [v2:172.21.15.187:6835/932692802,v1:172.21.15.187:6837/932692802] -- command(tid 0: {"prefix": "status"}) -- 0x555bca600650 con 0x555bca6914e0 2024-09-21T00:29:00.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.791+0000 7ff26566f640 1 --2- 172.21.15.112:0/3286511949 >> [v2:172.21.15.187:6835/932692802,v1:172.21.15.187:6837/932692802] conn(0x555bca6914e0 0x555bca6bc320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:00.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.791+0000 7ff23ffff640 10 client.6603 ms_handle_connect on v2:172.21.15.187:6835/932692802 2024-09-21T00:29:00.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.791+0000 7ff26566f640 1 --2- 172.21.15.112:0/3286511949 >> [v2:172.21.15.187:6835/932692802,v1:172.21.15.187:6837/932692802] conn(0x555bca6914e0 0x555bca6bc320 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9083dd8340b7baa server_cookie=6085d2afe180607b in_seq=0 out_seq=0 2024-09-21T00:29:00.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.791+0000 7ff23ffff640 1 -- 172.21.15.112:0/3286511949 <== mds.0 v2:172.21.15.187:6835/932692802 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x555bca600650 con 0x555bca6914e0 2024-09-21T00:29:00.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.791+0000 7ff23ffff640 10 client.6603 handle_command_reply: tid=0 2024-09-21T00:29:00.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.791+0000 7ff26e843740 1 client.6603 shutdown 2024-09-21T00:29:00.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.792+0000 7ff26e843740 1 -- 172.21.15.112:0/3286511949 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555bca2ca590 msgr2=0x555bca6e9620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:00.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.792+0000 7ff26e843740 1 --2- 172.21.15.112:0/3286511949 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555bca2ca590 0x555bca6e9620 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7ff25400a2a0 tx=0x7ff254062820 comp rx=0 tx=0).stop 2024-09-21T00:29:00.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.792+0000 7ff26e843740 1 -- 172.21.15.112:0/3286511949 shutdown_connections 2024-09-21T00:29:00.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.792+0000 7ff26e843740 1 --2- 172.21.15.112:0/3286511949 >> [v2:172.21.15.187:6835/932692802,v1:172.21.15.187:6837/932692802] conn(0x555bca6914e0 0x555bca6bc320 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:00.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.792+0000 7ff26e843740 1 --2- 172.21.15.112:0/3286511949 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555bca2eb270 0x555bca6e9b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:00.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.792+0000 7ff23ffff640 0 client.6603 ms_handle_reset on v2:172.21.15.187:6835/932692802 2024-09-21T00:29:00.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.792+0000 7ff26e843740 1 --2- 172.21.15.112:0/3286511949 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555bca2ca590 0x555bca6e9620 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:00.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.792+0000 7ff26e843740 1 --2- 172.21.15.112:0/3286511949 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x555bca2c23b0 0x555bca7549f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:00.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.792+0000 7ff26e843740 1 -- 172.21.15.112:0/3286511949 >> 172.21.15.112:0/3286511949 conn(0x555bca2df3c0 msgr2=0x555bca6da850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:00.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.792+0000 7ff26e843740 1 -- 172.21.15.112:0/3286511949 shutdown_connections 2024-09-21T00:29:00.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.792+0000 7ff26e843740 1 -- 172.21.15.112:0/3286511949 wait complete. 2024-09-21T00:29:00.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.792+0000 7ff26e843740 20 client.6603 trim_cache size 0 max 16384 2024-09-21T00:29:00.798 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-21T00:29:00.798 INFO:teuthology.orchestra.run.smithi112.stdout: "cluster_fsid": "8b34692b-dc65-49b2-a08f-26f707b96ac5", 2024-09-21T00:29:00.798 INFO:teuthology.orchestra.run.smithi112.stdout: "whoami": 0, 2024-09-21T00:29:00.798 INFO:teuthology.orchestra.run.smithi112.stdout: "id": 6486, 2024-09-21T00:29:00.798 INFO:teuthology.orchestra.run.smithi112.stdout: "want_state": "up:active", 2024-09-21T00:29:00.798 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "up:active", 2024-09-21T00:29:00.799 INFO:teuthology.orchestra.run.smithi112.stdout: "fs_name": "cephfs", 2024-09-21T00:29:00.799 INFO:teuthology.orchestra.run.smithi112.stdout: "rank_uptime": 5.7105275129999997, 2024-09-21T00:29:00.799 INFO:teuthology.orchestra.run.smithi112.stdout: "mdsmap_epoch": 55, 2024-09-21T00:29:00.799 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch": 83, 2024-09-21T00:29:00.799 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch_barrier": 83, 2024-09-21T00:29:00.799 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 11.021384485 2024-09-21T00:29:00.799 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-21T00:29:00.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.792+0000 7ff2678d3640 1 -- 172.21.15.112:0/3896539132 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff24c078bc0 msgr2=0x7ff24c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:00.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.792+0000 7ff2678d3640 1 --2- 172.21.15.112:0/3896539132 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff24c078bc0 0x7ff24c07b0d0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7ff2600b74e0 tx=0x7ff25000d040 comp rx=0 tx=0).stop 2024-09-21T00:29:00.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.792+0000 7ff2678d3640 1 -- 172.21.15.112:0/3896539132 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2600cab00 msgr2=0x7ff2600cef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:00.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.792+0000 7ff2678d3640 1 --2- 172.21.15.112:0/3896539132 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2600cab00 0x7ff2600cef80 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7ff25400c740 tx=0x7ff25400cc00 comp rx=0 tx=0).stop 2024-09-21T00:29:00.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.792+0000 7ff2678d3640 1 -- 172.21.15.112:0/3896539132 shutdown_connections 2024-09-21T00:29:00.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.792+0000 7ff2678d3640 1 --2- 172.21.15.112:0/3896539132 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff260158fc0 0x7ff2600cf4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:00.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.792+0000 7ff2678d3640 1 --2- 172.21.15.112:0/3896539132 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff24c078bc0 0x7ff24c07b0d0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:00.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.792+0000 7ff2678d3640 1 --2- 172.21.15.112:0/3896539132 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2600cab00 0x7ff2600cef80 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:00.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.792+0000 7ff2678d3640 1 --2- 172.21.15.112:0/3896539132 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff2600708b0 0x7ff2601f5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:00.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.792+0000 7ff2678d3640 1 -- 172.21.15.112:0/3896539132 >> 172.21.15.112:0/3896539132 conn(0x7ff260073fe0 msgr2=0x7ff260157190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:00.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.793+0000 7ff2678d3640 1 -- 172.21.15.112:0/3896539132 shutdown_connections 2024-09-21T00:29:00.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.793+0000 7ff2678d3640 1 -- 172.21.15.112:0/3896539132 wait complete. 2024-09-21T00:29:00.804 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T00:29:00.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.916+0000 7f23c9c12640 1 -- 172.21.15.112:0/1254191234 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23ac003b10 msgr2=0x7f23ac003f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:00.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.916+0000 7f23c9c12640 1 --2- 172.21.15.112:0/1254191234 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23ac003b10 0x7f23ac003f80 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f23b400bb20 tx=0x7f23ac005830 comp rx=0 tx=0).stop 2024-09-21T00:29:00.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.916+0000 7f23c3fff640 1 -- 172.21.15.112:0/1254191234 reap_dead start 2024-09-21T00:29:00.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.916+0000 7f23c9c12640 1 -- 172.21.15.112:0/1254191234 shutdown_connections 2024-09-21T00:29:00.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.916+0000 7f23c9c12640 1 -- 172.21.15.112:0/1254191234 >> 172.21.15.112:0/1254191234 conn(0x7f23c40b42e0 msgr2=0x7f23c40b4720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:00.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.916+0000 7f23c9c12640 1 -- 172.21.15.112:0/1254191234 shutdown_connections 2024-09-21T00:29:00.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.916+0000 7f23c9c12640 1 -- 172.21.15.112:0/1254191234 wait complete. 2024-09-21T00:29:00.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.918+0000 7f23c9c12640 1 Processor -- start 2024-09-21T00:29:00.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.918+0000 7f23c9c12640 1 -- start start 2024-09-21T00:29:00.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.919+0000 7f23c9c12640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f23ac003b10 0x7f23c4148080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:00.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.919+0000 7f23c9c12640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f23c414a840 0x7f23c414ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:00.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.920+0000 7f23c9c12640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23c4149a30 0x7f23c4149e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:00.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.920+0000 7f23c9c12640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f23c40c6f20 con 0x7f23c4149a30 2024-09-21T00:29:00.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.920+0000 7f23c9c12640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f23c40b3660 con 0x7f23ac003b10 2024-09-21T00:29:00.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.920+0000 7f23c9c12640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f23c40cd000 con 0x7f23c414a840 2024-09-21T00:29:00.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.920+0000 7f23c3fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23c4149a30 0x7f23c4149e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:00.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.920+0000 7f23c3fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23c4149a30 0x7f23c4149e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38284/0 (socket says 172.21.15.112:38284) 2024-09-21T00:29:00.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.920+0000 7f23c3fff640 1 -- 172.21.15.112:0/2567034998 learned_addr learned my addr 172.21.15.112:0/2567034998 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:29:00.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.920+0000 7f23c37fe640 1 --2- 172.21.15.112:0/2567034998 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f23ac003b10 0x7f23c4148080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:00.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.920+0000 7f23c2ffd640 1 --2- 172.21.15.112:0/2567034998 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f23c414a840 0x7f23c414ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:00.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.920+0000 7f23c3fff640 1 -- 172.21.15.112:0/2567034998 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f23c414a840 msgr2=0x7f23c414ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:00.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.920+0000 7f23c3fff640 1 --2- 172.21.15.112:0/2567034998 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f23c414a840 0x7f23c414ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:00.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.920+0000 7f23c3fff640 1 -- 172.21.15.112:0/2567034998 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f23ac003b10 msgr2=0x7f23c4148080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:00.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.920+0000 7f23c3fff640 1 --2- 172.21.15.112:0/2567034998 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f23ac003b10 0x7f23c4148080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:00.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.920+0000 7f23c3fff640 1 -- 172.21.15.112:0/2567034998 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23c414b190 con 0x7f23c4149a30 2024-09-21T00:29:00.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.920+0000 7f23c37fe640 1 --2- 172.21.15.112:0/2567034998 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f23ac003b10 0x7f23c4148080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:29:00.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.920+0000 7f23c3fff640 1 --2- 172.21.15.112:0/2567034998 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23c4149a30 0x7f23c4149e40 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f23c40b84d0 tx=0x7f23bc006720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:00.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.920+0000 7f23abfff640 1 -- 172.21.15.112:0/2567034998 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23bc00fa80 con 0x7f23c4149a30 2024-09-21T00:29:00.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.920+0000 7f23abfff640 1 -- 172.21.15.112:0/2567034998 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f23bc018070 con 0x7f23c4149a30 2024-09-21T00:29:00.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.920+0000 7f23c9c12640 1 -- 172.21.15.112:0/2567034998 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23c4148990 con 0x7f23c4149a30 2024-09-21T00:29:00.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.920+0000 7f23abfff640 1 -- 172.21.15.112:0/2567034998 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23bc011430 con 0x7f23c4149a30 2024-09-21T00:29:00.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.920+0000 7f23c9c12640 1 -- 172.21.15.112:0/2567034998 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23c4148f50 con 0x7f23c4149a30 2024-09-21T00:29:00.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.921+0000 7f23c9c12640 1 -- 172.21.15.112:0/2567034998 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23c40c62f0 con 0x7f23c4149a30 2024-09-21T00:29:00.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.922+0000 7f23abfff640 1 -- 172.21.15.112:0/2567034998 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f23bc0115d0 con 0x7f23c4149a30 2024-09-21T00:29:00.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.922+0000 7f23abfff640 1 --2- 172.21.15.112:0/2567034998 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f23b0078bc0 0x7f23b007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:00.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.923+0000 7f23abfff640 1 -- 172.21.15.112:0/2567034998 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 6101+0+0 (secure 0 0 0) 0x7f23bc09b490 con 0x7f23c4149a30 2024-09-21T00:29:00.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.923+0000 7f23c37fe640 1 --2- 172.21.15.112:0/2567034998 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f23b0078bc0 0x7f23b007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:00.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.923+0000 7f23c37fe640 1 --2- 172.21.15.112:0/2567034998 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f23b0078bc0 0x7f23b007b0d0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f23b4036000 tx=0x7f23b40046c0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:00.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:00.927+0000 7f23abfff640 1 -- 172.21.15.112:0/2567034998 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f23c414b190 con 0x7f23c4149a30 2024-09-21T00:29:01.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:01.142+0000 7f23c9c12640 1 -- 172.21.15.112:0/2567034998 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f23c40c7e20 con 0x7f23c4149a30 2024-09-21T00:29:01.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:01.142+0000 7f23abfff640 1 -- 172.21.15.112:0/2567034998 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v55) ==== 53+0+73 (secure 0 0 0) 0x7f23c40cd540 con 0x7f23c4149a30 2024-09-21T00:29:01.143 INFO:teuthology.orchestra.run.smithi112.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T00:29:01.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:01.145+0000 7f23c9c12640 1 -- 172.21.15.112:0/2567034998 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f23b0078bc0 msgr2=0x7f23b007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:01.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:01.145+0000 7f23c9c12640 1 --2- 172.21.15.112:0/2567034998 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f23b0078bc0 0x7f23b007b0d0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f23b4036000 tx=0x7f23b40046c0 comp rx=0 tx=0).stop 2024-09-21T00:29:01.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:01.145+0000 7f23c9c12640 1 -- 172.21.15.112:0/2567034998 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23c4149a30 msgr2=0x7f23c4149e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:01.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:01.145+0000 7f23c9c12640 1 --2- 172.21.15.112:0/2567034998 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23c4149a30 0x7f23c4149e40 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f23c40b84d0 tx=0x7f23bc006720 comp rx=0 tx=0).stop 2024-09-21T00:29:01.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:01.145+0000 7f23c9c12640 1 -- 172.21.15.112:0/2567034998 shutdown_connections 2024-09-21T00:29:01.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:01.145+0000 7f23c9c12640 1 --2- 172.21.15.112:0/2567034998 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f23b0078bc0 0x7f23b007b0d0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:01.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:01.145+0000 7f23c9c12640 1 --2- 172.21.15.112:0/2567034998 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23c4149a30 0x7f23c4149e40 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:01.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:01.145+0000 7f23c9c12640 1 --2- 172.21.15.112:0/2567034998 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f23c414a840 0x7f23c414ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:01.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:01.145+0000 7f23c9c12640 1 --2- 172.21.15.112:0/2567034998 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f23ac003b10 0x7f23c4148080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:01.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:01.145+0000 7f23c9c12640 1 -- 172.21.15.112:0/2567034998 >> 172.21.15.112:0/2567034998 conn(0x7f23c40b42e0 msgr2=0x7f23c40d06f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:01.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:01.145+0000 7f23c9c12640 1 -- 172.21.15.112:0/2567034998 shutdown_connections 2024-09-21T00:29:01.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:01.145+0000 7f23c9c12640 1 -- 172.21.15.112:0/2567034998 wait complete. 2024-09-21T00:29:01.157 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T00:29:01.157 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T00:29:01.157 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi112.front.sepia.ceph.com 2024-09-21T00:29:01.157 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T00:29:01.157 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T00:29:01.158 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T00:29:01.158 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-21T00:29:01.158 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T00:29:01.158 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T00:29:01.158 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:29:01.158 DEBUG:teuthology.orchestra.run.smithi112:> ip addr 2024-09-21T00:29:01.174 INFO:teuthology.orchestra.run.smithi112.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T00:29:01.174 INFO:teuthology.orchestra.run.smithi112.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T00:29:01.174 INFO:teuthology.orchestra.run.smithi112.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T00:29:01.174 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:29:01.174 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 ::1/128 scope host 2024-09-21T00:29:01.174 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:29:01.175 INFO:teuthology.orchestra.run.smithi112.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T00:29:01.175 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:d8:a5:2c brd ff:ff:ff:ff:ff:ff 2024-09-21T00:29:01.175 INFO:teuthology.orchestra.run.smithi112.stdout: altname enp7s0f0 2024-09-21T00:29:01.175 INFO:teuthology.orchestra.run.smithi112.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T00:29:01.175 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:d8:a5:2d brd ff:ff:ff:ff:ff:ff 2024-09-21T00:29:01.175 INFO:teuthology.orchestra.run.smithi112.stdout: altname enp7s0f1 2024-09-21T00:29:01.175 INFO:teuthology.orchestra.run.smithi112.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T00:29:01.175 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:8f:cb:68 brd ff:ff:ff:ff:ff:ff 2024-09-21T00:29:01.175 INFO:teuthology.orchestra.run.smithi112.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T00:29:01.175 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:8f:cb:69 brd ff:ff:ff:ff:ff:ff 2024-09-21T00:29:01.175 INFO:teuthology.orchestra.run.smithi112.stdout: inet 172.21.15.112/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-21T00:29:01.175 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft 42581sec preferred_lft 42581sec 2024-09-21T00:29:01.176 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::ec4:7aff:fe8f:cb69/64 scope link noprefixroute 2024-09-21T00:29:01.176 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:29:01.176 INFO:teuthology.orchestra.run.smithi112.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T00:29:01.176 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 1a:39:64:e7:fe:d7 brd ff:ff:ff:ff:ff:ff 2024-09-21T00:29:01.176 INFO:teuthology.orchestra.run.smithi112.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T00:29:01.176 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:29:01.176 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::5858:3dff:feaf:1774/64 scope link 2024-09-21T00:29:01.176 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:29:01.176 INFO:teuthology.orchestra.run.smithi112.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T00:29:01.176 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 1a:39:64:e7:fe:d7 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T00:29:01.176 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::1839:64ff:fee7:fed7/64 scope link 2024-09-21T00:29:01.176 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:29:01.177 INFO:teuthology.orchestra.run.smithi112.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T00:29:01.177 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 86:c9:6c:ef:e4:e9 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T00:29:01.177 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::84c9:6cff:feef:e4e9/64 scope link 2024-09-21T00:29:01.177 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:29:01.177 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:29:01.177 DEBUG:teuthology.orchestra.run.smithi112:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T00:29:01.238 INFO:teuthology.orchestra.run.smithi112.stdout:1 2024-09-21T00:29:01.239 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:29:01.239 DEBUG:teuthology.orchestra.run.smithi112:> route 2024-09-21T00:29:01.296 INFO:teuthology.orchestra.run.smithi112.stdout:Kernel IP routing table 2024-09-21T00:29:01.296 INFO:teuthology.orchestra.run.smithi112.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T00:29:01.296 INFO:teuthology.orchestra.run.smithi112.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-21T00:29:01.296 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-21T00:29:01.296 INFO:teuthology.orchestra.run.smithi112.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T00:29:01.296 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T00:29:01.296 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2024-09-21T00:29:01.297 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-21T00:29:01.297 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-21T00:29:01.297 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-21T00:29:01.297 DEBUG:teuthology.orchestra.run.smithi112:> ') 2024-09-21T00:29:01.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:01 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T00:29:01.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T00:29:01.422 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:29:01.422 DEBUG:teuthology.orchestra.run.smithi112:> ip netns list 2024-09-21T00:29:01.438 INFO:teuthology.orchestra.run.smithi112.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-21T00:29:01.439 INFO:teuthology.orchestra.run.smithi112.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T00:29:01.439 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T00:29:01.439 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:29:01.439 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link set brx.0 up 2024-09-21T00:29:01.503 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T00:29:01.503 DEBUG:teuthology.orchestra.run.smithi112:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T00:29:01.559 INFO:teuthology.orchestra.run.smithi112.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T00:29:01.559 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T00:29:01.559 DEBUG:teuthology.orchestra.run.smithi112:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T00:29:01.613 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-21T00:29:01.613 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:29:01.614 DEBUG:teuthology.orchestra.run.smithi112:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T00:29:01.707 INFO:teuthology.orchestra.run.smithi112.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T00:29:01.708 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T00:29:01.708 INFO:teuthology.orchestra.run.smithi112.stdout:invalid new device string format 2024-09-21T00:29:01.718 INFO:teuthology.orchestra.run.smithi112.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T00:29:01.718 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T00:29:01.718 INFO:teuthology.orchestra.run.smithi112.stdout:invalid new device string format 2024-09-21T00:29:01.719 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: resolved to: "172.21.15.112:3300,172.21.15.187:3300,172.21.15.187:3301" 2024-09-21T00:29:01.719 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T00:29:01.725 INFO:teuthology.orchestra.run.smithi112.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T00:29:01.725 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T00:29:01.725 INFO:teuthology.orchestra.run.smithi112.stdout:invalid new device string format 2024-09-21T00:29:01.726 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: resolved to: "172.21.15.112:3300,172.21.15.187:3300,172.21.15.187:3301" 2024-09-21T00:29:01.726 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T00:29:01.726 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: resolved to: "172.21.15.112:6789,172.21.15.187:6789,172.21.15.187:6790" 2024-09-21T00:29:01.726 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.112:6789,172.21.15.187:6789,172.21.15.187:6790:/ 2024-09-21T00:29:01.726 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=8b34692b-dc65-49b2-a08f-26f707b96ac5" will pass to kernel 2024-09-21T00:29:01.727 INFO:teuthology.orchestra.run.smithi112.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-21T00:29:01.727 INFO:teuthology.orchestra.run.smithi112.stdout: You just mounted a file system that supports labels which does not 2024-09-21T00:29:01.727 INFO:teuthology.orchestra.run.smithi112.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T00:29:01.727 INFO:teuthology.orchestra.run.smithi112.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T00:29:01.727 INFO:teuthology.orchestra.run.smithi112.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T00:29:01.728 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T00:29:01.728 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:29:01.729 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T00:29:01.754 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T00:29:01.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:01 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T00:29:01.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T00:29:01.857 INFO:teuthology.orchestra.run.smithi112.stdout:client6612 2024-09-21T00:29:01.857 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T00:29:01.857 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:29:01.873 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T00:29:02.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:02.025+0000 7fbff3982640 1 -- 172.21.15.112:0/3219996498 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbfd80040b0 msgr2=0x7fbfd8004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:02.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:02.025+0000 7fbff3982640 1 --2- 172.21.15.112:0/3219996498 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbfd80040b0 0x7fbfd8004800 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fbfec0c3be0 tx=0x7fbfd8005830 comp rx=0 tx=0).stop 2024-09-21T00:29:02.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:02.025+0000 7fbff1f1f640 1 -- 172.21.15.112:0/3219996498 reap_dead start 2024-09-21T00:29:02.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:02.025+0000 7fbff3982640 1 -- 172.21.15.112:0/3219996498 shutdown_connections 2024-09-21T00:29:02.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:02.025+0000 7fbff3982640 1 -- 172.21.15.112:0/3219996498 >> 172.21.15.112:0/3219996498 conn(0x7fbfec0b3580 msgr2=0x7fbfec0b3990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:02.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:02.025+0000 7fbff3982640 1 -- 172.21.15.112:0/3219996498 shutdown_connections 2024-09-21T00:29:02.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:02.025+0000 7fbff3982640 1 -- 172.21.15.112:0/3219996498 wait complete. 2024-09-21T00:29:02.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:02.027+0000 7fbff3982640 1 Processor -- start 2024-09-21T00:29:02.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:02.027+0000 7fbff3982640 1 -- start start 2024-09-21T00:29:02.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:02.028+0000 7fbff3982640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbfd80040b0 0x7fbfec14cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:02.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:02.028+0000 7fbff3982640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbfec1562d0 0x7fbfec15e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:02.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:02.028+0000 7fbff3982640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbfec155d80 0x7fbfec15ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:02.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:02.028+0000 7fbff3982640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbfec15cfa0 con 0x7fbfd80040b0 2024-09-21T00:29:02.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:02.028+0000 7fbff3982640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fbfec15ce20 con 0x7fbfec155d80 2024-09-21T00:29:02.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:02.028+0000 7fbff3982640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fbfec15d120 con 0x7fbfec1562d0 2024-09-21T00:29:02.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:02.028+0000 7fbff171e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbfd80040b0 0x7fbfec14cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:02.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:02.028+0000 7fbff171e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbfd80040b0 0x7fbfec14cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38304/0 (socket says 172.21.15.112:38304) 2024-09-21T00:29:02.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:02.028+0000 7fbff171e640 1 -- 172.21.15.112:0/1556519583 learned_addr learned my addr 172.21.15.112:0/1556519583 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:29:02.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:02.028+0000 7fbff1f1f640 1 --2- 172.21.15.112:0/1556519583 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbfec155d80 0x7fbfec15ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:02.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:02.028+0000 7fbff0f1d640 1 --2- 172.21.15.112:0/1556519583 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbfec1562d0 0x7fbfec15e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:02.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:02.028+0000 7fbff171e640 1 -- 172.21.15.112:0/1556519583 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbfec1562d0 msgr2=0x7fbfec15e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:02.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:02.028+0000 7fbff171e640 1 --2- 172.21.15.112:0/1556519583 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbfec1562d0 0x7fbfec15e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:02.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:02.028+0000 7fbff171e640 1 -- 172.21.15.112:0/1556519583 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbfec155d80 msgr2=0x7fbfec15ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:02.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:02.028+0000 7fbff171e640 1 --2- 172.21.15.112:0/1556519583 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbfec155d80 0x7fbfec15ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:02.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:02.028+0000 7fbff171e640 1 -- 172.21.15.112:0/1556519583 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfec0c6ec0 con 0x7fbfd80040b0 2024-09-21T00:29:02.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:02.028+0000 7fbff171e640 1 --2- 172.21.15.112:0/1556519583 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbfd80040b0 0x7fbfec14cdc0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fbfe8006680 tx=0x7fbfe80039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:02.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:02.029+0000 7fbfe1ffb640 1 -- 172.21.15.112:0/1556519583 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfe8015070 con 0x7fbfd80040b0 2024-09-21T00:29:02.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:02.029+0000 7fbfe1ffb640 1 -- 172.21.15.112:0/1556519583 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbfe8011920 con 0x7fbfd80040b0 2024-09-21T00:29:02.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:02.029+0000 7fbff3982640 1 -- 172.21.15.112:0/1556519583 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfec14d300 con 0x7fbfd80040b0 2024-09-21T00:29:02.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:02.029+0000 7fbfe1ffb640 1 -- 172.21.15.112:0/1556519583 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfe8011d70 con 0x7fbfd80040b0 2024-09-21T00:29:02.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:02.029+0000 7fbff3982640 1 -- 172.21.15.112:0/1556519583 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfec0c89e0 con 0x7fbfd80040b0 2024-09-21T00:29:02.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:02.030+0000 7fbff3982640 1 -- 172.21.15.112:0/1556519583 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfec0d05b0 con 0x7fbfd80040b0 2024-09-21T00:29:02.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:02.031+0000 7fbfe1ffb640 1 -- 172.21.15.112:0/1556519583 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fbfe8018070 con 0x7fbfd80040b0 2024-09-21T00:29:02.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:02.031+0000 7fbfe1ffb640 1 --2- 172.21.15.112:0/1556519583 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fbfd0078bc0 0x7fbfd007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:02.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:02.031+0000 7fbfe1ffb640 1 -- 172.21.15.112:0/1556519583 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 6101+0+0 (secure 0 0 0) 0x7fbfe809bc90 con 0x7fbfd80040b0 2024-09-21T00:29:02.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:02.031+0000 7fbff0f1d640 1 --2- 172.21.15.112:0/1556519583 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fbfd0078bc0 0x7fbfd007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:02.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:02.032+0000 7fbff0f1d640 1 --2- 172.21.15.112:0/1556519583 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fbfd0078bc0 0x7fbfd007b0d0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fbfdc035000 tx=0x7fbfdc0046c0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:02.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:02.036+0000 7fbfe1ffb640 1 -- 172.21.15.112:0/1556519583 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fbfec0d0f00 con 0x7fbfd80040b0 2024-09-21T00:29:02.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:02.253+0000 7fbff3982640 1 -- 172.21.15.112:0/1556519583 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fbfec0cc660 con 0x7fbfd80040b0 2024-09-21T00:29:02.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:02.253+0000 7fbfe1ffb640 1 -- 172.21.15.112:0/1556519583 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v55) ==== 53+0+73 (secure 0 0 0) 0x7fbfe8063f40 con 0x7fbfd80040b0 2024-09-21T00:29:02.254 INFO:teuthology.orchestra.run.smithi112.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T00:29:02.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:02.255+0000 7fbff3982640 1 -- 172.21.15.112:0/1556519583 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fbfd0078bc0 msgr2=0x7fbfd007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:02.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:02.255+0000 7fbff3982640 1 --2- 172.21.15.112:0/1556519583 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fbfd0078bc0 0x7fbfd007b0d0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fbfdc035000 tx=0x7fbfdc0046c0 comp rx=0 tx=0).stop 2024-09-21T00:29:02.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:02.255+0000 7fbff3982640 1 -- 172.21.15.112:0/1556519583 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbfd80040b0 msgr2=0x7fbfec14cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:02.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:02.255+0000 7fbff3982640 1 --2- 172.21.15.112:0/1556519583 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbfd80040b0 0x7fbfec14cdc0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fbfe8006680 tx=0x7fbfe80039c0 comp rx=0 tx=0).stop 2024-09-21T00:29:02.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:02.255+0000 7fbff3982640 1 -- 172.21.15.112:0/1556519583 shutdown_connections 2024-09-21T00:29:02.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:02.255+0000 7fbff3982640 1 --2- 172.21.15.112:0/1556519583 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbfec155d80 0x7fbfec15ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:02.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:02.255+0000 7fbff3982640 1 --2- 172.21.15.112:0/1556519583 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbfec1562d0 0x7fbfec15e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:02.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:02.255+0000 7fbff3982640 1 --2- 172.21.15.112:0/1556519583 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fbfd0078bc0 0x7fbfd007b0d0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:02.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:02.255+0000 7fbff3982640 1 --2- 172.21.15.112:0/1556519583 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbfd80040b0 0x7fbfec14cdc0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:02.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:02.255+0000 7fbff3982640 1 -- 172.21.15.112:0/1556519583 >> 172.21.15.112:0/1556519583 conn(0x7fbfec0b3580 msgr2=0x7fbfec06dd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:02.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:02.255+0000 7fbff3982640 1 -- 172.21.15.112:0/1556519583 shutdown_connections 2024-09-21T00:29:02.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:02.255+0000 7fbff3982640 1 -- 172.21.15.112:0/1556519583 wait complete. 2024-09-21T00:29:02.267 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T00:29:02.267 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T00:29:02.267 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi187.front.sepia.ceph.com 2024-09-21T00:29:02.268 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-21T00:29:02.268 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-21T00:29:02.268 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T00:29:02.268 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-21T00:29:02.268 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T00:29:02.268 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-21T00:29:02.268 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:29:02.268 DEBUG:teuthology.orchestra.run.smithi187:> ip addr 2024-09-21T00:29:02.288 INFO:teuthology.orchestra.run.smithi187.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T00:29:02.288 INFO:teuthology.orchestra.run.smithi187.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T00:29:02.288 INFO:teuthology.orchestra.run.smithi187.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T00:29:02.289 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:29:02.289 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 ::1/128 scope host 2024-09-21T00:29:02.289 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:29:02.289 INFO:teuthology.orchestra.run.smithi187.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T00:29:02.289 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 0c:c4:7a:d9:36:78 brd ff:ff:ff:ff:ff:ff 2024-09-21T00:29:02.289 INFO:teuthology.orchestra.run.smithi187.stdout: altname enp7s0f0 2024-09-21T00:29:02.289 INFO:teuthology.orchestra.run.smithi187.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T00:29:02.289 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 0c:c4:7a:d9:36:79 brd ff:ff:ff:ff:ff:ff 2024-09-21T00:29:02.289 INFO:teuthology.orchestra.run.smithi187.stdout: altname enp7s0f1 2024-09-21T00:29:02.289 INFO:teuthology.orchestra.run.smithi187.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T00:29:02.289 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 0c:c4:7a:88:77:b0 brd ff:ff:ff:ff:ff:ff 2024-09-21T00:29:02.289 INFO:teuthology.orchestra.run.smithi187.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T00:29:02.289 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 0c:c4:7a:88:77:b1 brd ff:ff:ff:ff:ff:ff 2024-09-21T00:29:02.289 INFO:teuthology.orchestra.run.smithi187.stdout: inet 172.21.15.187/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-21T00:29:02.290 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft 42568sec preferred_lft 42568sec 2024-09-21T00:29:02.290 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 fe80::ec4:7aff:fe88:77b1/64 scope link noprefixroute 2024-09-21T00:29:02.290 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:29:02.290 INFO:teuthology.orchestra.run.smithi187.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T00:29:02.290 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 86:3b:f6:5b:1b:32 brd ff:ff:ff:ff:ff:ff 2024-09-21T00:29:02.290 INFO:teuthology.orchestra.run.smithi187.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T00:29:02.290 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:29:02.290 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 fe80::ac6f:97ff:fe09:b82e/64 scope link 2024-09-21T00:29:02.290 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:29:02.290 INFO:teuthology.orchestra.run.smithi187.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T00:29:02.290 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether ca:0b:a0:f5:2b:b5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T00:29:02.290 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 fe80::c80b:a0ff:fef5:2bb5/64 scope link 2024-09-21T00:29:02.290 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:29:02.290 INFO:teuthology.orchestra.run.smithi187.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T00:29:02.291 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 86:3b:f6:5b:1b:32 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-21T00:29:02.291 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 fe80::843b:f6ff:fe5b:1b32/64 scope link 2024-09-21T00:29:02.291 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:29:02.291 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:29:02.291 DEBUG:teuthology.orchestra.run.smithi187:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T00:29:02.357 INFO:teuthology.orchestra.run.smithi187.stdout:1 2024-09-21T00:29:02.358 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:29:02.359 DEBUG:teuthology.orchestra.run.smithi187:> route 2024-09-21T00:29:02.418 INFO:teuthology.orchestra.run.smithi187.stdout:Kernel IP routing table 2024-09-21T00:29:02.418 INFO:teuthology.orchestra.run.smithi187.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T00:29:02.418 INFO:teuthology.orchestra.run.smithi187.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-21T00:29:02.418 INFO:teuthology.orchestra.run.smithi187.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-21T00:29:02.418 INFO:teuthology.orchestra.run.smithi187.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T00:29:02.419 DEBUG:teuthology.orchestra.run.smithi187:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T00:29:02.419 DEBUG:teuthology.orchestra.run.smithi187:> set -e 2024-09-21T00:29:02.419 DEBUG:teuthology.orchestra.run.smithi187:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-21T00:29:02.419 DEBUG:teuthology.orchestra.run.smithi187:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-21T00:29:02.420 DEBUG:teuthology.orchestra.run.smithi187:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-21T00:29:02.420 DEBUG:teuthology.orchestra.run.smithi187:> ') 2024-09-21T00:29:02.509 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:29:02 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T00:29:02.542 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:29:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T00:29:02.548 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:29:02.548 DEBUG:teuthology.orchestra.run.smithi187:> ip netns list 2024-09-21T00:29:02.563 INFO:teuthology.orchestra.run.smithi187.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-21T00:29:02.563 INFO:teuthology.orchestra.run.smithi187.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-21T00:29:02.564 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-21T00:29:02.564 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:29:02.564 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip link set brx.0 up 2024-09-21T00:29:02.629 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T00:29:02.629 DEBUG:teuthology.orchestra.run.smithi187:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-21T00:29:02.684 INFO:teuthology.orchestra.run.smithi187.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-21T00:29:02.684 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T00:29:02.685 DEBUG:teuthology.orchestra.run.smithi187:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-21T00:29:02.740 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-21T00:29:02.740 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:29:02.740 DEBUG:teuthology.orchestra.run.smithi187:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-21T00:29:02.834 INFO:teuthology.orchestra.run.smithi187.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-21T00:29:02.834 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-21T00:29:02.834 INFO:teuthology.orchestra.run.smithi187.stdout:invalid new device string format 2024-09-21T00:29:02.845 INFO:teuthology.orchestra.run.smithi187.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-21T00:29:02.845 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-21T00:29:02.845 INFO:teuthology.orchestra.run.smithi187.stdout:invalid new device string format 2024-09-21T00:29:02.845 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: resolved to: "172.21.15.112:3300,172.21.15.187:3300,172.21.15.187:3301" 2024-09-21T00:29:02.845 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T00:29:02.854 INFO:teuthology.orchestra.run.smithi187.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-21T00:29:02.854 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-21T00:29:02.854 INFO:teuthology.orchestra.run.smithi187.stdout:invalid new device string format 2024-09-21T00:29:02.854 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: resolved to: "172.21.15.112:3300,172.21.15.187:3300,172.21.15.187:3301" 2024-09-21T00:29:02.854 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T00:29:02.854 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: resolved to: "172.21.15.112:6789,172.21.15.187:6789,172.21.15.187:6790" 2024-09-21T00:29:02.854 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.112:6789,172.21.15.187:6789,172.21.15.187:6790:/ 2024-09-21T00:29:02.854 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=8b34692b-dc65-49b2-a08f-26f707b96ac5" will pass to kernel 2024-09-21T00:29:02.860 INFO:teuthology.orchestra.run.smithi187.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-09-21T00:29:02.860 INFO:teuthology.orchestra.run.smithi187.stdout: You just mounted a file system that supports labels which does not 2024-09-21T00:29:02.860 INFO:teuthology.orchestra.run.smithi187.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T00:29:02.860 INFO:teuthology.orchestra.run.smithi187.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T00:29:02.860 INFO:teuthology.orchestra.run.smithi187.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T00:29:02.860 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T00:29:02.860 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:29:02.860 DEBUG:teuthology.orchestra.run.smithi187:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-21T00:29:02.886 DEBUG:teuthology.orchestra.run.smithi187:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T00:29:02.974 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:29:02 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T00:29:02.976 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:29:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T00:29:02.982 INFO:teuthology.orchestra.run.smithi187.stdout:client6058 2024-09-21T00:29:02.982 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-09-21T00:29:02.983 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:29:02.998 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds ms_async_reap_threshold 1 2024-09-21T00:29:03.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:03.123+0000 7f619c68c640 1 -- 172.21.15.112:0/3763564574 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6180004d90 msgr2=0x7f6180005260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:03.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:03.123+0000 7f619c68c640 1 --2- 172.21.15.112:0/3763564574 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6180004d90 0x7f6180005260 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f618c006fd0 tx=0x7f6180005830 comp rx=0 tx=0).stop 2024-09-21T00:29:03.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:03.123+0000 7f619ac29640 1 -- 172.21.15.112:0/3763564574 reap_dead start 2024-09-21T00:29:03.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:03.123+0000 7f619c68c640 1 -- 172.21.15.112:0/3763564574 shutdown_connections 2024-09-21T00:29:03.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:03.123+0000 7f619c68c640 1 -- 172.21.15.112:0/3763564574 >> 172.21.15.112:0/3763564574 conn(0x7f61940cf220 msgr2=0x7f61940cf630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:03.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:03.123+0000 7f619c68c640 1 -- 172.21.15.112:0/3763564574 shutdown_connections 2024-09-21T00:29:03.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:03.123+0000 7f619c68c640 1 -- 172.21.15.112:0/3763564574 wait complete. 2024-09-21T00:29:03.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:03.124+0000 7f619c68c640 1 Processor -- start 2024-09-21T00:29:03.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:03.125+0000 7f619c68c640 1 -- start start 2024-09-21T00:29:03.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:03.126+0000 7f619c68c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6180004d90 0x7f619414a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:03.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:03.126+0000 7f619c68c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f61941527c0 0x7f61940c9330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:03.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:03.126+0000 7f619c68c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6194155f00 0x7f61940c9870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:03.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:03.126+0000 7f619c68c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f61940d2130 con 0x7f61941527c0 2024-09-21T00:29:03.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:03.126+0000 7f619c68c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f61940d1fb0 con 0x7f6194155f00 2024-09-21T00:29:03.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:03.126+0000 7f619c68c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f61940b26e0 con 0x7f6180004d90 2024-09-21T00:29:03.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:03.126+0000 7f6199c27640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f61941527c0 0x7f61940c9330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:03.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:03.126+0000 7f6199c27640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f61941527c0 0x7f61940c9330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38314/0 (socket says 172.21.15.112:38314) 2024-09-21T00:29:03.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:03.126+0000 7f6199c27640 1 -- 172.21.15.112:0/536539605 learned_addr learned my addr 172.21.15.112:0/536539605 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:29:03.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:03.126+0000 7f619ac29640 1 --2- 172.21.15.112:0/536539605 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6194155f00 0x7f61940c9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:03.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:03.126+0000 7f619a428640 1 --2- 172.21.15.112:0/536539605 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6180004d90 0x7f619414a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:03.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:03.126+0000 7f6199c27640 1 -- 172.21.15.112:0/536539605 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6180004d90 msgr2=0x7f619414a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:03.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:03.126+0000 7f6199c27640 1 --2- 172.21.15.112:0/536539605 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6180004d90 0x7f619414a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:03.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:03.126+0000 7f6199c27640 1 -- 172.21.15.112:0/536539605 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6194155f00 msgr2=0x7f61940c9870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:03.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:03.126+0000 7f6199c27640 1 --2- 172.21.15.112:0/536539605 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6194155f00 0x7f61940c9870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:03.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:03.126+0000 7f6199c27640 1 -- 172.21.15.112:0/536539605 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61940b6be0 con 0x7f61941527c0 2024-09-21T00:29:03.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:03.126+0000 7f619ac29640 1 --2- 172.21.15.112:0/536539605 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6194155f00 0x7f61940c9870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:29:03.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:03.126+0000 7f619a428640 1 --2- 172.21.15.112:0/536539605 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6180004d90 0x7f619414a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:29:03.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:03.126+0000 7f6199c27640 1 --2- 172.21.15.112:0/536539605 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f61941527c0 0x7f61940c9330 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f6184010aa0 tx=0x7f6184010f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:03.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:03.126+0000 7f618affd640 1 -- 172.21.15.112:0/536539605 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f618400ebc0 con 0x7f61941527c0 2024-09-21T00:29:03.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:03.127+0000 7f618affd640 1 -- 172.21.15.112:0/536539605 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f618400ed60 con 0x7f61941527c0 2024-09-21T00:29:03.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:03.127+0000 7f619c68c640 1 -- 172.21.15.112:0/536539605 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61940b6ec0 con 0x7f61941527c0 2024-09-21T00:29:03.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:03.127+0000 7f618affd640 1 -- 172.21.15.112:0/536539605 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61840116a0 con 0x7f61941527c0 2024-09-21T00:29:03.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:03.127+0000 7f619c68c640 1 -- 172.21.15.112:0/536539605 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61941559e0 con 0x7f61941527c0 2024-09-21T00:29:03.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:03.128+0000 7f619c68c640 1 -- 172.21.15.112:0/536539605 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61940b1fd0 con 0x7f61941527c0 2024-09-21T00:29:03.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:03.128+0000 7f618affd640 1 -- 172.21.15.112:0/536539605 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6184011840 con 0x7f61941527c0 2024-09-21T00:29:03.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:03.129+0000 7f618affd640 1 --2- 172.21.15.112:0/536539605 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f6178078bc0 0x7f617807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:03.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:03.129+0000 7f618affd640 1 -- 172.21.15.112:0/536539605 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 6101+0+0 (secure 0 0 0) 0x7f618409bb50 con 0x7f61941527c0 2024-09-21T00:29:03.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:03.129+0000 7f619a428640 1 --2- 172.21.15.112:0/536539605 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f6178078bc0 0x7f617807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:03.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:03.129+0000 7f619a428640 1 --2- 172.21.15.112:0/536539605 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f6178078bc0 0x7f617807b0d0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f619000bc30 tx=0x7f619000b2e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:03.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:03.133+0000 7f618affd640 1 -- 172.21.15.112:0/536539605 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f6184063e00 con 0x7f61941527c0 2024-09-21T00:29:03.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:03.310+0000 7f619c68c640 1 -- 172.21.15.112:0/536539605 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command([{prefix=config set, name=ms_async_reap_threshold}] v 0) -- 0x7f6194072250 con 0x7f61941527c0 2024-09-21T00:29:03.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:03.316+0000 7f618affd640 1 -- 172.21.15.112:0/536539605 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{prefix=config set, name=ms_async_reap_threshold}]=0 v43)=0 v43) ==== 121+0+0 (secure 0 0 0) 0x7f6184063700 con 0x7f61941527c0 2024-09-21T00:29:03.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:03.318+0000 7f619c68c640 1 -- 172.21.15.112:0/536539605 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f6178078bc0 msgr2=0x7f617807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:03.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:03.318+0000 7f619c68c640 1 --2- 172.21.15.112:0/536539605 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f6178078bc0 0x7f617807b0d0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f619000bc30 tx=0x7f619000b2e0 comp rx=0 tx=0).stop 2024-09-21T00:29:03.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:03.318+0000 7f619c68c640 1 -- 172.21.15.112:0/536539605 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f61941527c0 msgr2=0x7f61940c9330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:03.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:03.318+0000 7f619c68c640 1 --2- 172.21.15.112:0/536539605 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f61941527c0 0x7f61940c9330 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f6184010aa0 tx=0x7f6184010f60 comp rx=0 tx=0).stop 2024-09-21T00:29:03.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:03.318+0000 7f619c68c640 1 -- 172.21.15.112:0/536539605 shutdown_connections 2024-09-21T00:29:03.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:03.318+0000 7f619c68c640 1 --2- 172.21.15.112:0/536539605 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6194155f00 0x7f61940c9870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:03.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:03.318+0000 7f619c68c640 1 --2- 172.21.15.112:0/536539605 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f6178078bc0 0x7f617807b0d0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:03.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:03.318+0000 7f619c68c640 1 --2- 172.21.15.112:0/536539605 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f61941527c0 0x7f61940c9330 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:03.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:03.318+0000 7f619c68c640 1 --2- 172.21.15.112:0/536539605 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6180004d90 0x7f619414a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:03.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:03.318+0000 7f619c68c640 1 -- 172.21.15.112:0/536539605 >> 172.21.15.112:0/536539605 conn(0x7f61940cf220 msgr2=0x7f619415ab30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:03.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:03.318+0000 7f619c68c640 1 -- 172.21.15.112:0/536539605 shutdown_connections 2024-09-21T00:29:03.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:03.319+0000 7f619c68c640 1 -- 172.21.15.112:0/536539605 wait complete. 2024-09-21T00:29:03.331 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T00:29:03.331 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:29:03.347 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T00:29:03.347 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:29:03.403 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T00:29:03.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:03.555+0000 7f8cb37e6640 1 -- 172.21.15.112:0/2092787887 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8cac0ccae0 msgr2=0x7f8cac0ccef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:03.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:03.555+0000 7f8cb37e6640 1 --2- 172.21.15.112:0/2092787887 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8cac0ccae0 0x7f8cac0ccef0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f8cac0b4f40 tx=0x7f8ca402aa30 comp rx=0 tx=0).stop 2024-09-21T00:29:03.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:03.555+0000 7f8cb37e6640 1 -- 172.21.15.112:0/2092787887 shutdown_connections 2024-09-21T00:29:03.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:03.555+0000 7f8cb37e6640 1 --2- 172.21.15.112:0/2092787887 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8cac0ccae0 0x7f8cac0ccef0 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:03.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:03.555+0000 7f8cb37e6640 1 --2- 172.21.15.112:0/2092787887 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8cac0d0860 0x7f8cac15a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:03.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:03.555+0000 7f8cb37e6640 1 --2- 172.21.15.112:0/2092787887 >> v2:172.21.15.187:3300/0 conn(0x7f8cac0d2030 0x7f8cac15d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:03.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:03.555+0000 7f8cb37e6640 1 -- 172.21.15.112:0/2092787887 >> 172.21.15.112:0/2092787887 conn(0x7f8cac0b43f0 msgr2=0x7f8cac0b4830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:03.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:03.555+0000 7f8cb37e6640 1 -- 172.21.15.112:0/2092787887 shutdown_connections 2024-09-21T00:29:03.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:03.555+0000 7f8cb37e6640 1 -- 172.21.15.112:0/2092787887 wait complete. 2024-09-21T00:29:03.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:03.557+0000 7f8cb37e6640 1 Processor -- start 2024-09-21T00:29:03.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:03.557+0000 7f8cb37e6640 1 -- start start 2024-09-21T00:29:03.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:03.558+0000 7f8cb37e6640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8cac0ccae0 0x7f8cac152040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:03.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:03.558+0000 7f8cb37e6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8cac0d0860 0x7f8cac155450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:03.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:03.558+0000 7f8cb37e6640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8cac0d2030 0x7f8cac155990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:03.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:03.558+0000 7f8cb37e6640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8cac160700 con 0x7f8cac0d0860 2024-09-21T00:29:03.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:03.558+0000 7f8cb37e6640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8cac160580 con 0x7f8cac0d2030 2024-09-21T00:29:03.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:03.558+0000 7f8cb37e6640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f8cac160880 con 0x7f8cac0ccae0 2024-09-21T00:29:03.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:03.558+0000 7f8cb0d81640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8cac0d0860 0x7f8cac155450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:03.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:03.558+0000 7f8cb0d81640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8cac0d0860 0x7f8cac155450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38342/0 (socket says 172.21.15.112:38342) 2024-09-21T00:29:03.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:03.558+0000 7f8cb0d81640 1 -- 172.21.15.112:0/2333121952 learned_addr learned my addr 172.21.15.112:0/2333121952 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:29:03.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:03.558+0000 7f8cb1d83640 1 --2- 172.21.15.112:0/2333121952 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8cac0d2030 0x7f8cac155990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:03.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:03.558+0000 7f8cb1582640 1 --2- 172.21.15.112:0/2333121952 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8cac0ccae0 0x7f8cac152040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:03.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:03.558+0000 7f8cb0d81640 1 -- 172.21.15.112:0/2333121952 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8cac0ccae0 msgr2=0x7f8cac152040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:03.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:03.558+0000 7f8cb0d81640 1 --2- 172.21.15.112:0/2333121952 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8cac0ccae0 0x7f8cac152040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:03.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:03.558+0000 7f8cb0d81640 1 -- 172.21.15.112:0/2333121952 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8cac0d2030 msgr2=0x7f8cac155990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:03.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:03.558+0000 7f8cb0d81640 1 --2- 172.21.15.112:0/2333121952 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8cac0d2030 0x7f8cac155990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:03.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:03.558+0000 7f8cb0d81640 1 -- 172.21.15.112:0/2333121952 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cac0cb8a0 con 0x7f8cac0d0860 2024-09-21T00:29:03.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:03.558+0000 7f8cb1582640 1 --2- 172.21.15.112:0/2333121952 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8cac0ccae0 0x7f8cac152040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:29:03.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:03.558+0000 7f8cb0d81640 1 --2- 172.21.15.112:0/2333121952 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8cac0d0860 0x7f8cac155450 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c00b440 tx=0x7f8c9c00b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:03.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:03.559+0000 7f8ca1ffb640 1 -- 172.21.15.112:0/2333121952 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c9c015020 con 0x7f8cac0d0860 2024-09-21T00:29:03.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:03.559+0000 7f8ca1ffb640 1 -- 172.21.15.112:0/2333121952 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8c9c011980 con 0x7f8cac0d0860 2024-09-21T00:29:03.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:03.559+0000 7f8cb37e6640 1 -- 172.21.15.112:0/2333121952 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cac157820 con 0x7f8cac0d0860 2024-09-21T00:29:03.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:03.559+0000 7f8ca1ffb640 1 -- 172.21.15.112:0/2333121952 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c9c011dd0 con 0x7f8cac0d0860 2024-09-21T00:29:03.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:03.559+0000 7f8cb37e6640 1 -- 172.21.15.112:0/2333121952 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cac157db0 con 0x7f8cac0d0860 2024-09-21T00:29:03.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:03.560+0000 7f8cb37e6640 1 -- 172.21.15.112:0/2333121952 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8cac15a0c0 con 0x7f8cac0d0860 2024-09-21T00:29:03.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:03.560+0000 7f8ca1ffb640 1 -- 172.21.15.112:0/2333121952 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8c9c018020 con 0x7f8cac0d0860 2024-09-21T00:29:03.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:03.561+0000 7f8ca1ffb640 1 --2- 172.21.15.112:0/2333121952 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f8c90078bc0 0x7f8c9007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:03.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:03.561+0000 7f8ca1ffb640 1 -- 172.21.15.112:0/2333121952 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 6101+0+0 (secure 0 0 0) 0x7f8c9c09ae90 con 0x7f8cac0d0860 2024-09-21T00:29:03.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:03.561+0000 7f8cb1582640 1 --2- 172.21.15.112:0/2333121952 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f8c90078bc0 0x7f8c9007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:03.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:03.561+0000 7f8cb1582640 1 --2- 172.21.15.112:0/2333121952 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f8c90078bc0 0x7f8c9007b0d0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f8ca8005fd0 tx=0x7f8ca8005e20 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:03.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:03.565+0000 7f8ca1ffb640 1 -- 172.21.15.112:0/2333121952 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f8c9c01f020 con 0x7f8cac0d0860 2024-09-21T00:29:03.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:03.753+0000 7f8cb37e6640 1 -- 172.21.15.112:0/2333121952 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f8cac0b2cc0 con 0x7f8cac0d0860 2024-09-21T00:29:03.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:03.753+0000 7f8ca1ffb640 1 -- 172.21.15.112:0/2333121952 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v83) ==== 80+0+0 (secure 0 0 0) 0x7f8c9c00d040 con 0x7f8cac0d0860 2024-09-21T00:29:03.754 INFO:teuthology.orchestra.run.smithi112.stderr:listed 0 entries 2024-09-21T00:29:03.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:03.756+0000 7f8cb37e6640 1 -- 172.21.15.112:0/2333121952 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f8c90078bc0 msgr2=0x7f8c9007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:03.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:03.756+0000 7f8cb37e6640 1 --2- 172.21.15.112:0/2333121952 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f8c90078bc0 0x7f8c9007b0d0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f8ca8005fd0 tx=0x7f8ca8005e20 comp rx=0 tx=0).stop 2024-09-21T00:29:03.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:03.756+0000 7f8cb37e6640 1 -- 172.21.15.112:0/2333121952 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8cac0d0860 msgr2=0x7f8cac155450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:03.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:03.756+0000 7f8cb37e6640 1 --2- 172.21.15.112:0/2333121952 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8cac0d0860 0x7f8cac155450 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c00b440 tx=0x7f8c9c00b900 comp rx=0 tx=0).stop 2024-09-21T00:29:03.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:03.756+0000 7f8cb37e6640 1 -- 172.21.15.112:0/2333121952 shutdown_connections 2024-09-21T00:29:03.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:03.756+0000 7f8cb37e6640 1 --2- 172.21.15.112:0/2333121952 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8cac0d2030 0x7f8cac155990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:03.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:03.756+0000 7f8cb37e6640 1 --2- 172.21.15.112:0/2333121952 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f8c90078bc0 0x7f8c9007b0d0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:03.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:03.756+0000 7f8cb37e6640 1 --2- 172.21.15.112:0/2333121952 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8cac0d0860 0x7f8cac155450 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:03.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:03.756+0000 7f8cb37e6640 1 --2- 172.21.15.112:0/2333121952 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8cac0ccae0 0x7f8cac152040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:03.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:03.756+0000 7f8cb37e6640 1 -- 172.21.15.112:0/2333121952 >> 172.21.15.112:0/2333121952 conn(0x7f8cac0b43f0 msgr2=0x7f8cac15a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:03.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:03.756+0000 7f8cb37e6640 1 -- 172.21.15.112:0/2333121952 shutdown_connections 2024-09-21T00:29:03.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:03.756+0000 7f8cb37e6640 1 -- 172.21.15.112:0/2333121952 wait complete. 2024-09-21T00:29:03.768 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-21T00:29:03.768 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:29:03.769 DEBUG:teuthology.orchestra.run.smithi112:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-21T00:29:03.797 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi112.front.sepia.ceph.com 2024-09-21T00:29:03.797 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:29:03.798 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T00:29:03.853 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-09-21T00:29:03.853 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:29:03.870 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-09-21T00:29:03.871 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:29:03.928 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T00:29:04.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.041+0000 7f96b18d5640 1 -- 172.21.15.112:0/1825716325 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96ac15ea40 msgr2=0x7f96ac070250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:04.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.041+0000 7f96b18d5640 1 --2- 172.21.15.112:0/1825716325 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96ac15ea40 0x7f96ac070250 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f96ac075460 tx=0x7f96a402a480 comp rx=0 tx=0).stop 2024-09-21T00:29:04.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.041+0000 7f96b18d5640 1 -- 172.21.15.112:0/1825716325 shutdown_connections 2024-09-21T00:29:04.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.041+0000 7f96b18d5640 1 --2- 172.21.15.112:0/1825716325 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96ac15ea40 0x7f96ac070250 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:04.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.041+0000 7f96b18d5640 1 --2- 172.21.15.112:0/1825716325 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f96ac0d4a90 0x7f96ac0d38c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:04.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.041+0000 7f96b18d5640 1 -- 172.21.15.112:0/1825716325 >> 172.21.15.112:0/1825716325 conn(0x7f96ac09d8a0 msgr2=0x7f96ac08bfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:04.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.041+0000 7f96b18d5640 1 -- 172.21.15.112:0/1825716325 shutdown_connections 2024-09-21T00:29:04.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.041+0000 7f96b18d5640 1 -- 172.21.15.112:0/1825716325 wait complete. 2024-09-21T00:29:04.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.043+0000 7f96b18d5640 1 Processor -- start 2024-09-21T00:29:04.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.043+0000 7f96b18d5640 1 -- start start 2024-09-21T00:29:04.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.044+0000 7f96b18d5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96ac0d0ae0 0x7f96ac15a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:04.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.044+0000 7f96b18d5640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f96ac0d4a90 0x7f96ac06cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:04.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.044+0000 7f96b18d5640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f96ac15ea40 0x7f96ac06d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:04.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.044+0000 7f96b18d5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f96ac162b50 con 0x7f96ac0d0ae0 2024-09-21T00:29:04.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.044+0000 7f96b18d5640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f96ac1629d0 con 0x7f96ac0d4a90 2024-09-21T00:29:04.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.044+0000 7f96b18d5640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f96ac162cd0 con 0x7f96ac15ea40 2024-09-21T00:29:04.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.044+0000 7f96aaffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96ac0d0ae0 0x7f96ac15a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:04.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.044+0000 7f96aaffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96ac0d0ae0 0x7f96ac15a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38350/0 (socket says 172.21.15.112:38350) 2024-09-21T00:29:04.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.044+0000 7f96aaffd640 1 -- 172.21.15.112:0/355382376 learned_addr learned my addr 172.21.15.112:0/355382376 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:29:04.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.044+0000 7f96ab7fe640 1 --2- 172.21.15.112:0/355382376 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f96ac15ea40 0x7f96ac06d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:04.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.044+0000 7f96aa7fc640 1 --2- 172.21.15.112:0/355382376 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f96ac0d4a90 0x7f96ac06cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:04.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.044+0000 7f96aaffd640 1 -- 172.21.15.112:0/355382376 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f96ac15ea40 msgr2=0x7f96ac06d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:04.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.044+0000 7f96aaffd640 1 --2- 172.21.15.112:0/355382376 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f96ac15ea40 0x7f96ac06d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:04.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.044+0000 7f96aaffd640 1 -- 172.21.15.112:0/355382376 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f96ac0d4a90 msgr2=0x7f96ac06cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:04.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.044+0000 7f96aaffd640 1 --2- 172.21.15.112:0/355382376 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f96ac0d4a90 0x7f96ac06cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:04.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.044+0000 7f96aaffd640 1 -- 172.21.15.112:0/355382376 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96ac06fc70 con 0x7f96ac0d0ae0 2024-09-21T00:29:04.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.045+0000 7f96aaffd640 1 --2- 172.21.15.112:0/355382376 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96ac0d0ae0 0x7f96ac15a040 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f969c00ba80 tx=0x7f969c00bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:04.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.045+0000 7f9693fff640 1 -- 172.21.15.112:0/355382376 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f969c00cc00 con 0x7f96ac0d0ae0 2024-09-21T00:29:04.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.045+0000 7f9693fff640 1 -- 172.21.15.112:0/355382376 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f969c0039c0 con 0x7f96ac0d0ae0 2024-09-21T00:29:04.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.045+0000 7f96b18d5640 1 -- 172.21.15.112:0/355382376 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96ac0cc4a0 con 0x7f96ac0d0ae0 2024-09-21T00:29:04.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.045+0000 7f9693fff640 1 -- 172.21.15.112:0/355382376 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f969c003e50 con 0x7f96ac0d0ae0 2024-09-21T00:29:04.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.045+0000 7f96b18d5640 1 -- 172.21.15.112:0/355382376 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96ac06ff50 con 0x7f96ac0d0ae0 2024-09-21T00:29:04.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.046+0000 7f96b18d5640 1 -- 172.21.15.112:0/355382376 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96ac163210 con 0x7f96ac0d0ae0 2024-09-21T00:29:04.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.047+0000 7f9693fff640 1 -- 172.21.15.112:0/355382376 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f969c01b020 con 0x7f96ac0d0ae0 2024-09-21T00:29:04.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.047+0000 7f9693fff640 1 --2- 172.21.15.112:0/355382376 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f9698078d30 0x7f969807b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:04.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.047+0000 7f9693fff640 1 -- 172.21.15.112:0/355382376 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 6101+0+0 (secure 0 0 0) 0x7f969c09af60 con 0x7f96ac0d0ae0 2024-09-21T00:29:04.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.047+0000 7f96aa7fc640 1 --2- 172.21.15.112:0/355382376 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f9698078d30 0x7f969807b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:04.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.048+0000 7f96aa7fc640 1 --2- 172.21.15.112:0/355382376 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f9698078d30 0x7f969807b240 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f96a0005fd0 tx=0x7f96a0005d80 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:04.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.051+0000 7f9693fff640 1 -- 172.21.15.112:0/355382376 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f969c0632a0 con 0x7f96ac0d0ae0 2024-09-21T00:29:04.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.240+0000 7f96b18d5640 1 -- 172.21.15.112:0/355382376 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f96ac0d43f0 con 0x7f96ac0d0ae0 2024-09-21T00:29:04.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.240+0000 7f9693fff640 1 -- 172.21.15.112:0/355382376 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v83) ==== 80+0+0 (secure 0 0 0) 0x7f969c062ba0 con 0x7f96ac0d0ae0 2024-09-21T00:29:04.241 INFO:teuthology.orchestra.run.smithi112.stderr:listed 0 entries 2024-09-21T00:29:04.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.243+0000 7f96b18d5640 1 -- 172.21.15.112:0/355382376 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f9698078d30 msgr2=0x7f969807b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:04.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.243+0000 7f96b18d5640 1 --2- 172.21.15.112:0/355382376 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f9698078d30 0x7f969807b240 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f96a0005fd0 tx=0x7f96a0005d80 comp rx=0 tx=0).stop 2024-09-21T00:29:04.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.243+0000 7f96b18d5640 1 -- 172.21.15.112:0/355382376 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96ac0d0ae0 msgr2=0x7f96ac15a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:04.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.243+0000 7f96b18d5640 1 --2- 172.21.15.112:0/355382376 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96ac0d0ae0 0x7f96ac15a040 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f969c00ba80 tx=0x7f969c00bf40 comp rx=0 tx=0).stop 2024-09-21T00:29:04.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.243+0000 7f96b18d5640 1 -- 172.21.15.112:0/355382376 shutdown_connections 2024-09-21T00:29:04.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.243+0000 7f96b18d5640 1 --2- 172.21.15.112:0/355382376 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f96ac15ea40 0x7f96ac06d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:04.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.243+0000 7f96b18d5640 1 --2- 172.21.15.112:0/355382376 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f96ac0d4a90 0x7f96ac06cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:04.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.243+0000 7f96b18d5640 1 --2- 172.21.15.112:0/355382376 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f9698078d30 0x7f969807b240 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:04.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.243+0000 7f96b18d5640 1 --2- 172.21.15.112:0/355382376 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96ac0d0ae0 0x7f96ac15a040 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:04.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.243+0000 7f96b18d5640 1 -- 172.21.15.112:0/355382376 >> 172.21.15.112:0/355382376 conn(0x7f96ac09d8a0 msgr2=0x7f96ac15efd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:04.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.243+0000 7f96b18d5640 1 -- 172.21.15.112:0/355382376 shutdown_connections 2024-09-21T00:29:04.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.243+0000 7f96b18d5640 1 -- 172.21.15.112:0/355382376 wait complete. 2024-09-21T00:29:04.255 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-09-21T00:29:04.255 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:29:04.255 DEBUG:teuthology.orchestra.run.smithi187:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-09-21T00:29:04.285 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi187.front.sepia.ceph.com 2024-09-21T00:29:04.285 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:29:04.285 DEBUG:teuthology.orchestra.run.smithi187:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-21T00:29:04.342 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T00:29:04.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.454+0000 7f7aa9230640 1 -- 172.21.15.112:0/1708070250 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7aa40c6a00 msgr2=0x7f7aa40b2d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:04.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.454+0000 7f7aa9230640 1 --2- 172.21.15.112:0/1708070250 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7aa40c6a00 0x7f7aa40b2d40 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c009830 tx=0x7f7a9c02a410 comp rx=0 tx=0).stop 2024-09-21T00:29:04.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.454+0000 7f7aa9230640 1 -- 172.21.15.112:0/1708070250 shutdown_connections 2024-09-21T00:29:04.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.454+0000 7f7aa9230640 1 --2- 172.21.15.112:0/1708070250 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7aa40c6a00 0x7f7aa40b2d40 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:04.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.454+0000 7f7aa9230640 1 --2- 172.21.15.112:0/1708070250 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7aa40ccbf0 0x7f7aa40cd000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:04.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.454+0000 7f7aa9230640 1 -- 172.21.15.112:0/1708070250 >> 172.21.15.112:0/1708070250 conn(0x7f7aa40b42e0 msgr2=0x7f7aa40b4720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:04.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.454+0000 7f7aa9230640 1 -- 172.21.15.112:0/1708070250 shutdown_connections 2024-09-21T00:29:04.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.454+0000 7f7aa9230640 1 -- 172.21.15.112:0/1708070250 wait complete. 2024-09-21T00:29:04.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.456+0000 7f7aa9230640 1 Processor -- start 2024-09-21T00:29:04.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.456+0000 7f7aa9230640 1 -- start start 2024-09-21T00:29:04.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.457+0000 7f7aa9230640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7aa40c5ee0 0x7f7aa40caab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:04.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.457+0000 7f7aa9230640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7aa40c6a00 0x7f7aa41f6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:04.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.457+0000 7f7aa9230640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7aa40ccbf0 0x7f7aa40b6840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:04.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.457+0000 7f7aa9230640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa40b3400 con 0x7f7aa40c6a00 2024-09-21T00:29:04.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.457+0000 7f7aa9230640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa40b3280 con 0x7f7aa40ccbf0 2024-09-21T00:29:04.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.457+0000 7f7aa9230640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f7aa40b3580 con 0x7f7aa40c5ee0 2024-09-21T00:29:04.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.457+0000 7f7aa259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7aa40c6a00 0x7f7aa41f6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:04.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.457+0000 7f7aa259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7aa40c6a00 0x7f7aa41f6250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38366/0 (socket says 172.21.15.112:38366) 2024-09-21T00:29:04.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.457+0000 7f7aa259c640 1 -- 172.21.15.112:0/2778336398 learned_addr learned my addr 172.21.15.112:0/2778336398 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:29:04.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.457+0000 7f7aa359e640 1 --2- 172.21.15.112:0/2778336398 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7aa40ccbf0 0x7f7aa40b6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:04.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.457+0000 7f7aa2d9d640 1 --2- 172.21.15.112:0/2778336398 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7aa40c5ee0 0x7f7aa40caab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:04.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.458+0000 7f7aa259c640 1 -- 172.21.15.112:0/2778336398 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7aa40c5ee0 msgr2=0x7f7aa40caab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:04.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.458+0000 7f7aa259c640 1 --2- 172.21.15.112:0/2778336398 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7aa40c5ee0 0x7f7aa40caab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:04.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.458+0000 7f7aa259c640 1 -- 172.21.15.112:0/2778336398 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7aa40ccbf0 msgr2=0x7f7aa40b6840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:04.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.458+0000 7f7aa259c640 1 --2- 172.21.15.112:0/2778336398 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7aa40ccbf0 0x7f7aa40b6840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:04.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.458+0000 7f7aa259c640 1 -- 172.21.15.112:0/2778336398 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7aa406ea60 con 0x7f7aa40c6a00 2024-09-21T00:29:04.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.458+0000 7f7aa259c640 1 --2- 172.21.15.112:0/2778336398 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7aa40c6a00 0x7f7aa41f6250 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f7a9800c500 tx=0x7f7a9800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:04.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.458+0000 7f7a8b7fe640 1 -- 172.21.15.112:0/2778336398 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a98016020 con 0x7f7aa40c6a00 2024-09-21T00:29:04.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.458+0000 7f7a8b7fe640 1 -- 172.21.15.112:0/2778336398 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7a98012940 con 0x7f7aa40c6a00 2024-09-21T00:29:04.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.458+0000 7f7aa9230640 1 -- 172.21.15.112:0/2778336398 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7aa406ed40 con 0x7f7aa40c6a00 2024-09-21T00:29:04.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.458+0000 7f7a8b7fe640 1 -- 172.21.15.112:0/2778336398 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a98012d90 con 0x7f7aa40c6a00 2024-09-21T00:29:04.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.458+0000 7f7aa9230640 1 -- 172.21.15.112:0/2778336398 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7aa406f300 con 0x7f7aa40c6a00 2024-09-21T00:29:04.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.459+0000 7f7aa9230640 1 -- 172.21.15.112:0/2778336398 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7aa406e7d0 con 0x7f7aa40c6a00 2024-09-21T00:29:04.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.460+0000 7f7a8b7fe640 1 -- 172.21.15.112:0/2778336398 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f7a98019020 con 0x7f7aa40c6a00 2024-09-21T00:29:04.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.460+0000 7f7a8b7fe640 1 --2- 172.21.15.112:0/2778336398 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f7a90078b90 0x7f7a9007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:04.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.461+0000 7f7a8b7fe640 1 -- 172.21.15.112:0/2778336398 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 6101+0+0 (secure 0 0 0) 0x7f7a9809ae40 con 0x7f7aa40c6a00 2024-09-21T00:29:04.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.461+0000 7f7aa2d9d640 1 --2- 172.21.15.112:0/2778336398 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f7a90078b90 0x7f7a9007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:04.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.461+0000 7f7aa2d9d640 1 --2- 172.21.15.112:0/2778336398 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f7a90078b90 0x7f7a9007b0a0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f7a940099a0 tx=0x7f7a94009500 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:04.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.465+0000 7f7a8b7fe640 1 -- 172.21.15.112:0/2778336398 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f7a98020020 con 0x7f7aa40c6a00 2024-09-21T00:29:04.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.679+0000 7f7aa9230640 1 -- 172.21.15.112:0/2778336398 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7aa40cd000 con 0x7f7aa40c6a00 2024-09-21T00:29:04.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.679+0000 7f7a8b7fe640 1 -- 172.21.15.112:0/2778336398 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) ==== 94+0+5231 (secure 0 0 0) 0x7f7aa40cd540 con 0x7f7aa40c6a00 2024-09-21T00:29:04.681 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:29:04.681 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":55,"btime":"2024-09-21T00:28:58:105880+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5600,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6836/2708152171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":2708152171},{"type":"v1","addr":"172.21.15.187:6836","nonce":2708152171}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50},{"gid":6471,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6834/3529228075","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":3529228075},{"type":"v1","addr":"172.21.15.112:6834","nonce":3529228075}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50},{"gid":6474,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/101289292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":101289292},{"type":"v1","addr":"172.21.15.112:6835","nonce":101289292}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T00:28:54.453199+0000","modified":"2024-09-21T00:28:57.469605+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6486},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6486":{"gid":6486,"name":"d","rank":0,"incarnation":51,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6837/932692802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6835","nonce":932692802},{"type":"v1","addr":"172.21.15.187:6837","nonce":932692802}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6486,"qdb_cluster":[6486]},"id":6}]} 2024-09-21T00:29:04.681 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 55 2024-09-21T00:29:04.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.682+0000 7f7aa9230640 1 -- 172.21.15.112:0/2778336398 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f7a90078b90 msgr2=0x7f7a9007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:04.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.682+0000 7f7aa9230640 1 --2- 172.21.15.112:0/2778336398 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f7a90078b90 0x7f7a9007b0a0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f7a940099a0 tx=0x7f7a94009500 comp rx=0 tx=0).stop 2024-09-21T00:29:04.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.682+0000 7f7aa9230640 1 -- 172.21.15.112:0/2778336398 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7aa40c6a00 msgr2=0x7f7aa41f6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:04.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.682+0000 7f7aa9230640 1 --2- 172.21.15.112:0/2778336398 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7aa40c6a00 0x7f7aa41f6250 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f7a9800c500 tx=0x7f7a9800c9c0 comp rx=0 tx=0).stop 2024-09-21T00:29:04.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.682+0000 7f7aa9230640 1 -- 172.21.15.112:0/2778336398 shutdown_connections 2024-09-21T00:29:04.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.682+0000 7f7aa9230640 1 --2- 172.21.15.112:0/2778336398 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7aa40ccbf0 0x7f7aa40b6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:04.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.682+0000 7f7aa9230640 1 --2- 172.21.15.112:0/2778336398 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f7a90078b90 0x7f7a9007b0a0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:04.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.682+0000 7f7aa9230640 1 --2- 172.21.15.112:0/2778336398 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7aa40c6a00 0x7f7aa41f6250 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:04.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.682+0000 7f7aa9230640 1 --2- 172.21.15.112:0/2778336398 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7aa40c5ee0 0x7f7aa40caab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:04.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.682+0000 7f7aa9230640 1 -- 172.21.15.112:0/2778336398 >> 172.21.15.112:0/2778336398 conn(0x7f7aa40b42e0 msgr2=0x7f7aa40c70e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:04.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.682+0000 7f7aa9230640 1 -- 172.21.15.112:0/2778336398 shutdown_connections 2024-09-21T00:29:04.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.682+0000 7f7aa9230640 1 -- 172.21.15.112:0/2778336398 wait complete. 2024-09-21T00:29:04.694 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json perf dump 2024-09-21T00:29:04.823 INFO:teuthology.orchestra.run.smithi187.stdout:{"AsyncMessenger::Worker-0":{"msgr_recv_messages":52,"msgr_send_messages":51,"msgr_recv_bytes":13732,"msgr_send_bytes":47854,"msgr_created_connections":9,"msgr_active_connections":1,"msgr_running_total_time":0.011661797,"msgr_running_send_time":0.001722696,"msgr_running_recv_time":0.008155643,"msgr_running_fast_dispatch_time":0.000601006,"msgr_send_messages_queue_lat":{"avgcount":50,"sum":0.061547563,"avgtime":0.001230951},"msgr_handle_ack_lat":{"avgcount":29,"sum":0.000011145,"avgtime":0.000000384},"msgr_recv_encrypted_bytes":1872,"msgr_send_encrypted_bytes":448},"AsyncMessenger::Worker-1":{"msgr_recv_messages":73,"msgr_send_messages":66,"msgr_recv_bytes":130073,"msgr_send_bytes":25646,"msgr_created_connections":5,"msgr_active_connections":3,"msgr_running_total_time":0.009790612,"msgr_running_send_time":0.002381650,"msgr_running_recv_time":0.010414585,"msgr_running_fast_dispatch_time":0.000536639,"msgr_send_messages_queue_lat":{"avgcount":66,"sum":0.146865046,"avgtime":0.002225227},"msgr_handle_ack_lat":{"avgcount":5,"sum":0.000003110,"avgtime":0.000000622},"msgr_recv_encrypted_bytes":123968,"msgr_send_encrypted_bytes":9952},"AsyncMessenger::Worker-2":{"msgr_recv_messages":12,"msgr_send_messages":17,"msgr_recv_bytes":3205,"msgr_send_bytes":99283,"msgr_created_connections":5,"msgr_active_connections":3,"msgr_running_total_time":0.004477909,"msgr_running_send_time":0.000719494,"msgr_running_recv_time":0.003176694,"msgr_running_fast_dispatch_time":0.000369962,"msgr_send_messages_queue_lat":{"avgcount":15,"sum":0.125401447,"avgtime":0.008360096},"msgr_handle_ack_lat":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"msgr_recv_encrypted_bytes":160,"msgr_send_encrypted_bytes":93952},"cct":{"total_workers":1,"unhealthy_workers":0},"finisher-MDSRank":{"queue_len":0,"complete_latency":{"avgcount":49,"sum":0.004729036,"avgtime":0.000096510}},"finisher-PurgeQueue":{"queue_len":0,"complete_latency":{"avgcount":2,"sum":0.000029337,"avgtime":0.000014668}},"mds":{"request":29,"reply":29,"reply_latency":{"avgcount":29,"sum":0.005397349,"avgtime":0.000186115},"slow_reply":0,"forward":0,"dir_fetch_complete":0,"dir_fetch_keys":0,"dir_commit":12,"dir_split":0,"dir_merge":0,"inodes":11,"inodes_top":0,"inodes_bottom":11,"inodes_pin_tail":0,"inodes_pinned":10,"inodes_expired":0,"inodes_with_caps":0,"caps":0,"subtrees":2,"traverse":55,"traverse_hit":53,"traverse_forward":0,"traverse_discover":0,"traverse_dir_fetch":0,"traverse_remote_ino":0,"traverse_lock":0,"load_cent":0,"q":0,"exported":0,"exported_inodes":0,"imported":0,"imported_inodes":0,"openino_dir_fetch":0,"openino_backtrace_fetch":0,"openino_peer_discover":0,"root_rfiles":0,"root_rbytes":0,"root_rsnaps":0,"scrub_backtrace_fetch":0,"scrub_set_tag":0,"scrub_backtrace_repaired":0,"scrub_inotable_repaired":0,"scrub_dir_inodes":0,"scrub_dir_base_inodes":0,"scrub_dirfrag_rstats":0,"scrub_file_inodes":0,"handle_inode_file_caps":0,"ceph_cap_op_revoke":0,"ceph_cap_op_grant":0,"ceph_cap_op_trunc":0,"ceph_cap_op_flushsnap_ack":0,"ceph_cap_op_flush_ack":0,"handle_client_caps":0,"handle_client_caps_dirty":0,"handle_client_cap_release":0,"process_request_cap_release":1},"mds_cache":{"dir_update":0,"dir_update_receipt":0,"dir_try_discover":0,"dir_send_discover":0,"dir_handle_discover":0,"num_strays":0,"num_strays_delayed":0,"num_strays_enqueuing":0,"strays_created":0,"strays_enqueued":0,"strays_reintegrated":0,"strays_migrated":0,"num_recovering_processing":0,"num_recovering_enqueued":0,"num_recovering_prioritized":0,"recovery_started":0,"recovery_completed":0,"ireq_quiesce_path":0,"ireq_quiesce_inode":0,"ireq_enqueue_scrub":0,"ireq_exportdir":0,"ireq_flush":0,"ireq_fragmentdir":0,"ireq_fragstats":0,"ireq_inodestats":0,"uninline_started":0,"uninline_succeeded":0,"uninline_write_failed":0},"mds_log":{"evlrg":0,"evadd":8,"evex":0,"evtrm":0,"ev":8,"evexg":0,"evexd":0,"segadd":2,"segex":0,"segtrm":0,"seg":2,"segmjr":2,"segexg":0,"segexd":0,"expos":4194304,"wrpos":4197025,"rdpos":0,"jlat":{"avgcount":5,"sum":0.013628416,"avgtime":0.002725683},"replayed":0},"mds_mem":{"ino":13,"ino+":13,"ino-":0,"dir":12,"dir+":12,"dir-":0,"dn":11,"dn+":11,"dn-":0,"cap":1,"cap+":2,"cap-":1,"rss":110232,"heap":332222464},"mds_server":{"dispatch_client_request":29,"dispatch_server_request":0,"handle_client_request":29,"handle_client_session":6,"handle_peer_request":0,"req_create_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_getattr_latency":{"avgcount":26,"sum":0.004705391,"avgtime":0.000180976},"req_getfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_link_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookup_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookuphash_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupino_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupname_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupparent_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lssnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mkdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mknod_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mksnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_open_latency":{"avgcount":2,"sum":0.000282387,"avgtime":0.000141193},"req_readdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rename_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_renamesnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_snapdiff_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setattr_latency":{"avgcount":1,"sum":0.000409571,"avgtime":0.000409571},"req_setdirlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_symlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_unlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"cap_revoke_eviction":0,"cap_acquisition_throttle":0,"req_getvxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"mds_sessions":{"session_count":0,"session_add":2,"session_remove":2,"sessions_open":0,"sessions_stale":0,"total_load":28,"average_load":14,"avg_session_uptime":0,"mdthresh_evicted":0},"mempool":{"bloom_filter_bytes":0,"bloom_filter_items":0,"bluestore_alloc_bytes":0,"bluestore_alloc_items":0,"bluestore_cache_data_bytes":0,"bluestore_cache_data_items":0,"bluestore_cache_onode_bytes":0,"bluestore_cache_onode_items":0,"bluestore_cache_meta_bytes":0,"bluestore_cache_meta_items":0,"bluestore_cache_other_bytes":0,"bluestore_cache_other_items":0,"bluestore_cache_buffer_bytes":0,"bluestore_cache_buffer_items":0,"bluestore_extent_bytes":0,"bluestore_extent_items":0,"bluestore_blob_bytes":0,"bluestore_blob_items":0,"bluestore_shared_blob_bytes":0,"bluestore_shared_blob_items":0,"bluestore_inline_bl_bytes":0,"bluestore_inline_bl_items":0,"bluestore_fsck_bytes":0,"bluestore_fsck_items":0,"bluestore_txc_bytes":0,"bluestore_txc_items":0,"bluestore_writing_deferred_bytes":0,"bluestore_writing_deferred_items":0,"bluestore_writing_bytes":0,"bluestore_writing_items":0,"bluefs_bytes":0,"bluefs_items":0,"bluefs_file_reader_bytes":0,"bluefs_file_reader_items":0,"bluefs_file_writer_bytes":0,"bluefs_file_writer_items":0,"buffer_anon_bytes":208625,"buffer_anon_items":73,"buffer_meta_bytes":0,"buffer_meta_items":0,"osd_bytes":0,"osd_items":0,"osd_mapbl_bytes":0,"osd_mapbl_items":0,"osd_pglog_bytes":0,"osd_pglog_items":0,"osdmap_bytes":5424,"osdmap_items":76,"osdmap_mapping_bytes":0,"osdmap_mapping_items":0,"pgmap_bytes":0,"pgmap_items":0,"mds_co_bytes":53360,"mds_co_items":216,"unittest_1_bytes":0,"unittest_1_items":0,"unittest_2_bytes":0,"unittest_2_items":0},"objecter":{"op_active":0,"op_laggy":0,"op_send":43,"op_send_bytes":13841,"op_resend":30,"op_reply":43,"op_latency":{"avgcount":43,"sum":0.967984880,"avgtime":0.022511276},"op_inflight":0,"oplen_avg":{"avgcount":43,"sum":67},"op":43,"op_r":0,"op_w":43,"op_rmw":0,"op_pg":0,"osdop_stat":0,"osdop_create":11,"osdop_read":0,"osdop_write":5,"osdop_writefull":7,"osdop_writesame":0,"osdop_append":0,"osdop_zero":0,"osdop_truncate":0,"osdop_delete":6,"osdop_mapext":0,"osdop_sparse_read":0,"osdop_clonerange":0,"osdop_getxattr":0,"osdop_setxattr":22,"osdop_cmpxattr":0,"osdop_rmxattr":0,"osdop_resetxattrs":0,"osdop_call":0,"osdop_watch":0,"osdop_notify":0,"osdop_src_cmpxattr":0,"osdop_pgls":0,"osdop_pgls_filter":0,"osdop_other":0,"linger_active":0,"linger_send":0,"linger_resend":0,"linger_ping":0,"poolop_active":0,"poolop_send":0,"poolop_resend":0,"poolstat_active":0,"poolstat_send":0,"poolstat_resend":0,"statfs_active":0,"statfs_send":0,"statfs_resend":0,"command_active":0,"command_send":0,"command_resend":0,"map_epoch":83,"map_full":0,"map_inc":1,"osd_sessions":6,"osd_session_open":6,"osd_session_close":0,"osd_laggy":0,"omap_wr":15,"omap_rd":0,"omap_del":1},"oft":{"omap_total_objs":1,"omap_total_kv_pairs":0,"omap_total_updates":0,"omap_total_removes":0},"purge_queue":{"pq_executing_ops":0,"pq_executing_ops_high_water":0,"pq_executing":0,"pq_executing_high_water":0,"pq_executed_ops":0,"pq_executed":0,"pq_item_in_journal":0},"throttle-msgr_dispatch_throttler-mds":{"val":0,"max":104857600,"get_started":0,"get":132,"get_sum":131406,"get_or_fail_fail":0,"get_or_fail_success":132,"take":0,"take_sum":0,"put":132,"put_sum":131406,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_bytes":{"val":0,"max":104857600,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":43,"take_sum":13841,"put":43,"put_sum":13841,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_ops":{"val":0,"max":1024,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":43,"take_sum":43,"put":43,"put_sum":43,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle":{"val":0,"max":3758096384,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":0,"take_sum":0,"put":0,"put_sum":0,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle-0x55c6ab427380":{"val":0,"max":3758096384,"get_started":0,"get":8,"get_sum":2721,"get_or_fail_fail":0,"get_or_fail_success":8,"take":0,"take_sum":0,"put":5,"put_sum":2721,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"trackedop":{"slow_ops_count":0}} 2024-09-21T00:29:04.825 DEBUG:tasks.cephfs.filesystem:_json_asok output { "AsyncMessenger::Worker-0": { "msgr_active_connections": 1, "msgr_created_connections": 9, "msgr_handle_ack_lat": { "avgcount": 29, "avgtime": 3.84e-07, "sum": 1.1145e-05 }, "msgr_recv_bytes": 13732, "msgr_recv_encrypted_bytes": 1872, "msgr_recv_messages": 52, "msgr_running_fast_dispatch_time": 0.000601006, "msgr_running_recv_time": 0.008155643, "msgr_running_send_time": 0.001722696, "msgr_running_total_time": 0.011661797, "msgr_send_bytes": 47854, "msgr_send_encrypted_bytes": 448, "msgr_send_messages": 51, "msgr_send_messages_queue_lat": { "avgcount": 50, "avgtime": 0.001230951, "sum": 0.061547563 } }, "AsyncMessenger::Worker-1": { "msgr_active_connections": 3, "msgr_created_connections": 5, "msgr_handle_ack_lat": { "avgcount": 5, "avgtime": 6.22e-07, "sum": 3.11e-06 }, "msgr_recv_bytes": 130073, "msgr_recv_encrypted_bytes": 123968, "msgr_recv_messages": 73, "msgr_running_fast_dispatch_time": 0.000536639, "msgr_running_recv_time": 0.010414585, "msgr_running_send_time": 0.00238165, "msgr_running_total_time": 0.009790612, "msgr_send_bytes": 25646, "msgr_send_encrypted_bytes": 9952, "msgr_send_messages": 66, "msgr_send_messages_queue_lat": { "avgcount": 66, "avgtime": 0.002225227, "sum": 0.146865046 } }, "AsyncMessenger::Worker-2": { "msgr_active_connections": 3, "msgr_created_connections": 5, "msgr_handle_ack_lat": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "msgr_recv_bytes": 3205, "msgr_recv_encrypted_bytes": 160, "msgr_recv_messages": 12, "msgr_running_fast_dispatch_time": 0.000369962, "msgr_running_recv_time": 0.003176694, "msgr_running_send_time": 0.000719494, "msgr_running_total_time": 0.004477909, "msgr_send_bytes": 99283, "msgr_send_encrypted_bytes": 93952, "msgr_send_messages": 17, "msgr_send_messages_queue_lat": { "avgcount": 15, "avgtime": 0.008360096, "sum": 0.125401447 } }, "cct": { "total_workers": 1, "unhealthy_workers": 0 }, "finisher-MDSRank": { "complete_latency": { "avgcount": 49, "avgtime": 9.651e-05, "sum": 0.004729036 }, "queue_len": 0 }, "finisher-PurgeQueue": { "complete_latency": { "avgcount": 2, "avgtime": 1.4668e-05, "sum": 2.9337e-05 }, "queue_len": 0 }, "mds": { "caps": 0, "ceph_cap_op_flush_ack": 0, "ceph_cap_op_flushsnap_ack": 0, "ceph_cap_op_grant": 0, "ceph_cap_op_revoke": 0, "ceph_cap_op_trunc": 0, "dir_commit": 12, "dir_fetch_complete": 0, "dir_fetch_keys": 0, "dir_merge": 0, "dir_split": 0, "exported": 0, "exported_inodes": 0, "forward": 0, "handle_client_cap_release": 0, "handle_client_caps": 0, "handle_client_caps_dirty": 0, "handle_inode_file_caps": 0, "imported": 0, "imported_inodes": 0, "inodes": 11, "inodes_bottom": 11, "inodes_expired": 0, "inodes_pin_tail": 0, "inodes_pinned": 10, "inodes_top": 0, "inodes_with_caps": 0, "load_cent": 0, "openino_backtrace_fetch": 0, "openino_dir_fetch": 0, "openino_peer_discover": 0, "process_request_cap_release": 1, "q": 0, "reply": 29, "reply_latency": { "avgcount": 29, "avgtime": 0.000186115, "sum": 0.005397349 }, "request": 29, "root_rbytes": 0, "root_rfiles": 0, "root_rsnaps": 0, "scrub_backtrace_fetch": 0, "scrub_backtrace_repaired": 0, "scrub_dir_base_inodes": 0, "scrub_dir_inodes": 0, "scrub_dirfrag_rstats": 0, "scrub_file_inodes": 0, "scrub_inotable_repaired": 0, "scrub_set_tag": 0, "slow_reply": 0, "subtrees": 2, "traverse": 55, "traverse_dir_fetch": 0, "traverse_discover": 0, "traverse_forward": 0, "traverse_hit": 53, "traverse_lock": 0, "traverse_remote_ino": 0 }, "mds_cache": { "dir_handle_discover": 0, "dir_send_discover": 0, "dir_try_discover": 0, "dir_update": 0, "dir_update_receipt": 0, "ireq_enqueue_scrub": 0, "ireq_exportdir": 0, "ireq_flush": 0, "ireq_fragmentdir": 0, "ireq_fragstats": 0, "ireq_inodestats": 0, "ireq_quiesce_inode": 0, "ireq_quiesce_path": 0, "num_recovering_enqueued": 0, "num_recovering_prioritized": 0, "num_recovering_processing": 0, "num_strays": 0, "num_strays_delayed": 0, "num_strays_enqueuing": 0, "recovery_completed": 0, "recovery_started": 0, "strays_created": 0, "strays_enqueued": 0, "strays_migrated": 0, "strays_reintegrated": 0, "uninline_started": 0, "uninline_succeeded": 0, "uninline_write_failed": 0 }, "mds_log": { "ev": 8, "evadd": 8, "evex": 0, "evexd": 0, "evexg": 0, "evlrg": 0, "evtrm": 0, "expos": 4194304, "jlat": { "avgcount": 5, "avgtime": 0.002725683, "sum": 0.013628416 }, "rdpos": 0, "replayed": 0, "seg": 2, "segadd": 2, "segex": 0, "segexd": 0, "segexg": 0, "segmjr": 2, "segtrm": 0, "wrpos": 4197025 }, "mds_mem": { "cap": 1, "cap+": 2, "cap-": 1, "dir": 12, "dir+": 12, "dir-": 0, "dn": 11, "dn+": 11, "dn-": 0, "heap": 332222464, "ino": 13, "ino+": 13, "ino-": 0, "rss": 110232 }, "mds_server": { "cap_acquisition_throttle": 0, "cap_revoke_eviction": 0, "dispatch_client_request": 29, "dispatch_server_request": 0, "handle_client_request": 29, "handle_client_session": 6, "handle_peer_request": 0, "req_create_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_getattr_latency": { "avgcount": 26, "avgtime": 0.000180976, "sum": 0.004705391 }, "req_getfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_getvxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_link_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookup_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookuphash_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupino_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupname_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupparent_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lssnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mkdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mknod_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mksnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_open_latency": { "avgcount": 2, "avgtime": 0.000141193, "sum": 0.000282387 }, "req_readdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rename_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_renamesnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setattr_latency": { "avgcount": 1, "avgtime": 0.000409571, "sum": 0.000409571 }, "req_setdirlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_snapdiff_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_symlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_unlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "mds_sessions": { "average_load": 14, "avg_session_uptime": 0, "mdthresh_evicted": 0, "session_add": 2, "session_count": 0, "session_remove": 2, "sessions_open": 0, "sessions_stale": 0, "total_load": 28 }, "mempool": { "bloom_filter_bytes": 0, "bloom_filter_items": 0, "bluefs_bytes": 0, "bluefs_file_reader_bytes": 0, "bluefs_file_reader_items": 0, "bluefs_file_writer_bytes": 0, "bluefs_file_writer_items": 0, "bluefs_items": 0, "bluestore_alloc_bytes": 0, "bluestore_alloc_items": 0, "bluestore_blob_bytes": 0, "bluestore_blob_items": 0, "bluestore_cache_buffer_bytes": 0, "bluestore_cache_buffer_items": 0, "bluestore_cache_data_bytes": 0, "bluestore_cache_data_items": 0, "bluestore_cache_meta_bytes": 0, "bluestore_cache_meta_items": 0, "bluestore_cache_onode_bytes": 0, "bluestore_cache_onode_items": 0, "bluestore_cache_other_bytes": 0, "bluestore_cache_other_items": 0, "bluestore_extent_bytes": 0, "bluestore_extent_items": 0, "bluestore_fsck_bytes": 0, "bluestore_fsck_items": 0, "bluestore_inline_bl_bytes": 0, "bluestore_inline_bl_items": 0, "bluestore_shared_blob_bytes": 0, "bluestore_shared_blob_items": 0, "bluestore_txc_bytes": 0, "bluestore_txc_items": 0, "bluestore_writing_bytes": 0, "bluestore_writing_deferred_bytes": 0, "bluestore_writing_deferred_items": 0, "bluestore_writing_items": 0, "buffer_anon_bytes": 208625, "buffer_anon_items": 73, "buffer_meta_bytes": 0, "buffer_meta_items": 0, "mds_co_bytes": 53360, "mds_co_items": 216, "osd_bytes": 0, "osd_items": 0, "osd_mapbl_bytes": 0, "osd_mapbl_items": 0, "osd_pglog_bytes": 0, "osd_pglog_items": 0, "osdmap_bytes": 5424, "osdmap_items": 76, "osdmap_mapping_bytes": 0, "osdmap_mapping_items": 0, "pgmap_bytes": 0, "pgmap_items": 0, "unittest_1_bytes": 0, "unittest_1_items": 0, "unittest_2_bytes": 0, "unittest_2_items": 0 }, "objecter": { "command_active": 0, "command_resend": 0, "command_send": 0, "linger_active": 0, "linger_ping": 0, "linger_resend": 0, "linger_send": 0, "map_epoch": 83, "map_full": 0, "map_inc": 1, "omap_del": 1, "omap_rd": 0, "omap_wr": 15, "op": 43, "op_Infinitylight": 0, "op_active": 0, "op_laggy": 0, "op_latency": { "avgcount": 43, "avgtime": 0.022511276, "sum": 0.96798488 }, "op_pg": 0, "op_r": 0, "op_reply": 43, "op_resend": 30, "op_rmw": 0, "op_send": 43, "op_send_bytes": 13841, "op_w": 43, "oplen_avg": { "avgcount": 43, "sum": 67 }, "osd_laggy": 0, "osd_session_close": 0, "osd_session_open": 6, "osd_sessions": 6, "osdop_append": 0, "osdop_call": 0, "osdop_clonerange": 0, "osdop_cmpxattr": 0, "osdop_create": 11, "osdop_delete": 6, "osdop_getxattr": 0, "osdop_mapext": 0, "osdop_notify": 0, "osdop_other": 0, "osdop_pgls": 0, "osdop_pgls_filter": 0, "osdop_read": 0, "osdop_resetxattrs": 0, "osdop_rmxattr": 0, "osdop_setxattr": 22, "osdop_sparse_read": 0, "osdop_src_cmpxattr": 0, "osdop_stat": 0, "osdop_truncate": 0, "osdop_watch": 0, "osdop_write": 5, "osdop_writefull": 7, "osdop_writesame": 0, "osdop_zero": 0, "poolop_active": 0, "poolop_resend": 0, "poolop_send": 0, "poolstat_active": 0, "poolstat_resend": 0, "poolstat_send": 0, "statfs_active": 0, "statfs_resend": 0, "statfs_send": 0 }, "oft": { "omap_total_kv_pairs": 0, "omap_total_objs": 1, "omap_total_removes": 0, "omap_total_updates": 0 }, "purge_queue": { "pq_executed": 0, "pq_executed_ops": 0, "pq_executing": 0, "pq_executing_high_water": 0, "pq_executing_ops": 0, "pq_executing_ops_high_water": 0, "pq_item_in_journal": 0 }, "throttle-msgr_dispatch_throttler-mds": { "get": 132, "get_or_fail_fail": 0, "get_or_fail_success": 132, "get_started": 0, "get_sum": 131406, "max": 104857600, "put": 132, "put_sum": 131406, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_bytes": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 104857600, "put": 43, "put_sum": 13841, "take": 43, "take_sum": 13841, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_ops": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 1024, "put": 43, "put_sum": 43, "take": 43, "take_sum": 43, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 3758096384, "put": 0, "put_sum": 0, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle-0x55c6ab427380": { "get": 8, "get_or_fail_fail": 0, "get_or_fail_success": 8, "get_started": 0, "get_sum": 2721, "max": 3758096384, "put": 5, "put_sum": 2721, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "trackedop": { "slow_ops_count": 0 } } 2024-09-21T00:29:04.825 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.6:0 session ls 2024-09-21T00:29:04.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.944+0000 7f6a2995f640 1 -- 172.21.15.112:0/3564151278 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a240d3880 msgr2=0x7f6a24163260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:04.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.944+0000 7f6a2995f640 1 --2- 172.21.15.112:0/3564151278 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a240d3880 0x7f6a24163260 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f6a18009830 tx=0x7f6a1802a510 comp rx=0 tx=0).stop 2024-09-21T00:29:04.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.944+0000 7f6a2995f640 1 -- 172.21.15.112:0/3564151278 shutdown_connections 2024-09-21T00:29:04.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.944+0000 7f6a2995f640 1 --2- 172.21.15.112:0/3564151278 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a240d3880 0x7f6a24163260 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:04.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.944+0000 7f6a2995f640 1 --2- 172.21.15.112:0/3564151278 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6a240d48a0 0x7f6a240d3dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:04.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.944+0000 7f6a2995f640 1 -- 172.21.15.112:0/3564151278 >> 172.21.15.112:0/3564151278 conn(0x7f6a2408bfe0 msgr2=0x7f6a2409d890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:04.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.944+0000 7f6a2995f640 1 -- 172.21.15.112:0/3564151278 shutdown_connections 2024-09-21T00:29:04.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.944+0000 7f6a2995f640 1 -- 172.21.15.112:0/3564151278 wait complete. 2024-09-21T00:29:04.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.946+0000 7f6a2995f640 1 Processor -- start 2024-09-21T00:29:04.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.946+0000 7f6a2995f640 1 -- start start 2024-09-21T00:29:04.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.947+0000 7f6a2995f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a240d01b0 0x7f6a2415fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:04.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.947+0000 7f6a2995f640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6a240d3880 0x7f6a241f6a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:04.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.947+0000 7f6a2995f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6a240d48a0 0x7f6a240ca120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:04.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.947+0000 7f6a2995f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6a240d2490 con 0x7f6a240d01b0 2024-09-21T00:29:04.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.947+0000 7f6a2995f640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f6a240d2310 con 0x7f6a240d48a0 2024-09-21T00:29:04.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.947+0000 7f6a2995f640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f6a240d2610 con 0x7f6a240d3880 2024-09-21T00:29:04.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.947+0000 7f6a22ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a240d01b0 0x7f6a2415fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:04.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.947+0000 7f6a22ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a240d01b0 0x7f6a2415fac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38388/0 (socket says 172.21.15.112:38388) 2024-09-21T00:29:04.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.947+0000 7f6a22ffd640 1 -- 172.21.15.112:0/1069905293 learned_addr learned my addr 172.21.15.112:0/1069905293 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:29:04.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.947+0000 7f6a227fc640 1 --2- 172.21.15.112:0/1069905293 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6a240d3880 0x7f6a241f6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:04.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.947+0000 7f6a237fe640 1 --2- 172.21.15.112:0/1069905293 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6a240d48a0 0x7f6a240ca120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:04.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.947+0000 7f6a22ffd640 1 -- 172.21.15.112:0/1069905293 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6a240d3880 msgr2=0x7f6a241f6a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:04.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.947+0000 7f6a22ffd640 1 --2- 172.21.15.112:0/1069905293 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6a240d3880 0x7f6a241f6a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:04.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.947+0000 7f6a22ffd640 1 -- 172.21.15.112:0/1069905293 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6a240d48a0 msgr2=0x7f6a240ca120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:04.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.947+0000 7f6a22ffd640 1 --2- 172.21.15.112:0/1069905293 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6a240d48a0 0x7f6a240ca120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:04.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.947+0000 7f6a22ffd640 1 -- 172.21.15.112:0/1069905293 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a240c90a0 con 0x7f6a240d01b0 2024-09-21T00:29:04.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.948+0000 7f6a22ffd640 1 --2- 172.21.15.112:0/1069905293 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a240d01b0 0x7f6a2415fac0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f6a1400c970 tx=0x7f6a1400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:04.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.948+0000 7f6a0bfff640 1 -- 172.21.15.112:0/1069905293 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a14007bb0 con 0x7f6a240d01b0 2024-09-21T00:29:04.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.948+0000 7f6a0bfff640 1 -- 172.21.15.112:0/1069905293 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6a14007e40 con 0x7f6a240d01b0 2024-09-21T00:29:04.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.948+0000 7f6a0bfff640 1 -- 172.21.15.112:0/1069905293 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a14005280 con 0x7f6a240d01b0 2024-09-21T00:29:04.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.948+0000 7f6a2995f640 1 -- 172.21.15.112:0/1069905293 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a240c9380 con 0x7f6a240d01b0 2024-09-21T00:29:04.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.948+0000 7f6a2995f640 1 -- 172.21.15.112:0/1069905293 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a240c9940 con 0x7f6a240d01b0 2024-09-21T00:29:04.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.950+0000 7f6a0bfff640 1 -- 172.21.15.112:0/1069905293 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6a14005420 con 0x7f6a240d01b0 2024-09-21T00:29:04.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.951+0000 7f6a0bfff640 1 --2- 172.21.15.112:0/1069905293 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f6a10078b90 0x7f6a1007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:04.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.951+0000 7f6a0bfff640 1 -- 172.21.15.112:0/1069905293 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 6101+0+0 (secure 0 0 0) 0x7f6a1409b290 con 0x7f6a240d01b0 2024-09-21T00:29:04.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.951+0000 7f6a227fc640 1 --2- 172.21.15.112:0/1069905293 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f6a10078b90 0x7f6a1007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:04.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.951+0000 7f6a308c8740 1 Processor -- start 2024-09-21T00:29:04.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.951+0000 7f6a308c8740 1 -- start start 2024-09-21T00:29:04.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.952+0000 7f6a227fc640 1 --2- 172.21.15.112:0/1069905293 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f6a10078b90 0x7f6a1007b0a0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f6a18034000 tx=0x7f6a18004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:04.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.953+0000 7f6a308c8740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557a1c1c4390 0x557a1c5961d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:04.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.953+0000 7f6a308c8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557a1c19b4d0 0x557a1c596710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:04.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.953+0000 7f6a308c8740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x557a1c1a36b0 0x557a1c5b3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:04.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.953+0000 7f6a308c8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x557a1c4de8e0 con 0x557a1c19b4d0 2024-09-21T00:29:04.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.953+0000 7f6a308c8740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x557a1c4cdd10 con 0x557a1c1c4390 2024-09-21T00:29:04.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.953+0000 7f6a308c8740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x557a1c4d1b20 con 0x557a1c1a36b0 2024-09-21T00:29:04.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.953+0000 7f6a227fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557a1c19b4d0 0x557a1c596710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:04.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.953+0000 7f6a227fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557a1c19b4d0 0x557a1c596710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38404/0 (socket says 172.21.15.112:38404) 2024-09-21T00:29:04.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.953+0000 7f6a22ffd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557a1c1c4390 0x557a1c5961d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:04.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.953+0000 7f6a227fc640 1 -- 172.21.15.112:0/2858753705 learned_addr learned my addr 172.21.15.112:0/2858753705 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:29:04.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.953+0000 7f6a237fe640 1 --2- 172.21.15.112:0/2858753705 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x557a1c1a36b0 0x557a1c5b3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:04.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.953+0000 7f6a22ffd640 1 -- 172.21.15.112:0/2858753705 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x557a1c1a36b0 msgr2=0x557a1c5b3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:29:04.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.953+0000 7f6a22ffd640 1 --2- 172.21.15.112:0/2858753705 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x557a1c1a36b0 0x557a1c5b3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:04.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.953+0000 7f6a22ffd640 1 -- 172.21.15.112:0/2858753705 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557a1c19b4d0 msgr2=0x557a1c596710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:29:04.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.953+0000 7f6a22ffd640 1 --2- 172.21.15.112:0/2858753705 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557a1c19b4d0 0x557a1c596710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:04.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.953+0000 7f6a22ffd640 1 -- 172.21.15.112:0/2858753705 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a1c22f970 con 0x557a1c1c4390 2024-09-21T00:29:04.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.953+0000 7f6a227fc640 1 --2- 172.21.15.112:0/2858753705 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557a1c19b4d0 0x557a1c596710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:29:04.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.953+0000 7f6a22ffd640 1 --2- 172.21.15.112:0/2858753705 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557a1c1c4390 0x557a1c5961d0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f6a14099680 tx=0x7f6a14062840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c91d8e83427b99a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:04.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.954+0000 7f6a097fa640 1 -- 172.21.15.112:0/2858753705 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a1401d040 con 0x557a1c1c4390 2024-09-21T00:29:04.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.954+0000 7f6a097fa640 1 -- 172.21.15.112:0/2858753705 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6a14067a70 con 0x557a1c1c4390 2024-09-21T00:29:04.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.954+0000 7f6a097fa640 1 -- 172.21.15.112:0/2858753705 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a1406c920 con 0x557a1c1c4390 2024-09-21T00:29:04.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.954+0000 7f6a308c8740 1 -- 172.21.15.112:0/2858753705 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557a1c1c4390 msgr2=0x557a1c5961d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:04.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.954+0000 7f6a308c8740 1 --2- 172.21.15.112:0/2858753705 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557a1c1c4390 0x557a1c5961d0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f6a14099680 tx=0x7f6a14062840 comp rx=0 tx=0).stop 2024-09-21T00:29:04.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.954+0000 7f6a308c8740 1 -- 172.21.15.112:0/2858753705 shutdown_connections 2024-09-21T00:29:04.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.954+0000 7f6a308c8740 1 --2- 172.21.15.112:0/2858753705 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x557a1c1a36b0 0x557a1c5b3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:04.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.954+0000 7f6a308c8740 1 --2- 172.21.15.112:0/2858753705 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557a1c19b4d0 0x557a1c596710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:04.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.954+0000 7f6a308c8740 1 --2- 172.21.15.112:0/2858753705 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557a1c1c4390 0x557a1c5961d0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:04.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.954+0000 7f6a308c8740 1 -- 172.21.15.112:0/2858753705 >> 172.21.15.112:0/2858753705 conn(0x557a1c1b84e0 msgr2=0x557a1c56c490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:04.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.955+0000 7f6a308c8740 1 -- 172.21.15.112:0/2858753705 shutdown_connections 2024-09-21T00:29:04.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.955+0000 7f6a308c8740 1 -- 172.21.15.112:0/2858753705 wait complete. 2024-09-21T00:29:04.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.955+0000 7f6a308c8740 1 -- start start 2024-09-21T00:29:04.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.955+0000 7f6a308c8740 1 Processor -- start 2024-09-21T00:29:04.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.957+0000 7f6a308c8740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557a1c19b4d0 0x557a1c582cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:04.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.957+0000 7f6a308c8740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x557a1c1a36b0 0x557a1c5b3370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:04.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.957+0000 7f6a308c8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557a1c1c4390 0x557a1c5b38b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:04.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.957+0000 7f6a308c8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x557a1c23e740 con 0x557a1c1c4390 2024-09-21T00:29:04.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.957+0000 7f6a308c8740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x557a1c4737f0 con 0x557a1c19b4d0 2024-09-21T00:29:04.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.957+0000 7f6a308c8740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x557a1c596c50 con 0x557a1c1a36b0 2024-09-21T00:29:04.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.957+0000 7f6a308c8740 10 client.0 _ll_register_callbacks cb 0x557a1c4772a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:29:04.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.957+0000 7f6a237fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557a1c1c4390 0x557a1c5b38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:04.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.957+0000 7f6a237fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557a1c1c4390 0x557a1c5b38b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38414/0 (socket says 172.21.15.112:38414) 2024-09-21T00:29:04.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.957+0000 7f6a237fe640 1 -- 172.21.15.112:0/1641312017 learned_addr learned my addr 172.21.15.112:0/1641312017 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:29:04.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.957+0000 7f6a22ffd640 1 --2- 172.21.15.112:0/1641312017 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557a1c19b4d0 0x557a1c582cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:04.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.957+0000 7f6a227fc640 1 --2- 172.21.15.112:0/1641312017 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x557a1c1a36b0 0x557a1c5b3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:04.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.957+0000 7f6a237fe640 1 -- 172.21.15.112:0/1641312017 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x557a1c1a36b0 msgr2=0x557a1c5b3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:29:04.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.957+0000 7f6a237fe640 1 --2- 172.21.15.112:0/1641312017 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x557a1c1a36b0 0x557a1c5b3370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:04.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.957+0000 7f6a237fe640 1 -- 172.21.15.112:0/1641312017 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557a1c19b4d0 msgr2=0x557a1c582cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:29:04.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.957+0000 7f6a237fe640 1 --2- 172.21.15.112:0/1641312017 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557a1c19b4d0 0x557a1c582cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:04.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.957+0000 7f6a237fe640 1 -- 172.21.15.112:0/1641312017 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a1c2a67d0 con 0x557a1c1c4390 2024-09-21T00:29:04.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.957+0000 7f6a22ffd640 1 --2- 172.21.15.112:0/1641312017 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557a1c19b4d0 0x557a1c582cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:29:04.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.957+0000 7f6a237fe640 1 --2- 172.21.15.112:0/1641312017 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557a1c1c4390 0x557a1c5b38b0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f6a240cb580 tx=0x7f6a1c012a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4be96d12648482f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:04.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.957+0000 7f6a027fc640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T00:29:04.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.957+0000 7f6a027fc640 1 -- 172.21.15.112:0/1641312017 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a1c006d40 con 0x557a1c1c4390 2024-09-21T00:29:04.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.958+0000 7f6a027fc640 1 -- 172.21.15.112:0/1641312017 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 6101+0+0 (secure 0 0 0) 0x7f6a1c003970 con 0x557a1c1c4390 2024-09-21T00:29:04.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.958+0000 7f6a308c8740 1 -- 172.21.15.112:0/1641312017 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a14067a70 con 0x557a1c1c4390 2024-09-21T00:29:04.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.958+0000 7f6a027fc640 1 -- 172.21.15.112:0/1641312017 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f6a1c003b70 con 0x557a1c1c4390 2024-09-21T00:29:04.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.958+0000 7f6a308c8740 10 client.6657 fetch_fsmap learned FSMap version 55 2024-09-21T00:29:04.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.958+0000 7f6a308c8740 1 -- 172.21.15.112:0/1641312017 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=2+}) -- 0x557a1c5c3d90 con 0x557a1c1c4390 2024-09-21T00:29:04.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.958+0000 7f6a027fc640 1 -- 172.21.15.112:0/1641312017 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6a14067a70 con 0x557a1c1c4390 2024-09-21T00:29:04.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.958+0000 7f6a027fc640 1 -- 172.21.15.112:0/1641312017 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 55) ==== 3502+0+0 (secure 0 0 0) 0x7f6a1c005110 con 0x557a1c1c4390 2024-09-21T00:29:04.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.958+0000 7f6a308c8740 10 client.6657 fetch_fsmap finished waiting for FSMap version 55 2024-09-21T00:29:04.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.958+0000 7f6a308c8740 10 client.6657 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.d 2024-09-21T00:29:04.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.958+0000 7f6a308c8740 20 client.6657 populate_metadata read hostname 'smithi112' 2024-09-21T00:29:04.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.958+0000 7f6a308c8740 1 --2- 172.21.15.112:0/1641312017 >> [v2:172.21.15.187:6835/932692802,v1:172.21.15.187:6837/932692802] conn(0x557a1c182f60 0x557a1c5c0860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:04.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.958+0000 7f6a308c8740 4 client.6657 mds_command: new command op to 6486 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T00:29:04.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.958+0000 7f6a308c8740 1 -- 172.21.15.112:0/1641312017 --> [v2:172.21.15.187:6835/932692802,v1:172.21.15.187:6837/932692802] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557a1c4e0830 con 0x557a1c182f60 2024-09-21T00:29:04.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.958+0000 7f6a22ffd640 1 --2- 172.21.15.112:0/1641312017 >> [v2:172.21.15.187:6835/932692802,v1:172.21.15.187:6837/932692802] conn(0x557a1c182f60 0x557a1c5c0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:04.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.959+0000 7f6a027fc640 10 client.6657 ms_handle_connect on v2:172.21.15.187:6835/932692802 2024-09-21T00:29:04.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.959+0000 7f6a22ffd640 1 --2- 172.21.15.112:0/1641312017 >> [v2:172.21.15.187:6835/932692802,v1:172.21.15.187:6837/932692802] conn(0x557a1c182f60 0x557a1c5c0860 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6db0668df97a07ce server_cookie=7d10d1eb4249f91b in_seq=0 out_seq=0 2024-09-21T00:29:04.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.959+0000 7f6a027fc640 1 -- 172.21.15.112:0/1641312017 <== mds.0 v2:172.21.15.187:6835/932692802 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557a1c4e0830 con 0x557a1c182f60 2024-09-21T00:29:04.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.959+0000 7f6a027fc640 10 client.6657 handle_command_reply: tid=0 2024-09-21T00:29:04.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.959+0000 7f6a308c8740 1 client.6657 shutdown 2024-09-21T00:29:04.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.959+0000 7f6a308c8740 1 -- 172.21.15.112:0/1641312017 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557a1c1c4390 msgr2=0x557a1c5b38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:04.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.959+0000 7f6a308c8740 1 --2- 172.21.15.112:0/1641312017 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557a1c1c4390 0x557a1c5b38b0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f6a240cb580 tx=0x7f6a1c012a10 comp rx=0 tx=0).stop 2024-09-21T00:29:04.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.959+0000 7f6a308c8740 1 -- 172.21.15.112:0/1641312017 shutdown_connections 2024-09-21T00:29:04.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.959+0000 7f6a308c8740 1 --2- 172.21.15.112:0/1641312017 >> [v2:172.21.15.187:6835/932692802,v1:172.21.15.187:6837/932692802] conn(0x557a1c182f60 0x557a1c5c0860 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:04.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.959+0000 7f6a308c8740 1 --2- 172.21.15.112:0/1641312017 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557a1c1c4390 0x557a1c5b38b0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:04.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.959+0000 7f6a027fc640 0 client.6657 ms_handle_reset on v2:172.21.15.187:6835/932692802 2024-09-21T00:29:04.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.959+0000 7f6a308c8740 1 --2- 172.21.15.112:0/1641312017 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x557a1c1a36b0 0x557a1c5b3370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:04.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.959+0000 7f6a308c8740 1 --2- 172.21.15.112:0/1641312017 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557a1c19b4d0 0x557a1c582cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:04.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.959+0000 7f6a308c8740 1 -- 172.21.15.112:0/1641312017 >> 172.21.15.112:0/1641312017 conn(0x557a1c1b84e0 msgr2=0x557a1c5b2be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:04.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.960+0000 7f6a308c8740 1 -- 172.21.15.112:0/1641312017 shutdown_connections 2024-09-21T00:29:04.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.960+0000 7f6a308c8740 1 -- 172.21.15.112:0/1641312017 wait complete. 2024-09-21T00:29:04.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.960+0000 7f6a308c8740 20 client.6657 trim_cache size 0 max 16384 2024-09-21T00:29:04.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.978+0000 7f6a308c8740 1 Processor -- start 2024-09-21T00:29:04.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.978+0000 7f6a308c8740 1 -- start start 2024-09-21T00:29:04.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.979+0000 7f6a308c8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557a1c19b4d0 0x557a1c5acfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:04.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.979+0000 7f6a308c8740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x557a1c1a36b0 0x557a1c5b1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:04.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.979+0000 7f6a308c8740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557a1c1c4390 0x557a1c595e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:04.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.979+0000 7f6a308c8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6a242025e0 con 0x557a1c19b4d0 2024-09-21T00:29:04.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.979+0000 7f6a308c8740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f6a242015c0 con 0x557a1c1c4390 2024-09-21T00:29:04.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.979+0000 7f6a308c8740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f6a241550b0 con 0x557a1c1a36b0 2024-09-21T00:29:04.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.980+0000 7f6a22ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557a1c19b4d0 0x557a1c5acfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:04.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.980+0000 7f6a22ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557a1c19b4d0 0x557a1c5acfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38430/0 (socket says 172.21.15.112:38430) 2024-09-21T00:29:04.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.980+0000 7f6a22ffd640 1 -- 172.21.15.112:0/201401813 learned_addr learned my addr 172.21.15.112:0/201401813 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:29:04.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.980+0000 7f6a227fc640 1 --2- 172.21.15.112:0/201401813 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x557a1c1a36b0 0x557a1c5b1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:04.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.980+0000 7f6a237fe640 1 --2- 172.21.15.112:0/201401813 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557a1c1c4390 0x557a1c595e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:04.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.980+0000 7f6a22ffd640 1 -- 172.21.15.112:0/201401813 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x557a1c1a36b0 msgr2=0x557a1c5b1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:29:04.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.980+0000 7f6a22ffd640 1 --2- 172.21.15.112:0/201401813 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x557a1c1a36b0 0x557a1c5b1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:04.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.980+0000 7f6a22ffd640 1 -- 172.21.15.112:0/201401813 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557a1c1c4390 msgr2=0x557a1c595e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:29:04.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.980+0000 7f6a22ffd640 1 --2- 172.21.15.112:0/201401813 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557a1c1c4390 0x557a1c595e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:04.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.980+0000 7f6a22ffd640 1 -- 172.21.15.112:0/201401813 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a1c4d3fc0 con 0x557a1c19b4d0 2024-09-21T00:29:04.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.980+0000 7f6a237fe640 1 --2- 172.21.15.112:0/201401813 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557a1c1c4390 0x557a1c595e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:29:04.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.980+0000 7f6a22ffd640 1 --2- 172.21.15.112:0/201401813 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557a1c19b4d0 0x557a1c5acfe0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f6a14062dd0 tx=0x7f6a14066000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=368ea412c331e1c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:04.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.980+0000 7f6a08ff9640 1 -- 172.21.15.112:0/201401813 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a1401d040 con 0x557a1c19b4d0 2024-09-21T00:29:04.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.980+0000 7f6a08ff9640 1 -- 172.21.15.112:0/201401813 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6a14067a70 con 0x557a1c19b4d0 2024-09-21T00:29:04.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.980+0000 7f6a08ff9640 1 -- 172.21.15.112:0/201401813 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a14062590 con 0x557a1c19b4d0 2024-09-21T00:29:04.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.981+0000 7f6a308c8740 1 -- 172.21.15.112:0/201401813 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557a1c19b4d0 msgr2=0x557a1c5acfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:04.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.981+0000 7f6a308c8740 1 --2- 172.21.15.112:0/201401813 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557a1c19b4d0 0x557a1c5acfe0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f6a14062dd0 tx=0x7f6a14066000 comp rx=0 tx=0).stop 2024-09-21T00:29:04.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.981+0000 7f6a308c8740 1 -- 172.21.15.112:0/201401813 shutdown_connections 2024-09-21T00:29:04.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.981+0000 7f6a308c8740 1 --2- 172.21.15.112:0/201401813 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557a1c1c4390 0x557a1c595e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:04.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.981+0000 7f6a308c8740 1 --2- 172.21.15.112:0/201401813 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x557a1c1a36b0 0x557a1c5b1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:04.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.981+0000 7f6a308c8740 1 --2- 172.21.15.112:0/201401813 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557a1c19b4d0 0x557a1c5acfe0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:04.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.981+0000 7f6a308c8740 1 -- 172.21.15.112:0/201401813 >> 172.21.15.112:0/201401813 conn(0x557a1c1b84e0 msgr2=0x557a1c584fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:04.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.981+0000 7f6a308c8740 1 -- 172.21.15.112:0/201401813 shutdown_connections 2024-09-21T00:29:04.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.981+0000 7f6a308c8740 1 -- 172.21.15.112:0/201401813 wait complete. 2024-09-21T00:29:04.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.982+0000 7f6a308c8740 1 -- start start 2024-09-21T00:29:04.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.982+0000 7f6a308c8740 1 Processor -- start 2024-09-21T00:29:04.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.983+0000 7f6a308c8740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x557a1c19b4d0 0x557a1c626bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:04.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.983+0000 7f6a308c8740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557a1c1a36b0 0x557a1c595e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:04.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.983+0000 7f6a308c8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557a1c1c4390 0x557a1c5963b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:04.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.983+0000 7f6a308c8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x557a1c561ba0 con 0x557a1c1c4390 2024-09-21T00:29:04.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.983+0000 7f6a308c8740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f6a241f9be0 con 0x557a1c1a36b0 2024-09-21T00:29:04.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.983+0000 7f6a308c8740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x557a1c630700 con 0x557a1c19b4d0 2024-09-21T00:29:05.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.983+0000 7f6a308c8740 10 client.0 _ll_register_callbacks cb 0x7f6a1c001720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:29:05.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.983+0000 7f6a237fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557a1c1c4390 0x557a1c5963b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:05.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.984+0000 7f6a237fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557a1c1c4390 0x557a1c5963b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38442/0 (socket says 172.21.15.112:38442) 2024-09-21T00:29:05.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.984+0000 7f6a237fe640 1 -- 172.21.15.112:0/3153470218 learned_addr learned my addr 172.21.15.112:0/3153470218 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:29:05.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.984+0000 7f6a22ffd640 1 --2- 172.21.15.112:0/3153470218 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x557a1c19b4d0 0x557a1c626bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:05.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.984+0000 7f6a227fc640 1 --2- 172.21.15.112:0/3153470218 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557a1c1a36b0 0x557a1c595e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:05.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.984+0000 7f6a237fe640 1 -- 172.21.15.112:0/3153470218 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x557a1c19b4d0 msgr2=0x557a1c626bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:29:05.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.984+0000 7f6a237fe640 1 --2- 172.21.15.112:0/3153470218 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x557a1c19b4d0 0x557a1c626bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:05.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.984+0000 7f6a237fe640 1 -- 172.21.15.112:0/3153470218 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557a1c1a36b0 msgr2=0x557a1c595e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:29:05.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.984+0000 7f6a237fe640 1 --2- 172.21.15.112:0/3153470218 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557a1c1a36b0 0x557a1c595e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:05.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.984+0000 7f6a237fe640 1 -- 172.21.15.112:0/3153470218 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a1c627100 con 0x557a1c1c4390 2024-09-21T00:29:05.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.984+0000 7f6a227fc640 1 --2- 172.21.15.112:0/3153470218 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557a1c1a36b0 0x557a1c595e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:29:05.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.984+0000 7f6a237fe640 1 --2- 172.21.15.112:0/3153470218 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557a1c1c4390 0x557a1c5963b0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c0039a0 tx=0x7f6a1c0039d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7258a56fea49dfe9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:05.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.984+0000 7f6a027fc640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T00:29:05.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.984+0000 7f6a027fc640 1 -- 172.21.15.112:0/3153470218 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a1c02d020 con 0x557a1c1c4390 2024-09-21T00:29:05.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.984+0000 7f6a027fc640 1 -- 172.21.15.112:0/3153470218 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 6101+0+0 (secure 0 0 0) 0x7f6a1c021880 con 0x557a1c1c4390 2024-09-21T00:29:05.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.984+0000 7f6a308c8740 1 -- 172.21.15.112:0/3153470218 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a14067a70 con 0x557a1c1c4390 2024-09-21T00:29:05.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.984+0000 7f6a027fc640 1 -- 172.21.15.112:0/3153470218 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f6a1c021a80 con 0x557a1c1c4390 2024-09-21T00:29:05.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.984+0000 7f6a308c8740 10 client.6663 fetch_fsmap learned FSMap version 55 2024-09-21T00:29:05.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.984+0000 7f6a308c8740 1 -- 172.21.15.112:0/3153470218 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=2+}) -- 0x557a1c5d4b50 con 0x557a1c1c4390 2024-09-21T00:29:05.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.984+0000 7f6a027fc640 1 -- 172.21.15.112:0/3153470218 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6a14067a70 con 0x557a1c1c4390 2024-09-21T00:29:05.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.985+0000 7f6a027fc640 1 -- 172.21.15.112:0/3153470218 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 55) ==== 3502+0+0 (secure 0 0 0) 0x7f6a1c03a080 con 0x557a1c1c4390 2024-09-21T00:29:05.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.985+0000 7f6a308c8740 10 client.6663 fetch_fsmap finished waiting for FSMap version 55 2024-09-21T00:29:05.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.985+0000 7f6a308c8740 10 client.6663 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.d 2024-09-21T00:29:05.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.985+0000 7f6a308c8740 20 client.6663 populate_metadata read hostname 'smithi112' 2024-09-21T00:29:05.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.985+0000 7f6a308c8740 1 --2- 172.21.15.112:0/3153470218 >> [v2:172.21.15.187:6835/932692802,v1:172.21.15.187:6837/932692802] conn(0x557a1c6301e0 0x557a1c633ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:05.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.985+0000 7f6a308c8740 4 client.6663 mds_command: new command op to 6486 tid=0 multi_id=0 [{"prefix": "session ls"}] 2024-09-21T00:29:05.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.985+0000 7f6a308c8740 1 -- 172.21.15.112:0/3153470218 --> [v2:172.21.15.187:6835/932692802,v1:172.21.15.187:6837/932692802] -- command(tid 0: {"prefix": "session ls"}) -- 0x557a1c4dc2e0 con 0x557a1c6301e0 2024-09-21T00:29:05.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.985+0000 7f6a22ffd640 1 --2- 172.21.15.112:0/3153470218 >> [v2:172.21.15.187:6835/932692802,v1:172.21.15.187:6837/932692802] conn(0x557a1c6301e0 0x557a1c633ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:05.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.985+0000 7f6a027fc640 10 client.6663 ms_handle_connect on v2:172.21.15.187:6835/932692802 2024-09-21T00:29:05.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.985+0000 7f6a22ffd640 1 --2- 172.21.15.112:0/3153470218 >> [v2:172.21.15.187:6835/932692802,v1:172.21.15.187:6837/932692802] conn(0x557a1c6301e0 0x557a1c633ab0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b935772fe57d03d server_cookie=b1cb401ae6911c5e in_seq=0 out_seq=0 2024-09-21T00:29:05.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.985+0000 7f6a027fc640 1 -- 172.21.15.112:0/3153470218 <== mds.0 v2:172.21.15.187:6835/932692802 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557a1c4dc2e0 con 0x557a1c6301e0 2024-09-21T00:29:05.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.985+0000 7f6a027fc640 10 client.6663 handle_command_reply: tid=0 2024-09-21T00:29:05.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.985+0000 7f6a308c8740 1 client.6663 shutdown 2024-09-21T00:29:05.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.985+0000 7f6a308c8740 1 -- 172.21.15.112:0/3153470218 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557a1c1c4390 msgr2=0x557a1c5963b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:05.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.985+0000 7f6a308c8740 1 --2- 172.21.15.112:0/3153470218 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557a1c1c4390 0x557a1c5963b0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c0039a0 tx=0x7f6a1c0039d0 comp rx=0 tx=0).stop 2024-09-21T00:29:05.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.985+0000 7f6a308c8740 1 -- 172.21.15.112:0/3153470218 shutdown_connections 2024-09-21T00:29:05.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.985+0000 7f6a308c8740 1 --2- 172.21.15.112:0/3153470218 >> [v2:172.21.15.187:6835/932692802,v1:172.21.15.187:6837/932692802] conn(0x557a1c6301e0 0x557a1c633ab0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:05.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.985+0000 7f6a308c8740 1 --2- 172.21.15.112:0/3153470218 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557a1c1c4390 0x557a1c5963b0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:05.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.985+0000 7f6a027fc640 0 client.6663 ms_handle_reset on v2:172.21.15.187:6835/932692802 2024-09-21T00:29:05.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.985+0000 7f6a308c8740 1 --2- 172.21.15.112:0/3153470218 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557a1c1a36b0 0x557a1c595e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:05.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.985+0000 7f6a308c8740 1 --2- 172.21.15.112:0/3153470218 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x557a1c19b4d0 0x557a1c626bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:05.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.986+0000 7f6a308c8740 1 -- 172.21.15.112:0/3153470218 >> 172.21.15.112:0/3153470218 conn(0x557a1c1b84e0 msgr2=0x557a1c5828a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:05.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.986+0000 7f6a308c8740 1 -- 172.21.15.112:0/3153470218 shutdown_connections 2024-09-21T00:29:05.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.986+0000 7f6a308c8740 1 -- 172.21.15.112:0/3153470218 wait complete. 2024-09-21T00:29:05.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.986+0000 7f6a308c8740 20 client.6663 trim_cache size 0 max 16384 2024-09-21T00:29:05.003 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-09-21T00:29:05.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.986+0000 7f6a2995f640 1 -- 172.21.15.112:0/1069905293 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f6a10078b90 msgr2=0x7f6a1007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:05.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.986+0000 7f6a2995f640 1 --2- 172.21.15.112:0/1069905293 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f6a10078b90 0x7f6a1007b0a0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f6a18034000 tx=0x7f6a18004730 comp rx=0 tx=0).stop 2024-09-21T00:29:05.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.986+0000 7f6a2995f640 1 -- 172.21.15.112:0/1069905293 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a240d01b0 msgr2=0x7f6a2415fac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:05.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.986+0000 7f6a2995f640 1 --2- 172.21.15.112:0/1069905293 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a240d01b0 0x7f6a2415fac0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f6a1400c970 tx=0x7f6a1400ce30 comp rx=0 tx=0).stop 2024-09-21T00:29:05.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.986+0000 7f6a2995f640 1 -- 172.21.15.112:0/1069905293 shutdown_connections 2024-09-21T00:29:05.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.986+0000 7f6a2995f640 1 --2- 172.21.15.112:0/1069905293 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6a240d48a0 0x7f6a240ca120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:05.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.986+0000 7f6a2995f640 1 --2- 172.21.15.112:0/1069905293 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6a240d3880 0x7f6a241f6a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:05.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.986+0000 7f6a2995f640 1 --2- 172.21.15.112:0/1069905293 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f6a10078b90 0x7f6a1007b0a0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:05.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.986+0000 7f6a2995f640 1 --2- 172.21.15.112:0/1069905293 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a240d01b0 0x7f6a2415fac0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:05.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.986+0000 7f6a2995f640 1 -- 172.21.15.112:0/1069905293 >> 172.21.15.112:0/1069905293 conn(0x7f6a2408bfe0 msgr2=0x7f6a240b6220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:05.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.986+0000 7f6a2995f640 1 -- 172.21.15.112:0/1069905293 shutdown_connections 2024-09-21T00:29:05.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:04.986+0000 7f6a2995f640 1 -- 172.21.15.112:0/1069905293 wait complete. 2024-09-21T00:29:05.006 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json perf dump 2024-09-21T00:29:05.132 INFO:teuthology.orchestra.run.smithi187.stdout:{"AsyncMessenger::Worker-0":{"msgr_recv_messages":54,"msgr_send_messages":53,"msgr_recv_bytes":14022,"msgr_send_bytes":72497,"msgr_created_connections":11,"msgr_active_connections":1,"msgr_running_total_time":0.012221356,"msgr_running_send_time":0.001772666,"msgr_running_recv_time":0.008579642,"msgr_running_fast_dispatch_time":0.000601006,"msgr_send_messages_queue_lat":{"avgcount":52,"sum":0.061564504,"avgtime":0.001183932},"msgr_handle_ack_lat":{"avgcount":31,"sum":0.000011280,"avgtime":0.000000363},"msgr_recv_encrypted_bytes":1872,"msgr_send_encrypted_bytes":448},"AsyncMessenger::Worker-1":{"msgr_recv_messages":73,"msgr_send_messages":66,"msgr_recv_bytes":130073,"msgr_send_bytes":25646,"msgr_created_connections":5,"msgr_active_connections":3,"msgr_running_total_time":0.009790612,"msgr_running_send_time":0.002381650,"msgr_running_recv_time":0.010414585,"msgr_running_fast_dispatch_time":0.000536639,"msgr_send_messages_queue_lat":{"avgcount":66,"sum":0.146865046,"avgtime":0.002225227},"msgr_handle_ack_lat":{"avgcount":5,"sum":0.000003110,"avgtime":0.000000622},"msgr_recv_encrypted_bytes":123968,"msgr_send_encrypted_bytes":9952},"AsyncMessenger::Worker-2":{"msgr_recv_messages":12,"msgr_send_messages":17,"msgr_recv_bytes":3205,"msgr_send_bytes":99283,"msgr_created_connections":5,"msgr_active_connections":3,"msgr_running_total_time":0.004477909,"msgr_running_send_time":0.000719494,"msgr_running_recv_time":0.003176694,"msgr_running_fast_dispatch_time":0.000369962,"msgr_send_messages_queue_lat":{"avgcount":15,"sum":0.125401447,"avgtime":0.008360096},"msgr_handle_ack_lat":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"msgr_recv_encrypted_bytes":160,"msgr_send_encrypted_bytes":93952},"cct":{"total_workers":1,"unhealthy_workers":0},"finisher-MDSRank":{"queue_len":0,"complete_latency":{"avgcount":49,"sum":0.004729036,"avgtime":0.000096510}},"finisher-PurgeQueue":{"queue_len":0,"complete_latency":{"avgcount":2,"sum":0.000029337,"avgtime":0.000014668}},"mds":{"request":29,"reply":29,"reply_latency":{"avgcount":29,"sum":0.005397349,"avgtime":0.000186115},"slow_reply":0,"forward":0,"dir_fetch_complete":0,"dir_fetch_keys":0,"dir_commit":12,"dir_split":0,"dir_merge":0,"inodes":11,"inodes_top":0,"inodes_bottom":11,"inodes_pin_tail":0,"inodes_pinned":10,"inodes_expired":0,"inodes_with_caps":0,"caps":0,"subtrees":2,"traverse":55,"traverse_hit":53,"traverse_forward":0,"traverse_discover":0,"traverse_dir_fetch":0,"traverse_remote_ino":0,"traverse_lock":0,"load_cent":0,"q":0,"exported":0,"exported_inodes":0,"imported":0,"imported_inodes":0,"openino_dir_fetch":0,"openino_backtrace_fetch":0,"openino_peer_discover":0,"root_rfiles":0,"root_rbytes":0,"root_rsnaps":0,"scrub_backtrace_fetch":0,"scrub_set_tag":0,"scrub_backtrace_repaired":0,"scrub_inotable_repaired":0,"scrub_dir_inodes":0,"scrub_dir_base_inodes":0,"scrub_dirfrag_rstats":0,"scrub_file_inodes":0,"handle_inode_file_caps":0,"ceph_cap_op_revoke":0,"ceph_cap_op_grant":0,"ceph_cap_op_trunc":0,"ceph_cap_op_flushsnap_ack":0,"ceph_cap_op_flush_ack":0,"handle_client_caps":0,"handle_client_caps_dirty":0,"handle_client_cap_release":0,"process_request_cap_release":1},"mds_cache":{"dir_update":0,"dir_update_receipt":0,"dir_try_discover":0,"dir_send_discover":0,"dir_handle_discover":0,"num_strays":0,"num_strays_delayed":0,"num_strays_enqueuing":0,"strays_created":0,"strays_enqueued":0,"strays_reintegrated":0,"strays_migrated":0,"num_recovering_processing":0,"num_recovering_enqueued":0,"num_recovering_prioritized":0,"recovery_started":0,"recovery_completed":0,"ireq_quiesce_path":0,"ireq_quiesce_inode":0,"ireq_enqueue_scrub":0,"ireq_exportdir":0,"ireq_flush":0,"ireq_fragmentdir":0,"ireq_fragstats":0,"ireq_inodestats":0,"uninline_started":0,"uninline_succeeded":0,"uninline_write_failed":0},"mds_log":{"evlrg":0,"evadd":8,"evex":0,"evtrm":0,"ev":8,"evexg":0,"evexd":0,"segadd":2,"segex":0,"segtrm":0,"seg":2,"segmjr":2,"segexg":0,"segexd":0,"expos":4194304,"wrpos":4197025,"rdpos":0,"jlat":{"avgcount":5,"sum":0.013628416,"avgtime":0.002725683},"replayed":0},"mds_mem":{"ino":13,"ino+":13,"ino-":0,"dir":12,"dir+":12,"dir-":0,"dn":11,"dn+":11,"dn-":0,"cap":0,"cap+":2,"cap-":2,"rss":110360,"heap":332222464},"mds_server":{"dispatch_client_request":29,"dispatch_server_request":0,"handle_client_request":29,"handle_client_session":6,"handle_peer_request":0,"req_create_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_getattr_latency":{"avgcount":26,"sum":0.004705391,"avgtime":0.000180976},"req_getfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_link_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookup_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookuphash_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupino_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupname_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupparent_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lssnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mkdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mknod_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mksnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_open_latency":{"avgcount":2,"sum":0.000282387,"avgtime":0.000141193},"req_readdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rename_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_renamesnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_snapdiff_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setattr_latency":{"avgcount":1,"sum":0.000409571,"avgtime":0.000409571},"req_setdirlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_symlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_unlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"cap_revoke_eviction":0,"cap_acquisition_throttle":0,"req_getvxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"mds_sessions":{"session_count":0,"session_add":2,"session_remove":2,"sessions_open":0,"sessions_stale":0,"total_load":28,"average_load":14,"avg_session_uptime":0,"mdthresh_evicted":0},"mempool":{"bloom_filter_bytes":0,"bloom_filter_items":0,"bluestore_alloc_bytes":0,"bluestore_alloc_items":0,"bluestore_cache_data_bytes":0,"bluestore_cache_data_items":0,"bluestore_cache_onode_bytes":0,"bluestore_cache_onode_items":0,"bluestore_cache_meta_bytes":0,"bluestore_cache_meta_items":0,"bluestore_cache_other_bytes":0,"bluestore_cache_other_items":0,"bluestore_cache_buffer_bytes":0,"bluestore_cache_buffer_items":0,"bluestore_extent_bytes":0,"bluestore_extent_items":0,"bluestore_blob_bytes":0,"bluestore_blob_items":0,"bluestore_shared_blob_bytes":0,"bluestore_shared_blob_items":0,"bluestore_inline_bl_bytes":0,"bluestore_inline_bl_items":0,"bluestore_fsck_bytes":0,"bluestore_fsck_items":0,"bluestore_txc_bytes":0,"bluestore_txc_items":0,"bluestore_writing_deferred_bytes":0,"bluestore_writing_deferred_items":0,"bluestore_writing_bytes":0,"bluestore_writing_items":0,"bluefs_bytes":0,"bluefs_items":0,"bluefs_file_reader_bytes":0,"bluefs_file_reader_items":0,"bluefs_file_writer_bytes":0,"bluefs_file_writer_items":0,"buffer_anon_bytes":208625,"buffer_anon_items":73,"buffer_meta_bytes":0,"buffer_meta_items":0,"osd_bytes":0,"osd_items":0,"osd_mapbl_bytes":0,"osd_mapbl_items":0,"osd_pglog_bytes":0,"osd_pglog_items":0,"osdmap_bytes":5424,"osdmap_items":76,"osdmap_mapping_bytes":0,"osdmap_mapping_items":0,"pgmap_bytes":0,"pgmap_items":0,"mds_co_bytes":53360,"mds_co_items":216,"unittest_1_bytes":0,"unittest_1_items":0,"unittest_2_bytes":0,"unittest_2_items":0},"objecter":{"op_active":0,"op_laggy":0,"op_send":43,"op_send_bytes":13841,"op_resend":30,"op_reply":43,"op_latency":{"avgcount":43,"sum":0.967984880,"avgtime":0.022511276},"op_inflight":0,"oplen_avg":{"avgcount":43,"sum":67},"op":43,"op_r":0,"op_w":43,"op_rmw":0,"op_pg":0,"osdop_stat":0,"osdop_create":11,"osdop_read":0,"osdop_write":5,"osdop_writefull":7,"osdop_writesame":0,"osdop_append":0,"osdop_zero":0,"osdop_truncate":0,"osdop_delete":6,"osdop_mapext":0,"osdop_sparse_read":0,"osdop_clonerange":0,"osdop_getxattr":0,"osdop_setxattr":22,"osdop_cmpxattr":0,"osdop_rmxattr":0,"osdop_resetxattrs":0,"osdop_call":0,"osdop_watch":0,"osdop_notify":0,"osdop_src_cmpxattr":0,"osdop_pgls":0,"osdop_pgls_filter":0,"osdop_other":0,"linger_active":0,"linger_send":0,"linger_resend":0,"linger_ping":0,"poolop_active":0,"poolop_send":0,"poolop_resend":0,"poolstat_active":0,"poolstat_send":0,"poolstat_resend":0,"statfs_active":0,"statfs_send":0,"statfs_resend":0,"command_active":0,"command_send":0,"command_resend":0,"map_epoch":83,"map_full":0,"map_inc":1,"osd_sessions":6,"osd_session_open":6,"osd_session_close":0,"osd_laggy":0,"omap_wr":15,"omap_rd":0,"omap_del":1},"oft":{"omap_total_objs":1,"omap_total_kv_pairs":0,"omap_total_updates":0,"omap_total_removes":0},"purge_queue":{"pq_executing_ops":0,"pq_executing_ops_high_water":0,"pq_executing":0,"pq_executing_high_water":0,"pq_executed_ops":0,"pq_executed":0,"pq_item_in_journal":0},"throttle-msgr_dispatch_throttler-mds":{"val":0,"max":104857600,"get_started":0,"get":134,"get_sum":131516,"get_or_fail_fail":0,"get_or_fail_success":134,"take":0,"take_sum":0,"put":134,"put_sum":131516,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_bytes":{"val":0,"max":104857600,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":43,"take_sum":13841,"put":43,"put_sum":13841,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_ops":{"val":0,"max":1024,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":43,"take_sum":43,"put":43,"put_sum":43,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle":{"val":0,"max":3758096384,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":0,"take_sum":0,"put":0,"put_sum":0,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle-0x55c6ab427380":{"val":0,"max":3758096384,"get_started":0,"get":8,"get_sum":2721,"get_or_fail_fail":0,"get_or_fail_success":8,"take":0,"take_sum":0,"put":5,"put_sum":2721,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"trackedop":{"slow_ops_count":0}} 2024-09-21T00:29:05.134 DEBUG:tasks.cephfs.filesystem:_json_asok output { "AsyncMessenger::Worker-0": { "msgr_active_connections": 1, "msgr_created_connections": 11, "msgr_handle_ack_lat": { "avgcount": 31, "avgtime": 3.63e-07, "sum": 1.128e-05 }, "msgr_recv_bytes": 14022, "msgr_recv_encrypted_bytes": 1872, "msgr_recv_messages": 54, "msgr_running_fast_dispatch_time": 0.000601006, "msgr_running_recv_time": 0.008579642, "msgr_running_send_time": 0.001772666, "msgr_running_total_time": 0.012221356, "msgr_send_bytes": 72497, "msgr_send_encrypted_bytes": 448, "msgr_send_messages": 53, "msgr_send_messages_queue_lat": { "avgcount": 52, "avgtime": 0.001183932, "sum": 0.061564504 } }, "AsyncMessenger::Worker-1": { "msgr_active_connections": 3, "msgr_created_connections": 5, "msgr_handle_ack_lat": { "avgcount": 5, "avgtime": 6.22e-07, "sum": 3.11e-06 }, "msgr_recv_bytes": 130073, "msgr_recv_encrypted_bytes": 123968, "msgr_recv_messages": 73, "msgr_running_fast_dispatch_time": 0.000536639, "msgr_running_recv_time": 0.010414585, "msgr_running_send_time": 0.00238165, "msgr_running_total_time": 0.009790612, "msgr_send_bytes": 25646, "msgr_send_encrypted_bytes": 9952, "msgr_send_messages": 66, "msgr_send_messages_queue_lat": { "avgcount": 66, "avgtime": 0.002225227, "sum": 0.146865046 } }, "AsyncMessenger::Worker-2": { "msgr_active_connections": 3, "msgr_created_connections": 5, "msgr_handle_ack_lat": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "msgr_recv_bytes": 3205, "msgr_recv_encrypted_bytes": 160, "msgr_recv_messages": 12, "msgr_running_fast_dispatch_time": 0.000369962, "msgr_running_recv_time": 0.003176694, "msgr_running_send_time": 0.000719494, "msgr_running_total_time": 0.004477909, "msgr_send_bytes": 99283, "msgr_send_encrypted_bytes": 93952, "msgr_send_messages": 17, "msgr_send_messages_queue_lat": { "avgcount": 15, "avgtime": 0.008360096, "sum": 0.125401447 } }, "cct": { "total_workers": 1, "unhealthy_workers": 0 }, "finisher-MDSRank": { "complete_latency": { "avgcount": 49, "avgtime": 9.651e-05, "sum": 0.004729036 }, "queue_len": 0 }, "finisher-PurgeQueue": { "complete_latency": { "avgcount": 2, "avgtime": 1.4668e-05, "sum": 2.9337e-05 }, "queue_len": 0 }, "mds": { "caps": 0, "ceph_cap_op_flush_ack": 0, "ceph_cap_op_flushsnap_ack": 0, "ceph_cap_op_grant": 0, "ceph_cap_op_revoke": 0, "ceph_cap_op_trunc": 0, "dir_commit": 12, "dir_fetch_complete": 0, "dir_fetch_keys": 0, "dir_merge": 0, "dir_split": 0, "exported": 0, "exported_inodes": 0, "forward": 0, "handle_client_cap_release": 0, "handle_client_caps": 0, "handle_client_caps_dirty": 0, "handle_inode_file_caps": 0, "imported": 0, "imported_inodes": 0, "inodes": 11, "inodes_bottom": 11, "inodes_expired": 0, "inodes_pin_tail": 0, "inodes_pinned": 10, "inodes_top": 0, "inodes_with_caps": 0, "load_cent": 0, "openino_backtrace_fetch": 0, "openino_dir_fetch": 0, "openino_peer_discover": 0, "process_request_cap_release": 1, "q": 0, "reply": 29, "reply_latency": { "avgcount": 29, "avgtime": 0.000186115, "sum": 0.005397349 }, "request": 29, "root_rbytes": 0, "root_rfiles": 0, "root_rsnaps": 0, "scrub_backtrace_fetch": 0, "scrub_backtrace_repaired": 0, "scrub_dir_base_inodes": 0, "scrub_dir_inodes": 0, "scrub_dirfrag_rstats": 0, "scrub_file_inodes": 0, "scrub_inotable_repaired": 0, "scrub_set_tag": 0, "slow_reply": 0, "subtrees": 2, "traverse": 55, "traverse_dir_fetch": 0, "traverse_discover": 0, "traverse_forward": 0, "traverse_hit": 53, "traverse_lock": 0, "traverse_remote_ino": 0 }, "mds_cache": { "dir_handle_discover": 0, "dir_send_discover": 0, "dir_try_discover": 0, "dir_update": 0, "dir_update_receipt": 0, "ireq_enqueue_scrub": 0, "ireq_exportdir": 0, "ireq_flush": 0, "ireq_fragmentdir": 0, "ireq_fragstats": 0, "ireq_inodestats": 0, "ireq_quiesce_inode": 0, "ireq_quiesce_path": 0, "num_recovering_enqueued": 0, "num_recovering_prioritized": 0, "num_recovering_processing": 0, "num_strays": 0, "num_strays_delayed": 0, "num_strays_enqueuing": 0, "recovery_completed": 0, "recovery_started": 0, "strays_created": 0, "strays_enqueued": 0, "strays_migrated": 0, "strays_reintegrated": 0, "uninline_started": 0, "uninline_succeeded": 0, "uninline_write_failed": 0 }, "mds_log": { "ev": 8, "evadd": 8, "evex": 0, "evexd": 0, "evexg": 0, "evlrg": 0, "evtrm": 0, "expos": 4194304, "jlat": { "avgcount": 5, "avgtime": 0.002725683, "sum": 0.013628416 }, "rdpos": 0, "replayed": 0, "seg": 2, "segadd": 2, "segex": 0, "segexd": 0, "segexg": 0, "segmjr": 2, "segtrm": 0, "wrpos": 4197025 }, "mds_mem": { "cap": 0, "cap+": 2, "cap-": 2, "dir": 12, "dir+": 12, "dir-": 0, "dn": 11, "dn+": 11, "dn-": 0, "heap": 332222464, "ino": 13, "ino+": 13, "ino-": 0, "rss": 110360 }, "mds_server": { "cap_acquisition_throttle": 0, "cap_revoke_eviction": 0, "dispatch_client_request": 29, "dispatch_server_request": 0, "handle_client_request": 29, "handle_client_session": 6, "handle_peer_request": 0, "req_create_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_getattr_latency": { "avgcount": 26, "avgtime": 0.000180976, "sum": 0.004705391 }, "req_getfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_getvxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_link_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookup_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookuphash_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupino_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupname_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupparent_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lssnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mkdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mknod_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mksnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_open_latency": { "avgcount": 2, "avgtime": 0.000141193, "sum": 0.000282387 }, "req_readdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rename_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_renamesnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setattr_latency": { "avgcount": 1, "avgtime": 0.000409571, "sum": 0.000409571 }, "req_setdirlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_snapdiff_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_symlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_unlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "mds_sessions": { "average_load": 14, "avg_session_uptime": 0, "mdthresh_evicted": 0, "session_add": 2, "session_count": 0, "session_remove": 2, "sessions_open": 0, "sessions_stale": 0, "total_load": 28 }, "mempool": { "bloom_filter_bytes": 0, "bloom_filter_items": 0, "bluefs_bytes": 0, "bluefs_file_reader_bytes": 0, "bluefs_file_reader_items": 0, "bluefs_file_writer_bytes": 0, "bluefs_file_writer_items": 0, "bluefs_items": 0, "bluestore_alloc_bytes": 0, "bluestore_alloc_items": 0, "bluestore_blob_bytes": 0, "bluestore_blob_items": 0, "bluestore_cache_buffer_bytes": 0, "bluestore_cache_buffer_items": 0, "bluestore_cache_data_bytes": 0, "bluestore_cache_data_items": 0, "bluestore_cache_meta_bytes": 0, "bluestore_cache_meta_items": 0, "bluestore_cache_onode_bytes": 0, "bluestore_cache_onode_items": 0, "bluestore_cache_other_bytes": 0, "bluestore_cache_other_items": 0, "bluestore_extent_bytes": 0, "bluestore_extent_items": 0, "bluestore_fsck_bytes": 0, "bluestore_fsck_items": 0, "bluestore_inline_bl_bytes": 0, "bluestore_inline_bl_items": 0, "bluestore_shared_blob_bytes": 0, "bluestore_shared_blob_items": 0, "bluestore_txc_bytes": 0, "bluestore_txc_items": 0, "bluestore_writing_bytes": 0, "bluestore_writing_deferred_bytes": 0, "bluestore_writing_deferred_items": 0, "bluestore_writing_items": 0, "buffer_anon_bytes": 208625, "buffer_anon_items": 73, "buffer_meta_bytes": 0, "buffer_meta_items": 0, "mds_co_bytes": 53360, "mds_co_items": 216, "osd_bytes": 0, "osd_items": 0, "osd_mapbl_bytes": 0, "osd_mapbl_items": 0, "osd_pglog_bytes": 0, "osd_pglog_items": 0, "osdmap_bytes": 5424, "osdmap_items": 76, "osdmap_mapping_bytes": 0, "osdmap_mapping_items": 0, "pgmap_bytes": 0, "pgmap_items": 0, "unittest_1_bytes": 0, "unittest_1_items": 0, "unittest_2_bytes": 0, "unittest_2_items": 0 }, "objecter": { "command_active": 0, "command_resend": 0, "command_send": 0, "linger_active": 0, "linger_ping": 0, "linger_resend": 0, "linger_send": 0, "map_epoch": 83, "map_full": 0, "map_inc": 1, "omap_del": 1, "omap_rd": 0, "omap_wr": 15, "op": 43, "op_Infinitylight": 0, "op_active": 0, "op_laggy": 0, "op_latency": { "avgcount": 43, "avgtime": 0.022511276, "sum": 0.96798488 }, "op_pg": 0, "op_r": 0, "op_reply": 43, "op_resend": 30, "op_rmw": 0, "op_send": 43, "op_send_bytes": 13841, "op_w": 43, "oplen_avg": { "avgcount": 43, "sum": 67 }, "osd_laggy": 0, "osd_session_close": 0, "osd_session_open": 6, "osd_sessions": 6, "osdop_append": 0, "osdop_call": 0, "osdop_clonerange": 0, "osdop_cmpxattr": 0, "osdop_create": 11, "osdop_delete": 6, "osdop_getxattr": 0, "osdop_mapext": 0, "osdop_notify": 0, "osdop_other": 0, "osdop_pgls": 0, "osdop_pgls_filter": 0, "osdop_read": 0, "osdop_resetxattrs": 0, "osdop_rmxattr": 0, "osdop_setxattr": 22, "osdop_sparse_read": 0, "osdop_src_cmpxattr": 0, "osdop_stat": 0, "osdop_truncate": 0, "osdop_watch": 0, "osdop_write": 5, "osdop_writefull": 7, "osdop_writesame": 0, "osdop_zero": 0, "poolop_active": 0, "poolop_resend": 0, "poolop_send": 0, "poolstat_active": 0, "poolstat_resend": 0, "poolstat_send": 0, "statfs_active": 0, "statfs_resend": 0, "statfs_send": 0 }, "oft": { "omap_total_kv_pairs": 0, "omap_total_objs": 1, "omap_total_removes": 0, "omap_total_updates": 0 }, "purge_queue": { "pq_executed": 0, "pq_executed_ops": 0, "pq_executing": 0, "pq_executing_high_water": 0, "pq_executing_ops": 0, "pq_executing_ops_high_water": 0, "pq_item_in_journal": 0 }, "throttle-msgr_dispatch_throttler-mds": { "get": 134, "get_or_fail_fail": 0, "get_or_fail_success": 134, "get_started": 0, "get_sum": 131516, "max": 104857600, "put": 134, "put_sum": 131516, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_bytes": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 104857600, "put": 43, "put_sum": 13841, "take": 43, "take_sum": 13841, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_ops": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 1024, "put": 43, "put_sum": 43, "take": 43, "take_sum": 43, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 3758096384, "put": 0, "put_sum": 0, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle-0x55c6ab427380": { "get": 8, "get_or_fail_fail": 0, "get_or_fail_success": 8, "get_started": 0, "get_sum": 2721, "max": 3758096384, "put": 5, "put_sum": 2721, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "trackedop": { "slow_ops_count": 0 } } 2024-09-21T00:29:05.134 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-21T00:29:05.134 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T00:29:05.135 DEBUG:teuthology.orchestra.run.smithi112:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T00:29:05.179 DEBUG:teuthology.orchestra.run.smithi187:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T00:29:05.229 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T00:29:05.229 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T00:29:05.230 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T00:29:05.230 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:29:05.246 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056010,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1439 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248716k,nr_inodes=812179,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T00:29:05.247 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T00:29:05.247 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-09-21T00:29:05.247 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:29:05.263 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056013,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497440k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16835 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248720k,nr_inodes=812180,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497440k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T00:29:05.263 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T00:29:05.263 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T00:29:05.263 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:29:05.304 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056010,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1439 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248716k,nr_inodes=812179,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T00:29:05.304 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T00:29:05.304 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-09-21T00:29:05.304 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:29:05.321 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056013,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497440k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16835 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248720k,nr_inodes=812180,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497440k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T00:29:05.321 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T00:29:05.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:05.458+0000 7f910340e640 1 -- 172.21.15.112:0/1815641000 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90fc0b5ab0 msgr2=0x7f90fc0b0730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:05.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:05.458+0000 7f910340e640 1 --2- 172.21.15.112:0/1815641000 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90fc0b5ab0 0x7f90fc0b0730 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f90f8005fd0 tx=0x7f90f802a5d0 comp rx=0 tx=0).stop 2024-09-21T00:29:05.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:05.458+0000 7f910340e640 1 -- 172.21.15.112:0/1815641000 shutdown_connections 2024-09-21T00:29:05.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:05.458+0000 7f910340e640 1 --2- 172.21.15.112:0/1815641000 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f90fc15f7f0 0x7f90fc0cf0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:05.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:05.458+0000 7f910340e640 1 --2- 172.21.15.112:0/1815641000 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90fc0b5ab0 0x7f90fc0b0730 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:05.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:05.458+0000 7f910340e640 1 -- 172.21.15.112:0/1815641000 >> 172.21.15.112:0/1815641000 conn(0x7f90fc073fe0 msgr2=0x7f90fc0743f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:05.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:05.458+0000 7f910340e640 1 -- 172.21.15.112:0/1815641000 shutdown_connections 2024-09-21T00:29:05.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:05.458+0000 7f910340e640 1 -- 172.21.15.112:0/1815641000 wait complete. 2024-09-21T00:29:05.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:05.460+0000 7f910340e640 1 Processor -- start 2024-09-21T00:29:05.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:05.460+0000 7f910340e640 1 -- start start 2024-09-21T00:29:05.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:05.461+0000 7f910340e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90fc144dd0 0x7f90fc1f4750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:05.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:05.461+0000 7f910340e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f90fc0aaf50 0x7f90fc06d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:05.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:05.461+0000 7f910340e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f90fc0ab360 0x7f90fc06dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:05.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:05.461+0000 7f910340e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f90fc0cfda0 con 0x7f90fc144dd0 2024-09-21T00:29:05.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:05.461+0000 7f910340e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f90fc0cfc20 con 0x7f90fc0ab360 2024-09-21T00:29:05.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:05.461+0000 7f910340e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f90fc0cff20 con 0x7f90fc0aaf50 2024-09-21T00:29:05.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:05.461+0000 7f91011aa640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90fc144dd0 0x7f90fc1f4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:05.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:05.461+0000 7f91011aa640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90fc144dd0 0x7f90fc1f4750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38464/0 (socket says 172.21.15.112:38464) 2024-09-21T00:29:05.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:05.461+0000 7f91011aa640 1 -- 172.21.15.112:0/3955297082 learned_addr learned my addr 172.21.15.112:0/3955297082 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:29:05.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:05.461+0000 7f91019ab640 1 --2- 172.21.15.112:0/3955297082 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f90fc0ab360 0x7f90fc06dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:05.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:05.461+0000 7f91009a9640 1 --2- 172.21.15.112:0/3955297082 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f90fc0aaf50 0x7f90fc06d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:05.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:05.461+0000 7f91011aa640 1 -- 172.21.15.112:0/3955297082 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f90fc0aaf50 msgr2=0x7f90fc06d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:05.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:05.461+0000 7f91011aa640 1 --2- 172.21.15.112:0/3955297082 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f90fc0aaf50 0x7f90fc06d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:05.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:05.461+0000 7f91011aa640 1 -- 172.21.15.112:0/3955297082 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f90fc0ab360 msgr2=0x7f90fc06dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:05.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:05.461+0000 7f91011aa640 1 --2- 172.21.15.112:0/3955297082 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f90fc0ab360 0x7f90fc06dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:05.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:05.461+0000 7f91011aa640 1 -- 172.21.15.112:0/3955297082 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90fc1f83d0 con 0x7f90fc144dd0 2024-09-21T00:29:05.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:05.461+0000 7f91011aa640 1 --2- 172.21.15.112:0/3955297082 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90fc144dd0 0x7f90fc1f4750 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f90f8034040 tx=0x7f90f80038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:05.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:05.462+0000 7f90f1ffb640 1 -- 172.21.15.112:0/3955297082 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f803e030 con 0x7f90fc144dd0 2024-09-21T00:29:05.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:05.462+0000 7f90f1ffb640 1 -- 172.21.15.112:0/3955297082 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f90f8003ca0 con 0x7f90fc144dd0 2024-09-21T00:29:05.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:05.462+0000 7f910340e640 1 -- 172.21.15.112:0/3955297082 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90fc1f8840 con 0x7f90fc144dd0 2024-09-21T00:29:05.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:05.462+0000 7f90f1ffb640 1 -- 172.21.15.112:0/3955297082 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f8035040 con 0x7f90fc144dd0 2024-09-21T00:29:05.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:05.462+0000 7f910340e640 1 -- 172.21.15.112:0/3955297082 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90fc1f8e00 con 0x7f90fc144dd0 2024-09-21T00:29:05.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:05.463+0000 7f910340e640 1 -- 172.21.15.112:0/3955297082 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90fc0d0460 con 0x7f90fc144dd0 2024-09-21T00:29:05.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:05.463+0000 7f90f1ffb640 1 -- 172.21.15.112:0/3955297082 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f90f8041030 con 0x7f90fc144dd0 2024-09-21T00:29:05.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:05.464+0000 7f90f1ffb640 1 --2- 172.21.15.112:0/3955297082 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f90e0078b90 0x7f90e007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:05.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:05.464+0000 7f90f1ffb640 1 -- 172.21.15.112:0/3955297082 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 6101+0+0 (secure 0 0 0) 0x7f90f80bb390 con 0x7f90fc144dd0 2024-09-21T00:29:05.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:05.464+0000 7f91009a9640 1 --2- 172.21.15.112:0/3955297082 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f90e0078b90 0x7f90e007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:05.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:05.465+0000 7f91009a9640 1 --2- 172.21.15.112:0/3955297082 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f90e0078b90 0x7f90e007b0a0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f90fc0b04f0 tx=0x7f90ec009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:05.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:05.468+0000 7f90f1ffb640 1 -- 172.21.15.112:0/3955297082 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f90f8083110 con 0x7f90fc144dd0 2024-09-21T00:29:05.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:05.681+0000 7f910340e640 1 -- 172.21.15.112:0/3955297082 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f90fc074920 con 0x7f90fc144dd0 2024-09-21T00:29:05.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:05.681+0000 7f90f1ffb640 1 -- 172.21.15.112:0/3955297082 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) ==== 94+0+5231 (secure 0 0 0) 0x7f90f8087b40 con 0x7f90fc144dd0 2024-09-21T00:29:05.682 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:29:05.682 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":55,"btime":"2024-09-21T00:28:58:105880+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5600,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6836/2708152171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":2708152171},{"type":"v1","addr":"172.21.15.187:6836","nonce":2708152171}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50},{"gid":6471,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6834/3529228075","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":3529228075},{"type":"v1","addr":"172.21.15.112:6834","nonce":3529228075}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50},{"gid":6474,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/101289292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":101289292},{"type":"v1","addr":"172.21.15.112:6835","nonce":101289292}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T00:28:54.453199+0000","modified":"2024-09-21T00:28:57.469605+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6486},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6486":{"gid":6486,"name":"d","rank":0,"incarnation":51,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6837/932692802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6835","nonce":932692802},{"type":"v1","addr":"172.21.15.187:6837","nonce":932692802}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6486,"qdb_cluster":[6486]},"id":6}]} 2024-09-21T00:29:05.682 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 55 2024-09-21T00:29:05.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:05.683+0000 7f910340e640 1 -- 172.21.15.112:0/3955297082 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f90e0078b90 msgr2=0x7f90e007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:05.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:05.683+0000 7f910340e640 1 --2- 172.21.15.112:0/3955297082 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f90e0078b90 0x7f90e007b0a0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f90fc0b04f0 tx=0x7f90ec009040 comp rx=0 tx=0).stop 2024-09-21T00:29:05.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:05.683+0000 7f910340e640 1 -- 172.21.15.112:0/3955297082 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90fc144dd0 msgr2=0x7f90fc1f4750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:05.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:05.683+0000 7f910340e640 1 --2- 172.21.15.112:0/3955297082 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90fc144dd0 0x7f90fc1f4750 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f90f8034040 tx=0x7f90f80038a0 comp rx=0 tx=0).stop 2024-09-21T00:29:05.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:05.683+0000 7f910340e640 1 -- 172.21.15.112:0/3955297082 shutdown_connections 2024-09-21T00:29:05.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:05.683+0000 7f910340e640 1 --2- 172.21.15.112:0/3955297082 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f90fc0ab360 0x7f90fc06dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:05.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:05.683+0000 7f910340e640 1 --2- 172.21.15.112:0/3955297082 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f90fc0aaf50 0x7f90fc06d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:05.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:05.683+0000 7f910340e640 1 --2- 172.21.15.112:0/3955297082 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f90e0078b90 0x7f90e007b0a0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:05.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:05.683+0000 7f910340e640 1 --2- 172.21.15.112:0/3955297082 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90fc144dd0 0x7f90fc1f4750 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:05.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:05.683+0000 7f910340e640 1 -- 172.21.15.112:0/3955297082 >> 172.21.15.112:0/3955297082 conn(0x7f90fc073fe0 msgr2=0x7f90fc143f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:05.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:05.684+0000 7f910340e640 1 -- 172.21.15.112:0/3955297082 shutdown_connections 2024-09-21T00:29:05.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:05.684+0000 7f910340e640 1 -- 172.21.15.112:0/3955297082 wait complete. 2024-09-21T00:29:05.697 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T00:29:05.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:05.810+0000 7f58c4b90640 1 -- 172.21.15.112:0/520069660 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f58c00c6830 msgr2=0x7f58c00b1d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:05.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:05.810+0000 7f58c4b90640 1 --2- 172.21.15.112:0/520069660 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f58c00c6830 0x7f58c00b1d30 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f58c00b39c0 tx=0x7f58b802a980 comp rx=0 tx=0).stop 2024-09-21T00:29:05.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:05.810+0000 7f58c4b90640 1 -- 172.21.15.112:0/520069660 shutdown_connections 2024-09-21T00:29:05.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:05.810+0000 7f58c4b90640 1 --2- 172.21.15.112:0/520069660 >> v2:172.21.15.187:3300/0 conn(0x7f58c00c5ee0 0x7f58c00c62f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:05.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:05.810+0000 7f58c4b90640 1 --2- 172.21.15.112:0/520069660 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f58c00c6830 0x7f58c00b1d30 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:05.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:05.810+0000 7f58c4b90640 1 --2- 172.21.15.112:0/520069660 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f58c00ccbf0 0x7f58c00cd000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:05.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:05.810+0000 7f58c4b90640 1 -- 172.21.15.112:0/520069660 >> 172.21.15.112:0/520069660 conn(0x7f58c00b42e0 msgr2=0x7f58c00b4720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:05.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:05.810+0000 7f58c4b90640 1 -- 172.21.15.112:0/520069660 shutdown_connections 2024-09-21T00:29:05.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:05.810+0000 7f58c4b90640 1 -- 172.21.15.112:0/520069660 wait complete. 2024-09-21T00:29:05.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:05.812+0000 7f58c4b90640 1 Processor -- start 2024-09-21T00:29:05.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:05.812+0000 7f58c4b90640 1 -- start start 2024-09-21T00:29:05.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:05.813+0000 7f58c4b90640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f58c00c5ee0 0x7f58c015b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:05.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:05.814+0000 7f58c4b90640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f58c00c6830 0x7f58c00c9c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:05.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:05.814+0000 7f58c4b90640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f58c00ccbf0 0x7f58c00d2320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:05.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:05.814+0000 7f58c4b90640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f58c00b2270 con 0x7f58c00ccbf0 2024-09-21T00:29:05.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:05.814+0000 7f58c4b90640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f58c00c6e50 con 0x7f58c00c5ee0 2024-09-21T00:29:05.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:05.814+0000 7f58c4b90640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f58c00b23f0 con 0x7f58c00c6830 2024-09-21T00:29:05.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:05.814+0000 7f58bed9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f58c00ccbf0 0x7f58c00d2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:05.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:05.814+0000 7f58bed9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f58c00ccbf0 0x7f58c00d2320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38486/0 (socket says 172.21.15.112:38486) 2024-09-21T00:29:05.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:05.814+0000 7f58bed9d640 1 -- 172.21.15.112:0/3634430873 learned_addr learned my addr 172.21.15.112:0/3634430873 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:29:05.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:05.814+0000 7f58be59c640 1 --2- 172.21.15.112:0/3634430873 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f58c00c5ee0 0x7f58c015b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:05.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:05.814+0000 7f58bdd9b640 1 --2- 172.21.15.112:0/3634430873 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f58c00c6830 0x7f58c00c9c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:05.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:05.814+0000 7f58bed9d640 1 -- 172.21.15.112:0/3634430873 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f58c00c6830 msgr2=0x7f58c00c9c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:05.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:05.814+0000 7f58bed9d640 1 --2- 172.21.15.112:0/3634430873 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f58c00c6830 0x7f58c00c9c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:05.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:05.814+0000 7f58bed9d640 1 -- 172.21.15.112:0/3634430873 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f58c00c5ee0 msgr2=0x7f58c015b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:05.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:05.814+0000 7f58bed9d640 1 --2- 172.21.15.112:0/3634430873 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f58c00c5ee0 0x7f58c015b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:05.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:05.814+0000 7f58bed9d640 1 -- 172.21.15.112:0/3634430873 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58c01f30d0 con 0x7f58c00ccbf0 2024-09-21T00:29:05.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:05.814+0000 7f58bed9d640 1 --2- 172.21.15.112:0/3634430873 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f58c00ccbf0 0x7f58c00d2320 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f58b8033040 tx=0x7f58b8003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:05.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:05.814+0000 7f58aeffd640 1 -- 172.21.15.112:0/3634430873 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58b8034040 con 0x7f58c00ccbf0 2024-09-21T00:29:05.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:05.814+0000 7f58aeffd640 1 -- 172.21.15.112:0/3634430873 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f58b8004540 con 0x7f58c00ccbf0 2024-09-21T00:29:05.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:05.814+0000 7f58c4b90640 1 -- 172.21.15.112:0/3634430873 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58c01f33b0 con 0x7f58c00ccbf0 2024-09-21T00:29:05.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:05.814+0000 7f58aeffd640 1 -- 172.21.15.112:0/3634430873 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58b8045480 con 0x7f58c00ccbf0 2024-09-21T00:29:05.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:05.814+0000 7f58c4b90640 1 -- 172.21.15.112:0/3634430873 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58c01f3970 con 0x7f58c00ccbf0 2024-09-21T00:29:05.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:05.815+0000 7f58c4b90640 1 -- 172.21.15.112:0/3634430873 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58c00ca410 con 0x7f58c00ccbf0 2024-09-21T00:29:05.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:05.816+0000 7f58aeffd640 1 -- 172.21.15.112:0/3634430873 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f58b8037020 con 0x7f58c00ccbf0 2024-09-21T00:29:05.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:05.816+0000 7f58aeffd640 1 --2- 172.21.15.112:0/3634430873 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f58a0078b90 0x7f58a007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:05.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:05.817+0000 7f58aeffd640 1 -- 172.21.15.112:0/3634430873 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 6101+0+0 (secure 0 0 0) 0x7f58b80bc480 con 0x7f58c00ccbf0 2024-09-21T00:29:05.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:05.817+0000 7f58be59c640 1 --2- 172.21.15.112:0/3634430873 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f58a0078b90 0x7f58a007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:05.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:05.817+0000 7f58be59c640 1 --2- 172.21.15.112:0/3634430873 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f58a0078b90 0x7f58a007b0a0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f58b0009b30 tx=0x7f58b0005f90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:05.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:05.821+0000 7f58aeffd640 1 -- 172.21.15.112:0/3634430873 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f58c00ca410 con 0x7f58c00ccbf0 2024-09-21T00:29:06.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.032+0000 7f58c4b90640 1 -- 172.21.15.112:0/3634430873 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f58c00ca140 con 0x7f58c00ccbf0 2024-09-21T00:29:06.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.033+0000 7f58aeffd640 1 -- 172.21.15.112:0/3634430873 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) ==== 94+0+5231 (secure 0 0 0) 0x7f58b8084200 con 0x7f58c00ccbf0 2024-09-21T00:29:06.035 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:29:06.035 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":55,"btime":"2024-09-21T00:28:58:105880+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5600,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6836/2708152171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":2708152171},{"type":"v1","addr":"172.21.15.187:6836","nonce":2708152171}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50},{"gid":6471,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6834/3529228075","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":3529228075},{"type":"v1","addr":"172.21.15.112:6834","nonce":3529228075}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50},{"gid":6474,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/101289292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":101289292},{"type":"v1","addr":"172.21.15.112:6835","nonce":101289292}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T00:28:54.453199+0000","modified":"2024-09-21T00:28:57.469605+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6486},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6486":{"gid":6486,"name":"d","rank":0,"incarnation":51,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6837/932692802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6835","nonce":932692802},{"type":"v1","addr":"172.21.15.187:6837","nonce":932692802}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6486,"qdb_cluster":[6486]},"id":6}]} 2024-09-21T00:29:06.035 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 55 2024-09-21T00:29:06.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.035+0000 7f58c4b90640 1 -- 172.21.15.112:0/3634430873 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f58a0078b90 msgr2=0x7f58a007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:06.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.035+0000 7f58c4b90640 1 --2- 172.21.15.112:0/3634430873 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f58a0078b90 0x7f58a007b0a0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f58b0009b30 tx=0x7f58b0005f90 comp rx=0 tx=0).stop 2024-09-21T00:29:06.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.035+0000 7f58c4b90640 1 -- 172.21.15.112:0/3634430873 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f58c00ccbf0 msgr2=0x7f58c00d2320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:06.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.035+0000 7f58c4b90640 1 --2- 172.21.15.112:0/3634430873 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f58c00ccbf0 0x7f58c00d2320 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f58b8033040 tx=0x7f58b8003f10 comp rx=0 tx=0).stop 2024-09-21T00:29:06.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.035+0000 7f58c4b90640 1 -- 172.21.15.112:0/3634430873 shutdown_connections 2024-09-21T00:29:06.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.035+0000 7f58c4b90640 1 --2- 172.21.15.112:0/3634430873 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f58a0078b90 0x7f58a007b0a0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:06.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.035+0000 7f58c4b90640 1 --2- 172.21.15.112:0/3634430873 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f58c00ccbf0 0x7f58c00d2320 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:06.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.035+0000 7f58c4b90640 1 --2- 172.21.15.112:0/3634430873 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f58c00c6830 0x7f58c00c9c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:06.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.035+0000 7f58c4b90640 1 --2- 172.21.15.112:0/3634430873 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f58c00c5ee0 0x7f58c015b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:06.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.035+0000 7f58c4b90640 1 -- 172.21.15.112:0/3634430873 >> 172.21.15.112:0/3634430873 conn(0x7f58c00b42e0 msgr2=0x7f58c00721f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:06.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.035+0000 7f58c4b90640 1 -- 172.21.15.112:0/3634430873 shutdown_connections 2024-09-21T00:29:06.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.035+0000 7f58c4b90640 1 -- 172.21.15.112:0/3634430873 wait complete. 2024-09-21T00:29:06.049 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T00:29:06.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.159+0000 7f1053bb0640 1 -- 172.21.15.112:0/831212893 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f104c164eb0 msgr2=0x7f104c1672d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:06.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.159+0000 7f1053bb0640 1 --2- 172.21.15.112:0/831212893 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f104c164eb0 0x7f104c1672d0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f1044009830 tx=0x7f104402a3f0 comp rx=0 tx=0).stop 2024-09-21T00:29:06.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.159+0000 7f1053bb0640 1 -- 172.21.15.112:0/831212893 shutdown_connections 2024-09-21T00:29:06.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.159+0000 7f1053bb0640 1 --2- 172.21.15.112:0/831212893 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f104c164eb0 0x7f104c1672d0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:06.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.159+0000 7f1053bb0640 1 -- 172.21.15.112:0/831212893 >> 172.21.15.112:0/831212893 conn(0x7f104c0b42e0 msgr2=0x7f104c0b4720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:06.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.159+0000 7f1053bb0640 1 -- 172.21.15.112:0/831212893 shutdown_connections 2024-09-21T00:29:06.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.159+0000 7f1053bb0640 1 -- 172.21.15.112:0/831212893 wait complete. 2024-09-21T00:29:06.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.161+0000 7f1053bb0640 1 Processor -- start 2024-09-21T00:29:06.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.161+0000 7f1053bb0640 1 -- start start 2024-09-21T00:29:06.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.162+0000 7f1053bb0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f104c0b1d30 0x7f104c158ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:06.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.162+0000 7f1053bb0640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f104c0ccbf0 0x7f104c153ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:06.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.162+0000 7f1053bb0640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f104c164eb0 0x7f104c154e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:06.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.162+0000 7f1053bb0640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f104c167bd0 con 0x7f104c0b1d30 2024-09-21T00:29:06.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.162+0000 7f1053bb0640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f104c167a50 con 0x7f104c164eb0 2024-09-21T00:29:06.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.162+0000 7f1053bb0640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f104c167d50 con 0x7f104c0ccbf0 2024-09-21T00:29:06.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.162+0000 7f105194c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f104c0b1d30 0x7f104c158ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:06.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.162+0000 7f105194c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f104c0b1d30 0x7f104c158ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38494/0 (socket says 172.21.15.112:38494) 2024-09-21T00:29:06.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.162+0000 7f105194c640 1 -- 172.21.15.112:0/2158353121 learned_addr learned my addr 172.21.15.112:0/2158353121 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:29:06.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.162+0000 7f105214d640 1 --2- 172.21.15.112:0/2158353121 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f104c164eb0 0x7f104c154e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:06.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.162+0000 7f105114b640 1 --2- 172.21.15.112:0/2158353121 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f104c0ccbf0 0x7f104c153ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:06.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.162+0000 7f105194c640 1 -- 172.21.15.112:0/2158353121 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f104c0ccbf0 msgr2=0x7f104c153ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:06.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.162+0000 7f105194c640 1 --2- 172.21.15.112:0/2158353121 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f104c0ccbf0 0x7f104c153ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:06.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.162+0000 7f105194c640 1 -- 172.21.15.112:0/2158353121 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f104c164eb0 msgr2=0x7f104c154e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:06.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.162+0000 7f105194c640 1 --2- 172.21.15.112:0/2158353121 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f104c164eb0 0x7f104c154e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:06.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.162+0000 7f105194c640 1 -- 172.21.15.112:0/2158353121 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f104c0c96f0 con 0x7f104c0b1d30 2024-09-21T00:29:06.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.162+0000 7f105214d640 1 --2- 172.21.15.112:0/2158353121 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f104c164eb0 0x7f104c154e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:29:06.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.163+0000 7f105194c640 1 --2- 172.21.15.112:0/2158353121 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f104c0b1d30 0x7f104c158ea0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f103c00bab0 tx=0x7f103c00bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:06.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.163+0000 7f104a7fc640 1 -- 172.21.15.112:0/2158353121 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f103c019070 con 0x7f104c0b1d30 2024-09-21T00:29:06.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.163+0000 7f104a7fc640 1 -- 172.21.15.112:0/2158353121 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f103c00cd80 con 0x7f104c0b1d30 2024-09-21T00:29:06.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.163+0000 7f1053bb0640 1 -- 172.21.15.112:0/2158353121 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f104c154780 con 0x7f104c0b1d30 2024-09-21T00:29:06.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.163+0000 7f104a7fc640 1 -- 172.21.15.112:0/2158353121 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f103c0146a0 con 0x7f104c0b1d30 2024-09-21T00:29:06.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.163+0000 7f1053bb0640 1 -- 172.21.15.112:0/2158353121 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f104c1f6f20 con 0x7f104c0b1d30 2024-09-21T00:29:06.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.164+0000 7f1053bb0640 1 -- 172.21.15.112:0/2158353121 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f104c0cd000 con 0x7f104c0b1d30 2024-09-21T00:29:06.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.164+0000 7f104a7fc640 1 -- 172.21.15.112:0/2158353121 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f103c01c070 con 0x7f104c0b1d30 2024-09-21T00:29:06.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.165+0000 7f104a7fc640 1 --2- 172.21.15.112:0/2158353121 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1038078bc0 0x7f103807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:06.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.165+0000 7f104a7fc640 1 -- 172.21.15.112:0/2158353121 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 6101+0+0 (secure 0 0 0) 0x7f103c09ac10 con 0x7f104c0b1d30 2024-09-21T00:29:06.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.165+0000 7f105114b640 1 --2- 172.21.15.112:0/2158353121 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1038078bc0 0x7f103807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:06.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.166+0000 7f105114b640 1 --2- 172.21.15.112:0/2158353121 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1038078bc0 0x7f103807b0d0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f104c14a810 tx=0x7f1040009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:06.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.170+0000 7f104a7fc640 1 -- 172.21.15.112:0/2158353121 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f103c062f50 con 0x7f104c0b1d30 2024-09-21T00:29:06.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.350+0000 7f1053bb0640 1 -- 172.21.15.112:0/2158353121 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f104c0cc690 con 0x7f104c0b1d30 2024-09-21T00:29:06.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.351+0000 7f104a7fc640 1 -- 172.21.15.112:0/2158353121 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+16969 (secure 0 0 0) 0x7f103c062850 con 0x7f104c0b1d30 2024-09-21T00:29:06.353 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:29:06.353 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":83,"fsid":"8b34692b-dc65-49b2-a08f-26f707b96ac5","created":"2024-09-21T00:25:22.788490+0000","modified":"2024-09-21T00:28:56.093597+0000","last_up_change":"2024-09-21T00:25:36.161835+0000","last_in_change":"2024-09-21T00:25:26.346500+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T00:25:37.105006+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-09-21T00:28:52.107455+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-09-21T00:28:53.410975+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9c575086-67d2-4de2-b01b-d44dcc21d604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6809","nonce":755760621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6811","nonce":755760621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6815","nonce":755760621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6813","nonce":755760621}]},"public_addr":"172.21.15.112:6809/755760621","cluster_addr":"172.21.15.112:6811/755760621","heartbeat_back_addr":"172.21.15.112:6815/755760621","heartbeat_front_addr":"172.21.15.112:6813/755760621","state":["exists","up"]},{"osd":1,"uuid":"a983c227-a3ce-4bdd-82f1-438ba7e3ecd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6801","nonce":344932592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6803","nonce":344932592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6807","nonce":344932592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6805","nonce":344932592}]},"public_addr":"172.21.15.112:6801/344932592","cluster_addr":"172.21.15.112:6803/344932592","heartbeat_back_addr":"172.21.15.112:6807/344932592","heartbeat_front_addr":"172.21.15.112:6805/344932592","state":["exists","up"]},{"osd":2,"uuid":"1193c1d8-bfff-4a57-8200-5f69bf1ea3af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6825","nonce":407933616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6827","nonce":407933616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6831","nonce":407933616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6829","nonce":407933616}]},"public_addr":"172.21.15.112:6825/407933616","cluster_addr":"172.21.15.112:6827/407933616","heartbeat_back_addr":"172.21.15.112:6831/407933616","heartbeat_front_addr":"172.21.15.112:6829/407933616","state":["exists","up"]},{"osd":3,"uuid":"af04330a-36ad-40cc-a8dd-fa259b221de5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6817","nonce":4120164806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6819","nonce":4120164806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6823","nonce":4120164806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6821","nonce":4120164806}]},"public_addr":"172.21.15.112:6817/4120164806","cluster_addr":"172.21.15.112:6819/4120164806","heartbeat_back_addr":"172.21.15.112:6823/4120164806","heartbeat_front_addr":"172.21.15.112:6821/4120164806","state":["exists","up"]},{"osd":4,"uuid":"5b893111-292c-4c5a-b184-53ad38c7199e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6803","nonce":522077560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6805","nonce":522077560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6809","nonce":522077560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6807","nonce":522077560}]},"public_addr":"172.21.15.187:6803/522077560","cluster_addr":"172.21.15.187:6805/522077560","heartbeat_back_addr":"172.21.15.187:6809/522077560","heartbeat_front_addr":"172.21.15.187:6807/522077560","state":["exists","up"]},{"osd":5,"uuid":"c0b8e011-45fe-4c5e-925e-6be2a7f1868a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6827","nonce":3593144721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6829","nonce":3593144721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6833","nonce":3593144721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6831","nonce":3593144721}]},"public_addr":"172.21.15.187:6827/3593144721","cluster_addr":"172.21.15.187:6829/3593144721","heartbeat_back_addr":"172.21.15.187:6833/3593144721","heartbeat_front_addr":"172.21.15.187:6831/3593144721","state":["exists","up"]},{"osd":6,"uuid":"87d2d650-7489-4593-98a3-a1f433ce8dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6811","nonce":1494387657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6813","nonce":1494387657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6817","nonce":1494387657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6815","nonce":1494387657}]},"public_addr":"172.21.15.187:6811/1494387657","cluster_addr":"172.21.15.187:6813/1494387657","heartbeat_back_addr":"172.21.15.187:6817/1494387657","heartbeat_front_addr":"172.21.15.187:6815/1494387657","state":["exists","up"]},{"osd":7,"uuid":"c6b4f8e2-a1ee-4438-aa37-67ff4b6ef1f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6819","nonce":1624699842}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6821","nonce":1624699842}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6825","nonce":1624699842}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6823","nonce":1624699842}]},"public_addr":"172.21.15.187:6819/1624699842","cluster_addr":"172.21.15.187:6821/1624699842","heartbeat_back_addr":"172.21.15.187:6825/1624699842","heartbeat_front_addr":"172.21.15.187:6823/1624699842","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.873162+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.400506+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.664317+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.159791+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.202630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.113212+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.384786+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.448441+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T00:29:06.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.353+0000 7f1053bb0640 1 -- 172.21.15.112:0/2158353121 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1038078bc0 msgr2=0x7f103807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:06.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.353+0000 7f1053bb0640 1 --2- 172.21.15.112:0/2158353121 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1038078bc0 0x7f103807b0d0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f104c14a810 tx=0x7f1040009040 comp rx=0 tx=0).stop 2024-09-21T00:29:06.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.353+0000 7f1053bb0640 1 -- 172.21.15.112:0/2158353121 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f104c0b1d30 msgr2=0x7f104c158ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:06.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.353+0000 7f1053bb0640 1 --2- 172.21.15.112:0/2158353121 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f104c0b1d30 0x7f104c158ea0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f103c00bab0 tx=0x7f103c00bf70 comp rx=0 tx=0).stop 2024-09-21T00:29:06.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.353+0000 7f1053bb0640 1 -- 172.21.15.112:0/2158353121 shutdown_connections 2024-09-21T00:29:06.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.353+0000 7f1053bb0640 1 --2- 172.21.15.112:0/2158353121 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f104c164eb0 0x7f104c154e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:06.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.353+0000 7f1053bb0640 1 --2- 172.21.15.112:0/2158353121 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f104c0ccbf0 0x7f104c153ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:06.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.353+0000 7f1053bb0640 1 --2- 172.21.15.112:0/2158353121 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1038078bc0 0x7f103807b0d0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:06.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.353+0000 7f1053bb0640 1 --2- 172.21.15.112:0/2158353121 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f104c0b1d30 0x7f104c158ea0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:06.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.353+0000 7f1053bb0640 1 -- 172.21.15.112:0/2158353121 >> 172.21.15.112:0/2158353121 conn(0x7f104c0b42e0 msgr2=0x7f104c070990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:06.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.353+0000 7f1053bb0640 1 -- 172.21.15.112:0/2158353121 shutdown_connections 2024-09-21T00:29:06.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.353+0000 7f1053bb0640 1 -- 172.21.15.112:0/2158353121 wait complete. 2024-09-21T00:29:06.367 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-21T00:29:06.367 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T00:29:06.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.478+0000 7f7820227640 1 -- 172.21.15.112:0/3060091866 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f78180c5ee0 msgr2=0x7f78180c62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:06.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.478+0000 7f7820227640 1 --2- 172.21.15.112:0/3060091866 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f78180c5ee0 0x7f78180c62f0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f7818073f90 tx=0x7f780802a4d0 comp rx=0 tx=0).stop 2024-09-21T00:29:06.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.478+0000 7f7820227640 1 -- 172.21.15.112:0/3060091866 shutdown_connections 2024-09-21T00:29:06.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.478+0000 7f7820227640 1 --2- 172.21.15.112:0/3060091866 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f78180c5ee0 0x7f78180c62f0 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:06.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.478+0000 7f7820227640 1 --2- 172.21.15.112:0/3060091866 >> v2:172.21.15.187:3300/0 conn(0x7f78180ccbf0 0x7f78180cd000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:06.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.478+0000 7f7820227640 1 -- 172.21.15.112:0/3060091866 >> 172.21.15.112:0/3060091866 conn(0x7f78180b42e0 msgr2=0x7f78180b4720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:06.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.478+0000 7f7820227640 1 -- 172.21.15.112:0/3060091866 shutdown_connections 2024-09-21T00:29:06.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.478+0000 7f7820227640 1 -- 172.21.15.112:0/3060091866 wait complete. 2024-09-21T00:29:06.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.480+0000 7f7820227640 1 Processor -- start 2024-09-21T00:29:06.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.480+0000 7f7820227640 1 -- start start 2024-09-21T00:29:06.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.481+0000 7f7820227640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f78180c5ee0 0x7f78180b07c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:06.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.481+0000 7f7820227640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f78180c6830 0x7f7818156a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:06.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.481+0000 7f7820227640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f78180ccbf0 0x7f78181f6a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:06.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.481+0000 7f7820227640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f78180b3450 con 0x7f78180ccbf0 2024-09-21T00:29:06.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.481+0000 7f7820227640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f78180c6f90 con 0x7f78180c6830 2024-09-21T00:29:06.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.481+0000 7f7820227640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f78180b35d0 con 0x7f78180c5ee0 2024-09-21T00:29:06.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.481+0000 7f781e7c4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f78180ccbf0 0x7f78181f6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:06.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.481+0000 7f781e7c4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f78180ccbf0 0x7f78181f6a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39306/0 (socket says 172.21.15.112:39306) 2024-09-21T00:29:06.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.481+0000 7f781e7c4640 1 -- 172.21.15.112:0/2473888554 learned_addr learned my addr 172.21.15.112:0/2473888554 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:29:06.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.481+0000 7f781d7c2640 1 --2- 172.21.15.112:0/2473888554 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f78180c6830 0x7f7818156a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:06.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.481+0000 7f781dfc3640 1 --2- 172.21.15.112:0/2473888554 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f78180c5ee0 0x7f78180b07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:06.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.481+0000 7f781e7c4640 1 -- 172.21.15.112:0/2473888554 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f78180c5ee0 msgr2=0x7f78180b07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:06.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.481+0000 7f781e7c4640 1 --2- 172.21.15.112:0/2473888554 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f78180c5ee0 0x7f78180b07c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:06.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.481+0000 7f781e7c4640 1 -- 172.21.15.112:0/2473888554 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f78180c6830 msgr2=0x7f7818156a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:06.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.481+0000 7f781e7c4640 1 --2- 172.21.15.112:0/2473888554 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f78180c6830 0x7f7818156a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:06.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.481+0000 7f781e7c4640 1 -- 172.21.15.112:0/2473888554 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7818157760 con 0x7f78180ccbf0 2024-09-21T00:29:06.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.481+0000 7f781e7c4640 1 --2- 172.21.15.112:0/2473888554 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f78180ccbf0 0x7f78181f6a30 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f781000c700 tx=0x7f781000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:06.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.482+0000 7f780e7fc640 1 -- 172.21.15.112:0/2473888554 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7810003b70 con 0x7f78180ccbf0 2024-09-21T00:29:06.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.482+0000 7f780e7fc640 1 -- 172.21.15.112:0/2473888554 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7810003d10 con 0x7f78180ccbf0 2024-09-21T00:29:06.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.482+0000 7f780e7fc640 1 -- 172.21.15.112:0/2473888554 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f781000aa50 con 0x7f78180ccbf0 2024-09-21T00:29:06.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.482+0000 7f7820227640 1 -- 172.21.15.112:0/2473888554 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7818157a40 con 0x7f78180ccbf0 2024-09-21T00:29:06.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.482+0000 7f7820227640 1 -- 172.21.15.112:0/2473888554 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7818157d20 con 0x7f78180ccbf0 2024-09-21T00:29:06.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.483+0000 7f7820227640 1 -- 172.21.15.112:0/2473888554 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78180c6650 con 0x7f78180ccbf0 2024-09-21T00:29:06.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.483+0000 7f780e7fc640 1 -- 172.21.15.112:0/2473888554 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f781000abf0 con 0x7f78180ccbf0 2024-09-21T00:29:06.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.484+0000 7f780e7fc640 1 --2- 172.21.15.112:0/2473888554 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f77fc078b90 0x7f77fc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:06.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.484+0000 7f780e7fc640 1 -- 172.21.15.112:0/2473888554 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 6101+0+0 (secure 0 0 0) 0x7f781009ab50 con 0x7f78180ccbf0 2024-09-21T00:29:06.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.484+0000 7f781dfc3640 1 --2- 172.21.15.112:0/2473888554 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f77fc078b90 0x7f77fc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:06.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.485+0000 7f781dfc3640 1 --2- 172.21.15.112:0/2473888554 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f77fc078b90 0x7f77fc07b0a0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f7814005fd0 tx=0x7f781400b2e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:06.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.488+0000 7f780e7fc640 1 -- 172.21.15.112:0/2473888554 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f78180c6650 con 0x7f78180ccbf0 2024-09-21T00:29:06.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.700+0000 7f7820227640 1 -- 172.21.15.112:0/2473888554 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7818166310 con 0x7f78180ccbf0 2024-09-21T00:29:06.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.701+0000 7f780e7fc640 1 -- 172.21.15.112:0/2473888554 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) ==== 94+0+5231 (secure 0 0 0) 0x7f78180cd540 con 0x7f78180ccbf0 2024-09-21T00:29:06.703 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:29:06.703 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":55,"btime":"2024-09-21T00:28:58:105880+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5600,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6836/2708152171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":2708152171},{"type":"v1","addr":"172.21.15.187:6836","nonce":2708152171}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50},{"gid":6471,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6834/3529228075","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":3529228075},{"type":"v1","addr":"172.21.15.112:6834","nonce":3529228075}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50},{"gid":6474,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/101289292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":101289292},{"type":"v1","addr":"172.21.15.112:6835","nonce":101289292}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T00:28:54.453199+0000","modified":"2024-09-21T00:28:57.469605+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6486},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6486":{"gid":6486,"name":"d","rank":0,"incarnation":51,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6837/932692802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6835","nonce":932692802},{"type":"v1","addr":"172.21.15.187:6837","nonce":932692802}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6486,"qdb_cluster":[6486]},"id":6}]} 2024-09-21T00:29:06.703 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 55 2024-09-21T00:29:06.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.703+0000 7f7820227640 1 -- 172.21.15.112:0/2473888554 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f77fc078b90 msgr2=0x7f77fc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:06.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.703+0000 7f7820227640 1 --2- 172.21.15.112:0/2473888554 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f77fc078b90 0x7f77fc07b0a0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f7814005fd0 tx=0x7f781400b2e0 comp rx=0 tx=0).stop 2024-09-21T00:29:06.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.703+0000 7f7820227640 1 -- 172.21.15.112:0/2473888554 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f78180ccbf0 msgr2=0x7f78181f6a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:06.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.703+0000 7f7820227640 1 --2- 172.21.15.112:0/2473888554 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f78180ccbf0 0x7f78181f6a30 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f781000c700 tx=0x7f781000cbc0 comp rx=0 tx=0).stop 2024-09-21T00:29:06.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.703+0000 7f7820227640 1 -- 172.21.15.112:0/2473888554 shutdown_connections 2024-09-21T00:29:06.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.703+0000 7f7820227640 1 --2- 172.21.15.112:0/2473888554 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f77fc078b90 0x7f77fc07b0a0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:06.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.703+0000 7f7820227640 1 --2- 172.21.15.112:0/2473888554 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f78180ccbf0 0x7f78181f6a30 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:06.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.703+0000 7f7820227640 1 --2- 172.21.15.112:0/2473888554 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f78180c6830 0x7f7818156a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:06.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.703+0000 7f7820227640 1 --2- 172.21.15.112:0/2473888554 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f78180c5ee0 0x7f78180b07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:06.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.703+0000 7f7820227640 1 -- 172.21.15.112:0/2473888554 >> 172.21.15.112:0/2473888554 conn(0x7f78180b42e0 msgr2=0x7f781815b740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:06.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.703+0000 7f7820227640 1 -- 172.21.15.112:0/2473888554 shutdown_connections 2024-09-21T00:29:06.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.703+0000 7f7820227640 1 -- 172.21.15.112:0/2473888554 wait complete. 2024-09-21T00:29:06.716 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T00:29:06.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.827+0000 7f81e2994640 1 -- 172.21.15.112:0/40325951 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81dc0d01b0 msgr2=0x7f81dc0d05c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:06.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.827+0000 7f81e2994640 1 --2- 172.21.15.112:0/40325951 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81dc0d01b0 0x7f81dc0d05c0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f81d4005fd0 tx=0x7f81d402a8b0 comp rx=0 tx=0).stop 2024-09-21T00:29:06.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.827+0000 7f81e2994640 1 -- 172.21.15.112:0/40325951 shutdown_connections 2024-09-21T00:29:06.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.827+0000 7f81e2994640 1 --2- 172.21.15.112:0/40325951 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81dc0d01b0 0x7f81dc0d05c0 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:06.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.827+0000 7f81e2994640 1 -- 172.21.15.112:0/40325951 >> 172.21.15.112:0/40325951 conn(0x7f81dc08bfe0 msgr2=0x7f81dc09d890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:06.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.827+0000 7f81e2994640 1 -- 172.21.15.112:0/40325951 shutdown_connections 2024-09-21T00:29:06.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.827+0000 7f81e2994640 1 -- 172.21.15.112:0/40325951 wait complete. 2024-09-21T00:29:06.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.829+0000 7f81e2994640 1 Processor -- start 2024-09-21T00:29:06.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.829+0000 7f81e2994640 1 -- start start 2024-09-21T00:29:06.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.830+0000 7f81e2994640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81dc0d01b0 0x7f81dc0ca3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:06.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.830+0000 7f81e2994640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f81dc0d3880 0x7f81dc161570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:06.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.830+0000 7f81e2994640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f81dc0d48a0 0x7f81dc161ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:06.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.830+0000 7f81e2994640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f81dc0d2600 con 0x7f81dc0d01b0 2024-09-21T00:29:06.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.830+0000 7f81e2994640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f81dc0d2480 con 0x7f81dc0d48a0 2024-09-21T00:29:06.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.830+0000 7f81e2994640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f81dc0d2780 con 0x7f81dc0d3880 2024-09-21T00:29:06.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.830+0000 7f81dbfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81dc0d01b0 0x7f81dc0ca3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:06.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.830+0000 7f81dbfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81dc0d01b0 0x7f81dc0ca3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39312/0 (socket says 172.21.15.112:39312) 2024-09-21T00:29:06.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.830+0000 7f81dbfff640 1 -- 172.21.15.112:0/2953449157 learned_addr learned my addr 172.21.15.112:0/2953449157 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:29:06.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.831+0000 7f81e0f31640 1 --2- 172.21.15.112:0/2953449157 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f81dc0d48a0 0x7f81dc161ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:06.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.831+0000 7f81db7fe640 1 --2- 172.21.15.112:0/2953449157 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f81dc0d3880 0x7f81dc161570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:06.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.831+0000 7f81dbfff640 1 -- 172.21.15.112:0/2953449157 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f81dc0d3880 msgr2=0x7f81dc161570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:06.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.831+0000 7f81dbfff640 1 --2- 172.21.15.112:0/2953449157 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f81dc0d3880 0x7f81dc161570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:06.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.831+0000 7f81dbfff640 1 -- 172.21.15.112:0/2953449157 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f81dc0d48a0 msgr2=0x7f81dc161ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:06.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.831+0000 7f81dbfff640 1 --2- 172.21.15.112:0/2953449157 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f81dc0d48a0 0x7f81dc161ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:06.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.831+0000 7f81dbfff640 1 -- 172.21.15.112:0/2953449157 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81dc06f640 con 0x7f81dc0d01b0 2024-09-21T00:29:06.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.831+0000 7f81dbfff640 1 --2- 172.21.15.112:0/2953449157 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81dc0d01b0 0x7f81dc0ca3b0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f81cc00c3c0 tx=0x7f81cc00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:06.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.831+0000 7f81d8ff9640 1 -- 172.21.15.112:0/2953449157 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81cc017070 con 0x7f81dc0d01b0 2024-09-21T00:29:06.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.831+0000 7f81d8ff9640 1 -- 172.21.15.112:0/2953449157 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f81cc0035f0 con 0x7f81dc0d01b0 2024-09-21T00:29:06.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.831+0000 7f81e2994640 1 -- 172.21.15.112:0/2953449157 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81dc06f920 con 0x7f81dc0d01b0 2024-09-21T00:29:06.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.831+0000 7f81d8ff9640 1 -- 172.21.15.112:0/2953449157 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81cc003a80 con 0x7f81dc0d01b0 2024-09-21T00:29:06.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.831+0000 7f81e2994640 1 -- 172.21.15.112:0/2953449157 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81dc0cb670 con 0x7f81dc0d01b0 2024-09-21T00:29:06.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.832+0000 7f81e2994640 1 -- 172.21.15.112:0/2953449157 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81dc0b3d70 con 0x7f81dc0d01b0 2024-09-21T00:29:06.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.833+0000 7f81d8ff9640 1 -- 172.21.15.112:0/2953449157 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f81cc01a070 con 0x7f81dc0d01b0 2024-09-21T00:29:06.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.833+0000 7f81d8ff9640 1 --2- 172.21.15.112:0/2953449157 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f81c8078bc0 0x7f81c807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:06.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.833+0000 7f81d8ff9640 1 -- 172.21.15.112:0/2953449157 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 6101+0+0 (secure 0 0 0) 0x7f81cc09a1a0 con 0x7f81dc0d01b0 2024-09-21T00:29:06.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.833+0000 7f81db7fe640 1 --2- 172.21.15.112:0/2953449157 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f81c8078bc0 0x7f81c807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:06.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.834+0000 7f81db7fe640 1 --2- 172.21.15.112:0/2953449157 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f81c8078bc0 0x7f81c807b0d0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f81dc15e720 tx=0x7f81d0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:06.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:06.838+0000 7f81d8ff9640 1 -- 172.21.15.112:0/2953449157 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f81dc0b3d70 con 0x7f81dc0d01b0 2024-09-21T00:29:07.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.050+0000 7f81e2994640 1 -- 172.21.15.112:0/2953449157 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f81dc0d05c0 con 0x7f81dc0d01b0 2024-09-21T00:29:07.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.050+0000 7f81d8ff9640 1 -- 172.21.15.112:0/2953449157 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) ==== 94+0+5231 (secure 0 0 0) 0x7f81cc062450 con 0x7f81dc0d01b0 2024-09-21T00:29:07.052 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:29:07.052 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":55,"btime":"2024-09-21T00:28:58:105880+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5600,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6836/2708152171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":2708152171},{"type":"v1","addr":"172.21.15.187:6836","nonce":2708152171}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50},{"gid":6471,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6834/3529228075","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":3529228075},{"type":"v1","addr":"172.21.15.112:6834","nonce":3529228075}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50},{"gid":6474,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/101289292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":101289292},{"type":"v1","addr":"172.21.15.112:6835","nonce":101289292}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T00:28:54.453199+0000","modified":"2024-09-21T00:28:57.469605+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6486},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6486":{"gid":6486,"name":"d","rank":0,"incarnation":51,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6837/932692802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6835","nonce":932692802},{"type":"v1","addr":"172.21.15.187:6837","nonce":932692802}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6486,"qdb_cluster":[6486]},"id":6}]} 2024-09-21T00:29:07.053 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 55 2024-09-21T00:29:07.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.053+0000 7f81e2994640 1 -- 172.21.15.112:0/2953449157 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f81c8078bc0 msgr2=0x7f81c807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:07.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.053+0000 7f81e2994640 1 --2- 172.21.15.112:0/2953449157 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f81c8078bc0 0x7f81c807b0d0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f81dc15e720 tx=0x7f81d0008040 comp rx=0 tx=0).stop 2024-09-21T00:29:07.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.053+0000 7f81e2994640 1 -- 172.21.15.112:0/2953449157 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81dc0d01b0 msgr2=0x7f81dc0ca3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:07.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.053+0000 7f81e2994640 1 --2- 172.21.15.112:0/2953449157 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81dc0d01b0 0x7f81dc0ca3b0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f81cc00c3c0 tx=0x7f81cc00c880 comp rx=0 tx=0).stop 2024-09-21T00:29:07.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.053+0000 7f81e2994640 1 -- 172.21.15.112:0/2953449157 shutdown_connections 2024-09-21T00:29:07.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.053+0000 7f81e2994640 1 --2- 172.21.15.112:0/2953449157 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f81dc0d48a0 0x7f81dc161ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:07.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.053+0000 7f81e2994640 1 --2- 172.21.15.112:0/2953449157 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f81dc0d3880 0x7f81dc161570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:07.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.053+0000 7f81e2994640 1 --2- 172.21.15.112:0/2953449157 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f81c8078bc0 0x7f81c807b0d0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:07.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.053+0000 7f81e2994640 1 --2- 172.21.15.112:0/2953449157 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81dc0d01b0 0x7f81dc0ca3b0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:07.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.053+0000 7f81e2994640 1 -- 172.21.15.112:0/2953449157 >> 172.21.15.112:0/2953449157 conn(0x7f81dc08bfe0 msgr2=0x7f81dc068d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:07.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.053+0000 7f81e2994640 1 -- 172.21.15.112:0/2953449157 shutdown_connections 2024-09-21T00:29:07.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.053+0000 7f81e2994640 1 -- 172.21.15.112:0/2953449157 wait complete. 2024-09-21T00:29:07.066 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T00:29:07.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.179+0000 7f2efa24d640 1 -- 172.21.15.112:0/188013086 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2edc0099b0 msgr2=0x7f2edc00bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:07.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.179+0000 7f2efa24d640 1 --2- 172.21.15.112:0/188013086 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2edc0099b0 0x7f2edc00bdd0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f2eec006fd0 tx=0x7f2edc005830 comp rx=0 tx=0).stop 2024-09-21T00:29:07.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.179+0000 7f2efa24d640 1 -- 172.21.15.112:0/188013086 shutdown_connections 2024-09-21T00:29:07.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.179+0000 7f2efa24d640 1 --2- 172.21.15.112:0/188013086 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2edc0099b0 0x7f2edc00bdd0 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:07.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.179+0000 7f2efa24d640 1 -- 172.21.15.112:0/188013086 >> 172.21.15.112:0/188013086 conn(0x7f2ef40b42e0 msgr2=0x7f2ef40b4720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:07.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.179+0000 7f2efa24d640 1 -- 172.21.15.112:0/188013086 shutdown_connections 2024-09-21T00:29:07.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.179+0000 7f2efa24d640 1 -- 172.21.15.112:0/188013086 wait complete. 2024-09-21T00:29:07.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.181+0000 7f2efa24d640 1 Processor -- start 2024-09-21T00:29:07.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.181+0000 7f2efa24d640 1 -- start start 2024-09-21T00:29:07.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.182+0000 7f2efa24d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2edc0099b0 0x7f2ef40ad5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:07.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.182+0000 7f2efa24d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2ef40cb090 0x7f2ef40cb4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:07.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.182+0000 7f2efa24d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ef41543b0 0x7f2ef41547c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:07.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.182+0000 7f2efa24d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2ef40cd000 con 0x7f2ef41543b0 2024-09-21T00:29:07.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.182+0000 7f2efa24d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f2ef40b23a0 con 0x7f2ef40cb090 2024-09-21T00:29:07.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.182+0000 7f2efa24d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f2ef40cd180 con 0x7f2edc0099b0 2024-09-21T00:29:07.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.182+0000 7f2ef3fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ef41543b0 0x7f2ef41547c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:07.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.182+0000 7f2ef3fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ef41543b0 0x7f2ef41547c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39332/0 (socket says 172.21.15.112:39332) 2024-09-21T00:29:07.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.182+0000 7f2ef3fff640 1 -- 172.21.15.112:0/3438214324 learned_addr learned my addr 172.21.15.112:0/3438214324 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:29:07.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.182+0000 7f2ef2ffd640 1 --2- 172.21.15.112:0/3438214324 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2ef40cb090 0x7f2ef40cb4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:07.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.182+0000 7f2ef37fe640 1 --2- 172.21.15.112:0/3438214324 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2edc0099b0 0x7f2ef40ad5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:07.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.182+0000 7f2ef3fff640 1 -- 172.21.15.112:0/3438214324 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2edc0099b0 msgr2=0x7f2ef40ad5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:07.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.182+0000 7f2ef3fff640 1 --2- 172.21.15.112:0/3438214324 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2edc0099b0 0x7f2ef40ad5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:07.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.182+0000 7f2ef3fff640 1 -- 172.21.15.112:0/3438214324 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2ef40cb090 msgr2=0x7f2ef40cb4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:07.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.182+0000 7f2ef3fff640 1 --2- 172.21.15.112:0/3438214324 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2ef40cb090 0x7f2ef40cb4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:07.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.182+0000 7f2ef3fff640 1 -- 172.21.15.112:0/3438214324 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ef4154d00 con 0x7f2ef41543b0 2024-09-21T00:29:07.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.182+0000 7f2ef2ffd640 1 --2- 172.21.15.112:0/3438214324 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2ef40cb090 0x7f2ef40cb4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:29:07.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.183+0000 7f2ef3fff640 1 --2- 172.21.15.112:0/3438214324 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ef41543b0 0x7f2ef41547c0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f2eec02b490 tx=0x7f2eec004bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:07.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.183+0000 7f2edbfff640 1 -- 172.21.15.112:0/3438214324 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eec02b7e0 con 0x7f2ef41543b0 2024-09-21T00:29:07.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.183+0000 7f2edbfff640 1 -- 172.21.15.112:0/3438214324 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2eec02b980 con 0x7f2ef41543b0 2024-09-21T00:29:07.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.183+0000 7f2efa24d640 1 -- 172.21.15.112:0/3438214324 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ef40b01d0 con 0x7f2ef41543b0 2024-09-21T00:29:07.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.183+0000 7f2edbfff640 1 -- 172.21.15.112:0/3438214324 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eec02bd90 con 0x7f2ef41543b0 2024-09-21T00:29:07.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.183+0000 7f2efa24d640 1 -- 172.21.15.112:0/3438214324 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ef40b0790 con 0x7f2ef41543b0 2024-09-21T00:29:07.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.184+0000 7f2efa24d640 1 -- 172.21.15.112:0/3438214324 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ef40b3250 con 0x7f2ef41543b0 2024-09-21T00:29:07.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.185+0000 7f2edbfff640 1 -- 172.21.15.112:0/3438214324 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2eec03d030 con 0x7f2ef41543b0 2024-09-21T00:29:07.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.185+0000 7f2edbfff640 1 --2- 172.21.15.112:0/3438214324 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2ee0078bc0 0x7f2ee007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:07.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.185+0000 7f2edbfff640 1 -- 172.21.15.112:0/3438214324 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 6101+0+0 (secure 0 0 0) 0x7f2eec0ba780 con 0x7f2ef41543b0 2024-09-21T00:29:07.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.185+0000 7f2ef37fe640 1 --2- 172.21.15.112:0/3438214324 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2ee0078bc0 0x7f2ee007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:07.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.186+0000 7f2ef37fe640 1 --2- 172.21.15.112:0/3438214324 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2ee0078bc0 0x7f2ee007b0d0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f2ee4000c00 tx=0x7f2ee4006a20 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:07.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.190+0000 7f2edbfff640 1 -- 172.21.15.112:0/3438214324 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2ef4154d00 con 0x7f2ef41543b0 2024-09-21T00:29:07.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.369+0000 7f2efa24d640 1 -- 172.21.15.112:0/3438214324 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2ef40715d0 con 0x7f2ef41543b0 2024-09-21T00:29:07.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.370+0000 7f2edbfff640 1 -- 172.21.15.112:0/3438214324 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+16969 (secure 0 0 0) 0x7f2eec082a30 con 0x7f2ef41543b0 2024-09-21T00:29:07.372 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:29:07.372 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":83,"fsid":"8b34692b-dc65-49b2-a08f-26f707b96ac5","created":"2024-09-21T00:25:22.788490+0000","modified":"2024-09-21T00:28:56.093597+0000","last_up_change":"2024-09-21T00:25:36.161835+0000","last_in_change":"2024-09-21T00:25:26.346500+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T00:25:37.105006+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-09-21T00:28:52.107455+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-09-21T00:28:53.410975+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9c575086-67d2-4de2-b01b-d44dcc21d604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6809","nonce":755760621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6811","nonce":755760621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6815","nonce":755760621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6813","nonce":755760621}]},"public_addr":"172.21.15.112:6809/755760621","cluster_addr":"172.21.15.112:6811/755760621","heartbeat_back_addr":"172.21.15.112:6815/755760621","heartbeat_front_addr":"172.21.15.112:6813/755760621","state":["exists","up"]},{"osd":1,"uuid":"a983c227-a3ce-4bdd-82f1-438ba7e3ecd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6801","nonce":344932592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6803","nonce":344932592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6807","nonce":344932592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6805","nonce":344932592}]},"public_addr":"172.21.15.112:6801/344932592","cluster_addr":"172.21.15.112:6803/344932592","heartbeat_back_addr":"172.21.15.112:6807/344932592","heartbeat_front_addr":"172.21.15.112:6805/344932592","state":["exists","up"]},{"osd":2,"uuid":"1193c1d8-bfff-4a57-8200-5f69bf1ea3af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6825","nonce":407933616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6827","nonce":407933616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6831","nonce":407933616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6829","nonce":407933616}]},"public_addr":"172.21.15.112:6825/407933616","cluster_addr":"172.21.15.112:6827/407933616","heartbeat_back_addr":"172.21.15.112:6831/407933616","heartbeat_front_addr":"172.21.15.112:6829/407933616","state":["exists","up"]},{"osd":3,"uuid":"af04330a-36ad-40cc-a8dd-fa259b221de5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6817","nonce":4120164806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6819","nonce":4120164806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6823","nonce":4120164806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6821","nonce":4120164806}]},"public_addr":"172.21.15.112:6817/4120164806","cluster_addr":"172.21.15.112:6819/4120164806","heartbeat_back_addr":"172.21.15.112:6823/4120164806","heartbeat_front_addr":"172.21.15.112:6821/4120164806","state":["exists","up"]},{"osd":4,"uuid":"5b893111-292c-4c5a-b184-53ad38c7199e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6803","nonce":522077560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6805","nonce":522077560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6809","nonce":522077560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6807","nonce":522077560}]},"public_addr":"172.21.15.187:6803/522077560","cluster_addr":"172.21.15.187:6805/522077560","heartbeat_back_addr":"172.21.15.187:6809/522077560","heartbeat_front_addr":"172.21.15.187:6807/522077560","state":["exists","up"]},{"osd":5,"uuid":"c0b8e011-45fe-4c5e-925e-6be2a7f1868a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6827","nonce":3593144721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6829","nonce":3593144721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6833","nonce":3593144721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6831","nonce":3593144721}]},"public_addr":"172.21.15.187:6827/3593144721","cluster_addr":"172.21.15.187:6829/3593144721","heartbeat_back_addr":"172.21.15.187:6833/3593144721","heartbeat_front_addr":"172.21.15.187:6831/3593144721","state":["exists","up"]},{"osd":6,"uuid":"87d2d650-7489-4593-98a3-a1f433ce8dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6811","nonce":1494387657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6813","nonce":1494387657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6817","nonce":1494387657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6815","nonce":1494387657}]},"public_addr":"172.21.15.187:6811/1494387657","cluster_addr":"172.21.15.187:6813/1494387657","heartbeat_back_addr":"172.21.15.187:6817/1494387657","heartbeat_front_addr":"172.21.15.187:6815/1494387657","state":["exists","up"]},{"osd":7,"uuid":"c6b4f8e2-a1ee-4438-aa37-67ff4b6ef1f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6819","nonce":1624699842}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6821","nonce":1624699842}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6825","nonce":1624699842}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6823","nonce":1624699842}]},"public_addr":"172.21.15.187:6819/1624699842","cluster_addr":"172.21.15.187:6821/1624699842","heartbeat_back_addr":"172.21.15.187:6825/1624699842","heartbeat_front_addr":"172.21.15.187:6823/1624699842","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.873162+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.400506+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.664317+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.159791+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.202630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.113212+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.384786+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.448441+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T00:29:07.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.373+0000 7f2efa24d640 1 -- 172.21.15.112:0/3438214324 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2ee0078bc0 msgr2=0x7f2ee007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:07.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.373+0000 7f2efa24d640 1 --2- 172.21.15.112:0/3438214324 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2ee0078bc0 0x7f2ee007b0d0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f2ee4000c00 tx=0x7f2ee4006a20 comp rx=0 tx=0).stop 2024-09-21T00:29:07.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.373+0000 7f2efa24d640 1 -- 172.21.15.112:0/3438214324 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ef41543b0 msgr2=0x7f2ef41547c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:07.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.373+0000 7f2efa24d640 1 --2- 172.21.15.112:0/3438214324 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ef41543b0 0x7f2ef41547c0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f2eec02b490 tx=0x7f2eec004bb0 comp rx=0 tx=0).stop 2024-09-21T00:29:07.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.373+0000 7f2efa24d640 1 -- 172.21.15.112:0/3438214324 shutdown_connections 2024-09-21T00:29:07.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.373+0000 7f2efa24d640 1 --2- 172.21.15.112:0/3438214324 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2ee0078bc0 0x7f2ee007b0d0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:07.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.373+0000 7f2efa24d640 1 --2- 172.21.15.112:0/3438214324 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ef41543b0 0x7f2ef41547c0 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:07.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.373+0000 7f2efa24d640 1 --2- 172.21.15.112:0/3438214324 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2ef40cb090 0x7f2ef40cb4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:07.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.373+0000 7f2efa24d640 1 --2- 172.21.15.112:0/3438214324 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2edc0099b0 0x7f2ef40ad5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:07.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.373+0000 7f2efa24d640 1 -- 172.21.15.112:0/3438214324 >> 172.21.15.112:0/3438214324 conn(0x7f2ef40b42e0 msgr2=0x7f2ef40d03f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:07.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.373+0000 7f2efa24d640 1 -- 172.21.15.112:0/3438214324 shutdown_connections 2024-09-21T00:29:07.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.373+0000 7f2efa24d640 1 -- 172.21.15.112:0/3438214324 wait complete. 2024-09-21T00:29:07.386 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T00:29:07.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.497+0000 7f348bb12640 1 -- 172.21.15.112:0/851715984 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34840cf390 msgr2=0x7f348415bec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:07.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.497+0000 7f348bb12640 1 --2- 172.21.15.112:0/851715984 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34840cf390 0x7f348415bec0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f347c009ca0 tx=0x7f347c02aa30 comp rx=0 tx=0).stop 2024-09-21T00:29:07.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.497+0000 7f348bb12640 1 -- 172.21.15.112:0/851715984 shutdown_connections 2024-09-21T00:29:07.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.497+0000 7f348bb12640 1 --2- 172.21.15.112:0/851715984 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34840cf390 0x7f348415bec0 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:07.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.497+0000 7f348bb12640 1 --2- 172.21.15.112:0/851715984 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f34840cea40 0x7f34840cee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:07.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.497+0000 7f348bb12640 1 -- 172.21.15.112:0/851715984 >> 172.21.15.112:0/851715984 conn(0x7f34840cd5c0 msgr2=0x7f34840cd9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:07.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.497+0000 7f348bb12640 1 -- 172.21.15.112:0/851715984 shutdown_connections 2024-09-21T00:29:07.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.497+0000 7f348bb12640 1 -- 172.21.15.112:0/851715984 wait complete. 2024-09-21T00:29:07.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.499+0000 7f348bb12640 1 Processor -- start 2024-09-21T00:29:07.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.499+0000 7f348bb12640 1 -- start start 2024-09-21T00:29:07.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.500+0000 7f348bb12640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3484070440 0x7f34841fa3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:07.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.500+0000 7f348bb12640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f34840cea40 0x7f348415ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:07.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.500+0000 7f348bb12640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f34840cf390 0x7f3484158eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:07.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.500+0000 7f348bb12640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f348415c580 con 0x7f3484070440 2024-09-21T00:29:07.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.500+0000 7f348bb12640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f348415c400 con 0x7f34840cea40 2024-09-21T00:29:07.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.500+0000 7f348bb12640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f348415c700 con 0x7f34840cf390 2024-09-21T00:29:07.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.500+0000 7f34898ae640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3484070440 0x7f34841fa3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:07.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.500+0000 7f34898ae640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3484070440 0x7f34841fa3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39356/0 (socket says 172.21.15.112:39356) 2024-09-21T00:29:07.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.500+0000 7f34898ae640 1 -- 172.21.15.112:0/4145728467 learned_addr learned my addr 172.21.15.112:0/4145728467 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:29:07.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.500+0000 7f34890ad640 1 --2- 172.21.15.112:0/4145728467 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f34840cea40 0x7f348415ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:07.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.500+0000 7f348a0af640 1 --2- 172.21.15.112:0/4145728467 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f34840cf390 0x7f3484158eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:07.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.500+0000 7f34898ae640 1 -- 172.21.15.112:0/4145728467 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f34840cf390 msgr2=0x7f3484158eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:07.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.500+0000 7f34898ae640 1 --2- 172.21.15.112:0/4145728467 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f34840cf390 0x7f3484158eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:07.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.500+0000 7f34898ae640 1 -- 172.21.15.112:0/4145728467 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f34840cea40 msgr2=0x7f348415ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:07.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.500+0000 7f34898ae640 1 --2- 172.21.15.112:0/4145728467 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f34840cea40 0x7f348415ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:07.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.500+0000 7f34898ae640 1 -- 172.21.15.112:0/4145728467 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34841fa8f0 con 0x7f3484070440 2024-09-21T00:29:07.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.501+0000 7f34898ae640 1 --2- 172.21.15.112:0/4145728467 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3484070440 0x7f34841fa3b0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f348000b3f0 tx=0x7f348000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:07.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.501+0000 7f347a7fc640 1 -- 172.21.15.112:0/4145728467 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3480015020 con 0x7f3484070440 2024-09-21T00:29:07.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.501+0000 7f347a7fc640 1 -- 172.21.15.112:0/4145728467 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3480011920 con 0x7f3484070440 2024-09-21T00:29:07.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.501+0000 7f348bb12640 1 -- 172.21.15.112:0/4145728467 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f348415fcd0 con 0x7f3484070440 2024-09-21T00:29:07.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.501+0000 7f347a7fc640 1 -- 172.21.15.112:0/4145728467 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3480011d70 con 0x7f3484070440 2024-09-21T00:29:07.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.501+0000 7f348bb12640 1 -- 172.21.15.112:0/4145728467 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34841f9bb0 con 0x7f3484070440 2024-09-21T00:29:07.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.502+0000 7f348bb12640 1 -- 172.21.15.112:0/4145728467 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3484071150 con 0x7f3484070440 2024-09-21T00:29:07.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.502+0000 7f347a7fc640 1 -- 172.21.15.112:0/4145728467 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3480018020 con 0x7f3484070440 2024-09-21T00:29:07.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.503+0000 7f347a7fc640 1 --2- 172.21.15.112:0/4145728467 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f3468078b90 0x7f346807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:07.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.503+0000 7f347a7fc640 1 -- 172.21.15.112:0/4145728467 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 6101+0+0 (secure 0 0 0) 0x7f348009ad10 con 0x7f3484070440 2024-09-21T00:29:07.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.503+0000 7f34890ad640 1 --2- 172.21.15.112:0/4145728467 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f3468078b90 0x7f346807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:07.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.504+0000 7f34890ad640 1 --2- 172.21.15.112:0/4145728467 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f3468078b90 0x7f346807b0a0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f348414df00 tx=0x7f347400b2e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:07.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.507+0000 7f347a7fc640 1 -- 172.21.15.112:0/4145728467 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3484071150 con 0x7f3484070440 2024-09-21T00:29:07.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.723+0000 7f348bb12640 1 -- 172.21.15.112:0/4145728467 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f348415cc40 con 0x7f3484070440 2024-09-21T00:29:07.725 INFO:tasks.ceph.mon.a.smithi112.stderr:2024-09-21T00:29:07.723+0000 7f7656c13640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T00:29:07.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.725+0000 7f347a7fc640 1 -- 172.21.15.112:0/4145728467 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v56) ==== 161+0+0 (secure 0 0 0) 0x7f348001f020 con 0x7f3484070440 2024-09-21T00:29:07.726 INFO:teuthology.orchestra.run.smithi112.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T00:29:07.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.727+0000 7f348bb12640 1 -- 172.21.15.112:0/4145728467 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f3468078b90 msgr2=0x7f346807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:07.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.727+0000 7f348bb12640 1 --2- 172.21.15.112:0/4145728467 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f3468078b90 0x7f346807b0a0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f348414df00 tx=0x7f347400b2e0 comp rx=0 tx=0).stop 2024-09-21T00:29:07.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.727+0000 7f348bb12640 1 -- 172.21.15.112:0/4145728467 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3484070440 msgr2=0x7f34841fa3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:07.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.727+0000 7f348bb12640 1 --2- 172.21.15.112:0/4145728467 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3484070440 0x7f34841fa3b0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f348000b3f0 tx=0x7f348000b8b0 comp rx=0 tx=0).stop 2024-09-21T00:29:07.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.727+0000 7f348bb12640 1 -- 172.21.15.112:0/4145728467 shutdown_connections 2024-09-21T00:29:07.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.727+0000 7f348bb12640 1 --2- 172.21.15.112:0/4145728467 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f34840cf390 0x7f3484158eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:07.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.727+0000 7f348bb12640 1 --2- 172.21.15.112:0/4145728467 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f34840cea40 0x7f348415ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:07.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.727+0000 7f348bb12640 1 --2- 172.21.15.112:0/4145728467 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f3468078b90 0x7f346807b0a0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:07.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.727+0000 7f348bb12640 1 --2- 172.21.15.112:0/4145728467 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3484070440 0x7f34841fa3b0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:07.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.728+0000 7f348bb12640 1 -- 172.21.15.112:0/4145728467 >> 172.21.15.112:0/4145728467 conn(0x7f34840cd5c0 msgr2=0x7f3484155cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:07.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.728+0000 7f348bb12640 1 -- 172.21.15.112:0/4145728467 shutdown_connections 2024-09-21T00:29:07.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.728+0000 7f348bb12640 1 -- 172.21.15.112:0/4145728467 wait complete. 2024-09-21T00:29:07.730 INFO:tasks.ceph.mds.d.smithi187.stderr: -3452> 2024-09-21T00:28:49.768+0000 7f54a6488640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2024-09-21T00:29:07.732 INFO:tasks.ceph.mds.d.smithi187.stderr: -2062> 2024-09-21T00:29:00.415+0000 7f54a047c640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T00:29:07.741 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-21T00:29:07.787 INFO:tasks.ceph.mds.d.smithi187.stdout:starting mds.d at 2024-09-21T00:29:07.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.854+0000 7f484855e640 1 -- 172.21.15.112:0/3970655503 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48400d0c40 msgr2=0x7f484006d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:07.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.854+0000 7f484855e640 1 --2- 172.21.15.112:0/3970655503 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48400d0c40 0x7f484006d4f0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f4840072960 tx=0x7f483802a940 comp rx=0 tx=0).stop 2024-09-21T00:29:07.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.855+0000 7f484855e640 1 -- 172.21.15.112:0/3970655503 shutdown_connections 2024-09-21T00:29:07.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.855+0000 7f484855e640 1 --2- 172.21.15.112:0/3970655503 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48400d0c40 0x7f484006d4f0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:07.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.855+0000 7f484855e640 1 --2- 172.21.15.112:0/3970655503 >> v2:172.21.15.187:3300/0 conn(0x7f48400cc270 0x7f48400d0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:07.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.855+0000 7f484855e640 1 -- 172.21.15.112:0/3970655503 >> 172.21.15.112:0/3970655503 conn(0x7f48400b3580 msgr2=0x7f48400b3990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:07.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.855+0000 7f484855e640 1 -- 172.21.15.112:0/3970655503 shutdown_connections 2024-09-21T00:29:07.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.855+0000 7f484855e640 1 -- 172.21.15.112:0/3970655503 wait complete. 2024-09-21T00:29:07.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.856+0000 7f484855e640 1 Processor -- start 2024-09-21T00:29:07.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.856+0000 7f484855e640 1 -- start start 2024-09-21T00:29:07.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.858+0000 7f484855e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48400cc270 0x7f4840159590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:07.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.858+0000 7f484855e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f48400ce9e0 0x7f48400b1f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:07.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.858+0000 7f484855e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f48400d0c40 0x7f48400b2450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:07.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.858+0000 7f484855e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f48400705e0 con 0x7f48400cc270 2024-09-21T00:29:07.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.858+0000 7f484855e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f4840070460 con 0x7f48400d0c40 2024-09-21T00:29:07.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.858+0000 7f484855e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f48400af950 con 0x7f48400ce9e0 2024-09-21T00:29:07.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.858+0000 7f48462fa640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48400cc270 0x7f4840159590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:07.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.858+0000 7f48462fa640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48400cc270 0x7f4840159590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39372/0 (socket says 172.21.15.112:39372) 2024-09-21T00:29:07.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.858+0000 7f48462fa640 1 -- 172.21.15.112:0/865394590 learned_addr learned my addr 172.21.15.112:0/865394590 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:29:07.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.858+0000 7f4846afb640 1 --2- 172.21.15.112:0/865394590 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f48400d0c40 0x7f48400b2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:07.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.858+0000 7f4845af9640 1 --2- 172.21.15.112:0/865394590 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f48400ce9e0 0x7f48400b1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:07.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.858+0000 7f48462fa640 1 -- 172.21.15.112:0/865394590 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f48400ce9e0 msgr2=0x7f48400b1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:07.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.858+0000 7f48462fa640 1 --2- 172.21.15.112:0/865394590 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f48400ce9e0 0x7f48400b1f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:07.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.858+0000 7f48462fa640 1 -- 172.21.15.112:0/865394590 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f48400d0c40 msgr2=0x7f48400b2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:07.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.858+0000 7f48462fa640 1 --2- 172.21.15.112:0/865394590 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f48400d0c40 0x7f48400b2450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:07.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.858+0000 7f48462fa640 1 -- 172.21.15.112:0/865394590 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48401f3550 con 0x7f48400cc270 2024-09-21T00:29:07.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.858+0000 7f4846afb640 1 --2- 172.21.15.112:0/865394590 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f48400d0c40 0x7f48400b2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:29:07.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.858+0000 7f4845af9640 1 --2- 172.21.15.112:0/865394590 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f48400ce9e0 0x7f48400b1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:29:07.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.858+0000 7f48462fa640 1 --2- 172.21.15.112:0/865394590 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48400cc270 0x7f4840159590 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f483c00b470 tx=0x7f483c00b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:07.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.858+0000 7f4836ffd640 1 -- 172.21.15.112:0/865394590 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f483c015020 con 0x7f48400cc270 2024-09-21T00:29:07.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.858+0000 7f4836ffd640 1 -- 172.21.15.112:0/865394590 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f483c011940 con 0x7f48400cc270 2024-09-21T00:29:07.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.858+0000 7f484855e640 1 -- 172.21.15.112:0/865394590 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48401f3830 con 0x7f48400cc270 2024-09-21T00:29:07.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.858+0000 7f4836ffd640 1 -- 172.21.15.112:0/865394590 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f483c011d90 con 0x7f48400cc270 2024-09-21T00:29:07.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.859+0000 7f484855e640 1 -- 172.21.15.112:0/865394590 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48401f3df0 con 0x7f48400cc270 2024-09-21T00:29:07.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.859+0000 7f484855e640 1 -- 172.21.15.112:0/865394590 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48400695b0 con 0x7f48400cc270 2024-09-21T00:29:07.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.860+0000 7f4836ffd640 1 -- 172.21.15.112:0/865394590 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f483c018020 con 0x7f48400cc270 2024-09-21T00:29:07.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.861+0000 7f4836ffd640 1 --2- 172.21.15.112:0/865394590 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4824078bc0 0x7f482407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:07.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.861+0000 7f4836ffd640 1 -- 172.21.15.112:0/865394590 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 6187+0+0 (secure 0 0 0) 0x7f483c09ae70 con 0x7f48400cc270 2024-09-21T00:29:07.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.861+0000 7f4845af9640 1 --2- 172.21.15.112:0/865394590 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4824078bc0 0x7f482407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:07.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.862+0000 7f4845af9640 1 --2- 172.21.15.112:0/865394590 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4824078bc0 0x7f482407b0d0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f48300046b0 tx=0x7f4830009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:07.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:07.866+0000 7f4836ffd640 1 -- 172.21.15.112:0/865394590 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f483c01f020 con 0x7f48400cc270 2024-09-21T00:29:08.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:08.081+0000 7f484855e640 1 -- 172.21.15.112:0/865394590 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f484006db70 con 0x7f48400cc270 2024-09-21T00:29:08.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:08.726+0000 7f4836ffd640 1 -- 172.21.15.112:0/865394590 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v57) ==== 104+0+0 (secure 0 0 0) 0x7f483c0630d0 con 0x7f48400cc270 2024-09-21T00:29:08.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:08.729+0000 7f484855e640 1 -- 172.21.15.112:0/865394590 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4824078bc0 msgr2=0x7f482407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:08.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:08.729+0000 7f484855e640 1 --2- 172.21.15.112:0/865394590 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4824078bc0 0x7f482407b0d0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f48300046b0 tx=0x7f4830009040 comp rx=0 tx=0).stop 2024-09-21T00:29:08.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:08.729+0000 7f484855e640 1 -- 172.21.15.112:0/865394590 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48400cc270 msgr2=0x7f4840159590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:08.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:08.729+0000 7f484855e640 1 --2- 172.21.15.112:0/865394590 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48400cc270 0x7f4840159590 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f483c00b470 tx=0x7f483c00b930 comp rx=0 tx=0).stop 2024-09-21T00:29:08.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:08.729+0000 7f484855e640 1 -- 172.21.15.112:0/865394590 shutdown_connections 2024-09-21T00:29:08.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:08.729+0000 7f484855e640 1 --2- 172.21.15.112:0/865394590 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f48400d0c40 0x7f48400b2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:08.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:08.729+0000 7f484855e640 1 --2- 172.21.15.112:0/865394590 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f48400ce9e0 0x7f48400b1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:08.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:08.729+0000 7f484855e640 1 --2- 172.21.15.112:0/865394590 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4824078bc0 0x7f482407b0d0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:08.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:08.729+0000 7f484855e640 1 --2- 172.21.15.112:0/865394590 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48400cc270 0x7f4840159590 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:08.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:08.729+0000 7f484855e640 1 -- 172.21.15.112:0/865394590 >> 172.21.15.112:0/865394590 conn(0x7f48400b3580 msgr2=0x7f48400b0480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:08.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:08.729+0000 7f484855e640 1 -- 172.21.15.112:0/865394590 shutdown_connections 2024-09-21T00:29:08.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:08.729+0000 7f484855e640 1 -- 172.21.15.112:0/865394590 wait complete. 2024-09-21T00:29:08.743 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-21T00:29:08.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:08.861+0000 7f637f6a9640 1 -- 172.21.15.112:0/1567623544 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63780ccae0 msgr2=0x7f63780ccef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:08.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:08.861+0000 7f637f6a9640 1 --2- 172.21.15.112:0/1567623544 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63780ccae0 0x7f63780ccef0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f6370005fd0 tx=0x7f637002a8b0 comp rx=0 tx=0).stop 2024-09-21T00:29:08.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:08.861+0000 7f637f6a9640 1 -- 172.21.15.112:0/1567623544 shutdown_connections 2024-09-21T00:29:08.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:08.861+0000 7f637f6a9640 1 --2- 172.21.15.112:0/1567623544 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63780ccae0 0x7f63780ccef0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:08.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:08.861+0000 7f637f6a9640 1 --2- 172.21.15.112:0/1567623544 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f637815c6a0 0x7f63780ce5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:08.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:08.861+0000 7f637f6a9640 1 -- 172.21.15.112:0/1567623544 >> 172.21.15.112:0/1567623544 conn(0x7f63780b3f10 msgr2=0x7f63780b4320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:08.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:08.861+0000 7f637f6a9640 1 -- 172.21.15.112:0/1567623544 shutdown_connections 2024-09-21T00:29:08.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:08.861+0000 7f637f6a9640 1 -- 172.21.15.112:0/1567623544 wait complete. 2024-09-21T00:29:08.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:08.863+0000 7f637f6a9640 1 Processor -- start 2024-09-21T00:29:08.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:08.863+0000 7f637f6a9640 1 -- start start 2024-09-21T00:29:08.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:08.864+0000 7f637f6a9640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f63780b6590 0x7f63780cb550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:08.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:08.864+0000 7f637f6a9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63780ccae0 0x7f637815a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:08.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:08.864+0000 7f637f6a9640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f637815c6a0 0x7f63781f5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:08.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:08.864+0000 7f637f6a9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6378070b10 con 0x7f63780ccae0 2024-09-21T00:29:08.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:08.864+0000 7f637f6a9640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f6378070990 con 0x7f63780b6590 2024-09-21T00:29:08.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:08.864+0000 7f637f6a9640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f63780cd430 con 0x7f637815c6a0 2024-09-21T00:29:08.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:08.865+0000 7f637cc44640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63780ccae0 0x7f637815a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:08.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:08.865+0000 7f637cc44640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63780ccae0 0x7f637815a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39390/0 (socket says 172.21.15.112:39390) 2024-09-21T00:29:08.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:08.865+0000 7f637cc44640 1 -- 172.21.15.112:0/3196715442 learned_addr learned my addr 172.21.15.112:0/3196715442 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:29:08.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:08.865+0000 7f637d445640 1 --2- 172.21.15.112:0/3196715442 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f63780b6590 0x7f63780cb550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:08.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:08.865+0000 7f637dc46640 1 --2- 172.21.15.112:0/3196715442 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f637815c6a0 0x7f63781f5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:08.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:08.865+0000 7f637cc44640 1 -- 172.21.15.112:0/3196715442 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f637815c6a0 msgr2=0x7f63781f5cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:08.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:08.865+0000 7f637cc44640 1 --2- 172.21.15.112:0/3196715442 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f637815c6a0 0x7f63781f5cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:08.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:08.865+0000 7f637cc44640 1 -- 172.21.15.112:0/3196715442 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f63780b6590 msgr2=0x7f63780cb550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:08.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:08.865+0000 7f637cc44640 1 --2- 172.21.15.112:0/3196715442 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f63780b6590 0x7f63780cb550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:08.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:08.865+0000 7f637cc44640 1 -- 172.21.15.112:0/3196715442 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63781f9b10 con 0x7f63780ccae0 2024-09-21T00:29:08.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:08.865+0000 7f637dc46640 1 --2- 172.21.15.112:0/3196715442 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f637815c6a0 0x7f63781f5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:29:08.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:08.865+0000 7f637cc44640 1 --2- 172.21.15.112:0/3196715442 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63780ccae0 0x7f637815a050 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f636800c3c0 tx=0x7f636800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:08.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:08.865+0000 7f636dffb640 1 -- 172.21.15.112:0/3196715442 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6368017070 con 0x7f63780ccae0 2024-09-21T00:29:08.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:08.865+0000 7f636dffb640 1 -- 172.21.15.112:0/3196715442 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f63680035f0 con 0x7f63780ccae0 2024-09-21T00:29:08.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:08.865+0000 7f637f6a9640 1 -- 172.21.15.112:0/3196715442 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63781f9270 con 0x7f63780ccae0 2024-09-21T00:29:08.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:08.865+0000 7f636dffb640 1 -- 172.21.15.112:0/3196715442 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6368003a80 con 0x7f63780ccae0 2024-09-21T00:29:08.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:08.866+0000 7f637f6a9640 1 -- 172.21.15.112:0/3196715442 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63781f9830 con 0x7f63780ccae0 2024-09-21T00:29:08.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:08.866+0000 7f637f6a9640 1 -- 172.21.15.112:0/3196715442 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63780cd8e0 con 0x7f63780ccae0 2024-09-21T00:29:08.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:08.867+0000 7f636dffb640 1 -- 172.21.15.112:0/3196715442 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f636801a070 con 0x7f63780ccae0 2024-09-21T00:29:08.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:08.868+0000 7f636dffb640 1 --2- 172.21.15.112:0/3196715442 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f635c078b90 0x7f635c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:08.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:08.868+0000 7f636dffb640 1 -- 172.21.15.112:0/3196715442 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 6187+0+0 (secure 0 0 0) 0x7f636809a170 con 0x7f63780ccae0 2024-09-21T00:29:08.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:08.868+0000 7f637d445640 1 --2- 172.21.15.112:0/3196715442 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f635c078b90 0x7f635c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:08.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:08.868+0000 7f637d445640 1 --2- 172.21.15.112:0/3196715442 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f635c078b90 0x7f635c07b0a0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f6374009960 tx=0x7f6374009060 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:08.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:08.872+0000 7f636dffb640 1 -- 172.21.15.112:0/3196715442 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f6368061d80 con 0x7f63780ccae0 2024-09-21T00:29:09.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:09.048+0000 7f637f6a9640 1 -- 172.21.15.112:0/3196715442 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f6378068da0 con 0x7f63780ccae0 2024-09-21T00:29:09.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:09.731+0000 7f636dffb640 1 -- 172.21.15.112:0/3196715442 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v85) ==== 181+0+0 (secure 0 0 0) 0x7f63680667b0 con 0x7f63780ccae0 2024-09-21T00:29:09.733 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'cephfs_metadata' removed 2024-09-21T00:29:09.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:09.734+0000 7f637f6a9640 1 -- 172.21.15.112:0/3196715442 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f635c078b90 msgr2=0x7f635c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:09.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:09.734+0000 7f637f6a9640 1 --2- 172.21.15.112:0/3196715442 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f635c078b90 0x7f635c07b0a0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f6374009960 tx=0x7f6374009060 comp rx=0 tx=0).stop 2024-09-21T00:29:09.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:09.734+0000 7f637f6a9640 1 -- 172.21.15.112:0/3196715442 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63780ccae0 msgr2=0x7f637815a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:09.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:09.734+0000 7f637f6a9640 1 --2- 172.21.15.112:0/3196715442 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63780ccae0 0x7f637815a050 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f636800c3c0 tx=0x7f636800c880 comp rx=0 tx=0).stop 2024-09-21T00:29:09.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:09.734+0000 7f637f6a9640 1 -- 172.21.15.112:0/3196715442 shutdown_connections 2024-09-21T00:29:09.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:09.734+0000 7f637f6a9640 1 --2- 172.21.15.112:0/3196715442 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f637815c6a0 0x7f63781f5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:09.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:09.734+0000 7f637f6a9640 1 --2- 172.21.15.112:0/3196715442 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f635c078b90 0x7f635c07b0a0 secure :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f6374009960 tx=0x7f6374009060 comp rx=0 tx=0).stop 2024-09-21T00:29:09.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:09.734+0000 7f637f6a9640 1 --2- 172.21.15.112:0/3196715442 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63780ccae0 0x7f637815a050 secure :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f636800c3c0 tx=0x7f636800c880 comp rx=0 tx=0).stop 2024-09-21T00:29:09.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:09.734+0000 7f637f6a9640 1 --2- 172.21.15.112:0/3196715442 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f63780b6590 0x7f63780cb550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:09.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:09.734+0000 7f637f6a9640 1 -- 172.21.15.112:0/3196715442 >> 172.21.15.112:0/3196715442 conn(0x7f63780b3f10 msgr2=0x7f637815eda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:09.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:09.735+0000 7f637f6a9640 1 -- 172.21.15.112:0/3196715442 shutdown_connections 2024-09-21T00:29:09.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:09.735+0000 7f637f6a9640 1 -- 172.21.15.112:0/3196715442 wait complete. 2024-09-21T00:29:09.750 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-21T00:29:09.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:09.867+0000 7f46a36b2640 1 -- 172.21.15.112:0/3561610528 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f469c0b7db0 msgr2=0x7f469c0b5db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:09.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:09.867+0000 7f46a36b2640 1 --2- 172.21.15.112:0/3561610528 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f469c0b7db0 0x7f469c0b5db0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f4694005fd0 tx=0x7f469402a8b0 comp rx=0 tx=0).stop 2024-09-21T00:29:09.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:09.867+0000 7f46a36b2640 1 -- 172.21.15.112:0/3561610528 shutdown_connections 2024-09-21T00:29:09.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:09.867+0000 7f46a36b2640 1 --2- 172.21.15.112:0/3561610528 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f469c0b7db0 0x7f469c0b5db0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:09.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:09.867+0000 7f46a36b2640 1 --2- 172.21.15.112:0/3561610528 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f469c0ccb50 0x7f469c0d0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:09.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:09.867+0000 7f46a36b2640 1 -- 172.21.15.112:0/3561610528 >> 172.21.15.112:0/3561610528 conn(0x7f469c0b47d0 msgr2=0x7f469c0d2470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:09.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:09.867+0000 7f46a36b2640 1 -- 172.21.15.112:0/3561610528 shutdown_connections 2024-09-21T00:29:09.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:09.867+0000 7f46a36b2640 1 -- 172.21.15.112:0/3561610528 wait complete. 2024-09-21T00:29:09.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:09.869+0000 7f46a36b2640 1 Processor -- start 2024-09-21T00:29:09.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:09.869+0000 7f46a36b2640 1 -- start start 2024-09-21T00:29:09.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:09.870+0000 7f46a36b2640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f469c0b7db0 0x7f469c0b0b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:09.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:09.870+0000 7f46a36b2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f469c0ccb50 0x7f469c0b1050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:09.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:09.870+0000 7f46a36b2640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f469c15e9b0 0x7f469c0aeb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:09.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:09.870+0000 7f46a36b2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f469c0cdeb0 con 0x7f469c0ccb50 2024-09-21T00:29:09.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:09.870+0000 7f46a36b2640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f469c0cdd30 con 0x7f469c0b7db0 2024-09-21T00:29:09.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:09.870+0000 7f46a36b2640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f469c0ce030 con 0x7f469c15e9b0 2024-09-21T00:29:09.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:09.870+0000 7f46a0c4d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f469c0ccb50 0x7f469c0b1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:09.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:09.870+0000 7f46a0c4d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f469c0ccb50 0x7f469c0b1050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39394/0 (socket says 172.21.15.112:39394) 2024-09-21T00:29:09.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:09.870+0000 7f46a0c4d640 1 -- 172.21.15.112:0/2595411325 learned_addr learned my addr 172.21.15.112:0/2595411325 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:29:09.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:09.870+0000 7f46a144e640 1 --2- 172.21.15.112:0/2595411325 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f469c0b7db0 0x7f469c0b0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:09.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:09.870+0000 7f46a1c4f640 1 --2- 172.21.15.112:0/2595411325 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f469c15e9b0 0x7f469c0aeb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:09.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:09.870+0000 7f46a0c4d640 1 -- 172.21.15.112:0/2595411325 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f469c15e9b0 msgr2=0x7f469c0aeb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:09.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:09.870+0000 7f46a0c4d640 1 --2- 172.21.15.112:0/2595411325 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f469c15e9b0 0x7f469c0aeb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:09.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:09.870+0000 7f46a0c4d640 1 -- 172.21.15.112:0/2595411325 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f469c0b7db0 msgr2=0x7f469c0b0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:09.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:09.870+0000 7f46a0c4d640 1 --2- 172.21.15.112:0/2595411325 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f469c0b7db0 0x7f469c0b0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:09.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:09.870+0000 7f46a0c4d640 1 -- 172.21.15.112:0/2595411325 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f469c0ada70 con 0x7f469c0ccb50 2024-09-21T00:29:09.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:09.870+0000 7f46a144e640 1 --2- 172.21.15.112:0/2595411325 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f469c0b7db0 0x7f469c0b0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:29:09.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:09.871+0000 7f46a0c4d640 1 --2- 172.21.15.112:0/2595411325 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f469c0ccb50 0x7f469c0b1050 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f468c001eb0 tx=0x7f468c00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:09.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:09.871+0000 7f4691ffb640 1 -- 172.21.15.112:0/2595411325 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f468c011790 con 0x7f469c0ccb50 2024-09-21T00:29:09.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:09.871+0000 7f4691ffb640 1 -- 172.21.15.112:0/2595411325 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f468c017070 con 0x7f469c0ccb50 2024-09-21T00:29:09.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:09.871+0000 7f46a36b2640 1 -- 172.21.15.112:0/2595411325 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f469c0add50 con 0x7f469c0ccb50 2024-09-21T00:29:09.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:09.871+0000 7f4691ffb640 1 -- 172.21.15.112:0/2595411325 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f468c011c60 con 0x7f469c0ccb50 2024-09-21T00:29:09.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:09.871+0000 7f46a36b2640 1 -- 172.21.15.112:0/2595411325 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f469c0ae310 con 0x7f469c0ccb50 2024-09-21T00:29:09.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:09.872+0000 7f46a36b2640 1 -- 172.21.15.112:0/2595411325 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f469c0ce550 con 0x7f469c0ccb50 2024-09-21T00:29:09.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:09.873+0000 7f4691ffb640 1 -- 172.21.15.112:0/2595411325 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f468c01c460 con 0x7f469c0ccb50 2024-09-21T00:29:09.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:09.873+0000 7f4691ffb640 1 --2- 172.21.15.112:0/2595411325 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4680078b90 0x7f468007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:09.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:09.873+0000 7f4691ffb640 1 -- 172.21.15.112:0/2595411325 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5749+0+0 (secure 0 0 0) 0x7f468c09aa20 con 0x7f469c0ccb50 2024-09-21T00:29:09.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:09.873+0000 7f46a144e640 1 --2- 172.21.15.112:0/2595411325 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4680078b90 0x7f468007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:09.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:09.874+0000 7f46a144e640 1 --2- 172.21.15.112:0/2595411325 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4680078b90 0x7f468007b0a0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f46980095f0 tx=0x7f4698005f90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:09.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:09.878+0000 7f4691ffb640 1 -- 172.21.15.112:0/2595411325 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f468c0627f0 con 0x7f469c0ccb50 2024-09-21T00:29:10.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:10.055+0000 7f46a36b2640 1 -- 172.21.15.112:0/2595411325 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f469c0b37b0 con 0x7f469c0ccb50 2024-09-21T00:29:10.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:10.737+0000 7f4691ffb640 1 -- 172.21.15.112:0/2595411325 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v86) ==== 169+0+0 (secure 0 0 0) 0x7f468c067220 con 0x7f469c0ccb50 2024-09-21T00:29:10.739 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'cephfs_data' removed 2024-09-21T00:29:10.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:10.741+0000 7f46a36b2640 1 -- 172.21.15.112:0/2595411325 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4680078b90 msgr2=0x7f468007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:10.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:10.741+0000 7f46a36b2640 1 --2- 172.21.15.112:0/2595411325 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4680078b90 0x7f468007b0a0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f46980095f0 tx=0x7f4698005f90 comp rx=0 tx=0).stop 2024-09-21T00:29:10.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:10.741+0000 7f46a36b2640 1 -- 172.21.15.112:0/2595411325 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f469c0ccb50 msgr2=0x7f469c0b1050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:10.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:10.741+0000 7f46a36b2640 1 --2- 172.21.15.112:0/2595411325 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f469c0ccb50 0x7f469c0b1050 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f468c001eb0 tx=0x7f468c00c890 comp rx=0 tx=0).stop 2024-09-21T00:29:10.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:10.741+0000 7f46a36b2640 1 -- 172.21.15.112:0/2595411325 shutdown_connections 2024-09-21T00:29:10.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:10.741+0000 7f46a36b2640 1 --2- 172.21.15.112:0/2595411325 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f469c15e9b0 0x7f469c0aeb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:10.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:10.741+0000 7f46a36b2640 1 --2- 172.21.15.112:0/2595411325 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4680078b90 0x7f468007b0a0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:10.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:10.741+0000 7f46a36b2640 1 --2- 172.21.15.112:0/2595411325 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f469c0ccb50 0x7f469c0b1050 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:10.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:10.741+0000 7f46a36b2640 1 --2- 172.21.15.112:0/2595411325 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f469c0b7db0 0x7f469c0b0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:10.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:10.741+0000 7f46a36b2640 1 -- 172.21.15.112:0/2595411325 >> 172.21.15.112:0/2595411325 conn(0x7f469c0b47d0 msgr2=0x7f469c0b2be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:10.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:10.742+0000 7f46a36b2640 1 -- 172.21.15.112:0/2595411325 shutdown_connections 2024-09-21T00:29:10.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:10.742+0000 7f46a36b2640 1 -- 172.21.15.112:0/2595411325 wait complete. 2024-09-21T00:29:10.757 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 31 2024-09-21T00:29:10.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:10.870+0000 7fdff597b640 1 -- 172.21.15.112:0/2008090077 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdff00d01b0 msgr2=0x7fdff00d05c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:10.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:10.870+0000 7fdff597b640 1 --2- 172.21.15.112:0/2008090077 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdff00d01b0 0x7fdff00d05c0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fdff00d2050 tx=0x7fdfe802aa30 comp rx=0 tx=0).stop 2024-09-21T00:29:10.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:10.870+0000 7fdff597b640 1 -- 172.21.15.112:0/2008090077 shutdown_connections 2024-09-21T00:29:10.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:10.870+0000 7fdff597b640 1 --2- 172.21.15.112:0/2008090077 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdff00d01b0 0x7fdff00d05c0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:10.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:10.870+0000 7fdff597b640 1 --2- 172.21.15.112:0/2008090077 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdff00d3880 0x7fdff0163260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:10.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:10.870+0000 7fdff597b640 1 --2- 172.21.15.112:0/2008090077 >> v2:172.21.15.187:3300/0 conn(0x7fdff00d48a0 0x7fdff00d3dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:10.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:10.870+0000 7fdff597b640 1 -- 172.21.15.112:0/2008090077 >> 172.21.15.112:0/2008090077 conn(0x7fdff008bfe0 msgr2=0x7fdff009d890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:10.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:10.870+0000 7fdff597b640 1 -- 172.21.15.112:0/2008090077 shutdown_connections 2024-09-21T00:29:10.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:10.870+0000 7fdff597b640 1 -- 172.21.15.112:0/2008090077 wait complete. 2024-09-21T00:29:10.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:10.872+0000 7fdff597b640 1 Processor -- start 2024-09-21T00:29:10.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:10.872+0000 7fdff597b640 1 -- start start 2024-09-21T00:29:10.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:10.873+0000 7fdff597b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdff00d01b0 0x7fdff00cdeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:10.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:10.873+0000 7fdff597b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdff00d3880 0x7fdff0154c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:10.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:10.873+0000 7fdff597b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdff00d48a0 0x7fdff015f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:10.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:10.873+0000 7fdff597b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fdff00d2460 con 0x7fdff00d3880 2024-09-21T00:29:10.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:10.873+0000 7fdff597b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fdff00d22e0 con 0x7fdff00d48a0 2024-09-21T00:29:10.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:10.873+0000 7fdff597b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fdff00d25e0 con 0x7fdff00d01b0 2024-09-21T00:29:10.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:10.873+0000 7fdfe67fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdff00d3880 0x7fdff0154c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:10.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:10.873+0000 7fdfe67fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdff00d3880 0x7fdff0154c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39404/0 (socket says 172.21.15.112:39404) 2024-09-21T00:29:10.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:10.873+0000 7fdfe67fc640 1 -- 172.21.15.112:0/2923623151 learned_addr learned my addr 172.21.15.112:0/2923623151 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:29:10.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:10.873+0000 7fdfef7fe640 1 --2- 172.21.15.112:0/2923623151 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdff00d48a0 0x7fdff015f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:10.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:10.873+0000 7fdfeeffd640 1 --2- 172.21.15.112:0/2923623151 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdff00d01b0 0x7fdff00cdeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:10.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:10.874+0000 7fdfe67fc640 1 -- 172.21.15.112:0/2923623151 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdff00d01b0 msgr2=0x7fdff00cdeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:10.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:10.874+0000 7fdfe67fc640 1 --2- 172.21.15.112:0/2923623151 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdff00d01b0 0x7fdff00cdeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:10.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:10.874+0000 7fdfe67fc640 1 -- 172.21.15.112:0/2923623151 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdff00d48a0 msgr2=0x7fdff015f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:10.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:10.874+0000 7fdfe67fc640 1 --2- 172.21.15.112:0/2923623151 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdff00d48a0 0x7fdff015f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:10.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:10.874+0000 7fdfe67fc640 1 -- 172.21.15.112:0/2923623151 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdff01f54a0 con 0x7fdff00d3880 2024-09-21T00:29:10.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:10.874+0000 7fdfe67fc640 1 --2- 172.21.15.112:0/2923623151 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdff00d3880 0x7fdff0154c90 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fdfd800c970 tx=0x7fdfd800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:10.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:10.874+0000 7fdfecff9640 1 -- 172.21.15.112:0/2923623151 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfd8007bb0 con 0x7fdff00d3880 2024-09-21T00:29:10.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:10.874+0000 7fdfecff9640 1 -- 172.21.15.112:0/2923623151 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdfd8007e40 con 0x7fdff00d3880 2024-09-21T00:29:10.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:10.874+0000 7fdff597b640 1 -- 172.21.15.112:0/2923623151 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdff01f5780 con 0x7fdff00d3880 2024-09-21T00:29:10.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:10.874+0000 7fdfecff9640 1 -- 172.21.15.112:0/2923623151 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfd800f6b0 con 0x7fdff00d3880 2024-09-21T00:29:10.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:10.874+0000 7fdff597b640 1 -- 172.21.15.112:0/2923623151 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdff01f5d40 con 0x7fdff00d3880 2024-09-21T00:29:10.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:10.875+0000 7fdff597b640 1 -- 172.21.15.112:0/2923623151 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdff00d2b00 con 0x7fdff00d3880 2024-09-21T00:29:10.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:10.876+0000 7fdfecff9640 1 -- 172.21.15.112:0/2923623151 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fdfd8018400 con 0x7fdff00d3880 2024-09-21T00:29:10.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:10.876+0000 7fdfecff9640 1 --2- 172.21.15.112:0/2923623151 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fdfd0078bc0 0x7fdfd007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:10.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:10.876+0000 7fdfecff9640 1 -- 172.21.15.112:0/2923623151 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5335+0+0 (secure 0 0 0) 0x7fdfd809b110 con 0x7fdff00d3880 2024-09-21T00:29:10.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:10.876+0000 7fdfeeffd640 1 --2- 172.21.15.112:0/2923623151 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fdfd0078bc0 0x7fdfd007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:10.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:10.877+0000 7fdfeeffd640 1 --2- 172.21.15.112:0/2923623151 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fdfd0078bc0 0x7fdfd007b0d0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fdfe000bb50 tx=0x7fdfe000b2e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:10.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:10.881+0000 7fdfecff9640 1 -- 172.21.15.112:0/2923623151 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fdfd8014030 con 0x7fdff00d3880 2024-09-21T00:29:11.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:11.057+0000 7fdff597b640 1 -- 172.21.15.112:0/2923623151 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "config reset", "num": 31} v 0) -- 0x7fdff00b6f90 con 0x7fdff00d3880 2024-09-21T00:29:11.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:11.061+0000 7fdfecff9640 1 -- 172.21.15.112:0/2923623151 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 31}]=0 v44) ==== 71+0+0 (secure 0 0 0) 0x7fdfd806ae20 con 0x7fdff00d3880 2024-09-21T00:29:11.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:11.063+0000 7fdff597b640 1 -- 172.21.15.112:0/2923623151 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fdfd0078bc0 msgr2=0x7fdfd007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:11.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:11.063+0000 7fdff597b640 1 --2- 172.21.15.112:0/2923623151 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fdfd0078bc0 0x7fdfd007b0d0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fdfe000bb50 tx=0x7fdfe000b2e0 comp rx=0 tx=0).stop 2024-09-21T00:29:11.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:11.063+0000 7fdff597b640 1 -- 172.21.15.112:0/2923623151 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdff00d3880 msgr2=0x7fdff0154c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:11.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:11.063+0000 7fdff597b640 1 --2- 172.21.15.112:0/2923623151 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdff00d3880 0x7fdff0154c90 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fdfd800c970 tx=0x7fdfd800ce30 comp rx=0 tx=0).stop 2024-09-21T00:29:11.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:11.063+0000 7fdff597b640 1 -- 172.21.15.112:0/2923623151 shutdown_connections 2024-09-21T00:29:11.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:11.063+0000 7fdff597b640 1 --2- 172.21.15.112:0/2923623151 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdff00d48a0 0x7fdff015f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:11.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:11.063+0000 7fdff597b640 1 --2- 172.21.15.112:0/2923623151 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fdfd0078bc0 0x7fdfd007b0d0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:11.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:11.063+0000 7fdff597b640 1 --2- 172.21.15.112:0/2923623151 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdff00d3880 0x7fdff0154c90 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:11.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:11.063+0000 7fdff597b640 1 --2- 172.21.15.112:0/2923623151 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdff00d01b0 0x7fdff00cdeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:11.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:11.063+0000 7fdff597b640 1 -- 172.21.15.112:0/2923623151 >> 172.21.15.112:0/2923623151 conn(0x7fdff008bfe0 msgr2=0x7fdff00b1310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:11.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:11.063+0000 7fdff597b640 1 -- 172.21.15.112:0/2923623151 shutdown_connections 2024-09-21T00:29:11.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:11.063+0000 7fdff597b640 1 -- 172.21.15.112:0/2923623151 wait complete. 2024-09-21T00:29:11.076 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_sessionmap.TestSessionMap.test_tell_conn_close' 2024-09-21T00:29:11.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:11.196+0000 7f399f48b640 1 -- 172.21.15.112:0/2252482020 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f39980750a0 msgr2=0x7f399815f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:11.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:11.196+0000 7f399f48b640 1 --2- 172.21.15.112:0/2252482020 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f39980750a0 0x7f399815f040 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f3990005fd0 tx=0x7f399002aa00 comp rx=0 tx=0).stop 2024-09-21T00:29:11.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:11.196+0000 7f399f48b640 1 -- 172.21.15.112:0/2252482020 shutdown_connections 2024-09-21T00:29:11.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:11.196+0000 7f399f48b640 1 --2- 172.21.15.112:0/2252482020 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f39980750a0 0x7f399815f040 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:11.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:11.196+0000 7f399f48b640 1 --2- 172.21.15.112:0/2252482020 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f39980708b0 0x7f3998158fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:11.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:11.196+0000 7f399f48b640 1 -- 172.21.15.112:0/2252482020 >> 172.21.15.112:0/2252482020 conn(0x7f3998073fe0 msgr2=0x7f39980743f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:11.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:11.196+0000 7f399f48b640 1 -- 172.21.15.112:0/2252482020 shutdown_connections 2024-09-21T00:29:11.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:11.196+0000 7f399f48b640 1 -- 172.21.15.112:0/2252482020 wait complete. 2024-09-21T00:29:11.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:11.198+0000 7f399f48b640 1 Processor -- start 2024-09-21T00:29:11.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:11.198+0000 7f399f48b640 1 -- start start 2024-09-21T00:29:11.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:11.199+0000 7f399f48b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f39980708b0 0x7f3998070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:11.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:11.199+0000 7f399f48b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3998074c90 0x7f3998152480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:11.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:11.199+0000 7f399f48b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f39980750a0 0x7f39981f9120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:11.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:11.199+0000 7f399f48b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f399815f700 con 0x7f39980708b0 2024-09-21T00:29:11.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:11.199+0000 7f399f48b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f399815f580 con 0x7f3998074c90 2024-09-21T00:29:11.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:11.199+0000 7f399f48b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f399815f880 con 0x7f39980750a0 2024-09-21T00:29:11.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:11.199+0000 7f399d227640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f39980708b0 0x7f3998070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:11.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:11.200+0000 7f399d227640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f39980708b0 0x7f3998070260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39432/0 (socket says 172.21.15.112:39432) 2024-09-21T00:29:11.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:11.200+0000 7f399d227640 1 -- 172.21.15.112:0/3259194678 learned_addr learned my addr 172.21.15.112:0/3259194678 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:29:11.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:11.200+0000 7f399da28640 1 --2- 172.21.15.112:0/3259194678 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f39980750a0 0x7f39981f9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:11.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:11.200+0000 7f399ca26640 1 --2- 172.21.15.112:0/3259194678 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3998074c90 0x7f3998152480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:11.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:11.200+0000 7f399d227640 1 -- 172.21.15.112:0/3259194678 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f39980750a0 msgr2=0x7f39981f9120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:11.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:11.200+0000 7f399d227640 1 --2- 172.21.15.112:0/3259194678 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f39980750a0 0x7f39981f9120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:11.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:11.200+0000 7f399d227640 1 -- 172.21.15.112:0/3259194678 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3998074c90 msgr2=0x7f3998152480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:11.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:11.200+0000 7f399d227640 1 --2- 172.21.15.112:0/3259194678 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3998074c90 0x7f3998152480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:11.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:11.200+0000 7f399d227640 1 -- 172.21.15.112:0/3259194678 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3998152e30 con 0x7f39980708b0 2024-09-21T00:29:11.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:11.200+0000 7f399d227640 1 --2- 172.21.15.112:0/3259194678 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f39980708b0 0x7f3998070260 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f399400d750 tx=0x7f399400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:11.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:11.200+0000 7f398dffb640 1 -- 172.21.15.112:0/3259194678 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3994017070 con 0x7f39980708b0 2024-09-21T00:29:11.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:11.200+0000 7f398dffb640 1 -- 172.21.15.112:0/3259194678 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3994009b20 con 0x7f39980708b0 2024-09-21T00:29:11.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:11.200+0000 7f399f48b640 1 -- 172.21.15.112:0/3259194678 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3998153110 con 0x7f39980708b0 2024-09-21T00:29:11.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:11.200+0000 7f398dffb640 1 -- 172.21.15.112:0/3259194678 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3994010460 con 0x7f39980708b0 2024-09-21T00:29:11.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:11.200+0000 7f399f48b640 1 -- 172.21.15.112:0/3259194678 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3998156780 con 0x7f39980708b0 2024-09-21T00:29:11.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:11.201+0000 7f399f48b640 1 -- 172.21.15.112:0/3259194678 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39981569e0 con 0x7f39980708b0 2024-09-21T00:29:11.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:11.202+0000 7f398dffb640 1 -- 172.21.15.112:0/3259194678 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f399401a070 con 0x7f39980708b0 2024-09-21T00:29:11.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:11.202+0000 7f398dffb640 1 --2- 172.21.15.112:0/3259194678 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f397c078bc0 0x7f397c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:11.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:11.203+0000 7f398dffb640 1 -- 172.21.15.112:0/3259194678 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5335+0+0 (secure 0 0 0) 0x7f399409acc0 con 0x7f39980708b0 2024-09-21T00:29:11.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:11.203+0000 7f399ca26640 1 --2- 172.21.15.112:0/3259194678 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f397c078bc0 0x7f397c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:11.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:11.203+0000 7f399ca26640 1 --2- 172.21.15.112:0/3259194678 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f397c078bc0 0x7f397c07b0d0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f39980b2510 tx=0x7f3988006f90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:11.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:11.207+0000 7f398dffb640 1 -- 172.21.15.112:0/3259194678 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3994062c20 con 0x7f39980708b0 2024-09-21T00:29:11.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:11.452+0000 7f399f48b640 1 -- 172.21.15.112:0/3259194678 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_sessionmap.TestSessionMap.test_tell_conn_close"]} v 0) -- 0x7f399815fd90 con 0x7f39980708b0 2024-09-21T00:29:11.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:11.738+0000 7f398dffb640 1 -- 172.21.15.112:0/3259194678 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_sessionmap.TestSessionMap.test_tell_conn_close"]}]=0 v208) ==== 143+0+0 (secure 0 0 0) 0x7f3994067650 con 0x7f39980708b0 2024-09-21T00:29:11.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:11.741+0000 7f399f48b640 1 -- 172.21.15.112:0/3259194678 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f397c078bc0 msgr2=0x7f397c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:11.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:11.741+0000 7f399f48b640 1 --2- 172.21.15.112:0/3259194678 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f397c078bc0 0x7f397c07b0d0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f39980b2510 tx=0x7f3988006f90 comp rx=0 tx=0).stop 2024-09-21T00:29:11.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:11.741+0000 7f399f48b640 1 -- 172.21.15.112:0/3259194678 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f39980708b0 msgr2=0x7f3998070260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:11.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:11.741+0000 7f399f48b640 1 --2- 172.21.15.112:0/3259194678 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f39980708b0 0x7f3998070260 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f399400d750 tx=0x7f399400dc10 comp rx=0 tx=0).stop 2024-09-21T00:29:11.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:11.741+0000 7f399f48b640 1 -- 172.21.15.112:0/3259194678 shutdown_connections 2024-09-21T00:29:11.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:11.741+0000 7f399f48b640 1 --2- 172.21.15.112:0/3259194678 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f39980750a0 0x7f39981f9120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:11.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:11.741+0000 7f399f48b640 1 --2- 172.21.15.112:0/3259194678 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3998074c90 0x7f3998152480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:11.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:11.741+0000 7f399f48b640 1 --2- 172.21.15.112:0/3259194678 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f397c078bc0 0x7f397c07b0d0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:11.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:11.741+0000 7f399f48b640 1 --2- 172.21.15.112:0/3259194678 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f39980708b0 0x7f3998070260 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:11.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:11.741+0000 7f399f48b640 1 -- 172.21.15.112:0/3259194678 >> 172.21.15.112:0/3259194678 conn(0x7f3998073fe0 msgr2=0x7f39981e8e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:11.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:11.741+0000 7f399f48b640 1 -- 172.21.15.112:0/3259194678 shutdown_connections 2024-09-21T00:29:11.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:11.741+0000 7f399f48b640 1 -- 172.21.15.112:0/3259194678 wait complete. 2024-09-21T00:29:11.755 INFO:tasks.cephfs_test_runner:That when a `tell` command is sent using the python CLI, ... ok 2024-09-21T00:29:11.755 INFO:tasks.cephfs_test_runner:Starting test: test_tell_session_drop (tasks.cephfs.test_sessionmap.TestSessionMap) That when a `tell` command is sent using the python CLI, 2024-09-21T00:29:11.755 INFO:tasks.cephfs_test_runner:test_tell_session_drop (tasks.cephfs.test_sessionmap.TestSessionMap) 2024-09-21T00:29:11.755 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T00:29:11.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:11.867+0000 7fbd7e270640 1 -- 172.21.15.112:0/3202691850 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbd60005330 msgr2=0x7fbd6000b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:11.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:11.867+0000 7fbd7e270640 1 --2- 172.21.15.112:0/3202691850 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbd60005330 0x7fbd6000b9e0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fbd70005fd0 tx=0x7fbd60005830 comp rx=0 tx=0).stop 2024-09-21T00:29:11.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:11.867+0000 7fbd7e270640 1 -- 172.21.15.112:0/3202691850 shutdown_connections 2024-09-21T00:29:11.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:11.867+0000 7fbd7e270640 1 --2- 172.21.15.112:0/3202691850 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbd60005330 0x7fbd6000b9e0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:11.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:11.867+0000 7fbd7e270640 1 -- 172.21.15.112:0/3202691850 >> 172.21.15.112:0/3202691850 conn(0x7fbd780b43f0 msgr2=0x7fbd780b4830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:11.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:11.867+0000 7fbd7e270640 1 -- 172.21.15.112:0/3202691850 shutdown_connections 2024-09-21T00:29:11.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:11.867+0000 7fbd7e270640 1 -- 172.21.15.112:0/3202691850 wait complete. 2024-09-21T00:29:11.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:11.869+0000 7fbd7e270640 1 Processor -- start 2024-09-21T00:29:11.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:11.869+0000 7fbd7e270640 1 -- start start 2024-09-21T00:29:11.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:11.870+0000 7fbd7e270640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbd60005330 0x7fbd781586d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:11.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:11.870+0000 7fbd7e270640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbd78158d60 0x7fbd780cbae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:11.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:11.870+0000 7fbd7e270640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbd780cc020 0x7fbd780c7270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:11.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:11.870+0000 7fbd7e270640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbd7815a450 con 0x7fbd78158d60 2024-09-21T00:29:11.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:11.870+0000 7fbd7e270640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fbd7815a2d0 con 0x7fbd60005330 2024-09-21T00:29:11.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:11.870+0000 7fbd7e270640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fbd7815a5d0 con 0x7fbd780cc020 2024-09-21T00:29:11.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:11.870+0000 7fbd777fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbd78158d60 0x7fbd780cbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:11.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:11.870+0000 7fbd777fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbd78158d60 0x7fbd780cbae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39444/0 (socket says 172.21.15.112:39444) 2024-09-21T00:29:11.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:11.870+0000 7fbd777fe640 1 -- 172.21.15.112:0/4174366897 learned_addr learned my addr 172.21.15.112:0/4174366897 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:29:11.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:11.871+0000 7fbd7c80d640 1 --2- 172.21.15.112:0/4174366897 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbd780cc020 0x7fbd780c7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:11.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:11.871+0000 7fbd77fff640 1 --2- 172.21.15.112:0/4174366897 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbd60005330 0x7fbd781586d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:11.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:11.871+0000 7fbd777fe640 1 -- 172.21.15.112:0/4174366897 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbd780cc020 msgr2=0x7fbd780c7270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:11.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:11.871+0000 7fbd777fe640 1 --2- 172.21.15.112:0/4174366897 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbd780cc020 0x7fbd780c7270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:11.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:11.871+0000 7fbd777fe640 1 -- 172.21.15.112:0/4174366897 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbd60005330 msgr2=0x7fbd781586d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:11.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:11.871+0000 7fbd777fe640 1 --2- 172.21.15.112:0/4174366897 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbd60005330 0x7fbd781586d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:11.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:11.871+0000 7fbd777fe640 1 -- 172.21.15.112:0/4174366897 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd780cc430 con 0x7fbd78158d60 2024-09-21T00:29:11.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:11.871+0000 7fbd77fff640 1 --2- 172.21.15.112:0/4174366897 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbd60005330 0x7fbd781586d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:29:11.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:11.871+0000 7fbd777fe640 1 --2- 172.21.15.112:0/4174366897 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbd78158d60 0x7fbd780cbae0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c00e570 tx=0x7fbd6c00ea30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:11.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:11.871+0000 7fbd74ff9640 1 -- 172.21.15.112:0/4174366897 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd6c015020 con 0x7fbd78158d60 2024-09-21T00:29:11.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:11.871+0000 7fbd74ff9640 1 -- 172.21.15.112:0/4174366897 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbd6c010a40 con 0x7fbd78158d60 2024-09-21T00:29:11.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:11.871+0000 7fbd7e270640 1 -- 172.21.15.112:0/4174366897 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd780ca430 con 0x7fbd78158d60 2024-09-21T00:29:11.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:11.871+0000 7fbd74ff9640 1 -- 172.21.15.112:0/4174366897 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd6c011460 con 0x7fbd78158d60 2024-09-21T00:29:11.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:11.871+0000 7fbd7e270640 1 -- 172.21.15.112:0/4174366897 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd781f3b90 con 0x7fbd78158d60 2024-09-21T00:29:11.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:11.872+0000 7fbd7e270640 1 -- 172.21.15.112:0/4174366897 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd780ccca0 con 0x7fbd78158d60 2024-09-21T00:29:11.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:11.873+0000 7fbd74ff9640 1 -- 172.21.15.112:0/4174366897 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fbd6c018020 con 0x7fbd78158d60 2024-09-21T00:29:11.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:11.873+0000 7fbd74ff9640 1 --2- 172.21.15.112:0/4174366897 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fbd64078b90 0x7fbd6407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:11.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:11.873+0000 7fbd74ff9640 1 -- 172.21.15.112:0/4174366897 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5335+0+0 (secure 0 0 0) 0x7fbd6c09af80 con 0x7fbd78158d60 2024-09-21T00:29:11.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:11.873+0000 7fbd77fff640 1 --2- 172.21.15.112:0/4174366897 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fbd64078b90 0x7fbd6407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:11.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:11.874+0000 7fbd77fff640 1 --2- 172.21.15.112:0/4174366897 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fbd64078b90 0x7fbd6407b0a0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fbd68000c00 tx=0x7fbd68006180 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:11.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:11.878+0000 7fbd74ff9640 1 -- 172.21.15.112:0/4174366897 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fbd6c01f030 con 0x7fbd78158d60 2024-09-21T00:29:12.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:12.055+0000 7fbd7e270640 1 -- 172.21.15.112:0/4174366897 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbd7815b120 con 0x7fbd78158d60 2024-09-21T00:29:12.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:12.056+0000 7fbd74ff9640 1 -- 172.21.15.112:0/4174366897 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+13330 (secure 0 0 0) 0x7fbd780cc430 con 0x7fbd78158d60 2024-09-21T00:29:12.058 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:29:12.058 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":86,"fsid":"8b34692b-dc65-49b2-a08f-26f707b96ac5","created":"2024-09-21T00:25:22.788490+0000","modified":"2024-09-21T00:29:10.733339+0000","last_up_change":"2024-09-21T00:25:36.161835+0000","last_in_change":"2024-09-21T00:25:26.346500+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T00:25:37.105006+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9c575086-67d2-4de2-b01b-d44dcc21d604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6809","nonce":755760621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6811","nonce":755760621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6815","nonce":755760621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6813","nonce":755760621}]},"public_addr":"172.21.15.112:6809/755760621","cluster_addr":"172.21.15.112:6811/755760621","heartbeat_back_addr":"172.21.15.112:6815/755760621","heartbeat_front_addr":"172.21.15.112:6813/755760621","state":["exists","up"]},{"osd":1,"uuid":"a983c227-a3ce-4bdd-82f1-438ba7e3ecd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6801","nonce":344932592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6803","nonce":344932592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6807","nonce":344932592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6805","nonce":344932592}]},"public_addr":"172.21.15.112:6801/344932592","cluster_addr":"172.21.15.112:6803/344932592","heartbeat_back_addr":"172.21.15.112:6807/344932592","heartbeat_front_addr":"172.21.15.112:6805/344932592","state":["exists","up"]},{"osd":2,"uuid":"1193c1d8-bfff-4a57-8200-5f69bf1ea3af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6825","nonce":407933616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6827","nonce":407933616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6831","nonce":407933616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6829","nonce":407933616}]},"public_addr":"172.21.15.112:6825/407933616","cluster_addr":"172.21.15.112:6827/407933616","heartbeat_back_addr":"172.21.15.112:6831/407933616","heartbeat_front_addr":"172.21.15.112:6829/407933616","state":["exists","up"]},{"osd":3,"uuid":"af04330a-36ad-40cc-a8dd-fa259b221de5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6817","nonce":4120164806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6819","nonce":4120164806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6823","nonce":4120164806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6821","nonce":4120164806}]},"public_addr":"172.21.15.112:6817/4120164806","cluster_addr":"172.21.15.112:6819/4120164806","heartbeat_back_addr":"172.21.15.112:6823/4120164806","heartbeat_front_addr":"172.21.15.112:6821/4120164806","state":["exists","up"]},{"osd":4,"uuid":"5b893111-292c-4c5a-b184-53ad38c7199e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6803","nonce":522077560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6805","nonce":522077560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6809","nonce":522077560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6807","nonce":522077560}]},"public_addr":"172.21.15.187:6803/522077560","cluster_addr":"172.21.15.187:6805/522077560","heartbeat_back_addr":"172.21.15.187:6809/522077560","heartbeat_front_addr":"172.21.15.187:6807/522077560","state":["exists","up"]},{"osd":5,"uuid":"c0b8e011-45fe-4c5e-925e-6be2a7f1868a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6827","nonce":3593144721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6829","nonce":3593144721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6833","nonce":3593144721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6831","nonce":3593144721}]},"public_addr":"172.21.15.187:6827/3593144721","cluster_addr":"172.21.15.187:6829/3593144721","heartbeat_back_addr":"172.21.15.187:6833/3593144721","heartbeat_front_addr":"172.21.15.187:6831/3593144721","state":["exists","up"]},{"osd":6,"uuid":"87d2d650-7489-4593-98a3-a1f433ce8dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6811","nonce":1494387657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6813","nonce":1494387657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6817","nonce":1494387657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6815","nonce":1494387657}]},"public_addr":"172.21.15.187:6811/1494387657","cluster_addr":"172.21.15.187:6813/1494387657","heartbeat_back_addr":"172.21.15.187:6817/1494387657","heartbeat_front_addr":"172.21.15.187:6815/1494387657","state":["exists","up"]},{"osd":7,"uuid":"c6b4f8e2-a1ee-4438-aa37-67ff4b6ef1f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6819","nonce":1624699842}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6821","nonce":1624699842}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6825","nonce":1624699842}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6823","nonce":1624699842}]},"public_addr":"172.21.15.187:6819/1624699842","cluster_addr":"172.21.15.187:6821/1624699842","heartbeat_back_addr":"172.21.15.187:6825/1624699842","heartbeat_front_addr":"172.21.15.187:6823/1624699842","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.873162+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.400506+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.664317+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.159791+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.202630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.113212+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.384786+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.448441+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.187:6837/932692802":"2024-09-22T00:29:07.724943+0000","172.21.15.187:6835/932692802":"2024-09-22T00:29:07.724943+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T00:29:12.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:12.058+0000 7fbd7e270640 1 -- 172.21.15.112:0/4174366897 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fbd64078b90 msgr2=0x7fbd6407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:12.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:12.058+0000 7fbd7e270640 1 --2- 172.21.15.112:0/4174366897 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fbd64078b90 0x7fbd6407b0a0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fbd68000c00 tx=0x7fbd68006180 comp rx=0 tx=0).stop 2024-09-21T00:29:12.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:12.058+0000 7fbd7e270640 1 -- 172.21.15.112:0/4174366897 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbd78158d60 msgr2=0x7fbd780cbae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:12.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:12.058+0000 7fbd7e270640 1 --2- 172.21.15.112:0/4174366897 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbd78158d60 0x7fbd780cbae0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c00e570 tx=0x7fbd6c00ea30 comp rx=0 tx=0).stop 2024-09-21T00:29:12.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:12.058+0000 7fbd7e270640 1 -- 172.21.15.112:0/4174366897 shutdown_connections 2024-09-21T00:29:12.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:12.058+0000 7fbd7e270640 1 --2- 172.21.15.112:0/4174366897 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbd780cc020 0x7fbd780c7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:12.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:12.058+0000 7fbd7e270640 1 --2- 172.21.15.112:0/4174366897 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fbd64078b90 0x7fbd6407b0a0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:12.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:12.058+0000 7fbd7e270640 1 --2- 172.21.15.112:0/4174366897 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbd78158d60 0x7fbd780cbae0 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:12.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:12.058+0000 7fbd7e270640 1 --2- 172.21.15.112:0/4174366897 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbd60005330 0x7fbd781586d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:12.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:12.058+0000 7fbd7e270640 1 -- 172.21.15.112:0/4174366897 >> 172.21.15.112:0/4174366897 conn(0x7fbd780b43f0 msgr2=0x7fbd78159600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:12.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:12.059+0000 7fbd7e270640 1 -- 172.21.15.112:0/4174366897 shutdown_connections 2024-09-21T00:29:12.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:12.059+0000 7fbd7e270640 1 -- 172.21.15.112:0/4174366897 wait complete. 2024-09-21T00:29:12.072 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T00:25:37.105006+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-21T00:29:12.072 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-21T00:29:12.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:12.186+0000 7fd3fd494640 1 -- 172.21.15.112:0/1997178968 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3e4004800 msgr2=0x7fd3e4004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:12.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:12.186+0000 7fd3fd494640 1 --2- 172.21.15.112:0/1997178968 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3e4004800 0x7fd3e4004c90 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fd3e000baf0 tx=0x7fd3e4005830 comp rx=0 tx=0).stop 2024-09-21T00:29:12.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:12.186+0000 7fd3fd494640 1 -- 172.21.15.112:0/1997178968 shutdown_connections 2024-09-21T00:29:12.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:12.186+0000 7fd3fd494640 1 --2- 172.21.15.112:0/1997178968 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3e4004800 0x7fd3e4004c90 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:12.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:12.186+0000 7fd3fd494640 1 -- 172.21.15.112:0/1997178968 >> 172.21.15.112:0/1997178968 conn(0x7fd3f8073fe0 msgr2=0x7fd3f80743f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:12.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:12.186+0000 7fd3fd494640 1 -- 172.21.15.112:0/1997178968 shutdown_connections 2024-09-21T00:29:12.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:12.187+0000 7fd3fd494640 1 -- 172.21.15.112:0/1997178968 wait complete. 2024-09-21T00:29:12.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:12.188+0000 7fd3fd494640 1 Processor -- start 2024-09-21T00:29:12.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:12.188+0000 7fd3fd494640 1 -- start start 2024-09-21T00:29:12.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:12.190+0000 7fd3fd494640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd3e4004800 0x7fd3f813ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:12.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:12.190+0000 7fd3fd494640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd3f8143d80 0x7fd3f80ca5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:12.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:12.190+0000 7fd3fd494640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3f80b59a0 0x7fd3f80b5db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:12.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:12.190+0000 7fd3fd494640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd3f815ca00 con 0x7fd3f80b59a0 2024-09-21T00:29:12.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:12.190+0000 7fd3fd494640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd3f8074bb0 con 0x7fd3f8143d80 2024-09-21T00:29:12.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:12.190+0000 7fd3fd494640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd3f8157190 con 0x7fd3e4004800 2024-09-21T00:29:12.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:12.190+0000 7fd3f77fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3f80b59a0 0x7fd3f80b5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:12.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:12.190+0000 7fd3f77fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3f80b59a0 0x7fd3f80b5db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39452/0 (socket says 172.21.15.112:39452) 2024-09-21T00:29:12.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:12.190+0000 7fd3f77fe640 1 -- 172.21.15.112:0/564152821 learned_addr learned my addr 172.21.15.112:0/564152821 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:29:12.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:12.190+0000 7fd3f67fc640 1 --2- 172.21.15.112:0/564152821 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd3f8143d80 0x7fd3f80ca5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:12.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:12.190+0000 7fd3f6ffd640 1 --2- 172.21.15.112:0/564152821 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd3e4004800 0x7fd3f813ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:12.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:12.190+0000 7fd3f77fe640 1 -- 172.21.15.112:0/564152821 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd3e4004800 msgr2=0x7fd3f813ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:12.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:12.190+0000 7fd3f77fe640 1 --2- 172.21.15.112:0/564152821 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd3e4004800 0x7fd3f813ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:12.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:12.190+0000 7fd3f77fe640 1 -- 172.21.15.112:0/564152821 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd3f8143d80 msgr2=0x7fd3f80ca5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:12.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:12.190+0000 7fd3f77fe640 1 --2- 172.21.15.112:0/564152821 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd3f8143d80 0x7fd3f80ca5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:12.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:12.190+0000 7fd3f77fe640 1 -- 172.21.15.112:0/564152821 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3f80b62f0 con 0x7fd3f80b59a0 2024-09-21T00:29:12.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:12.190+0000 7fd3f77fe640 1 --2- 172.21.15.112:0/564152821 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3f80b59a0 0x7fd3f80b5db0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fd3f0000c00 tx=0x7fd3f000e6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:12.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:12.190+0000 7fd3ef7fe640 1 -- 172.21.15.112:0/564152821 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3f0011660 con 0x7fd3f80b59a0 2024-09-21T00:29:12.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:12.190+0000 7fd3ef7fe640 1 -- 172.21.15.112:0/564152821 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd3f0017070 con 0x7fd3f80b59a0 2024-09-21T00:29:12.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:12.191+0000 7fd3fd494640 1 -- 172.21.15.112:0/564152821 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3f80caf10 con 0x7fd3f80b59a0 2024-09-21T00:29:12.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:12.191+0000 7fd3ef7fe640 1 -- 172.21.15.112:0/564152821 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3f0011bb0 con 0x7fd3f80b59a0 2024-09-21T00:29:12.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:12.191+0000 7fd3fd494640 1 -- 172.21.15.112:0/564152821 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3f81f8da0 con 0x7fd3f80b59a0 2024-09-21T00:29:12.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:12.192+0000 7fd3fd494640 1 -- 172.21.15.112:0/564152821 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3f80ccfc0 con 0x7fd3f80b59a0 2024-09-21T00:29:12.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:12.193+0000 7fd3ef7fe640 1 -- 172.21.15.112:0/564152821 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd3f00253d0 con 0x7fd3f80b59a0 2024-09-21T00:29:12.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:12.193+0000 7fd3ef7fe640 1 --2- 172.21.15.112:0/564152821 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fd3d8078bc0 0x7fd3d807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:12.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:12.193+0000 7fd3ef7fe640 1 -- 172.21.15.112:0/564152821 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5335+0+0 (secure 0 0 0) 0x7fd3f009b830 con 0x7fd3f80b59a0 2024-09-21T00:29:12.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:12.193+0000 7fd3f6ffd640 1 --2- 172.21.15.112:0/564152821 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fd3d8078bc0 0x7fd3d807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:12.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:12.194+0000 7fd3f6ffd640 1 --2- 172.21.15.112:0/564152821 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fd3d8078bc0 0x7fd3d807b0d0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fd3e8004fd0 tx=0x7fd3e8004ad0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:12.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:12.197+0000 7fd3ef7fe640 1 -- 172.21.15.112:0/564152821 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd3f80b62f0 con 0x7fd3f80b59a0 2024-09-21T00:29:12.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:12.373+0000 7fd3fd494640 1 -- 172.21.15.112:0/564152821 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd3f80cb6c0 con 0x7fd3f80b59a0 2024-09-21T00:29:12.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:12.373+0000 7fd3ef7fe640 1 -- 172.21.15.112:0/564152821 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v86) ==== 93+0+10 (secure 0 0 0) 0x7fd3f0063990 con 0x7fd3f80b59a0 2024-09-21T00:29:12.375 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2024-09-21T00:29:12.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:12.376+0000 7fd3fd494640 1 -- 172.21.15.112:0/564152821 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fd3d8078bc0 msgr2=0x7fd3d807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:12.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:12.376+0000 7fd3fd494640 1 --2- 172.21.15.112:0/564152821 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fd3d8078bc0 0x7fd3d807b0d0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fd3e8004fd0 tx=0x7fd3e8004ad0 comp rx=0 tx=0).stop 2024-09-21T00:29:12.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:12.376+0000 7fd3fd494640 1 -- 172.21.15.112:0/564152821 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3f80b59a0 msgr2=0x7fd3f80b5db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:12.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:12.376+0000 7fd3fd494640 1 --2- 172.21.15.112:0/564152821 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3f80b59a0 0x7fd3f80b5db0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fd3f0000c00 tx=0x7fd3f000e6b0 comp rx=0 tx=0).stop 2024-09-21T00:29:12.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:12.376+0000 7fd3fd494640 1 -- 172.21.15.112:0/564152821 shutdown_connections 2024-09-21T00:29:12.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:12.376+0000 7fd3fd494640 1 --2- 172.21.15.112:0/564152821 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fd3d8078bc0 0x7fd3d807b0d0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:12.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:12.376+0000 7fd3fd494640 1 --2- 172.21.15.112:0/564152821 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3f80b59a0 0x7fd3f80b5db0 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:12.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:12.376+0000 7fd3fd494640 1 --2- 172.21.15.112:0/564152821 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd3f8143d80 0x7fd3f80ca5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:12.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:12.376+0000 7fd3fd494640 1 --2- 172.21.15.112:0/564152821 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd3e4004800 0x7fd3f813ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:12.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:12.376+0000 7fd3fd494640 1 -- 172.21.15.112:0/564152821 >> 172.21.15.112:0/564152821 conn(0x7fd3f8073fe0 msgr2=0x7fd3f80b0eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:12.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:12.376+0000 7fd3fd494640 1 -- 172.21.15.112:0/564152821 shutdown_connections 2024-09-21T00:29:12.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:12.376+0000 7fd3fd494640 1 -- 172.21.15.112:0/564152821 wait complete. 2024-09-21T00:29:12.389 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_sessionmap.TestSessionMap.test_tell_session_drop' 2024-09-21T00:29:12.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:12.500+0000 7ff841269640 1 -- 172.21.15.112:0/2015331544 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff83c15c3c0 msgr2=0x7ff83c0b3260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:12.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:12.500+0000 7ff841269640 1 --2- 172.21.15.112:0/2015331544 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff83c15c3c0 0x7ff83c0b3260 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff82c005fd0 tx=0x7ff82c02a980 comp rx=0 tx=0).stop 2024-09-21T00:29:12.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:12.500+0000 7ff841269640 1 -- 172.21.15.112:0/2015331544 shutdown_connections 2024-09-21T00:29:12.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:12.500+0000 7ff841269640 1 --2- 172.21.15.112:0/2015331544 >> v2:172.21.15.187:3300/0 conn(0x7ff83c0b37a0 0x7ff83c0b6030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:12.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:12.500+0000 7ff841269640 1 --2- 172.21.15.112:0/2015331544 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff83c15c3c0 0x7ff83c0b3260 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:12.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:12.500+0000 7ff841269640 1 -- 172.21.15.112:0/2015331544 >> 172.21.15.112:0/2015331544 conn(0x7ff83c08dcb0 msgr2=0x7ff83c09d8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:12.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:12.500+0000 7ff841269640 1 -- 172.21.15.112:0/2015331544 shutdown_connections 2024-09-21T00:29:12.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:12.500+0000 7ff841269640 1 -- 172.21.15.112:0/2015331544 wait complete. 2024-09-21T00:29:12.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:12.502+0000 7ff841269640 1 Processor -- start 2024-09-21T00:29:12.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:12.502+0000 7ff841269640 1 -- start start 2024-09-21T00:29:12.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:12.503+0000 7ff841269640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff83c0b37a0 0x7ff83c06e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:12.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:12.503+0000 7ff841269640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff83c0b3bb0 0x7ff83c15a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:12.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:12.503+0000 7ff841269640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff83c15c3c0 0x7ff83c0a1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:12.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:12.503+0000 7ff841269640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff83c163180 con 0x7ff83c0b37a0 2024-09-21T00:29:12.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:12.503+0000 7ff841269640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff83c0d0d30 con 0x7ff83c15c3c0 2024-09-21T00:29:12.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:12.503+0000 7ff841269640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff83c163300 con 0x7ff83c0b3bb0 2024-09-21T00:29:12.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:12.503+0000 7ff83affd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff83c0b37a0 0x7ff83c06e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:12.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:12.503+0000 7ff83affd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff83c0b37a0 0x7ff83c06e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39460/0 (socket says 172.21.15.112:39460) 2024-09-21T00:29:12.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:12.503+0000 7ff83affd640 1 -- 172.21.15.112:0/740230162 learned_addr learned my addr 172.21.15.112:0/740230162 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:29:12.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:12.503+0000 7ff83a7fc640 1 --2- 172.21.15.112:0/740230162 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff83c0b3bb0 0x7ff83c15a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:12.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:12.503+0000 7ff83b7fe640 1 --2- 172.21.15.112:0/740230162 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff83c15c3c0 0x7ff83c0a1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:12.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:12.504+0000 7ff83affd640 1 -- 172.21.15.112:0/740230162 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff83c0b3bb0 msgr2=0x7ff83c15a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:12.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:12.504+0000 7ff83affd640 1 --2- 172.21.15.112:0/740230162 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff83c0b3bb0 0x7ff83c15a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:12.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:12.504+0000 7ff83affd640 1 -- 172.21.15.112:0/740230162 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff83c15c3c0 msgr2=0x7ff83c0a1bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:12.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:12.504+0000 7ff83affd640 1 --2- 172.21.15.112:0/740230162 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff83c15c3c0 0x7ff83c0a1bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:12.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:12.504+0000 7ff83affd640 1 -- 172.21.15.112:0/740230162 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff83c0cc4f0 con 0x7ff83c0b37a0 2024-09-21T00:29:12.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:12.504+0000 7ff83a7fc640 1 --2- 172.21.15.112:0/740230162 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff83c0b3bb0 0x7ff83c15a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:29:12.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:12.504+0000 7ff83affd640 1 --2- 172.21.15.112:0/740230162 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff83c0b37a0 0x7ff83c06e600 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7ff82c006010 tx=0x7ff82c003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:12.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:12.504+0000 7ff8237fe640 1 -- 172.21.15.112:0/740230162 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff82c034040 con 0x7ff83c0b37a0 2024-09-21T00:29:12.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:12.504+0000 7ff841269640 1 -- 172.21.15.112:0/740230162 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff83c06ff00 con 0x7ff83c0b37a0 2024-09-21T00:29:12.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:12.504+0000 7ff8237fe640 1 -- 172.21.15.112:0/740230162 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff82c004e30 con 0x7ff83c0b37a0 2024-09-21T00:29:12.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:12.504+0000 7ff8237fe640 1 -- 172.21.15.112:0/740230162 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff82c0044b0 con 0x7ff83c0b37a0 2024-09-21T00:29:12.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:12.504+0000 7ff841269640 1 -- 172.21.15.112:0/740230162 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff83c0cc210 con 0x7ff83c0b37a0 2024-09-21T00:29:12.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:12.505+0000 7ff841269640 1 -- 172.21.15.112:0/740230162 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff83c0b3260 con 0x7ff83c0b37a0 2024-09-21T00:29:12.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:12.506+0000 7ff8237fe640 1 -- 172.21.15.112:0/740230162 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff82c037020 con 0x7ff83c0b37a0 2024-09-21T00:29:12.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:12.506+0000 7ff8237fe640 1 --2- 172.21.15.112:0/740230162 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff828078b90 0x7ff82807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:12.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:12.506+0000 7ff8237fe640 1 -- 172.21.15.112:0/740230162 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5335+0+0 (secure 0 0 0) 0x7ff82c0bbfc0 con 0x7ff83c0b37a0 2024-09-21T00:29:12.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:12.506+0000 7ff83a7fc640 1 --2- 172.21.15.112:0/740230162 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff828078b90 0x7ff82807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:12.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:12.507+0000 7ff83a7fc640 1 --2- 172.21.15.112:0/740230162 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff828078b90 0x7ff82807b0a0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7ff83c15b2a0 tx=0x7ff830005f90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:12.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:12.511+0000 7ff8237fe640 1 -- 172.21.15.112:0/740230162 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ff83c0cc4f0 con 0x7ff83c0b37a0 2024-09-21T00:29:12.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:12.754+0000 7ff841269640 1 -- 172.21.15.112:0/740230162 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_sessionmap.TestSessionMap.test_tell_session_drop"]} v 0) -- 0x7ff83c15f7f0 con 0x7ff83c0b37a0 2024-09-21T00:29:13.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:13.747+0000 7ff8237fe640 1 -- 172.21.15.112:0/740230162 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_sessionmap.TestSessionMap.test_tell_session_drop"]}]=0 v210) ==== 148+0+0 (secure 0 0 0) 0x7ff82c083ea0 con 0x7ff83c0b37a0 2024-09-21T00:29:13.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:13.750+0000 7ff841269640 1 -- 172.21.15.112:0/740230162 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff828078b90 msgr2=0x7ff82807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:13.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:13.750+0000 7ff841269640 1 --2- 172.21.15.112:0/740230162 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff828078b90 0x7ff82807b0a0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7ff83c15b2a0 tx=0x7ff830005f90 comp rx=0 tx=0).stop 2024-09-21T00:29:13.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:13.750+0000 7ff841269640 1 -- 172.21.15.112:0/740230162 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff83c0b37a0 msgr2=0x7ff83c06e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:13.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:13.750+0000 7ff841269640 1 --2- 172.21.15.112:0/740230162 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff83c0b37a0 0x7ff83c06e600 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7ff82c006010 tx=0x7ff82c003f10 comp rx=0 tx=0).stop 2024-09-21T00:29:13.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:13.750+0000 7ff841269640 1 -- 172.21.15.112:0/740230162 shutdown_connections 2024-09-21T00:29:13.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:13.750+0000 7ff841269640 1 --2- 172.21.15.112:0/740230162 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff83c15c3c0 0x7ff83c0a1bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:13.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:13.750+0000 7ff841269640 1 --2- 172.21.15.112:0/740230162 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff83c0b3bb0 0x7ff83c15a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:13.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:13.750+0000 7ff841269640 1 --2- 172.21.15.112:0/740230162 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff828078b90 0x7ff82807b0a0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:13.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:13.750+0000 7ff841269640 1 --2- 172.21.15.112:0/740230162 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff83c0b37a0 0x7ff83c06e600 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:13.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:13.750+0000 7ff841269640 1 -- 172.21.15.112:0/740230162 >> 172.21.15.112:0/740230162 conn(0x7ff83c08dcb0 msgr2=0x7ff83c0d4390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:13.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:13.750+0000 7ff841269640 1 -- 172.21.15.112:0/740230162 shutdown_connections 2024-09-21T00:29:13.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:13.750+0000 7ff841269640 1 -- 172.21.15.112:0/740230162 wait complete. 2024-09-21T00:29:13.767 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-21T00:29:13.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:13.884+0000 7ffa04ac5640 1 -- 172.21.15.112:0/1459033403 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff9ec004f40 msgr2=0x7ff9ec00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:13.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:13.884+0000 7ffa04ac5640 1 --2- 172.21.15.112:0/1459033403 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff9ec004f40 0x7ff9ec00b9c0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7ff9f8009ed0 tx=0x7ff9ec005830 comp rx=0 tx=0).stop 2024-09-21T00:29:13.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:13.884+0000 7ff9fed9d640 1 -- 172.21.15.112:0/1459033403 reap_dead start 2024-09-21T00:29:13.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:13.884+0000 7ffa04ac5640 1 -- 172.21.15.112:0/1459033403 shutdown_connections 2024-09-21T00:29:13.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:13.884+0000 7ffa04ac5640 1 -- 172.21.15.112:0/1459033403 >> 172.21.15.112:0/1459033403 conn(0x7ffa000b42e0 msgr2=0x7ffa000b4720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:13.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:13.884+0000 7ffa04ac5640 1 -- 172.21.15.112:0/1459033403 shutdown_connections 2024-09-21T00:29:13.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:13.884+0000 7ffa04ac5640 1 -- 172.21.15.112:0/1459033403 wait complete. 2024-09-21T00:29:13.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:13.886+0000 7ffa04ac5640 1 Processor -- start 2024-09-21T00:29:13.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:13.886+0000 7ffa04ac5640 1 -- start start 2024-09-21T00:29:13.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:13.887+0000 7ffa04ac5640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff9ec004f40 0x7ffa0014f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:13.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:13.887+0000 7ffa04ac5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa00155500 0x7ffa0015a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:13.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:13.887+0000 7ffa04ac5640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffa0015a8e0 0x7ffa001f3470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:13.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:13.887+0000 7ffa04ac5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ffa00165650 con 0x7ffa00155500 2024-09-21T00:29:13.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:13.887+0000 7ffa04ac5640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ffa001654d0 con 0x7ffa0015a8e0 2024-09-21T00:29:13.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:13.887+0000 7ffa04ac5640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ffa001657d0 con 0x7ff9ec004f40 2024-09-21T00:29:13.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:13.888+0000 7ff9f5d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa00155500 0x7ffa0015a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:13.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:13.888+0000 7ff9f5d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa00155500 0x7ffa0015a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39474/0 (socket says 172.21.15.112:39474) 2024-09-21T00:29:13.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:13.888+0000 7ff9f5d9b640 1 -- 172.21.15.112:0/3793922566 learned_addr learned my addr 172.21.15.112:0/3793922566 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:29:13.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:13.888+0000 7ff9fe59c640 1 --2- 172.21.15.112:0/3793922566 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff9ec004f40 0x7ffa0014f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:13.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:13.888+0000 7ff9fed9d640 1 --2- 172.21.15.112:0/3793922566 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffa0015a8e0 0x7ffa001f3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:13.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:13.888+0000 7ff9f5d9b640 1 -- 172.21.15.112:0/3793922566 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff9ec004f40 msgr2=0x7ffa0014f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:13.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:13.888+0000 7ff9f5d9b640 1 --2- 172.21.15.112:0/3793922566 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff9ec004f40 0x7ffa0014f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:13.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:13.888+0000 7ff9f5d9b640 1 -- 172.21.15.112:0/3793922566 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffa0015a8e0 msgr2=0x7ffa001f3470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:13.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:13.888+0000 7ff9f5d9b640 1 --2- 172.21.15.112:0/3793922566 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffa0015a8e0 0x7ffa001f3470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:13.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:13.888+0000 7ff9f5d9b640 1 -- 172.21.15.112:0/3793922566 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa0015acf0 con 0x7ffa00155500 2024-09-21T00:29:13.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:13.888+0000 7ff9fed9d640 1 --2- 172.21.15.112:0/3793922566 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffa0015a8e0 0x7ffa001f3470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:29:13.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:13.888+0000 7ff9f5d9b640 1 --2- 172.21.15.112:0/3793922566 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa00155500 0x7ffa0015a3a0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7ff9e8003f60 tx=0x7ff9e8003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:13.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:13.888+0000 7ff9f77fe640 1 -- 172.21.15.112:0/3793922566 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9e8011640 con 0x7ffa00155500 2024-09-21T00:29:13.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:13.888+0000 7ff9f77fe640 1 -- 172.21.15.112:0/3793922566 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff9e8016020 con 0x7ffa00155500 2024-09-21T00:29:13.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:13.888+0000 7ffa04ac5640 1 -- 172.21.15.112:0/3793922566 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa001500c0 con 0x7ffa00155500 2024-09-21T00:29:13.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:13.888+0000 7ff9f77fe640 1 -- 172.21.15.112:0/3793922566 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9e8011a60 con 0x7ffa00155500 2024-09-21T00:29:13.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:13.888+0000 7ffa04ac5640 1 -- 172.21.15.112:0/3793922566 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa001f6a60 con 0x7ffa00155500 2024-09-21T00:29:13.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:13.889+0000 7ffa04ac5640 1 -- 172.21.15.112:0/3793922566 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa00154d20 con 0x7ffa00155500 2024-09-21T00:29:13.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:13.890+0000 7ff9f77fe640 1 -- 172.21.15.112:0/3793922566 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff9e8011c00 con 0x7ffa00155500 2024-09-21T00:29:13.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:13.890+0000 7ff9f77fe640 1 --2- 172.21.15.112:0/3793922566 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff9e0078b90 0x7ff9e007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:13.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:13.891+0000 7ff9f77fe640 1 -- 172.21.15.112:0/3793922566 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5335+0+0 (secure 0 0 0) 0x7ff9e809b8d0 con 0x7ffa00155500 2024-09-21T00:29:13.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:13.891+0000 7ff9fe59c640 1 --2- 172.21.15.112:0/3793922566 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff9e0078b90 0x7ff9e007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:13.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:13.891+0000 7ff9fe59c640 1 --2- 172.21.15.112:0/3793922566 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff9e0078b90 0x7ff9e007b0a0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7ff9f0006b90 tx=0x7ff9f00067a0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:13.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:13.895+0000 7ff9f77fe640 1 -- 172.21.15.112:0/3793922566 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ff9e8020020 con 0x7ffa00155500 2024-09-21T00:29:14.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:14.072+0000 7ffa04ac5640 1 -- 172.21.15.112:0/3793922566 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7ffa000cd000 con 0x7ffa00155500 2024-09-21T00:29:14.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:14.072+0000 7ff9f77fe640 1 -- 172.21.15.112:0/3793922566 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v44) ==== 59+0+2294 (secure 0 0 0) 0x7ffa0015acf0 con 0x7ffa00155500 2024-09-21T00:29:14.074 INFO:teuthology.orchestra.run.smithi112.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-21T00:29:14.074 INFO:teuthology.orchestra.run.smithi112.stdout:mon advanced mon_op_complaint_time 120 2024-09-21T00:29:14.074 INFO:teuthology.orchestra.run.smithi112.stdout:mgr advanced client_mount_timeout 30 2024-09-21T00:29:14.074 INFO:teuthology.orchestra.run.smithi112.stdout:mgr advanced debug_client 20/20 2024-09-21T00:29:14.074 INFO:teuthology.orchestra.run.smithi112.stdout:mgr advanced debug_mgr 20/20 2024-09-21T00:29:14.074 INFO:teuthology.orchestra.run.smithi112.stdout:mgr advanced debug_ms 1/1 2024-09-21T00:29:14.075 INFO:teuthology.orchestra.run.smithi112.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-21T00:29:14.075 INFO:teuthology.orchestra.run.smithi112.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-21T00:29:14.075 INFO:teuthology.orchestra.run.smithi112.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 5090.401743 2024-09-21T00:29:14.075 INFO:teuthology.orchestra.run.smithi112.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 7876.743127 2024-09-21T00:29:14.075 INFO:teuthology.orchestra.run.smithi112.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 4865.073913 2024-09-21T00:29:14.075 INFO:teuthology.orchestra.run.smithi112.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 5117.206240 2024-09-21T00:29:14.075 INFO:teuthology.orchestra.run.smithi112.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 4329.674604 2024-09-21T00:29:14.075 INFO:teuthology.orchestra.run.smithi112.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4322.652151 2024-09-21T00:29:14.075 INFO:teuthology.orchestra.run.smithi112.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 4244.674337 2024-09-21T00:29:14.075 INFO:teuthology.orchestra.run.smithi112.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 4472.316624 2024-09-21T00:29:14.075 INFO:teuthology.orchestra.run.smithi112.stdout:mds advanced debug_mds 20/20 2024-09-21T00:29:14.075 INFO:teuthology.orchestra.run.smithi112.stdout:mds advanced debug_mds_balancer 20/20 2024-09-21T00:29:14.075 INFO:teuthology.orchestra.run.smithi112.stdout:mds advanced debug_ms 2/2 2024-09-21T00:29:14.076 INFO:teuthology.orchestra.run.smithi112.stdout:mds dev mds_debug_frag true 2024-09-21T00:29:14.076 INFO:teuthology.orchestra.run.smithi112.stdout:mds dev mds_debug_scatterstat true 2024-09-21T00:29:14.076 INFO:teuthology.orchestra.run.smithi112.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-21T00:29:14.076 INFO:teuthology.orchestra.run.smithi112.stdout:mds dev mds_verify_scatter true 2024-09-21T00:29:14.076 INFO:teuthology.orchestra.run.smithi112.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-21T00:29:14.076 INFO:teuthology.orchestra.run.smithi112.stdout:mds advanced rados_mon_op_timeout 900 2024-09-21T00:29:14.076 INFO:teuthology.orchestra.run.smithi112.stdout:mds advanced rados_osd_op_timeout 900 2024-09-21T00:29:14.076 INFO:teuthology.orchestra.run.smithi112.stdout:client advanced client_mount_timeout 600 2024-09-21T00:29:14.076 INFO:teuthology.orchestra.run.smithi112.stdout:client advanced debug_client 20/20 2024-09-21T00:29:14.076 INFO:teuthology.orchestra.run.smithi112.stdout:client advanced debug_ms 1/1 2024-09-21T00:29:14.076 INFO:teuthology.orchestra.run.smithi112.stdout:client advanced rados_mon_op_timeout 900 2024-09-21T00:29:14.076 INFO:teuthology.orchestra.run.smithi112.stdout:client advanced rados_osd_op_timeout 900 2024-09-21T00:29:14.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:14.074+0000 7ffa04ac5640 1 -- 172.21.15.112:0/3793922566 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff9e0078b90 msgr2=0x7ff9e007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:14.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:14.074+0000 7ffa04ac5640 1 --2- 172.21.15.112:0/3793922566 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff9e0078b90 0x7ff9e007b0a0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7ff9f0006b90 tx=0x7ff9f00067a0 comp rx=0 tx=0).stop 2024-09-21T00:29:14.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:14.074+0000 7ffa04ac5640 1 -- 172.21.15.112:0/3793922566 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa00155500 msgr2=0x7ffa0015a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:14.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:14.074+0000 7ffa04ac5640 1 --2- 172.21.15.112:0/3793922566 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa00155500 0x7ffa0015a3a0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7ff9e8003f60 tx=0x7ff9e8003f90 comp rx=0 tx=0).stop 2024-09-21T00:29:14.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:14.074+0000 7ffa04ac5640 1 -- 172.21.15.112:0/3793922566 shutdown_connections 2024-09-21T00:29:14.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:14.074+0000 7ffa04ac5640 1 --2- 172.21.15.112:0/3793922566 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffa0015a8e0 0x7ffa001f3470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:14.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:14.074+0000 7ffa04ac5640 1 --2- 172.21.15.112:0/3793922566 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff9e0078b90 0x7ff9e007b0a0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:14.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:14.074+0000 7ffa04ac5640 1 --2- 172.21.15.112:0/3793922566 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa00155500 0x7ffa0015a3a0 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:14.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:14.074+0000 7ffa04ac5640 1 --2- 172.21.15.112:0/3793922566 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff9ec004f40 0x7ffa0014f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:14.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:14.074+0000 7ffa04ac5640 1 -- 172.21.15.112:0/3793922566 >> 172.21.15.112:0/3793922566 conn(0x7ffa000b42e0 msgr2=0x7ffa0015b180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:14.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:14.074+0000 7ffa04ac5640 1 -- 172.21.15.112:0/3793922566 shutdown_connections 2024-09-21T00:29:14.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:14.074+0000 7ffa04ac5640 1 -- 172.21.15.112:0/3793922566 wait complete. 2024-09-21T00:29:14.087 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-21T00:29:14.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:14.202+0000 7f1c040d3640 1 -- 172.21.15.112:0/2510517571 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1be80099b0 msgr2=0x7f1be800bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:14.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:14.202+0000 7f1c040d3640 1 --2- 172.21.15.112:0/2510517571 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1be80099b0 0x7f1be800bdd0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f1bf4005fd0 tx=0x7f1be8005830 comp rx=0 tx=0).stop 2024-09-21T00:29:14.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:14.202+0000 7f1c040d3640 1 -- 172.21.15.112:0/2510517571 shutdown_connections 2024-09-21T00:29:14.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:14.202+0000 7f1c040d3640 1 --2- 172.21.15.112:0/2510517571 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1be80099b0 0x7f1be800bdd0 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:14.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:14.202+0000 7f1c040d3640 1 -- 172.21.15.112:0/2510517571 >> 172.21.15.112:0/2510517571 conn(0x7f1bfc0b42e0 msgr2=0x7f1bfc0b4720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:14.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:14.202+0000 7f1c040d3640 1 -- 172.21.15.112:0/2510517571 shutdown_connections 2024-09-21T00:29:14.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:14.202+0000 7f1c040d3640 1 -- 172.21.15.112:0/2510517571 wait complete. 2024-09-21T00:29:14.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:14.204+0000 7f1c040d3640 1 Processor -- start 2024-09-21T00:29:14.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:14.204+0000 7f1c040d3640 1 -- start start 2024-09-21T00:29:14.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:14.205+0000 7f1c040d3640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1be80099b0 0x7f1bfc1475c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:14.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:14.205+0000 7f1c040d3640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1bfc1f36a0 0x7f1bfc15a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:14.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:14.205+0000 7f1c040d3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1bfc154840 0x7f1bfc15a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:14.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:14.205+0000 7f1c040d3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1bfc0cd390 con 0x7f1bfc154840 2024-09-21T00:29:14.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:14.205+0000 7f1c040d3640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f1bfc0cd210 con 0x7f1bfc1f36a0 2024-09-21T00:29:14.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:14.205+0000 7f1c040d3640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f1bfc0cd510 con 0x7f1be80099b0 2024-09-21T00:29:14.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:14.205+0000 7f1c02670640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1bfc154840 0x7f1bfc15a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:14.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:14.205+0000 7f1c02670640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1bfc154840 0x7f1bfc15a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39494/0 (socket says 172.21.15.112:39494) 2024-09-21T00:29:14.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:14.205+0000 7f1c02670640 1 -- 172.21.15.112:0/3392543079 learned_addr learned my addr 172.21.15.112:0/3392543079 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:29:14.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:14.205+0000 7f1c02670640 1 -- 172.21.15.112:0/3392543079 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1be80099b0 msgr2=0x7f1bfc1475c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:14.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:14.205+0000 7f1c02670640 1 --2- 172.21.15.112:0/3392543079 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1be80099b0 0x7f1bfc1475c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:14.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:14.205+0000 7f1c02670640 1 -- 172.21.15.112:0/3392543079 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1bfc1f36a0 msgr2=0x7f1bfc15a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:14.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:14.205+0000 7f1c02670640 1 --2- 172.21.15.112:0/3392543079 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1bfc1f36a0 0x7f1bfc15a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:14.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:14.205+0000 7f1c02670640 1 -- 172.21.15.112:0/3392543079 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bfc15b090 con 0x7f1bfc154840 2024-09-21T00:29:14.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:14.205+0000 7f1c02670640 1 --2- 172.21.15.112:0/3392543079 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1bfc154840 0x7f1bfc15a950 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f1bf4009fd0 tx=0x7f1bf4033e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:14.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:14.206+0000 7f1bf27fc640 1 -- 172.21.15.112:0/3392543079 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bf4038040 con 0x7f1bfc154840 2024-09-21T00:29:14.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:14.206+0000 7f1bf27fc640 1 -- 172.21.15.112:0/3392543079 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1bf4041070 con 0x7f1bfc154840 2024-09-21T00:29:14.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:14.206+0000 7f1c040d3640 1 -- 172.21.15.112:0/3392543079 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bfc0c9570 con 0x7f1bfc154840 2024-09-21T00:29:14.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:14.206+0000 7f1bf27fc640 1 -- 172.21.15.112:0/3392543079 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bf4009200 con 0x7f1bfc154840 2024-09-21T00:29:14.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:14.206+0000 7f1c040d3640 1 -- 172.21.15.112:0/3392543079 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bfc0c9b30 con 0x7f1bfc154840 2024-09-21T00:29:14.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:14.207+0000 7f1c040d3640 1 -- 172.21.15.112:0/3392543079 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1bfc0cc7d0 con 0x7f1bfc154840 2024-09-21T00:29:14.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:14.207+0000 7f1bf27fc640 1 -- 172.21.15.112:0/3392543079 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1bf403c9a0 con 0x7f1bfc154840 2024-09-21T00:29:14.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:14.208+0000 7f1bf27fc640 1 --2- 172.21.15.112:0/3392543079 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1be0078bc0 0x7f1be007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:14.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:14.208+0000 7f1bf27fc640 1 -- 172.21.15.112:0/3392543079 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5335+0+0 (secure 0 0 0) 0x7f1bf40bbaa0 con 0x7f1bfc154840 2024-09-21T00:29:14.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:14.208+0000 7f1c01e6f640 1 --2- 172.21.15.112:0/3392543079 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1be0078bc0 0x7f1be007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:14.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:14.209+0000 7f1c01e6f640 1 --2- 172.21.15.112:0/3392543079 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1be0078bc0 0x7f1be007b0d0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f1bf8001710 tx=0x7f1bf8001040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:14.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:14.213+0000 7f1bf27fc640 1 -- 172.21.15.112:0/3392543079 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1bf40840e0 con 0x7f1bfc154840 2024-09-21T00:29:14.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:14.387+0000 7f1c040d3640 1 -- 172.21.15.112:0/3392543079 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f1bfc0c6180 con 0x7f1bfc154840 2024-09-21T00:29:14.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:14.391+0000 7f1bf27fc640 1 -- 172.21.15.112:0/3392543079 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v45)=0 v45) ==== 122+0+0 (secure 0 0 0) 0x7f1bf40839e0 con 0x7f1bfc154840 2024-09-21T00:29:14.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:14.393+0000 7f1c040d3640 1 -- 172.21.15.112:0/3392543079 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1be0078bc0 msgr2=0x7f1be007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:14.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:14.393+0000 7f1c040d3640 1 --2- 172.21.15.112:0/3392543079 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1be0078bc0 0x7f1be007b0d0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f1bf8001710 tx=0x7f1bf8001040 comp rx=0 tx=0).stop 2024-09-21T00:29:14.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:14.393+0000 7f1c040d3640 1 -- 172.21.15.112:0/3392543079 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1bfc154840 msgr2=0x7f1bfc15a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:14.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:14.393+0000 7f1c040d3640 1 --2- 172.21.15.112:0/3392543079 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1bfc154840 0x7f1bfc15a950 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f1bf4009fd0 tx=0x7f1bf4033e40 comp rx=0 tx=0).stop 2024-09-21T00:29:14.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:14.393+0000 7f1c040d3640 1 -- 172.21.15.112:0/3392543079 shutdown_connections 2024-09-21T00:29:14.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:14.393+0000 7f1c040d3640 1 --2- 172.21.15.112:0/3392543079 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1be0078bc0 0x7f1be007b0d0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:14.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:14.393+0000 7f1c040d3640 1 --2- 172.21.15.112:0/3392543079 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1bfc154840 0x7f1bfc15a950 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:14.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:14.393+0000 7f1c040d3640 1 --2- 172.21.15.112:0/3392543079 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1bfc1f36a0 0x7f1bfc15a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:14.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:14.393+0000 7f1c040d3640 1 --2- 172.21.15.112:0/3392543079 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1be80099b0 0x7f1bfc1475c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:14.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:14.393+0000 7f1c040d3640 1 -- 172.21.15.112:0/3392543079 >> 172.21.15.112:0/3392543079 conn(0x7f1bfc0b42e0 msgr2=0x7f1bfc0c6390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:14.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:14.393+0000 7f1c040d3640 1 -- 172.21.15.112:0/3392543079 shutdown_connections 2024-09-21T00:29:14.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:14.393+0000 7f1c040d3640 1 -- 172.21.15.112:0/3392543079 wait complete. 2024-09-21T00:29:14.406 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T00:29:14.406 DEBUG:teuthology.orchestra.run.smithi112:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T00:29:14.444 DEBUG:teuthology.orchestra.run.smithi187:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T00:29:14.502 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T00:29:14.502 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T00:29:14.502 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:29:14.518 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056010,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1439 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248716k,nr_inodes=812179,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T00:29:14.518 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-09-21T00:29:14.518 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:29:14.536 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056013,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497440k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16835 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248720k,nr_inodes=812180,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497440k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T00:29:14.536 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T00:29:14.536 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:29:14.573 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056010,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1439 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248716k,nr_inodes=812179,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T00:29:14.574 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-09-21T00:29:14.574 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:29:14.593 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056013,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497440k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16835 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248720k,nr_inodes=812180,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497440k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T00:29:14.593 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-21T00:29:14.593 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T00:29:14.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:14.724+0000 7ffbb712b640 1 -- 172.21.15.112:0/3901940991 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb9c004800 msgr2=0x7ffb9c004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:14.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:14.724+0000 7ffbb712b640 1 --2- 172.21.15.112:0/3901940991 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb9c004800 0x7ffb9c004c10 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7ffb9800bb20 tx=0x7ffb9c005830 comp rx=0 tx=0).stop 2024-09-21T00:29:14.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:14.724+0000 7ffbb56c8640 1 -- 172.21.15.112:0/3901940991 reap_dead start 2024-09-21T00:29:14.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:14.724+0000 7ffbb712b640 1 -- 172.21.15.112:0/3901940991 shutdown_connections 2024-09-21T00:29:14.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:14.724+0000 7ffbb712b640 1 -- 172.21.15.112:0/3901940991 >> 172.21.15.112:0/3901940991 conn(0x7ffbb00cf220 msgr2=0x7ffbb00cf630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:14.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:14.724+0000 7ffbb712b640 1 -- 172.21.15.112:0/3901940991 shutdown_connections 2024-09-21T00:29:14.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:14.724+0000 7ffbb712b640 1 -- 172.21.15.112:0/3901940991 wait complete. 2024-09-21T00:29:14.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:14.726+0000 7ffbb712b640 1 Processor -- start 2024-09-21T00:29:14.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:14.726+0000 7ffbb712b640 1 -- start start 2024-09-21T00:29:14.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:14.727+0000 7ffbb712b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffb9c004800 0x7ffbb015f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:14.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:14.727+0000 7ffbb712b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffbb015d8c0 0x7ffbb01f3040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:14.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:14.727+0000 7ffbb712b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffbb015e590 0x7ffbb015e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:14.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:14.727+0000 7ffbb712b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ffbb00d2130 con 0x7ffbb015e590 2024-09-21T00:29:14.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:14.727+0000 7ffbb712b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ffbb00d1fb0 con 0x7ffbb015d8c0 2024-09-21T00:29:14.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:14.727+0000 7ffbb712b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ffbb00b48f0 con 0x7ffb9c004800 2024-09-21T00:29:14.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:14.727+0000 7ffbb56c8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffbb015e590 0x7ffbb015e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:14.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:14.727+0000 7ffbb56c8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffbb015e590 0x7ffbb015e9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39516/0 (socket says 172.21.15.112:39516) 2024-09-21T00:29:14.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:14.727+0000 7ffbb56c8640 1 -- 172.21.15.112:0/861838869 learned_addr learned my addr 172.21.15.112:0/861838869 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:29:14.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:14.727+0000 7ffbb4ec7640 1 --2- 172.21.15.112:0/861838869 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffb9c004800 0x7ffbb015f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:14.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:14.727+0000 7ffbaffff640 1 --2- 172.21.15.112:0/861838869 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffbb015d8c0 0x7ffbb01f3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:14.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:14.728+0000 7ffbb56c8640 1 -- 172.21.15.112:0/861838869 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffb9c004800 msgr2=0x7ffbb015f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:14.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:14.728+0000 7ffbb56c8640 1 --2- 172.21.15.112:0/861838869 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffb9c004800 0x7ffbb015f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:14.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:14.728+0000 7ffbb56c8640 1 -- 172.21.15.112:0/861838869 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffbb015d8c0 msgr2=0x7ffbb01f3040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:14.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:14.728+0000 7ffbb56c8640 1 --2- 172.21.15.112:0/861838869 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffbb015d8c0 0x7ffbb01f3040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:14.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:14.728+0000 7ffbb56c8640 1 -- 172.21.15.112:0/861838869 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbb01f3b70 con 0x7ffbb015e590 2024-09-21T00:29:14.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:14.728+0000 7ffbb56c8640 1 --2- 172.21.15.112:0/861838869 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffbb015e590 0x7ffbb015e9a0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7ffbb015ce60 tx=0x7ffba800f720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:14.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:14.728+0000 7ffbad7fa640 1 -- 172.21.15.112:0/861838869 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffba8011630 con 0x7ffbb015e590 2024-09-21T00:29:14.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:14.728+0000 7ffbad7fa640 1 -- 172.21.15.112:0/861838869 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffba8018070 con 0x7ffbb015e590 2024-09-21T00:29:14.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:14.728+0000 7ffbad7fa640 1 -- 172.21.15.112:0/861838869 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffba8011a70 con 0x7ffbb015e590 2024-09-21T00:29:14.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:14.728+0000 7ffbb712b640 1 -- 172.21.15.112:0/861838869 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbb01f3e50 con 0x7ffbb015e590 2024-09-21T00:29:14.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:14.728+0000 7ffbb712b640 1 -- 172.21.15.112:0/861838869 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbb015d3a0 con 0x7ffbb015e590 2024-09-21T00:29:14.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:14.729+0000 7ffbb712b640 1 -- 172.21.15.112:0/861838869 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffbb01f38e0 con 0x7ffbb015e590 2024-09-21T00:29:14.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:14.730+0000 7ffbad7fa640 1 -- 172.21.15.112:0/861838869 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ffba8011c10 con 0x7ffbb015e590 2024-09-21T00:29:14.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:14.730+0000 7ffbad7fa640 1 --2- 172.21.15.112:0/861838869 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ffb90078b90 0x7ffb9007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:14.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:14.730+0000 7ffbad7fa640 1 -- 172.21.15.112:0/861838869 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5335+0+0 (secure 0 0 0) 0x7ffba809ac20 con 0x7ffbb015e590 2024-09-21T00:29:14.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:14.730+0000 7ffbb4ec7640 1 --2- 172.21.15.112:0/861838869 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ffb90078b90 0x7ffb9007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:14.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:14.731+0000 7ffbb4ec7640 1 --2- 172.21.15.112:0/861838869 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ffb90078b90 0x7ffb9007b0a0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7ffba0006fd0 tx=0x7ffba0006b20 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:14.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:14.735+0000 7ffbad7fa640 1 -- 172.21.15.112:0/861838869 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ffba8062c10 con 0x7ffbb015e590 2024-09-21T00:29:14.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:14.949+0000 7ffbb712b640 1 -- 172.21.15.112:0/861838869 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffbb00ccb00 con 0x7ffbb015e590 2024-09-21T00:29:14.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:14.950+0000 7ffbad7fa640 1 -- 172.21.15.112:0/861838869 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) ==== 94+0+3917 (secure 0 0 0) 0x7ffbb00ccb00 con 0x7ffbb015e590 2024-09-21T00:29:14.951 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:29:14.951 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":57,"btime":"2024-09-21T00:29:08:726675+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5600,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6836/2708152171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":2708152171},{"type":"v1","addr":"172.21.15.187:6836","nonce":2708152171}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50},{"gid":5750,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6837/553746937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6835","nonce":553746937},{"type":"v1","addr":"172.21.15.187:6837","nonce":553746937}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":6471,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6834/3529228075","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":3529228075},{"type":"v1","addr":"172.21.15.112:6834","nonce":3529228075}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50},{"gid":6474,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/101289292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":101289292},{"type":"v1","addr":"172.21.15.112:6835","nonce":101289292}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50}],"filesystems":[]} 2024-09-21T00:29:14.952 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 57 2024-09-21T00:29:14.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:14.952+0000 7ffbb712b640 1 -- 172.21.15.112:0/861838869 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ffb90078b90 msgr2=0x7ffb9007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:14.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:14.952+0000 7ffbb712b640 1 --2- 172.21.15.112:0/861838869 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ffb90078b90 0x7ffb9007b0a0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7ffba0006fd0 tx=0x7ffba0006b20 comp rx=0 tx=0).stop 2024-09-21T00:29:14.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:14.952+0000 7ffbb712b640 1 -- 172.21.15.112:0/861838869 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffbb015e590 msgr2=0x7ffbb015e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:14.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:14.952+0000 7ffbb712b640 1 --2- 172.21.15.112:0/861838869 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffbb015e590 0x7ffbb015e9a0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7ffbb015ce60 tx=0x7ffba800f720 comp rx=0 tx=0).stop 2024-09-21T00:29:14.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:14.952+0000 7ffbb712b640 1 -- 172.21.15.112:0/861838869 shutdown_connections 2024-09-21T00:29:14.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:14.952+0000 7ffbb712b640 1 --2- 172.21.15.112:0/861838869 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ffb90078b90 0x7ffb9007b0a0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:14.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:14.952+0000 7ffbb712b640 1 --2- 172.21.15.112:0/861838869 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffbb015e590 0x7ffbb015e9a0 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:14.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:14.952+0000 7ffbb712b640 1 --2- 172.21.15.112:0/861838869 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffbb015d8c0 0x7ffbb01f3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:14.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:14.952+0000 7ffbb712b640 1 --2- 172.21.15.112:0/861838869 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffb9c004800 0x7ffbb015f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:14.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:14.952+0000 7ffbb712b640 1 -- 172.21.15.112:0/861838869 >> 172.21.15.112:0/861838869 conn(0x7ffbb00cf220 msgr2=0x7ffbb015b0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:14.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:14.952+0000 7ffbb712b640 1 -- 172.21.15.112:0/861838869 shutdown_connections 2024-09-21T00:29:14.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:14.952+0000 7ffbb712b640 1 -- 172.21.15.112:0/861838869 wait complete. 2024-09-21T00:29:14.965 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-21T00:29:14.965 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-21T00:29:14.966 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-21T00:29:14.966 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T00:29:14.966 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-21T00:29:14.966 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-21T00:29:14.966 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-21T00:29:14.966 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T00:29:14.966 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-21T00:29:14.967 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-21T00:29:14.967 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-21T00:29:14.967 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T00:29:14.967 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-21T00:29:14.967 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-21T00:29:14.967 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-21T00:29:14.967 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T00:29:14.975 INFO:tasks.ceph.mds.a:Stopped 2024-09-21T00:29:14.975 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-21T00:29:14.976 INFO:tasks.ceph.mds.c:Stopped 2024-09-21T00:29:14.976 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-21T00:29:14.980 INFO:tasks.ceph.mds.b:Stopped 2024-09-21T00:29:14.980 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-21T00:29:14.982 INFO:tasks.ceph.mds.d:Stopped 2024-09-21T00:29:14.983 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-21T00:29:15.018 INFO:tasks.ceph.mds.a:Started 2024-09-21T00:29:15.018 DEBUG:teuthology.parallel:result is None 2024-09-21T00:29:15.018 INFO:tasks.ceph.mds.c:Started 2024-09-21T00:29:15.019 DEBUG:teuthology.parallel:result is None 2024-09-21T00:29:15.024 INFO:tasks.ceph.mds.b:Started 2024-09-21T00:29:15.024 DEBUG:teuthology.parallel:result is None 2024-09-21T00:29:15.025 INFO:tasks.ceph.mds.d:Started 2024-09-21T00:29:15.025 DEBUG:teuthology.parallel:result is None 2024-09-21T00:29:15.026 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-21T00:29:15.118 INFO:tasks.ceph.mds.a.smithi112.stdout:starting mds.a at 2024-09-21T00:29:15.119 INFO:tasks.ceph.mds.c.smithi112.stdout:starting mds.c at 2024-09-21T00:29:15.120 INFO:tasks.ceph.mds.d.smithi187.stdout:starting mds.d at 2024-09-21T00:29:15.121 INFO:tasks.ceph.mds.b.smithi187.stdout:starting mds.b at 2024-09-21T00:29:15.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:15.142+0000 7f588400e640 1 -- 172.21.15.112:0/20798572 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f587c0b3bb0 msgr2=0x7f587c0b6740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:15.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:15.142+0000 7f588400e640 1 --2- 172.21.15.112:0/20798572 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f587c0b3bb0 0x7f587c0b6740 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f5874005fd0 tx=0x7f587402a7a0 comp rx=0 tx=0).stop 2024-09-21T00:29:15.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:15.142+0000 7f588400e640 1 -- 172.21.15.112:0/20798572 shutdown_connections 2024-09-21T00:29:15.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:15.142+0000 7f588400e640 1 --2- 172.21.15.112:0/20798572 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f587c0b3bb0 0x7f587c0b6740 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:15.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:15.142+0000 7f588400e640 1 --2- 172.21.15.112:0/20798572 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f587c15c3c0 0x7f587c0b3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:15.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:15.142+0000 7f588400e640 1 -- 172.21.15.112:0/20798572 >> 172.21.15.112:0/20798572 conn(0x7f587c08dcb0 msgr2=0x7f587c09d8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:15.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:15.142+0000 7f588400e640 1 -- 172.21.15.112:0/20798572 shutdown_connections 2024-09-21T00:29:15.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:15.142+0000 7f588400e640 1 -- 172.21.15.112:0/20798572 wait complete. 2024-09-21T00:29:15.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:15.144+0000 7f588400e640 1 Processor -- start 2024-09-21T00:29:15.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:15.144+0000 7f588400e640 1 -- start start 2024-09-21T00:29:15.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:15.146+0000 7f588400e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f587c0b37a0 0x7f587c1518e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:15.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:15.146+0000 7f588400e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f587c0b3bb0 0x7f587c06e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:15.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:15.146+0000 7f588400e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f587c15c3c0 0x7f587c159fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:15.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:15.146+0000 7f588400e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f587c163180 con 0x7f587c15c3c0 2024-09-21T00:29:15.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:15.146+0000 7f588400e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f587c0d0d30 con 0x7f587c0b37a0 2024-09-21T00:29:15.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:15.146+0000 7f588400e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f587c163300 con 0x7f587c0b3bb0 2024-09-21T00:29:15.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:15.146+0000 7f58825ab640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f587c15c3c0 0x7f587c159fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:15.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:15.146+0000 7f5881daa640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f587c0b37a0 0x7f587c1518e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:15.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:15.146+0000 7f58825ab640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f587c15c3c0 0x7f587c159fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39548/0 (socket says 172.21.15.112:39548) 2024-09-21T00:29:15.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:15.146+0000 7f58825ab640 1 -- 172.21.15.112:0/3047033886 learned_addr learned my addr 172.21.15.112:0/3047033886 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:29:15.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:15.146+0000 7f58815a9640 1 --2- 172.21.15.112:0/3047033886 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f587c0b3bb0 0x7f587c06e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:15.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:15.146+0000 7f58825ab640 1 -- 172.21.15.112:0/3047033886 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f587c0b3bb0 msgr2=0x7f587c06e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:15.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:15.146+0000 7f58825ab640 1 --2- 172.21.15.112:0/3047033886 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f587c0b3bb0 0x7f587c06e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:15.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:15.146+0000 7f58825ab640 1 -- 172.21.15.112:0/3047033886 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f587c0b37a0 msgr2=0x7f587c1518e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:15.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:15.146+0000 7f58825ab640 1 --2- 172.21.15.112:0/3047033886 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f587c0b37a0 0x7f587c1518e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:15.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:15.146+0000 7f58825ab640 1 -- 172.21.15.112:0/3047033886 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f587c06cde0 con 0x7f587c15c3c0 2024-09-21T00:29:15.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:15.146+0000 7f5881daa640 1 --2- 172.21.15.112:0/3047033886 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f587c0b37a0 0x7f587c1518e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:29:15.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:15.147+0000 7f58825ab640 1 --2- 172.21.15.112:0/3047033886 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f587c15c3c0 0x7f587c159fb0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f5874006010 tx=0x7f5874032c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:15.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:15.147+0000 7f586e7fc640 1 -- 172.21.15.112:0/3047033886 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5874041020 con 0x7f587c15c3c0 2024-09-21T00:29:15.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:15.147+0000 7f586e7fc640 1 -- 172.21.15.112:0/3047033886 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5874032e50 con 0x7f587c15c3c0 2024-09-21T00:29:15.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:15.147+0000 7f588400e640 1 -- 172.21.15.112:0/3047033886 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f587c06d0c0 con 0x7f587c15c3c0 2024-09-21T00:29:15.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:15.147+0000 7f586e7fc640 1 -- 172.21.15.112:0/3047033886 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5874036040 con 0x7f587c15c3c0 2024-09-21T00:29:15.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:15.147+0000 7f588400e640 1 -- 172.21.15.112:0/3047033886 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f587c06d680 con 0x7f587c15c3c0 2024-09-21T00:29:15.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:15.148+0000 7f588400e640 1 -- 172.21.15.112:0/3047033886 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f587c0b3260 con 0x7f587c15c3c0 2024-09-21T00:29:15.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:15.149+0000 7f586e7fc640 1 -- 172.21.15.112:0/3047033886 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5874044020 con 0x7f587c15c3c0 2024-09-21T00:29:15.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:15.149+0000 7f586e7fc640 1 --2- 172.21.15.112:0/3047033886 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f5860078bc0 0x7f586007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:15.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:15.149+0000 7f586e7fc640 1 -- 172.21.15.112:0/3047033886 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5335+0+0 (secure 0 0 0) 0x7f58740bba70 con 0x7f587c15c3c0 2024-09-21T00:29:15.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:15.150+0000 7f5881daa640 1 --2- 172.21.15.112:0/3047033886 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f5860078bc0 0x7f586007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:15.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:15.150+0000 7f5881daa640 1 --2- 172.21.15.112:0/3047033886 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f5860078bc0 0x7f586007b0d0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f5870011440 tx=0x7f5870005f30 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:15.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:15.154+0000 7f586e7fc640 1 -- 172.21.15.112:0/3047033886 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f587404b030 con 0x7f587c15c3c0 2024-09-21T00:29:15.343 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:15.341+0000 7f588400e640 1 -- 172.21.15.112:0/3047033886 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f587c0b22b0 con 0x7f587c15c3c0 2024-09-21T00:29:15.360 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T00:29:15.363 DEBUG:teuthology.orchestra.run.smithi187:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T00:29:15.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:15.762+0000 7f586e7fc640 1 -- 172.21.15.112:0/3047033886 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v88) ==== 97+0+0 (secure 0 0 0) 0x7f58740839d0 con 0x7f587c15c3c0 2024-09-21T00:29:15.764 INFO:teuthology.orchestra.run.smithi112.stderr: removed all blocklist entries 2024-09-21T00:29:15.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:15.766+0000 7f588400e640 1 -- 172.21.15.112:0/3047033886 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f5860078bc0 msgr2=0x7f586007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:15.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:15.766+0000 7f588400e640 1 --2- 172.21.15.112:0/3047033886 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f5860078bc0 0x7f586007b0d0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f5870011440 tx=0x7f5870005f30 comp rx=0 tx=0).stop 2024-09-21T00:29:15.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:15.766+0000 7f588400e640 1 -- 172.21.15.112:0/3047033886 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f587c15c3c0 msgr2=0x7f587c159fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:15.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:15.766+0000 7f588400e640 1 --2- 172.21.15.112:0/3047033886 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f587c15c3c0 0x7f587c159fb0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f5874006010 tx=0x7f5874032c60 comp rx=0 tx=0).stop 2024-09-21T00:29:15.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:15.766+0000 7f588400e640 1 -- 172.21.15.112:0/3047033886 shutdown_connections 2024-09-21T00:29:15.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:15.766+0000 7f588400e640 1 --2- 172.21.15.112:0/3047033886 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f5860078bc0 0x7f586007b0d0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:15.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:15.766+0000 7f588400e640 1 --2- 172.21.15.112:0/3047033886 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f587c15c3c0 0x7f587c159fb0 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:15.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:15.766+0000 7f588400e640 1 --2- 172.21.15.112:0/3047033886 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f587c0b3bb0 0x7f587c06e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:15.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:15.766+0000 7f588400e640 1 --2- 172.21.15.112:0/3047033886 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f587c0b37a0 0x7f587c1518e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:15.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:15.766+0000 7f588400e640 1 -- 172.21.15.112:0/3047033886 >> 172.21.15.112:0/3047033886 conn(0x7f587c08dcb0 msgr2=0x7f587c160000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:15.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:15.766+0000 7f588400e640 1 -- 172.21.15.112:0/3047033886 shutdown_connections 2024-09-21T00:29:15.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:15.766+0000 7f588400e640 1 -- 172.21.15.112:0/3047033886 wait complete. 2024-09-21T00:29:15.782 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-21T00:29:15.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:15.893+0000 7f4b67e7e640 1 -- 172.21.15.112:0/139597448 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b4c003b30 msgr2=0x7f4b4c003fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:15.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:15.893+0000 7f4b67e7e640 1 --2- 172.21.15.112:0/139597448 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b4c003b30 0x7f4b4c003fa0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c009ed0 tx=0x7f4b4c005830 comp rx=0 tx=0).stop 2024-09-21T00:29:15.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:15.893+0000 7f4b67e7e640 1 -- 172.21.15.112:0/139597448 shutdown_connections 2024-09-21T00:29:15.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:15.893+0000 7f4b67e7e640 1 --2- 172.21.15.112:0/139597448 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b4c003b30 0x7f4b4c003fa0 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:15.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:15.893+0000 7f4b67e7e640 1 -- 172.21.15.112:0/139597448 >> 172.21.15.112:0/139597448 conn(0x7f4b60073fe0 msgr2=0x7f4b600743f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:15.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:15.894+0000 7f4b67e7e640 1 -- 172.21.15.112:0/139597448 shutdown_connections 2024-09-21T00:29:15.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:15.894+0000 7f4b67e7e640 1 -- 172.21.15.112:0/139597448 wait complete. 2024-09-21T00:29:15.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:15.895+0000 7f4b67e7e640 1 Processor -- start 2024-09-21T00:29:15.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:15.895+0000 7f4b67e7e640 1 -- start start 2024-09-21T00:29:15.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:15.897+0000 7f4b67e7e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4b4c003b30 0x7f4b6006d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:15.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:15.897+0000 7f4b67e7e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4b601f2d00 0x7f4b6006dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:15.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:15.897+0000 7f4b67e7e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b600ac2b0 0x7f4b600ac6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:15.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:15.897+0000 7f4b67e7e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4b600b0b70 con 0x7f4b600ac2b0 2024-09-21T00:29:15.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:15.897+0000 7f4b67e7e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f4b600b09f0 con 0x7f4b601f2d00 2024-09-21T00:29:15.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:15.897+0000 7f4b67e7e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f4b6015f0e0 con 0x7f4b4c003b30 2024-09-21T00:29:15.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:15.897+0000 7f4b6641b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b600ac2b0 0x7f4b600ac6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:15.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:15.897+0000 7f4b6641b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b600ac2b0 0x7f4b600ac6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39572/0 (socket says 172.21.15.112:39572) 2024-09-21T00:29:15.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:15.897+0000 7f4b6641b640 1 -- 172.21.15.112:0/1518393472 learned_addr learned my addr 172.21.15.112:0/1518393472 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:29:15.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:15.897+0000 7f4b65c1a640 1 --2- 172.21.15.112:0/1518393472 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4b4c003b30 0x7f4b6006d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:15.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:15.897+0000 7f4b65419640 1 --2- 172.21.15.112:0/1518393472 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4b601f2d00 0x7f4b6006dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:15.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:15.897+0000 7f4b6641b640 1 -- 172.21.15.112:0/1518393472 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4b4c003b30 msgr2=0x7f4b6006d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:15.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:15.897+0000 7f4b6641b640 1 --2- 172.21.15.112:0/1518393472 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4b4c003b30 0x7f4b6006d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:15.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:15.897+0000 7f4b6641b640 1 -- 172.21.15.112:0/1518393472 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4b601f2d00 msgr2=0x7f4b6006dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:15.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:15.897+0000 7f4b6641b640 1 --2- 172.21.15.112:0/1518393472 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4b601f2d00 0x7f4b6006dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:15.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:15.897+0000 7f4b6641b640 1 -- 172.21.15.112:0/1518393472 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b600aff70 con 0x7f4b600ac2b0 2024-09-21T00:29:15.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:15.897+0000 7f4b65419640 1 --2- 172.21.15.112:0/1518393472 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4b601f2d00 0x7f4b6006dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:29:15.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:15.897+0000 7f4b6641b640 1 --2- 172.21.15.112:0/1518393472 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b600ac2b0 0x7f4b600ac6c0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f4b600b6370 tx=0x7f4b58001260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:15.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:15.897+0000 7f4b567fc640 1 -- 172.21.15.112:0/1518393472 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b58011460 con 0x7f4b600ac2b0 2024-09-21T00:29:15.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:15.897+0000 7f4b567fc640 1 -- 172.21.15.112:0/1518393472 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4b58017070 con 0x7f4b600ac2b0 2024-09-21T00:29:15.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:15.897+0000 7f4b67e7e640 1 -- 172.21.15.112:0/1518393472 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b601f3d20 con 0x7f4b600ac2b0 2024-09-21T00:29:15.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:15.897+0000 7f4b567fc640 1 -- 172.21.15.112:0/1518393472 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b580118c0 con 0x7f4b600ac2b0 2024-09-21T00:29:15.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:15.897+0000 7f4b67e7e640 1 -- 172.21.15.112:0/1518393472 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b601f3760 con 0x7f4b600ac2b0 2024-09-21T00:29:15.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:15.898+0000 7f4b67e7e640 1 -- 172.21.15.112:0/1518393472 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b601f31f0 con 0x7f4b600ac2b0 2024-09-21T00:29:15.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:15.899+0000 7f4b567fc640 1 -- 172.21.15.112:0/1518393472 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f4b58011a60 con 0x7f4b600ac2b0 2024-09-21T00:29:15.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:15.900+0000 7f4b567fc640 1 --2- 172.21.15.112:0/1518393472 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4b44078b90 0x7f4b4407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:15.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:15.900+0000 7f4b567fc640 1 -- 172.21.15.112:0/1518393472 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5249+0+0 (secure 0 0 0) 0x7f4b5809ae90 con 0x7f4b600ac2b0 2024-09-21T00:29:15.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:15.900+0000 7f4b65c1a640 1 --2- 172.21.15.112:0/1518393472 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4b44078b90 0x7f4b4407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:15.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:15.900+0000 7f4b65c1a640 1 --2- 172.21.15.112:0/1518393472 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4b44078b90 0x7f4b4407b0a0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c035000 tx=0x7f4b5c004a40 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:15.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:15.904+0000 7f4b567fc640 1 -- 172.21.15.112:0/1518393472 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f4b580635b0 con 0x7f4b600ac2b0 2024-09-21T00:29:16.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:16.148+0000 7f4b67e7e640 1 -- 172.21.15.112:0/1518393472 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f4b6015f640 con 0x7f4b600ac2b0 2024-09-21T00:29:16.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:16.148+0000 7f4b567fc640 1 -- 172.21.15.112:0/1518393472 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v19) ==== 80+0+5304 (secure 0 0 0) 0x7f4b58062eb0 con 0x7f4b600ac2b0 2024-09-21T00:29:16.150 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:29:16.150 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-21T00:29:16.150 INFO:teuthology.orchestra.run.smithi112.stdout: "auth_dump": [ 2024-09-21T00:29:16.150 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:29:16.151 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "mds.a", 2024-09-21T00:29:16.151 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBPEu5m3capAhAAczRM4L8ldX985NW4ep3w5Q==", 2024-09-21T00:29:16.151 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:29:16.151 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2024-09-21T00:29:16.151 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow *", 2024-09-21T00:29:16.151 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow *", 2024-09-21T00:29:16.151 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2024-09-21T00:29:16.151 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:29:16.151 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:29:16.151 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:29:16.151 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "mds.b", 2024-09-21T00:29:16.152 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBPEu5mxlDlDxAAtqngbTOhPqXDjFhJpSCLHg==", 2024-09-21T00:29:16.152 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:29:16.152 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2024-09-21T00:29:16.152 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow *", 2024-09-21T00:29:16.152 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow *", 2024-09-21T00:29:16.152 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2024-09-21T00:29:16.152 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:29:16.152 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:29:16.152 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:29:16.152 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "mds.c", 2024-09-21T00:29:16.152 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBPEu5mXNVRChAA1qsPbEybmY16jw5Jwk/v1A==", 2024-09-21T00:29:16.152 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:29:16.153 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2024-09-21T00:29:16.153 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow *", 2024-09-21T00:29:16.153 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow *", 2024-09-21T00:29:16.153 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2024-09-21T00:29:16.153 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:29:16.153 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:29:16.153 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:29:16.153 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "mds.d", 2024-09-21T00:29:16.153 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBPEu5mnzySFxAAX3VfTgGYGcDweI7rPf3xvw==", 2024-09-21T00:29:16.153 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:29:16.153 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2024-09-21T00:29:16.153 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow *", 2024-09-21T00:29:16.153 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow *", 2024-09-21T00:29:16.154 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2024-09-21T00:29:16.154 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:29:16.154 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:29:16.154 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:29:16.154 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.0", 2024-09-21T00:29:16.154 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBSEu5moQg4OBAAi8mzLx7tWnORlrL64/MWlw==", 2024-09-21T00:29:16.154 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:29:16.154 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2024-09-21T00:29:16.154 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2024-09-21T00:29:16.154 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2024-09-21T00:29:16.154 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:29:16.154 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:29:16.155 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:29:16.155 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.1", 2024-09-21T00:29:16.155 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBWEu5mxfobDhAAi7I6LYq9udB0W8bnYaBY9Q==", 2024-09-21T00:29:16.155 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:29:16.155 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2024-09-21T00:29:16.155 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2024-09-21T00:29:16.155 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2024-09-21T00:29:16.155 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:29:16.155 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:29:16.155 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:29:16.155 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.2", 2024-09-21T00:29:16.155 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBZEu5mDqC/EhAAwzTbj22ocR4xJ8DMJvN7Lg==", 2024-09-21T00:29:16.156 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:29:16.156 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2024-09-21T00:29:16.156 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2024-09-21T00:29:16.156 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2024-09-21T00:29:16.156 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:29:16.156 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:29:16.156 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:29:16.156 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.3", 2024-09-21T00:29:16.156 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBcEu5mwweGJBAAT3hZn5pto0aWt6BwGuaqcw==", 2024-09-21T00:29:16.156 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:29:16.156 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2024-09-21T00:29:16.156 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2024-09-21T00:29:16.156 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2024-09-21T00:29:16.157 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:29:16.157 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:29:16.157 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:29:16.157 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.4", 2024-09-21T00:29:16.157 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBjEu5mIPd3ARAAkxTYphn38uKh48UUVEB+Fg==", 2024-09-21T00:29:16.157 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:29:16.157 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2024-09-21T00:29:16.157 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2024-09-21T00:29:16.157 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2024-09-21T00:29:16.157 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:29:16.157 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:29:16.157 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:29:16.158 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.5", 2024-09-21T00:29:16.158 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBmEu5mP9IVBBAAftfK5V1K4vYRyFyJS2UzKQ==", 2024-09-21T00:29:16.158 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:29:16.158 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2024-09-21T00:29:16.158 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2024-09-21T00:29:16.158 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2024-09-21T00:29:16.158 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:29:16.158 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:29:16.158 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:29:16.158 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.6", 2024-09-21T00:29:16.158 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBpEu5mmjiyFRAAbiW2M8JE0jJOaNg1u+1kgw==", 2024-09-21T00:29:16.158 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:29:16.159 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2024-09-21T00:29:16.159 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2024-09-21T00:29:16.159 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2024-09-21T00:29:16.159 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:29:16.159 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:29:16.159 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:29:16.159 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.7", 2024-09-21T00:29:16.159 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBsEu5moklPKBAA6hzVvgvpmZMkPaFFce4mqw==", 2024-09-21T00:29:16.159 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:29:16.159 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2024-09-21T00:29:16.159 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2024-09-21T00:29:16.159 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2024-09-21T00:29:16.159 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:29:16.160 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:29:16.160 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:29:16.160 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "client.0", 2024-09-21T00:29:16.160 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBPEu5m49NPHBAAl/rcjiE8xYI+XdWwvZ6FOw==", 2024-09-21T00:29:16.160 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:29:16.160 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2024-09-21T00:29:16.160 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow r", 2024-09-21T00:29:16.160 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-21T00:29:16.160 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:29:16.160 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:29:16.160 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:29:16.160 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "client.1", 2024-09-21T00:29:16.161 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBPEu5mG33aIxAAirQhTlVyF/pRJx93T/0bIQ==", 2024-09-21T00:29:16.161 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:29:16.161 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2024-09-21T00:29:16.161 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow r", 2024-09-21T00:29:16.161 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow rw", 2024-09-21T00:29:16.161 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow rwx" 2024-09-21T00:29:16.161 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:29:16.161 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:29:16.161 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:29:16.161 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "client.2", 2024-09-21T00:29:16.161 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBPEu5mreUUIBAAELeN1N58srw6GqBrj8UOoQ==", 2024-09-21T00:29:16.161 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:29:16.162 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2024-09-21T00:29:16.162 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow r", 2024-09-21T00:29:16.162 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow rw", 2024-09-21T00:29:16.162 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow rwx" 2024-09-21T00:29:16.162 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:29:16.162 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:29:16.162 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:29:16.162 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "client.3", 2024-09-21T00:29:16.162 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBPEu5mcTmiJxAAY6bmTtow0R0heJHjx0ahDQ==", 2024-09-21T00:29:16.162 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:29:16.162 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2024-09-21T00:29:16.162 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow r", 2024-09-21T00:29:16.163 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow rw", 2024-09-21T00:29:16.163 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow rwx" 2024-09-21T00:29:16.163 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:29:16.163 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:29:16.163 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:29:16.163 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "client.admin", 2024-09-21T00:29:16.163 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBOEu5mTxhNIBAAlStXGsGd/okXhakY6WI/4w==", 2024-09-21T00:29:16.163 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:29:16.163 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow *", 2024-09-21T00:29:16.163 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow *", 2024-09-21T00:29:16.163 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow *", 2024-09-21T00:29:16.163 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2024-09-21T00:29:16.164 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:29:16.164 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:29:16.164 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:29:16.164 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "mgr.x", 2024-09-21T00:29:16.164 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBOEu5mjGpDOhAAJFudb2wZlOBjo8Ss65Q+eg==", 2024-09-21T00:29:16.164 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:29:16.164 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow *", 2024-09-21T00:29:16.164 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile mgr", 2024-09-21T00:29:16.164 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2024-09-21T00:29:16.164 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:29:16.164 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:29:16.164 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:29:16.165 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "mgr.y", 2024-09-21T00:29:16.165 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBOEu5myGjKNRAA9cMDObZYYOXMGdv2mOXkJA==", 2024-09-21T00:29:16.165 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:29:16.165 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow *", 2024-09-21T00:29:16.165 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile mgr", 2024-09-21T00:29:16.165 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2024-09-21T00:29:16.165 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:29:16.165 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:29:16.165 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-09-21T00:29:16.165 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-21T00:29:16.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:16.150+0000 7f4b67e7e640 1 -- 172.21.15.112:0/1518393472 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4b44078b90 msgr2=0x7f4b4407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:16.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:16.150+0000 7f4b67e7e640 1 --2- 172.21.15.112:0/1518393472 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4b44078b90 0x7f4b4407b0a0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c035000 tx=0x7f4b5c004a40 comp rx=0 tx=0).stop 2024-09-21T00:29:16.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:16.150+0000 7f4b67e7e640 1 -- 172.21.15.112:0/1518393472 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b600ac2b0 msgr2=0x7f4b600ac6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:16.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:16.150+0000 7f4b67e7e640 1 --2- 172.21.15.112:0/1518393472 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b600ac2b0 0x7f4b600ac6c0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f4b600b6370 tx=0x7f4b58001260 comp rx=0 tx=0).stop 2024-09-21T00:29:16.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:16.150+0000 7f4b67e7e640 1 -- 172.21.15.112:0/1518393472 shutdown_connections 2024-09-21T00:29:16.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:16.150+0000 7f4b67e7e640 1 --2- 172.21.15.112:0/1518393472 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4b44078b90 0x7f4b4407b0a0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:16.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:16.150+0000 7f4b67e7e640 1 --2- 172.21.15.112:0/1518393472 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b600ac2b0 0x7f4b600ac6c0 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:16.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:16.150+0000 7f4b67e7e640 1 --2- 172.21.15.112:0/1518393472 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4b601f2d00 0x7f4b6006dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:16.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:16.150+0000 7f4b67e7e640 1 --2- 172.21.15.112:0/1518393472 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4b4c003b30 0x7f4b6006d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:16.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:16.150+0000 7f4b67e7e640 1 -- 172.21.15.112:0/1518393472 >> 172.21.15.112:0/1518393472 conn(0x7f4b60073fe0 msgr2=0x7f4b600b1170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:16.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:16.150+0000 7f4b67e7e640 1 -- 172.21.15.112:0/1518393472 shutdown_connections 2024-09-21T00:29:16.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:16.150+0000 7f4b67e7e640 1 -- 172.21.15.112:0/1518393472 wait complete. 2024-09-21T00:29:16.168 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-21T00:29:16.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:16.279+0000 7f1388a69640 1 -- 172.21.15.112:0/285073439 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1370004f40 msgr2=0x7f137000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:16.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:16.279+0000 7f1388a69640 1 --2- 172.21.15.112:0/285073439 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1370004f40 0x7f137000b9c0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f137400c310 tx=0x7f1370005830 comp rx=0 tx=0).stop 2024-09-21T00:29:16.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:16.279+0000 7f1382d9d640 1 -- 172.21.15.112:0/285073439 reap_dead start 2024-09-21T00:29:16.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:16.279+0000 7f1388a69640 1 -- 172.21.15.112:0/285073439 shutdown_connections 2024-09-21T00:29:16.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:16.279+0000 7f1388a69640 1 -- 172.21.15.112:0/285073439 >> 172.21.15.112:0/285073439 conn(0x7f13840b43f0 msgr2=0x7f13840b4830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:16.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:16.279+0000 7f1388a69640 1 -- 172.21.15.112:0/285073439 shutdown_connections 2024-09-21T00:29:16.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:16.279+0000 7f1388a69640 1 -- 172.21.15.112:0/285073439 wait complete. 2024-09-21T00:29:16.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:16.281+0000 7f1388a69640 1 Processor -- start 2024-09-21T00:29:16.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:16.281+0000 7f1388a69640 1 -- start start 2024-09-21T00:29:16.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:16.283+0000 7f1388a69640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1370004f40 0x7f13841f62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:16.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:16.283+0000 7f1388a69640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f13840b5ca0 0x7f13840711d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:16.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:16.283+0000 7f1388a69640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1384071710 0x7f1384156310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:16.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:16.283+0000 7f1388a69640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1384161620 con 0x7f1384071710 2024-09-21T00:29:16.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:16.283+0000 7f1388a69640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f13841614a0 con 0x7f13840b5ca0 2024-09-21T00:29:16.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:16.283+0000 7f1388a69640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f13841617a0 con 0x7f1370004f40 2024-09-21T00:29:16.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:16.283+0000 7f1382d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1384071710 0x7f1384156310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:16.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:16.283+0000 7f1382d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1384071710 0x7f1384156310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48490/0 (socket says 172.21.15.112:48490) 2024-09-21T00:29:16.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:16.283+0000 7f1382d9d640 1 -- 172.21.15.112:0/314319967 learned_addr learned my addr 172.21.15.112:0/314319967 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:29:16.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:16.283+0000 7f138259c640 1 --2- 172.21.15.112:0/314319967 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1370004f40 0x7f13841f62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:16.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:16.283+0000 7f1381d9b640 1 --2- 172.21.15.112:0/314319967 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f13840b5ca0 0x7f13840711d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:16.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:16.283+0000 7f1382d9d640 1 -- 172.21.15.112:0/314319967 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1370004f40 msgr2=0x7f13841f62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:16.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:16.283+0000 7f1382d9d640 1 --2- 172.21.15.112:0/314319967 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1370004f40 0x7f13841f62b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:16.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:16.283+0000 7f1382d9d640 1 -- 172.21.15.112:0/314319967 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f13840b5ca0 msgr2=0x7f13840711d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:16.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:16.283+0000 7f1382d9d640 1 --2- 172.21.15.112:0/314319967 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f13840b5ca0 0x7f13840711d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:16.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:16.283+0000 7f1382d9d640 1 -- 172.21.15.112:0/314319967 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1384071b20 con 0x7f1384071710 2024-09-21T00:29:16.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:16.283+0000 7f1381d9b640 1 --2- 172.21.15.112:0/314319967 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f13840b5ca0 0x7f13840711d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:29:16.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:16.283+0000 7f1382d9d640 1 --2- 172.21.15.112:0/314319967 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1384071710 0x7f1384156310 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f137402b180 tx=0x7f137402ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:16.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:16.283+0000 7f137affd640 1 -- 172.21.15.112:0/314319967 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1374031070 con 0x7f1384071710 2024-09-21T00:29:16.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:16.283+0000 7f137affd640 1 -- 172.21.15.112:0/314319967 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f13740014f0 con 0x7f1384071710 2024-09-21T00:29:16.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:16.283+0000 7f1388a69640 1 -- 172.21.15.112:0/314319967 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13840b6720 con 0x7f1384071710 2024-09-21T00:29:16.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:16.283+0000 7f137affd640 1 -- 172.21.15.112:0/314319967 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1374006c90 con 0x7f1384071710 2024-09-21T00:29:16.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:16.283+0000 7f1388a69640 1 -- 172.21.15.112:0/314319967 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13840b6ce0 con 0x7f1384071710 2024-09-21T00:29:16.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:16.284+0000 7f1388a69640 1 -- 172.21.15.112:0/314319967 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f138406ad00 con 0x7f1384071710 2024-09-21T00:29:16.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:16.285+0000 7f137affd640 1 -- 172.21.15.112:0/314319967 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1374034070 con 0x7f1384071710 2024-09-21T00:29:16.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:16.285+0000 7f137affd640 1 --2- 172.21.15.112:0/314319967 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1364078b90 0x7f136407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:16.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:16.286+0000 7f137affd640 1 -- 172.21.15.112:0/314319967 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5249+0+0 (secure 0 0 0) 0x7f13740ba8d0 con 0x7f1384071710 2024-09-21T00:29:16.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:16.286+0000 7f138259c640 1 --2- 172.21.15.112:0/314319967 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1364078b90 0x7f136407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:16.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:16.286+0000 7f138259c640 1 --2- 172.21.15.112:0/314319967 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1364078b90 0x7f136407b0a0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f137c005fd0 tx=0x7f137c009530 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:16.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:16.290+0000 7f137affd640 1 -- 172.21.15.112:0/314319967 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f138406ad00 con 0x7f1384071710 2024-09-21T00:29:16.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:16.470+0000 7f1388a69640 1 -- 172.21.15.112:0/314319967 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f13840b64b0 con 0x7f1384071710 2024-09-21T00:29:16.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:16.470+0000 7f137affd640 1 -- 172.21.15.112:0/314319967 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v90) ==== 84+0+65 (secure 0 0 0) 0x7f1374082e50 con 0x7f1384071710 2024-09-21T00:29:16.472 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:29:16.472 INFO:teuthology.orchestra.run.smithi112.stdout:[ 2024-09-21T00:29:16.472 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:29:16.472 INFO:teuthology.orchestra.run.smithi112.stdout: "poolnum": 1, 2024-09-21T00:29:16.473 INFO:teuthology.orchestra.run.smithi112.stdout: "poolname": ".mgr" 2024-09-21T00:29:16.473 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:29:16.473 INFO:teuthology.orchestra.run.smithi112.stdout:] 2024-09-21T00:29:16.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:16.472+0000 7f1388a69640 1 -- 172.21.15.112:0/314319967 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1364078b90 msgr2=0x7f136407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:16.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:16.472+0000 7f1388a69640 1 --2- 172.21.15.112:0/314319967 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1364078b90 0x7f136407b0a0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f137c005fd0 tx=0x7f137c009530 comp rx=0 tx=0).stop 2024-09-21T00:29:16.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:16.472+0000 7f1388a69640 1 -- 172.21.15.112:0/314319967 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1384071710 msgr2=0x7f1384156310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:16.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:16.472+0000 7f1388a69640 1 --2- 172.21.15.112:0/314319967 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1384071710 0x7f1384156310 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f137402b180 tx=0x7f137402ec40 comp rx=0 tx=0).stop 2024-09-21T00:29:16.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:16.472+0000 7f1388a69640 1 -- 172.21.15.112:0/314319967 shutdown_connections 2024-09-21T00:29:16.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:16.472+0000 7f1388a69640 1 --2- 172.21.15.112:0/314319967 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1364078b90 0x7f136407b0a0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:16.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:16.472+0000 7f1388a69640 1 --2- 172.21.15.112:0/314319967 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1384071710 0x7f1384156310 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:16.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:16.472+0000 7f1388a69640 1 --2- 172.21.15.112:0/314319967 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f13840b5ca0 0x7f13840711d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:16.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:16.472+0000 7f1388a69640 1 --2- 172.21.15.112:0/314319967 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1370004f40 0x7f13841f62b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:16.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:16.472+0000 7f1388a69640 1 -- 172.21.15.112:0/314319967 >> 172.21.15.112:0/314319967 conn(0x7f13840b43f0 msgr2=0x7f1384158760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:16.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:16.472+0000 7f1388a69640 1 -- 172.21.15.112:0/314319967 shutdown_connections 2024-09-21T00:29:16.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:16.472+0000 7f1388a69640 1 -- 172.21.15.112:0/314319967 wait complete. 2024-09-21T00:29:16.485 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-21T00:29:16.485 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-21T00:29:16.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:16.600+0000 7fe583e16640 1 -- 172.21.15.112:0/3914425795 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe5680053a0 msgr2=0x7fe56800ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:16.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:16.600+0000 7fe583e16640 1 --2- 172.21.15.112:0/3914425795 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe5680053a0 0x7fe56800ba50 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fe574005fd0 tx=0x7fe568005830 comp rx=0 tx=0).stop 2024-09-21T00:29:16.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:16.600+0000 7fe583e16640 1 -- 172.21.15.112:0/3914425795 shutdown_connections 2024-09-21T00:29:16.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:16.600+0000 7fe583e16640 1 --2- 172.21.15.112:0/3914425795 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe5680053a0 0x7fe56800ba50 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:16.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:16.600+0000 7fe583e16640 1 -- 172.21.15.112:0/3914425795 >> 172.21.15.112:0/3914425795 conn(0x7fe57c0b47d0 msgr2=0x7fe57c0d2470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:16.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:16.600+0000 7fe583e16640 1 -- 172.21.15.112:0/3914425795 shutdown_connections 2024-09-21T00:29:16.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:16.600+0000 7fe583e16640 1 -- 172.21.15.112:0/3914425795 wait complete. 2024-09-21T00:29:16.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:16.602+0000 7fe583e16640 1 Processor -- start 2024-09-21T00:29:16.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:16.602+0000 7fe583e16640 1 -- start start 2024-09-21T00:29:16.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:16.603+0000 7fe583e16640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe5680053a0 0x7fe57c149760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:16.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:16.603+0000 7fe583e16640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe57c158ed0 0x7fe57c0cb820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:16.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:16.603+0000 7fe583e16640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe57c1f3550 0x7fe57c0cbd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:16.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:16.603+0000 7fe583e16640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe57c15d950 con 0x7fe57c158ed0 2024-09-21T00:29:16.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:16.603+0000 7fe583e16640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe57c15d7d0 con 0x7fe5680053a0 2024-09-21T00:29:16.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:16.603+0000 7fe583e16640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe57c15dad0 con 0x7fe57c1f3550 2024-09-21T00:29:16.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:16.604+0000 7fe5813b1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe57c158ed0 0x7fe57c0cb820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:16.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:16.604+0000 7fe5813b1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe57c158ed0 0x7fe57c0cb820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48506/0 (socket says 172.21.15.112:48506) 2024-09-21T00:29:16.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:16.604+0000 7fe5813b1640 1 -- 172.21.15.112:0/1659378064 learned_addr learned my addr 172.21.15.112:0/1659378064 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:29:16.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:16.604+0000 7fe581bb2640 1 --2- 172.21.15.112:0/1659378064 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe5680053a0 0x7fe57c149760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:16.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:16.604+0000 7fe5823b3640 1 --2- 172.21.15.112:0/1659378064 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe57c1f3550 0x7fe57c0cbd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:16.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:16.604+0000 7fe5813b1640 1 -- 172.21.15.112:0/1659378064 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe57c1f3550 msgr2=0x7fe57c0cbd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:16.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:16.604+0000 7fe5813b1640 1 --2- 172.21.15.112:0/1659378064 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe57c1f3550 0x7fe57c0cbd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:16.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:16.604+0000 7fe5813b1640 1 -- 172.21.15.112:0/1659378064 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe5680053a0 msgr2=0x7fe57c149760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:16.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:16.604+0000 7fe5813b1640 1 --2- 172.21.15.112:0/1659378064 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe5680053a0 0x7fe57c149760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:16.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:16.604+0000 7fe5813b1640 1 -- 172.21.15.112:0/1659378064 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe57c14b8c0 con 0x7fe57c158ed0 2024-09-21T00:29:16.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:16.604+0000 7fe581bb2640 1 --2- 172.21.15.112:0/1659378064 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe5680053a0 0x7fe57c149760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:29:16.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:16.604+0000 7fe5823b3640 1 --2- 172.21.15.112:0/1659378064 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe57c1f3550 0x7fe57c0cbd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:29:16.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:16.604+0000 7fe5813b1640 1 --2- 172.21.15.112:0/1659378064 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe57c158ed0 0x7fe57c0cb820 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fe56c003e10 tx=0x7fe56c001630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:16.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:16.604+0000 7fe5727fc640 1 -- 172.21.15.112:0/1659378064 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe56c015020 con 0x7fe57c158ed0 2024-09-21T00:29:16.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:16.604+0000 7fe5727fc640 1 -- 172.21.15.112:0/1659378064 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe56c011840 con 0x7fe57c158ed0 2024-09-21T00:29:16.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:16.604+0000 7fe583e16640 1 -- 172.21.15.112:0/1659378064 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe57c14bba0 con 0x7fe57c158ed0 2024-09-21T00:29:16.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:16.604+0000 7fe5727fc640 1 -- 172.21.15.112:0/1659378064 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe56c011c70 con 0x7fe57c158ed0 2024-09-21T00:29:16.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:16.604+0000 7fe583e16640 1 -- 172.21.15.112:0/1659378064 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe57c14c160 con 0x7fe57c158ed0 2024-09-21T00:29:16.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:16.605+0000 7fe583e16640 1 -- 172.21.15.112:0/1659378064 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe57c0ccba0 con 0x7fe57c158ed0 2024-09-21T00:29:16.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:16.606+0000 7fe5727fc640 1 -- 172.21.15.112:0/1659378064 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe56c018020 con 0x7fe57c158ed0 2024-09-21T00:29:16.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:16.606+0000 7fe5727fc640 1 --2- 172.21.15.112:0/1659378064 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fe560078b90 0x7fe56007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:16.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:16.607+0000 7fe5727fc640 1 -- 172.21.15.112:0/1659378064 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5249+0+0 (secure 0 0 0) 0x7fe56c09b5a0 con 0x7fe57c158ed0 2024-09-21T00:29:16.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:16.607+0000 7fe581bb2640 1 --2- 172.21.15.112:0/1659378064 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fe560078b90 0x7fe56007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:16.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:16.607+0000 7fe581bb2640 1 --2- 172.21.15.112:0/1659378064 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fe560078b90 0x7fe56007b0a0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fe578001eb0 tx=0x7fe578001620 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:16.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:16.611+0000 7fe5727fc640 1 -- 172.21.15.112:0/1659378064 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe56c01f020 con 0x7fe57c158ed0 2024-09-21T00:29:16.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:16.788+0000 7fe583e16640 1 -- 172.21.15.112:0/1659378064 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fe57c14b650 con 0x7fe57c158ed0 2024-09-21T00:29:17.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:17.773+0000 7fe5727fc640 1 -- 172.21.15.112:0/1659378064 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v91) ==== 138+0+0 (secure 0 0 0) 0x7fe56c063cc0 con 0x7fe57c158ed0 2024-09-21T00:29:17.775 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'cephfs_metadata' created 2024-09-21T00:29:17.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:17.776+0000 7fe583e16640 1 -- 172.21.15.112:0/1659378064 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fe560078b90 msgr2=0x7fe56007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:17.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:17.776+0000 7fe583e16640 1 --2- 172.21.15.112:0/1659378064 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fe560078b90 0x7fe56007b0a0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fe578001eb0 tx=0x7fe578001620 comp rx=0 tx=0).stop 2024-09-21T00:29:17.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:17.776+0000 7fe583e16640 1 -- 172.21.15.112:0/1659378064 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe57c158ed0 msgr2=0x7fe57c0cb820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:17.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:17.776+0000 7fe583e16640 1 --2- 172.21.15.112:0/1659378064 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe57c158ed0 0x7fe57c0cb820 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fe56c003e10 tx=0x7fe56c001630 comp rx=0 tx=0).stop 2024-09-21T00:29:17.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:17.776+0000 7fe583e16640 1 -- 172.21.15.112:0/1659378064 shutdown_connections 2024-09-21T00:29:17.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:17.776+0000 7fe583e16640 1 --2- 172.21.15.112:0/1659378064 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe57c1f3550 0x7fe57c0cbd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:17.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:17.776+0000 7fe583e16640 1 --2- 172.21.15.112:0/1659378064 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fe560078b90 0x7fe56007b0a0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:17.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:17.776+0000 7fe583e16640 1 --2- 172.21.15.112:0/1659378064 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe57c158ed0 0x7fe57c0cb820 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:17.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:17.776+0000 7fe583e16640 1 --2- 172.21.15.112:0/1659378064 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe5680053a0 0x7fe57c149760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:17.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:17.776+0000 7fe583e16640 1 -- 172.21.15.112:0/1659378064 >> 172.21.15.112:0/1659378064 conn(0x7fe57c0b47d0 msgr2=0x7fe57c0cee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:17.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:17.777+0000 7fe583e16640 1 -- 172.21.15.112:0/1659378064 shutdown_connections 2024-09-21T00:29:17.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:17.777+0000 7fe583e16640 1 -- 172.21.15.112:0/1659378064 wait complete. 2024-09-21T00:29:17.796 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-21T00:29:17.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:17.919+0000 7f2f4d790640 1 -- 172.21.15.112:0/3621175127 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f4815f3a0 msgr2=0x7f2f4815f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:17.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:17.919+0000 7f2f4d790640 1 --2- 172.21.15.112:0/3621175127 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f4815f3a0 0x7f2f4815f7b0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f2f480d17f0 tx=0x7f2f4002aa30 comp rx=0 tx=0).stop 2024-09-21T00:29:17.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:17.920+0000 7f2f4d790640 1 -- 172.21.15.112:0/3621175127 shutdown_connections 2024-09-21T00:29:17.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:17.920+0000 7f2f4d790640 1 --2- 172.21.15.112:0/3621175127 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f4815f3a0 0x7f2f4815f7b0 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:17.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:17.920+0000 7f2f4d790640 1 --2- 172.21.15.112:0/3621175127 >> v2:172.21.15.187:3300/0 conn(0x7f2f480b3970 0x7f2f4815ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:17.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:17.920+0000 7f2f4d790640 1 -- 172.21.15.112:0/3621175127 >> 172.21.15.112:0/3621175127 conn(0x7f2f4808dcb0 msgr2=0x7f2f4809d8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:17.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:17.920+0000 7f2f4d790640 1 -- 172.21.15.112:0/3621175127 shutdown_connections 2024-09-21T00:29:17.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:17.920+0000 7f2f4d790640 1 -- 172.21.15.112:0/3621175127 wait complete. 2024-09-21T00:29:17.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:17.921+0000 7f2f4d790640 1 Processor -- start 2024-09-21T00:29:17.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:17.921+0000 7f2f4d790640 1 -- start start 2024-09-21T00:29:17.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:17.923+0000 7f2f4d790640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2f480b3970 0x7f2f480cd9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:17.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:17.923+0000 7f2f4d790640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f4815c3c0 0x7f2f4815ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:17.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:17.923+0000 7f2f4d790640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2f4815f3a0 0x7f2f4815b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:17.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:17.923+0000 7f2f4d790640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2f48163180 con 0x7f2f4815c3c0 2024-09-21T00:29:17.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:17.923+0000 7f2f4d790640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f2f480d0d30 con 0x7f2f480b3970 2024-09-21T00:29:17.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:17.923+0000 7f2f4d790640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f2f48163300 con 0x7f2f4815f3a0 2024-09-21T00:29:17.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:17.923+0000 7f2f3ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f4815c3c0 0x7f2f4815ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:17.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:17.923+0000 7f2f3ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f4815c3c0 0x7f2f4815ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48530/0 (socket says 172.21.15.112:48530) 2024-09-21T00:29:17.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:17.923+0000 7f2f3ffff640 1 -- 172.21.15.112:0/1582719971 learned_addr learned my addr 172.21.15.112:0/1582719971 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:29:17.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:17.923+0000 7f2f46ffd640 1 --2- 172.21.15.112:0/1582719971 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2f480b3970 0x7f2f480cd9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:17.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:17.923+0000 7f2f477fe640 1 --2- 172.21.15.112:0/1582719971 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2f4815f3a0 0x7f2f4815b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:17.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:17.923+0000 7f2f3ffff640 1 -- 172.21.15.112:0/1582719971 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2f4815f3a0 msgr2=0x7f2f4815b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:17.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:17.923+0000 7f2f3ffff640 1 --2- 172.21.15.112:0/1582719971 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2f4815f3a0 0x7f2f4815b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:17.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:17.923+0000 7f2f3ffff640 1 -- 172.21.15.112:0/1582719971 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2f480b3970 msgr2=0x7f2f480cd9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:17.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:17.923+0000 7f2f3ffff640 1 --2- 172.21.15.112:0/1582719971 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2f480b3970 0x7f2f480cd9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:17.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:17.923+0000 7f2f3ffff640 1 -- 172.21.15.112:0/1582719971 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f481f9b80 con 0x7f2f4815c3c0 2024-09-21T00:29:17.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:17.923+0000 7f2f46ffd640 1 --2- 172.21.15.112:0/1582719971 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2f480b3970 0x7f2f480cd9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:29:17.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:17.923+0000 7f2f477fe640 1 --2- 172.21.15.112:0/1582719971 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2f4815f3a0 0x7f2f4815b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:29:17.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:17.923+0000 7f2f3ffff640 1 --2- 172.21.15.112:0/1582719971 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f4815c3c0 0x7f2f4815ac60 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f2f3000d750 tx=0x7f2f3000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:17.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:17.923+0000 7f2f3f7fe640 1 -- 172.21.15.112:0/1582719971 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f30017070 con 0x7f2f4815c3c0 2024-09-21T00:29:17.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:17.923+0000 7f2f3f7fe640 1 -- 172.21.15.112:0/1582719971 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2f30009b20 con 0x7f2f4815c3c0 2024-09-21T00:29:17.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:17.923+0000 7f2f4d790640 1 -- 172.21.15.112:0/1582719971 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f48070020 con 0x7f2f4815c3c0 2024-09-21T00:29:17.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:17.924+0000 7f2f3f7fe640 1 -- 172.21.15.112:0/1582719971 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f30010460 con 0x7f2f4815c3c0 2024-09-21T00:29:17.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:17.924+0000 7f2f4d790640 1 -- 172.21.15.112:0/1582719971 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f481f90a0 con 0x7f2f4815c3c0 2024-09-21T00:29:17.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:17.924+0000 7f2f4d790640 1 -- 172.21.15.112:0/1582719971 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f4815ee60 con 0x7f2f4815c3c0 2024-09-21T00:29:17.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:17.925+0000 7f2f3f7fe640 1 -- 172.21.15.112:0/1582719971 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2f3001a070 con 0x7f2f4815c3c0 2024-09-21T00:29:17.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:17.926+0000 7f2f3f7fe640 1 --2- 172.21.15.112:0/1582719971 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2f28078b90 0x7f2f2807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:17.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:17.926+0000 7f2f3f7fe640 1 -- 172.21.15.112:0/1582719971 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5619+0+0 (secure 0 0 0) 0x7f2f3009ae20 con 0x7f2f4815c3c0 2024-09-21T00:29:17.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:17.926+0000 7f2f46ffd640 1 --2- 172.21.15.112:0/1582719971 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2f28078b90 0x7f2f2807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:17.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:17.927+0000 7f2f46ffd640 1 --2- 172.21.15.112:0/1582719971 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2f28078b90 0x7f2f2807b0a0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f2f380016d0 tx=0x7f2f38009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:17.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:17.931+0000 7f2f3f7fe640 1 -- 172.21.15.112:0/1582719971 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2f300633d0 con 0x7f2f4815c3c0 2024-09-21T00:29:18.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:18.109+0000 7f2f4d790640 1 -- 172.21.15.112:0/1582719971 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f2f4806fdb0 con 0x7f2f4815c3c0 2024-09-21T00:29:18.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:18.776+0000 7f2f3f7fe640 1 -- 172.21.15.112:0/1582719971 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v92) ==== 170+0+0 (secure 0 0 0) 0x7f2f30062cd0 con 0x7f2f4815c3c0 2024-09-21T00:29:18.777 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'cephfs_data' created 2024-09-21T00:29:18.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:18.778+0000 7f2f4d790640 1 -- 172.21.15.112:0/1582719971 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2f28078b90 msgr2=0x7f2f2807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:18.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:18.778+0000 7f2f4d790640 1 --2- 172.21.15.112:0/1582719971 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2f28078b90 0x7f2f2807b0a0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f2f380016d0 tx=0x7f2f38009040 comp rx=0 tx=0).stop 2024-09-21T00:29:18.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:18.778+0000 7f2f4d790640 1 -- 172.21.15.112:0/1582719971 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f4815c3c0 msgr2=0x7f2f4815ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:18.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:18.778+0000 7f2f4d790640 1 --2- 172.21.15.112:0/1582719971 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f4815c3c0 0x7f2f4815ac60 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f2f3000d750 tx=0x7f2f3000dc10 comp rx=0 tx=0).stop 2024-09-21T00:29:18.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:18.778+0000 7f2f4d790640 1 -- 172.21.15.112:0/1582719971 shutdown_connections 2024-09-21T00:29:18.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:18.778+0000 7f2f4d790640 1 --2- 172.21.15.112:0/1582719971 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2f4815f3a0 0x7f2f4815b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:18.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:18.778+0000 7f2f4d790640 1 --2- 172.21.15.112:0/1582719971 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2f28078b90 0x7f2f2807b0a0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:18.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:18.778+0000 7f2f4d790640 1 --2- 172.21.15.112:0/1582719971 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f4815c3c0 0x7f2f4815ac60 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:18.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:18.778+0000 7f2f4d790640 1 --2- 172.21.15.112:0/1582719971 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2f480b3970 0x7f2f480cd9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:18.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:18.778+0000 7f2f4d790640 1 -- 172.21.15.112:0/1582719971 >> 172.21.15.112:0/1582719971 conn(0x7f2f4808dcb0 msgr2=0x7f2f4815d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:18.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:18.779+0000 7f2f4d790640 1 -- 172.21.15.112:0/1582719971 shutdown_connections 2024-09-21T00:29:18.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:18.779+0000 7f2f4d790640 1 -- 172.21.15.112:0/1582719971 wait complete. 2024-09-21T00:29:18.801 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-21T00:29:18.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:18.916+0000 7f3d95557640 1 -- 172.21.15.112:0/111000126 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d78004850 msgr2=0x7f3d78004d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:18.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:18.916+0000 7f3d95557640 1 --2- 172.21.15.112:0/111000126 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d78004850 0x7f3d78004d20 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f3d84001830 tx=0x7f3d78005830 comp rx=0 tx=0).stop 2024-09-21T00:29:18.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:18.917+0000 7f3d95557640 1 -- 172.21.15.112:0/111000126 shutdown_connections 2024-09-21T00:29:18.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:18.917+0000 7f3d95557640 1 --2- 172.21.15.112:0/111000126 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d78004850 0x7f3d78004d20 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:18.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:18.917+0000 7f3d95557640 1 -- 172.21.15.112:0/111000126 >> 172.21.15.112:0/111000126 conn(0x7f3d9008bfe0 msgr2=0x7f3d9009d890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:18.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:18.917+0000 7f3d95557640 1 -- 172.21.15.112:0/111000126 shutdown_connections 2024-09-21T00:29:18.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:18.917+0000 7f3d95557640 1 -- 172.21.15.112:0/111000126 wait complete. 2024-09-21T00:29:18.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:18.918+0000 7f3d95557640 1 Processor -- start 2024-09-21T00:29:18.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:18.919+0000 7f3d95557640 1 -- start start 2024-09-21T00:29:18.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:18.920+0000 7f3d95557640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3d78004850 0x7f3d90160d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:18.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:18.920+0000 7f3d95557640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3d901f58b0 0x7f3d9014f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:18.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:18.920+0000 7f3d95557640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d900b4440 0x7f3d900b4850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:18.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:18.920+0000 7f3d95557640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3d901633e0 con 0x7f3d900b4440 2024-09-21T00:29:18.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:18.920+0000 7f3d95557640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3d90163260 con 0x7f3d901f58b0 2024-09-21T00:29:18.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:18.920+0000 7f3d95557640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3d90163560 con 0x7f3d78004850 2024-09-21T00:29:18.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:18.920+0000 7f3d8f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d900b4440 0x7f3d900b4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:18.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:18.920+0000 7f3d8f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d900b4440 0x7f3d900b4850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48554/0 (socket says 172.21.15.112:48554) 2024-09-21T00:29:18.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:18.920+0000 7f3d8f7fe640 1 -- 172.21.15.112:0/3036254261 learned_addr learned my addr 172.21.15.112:0/3036254261 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:29:18.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:18.920+0000 7f3d8e7fc640 1 --2- 172.21.15.112:0/3036254261 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3d901f58b0 0x7f3d9014f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:18.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:18.920+0000 7f3d8effd640 1 --2- 172.21.15.112:0/3036254261 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3d78004850 0x7f3d90160d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:18.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:18.920+0000 7f3d8f7fe640 1 -- 172.21.15.112:0/3036254261 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3d78004850 msgr2=0x7f3d90160d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:18.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:18.920+0000 7f3d8f7fe640 1 --2- 172.21.15.112:0/3036254261 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3d78004850 0x7f3d90160d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:18.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:18.920+0000 7f3d8f7fe640 1 -- 172.21.15.112:0/3036254261 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3d901f58b0 msgr2=0x7f3d9014f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:18.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:18.920+0000 7f3d8f7fe640 1 --2- 172.21.15.112:0/3036254261 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3d901f58b0 0x7f3d9014f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:18.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:18.920+0000 7f3d8f7fe640 1 -- 172.21.15.112:0/3036254261 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d900b4d90 con 0x7f3d900b4440 2024-09-21T00:29:18.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:18.920+0000 7f3d8e7fc640 1 --2- 172.21.15.112:0/3036254261 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3d901f58b0 0x7f3d9014f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:29:18.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:18.920+0000 7f3d8f7fe640 1 --2- 172.21.15.112:0/3036254261 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d900b4440 0x7f3d900b4850 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f3d88002230 tx=0x7f3d8800eb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:18.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:18.920+0000 7f3d777fe640 1 -- 172.21.15.112:0/3036254261 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d88010890 con 0x7f3d900b4440 2024-09-21T00:29:18.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:18.920+0000 7f3d777fe640 1 -- 172.21.15.112:0/3036254261 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3d88017070 con 0x7f3d900b4440 2024-09-21T00:29:18.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:18.921+0000 7f3d95557640 1 -- 172.21.15.112:0/3036254261 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d901612b0 con 0x7f3d900b4440 2024-09-21T00:29:18.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:18.921+0000 7f3d777fe640 1 -- 172.21.15.112:0/3036254261 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d88010da0 con 0x7f3d900b4440 2024-09-21T00:29:18.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:18.921+0000 7f3d95557640 1 -- 172.21.15.112:0/3036254261 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d901f6380 con 0x7f3d900b4440 2024-09-21T00:29:18.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:18.922+0000 7f3d95557640 1 -- 172.21.15.112:0/3036254261 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d900d05c0 con 0x7f3d900b4440 2024-09-21T00:29:18.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:18.922+0000 7f3d777fe640 1 -- 172.21.15.112:0/3036254261 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3d8801c460 con 0x7f3d900b4440 2024-09-21T00:29:18.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:18.923+0000 7f3d777fe640 1 --2- 172.21.15.112:0/3036254261 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f3d7c078b90 0x7f3d7c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:18.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:18.923+0000 7f3d777fe640 1 -- 172.21.15.112:0/3036254261 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 6001+0+0 (secure 0 0 0) 0x7f3d8809b240 con 0x7f3d900b4440 2024-09-21T00:29:18.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:18.923+0000 7f3d8effd640 1 --2- 172.21.15.112:0/3036254261 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f3d7c078b90 0x7f3d7c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:18.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:18.924+0000 7f3d8effd640 1 --2- 172.21.15.112:0/3036254261 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f3d7c078b90 0x7f3d7c07b0a0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f3d80005eb0 tx=0x7f3d80005f90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:18.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:18.928+0000 7f3d777fe640 1 -- 172.21.15.112:0/3036254261 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3d900b4d90 con 0x7f3d900b4440 2024-09-21T00:29:19.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:19.144+0000 7f3d95557640 1 -- 172.21.15.112:0/3036254261 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f3d900b6f90 con 0x7f3d900b4440 2024-09-21T00:29:19.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:19.796+0000 7f3d777fe640 1 -- 172.21.15.112:0/3036254261 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 15 and data pool 16 v60) ==== 174+0+0 (secure 0 0 0) 0x7f3d88063550 con 0x7f3d900b4440 2024-09-21T00:29:19.802 INFO:teuthology.orchestra.run.smithi112.stderr:new fs with metadata pool 15 and data pool 16 2024-09-21T00:29:19.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:19.813+0000 7f3d95557640 1 -- 172.21.15.112:0/3036254261 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f3d7c078b90 msgr2=0x7f3d7c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:19.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:19.813+0000 7f3d95557640 1 --2- 172.21.15.112:0/3036254261 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f3d7c078b90 0x7f3d7c07b0a0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f3d80005eb0 tx=0x7f3d80005f90 comp rx=0 tx=0).stop 2024-09-21T00:29:19.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:19.813+0000 7f3d95557640 1 -- 172.21.15.112:0/3036254261 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d900b4440 msgr2=0x7f3d900b4850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:19.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:19.813+0000 7f3d95557640 1 --2- 172.21.15.112:0/3036254261 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d900b4440 0x7f3d900b4850 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f3d88002230 tx=0x7f3d8800eb00 comp rx=0 tx=0).stop 2024-09-21T00:29:19.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:19.815+0000 7f3d95557640 1 -- 172.21.15.112:0/3036254261 shutdown_connections 2024-09-21T00:29:19.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:19.815+0000 7f3d95557640 1 --2- 172.21.15.112:0/3036254261 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f3d7c078b90 0x7f3d7c07b0a0 secure :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f3d80005eb0 tx=0x7f3d80005f90 comp rx=0 tx=0).stop 2024-09-21T00:29:19.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:19.815+0000 7f3d95557640 1 --2- 172.21.15.112:0/3036254261 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d900b4440 0x7f3d900b4850 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:19.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:19.815+0000 7f3d95557640 1 --2- 172.21.15.112:0/3036254261 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3d901f58b0 0x7f3d9014f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:19.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:19.815+0000 7f3d95557640 1 --2- 172.21.15.112:0/3036254261 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3d78004850 0x7f3d90160d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:19.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:19.815+0000 7f3d95557640 1 -- 172.21.15.112:0/3036254261 >> 172.21.15.112:0/3036254261 conn(0x7f3d9008bfe0 msgr2=0x7f3d9015eb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:19.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:19.819+0000 7f3d95557640 1 -- 172.21.15.112:0/3036254261 shutdown_connections 2024-09-21T00:29:19.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:19.820+0000 7f3d95557640 1 -- 172.21.15.112:0/3036254261 wait complete. 2024-09-21T00:29:19.840 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T00:29:19.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:19.954+0000 7f6f0ef69640 1 -- 172.21.15.112:0/132623264 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6f080ccae0 msgr2=0x7f6f080ccef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:19.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:19.954+0000 7f6f0ef69640 1 --2- 172.21.15.112:0/132623264 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6f080ccae0 0x7f6f080ccef0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f6f0815cd80 tx=0x7f6f0002a8b0 comp rx=0 tx=0).stop 2024-09-21T00:29:19.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:19.954+0000 7f6f0ef69640 1 -- 172.21.15.112:0/132623264 shutdown_connections 2024-09-21T00:29:19.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:19.954+0000 7f6f0ef69640 1 --2- 172.21.15.112:0/132623264 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6f080ccae0 0x7f6f080ccef0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:19.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:19.954+0000 7f6f0ef69640 1 --2- 172.21.15.112:0/132623264 >> v2:172.21.15.187:3300/0 conn(0x7f6f0815d870 0x7f6f080d0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:19.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:19.954+0000 7f6f0ef69640 1 -- 172.21.15.112:0/132623264 >> 172.21.15.112:0/132623264 conn(0x7f6f080b43f0 msgr2=0x7f6f080b4830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:19.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:19.954+0000 7f6f0ef69640 1 -- 172.21.15.112:0/132623264 shutdown_connections 2024-09-21T00:29:19.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:19.954+0000 7f6f0ef69640 1 -- 172.21.15.112:0/132623264 wait complete. 2024-09-21T00:29:19.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:19.955+0000 7f6f0ef69640 1 Processor -- start 2024-09-21T00:29:19.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:19.956+0000 7f6f0ef69640 1 -- start start 2024-09-21T00:29:19.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:19.957+0000 7f6f0ef69640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6f080ccae0 0x7f6f080c5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:19.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:19.957+0000 7f6f0ef69640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6f080d2030 0x7f6f081476c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:19.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:19.957+0000 7f6f0ef69640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6f0815d870 0x7f6f081e9370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:19.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:19.957+0000 7f6f0ef69640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6f080b2280 con 0x7f6f080d2030 2024-09-21T00:29:19.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:19.957+0000 7f6f0ef69640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f6f080b2100 con 0x7f6f080ccae0 2024-09-21T00:29:19.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:19.957+0000 7f6f0ef69640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f6f080b2400 con 0x7f6f0815d870 2024-09-21T00:29:19.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:19.957+0000 7f6f07fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6f080d2030 0x7f6f081476c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:19.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:19.957+0000 7f6f07fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6f080d2030 0x7f6f081476c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48578/0 (socket says 172.21.15.112:48578) 2024-09-21T00:29:19.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:19.957+0000 7f6f07fff640 1 -- 172.21.15.112:0/2258918102 learned_addr learned my addr 172.21.15.112:0/2258918102 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:29:19.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:19.957+0000 7f6f0cd05640 1 --2- 172.21.15.112:0/2258918102 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6f080ccae0 0x7f6f080c5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:19.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:19.957+0000 7f6f0d506640 1 --2- 172.21.15.112:0/2258918102 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6f0815d870 0x7f6f081e9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:19.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:19.957+0000 7f6f07fff640 1 -- 172.21.15.112:0/2258918102 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6f0815d870 msgr2=0x7f6f081e9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:19.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:19.957+0000 7f6f07fff640 1 --2- 172.21.15.112:0/2258918102 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6f0815d870 0x7f6f081e9370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:19.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:19.957+0000 7f6f07fff640 1 -- 172.21.15.112:0/2258918102 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6f080ccae0 msgr2=0x7f6f080c5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:19.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:19.957+0000 7f6f07fff640 1 --2- 172.21.15.112:0/2258918102 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6f080ccae0 0x7f6f080c5fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:19.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:19.957+0000 7f6f07fff640 1 -- 172.21.15.112:0/2258918102 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f080cab00 con 0x7f6f080d2030 2024-09-21T00:29:19.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:19.957+0000 7f6f0d506640 1 --2- 172.21.15.112:0/2258918102 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6f0815d870 0x7f6f081e9370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:29:19.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:19.958+0000 7f6f07fff640 1 --2- 172.21.15.112:0/2258918102 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6f080d2030 0x7f6f081476c0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f6efc00c9a0 tx=0x7f6efc00ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:19.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:19.958+0000 7f6f057fa640 1 -- 172.21.15.112:0/2258918102 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6efc007b50 con 0x7f6f080d2030 2024-09-21T00:29:19.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:19.958+0000 7f6f057fa640 1 -- 172.21.15.112:0/2258918102 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6efc007de0 con 0x7f6f080d2030 2024-09-21T00:29:19.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:19.958+0000 7f6f0ef69640 1 -- 172.21.15.112:0/2258918102 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f080cade0 con 0x7f6f080d2030 2024-09-21T00:29:19.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:19.958+0000 7f6f057fa640 1 -- 172.21.15.112:0/2258918102 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6efc00f6b0 con 0x7f6f080d2030 2024-09-21T00:29:19.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:19.958+0000 7f6f0ef69640 1 -- 172.21.15.112:0/2258918102 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f080cb3a0 con 0x7f6f080d2030 2024-09-21T00:29:19.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:19.959+0000 7f6f0ef69640 1 -- 172.21.15.112:0/2258918102 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f080ca870 con 0x7f6f080d2030 2024-09-21T00:29:19.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:19.960+0000 7f6f057fa640 1 -- 172.21.15.112:0/2258918102 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6efc00f850 con 0x7f6f080d2030 2024-09-21T00:29:19.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:19.960+0000 7f6f057fa640 1 --2- 172.21.15.112:0/2258918102 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f6ef4078b90 0x7f6ef407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:19.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:19.960+0000 7f6f057fa640 1 -- 172.21.15.112:0/2258918102 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6101+0+0 (secure 0 0 0) 0x7f6efc09b540 con 0x7f6f080d2030 2024-09-21T00:29:19.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:19.960+0000 7f6f0cd05640 1 --2- 172.21.15.112:0/2258918102 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f6ef4078b90 0x7f6ef407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:19.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:19.961+0000 7f6f0cd05640 1 --2- 172.21.15.112:0/2258918102 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f6ef4078b90 0x7f6ef407b0a0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f6ef8007d60 tx=0x7f6ef8007cf0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:19.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:19.965+0000 7f6f057fa640 1 -- 172.21.15.112:0/2258918102 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f6efc014030 con 0x7f6f080d2030 2024-09-21T00:29:20.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:20.144+0000 7f6f0ef69640 1 -- 172.21.15.112:0/2258918102 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6f080cd860 con 0x7f6f080d2030 2024-09-21T00:29:20.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:20.145+0000 7f6f057fa640 1 -- 172.21.15.112:0/2258918102 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+16970 (secure 0 0 0) 0x7f6efc063800 con 0x7f6f080d2030 2024-09-21T00:29:20.147 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:29:20.147 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":93,"fsid":"8b34692b-dc65-49b2-a08f-26f707b96ac5","created":"2024-09-21T00:25:22.788490+0000","modified":"2024-09-21T00:29:19.145837+0000","last_up_change":"2024-09-21T00:25:36.161835+0000","last_in_change":"2024-09-21T00:25:26.346500+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T00:25:37.105006+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-09-21T00:29:16.794043+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-09-21T00:29:18.114604+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9c575086-67d2-4de2-b01b-d44dcc21d604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6809","nonce":755760621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6811","nonce":755760621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6815","nonce":755760621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6813","nonce":755760621}]},"public_addr":"172.21.15.112:6809/755760621","cluster_addr":"172.21.15.112:6811/755760621","heartbeat_back_addr":"172.21.15.112:6815/755760621","heartbeat_front_addr":"172.21.15.112:6813/755760621","state":["exists","up"]},{"osd":1,"uuid":"a983c227-a3ce-4bdd-82f1-438ba7e3ecd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6801","nonce":344932592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6803","nonce":344932592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6807","nonce":344932592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6805","nonce":344932592}]},"public_addr":"172.21.15.112:6801/344932592","cluster_addr":"172.21.15.112:6803/344932592","heartbeat_back_addr":"172.21.15.112:6807/344932592","heartbeat_front_addr":"172.21.15.112:6805/344932592","state":["exists","up"]},{"osd":2,"uuid":"1193c1d8-bfff-4a57-8200-5f69bf1ea3af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6825","nonce":407933616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6827","nonce":407933616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6831","nonce":407933616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6829","nonce":407933616}]},"public_addr":"172.21.15.112:6825/407933616","cluster_addr":"172.21.15.112:6827/407933616","heartbeat_back_addr":"172.21.15.112:6831/407933616","heartbeat_front_addr":"172.21.15.112:6829/407933616","state":["exists","up"]},{"osd":3,"uuid":"af04330a-36ad-40cc-a8dd-fa259b221de5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6817","nonce":4120164806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6819","nonce":4120164806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6823","nonce":4120164806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6821","nonce":4120164806}]},"public_addr":"172.21.15.112:6817/4120164806","cluster_addr":"172.21.15.112:6819/4120164806","heartbeat_back_addr":"172.21.15.112:6823/4120164806","heartbeat_front_addr":"172.21.15.112:6821/4120164806","state":["exists","up"]},{"osd":4,"uuid":"5b893111-292c-4c5a-b184-53ad38c7199e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6803","nonce":522077560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6805","nonce":522077560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6809","nonce":522077560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6807","nonce":522077560}]},"public_addr":"172.21.15.187:6803/522077560","cluster_addr":"172.21.15.187:6805/522077560","heartbeat_back_addr":"172.21.15.187:6809/522077560","heartbeat_front_addr":"172.21.15.187:6807/522077560","state":["exists","up"]},{"osd":5,"uuid":"c0b8e011-45fe-4c5e-925e-6be2a7f1868a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6827","nonce":3593144721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6829","nonce":3593144721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6833","nonce":3593144721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6831","nonce":3593144721}]},"public_addr":"172.21.15.187:6827/3593144721","cluster_addr":"172.21.15.187:6829/3593144721","heartbeat_back_addr":"172.21.15.187:6833/3593144721","heartbeat_front_addr":"172.21.15.187:6831/3593144721","state":["exists","up"]},{"osd":6,"uuid":"87d2d650-7489-4593-98a3-a1f433ce8dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6811","nonce":1494387657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6813","nonce":1494387657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6817","nonce":1494387657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6815","nonce":1494387657}]},"public_addr":"172.21.15.187:6811/1494387657","cluster_addr":"172.21.15.187:6813/1494387657","heartbeat_back_addr":"172.21.15.187:6817/1494387657","heartbeat_front_addr":"172.21.15.187:6815/1494387657","state":["exists","up"]},{"osd":7,"uuid":"c6b4f8e2-a1ee-4438-aa37-67ff4b6ef1f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6819","nonce":1624699842}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6821","nonce":1624699842}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6825","nonce":1624699842}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6823","nonce":1624699842}]},"public_addr":"172.21.15.187:6819/1624699842","cluster_addr":"172.21.15.187:6821/1624699842","heartbeat_back_addr":"172.21.15.187:6825/1624699842","heartbeat_front_addr":"172.21.15.187:6823/1624699842","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.873162+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.400506+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.664317+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.159791+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.202630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.113212+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.384786+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.448441+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T00:29:20.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:20.147+0000 7f6f0ef69640 1 -- 172.21.15.112:0/2258918102 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f6ef4078b90 msgr2=0x7f6ef407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:20.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:20.147+0000 7f6f0ef69640 1 --2- 172.21.15.112:0/2258918102 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f6ef4078b90 0x7f6ef407b0a0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f6ef8007d60 tx=0x7f6ef8007cf0 comp rx=0 tx=0).stop 2024-09-21T00:29:20.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:20.147+0000 7f6f0ef69640 1 -- 172.21.15.112:0/2258918102 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6f080d2030 msgr2=0x7f6f081476c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:20.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:20.147+0000 7f6f0ef69640 1 --2- 172.21.15.112:0/2258918102 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6f080d2030 0x7f6f081476c0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f6efc00c9a0 tx=0x7f6efc00ce60 comp rx=0 tx=0).stop 2024-09-21T00:29:20.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:20.148+0000 7f6f0ef69640 1 -- 172.21.15.112:0/2258918102 shutdown_connections 2024-09-21T00:29:20.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:20.148+0000 7f6f0ef69640 1 --2- 172.21.15.112:0/2258918102 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6f0815d870 0x7f6f081e9370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:20.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:20.148+0000 7f6f0ef69640 1 --2- 172.21.15.112:0/2258918102 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f6ef4078b90 0x7f6ef407b0a0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:20.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:20.148+0000 7f6f0ef69640 1 --2- 172.21.15.112:0/2258918102 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6f080d2030 0x7f6f081476c0 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:20.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:20.148+0000 7f6f0ef69640 1 --2- 172.21.15.112:0/2258918102 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6f080ccae0 0x7f6f080c5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:20.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:20.148+0000 7f6f0ef69640 1 -- 172.21.15.112:0/2258918102 >> 172.21.15.112:0/2258918102 conn(0x7f6f080b43f0 msgr2=0x7f6f0815fb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:20.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:20.148+0000 7f6f0ef69640 1 -- 172.21.15.112:0/2258918102 shutdown_connections 2024-09-21T00:29:20.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:20.148+0000 7f6f0ef69640 1 -- 172.21.15.112:0/2258918102 wait complete. 2024-09-21T00:29:20.161 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T00:29:20.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:20.271+0000 7fcaa9757640 1 -- 172.21.15.112:0/2551523326 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcaa40c6a00 msgr2=0x7fcaa40b2d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:20.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:20.271+0000 7fcaa9757640 1 --2- 172.21.15.112:0/2551523326 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcaa40c6a00 0x7fcaa40b2d40 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fcaa4073f90 tx=0x7fca9c02a790 comp rx=0 tx=0).stop 2024-09-21T00:29:20.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:20.271+0000 7fcaa9757640 1 -- 172.21.15.112:0/2551523326 shutdown_connections 2024-09-21T00:29:20.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:20.271+0000 7fcaa9757640 1 --2- 172.21.15.112:0/2551523326 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcaa40c6a00 0x7fcaa40b2d40 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:20.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:20.271+0000 7fcaa9757640 1 --2- 172.21.15.112:0/2551523326 >> v2:172.21.15.187:3300/0 conn(0x7fcaa40ccbf0 0x7fcaa40cd000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:20.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:20.271+0000 7fcaa9757640 1 -- 172.21.15.112:0/2551523326 >> 172.21.15.112:0/2551523326 conn(0x7fcaa40b42e0 msgr2=0x7fcaa40b4720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:20.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:20.271+0000 7fcaa9757640 1 -- 172.21.15.112:0/2551523326 shutdown_connections 2024-09-21T00:29:20.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:20.271+0000 7fcaa9757640 1 -- 172.21.15.112:0/2551523326 wait complete. 2024-09-21T00:29:20.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:20.273+0000 7fcaa9757640 1 Processor -- start 2024-09-21T00:29:20.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:20.273+0000 7fcaa9757640 1 -- start start 2024-09-21T00:29:20.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:20.274+0000 7fcaa9757640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcaa40c5ee0 0x7fcaa40c72b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:20.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:20.274+0000 7fcaa9757640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcaa40c6a00 0x7fcaa4155f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:20.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:20.275+0000 7fcaa9757640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcaa40ccbf0 0x7fcaa41564c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:20.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:20.275+0000 7fcaa9757640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa40b3400 con 0x7fcaa40c5ee0 2024-09-21T00:29:20.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:20.275+0000 7fcaa9757640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa40b3280 con 0x7fcaa40c6a00 2024-09-21T00:29:20.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:20.275+0000 7fcaa9757640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fcaa40b3580 con 0x7fcaa40ccbf0 2024-09-21T00:29:20.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:20.275+0000 7fcaa2ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcaa40c5ee0 0x7fcaa40c72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:20.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:20.275+0000 7fcaa2ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcaa40c5ee0 0x7fcaa40c72b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48596/0 (socket says 172.21.15.112:48596) 2024-09-21T00:29:20.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:20.275+0000 7fcaa2ffd640 1 -- 172.21.15.112:0/1557548495 learned_addr learned my addr 172.21.15.112:0/1557548495 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:29:20.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:20.275+0000 7fcaa37fe640 1 --2- 172.21.15.112:0/1557548495 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcaa40ccbf0 0x7fcaa41564c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:20.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:20.275+0000 7fca9a7fc640 1 --2- 172.21.15.112:0/1557548495 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcaa40c6a00 0x7fcaa4155f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:20.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:20.275+0000 7fcaa2ffd640 1 -- 172.21.15.112:0/1557548495 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcaa40ccbf0 msgr2=0x7fcaa41564c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:20.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:20.275+0000 7fcaa2ffd640 1 --2- 172.21.15.112:0/1557548495 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcaa40ccbf0 0x7fcaa41564c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:20.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:20.275+0000 7fcaa2ffd640 1 -- 172.21.15.112:0/1557548495 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcaa40c6a00 msgr2=0x7fcaa4155f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:20.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:20.275+0000 7fcaa2ffd640 1 --2- 172.21.15.112:0/1557548495 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcaa40c6a00 0x7fcaa4155f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:20.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:20.275+0000 7fcaa2ffd640 1 -- 172.21.15.112:0/1557548495 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcaa4156a00 con 0x7fcaa40c5ee0 2024-09-21T00:29:20.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:20.275+0000 7fcaa37fe640 1 --2- 172.21.15.112:0/1557548495 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcaa40ccbf0 0x7fcaa41564c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:29:20.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:20.275+0000 7fca9a7fc640 1 --2- 172.21.15.112:0/1557548495 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcaa40c6a00 0x7fcaa4155f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:29:20.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:20.275+0000 7fcaa2ffd640 1 --2- 172.21.15.112:0/1557548495 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcaa40c5ee0 0x7fcaa40c72b0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fca9400d770 tx=0x7fca9400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:20.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:20.275+0000 7fca9bfff640 1 -- 172.21.15.112:0/1557548495 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca94017070 con 0x7fcaa40c5ee0 2024-09-21T00:29:20.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:20.275+0000 7fca9bfff640 1 -- 172.21.15.112:0/1557548495 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fca94009d20 con 0x7fcaa40c5ee0 2024-09-21T00:29:20.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:20.275+0000 7fcaa9757640 1 -- 172.21.15.112:0/1557548495 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcaa41f9160 con 0x7fcaa40c5ee0 2024-09-21T00:29:20.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:20.275+0000 7fca9bfff640 1 -- 172.21.15.112:0/1557548495 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca940106c0 con 0x7fcaa40c5ee0 2024-09-21T00:29:20.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:20.275+0000 7fcaa9757640 1 -- 172.21.15.112:0/1557548495 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcaa41f9720 con 0x7fcaa40c5ee0 2024-09-21T00:29:20.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:20.276+0000 7fcaa9757640 1 -- 172.21.15.112:0/1557548495 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcaa40c6490 con 0x7fcaa40c5ee0 2024-09-21T00:29:20.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:20.277+0000 7fca9bfff640 1 -- 172.21.15.112:0/1557548495 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fca9401a070 con 0x7fcaa40c5ee0 2024-09-21T00:29:20.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:20.278+0000 7fca9bfff640 1 --2- 172.21.15.112:0/1557548495 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fca84078b90 0x7fca8407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:20.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:20.278+0000 7fca9bfff640 1 -- 172.21.15.112:0/1557548495 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6101+0+0 (secure 0 0 0) 0x7fca9409b0d0 con 0x7fcaa40c5ee0 2024-09-21T00:29:20.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:20.278+0000 7fca9a7fc640 1 --2- 172.21.15.112:0/1557548495 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fca84078b90 0x7fca8407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:20.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:20.279+0000 7fca9a7fc640 1 --2- 172.21.15.112:0/1557548495 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fca84078b90 0x7fca8407b0a0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fcaa406e830 tx=0x7fca8c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:20.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:20.282+0000 7fca9bfff640 1 -- 172.21.15.112:0/1557548495 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fca94063380 con 0x7fcaa40c5ee0 2024-09-21T00:29:20.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:20.460+0000 7fcaa9757640 1 -- 172.21.15.112:0/1557548495 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcaa4075bc0 con 0x7fcaa40c5ee0 2024-09-21T00:29:20.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:20.461+0000 7fca9bfff640 1 -- 172.21.15.112:0/1557548495 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+16970 (secure 0 0 0) 0x7fca94062c80 con 0x7fcaa40c5ee0 2024-09-21T00:29:20.463 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:29:20.463 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":93,"fsid":"8b34692b-dc65-49b2-a08f-26f707b96ac5","created":"2024-09-21T00:25:22.788490+0000","modified":"2024-09-21T00:29:19.145837+0000","last_up_change":"2024-09-21T00:25:36.161835+0000","last_in_change":"2024-09-21T00:25:26.346500+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T00:25:37.105006+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-09-21T00:29:16.794043+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-09-21T00:29:18.114604+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9c575086-67d2-4de2-b01b-d44dcc21d604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6809","nonce":755760621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6811","nonce":755760621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6815","nonce":755760621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6813","nonce":755760621}]},"public_addr":"172.21.15.112:6809/755760621","cluster_addr":"172.21.15.112:6811/755760621","heartbeat_back_addr":"172.21.15.112:6815/755760621","heartbeat_front_addr":"172.21.15.112:6813/755760621","state":["exists","up"]},{"osd":1,"uuid":"a983c227-a3ce-4bdd-82f1-438ba7e3ecd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6801","nonce":344932592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6803","nonce":344932592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6807","nonce":344932592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6805","nonce":344932592}]},"public_addr":"172.21.15.112:6801/344932592","cluster_addr":"172.21.15.112:6803/344932592","heartbeat_back_addr":"172.21.15.112:6807/344932592","heartbeat_front_addr":"172.21.15.112:6805/344932592","state":["exists","up"]},{"osd":2,"uuid":"1193c1d8-bfff-4a57-8200-5f69bf1ea3af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6825","nonce":407933616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6827","nonce":407933616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6831","nonce":407933616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6829","nonce":407933616}]},"public_addr":"172.21.15.112:6825/407933616","cluster_addr":"172.21.15.112:6827/407933616","heartbeat_back_addr":"172.21.15.112:6831/407933616","heartbeat_front_addr":"172.21.15.112:6829/407933616","state":["exists","up"]},{"osd":3,"uuid":"af04330a-36ad-40cc-a8dd-fa259b221de5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6817","nonce":4120164806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6819","nonce":4120164806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6823","nonce":4120164806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6821","nonce":4120164806}]},"public_addr":"172.21.15.112:6817/4120164806","cluster_addr":"172.21.15.112:6819/4120164806","heartbeat_back_addr":"172.21.15.112:6823/4120164806","heartbeat_front_addr":"172.21.15.112:6821/4120164806","state":["exists","up"]},{"osd":4,"uuid":"5b893111-292c-4c5a-b184-53ad38c7199e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6803","nonce":522077560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6805","nonce":522077560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6809","nonce":522077560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6807","nonce":522077560}]},"public_addr":"172.21.15.187:6803/522077560","cluster_addr":"172.21.15.187:6805/522077560","heartbeat_back_addr":"172.21.15.187:6809/522077560","heartbeat_front_addr":"172.21.15.187:6807/522077560","state":["exists","up"]},{"osd":5,"uuid":"c0b8e011-45fe-4c5e-925e-6be2a7f1868a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6827","nonce":3593144721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6829","nonce":3593144721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6833","nonce":3593144721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6831","nonce":3593144721}]},"public_addr":"172.21.15.187:6827/3593144721","cluster_addr":"172.21.15.187:6829/3593144721","heartbeat_back_addr":"172.21.15.187:6833/3593144721","heartbeat_front_addr":"172.21.15.187:6831/3593144721","state":["exists","up"]},{"osd":6,"uuid":"87d2d650-7489-4593-98a3-a1f433ce8dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6811","nonce":1494387657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6813","nonce":1494387657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6817","nonce":1494387657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6815","nonce":1494387657}]},"public_addr":"172.21.15.187:6811/1494387657","cluster_addr":"172.21.15.187:6813/1494387657","heartbeat_back_addr":"172.21.15.187:6817/1494387657","heartbeat_front_addr":"172.21.15.187:6815/1494387657","state":["exists","up"]},{"osd":7,"uuid":"c6b4f8e2-a1ee-4438-aa37-67ff4b6ef1f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6819","nonce":1624699842}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6821","nonce":1624699842}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6825","nonce":1624699842}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6823","nonce":1624699842}]},"public_addr":"172.21.15.187:6819/1624699842","cluster_addr":"172.21.15.187:6821/1624699842","heartbeat_back_addr":"172.21.15.187:6825/1624699842","heartbeat_front_addr":"172.21.15.187:6823/1624699842","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.873162+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.400506+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.664317+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.159791+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.202630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.113212+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.384786+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.448441+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T00:29:20.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:20.464+0000 7fcaa9757640 1 -- 172.21.15.112:0/1557548495 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fca84078b90 msgr2=0x7fca8407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:20.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:20.464+0000 7fcaa9757640 1 --2- 172.21.15.112:0/1557548495 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fca84078b90 0x7fca8407b0a0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fcaa406e830 tx=0x7fca8c009040 comp rx=0 tx=0).stop 2024-09-21T00:29:20.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:20.464+0000 7fcaa9757640 1 -- 172.21.15.112:0/1557548495 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcaa40c5ee0 msgr2=0x7fcaa40c72b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:20.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:20.464+0000 7fcaa9757640 1 --2- 172.21.15.112:0/1557548495 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcaa40c5ee0 0x7fcaa40c72b0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fca9400d770 tx=0x7fca9400dc30 comp rx=0 tx=0).stop 2024-09-21T00:29:20.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:20.464+0000 7fcaa9757640 1 -- 172.21.15.112:0/1557548495 shutdown_connections 2024-09-21T00:29:20.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:20.464+0000 7fcaa9757640 1 --2- 172.21.15.112:0/1557548495 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcaa40ccbf0 0x7fcaa41564c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:20.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:20.464+0000 7fcaa9757640 1 --2- 172.21.15.112:0/1557548495 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcaa40c6a00 0x7fcaa4155f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:20.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:20.464+0000 7fcaa9757640 1 --2- 172.21.15.112:0/1557548495 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fca84078b90 0x7fca8407b0a0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:20.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:20.464+0000 7fcaa9757640 1 --2- 172.21.15.112:0/1557548495 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcaa40c5ee0 0x7fcaa40c72b0 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:20.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:20.464+0000 7fcaa9757640 1 -- 172.21.15.112:0/1557548495 >> 172.21.15.112:0/1557548495 conn(0x7fcaa40b42e0 msgr2=0x7fcaa415b6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:20.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:20.464+0000 7fcaa9757640 1 -- 172.21.15.112:0/1557548495 shutdown_connections 2024-09-21T00:29:20.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:20.464+0000 7fcaa9757640 1 -- 172.21.15.112:0/1557548495 wait complete. 2024-09-21T00:29:20.477 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-21T00:29:20.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:20.591+0000 7fcec4230640 1 -- 172.21.15.112:0/2878935463 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcebc0b4170 msgr2=0x7fcebc0b4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:20.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:20.591+0000 7fcec4230640 1 --2- 172.21.15.112:0/2878935463 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcebc0b4170 0x7fcebc0b4580 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fceb4005fd0 tx=0x7fceb402a810 comp rx=0 tx=0).stop 2024-09-21T00:29:20.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:20.591+0000 7fcec4230640 1 -- 172.21.15.112:0/2878935463 shutdown_connections 2024-09-21T00:29:20.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:20.591+0000 7fcec4230640 1 --2- 172.21.15.112:0/2878935463 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcebc0b4170 0x7fcebc0b4580 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:20.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:20.591+0000 7fcec4230640 1 --2- 172.21.15.112:0/2878935463 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcebc0d02f0 0x7fcebc15f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:20.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:20.591+0000 7fcec4230640 1 -- 172.21.15.112:0/2878935463 >> 172.21.15.112:0/2878935463 conn(0x7fcebc0cf220 msgr2=0x7fcebc0cf630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:20.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:20.591+0000 7fcec4230640 1 -- 172.21.15.112:0/2878935463 shutdown_connections 2024-09-21T00:29:20.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:20.591+0000 7fcec4230640 1 -- 172.21.15.112:0/2878935463 wait complete. 2024-09-21T00:29:20.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:20.592+0000 7fcec4230640 1 Processor -- start 2024-09-21T00:29:20.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:20.592+0000 7fcec4230640 1 -- start start 2024-09-21T00:29:20.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:20.594+0000 7fcec4230640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcebc0b4170 0x7fcebc158370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:20.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:20.594+0000 7fcec4230640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcebc0d02f0 0x7fcebc14eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:20.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:20.594+0000 7fcec4230640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcebc0d0960 0x7fcebc0b6830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:20.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:20.594+0000 7fcec4230640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcebc0ccd60 con 0x7fcebc0d0960 2024-09-21T00:29:20.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:20.594+0000 7fcec4230640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fcebc0ccbe0 con 0x7fcebc0b4170 2024-09-21T00:29:20.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:20.594+0000 7fcec4230640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fcebc0ccee0 con 0x7fcebc0d02f0 2024-09-21T00:29:20.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:20.594+0000 7fcec27cd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcebc0d0960 0x7fcebc0b6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:20.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:20.594+0000 7fcec27cd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcebc0d0960 0x7fcebc0b6830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48608/0 (socket says 172.21.15.112:48608) 2024-09-21T00:29:20.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:20.594+0000 7fcec27cd640 1 -- 172.21.15.112:0/1969731003 learned_addr learned my addr 172.21.15.112:0/1969731003 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:29:20.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:20.594+0000 7fcec1fcc640 1 --2- 172.21.15.112:0/1969731003 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcebc0b4170 0x7fcebc158370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:20.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:20.594+0000 7fcec17cb640 1 --2- 172.21.15.112:0/1969731003 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcebc0d02f0 0x7fcebc14eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:20.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:20.594+0000 7fcec27cd640 1 -- 172.21.15.112:0/1969731003 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcebc0d02f0 msgr2=0x7fcebc14eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:20.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:20.594+0000 7fcec27cd640 1 --2- 172.21.15.112:0/1969731003 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcebc0d02f0 0x7fcebc14eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:20.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:20.594+0000 7fcec27cd640 1 -- 172.21.15.112:0/1969731003 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcebc0b4170 msgr2=0x7fcebc158370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:20.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:20.594+0000 7fcec27cd640 1 --2- 172.21.15.112:0/1969731003 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcebc0b4170 0x7fcebc158370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:20.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:20.594+0000 7fcec27cd640 1 -- 172.21.15.112:0/1969731003 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcebc0b16a0 con 0x7fcebc0d0960 2024-09-21T00:29:20.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:20.594+0000 7fcec27cd640 1 --2- 172.21.15.112:0/1969731003 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcebc0d0960 0x7fcebc0b6830 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fceb4035040 tx=0x7fceb4031650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:20.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:20.595+0000 7fceb27fc640 1 -- 172.21.15.112:0/1969731003 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceb4041070 con 0x7fcebc0d0960 2024-09-21T00:29:20.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:20.595+0000 7fceb27fc640 1 -- 172.21.15.112:0/1969731003 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fceb4031a50 con 0x7fcebc0d0960 2024-09-21T00:29:20.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:20.595+0000 7fceb27fc640 1 -- 172.21.15.112:0/1969731003 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceb4037040 con 0x7fcebc0d0960 2024-09-21T00:29:20.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:20.595+0000 7fcec4230640 1 -- 172.21.15.112:0/1969731003 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcebc0706f0 con 0x7fcebc0d0960 2024-09-21T00:29:20.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:20.595+0000 7fcec4230640 1 -- 172.21.15.112:0/1969731003 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcebc070cb0 con 0x7fcebc0d0960 2024-09-21T00:29:20.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:20.596+0000 7fcec4230640 1 -- 172.21.15.112:0/1969731003 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcebc0cd340 con 0x7fcebc0d0960 2024-09-21T00:29:20.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:20.596+0000 7fceb27fc640 1 -- 172.21.15.112:0/1969731003 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fceb4044070 con 0x7fcebc0d0960 2024-09-21T00:29:20.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:20.597+0000 7fceb27fc640 1 --2- 172.21.15.112:0/1969731003 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fcea0078bc0 0x7fcea007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:20.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:20.597+0000 7fceb27fc640 1 -- 172.21.15.112:0/1969731003 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6101+0+0 (secure 0 0 0) 0x7fceb40bc2a0 con 0x7fcebc0d0960 2024-09-21T00:29:20.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:20.597+0000 7fcec1fcc640 1 --2- 172.21.15.112:0/1969731003 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fcea0078bc0 0x7fcea007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:20.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:20.598+0000 7fcec1fcc640 1 --2- 172.21.15.112:0/1969731003 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fcea0078bc0 0x7fcea007b0d0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fceb8007d60 tx=0x7fceb8007cf0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:20.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:20.601+0000 7fceb27fc640 1 -- 172.21.15.112:0/1969731003 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fceb4083f00 con 0x7fcebc0d0960 2024-09-21T00:29:20.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:20.818+0000 7fcec4230640 1 -- 172.21.15.112:0/1969731003 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fcebc0b1430 con 0x7fcebc0d0960 2024-09-21T00:29:21.775 INFO:tasks.ceph.mds.c.smithi112.stderr:2024-09-21T00:29:21.774+0000 7f07c0c0c640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T00:29:21.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:21.808+0000 7fceb27fc640 1 -- 172.21.15.112:0/1969731003 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v63) ==== 118+0+0 (secure 0 0 0) 0x7fceb40bc0c0 con 0x7fcebc0d0960 2024-09-21T00:29:21.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:21.811+0000 7fcec4230640 1 -- 172.21.15.112:0/1969731003 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fcea0078bc0 msgr2=0x7fcea007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:21.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:21.811+0000 7fcec4230640 1 --2- 172.21.15.112:0/1969731003 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fcea0078bc0 0x7fcea007b0d0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fceb8007d60 tx=0x7fceb8007cf0 comp rx=0 tx=0).stop 2024-09-21T00:29:21.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:21.811+0000 7fcec4230640 1 -- 172.21.15.112:0/1969731003 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcebc0d0960 msgr2=0x7fcebc0b6830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:21.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:21.811+0000 7fcec4230640 1 --2- 172.21.15.112:0/1969731003 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcebc0d0960 0x7fcebc0b6830 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fceb4035040 tx=0x7fceb4031650 comp rx=0 tx=0).stop 2024-09-21T00:29:21.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:21.811+0000 7fcec4230640 1 -- 172.21.15.112:0/1969731003 shutdown_connections 2024-09-21T00:29:21.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:21.811+0000 7fcec4230640 1 --2- 172.21.15.112:0/1969731003 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fcea0078bc0 0x7fcea007b0d0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:21.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:21.811+0000 7fcec4230640 1 --2- 172.21.15.112:0/1969731003 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcebc0d0960 0x7fcebc0b6830 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:21.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:21.811+0000 7fcec4230640 1 --2- 172.21.15.112:0/1969731003 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcebc0d02f0 0x7fcebc14eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:21.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:21.811+0000 7fcec4230640 1 --2- 172.21.15.112:0/1969731003 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcebc0b4170 0x7fcebc158370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:21.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:21.811+0000 7fcec4230640 1 -- 172.21.15.112:0/1969731003 >> 172.21.15.112:0/1969731003 conn(0x7fcebc0cf220 msgr2=0x7fcebc0d2cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:21.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:21.811+0000 7fcec4230640 1 -- 172.21.15.112:0/1969731003 shutdown_connections 2024-09-21T00:29:21.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:21.811+0000 7fcec4230640 1 -- 172.21.15.112:0/1969731003 wait complete. 2024-09-21T00:29:21.827 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-21T00:29:21.827 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-21T00:29:21.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:21.947+0000 7fdc391da640 1 -- 172.21.15.112:0/979842541 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc20003b10 msgr2=0x7fdc20003f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:21.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:21.947+0000 7fdc391da640 1 --2- 172.21.15.112:0/979842541 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc20003b10 0x7fdc20003f80 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fdc24001dc0 tx=0x7fdc20005830 comp rx=0 tx=0).stop 2024-09-21T00:29:21.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:21.947+0000 7fdc3359e640 1 -- 172.21.15.112:0/979842541 reap_dead start 2024-09-21T00:29:21.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:21.947+0000 7fdc391da640 1 -- 172.21.15.112:0/979842541 shutdown_connections 2024-09-21T00:29:21.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:21.947+0000 7fdc391da640 1 -- 172.21.15.112:0/979842541 >> 172.21.15.112:0/979842541 conn(0x7fdc340cee10 msgr2=0x7fdc340cf220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:21.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:21.947+0000 7fdc391da640 1 -- 172.21.15.112:0/979842541 shutdown_connections 2024-09-21T00:29:21.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:21.947+0000 7fdc391da640 1 -- 172.21.15.112:0/979842541 wait complete. 2024-09-21T00:29:21.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:21.949+0000 7fdc391da640 1 Processor -- start 2024-09-21T00:29:21.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:21.949+0000 7fdc391da640 1 -- start start 2024-09-21T00:29:21.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:21.950+0000 7fdc391da640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc20003b10 0x7fdc34070ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:21.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:21.950+0000 7fdc391da640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdc34158830 0x7fdc3414c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:21.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:21.950+0000 7fdc391da640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdc3414c7c0 0x7fdc341f7010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:21.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:21.950+0000 7fdc391da640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fdc340ce820 con 0x7fdc20003b10 2024-09-21T00:29:21.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:21.950+0000 7fdc391da640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fdc340ce6a0 con 0x7fdc34158830 2024-09-21T00:29:21.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:21.950+0000 7fdc391da640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fdc340ce9a0 con 0x7fdc3414c7c0 2024-09-21T00:29:21.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:21.950+0000 7fdc32d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc20003b10 0x7fdc34070ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:21.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:21.950+0000 7fdc32d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc20003b10 0x7fdc34070ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48620/0 (socket says 172.21.15.112:48620) 2024-09-21T00:29:21.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:21.950+0000 7fdc32d9d640 1 -- 172.21.15.112:0/246270089 learned_addr learned my addr 172.21.15.112:0/246270089 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:29:21.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:21.951+0000 7fdc3359e640 1 --2- 172.21.15.112:0/246270089 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdc3414c7c0 0x7fdc341f7010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:21.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:21.951+0000 7fdc2a59c640 1 --2- 172.21.15.112:0/246270089 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdc34158830 0x7fdc3414c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:21.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:21.951+0000 7fdc32d9d640 1 -- 172.21.15.112:0/246270089 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdc3414c7c0 msgr2=0x7fdc341f7010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:21.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:21.951+0000 7fdc32d9d640 1 --2- 172.21.15.112:0/246270089 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdc3414c7c0 0x7fdc341f7010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:21.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:21.951+0000 7fdc32d9d640 1 -- 172.21.15.112:0/246270089 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdc34158830 msgr2=0x7fdc3414c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:21.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:21.951+0000 7fdc32d9d640 1 --2- 172.21.15.112:0/246270089 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdc34158830 0x7fdc3414c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:21.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:21.951+0000 7fdc32d9d640 1 -- 172.21.15.112:0/246270089 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc341f4810 con 0x7fdc20003b10 2024-09-21T00:29:21.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:21.951+0000 7fdc3359e640 1 --2- 172.21.15.112:0/246270089 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdc3414c7c0 0x7fdc341f7010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:29:21.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:21.951+0000 7fdc2a59c640 1 --2- 172.21.15.112:0/246270089 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdc34158830 0x7fdc3414c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:29:21.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:21.951+0000 7fdc32d9d640 1 --2- 172.21.15.112:0/246270089 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc20003b10 0x7fdc34070ca0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fdc2402acc0 tx=0x7fdc24006db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:21.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:21.951+0000 7fdc2bfff640 1 -- 172.21.15.112:0/246270089 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc24006250 con 0x7fdc20003b10 2024-09-21T00:29:21.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:21.951+0000 7fdc2bfff640 1 -- 172.21.15.112:0/246270089 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdc240064e0 con 0x7fdc20003b10 2024-09-21T00:29:21.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:21.951+0000 7fdc391da640 1 -- 172.21.15.112:0/246270089 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc340c9650 con 0x7fdc20003b10 2024-09-21T00:29:21.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:21.951+0000 7fdc2bfff640 1 -- 172.21.15.112:0/246270089 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc240068b0 con 0x7fdc20003b10 2024-09-21T00:29:21.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:21.951+0000 7fdc391da640 1 -- 172.21.15.112:0/246270089 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc340c9c10 con 0x7fdc20003b10 2024-09-21T00:29:21.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:21.952+0000 7fdc391da640 1 -- 172.21.15.112:0/246270089 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc3415bf50 con 0x7fdc20003b10 2024-09-21T00:29:21.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:21.953+0000 7fdc2bfff640 1 -- 172.21.15.112:0/246270089 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fdc24033480 con 0x7fdc20003b10 2024-09-21T00:29:21.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:21.953+0000 7fdc2bfff640 1 --2- 172.21.15.112:0/246270089 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fdc14078d30 0x7fdc1407b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:21.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:21.953+0000 7fdc2bfff640 1 -- 172.21.15.112:0/246270089 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6101+0+0 (secure 0 0 0) 0x7fdc240bb560 con 0x7fdc20003b10 2024-09-21T00:29:21.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:21.954+0000 7fdc2a59c640 1 --2- 172.21.15.112:0/246270089 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fdc14078d30 0x7fdc1407b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:21.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:21.954+0000 7fdc2a59c640 1 --2- 172.21.15.112:0/246270089 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fdc14078d30 0x7fdc1407b240 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fdc34072140 tx=0x7fdc1c005f90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:21.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:21.958+0000 7fdc2bfff640 1 -- 172.21.15.112:0/246270089 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fdc3415bf50 con 0x7fdc20003b10 2024-09-21T00:29:22.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:22.170+0000 7fdc391da640 1 -- 172.21.15.112:0/246270089 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fdc340c9440 con 0x7fdc20003b10 2024-09-21T00:29:22.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:22.810+0000 7fdc2bfff640 1 -- 172.21.15.112:0/246270089 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v64) ==== 122+0+0 (secure 0 0 0) 0x7fdc240831c0 con 0x7fdc20003b10 2024-09-21T00:29:22.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:22.812+0000 7fdc391da640 1 -- 172.21.15.112:0/246270089 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fdc14078d30 msgr2=0x7fdc1407b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:22.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:22.812+0000 7fdc391da640 1 --2- 172.21.15.112:0/246270089 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fdc14078d30 0x7fdc1407b240 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fdc34072140 tx=0x7fdc1c005f90 comp rx=0 tx=0).stop 2024-09-21T00:29:22.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:22.812+0000 7fdc391da640 1 -- 172.21.15.112:0/246270089 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc20003b10 msgr2=0x7fdc34070ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:22.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:22.812+0000 7fdc391da640 1 --2- 172.21.15.112:0/246270089 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc20003b10 0x7fdc34070ca0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fdc2402acc0 tx=0x7fdc24006db0 comp rx=0 tx=0).stop 2024-09-21T00:29:22.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:22.813+0000 7fdc391da640 1 -- 172.21.15.112:0/246270089 shutdown_connections 2024-09-21T00:29:22.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:22.813+0000 7fdc391da640 1 --2- 172.21.15.112:0/246270089 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdc3414c7c0 0x7fdc341f7010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:22.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:22.813+0000 7fdc391da640 1 --2- 172.21.15.112:0/246270089 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdc34158830 0x7fdc3414c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:22.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:22.813+0000 7fdc391da640 1 --2- 172.21.15.112:0/246270089 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fdc14078d30 0x7fdc1407b240 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:22.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:22.813+0000 7fdc391da640 1 --2- 172.21.15.112:0/246270089 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc20003b10 0x7fdc34070ca0 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:22.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:22.813+0000 7fdc391da640 1 -- 172.21.15.112:0/246270089 >> 172.21.15.112:0/246270089 conn(0x7fdc340cee10 msgr2=0x7fdc3415b840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:22.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:22.813+0000 7fdc391da640 1 -- 172.21.15.112:0/246270089 shutdown_connections 2024-09-21T00:29:22.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:22.813+0000 7fdc391da640 1 -- 172.21.15.112:0/246270089 wait complete. 2024-09-21T00:29:22.827 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-21T00:29:22.828 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T00:29:22.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:22.942+0000 7f0ed381d640 1 -- 172.21.15.112:0/3116316661 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ecc0d0c40 msgr2=0x7f0ecc06d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:22.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:22.942+0000 7f0ed381d640 1 --2- 172.21.15.112:0/3116316661 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ecc0d0c40 0x7f0ecc06d4f0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f0ecc072960 tx=0x7f0ec402a940 comp rx=0 tx=0).stop 2024-09-21T00:29:22.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:22.942+0000 7f0ed381d640 1 -- 172.21.15.112:0/3116316661 shutdown_connections 2024-09-21T00:29:22.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:22.942+0000 7f0ed381d640 1 --2- 172.21.15.112:0/3116316661 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ecc0d0c40 0x7f0ecc06d4f0 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:22.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:22.942+0000 7f0ed381d640 1 --2- 172.21.15.112:0/3116316661 >> v2:172.21.15.187:3300/0 conn(0x7f0ecc0cc270 0x7f0ecc0d0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:22.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:22.942+0000 7f0ed381d640 1 -- 172.21.15.112:0/3116316661 >> 172.21.15.112:0/3116316661 conn(0x7f0ecc0b3580 msgr2=0x7f0ecc0b3990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:22.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:22.942+0000 7f0ed381d640 1 -- 172.21.15.112:0/3116316661 shutdown_connections 2024-09-21T00:29:22.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:22.942+0000 7f0ed381d640 1 -- 172.21.15.112:0/3116316661 wait complete. 2024-09-21T00:29:22.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:22.943+0000 7f0ed381d640 1 Processor -- start 2024-09-21T00:29:22.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:22.944+0000 7f0ed381d640 1 -- start start 2024-09-21T00:29:22.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:22.945+0000 7f0ed381d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ecc0cc270 0x7f0ecc1582c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:22.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:22.945+0000 7f0ed381d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0ecc0ce9e0 0x7f0ecc1e8270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:22.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:22.945+0000 7f0ed381d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ecc0d0c40 0x7f0ecc1e87b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:22.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:22.945+0000 7f0ed381d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0ecc0705e0 con 0x7f0ecc0cc270 2024-09-21T00:29:22.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:22.945+0000 7f0ed381d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0ecc070460 con 0x7f0ecc0d0c40 2024-09-21T00:29:22.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:22.945+0000 7f0ed381d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f0ecc0af950 con 0x7f0ecc0ce9e0 2024-09-21T00:29:22.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:22.945+0000 7f0ed15b9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ecc0cc270 0x7f0ecc1582c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:22.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:22.945+0000 7f0ed15b9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ecc0cc270 0x7f0ecc1582c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48644/0 (socket says 172.21.15.112:48644) 2024-09-21T00:29:22.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:22.945+0000 7f0ed15b9640 1 -- 172.21.15.112:0/778624548 learned_addr learned my addr 172.21.15.112:0/778624548 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:29:22.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:22.945+0000 7f0ed1dba640 1 --2- 172.21.15.112:0/778624548 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ecc0d0c40 0x7f0ecc1e87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:22.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:22.945+0000 7f0ed0db8640 1 --2- 172.21.15.112:0/778624548 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0ecc0ce9e0 0x7f0ecc1e8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:22.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:22.945+0000 7f0ed15b9640 1 -- 172.21.15.112:0/778624548 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0ecc0ce9e0 msgr2=0x7f0ecc1e8270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:22.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:22.945+0000 7f0ed15b9640 1 --2- 172.21.15.112:0/778624548 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0ecc0ce9e0 0x7f0ecc1e8270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:22.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:22.945+0000 7f0ed15b9640 1 -- 172.21.15.112:0/778624548 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ecc0d0c40 msgr2=0x7f0ecc1e87b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:22.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:22.945+0000 7f0ed15b9640 1 --2- 172.21.15.112:0/778624548 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ecc0d0c40 0x7f0ecc1e87b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:22.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:22.945+0000 7f0ed15b9640 1 -- 172.21.15.112:0/778624548 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ecc09d700 con 0x7f0ecc0cc270 2024-09-21T00:29:22.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:22.945+0000 7f0ed15b9640 1 --2- 172.21.15.112:0/778624548 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ecc0cc270 0x7f0ecc1582c0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f0ec800b410 tx=0x7f0ec800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:22.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:22.946+0000 7f0ec1ffb640 1 -- 172.21.15.112:0/778624548 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ec8015020 con 0x7f0ecc0cc270 2024-09-21T00:29:22.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:22.946+0000 7f0ec1ffb640 1 -- 172.21.15.112:0/778624548 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0ec8011940 con 0x7f0ecc0cc270 2024-09-21T00:29:22.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:22.946+0000 7f0ed381d640 1 -- 172.21.15.112:0/778624548 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ecc1f9ca0 con 0x7f0ecc0cc270 2024-09-21T00:29:22.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:22.946+0000 7f0ec1ffb640 1 -- 172.21.15.112:0/778624548 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ec8011d90 con 0x7f0ecc0cc270 2024-09-21T00:29:22.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:22.946+0000 7f0ed381d640 1 -- 172.21.15.112:0/778624548 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ecc09bcb0 con 0x7f0ecc0cc270 2024-09-21T00:29:22.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:22.947+0000 7f0ed381d640 1 -- 172.21.15.112:0/778624548 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ecc09a210 con 0x7f0ecc0cc270 2024-09-21T00:29:22.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:22.947+0000 7f0ec1ffb640 1 -- 172.21.15.112:0/778624548 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0ec8018020 con 0x7f0ecc0cc270 2024-09-21T00:29:22.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:22.948+0000 7f0ec1ffb640 1 --2- 172.21.15.112:0/778624548 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f0eb0078bc0 0x7f0eb007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:22.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:22.948+0000 7f0ec1ffb640 1 -- 172.21.15.112:0/778624548 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6101+0+0 (secure 0 0 0) 0x7f0ec809ade0 con 0x7f0ecc0cc270 2024-09-21T00:29:22.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:22.948+0000 7f0ed0db8640 1 --2- 172.21.15.112:0/778624548 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f0eb0078bc0 0x7f0eb007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:22.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:22.949+0000 7f0ed0db8640 1 --2- 172.21.15.112:0/778624548 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f0eb0078bc0 0x7f0eb007b0d0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc007fd0 tx=0x7f0ebc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:22.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:22.952+0000 7f0ec1ffb640 1 -- 172.21.15.112:0/778624548 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f0ec801f020 con 0x7f0ecc0cc270 2024-09-21T00:29:23.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.164+0000 7f0ed381d640 1 -- 172.21.15.112:0/778624548 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0ecc06dbc0 con 0x7f0ecc0cc270 2024-09-21T00:29:23.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.165+0000 7f0ec1ffb640 1 -- 172.21.15.112:0/778624548 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) ==== 94+0+5237 (secure 0 0 0) 0x7f0ec8063120 con 0x7f0ecc0cc270 2024-09-21T00:29:23.167 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:29:23.167 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":64,"btime":"2024-09-21T00:29:22:809921+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5804,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/3818262153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":3818262153},{"type":"v1","addr":"172.21.15.187:6835","nonce":3818262153}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59},{"gid":6214,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6837/1321768683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":1321768683},{"type":"v1","addr":"172.21.15.187:6837","nonce":1321768683}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59},{"gid":6801,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6834/4037012343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":4037012343},{"type":"v1","addr":"172.21.15.112:6834","nonce":4037012343}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T00:29:19.145794+0000","modified":"2024-09-21T00:29:22.172462+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6804},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6804":{"gid":6804,"name":"c","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6835/2797048793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":2797048793},{"type":"v1","addr":"172.21.15.112:6835","nonce":2797048793}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6804,"qdb_cluster":[6804]},"id":7}]} 2024-09-21T00:29:23.167 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 64 2024-09-21T00:29:23.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.167+0000 7f0ed381d640 1 -- 172.21.15.112:0/778624548 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f0eb0078bc0 msgr2=0x7f0eb007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:23.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.167+0000 7f0ed381d640 1 --2- 172.21.15.112:0/778624548 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f0eb0078bc0 0x7f0eb007b0d0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc007fd0 tx=0x7f0ebc009040 comp rx=0 tx=0).stop 2024-09-21T00:29:23.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.167+0000 7f0ed381d640 1 -- 172.21.15.112:0/778624548 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ecc0cc270 msgr2=0x7f0ecc1582c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:23.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.167+0000 7f0ed381d640 1 --2- 172.21.15.112:0/778624548 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ecc0cc270 0x7f0ecc1582c0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f0ec800b410 tx=0x7f0ec800b8d0 comp rx=0 tx=0).stop 2024-09-21T00:29:23.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.167+0000 7f0ed381d640 1 -- 172.21.15.112:0/778624548 shutdown_connections 2024-09-21T00:29:23.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.167+0000 7f0ed381d640 1 --2- 172.21.15.112:0/778624548 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ecc0d0c40 0x7f0ecc1e87b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:23.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.167+0000 7f0ed381d640 1 --2- 172.21.15.112:0/778624548 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0ecc0ce9e0 0x7f0ecc1e8270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:23.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.167+0000 7f0ed381d640 1 --2- 172.21.15.112:0/778624548 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f0eb0078bc0 0x7f0eb007b0d0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:23.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.167+0000 7f0ed381d640 1 --2- 172.21.15.112:0/778624548 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ecc0cc270 0x7f0ecc1582c0 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:23.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.167+0000 7f0ed381d640 1 -- 172.21.15.112:0/778624548 >> 172.21.15.112:0/778624548 conn(0x7f0ecc0b3580 msgr2=0x7f0ecc0ad040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:23.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.167+0000 7f0ed381d640 1 -- 172.21.15.112:0/778624548 shutdown_connections 2024-09-21T00:29:23.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.167+0000 7f0ed381d640 1 -- 172.21.15.112:0/778624548 wait complete. 2024-09-21T00:29:23.180 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T00:29:23.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.290+0000 7f8630427640 1 -- 172.21.15.112:0/4003996948 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f86280ccbf0 msgr2=0x7f86280cd000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:23.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.290+0000 7f8630427640 1 --2- 172.21.15.112:0/4003996948 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f86280ccbf0 0x7f86280cd000 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f86280cde90 tx=0x7f861802a790 comp rx=0 tx=0).stop 2024-09-21T00:29:23.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.290+0000 7f8630427640 1 -- 172.21.15.112:0/4003996948 shutdown_connections 2024-09-21T00:29:23.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.290+0000 7f8630427640 1 --2- 172.21.15.112:0/4003996948 >> v2:172.21.15.187:3300/0 conn(0x7f86280c6a00 0x7f86280b2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:23.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.290+0000 7f8630427640 1 --2- 172.21.15.112:0/4003996948 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f86280ccbf0 0x7f86280cd000 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:23.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.290+0000 7f8630427640 1 -- 172.21.15.112:0/4003996948 >> 172.21.15.112:0/4003996948 conn(0x7f86280b42e0 msgr2=0x7f86280b4720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:23.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.290+0000 7f8630427640 1 -- 172.21.15.112:0/4003996948 shutdown_connections 2024-09-21T00:29:23.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.290+0000 7f8630427640 1 -- 172.21.15.112:0/4003996948 wait complete. 2024-09-21T00:29:23.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.292+0000 7f8630427640 1 Processor -- start 2024-09-21T00:29:23.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.292+0000 7f8630427640 1 -- start start 2024-09-21T00:29:23.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.293+0000 7f8630427640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f86280c5ee0 0x7f86281512f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:23.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.293+0000 7f8630427640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f86280c6a00 0x7f862814fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:23.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.293+0000 7f8630427640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f86280ccbf0 0x7f86281f6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:23.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.293+0000 7f8630427640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f86280b3610 con 0x7f86280ccbf0 2024-09-21T00:29:23.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.293+0000 7f8630427640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f86280b3490 con 0x7f86280c5ee0 2024-09-21T00:29:23.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.293+0000 7f8630427640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f86280b3790 con 0x7f86280c6a00 2024-09-21T00:29:23.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.293+0000 7f862e9c4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f86280ccbf0 0x7f86281f6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:23.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.293+0000 7f862e9c4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f86280ccbf0 0x7f86281f6aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48668/0 (socket says 172.21.15.112:48668) 2024-09-21T00:29:23.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.293+0000 7f862e9c4640 1 -- 172.21.15.112:0/3114407626 learned_addr learned my addr 172.21.15.112:0/3114407626 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:29:23.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.293+0000 7f862e1c3640 1 --2- 172.21.15.112:0/3114407626 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f86280c5ee0 0x7f86281512f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:23.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.293+0000 7f862d9c2640 1 --2- 172.21.15.112:0/3114407626 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f86280c6a00 0x7f862814fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:23.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.294+0000 7f862e9c4640 1 -- 172.21.15.112:0/3114407626 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f86280c6a00 msgr2=0x7f862814fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:23.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.294+0000 7f862e9c4640 1 --2- 172.21.15.112:0/3114407626 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f86280c6a00 0x7f862814fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:23.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.294+0000 7f862e9c4640 1 -- 172.21.15.112:0/3114407626 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f86280c5ee0 msgr2=0x7f86281512f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:23.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.294+0000 7f862e9c4640 1 --2- 172.21.15.112:0/3114407626 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f86280c5ee0 0x7f86281512f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:23.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.294+0000 7f862e9c4640 1 -- 172.21.15.112:0/3114407626 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8628150750 con 0x7f86280ccbf0 2024-09-21T00:29:23.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.294+0000 7f862e9c4640 1 --2- 172.21.15.112:0/3114407626 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f86280ccbf0 0x7f86281f6aa0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f862000c940 tx=0x7f862000ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:23.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.294+0000 7f8626ffd640 1 -- 172.21.15.112:0/3114407626 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8620004460 con 0x7f86280ccbf0 2024-09-21T00:29:23.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.294+0000 7f8626ffd640 1 -- 172.21.15.112:0/3114407626 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f86200077a0 con 0x7f86280ccbf0 2024-09-21T00:29:23.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.294+0000 7f8626ffd640 1 -- 172.21.15.112:0/3114407626 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8620007bd0 con 0x7f86280ccbf0 2024-09-21T00:29:23.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.294+0000 7f8630427640 1 -- 172.21.15.112:0/3114407626 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8628150a30 con 0x7f86280ccbf0 2024-09-21T00:29:23.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.294+0000 7f8630427640 1 -- 172.21.15.112:0/3114407626 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8628153330 con 0x7f86280ccbf0 2024-09-21T00:29:23.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.295+0000 7f8630427640 1 -- 172.21.15.112:0/3114407626 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86280c6790 con 0x7f86280ccbf0 2024-09-21T00:29:23.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.296+0000 7f8626ffd640 1 -- 172.21.15.112:0/3114407626 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f86200035f0 con 0x7f86280ccbf0 2024-09-21T00:29:23.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.296+0000 7f8626ffd640 1 --2- 172.21.15.112:0/3114407626 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f8614078b90 0x7f861407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:23.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.296+0000 7f8626ffd640 1 -- 172.21.15.112:0/3114407626 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6101+0+0 (secure 0 0 0) 0x7f862009b560 con 0x7f86280ccbf0 2024-09-21T00:29:23.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.296+0000 7f862e1c3640 1 --2- 172.21.15.112:0/3114407626 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f8614078b90 0x7f861407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:23.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.297+0000 7f862e1c3640 1 --2- 172.21.15.112:0/3114407626 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f8614078b90 0x7f861407b0a0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f8618033000 tx=0x7f8618004ab0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:23.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.301+0000 7f8626ffd640 1 -- 172.21.15.112:0/3114407626 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f86280c6790 con 0x7f86280ccbf0 2024-09-21T00:29:23.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.479+0000 7f8630427640 1 -- 172.21.15.112:0/3114407626 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f86280cd000 con 0x7f86280ccbf0 2024-09-21T00:29:23.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.480+0000 7f8626ffd640 1 -- 172.21.15.112:0/3114407626 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+16957 (secure 0 0 0) 0x7f8620013030 con 0x7f86280ccbf0 2024-09-21T00:29:23.482 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:29:23.482 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":94,"fsid":"8b34692b-dc65-49b2-a08f-26f707b96ac5","created":"2024-09-21T00:25:22.788490+0000","modified":"2024-09-21T00:29:20.797323+0000","last_up_change":"2024-09-21T00:25:36.161835+0000","last_in_change":"2024-09-21T00:25:26.346500+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T00:25:37.105006+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-09-21T00:29:16.794043+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-09-21T00:29:18.114604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9c575086-67d2-4de2-b01b-d44dcc21d604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6809","nonce":755760621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6811","nonce":755760621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6815","nonce":755760621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6813","nonce":755760621}]},"public_addr":"172.21.15.112:6809/755760621","cluster_addr":"172.21.15.112:6811/755760621","heartbeat_back_addr":"172.21.15.112:6815/755760621","heartbeat_front_addr":"172.21.15.112:6813/755760621","state":["exists","up"]},{"osd":1,"uuid":"a983c227-a3ce-4bdd-82f1-438ba7e3ecd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6801","nonce":344932592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6803","nonce":344932592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6807","nonce":344932592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6805","nonce":344932592}]},"public_addr":"172.21.15.112:6801/344932592","cluster_addr":"172.21.15.112:6803/344932592","heartbeat_back_addr":"172.21.15.112:6807/344932592","heartbeat_front_addr":"172.21.15.112:6805/344932592","state":["exists","up"]},{"osd":2,"uuid":"1193c1d8-bfff-4a57-8200-5f69bf1ea3af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6825","nonce":407933616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6827","nonce":407933616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6831","nonce":407933616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6829","nonce":407933616}]},"public_addr":"172.21.15.112:6825/407933616","cluster_addr":"172.21.15.112:6827/407933616","heartbeat_back_addr":"172.21.15.112:6831/407933616","heartbeat_front_addr":"172.21.15.112:6829/407933616","state":["exists","up"]},{"osd":3,"uuid":"af04330a-36ad-40cc-a8dd-fa259b221de5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6817","nonce":4120164806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6819","nonce":4120164806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6823","nonce":4120164806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6821","nonce":4120164806}]},"public_addr":"172.21.15.112:6817/4120164806","cluster_addr":"172.21.15.112:6819/4120164806","heartbeat_back_addr":"172.21.15.112:6823/4120164806","heartbeat_front_addr":"172.21.15.112:6821/4120164806","state":["exists","up"]},{"osd":4,"uuid":"5b893111-292c-4c5a-b184-53ad38c7199e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6803","nonce":522077560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6805","nonce":522077560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6809","nonce":522077560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6807","nonce":522077560}]},"public_addr":"172.21.15.187:6803/522077560","cluster_addr":"172.21.15.187:6805/522077560","heartbeat_back_addr":"172.21.15.187:6809/522077560","heartbeat_front_addr":"172.21.15.187:6807/522077560","state":["exists","up"]},{"osd":5,"uuid":"c0b8e011-45fe-4c5e-925e-6be2a7f1868a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6827","nonce":3593144721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6829","nonce":3593144721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6833","nonce":3593144721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6831","nonce":3593144721}]},"public_addr":"172.21.15.187:6827/3593144721","cluster_addr":"172.21.15.187:6829/3593144721","heartbeat_back_addr":"172.21.15.187:6833/3593144721","heartbeat_front_addr":"172.21.15.187:6831/3593144721","state":["exists","up"]},{"osd":6,"uuid":"87d2d650-7489-4593-98a3-a1f433ce8dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6811","nonce":1494387657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6813","nonce":1494387657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6817","nonce":1494387657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6815","nonce":1494387657}]},"public_addr":"172.21.15.187:6811/1494387657","cluster_addr":"172.21.15.187:6813/1494387657","heartbeat_back_addr":"172.21.15.187:6817/1494387657","heartbeat_front_addr":"172.21.15.187:6815/1494387657","state":["exists","up"]},{"osd":7,"uuid":"c6b4f8e2-a1ee-4438-aa37-67ff4b6ef1f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6819","nonce":1624699842}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6821","nonce":1624699842}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6825","nonce":1624699842}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6823","nonce":1624699842}]},"public_addr":"172.21.15.187:6819/1624699842","cluster_addr":"172.21.15.187:6821/1624699842","heartbeat_back_addr":"172.21.15.187:6825/1624699842","heartbeat_front_addr":"172.21.15.187:6823/1624699842","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.873162+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.400506+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.664317+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.159791+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.202630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.113212+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.384786+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.448441+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T00:29:23.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.482+0000 7f8630427640 1 -- 172.21.15.112:0/3114407626 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f8614078b90 msgr2=0x7f861407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:23.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.482+0000 7f8630427640 1 --2- 172.21.15.112:0/3114407626 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f8614078b90 0x7f861407b0a0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f8618033000 tx=0x7f8618004ab0 comp rx=0 tx=0).stop 2024-09-21T00:29:23.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.482+0000 7f8630427640 1 -- 172.21.15.112:0/3114407626 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f86280ccbf0 msgr2=0x7f86281f6aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:23.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.482+0000 7f8630427640 1 --2- 172.21.15.112:0/3114407626 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f86280ccbf0 0x7f86281f6aa0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f862000c940 tx=0x7f862000ce00 comp rx=0 tx=0).stop 2024-09-21T00:29:23.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.483+0000 7f8630427640 1 -- 172.21.15.112:0/3114407626 shutdown_connections 2024-09-21T00:29:23.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.483+0000 7f8630427640 1 --2- 172.21.15.112:0/3114407626 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f8614078b90 0x7f861407b0a0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:23.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.483+0000 7f8630427640 1 --2- 172.21.15.112:0/3114407626 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f86280ccbf0 0x7f86281f6aa0 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:23.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.483+0000 7f8630427640 1 --2- 172.21.15.112:0/3114407626 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f86280c6a00 0x7f862814fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:23.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.483+0000 7f8630427640 1 --2- 172.21.15.112:0/3114407626 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f86280c5ee0 0x7f86281512f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:23.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.483+0000 7f8630427640 1 -- 172.21.15.112:0/3114407626 >> 172.21.15.112:0/3114407626 conn(0x7f86280b42e0 msgr2=0x7f8628072010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:23.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.483+0000 7f8630427640 1 -- 172.21.15.112:0/3114407626 shutdown_connections 2024-09-21T00:29:23.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.483+0000 7f8630427640 1 -- 172.21.15.112:0/3114407626 wait complete. 2024-09-21T00:29:23.496 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T00:29:23.496 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T00:29:23.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.607+0000 7fa04a87e640 1 -- 172.21.15.112:0/1146085030 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0440b5ab0 msgr2=0x7fa0440b0730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:23.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.607+0000 7fa04a87e640 1 --2- 172.21.15.112:0/1146085030 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0440b5ab0 0x7fa0440b0730 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fa0440ce670 tx=0x7fa03402a560 comp rx=0 tx=0).stop 2024-09-21T00:29:23.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.607+0000 7fa04a87e640 1 -- 172.21.15.112:0/1146085030 shutdown_connections 2024-09-21T00:29:23.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.607+0000 7fa04a87e640 1 --2- 172.21.15.112:0/1146085030 >> v2:172.21.15.187:3300/0 conn(0x7fa04415f7f0 0x7fa0440cf0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:23.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.607+0000 7fa04a87e640 1 --2- 172.21.15.112:0/1146085030 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0440b5ab0 0x7fa0440b0730 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:23.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.607+0000 7fa04a87e640 1 -- 172.21.15.112:0/1146085030 >> 172.21.15.112:0/1146085030 conn(0x7fa044073fe0 msgr2=0x7fa0440743f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:23.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.607+0000 7fa04a87e640 1 -- 172.21.15.112:0/1146085030 shutdown_connections 2024-09-21T00:29:23.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.607+0000 7fa04a87e640 1 -- 172.21.15.112:0/1146085030 wait complete. 2024-09-21T00:29:23.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.609+0000 7fa04a87e640 1 Processor -- start 2024-09-21T00:29:23.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.609+0000 7fa04a87e640 1 -- start start 2024-09-21T00:29:23.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.610+0000 7fa04a87e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa044144dd0 0x7fa044142750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:23.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.610+0000 7fa04a87e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa044142c90 0x7fa0441405d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:23.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.610+0000 7fa04a87e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0441f5730 0x7fa044140b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:23.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.610+0000 7fa04a87e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa0440cfdd0 con 0x7fa0441f5730 2024-09-21T00:29:23.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.610+0000 7fa04a87e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa0440cfc50 con 0x7fa044142c90 2024-09-21T00:29:23.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.610+0000 7fa04a87e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa0440cff50 con 0x7fa044144dd0 2024-09-21T00:29:23.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.610+0000 7fa048e1b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0441f5730 0x7fa044140b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:23.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.610+0000 7fa048e1b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0441f5730 0x7fa044140b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48686/0 (socket says 172.21.15.112:48686) 2024-09-21T00:29:23.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.610+0000 7fa048e1b640 1 -- 172.21.15.112:0/444547612 learned_addr learned my addr 172.21.15.112:0/444547612 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:29:23.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.610+0000 7fa043fff640 1 --2- 172.21.15.112:0/444547612 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa044144dd0 0x7fa044142750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:23.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.610+0000 7fa0437fe640 1 --2- 172.21.15.112:0/444547612 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa044142c90 0x7fa0441405d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:23.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.610+0000 7fa048e1b640 1 -- 172.21.15.112:0/444547612 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa044144dd0 msgr2=0x7fa044142750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:23.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.610+0000 7fa048e1b640 1 --2- 172.21.15.112:0/444547612 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa044144dd0 0x7fa044142750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:23.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.610+0000 7fa048e1b640 1 -- 172.21.15.112:0/444547612 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa044142c90 msgr2=0x7fa0441405d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:23.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.610+0000 7fa048e1b640 1 --2- 172.21.15.112:0/444547612 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa044142c90 0x7fa0441405d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:23.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.610+0000 7fa048e1b640 1 -- 172.21.15.112:0/444547612 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0441e8770 con 0x7fa0441f5730 2024-09-21T00:29:23.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.611+0000 7fa048e1b640 1 --2- 172.21.15.112:0/444547612 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0441f5730 0x7fa044140b10 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fa03c00b9e0 tx=0x7fa03c00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:23.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.611+0000 7fa040ff9640 1 -- 172.21.15.112:0/444547612 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa03c019070 con 0x7fa0441f5730 2024-09-21T00:29:23.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.611+0000 7fa040ff9640 1 -- 172.21.15.112:0/444547612 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa03c00cc30 con 0x7fa0441f5730 2024-09-21T00:29:23.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.611+0000 7fa04a87e640 1 -- 172.21.15.112:0/444547612 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0441e8a50 con 0x7fa0441f5730 2024-09-21T00:29:23.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.611+0000 7fa040ff9640 1 -- 172.21.15.112:0/444547612 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa03c0146a0 con 0x7fa0441f5730 2024-09-21T00:29:23.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.611+0000 7fa04a87e640 1 -- 172.21.15.112:0/444547612 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0440aeac0 con 0x7fa0441f5730 2024-09-21T00:29:23.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.612+0000 7fa04a87e640 1 -- 172.21.15.112:0/444547612 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0440d0480 con 0x7fa0441f5730 2024-09-21T00:29:23.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.612+0000 7fa040ff9640 1 -- 172.21.15.112:0/444547612 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa03c01c070 con 0x7fa0441f5730 2024-09-21T00:29:23.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.613+0000 7fa040ff9640 1 --2- 172.21.15.112:0/444547612 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fa030078bc0 0x7fa03007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:23.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.613+0000 7fa040ff9640 1 -- 172.21.15.112:0/444547612 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6101+0+0 (secure 0 0 0) 0x7fa03c09aa70 con 0x7fa0441f5730 2024-09-21T00:29:23.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.613+0000 7fa043fff640 1 --2- 172.21.15.112:0/444547612 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fa030078bc0 0x7fa03007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:23.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.614+0000 7fa043fff640 1 --2- 172.21.15.112:0/444547612 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fa030078bc0 0x7fa03007b0d0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fa034033000 tx=0x7fa034004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:23.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.618+0000 7fa040ff9640 1 -- 172.21.15.112:0/444547612 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa03c00ce20 con 0x7fa0441f5730 2024-09-21T00:29:23.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.793+0000 7fa04a87e640 1 -- 172.21.15.112:0/444547612 --> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa0441f5ca0 con 0x7fa030078bc0 2024-09-21T00:29:23.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.798+0000 7fa040ff9640 1 -- 172.21.15.112:0/444547612 <== mgr.4103 v2:172.21.15.187:6800/2921522906 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+250705 (secure 0 0 0) 0x7fa0441f5ca0 con 0x7fa030078bc0 2024-09-21T00:29:23.802 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:29:23.803 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2024-09-21T00:29:23.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.801+0000 7fa04a87e640 1 -- 172.21.15.112:0/444547612 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fa030078bc0 msgr2=0x7fa03007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:23.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.801+0000 7fa04a87e640 1 --2- 172.21.15.112:0/444547612 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fa030078bc0 0x7fa03007b0d0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fa034033000 tx=0x7fa034004730 comp rx=0 tx=0).stop 2024-09-21T00:29:23.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.801+0000 7fa04a87e640 1 -- 172.21.15.112:0/444547612 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0441f5730 msgr2=0x7fa044140b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:23.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.801+0000 7fa04a87e640 1 --2- 172.21.15.112:0/444547612 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0441f5730 0x7fa044140b10 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fa03c00b9e0 tx=0x7fa03c00bea0 comp rx=0 tx=0).stop 2024-09-21T00:29:23.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.801+0000 7fa04a87e640 1 -- 172.21.15.112:0/444547612 shutdown_connections 2024-09-21T00:29:23.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.801+0000 7fa04a87e640 1 --2- 172.21.15.112:0/444547612 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fa030078bc0 0x7fa03007b0d0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:23.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.801+0000 7fa04a87e640 1 --2- 172.21.15.112:0/444547612 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0441f5730 0x7fa044140b10 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:23.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.801+0000 7fa04a87e640 1 --2- 172.21.15.112:0/444547612 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa044142c90 0x7fa0441405d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:23.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.801+0000 7fa04a87e640 1 --2- 172.21.15.112:0/444547612 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa044144dd0 0x7fa044142750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:23.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.801+0000 7fa04a87e640 1 -- 172.21.15.112:0/444547612 >> 172.21.15.112:0/444547612 conn(0x7fa044073fe0 msgr2=0x7fa044143ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:23.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.801+0000 7fa04a87e640 1 -- 172.21.15.112:0/444547612 shutdown_connections 2024-09-21T00:29:23.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.801+0000 7fa04a87e640 1 -- 172.21.15.112:0/444547612 wait complete. 2024-09-21T00:29:23.814 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":202,"stamp":"2024-09-21T00:29:23.144482+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592744,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":137,"num_write_kb":1635,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":105,"ondisk_log_size":105,"up":180,"acting":180,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":108,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":259712,"kb_used_data":10192,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754715008,"statfs":{"total":773094113280,"available":772828168192,"internally_reserved":0,"allocated":10436608,"data_stored":5343879,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2376,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.368600"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.807159+0000","last_change":"2024-09-21T00:29:19.813350+0000","last_active":"2024-09-21T00:29:20.807159+0000","last_peered":"2024-09-21T00:29:20.807159+0000","last_clean":"2024-09-21T00:29:20.807159+0000","last_became_active":"2024-09-21T00:29:19.812123+0000","last_became_peered":"2024-09-21T00:29:19.812123+0000","last_unstale":"2024-09-21T00:29:20.807159+0000","last_undegraded":"2024-09-21T00:29:20.807159+0000","last_fullsized":"2024-09-21T00:29:20.807159+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:17:13.647509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022537+0000","last_change":"2024-09-21T00:29:19.807870+0000","last_active":"2024-09-21T00:29:21.022537+0000","last_peered":"2024-09-21T00:29:21.022537+0000","last_clean":"2024-09-21T00:29:21.022537+0000","last_became_active":"2024-09-21T00:29:19.807563+0000","last_became_peered":"2024-09-21T00:29:19.807563+0000","last_unstale":"2024-09-21T00:29:21.022537+0000","last_undegraded":"2024-09-21T00:29:21.022537+0000","last_fullsized":"2024-09-21T00:29:21.022537+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:26:29.645186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.807064+0000","last_change":"2024-09-21T00:29:19.804321+0000","last_active":"2024-09-21T00:29:20.807064+0000","last_peered":"2024-09-21T00:29:20.807064+0000","last_clean":"2024-09-21T00:29:20.807064+0000","last_became_active":"2024-09-21T00:29:19.804013+0000","last_became_peered":"2024-09-21T00:29:19.804013+0000","last_unstale":"2024-09-21T00:29:20.807064+0000","last_undegraded":"2024-09-21T00:29:20.807064+0000","last_fullsized":"2024-09-21T00:29:20.807064+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:47:15.426188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.021814+0000","last_change":"2024-09-21T00:29:19.811466+0000","last_active":"2024-09-21T00:29:21.021814+0000","last_peered":"2024-09-21T00:29:21.021814+0000","last_clean":"2024-09-21T00:29:21.021814+0000","last_became_active":"2024-09-21T00:29:19.811130+0000","last_became_peered":"2024-09-21T00:29:19.811130+0000","last_unstale":"2024-09-21T00:29:21.021814+0000","last_undegraded":"2024-09-21T00:29:21.021814+0000","last_fullsized":"2024-09-21T00:29:21.021814+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:44:42.287171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:18.775092+0000","last_change":"2024-09-21T00:29:18.775092+0000","last_active":"2024-09-21T00:29:18.775092+0000","last_peered":"2024-09-21T00:29:18.775092+0000","last_clean":"2024-09-21T00:29:18.775092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:18.775092+0000","last_undegraded":"2024-09-21T00:29:18.775092+0000","last_fullsized":"2024-09-21T00:29:18.775092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022508+0000","last_change":"2024-09-21T00:29:19.822580+0000","last_active":"2024-09-21T00:29:21.022508+0000","last_peered":"2024-09-21T00:29:21.022508+0000","last_clean":"2024-09-21T00:29:21.022508+0000","last_became_active":"2024-09-21T00:29:19.822239+0000","last_became_peered":"2024-09-21T00:29:19.822239+0000","last_unstale":"2024-09-21T00:29:21.022508+0000","last_undegraded":"2024-09-21T00:29:21.022508+0000","last_fullsized":"2024-09-21T00:29:21.022508+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:46:09.814298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.021489+0000","last_change":"2024-09-21T00:29:19.822649+0000","last_active":"2024-09-21T00:29:21.021489+0000","last_peered":"2024-09-21T00:29:21.021489+0000","last_clean":"2024-09-21T00:29:21.021489+0000","last_became_active":"2024-09-21T00:29:19.822194+0000","last_became_peered":"2024-09-21T00:29:19.822194+0000","last_unstale":"2024-09-21T00:29:21.021489+0000","last_undegraded":"2024-09-21T00:29:21.021489+0000","last_fullsized":"2024-09-21T00:29:21.021489+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:09:42.739510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022777+0000","last_change":"2024-09-21T00:29:19.816934+0000","last_active":"2024-09-21T00:29:21.022777+0000","last_peered":"2024-09-21T00:29:21.022777+0000","last_clean":"2024-09-21T00:29:21.022777+0000","last_became_active":"2024-09-21T00:29:19.815942+0000","last_became_peered":"2024-09-21T00:29:19.815942+0000","last_unstale":"2024-09-21T00:29:21.022777+0000","last_undegraded":"2024-09-21T00:29:21.022777+0000","last_fullsized":"2024-09-21T00:29:21.022777+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:35:25.434622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022456+0000","last_change":"2024-09-21T00:29:19.816919+0000","last_active":"2024-09-21T00:29:21.022456+0000","last_peered":"2024-09-21T00:29:21.022456+0000","last_clean":"2024-09-21T00:29:21.022456+0000","last_became_active":"2024-09-21T00:29:19.816530+0000","last_became_peered":"2024-09-21T00:29:19.816530+0000","last_unstale":"2024-09-21T00:29:21.022456+0000","last_undegraded":"2024-09-21T00:29:21.022456+0000","last_fullsized":"2024-09-21T00:29:21.022456+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:40:46.095060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:18.775092+0000","last_change":"2024-09-21T00:29:18.775092+0000","last_active":"2024-09-21T00:29:18.775092+0000","last_peered":"2024-09-21T00:29:18.775092+0000","last_clean":"2024-09-21T00:29:18.775092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:18.775092+0000","last_undegraded":"2024-09-21T00:29:18.775092+0000","last_fullsized":"2024-09-21T00:29:18.775092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.809793+0000","last_change":"2024-09-21T00:29:19.814490+0000","last_active":"2024-09-21T00:29:20.809793+0000","last_peered":"2024-09-21T00:29:20.809793+0000","last_clean":"2024-09-21T00:29:20.809793+0000","last_became_active":"2024-09-21T00:29:19.813188+0000","last_became_peered":"2024-09-21T00:29:19.813188+0000","last_unstale":"2024-09-21T00:29:20.809793+0000","last_undegraded":"2024-09-21T00:29:20.809793+0000","last_fullsized":"2024-09-21T00:29:20.809793+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:59:50.910192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.810780+0000","last_change":"2024-09-21T00:29:19.811796+0000","last_active":"2024-09-21T00:29:20.810780+0000","last_peered":"2024-09-21T00:29:20.810780+0000","last_clean":"2024-09-21T00:29:20.810780+0000","last_became_active":"2024-09-21T00:29:19.811496+0000","last_became_peered":"2024-09-21T00:29:19.811496+0000","last_unstale":"2024-09-21T00:29:20.810780+0000","last_undegraded":"2024-09-21T00:29:20.810780+0000","last_fullsized":"2024-09-21T00:29:20.810780+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:59:41.316661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022019+0000","last_change":"2024-09-21T00:29:19.818860+0000","last_active":"2024-09-21T00:29:21.022019+0000","last_peered":"2024-09-21T00:29:21.022019+0000","last_clean":"2024-09-21T00:29:21.022019+0000","last_became_active":"2024-09-21T00:29:19.818552+0000","last_became_peered":"2024-09-21T00:29:19.818552+0000","last_unstale":"2024-09-21T00:29:21.022019+0000","last_undegraded":"2024-09-21T00:29:21.022019+0000","last_fullsized":"2024-09-21T00:29:21.022019+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:37:22.821347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.021673+0000","last_change":"2024-09-21T00:29:19.820196+0000","last_active":"2024-09-21T00:29:21.021673+0000","last_peered":"2024-09-21T00:29:21.021673+0000","last_clean":"2024-09-21T00:29:21.021673+0000","last_became_active":"2024-09-21T00:29:19.819691+0000","last_became_peered":"2024-09-21T00:29:19.819691+0000","last_unstale":"2024-09-21T00:29:21.021673+0000","last_undegraded":"2024-09-21T00:29:21.021673+0000","last_fullsized":"2024-09-21T00:29:21.021673+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:48:11.528107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.021642+0000","last_change":"2024-09-21T00:29:19.810601+0000","last_active":"2024-09-21T00:29:21.021642+0000","last_peered":"2024-09-21T00:29:21.021642+0000","last_clean":"2024-09-21T00:29:21.021642+0000","last_became_active":"2024-09-21T00:29:19.810272+0000","last_became_peered":"2024-09-21T00:29:19.810272+0000","last_unstale":"2024-09-21T00:29:21.021642+0000","last_undegraded":"2024-09-21T00:29:21.021642+0000","last_fullsized":"2024-09-21T00:29:21.021642+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:28:10.979181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.021511+0000","last_change":"2024-09-21T00:29:19.818725+0000","last_active":"2024-09-21T00:29:21.021511+0000","last_peered":"2024-09-21T00:29:21.021511+0000","last_clean":"2024-09-21T00:29:21.021511+0000","last_became_active":"2024-09-21T00:29:19.818406+0000","last_became_peered":"2024-09-21T00:29:19.818406+0000","last_unstale":"2024-09-21T00:29:21.021511+0000","last_undegraded":"2024-09-21T00:29:21.021511+0000","last_fullsized":"2024-09-21T00:29:21.021511+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:13:53.211350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.807387+0000","last_change":"2024-09-21T00:29:19.803557+0000","last_active":"2024-09-21T00:29:20.807387+0000","last_peered":"2024-09-21T00:29:20.807387+0000","last_clean":"2024-09-21T00:29:20.807387+0000","last_became_active":"2024-09-21T00:29:19.802868+0000","last_became_peered":"2024-09-21T00:29:19.802868+0000","last_unstale":"2024-09-21T00:29:20.807387+0000","last_undegraded":"2024-09-21T00:29:20.807387+0000","last_fullsized":"2024-09-21T00:29:20.807387+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:17:38.099624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022370+0000","last_change":"2024-09-21T00:29:19.811473+0000","last_active":"2024-09-21T00:29:21.022370+0000","last_peered":"2024-09-21T00:29:21.022370+0000","last_clean":"2024-09-21T00:29:21.022370+0000","last_became_active":"2024-09-21T00:29:19.811129+0000","last_became_peered":"2024-09-21T00:29:19.811129+0000","last_unstale":"2024-09-21T00:29:21.022370+0000","last_undegraded":"2024-09-21T00:29:21.022370+0000","last_fullsized":"2024-09-21T00:29:21.022370+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:26.631458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.807233+0000","last_change":"2024-09-21T00:29:19.809929+0000","last_active":"2024-09-21T00:29:20.807233+0000","last_peered":"2024-09-21T00:29:20.807233+0000","last_clean":"2024-09-21T00:29:20.807233+0000","last_became_active":"2024-09-21T00:29:19.809541+0000","last_became_peered":"2024-09-21T00:29:19.809541+0000","last_unstale":"2024-09-21T00:29:20.807233+0000","last_undegraded":"2024-09-21T00:29:20.807233+0000","last_fullsized":"2024-09-21T00:29:20.807233+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:34:11.115991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.810853+0000","last_change":"2024-09-21T00:29:19.808754+0000","last_active":"2024-09-21T00:29:20.810853+0000","last_peered":"2024-09-21T00:29:20.810853+0000","last_clean":"2024-09-21T00:29:20.810853+0000","last_became_active":"2024-09-21T00:29:19.807857+0000","last_became_peered":"2024-09-21T00:29:19.807857+0000","last_unstale":"2024-09-21T00:29:20.810853+0000","last_undegraded":"2024-09-21T00:29:20.810853+0000","last_fullsized":"2024-09-21T00:29:20.810853+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:53:45.848291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.023011+0000","last_change":"2024-09-21T00:29:19.815917+0000","last_active":"2024-09-21T00:29:21.023011+0000","last_peered":"2024-09-21T00:29:21.023011+0000","last_clean":"2024-09-21T00:29:21.023011+0000","last_became_active":"2024-09-21T00:29:19.815559+0000","last_became_peered":"2024-09-21T00:29:19.815559+0000","last_unstale":"2024-09-21T00:29:21.023011+0000","last_undegraded":"2024-09-21T00:29:21.023011+0000","last_fullsized":"2024-09-21T00:29:21.023011+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:47:06.729633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:18.775092+0000","last_change":"2024-09-21T00:29:18.775092+0000","last_active":"2024-09-21T00:29:18.775092+0000","last_peered":"2024-09-21T00:29:18.775092+0000","last_clean":"2024-09-21T00:29:18.775092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:18.775092+0000","last_undegraded":"2024-09-21T00:29:18.775092+0000","last_fullsized":"2024-09-21T00:29:18.775092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.807054+0000","last_change":"2024-09-21T00:29:19.802421+0000","last_active":"2024-09-21T00:29:20.807054+0000","last_peered":"2024-09-21T00:29:20.807054+0000","last_clean":"2024-09-21T00:29:20.807054+0000","last_became_active":"2024-09-21T00:29:19.802074+0000","last_became_peered":"2024-09-21T00:29:19.802074+0000","last_unstale":"2024-09-21T00:29:20.807054+0000","last_undegraded":"2024-09-21T00:29:20.807054+0000","last_fullsized":"2024-09-21T00:29:20.807054+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:55:22.327108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:18.775092+0000","last_change":"2024-09-21T00:29:18.775092+0000","last_active":"2024-09-21T00:29:18.775092+0000","last_peered":"2024-09-21T00:29:18.775092+0000","last_clean":"2024-09-21T00:29:18.775092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:18.775092+0000","last_undegraded":"2024-09-21T00:29:18.775092+0000","last_fullsized":"2024-09-21T00:29:18.775092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.810198+0000","last_change":"2024-09-21T00:29:19.802920+0000","last_active":"2024-09-21T00:29:20.810198+0000","last_peered":"2024-09-21T00:29:20.810198+0000","last_clean":"2024-09-21T00:29:20.810198+0000","last_became_active":"2024-09-21T00:29:19.802584+0000","last_became_peered":"2024-09-21T00:29:19.802584+0000","last_unstale":"2024-09-21T00:29:20.810198+0000","last_undegraded":"2024-09-21T00:29:20.810198+0000","last_fullsized":"2024-09-21T00:29:20.810198+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:38:27.387471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.810402+0000","last_change":"2024-09-21T00:29:19.808064+0000","last_active":"2024-09-21T00:29:20.810402+0000","last_peered":"2024-09-21T00:29:20.810402+0000","last_clean":"2024-09-21T00:29:20.810402+0000","last_became_active":"2024-09-21T00:29:19.807712+0000","last_became_peered":"2024-09-21T00:29:19.807712+0000","last_unstale":"2024-09-21T00:29:20.810402+0000","last_undegraded":"2024-09-21T00:29:20.810402+0000","last_fullsized":"2024-09-21T00:29:20.810402+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:45:54.930710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.810053+0000","last_change":"2024-09-21T00:29:19.811883+0000","last_active":"2024-09-21T00:29:20.810053+0000","last_peered":"2024-09-21T00:29:20.810053+0000","last_clean":"2024-09-21T00:29:20.810053+0000","last_became_active":"2024-09-21T00:29:19.811584+0000","last_became_peered":"2024-09-21T00:29:19.811584+0000","last_unstale":"2024-09-21T00:29:20.810053+0000","last_undegraded":"2024-09-21T00:29:20.810053+0000","last_fullsized":"2024-09-21T00:29:20.810053+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:00:59.593942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022016+0000","last_change":"2024-09-21T00:29:19.816999+0000","last_active":"2024-09-21T00:29:21.022016+0000","last_peered":"2024-09-21T00:29:21.022016+0000","last_clean":"2024-09-21T00:29:21.022016+0000","last_became_active":"2024-09-21T00:29:19.814905+0000","last_became_peered":"2024-09-21T00:29:19.814905+0000","last_unstale":"2024-09-21T00:29:21.022016+0000","last_undegraded":"2024-09-21T00:29:21.022016+0000","last_fullsized":"2024-09-21T00:29:21.022016+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:46:57.412342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.023130+0000","last_change":"2024-09-21T00:29:19.815104+0000","last_active":"2024-09-21T00:29:21.023130+0000","last_peered":"2024-09-21T00:29:21.023130+0000","last_clean":"2024-09-21T00:29:21.023130+0000","last_became_active":"2024-09-21T00:29:19.814806+0000","last_became_peered":"2024-09-21T00:29:19.814806+0000","last_unstale":"2024-09-21T00:29:21.023130+0000","last_undegraded":"2024-09-21T00:29:21.023130+0000","last_fullsized":"2024-09-21T00:29:21.023130+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:14:39.263553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.810509+0000","last_change":"2024-09-21T00:29:19.802931+0000","last_active":"2024-09-21T00:29:20.810509+0000","last_peered":"2024-09-21T00:29:20.810509+0000","last_clean":"2024-09-21T00:29:20.810509+0000","last_became_active":"2024-09-21T00:29:19.802606+0000","last_became_peered":"2024-09-21T00:29:19.802606+0000","last_unstale":"2024-09-21T00:29:20.810509+0000","last_undegraded":"2024-09-21T00:29:20.810509+0000","last_fullsized":"2024-09-21T00:29:20.810509+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:56:07.296305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022325+0000","last_change":"2024-09-21T00:29:19.812717+0000","last_active":"2024-09-21T00:29:21.022325+0000","last_peered":"2024-09-21T00:29:21.022325+0000","last_clean":"2024-09-21T00:29:21.022325+0000","last_became_active":"2024-09-21T00:29:19.812402+0000","last_became_peered":"2024-09-21T00:29:19.812402+0000","last_unstale":"2024-09-21T00:29:21.022325+0000","last_undegraded":"2024-09-21T00:29:21.022325+0000","last_fullsized":"2024-09-21T00:29:21.022325+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:26:39.938614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022839+0000","last_change":"2024-09-21T00:29:19.816588+0000","last_active":"2024-09-21T00:29:21.022839+0000","last_peered":"2024-09-21T00:29:21.022839+0000","last_clean":"2024-09-21T00:29:21.022839+0000","last_became_active":"2024-09-21T00:29:19.815574+0000","last_became_peered":"2024-09-21T00:29:19.815574+0000","last_unstale":"2024-09-21T00:29:21.022839+0000","last_undegraded":"2024-09-21T00:29:21.022839+0000","last_fullsized":"2024-09-21T00:29:21.022839+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:47:08.243928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.807629+0000","last_change":"2024-09-21T00:29:19.810747+0000","last_active":"2024-09-21T00:29:20.807629+0000","last_peered":"2024-09-21T00:29:20.807629+0000","last_clean":"2024-09-21T00:29:20.807629+0000","last_became_active":"2024-09-21T00:29:19.810429+0000","last_became_peered":"2024-09-21T00:29:19.810429+0000","last_unstale":"2024-09-21T00:29:20.807629+0000","last_undegraded":"2024-09-21T00:29:20.807629+0000","last_fullsized":"2024-09-21T00:29:20.807629+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:21:05.578580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"93'4","reported_seq":19,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022399+0000","last_change":"2024-09-21T00:29:18.798208+0000","last_active":"2024-09-21T00:29:21.022399+0000","last_peered":"2024-09-21T00:29:21.022399+0000","last_clean":"2024-09-21T00:29:21.022399+0000","last_became_active":"2024-09-21T00:29:18.797857+0000","last_became_peered":"2024-09-21T00:29:18.797857+0000","last_unstale":"2024-09-21T00:29:21.022399+0000","last_undegraded":"2024-09-21T00:29:21.022399+0000","last_fullsized":"2024-09-21T00:29:21.022399+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T00:37:31.452087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.021779+0000","last_change":"2024-09-21T00:29:19.822277+0000","last_active":"2024-09-21T00:29:21.021779+0000","last_peered":"2024-09-21T00:29:21.021779+0000","last_clean":"2024-09-21T00:29:21.021779+0000","last_became_active":"2024-09-21T00:29:19.821917+0000","last_became_peered":"2024-09-21T00:29:19.821917+0000","last_unstale":"2024-09-21T00:29:21.021779+0000","last_undegraded":"2024-09-21T00:29:21.021779+0000","last_fullsized":"2024-09-21T00:29:21.021779+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:04:07.143491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"93'1","reported_seq":16,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022961+0000","last_change":"2024-09-21T00:29:18.799882+0000","last_active":"2024-09-21T00:29:21.022961+0000","last_peered":"2024-09-21T00:29:21.022961+0000","last_clean":"2024-09-21T00:29:21.022961+0000","last_became_active":"2024-09-21T00:29:18.799524+0000","last_became_peered":"2024-09-21T00:29:18.799524+0000","last_unstale":"2024-09-21T00:29:21.022961+0000","last_undegraded":"2024-09-21T00:29:21.022961+0000","last_fullsized":"2024-09-21T00:29:21.022961+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:39:17.740976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.810993+0000","last_change":"2024-09-21T00:29:19.809700+0000","last_active":"2024-09-21T00:29:20.810993+0000","last_peered":"2024-09-21T00:29:20.810993+0000","last_clean":"2024-09-21T00:29:20.810993+0000","last_became_active":"2024-09-21T00:29:19.809343+0000","last_became_peered":"2024-09-21T00:29:19.809343+0000","last_unstale":"2024-09-21T00:29:20.810993+0000","last_undegraded":"2024-09-21T00:29:20.810993+0000","last_fullsized":"2024-09-21T00:29:20.810993+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:28:50.502430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"93'2","reported_seq":17,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.810949+0000","last_change":"2024-09-21T00:29:18.797271+0000","last_active":"2024-09-21T00:29:20.810949+0000","last_peered":"2024-09-21T00:29:20.810949+0000","last_clean":"2024-09-21T00:29:20.810949+0000","last_became_active":"2024-09-21T00:29:18.796656+0000","last_became_peered":"2024-09-21T00:29:18.796656+0000","last_unstale":"2024-09-21T00:29:20.810949+0000","last_undegraded":"2024-09-21T00:29:20.810949+0000","last_fullsized":"2024-09-21T00:29:20.810949+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:28:00.490269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.810191+0000","last_change":"2024-09-21T00:29:19.802323+0000","last_active":"2024-09-21T00:29:20.810191+0000","last_peered":"2024-09-21T00:29:20.810191+0000","last_clean":"2024-09-21T00:29:20.810191+0000","last_became_active":"2024-09-21T00:29:19.801998+0000","last_became_peered":"2024-09-21T00:29:19.801998+0000","last_unstale":"2024-09-21T00:29:20.810191+0000","last_undegraded":"2024-09-21T00:29:20.810191+0000","last_fullsized":"2024-09-21T00:29:20.810191+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:31:43.191165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-09-21T00:29:17.783208+0000","last_change":"2024-09-21T00:29:17.783208+0000","last_active":"2024-09-21T00:29:17.769711+0000","last_peered":"2024-09-21T00:29:17.769711+0000","last_clean":"2024-09-21T00:29:17.769711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:17.783208+0000","last_undegraded":"2024-09-21T00:29:17.783208+0000","last_fullsized":"2024-09-21T00:29:17.783208+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022055+0000","last_change":"2024-09-21T00:29:19.821679+0000","last_active":"2024-09-21T00:29:21.022055+0000","last_peered":"2024-09-21T00:29:21.022055+0000","last_clean":"2024-09-21T00:29:21.022055+0000","last_became_active":"2024-09-21T00:29:19.821331+0000","last_became_peered":"2024-09-21T00:29:19.821331+0000","last_unstale":"2024-09-21T00:29:21.022055+0000","last_undegraded":"2024-09-21T00:29:21.022055+0000","last_fullsized":"2024-09-21T00:29:21.022055+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:13:53.211350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"93'1","reported_seq":16,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.021871+0000","last_change":"2024-09-21T00:29:18.797442+0000","last_active":"2024-09-21T00:29:21.021871+0000","last_peered":"2024-09-21T00:29:21.021871+0000","last_clean":"2024-09-21T00:29:21.021871+0000","last_became_active":"2024-09-21T00:29:18.797128+0000","last_became_peered":"2024-09-21T00:29:18.797128+0000","last_unstale":"2024-09-21T00:29:21.021871+0000","last_undegraded":"2024-09-21T00:29:21.021871+0000","last_fullsized":"2024-09-21T00:29:21.021871+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:22:52.216862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.807399+0000","last_change":"2024-09-21T00:29:19.808878+0000","last_active":"2024-09-21T00:29:20.807399+0000","last_peered":"2024-09-21T00:29:20.807399+0000","last_clean":"2024-09-21T00:29:20.807399+0000","last_became_active":"2024-09-21T00:29:19.808577+0000","last_became_peered":"2024-09-21T00:29:19.808577+0000","last_unstale":"2024-09-21T00:29:20.807399+0000","last_undegraded":"2024-09-21T00:29:20.807399+0000","last_fullsized":"2024-09-21T00:29:20.807399+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:34:29.883537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"93'1","reported_seq":16,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.810090+0000","last_change":"2024-09-21T00:29:18.801027+0000","last_active":"2024-09-21T00:29:20.810090+0000","last_peered":"2024-09-21T00:29:20.810090+0000","last_clean":"2024-09-21T00:29:20.810090+0000","last_became_active":"2024-09-21T00:29:18.800693+0000","last_became_peered":"2024-09-21T00:29:18.800693+0000","last_unstale":"2024-09-21T00:29:20.810090+0000","last_undegraded":"2024-09-21T00:29:20.810090+0000","last_fullsized":"2024-09-21T00:29:20.810090+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:17:06.642779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022624+0000","last_change":"2024-09-21T00:29:19.815242+0000","last_active":"2024-09-21T00:29:21.022624+0000","last_peered":"2024-09-21T00:29:21.022624+0000","last_clean":"2024-09-21T00:29:21.022624+0000","last_became_active":"2024-09-21T00:29:19.814858+0000","last_became_peered":"2024-09-21T00:29:19.814858+0000","last_unstale":"2024-09-21T00:29:21.022624+0000","last_undegraded":"2024-09-21T00:29:21.022624+0000","last_fullsized":"2024-09-21T00:29:21.022624+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:35:06.851240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.809872+0000","last_change":"2024-09-21T00:29:18.797155+0000","last_active":"2024-09-21T00:29:20.809872+0000","last_peered":"2024-09-21T00:29:20.809872+0000","last_clean":"2024-09-21T00:29:20.809872+0000","last_became_active":"2024-09-21T00:29:18.795953+0000","last_became_peered":"2024-09-21T00:29:18.795953+0000","last_unstale":"2024-09-21T00:29:20.809872+0000","last_undegraded":"2024-09-21T00:29:20.809872+0000","last_fullsized":"2024-09-21T00:29:20.809872+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T00:54:16.170396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022428+0000","last_change":"2024-09-21T00:29:19.817180+0000","last_active":"2024-09-21T00:29:21.022428+0000","last_peered":"2024-09-21T00:29:21.022428+0000","last_clean":"2024-09-21T00:29:21.022428+0000","last_became_active":"2024-09-21T00:29:19.816859+0000","last_became_peered":"2024-09-21T00:29:19.816859+0000","last_unstale":"2024-09-21T00:29:21.022428+0000","last_undegraded":"2024-09-21T00:29:21.022428+0000","last_fullsized":"2024-09-21T00:29:21.022428+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:30:01.904485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.021605+0000","last_change":"2024-09-21T00:29:18.793095+0000","last_active":"2024-09-21T00:29:21.021605+0000","last_peered":"2024-09-21T00:29:21.021605+0000","last_clean":"2024-09-21T00:29:21.021605+0000","last_became_active":"2024-09-21T00:29:18.792693+0000","last_became_peered":"2024-09-21T00:29:18.792693+0000","last_unstale":"2024-09-21T00:29:21.021605+0000","last_undegraded":"2024-09-21T00:29:21.021605+0000","last_fullsized":"2024-09-21T00:29:21.021605+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:56:18.147899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.807542+0000","last_change":"2024-09-21T00:29:19.815545+0000","last_active":"2024-09-21T00:29:20.807542+0000","last_peered":"2024-09-21T00:29:20.807542+0000","last_clean":"2024-09-21T00:29:20.807542+0000","last_became_active":"2024-09-21T00:29:19.811424+0000","last_became_peered":"2024-09-21T00:29:19.811424+0000","last_unstale":"2024-09-21T00:29:20.807542+0000","last_undegraded":"2024-09-21T00:29:20.807542+0000","last_fullsized":"2024-09-21T00:29:20.807542+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:20:57.334693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.810327+0000","last_change":"2024-09-21T00:29:18.796965+0000","last_active":"2024-09-21T00:29:20.810327+0000","last_peered":"2024-09-21T00:29:20.810327+0000","last_clean":"2024-09-21T00:29:20.810327+0000","last_became_active":"2024-09-21T00:29:18.796548+0000","last_became_peered":"2024-09-21T00:29:18.796548+0000","last_unstale":"2024-09-21T00:29:20.810327+0000","last_undegraded":"2024-09-21T00:29:20.810327+0000","last_fullsized":"2024-09-21T00:29:20.810327+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:00:18.177473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:18.775092+0000","last_change":"2024-09-21T00:29:18.775092+0000","last_active":"2024-09-21T00:29:18.775092+0000","last_peered":"2024-09-21T00:29:18.775092+0000","last_clean":"2024-09-21T00:29:18.775092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:18.775092+0000","last_undegraded":"2024-09-21T00:29:18.775092+0000","last_fullsized":"2024-09-21T00:29:18.775092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.807185+0000","last_change":"2024-09-21T00:29:18.798831+0000","last_active":"2024-09-21T00:29:20.807185+0000","last_peered":"2024-09-21T00:29:20.807185+0000","last_clean":"2024-09-21T00:29:20.807185+0000","last_became_active":"2024-09-21T00:29:18.798551+0000","last_became_peered":"2024-09-21T00:29:18.798551+0000","last_unstale":"2024-09-21T00:29:20.807185+0000","last_undegraded":"2024-09-21T00:29:20.807185+0000","last_fullsized":"2024-09-21T00:29:20.807185+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:12:30.180725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022749+0000","last_change":"2024-09-21T00:29:19.812082+0000","last_active":"2024-09-21T00:29:21.022749+0000","last_peered":"2024-09-21T00:29:21.022749+0000","last_clean":"2024-09-21T00:29:21.022749+0000","last_became_active":"2024-09-21T00:29:19.811773+0000","last_became_peered":"2024-09-21T00:29:19.811773+0000","last_unstale":"2024-09-21T00:29:21.022749+0000","last_undegraded":"2024-09-21T00:29:21.022749+0000","last_fullsized":"2024-09-21T00:29:21.022749+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:17:43.069499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"93'3","reported_seq":18,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.021859+0000","last_change":"2024-09-21T00:29:18.794178+0000","last_active":"2024-09-21T00:29:21.021859+0000","last_peered":"2024-09-21T00:29:21.021859+0000","last_clean":"2024-09-21T00:29:21.021859+0000","last_became_active":"2024-09-21T00:29:18.793851+0000","last_became_peered":"2024-09-21T00:29:18.793851+0000","last_unstale":"2024-09-21T00:29:21.021859+0000","last_undegraded":"2024-09-21T00:29:21.021859+0000","last_fullsized":"2024-09-21T00:29:21.021859+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:52:27.563468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:18.775092+0000","last_change":"2024-09-21T00:29:18.775092+0000","last_active":"2024-09-21T00:29:18.775092+0000","last_peered":"2024-09-21T00:29:18.775092+0000","last_clean":"2024-09-21T00:29:18.775092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:18.775092+0000","last_undegraded":"2024-09-21T00:29:18.775092+0000","last_fullsized":"2024-09-21T00:29:18.775092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.14","version":"93'6","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.021896+0000","last_change":"2024-09-21T00:29:18.800236+0000","last_active":"2024-09-21T00:29:21.021896+0000","last_peered":"2024-09-21T00:29:21.021896+0000","last_clean":"2024-09-21T00:29:21.021896+0000","last_became_active":"2024-09-21T00:29:18.799899+0000","last_became_peered":"2024-09-21T00:29:18.799899+0000","last_unstale":"2024-09-21T00:29:21.021896+0000","last_undegraded":"2024-09-21T00:29:21.021896+0000","last_fullsized":"2024-09-21T00:29:21.021896+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:29:10.908555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.809551+0000","last_change":"2024-09-21T00:29:19.811833+0000","last_active":"2024-09-21T00:29:20.809551+0000","last_peered":"2024-09-21T00:29:20.809551+0000","last_clean":"2024-09-21T00:29:20.809551+0000","last_became_active":"2024-09-21T00:29:19.811519+0000","last_became_peered":"2024-09-21T00:29:19.811519+0000","last_unstale":"2024-09-21T00:29:20.809551+0000","last_undegraded":"2024-09-21T00:29:20.809551+0000","last_fullsized":"2024-09-21T00:29:20.809551+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:52:28.568849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-09-21T00:29:17.784261+0000","last_change":"2024-09-21T00:29:17.784261+0000","last_active":"2024-09-21T00:29:17.769711+0000","last_peered":"2024-09-21T00:29:17.769711+0000","last_clean":"2024-09-21T00:29:17.769711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:17.784261+0000","last_undegraded":"2024-09-21T00:29:17.784261+0000","last_fullsized":"2024-09-21T00:29:17.784261+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022489+0000","last_change":"2024-09-21T00:29:19.814412+0000","last_active":"2024-09-21T00:29:21.022489+0000","last_peered":"2024-09-21T00:29:21.022489+0000","last_clean":"2024-09-21T00:29:21.022489+0000","last_became_active":"2024-09-21T00:29:19.813923+0000","last_became_peered":"2024-09-21T00:29:19.813923+0000","last_unstale":"2024-09-21T00:29:21.022489+0000","last_undegraded":"2024-09-21T00:29:21.022489+0000","last_fullsized":"2024-09-21T00:29:21.022489+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:42:28.398366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022221+0000","last_change":"2024-09-21T00:29:18.788248+0000","last_active":"2024-09-21T00:29:21.022221+0000","last_peered":"2024-09-21T00:29:21.022221+0000","last_clean":"2024-09-21T00:29:21.022221+0000","last_became_active":"2024-09-21T00:29:18.787904+0000","last_became_peered":"2024-09-21T00:29:18.787904+0000","last_unstale":"2024-09-21T00:29:21.022221+0000","last_undegraded":"2024-09-21T00:29:21.022221+0000","last_fullsized":"2024-09-21T00:29:21.022221+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:21:04.691629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.023078+0000","last_change":"2024-09-21T00:29:19.814341+0000","last_active":"2024-09-21T00:29:21.023078+0000","last_peered":"2024-09-21T00:29:21.023078+0000","last_clean":"2024-09-21T00:29:21.023078+0000","last_became_active":"2024-09-21T00:29:19.814052+0000","last_became_peered":"2024-09-21T00:29:19.814052+0000","last_unstale":"2024-09-21T00:29:21.023078+0000","last_undegraded":"2024-09-21T00:29:21.023078+0000","last_fullsized":"2024-09-21T00:29:21.023078+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:58:18.707835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.021901+0000","last_change":"2024-09-21T00:29:18.800084+0000","last_active":"2024-09-21T00:29:21.021901+0000","last_peered":"2024-09-21T00:29:21.021901+0000","last_clean":"2024-09-21T00:29:21.021901+0000","last_became_active":"2024-09-21T00:29:18.799797+0000","last_became_peered":"2024-09-21T00:29:18.799797+0000","last_unstale":"2024-09-21T00:29:21.021901+0000","last_undegraded":"2024-09-21T00:29:21.021901+0000","last_fullsized":"2024-09-21T00:29:21.021901+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T00:51:35.304058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.807595+0000","last_change":"2024-09-21T00:29:19.804683+0000","last_active":"2024-09-21T00:29:20.807595+0000","last_peered":"2024-09-21T00:29:20.807595+0000","last_clean":"2024-09-21T00:29:20.807595+0000","last_became_active":"2024-09-21T00:29:19.803246+0000","last_became_peered":"2024-09-21T00:29:19.803246+0000","last_unstale":"2024-09-21T00:29:20.807595+0000","last_undegraded":"2024-09-21T00:29:20.807595+0000","last_fullsized":"2024-09-21T00:29:20.807595+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:46:26.637135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"93'8","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.810678+0000","last_change":"2024-09-21T00:29:18.796567+0000","last_active":"2024-09-21T00:29:20.810678+0000","last_peered":"2024-09-21T00:29:20.810678+0000","last_clean":"2024-09-21T00:29:20.810678+0000","last_became_active":"2024-09-21T00:29:18.796255+0000","last_became_peered":"2024-09-21T00:29:18.796255+0000","last_unstale":"2024-09-21T00:29:20.810678+0000","last_undegraded":"2024-09-21T00:29:20.810678+0000","last_fullsized":"2024-09-21T00:29:20.810678+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:04:53.438244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.023031+0000","last_change":"2024-09-21T00:29:19.817234+0000","last_active":"2024-09-21T00:29:21.023031+0000","last_peered":"2024-09-21T00:29:21.023031+0000","last_clean":"2024-09-21T00:29:21.023031+0000","last_became_active":"2024-09-21T00:29:19.816273+0000","last_became_peered":"2024-09-21T00:29:19.816273+0000","last_unstale":"2024-09-21T00:29:21.023031+0000","last_undegraded":"2024-09-21T00:29:21.023031+0000","last_fullsized":"2024-09-21T00:29:21.023031+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:45:02.009239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"93'1","reported_seq":16,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022386+0000","last_change":"2024-09-21T00:29:18.799579+0000","last_active":"2024-09-21T00:29:21.022386+0000","last_peered":"2024-09-21T00:29:21.022386+0000","last_clean":"2024-09-21T00:29:21.022386+0000","last_became_active":"2024-09-21T00:29:18.799238+0000","last_became_peered":"2024-09-21T00:29:18.799238+0000","last_unstale":"2024-09-21T00:29:21.022386+0000","last_undegraded":"2024-09-21T00:29:21.022386+0000","last_fullsized":"2024-09-21T00:29:21.022386+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:01:35.301014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022278+0000","last_change":"2024-09-21T00:29:19.814239+0000","last_active":"2024-09-21T00:29:21.022278+0000","last_peered":"2024-09-21T00:29:21.022278+0000","last_clean":"2024-09-21T00:29:21.022278+0000","last_became_active":"2024-09-21T00:29:19.813892+0000","last_became_peered":"2024-09-21T00:29:19.813892+0000","last_unstale":"2024-09-21T00:29:21.022278+0000","last_undegraded":"2024-09-21T00:29:21.022278+0000","last_fullsized":"2024-09-21T00:29:21.022278+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:02:55.200607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"93'1","reported_seq":16,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.021997+0000","last_change":"2024-09-21T00:29:18.786505+0000","last_active":"2024-09-21T00:29:21.021997+0000","last_peered":"2024-09-21T00:29:21.021997+0000","last_clean":"2024-09-21T00:29:21.021997+0000","last_became_active":"2024-09-21T00:29:18.786141+0000","last_became_peered":"2024-09-21T00:29:18.786141+0000","last_unstale":"2024-09-21T00:29:21.021997+0000","last_undegraded":"2024-09-21T00:29:21.021997+0000","last_fullsized":"2024-09-21T00:29:21.021997+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:59:49.904811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:18.775092+0000","last_change":"2024-09-21T00:29:18.775092+0000","last_active":"2024-09-21T00:29:18.775092+0000","last_peered":"2024-09-21T00:29:18.775092+0000","last_clean":"2024-09-21T00:29:18.775092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:18.775092+0000","last_undegraded":"2024-09-21T00:29:18.775092+0000","last_fullsized":"2024-09-21T00:29:18.775092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.809671+0000","last_change":"2024-09-21T00:29:18.802388+0000","last_active":"2024-09-21T00:29:20.809671+0000","last_peered":"2024-09-21T00:29:20.809671+0000","last_clean":"2024-09-21T00:29:20.809671+0000","last_became_active":"2024-09-21T00:29:18.802060+0000","last_became_peered":"2024-09-21T00:29:18.802060+0000","last_unstale":"2024-09-21T00:29:20.809671+0000","last_undegraded":"2024-09-21T00:29:20.809671+0000","last_fullsized":"2024-09-21T00:29:20.809671+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:26:13.866796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.807105+0000","last_change":"2024-09-21T00:29:19.812924+0000","last_active":"2024-09-21T00:29:20.807105+0000","last_peered":"2024-09-21T00:29:20.807105+0000","last_clean":"2024-09-21T00:29:20.807105+0000","last_became_active":"2024-09-21T00:29:19.812607+0000","last_became_peered":"2024-09-21T00:29:19.812607+0000","last_unstale":"2024-09-21T00:29:20.807105+0000","last_undegraded":"2024-09-21T00:29:20.807105+0000","last_fullsized":"2024-09-21T00:29:20.807105+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:11:09.667419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"93'2","reported_seq":17,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.023167+0000","last_change":"2024-09-21T00:29:18.796630+0000","last_active":"2024-09-21T00:29:21.023167+0000","last_peered":"2024-09-21T00:29:21.023167+0000","last_clean":"2024-09-21T00:29:21.023167+0000","last_became_active":"2024-09-21T00:29:18.796317+0000","last_became_peered":"2024-09-21T00:29:18.796317+0000","last_unstale":"2024-09-21T00:29:21.023167+0000","last_undegraded":"2024-09-21T00:29:21.023167+0000","last_fullsized":"2024-09-21T00:29:21.023167+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:50:16.709606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.807330+0000","last_change":"2024-09-21T00:29:19.801051+0000","last_active":"2024-09-21T00:29:20.807330+0000","last_peered":"2024-09-21T00:29:20.807330+0000","last_clean":"2024-09-21T00:29:20.807330+0000","last_became_active":"2024-09-21T00:29:19.798834+0000","last_became_peered":"2024-09-21T00:29:19.798834+0000","last_unstale":"2024-09-21T00:29:20.807330+0000","last_undegraded":"2024-09-21T00:29:20.807330+0000","last_fullsized":"2024-09-21T00:29:20.807330+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:20:15.757612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022535+0000","last_change":"2024-09-21T00:29:18.796311+0000","last_active":"2024-09-21T00:29:21.022535+0000","last_peered":"2024-09-21T00:29:21.022535+0000","last_clean":"2024-09-21T00:29:21.022535+0000","last_became_active":"2024-09-21T00:29:18.795425+0000","last_became_peered":"2024-09-21T00:29:18.795425+0000","last_unstale":"2024-09-21T00:29:21.022535+0000","last_undegraded":"2024-09-21T00:29:21.022535+0000","last_fullsized":"2024-09-21T00:29:21.022535+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:12:21.051042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022415+0000","last_change":"2024-09-21T00:29:19.813509+0000","last_active":"2024-09-21T00:29:21.022415+0000","last_peered":"2024-09-21T00:29:21.022415+0000","last_clean":"2024-09-21T00:29:21.022415+0000","last_became_active":"2024-09-21T00:29:19.813162+0000","last_became_peered":"2024-09-21T00:29:19.813162+0000","last_unstale":"2024-09-21T00:29:21.022415+0000","last_undegraded":"2024-09-21T00:29:21.022415+0000","last_fullsized":"2024-09-21T00:29:21.022415+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:48:11.528107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022396+0000","last_change":"2024-09-21T00:29:18.799921+0000","last_active":"2024-09-21T00:29:21.022396+0000","last_peered":"2024-09-21T00:29:21.022396+0000","last_clean":"2024-09-21T00:29:21.022396+0000","last_became_active":"2024-09-21T00:29:18.799614+0000","last_became_peered":"2024-09-21T00:29:18.799614+0000","last_unstale":"2024-09-21T00:29:21.022396+0000","last_undegraded":"2024-09-21T00:29:21.022396+0000","last_fullsized":"2024-09-21T00:29:21.022396+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:33:56.930832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022472+0000","last_change":"2024-09-21T00:29:19.813609+0000","last_active":"2024-09-21T00:29:21.022472+0000","last_peered":"2024-09-21T00:29:21.022472+0000","last_clean":"2024-09-21T00:29:21.022472+0000","last_became_active":"2024-09-21T00:29:19.813291+0000","last_became_peered":"2024-09-21T00:29:19.813291+0000","last_unstale":"2024-09-21T00:29:21.022472+0000","last_undegraded":"2024-09-21T00:29:21.022472+0000","last_fullsized":"2024-09-21T00:29:21.022472+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:48:12.895949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022964+0000","last_change":"2024-09-21T00:29:18.796499+0000","last_active":"2024-09-21T00:29:21.022964+0000","last_peered":"2024-09-21T00:29:21.022964+0000","last_clean":"2024-09-21T00:29:21.022964+0000","last_became_active":"2024-09-21T00:29:18.796175+0000","last_became_peered":"2024-09-21T00:29:18.796175+0000","last_unstale":"2024-09-21T00:29:21.022964+0000","last_undegraded":"2024-09-21T00:29:21.022964+0000","last_fullsized":"2024-09-21T00:29:21.022964+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:58:46.110270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.807498+0000","last_change":"2024-09-21T00:29:19.815935+0000","last_active":"2024-09-21T00:29:20.807498+0000","last_peered":"2024-09-21T00:29:20.807498+0000","last_clean":"2024-09-21T00:29:20.807498+0000","last_became_active":"2024-09-21T00:29:19.814883+0000","last_became_peered":"2024-09-21T00:29:19.814883+0000","last_unstale":"2024-09-21T00:29:20.807498+0000","last_undegraded":"2024-09-21T00:29:20.807498+0000","last_fullsized":"2024-09-21T00:29:20.807498+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:40:18.526456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-09-21T00:29:17.781825+0000","last_change":"2024-09-21T00:29:17.781825+0000","last_active":"2024-09-21T00:29:17.769711+0000","last_peered":"2024-09-21T00:29:17.769711+0000","last_clean":"2024-09-21T00:29:17.769711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:17.781825+0000","last_undegraded":"2024-09-21T00:29:17.781825+0000","last_fullsized":"2024-09-21T00:29:17.781825+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.809738+0000","last_change":"2024-09-21T00:29:19.806248+0000","last_active":"2024-09-21T00:29:20.809738+0000","last_peered":"2024-09-21T00:29:20.809738+0000","last_clean":"2024-09-21T00:29:20.809738+0000","last_became_active":"2024-09-21T00:29:19.805916+0000","last_became_peered":"2024-09-21T00:29:19.805916+0000","last_unstale":"2024-09-21T00:29:20.809738+0000","last_undegraded":"2024-09-21T00:29:20.809738+0000","last_fullsized":"2024-09-21T00:29:20.809738+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:42:42.562148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"93'1","reported_seq":16,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.807289+0000","last_change":"2024-09-21T00:29:18.797446+0000","last_active":"2024-09-21T00:29:20.807289+0000","last_peered":"2024-09-21T00:29:20.807289+0000","last_clean":"2024-09-21T00:29:20.807289+0000","last_became_active":"2024-09-21T00:29:18.797076+0000","last_became_peered":"2024-09-21T00:29:18.797076+0000","last_unstale":"2024-09-21T00:29:20.807289+0000","last_undegraded":"2024-09-21T00:29:20.807289+0000","last_fullsized":"2024-09-21T00:29:20.807289+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:06:17.566926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022312+0000","last_change":"2024-09-21T00:29:19.812384+0000","last_active":"2024-09-21T00:29:21.022312+0000","last_peered":"2024-09-21T00:29:21.022312+0000","last_clean":"2024-09-21T00:29:21.022312+0000","last_became_active":"2024-09-21T00:29:19.812077+0000","last_became_peered":"2024-09-21T00:29:19.812077+0000","last_unstale":"2024-09-21T00:29:21.022312+0000","last_undegraded":"2024-09-21T00:29:21.022312+0000","last_fullsized":"2024-09-21T00:29:21.022312+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:28:10.979181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022603+0000","last_change":"2024-09-21T00:29:18.796908+0000","last_active":"2024-09-21T00:29:21.022603+0000","last_peered":"2024-09-21T00:29:21.022603+0000","last_clean":"2024-09-21T00:29:21.022603+0000","last_became_active":"2024-09-21T00:29:18.796559+0000","last_became_peered":"2024-09-21T00:29:18.796559+0000","last_unstale":"2024-09-21T00:29:21.022603+0000","last_undegraded":"2024-09-21T00:29:21.022603+0000","last_fullsized":"2024-09-21T00:29:21.022603+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:27:50.568077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.809937+0000","last_change":"2024-09-21T00:29:19.808826+0000","last_active":"2024-09-21T00:29:20.809937+0000","last_peered":"2024-09-21T00:29:20.809937+0000","last_clean":"2024-09-21T00:29:20.809937+0000","last_became_active":"2024-09-21T00:29:19.808196+0000","last_became_peered":"2024-09-21T00:29:19.808196+0000","last_unstale":"2024-09-21T00:29:20.809937+0000","last_undegraded":"2024-09-21T00:29:20.809937+0000","last_fullsized":"2024-09-21T00:29:20.809937+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:22:53.222243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-09-21T00:29:17.782729+0000","last_change":"2024-09-21T00:29:17.782729+0000","last_active":"2024-09-21T00:29:17.769711+0000","last_peered":"2024-09-21T00:29:17.769711+0000","last_clean":"2024-09-21T00:29:17.769711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:17.782729+0000","last_undegraded":"2024-09-21T00:29:17.782729+0000","last_fullsized":"2024-09-21T00:29:17.782729+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.021641+0000","last_change":"2024-09-21T00:29:19.823036+0000","last_active":"2024-09-21T00:29:21.021641+0000","last_peered":"2024-09-21T00:29:21.021641+0000","last_clean":"2024-09-21T00:29:21.021641+0000","last_became_active":"2024-09-21T00:29:19.822688+0000","last_became_peered":"2024-09-21T00:29:19.822688+0000","last_unstale":"2024-09-21T00:29:21.021641+0000","last_undegraded":"2024-09-21T00:29:21.021641+0000","last_fullsized":"2024-09-21T00:29:21.021641+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:54:44.835987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.809976+0000","last_change":"2024-09-21T00:29:18.795683+0000","last_active":"2024-09-21T00:29:20.809976+0000","last_peered":"2024-09-21T00:29:20.809976+0000","last_clean":"2024-09-21T00:29:20.809976+0000","last_became_active":"2024-09-21T00:29:18.795363+0000","last_became_peered":"2024-09-21T00:29:18.795363+0000","last_unstale":"2024-09-21T00:29:20.809976+0000","last_undegraded":"2024-09-21T00:29:20.809976+0000","last_fullsized":"2024-09-21T00:29:20.809976+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:20:55.296469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022149+0000","last_change":"2024-09-21T00:29:19.812845+0000","last_active":"2024-09-21T00:29:21.022149+0000","last_peered":"2024-09-21T00:29:21.022149+0000","last_clean":"2024-09-21T00:29:21.022149+0000","last_became_active":"2024-09-21T00:29:19.812536+0000","last_became_peered":"2024-09-21T00:29:19.812536+0000","last_unstale":"2024-09-21T00:29:21.022149+0000","last_undegraded":"2024-09-21T00:29:21.022149+0000","last_fullsized":"2024-09-21T00:29:21.022149+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:48:44.631863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022864+0000","last_change":"2024-09-21T00:29:18.800233+0000","last_active":"2024-09-21T00:29:21.022864+0000","last_peered":"2024-09-21T00:29:21.022864+0000","last_clean":"2024-09-21T00:29:21.022864+0000","last_became_active":"2024-09-21T00:29:18.799952+0000","last_became_peered":"2024-09-21T00:29:18.799952+0000","last_unstale":"2024-09-21T00:29:21.022864+0000","last_undegraded":"2024-09-21T00:29:21.022864+0000","last_fullsized":"2024-09-21T00:29:21.022864+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:21:56.870500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.021841+0000","last_change":"2024-09-21T00:29:19.816067+0000","last_active":"2024-09-21T00:29:21.021841+0000","last_peered":"2024-09-21T00:29:21.021841+0000","last_clean":"2024-09-21T00:29:21.021841+0000","last_became_active":"2024-09-21T00:29:19.815714+0000","last_became_peered":"2024-09-21T00:29:19.815714+0000","last_unstale":"2024-09-21T00:29:21.021841+0000","last_undegraded":"2024-09-21T00:29:21.021841+0000","last_fullsized":"2024-09-21T00:29:21.021841+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:30:01.904485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022492+0000","last_change":"2024-09-21T00:29:18.796212+0000","last_active":"2024-09-21T00:29:21.022492+0000","last_peered":"2024-09-21T00:29:21.022492+0000","last_clean":"2024-09-21T00:29:21.022492+0000","last_became_active":"2024-09-21T00:29:18.795864+0000","last_became_peered":"2024-09-21T00:29:18.795864+0000","last_unstale":"2024-09-21T00:29:21.022492+0000","last_undegraded":"2024-09-21T00:29:21.022492+0000","last_fullsized":"2024-09-21T00:29:21.022492+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:57:43.771137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.810604+0000","last_change":"2024-09-21T00:29:19.811714+0000","last_active":"2024-09-21T00:29:20.810604+0000","last_peered":"2024-09-21T00:29:20.810604+0000","last_clean":"2024-09-21T00:29:20.810604+0000","last_became_active":"2024-09-21T00:29:19.809202+0000","last_became_peered":"2024-09-21T00:29:19.809202+0000","last_unstale":"2024-09-21T00:29:20.810604+0000","last_undegraded":"2024-09-21T00:29:20.810604+0000","last_fullsized":"2024-09-21T00:29:20.810604+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:44:42.287171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.023051+0000","last_change":"2024-09-21T00:29:18.801879+0000","last_active":"2024-09-21T00:29:21.023051+0000","last_peered":"2024-09-21T00:29:21.023051+0000","last_clean":"2024-09-21T00:29:21.023051+0000","last_became_active":"2024-09-21T00:29:18.800708+0000","last_became_peered":"2024-09-21T00:29:18.800708+0000","last_unstale":"2024-09-21T00:29:21.023051+0000","last_undegraded":"2024-09-21T00:29:21.023051+0000","last_fullsized":"2024-09-21T00:29:21.023051+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:48:40.413052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.809702+0000","last_change":"2024-09-21T00:29:19.798077+0000","last_active":"2024-09-21T00:29:20.809702+0000","last_peered":"2024-09-21T00:29:20.809702+0000","last_clean":"2024-09-21T00:29:20.809702+0000","last_became_active":"2024-09-21T00:29:19.797724+0000","last_became_peered":"2024-09-21T00:29:19.797724+0000","last_unstale":"2024-09-21T00:29:20.809702+0000","last_undegraded":"2024-09-21T00:29:20.809702+0000","last_fullsized":"2024-09-21T00:29:20.809702+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:35:16.985174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"93'2","reported_seq":17,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.023087+0000","last_change":"2024-09-21T00:29:18.800162+0000","last_active":"2024-09-21T00:29:21.023087+0000","last_peered":"2024-09-21T00:29:21.023087+0000","last_clean":"2024-09-21T00:29:21.023087+0000","last_became_active":"2024-09-21T00:29:18.798939+0000","last_became_peered":"2024-09-21T00:29:18.798939+0000","last_unstale":"2024-09-21T00:29:21.023087+0000","last_undegraded":"2024-09-21T00:29:21.023087+0000","last_fullsized":"2024-09-21T00:29:21.023087+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:28:05.362595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":244,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.021503+0000","last_change":"2024-09-21T00:25:38.725792+0000","last_active":"2024-09-21T00:29:21.021503+0000","last_peered":"2024-09-21T00:29:21.021503+0000","last_clean":"2024-09-21T00:29:21.021503+0000","last_became_active":"2024-09-21T00:25:38.724668+0000","last_became_peered":"2024-09-21T00:25:38.724668+0000","last_unstale":"2024-09-21T00:29:21.021503+0000","last_undegraded":"2024-09-21T00:29:21.021503+0000","last_fullsized":"2024-09-21T00:29:21.021503+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:37.163216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:37.163216+0000","last_clean_scrub_stamp":"2024-09-21T00:25:37.163216+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:14:36.588660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":114,"acting":114,"num_store_stats":7},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2376,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":33,"ondisk_log_size":33,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509488,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32908,"kb_used_data":1720,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338932,"statfs":{"total":96636764160,"available":96603066368,"internally_reserved":0,"allocated":1761280,"data_stored":1116174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509489,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32336,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339504,"statfs":{"total":96636764160,"available":96603652096,"internally_reserved":0,"allocated":1167360,"data_stored":521170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509488,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32308,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339532,"statfs":{"total":96636764160,"available":96603680768,"internally_reserved":0,"allocated":1146880,"data_stored":518954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509487,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32328,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339512,"statfs":{"total":96636764160,"available":96603660288,"internally_reserved":0,"allocated":1167360,"data_stored":525738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509489,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32328,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339512,"statfs":{"total":96636764160,"available":96603660288,"internally_reserved":0,"allocated":1163264,"data_stored":526228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509489,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32324,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339516,"statfs":{"total":96636764160,"available":96603664384,"internally_reserved":0,"allocated":1163264,"data_stored":525680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509489,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32904,"kb_used_data":1716,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338936,"statfs":{"total":96636764160,"available":96603070464,"internally_reserved":0,"allocated":1757184,"data_stored":1111024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574896,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32276,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339564,"statfs":{"total":96636764160,"available":96603713536,"internally_reserved":0,"allocated":1110016,"data_stored":498911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T00:29:23.818 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T00:29:23.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.966+0000 7f5c45ef2640 1 -- 172.21.15.112:0/1153564794 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c28004d40 msgr2=0x7f5c28005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:23.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.966+0000 7f5c45ef2640 1 --2- 172.21.15.112:0/1153564794 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c28004d40 0x7f5c28005150 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f5c3800c760 tx=0x7f5c28005830 comp rx=0 tx=0).stop 2024-09-21T00:29:23.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.966+0000 7f5c3ffff640 1 -- 172.21.15.112:0/1153564794 reap_dead start 2024-09-21T00:29:23.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.966+0000 7f5c45ef2640 1 -- 172.21.15.112:0/1153564794 shutdown_connections 2024-09-21T00:29:23.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.966+0000 7f5c45ef2640 1 -- 172.21.15.112:0/1153564794 >> 172.21.15.112:0/1153564794 conn(0x7f5c400b42e0 msgr2=0x7f5c400b4720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:23.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.966+0000 7f5c45ef2640 1 -- 172.21.15.112:0/1153564794 shutdown_connections 2024-09-21T00:29:23.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.966+0000 7f5c45ef2640 1 -- 172.21.15.112:0/1153564794 wait complete. 2024-09-21T00:29:23.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.968+0000 7f5c45ef2640 1 Processor -- start 2024-09-21T00:29:23.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.968+0000 7f5c45ef2640 1 -- start start 2024-09-21T00:29:23.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.970+0000 7f5c45ef2640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5c401f8000 0x7f5c4014f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:23.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.970+0000 7f5c45ef2640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5c400ac8a0 0x7f5c40155000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:23.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.970+0000 7f5c45ef2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c40155540 0x7f5c400b0a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:23.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.970+0000 7f5c45ef2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5c400b3450 con 0x7f5c40155540 2024-09-21T00:29:23.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.970+0000 7f5c45ef2640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5c400c6e90 con 0x7f5c400ac8a0 2024-09-21T00:29:23.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.970+0000 7f5c45ef2640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f5c400b35d0 con 0x7f5c401f8000 2024-09-21T00:29:23.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.970+0000 7f5c3ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c40155540 0x7f5c400b0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:23.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.970+0000 7f5c3ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c40155540 0x7f5c400b0a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48708/0 (socket says 172.21.15.112:48708) 2024-09-21T00:29:23.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.970+0000 7f5c3ffff640 1 -- 172.21.15.112:0/712978518 learned_addr learned my addr 172.21.15.112:0/712978518 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:29:23.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.970+0000 7f5c3f7fe640 1 --2- 172.21.15.112:0/712978518 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5c401f8000 0x7f5c4014f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:23.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.970+0000 7f5c3effd640 1 --2- 172.21.15.112:0/712978518 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5c400ac8a0 0x7f5c40155000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:23.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.970+0000 7f5c3ffff640 1 -- 172.21.15.112:0/712978518 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5c401f8000 msgr2=0x7f5c4014f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:23.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.970+0000 7f5c3ffff640 1 --2- 172.21.15.112:0/712978518 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5c401f8000 0x7f5c4014f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:23.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.970+0000 7f5c3ffff640 1 -- 172.21.15.112:0/712978518 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5c400ac8a0 msgr2=0x7f5c40155000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:23.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.970+0000 7f5c3ffff640 1 --2- 172.21.15.112:0/712978518 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5c400ac8a0 0x7f5c40155000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:23.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.970+0000 7f5c3ffff640 1 -- 172.21.15.112:0/712978518 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c4014fa30 con 0x7f5c40155540 2024-09-21T00:29:23.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.970+0000 7f5c3ffff640 1 --2- 172.21.15.112:0/712978518 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c40155540 0x7f5c400b0a90 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f5c3800c880 tx=0x7f5c3802b570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:23.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.970+0000 7f5c27fff640 1 -- 172.21.15.112:0/712978518 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c3800c380 con 0x7f5c40155540 2024-09-21T00:29:23.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.970+0000 7f5c27fff640 1 -- 172.21.15.112:0/712978518 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5c38004b40 con 0x7f5c40155540 2024-09-21T00:29:23.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.970+0000 7f5c45ef2640 1 -- 172.21.15.112:0/712978518 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c400caa70 con 0x7f5c40155540 2024-09-21T00:29:23.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.970+0000 7f5c27fff640 1 -- 172.21.15.112:0/712978518 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c3802b9b0 con 0x7f5c40155540 2024-09-21T00:29:23.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.970+0000 7f5c45ef2640 1 -- 172.21.15.112:0/712978518 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c400cb030 con 0x7f5c40155540 2024-09-21T00:29:23.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.971+0000 7f5c45ef2640 1 -- 172.21.15.112:0/712978518 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c400cd2b0 con 0x7f5c40155540 2024-09-21T00:29:23.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.972+0000 7f5c27fff640 1 -- 172.21.15.112:0/712978518 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5c3803d030 con 0x7f5c40155540 2024-09-21T00:29:23.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.972+0000 7f5c27fff640 1 --2- 172.21.15.112:0/712978518 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f5c2c078b90 0x7f5c2c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:23.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.973+0000 7f5c27fff640 1 -- 172.21.15.112:0/712978518 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6101+0+0 (secure 0 0 0) 0x7f5c380ba5b0 con 0x7f5c40155540 2024-09-21T00:29:23.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.973+0000 7f5c3f7fe640 1 --2- 172.21.15.112:0/712978518 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f5c2c078b90 0x7f5c2c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:23.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.973+0000 7f5c3f7fe640 1 --2- 172.21.15.112:0/712978518 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f5c2c078b90 0x7f5c2c07b0a0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f5c30009a90 tx=0x7f5c30005f90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:23.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:23.977+0000 7f5c27fff640 1 -- 172.21.15.112:0/712978518 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5c400cd2b0 con 0x7f5c40155540 2024-09-21T00:29:24.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:24.150+0000 7f5c45ef2640 1 -- 172.21.15.112:0/712978518 --> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5c400c64b0 con 0x7f5c2c078b90 2024-09-21T00:29:24.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:24.155+0000 7f5c27fff640 1 -- 172.21.15.112:0/712978518 <== mgr.4103 v2:172.21.15.187:6800/2921522906 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+250705 (secure 0 0 0) 0x7f5c400c64b0 con 0x7f5c2c078b90 2024-09-21T00:29:24.159 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:29:24.161 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2024-09-21T00:29:24.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:24.159+0000 7f5c45ef2640 1 -- 172.21.15.112:0/712978518 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f5c2c078b90 msgr2=0x7f5c2c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:24.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:24.159+0000 7f5c45ef2640 1 --2- 172.21.15.112:0/712978518 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f5c2c078b90 0x7f5c2c07b0a0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f5c30009a90 tx=0x7f5c30005f90 comp rx=0 tx=0).stop 2024-09-21T00:29:24.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:24.159+0000 7f5c45ef2640 1 -- 172.21.15.112:0/712978518 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c40155540 msgr2=0x7f5c400b0a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:24.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:24.159+0000 7f5c45ef2640 1 --2- 172.21.15.112:0/712978518 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c40155540 0x7f5c400b0a90 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f5c3800c880 tx=0x7f5c3802b570 comp rx=0 tx=0).stop 2024-09-21T00:29:24.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:24.159+0000 7f5c45ef2640 1 -- 172.21.15.112:0/712978518 shutdown_connections 2024-09-21T00:29:24.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:24.159+0000 7f5c45ef2640 1 --2- 172.21.15.112:0/712978518 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f5c2c078b90 0x7f5c2c07b0a0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:24.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:24.159+0000 7f5c45ef2640 1 --2- 172.21.15.112:0/712978518 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c40155540 0x7f5c400b0a90 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:24.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:24.159+0000 7f5c45ef2640 1 --2- 172.21.15.112:0/712978518 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5c400ac8a0 0x7f5c40155000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:24.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:24.159+0000 7f5c45ef2640 1 --2- 172.21.15.112:0/712978518 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5c401f8000 0x7f5c4014f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:24.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:24.159+0000 7f5c45ef2640 1 -- 172.21.15.112:0/712978518 >> 172.21.15.112:0/712978518 conn(0x7f5c400b42e0 msgr2=0x7f5c4015ab10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:24.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:24.159+0000 7f5c45ef2640 1 -- 172.21.15.112:0/712978518 shutdown_connections 2024-09-21T00:29:24.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:24.159+0000 7f5c45ef2640 1 -- 172.21.15.112:0/712978518 wait complete. 2024-09-21T00:29:24.172 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":202,"stamp":"2024-09-21T00:29:23.144482+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592744,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":137,"num_write_kb":1635,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":105,"ondisk_log_size":105,"up":180,"acting":180,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":108,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":259712,"kb_used_data":10192,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754715008,"statfs":{"total":773094113280,"available":772828168192,"internally_reserved":0,"allocated":10436608,"data_stored":5343879,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2376,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.368600"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.807159+0000","last_change":"2024-09-21T00:29:19.813350+0000","last_active":"2024-09-21T00:29:20.807159+0000","last_peered":"2024-09-21T00:29:20.807159+0000","last_clean":"2024-09-21T00:29:20.807159+0000","last_became_active":"2024-09-21T00:29:19.812123+0000","last_became_peered":"2024-09-21T00:29:19.812123+0000","last_unstale":"2024-09-21T00:29:20.807159+0000","last_undegraded":"2024-09-21T00:29:20.807159+0000","last_fullsized":"2024-09-21T00:29:20.807159+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:17:13.647509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022537+0000","last_change":"2024-09-21T00:29:19.807870+0000","last_active":"2024-09-21T00:29:21.022537+0000","last_peered":"2024-09-21T00:29:21.022537+0000","last_clean":"2024-09-21T00:29:21.022537+0000","last_became_active":"2024-09-21T00:29:19.807563+0000","last_became_peered":"2024-09-21T00:29:19.807563+0000","last_unstale":"2024-09-21T00:29:21.022537+0000","last_undegraded":"2024-09-21T00:29:21.022537+0000","last_fullsized":"2024-09-21T00:29:21.022537+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:26:29.645186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.807064+0000","last_change":"2024-09-21T00:29:19.804321+0000","last_active":"2024-09-21T00:29:20.807064+0000","last_peered":"2024-09-21T00:29:20.807064+0000","last_clean":"2024-09-21T00:29:20.807064+0000","last_became_active":"2024-09-21T00:29:19.804013+0000","last_became_peered":"2024-09-21T00:29:19.804013+0000","last_unstale":"2024-09-21T00:29:20.807064+0000","last_undegraded":"2024-09-21T00:29:20.807064+0000","last_fullsized":"2024-09-21T00:29:20.807064+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:47:15.426188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.021814+0000","last_change":"2024-09-21T00:29:19.811466+0000","last_active":"2024-09-21T00:29:21.021814+0000","last_peered":"2024-09-21T00:29:21.021814+0000","last_clean":"2024-09-21T00:29:21.021814+0000","last_became_active":"2024-09-21T00:29:19.811130+0000","last_became_peered":"2024-09-21T00:29:19.811130+0000","last_unstale":"2024-09-21T00:29:21.021814+0000","last_undegraded":"2024-09-21T00:29:21.021814+0000","last_fullsized":"2024-09-21T00:29:21.021814+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:44:42.287171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:18.775092+0000","last_change":"2024-09-21T00:29:18.775092+0000","last_active":"2024-09-21T00:29:18.775092+0000","last_peered":"2024-09-21T00:29:18.775092+0000","last_clean":"2024-09-21T00:29:18.775092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:18.775092+0000","last_undegraded":"2024-09-21T00:29:18.775092+0000","last_fullsized":"2024-09-21T00:29:18.775092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022508+0000","last_change":"2024-09-21T00:29:19.822580+0000","last_active":"2024-09-21T00:29:21.022508+0000","last_peered":"2024-09-21T00:29:21.022508+0000","last_clean":"2024-09-21T00:29:21.022508+0000","last_became_active":"2024-09-21T00:29:19.822239+0000","last_became_peered":"2024-09-21T00:29:19.822239+0000","last_unstale":"2024-09-21T00:29:21.022508+0000","last_undegraded":"2024-09-21T00:29:21.022508+0000","last_fullsized":"2024-09-21T00:29:21.022508+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:46:09.814298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.021489+0000","last_change":"2024-09-21T00:29:19.822649+0000","last_active":"2024-09-21T00:29:21.021489+0000","last_peered":"2024-09-21T00:29:21.021489+0000","last_clean":"2024-09-21T00:29:21.021489+0000","last_became_active":"2024-09-21T00:29:19.822194+0000","last_became_peered":"2024-09-21T00:29:19.822194+0000","last_unstale":"2024-09-21T00:29:21.021489+0000","last_undegraded":"2024-09-21T00:29:21.021489+0000","last_fullsized":"2024-09-21T00:29:21.021489+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:09:42.739510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022777+0000","last_change":"2024-09-21T00:29:19.816934+0000","last_active":"2024-09-21T00:29:21.022777+0000","last_peered":"2024-09-21T00:29:21.022777+0000","last_clean":"2024-09-21T00:29:21.022777+0000","last_became_active":"2024-09-21T00:29:19.815942+0000","last_became_peered":"2024-09-21T00:29:19.815942+0000","last_unstale":"2024-09-21T00:29:21.022777+0000","last_undegraded":"2024-09-21T00:29:21.022777+0000","last_fullsized":"2024-09-21T00:29:21.022777+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:35:25.434622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022456+0000","last_change":"2024-09-21T00:29:19.816919+0000","last_active":"2024-09-21T00:29:21.022456+0000","last_peered":"2024-09-21T00:29:21.022456+0000","last_clean":"2024-09-21T00:29:21.022456+0000","last_became_active":"2024-09-21T00:29:19.816530+0000","last_became_peered":"2024-09-21T00:29:19.816530+0000","last_unstale":"2024-09-21T00:29:21.022456+0000","last_undegraded":"2024-09-21T00:29:21.022456+0000","last_fullsized":"2024-09-21T00:29:21.022456+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:40:46.095060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:18.775092+0000","last_change":"2024-09-21T00:29:18.775092+0000","last_active":"2024-09-21T00:29:18.775092+0000","last_peered":"2024-09-21T00:29:18.775092+0000","last_clean":"2024-09-21T00:29:18.775092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:18.775092+0000","last_undegraded":"2024-09-21T00:29:18.775092+0000","last_fullsized":"2024-09-21T00:29:18.775092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.809793+0000","last_change":"2024-09-21T00:29:19.814490+0000","last_active":"2024-09-21T00:29:20.809793+0000","last_peered":"2024-09-21T00:29:20.809793+0000","last_clean":"2024-09-21T00:29:20.809793+0000","last_became_active":"2024-09-21T00:29:19.813188+0000","last_became_peered":"2024-09-21T00:29:19.813188+0000","last_unstale":"2024-09-21T00:29:20.809793+0000","last_undegraded":"2024-09-21T00:29:20.809793+0000","last_fullsized":"2024-09-21T00:29:20.809793+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:59:50.910192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.810780+0000","last_change":"2024-09-21T00:29:19.811796+0000","last_active":"2024-09-21T00:29:20.810780+0000","last_peered":"2024-09-21T00:29:20.810780+0000","last_clean":"2024-09-21T00:29:20.810780+0000","last_became_active":"2024-09-21T00:29:19.811496+0000","last_became_peered":"2024-09-21T00:29:19.811496+0000","last_unstale":"2024-09-21T00:29:20.810780+0000","last_undegraded":"2024-09-21T00:29:20.810780+0000","last_fullsized":"2024-09-21T00:29:20.810780+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:59:41.316661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022019+0000","last_change":"2024-09-21T00:29:19.818860+0000","last_active":"2024-09-21T00:29:21.022019+0000","last_peered":"2024-09-21T00:29:21.022019+0000","last_clean":"2024-09-21T00:29:21.022019+0000","last_became_active":"2024-09-21T00:29:19.818552+0000","last_became_peered":"2024-09-21T00:29:19.818552+0000","last_unstale":"2024-09-21T00:29:21.022019+0000","last_undegraded":"2024-09-21T00:29:21.022019+0000","last_fullsized":"2024-09-21T00:29:21.022019+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:37:22.821347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.021673+0000","last_change":"2024-09-21T00:29:19.820196+0000","last_active":"2024-09-21T00:29:21.021673+0000","last_peered":"2024-09-21T00:29:21.021673+0000","last_clean":"2024-09-21T00:29:21.021673+0000","last_became_active":"2024-09-21T00:29:19.819691+0000","last_became_peered":"2024-09-21T00:29:19.819691+0000","last_unstale":"2024-09-21T00:29:21.021673+0000","last_undegraded":"2024-09-21T00:29:21.021673+0000","last_fullsized":"2024-09-21T00:29:21.021673+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:48:11.528107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.021642+0000","last_change":"2024-09-21T00:29:19.810601+0000","last_active":"2024-09-21T00:29:21.021642+0000","last_peered":"2024-09-21T00:29:21.021642+0000","last_clean":"2024-09-21T00:29:21.021642+0000","last_became_active":"2024-09-21T00:29:19.810272+0000","last_became_peered":"2024-09-21T00:29:19.810272+0000","last_unstale":"2024-09-21T00:29:21.021642+0000","last_undegraded":"2024-09-21T00:29:21.021642+0000","last_fullsized":"2024-09-21T00:29:21.021642+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:28:10.979181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.021511+0000","last_change":"2024-09-21T00:29:19.818725+0000","last_active":"2024-09-21T00:29:21.021511+0000","last_peered":"2024-09-21T00:29:21.021511+0000","last_clean":"2024-09-21T00:29:21.021511+0000","last_became_active":"2024-09-21T00:29:19.818406+0000","last_became_peered":"2024-09-21T00:29:19.818406+0000","last_unstale":"2024-09-21T00:29:21.021511+0000","last_undegraded":"2024-09-21T00:29:21.021511+0000","last_fullsized":"2024-09-21T00:29:21.021511+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:13:53.211350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.807387+0000","last_change":"2024-09-21T00:29:19.803557+0000","last_active":"2024-09-21T00:29:20.807387+0000","last_peered":"2024-09-21T00:29:20.807387+0000","last_clean":"2024-09-21T00:29:20.807387+0000","last_became_active":"2024-09-21T00:29:19.802868+0000","last_became_peered":"2024-09-21T00:29:19.802868+0000","last_unstale":"2024-09-21T00:29:20.807387+0000","last_undegraded":"2024-09-21T00:29:20.807387+0000","last_fullsized":"2024-09-21T00:29:20.807387+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:17:38.099624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022370+0000","last_change":"2024-09-21T00:29:19.811473+0000","last_active":"2024-09-21T00:29:21.022370+0000","last_peered":"2024-09-21T00:29:21.022370+0000","last_clean":"2024-09-21T00:29:21.022370+0000","last_became_active":"2024-09-21T00:29:19.811129+0000","last_became_peered":"2024-09-21T00:29:19.811129+0000","last_unstale":"2024-09-21T00:29:21.022370+0000","last_undegraded":"2024-09-21T00:29:21.022370+0000","last_fullsized":"2024-09-21T00:29:21.022370+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:26.631458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.807233+0000","last_change":"2024-09-21T00:29:19.809929+0000","last_active":"2024-09-21T00:29:20.807233+0000","last_peered":"2024-09-21T00:29:20.807233+0000","last_clean":"2024-09-21T00:29:20.807233+0000","last_became_active":"2024-09-21T00:29:19.809541+0000","last_became_peered":"2024-09-21T00:29:19.809541+0000","last_unstale":"2024-09-21T00:29:20.807233+0000","last_undegraded":"2024-09-21T00:29:20.807233+0000","last_fullsized":"2024-09-21T00:29:20.807233+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:34:11.115991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.810853+0000","last_change":"2024-09-21T00:29:19.808754+0000","last_active":"2024-09-21T00:29:20.810853+0000","last_peered":"2024-09-21T00:29:20.810853+0000","last_clean":"2024-09-21T00:29:20.810853+0000","last_became_active":"2024-09-21T00:29:19.807857+0000","last_became_peered":"2024-09-21T00:29:19.807857+0000","last_unstale":"2024-09-21T00:29:20.810853+0000","last_undegraded":"2024-09-21T00:29:20.810853+0000","last_fullsized":"2024-09-21T00:29:20.810853+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:53:45.848291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.023011+0000","last_change":"2024-09-21T00:29:19.815917+0000","last_active":"2024-09-21T00:29:21.023011+0000","last_peered":"2024-09-21T00:29:21.023011+0000","last_clean":"2024-09-21T00:29:21.023011+0000","last_became_active":"2024-09-21T00:29:19.815559+0000","last_became_peered":"2024-09-21T00:29:19.815559+0000","last_unstale":"2024-09-21T00:29:21.023011+0000","last_undegraded":"2024-09-21T00:29:21.023011+0000","last_fullsized":"2024-09-21T00:29:21.023011+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:47:06.729633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:18.775092+0000","last_change":"2024-09-21T00:29:18.775092+0000","last_active":"2024-09-21T00:29:18.775092+0000","last_peered":"2024-09-21T00:29:18.775092+0000","last_clean":"2024-09-21T00:29:18.775092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:18.775092+0000","last_undegraded":"2024-09-21T00:29:18.775092+0000","last_fullsized":"2024-09-21T00:29:18.775092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.807054+0000","last_change":"2024-09-21T00:29:19.802421+0000","last_active":"2024-09-21T00:29:20.807054+0000","last_peered":"2024-09-21T00:29:20.807054+0000","last_clean":"2024-09-21T00:29:20.807054+0000","last_became_active":"2024-09-21T00:29:19.802074+0000","last_became_peered":"2024-09-21T00:29:19.802074+0000","last_unstale":"2024-09-21T00:29:20.807054+0000","last_undegraded":"2024-09-21T00:29:20.807054+0000","last_fullsized":"2024-09-21T00:29:20.807054+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:55:22.327108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:18.775092+0000","last_change":"2024-09-21T00:29:18.775092+0000","last_active":"2024-09-21T00:29:18.775092+0000","last_peered":"2024-09-21T00:29:18.775092+0000","last_clean":"2024-09-21T00:29:18.775092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:18.775092+0000","last_undegraded":"2024-09-21T00:29:18.775092+0000","last_fullsized":"2024-09-21T00:29:18.775092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.810198+0000","last_change":"2024-09-21T00:29:19.802920+0000","last_active":"2024-09-21T00:29:20.810198+0000","last_peered":"2024-09-21T00:29:20.810198+0000","last_clean":"2024-09-21T00:29:20.810198+0000","last_became_active":"2024-09-21T00:29:19.802584+0000","last_became_peered":"2024-09-21T00:29:19.802584+0000","last_unstale":"2024-09-21T00:29:20.810198+0000","last_undegraded":"2024-09-21T00:29:20.810198+0000","last_fullsized":"2024-09-21T00:29:20.810198+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:38:27.387471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.810402+0000","last_change":"2024-09-21T00:29:19.808064+0000","last_active":"2024-09-21T00:29:20.810402+0000","last_peered":"2024-09-21T00:29:20.810402+0000","last_clean":"2024-09-21T00:29:20.810402+0000","last_became_active":"2024-09-21T00:29:19.807712+0000","last_became_peered":"2024-09-21T00:29:19.807712+0000","last_unstale":"2024-09-21T00:29:20.810402+0000","last_undegraded":"2024-09-21T00:29:20.810402+0000","last_fullsized":"2024-09-21T00:29:20.810402+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:45:54.930710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.810053+0000","last_change":"2024-09-21T00:29:19.811883+0000","last_active":"2024-09-21T00:29:20.810053+0000","last_peered":"2024-09-21T00:29:20.810053+0000","last_clean":"2024-09-21T00:29:20.810053+0000","last_became_active":"2024-09-21T00:29:19.811584+0000","last_became_peered":"2024-09-21T00:29:19.811584+0000","last_unstale":"2024-09-21T00:29:20.810053+0000","last_undegraded":"2024-09-21T00:29:20.810053+0000","last_fullsized":"2024-09-21T00:29:20.810053+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:00:59.593942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022016+0000","last_change":"2024-09-21T00:29:19.816999+0000","last_active":"2024-09-21T00:29:21.022016+0000","last_peered":"2024-09-21T00:29:21.022016+0000","last_clean":"2024-09-21T00:29:21.022016+0000","last_became_active":"2024-09-21T00:29:19.814905+0000","last_became_peered":"2024-09-21T00:29:19.814905+0000","last_unstale":"2024-09-21T00:29:21.022016+0000","last_undegraded":"2024-09-21T00:29:21.022016+0000","last_fullsized":"2024-09-21T00:29:21.022016+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:46:57.412342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.023130+0000","last_change":"2024-09-21T00:29:19.815104+0000","last_active":"2024-09-21T00:29:21.023130+0000","last_peered":"2024-09-21T00:29:21.023130+0000","last_clean":"2024-09-21T00:29:21.023130+0000","last_became_active":"2024-09-21T00:29:19.814806+0000","last_became_peered":"2024-09-21T00:29:19.814806+0000","last_unstale":"2024-09-21T00:29:21.023130+0000","last_undegraded":"2024-09-21T00:29:21.023130+0000","last_fullsized":"2024-09-21T00:29:21.023130+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:14:39.263553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.810509+0000","last_change":"2024-09-21T00:29:19.802931+0000","last_active":"2024-09-21T00:29:20.810509+0000","last_peered":"2024-09-21T00:29:20.810509+0000","last_clean":"2024-09-21T00:29:20.810509+0000","last_became_active":"2024-09-21T00:29:19.802606+0000","last_became_peered":"2024-09-21T00:29:19.802606+0000","last_unstale":"2024-09-21T00:29:20.810509+0000","last_undegraded":"2024-09-21T00:29:20.810509+0000","last_fullsized":"2024-09-21T00:29:20.810509+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:56:07.296305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022325+0000","last_change":"2024-09-21T00:29:19.812717+0000","last_active":"2024-09-21T00:29:21.022325+0000","last_peered":"2024-09-21T00:29:21.022325+0000","last_clean":"2024-09-21T00:29:21.022325+0000","last_became_active":"2024-09-21T00:29:19.812402+0000","last_became_peered":"2024-09-21T00:29:19.812402+0000","last_unstale":"2024-09-21T00:29:21.022325+0000","last_undegraded":"2024-09-21T00:29:21.022325+0000","last_fullsized":"2024-09-21T00:29:21.022325+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:26:39.938614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022839+0000","last_change":"2024-09-21T00:29:19.816588+0000","last_active":"2024-09-21T00:29:21.022839+0000","last_peered":"2024-09-21T00:29:21.022839+0000","last_clean":"2024-09-21T00:29:21.022839+0000","last_became_active":"2024-09-21T00:29:19.815574+0000","last_became_peered":"2024-09-21T00:29:19.815574+0000","last_unstale":"2024-09-21T00:29:21.022839+0000","last_undegraded":"2024-09-21T00:29:21.022839+0000","last_fullsized":"2024-09-21T00:29:21.022839+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:47:08.243928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.807629+0000","last_change":"2024-09-21T00:29:19.810747+0000","last_active":"2024-09-21T00:29:20.807629+0000","last_peered":"2024-09-21T00:29:20.807629+0000","last_clean":"2024-09-21T00:29:20.807629+0000","last_became_active":"2024-09-21T00:29:19.810429+0000","last_became_peered":"2024-09-21T00:29:19.810429+0000","last_unstale":"2024-09-21T00:29:20.807629+0000","last_undegraded":"2024-09-21T00:29:20.807629+0000","last_fullsized":"2024-09-21T00:29:20.807629+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:21:05.578580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"93'4","reported_seq":19,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022399+0000","last_change":"2024-09-21T00:29:18.798208+0000","last_active":"2024-09-21T00:29:21.022399+0000","last_peered":"2024-09-21T00:29:21.022399+0000","last_clean":"2024-09-21T00:29:21.022399+0000","last_became_active":"2024-09-21T00:29:18.797857+0000","last_became_peered":"2024-09-21T00:29:18.797857+0000","last_unstale":"2024-09-21T00:29:21.022399+0000","last_undegraded":"2024-09-21T00:29:21.022399+0000","last_fullsized":"2024-09-21T00:29:21.022399+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T00:37:31.452087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.021779+0000","last_change":"2024-09-21T00:29:19.822277+0000","last_active":"2024-09-21T00:29:21.021779+0000","last_peered":"2024-09-21T00:29:21.021779+0000","last_clean":"2024-09-21T00:29:21.021779+0000","last_became_active":"2024-09-21T00:29:19.821917+0000","last_became_peered":"2024-09-21T00:29:19.821917+0000","last_unstale":"2024-09-21T00:29:21.021779+0000","last_undegraded":"2024-09-21T00:29:21.021779+0000","last_fullsized":"2024-09-21T00:29:21.021779+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:04:07.143491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"93'1","reported_seq":16,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022961+0000","last_change":"2024-09-21T00:29:18.799882+0000","last_active":"2024-09-21T00:29:21.022961+0000","last_peered":"2024-09-21T00:29:21.022961+0000","last_clean":"2024-09-21T00:29:21.022961+0000","last_became_active":"2024-09-21T00:29:18.799524+0000","last_became_peered":"2024-09-21T00:29:18.799524+0000","last_unstale":"2024-09-21T00:29:21.022961+0000","last_undegraded":"2024-09-21T00:29:21.022961+0000","last_fullsized":"2024-09-21T00:29:21.022961+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:39:17.740976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.810993+0000","last_change":"2024-09-21T00:29:19.809700+0000","last_active":"2024-09-21T00:29:20.810993+0000","last_peered":"2024-09-21T00:29:20.810993+0000","last_clean":"2024-09-21T00:29:20.810993+0000","last_became_active":"2024-09-21T00:29:19.809343+0000","last_became_peered":"2024-09-21T00:29:19.809343+0000","last_unstale":"2024-09-21T00:29:20.810993+0000","last_undegraded":"2024-09-21T00:29:20.810993+0000","last_fullsized":"2024-09-21T00:29:20.810993+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:28:50.502430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"93'2","reported_seq":17,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.810949+0000","last_change":"2024-09-21T00:29:18.797271+0000","last_active":"2024-09-21T00:29:20.810949+0000","last_peered":"2024-09-21T00:29:20.810949+0000","last_clean":"2024-09-21T00:29:20.810949+0000","last_became_active":"2024-09-21T00:29:18.796656+0000","last_became_peered":"2024-09-21T00:29:18.796656+0000","last_unstale":"2024-09-21T00:29:20.810949+0000","last_undegraded":"2024-09-21T00:29:20.810949+0000","last_fullsized":"2024-09-21T00:29:20.810949+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:28:00.490269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.810191+0000","last_change":"2024-09-21T00:29:19.802323+0000","last_active":"2024-09-21T00:29:20.810191+0000","last_peered":"2024-09-21T00:29:20.810191+0000","last_clean":"2024-09-21T00:29:20.810191+0000","last_became_active":"2024-09-21T00:29:19.801998+0000","last_became_peered":"2024-09-21T00:29:19.801998+0000","last_unstale":"2024-09-21T00:29:20.810191+0000","last_undegraded":"2024-09-21T00:29:20.810191+0000","last_fullsized":"2024-09-21T00:29:20.810191+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:31:43.191165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-09-21T00:29:17.783208+0000","last_change":"2024-09-21T00:29:17.783208+0000","last_active":"2024-09-21T00:29:17.769711+0000","last_peered":"2024-09-21T00:29:17.769711+0000","last_clean":"2024-09-21T00:29:17.769711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:17.783208+0000","last_undegraded":"2024-09-21T00:29:17.783208+0000","last_fullsized":"2024-09-21T00:29:17.783208+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022055+0000","last_change":"2024-09-21T00:29:19.821679+0000","last_active":"2024-09-21T00:29:21.022055+0000","last_peered":"2024-09-21T00:29:21.022055+0000","last_clean":"2024-09-21T00:29:21.022055+0000","last_became_active":"2024-09-21T00:29:19.821331+0000","last_became_peered":"2024-09-21T00:29:19.821331+0000","last_unstale":"2024-09-21T00:29:21.022055+0000","last_undegraded":"2024-09-21T00:29:21.022055+0000","last_fullsized":"2024-09-21T00:29:21.022055+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:13:53.211350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"93'1","reported_seq":16,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.021871+0000","last_change":"2024-09-21T00:29:18.797442+0000","last_active":"2024-09-21T00:29:21.021871+0000","last_peered":"2024-09-21T00:29:21.021871+0000","last_clean":"2024-09-21T00:29:21.021871+0000","last_became_active":"2024-09-21T00:29:18.797128+0000","last_became_peered":"2024-09-21T00:29:18.797128+0000","last_unstale":"2024-09-21T00:29:21.021871+0000","last_undegraded":"2024-09-21T00:29:21.021871+0000","last_fullsized":"2024-09-21T00:29:21.021871+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:22:52.216862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.807399+0000","last_change":"2024-09-21T00:29:19.808878+0000","last_active":"2024-09-21T00:29:20.807399+0000","last_peered":"2024-09-21T00:29:20.807399+0000","last_clean":"2024-09-21T00:29:20.807399+0000","last_became_active":"2024-09-21T00:29:19.808577+0000","last_became_peered":"2024-09-21T00:29:19.808577+0000","last_unstale":"2024-09-21T00:29:20.807399+0000","last_undegraded":"2024-09-21T00:29:20.807399+0000","last_fullsized":"2024-09-21T00:29:20.807399+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:34:29.883537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"93'1","reported_seq":16,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.810090+0000","last_change":"2024-09-21T00:29:18.801027+0000","last_active":"2024-09-21T00:29:20.810090+0000","last_peered":"2024-09-21T00:29:20.810090+0000","last_clean":"2024-09-21T00:29:20.810090+0000","last_became_active":"2024-09-21T00:29:18.800693+0000","last_became_peered":"2024-09-21T00:29:18.800693+0000","last_unstale":"2024-09-21T00:29:20.810090+0000","last_undegraded":"2024-09-21T00:29:20.810090+0000","last_fullsized":"2024-09-21T00:29:20.810090+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:17:06.642779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022624+0000","last_change":"2024-09-21T00:29:19.815242+0000","last_active":"2024-09-21T00:29:21.022624+0000","last_peered":"2024-09-21T00:29:21.022624+0000","last_clean":"2024-09-21T00:29:21.022624+0000","last_became_active":"2024-09-21T00:29:19.814858+0000","last_became_peered":"2024-09-21T00:29:19.814858+0000","last_unstale":"2024-09-21T00:29:21.022624+0000","last_undegraded":"2024-09-21T00:29:21.022624+0000","last_fullsized":"2024-09-21T00:29:21.022624+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:35:06.851240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.809872+0000","last_change":"2024-09-21T00:29:18.797155+0000","last_active":"2024-09-21T00:29:20.809872+0000","last_peered":"2024-09-21T00:29:20.809872+0000","last_clean":"2024-09-21T00:29:20.809872+0000","last_became_active":"2024-09-21T00:29:18.795953+0000","last_became_peered":"2024-09-21T00:29:18.795953+0000","last_unstale":"2024-09-21T00:29:20.809872+0000","last_undegraded":"2024-09-21T00:29:20.809872+0000","last_fullsized":"2024-09-21T00:29:20.809872+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T00:54:16.170396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022428+0000","last_change":"2024-09-21T00:29:19.817180+0000","last_active":"2024-09-21T00:29:21.022428+0000","last_peered":"2024-09-21T00:29:21.022428+0000","last_clean":"2024-09-21T00:29:21.022428+0000","last_became_active":"2024-09-21T00:29:19.816859+0000","last_became_peered":"2024-09-21T00:29:19.816859+0000","last_unstale":"2024-09-21T00:29:21.022428+0000","last_undegraded":"2024-09-21T00:29:21.022428+0000","last_fullsized":"2024-09-21T00:29:21.022428+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:30:01.904485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.021605+0000","last_change":"2024-09-21T00:29:18.793095+0000","last_active":"2024-09-21T00:29:21.021605+0000","last_peered":"2024-09-21T00:29:21.021605+0000","last_clean":"2024-09-21T00:29:21.021605+0000","last_became_active":"2024-09-21T00:29:18.792693+0000","last_became_peered":"2024-09-21T00:29:18.792693+0000","last_unstale":"2024-09-21T00:29:21.021605+0000","last_undegraded":"2024-09-21T00:29:21.021605+0000","last_fullsized":"2024-09-21T00:29:21.021605+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:56:18.147899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.807542+0000","last_change":"2024-09-21T00:29:19.815545+0000","last_active":"2024-09-21T00:29:20.807542+0000","last_peered":"2024-09-21T00:29:20.807542+0000","last_clean":"2024-09-21T00:29:20.807542+0000","last_became_active":"2024-09-21T00:29:19.811424+0000","last_became_peered":"2024-09-21T00:29:19.811424+0000","last_unstale":"2024-09-21T00:29:20.807542+0000","last_undegraded":"2024-09-21T00:29:20.807542+0000","last_fullsized":"2024-09-21T00:29:20.807542+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:20:57.334693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.810327+0000","last_change":"2024-09-21T00:29:18.796965+0000","last_active":"2024-09-21T00:29:20.810327+0000","last_peered":"2024-09-21T00:29:20.810327+0000","last_clean":"2024-09-21T00:29:20.810327+0000","last_became_active":"2024-09-21T00:29:18.796548+0000","last_became_peered":"2024-09-21T00:29:18.796548+0000","last_unstale":"2024-09-21T00:29:20.810327+0000","last_undegraded":"2024-09-21T00:29:20.810327+0000","last_fullsized":"2024-09-21T00:29:20.810327+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:00:18.177473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:18.775092+0000","last_change":"2024-09-21T00:29:18.775092+0000","last_active":"2024-09-21T00:29:18.775092+0000","last_peered":"2024-09-21T00:29:18.775092+0000","last_clean":"2024-09-21T00:29:18.775092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:18.775092+0000","last_undegraded":"2024-09-21T00:29:18.775092+0000","last_fullsized":"2024-09-21T00:29:18.775092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.807185+0000","last_change":"2024-09-21T00:29:18.798831+0000","last_active":"2024-09-21T00:29:20.807185+0000","last_peered":"2024-09-21T00:29:20.807185+0000","last_clean":"2024-09-21T00:29:20.807185+0000","last_became_active":"2024-09-21T00:29:18.798551+0000","last_became_peered":"2024-09-21T00:29:18.798551+0000","last_unstale":"2024-09-21T00:29:20.807185+0000","last_undegraded":"2024-09-21T00:29:20.807185+0000","last_fullsized":"2024-09-21T00:29:20.807185+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:12:30.180725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022749+0000","last_change":"2024-09-21T00:29:19.812082+0000","last_active":"2024-09-21T00:29:21.022749+0000","last_peered":"2024-09-21T00:29:21.022749+0000","last_clean":"2024-09-21T00:29:21.022749+0000","last_became_active":"2024-09-21T00:29:19.811773+0000","last_became_peered":"2024-09-21T00:29:19.811773+0000","last_unstale":"2024-09-21T00:29:21.022749+0000","last_undegraded":"2024-09-21T00:29:21.022749+0000","last_fullsized":"2024-09-21T00:29:21.022749+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:17:43.069499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"93'3","reported_seq":18,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.021859+0000","last_change":"2024-09-21T00:29:18.794178+0000","last_active":"2024-09-21T00:29:21.021859+0000","last_peered":"2024-09-21T00:29:21.021859+0000","last_clean":"2024-09-21T00:29:21.021859+0000","last_became_active":"2024-09-21T00:29:18.793851+0000","last_became_peered":"2024-09-21T00:29:18.793851+0000","last_unstale":"2024-09-21T00:29:21.021859+0000","last_undegraded":"2024-09-21T00:29:21.021859+0000","last_fullsized":"2024-09-21T00:29:21.021859+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:52:27.563468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:18.775092+0000","last_change":"2024-09-21T00:29:18.775092+0000","last_active":"2024-09-21T00:29:18.775092+0000","last_peered":"2024-09-21T00:29:18.775092+0000","last_clean":"2024-09-21T00:29:18.775092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:18.775092+0000","last_undegraded":"2024-09-21T00:29:18.775092+0000","last_fullsized":"2024-09-21T00:29:18.775092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.14","version":"93'6","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.021896+0000","last_change":"2024-09-21T00:29:18.800236+0000","last_active":"2024-09-21T00:29:21.021896+0000","last_peered":"2024-09-21T00:29:21.021896+0000","last_clean":"2024-09-21T00:29:21.021896+0000","last_became_active":"2024-09-21T00:29:18.799899+0000","last_became_peered":"2024-09-21T00:29:18.799899+0000","last_unstale":"2024-09-21T00:29:21.021896+0000","last_undegraded":"2024-09-21T00:29:21.021896+0000","last_fullsized":"2024-09-21T00:29:21.021896+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:29:10.908555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.809551+0000","last_change":"2024-09-21T00:29:19.811833+0000","last_active":"2024-09-21T00:29:20.809551+0000","last_peered":"2024-09-21T00:29:20.809551+0000","last_clean":"2024-09-21T00:29:20.809551+0000","last_became_active":"2024-09-21T00:29:19.811519+0000","last_became_peered":"2024-09-21T00:29:19.811519+0000","last_unstale":"2024-09-21T00:29:20.809551+0000","last_undegraded":"2024-09-21T00:29:20.809551+0000","last_fullsized":"2024-09-21T00:29:20.809551+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:52:28.568849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-09-21T00:29:17.784261+0000","last_change":"2024-09-21T00:29:17.784261+0000","last_active":"2024-09-21T00:29:17.769711+0000","last_peered":"2024-09-21T00:29:17.769711+0000","last_clean":"2024-09-21T00:29:17.769711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:17.784261+0000","last_undegraded":"2024-09-21T00:29:17.784261+0000","last_fullsized":"2024-09-21T00:29:17.784261+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022489+0000","last_change":"2024-09-21T00:29:19.814412+0000","last_active":"2024-09-21T00:29:21.022489+0000","last_peered":"2024-09-21T00:29:21.022489+0000","last_clean":"2024-09-21T00:29:21.022489+0000","last_became_active":"2024-09-21T00:29:19.813923+0000","last_became_peered":"2024-09-21T00:29:19.813923+0000","last_unstale":"2024-09-21T00:29:21.022489+0000","last_undegraded":"2024-09-21T00:29:21.022489+0000","last_fullsized":"2024-09-21T00:29:21.022489+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:42:28.398366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022221+0000","last_change":"2024-09-21T00:29:18.788248+0000","last_active":"2024-09-21T00:29:21.022221+0000","last_peered":"2024-09-21T00:29:21.022221+0000","last_clean":"2024-09-21T00:29:21.022221+0000","last_became_active":"2024-09-21T00:29:18.787904+0000","last_became_peered":"2024-09-21T00:29:18.787904+0000","last_unstale":"2024-09-21T00:29:21.022221+0000","last_undegraded":"2024-09-21T00:29:21.022221+0000","last_fullsized":"2024-09-21T00:29:21.022221+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:21:04.691629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.023078+0000","last_change":"2024-09-21T00:29:19.814341+0000","last_active":"2024-09-21T00:29:21.023078+0000","last_peered":"2024-09-21T00:29:21.023078+0000","last_clean":"2024-09-21T00:29:21.023078+0000","last_became_active":"2024-09-21T00:29:19.814052+0000","last_became_peered":"2024-09-21T00:29:19.814052+0000","last_unstale":"2024-09-21T00:29:21.023078+0000","last_undegraded":"2024-09-21T00:29:21.023078+0000","last_fullsized":"2024-09-21T00:29:21.023078+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:58:18.707835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.021901+0000","last_change":"2024-09-21T00:29:18.800084+0000","last_active":"2024-09-21T00:29:21.021901+0000","last_peered":"2024-09-21T00:29:21.021901+0000","last_clean":"2024-09-21T00:29:21.021901+0000","last_became_active":"2024-09-21T00:29:18.799797+0000","last_became_peered":"2024-09-21T00:29:18.799797+0000","last_unstale":"2024-09-21T00:29:21.021901+0000","last_undegraded":"2024-09-21T00:29:21.021901+0000","last_fullsized":"2024-09-21T00:29:21.021901+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T00:51:35.304058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.807595+0000","last_change":"2024-09-21T00:29:19.804683+0000","last_active":"2024-09-21T00:29:20.807595+0000","last_peered":"2024-09-21T00:29:20.807595+0000","last_clean":"2024-09-21T00:29:20.807595+0000","last_became_active":"2024-09-21T00:29:19.803246+0000","last_became_peered":"2024-09-21T00:29:19.803246+0000","last_unstale":"2024-09-21T00:29:20.807595+0000","last_undegraded":"2024-09-21T00:29:20.807595+0000","last_fullsized":"2024-09-21T00:29:20.807595+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:46:26.637135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"93'8","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.810678+0000","last_change":"2024-09-21T00:29:18.796567+0000","last_active":"2024-09-21T00:29:20.810678+0000","last_peered":"2024-09-21T00:29:20.810678+0000","last_clean":"2024-09-21T00:29:20.810678+0000","last_became_active":"2024-09-21T00:29:18.796255+0000","last_became_peered":"2024-09-21T00:29:18.796255+0000","last_unstale":"2024-09-21T00:29:20.810678+0000","last_undegraded":"2024-09-21T00:29:20.810678+0000","last_fullsized":"2024-09-21T00:29:20.810678+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:04:53.438244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.023031+0000","last_change":"2024-09-21T00:29:19.817234+0000","last_active":"2024-09-21T00:29:21.023031+0000","last_peered":"2024-09-21T00:29:21.023031+0000","last_clean":"2024-09-21T00:29:21.023031+0000","last_became_active":"2024-09-21T00:29:19.816273+0000","last_became_peered":"2024-09-21T00:29:19.816273+0000","last_unstale":"2024-09-21T00:29:21.023031+0000","last_undegraded":"2024-09-21T00:29:21.023031+0000","last_fullsized":"2024-09-21T00:29:21.023031+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:45:02.009239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"93'1","reported_seq":16,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022386+0000","last_change":"2024-09-21T00:29:18.799579+0000","last_active":"2024-09-21T00:29:21.022386+0000","last_peered":"2024-09-21T00:29:21.022386+0000","last_clean":"2024-09-21T00:29:21.022386+0000","last_became_active":"2024-09-21T00:29:18.799238+0000","last_became_peered":"2024-09-21T00:29:18.799238+0000","last_unstale":"2024-09-21T00:29:21.022386+0000","last_undegraded":"2024-09-21T00:29:21.022386+0000","last_fullsized":"2024-09-21T00:29:21.022386+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:01:35.301014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022278+0000","last_change":"2024-09-21T00:29:19.814239+0000","last_active":"2024-09-21T00:29:21.022278+0000","last_peered":"2024-09-21T00:29:21.022278+0000","last_clean":"2024-09-21T00:29:21.022278+0000","last_became_active":"2024-09-21T00:29:19.813892+0000","last_became_peered":"2024-09-21T00:29:19.813892+0000","last_unstale":"2024-09-21T00:29:21.022278+0000","last_undegraded":"2024-09-21T00:29:21.022278+0000","last_fullsized":"2024-09-21T00:29:21.022278+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:02:55.200607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"93'1","reported_seq":16,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.021997+0000","last_change":"2024-09-21T00:29:18.786505+0000","last_active":"2024-09-21T00:29:21.021997+0000","last_peered":"2024-09-21T00:29:21.021997+0000","last_clean":"2024-09-21T00:29:21.021997+0000","last_became_active":"2024-09-21T00:29:18.786141+0000","last_became_peered":"2024-09-21T00:29:18.786141+0000","last_unstale":"2024-09-21T00:29:21.021997+0000","last_undegraded":"2024-09-21T00:29:21.021997+0000","last_fullsized":"2024-09-21T00:29:21.021997+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:59:49.904811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:18.775092+0000","last_change":"2024-09-21T00:29:18.775092+0000","last_active":"2024-09-21T00:29:18.775092+0000","last_peered":"2024-09-21T00:29:18.775092+0000","last_clean":"2024-09-21T00:29:18.775092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:18.775092+0000","last_undegraded":"2024-09-21T00:29:18.775092+0000","last_fullsized":"2024-09-21T00:29:18.775092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.809671+0000","last_change":"2024-09-21T00:29:18.802388+0000","last_active":"2024-09-21T00:29:20.809671+0000","last_peered":"2024-09-21T00:29:20.809671+0000","last_clean":"2024-09-21T00:29:20.809671+0000","last_became_active":"2024-09-21T00:29:18.802060+0000","last_became_peered":"2024-09-21T00:29:18.802060+0000","last_unstale":"2024-09-21T00:29:20.809671+0000","last_undegraded":"2024-09-21T00:29:20.809671+0000","last_fullsized":"2024-09-21T00:29:20.809671+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:26:13.866796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.807105+0000","last_change":"2024-09-21T00:29:19.812924+0000","last_active":"2024-09-21T00:29:20.807105+0000","last_peered":"2024-09-21T00:29:20.807105+0000","last_clean":"2024-09-21T00:29:20.807105+0000","last_became_active":"2024-09-21T00:29:19.812607+0000","last_became_peered":"2024-09-21T00:29:19.812607+0000","last_unstale":"2024-09-21T00:29:20.807105+0000","last_undegraded":"2024-09-21T00:29:20.807105+0000","last_fullsized":"2024-09-21T00:29:20.807105+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:11:09.667419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"93'2","reported_seq":17,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.023167+0000","last_change":"2024-09-21T00:29:18.796630+0000","last_active":"2024-09-21T00:29:21.023167+0000","last_peered":"2024-09-21T00:29:21.023167+0000","last_clean":"2024-09-21T00:29:21.023167+0000","last_became_active":"2024-09-21T00:29:18.796317+0000","last_became_peered":"2024-09-21T00:29:18.796317+0000","last_unstale":"2024-09-21T00:29:21.023167+0000","last_undegraded":"2024-09-21T00:29:21.023167+0000","last_fullsized":"2024-09-21T00:29:21.023167+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:50:16.709606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.807330+0000","last_change":"2024-09-21T00:29:19.801051+0000","last_active":"2024-09-21T00:29:20.807330+0000","last_peered":"2024-09-21T00:29:20.807330+0000","last_clean":"2024-09-21T00:29:20.807330+0000","last_became_active":"2024-09-21T00:29:19.798834+0000","last_became_peered":"2024-09-21T00:29:19.798834+0000","last_unstale":"2024-09-21T00:29:20.807330+0000","last_undegraded":"2024-09-21T00:29:20.807330+0000","last_fullsized":"2024-09-21T00:29:20.807330+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:20:15.757612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022535+0000","last_change":"2024-09-21T00:29:18.796311+0000","last_active":"2024-09-21T00:29:21.022535+0000","last_peered":"2024-09-21T00:29:21.022535+0000","last_clean":"2024-09-21T00:29:21.022535+0000","last_became_active":"2024-09-21T00:29:18.795425+0000","last_became_peered":"2024-09-21T00:29:18.795425+0000","last_unstale":"2024-09-21T00:29:21.022535+0000","last_undegraded":"2024-09-21T00:29:21.022535+0000","last_fullsized":"2024-09-21T00:29:21.022535+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:12:21.051042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022415+0000","last_change":"2024-09-21T00:29:19.813509+0000","last_active":"2024-09-21T00:29:21.022415+0000","last_peered":"2024-09-21T00:29:21.022415+0000","last_clean":"2024-09-21T00:29:21.022415+0000","last_became_active":"2024-09-21T00:29:19.813162+0000","last_became_peered":"2024-09-21T00:29:19.813162+0000","last_unstale":"2024-09-21T00:29:21.022415+0000","last_undegraded":"2024-09-21T00:29:21.022415+0000","last_fullsized":"2024-09-21T00:29:21.022415+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:48:11.528107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022396+0000","last_change":"2024-09-21T00:29:18.799921+0000","last_active":"2024-09-21T00:29:21.022396+0000","last_peered":"2024-09-21T00:29:21.022396+0000","last_clean":"2024-09-21T00:29:21.022396+0000","last_became_active":"2024-09-21T00:29:18.799614+0000","last_became_peered":"2024-09-21T00:29:18.799614+0000","last_unstale":"2024-09-21T00:29:21.022396+0000","last_undegraded":"2024-09-21T00:29:21.022396+0000","last_fullsized":"2024-09-21T00:29:21.022396+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:33:56.930832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022472+0000","last_change":"2024-09-21T00:29:19.813609+0000","last_active":"2024-09-21T00:29:21.022472+0000","last_peered":"2024-09-21T00:29:21.022472+0000","last_clean":"2024-09-21T00:29:21.022472+0000","last_became_active":"2024-09-21T00:29:19.813291+0000","last_became_peered":"2024-09-21T00:29:19.813291+0000","last_unstale":"2024-09-21T00:29:21.022472+0000","last_undegraded":"2024-09-21T00:29:21.022472+0000","last_fullsized":"2024-09-21T00:29:21.022472+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:48:12.895949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022964+0000","last_change":"2024-09-21T00:29:18.796499+0000","last_active":"2024-09-21T00:29:21.022964+0000","last_peered":"2024-09-21T00:29:21.022964+0000","last_clean":"2024-09-21T00:29:21.022964+0000","last_became_active":"2024-09-21T00:29:18.796175+0000","last_became_peered":"2024-09-21T00:29:18.796175+0000","last_unstale":"2024-09-21T00:29:21.022964+0000","last_undegraded":"2024-09-21T00:29:21.022964+0000","last_fullsized":"2024-09-21T00:29:21.022964+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:58:46.110270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.807498+0000","last_change":"2024-09-21T00:29:19.815935+0000","last_active":"2024-09-21T00:29:20.807498+0000","last_peered":"2024-09-21T00:29:20.807498+0000","last_clean":"2024-09-21T00:29:20.807498+0000","last_became_active":"2024-09-21T00:29:19.814883+0000","last_became_peered":"2024-09-21T00:29:19.814883+0000","last_unstale":"2024-09-21T00:29:20.807498+0000","last_undegraded":"2024-09-21T00:29:20.807498+0000","last_fullsized":"2024-09-21T00:29:20.807498+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:40:18.526456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-09-21T00:29:17.781825+0000","last_change":"2024-09-21T00:29:17.781825+0000","last_active":"2024-09-21T00:29:17.769711+0000","last_peered":"2024-09-21T00:29:17.769711+0000","last_clean":"2024-09-21T00:29:17.769711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:17.781825+0000","last_undegraded":"2024-09-21T00:29:17.781825+0000","last_fullsized":"2024-09-21T00:29:17.781825+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.809738+0000","last_change":"2024-09-21T00:29:19.806248+0000","last_active":"2024-09-21T00:29:20.809738+0000","last_peered":"2024-09-21T00:29:20.809738+0000","last_clean":"2024-09-21T00:29:20.809738+0000","last_became_active":"2024-09-21T00:29:19.805916+0000","last_became_peered":"2024-09-21T00:29:19.805916+0000","last_unstale":"2024-09-21T00:29:20.809738+0000","last_undegraded":"2024-09-21T00:29:20.809738+0000","last_fullsized":"2024-09-21T00:29:20.809738+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:42:42.562148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"93'1","reported_seq":16,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.807289+0000","last_change":"2024-09-21T00:29:18.797446+0000","last_active":"2024-09-21T00:29:20.807289+0000","last_peered":"2024-09-21T00:29:20.807289+0000","last_clean":"2024-09-21T00:29:20.807289+0000","last_became_active":"2024-09-21T00:29:18.797076+0000","last_became_peered":"2024-09-21T00:29:18.797076+0000","last_unstale":"2024-09-21T00:29:20.807289+0000","last_undegraded":"2024-09-21T00:29:20.807289+0000","last_fullsized":"2024-09-21T00:29:20.807289+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:06:17.566926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022312+0000","last_change":"2024-09-21T00:29:19.812384+0000","last_active":"2024-09-21T00:29:21.022312+0000","last_peered":"2024-09-21T00:29:21.022312+0000","last_clean":"2024-09-21T00:29:21.022312+0000","last_became_active":"2024-09-21T00:29:19.812077+0000","last_became_peered":"2024-09-21T00:29:19.812077+0000","last_unstale":"2024-09-21T00:29:21.022312+0000","last_undegraded":"2024-09-21T00:29:21.022312+0000","last_fullsized":"2024-09-21T00:29:21.022312+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:28:10.979181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022603+0000","last_change":"2024-09-21T00:29:18.796908+0000","last_active":"2024-09-21T00:29:21.022603+0000","last_peered":"2024-09-21T00:29:21.022603+0000","last_clean":"2024-09-21T00:29:21.022603+0000","last_became_active":"2024-09-21T00:29:18.796559+0000","last_became_peered":"2024-09-21T00:29:18.796559+0000","last_unstale":"2024-09-21T00:29:21.022603+0000","last_undegraded":"2024-09-21T00:29:21.022603+0000","last_fullsized":"2024-09-21T00:29:21.022603+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:27:50.568077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.809937+0000","last_change":"2024-09-21T00:29:19.808826+0000","last_active":"2024-09-21T00:29:20.809937+0000","last_peered":"2024-09-21T00:29:20.809937+0000","last_clean":"2024-09-21T00:29:20.809937+0000","last_became_active":"2024-09-21T00:29:19.808196+0000","last_became_peered":"2024-09-21T00:29:19.808196+0000","last_unstale":"2024-09-21T00:29:20.809937+0000","last_undegraded":"2024-09-21T00:29:20.809937+0000","last_fullsized":"2024-09-21T00:29:20.809937+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:22:53.222243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-09-21T00:29:17.782729+0000","last_change":"2024-09-21T00:29:17.782729+0000","last_active":"2024-09-21T00:29:17.769711+0000","last_peered":"2024-09-21T00:29:17.769711+0000","last_clean":"2024-09-21T00:29:17.769711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:17.782729+0000","last_undegraded":"2024-09-21T00:29:17.782729+0000","last_fullsized":"2024-09-21T00:29:17.782729+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.021641+0000","last_change":"2024-09-21T00:29:19.823036+0000","last_active":"2024-09-21T00:29:21.021641+0000","last_peered":"2024-09-21T00:29:21.021641+0000","last_clean":"2024-09-21T00:29:21.021641+0000","last_became_active":"2024-09-21T00:29:19.822688+0000","last_became_peered":"2024-09-21T00:29:19.822688+0000","last_unstale":"2024-09-21T00:29:21.021641+0000","last_undegraded":"2024-09-21T00:29:21.021641+0000","last_fullsized":"2024-09-21T00:29:21.021641+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:54:44.835987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.809976+0000","last_change":"2024-09-21T00:29:18.795683+0000","last_active":"2024-09-21T00:29:20.809976+0000","last_peered":"2024-09-21T00:29:20.809976+0000","last_clean":"2024-09-21T00:29:20.809976+0000","last_became_active":"2024-09-21T00:29:18.795363+0000","last_became_peered":"2024-09-21T00:29:18.795363+0000","last_unstale":"2024-09-21T00:29:20.809976+0000","last_undegraded":"2024-09-21T00:29:20.809976+0000","last_fullsized":"2024-09-21T00:29:20.809976+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:20:55.296469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022149+0000","last_change":"2024-09-21T00:29:19.812845+0000","last_active":"2024-09-21T00:29:21.022149+0000","last_peered":"2024-09-21T00:29:21.022149+0000","last_clean":"2024-09-21T00:29:21.022149+0000","last_became_active":"2024-09-21T00:29:19.812536+0000","last_became_peered":"2024-09-21T00:29:19.812536+0000","last_unstale":"2024-09-21T00:29:21.022149+0000","last_undegraded":"2024-09-21T00:29:21.022149+0000","last_fullsized":"2024-09-21T00:29:21.022149+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:48:44.631863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022864+0000","last_change":"2024-09-21T00:29:18.800233+0000","last_active":"2024-09-21T00:29:21.022864+0000","last_peered":"2024-09-21T00:29:21.022864+0000","last_clean":"2024-09-21T00:29:21.022864+0000","last_became_active":"2024-09-21T00:29:18.799952+0000","last_became_peered":"2024-09-21T00:29:18.799952+0000","last_unstale":"2024-09-21T00:29:21.022864+0000","last_undegraded":"2024-09-21T00:29:21.022864+0000","last_fullsized":"2024-09-21T00:29:21.022864+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:21:56.870500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.021841+0000","last_change":"2024-09-21T00:29:19.816067+0000","last_active":"2024-09-21T00:29:21.021841+0000","last_peered":"2024-09-21T00:29:21.021841+0000","last_clean":"2024-09-21T00:29:21.021841+0000","last_became_active":"2024-09-21T00:29:19.815714+0000","last_became_peered":"2024-09-21T00:29:19.815714+0000","last_unstale":"2024-09-21T00:29:21.021841+0000","last_undegraded":"2024-09-21T00:29:21.021841+0000","last_fullsized":"2024-09-21T00:29:21.021841+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:30:01.904485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022492+0000","last_change":"2024-09-21T00:29:18.796212+0000","last_active":"2024-09-21T00:29:21.022492+0000","last_peered":"2024-09-21T00:29:21.022492+0000","last_clean":"2024-09-21T00:29:21.022492+0000","last_became_active":"2024-09-21T00:29:18.795864+0000","last_became_peered":"2024-09-21T00:29:18.795864+0000","last_unstale":"2024-09-21T00:29:21.022492+0000","last_undegraded":"2024-09-21T00:29:21.022492+0000","last_fullsized":"2024-09-21T00:29:21.022492+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:57:43.771137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.810604+0000","last_change":"2024-09-21T00:29:19.811714+0000","last_active":"2024-09-21T00:29:20.810604+0000","last_peered":"2024-09-21T00:29:20.810604+0000","last_clean":"2024-09-21T00:29:20.810604+0000","last_became_active":"2024-09-21T00:29:19.809202+0000","last_became_peered":"2024-09-21T00:29:19.809202+0000","last_unstale":"2024-09-21T00:29:20.810604+0000","last_undegraded":"2024-09-21T00:29:20.810604+0000","last_fullsized":"2024-09-21T00:29:20.810604+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:44:42.287171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.023051+0000","last_change":"2024-09-21T00:29:18.801879+0000","last_active":"2024-09-21T00:29:21.023051+0000","last_peered":"2024-09-21T00:29:21.023051+0000","last_clean":"2024-09-21T00:29:21.023051+0000","last_became_active":"2024-09-21T00:29:18.800708+0000","last_became_peered":"2024-09-21T00:29:18.800708+0000","last_unstale":"2024-09-21T00:29:21.023051+0000","last_undegraded":"2024-09-21T00:29:21.023051+0000","last_fullsized":"2024-09-21T00:29:21.023051+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:48:40.413052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.809702+0000","last_change":"2024-09-21T00:29:19.798077+0000","last_active":"2024-09-21T00:29:20.809702+0000","last_peered":"2024-09-21T00:29:20.809702+0000","last_clean":"2024-09-21T00:29:20.809702+0000","last_became_active":"2024-09-21T00:29:19.797724+0000","last_became_peered":"2024-09-21T00:29:19.797724+0000","last_unstale":"2024-09-21T00:29:20.809702+0000","last_undegraded":"2024-09-21T00:29:20.809702+0000","last_fullsized":"2024-09-21T00:29:20.809702+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:35:16.985174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"93'2","reported_seq":17,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.023087+0000","last_change":"2024-09-21T00:29:18.800162+0000","last_active":"2024-09-21T00:29:21.023087+0000","last_peered":"2024-09-21T00:29:21.023087+0000","last_clean":"2024-09-21T00:29:21.023087+0000","last_became_active":"2024-09-21T00:29:18.798939+0000","last_became_peered":"2024-09-21T00:29:18.798939+0000","last_unstale":"2024-09-21T00:29:21.023087+0000","last_undegraded":"2024-09-21T00:29:21.023087+0000","last_fullsized":"2024-09-21T00:29:21.023087+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:28:05.362595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":244,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.021503+0000","last_change":"2024-09-21T00:25:38.725792+0000","last_active":"2024-09-21T00:29:21.021503+0000","last_peered":"2024-09-21T00:29:21.021503+0000","last_clean":"2024-09-21T00:29:21.021503+0000","last_became_active":"2024-09-21T00:25:38.724668+0000","last_became_peered":"2024-09-21T00:25:38.724668+0000","last_unstale":"2024-09-21T00:29:21.021503+0000","last_undegraded":"2024-09-21T00:29:21.021503+0000","last_fullsized":"2024-09-21T00:29:21.021503+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:37.163216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:37.163216+0000","last_clean_scrub_stamp":"2024-09-21T00:25:37.163216+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:14:36.588660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":114,"acting":114,"num_store_stats":7},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2376,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":33,"ondisk_log_size":33,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509488,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32908,"kb_used_data":1720,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338932,"statfs":{"total":96636764160,"available":96603066368,"internally_reserved":0,"allocated":1761280,"data_stored":1116174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509489,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32336,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339504,"statfs":{"total":96636764160,"available":96603652096,"internally_reserved":0,"allocated":1167360,"data_stored":521170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509488,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32308,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339532,"statfs":{"total":96636764160,"available":96603680768,"internally_reserved":0,"allocated":1146880,"data_stored":518954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509487,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32328,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339512,"statfs":{"total":96636764160,"available":96603660288,"internally_reserved":0,"allocated":1167360,"data_stored":525738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509489,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32328,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339512,"statfs":{"total":96636764160,"available":96603660288,"internally_reserved":0,"allocated":1163264,"data_stored":526228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509489,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32324,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339516,"statfs":{"total":96636764160,"available":96603664384,"internally_reserved":0,"allocated":1163264,"data_stored":525680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509489,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32904,"kb_used_data":1716,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338936,"statfs":{"total":96636764160,"available":96603070464,"internally_reserved":0,"allocated":1757184,"data_stored":1111024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574896,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32276,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339564,"statfs":{"total":96636764160,"available":96603713536,"internally_reserved":0,"allocated":1110016,"data_stored":498911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T00:29:24.176 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T00:29:24.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:24.323+0000 7f9e1a4b4640 1 -- 172.21.15.112:0/1515316612 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e1415de90 msgr2=0x7f9e1415e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:24.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:24.323+0000 7f9e1a4b4640 1 --2- 172.21.15.112:0/1515316612 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e1415de90 0x7f9e1415e2a0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f9e0c005fd0 tx=0x7f9e0c02a970 comp rx=0 tx=0).stop 2024-09-21T00:29:24.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:24.323+0000 7f9e1a4b4640 1 -- 172.21.15.112:0/1515316612 shutdown_connections 2024-09-21T00:29:24.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:24.323+0000 7f9e1a4b4640 1 --2- 172.21.15.112:0/1515316612 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e1415de90 0x7f9e1415e2a0 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:24.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:24.323+0000 7f9e1a4b4640 1 --2- 172.21.15.112:0/1515316612 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9e1415cca0 0x7f9e1415d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:24.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:24.323+0000 7f9e1a4b4640 1 -- 172.21.15.112:0/1515316612 >> 172.21.15.112:0/1515316612 conn(0x7f9e14073fe0 msgr2=0x7f9e140743f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:24.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:24.323+0000 7f9e1a4b4640 1 -- 172.21.15.112:0/1515316612 shutdown_connections 2024-09-21T00:29:24.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:24.323+0000 7f9e1a4b4640 1 -- 172.21.15.112:0/1515316612 wait complete. 2024-09-21T00:29:24.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:24.325+0000 7f9e1a4b4640 1 Processor -- start 2024-09-21T00:29:24.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:24.325+0000 7f9e1a4b4640 1 -- start start 2024-09-21T00:29:24.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:24.326+0000 7f9e1a4b4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e14156c90 0x7f9e141f6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:24.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:24.326+0000 7f9e1a4b4640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9e1415cca0 0x7f9e14147510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:24.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:24.326+0000 7f9e1a4b4640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9e1415de90 0x7f9e14147a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:24.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:24.326+0000 7f9e1a4b4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9e14070520 con 0x7f9e14156c90 2024-09-21T00:29:24.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:24.326+0000 7f9e1a4b4640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9e140703a0 con 0x7f9e1415cca0 2024-09-21T00:29:24.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:24.326+0000 7f9e1a4b4640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f9e140706a0 con 0x7f9e1415de90 2024-09-21T00:29:24.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:24.326+0000 7f9e13fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e14156c90 0x7f9e141f6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:24.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:24.326+0000 7f9e13fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e14156c90 0x7f9e141f6240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48728/0 (socket says 172.21.15.112:48728) 2024-09-21T00:29:24.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:24.326+0000 7f9e13fff640 1 -- 172.21.15.112:0/2227581039 learned_addr learned my addr 172.21.15.112:0/2227581039 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:29:24.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:24.327+0000 7f9e137fe640 1 --2- 172.21.15.112:0/2227581039 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9e1415cca0 0x7f9e14147510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:24.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:24.327+0000 7f9e18a51640 1 --2- 172.21.15.112:0/2227581039 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9e1415de90 0x7f9e14147a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:24.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:24.327+0000 7f9e13fff640 1 -- 172.21.15.112:0/2227581039 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9e1415de90 msgr2=0x7f9e14147a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:24.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:24.327+0000 7f9e13fff640 1 --2- 172.21.15.112:0/2227581039 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9e1415de90 0x7f9e14147a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:24.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:24.327+0000 7f9e13fff640 1 -- 172.21.15.112:0/2227581039 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9e1415cca0 msgr2=0x7f9e14147510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:24.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:24.327+0000 7f9e13fff640 1 --2- 172.21.15.112:0/2227581039 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9e1415cca0 0x7f9e14147510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:24.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:24.327+0000 7f9e13fff640 1 -- 172.21.15.112:0/2227581039 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e140a9550 con 0x7f9e14156c90 2024-09-21T00:29:24.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:24.327+0000 7f9e137fe640 1 --2- 172.21.15.112:0/2227581039 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9e1415cca0 0x7f9e14147510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:29:24.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:24.327+0000 7f9e13fff640 1 --2- 172.21.15.112:0/2227581039 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e14156c90 0x7f9e141f6240 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f9e0400b3f0 tx=0x7f9e0400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:24.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:24.327+0000 7f9e10ff9640 1 -- 172.21.15.112:0/2227581039 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e04015020 con 0x7f9e14156c90 2024-09-21T00:29:24.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:24.327+0000 7f9e10ff9640 1 -- 172.21.15.112:0/2227581039 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9e04011920 con 0x7f9e14156c90 2024-09-21T00:29:24.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:24.327+0000 7f9e1a4b4640 1 -- 172.21.15.112:0/2227581039 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e140a9830 con 0x7f9e14156c90 2024-09-21T00:29:24.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:24.327+0000 7f9e10ff9640 1 -- 172.21.15.112:0/2227581039 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e04011d70 con 0x7f9e14156c90 2024-09-21T00:29:24.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:24.327+0000 7f9e1a4b4640 1 -- 172.21.15.112:0/2227581039 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e140a9df0 con 0x7f9e14156c90 2024-09-21T00:29:24.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:24.328+0000 7f9e1a4b4640 1 -- 172.21.15.112:0/2227581039 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e14070c40 con 0x7f9e14156c90 2024-09-21T00:29:24.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:24.329+0000 7f9e10ff9640 1 -- 172.21.15.112:0/2227581039 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9e04018020 con 0x7f9e14156c90 2024-09-21T00:29:24.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:24.329+0000 7f9e10ff9640 1 --2- 172.21.15.112:0/2227581039 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f9e00078b90 0x7f9e0007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:24.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:24.329+0000 7f9e10ff9640 1 -- 172.21.15.112:0/2227581039 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6101+0+0 (secure 0 0 0) 0x7f9e0409abd0 con 0x7f9e14156c90 2024-09-21T00:29:24.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:24.329+0000 7f9e137fe640 1 --2- 172.21.15.112:0/2227581039 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f9e00078b90 0x7f9e0007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:24.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:24.330+0000 7f9e137fe640 1 --2- 172.21.15.112:0/2227581039 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f9e00078b90 0x7f9e0007b0a0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f9e141f3980 tx=0x7f9e08008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:24.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:24.334+0000 7f9e10ff9640 1 -- 172.21.15.112:0/2227581039 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f9e0401f030 con 0x7f9e14156c90 2024-09-21T00:29:24.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:24.509+0000 7f9e1a4b4640 1 -- 172.21.15.112:0/2227581039 --> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9e1406f6e0 con 0x7f9e00078b90 2024-09-21T00:29:24.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:24.514+0000 7f9e10ff9640 1 -- 172.21.15.112:0/2227581039 <== mgr.4103 v2:172.21.15.187:6800/2921522906 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+250705 (secure 0 0 0) 0x7f9e1406f6e0 con 0x7f9e00078b90 2024-09-21T00:29:24.518 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:29:24.519 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2024-09-21T00:29:24.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:24.517+0000 7f9e1a4b4640 1 -- 172.21.15.112:0/2227581039 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f9e00078b90 msgr2=0x7f9e0007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:24.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:24.517+0000 7f9e1a4b4640 1 --2- 172.21.15.112:0/2227581039 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f9e00078b90 0x7f9e0007b0a0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f9e141f3980 tx=0x7f9e08008040 comp rx=0 tx=0).stop 2024-09-21T00:29:24.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:24.517+0000 7f9e1a4b4640 1 -- 172.21.15.112:0/2227581039 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e14156c90 msgr2=0x7f9e141f6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:24.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:24.517+0000 7f9e1a4b4640 1 --2- 172.21.15.112:0/2227581039 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e14156c90 0x7f9e141f6240 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f9e0400b3f0 tx=0x7f9e0400b8b0 comp rx=0 tx=0).stop 2024-09-21T00:29:24.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:24.517+0000 7f9e1a4b4640 1 -- 172.21.15.112:0/2227581039 shutdown_connections 2024-09-21T00:29:24.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:24.517+0000 7f9e1a4b4640 1 --2- 172.21.15.112:0/2227581039 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9e1415de90 0x7f9e14147a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:24.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:24.517+0000 7f9e1a4b4640 1 --2- 172.21.15.112:0/2227581039 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9e1415cca0 0x7f9e14147510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:24.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:24.517+0000 7f9e1a4b4640 1 --2- 172.21.15.112:0/2227581039 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f9e00078b90 0x7f9e0007b0a0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:24.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:24.517+0000 7f9e1a4b4640 1 --2- 172.21.15.112:0/2227581039 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e14156c90 0x7f9e141f6240 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:24.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:24.517+0000 7f9e1a4b4640 1 -- 172.21.15.112:0/2227581039 >> 172.21.15.112:0/2227581039 conn(0x7f9e14073fe0 msgr2=0x7f9e14156140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:24.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:24.517+0000 7f9e1a4b4640 1 -- 172.21.15.112:0/2227581039 shutdown_connections 2024-09-21T00:29:24.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:24.517+0000 7f9e1a4b4640 1 -- 172.21.15.112:0/2227581039 wait complete. 2024-09-21T00:29:24.530 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":202,"stamp":"2024-09-21T00:29:23.144482+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592744,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":137,"num_write_kb":1635,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":105,"ondisk_log_size":105,"up":180,"acting":180,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":108,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":259712,"kb_used_data":10192,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754715008,"statfs":{"total":773094113280,"available":772828168192,"internally_reserved":0,"allocated":10436608,"data_stored":5343879,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2376,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.368600"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.807159+0000","last_change":"2024-09-21T00:29:19.813350+0000","last_active":"2024-09-21T00:29:20.807159+0000","last_peered":"2024-09-21T00:29:20.807159+0000","last_clean":"2024-09-21T00:29:20.807159+0000","last_became_active":"2024-09-21T00:29:19.812123+0000","last_became_peered":"2024-09-21T00:29:19.812123+0000","last_unstale":"2024-09-21T00:29:20.807159+0000","last_undegraded":"2024-09-21T00:29:20.807159+0000","last_fullsized":"2024-09-21T00:29:20.807159+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:17:13.647509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022537+0000","last_change":"2024-09-21T00:29:19.807870+0000","last_active":"2024-09-21T00:29:21.022537+0000","last_peered":"2024-09-21T00:29:21.022537+0000","last_clean":"2024-09-21T00:29:21.022537+0000","last_became_active":"2024-09-21T00:29:19.807563+0000","last_became_peered":"2024-09-21T00:29:19.807563+0000","last_unstale":"2024-09-21T00:29:21.022537+0000","last_undegraded":"2024-09-21T00:29:21.022537+0000","last_fullsized":"2024-09-21T00:29:21.022537+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:26:29.645186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.807064+0000","last_change":"2024-09-21T00:29:19.804321+0000","last_active":"2024-09-21T00:29:20.807064+0000","last_peered":"2024-09-21T00:29:20.807064+0000","last_clean":"2024-09-21T00:29:20.807064+0000","last_became_active":"2024-09-21T00:29:19.804013+0000","last_became_peered":"2024-09-21T00:29:19.804013+0000","last_unstale":"2024-09-21T00:29:20.807064+0000","last_undegraded":"2024-09-21T00:29:20.807064+0000","last_fullsized":"2024-09-21T00:29:20.807064+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:47:15.426188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.021814+0000","last_change":"2024-09-21T00:29:19.811466+0000","last_active":"2024-09-21T00:29:21.021814+0000","last_peered":"2024-09-21T00:29:21.021814+0000","last_clean":"2024-09-21T00:29:21.021814+0000","last_became_active":"2024-09-21T00:29:19.811130+0000","last_became_peered":"2024-09-21T00:29:19.811130+0000","last_unstale":"2024-09-21T00:29:21.021814+0000","last_undegraded":"2024-09-21T00:29:21.021814+0000","last_fullsized":"2024-09-21T00:29:21.021814+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:44:42.287171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:18.775092+0000","last_change":"2024-09-21T00:29:18.775092+0000","last_active":"2024-09-21T00:29:18.775092+0000","last_peered":"2024-09-21T00:29:18.775092+0000","last_clean":"2024-09-21T00:29:18.775092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:18.775092+0000","last_undegraded":"2024-09-21T00:29:18.775092+0000","last_fullsized":"2024-09-21T00:29:18.775092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022508+0000","last_change":"2024-09-21T00:29:19.822580+0000","last_active":"2024-09-21T00:29:21.022508+0000","last_peered":"2024-09-21T00:29:21.022508+0000","last_clean":"2024-09-21T00:29:21.022508+0000","last_became_active":"2024-09-21T00:29:19.822239+0000","last_became_peered":"2024-09-21T00:29:19.822239+0000","last_unstale":"2024-09-21T00:29:21.022508+0000","last_undegraded":"2024-09-21T00:29:21.022508+0000","last_fullsized":"2024-09-21T00:29:21.022508+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:46:09.814298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.021489+0000","last_change":"2024-09-21T00:29:19.822649+0000","last_active":"2024-09-21T00:29:21.021489+0000","last_peered":"2024-09-21T00:29:21.021489+0000","last_clean":"2024-09-21T00:29:21.021489+0000","last_became_active":"2024-09-21T00:29:19.822194+0000","last_became_peered":"2024-09-21T00:29:19.822194+0000","last_unstale":"2024-09-21T00:29:21.021489+0000","last_undegraded":"2024-09-21T00:29:21.021489+0000","last_fullsized":"2024-09-21T00:29:21.021489+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:09:42.739510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022777+0000","last_change":"2024-09-21T00:29:19.816934+0000","last_active":"2024-09-21T00:29:21.022777+0000","last_peered":"2024-09-21T00:29:21.022777+0000","last_clean":"2024-09-21T00:29:21.022777+0000","last_became_active":"2024-09-21T00:29:19.815942+0000","last_became_peered":"2024-09-21T00:29:19.815942+0000","last_unstale":"2024-09-21T00:29:21.022777+0000","last_undegraded":"2024-09-21T00:29:21.022777+0000","last_fullsized":"2024-09-21T00:29:21.022777+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:35:25.434622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022456+0000","last_change":"2024-09-21T00:29:19.816919+0000","last_active":"2024-09-21T00:29:21.022456+0000","last_peered":"2024-09-21T00:29:21.022456+0000","last_clean":"2024-09-21T00:29:21.022456+0000","last_became_active":"2024-09-21T00:29:19.816530+0000","last_became_peered":"2024-09-21T00:29:19.816530+0000","last_unstale":"2024-09-21T00:29:21.022456+0000","last_undegraded":"2024-09-21T00:29:21.022456+0000","last_fullsized":"2024-09-21T00:29:21.022456+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:40:46.095060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:18.775092+0000","last_change":"2024-09-21T00:29:18.775092+0000","last_active":"2024-09-21T00:29:18.775092+0000","last_peered":"2024-09-21T00:29:18.775092+0000","last_clean":"2024-09-21T00:29:18.775092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:18.775092+0000","last_undegraded":"2024-09-21T00:29:18.775092+0000","last_fullsized":"2024-09-21T00:29:18.775092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.809793+0000","last_change":"2024-09-21T00:29:19.814490+0000","last_active":"2024-09-21T00:29:20.809793+0000","last_peered":"2024-09-21T00:29:20.809793+0000","last_clean":"2024-09-21T00:29:20.809793+0000","last_became_active":"2024-09-21T00:29:19.813188+0000","last_became_peered":"2024-09-21T00:29:19.813188+0000","last_unstale":"2024-09-21T00:29:20.809793+0000","last_undegraded":"2024-09-21T00:29:20.809793+0000","last_fullsized":"2024-09-21T00:29:20.809793+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:59:50.910192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.810780+0000","last_change":"2024-09-21T00:29:19.811796+0000","last_active":"2024-09-21T00:29:20.810780+0000","last_peered":"2024-09-21T00:29:20.810780+0000","last_clean":"2024-09-21T00:29:20.810780+0000","last_became_active":"2024-09-21T00:29:19.811496+0000","last_became_peered":"2024-09-21T00:29:19.811496+0000","last_unstale":"2024-09-21T00:29:20.810780+0000","last_undegraded":"2024-09-21T00:29:20.810780+0000","last_fullsized":"2024-09-21T00:29:20.810780+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:59:41.316661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022019+0000","last_change":"2024-09-21T00:29:19.818860+0000","last_active":"2024-09-21T00:29:21.022019+0000","last_peered":"2024-09-21T00:29:21.022019+0000","last_clean":"2024-09-21T00:29:21.022019+0000","last_became_active":"2024-09-21T00:29:19.818552+0000","last_became_peered":"2024-09-21T00:29:19.818552+0000","last_unstale":"2024-09-21T00:29:21.022019+0000","last_undegraded":"2024-09-21T00:29:21.022019+0000","last_fullsized":"2024-09-21T00:29:21.022019+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:37:22.821347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.021673+0000","last_change":"2024-09-21T00:29:19.820196+0000","last_active":"2024-09-21T00:29:21.021673+0000","last_peered":"2024-09-21T00:29:21.021673+0000","last_clean":"2024-09-21T00:29:21.021673+0000","last_became_active":"2024-09-21T00:29:19.819691+0000","last_became_peered":"2024-09-21T00:29:19.819691+0000","last_unstale":"2024-09-21T00:29:21.021673+0000","last_undegraded":"2024-09-21T00:29:21.021673+0000","last_fullsized":"2024-09-21T00:29:21.021673+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:48:11.528107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.021642+0000","last_change":"2024-09-21T00:29:19.810601+0000","last_active":"2024-09-21T00:29:21.021642+0000","last_peered":"2024-09-21T00:29:21.021642+0000","last_clean":"2024-09-21T00:29:21.021642+0000","last_became_active":"2024-09-21T00:29:19.810272+0000","last_became_peered":"2024-09-21T00:29:19.810272+0000","last_unstale":"2024-09-21T00:29:21.021642+0000","last_undegraded":"2024-09-21T00:29:21.021642+0000","last_fullsized":"2024-09-21T00:29:21.021642+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:28:10.979181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.021511+0000","last_change":"2024-09-21T00:29:19.818725+0000","last_active":"2024-09-21T00:29:21.021511+0000","last_peered":"2024-09-21T00:29:21.021511+0000","last_clean":"2024-09-21T00:29:21.021511+0000","last_became_active":"2024-09-21T00:29:19.818406+0000","last_became_peered":"2024-09-21T00:29:19.818406+0000","last_unstale":"2024-09-21T00:29:21.021511+0000","last_undegraded":"2024-09-21T00:29:21.021511+0000","last_fullsized":"2024-09-21T00:29:21.021511+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:13:53.211350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.807387+0000","last_change":"2024-09-21T00:29:19.803557+0000","last_active":"2024-09-21T00:29:20.807387+0000","last_peered":"2024-09-21T00:29:20.807387+0000","last_clean":"2024-09-21T00:29:20.807387+0000","last_became_active":"2024-09-21T00:29:19.802868+0000","last_became_peered":"2024-09-21T00:29:19.802868+0000","last_unstale":"2024-09-21T00:29:20.807387+0000","last_undegraded":"2024-09-21T00:29:20.807387+0000","last_fullsized":"2024-09-21T00:29:20.807387+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:17:38.099624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022370+0000","last_change":"2024-09-21T00:29:19.811473+0000","last_active":"2024-09-21T00:29:21.022370+0000","last_peered":"2024-09-21T00:29:21.022370+0000","last_clean":"2024-09-21T00:29:21.022370+0000","last_became_active":"2024-09-21T00:29:19.811129+0000","last_became_peered":"2024-09-21T00:29:19.811129+0000","last_unstale":"2024-09-21T00:29:21.022370+0000","last_undegraded":"2024-09-21T00:29:21.022370+0000","last_fullsized":"2024-09-21T00:29:21.022370+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:26.631458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.807233+0000","last_change":"2024-09-21T00:29:19.809929+0000","last_active":"2024-09-21T00:29:20.807233+0000","last_peered":"2024-09-21T00:29:20.807233+0000","last_clean":"2024-09-21T00:29:20.807233+0000","last_became_active":"2024-09-21T00:29:19.809541+0000","last_became_peered":"2024-09-21T00:29:19.809541+0000","last_unstale":"2024-09-21T00:29:20.807233+0000","last_undegraded":"2024-09-21T00:29:20.807233+0000","last_fullsized":"2024-09-21T00:29:20.807233+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:34:11.115991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.810853+0000","last_change":"2024-09-21T00:29:19.808754+0000","last_active":"2024-09-21T00:29:20.810853+0000","last_peered":"2024-09-21T00:29:20.810853+0000","last_clean":"2024-09-21T00:29:20.810853+0000","last_became_active":"2024-09-21T00:29:19.807857+0000","last_became_peered":"2024-09-21T00:29:19.807857+0000","last_unstale":"2024-09-21T00:29:20.810853+0000","last_undegraded":"2024-09-21T00:29:20.810853+0000","last_fullsized":"2024-09-21T00:29:20.810853+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:53:45.848291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.023011+0000","last_change":"2024-09-21T00:29:19.815917+0000","last_active":"2024-09-21T00:29:21.023011+0000","last_peered":"2024-09-21T00:29:21.023011+0000","last_clean":"2024-09-21T00:29:21.023011+0000","last_became_active":"2024-09-21T00:29:19.815559+0000","last_became_peered":"2024-09-21T00:29:19.815559+0000","last_unstale":"2024-09-21T00:29:21.023011+0000","last_undegraded":"2024-09-21T00:29:21.023011+0000","last_fullsized":"2024-09-21T00:29:21.023011+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:47:06.729633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:18.775092+0000","last_change":"2024-09-21T00:29:18.775092+0000","last_active":"2024-09-21T00:29:18.775092+0000","last_peered":"2024-09-21T00:29:18.775092+0000","last_clean":"2024-09-21T00:29:18.775092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:18.775092+0000","last_undegraded":"2024-09-21T00:29:18.775092+0000","last_fullsized":"2024-09-21T00:29:18.775092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.807054+0000","last_change":"2024-09-21T00:29:19.802421+0000","last_active":"2024-09-21T00:29:20.807054+0000","last_peered":"2024-09-21T00:29:20.807054+0000","last_clean":"2024-09-21T00:29:20.807054+0000","last_became_active":"2024-09-21T00:29:19.802074+0000","last_became_peered":"2024-09-21T00:29:19.802074+0000","last_unstale":"2024-09-21T00:29:20.807054+0000","last_undegraded":"2024-09-21T00:29:20.807054+0000","last_fullsized":"2024-09-21T00:29:20.807054+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:55:22.327108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:18.775092+0000","last_change":"2024-09-21T00:29:18.775092+0000","last_active":"2024-09-21T00:29:18.775092+0000","last_peered":"2024-09-21T00:29:18.775092+0000","last_clean":"2024-09-21T00:29:18.775092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:18.775092+0000","last_undegraded":"2024-09-21T00:29:18.775092+0000","last_fullsized":"2024-09-21T00:29:18.775092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.810198+0000","last_change":"2024-09-21T00:29:19.802920+0000","last_active":"2024-09-21T00:29:20.810198+0000","last_peered":"2024-09-21T00:29:20.810198+0000","last_clean":"2024-09-21T00:29:20.810198+0000","last_became_active":"2024-09-21T00:29:19.802584+0000","last_became_peered":"2024-09-21T00:29:19.802584+0000","last_unstale":"2024-09-21T00:29:20.810198+0000","last_undegraded":"2024-09-21T00:29:20.810198+0000","last_fullsized":"2024-09-21T00:29:20.810198+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:38:27.387471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.810402+0000","last_change":"2024-09-21T00:29:19.808064+0000","last_active":"2024-09-21T00:29:20.810402+0000","last_peered":"2024-09-21T00:29:20.810402+0000","last_clean":"2024-09-21T00:29:20.810402+0000","last_became_active":"2024-09-21T00:29:19.807712+0000","last_became_peered":"2024-09-21T00:29:19.807712+0000","last_unstale":"2024-09-21T00:29:20.810402+0000","last_undegraded":"2024-09-21T00:29:20.810402+0000","last_fullsized":"2024-09-21T00:29:20.810402+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:45:54.930710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.810053+0000","last_change":"2024-09-21T00:29:19.811883+0000","last_active":"2024-09-21T00:29:20.810053+0000","last_peered":"2024-09-21T00:29:20.810053+0000","last_clean":"2024-09-21T00:29:20.810053+0000","last_became_active":"2024-09-21T00:29:19.811584+0000","last_became_peered":"2024-09-21T00:29:19.811584+0000","last_unstale":"2024-09-21T00:29:20.810053+0000","last_undegraded":"2024-09-21T00:29:20.810053+0000","last_fullsized":"2024-09-21T00:29:20.810053+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:00:59.593942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022016+0000","last_change":"2024-09-21T00:29:19.816999+0000","last_active":"2024-09-21T00:29:21.022016+0000","last_peered":"2024-09-21T00:29:21.022016+0000","last_clean":"2024-09-21T00:29:21.022016+0000","last_became_active":"2024-09-21T00:29:19.814905+0000","last_became_peered":"2024-09-21T00:29:19.814905+0000","last_unstale":"2024-09-21T00:29:21.022016+0000","last_undegraded":"2024-09-21T00:29:21.022016+0000","last_fullsized":"2024-09-21T00:29:21.022016+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:46:57.412342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.023130+0000","last_change":"2024-09-21T00:29:19.815104+0000","last_active":"2024-09-21T00:29:21.023130+0000","last_peered":"2024-09-21T00:29:21.023130+0000","last_clean":"2024-09-21T00:29:21.023130+0000","last_became_active":"2024-09-21T00:29:19.814806+0000","last_became_peered":"2024-09-21T00:29:19.814806+0000","last_unstale":"2024-09-21T00:29:21.023130+0000","last_undegraded":"2024-09-21T00:29:21.023130+0000","last_fullsized":"2024-09-21T00:29:21.023130+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:14:39.263553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.810509+0000","last_change":"2024-09-21T00:29:19.802931+0000","last_active":"2024-09-21T00:29:20.810509+0000","last_peered":"2024-09-21T00:29:20.810509+0000","last_clean":"2024-09-21T00:29:20.810509+0000","last_became_active":"2024-09-21T00:29:19.802606+0000","last_became_peered":"2024-09-21T00:29:19.802606+0000","last_unstale":"2024-09-21T00:29:20.810509+0000","last_undegraded":"2024-09-21T00:29:20.810509+0000","last_fullsized":"2024-09-21T00:29:20.810509+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:56:07.296305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022325+0000","last_change":"2024-09-21T00:29:19.812717+0000","last_active":"2024-09-21T00:29:21.022325+0000","last_peered":"2024-09-21T00:29:21.022325+0000","last_clean":"2024-09-21T00:29:21.022325+0000","last_became_active":"2024-09-21T00:29:19.812402+0000","last_became_peered":"2024-09-21T00:29:19.812402+0000","last_unstale":"2024-09-21T00:29:21.022325+0000","last_undegraded":"2024-09-21T00:29:21.022325+0000","last_fullsized":"2024-09-21T00:29:21.022325+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:26:39.938614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022839+0000","last_change":"2024-09-21T00:29:19.816588+0000","last_active":"2024-09-21T00:29:21.022839+0000","last_peered":"2024-09-21T00:29:21.022839+0000","last_clean":"2024-09-21T00:29:21.022839+0000","last_became_active":"2024-09-21T00:29:19.815574+0000","last_became_peered":"2024-09-21T00:29:19.815574+0000","last_unstale":"2024-09-21T00:29:21.022839+0000","last_undegraded":"2024-09-21T00:29:21.022839+0000","last_fullsized":"2024-09-21T00:29:21.022839+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:47:08.243928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.807629+0000","last_change":"2024-09-21T00:29:19.810747+0000","last_active":"2024-09-21T00:29:20.807629+0000","last_peered":"2024-09-21T00:29:20.807629+0000","last_clean":"2024-09-21T00:29:20.807629+0000","last_became_active":"2024-09-21T00:29:19.810429+0000","last_became_peered":"2024-09-21T00:29:19.810429+0000","last_unstale":"2024-09-21T00:29:20.807629+0000","last_undegraded":"2024-09-21T00:29:20.807629+0000","last_fullsized":"2024-09-21T00:29:20.807629+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:21:05.578580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"93'4","reported_seq":19,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022399+0000","last_change":"2024-09-21T00:29:18.798208+0000","last_active":"2024-09-21T00:29:21.022399+0000","last_peered":"2024-09-21T00:29:21.022399+0000","last_clean":"2024-09-21T00:29:21.022399+0000","last_became_active":"2024-09-21T00:29:18.797857+0000","last_became_peered":"2024-09-21T00:29:18.797857+0000","last_unstale":"2024-09-21T00:29:21.022399+0000","last_undegraded":"2024-09-21T00:29:21.022399+0000","last_fullsized":"2024-09-21T00:29:21.022399+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T00:37:31.452087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.021779+0000","last_change":"2024-09-21T00:29:19.822277+0000","last_active":"2024-09-21T00:29:21.021779+0000","last_peered":"2024-09-21T00:29:21.021779+0000","last_clean":"2024-09-21T00:29:21.021779+0000","last_became_active":"2024-09-21T00:29:19.821917+0000","last_became_peered":"2024-09-21T00:29:19.821917+0000","last_unstale":"2024-09-21T00:29:21.021779+0000","last_undegraded":"2024-09-21T00:29:21.021779+0000","last_fullsized":"2024-09-21T00:29:21.021779+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:04:07.143491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"93'1","reported_seq":16,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022961+0000","last_change":"2024-09-21T00:29:18.799882+0000","last_active":"2024-09-21T00:29:21.022961+0000","last_peered":"2024-09-21T00:29:21.022961+0000","last_clean":"2024-09-21T00:29:21.022961+0000","last_became_active":"2024-09-21T00:29:18.799524+0000","last_became_peered":"2024-09-21T00:29:18.799524+0000","last_unstale":"2024-09-21T00:29:21.022961+0000","last_undegraded":"2024-09-21T00:29:21.022961+0000","last_fullsized":"2024-09-21T00:29:21.022961+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:39:17.740976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.810993+0000","last_change":"2024-09-21T00:29:19.809700+0000","last_active":"2024-09-21T00:29:20.810993+0000","last_peered":"2024-09-21T00:29:20.810993+0000","last_clean":"2024-09-21T00:29:20.810993+0000","last_became_active":"2024-09-21T00:29:19.809343+0000","last_became_peered":"2024-09-21T00:29:19.809343+0000","last_unstale":"2024-09-21T00:29:20.810993+0000","last_undegraded":"2024-09-21T00:29:20.810993+0000","last_fullsized":"2024-09-21T00:29:20.810993+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:28:50.502430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"93'2","reported_seq":17,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.810949+0000","last_change":"2024-09-21T00:29:18.797271+0000","last_active":"2024-09-21T00:29:20.810949+0000","last_peered":"2024-09-21T00:29:20.810949+0000","last_clean":"2024-09-21T00:29:20.810949+0000","last_became_active":"2024-09-21T00:29:18.796656+0000","last_became_peered":"2024-09-21T00:29:18.796656+0000","last_unstale":"2024-09-21T00:29:20.810949+0000","last_undegraded":"2024-09-21T00:29:20.810949+0000","last_fullsized":"2024-09-21T00:29:20.810949+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:28:00.490269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.810191+0000","last_change":"2024-09-21T00:29:19.802323+0000","last_active":"2024-09-21T00:29:20.810191+0000","last_peered":"2024-09-21T00:29:20.810191+0000","last_clean":"2024-09-21T00:29:20.810191+0000","last_became_active":"2024-09-21T00:29:19.801998+0000","last_became_peered":"2024-09-21T00:29:19.801998+0000","last_unstale":"2024-09-21T00:29:20.810191+0000","last_undegraded":"2024-09-21T00:29:20.810191+0000","last_fullsized":"2024-09-21T00:29:20.810191+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:31:43.191165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-09-21T00:29:17.783208+0000","last_change":"2024-09-21T00:29:17.783208+0000","last_active":"2024-09-21T00:29:17.769711+0000","last_peered":"2024-09-21T00:29:17.769711+0000","last_clean":"2024-09-21T00:29:17.769711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:17.783208+0000","last_undegraded":"2024-09-21T00:29:17.783208+0000","last_fullsized":"2024-09-21T00:29:17.783208+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022055+0000","last_change":"2024-09-21T00:29:19.821679+0000","last_active":"2024-09-21T00:29:21.022055+0000","last_peered":"2024-09-21T00:29:21.022055+0000","last_clean":"2024-09-21T00:29:21.022055+0000","last_became_active":"2024-09-21T00:29:19.821331+0000","last_became_peered":"2024-09-21T00:29:19.821331+0000","last_unstale":"2024-09-21T00:29:21.022055+0000","last_undegraded":"2024-09-21T00:29:21.022055+0000","last_fullsized":"2024-09-21T00:29:21.022055+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:13:53.211350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"93'1","reported_seq":16,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.021871+0000","last_change":"2024-09-21T00:29:18.797442+0000","last_active":"2024-09-21T00:29:21.021871+0000","last_peered":"2024-09-21T00:29:21.021871+0000","last_clean":"2024-09-21T00:29:21.021871+0000","last_became_active":"2024-09-21T00:29:18.797128+0000","last_became_peered":"2024-09-21T00:29:18.797128+0000","last_unstale":"2024-09-21T00:29:21.021871+0000","last_undegraded":"2024-09-21T00:29:21.021871+0000","last_fullsized":"2024-09-21T00:29:21.021871+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:22:52.216862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.807399+0000","last_change":"2024-09-21T00:29:19.808878+0000","last_active":"2024-09-21T00:29:20.807399+0000","last_peered":"2024-09-21T00:29:20.807399+0000","last_clean":"2024-09-21T00:29:20.807399+0000","last_became_active":"2024-09-21T00:29:19.808577+0000","last_became_peered":"2024-09-21T00:29:19.808577+0000","last_unstale":"2024-09-21T00:29:20.807399+0000","last_undegraded":"2024-09-21T00:29:20.807399+0000","last_fullsized":"2024-09-21T00:29:20.807399+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:34:29.883537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"93'1","reported_seq":16,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.810090+0000","last_change":"2024-09-21T00:29:18.801027+0000","last_active":"2024-09-21T00:29:20.810090+0000","last_peered":"2024-09-21T00:29:20.810090+0000","last_clean":"2024-09-21T00:29:20.810090+0000","last_became_active":"2024-09-21T00:29:18.800693+0000","last_became_peered":"2024-09-21T00:29:18.800693+0000","last_unstale":"2024-09-21T00:29:20.810090+0000","last_undegraded":"2024-09-21T00:29:20.810090+0000","last_fullsized":"2024-09-21T00:29:20.810090+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:17:06.642779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022624+0000","last_change":"2024-09-21T00:29:19.815242+0000","last_active":"2024-09-21T00:29:21.022624+0000","last_peered":"2024-09-21T00:29:21.022624+0000","last_clean":"2024-09-21T00:29:21.022624+0000","last_became_active":"2024-09-21T00:29:19.814858+0000","last_became_peered":"2024-09-21T00:29:19.814858+0000","last_unstale":"2024-09-21T00:29:21.022624+0000","last_undegraded":"2024-09-21T00:29:21.022624+0000","last_fullsized":"2024-09-21T00:29:21.022624+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:35:06.851240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.809872+0000","last_change":"2024-09-21T00:29:18.797155+0000","last_active":"2024-09-21T00:29:20.809872+0000","last_peered":"2024-09-21T00:29:20.809872+0000","last_clean":"2024-09-21T00:29:20.809872+0000","last_became_active":"2024-09-21T00:29:18.795953+0000","last_became_peered":"2024-09-21T00:29:18.795953+0000","last_unstale":"2024-09-21T00:29:20.809872+0000","last_undegraded":"2024-09-21T00:29:20.809872+0000","last_fullsized":"2024-09-21T00:29:20.809872+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T00:54:16.170396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022428+0000","last_change":"2024-09-21T00:29:19.817180+0000","last_active":"2024-09-21T00:29:21.022428+0000","last_peered":"2024-09-21T00:29:21.022428+0000","last_clean":"2024-09-21T00:29:21.022428+0000","last_became_active":"2024-09-21T00:29:19.816859+0000","last_became_peered":"2024-09-21T00:29:19.816859+0000","last_unstale":"2024-09-21T00:29:21.022428+0000","last_undegraded":"2024-09-21T00:29:21.022428+0000","last_fullsized":"2024-09-21T00:29:21.022428+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:30:01.904485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.021605+0000","last_change":"2024-09-21T00:29:18.793095+0000","last_active":"2024-09-21T00:29:21.021605+0000","last_peered":"2024-09-21T00:29:21.021605+0000","last_clean":"2024-09-21T00:29:21.021605+0000","last_became_active":"2024-09-21T00:29:18.792693+0000","last_became_peered":"2024-09-21T00:29:18.792693+0000","last_unstale":"2024-09-21T00:29:21.021605+0000","last_undegraded":"2024-09-21T00:29:21.021605+0000","last_fullsized":"2024-09-21T00:29:21.021605+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:56:18.147899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.807542+0000","last_change":"2024-09-21T00:29:19.815545+0000","last_active":"2024-09-21T00:29:20.807542+0000","last_peered":"2024-09-21T00:29:20.807542+0000","last_clean":"2024-09-21T00:29:20.807542+0000","last_became_active":"2024-09-21T00:29:19.811424+0000","last_became_peered":"2024-09-21T00:29:19.811424+0000","last_unstale":"2024-09-21T00:29:20.807542+0000","last_undegraded":"2024-09-21T00:29:20.807542+0000","last_fullsized":"2024-09-21T00:29:20.807542+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:20:57.334693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.810327+0000","last_change":"2024-09-21T00:29:18.796965+0000","last_active":"2024-09-21T00:29:20.810327+0000","last_peered":"2024-09-21T00:29:20.810327+0000","last_clean":"2024-09-21T00:29:20.810327+0000","last_became_active":"2024-09-21T00:29:18.796548+0000","last_became_peered":"2024-09-21T00:29:18.796548+0000","last_unstale":"2024-09-21T00:29:20.810327+0000","last_undegraded":"2024-09-21T00:29:20.810327+0000","last_fullsized":"2024-09-21T00:29:20.810327+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:00:18.177473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:18.775092+0000","last_change":"2024-09-21T00:29:18.775092+0000","last_active":"2024-09-21T00:29:18.775092+0000","last_peered":"2024-09-21T00:29:18.775092+0000","last_clean":"2024-09-21T00:29:18.775092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:18.775092+0000","last_undegraded":"2024-09-21T00:29:18.775092+0000","last_fullsized":"2024-09-21T00:29:18.775092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.807185+0000","last_change":"2024-09-21T00:29:18.798831+0000","last_active":"2024-09-21T00:29:20.807185+0000","last_peered":"2024-09-21T00:29:20.807185+0000","last_clean":"2024-09-21T00:29:20.807185+0000","last_became_active":"2024-09-21T00:29:18.798551+0000","last_became_peered":"2024-09-21T00:29:18.798551+0000","last_unstale":"2024-09-21T00:29:20.807185+0000","last_undegraded":"2024-09-21T00:29:20.807185+0000","last_fullsized":"2024-09-21T00:29:20.807185+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:12:30.180725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022749+0000","last_change":"2024-09-21T00:29:19.812082+0000","last_active":"2024-09-21T00:29:21.022749+0000","last_peered":"2024-09-21T00:29:21.022749+0000","last_clean":"2024-09-21T00:29:21.022749+0000","last_became_active":"2024-09-21T00:29:19.811773+0000","last_became_peered":"2024-09-21T00:29:19.811773+0000","last_unstale":"2024-09-21T00:29:21.022749+0000","last_undegraded":"2024-09-21T00:29:21.022749+0000","last_fullsized":"2024-09-21T00:29:21.022749+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:17:43.069499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"93'3","reported_seq":18,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.021859+0000","last_change":"2024-09-21T00:29:18.794178+0000","last_active":"2024-09-21T00:29:21.021859+0000","last_peered":"2024-09-21T00:29:21.021859+0000","last_clean":"2024-09-21T00:29:21.021859+0000","last_became_active":"2024-09-21T00:29:18.793851+0000","last_became_peered":"2024-09-21T00:29:18.793851+0000","last_unstale":"2024-09-21T00:29:21.021859+0000","last_undegraded":"2024-09-21T00:29:21.021859+0000","last_fullsized":"2024-09-21T00:29:21.021859+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:52:27.563468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:18.775092+0000","last_change":"2024-09-21T00:29:18.775092+0000","last_active":"2024-09-21T00:29:18.775092+0000","last_peered":"2024-09-21T00:29:18.775092+0000","last_clean":"2024-09-21T00:29:18.775092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:18.775092+0000","last_undegraded":"2024-09-21T00:29:18.775092+0000","last_fullsized":"2024-09-21T00:29:18.775092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.14","version":"93'6","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.021896+0000","last_change":"2024-09-21T00:29:18.800236+0000","last_active":"2024-09-21T00:29:21.021896+0000","last_peered":"2024-09-21T00:29:21.021896+0000","last_clean":"2024-09-21T00:29:21.021896+0000","last_became_active":"2024-09-21T00:29:18.799899+0000","last_became_peered":"2024-09-21T00:29:18.799899+0000","last_unstale":"2024-09-21T00:29:21.021896+0000","last_undegraded":"2024-09-21T00:29:21.021896+0000","last_fullsized":"2024-09-21T00:29:21.021896+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:29:10.908555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.809551+0000","last_change":"2024-09-21T00:29:19.811833+0000","last_active":"2024-09-21T00:29:20.809551+0000","last_peered":"2024-09-21T00:29:20.809551+0000","last_clean":"2024-09-21T00:29:20.809551+0000","last_became_active":"2024-09-21T00:29:19.811519+0000","last_became_peered":"2024-09-21T00:29:19.811519+0000","last_unstale":"2024-09-21T00:29:20.809551+0000","last_undegraded":"2024-09-21T00:29:20.809551+0000","last_fullsized":"2024-09-21T00:29:20.809551+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:52:28.568849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-09-21T00:29:17.784261+0000","last_change":"2024-09-21T00:29:17.784261+0000","last_active":"2024-09-21T00:29:17.769711+0000","last_peered":"2024-09-21T00:29:17.769711+0000","last_clean":"2024-09-21T00:29:17.769711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:17.784261+0000","last_undegraded":"2024-09-21T00:29:17.784261+0000","last_fullsized":"2024-09-21T00:29:17.784261+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022489+0000","last_change":"2024-09-21T00:29:19.814412+0000","last_active":"2024-09-21T00:29:21.022489+0000","last_peered":"2024-09-21T00:29:21.022489+0000","last_clean":"2024-09-21T00:29:21.022489+0000","last_became_active":"2024-09-21T00:29:19.813923+0000","last_became_peered":"2024-09-21T00:29:19.813923+0000","last_unstale":"2024-09-21T00:29:21.022489+0000","last_undegraded":"2024-09-21T00:29:21.022489+0000","last_fullsized":"2024-09-21T00:29:21.022489+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:42:28.398366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022221+0000","last_change":"2024-09-21T00:29:18.788248+0000","last_active":"2024-09-21T00:29:21.022221+0000","last_peered":"2024-09-21T00:29:21.022221+0000","last_clean":"2024-09-21T00:29:21.022221+0000","last_became_active":"2024-09-21T00:29:18.787904+0000","last_became_peered":"2024-09-21T00:29:18.787904+0000","last_unstale":"2024-09-21T00:29:21.022221+0000","last_undegraded":"2024-09-21T00:29:21.022221+0000","last_fullsized":"2024-09-21T00:29:21.022221+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:21:04.691629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.023078+0000","last_change":"2024-09-21T00:29:19.814341+0000","last_active":"2024-09-21T00:29:21.023078+0000","last_peered":"2024-09-21T00:29:21.023078+0000","last_clean":"2024-09-21T00:29:21.023078+0000","last_became_active":"2024-09-21T00:29:19.814052+0000","last_became_peered":"2024-09-21T00:29:19.814052+0000","last_unstale":"2024-09-21T00:29:21.023078+0000","last_undegraded":"2024-09-21T00:29:21.023078+0000","last_fullsized":"2024-09-21T00:29:21.023078+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:58:18.707835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.021901+0000","last_change":"2024-09-21T00:29:18.800084+0000","last_active":"2024-09-21T00:29:21.021901+0000","last_peered":"2024-09-21T00:29:21.021901+0000","last_clean":"2024-09-21T00:29:21.021901+0000","last_became_active":"2024-09-21T00:29:18.799797+0000","last_became_peered":"2024-09-21T00:29:18.799797+0000","last_unstale":"2024-09-21T00:29:21.021901+0000","last_undegraded":"2024-09-21T00:29:21.021901+0000","last_fullsized":"2024-09-21T00:29:21.021901+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T00:51:35.304058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.807595+0000","last_change":"2024-09-21T00:29:19.804683+0000","last_active":"2024-09-21T00:29:20.807595+0000","last_peered":"2024-09-21T00:29:20.807595+0000","last_clean":"2024-09-21T00:29:20.807595+0000","last_became_active":"2024-09-21T00:29:19.803246+0000","last_became_peered":"2024-09-21T00:29:19.803246+0000","last_unstale":"2024-09-21T00:29:20.807595+0000","last_undegraded":"2024-09-21T00:29:20.807595+0000","last_fullsized":"2024-09-21T00:29:20.807595+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:46:26.637135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"93'8","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.810678+0000","last_change":"2024-09-21T00:29:18.796567+0000","last_active":"2024-09-21T00:29:20.810678+0000","last_peered":"2024-09-21T00:29:20.810678+0000","last_clean":"2024-09-21T00:29:20.810678+0000","last_became_active":"2024-09-21T00:29:18.796255+0000","last_became_peered":"2024-09-21T00:29:18.796255+0000","last_unstale":"2024-09-21T00:29:20.810678+0000","last_undegraded":"2024-09-21T00:29:20.810678+0000","last_fullsized":"2024-09-21T00:29:20.810678+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:04:53.438244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.023031+0000","last_change":"2024-09-21T00:29:19.817234+0000","last_active":"2024-09-21T00:29:21.023031+0000","last_peered":"2024-09-21T00:29:21.023031+0000","last_clean":"2024-09-21T00:29:21.023031+0000","last_became_active":"2024-09-21T00:29:19.816273+0000","last_became_peered":"2024-09-21T00:29:19.816273+0000","last_unstale":"2024-09-21T00:29:21.023031+0000","last_undegraded":"2024-09-21T00:29:21.023031+0000","last_fullsized":"2024-09-21T00:29:21.023031+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:45:02.009239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"93'1","reported_seq":16,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022386+0000","last_change":"2024-09-21T00:29:18.799579+0000","last_active":"2024-09-21T00:29:21.022386+0000","last_peered":"2024-09-21T00:29:21.022386+0000","last_clean":"2024-09-21T00:29:21.022386+0000","last_became_active":"2024-09-21T00:29:18.799238+0000","last_became_peered":"2024-09-21T00:29:18.799238+0000","last_unstale":"2024-09-21T00:29:21.022386+0000","last_undegraded":"2024-09-21T00:29:21.022386+0000","last_fullsized":"2024-09-21T00:29:21.022386+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:01:35.301014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022278+0000","last_change":"2024-09-21T00:29:19.814239+0000","last_active":"2024-09-21T00:29:21.022278+0000","last_peered":"2024-09-21T00:29:21.022278+0000","last_clean":"2024-09-21T00:29:21.022278+0000","last_became_active":"2024-09-21T00:29:19.813892+0000","last_became_peered":"2024-09-21T00:29:19.813892+0000","last_unstale":"2024-09-21T00:29:21.022278+0000","last_undegraded":"2024-09-21T00:29:21.022278+0000","last_fullsized":"2024-09-21T00:29:21.022278+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:02:55.200607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"93'1","reported_seq":16,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.021997+0000","last_change":"2024-09-21T00:29:18.786505+0000","last_active":"2024-09-21T00:29:21.021997+0000","last_peered":"2024-09-21T00:29:21.021997+0000","last_clean":"2024-09-21T00:29:21.021997+0000","last_became_active":"2024-09-21T00:29:18.786141+0000","last_became_peered":"2024-09-21T00:29:18.786141+0000","last_unstale":"2024-09-21T00:29:21.021997+0000","last_undegraded":"2024-09-21T00:29:21.021997+0000","last_fullsized":"2024-09-21T00:29:21.021997+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:59:49.904811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:18.775092+0000","last_change":"2024-09-21T00:29:18.775092+0000","last_active":"2024-09-21T00:29:18.775092+0000","last_peered":"2024-09-21T00:29:18.775092+0000","last_clean":"2024-09-21T00:29:18.775092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:18.775092+0000","last_undegraded":"2024-09-21T00:29:18.775092+0000","last_fullsized":"2024-09-21T00:29:18.775092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.809671+0000","last_change":"2024-09-21T00:29:18.802388+0000","last_active":"2024-09-21T00:29:20.809671+0000","last_peered":"2024-09-21T00:29:20.809671+0000","last_clean":"2024-09-21T00:29:20.809671+0000","last_became_active":"2024-09-21T00:29:18.802060+0000","last_became_peered":"2024-09-21T00:29:18.802060+0000","last_unstale":"2024-09-21T00:29:20.809671+0000","last_undegraded":"2024-09-21T00:29:20.809671+0000","last_fullsized":"2024-09-21T00:29:20.809671+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:26:13.866796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.807105+0000","last_change":"2024-09-21T00:29:19.812924+0000","last_active":"2024-09-21T00:29:20.807105+0000","last_peered":"2024-09-21T00:29:20.807105+0000","last_clean":"2024-09-21T00:29:20.807105+0000","last_became_active":"2024-09-21T00:29:19.812607+0000","last_became_peered":"2024-09-21T00:29:19.812607+0000","last_unstale":"2024-09-21T00:29:20.807105+0000","last_undegraded":"2024-09-21T00:29:20.807105+0000","last_fullsized":"2024-09-21T00:29:20.807105+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:11:09.667419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"93'2","reported_seq":17,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.023167+0000","last_change":"2024-09-21T00:29:18.796630+0000","last_active":"2024-09-21T00:29:21.023167+0000","last_peered":"2024-09-21T00:29:21.023167+0000","last_clean":"2024-09-21T00:29:21.023167+0000","last_became_active":"2024-09-21T00:29:18.796317+0000","last_became_peered":"2024-09-21T00:29:18.796317+0000","last_unstale":"2024-09-21T00:29:21.023167+0000","last_undegraded":"2024-09-21T00:29:21.023167+0000","last_fullsized":"2024-09-21T00:29:21.023167+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:50:16.709606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.807330+0000","last_change":"2024-09-21T00:29:19.801051+0000","last_active":"2024-09-21T00:29:20.807330+0000","last_peered":"2024-09-21T00:29:20.807330+0000","last_clean":"2024-09-21T00:29:20.807330+0000","last_became_active":"2024-09-21T00:29:19.798834+0000","last_became_peered":"2024-09-21T00:29:19.798834+0000","last_unstale":"2024-09-21T00:29:20.807330+0000","last_undegraded":"2024-09-21T00:29:20.807330+0000","last_fullsized":"2024-09-21T00:29:20.807330+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:20:15.757612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022535+0000","last_change":"2024-09-21T00:29:18.796311+0000","last_active":"2024-09-21T00:29:21.022535+0000","last_peered":"2024-09-21T00:29:21.022535+0000","last_clean":"2024-09-21T00:29:21.022535+0000","last_became_active":"2024-09-21T00:29:18.795425+0000","last_became_peered":"2024-09-21T00:29:18.795425+0000","last_unstale":"2024-09-21T00:29:21.022535+0000","last_undegraded":"2024-09-21T00:29:21.022535+0000","last_fullsized":"2024-09-21T00:29:21.022535+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:12:21.051042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022415+0000","last_change":"2024-09-21T00:29:19.813509+0000","last_active":"2024-09-21T00:29:21.022415+0000","last_peered":"2024-09-21T00:29:21.022415+0000","last_clean":"2024-09-21T00:29:21.022415+0000","last_became_active":"2024-09-21T00:29:19.813162+0000","last_became_peered":"2024-09-21T00:29:19.813162+0000","last_unstale":"2024-09-21T00:29:21.022415+0000","last_undegraded":"2024-09-21T00:29:21.022415+0000","last_fullsized":"2024-09-21T00:29:21.022415+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:48:11.528107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022396+0000","last_change":"2024-09-21T00:29:18.799921+0000","last_active":"2024-09-21T00:29:21.022396+0000","last_peered":"2024-09-21T00:29:21.022396+0000","last_clean":"2024-09-21T00:29:21.022396+0000","last_became_active":"2024-09-21T00:29:18.799614+0000","last_became_peered":"2024-09-21T00:29:18.799614+0000","last_unstale":"2024-09-21T00:29:21.022396+0000","last_undegraded":"2024-09-21T00:29:21.022396+0000","last_fullsized":"2024-09-21T00:29:21.022396+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:33:56.930832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022472+0000","last_change":"2024-09-21T00:29:19.813609+0000","last_active":"2024-09-21T00:29:21.022472+0000","last_peered":"2024-09-21T00:29:21.022472+0000","last_clean":"2024-09-21T00:29:21.022472+0000","last_became_active":"2024-09-21T00:29:19.813291+0000","last_became_peered":"2024-09-21T00:29:19.813291+0000","last_unstale":"2024-09-21T00:29:21.022472+0000","last_undegraded":"2024-09-21T00:29:21.022472+0000","last_fullsized":"2024-09-21T00:29:21.022472+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:48:12.895949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022964+0000","last_change":"2024-09-21T00:29:18.796499+0000","last_active":"2024-09-21T00:29:21.022964+0000","last_peered":"2024-09-21T00:29:21.022964+0000","last_clean":"2024-09-21T00:29:21.022964+0000","last_became_active":"2024-09-21T00:29:18.796175+0000","last_became_peered":"2024-09-21T00:29:18.796175+0000","last_unstale":"2024-09-21T00:29:21.022964+0000","last_undegraded":"2024-09-21T00:29:21.022964+0000","last_fullsized":"2024-09-21T00:29:21.022964+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:58:46.110270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.807498+0000","last_change":"2024-09-21T00:29:19.815935+0000","last_active":"2024-09-21T00:29:20.807498+0000","last_peered":"2024-09-21T00:29:20.807498+0000","last_clean":"2024-09-21T00:29:20.807498+0000","last_became_active":"2024-09-21T00:29:19.814883+0000","last_became_peered":"2024-09-21T00:29:19.814883+0000","last_unstale":"2024-09-21T00:29:20.807498+0000","last_undegraded":"2024-09-21T00:29:20.807498+0000","last_fullsized":"2024-09-21T00:29:20.807498+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:40:18.526456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-09-21T00:29:17.781825+0000","last_change":"2024-09-21T00:29:17.781825+0000","last_active":"2024-09-21T00:29:17.769711+0000","last_peered":"2024-09-21T00:29:17.769711+0000","last_clean":"2024-09-21T00:29:17.769711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:17.781825+0000","last_undegraded":"2024-09-21T00:29:17.781825+0000","last_fullsized":"2024-09-21T00:29:17.781825+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.809738+0000","last_change":"2024-09-21T00:29:19.806248+0000","last_active":"2024-09-21T00:29:20.809738+0000","last_peered":"2024-09-21T00:29:20.809738+0000","last_clean":"2024-09-21T00:29:20.809738+0000","last_became_active":"2024-09-21T00:29:19.805916+0000","last_became_peered":"2024-09-21T00:29:19.805916+0000","last_unstale":"2024-09-21T00:29:20.809738+0000","last_undegraded":"2024-09-21T00:29:20.809738+0000","last_fullsized":"2024-09-21T00:29:20.809738+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:42:42.562148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"93'1","reported_seq":16,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.807289+0000","last_change":"2024-09-21T00:29:18.797446+0000","last_active":"2024-09-21T00:29:20.807289+0000","last_peered":"2024-09-21T00:29:20.807289+0000","last_clean":"2024-09-21T00:29:20.807289+0000","last_became_active":"2024-09-21T00:29:18.797076+0000","last_became_peered":"2024-09-21T00:29:18.797076+0000","last_unstale":"2024-09-21T00:29:20.807289+0000","last_undegraded":"2024-09-21T00:29:20.807289+0000","last_fullsized":"2024-09-21T00:29:20.807289+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:06:17.566926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022312+0000","last_change":"2024-09-21T00:29:19.812384+0000","last_active":"2024-09-21T00:29:21.022312+0000","last_peered":"2024-09-21T00:29:21.022312+0000","last_clean":"2024-09-21T00:29:21.022312+0000","last_became_active":"2024-09-21T00:29:19.812077+0000","last_became_peered":"2024-09-21T00:29:19.812077+0000","last_unstale":"2024-09-21T00:29:21.022312+0000","last_undegraded":"2024-09-21T00:29:21.022312+0000","last_fullsized":"2024-09-21T00:29:21.022312+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:28:10.979181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022603+0000","last_change":"2024-09-21T00:29:18.796908+0000","last_active":"2024-09-21T00:29:21.022603+0000","last_peered":"2024-09-21T00:29:21.022603+0000","last_clean":"2024-09-21T00:29:21.022603+0000","last_became_active":"2024-09-21T00:29:18.796559+0000","last_became_peered":"2024-09-21T00:29:18.796559+0000","last_unstale":"2024-09-21T00:29:21.022603+0000","last_undegraded":"2024-09-21T00:29:21.022603+0000","last_fullsized":"2024-09-21T00:29:21.022603+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:27:50.568077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.809937+0000","last_change":"2024-09-21T00:29:19.808826+0000","last_active":"2024-09-21T00:29:20.809937+0000","last_peered":"2024-09-21T00:29:20.809937+0000","last_clean":"2024-09-21T00:29:20.809937+0000","last_became_active":"2024-09-21T00:29:19.808196+0000","last_became_peered":"2024-09-21T00:29:19.808196+0000","last_unstale":"2024-09-21T00:29:20.809937+0000","last_undegraded":"2024-09-21T00:29:20.809937+0000","last_fullsized":"2024-09-21T00:29:20.809937+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:22:53.222243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-09-21T00:29:17.782729+0000","last_change":"2024-09-21T00:29:17.782729+0000","last_active":"2024-09-21T00:29:17.769711+0000","last_peered":"2024-09-21T00:29:17.769711+0000","last_clean":"2024-09-21T00:29:17.769711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:17.782729+0000","last_undegraded":"2024-09-21T00:29:17.782729+0000","last_fullsized":"2024-09-21T00:29:17.782729+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.021641+0000","last_change":"2024-09-21T00:29:19.823036+0000","last_active":"2024-09-21T00:29:21.021641+0000","last_peered":"2024-09-21T00:29:21.021641+0000","last_clean":"2024-09-21T00:29:21.021641+0000","last_became_active":"2024-09-21T00:29:19.822688+0000","last_became_peered":"2024-09-21T00:29:19.822688+0000","last_unstale":"2024-09-21T00:29:21.021641+0000","last_undegraded":"2024-09-21T00:29:21.021641+0000","last_fullsized":"2024-09-21T00:29:21.021641+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:54:44.835987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.809976+0000","last_change":"2024-09-21T00:29:18.795683+0000","last_active":"2024-09-21T00:29:20.809976+0000","last_peered":"2024-09-21T00:29:20.809976+0000","last_clean":"2024-09-21T00:29:20.809976+0000","last_became_active":"2024-09-21T00:29:18.795363+0000","last_became_peered":"2024-09-21T00:29:18.795363+0000","last_unstale":"2024-09-21T00:29:20.809976+0000","last_undegraded":"2024-09-21T00:29:20.809976+0000","last_fullsized":"2024-09-21T00:29:20.809976+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:20:55.296469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022149+0000","last_change":"2024-09-21T00:29:19.812845+0000","last_active":"2024-09-21T00:29:21.022149+0000","last_peered":"2024-09-21T00:29:21.022149+0000","last_clean":"2024-09-21T00:29:21.022149+0000","last_became_active":"2024-09-21T00:29:19.812536+0000","last_became_peered":"2024-09-21T00:29:19.812536+0000","last_unstale":"2024-09-21T00:29:21.022149+0000","last_undegraded":"2024-09-21T00:29:21.022149+0000","last_fullsized":"2024-09-21T00:29:21.022149+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:48:44.631863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022864+0000","last_change":"2024-09-21T00:29:18.800233+0000","last_active":"2024-09-21T00:29:21.022864+0000","last_peered":"2024-09-21T00:29:21.022864+0000","last_clean":"2024-09-21T00:29:21.022864+0000","last_became_active":"2024-09-21T00:29:18.799952+0000","last_became_peered":"2024-09-21T00:29:18.799952+0000","last_unstale":"2024-09-21T00:29:21.022864+0000","last_undegraded":"2024-09-21T00:29:21.022864+0000","last_fullsized":"2024-09-21T00:29:21.022864+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:21:56.870500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.021841+0000","last_change":"2024-09-21T00:29:19.816067+0000","last_active":"2024-09-21T00:29:21.021841+0000","last_peered":"2024-09-21T00:29:21.021841+0000","last_clean":"2024-09-21T00:29:21.021841+0000","last_became_active":"2024-09-21T00:29:19.815714+0000","last_became_peered":"2024-09-21T00:29:19.815714+0000","last_unstale":"2024-09-21T00:29:21.021841+0000","last_undegraded":"2024-09-21T00:29:21.021841+0000","last_fullsized":"2024-09-21T00:29:21.021841+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:30:01.904485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022492+0000","last_change":"2024-09-21T00:29:18.796212+0000","last_active":"2024-09-21T00:29:21.022492+0000","last_peered":"2024-09-21T00:29:21.022492+0000","last_clean":"2024-09-21T00:29:21.022492+0000","last_became_active":"2024-09-21T00:29:18.795864+0000","last_became_peered":"2024-09-21T00:29:18.795864+0000","last_unstale":"2024-09-21T00:29:21.022492+0000","last_undegraded":"2024-09-21T00:29:21.022492+0000","last_fullsized":"2024-09-21T00:29:21.022492+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:57:43.771137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.810604+0000","last_change":"2024-09-21T00:29:19.811714+0000","last_active":"2024-09-21T00:29:20.810604+0000","last_peered":"2024-09-21T00:29:20.810604+0000","last_clean":"2024-09-21T00:29:20.810604+0000","last_became_active":"2024-09-21T00:29:19.809202+0000","last_became_peered":"2024-09-21T00:29:19.809202+0000","last_unstale":"2024-09-21T00:29:20.810604+0000","last_undegraded":"2024-09-21T00:29:20.810604+0000","last_fullsized":"2024-09-21T00:29:20.810604+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:44:42.287171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.023051+0000","last_change":"2024-09-21T00:29:18.801879+0000","last_active":"2024-09-21T00:29:21.023051+0000","last_peered":"2024-09-21T00:29:21.023051+0000","last_clean":"2024-09-21T00:29:21.023051+0000","last_became_active":"2024-09-21T00:29:18.800708+0000","last_became_peered":"2024-09-21T00:29:18.800708+0000","last_unstale":"2024-09-21T00:29:21.023051+0000","last_undegraded":"2024-09-21T00:29:21.023051+0000","last_fullsized":"2024-09-21T00:29:21.023051+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:48:40.413052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.809702+0000","last_change":"2024-09-21T00:29:19.798077+0000","last_active":"2024-09-21T00:29:20.809702+0000","last_peered":"2024-09-21T00:29:20.809702+0000","last_clean":"2024-09-21T00:29:20.809702+0000","last_became_active":"2024-09-21T00:29:19.797724+0000","last_became_peered":"2024-09-21T00:29:19.797724+0000","last_unstale":"2024-09-21T00:29:20.809702+0000","last_undegraded":"2024-09-21T00:29:20.809702+0000","last_fullsized":"2024-09-21T00:29:20.809702+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:35:16.985174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"93'2","reported_seq":17,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.023087+0000","last_change":"2024-09-21T00:29:18.800162+0000","last_active":"2024-09-21T00:29:21.023087+0000","last_peered":"2024-09-21T00:29:21.023087+0000","last_clean":"2024-09-21T00:29:21.023087+0000","last_became_active":"2024-09-21T00:29:18.798939+0000","last_became_peered":"2024-09-21T00:29:18.798939+0000","last_unstale":"2024-09-21T00:29:21.023087+0000","last_undegraded":"2024-09-21T00:29:21.023087+0000","last_fullsized":"2024-09-21T00:29:21.023087+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:28:05.362595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":244,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.021503+0000","last_change":"2024-09-21T00:25:38.725792+0000","last_active":"2024-09-21T00:29:21.021503+0000","last_peered":"2024-09-21T00:29:21.021503+0000","last_clean":"2024-09-21T00:29:21.021503+0000","last_became_active":"2024-09-21T00:25:38.724668+0000","last_became_peered":"2024-09-21T00:25:38.724668+0000","last_unstale":"2024-09-21T00:29:21.021503+0000","last_undegraded":"2024-09-21T00:29:21.021503+0000","last_fullsized":"2024-09-21T00:29:21.021503+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:37.163216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:37.163216+0000","last_clean_scrub_stamp":"2024-09-21T00:25:37.163216+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:14:36.588660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":114,"acting":114,"num_store_stats":7},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2376,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":33,"ondisk_log_size":33,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509488,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32908,"kb_used_data":1720,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338932,"statfs":{"total":96636764160,"available":96603066368,"internally_reserved":0,"allocated":1761280,"data_stored":1116174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509489,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32336,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339504,"statfs":{"total":96636764160,"available":96603652096,"internally_reserved":0,"allocated":1167360,"data_stored":521170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509488,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32308,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339532,"statfs":{"total":96636764160,"available":96603680768,"internally_reserved":0,"allocated":1146880,"data_stored":518954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509487,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32328,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339512,"statfs":{"total":96636764160,"available":96603660288,"internally_reserved":0,"allocated":1167360,"data_stored":525738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509489,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32328,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339512,"statfs":{"total":96636764160,"available":96603660288,"internally_reserved":0,"allocated":1163264,"data_stored":526228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509489,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32324,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339516,"statfs":{"total":96636764160,"available":96603664384,"internally_reserved":0,"allocated":1163264,"data_stored":525680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509489,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32904,"kb_used_data":1716,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338936,"statfs":{"total":96636764160,"available":96603070464,"internally_reserved":0,"allocated":1757184,"data_stored":1111024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574896,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32276,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339564,"statfs":{"total":96636764160,"available":96603713536,"internally_reserved":0,"allocated":1110016,"data_stored":498911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T00:29:24.581 INFO:tasks.ceph.ceph_manager.ceph:PG 16.3b is not active+clean 2024-09-21T00:29:24.581 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:29:18.775092+0000', 'last_change': '2024-09-21T00:29:18.775092+0000', 'last_active': '2024-09-21T00:29:18.775092+0000', 'last_peered': '2024-09-21T00:29:18.775092+0000', 'last_clean': '2024-09-21T00:29:18.775092+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:29:18.775092+0000', 'last_undegraded': '2024-09-21T00:29:18.775092+0000', 'last_fullsized': '2024-09-21T00:29:18.775092+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:29:18.775092+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:29:18.775092+0000', 'last_clean_scrub_stamp': '2024-09-21T00:29:18.775092+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:29:24.582 INFO:tasks.ceph.ceph_manager.ceph:PG 16.36 is not active+clean 2024-09-21T00:29:24.582 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:29:18.775092+0000', 'last_change': '2024-09-21T00:29:18.775092+0000', 'last_active': '2024-09-21T00:29:18.775092+0000', 'last_peered': '2024-09-21T00:29:18.775092+0000', 'last_clean': '2024-09-21T00:29:18.775092+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:29:18.775092+0000', 'last_undegraded': '2024-09-21T00:29:18.775092+0000', 'last_fullsized': '2024-09-21T00:29:18.775092+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:29:18.775092+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:29:18.775092+0000', 'last_clean_scrub_stamp': '2024-09-21T00:29:18.775092+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:29:24.582 INFO:tasks.ceph.ceph_manager.ceph:PG 16.2a is not active+clean 2024-09-21T00:29:24.582 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:29:18.775092+0000', 'last_change': '2024-09-21T00:29:18.775092+0000', 'last_active': '2024-09-21T00:29:18.775092+0000', 'last_peered': '2024-09-21T00:29:18.775092+0000', 'last_clean': '2024-09-21T00:29:18.775092+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:29:18.775092+0000', 'last_undegraded': '2024-09-21T00:29:18.775092+0000', 'last_fullsized': '2024-09-21T00:29:18.775092+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:29:18.775092+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:29:18.775092+0000', 'last_clean_scrub_stamp': '2024-09-21T00:29:18.775092+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:29:24.582 INFO:tasks.ceph.ceph_manager.ceph:PG 16.28 is not active+clean 2024-09-21T00:29:24.582 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:29:18.775092+0000', 'last_change': '2024-09-21T00:29:18.775092+0000', 'last_active': '2024-09-21T00:29:18.775092+0000', 'last_peered': '2024-09-21T00:29:18.775092+0000', 'last_clean': '2024-09-21T00:29:18.775092+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:29:18.775092+0000', 'last_undegraded': '2024-09-21T00:29:18.775092+0000', 'last_fullsized': '2024-09-21T00:29:18.775092+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:29:18.775092+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:29:18.775092+0000', 'last_clean_scrub_stamp': '2024-09-21T00:29:18.775092+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:29:24.582 INFO:tasks.ceph.ceph_manager.ceph:PG 15.1c is not active+clean 2024-09-21T00:29:24.582 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 91, 'state': 'creating+peering', 'last_fresh': '2024-09-21T00:29:17.783208+0000', 'last_change': '2024-09-21T00:29:17.783208+0000', 'last_active': '2024-09-21T00:29:17.769711+0000', 'last_peered': '2024-09-21T00:29:17.769711+0000', 'last_clean': '2024-09-21T00:29:17.769711+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:29:17.783208+0000', 'last_undegraded': '2024-09-21T00:29:17.783208+0000', 'last_fullsized': '2024-09-21T00:29:17.783208+0000', 'mapping_epoch': 91, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 91, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:29:17.769711+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:29:17.769711+0000', 'last_clean_scrub_stamp': '2024-09-21T00:29:17.769711+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-21T00:29:24.582 INFO:tasks.ceph.ceph_manager.ceph:PG 16.9 is not active+clean 2024-09-21T00:29:24.583 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:29:18.775092+0000', 'last_change': '2024-09-21T00:29:18.775092+0000', 'last_active': '2024-09-21T00:29:18.775092+0000', 'last_peered': '2024-09-21T00:29:18.775092+0000', 'last_clean': '2024-09-21T00:29:18.775092+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:29:18.775092+0000', 'last_undegraded': '2024-09-21T00:29:18.775092+0000', 'last_fullsized': '2024-09-21T00:29:18.775092+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:29:18.775092+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:29:18.775092+0000', 'last_clean_scrub_stamp': '2024-09-21T00:29:18.775092+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:29:24.583 INFO:tasks.ceph.ceph_manager.ceph:PG 16.b is not active+clean 2024-09-21T00:29:24.583 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:29:18.775092+0000', 'last_change': '2024-09-21T00:29:18.775092+0000', 'last_active': '2024-09-21T00:29:18.775092+0000', 'last_peered': '2024-09-21T00:29:18.775092+0000', 'last_clean': '2024-09-21T00:29:18.775092+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:29:18.775092+0000', 'last_undegraded': '2024-09-21T00:29:18.775092+0000', 'last_fullsized': '2024-09-21T00:29:18.775092+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:29:18.775092+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:29:18.775092+0000', 'last_clean_scrub_stamp': '2024-09-21T00:29:18.775092+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:29:24.583 INFO:tasks.ceph.ceph_manager.ceph:PG 15.13 is not active+clean 2024-09-21T00:29:24.583 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 91, 'state': 'creating+peering', 'last_fresh': '2024-09-21T00:29:17.784261+0000', 'last_change': '2024-09-21T00:29:17.784261+0000', 'last_active': '2024-09-21T00:29:17.769711+0000', 'last_peered': '2024-09-21T00:29:17.769711+0000', 'last_clean': '2024-09-21T00:29:17.769711+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:29:17.784261+0000', 'last_undegraded': '2024-09-21T00:29:17.784261+0000', 'last_fullsized': '2024-09-21T00:29:17.784261+0000', 'mapping_epoch': 91, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 91, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:29:17.769711+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:29:17.769711+0000', 'last_clean_scrub_stamp': '2024-09-21T00:29:17.769711+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-21T00:29:24.583 INFO:tasks.ceph.ceph_manager.ceph:PG 16.13 is not active+clean 2024-09-21T00:29:24.583 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:29:18.775092+0000', 'last_change': '2024-09-21T00:29:18.775092+0000', 'last_active': '2024-09-21T00:29:18.775092+0000', 'last_peered': '2024-09-21T00:29:18.775092+0000', 'last_clean': '2024-09-21T00:29:18.775092+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:29:18.775092+0000', 'last_undegraded': '2024-09-21T00:29:18.775092+0000', 'last_fullsized': '2024-09-21T00:29:18.775092+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:29:18.775092+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:29:18.775092+0000', 'last_clean_scrub_stamp': '2024-09-21T00:29:18.775092+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:29:24.583 INFO:tasks.ceph.ceph_manager.ceph:PG 15.7 is not active+clean 2024-09-21T00:29:24.583 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 91, 'state': 'creating+peering', 'last_fresh': '2024-09-21T00:29:17.781825+0000', 'last_change': '2024-09-21T00:29:17.781825+0000', 'last_active': '2024-09-21T00:29:17.769711+0000', 'last_peered': '2024-09-21T00:29:17.769711+0000', 'last_clean': '2024-09-21T00:29:17.769711+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:29:17.781825+0000', 'last_undegraded': '2024-09-21T00:29:17.781825+0000', 'last_fullsized': '2024-09-21T00:29:17.781825+0000', 'mapping_epoch': 91, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 91, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:29:17.769711+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:29:17.769711+0000', 'last_clean_scrub_stamp': '2024-09-21T00:29:17.769711+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-21T00:29:24.583 INFO:tasks.ceph.ceph_manager.ceph:PG 15.4 is not active+clean 2024-09-21T00:29:24.584 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 91, 'state': 'creating+peering', 'last_fresh': '2024-09-21T00:29:17.782729+0000', 'last_change': '2024-09-21T00:29:17.782729+0000', 'last_active': '2024-09-21T00:29:17.769711+0000', 'last_peered': '2024-09-21T00:29:17.769711+0000', 'last_clean': '2024-09-21T00:29:17.769711+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:29:17.782729+0000', 'last_undegraded': '2024-09-21T00:29:17.782729+0000', 'last_fullsized': '2024-09-21T00:29:17.782729+0000', 'mapping_epoch': 91, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 91, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:29:17.769711+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:29:17.769711+0000', 'last_clean_scrub_stamp': '2024-09-21T00:29:17.769711+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-21T00:29:24.584 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-21T00:29:24.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:24.693+0000 7f8b64d2b640 1 -- 172.21.15.112:0/3616770893 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8b60144dd0 msgr2=0x7f8b600cf0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:24.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:24.693+0000 7f8b64d2b640 1 --2- 172.21.15.112:0/3616770893 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8b60144dd0 0x7f8b600cf0c0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f8b50009830 tx=0x7f8b5002a510 comp rx=0 tx=0).stop 2024-09-21T00:29:24.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:24.694+0000 7f8b64d2b640 1 -- 172.21.15.112:0/3616770893 shutdown_connections 2024-09-21T00:29:24.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:24.694+0000 7f8b64d2b640 1 --2- 172.21.15.112:0/3616770893 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8b60144dd0 0x7f8b600cf0c0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:24.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:24.694+0000 7f8b64d2b640 1 --2- 172.21.15.112:0/3616770893 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8b6015f7f0 0x7f8b600b0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:24.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:24.694+0000 7f8b64d2b640 1 -- 172.21.15.112:0/3616770893 >> 172.21.15.112:0/3616770893 conn(0x7f8b60073fe0 msgr2=0x7f8b600743f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:24.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:24.694+0000 7f8b64d2b640 1 -- 172.21.15.112:0/3616770893 shutdown_connections 2024-09-21T00:29:24.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:24.694+0000 7f8b64d2b640 1 -- 172.21.15.112:0/3616770893 wait complete. 2024-09-21T00:29:24.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:24.695+0000 7f8b64d2b640 1 Processor -- start 2024-09-21T00:29:24.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:24.696+0000 7f8b64d2b640 1 -- start start 2024-09-21T00:29:24.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:24.697+0000 7f8b64d2b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8b60144dd0 0x7f8b601f3de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:24.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:24.697+0000 7f8b64d2b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8b601f68a0 0x7f8b6015ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:24.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:24.697+0000 7f8b64d2b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8b6013d690 0x7f8b6013daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:24.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:24.697+0000 7f8b64d2b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8b600cfda0 con 0x7f8b60144dd0 2024-09-21T00:29:24.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:24.697+0000 7f8b64d2b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8b600cfc20 con 0x7f8b601f68a0 2024-09-21T00:29:24.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:24.697+0000 7f8b64d2b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f8b600cff20 con 0x7f8b6013d690 2024-09-21T00:29:24.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:24.697+0000 7f8b5e59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8b60144dd0 0x7f8b601f3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:24.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:24.697+0000 7f8b5e59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8b60144dd0 0x7f8b601f3de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48740/0 (socket says 172.21.15.112:48740) 2024-09-21T00:29:24.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:24.697+0000 7f8b5e59c640 1 -- 172.21.15.112:0/3530607176 learned_addr learned my addr 172.21.15.112:0/3530607176 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:29:24.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:24.697+0000 7f8b5dd9b640 1 --2- 172.21.15.112:0/3530607176 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8b601f68a0 0x7f8b6015ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:24.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:24.697+0000 7f8b5ed9d640 1 --2- 172.21.15.112:0/3530607176 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8b6013d690 0x7f8b6013daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:24.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:24.697+0000 7f8b5e59c640 1 -- 172.21.15.112:0/3530607176 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8b6013d690 msgr2=0x7f8b6013daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:24.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:24.697+0000 7f8b5e59c640 1 --2- 172.21.15.112:0/3530607176 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8b6013d690 0x7f8b6013daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:24.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:24.697+0000 7f8b5e59c640 1 -- 172.21.15.112:0/3530607176 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8b601f68a0 msgr2=0x7f8b6015ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:24.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:24.697+0000 7f8b5e59c640 1 --2- 172.21.15.112:0/3530607176 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8b601f68a0 0x7f8b6015ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:24.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:24.697+0000 7f8b5e59c640 1 -- 172.21.15.112:0/3530607176 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b601f7360 con 0x7f8b60144dd0 2024-09-21T00:29:24.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:24.698+0000 7f8b5e59c640 1 --2- 172.21.15.112:0/3530607176 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8b60144dd0 0x7f8b601f3de0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f8b5800cb40 tx=0x7f8b5800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:24.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:24.698+0000 7f8b56ffd640 1 -- 172.21.15.112:0/3530607176 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b58016020 con 0x7f8b60144dd0 2024-09-21T00:29:24.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:24.698+0000 7f8b64d2b640 1 -- 172.21.15.112:0/3530607176 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b601f7640 con 0x7f8b60144dd0 2024-09-21T00:29:24.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:24.698+0000 7f8b56ffd640 1 -- 172.21.15.112:0/3530607176 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8b5800adb0 con 0x7f8b60144dd0 2024-09-21T00:29:24.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:24.698+0000 7f8b56ffd640 1 -- 172.21.15.112:0/3530607176 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b58012680 con 0x7f8b60144dd0 2024-09-21T00:29:24.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:24.698+0000 7f8b64d2b640 1 -- 172.21.15.112:0/3530607176 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b601f32b0 con 0x7f8b60144dd0 2024-09-21T00:29:24.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:24.699+0000 7f8b64d2b640 1 -- 172.21.15.112:0/3530607176 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b600d0440 con 0x7f8b60144dd0 2024-09-21T00:29:24.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:24.700+0000 7f8b56ffd640 1 -- 172.21.15.112:0/3530607176 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8b58019020 con 0x7f8b60144dd0 2024-09-21T00:29:24.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:24.700+0000 7f8b56ffd640 1 --2- 172.21.15.112:0/3530607176 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f8b40078b90 0x7f8b4007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:24.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:24.700+0000 7f8b56ffd640 1 -- 172.21.15.112:0/3530607176 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6101+0+0 (secure 0 0 0) 0x7f8b5809b3d0 con 0x7f8b60144dd0 2024-09-21T00:29:24.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:24.700+0000 7f8b5dd9b640 1 --2- 172.21.15.112:0/3530607176 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f8b40078b90 0x7f8b4007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:24.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:24.701+0000 7f8b5dd9b640 1 --2- 172.21.15.112:0/3530607176 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f8b40078b90 0x7f8b4007b0a0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f8b480095f0 tx=0x7f8b48005f90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:24.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:24.705+0000 7f8b56ffd640 1 -- 172.21.15.112:0/3530607176 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f8b58020020 con 0x7f8b60144dd0 2024-09-21T00:29:24.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:24.954+0000 7f8b64d2b640 1 -- 172.21.15.112:0/3530607176 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f8b60074920 con 0x7f8b60144dd0 2024-09-21T00:29:24.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:24.955+0000 7f8b56ffd640 1 -- 172.21.15.112:0/3530607176 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1636 (secure 0 0 0) 0x7f8b58063040 con 0x7f8b60144dd0 2024-09-21T00:29:24.956 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:29:24.956 INFO:teuthology.orchestra.run.smithi112.stdout:{"fsid":"8b34692b-dc65-49b2-a08f-26f707b96ac5","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":242,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":94,"num_osds":8,"num_up_osds":8,"osd_up_since":1726878336,"num_in_osds":8,"osd_in_since":1726878326,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":86},{"state_name":"unknown","count":7},{"state_name":"creating+peering","count":4}],"num_pgs":97,"num_pools":3,"num_objects":20,"data_bytes":592744,"bytes_used":265945088,"bytes_avail":772828168192,"bytes_total":773094113280,"unknown_pgs_ratio":0.072164945304393768,"inactive_pgs_ratio":0.041237112134695053,"write_bytes_sec":1335,"read_op_per_sec":0,"write_op_per_sec":6},"fsmap":{"epoch":64,"btime":"2024-09-21T00:29:22:809921+0000","id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"c","status":"up:active","gid":6804}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-21T00:28:43.134478+0000","services":{"mds":{"daemons":{"summary":"","b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-21T00:29:24.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:24.957+0000 7f8b64d2b640 1 -- 172.21.15.112:0/3530607176 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f8b40078b90 msgr2=0x7f8b4007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:24.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:24.957+0000 7f8b64d2b640 1 --2- 172.21.15.112:0/3530607176 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f8b40078b90 0x7f8b4007b0a0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f8b480095f0 tx=0x7f8b48005f90 comp rx=0 tx=0).stop 2024-09-21T00:29:24.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:24.957+0000 7f8b64d2b640 1 -- 172.21.15.112:0/3530607176 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8b60144dd0 msgr2=0x7f8b601f3de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:24.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:24.957+0000 7f8b64d2b640 1 --2- 172.21.15.112:0/3530607176 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8b60144dd0 0x7f8b601f3de0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f8b5800cb40 tx=0x7f8b5800a060 comp rx=0 tx=0).stop 2024-09-21T00:29:24.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:24.957+0000 7f8b64d2b640 1 -- 172.21.15.112:0/3530607176 shutdown_connections 2024-09-21T00:29:24.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:24.957+0000 7f8b64d2b640 1 --2- 172.21.15.112:0/3530607176 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8b6013d690 0x7f8b6013daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:24.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:24.957+0000 7f8b64d2b640 1 --2- 172.21.15.112:0/3530607176 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8b601f68a0 0x7f8b6015ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:24.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:24.957+0000 7f8b64d2b640 1 --2- 172.21.15.112:0/3530607176 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f8b40078b90 0x7f8b4007b0a0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:24.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:24.957+0000 7f8b64d2b640 1 --2- 172.21.15.112:0/3530607176 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8b60144dd0 0x7f8b601f3de0 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:24.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:24.957+0000 7f8b64d2b640 1 -- 172.21.15.112:0/3530607176 >> 172.21.15.112:0/3530607176 conn(0x7f8b60073fe0 msgr2=0x7f8b600b1150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:24.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:24.957+0000 7f8b64d2b640 1 -- 172.21.15.112:0/3530607176 shutdown_connections 2024-09-21T00:29:24.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:24.957+0000 7f8b64d2b640 1 -- 172.21.15.112:0/3530607176 wait complete. 2024-09-21T00:29:24.969 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-21T00:29:24.970 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T00:29:25.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:25.080+0000 7f2ce1a77640 1 -- 172.21.15.112:0/2910232541 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2cdc0ccae0 msgr2=0x7f2cdc0ccef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:25.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:25.080+0000 7f2ce1a77640 1 --2- 172.21.15.112:0/2910232541 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2cdc0ccae0 0x7f2cdc0ccef0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f2cdc15cd80 tx=0x7f2cd402a530 comp rx=0 tx=0).stop 2024-09-21T00:29:25.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:25.080+0000 7f2ce1a77640 1 -- 172.21.15.112:0/2910232541 shutdown_connections 2024-09-21T00:29:25.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:25.080+0000 7f2ce1a77640 1 --2- 172.21.15.112:0/2910232541 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2cdc0ccae0 0x7f2cdc0ccef0 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:25.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:25.080+0000 7f2ce1a77640 1 --2- 172.21.15.112:0/2910232541 >> v2:172.21.15.187:3300/0 conn(0x7f2cdc15d870 0x7f2cdc0d0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:25.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:25.080+0000 7f2ce1a77640 1 -- 172.21.15.112:0/2910232541 >> 172.21.15.112:0/2910232541 conn(0x7f2cdc0b43f0 msgr2=0x7f2cdc0b4830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:25.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:25.080+0000 7f2ce1a77640 1 -- 172.21.15.112:0/2910232541 shutdown_connections 2024-09-21T00:29:25.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:25.080+0000 7f2ce1a77640 1 -- 172.21.15.112:0/2910232541 wait complete. 2024-09-21T00:29:25.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:25.081+0000 7f2ce1a77640 1 Processor -- start 2024-09-21T00:29:25.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:25.082+0000 7f2ce1a77640 1 -- start start 2024-09-21T00:29:25.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:25.083+0000 7f2ce1a77640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2cdc0ccae0 0x7f2cdc0b7e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:25.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:25.083+0000 7f2ce1a77640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2cdc0d2030 0x7f2cdc159270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:25.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:25.083+0000 7f2ce1a77640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2cdc15d870 0x7f2cdc1f6d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:25.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:25.083+0000 7f2ce1a77640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2cdc0b2280 con 0x7f2cdc15d870 2024-09-21T00:29:25.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:25.083+0000 7f2ce1a77640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f2cdc0b2100 con 0x7f2cdc0ccae0 2024-09-21T00:29:25.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:25.083+0000 7f2ce1a77640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f2cdc0b2400 con 0x7f2cdc0d2030 2024-09-21T00:29:25.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:25.083+0000 7f2cdbfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2cdc15d870 0x7f2cdc1f6d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:25.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:25.083+0000 7f2cdbfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2cdc15d870 0x7f2cdc1f6d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48752/0 (socket says 172.21.15.112:48752) 2024-09-21T00:29:25.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:25.083+0000 7f2cdbfff640 1 -- 172.21.15.112:0/3731277279 learned_addr learned my addr 172.21.15.112:0/3731277279 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:29:25.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:25.083+0000 7f2cdaffd640 1 --2- 172.21.15.112:0/3731277279 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2cdc0d2030 0x7f2cdc159270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:25.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:25.083+0000 7f2cdb7fe640 1 --2- 172.21.15.112:0/3731277279 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2cdc0ccae0 0x7f2cdc0b7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:25.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:25.083+0000 7f2cdbfff640 1 -- 172.21.15.112:0/3731277279 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2cdc0d2030 msgr2=0x7f2cdc159270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:25.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:25.083+0000 7f2cdbfff640 1 --2- 172.21.15.112:0/3731277279 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2cdc0d2030 0x7f2cdc159270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:25.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:25.083+0000 7f2cdbfff640 1 -- 172.21.15.112:0/3731277279 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2cdc0ccae0 msgr2=0x7f2cdc0b7e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:25.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:25.083+0000 7f2cdbfff640 1 --2- 172.21.15.112:0/3731277279 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2cdc0ccae0 0x7f2cdc0b7e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:25.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:25.083+0000 7f2cdbfff640 1 -- 172.21.15.112:0/3731277279 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cdc0c7ac0 con 0x7f2cdc15d870 2024-09-21T00:29:25.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:25.083+0000 7f2cdaffd640 1 --2- 172.21.15.112:0/3731277279 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2cdc0d2030 0x7f2cdc159270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:29:25.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:25.083+0000 7f2cdbfff640 1 --2- 172.21.15.112:0/3731277279 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2cdc15d870 0x7f2cdc1f6d50 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f2cd4009920 tx=0x7f2cd4033c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:25.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:25.084+0000 7f2cc3fff640 1 -- 172.21.15.112:0/3731277279 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cd4031b60 con 0x7f2cdc15d870 2024-09-21T00:29:25.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:25.084+0000 7f2cc3fff640 1 -- 172.21.15.112:0/3731277279 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2cd4031e10 con 0x7f2cdc15d870 2024-09-21T00:29:25.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:25.084+0000 7f2ce1a77640 1 -- 172.21.15.112:0/3731277279 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cdc0c7da0 con 0x7f2cdc15d870 2024-09-21T00:29:25.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:25.084+0000 7f2cc3fff640 1 -- 172.21.15.112:0/3731277279 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cd4005260 con 0x7f2cdc15d870 2024-09-21T00:29:25.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:25.084+0000 7f2ce1a77640 1 -- 172.21.15.112:0/3731277279 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cdc0c8360 con 0x7f2cdc15d870 2024-09-21T00:29:25.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:25.085+0000 7f2ce1a77640 1 -- 172.21.15.112:0/3731277279 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cdc0b2920 con 0x7f2cdc15d870 2024-09-21T00:29:25.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:25.085+0000 7f2cc3fff640 1 -- 172.21.15.112:0/3731277279 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2cd4005400 con 0x7f2cdc15d870 2024-09-21T00:29:25.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:25.086+0000 7f2cc3fff640 1 --2- 172.21.15.112:0/3731277279 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2cc8078b90 0x7f2cc807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:25.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:25.086+0000 7f2cc3fff640 1 -- 172.21.15.112:0/3731277279 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6101+0+0 (secure 0 0 0) 0x7f2cd40bc070 con 0x7f2cdc15d870 2024-09-21T00:29:25.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:25.086+0000 7f2cdb7fe640 1 --2- 172.21.15.112:0/3731277279 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2cc8078b90 0x7f2cc807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:25.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:25.087+0000 7f2cdb7fe640 1 --2- 172.21.15.112:0/3731277279 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2cc8078b90 0x7f2cc807b0a0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc0020b0 tx=0x7f2ccc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:25.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:25.090+0000 7f2cc3fff640 1 -- 172.21.15.112:0/3731277279 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2cd403f030 con 0x7f2cdc15d870 2024-09-21T00:29:25.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:25.264+0000 7f2ce1a77640 1 -- 172.21.15.112:0/3731277279 --> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2cdc0cdb20 con 0x7f2cc8078b90 2024-09-21T00:29:25.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:25.269+0000 7f2cc3fff640 1 -- 172.21.15.112:0/3731277279 <== mgr.4103 v2:172.21.15.187:6800/2921522906 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252072 (secure 0 0 0) 0x7f2ccc00ec20 con 0x7f2cc8078b90 2024-09-21T00:29:25.272 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:29:25.277 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2024-09-21T00:29:25.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:25.272+0000 7f2ce1a77640 1 -- 172.21.15.112:0/3731277279 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2cc8078b90 msgr2=0x7f2cc807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:25.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:25.272+0000 7f2ce1a77640 1 --2- 172.21.15.112:0/3731277279 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2cc8078b90 0x7f2cc807b0a0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc0020b0 tx=0x7f2ccc009040 comp rx=0 tx=0).stop 2024-09-21T00:29:25.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:25.272+0000 7f2ce1a77640 1 -- 172.21.15.112:0/3731277279 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2cdc15d870 msgr2=0x7f2cdc1f6d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:25.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:25.272+0000 7f2ce1a77640 1 --2- 172.21.15.112:0/3731277279 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2cdc15d870 0x7f2cdc1f6d50 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f2cd4009920 tx=0x7f2cd4033c10 comp rx=0 tx=0).stop 2024-09-21T00:29:25.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:25.272+0000 7f2ce1a77640 1 -- 172.21.15.112:0/3731277279 shutdown_connections 2024-09-21T00:29:25.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:25.272+0000 7f2ce1a77640 1 --2- 172.21.15.112:0/3731277279 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2cc8078b90 0x7f2cc807b0a0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:25.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:25.272+0000 7f2ce1a77640 1 --2- 172.21.15.112:0/3731277279 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2cdc15d870 0x7f2cdc1f6d50 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:25.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:25.272+0000 7f2ce1a77640 1 --2- 172.21.15.112:0/3731277279 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2cdc0d2030 0x7f2cdc159270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:25.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:25.272+0000 7f2ce1a77640 1 --2- 172.21.15.112:0/3731277279 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2cdc0ccae0 0x7f2cdc0b7e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:25.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:25.272+0000 7f2ce1a77640 1 -- 172.21.15.112:0/3731277279 >> 172.21.15.112:0/3731277279 conn(0x7f2cdc0b43f0 msgr2=0x7f2cdc0b4800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:25.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:25.272+0000 7f2ce1a77640 1 -- 172.21.15.112:0/3731277279 shutdown_connections 2024-09-21T00:29:25.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:25.272+0000 7f2ce1a77640 1 -- 172.21.15.112:0/3731277279 wait complete. 2024-09-21T00:29:25.286 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":203,"stamp":"2024-09-21T00:29:25.145036+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":121,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":259764,"kb_used_data":10244,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754714956,"statfs":{"total":773094113280,"available":772828114944,"internally_reserved":0,"allocated":10489856,"data_stored":5370648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.366734"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.807159+0000","last_change":"2024-09-21T00:29:19.813350+0000","last_active":"2024-09-21T00:29:20.807159+0000","last_peered":"2024-09-21T00:29:20.807159+0000","last_clean":"2024-09-21T00:29:20.807159+0000","last_became_active":"2024-09-21T00:29:19.812123+0000","last_became_peered":"2024-09-21T00:29:19.812123+0000","last_unstale":"2024-09-21T00:29:20.807159+0000","last_undegraded":"2024-09-21T00:29:20.807159+0000","last_fullsized":"2024-09-21T00:29:20.807159+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:17:13.647509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022537+0000","last_change":"2024-09-21T00:29:19.807870+0000","last_active":"2024-09-21T00:29:21.022537+0000","last_peered":"2024-09-21T00:29:21.022537+0000","last_clean":"2024-09-21T00:29:21.022537+0000","last_became_active":"2024-09-21T00:29:19.807563+0000","last_became_peered":"2024-09-21T00:29:19.807563+0000","last_unstale":"2024-09-21T00:29:21.022537+0000","last_undegraded":"2024-09-21T00:29:21.022537+0000","last_fullsized":"2024-09-21T00:29:21.022537+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:26:29.645186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.807064+0000","last_change":"2024-09-21T00:29:19.804321+0000","last_active":"2024-09-21T00:29:20.807064+0000","last_peered":"2024-09-21T00:29:20.807064+0000","last_clean":"2024-09-21T00:29:20.807064+0000","last_became_active":"2024-09-21T00:29:19.804013+0000","last_became_peered":"2024-09-21T00:29:19.804013+0000","last_unstale":"2024-09-21T00:29:20.807064+0000","last_undegraded":"2024-09-21T00:29:20.807064+0000","last_fullsized":"2024-09-21T00:29:20.807064+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:47:15.426188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.021814+0000","last_change":"2024-09-21T00:29:19.811466+0000","last_active":"2024-09-21T00:29:21.021814+0000","last_peered":"2024-09-21T00:29:21.021814+0000","last_clean":"2024-09-21T00:29:21.021814+0000","last_became_active":"2024-09-21T00:29:19.811130+0000","last_became_peered":"2024-09-21T00:29:19.811130+0000","last_unstale":"2024-09-21T00:29:21.021814+0000","last_undegraded":"2024-09-21T00:29:21.021814+0000","last_fullsized":"2024-09-21T00:29:21.021814+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:44:42.287171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022147+0000","last_change":"2024-09-21T00:29:19.801339+0000","last_active":"2024-09-21T00:29:21.022147+0000","last_peered":"2024-09-21T00:29:21.022147+0000","last_clean":"2024-09-21T00:29:21.022147+0000","last_became_active":"2024-09-21T00:29:19.800996+0000","last_became_peered":"2024-09-21T00:29:19.800996+0000","last_unstale":"2024-09-21T00:29:21.022147+0000","last_undegraded":"2024-09-21T00:29:21.022147+0000","last_fullsized":"2024-09-21T00:29:21.022147+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:26:43.393876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022508+0000","last_change":"2024-09-21T00:29:19.822580+0000","last_active":"2024-09-21T00:29:21.022508+0000","last_peered":"2024-09-21T00:29:21.022508+0000","last_clean":"2024-09-21T00:29:21.022508+0000","last_became_active":"2024-09-21T00:29:19.822239+0000","last_became_peered":"2024-09-21T00:29:19.822239+0000","last_unstale":"2024-09-21T00:29:21.022508+0000","last_undegraded":"2024-09-21T00:29:21.022508+0000","last_fullsized":"2024-09-21T00:29:21.022508+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:46:09.814298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.021489+0000","last_change":"2024-09-21T00:29:19.822649+0000","last_active":"2024-09-21T00:29:21.021489+0000","last_peered":"2024-09-21T00:29:21.021489+0000","last_clean":"2024-09-21T00:29:21.021489+0000","last_became_active":"2024-09-21T00:29:19.822194+0000","last_became_peered":"2024-09-21T00:29:19.822194+0000","last_unstale":"2024-09-21T00:29:21.021489+0000","last_undegraded":"2024-09-21T00:29:21.021489+0000","last_fullsized":"2024-09-21T00:29:21.021489+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:09:42.739510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022777+0000","last_change":"2024-09-21T00:29:19.816934+0000","last_active":"2024-09-21T00:29:21.022777+0000","last_peered":"2024-09-21T00:29:21.022777+0000","last_clean":"2024-09-21T00:29:21.022777+0000","last_became_active":"2024-09-21T00:29:19.815942+0000","last_became_peered":"2024-09-21T00:29:19.815942+0000","last_unstale":"2024-09-21T00:29:21.022777+0000","last_undegraded":"2024-09-21T00:29:21.022777+0000","last_fullsized":"2024-09-21T00:29:21.022777+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:35:25.434622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022456+0000","last_change":"2024-09-21T00:29:19.816919+0000","last_active":"2024-09-21T00:29:21.022456+0000","last_peered":"2024-09-21T00:29:21.022456+0000","last_clean":"2024-09-21T00:29:21.022456+0000","last_became_active":"2024-09-21T00:29:19.816530+0000","last_became_peered":"2024-09-21T00:29:19.816530+0000","last_unstale":"2024-09-21T00:29:21.022456+0000","last_undegraded":"2024-09-21T00:29:21.022456+0000","last_fullsized":"2024-09-21T00:29:21.022456+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:40:46.095060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.021554+0000","last_change":"2024-09-21T00:29:19.809269+0000","last_active":"2024-09-21T00:29:21.021554+0000","last_peered":"2024-09-21T00:29:21.021554+0000","last_clean":"2024-09-21T00:29:21.021554+0000","last_became_active":"2024-09-21T00:29:19.808942+0000","last_became_peered":"2024-09-21T00:29:19.808942+0000","last_unstale":"2024-09-21T00:29:21.021554+0000","last_undegraded":"2024-09-21T00:29:21.021554+0000","last_fullsized":"2024-09-21T00:29:21.021554+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:58:57.469916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.809793+0000","last_change":"2024-09-21T00:29:19.814490+0000","last_active":"2024-09-21T00:29:20.809793+0000","last_peered":"2024-09-21T00:29:20.809793+0000","last_clean":"2024-09-21T00:29:20.809793+0000","last_became_active":"2024-09-21T00:29:19.813188+0000","last_became_peered":"2024-09-21T00:29:19.813188+0000","last_unstale":"2024-09-21T00:29:20.809793+0000","last_undegraded":"2024-09-21T00:29:20.809793+0000","last_fullsized":"2024-09-21T00:29:20.809793+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:59:50.910192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.810780+0000","last_change":"2024-09-21T00:29:19.811796+0000","last_active":"2024-09-21T00:29:20.810780+0000","last_peered":"2024-09-21T00:29:20.810780+0000","last_clean":"2024-09-21T00:29:20.810780+0000","last_became_active":"2024-09-21T00:29:19.811496+0000","last_became_peered":"2024-09-21T00:29:19.811496+0000","last_unstale":"2024-09-21T00:29:20.810780+0000","last_undegraded":"2024-09-21T00:29:20.810780+0000","last_fullsized":"2024-09-21T00:29:20.810780+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:59:41.316661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022019+0000","last_change":"2024-09-21T00:29:19.818860+0000","last_active":"2024-09-21T00:29:21.022019+0000","last_peered":"2024-09-21T00:29:21.022019+0000","last_clean":"2024-09-21T00:29:21.022019+0000","last_became_active":"2024-09-21T00:29:19.818552+0000","last_became_peered":"2024-09-21T00:29:19.818552+0000","last_unstale":"2024-09-21T00:29:21.022019+0000","last_undegraded":"2024-09-21T00:29:21.022019+0000","last_fullsized":"2024-09-21T00:29:21.022019+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:37:22.821347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.021673+0000","last_change":"2024-09-21T00:29:19.820196+0000","last_active":"2024-09-21T00:29:21.021673+0000","last_peered":"2024-09-21T00:29:21.021673+0000","last_clean":"2024-09-21T00:29:21.021673+0000","last_became_active":"2024-09-21T00:29:19.819691+0000","last_became_peered":"2024-09-21T00:29:19.819691+0000","last_unstale":"2024-09-21T00:29:21.021673+0000","last_undegraded":"2024-09-21T00:29:21.021673+0000","last_fullsized":"2024-09-21T00:29:21.021673+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:48:11.528107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.021642+0000","last_change":"2024-09-21T00:29:19.810601+0000","last_active":"2024-09-21T00:29:21.021642+0000","last_peered":"2024-09-21T00:29:21.021642+0000","last_clean":"2024-09-21T00:29:21.021642+0000","last_became_active":"2024-09-21T00:29:19.810272+0000","last_became_peered":"2024-09-21T00:29:19.810272+0000","last_unstale":"2024-09-21T00:29:21.021642+0000","last_undegraded":"2024-09-21T00:29:21.021642+0000","last_fullsized":"2024-09-21T00:29:21.021642+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:28:10.979181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.021511+0000","last_change":"2024-09-21T00:29:19.818725+0000","last_active":"2024-09-21T00:29:21.021511+0000","last_peered":"2024-09-21T00:29:21.021511+0000","last_clean":"2024-09-21T00:29:21.021511+0000","last_became_active":"2024-09-21T00:29:19.818406+0000","last_became_peered":"2024-09-21T00:29:19.818406+0000","last_unstale":"2024-09-21T00:29:21.021511+0000","last_undegraded":"2024-09-21T00:29:21.021511+0000","last_fullsized":"2024-09-21T00:29:21.021511+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:13:53.211350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.807387+0000","last_change":"2024-09-21T00:29:19.803557+0000","last_active":"2024-09-21T00:29:20.807387+0000","last_peered":"2024-09-21T00:29:20.807387+0000","last_clean":"2024-09-21T00:29:20.807387+0000","last_became_active":"2024-09-21T00:29:19.802868+0000","last_became_peered":"2024-09-21T00:29:19.802868+0000","last_unstale":"2024-09-21T00:29:20.807387+0000","last_undegraded":"2024-09-21T00:29:20.807387+0000","last_fullsized":"2024-09-21T00:29:20.807387+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:17:38.099624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022370+0000","last_change":"2024-09-21T00:29:19.811473+0000","last_active":"2024-09-21T00:29:21.022370+0000","last_peered":"2024-09-21T00:29:21.022370+0000","last_clean":"2024-09-21T00:29:21.022370+0000","last_became_active":"2024-09-21T00:29:19.811129+0000","last_became_peered":"2024-09-21T00:29:19.811129+0000","last_unstale":"2024-09-21T00:29:21.022370+0000","last_undegraded":"2024-09-21T00:29:21.022370+0000","last_fullsized":"2024-09-21T00:29:21.022370+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:26.631458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.807233+0000","last_change":"2024-09-21T00:29:19.809929+0000","last_active":"2024-09-21T00:29:20.807233+0000","last_peered":"2024-09-21T00:29:20.807233+0000","last_clean":"2024-09-21T00:29:20.807233+0000","last_became_active":"2024-09-21T00:29:19.809541+0000","last_became_peered":"2024-09-21T00:29:19.809541+0000","last_unstale":"2024-09-21T00:29:20.807233+0000","last_undegraded":"2024-09-21T00:29:20.807233+0000","last_fullsized":"2024-09-21T00:29:20.807233+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:34:11.115991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.810853+0000","last_change":"2024-09-21T00:29:19.808754+0000","last_active":"2024-09-21T00:29:20.810853+0000","last_peered":"2024-09-21T00:29:20.810853+0000","last_clean":"2024-09-21T00:29:20.810853+0000","last_became_active":"2024-09-21T00:29:19.807857+0000","last_became_peered":"2024-09-21T00:29:19.807857+0000","last_unstale":"2024-09-21T00:29:20.810853+0000","last_undegraded":"2024-09-21T00:29:20.810853+0000","last_fullsized":"2024-09-21T00:29:20.810853+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:53:45.848291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.023011+0000","last_change":"2024-09-21T00:29:19.815917+0000","last_active":"2024-09-21T00:29:21.023011+0000","last_peered":"2024-09-21T00:29:21.023011+0000","last_clean":"2024-09-21T00:29:21.023011+0000","last_became_active":"2024-09-21T00:29:19.815559+0000","last_became_peered":"2024-09-21T00:29:19.815559+0000","last_unstale":"2024-09-21T00:29:21.023011+0000","last_undegraded":"2024-09-21T00:29:21.023011+0000","last_fullsized":"2024-09-21T00:29:21.023011+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:47:06.729633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.021820+0000","last_change":"2024-09-21T00:29:19.803150+0000","last_active":"2024-09-21T00:29:21.021820+0000","last_peered":"2024-09-21T00:29:21.021820+0000","last_clean":"2024-09-21T00:29:21.021820+0000","last_became_active":"2024-09-21T00:29:19.802841+0000","last_became_peered":"2024-09-21T00:29:19.802841+0000","last_unstale":"2024-09-21T00:29:21.021820+0000","last_undegraded":"2024-09-21T00:29:21.021820+0000","last_fullsized":"2024-09-21T00:29:21.021820+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:31:42.414674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.807054+0000","last_change":"2024-09-21T00:29:19.802421+0000","last_active":"2024-09-21T00:29:20.807054+0000","last_peered":"2024-09-21T00:29:20.807054+0000","last_clean":"2024-09-21T00:29:20.807054+0000","last_became_active":"2024-09-21T00:29:19.802074+0000","last_became_peered":"2024-09-21T00:29:19.802074+0000","last_unstale":"2024-09-21T00:29:20.807054+0000","last_undegraded":"2024-09-21T00:29:20.807054+0000","last_fullsized":"2024-09-21T00:29:20.807054+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:55:22.327108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.021168+0000","last_change":"2024-09-21T00:29:19.806258+0000","last_active":"2024-09-21T00:29:21.021168+0000","last_peered":"2024-09-21T00:29:21.021168+0000","last_clean":"2024-09-21T00:29:21.021168+0000","last_became_active":"2024-09-21T00:29:19.805930+0000","last_became_peered":"2024-09-21T00:29:19.805930+0000","last_unstale":"2024-09-21T00:29:21.021168+0000","last_undegraded":"2024-09-21T00:29:21.021168+0000","last_fullsized":"2024-09-21T00:29:21.021168+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:46:50.551185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.810198+0000","last_change":"2024-09-21T00:29:19.802920+0000","last_active":"2024-09-21T00:29:20.810198+0000","last_peered":"2024-09-21T00:29:20.810198+0000","last_clean":"2024-09-21T00:29:20.810198+0000","last_became_active":"2024-09-21T00:29:19.802584+0000","last_became_peered":"2024-09-21T00:29:19.802584+0000","last_unstale":"2024-09-21T00:29:20.810198+0000","last_undegraded":"2024-09-21T00:29:20.810198+0000","last_fullsized":"2024-09-21T00:29:20.810198+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:38:27.387471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.810402+0000","last_change":"2024-09-21T00:29:19.808064+0000","last_active":"2024-09-21T00:29:20.810402+0000","last_peered":"2024-09-21T00:29:20.810402+0000","last_clean":"2024-09-21T00:29:20.810402+0000","last_became_active":"2024-09-21T00:29:19.807712+0000","last_became_peered":"2024-09-21T00:29:19.807712+0000","last_unstale":"2024-09-21T00:29:20.810402+0000","last_undegraded":"2024-09-21T00:29:20.810402+0000","last_fullsized":"2024-09-21T00:29:20.810402+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:45:54.930710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.810053+0000","last_change":"2024-09-21T00:29:19.811883+0000","last_active":"2024-09-21T00:29:20.810053+0000","last_peered":"2024-09-21T00:29:20.810053+0000","last_clean":"2024-09-21T00:29:20.810053+0000","last_became_active":"2024-09-21T00:29:19.811584+0000","last_became_peered":"2024-09-21T00:29:19.811584+0000","last_unstale":"2024-09-21T00:29:20.810053+0000","last_undegraded":"2024-09-21T00:29:20.810053+0000","last_fullsized":"2024-09-21T00:29:20.810053+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:00:59.593942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022016+0000","last_change":"2024-09-21T00:29:19.816999+0000","last_active":"2024-09-21T00:29:21.022016+0000","last_peered":"2024-09-21T00:29:21.022016+0000","last_clean":"2024-09-21T00:29:21.022016+0000","last_became_active":"2024-09-21T00:29:19.814905+0000","last_became_peered":"2024-09-21T00:29:19.814905+0000","last_unstale":"2024-09-21T00:29:21.022016+0000","last_undegraded":"2024-09-21T00:29:21.022016+0000","last_fullsized":"2024-09-21T00:29:21.022016+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:46:57.412342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.023130+0000","last_change":"2024-09-21T00:29:19.815104+0000","last_active":"2024-09-21T00:29:21.023130+0000","last_peered":"2024-09-21T00:29:21.023130+0000","last_clean":"2024-09-21T00:29:21.023130+0000","last_became_active":"2024-09-21T00:29:19.814806+0000","last_became_peered":"2024-09-21T00:29:19.814806+0000","last_unstale":"2024-09-21T00:29:21.023130+0000","last_undegraded":"2024-09-21T00:29:21.023130+0000","last_fullsized":"2024-09-21T00:29:21.023130+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:14:39.263553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.810509+0000","last_change":"2024-09-21T00:29:19.802931+0000","last_active":"2024-09-21T00:29:20.810509+0000","last_peered":"2024-09-21T00:29:20.810509+0000","last_clean":"2024-09-21T00:29:20.810509+0000","last_became_active":"2024-09-21T00:29:19.802606+0000","last_became_peered":"2024-09-21T00:29:19.802606+0000","last_unstale":"2024-09-21T00:29:20.810509+0000","last_undegraded":"2024-09-21T00:29:20.810509+0000","last_fullsized":"2024-09-21T00:29:20.810509+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:56:07.296305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022325+0000","last_change":"2024-09-21T00:29:19.812717+0000","last_active":"2024-09-21T00:29:21.022325+0000","last_peered":"2024-09-21T00:29:21.022325+0000","last_clean":"2024-09-21T00:29:21.022325+0000","last_became_active":"2024-09-21T00:29:19.812402+0000","last_became_peered":"2024-09-21T00:29:19.812402+0000","last_unstale":"2024-09-21T00:29:21.022325+0000","last_undegraded":"2024-09-21T00:29:21.022325+0000","last_fullsized":"2024-09-21T00:29:21.022325+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:26:39.938614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022839+0000","last_change":"2024-09-21T00:29:19.816588+0000","last_active":"2024-09-21T00:29:21.022839+0000","last_peered":"2024-09-21T00:29:21.022839+0000","last_clean":"2024-09-21T00:29:21.022839+0000","last_became_active":"2024-09-21T00:29:19.815574+0000","last_became_peered":"2024-09-21T00:29:19.815574+0000","last_unstale":"2024-09-21T00:29:21.022839+0000","last_undegraded":"2024-09-21T00:29:21.022839+0000","last_fullsized":"2024-09-21T00:29:21.022839+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:47:08.243928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.807629+0000","last_change":"2024-09-21T00:29:19.810747+0000","last_active":"2024-09-21T00:29:20.807629+0000","last_peered":"2024-09-21T00:29:20.807629+0000","last_clean":"2024-09-21T00:29:20.807629+0000","last_became_active":"2024-09-21T00:29:19.810429+0000","last_became_peered":"2024-09-21T00:29:19.810429+0000","last_unstale":"2024-09-21T00:29:20.807629+0000","last_undegraded":"2024-09-21T00:29:20.807629+0000","last_fullsized":"2024-09-21T00:29:20.807629+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:21:05.578580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"93'4","reported_seq":19,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022399+0000","last_change":"2024-09-21T00:29:18.798208+0000","last_active":"2024-09-21T00:29:21.022399+0000","last_peered":"2024-09-21T00:29:21.022399+0000","last_clean":"2024-09-21T00:29:21.022399+0000","last_became_active":"2024-09-21T00:29:18.797857+0000","last_became_peered":"2024-09-21T00:29:18.797857+0000","last_unstale":"2024-09-21T00:29:21.022399+0000","last_undegraded":"2024-09-21T00:29:21.022399+0000","last_fullsized":"2024-09-21T00:29:21.022399+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T00:37:31.452087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.021779+0000","last_change":"2024-09-21T00:29:19.822277+0000","last_active":"2024-09-21T00:29:21.021779+0000","last_peered":"2024-09-21T00:29:21.021779+0000","last_clean":"2024-09-21T00:29:21.021779+0000","last_became_active":"2024-09-21T00:29:19.821917+0000","last_became_peered":"2024-09-21T00:29:19.821917+0000","last_unstale":"2024-09-21T00:29:21.021779+0000","last_undegraded":"2024-09-21T00:29:21.021779+0000","last_fullsized":"2024-09-21T00:29:21.021779+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:04:07.143491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"93'1","reported_seq":16,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022961+0000","last_change":"2024-09-21T00:29:18.799882+0000","last_active":"2024-09-21T00:29:21.022961+0000","last_peered":"2024-09-21T00:29:21.022961+0000","last_clean":"2024-09-21T00:29:21.022961+0000","last_became_active":"2024-09-21T00:29:18.799524+0000","last_became_peered":"2024-09-21T00:29:18.799524+0000","last_unstale":"2024-09-21T00:29:21.022961+0000","last_undegraded":"2024-09-21T00:29:21.022961+0000","last_fullsized":"2024-09-21T00:29:21.022961+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:39:17.740976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.810993+0000","last_change":"2024-09-21T00:29:19.809700+0000","last_active":"2024-09-21T00:29:20.810993+0000","last_peered":"2024-09-21T00:29:20.810993+0000","last_clean":"2024-09-21T00:29:20.810993+0000","last_became_active":"2024-09-21T00:29:19.809343+0000","last_became_peered":"2024-09-21T00:29:19.809343+0000","last_unstale":"2024-09-21T00:29:20.810993+0000","last_undegraded":"2024-09-21T00:29:20.810993+0000","last_fullsized":"2024-09-21T00:29:20.810993+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:28:50.502430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"93'2","reported_seq":17,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.810949+0000","last_change":"2024-09-21T00:29:18.797271+0000","last_active":"2024-09-21T00:29:20.810949+0000","last_peered":"2024-09-21T00:29:20.810949+0000","last_clean":"2024-09-21T00:29:20.810949+0000","last_became_active":"2024-09-21T00:29:18.796656+0000","last_became_peered":"2024-09-21T00:29:18.796656+0000","last_unstale":"2024-09-21T00:29:20.810949+0000","last_undegraded":"2024-09-21T00:29:20.810949+0000","last_fullsized":"2024-09-21T00:29:20.810949+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:28:00.490269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.810191+0000","last_change":"2024-09-21T00:29:19.802323+0000","last_active":"2024-09-21T00:29:20.810191+0000","last_peered":"2024-09-21T00:29:20.810191+0000","last_clean":"2024-09-21T00:29:20.810191+0000","last_became_active":"2024-09-21T00:29:19.801998+0000","last_became_peered":"2024-09-21T00:29:19.801998+0000","last_unstale":"2024-09-21T00:29:20.810191+0000","last_undegraded":"2024-09-21T00:29:20.810191+0000","last_fullsized":"2024-09-21T00:29:20.810191+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:31:43.191165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"93'2","reported_seq":17,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.021788+0000","last_change":"2024-09-21T00:29:18.798679+0000","last_active":"2024-09-21T00:29:21.021788+0000","last_peered":"2024-09-21T00:29:21.021788+0000","last_clean":"2024-09-21T00:29:21.021788+0000","last_became_active":"2024-09-21T00:29:18.798341+0000","last_became_peered":"2024-09-21T00:29:18.798341+0000","last_unstale":"2024-09-21T00:29:21.021788+0000","last_undegraded":"2024-09-21T00:29:21.021788+0000","last_fullsized":"2024-09-21T00:29:21.021788+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:29:04.021021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022055+0000","last_change":"2024-09-21T00:29:19.821679+0000","last_active":"2024-09-21T00:29:21.022055+0000","last_peered":"2024-09-21T00:29:21.022055+0000","last_clean":"2024-09-21T00:29:21.022055+0000","last_became_active":"2024-09-21T00:29:19.821331+0000","last_became_peered":"2024-09-21T00:29:19.821331+0000","last_unstale":"2024-09-21T00:29:21.022055+0000","last_undegraded":"2024-09-21T00:29:21.022055+0000","last_fullsized":"2024-09-21T00:29:21.022055+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:13:53.211350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"93'1","reported_seq":16,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.021871+0000","last_change":"2024-09-21T00:29:18.797442+0000","last_active":"2024-09-21T00:29:21.021871+0000","last_peered":"2024-09-21T00:29:21.021871+0000","last_clean":"2024-09-21T00:29:21.021871+0000","last_became_active":"2024-09-21T00:29:18.797128+0000","last_became_peered":"2024-09-21T00:29:18.797128+0000","last_unstale":"2024-09-21T00:29:21.021871+0000","last_undegraded":"2024-09-21T00:29:21.021871+0000","last_fullsized":"2024-09-21T00:29:21.021871+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:22:52.216862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.807399+0000","last_change":"2024-09-21T00:29:19.808878+0000","last_active":"2024-09-21T00:29:20.807399+0000","last_peered":"2024-09-21T00:29:20.807399+0000","last_clean":"2024-09-21T00:29:20.807399+0000","last_became_active":"2024-09-21T00:29:19.808577+0000","last_became_peered":"2024-09-21T00:29:19.808577+0000","last_unstale":"2024-09-21T00:29:20.807399+0000","last_undegraded":"2024-09-21T00:29:20.807399+0000","last_fullsized":"2024-09-21T00:29:20.807399+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:34:29.883537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"93'1","reported_seq":16,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.810090+0000","last_change":"2024-09-21T00:29:18.801027+0000","last_active":"2024-09-21T00:29:20.810090+0000","last_peered":"2024-09-21T00:29:20.810090+0000","last_clean":"2024-09-21T00:29:20.810090+0000","last_became_active":"2024-09-21T00:29:18.800693+0000","last_became_peered":"2024-09-21T00:29:18.800693+0000","last_unstale":"2024-09-21T00:29:20.810090+0000","last_undegraded":"2024-09-21T00:29:20.810090+0000","last_fullsized":"2024-09-21T00:29:20.810090+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:17:06.642779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022624+0000","last_change":"2024-09-21T00:29:19.815242+0000","last_active":"2024-09-21T00:29:21.022624+0000","last_peered":"2024-09-21T00:29:21.022624+0000","last_clean":"2024-09-21T00:29:21.022624+0000","last_became_active":"2024-09-21T00:29:19.814858+0000","last_became_peered":"2024-09-21T00:29:19.814858+0000","last_unstale":"2024-09-21T00:29:21.022624+0000","last_undegraded":"2024-09-21T00:29:21.022624+0000","last_fullsized":"2024-09-21T00:29:21.022624+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:35:06.851240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.809872+0000","last_change":"2024-09-21T00:29:18.797155+0000","last_active":"2024-09-21T00:29:20.809872+0000","last_peered":"2024-09-21T00:29:20.809872+0000","last_clean":"2024-09-21T00:29:20.809872+0000","last_became_active":"2024-09-21T00:29:18.795953+0000","last_became_peered":"2024-09-21T00:29:18.795953+0000","last_unstale":"2024-09-21T00:29:20.809872+0000","last_undegraded":"2024-09-21T00:29:20.809872+0000","last_fullsized":"2024-09-21T00:29:20.809872+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T00:54:16.170396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022428+0000","last_change":"2024-09-21T00:29:19.817180+0000","last_active":"2024-09-21T00:29:21.022428+0000","last_peered":"2024-09-21T00:29:21.022428+0000","last_clean":"2024-09-21T00:29:21.022428+0000","last_became_active":"2024-09-21T00:29:19.816859+0000","last_became_peered":"2024-09-21T00:29:19.816859+0000","last_unstale":"2024-09-21T00:29:21.022428+0000","last_undegraded":"2024-09-21T00:29:21.022428+0000","last_fullsized":"2024-09-21T00:29:21.022428+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:30:01.904485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.021605+0000","last_change":"2024-09-21T00:29:18.793095+0000","last_active":"2024-09-21T00:29:21.021605+0000","last_peered":"2024-09-21T00:29:21.021605+0000","last_clean":"2024-09-21T00:29:21.021605+0000","last_became_active":"2024-09-21T00:29:18.792693+0000","last_became_peered":"2024-09-21T00:29:18.792693+0000","last_unstale":"2024-09-21T00:29:21.021605+0000","last_undegraded":"2024-09-21T00:29:21.021605+0000","last_fullsized":"2024-09-21T00:29:21.021605+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:56:18.147899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.807542+0000","last_change":"2024-09-21T00:29:19.815545+0000","last_active":"2024-09-21T00:29:20.807542+0000","last_peered":"2024-09-21T00:29:20.807542+0000","last_clean":"2024-09-21T00:29:20.807542+0000","last_became_active":"2024-09-21T00:29:19.811424+0000","last_became_peered":"2024-09-21T00:29:19.811424+0000","last_unstale":"2024-09-21T00:29:20.807542+0000","last_undegraded":"2024-09-21T00:29:20.807542+0000","last_fullsized":"2024-09-21T00:29:20.807542+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:20:57.334693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.810327+0000","last_change":"2024-09-21T00:29:18.796965+0000","last_active":"2024-09-21T00:29:20.810327+0000","last_peered":"2024-09-21T00:29:20.810327+0000","last_clean":"2024-09-21T00:29:20.810327+0000","last_became_active":"2024-09-21T00:29:18.796548+0000","last_became_peered":"2024-09-21T00:29:18.796548+0000","last_unstale":"2024-09-21T00:29:20.810327+0000","last_undegraded":"2024-09-21T00:29:20.810327+0000","last_fullsized":"2024-09-21T00:29:20.810327+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:00:18.177473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.021362+0000","last_change":"2024-09-21T00:29:19.814792+0000","last_active":"2024-09-21T00:29:21.021362+0000","last_peered":"2024-09-21T00:29:21.021362+0000","last_clean":"2024-09-21T00:29:21.021362+0000","last_became_active":"2024-09-21T00:29:19.814147+0000","last_became_peered":"2024-09-21T00:29:19.814147+0000","last_unstale":"2024-09-21T00:29:21.021362+0000","last_undegraded":"2024-09-21T00:29:21.021362+0000","last_fullsized":"2024-09-21T00:29:21.021362+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:01:27.998993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.807185+0000","last_change":"2024-09-21T00:29:18.798831+0000","last_active":"2024-09-21T00:29:20.807185+0000","last_peered":"2024-09-21T00:29:20.807185+0000","last_clean":"2024-09-21T00:29:20.807185+0000","last_became_active":"2024-09-21T00:29:18.798551+0000","last_became_peered":"2024-09-21T00:29:18.798551+0000","last_unstale":"2024-09-21T00:29:20.807185+0000","last_undegraded":"2024-09-21T00:29:20.807185+0000","last_fullsized":"2024-09-21T00:29:20.807185+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:12:30.180725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022749+0000","last_change":"2024-09-21T00:29:19.812082+0000","last_active":"2024-09-21T00:29:21.022749+0000","last_peered":"2024-09-21T00:29:21.022749+0000","last_clean":"2024-09-21T00:29:21.022749+0000","last_became_active":"2024-09-21T00:29:19.811773+0000","last_became_peered":"2024-09-21T00:29:19.811773+0000","last_unstale":"2024-09-21T00:29:21.022749+0000","last_undegraded":"2024-09-21T00:29:21.022749+0000","last_fullsized":"2024-09-21T00:29:21.022749+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:17:43.069499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"93'3","reported_seq":18,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.021859+0000","last_change":"2024-09-21T00:29:18.794178+0000","last_active":"2024-09-21T00:29:21.021859+0000","last_peered":"2024-09-21T00:29:21.021859+0000","last_clean":"2024-09-21T00:29:21.021859+0000","last_became_active":"2024-09-21T00:29:18.793851+0000","last_became_peered":"2024-09-21T00:29:18.793851+0000","last_unstale":"2024-09-21T00:29:21.021859+0000","last_undegraded":"2024-09-21T00:29:21.021859+0000","last_fullsized":"2024-09-21T00:29:21.021859+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:52:27.563468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022394+0000","last_change":"2024-09-21T00:29:19.814438+0000","last_active":"2024-09-21T00:29:21.022394+0000","last_peered":"2024-09-21T00:29:21.022394+0000","last_clean":"2024-09-21T00:29:21.022394+0000","last_became_active":"2024-09-21T00:29:19.814085+0000","last_became_peered":"2024-09-21T00:29:19.814085+0000","last_unstale":"2024-09-21T00:29:21.022394+0000","last_undegraded":"2024-09-21T00:29:21.022394+0000","last_fullsized":"2024-09-21T00:29:21.022394+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:08:38.851080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"93'6","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.021896+0000","last_change":"2024-09-21T00:29:18.800236+0000","last_active":"2024-09-21T00:29:21.021896+0000","last_peered":"2024-09-21T00:29:21.021896+0000","last_clean":"2024-09-21T00:29:21.021896+0000","last_became_active":"2024-09-21T00:29:18.799899+0000","last_became_peered":"2024-09-21T00:29:18.799899+0000","last_unstale":"2024-09-21T00:29:21.021896+0000","last_undegraded":"2024-09-21T00:29:21.021896+0000","last_fullsized":"2024-09-21T00:29:21.021896+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:29:10.908555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.809551+0000","last_change":"2024-09-21T00:29:19.811833+0000","last_active":"2024-09-21T00:29:20.809551+0000","last_peered":"2024-09-21T00:29:20.809551+0000","last_clean":"2024-09-21T00:29:20.809551+0000","last_became_active":"2024-09-21T00:29:19.811519+0000","last_became_peered":"2024-09-21T00:29:19.811519+0000","last_unstale":"2024-09-21T00:29:20.809551+0000","last_undegraded":"2024-09-21T00:29:20.809551+0000","last_fullsized":"2024-09-21T00:29:20.809551+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:52:28.568849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"93'2","reported_seq":17,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022457+0000","last_change":"2024-09-21T00:29:18.801761+0000","last_active":"2024-09-21T00:29:21.022457+0000","last_peered":"2024-09-21T00:29:21.022457+0000","last_clean":"2024-09-21T00:29:21.022457+0000","last_became_active":"2024-09-21T00:29:18.801422+0000","last_became_peered":"2024-09-21T00:29:18.801422+0000","last_unstale":"2024-09-21T00:29:21.022457+0000","last_undegraded":"2024-09-21T00:29:21.022457+0000","last_fullsized":"2024-09-21T00:29:21.022457+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:55:45.769871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022489+0000","last_change":"2024-09-21T00:29:19.814412+0000","last_active":"2024-09-21T00:29:21.022489+0000","last_peered":"2024-09-21T00:29:21.022489+0000","last_clean":"2024-09-21T00:29:21.022489+0000","last_became_active":"2024-09-21T00:29:19.813923+0000","last_became_peered":"2024-09-21T00:29:19.813923+0000","last_unstale":"2024-09-21T00:29:21.022489+0000","last_undegraded":"2024-09-21T00:29:21.022489+0000","last_fullsized":"2024-09-21T00:29:21.022489+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:42:28.398366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022221+0000","last_change":"2024-09-21T00:29:18.788248+0000","last_active":"2024-09-21T00:29:21.022221+0000","last_peered":"2024-09-21T00:29:21.022221+0000","last_clean":"2024-09-21T00:29:21.022221+0000","last_became_active":"2024-09-21T00:29:18.787904+0000","last_became_peered":"2024-09-21T00:29:18.787904+0000","last_unstale":"2024-09-21T00:29:21.022221+0000","last_undegraded":"2024-09-21T00:29:21.022221+0000","last_fullsized":"2024-09-21T00:29:21.022221+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:21:04.691629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.023078+0000","last_change":"2024-09-21T00:29:19.814341+0000","last_active":"2024-09-21T00:29:21.023078+0000","last_peered":"2024-09-21T00:29:21.023078+0000","last_clean":"2024-09-21T00:29:21.023078+0000","last_became_active":"2024-09-21T00:29:19.814052+0000","last_became_peered":"2024-09-21T00:29:19.814052+0000","last_unstale":"2024-09-21T00:29:21.023078+0000","last_undegraded":"2024-09-21T00:29:21.023078+0000","last_fullsized":"2024-09-21T00:29:21.023078+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:58:18.707835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.021901+0000","last_change":"2024-09-21T00:29:18.800084+0000","last_active":"2024-09-21T00:29:21.021901+0000","last_peered":"2024-09-21T00:29:21.021901+0000","last_clean":"2024-09-21T00:29:21.021901+0000","last_became_active":"2024-09-21T00:29:18.799797+0000","last_became_peered":"2024-09-21T00:29:18.799797+0000","last_unstale":"2024-09-21T00:29:21.021901+0000","last_undegraded":"2024-09-21T00:29:21.021901+0000","last_fullsized":"2024-09-21T00:29:21.021901+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T00:51:35.304058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.807595+0000","last_change":"2024-09-21T00:29:19.804683+0000","last_active":"2024-09-21T00:29:20.807595+0000","last_peered":"2024-09-21T00:29:20.807595+0000","last_clean":"2024-09-21T00:29:20.807595+0000","last_became_active":"2024-09-21T00:29:19.803246+0000","last_became_peered":"2024-09-21T00:29:19.803246+0000","last_unstale":"2024-09-21T00:29:20.807595+0000","last_undegraded":"2024-09-21T00:29:20.807595+0000","last_fullsized":"2024-09-21T00:29:20.807595+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:46:26.637135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"93'8","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.810678+0000","last_change":"2024-09-21T00:29:18.796567+0000","last_active":"2024-09-21T00:29:20.810678+0000","last_peered":"2024-09-21T00:29:20.810678+0000","last_clean":"2024-09-21T00:29:20.810678+0000","last_became_active":"2024-09-21T00:29:18.796255+0000","last_became_peered":"2024-09-21T00:29:18.796255+0000","last_unstale":"2024-09-21T00:29:20.810678+0000","last_undegraded":"2024-09-21T00:29:20.810678+0000","last_fullsized":"2024-09-21T00:29:20.810678+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:04:53.438244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.023031+0000","last_change":"2024-09-21T00:29:19.817234+0000","last_active":"2024-09-21T00:29:21.023031+0000","last_peered":"2024-09-21T00:29:21.023031+0000","last_clean":"2024-09-21T00:29:21.023031+0000","last_became_active":"2024-09-21T00:29:19.816273+0000","last_became_peered":"2024-09-21T00:29:19.816273+0000","last_unstale":"2024-09-21T00:29:21.023031+0000","last_undegraded":"2024-09-21T00:29:21.023031+0000","last_fullsized":"2024-09-21T00:29:21.023031+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:45:02.009239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"93'1","reported_seq":16,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022386+0000","last_change":"2024-09-21T00:29:18.799579+0000","last_active":"2024-09-21T00:29:21.022386+0000","last_peered":"2024-09-21T00:29:21.022386+0000","last_clean":"2024-09-21T00:29:21.022386+0000","last_became_active":"2024-09-21T00:29:18.799238+0000","last_became_peered":"2024-09-21T00:29:18.799238+0000","last_unstale":"2024-09-21T00:29:21.022386+0000","last_undegraded":"2024-09-21T00:29:21.022386+0000","last_fullsized":"2024-09-21T00:29:21.022386+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:01:35.301014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022278+0000","last_change":"2024-09-21T00:29:19.814239+0000","last_active":"2024-09-21T00:29:21.022278+0000","last_peered":"2024-09-21T00:29:21.022278+0000","last_clean":"2024-09-21T00:29:21.022278+0000","last_became_active":"2024-09-21T00:29:19.813892+0000","last_became_peered":"2024-09-21T00:29:19.813892+0000","last_unstale":"2024-09-21T00:29:21.022278+0000","last_undegraded":"2024-09-21T00:29:21.022278+0000","last_fullsized":"2024-09-21T00:29:21.022278+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:02:55.200607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"93'1","reported_seq":16,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.021997+0000","last_change":"2024-09-21T00:29:18.786505+0000","last_active":"2024-09-21T00:29:21.021997+0000","last_peered":"2024-09-21T00:29:21.021997+0000","last_clean":"2024-09-21T00:29:21.021997+0000","last_became_active":"2024-09-21T00:29:18.786141+0000","last_became_peered":"2024-09-21T00:29:18.786141+0000","last_unstale":"2024-09-21T00:29:21.021997+0000","last_undegraded":"2024-09-21T00:29:21.021997+0000","last_fullsized":"2024-09-21T00:29:21.021997+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:59:49.904811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022402+0000","last_change":"2024-09-21T00:29:19.810193+0000","last_active":"2024-09-21T00:29:21.022402+0000","last_peered":"2024-09-21T00:29:21.022402+0000","last_clean":"2024-09-21T00:29:21.022402+0000","last_became_active":"2024-09-21T00:29:19.809872+0000","last_became_peered":"2024-09-21T00:29:19.809872+0000","last_unstale":"2024-09-21T00:29:21.022402+0000","last_undegraded":"2024-09-21T00:29:21.022402+0000","last_fullsized":"2024-09-21T00:29:21.022402+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:42:51.392778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.809671+0000","last_change":"2024-09-21T00:29:18.802388+0000","last_active":"2024-09-21T00:29:20.809671+0000","last_peered":"2024-09-21T00:29:20.809671+0000","last_clean":"2024-09-21T00:29:20.809671+0000","last_became_active":"2024-09-21T00:29:18.802060+0000","last_became_peered":"2024-09-21T00:29:18.802060+0000","last_unstale":"2024-09-21T00:29:20.809671+0000","last_undegraded":"2024-09-21T00:29:20.809671+0000","last_fullsized":"2024-09-21T00:29:20.809671+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:26:13.866796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.807105+0000","last_change":"2024-09-21T00:29:19.812924+0000","last_active":"2024-09-21T00:29:20.807105+0000","last_peered":"2024-09-21T00:29:20.807105+0000","last_clean":"2024-09-21T00:29:20.807105+0000","last_became_active":"2024-09-21T00:29:19.812607+0000","last_became_peered":"2024-09-21T00:29:19.812607+0000","last_unstale":"2024-09-21T00:29:20.807105+0000","last_undegraded":"2024-09-21T00:29:20.807105+0000","last_fullsized":"2024-09-21T00:29:20.807105+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:11:09.667419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"93'2","reported_seq":17,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.023167+0000","last_change":"2024-09-21T00:29:18.796630+0000","last_active":"2024-09-21T00:29:21.023167+0000","last_peered":"2024-09-21T00:29:21.023167+0000","last_clean":"2024-09-21T00:29:21.023167+0000","last_became_active":"2024-09-21T00:29:18.796317+0000","last_became_peered":"2024-09-21T00:29:18.796317+0000","last_unstale":"2024-09-21T00:29:21.023167+0000","last_undegraded":"2024-09-21T00:29:21.023167+0000","last_fullsized":"2024-09-21T00:29:21.023167+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:50:16.709606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.807330+0000","last_change":"2024-09-21T00:29:19.801051+0000","last_active":"2024-09-21T00:29:20.807330+0000","last_peered":"2024-09-21T00:29:20.807330+0000","last_clean":"2024-09-21T00:29:20.807330+0000","last_became_active":"2024-09-21T00:29:19.798834+0000","last_became_peered":"2024-09-21T00:29:19.798834+0000","last_unstale":"2024-09-21T00:29:20.807330+0000","last_undegraded":"2024-09-21T00:29:20.807330+0000","last_fullsized":"2024-09-21T00:29:20.807330+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:20:15.757612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022535+0000","last_change":"2024-09-21T00:29:18.796311+0000","last_active":"2024-09-21T00:29:21.022535+0000","last_peered":"2024-09-21T00:29:21.022535+0000","last_clean":"2024-09-21T00:29:21.022535+0000","last_became_active":"2024-09-21T00:29:18.795425+0000","last_became_peered":"2024-09-21T00:29:18.795425+0000","last_unstale":"2024-09-21T00:29:21.022535+0000","last_undegraded":"2024-09-21T00:29:21.022535+0000","last_fullsized":"2024-09-21T00:29:21.022535+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:12:21.051042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022415+0000","last_change":"2024-09-21T00:29:19.813509+0000","last_active":"2024-09-21T00:29:21.022415+0000","last_peered":"2024-09-21T00:29:21.022415+0000","last_clean":"2024-09-21T00:29:21.022415+0000","last_became_active":"2024-09-21T00:29:19.813162+0000","last_became_peered":"2024-09-21T00:29:19.813162+0000","last_unstale":"2024-09-21T00:29:21.022415+0000","last_undegraded":"2024-09-21T00:29:21.022415+0000","last_fullsized":"2024-09-21T00:29:21.022415+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:48:11.528107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022396+0000","last_change":"2024-09-21T00:29:18.799921+0000","last_active":"2024-09-21T00:29:21.022396+0000","last_peered":"2024-09-21T00:29:21.022396+0000","last_clean":"2024-09-21T00:29:21.022396+0000","last_became_active":"2024-09-21T00:29:18.799614+0000","last_became_peered":"2024-09-21T00:29:18.799614+0000","last_unstale":"2024-09-21T00:29:21.022396+0000","last_undegraded":"2024-09-21T00:29:21.022396+0000","last_fullsized":"2024-09-21T00:29:21.022396+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:33:56.930832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022472+0000","last_change":"2024-09-21T00:29:19.813609+0000","last_active":"2024-09-21T00:29:21.022472+0000","last_peered":"2024-09-21T00:29:21.022472+0000","last_clean":"2024-09-21T00:29:21.022472+0000","last_became_active":"2024-09-21T00:29:19.813291+0000","last_became_peered":"2024-09-21T00:29:19.813291+0000","last_unstale":"2024-09-21T00:29:21.022472+0000","last_undegraded":"2024-09-21T00:29:21.022472+0000","last_fullsized":"2024-09-21T00:29:21.022472+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:48:12.895949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022964+0000","last_change":"2024-09-21T00:29:18.796499+0000","last_active":"2024-09-21T00:29:21.022964+0000","last_peered":"2024-09-21T00:29:21.022964+0000","last_clean":"2024-09-21T00:29:21.022964+0000","last_became_active":"2024-09-21T00:29:18.796175+0000","last_became_peered":"2024-09-21T00:29:18.796175+0000","last_unstale":"2024-09-21T00:29:21.022964+0000","last_undegraded":"2024-09-21T00:29:21.022964+0000","last_fullsized":"2024-09-21T00:29:21.022964+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:58:46.110270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.807498+0000","last_change":"2024-09-21T00:29:19.815935+0000","last_active":"2024-09-21T00:29:20.807498+0000","last_peered":"2024-09-21T00:29:20.807498+0000","last_clean":"2024-09-21T00:29:20.807498+0000","last_became_active":"2024-09-21T00:29:19.814883+0000","last_became_peered":"2024-09-21T00:29:19.814883+0000","last_unstale":"2024-09-21T00:29:20.807498+0000","last_undegraded":"2024-09-21T00:29:20.807498+0000","last_fullsized":"2024-09-21T00:29:20.807498+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:40:18.526456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"93'2","reported_seq":17,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.021237+0000","last_change":"2024-09-21T00:29:18.802766+0000","last_active":"2024-09-21T00:29:21.021237+0000","last_peered":"2024-09-21T00:29:21.021237+0000","last_clean":"2024-09-21T00:29:21.021237+0000","last_became_active":"2024-09-21T00:29:18.802061+0000","last_became_peered":"2024-09-21T00:29:18.802061+0000","last_unstale":"2024-09-21T00:29:21.021237+0000","last_undegraded":"2024-09-21T00:29:21.021237+0000","last_fullsized":"2024-09-21T00:29:21.021237+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:43:45.169355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.809738+0000","last_change":"2024-09-21T00:29:19.806248+0000","last_active":"2024-09-21T00:29:20.809738+0000","last_peered":"2024-09-21T00:29:20.809738+0000","last_clean":"2024-09-21T00:29:20.809738+0000","last_became_active":"2024-09-21T00:29:19.805916+0000","last_became_peered":"2024-09-21T00:29:19.805916+0000","last_unstale":"2024-09-21T00:29:20.809738+0000","last_undegraded":"2024-09-21T00:29:20.809738+0000","last_fullsized":"2024-09-21T00:29:20.809738+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:42:42.562148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"93'1","reported_seq":16,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.807289+0000","last_change":"2024-09-21T00:29:18.797446+0000","last_active":"2024-09-21T00:29:20.807289+0000","last_peered":"2024-09-21T00:29:20.807289+0000","last_clean":"2024-09-21T00:29:20.807289+0000","last_became_active":"2024-09-21T00:29:18.797076+0000","last_became_peered":"2024-09-21T00:29:18.797076+0000","last_unstale":"2024-09-21T00:29:20.807289+0000","last_undegraded":"2024-09-21T00:29:20.807289+0000","last_fullsized":"2024-09-21T00:29:20.807289+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:06:17.566926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022312+0000","last_change":"2024-09-21T00:29:19.812384+0000","last_active":"2024-09-21T00:29:21.022312+0000","last_peered":"2024-09-21T00:29:21.022312+0000","last_clean":"2024-09-21T00:29:21.022312+0000","last_became_active":"2024-09-21T00:29:19.812077+0000","last_became_peered":"2024-09-21T00:29:19.812077+0000","last_unstale":"2024-09-21T00:29:21.022312+0000","last_undegraded":"2024-09-21T00:29:21.022312+0000","last_fullsized":"2024-09-21T00:29:21.022312+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:28:10.979181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022603+0000","last_change":"2024-09-21T00:29:18.796908+0000","last_active":"2024-09-21T00:29:21.022603+0000","last_peered":"2024-09-21T00:29:21.022603+0000","last_clean":"2024-09-21T00:29:21.022603+0000","last_became_active":"2024-09-21T00:29:18.796559+0000","last_became_peered":"2024-09-21T00:29:18.796559+0000","last_unstale":"2024-09-21T00:29:21.022603+0000","last_undegraded":"2024-09-21T00:29:21.022603+0000","last_fullsized":"2024-09-21T00:29:21.022603+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:27:50.568077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.809937+0000","last_change":"2024-09-21T00:29:19.808826+0000","last_active":"2024-09-21T00:29:20.809937+0000","last_peered":"2024-09-21T00:29:20.809937+0000","last_clean":"2024-09-21T00:29:20.809937+0000","last_became_active":"2024-09-21T00:29:19.808196+0000","last_became_peered":"2024-09-21T00:29:19.808196+0000","last_unstale":"2024-09-21T00:29:20.809937+0000","last_undegraded":"2024-09-21T00:29:20.809937+0000","last_fullsized":"2024-09-21T00:29:20.809937+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:22:53.222243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.021895+0000","last_change":"2024-09-21T00:29:18.802804+0000","last_active":"2024-09-21T00:29:21.021895+0000","last_peered":"2024-09-21T00:29:21.021895+0000","last_clean":"2024-09-21T00:29:21.021895+0000","last_became_active":"2024-09-21T00:29:18.802469+0000","last_became_peered":"2024-09-21T00:29:18.802469+0000","last_unstale":"2024-09-21T00:29:21.021895+0000","last_undegraded":"2024-09-21T00:29:21.021895+0000","last_fullsized":"2024-09-21T00:29:21.021895+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:27:36.567501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.021641+0000","last_change":"2024-09-21T00:29:19.823036+0000","last_active":"2024-09-21T00:29:21.021641+0000","last_peered":"2024-09-21T00:29:21.021641+0000","last_clean":"2024-09-21T00:29:21.021641+0000","last_became_active":"2024-09-21T00:29:19.822688+0000","last_became_peered":"2024-09-21T00:29:19.822688+0000","last_unstale":"2024-09-21T00:29:21.021641+0000","last_undegraded":"2024-09-21T00:29:21.021641+0000","last_fullsized":"2024-09-21T00:29:21.021641+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:54:44.835987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.809976+0000","last_change":"2024-09-21T00:29:18.795683+0000","last_active":"2024-09-21T00:29:20.809976+0000","last_peered":"2024-09-21T00:29:20.809976+0000","last_clean":"2024-09-21T00:29:20.809976+0000","last_became_active":"2024-09-21T00:29:18.795363+0000","last_became_peered":"2024-09-21T00:29:18.795363+0000","last_unstale":"2024-09-21T00:29:20.809976+0000","last_undegraded":"2024-09-21T00:29:20.809976+0000","last_fullsized":"2024-09-21T00:29:20.809976+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:20:55.296469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022149+0000","last_change":"2024-09-21T00:29:19.812845+0000","last_active":"2024-09-21T00:29:21.022149+0000","last_peered":"2024-09-21T00:29:21.022149+0000","last_clean":"2024-09-21T00:29:21.022149+0000","last_became_active":"2024-09-21T00:29:19.812536+0000","last_became_peered":"2024-09-21T00:29:19.812536+0000","last_unstale":"2024-09-21T00:29:21.022149+0000","last_undegraded":"2024-09-21T00:29:21.022149+0000","last_fullsized":"2024-09-21T00:29:21.022149+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:48:44.631863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022864+0000","last_change":"2024-09-21T00:29:18.800233+0000","last_active":"2024-09-21T00:29:21.022864+0000","last_peered":"2024-09-21T00:29:21.022864+0000","last_clean":"2024-09-21T00:29:21.022864+0000","last_became_active":"2024-09-21T00:29:18.799952+0000","last_became_peered":"2024-09-21T00:29:18.799952+0000","last_unstale":"2024-09-21T00:29:21.022864+0000","last_undegraded":"2024-09-21T00:29:21.022864+0000","last_fullsized":"2024-09-21T00:29:21.022864+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:21:56.870500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.021841+0000","last_change":"2024-09-21T00:29:19.816067+0000","last_active":"2024-09-21T00:29:21.021841+0000","last_peered":"2024-09-21T00:29:21.021841+0000","last_clean":"2024-09-21T00:29:21.021841+0000","last_became_active":"2024-09-21T00:29:19.815714+0000","last_became_peered":"2024-09-21T00:29:19.815714+0000","last_unstale":"2024-09-21T00:29:21.021841+0000","last_undegraded":"2024-09-21T00:29:21.021841+0000","last_fullsized":"2024-09-21T00:29:21.021841+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:30:01.904485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022492+0000","last_change":"2024-09-21T00:29:18.796212+0000","last_active":"2024-09-21T00:29:21.022492+0000","last_peered":"2024-09-21T00:29:21.022492+0000","last_clean":"2024-09-21T00:29:21.022492+0000","last_became_active":"2024-09-21T00:29:18.795864+0000","last_became_peered":"2024-09-21T00:29:18.795864+0000","last_unstale":"2024-09-21T00:29:21.022492+0000","last_undegraded":"2024-09-21T00:29:21.022492+0000","last_fullsized":"2024-09-21T00:29:21.022492+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:57:43.771137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.810604+0000","last_change":"2024-09-21T00:29:19.811714+0000","last_active":"2024-09-21T00:29:20.810604+0000","last_peered":"2024-09-21T00:29:20.810604+0000","last_clean":"2024-09-21T00:29:20.810604+0000","last_became_active":"2024-09-21T00:29:19.809202+0000","last_became_peered":"2024-09-21T00:29:19.809202+0000","last_unstale":"2024-09-21T00:29:20.810604+0000","last_undegraded":"2024-09-21T00:29:20.810604+0000","last_fullsized":"2024-09-21T00:29:20.810604+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:44:42.287171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.023051+0000","last_change":"2024-09-21T00:29:18.801879+0000","last_active":"2024-09-21T00:29:21.023051+0000","last_peered":"2024-09-21T00:29:21.023051+0000","last_clean":"2024-09-21T00:29:21.023051+0000","last_became_active":"2024-09-21T00:29:18.800708+0000","last_became_peered":"2024-09-21T00:29:18.800708+0000","last_unstale":"2024-09-21T00:29:21.023051+0000","last_undegraded":"2024-09-21T00:29:21.023051+0000","last_fullsized":"2024-09-21T00:29:21.023051+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:48:40.413052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.809702+0000","last_change":"2024-09-21T00:29:19.798077+0000","last_active":"2024-09-21T00:29:20.809702+0000","last_peered":"2024-09-21T00:29:20.809702+0000","last_clean":"2024-09-21T00:29:20.809702+0000","last_became_active":"2024-09-21T00:29:19.797724+0000","last_became_peered":"2024-09-21T00:29:19.797724+0000","last_unstale":"2024-09-21T00:29:20.809702+0000","last_undegraded":"2024-09-21T00:29:20.809702+0000","last_fullsized":"2024-09-21T00:29:20.809702+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:35:16.985174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"93'2","reported_seq":17,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.023087+0000","last_change":"2024-09-21T00:29:18.800162+0000","last_active":"2024-09-21T00:29:21.023087+0000","last_peered":"2024-09-21T00:29:21.023087+0000","last_clean":"2024-09-21T00:29:21.023087+0000","last_became_active":"2024-09-21T00:29:18.798939+0000","last_became_peered":"2024-09-21T00:29:18.798939+0000","last_unstale":"2024-09-21T00:29:21.023087+0000","last_undegraded":"2024-09-21T00:29:21.023087+0000","last_fullsized":"2024-09-21T00:29:21.023087+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:28:05.362595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":244,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.021503+0000","last_change":"2024-09-21T00:25:38.725792+0000","last_active":"2024-09-21T00:29:21.021503+0000","last_peered":"2024-09-21T00:29:21.021503+0000","last_clean":"2024-09-21T00:29:21.021503+0000","last_became_active":"2024-09-21T00:25:38.724668+0000","last_became_peered":"2024-09-21T00:25:38.724668+0000","last_unstale":"2024-09-21T00:29:21.021503+0000","last_undegraded":"2024-09-21T00:29:21.021503+0000","last_fullsized":"2024-09-21T00:29:21.021503+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:37.163216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:37.163216+0000","last_clean_scrub_stamp":"2024-09-21T00:25:37.163216+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:14:36.588660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509488,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32908,"kb_used_data":1720,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338932,"statfs":{"total":96636764160,"available":96603066368,"internally_reserved":0,"allocated":1761280,"data_stored":1116174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509489,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32336,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339504,"statfs":{"total":96636764160,"available":96603652096,"internally_reserved":0,"allocated":1167360,"data_stored":521170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509488,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32308,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339532,"statfs":{"total":96636764160,"available":96603680768,"internally_reserved":0,"allocated":1146880,"data_stored":518954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509487,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32328,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339512,"statfs":{"total":96636764160,"available":96603660288,"internally_reserved":0,"allocated":1167360,"data_stored":525738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509489,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32328,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339512,"statfs":{"total":96636764160,"available":96603660288,"internally_reserved":0,"allocated":1163264,"data_stored":526228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509489,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32324,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339516,"statfs":{"total":96636764160,"available":96603664384,"internally_reserved":0,"allocated":1163264,"data_stored":525680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509489,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32904,"kb_used_data":1716,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338936,"statfs":{"total":96636764160,"available":96603070464,"internally_reserved":0,"allocated":1757184,"data_stored":1111024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574897,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32328,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339512,"statfs":{"total":96636764160,"available":96603660288,"internally_reserved":0,"allocated":1163264,"data_stored":525680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T00:29:28.290 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T00:29:28.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:28.415+0000 7f75a90b2640 1 -- 172.21.15.112:0/1754832096 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7590003b10 msgr2=0x7f7590003f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:28.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:28.415+0000 7f75a90b2640 1 --2- 172.21.15.112:0/1754832096 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7590003b10 0x7f7590003f80 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f759400c260 tx=0x7f7590005830 comp rx=0 tx=0).stop 2024-09-21T00:29:28.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:28.415+0000 7f75a359e640 1 -- 172.21.15.112:0/1754832096 reap_dead start 2024-09-21T00:29:28.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:28.415+0000 7f75a90b2640 1 -- 172.21.15.112:0/1754832096 shutdown_connections 2024-09-21T00:29:28.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:28.415+0000 7f75a90b2640 1 -- 172.21.15.112:0/1754832096 >> 172.21.15.112:0/1754832096 conn(0x7f75a40b43f0 msgr2=0x7f75a40b4830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:28.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:28.416+0000 7f75a90b2640 1 -- 172.21.15.112:0/1754832096 shutdown_connections 2024-09-21T00:29:28.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:28.416+0000 7f75a90b2640 1 -- 172.21.15.112:0/1754832096 wait complete. 2024-09-21T00:29:28.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:28.417+0000 7f75a90b2640 1 Processor -- start 2024-09-21T00:29:28.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:28.418+0000 7f75a90b2640 1 -- start start 2024-09-21T00:29:28.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:28.419+0000 7f75a90b2640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7590003b10 0x7f75a4158f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:28.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:28.419+0000 7f75a90b2640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f75a41f39e0 0x7f75a4159440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:28.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:28.419+0000 7f75a90b2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75a41f70d0 0x7f75a41f74e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:28.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:28.419+0000 7f75a90b2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f75a41636c0 con 0x7f75a41f70d0 2024-09-21T00:29:28.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:28.419+0000 7f75a90b2640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f75a40d0d70 con 0x7f75a41f39e0 2024-09-21T00:29:28.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:28.419+0000 7f75a90b2640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f75a4163840 con 0x7f7590003b10 2024-09-21T00:29:28.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:28.419+0000 7f75a359e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75a41f70d0 0x7f75a41f74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:28.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:28.419+0000 7f75a359e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75a41f70d0 0x7f75a41f74e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57818/0 (socket says 172.21.15.112:57818) 2024-09-21T00:29:28.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:28.419+0000 7f75a359e640 1 -- 172.21.15.112:0/3260393155 learned_addr learned my addr 172.21.15.112:0/3260393155 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:29:28.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:28.419+0000 7f759a59c640 1 --2- 172.21.15.112:0/3260393155 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f75a41f39e0 0x7f75a4159440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:28.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:28.419+0000 7f75a2d9d640 1 --2- 172.21.15.112:0/3260393155 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7590003b10 0x7f75a4158f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:28.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:28.420+0000 7f75a359e640 1 -- 172.21.15.112:0/3260393155 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7590003b10 msgr2=0x7f75a4158f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:28.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:28.420+0000 7f75a359e640 1 --2- 172.21.15.112:0/3260393155 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7590003b10 0x7f75a4158f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:28.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:28.420+0000 7f75a359e640 1 -- 172.21.15.112:0/3260393155 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f75a41f39e0 msgr2=0x7f75a4159440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:28.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:28.420+0000 7f75a359e640 1 --2- 172.21.15.112:0/3260393155 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f75a41f39e0 0x7f75a4159440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:28.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:28.420+0000 7f75a359e640 1 -- 172.21.15.112:0/3260393155 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75a40b6e20 con 0x7f75a41f70d0 2024-09-21T00:29:28.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:28.420+0000 7f75a359e640 1 --2- 172.21.15.112:0/3260393155 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75a41f70d0 0x7f75a41f74e0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f75a40cff10 tx=0x7f759c006430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:28.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:28.420+0000 7f759bfff640 1 -- 172.21.15.112:0/3260393155 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f759c011840 con 0x7f75a41f70d0 2024-09-21T00:29:28.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:28.420+0000 7f759bfff640 1 -- 172.21.15.112:0/3260393155 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f759c018070 con 0x7f75a41f70d0 2024-09-21T00:29:28.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:28.420+0000 7f75a90b2640 1 -- 172.21.15.112:0/3260393155 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75a41f4130 con 0x7f75a41f70d0 2024-09-21T00:29:28.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:28.420+0000 7f759bfff640 1 -- 172.21.15.112:0/3260393155 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f759c011c80 con 0x7f75a41f70d0 2024-09-21T00:29:28.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:28.420+0000 7f75a90b2640 1 -- 172.21.15.112:0/3260393155 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75a41f46f0 con 0x7f75a41f70d0 2024-09-21T00:29:28.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:28.421+0000 7f75a90b2640 1 -- 172.21.15.112:0/3260393155 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75a41f3ea0 con 0x7f75a41f70d0 2024-09-21T00:29:28.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:28.422+0000 7f759bfff640 1 -- 172.21.15.112:0/3260393155 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f759c01d460 con 0x7f75a41f70d0 2024-09-21T00:29:28.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:28.422+0000 7f759bfff640 1 --2- 172.21.15.112:0/3260393155 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f7584078b90 0x7f758407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:28.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:28.422+0000 7f759bfff640 1 -- 172.21.15.112:0/3260393155 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6188+0+0 (secure 0 0 0) 0x7f759c09b6e0 con 0x7f75a41f70d0 2024-09-21T00:29:28.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:28.422+0000 7f75a2d9d640 1 --2- 172.21.15.112:0/3260393155 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f7584078b90 0x7f758407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:28.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:28.423+0000 7f75a2d9d640 1 --2- 172.21.15.112:0/3260393155 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f7584078b90 0x7f758407b0a0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f7594033000 tx=0x7f7594038040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:28.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:28.427+0000 7f759bfff640 1 -- 172.21.15.112:0/3260393155 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f759c011e20 con 0x7f75a41f70d0 2024-09-21T00:29:28.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:28.600+0000 7f75a90b2640 1 -- 172.21.15.112:0/3260393155 --> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f75a40b6ff0 con 0x7f7584078b90 2024-09-21T00:29:28.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:28.606+0000 7f759bfff640 1 -- 172.21.15.112:0/3260393155 <== mgr.4103 v2:172.21.15.187:6800/2921522906 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251986 (secure 0 0 0) 0x7f75a40b6ff0 con 0x7f7584078b90 2024-09-21T00:29:28.609 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:29:28.611 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2024-09-21T00:29:28.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:28.608+0000 7f75a90b2640 1 -- 172.21.15.112:0/3260393155 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f7584078b90 msgr2=0x7f758407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:28.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:28.608+0000 7f75a90b2640 1 --2- 172.21.15.112:0/3260393155 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f7584078b90 0x7f758407b0a0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f7594033000 tx=0x7f7594038040 comp rx=0 tx=0).stop 2024-09-21T00:29:28.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:28.608+0000 7f75a90b2640 1 -- 172.21.15.112:0/3260393155 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75a41f70d0 msgr2=0x7f75a41f74e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:28.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:28.608+0000 7f75a90b2640 1 --2- 172.21.15.112:0/3260393155 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75a41f70d0 0x7f75a41f74e0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f75a40cff10 tx=0x7f759c006430 comp rx=0 tx=0).stop 2024-09-21T00:29:28.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:28.608+0000 7f75a90b2640 1 -- 172.21.15.112:0/3260393155 shutdown_connections 2024-09-21T00:29:28.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:28.608+0000 7f75a90b2640 1 --2- 172.21.15.112:0/3260393155 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f7584078b90 0x7f758407b0a0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:28.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:28.608+0000 7f75a90b2640 1 --2- 172.21.15.112:0/3260393155 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75a41f70d0 0x7f75a41f74e0 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:28.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:28.608+0000 7f75a90b2640 1 --2- 172.21.15.112:0/3260393155 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f75a41f39e0 0x7f75a4159440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:28.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:28.608+0000 7f75a90b2640 1 --2- 172.21.15.112:0/3260393155 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7590003b10 0x7f75a4158f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:28.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:28.608+0000 7f75a90b2640 1 -- 172.21.15.112:0/3260393155 >> 172.21.15.112:0/3260393155 conn(0x7f75a40b43f0 msgr2=0x7f75a415beb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:28.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:28.608+0000 7f75a90b2640 1 -- 172.21.15.112:0/3260393155 shutdown_connections 2024-09-21T00:29:28.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:28.608+0000 7f75a90b2640 1 -- 172.21.15.112:0/3260393155 wait complete. 2024-09-21T00:29:28.621 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":205,"stamp":"2024-09-21T00:29:27.842342+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":181,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":259800,"kb_used_data":10280,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754714920,"statfs":{"total":773094113280,"available":772828078080,"internally_reserved":0,"allocated":10526720,"data_stored":5390724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.050438"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.807159+0000","last_change":"2024-09-21T00:29:19.813350+0000","last_active":"2024-09-21T00:29:20.807159+0000","last_peered":"2024-09-21T00:29:20.807159+0000","last_clean":"2024-09-21T00:29:20.807159+0000","last_became_active":"2024-09-21T00:29:19.812123+0000","last_became_peered":"2024-09-21T00:29:19.812123+0000","last_unstale":"2024-09-21T00:29:20.807159+0000","last_undegraded":"2024-09-21T00:29:20.807159+0000","last_fullsized":"2024-09-21T00:29:20.807159+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:17:13.647509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022537+0000","last_change":"2024-09-21T00:29:19.807870+0000","last_active":"2024-09-21T00:29:21.022537+0000","last_peered":"2024-09-21T00:29:21.022537+0000","last_clean":"2024-09-21T00:29:21.022537+0000","last_became_active":"2024-09-21T00:29:19.807563+0000","last_became_peered":"2024-09-21T00:29:19.807563+0000","last_unstale":"2024-09-21T00:29:21.022537+0000","last_undegraded":"2024-09-21T00:29:21.022537+0000","last_fullsized":"2024-09-21T00:29:21.022537+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:26:29.645186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.807064+0000","last_change":"2024-09-21T00:29:19.804321+0000","last_active":"2024-09-21T00:29:20.807064+0000","last_peered":"2024-09-21T00:29:20.807064+0000","last_clean":"2024-09-21T00:29:20.807064+0000","last_became_active":"2024-09-21T00:29:19.804013+0000","last_became_peered":"2024-09-21T00:29:19.804013+0000","last_unstale":"2024-09-21T00:29:20.807064+0000","last_undegraded":"2024-09-21T00:29:20.807064+0000","last_fullsized":"2024-09-21T00:29:20.807064+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:47:15.426188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.021814+0000","last_change":"2024-09-21T00:29:19.811466+0000","last_active":"2024-09-21T00:29:21.021814+0000","last_peered":"2024-09-21T00:29:21.021814+0000","last_clean":"2024-09-21T00:29:21.021814+0000","last_became_active":"2024-09-21T00:29:19.811130+0000","last_became_peered":"2024-09-21T00:29:19.811130+0000","last_unstale":"2024-09-21T00:29:21.021814+0000","last_undegraded":"2024-09-21T00:29:21.021814+0000","last_fullsized":"2024-09-21T00:29:21.021814+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:44:42.287171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022147+0000","last_change":"2024-09-21T00:29:19.801339+0000","last_active":"2024-09-21T00:29:21.022147+0000","last_peered":"2024-09-21T00:29:21.022147+0000","last_clean":"2024-09-21T00:29:21.022147+0000","last_became_active":"2024-09-21T00:29:19.800996+0000","last_became_peered":"2024-09-21T00:29:19.800996+0000","last_unstale":"2024-09-21T00:29:21.022147+0000","last_undegraded":"2024-09-21T00:29:21.022147+0000","last_fullsized":"2024-09-21T00:29:21.022147+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:26:43.393876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022508+0000","last_change":"2024-09-21T00:29:19.822580+0000","last_active":"2024-09-21T00:29:21.022508+0000","last_peered":"2024-09-21T00:29:21.022508+0000","last_clean":"2024-09-21T00:29:21.022508+0000","last_became_active":"2024-09-21T00:29:19.822239+0000","last_became_peered":"2024-09-21T00:29:19.822239+0000","last_unstale":"2024-09-21T00:29:21.022508+0000","last_undegraded":"2024-09-21T00:29:21.022508+0000","last_fullsized":"2024-09-21T00:29:21.022508+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:46:09.814298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.021489+0000","last_change":"2024-09-21T00:29:19.822649+0000","last_active":"2024-09-21T00:29:21.021489+0000","last_peered":"2024-09-21T00:29:21.021489+0000","last_clean":"2024-09-21T00:29:21.021489+0000","last_became_active":"2024-09-21T00:29:19.822194+0000","last_became_peered":"2024-09-21T00:29:19.822194+0000","last_unstale":"2024-09-21T00:29:21.021489+0000","last_undegraded":"2024-09-21T00:29:21.021489+0000","last_fullsized":"2024-09-21T00:29:21.021489+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:09:42.739510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022777+0000","last_change":"2024-09-21T00:29:19.816934+0000","last_active":"2024-09-21T00:29:21.022777+0000","last_peered":"2024-09-21T00:29:21.022777+0000","last_clean":"2024-09-21T00:29:21.022777+0000","last_became_active":"2024-09-21T00:29:19.815942+0000","last_became_peered":"2024-09-21T00:29:19.815942+0000","last_unstale":"2024-09-21T00:29:21.022777+0000","last_undegraded":"2024-09-21T00:29:21.022777+0000","last_fullsized":"2024-09-21T00:29:21.022777+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:35:25.434622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022456+0000","last_change":"2024-09-21T00:29:19.816919+0000","last_active":"2024-09-21T00:29:21.022456+0000","last_peered":"2024-09-21T00:29:21.022456+0000","last_clean":"2024-09-21T00:29:21.022456+0000","last_became_active":"2024-09-21T00:29:19.816530+0000","last_became_peered":"2024-09-21T00:29:19.816530+0000","last_unstale":"2024-09-21T00:29:21.022456+0000","last_undegraded":"2024-09-21T00:29:21.022456+0000","last_fullsized":"2024-09-21T00:29:21.022456+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:40:46.095060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.021554+0000","last_change":"2024-09-21T00:29:19.809269+0000","last_active":"2024-09-21T00:29:21.021554+0000","last_peered":"2024-09-21T00:29:21.021554+0000","last_clean":"2024-09-21T00:29:21.021554+0000","last_became_active":"2024-09-21T00:29:19.808942+0000","last_became_peered":"2024-09-21T00:29:19.808942+0000","last_unstale":"2024-09-21T00:29:21.021554+0000","last_undegraded":"2024-09-21T00:29:21.021554+0000","last_fullsized":"2024-09-21T00:29:21.021554+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:58:57.469916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.809793+0000","last_change":"2024-09-21T00:29:19.814490+0000","last_active":"2024-09-21T00:29:20.809793+0000","last_peered":"2024-09-21T00:29:20.809793+0000","last_clean":"2024-09-21T00:29:20.809793+0000","last_became_active":"2024-09-21T00:29:19.813188+0000","last_became_peered":"2024-09-21T00:29:19.813188+0000","last_unstale":"2024-09-21T00:29:20.809793+0000","last_undegraded":"2024-09-21T00:29:20.809793+0000","last_fullsized":"2024-09-21T00:29:20.809793+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:59:50.910192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.810780+0000","last_change":"2024-09-21T00:29:19.811796+0000","last_active":"2024-09-21T00:29:20.810780+0000","last_peered":"2024-09-21T00:29:20.810780+0000","last_clean":"2024-09-21T00:29:20.810780+0000","last_became_active":"2024-09-21T00:29:19.811496+0000","last_became_peered":"2024-09-21T00:29:19.811496+0000","last_unstale":"2024-09-21T00:29:20.810780+0000","last_undegraded":"2024-09-21T00:29:20.810780+0000","last_fullsized":"2024-09-21T00:29:20.810780+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:59:41.316661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022019+0000","last_change":"2024-09-21T00:29:19.818860+0000","last_active":"2024-09-21T00:29:21.022019+0000","last_peered":"2024-09-21T00:29:21.022019+0000","last_clean":"2024-09-21T00:29:21.022019+0000","last_became_active":"2024-09-21T00:29:19.818552+0000","last_became_peered":"2024-09-21T00:29:19.818552+0000","last_unstale":"2024-09-21T00:29:21.022019+0000","last_undegraded":"2024-09-21T00:29:21.022019+0000","last_fullsized":"2024-09-21T00:29:21.022019+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:37:22.821347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.021673+0000","last_change":"2024-09-21T00:29:19.820196+0000","last_active":"2024-09-21T00:29:21.021673+0000","last_peered":"2024-09-21T00:29:21.021673+0000","last_clean":"2024-09-21T00:29:21.021673+0000","last_became_active":"2024-09-21T00:29:19.819691+0000","last_became_peered":"2024-09-21T00:29:19.819691+0000","last_unstale":"2024-09-21T00:29:21.021673+0000","last_undegraded":"2024-09-21T00:29:21.021673+0000","last_fullsized":"2024-09-21T00:29:21.021673+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:48:11.528107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.021642+0000","last_change":"2024-09-21T00:29:19.810601+0000","last_active":"2024-09-21T00:29:21.021642+0000","last_peered":"2024-09-21T00:29:21.021642+0000","last_clean":"2024-09-21T00:29:21.021642+0000","last_became_active":"2024-09-21T00:29:19.810272+0000","last_became_peered":"2024-09-21T00:29:19.810272+0000","last_unstale":"2024-09-21T00:29:21.021642+0000","last_undegraded":"2024-09-21T00:29:21.021642+0000","last_fullsized":"2024-09-21T00:29:21.021642+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:28:10.979181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.021511+0000","last_change":"2024-09-21T00:29:19.818725+0000","last_active":"2024-09-21T00:29:21.021511+0000","last_peered":"2024-09-21T00:29:21.021511+0000","last_clean":"2024-09-21T00:29:21.021511+0000","last_became_active":"2024-09-21T00:29:19.818406+0000","last_became_peered":"2024-09-21T00:29:19.818406+0000","last_unstale":"2024-09-21T00:29:21.021511+0000","last_undegraded":"2024-09-21T00:29:21.021511+0000","last_fullsized":"2024-09-21T00:29:21.021511+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:13:53.211350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.807387+0000","last_change":"2024-09-21T00:29:19.803557+0000","last_active":"2024-09-21T00:29:20.807387+0000","last_peered":"2024-09-21T00:29:20.807387+0000","last_clean":"2024-09-21T00:29:20.807387+0000","last_became_active":"2024-09-21T00:29:19.802868+0000","last_became_peered":"2024-09-21T00:29:19.802868+0000","last_unstale":"2024-09-21T00:29:20.807387+0000","last_undegraded":"2024-09-21T00:29:20.807387+0000","last_fullsized":"2024-09-21T00:29:20.807387+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:17:38.099624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022370+0000","last_change":"2024-09-21T00:29:19.811473+0000","last_active":"2024-09-21T00:29:21.022370+0000","last_peered":"2024-09-21T00:29:21.022370+0000","last_clean":"2024-09-21T00:29:21.022370+0000","last_became_active":"2024-09-21T00:29:19.811129+0000","last_became_peered":"2024-09-21T00:29:19.811129+0000","last_unstale":"2024-09-21T00:29:21.022370+0000","last_undegraded":"2024-09-21T00:29:21.022370+0000","last_fullsized":"2024-09-21T00:29:21.022370+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:26.631458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.807233+0000","last_change":"2024-09-21T00:29:19.809929+0000","last_active":"2024-09-21T00:29:20.807233+0000","last_peered":"2024-09-21T00:29:20.807233+0000","last_clean":"2024-09-21T00:29:20.807233+0000","last_became_active":"2024-09-21T00:29:19.809541+0000","last_became_peered":"2024-09-21T00:29:19.809541+0000","last_unstale":"2024-09-21T00:29:20.807233+0000","last_undegraded":"2024-09-21T00:29:20.807233+0000","last_fullsized":"2024-09-21T00:29:20.807233+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:34:11.115991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.810853+0000","last_change":"2024-09-21T00:29:19.808754+0000","last_active":"2024-09-21T00:29:20.810853+0000","last_peered":"2024-09-21T00:29:20.810853+0000","last_clean":"2024-09-21T00:29:20.810853+0000","last_became_active":"2024-09-21T00:29:19.807857+0000","last_became_peered":"2024-09-21T00:29:19.807857+0000","last_unstale":"2024-09-21T00:29:20.810853+0000","last_undegraded":"2024-09-21T00:29:20.810853+0000","last_fullsized":"2024-09-21T00:29:20.810853+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:53:45.848291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.023011+0000","last_change":"2024-09-21T00:29:19.815917+0000","last_active":"2024-09-21T00:29:21.023011+0000","last_peered":"2024-09-21T00:29:21.023011+0000","last_clean":"2024-09-21T00:29:21.023011+0000","last_became_active":"2024-09-21T00:29:19.815559+0000","last_became_peered":"2024-09-21T00:29:19.815559+0000","last_unstale":"2024-09-21T00:29:21.023011+0000","last_undegraded":"2024-09-21T00:29:21.023011+0000","last_fullsized":"2024-09-21T00:29:21.023011+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:47:06.729633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.021820+0000","last_change":"2024-09-21T00:29:19.803150+0000","last_active":"2024-09-21T00:29:21.021820+0000","last_peered":"2024-09-21T00:29:21.021820+0000","last_clean":"2024-09-21T00:29:21.021820+0000","last_became_active":"2024-09-21T00:29:19.802841+0000","last_became_peered":"2024-09-21T00:29:19.802841+0000","last_unstale":"2024-09-21T00:29:21.021820+0000","last_undegraded":"2024-09-21T00:29:21.021820+0000","last_fullsized":"2024-09-21T00:29:21.021820+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:31:42.414674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.807054+0000","last_change":"2024-09-21T00:29:19.802421+0000","last_active":"2024-09-21T00:29:20.807054+0000","last_peered":"2024-09-21T00:29:20.807054+0000","last_clean":"2024-09-21T00:29:20.807054+0000","last_became_active":"2024-09-21T00:29:19.802074+0000","last_became_peered":"2024-09-21T00:29:19.802074+0000","last_unstale":"2024-09-21T00:29:20.807054+0000","last_undegraded":"2024-09-21T00:29:20.807054+0000","last_fullsized":"2024-09-21T00:29:20.807054+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:55:22.327108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.021168+0000","last_change":"2024-09-21T00:29:19.806258+0000","last_active":"2024-09-21T00:29:21.021168+0000","last_peered":"2024-09-21T00:29:21.021168+0000","last_clean":"2024-09-21T00:29:21.021168+0000","last_became_active":"2024-09-21T00:29:19.805930+0000","last_became_peered":"2024-09-21T00:29:19.805930+0000","last_unstale":"2024-09-21T00:29:21.021168+0000","last_undegraded":"2024-09-21T00:29:21.021168+0000","last_fullsized":"2024-09-21T00:29:21.021168+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:46:50.551185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.810198+0000","last_change":"2024-09-21T00:29:19.802920+0000","last_active":"2024-09-21T00:29:20.810198+0000","last_peered":"2024-09-21T00:29:20.810198+0000","last_clean":"2024-09-21T00:29:20.810198+0000","last_became_active":"2024-09-21T00:29:19.802584+0000","last_became_peered":"2024-09-21T00:29:19.802584+0000","last_unstale":"2024-09-21T00:29:20.810198+0000","last_undegraded":"2024-09-21T00:29:20.810198+0000","last_fullsized":"2024-09-21T00:29:20.810198+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:38:27.387471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.810402+0000","last_change":"2024-09-21T00:29:19.808064+0000","last_active":"2024-09-21T00:29:20.810402+0000","last_peered":"2024-09-21T00:29:20.810402+0000","last_clean":"2024-09-21T00:29:20.810402+0000","last_became_active":"2024-09-21T00:29:19.807712+0000","last_became_peered":"2024-09-21T00:29:19.807712+0000","last_unstale":"2024-09-21T00:29:20.810402+0000","last_undegraded":"2024-09-21T00:29:20.810402+0000","last_fullsized":"2024-09-21T00:29:20.810402+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:45:54.930710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.810053+0000","last_change":"2024-09-21T00:29:19.811883+0000","last_active":"2024-09-21T00:29:20.810053+0000","last_peered":"2024-09-21T00:29:20.810053+0000","last_clean":"2024-09-21T00:29:20.810053+0000","last_became_active":"2024-09-21T00:29:19.811584+0000","last_became_peered":"2024-09-21T00:29:19.811584+0000","last_unstale":"2024-09-21T00:29:20.810053+0000","last_undegraded":"2024-09-21T00:29:20.810053+0000","last_fullsized":"2024-09-21T00:29:20.810053+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:00:59.593942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022016+0000","last_change":"2024-09-21T00:29:19.816999+0000","last_active":"2024-09-21T00:29:21.022016+0000","last_peered":"2024-09-21T00:29:21.022016+0000","last_clean":"2024-09-21T00:29:21.022016+0000","last_became_active":"2024-09-21T00:29:19.814905+0000","last_became_peered":"2024-09-21T00:29:19.814905+0000","last_unstale":"2024-09-21T00:29:21.022016+0000","last_undegraded":"2024-09-21T00:29:21.022016+0000","last_fullsized":"2024-09-21T00:29:21.022016+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:46:57.412342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.023130+0000","last_change":"2024-09-21T00:29:19.815104+0000","last_active":"2024-09-21T00:29:21.023130+0000","last_peered":"2024-09-21T00:29:21.023130+0000","last_clean":"2024-09-21T00:29:21.023130+0000","last_became_active":"2024-09-21T00:29:19.814806+0000","last_became_peered":"2024-09-21T00:29:19.814806+0000","last_unstale":"2024-09-21T00:29:21.023130+0000","last_undegraded":"2024-09-21T00:29:21.023130+0000","last_fullsized":"2024-09-21T00:29:21.023130+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:14:39.263553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.810509+0000","last_change":"2024-09-21T00:29:19.802931+0000","last_active":"2024-09-21T00:29:20.810509+0000","last_peered":"2024-09-21T00:29:20.810509+0000","last_clean":"2024-09-21T00:29:20.810509+0000","last_became_active":"2024-09-21T00:29:19.802606+0000","last_became_peered":"2024-09-21T00:29:19.802606+0000","last_unstale":"2024-09-21T00:29:20.810509+0000","last_undegraded":"2024-09-21T00:29:20.810509+0000","last_fullsized":"2024-09-21T00:29:20.810509+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:56:07.296305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022325+0000","last_change":"2024-09-21T00:29:19.812717+0000","last_active":"2024-09-21T00:29:21.022325+0000","last_peered":"2024-09-21T00:29:21.022325+0000","last_clean":"2024-09-21T00:29:21.022325+0000","last_became_active":"2024-09-21T00:29:19.812402+0000","last_became_peered":"2024-09-21T00:29:19.812402+0000","last_unstale":"2024-09-21T00:29:21.022325+0000","last_undegraded":"2024-09-21T00:29:21.022325+0000","last_fullsized":"2024-09-21T00:29:21.022325+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:26:39.938614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022839+0000","last_change":"2024-09-21T00:29:19.816588+0000","last_active":"2024-09-21T00:29:21.022839+0000","last_peered":"2024-09-21T00:29:21.022839+0000","last_clean":"2024-09-21T00:29:21.022839+0000","last_became_active":"2024-09-21T00:29:19.815574+0000","last_became_peered":"2024-09-21T00:29:19.815574+0000","last_unstale":"2024-09-21T00:29:21.022839+0000","last_undegraded":"2024-09-21T00:29:21.022839+0000","last_fullsized":"2024-09-21T00:29:21.022839+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:47:08.243928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.807629+0000","last_change":"2024-09-21T00:29:19.810747+0000","last_active":"2024-09-21T00:29:20.807629+0000","last_peered":"2024-09-21T00:29:20.807629+0000","last_clean":"2024-09-21T00:29:20.807629+0000","last_became_active":"2024-09-21T00:29:19.810429+0000","last_became_peered":"2024-09-21T00:29:19.810429+0000","last_unstale":"2024-09-21T00:29:20.807629+0000","last_undegraded":"2024-09-21T00:29:20.807629+0000","last_fullsized":"2024-09-21T00:29:20.807629+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:21:05.578580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"93'4","reported_seq":19,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022399+0000","last_change":"2024-09-21T00:29:18.798208+0000","last_active":"2024-09-21T00:29:21.022399+0000","last_peered":"2024-09-21T00:29:21.022399+0000","last_clean":"2024-09-21T00:29:21.022399+0000","last_became_active":"2024-09-21T00:29:18.797857+0000","last_became_peered":"2024-09-21T00:29:18.797857+0000","last_unstale":"2024-09-21T00:29:21.022399+0000","last_undegraded":"2024-09-21T00:29:21.022399+0000","last_fullsized":"2024-09-21T00:29:21.022399+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T00:37:31.452087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.021779+0000","last_change":"2024-09-21T00:29:19.822277+0000","last_active":"2024-09-21T00:29:21.021779+0000","last_peered":"2024-09-21T00:29:21.021779+0000","last_clean":"2024-09-21T00:29:21.021779+0000","last_became_active":"2024-09-21T00:29:19.821917+0000","last_became_peered":"2024-09-21T00:29:19.821917+0000","last_unstale":"2024-09-21T00:29:21.021779+0000","last_undegraded":"2024-09-21T00:29:21.021779+0000","last_fullsized":"2024-09-21T00:29:21.021779+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:04:07.143491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"93'1","reported_seq":16,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022961+0000","last_change":"2024-09-21T00:29:18.799882+0000","last_active":"2024-09-21T00:29:21.022961+0000","last_peered":"2024-09-21T00:29:21.022961+0000","last_clean":"2024-09-21T00:29:21.022961+0000","last_became_active":"2024-09-21T00:29:18.799524+0000","last_became_peered":"2024-09-21T00:29:18.799524+0000","last_unstale":"2024-09-21T00:29:21.022961+0000","last_undegraded":"2024-09-21T00:29:21.022961+0000","last_fullsized":"2024-09-21T00:29:21.022961+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:39:17.740976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.810993+0000","last_change":"2024-09-21T00:29:19.809700+0000","last_active":"2024-09-21T00:29:20.810993+0000","last_peered":"2024-09-21T00:29:20.810993+0000","last_clean":"2024-09-21T00:29:20.810993+0000","last_became_active":"2024-09-21T00:29:19.809343+0000","last_became_peered":"2024-09-21T00:29:19.809343+0000","last_unstale":"2024-09-21T00:29:20.810993+0000","last_undegraded":"2024-09-21T00:29:20.810993+0000","last_fullsized":"2024-09-21T00:29:20.810993+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:28:50.502430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"93'2","reported_seq":17,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.810949+0000","last_change":"2024-09-21T00:29:18.797271+0000","last_active":"2024-09-21T00:29:20.810949+0000","last_peered":"2024-09-21T00:29:20.810949+0000","last_clean":"2024-09-21T00:29:20.810949+0000","last_became_active":"2024-09-21T00:29:18.796656+0000","last_became_peered":"2024-09-21T00:29:18.796656+0000","last_unstale":"2024-09-21T00:29:20.810949+0000","last_undegraded":"2024-09-21T00:29:20.810949+0000","last_fullsized":"2024-09-21T00:29:20.810949+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:28:00.490269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.810191+0000","last_change":"2024-09-21T00:29:19.802323+0000","last_active":"2024-09-21T00:29:20.810191+0000","last_peered":"2024-09-21T00:29:20.810191+0000","last_clean":"2024-09-21T00:29:20.810191+0000","last_became_active":"2024-09-21T00:29:19.801998+0000","last_became_peered":"2024-09-21T00:29:19.801998+0000","last_unstale":"2024-09-21T00:29:20.810191+0000","last_undegraded":"2024-09-21T00:29:20.810191+0000","last_fullsized":"2024-09-21T00:29:20.810191+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:31:43.191165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"93'2","reported_seq":17,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.021788+0000","last_change":"2024-09-21T00:29:18.798679+0000","last_active":"2024-09-21T00:29:21.021788+0000","last_peered":"2024-09-21T00:29:21.021788+0000","last_clean":"2024-09-21T00:29:21.021788+0000","last_became_active":"2024-09-21T00:29:18.798341+0000","last_became_peered":"2024-09-21T00:29:18.798341+0000","last_unstale":"2024-09-21T00:29:21.021788+0000","last_undegraded":"2024-09-21T00:29:21.021788+0000","last_fullsized":"2024-09-21T00:29:21.021788+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:29:04.021021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022055+0000","last_change":"2024-09-21T00:29:19.821679+0000","last_active":"2024-09-21T00:29:21.022055+0000","last_peered":"2024-09-21T00:29:21.022055+0000","last_clean":"2024-09-21T00:29:21.022055+0000","last_became_active":"2024-09-21T00:29:19.821331+0000","last_became_peered":"2024-09-21T00:29:19.821331+0000","last_unstale":"2024-09-21T00:29:21.022055+0000","last_undegraded":"2024-09-21T00:29:21.022055+0000","last_fullsized":"2024-09-21T00:29:21.022055+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:13:53.211350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"93'1","reported_seq":16,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.021871+0000","last_change":"2024-09-21T00:29:18.797442+0000","last_active":"2024-09-21T00:29:21.021871+0000","last_peered":"2024-09-21T00:29:21.021871+0000","last_clean":"2024-09-21T00:29:21.021871+0000","last_became_active":"2024-09-21T00:29:18.797128+0000","last_became_peered":"2024-09-21T00:29:18.797128+0000","last_unstale":"2024-09-21T00:29:21.021871+0000","last_undegraded":"2024-09-21T00:29:21.021871+0000","last_fullsized":"2024-09-21T00:29:21.021871+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:22:52.216862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.807399+0000","last_change":"2024-09-21T00:29:19.808878+0000","last_active":"2024-09-21T00:29:20.807399+0000","last_peered":"2024-09-21T00:29:20.807399+0000","last_clean":"2024-09-21T00:29:20.807399+0000","last_became_active":"2024-09-21T00:29:19.808577+0000","last_became_peered":"2024-09-21T00:29:19.808577+0000","last_unstale":"2024-09-21T00:29:20.807399+0000","last_undegraded":"2024-09-21T00:29:20.807399+0000","last_fullsized":"2024-09-21T00:29:20.807399+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:34:29.883537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"93'1","reported_seq":16,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.810090+0000","last_change":"2024-09-21T00:29:18.801027+0000","last_active":"2024-09-21T00:29:20.810090+0000","last_peered":"2024-09-21T00:29:20.810090+0000","last_clean":"2024-09-21T00:29:20.810090+0000","last_became_active":"2024-09-21T00:29:18.800693+0000","last_became_peered":"2024-09-21T00:29:18.800693+0000","last_unstale":"2024-09-21T00:29:20.810090+0000","last_undegraded":"2024-09-21T00:29:20.810090+0000","last_fullsized":"2024-09-21T00:29:20.810090+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:17:06.642779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022624+0000","last_change":"2024-09-21T00:29:19.815242+0000","last_active":"2024-09-21T00:29:21.022624+0000","last_peered":"2024-09-21T00:29:21.022624+0000","last_clean":"2024-09-21T00:29:21.022624+0000","last_became_active":"2024-09-21T00:29:19.814858+0000","last_became_peered":"2024-09-21T00:29:19.814858+0000","last_unstale":"2024-09-21T00:29:21.022624+0000","last_undegraded":"2024-09-21T00:29:21.022624+0000","last_fullsized":"2024-09-21T00:29:21.022624+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:35:06.851240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.809872+0000","last_change":"2024-09-21T00:29:18.797155+0000","last_active":"2024-09-21T00:29:20.809872+0000","last_peered":"2024-09-21T00:29:20.809872+0000","last_clean":"2024-09-21T00:29:20.809872+0000","last_became_active":"2024-09-21T00:29:18.795953+0000","last_became_peered":"2024-09-21T00:29:18.795953+0000","last_unstale":"2024-09-21T00:29:20.809872+0000","last_undegraded":"2024-09-21T00:29:20.809872+0000","last_fullsized":"2024-09-21T00:29:20.809872+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T00:54:16.170396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022428+0000","last_change":"2024-09-21T00:29:19.817180+0000","last_active":"2024-09-21T00:29:21.022428+0000","last_peered":"2024-09-21T00:29:21.022428+0000","last_clean":"2024-09-21T00:29:21.022428+0000","last_became_active":"2024-09-21T00:29:19.816859+0000","last_became_peered":"2024-09-21T00:29:19.816859+0000","last_unstale":"2024-09-21T00:29:21.022428+0000","last_undegraded":"2024-09-21T00:29:21.022428+0000","last_fullsized":"2024-09-21T00:29:21.022428+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:30:01.904485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.021605+0000","last_change":"2024-09-21T00:29:18.793095+0000","last_active":"2024-09-21T00:29:21.021605+0000","last_peered":"2024-09-21T00:29:21.021605+0000","last_clean":"2024-09-21T00:29:21.021605+0000","last_became_active":"2024-09-21T00:29:18.792693+0000","last_became_peered":"2024-09-21T00:29:18.792693+0000","last_unstale":"2024-09-21T00:29:21.021605+0000","last_undegraded":"2024-09-21T00:29:21.021605+0000","last_fullsized":"2024-09-21T00:29:21.021605+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:56:18.147899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.807542+0000","last_change":"2024-09-21T00:29:19.815545+0000","last_active":"2024-09-21T00:29:20.807542+0000","last_peered":"2024-09-21T00:29:20.807542+0000","last_clean":"2024-09-21T00:29:20.807542+0000","last_became_active":"2024-09-21T00:29:19.811424+0000","last_became_peered":"2024-09-21T00:29:19.811424+0000","last_unstale":"2024-09-21T00:29:20.807542+0000","last_undegraded":"2024-09-21T00:29:20.807542+0000","last_fullsized":"2024-09-21T00:29:20.807542+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:20:57.334693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.810327+0000","last_change":"2024-09-21T00:29:18.796965+0000","last_active":"2024-09-21T00:29:20.810327+0000","last_peered":"2024-09-21T00:29:20.810327+0000","last_clean":"2024-09-21T00:29:20.810327+0000","last_became_active":"2024-09-21T00:29:18.796548+0000","last_became_peered":"2024-09-21T00:29:18.796548+0000","last_unstale":"2024-09-21T00:29:20.810327+0000","last_undegraded":"2024-09-21T00:29:20.810327+0000","last_fullsized":"2024-09-21T00:29:20.810327+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:00:18.177473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.021362+0000","last_change":"2024-09-21T00:29:19.814792+0000","last_active":"2024-09-21T00:29:21.021362+0000","last_peered":"2024-09-21T00:29:21.021362+0000","last_clean":"2024-09-21T00:29:21.021362+0000","last_became_active":"2024-09-21T00:29:19.814147+0000","last_became_peered":"2024-09-21T00:29:19.814147+0000","last_unstale":"2024-09-21T00:29:21.021362+0000","last_undegraded":"2024-09-21T00:29:21.021362+0000","last_fullsized":"2024-09-21T00:29:21.021362+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:01:27.998993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.807185+0000","last_change":"2024-09-21T00:29:18.798831+0000","last_active":"2024-09-21T00:29:20.807185+0000","last_peered":"2024-09-21T00:29:20.807185+0000","last_clean":"2024-09-21T00:29:20.807185+0000","last_became_active":"2024-09-21T00:29:18.798551+0000","last_became_peered":"2024-09-21T00:29:18.798551+0000","last_unstale":"2024-09-21T00:29:20.807185+0000","last_undegraded":"2024-09-21T00:29:20.807185+0000","last_fullsized":"2024-09-21T00:29:20.807185+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:12:30.180725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022749+0000","last_change":"2024-09-21T00:29:19.812082+0000","last_active":"2024-09-21T00:29:21.022749+0000","last_peered":"2024-09-21T00:29:21.022749+0000","last_clean":"2024-09-21T00:29:21.022749+0000","last_became_active":"2024-09-21T00:29:19.811773+0000","last_became_peered":"2024-09-21T00:29:19.811773+0000","last_unstale":"2024-09-21T00:29:21.022749+0000","last_undegraded":"2024-09-21T00:29:21.022749+0000","last_fullsized":"2024-09-21T00:29:21.022749+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:17:43.069499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"93'3","reported_seq":18,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.021859+0000","last_change":"2024-09-21T00:29:18.794178+0000","last_active":"2024-09-21T00:29:21.021859+0000","last_peered":"2024-09-21T00:29:21.021859+0000","last_clean":"2024-09-21T00:29:21.021859+0000","last_became_active":"2024-09-21T00:29:18.793851+0000","last_became_peered":"2024-09-21T00:29:18.793851+0000","last_unstale":"2024-09-21T00:29:21.021859+0000","last_undegraded":"2024-09-21T00:29:21.021859+0000","last_fullsized":"2024-09-21T00:29:21.021859+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:52:27.563468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022394+0000","last_change":"2024-09-21T00:29:19.814438+0000","last_active":"2024-09-21T00:29:21.022394+0000","last_peered":"2024-09-21T00:29:21.022394+0000","last_clean":"2024-09-21T00:29:21.022394+0000","last_became_active":"2024-09-21T00:29:19.814085+0000","last_became_peered":"2024-09-21T00:29:19.814085+0000","last_unstale":"2024-09-21T00:29:21.022394+0000","last_undegraded":"2024-09-21T00:29:21.022394+0000","last_fullsized":"2024-09-21T00:29:21.022394+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:08:38.851080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"93'6","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.021896+0000","last_change":"2024-09-21T00:29:18.800236+0000","last_active":"2024-09-21T00:29:21.021896+0000","last_peered":"2024-09-21T00:29:21.021896+0000","last_clean":"2024-09-21T00:29:21.021896+0000","last_became_active":"2024-09-21T00:29:18.799899+0000","last_became_peered":"2024-09-21T00:29:18.799899+0000","last_unstale":"2024-09-21T00:29:21.021896+0000","last_undegraded":"2024-09-21T00:29:21.021896+0000","last_fullsized":"2024-09-21T00:29:21.021896+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:29:10.908555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.809551+0000","last_change":"2024-09-21T00:29:19.811833+0000","last_active":"2024-09-21T00:29:20.809551+0000","last_peered":"2024-09-21T00:29:20.809551+0000","last_clean":"2024-09-21T00:29:20.809551+0000","last_became_active":"2024-09-21T00:29:19.811519+0000","last_became_peered":"2024-09-21T00:29:19.811519+0000","last_unstale":"2024-09-21T00:29:20.809551+0000","last_undegraded":"2024-09-21T00:29:20.809551+0000","last_fullsized":"2024-09-21T00:29:20.809551+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:52:28.568849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"93'2","reported_seq":17,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022457+0000","last_change":"2024-09-21T00:29:18.801761+0000","last_active":"2024-09-21T00:29:21.022457+0000","last_peered":"2024-09-21T00:29:21.022457+0000","last_clean":"2024-09-21T00:29:21.022457+0000","last_became_active":"2024-09-21T00:29:18.801422+0000","last_became_peered":"2024-09-21T00:29:18.801422+0000","last_unstale":"2024-09-21T00:29:21.022457+0000","last_undegraded":"2024-09-21T00:29:21.022457+0000","last_fullsized":"2024-09-21T00:29:21.022457+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:55:45.769871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022489+0000","last_change":"2024-09-21T00:29:19.814412+0000","last_active":"2024-09-21T00:29:21.022489+0000","last_peered":"2024-09-21T00:29:21.022489+0000","last_clean":"2024-09-21T00:29:21.022489+0000","last_became_active":"2024-09-21T00:29:19.813923+0000","last_became_peered":"2024-09-21T00:29:19.813923+0000","last_unstale":"2024-09-21T00:29:21.022489+0000","last_undegraded":"2024-09-21T00:29:21.022489+0000","last_fullsized":"2024-09-21T00:29:21.022489+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:42:28.398366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022221+0000","last_change":"2024-09-21T00:29:18.788248+0000","last_active":"2024-09-21T00:29:21.022221+0000","last_peered":"2024-09-21T00:29:21.022221+0000","last_clean":"2024-09-21T00:29:21.022221+0000","last_became_active":"2024-09-21T00:29:18.787904+0000","last_became_peered":"2024-09-21T00:29:18.787904+0000","last_unstale":"2024-09-21T00:29:21.022221+0000","last_undegraded":"2024-09-21T00:29:21.022221+0000","last_fullsized":"2024-09-21T00:29:21.022221+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:21:04.691629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.023078+0000","last_change":"2024-09-21T00:29:19.814341+0000","last_active":"2024-09-21T00:29:21.023078+0000","last_peered":"2024-09-21T00:29:21.023078+0000","last_clean":"2024-09-21T00:29:21.023078+0000","last_became_active":"2024-09-21T00:29:19.814052+0000","last_became_peered":"2024-09-21T00:29:19.814052+0000","last_unstale":"2024-09-21T00:29:21.023078+0000","last_undegraded":"2024-09-21T00:29:21.023078+0000","last_fullsized":"2024-09-21T00:29:21.023078+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:58:18.707835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.021901+0000","last_change":"2024-09-21T00:29:18.800084+0000","last_active":"2024-09-21T00:29:21.021901+0000","last_peered":"2024-09-21T00:29:21.021901+0000","last_clean":"2024-09-21T00:29:21.021901+0000","last_became_active":"2024-09-21T00:29:18.799797+0000","last_became_peered":"2024-09-21T00:29:18.799797+0000","last_unstale":"2024-09-21T00:29:21.021901+0000","last_undegraded":"2024-09-21T00:29:21.021901+0000","last_fullsized":"2024-09-21T00:29:21.021901+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T00:51:35.304058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.807595+0000","last_change":"2024-09-21T00:29:19.804683+0000","last_active":"2024-09-21T00:29:20.807595+0000","last_peered":"2024-09-21T00:29:20.807595+0000","last_clean":"2024-09-21T00:29:20.807595+0000","last_became_active":"2024-09-21T00:29:19.803246+0000","last_became_peered":"2024-09-21T00:29:19.803246+0000","last_unstale":"2024-09-21T00:29:20.807595+0000","last_undegraded":"2024-09-21T00:29:20.807595+0000","last_fullsized":"2024-09-21T00:29:20.807595+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:46:26.637135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"93'8","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.810678+0000","last_change":"2024-09-21T00:29:18.796567+0000","last_active":"2024-09-21T00:29:20.810678+0000","last_peered":"2024-09-21T00:29:20.810678+0000","last_clean":"2024-09-21T00:29:20.810678+0000","last_became_active":"2024-09-21T00:29:18.796255+0000","last_became_peered":"2024-09-21T00:29:18.796255+0000","last_unstale":"2024-09-21T00:29:20.810678+0000","last_undegraded":"2024-09-21T00:29:20.810678+0000","last_fullsized":"2024-09-21T00:29:20.810678+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:04:53.438244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.023031+0000","last_change":"2024-09-21T00:29:19.817234+0000","last_active":"2024-09-21T00:29:21.023031+0000","last_peered":"2024-09-21T00:29:21.023031+0000","last_clean":"2024-09-21T00:29:21.023031+0000","last_became_active":"2024-09-21T00:29:19.816273+0000","last_became_peered":"2024-09-21T00:29:19.816273+0000","last_unstale":"2024-09-21T00:29:21.023031+0000","last_undegraded":"2024-09-21T00:29:21.023031+0000","last_fullsized":"2024-09-21T00:29:21.023031+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:45:02.009239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"93'1","reported_seq":16,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022386+0000","last_change":"2024-09-21T00:29:18.799579+0000","last_active":"2024-09-21T00:29:21.022386+0000","last_peered":"2024-09-21T00:29:21.022386+0000","last_clean":"2024-09-21T00:29:21.022386+0000","last_became_active":"2024-09-21T00:29:18.799238+0000","last_became_peered":"2024-09-21T00:29:18.799238+0000","last_unstale":"2024-09-21T00:29:21.022386+0000","last_undegraded":"2024-09-21T00:29:21.022386+0000","last_fullsized":"2024-09-21T00:29:21.022386+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:01:35.301014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022278+0000","last_change":"2024-09-21T00:29:19.814239+0000","last_active":"2024-09-21T00:29:21.022278+0000","last_peered":"2024-09-21T00:29:21.022278+0000","last_clean":"2024-09-21T00:29:21.022278+0000","last_became_active":"2024-09-21T00:29:19.813892+0000","last_became_peered":"2024-09-21T00:29:19.813892+0000","last_unstale":"2024-09-21T00:29:21.022278+0000","last_undegraded":"2024-09-21T00:29:21.022278+0000","last_fullsized":"2024-09-21T00:29:21.022278+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:02:55.200607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"93'1","reported_seq":16,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.021997+0000","last_change":"2024-09-21T00:29:18.786505+0000","last_active":"2024-09-21T00:29:21.021997+0000","last_peered":"2024-09-21T00:29:21.021997+0000","last_clean":"2024-09-21T00:29:21.021997+0000","last_became_active":"2024-09-21T00:29:18.786141+0000","last_became_peered":"2024-09-21T00:29:18.786141+0000","last_unstale":"2024-09-21T00:29:21.021997+0000","last_undegraded":"2024-09-21T00:29:21.021997+0000","last_fullsized":"2024-09-21T00:29:21.021997+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:59:49.904811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022402+0000","last_change":"2024-09-21T00:29:19.810193+0000","last_active":"2024-09-21T00:29:21.022402+0000","last_peered":"2024-09-21T00:29:21.022402+0000","last_clean":"2024-09-21T00:29:21.022402+0000","last_became_active":"2024-09-21T00:29:19.809872+0000","last_became_peered":"2024-09-21T00:29:19.809872+0000","last_unstale":"2024-09-21T00:29:21.022402+0000","last_undegraded":"2024-09-21T00:29:21.022402+0000","last_fullsized":"2024-09-21T00:29:21.022402+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:42:51.392778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.809671+0000","last_change":"2024-09-21T00:29:18.802388+0000","last_active":"2024-09-21T00:29:20.809671+0000","last_peered":"2024-09-21T00:29:20.809671+0000","last_clean":"2024-09-21T00:29:20.809671+0000","last_became_active":"2024-09-21T00:29:18.802060+0000","last_became_peered":"2024-09-21T00:29:18.802060+0000","last_unstale":"2024-09-21T00:29:20.809671+0000","last_undegraded":"2024-09-21T00:29:20.809671+0000","last_fullsized":"2024-09-21T00:29:20.809671+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:26:13.866796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.807105+0000","last_change":"2024-09-21T00:29:19.812924+0000","last_active":"2024-09-21T00:29:20.807105+0000","last_peered":"2024-09-21T00:29:20.807105+0000","last_clean":"2024-09-21T00:29:20.807105+0000","last_became_active":"2024-09-21T00:29:19.812607+0000","last_became_peered":"2024-09-21T00:29:19.812607+0000","last_unstale":"2024-09-21T00:29:20.807105+0000","last_undegraded":"2024-09-21T00:29:20.807105+0000","last_fullsized":"2024-09-21T00:29:20.807105+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:11:09.667419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"93'2","reported_seq":17,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.023167+0000","last_change":"2024-09-21T00:29:18.796630+0000","last_active":"2024-09-21T00:29:21.023167+0000","last_peered":"2024-09-21T00:29:21.023167+0000","last_clean":"2024-09-21T00:29:21.023167+0000","last_became_active":"2024-09-21T00:29:18.796317+0000","last_became_peered":"2024-09-21T00:29:18.796317+0000","last_unstale":"2024-09-21T00:29:21.023167+0000","last_undegraded":"2024-09-21T00:29:21.023167+0000","last_fullsized":"2024-09-21T00:29:21.023167+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:50:16.709606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.807330+0000","last_change":"2024-09-21T00:29:19.801051+0000","last_active":"2024-09-21T00:29:20.807330+0000","last_peered":"2024-09-21T00:29:20.807330+0000","last_clean":"2024-09-21T00:29:20.807330+0000","last_became_active":"2024-09-21T00:29:19.798834+0000","last_became_peered":"2024-09-21T00:29:19.798834+0000","last_unstale":"2024-09-21T00:29:20.807330+0000","last_undegraded":"2024-09-21T00:29:20.807330+0000","last_fullsized":"2024-09-21T00:29:20.807330+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:20:15.757612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022535+0000","last_change":"2024-09-21T00:29:18.796311+0000","last_active":"2024-09-21T00:29:21.022535+0000","last_peered":"2024-09-21T00:29:21.022535+0000","last_clean":"2024-09-21T00:29:21.022535+0000","last_became_active":"2024-09-21T00:29:18.795425+0000","last_became_peered":"2024-09-21T00:29:18.795425+0000","last_unstale":"2024-09-21T00:29:21.022535+0000","last_undegraded":"2024-09-21T00:29:21.022535+0000","last_fullsized":"2024-09-21T00:29:21.022535+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:12:21.051042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022415+0000","last_change":"2024-09-21T00:29:19.813509+0000","last_active":"2024-09-21T00:29:21.022415+0000","last_peered":"2024-09-21T00:29:21.022415+0000","last_clean":"2024-09-21T00:29:21.022415+0000","last_became_active":"2024-09-21T00:29:19.813162+0000","last_became_peered":"2024-09-21T00:29:19.813162+0000","last_unstale":"2024-09-21T00:29:21.022415+0000","last_undegraded":"2024-09-21T00:29:21.022415+0000","last_fullsized":"2024-09-21T00:29:21.022415+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:48:11.528107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022396+0000","last_change":"2024-09-21T00:29:18.799921+0000","last_active":"2024-09-21T00:29:21.022396+0000","last_peered":"2024-09-21T00:29:21.022396+0000","last_clean":"2024-09-21T00:29:21.022396+0000","last_became_active":"2024-09-21T00:29:18.799614+0000","last_became_peered":"2024-09-21T00:29:18.799614+0000","last_unstale":"2024-09-21T00:29:21.022396+0000","last_undegraded":"2024-09-21T00:29:21.022396+0000","last_fullsized":"2024-09-21T00:29:21.022396+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:33:56.930832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022472+0000","last_change":"2024-09-21T00:29:19.813609+0000","last_active":"2024-09-21T00:29:21.022472+0000","last_peered":"2024-09-21T00:29:21.022472+0000","last_clean":"2024-09-21T00:29:21.022472+0000","last_became_active":"2024-09-21T00:29:19.813291+0000","last_became_peered":"2024-09-21T00:29:19.813291+0000","last_unstale":"2024-09-21T00:29:21.022472+0000","last_undegraded":"2024-09-21T00:29:21.022472+0000","last_fullsized":"2024-09-21T00:29:21.022472+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:48:12.895949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022964+0000","last_change":"2024-09-21T00:29:18.796499+0000","last_active":"2024-09-21T00:29:21.022964+0000","last_peered":"2024-09-21T00:29:21.022964+0000","last_clean":"2024-09-21T00:29:21.022964+0000","last_became_active":"2024-09-21T00:29:18.796175+0000","last_became_peered":"2024-09-21T00:29:18.796175+0000","last_unstale":"2024-09-21T00:29:21.022964+0000","last_undegraded":"2024-09-21T00:29:21.022964+0000","last_fullsized":"2024-09-21T00:29:21.022964+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:58:46.110270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.807498+0000","last_change":"2024-09-21T00:29:19.815935+0000","last_active":"2024-09-21T00:29:20.807498+0000","last_peered":"2024-09-21T00:29:20.807498+0000","last_clean":"2024-09-21T00:29:20.807498+0000","last_became_active":"2024-09-21T00:29:19.814883+0000","last_became_peered":"2024-09-21T00:29:19.814883+0000","last_unstale":"2024-09-21T00:29:20.807498+0000","last_undegraded":"2024-09-21T00:29:20.807498+0000","last_fullsized":"2024-09-21T00:29:20.807498+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:40:18.526456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"93'2","reported_seq":17,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.021237+0000","last_change":"2024-09-21T00:29:18.802766+0000","last_active":"2024-09-21T00:29:21.021237+0000","last_peered":"2024-09-21T00:29:21.021237+0000","last_clean":"2024-09-21T00:29:21.021237+0000","last_became_active":"2024-09-21T00:29:18.802061+0000","last_became_peered":"2024-09-21T00:29:18.802061+0000","last_unstale":"2024-09-21T00:29:21.021237+0000","last_undegraded":"2024-09-21T00:29:21.021237+0000","last_fullsized":"2024-09-21T00:29:21.021237+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:43:45.169355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.809738+0000","last_change":"2024-09-21T00:29:19.806248+0000","last_active":"2024-09-21T00:29:20.809738+0000","last_peered":"2024-09-21T00:29:20.809738+0000","last_clean":"2024-09-21T00:29:20.809738+0000","last_became_active":"2024-09-21T00:29:19.805916+0000","last_became_peered":"2024-09-21T00:29:19.805916+0000","last_unstale":"2024-09-21T00:29:20.809738+0000","last_undegraded":"2024-09-21T00:29:20.809738+0000","last_fullsized":"2024-09-21T00:29:20.809738+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:42:42.562148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"93'1","reported_seq":16,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.807289+0000","last_change":"2024-09-21T00:29:18.797446+0000","last_active":"2024-09-21T00:29:20.807289+0000","last_peered":"2024-09-21T00:29:20.807289+0000","last_clean":"2024-09-21T00:29:20.807289+0000","last_became_active":"2024-09-21T00:29:18.797076+0000","last_became_peered":"2024-09-21T00:29:18.797076+0000","last_unstale":"2024-09-21T00:29:20.807289+0000","last_undegraded":"2024-09-21T00:29:20.807289+0000","last_fullsized":"2024-09-21T00:29:20.807289+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:06:17.566926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022312+0000","last_change":"2024-09-21T00:29:19.812384+0000","last_active":"2024-09-21T00:29:21.022312+0000","last_peered":"2024-09-21T00:29:21.022312+0000","last_clean":"2024-09-21T00:29:21.022312+0000","last_became_active":"2024-09-21T00:29:19.812077+0000","last_became_peered":"2024-09-21T00:29:19.812077+0000","last_unstale":"2024-09-21T00:29:21.022312+0000","last_undegraded":"2024-09-21T00:29:21.022312+0000","last_fullsized":"2024-09-21T00:29:21.022312+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:28:10.979181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022603+0000","last_change":"2024-09-21T00:29:18.796908+0000","last_active":"2024-09-21T00:29:21.022603+0000","last_peered":"2024-09-21T00:29:21.022603+0000","last_clean":"2024-09-21T00:29:21.022603+0000","last_became_active":"2024-09-21T00:29:18.796559+0000","last_became_peered":"2024-09-21T00:29:18.796559+0000","last_unstale":"2024-09-21T00:29:21.022603+0000","last_undegraded":"2024-09-21T00:29:21.022603+0000","last_fullsized":"2024-09-21T00:29:21.022603+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:27:50.568077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.809937+0000","last_change":"2024-09-21T00:29:19.808826+0000","last_active":"2024-09-21T00:29:20.809937+0000","last_peered":"2024-09-21T00:29:20.809937+0000","last_clean":"2024-09-21T00:29:20.809937+0000","last_became_active":"2024-09-21T00:29:19.808196+0000","last_became_peered":"2024-09-21T00:29:19.808196+0000","last_unstale":"2024-09-21T00:29:20.809937+0000","last_undegraded":"2024-09-21T00:29:20.809937+0000","last_fullsized":"2024-09-21T00:29:20.809937+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:22:53.222243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.021895+0000","last_change":"2024-09-21T00:29:18.802804+0000","last_active":"2024-09-21T00:29:21.021895+0000","last_peered":"2024-09-21T00:29:21.021895+0000","last_clean":"2024-09-21T00:29:21.021895+0000","last_became_active":"2024-09-21T00:29:18.802469+0000","last_became_peered":"2024-09-21T00:29:18.802469+0000","last_unstale":"2024-09-21T00:29:21.021895+0000","last_undegraded":"2024-09-21T00:29:21.021895+0000","last_fullsized":"2024-09-21T00:29:21.021895+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:27:36.567501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.021641+0000","last_change":"2024-09-21T00:29:19.823036+0000","last_active":"2024-09-21T00:29:21.021641+0000","last_peered":"2024-09-21T00:29:21.021641+0000","last_clean":"2024-09-21T00:29:21.021641+0000","last_became_active":"2024-09-21T00:29:19.822688+0000","last_became_peered":"2024-09-21T00:29:19.822688+0000","last_unstale":"2024-09-21T00:29:21.021641+0000","last_undegraded":"2024-09-21T00:29:21.021641+0000","last_fullsized":"2024-09-21T00:29:21.021641+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:54:44.835987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.809976+0000","last_change":"2024-09-21T00:29:18.795683+0000","last_active":"2024-09-21T00:29:20.809976+0000","last_peered":"2024-09-21T00:29:20.809976+0000","last_clean":"2024-09-21T00:29:20.809976+0000","last_became_active":"2024-09-21T00:29:18.795363+0000","last_became_peered":"2024-09-21T00:29:18.795363+0000","last_unstale":"2024-09-21T00:29:20.809976+0000","last_undegraded":"2024-09-21T00:29:20.809976+0000","last_fullsized":"2024-09-21T00:29:20.809976+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:20:55.296469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022149+0000","last_change":"2024-09-21T00:29:19.812845+0000","last_active":"2024-09-21T00:29:21.022149+0000","last_peered":"2024-09-21T00:29:21.022149+0000","last_clean":"2024-09-21T00:29:21.022149+0000","last_became_active":"2024-09-21T00:29:19.812536+0000","last_became_peered":"2024-09-21T00:29:19.812536+0000","last_unstale":"2024-09-21T00:29:21.022149+0000","last_undegraded":"2024-09-21T00:29:21.022149+0000","last_fullsized":"2024-09-21T00:29:21.022149+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:48:44.631863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022864+0000","last_change":"2024-09-21T00:29:18.800233+0000","last_active":"2024-09-21T00:29:21.022864+0000","last_peered":"2024-09-21T00:29:21.022864+0000","last_clean":"2024-09-21T00:29:21.022864+0000","last_became_active":"2024-09-21T00:29:18.799952+0000","last_became_peered":"2024-09-21T00:29:18.799952+0000","last_unstale":"2024-09-21T00:29:21.022864+0000","last_undegraded":"2024-09-21T00:29:21.022864+0000","last_fullsized":"2024-09-21T00:29:21.022864+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:21:56.870500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.021841+0000","last_change":"2024-09-21T00:29:19.816067+0000","last_active":"2024-09-21T00:29:21.021841+0000","last_peered":"2024-09-21T00:29:21.021841+0000","last_clean":"2024-09-21T00:29:21.021841+0000","last_became_active":"2024-09-21T00:29:19.815714+0000","last_became_peered":"2024-09-21T00:29:19.815714+0000","last_unstale":"2024-09-21T00:29:21.021841+0000","last_undegraded":"2024-09-21T00:29:21.021841+0000","last_fullsized":"2024-09-21T00:29:21.021841+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:30:01.904485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.022492+0000","last_change":"2024-09-21T00:29:18.796212+0000","last_active":"2024-09-21T00:29:21.022492+0000","last_peered":"2024-09-21T00:29:21.022492+0000","last_clean":"2024-09-21T00:29:21.022492+0000","last_became_active":"2024-09-21T00:29:18.795864+0000","last_became_peered":"2024-09-21T00:29:18.795864+0000","last_unstale":"2024-09-21T00:29:21.022492+0000","last_undegraded":"2024-09-21T00:29:21.022492+0000","last_fullsized":"2024-09-21T00:29:21.022492+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:57:43.771137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.810604+0000","last_change":"2024-09-21T00:29:19.811714+0000","last_active":"2024-09-21T00:29:20.810604+0000","last_peered":"2024-09-21T00:29:20.810604+0000","last_clean":"2024-09-21T00:29:20.810604+0000","last_became_active":"2024-09-21T00:29:19.809202+0000","last_became_peered":"2024-09-21T00:29:19.809202+0000","last_unstale":"2024-09-21T00:29:20.810604+0000","last_undegraded":"2024-09-21T00:29:20.810604+0000","last_fullsized":"2024-09-21T00:29:20.810604+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:44:42.287171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.023051+0000","last_change":"2024-09-21T00:29:18.801879+0000","last_active":"2024-09-21T00:29:21.023051+0000","last_peered":"2024-09-21T00:29:21.023051+0000","last_clean":"2024-09-21T00:29:21.023051+0000","last_became_active":"2024-09-21T00:29:18.800708+0000","last_became_peered":"2024-09-21T00:29:18.800708+0000","last_unstale":"2024-09-21T00:29:21.023051+0000","last_undegraded":"2024-09-21T00:29:21.023051+0000","last_fullsized":"2024-09-21T00:29:21.023051+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:48:40.413052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:20.809702+0000","last_change":"2024-09-21T00:29:19.798077+0000","last_active":"2024-09-21T00:29:20.809702+0000","last_peered":"2024-09-21T00:29:20.809702+0000","last_clean":"2024-09-21T00:29:20.809702+0000","last_became_active":"2024-09-21T00:29:19.797724+0000","last_became_peered":"2024-09-21T00:29:19.797724+0000","last_unstale":"2024-09-21T00:29:20.809702+0000","last_undegraded":"2024-09-21T00:29:20.809702+0000","last_fullsized":"2024-09-21T00:29:20.809702+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:18.775092+0000","last_clean_scrub_stamp":"2024-09-21T00:29:18.775092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:35:16.985174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"93'2","reported_seq":17,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.023087+0000","last_change":"2024-09-21T00:29:18.800162+0000","last_active":"2024-09-21T00:29:21.023087+0000","last_peered":"2024-09-21T00:29:21.023087+0000","last_clean":"2024-09-21T00:29:21.023087+0000","last_became_active":"2024-09-21T00:29:18.798939+0000","last_became_peered":"2024-09-21T00:29:18.798939+0000","last_unstale":"2024-09-21T00:29:21.023087+0000","last_undegraded":"2024-09-21T00:29:21.023087+0000","last_fullsized":"2024-09-21T00:29:21.023087+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:17.769711+0000","last_clean_scrub_stamp":"2024-09-21T00:29:17.769711+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:28:05.362595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":244,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-21T00:29:21.021503+0000","last_change":"2024-09-21T00:25:38.725792+0000","last_active":"2024-09-21T00:29:21.021503+0000","last_peered":"2024-09-21T00:29:21.021503+0000","last_clean":"2024-09-21T00:29:21.021503+0000","last_became_active":"2024-09-21T00:25:38.724668+0000","last_became_peered":"2024-09-21T00:25:38.724668+0000","last_unstale":"2024-09-21T00:29:21.021503+0000","last_undegraded":"2024-09-21T00:29:21.021503+0000","last_fullsized":"2024-09-21T00:29:21.021503+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:37.163216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:37.163216+0000","last_clean_scrub_stamp":"2024-09-21T00:25:37.163216+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:14:36.588660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509489,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32908,"kb_used_data":1720,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338932,"statfs":{"total":96636764160,"available":96603066368,"internally_reserved":0,"allocated":1761280,"data_stored":1116174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509490,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32340,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339500,"statfs":{"total":96636764160,"available":96603648000,"internally_reserved":0,"allocated":1179648,"data_stored":527862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509489,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32324,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339516,"statfs":{"total":96636764160,"available":96603664384,"internally_reserved":0,"allocated":1159168,"data_stored":525646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509488,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32328,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339512,"statfs":{"total":96636764160,"available":96603660288,"internally_reserved":0,"allocated":1167360,"data_stored":525738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509490,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32328,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339512,"statfs":{"total":96636764160,"available":96603660288,"internally_reserved":0,"allocated":1163264,"data_stored":526228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509490,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32324,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339516,"statfs":{"total":96636764160,"available":96603664384,"internally_reserved":0,"allocated":1163264,"data_stored":525680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509490,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32920,"kb_used_data":1728,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338920,"statfs":{"total":96636764160,"available":96603054080,"internally_reserved":0,"allocated":1769472,"data_stored":1117716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574897,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32328,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339512,"statfs":{"total":96636764160,"available":96603660288,"internally_reserved":0,"allocated":1163264,"data_stored":525680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T00:29:28.625 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T00:29:28.625 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-21T00:29:28.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:28.779+0000 7f7801d90640 1 -- 172.21.15.112:0/723100704 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77fc0d48a0 msgr2=0x7f77fc0d3dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:28.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:28.779+0000 7f7801d90640 1 --2- 172.21.15.112:0/723100704 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77fc0d48a0 0x7f77fc0d3dc0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f77fc0cea80 tx=0x7f77ec02a560 comp rx=0 tx=0).stop 2024-09-21T00:29:28.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:28.779+0000 7f7801d90640 1 -- 172.21.15.112:0/723100704 shutdown_connections 2024-09-21T00:29:28.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:28.779+0000 7f7801d90640 1 --2- 172.21.15.112:0/723100704 >> v2:172.21.15.187:3300/0 conn(0x7f77fc0d3880 0x7f77fc163260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:28.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:28.779+0000 7f7801d90640 1 --2- 172.21.15.112:0/723100704 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f77fc0d01b0 0x7f77fc0d05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:28.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:28.779+0000 7f7801d90640 1 --2- 172.21.15.112:0/723100704 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77fc0d48a0 0x7f77fc0d3dc0 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:28.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:28.779+0000 7f7801d90640 1 -- 172.21.15.112:0/723100704 >> 172.21.15.112:0/723100704 conn(0x7f77fc08bfe0 msgr2=0x7f77fc09d890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:28.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:28.779+0000 7f7801d90640 1 -- 172.21.15.112:0/723100704 shutdown_connections 2024-09-21T00:29:28.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:28.780+0000 7f7801d90640 1 -- 172.21.15.112:0/723100704 wait complete. 2024-09-21T00:29:28.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:28.781+0000 7f7801d90640 1 Processor -- start 2024-09-21T00:29:28.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:28.781+0000 7f7801d90640 1 -- start start 2024-09-21T00:29:28.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:28.783+0000 7f7801d90640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77fc0d01b0 0x7f77fc161be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:28.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:28.783+0000 7f7801d90640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f77fc0d3880 0x7f77fc1545b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:28.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:28.783+0000 7f7801d90640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f77fc0d48a0 0x7f77fc0b3370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:28.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:28.783+0000 7f7801d90640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f77fc0d2250 con 0x7f77fc0d01b0 2024-09-21T00:29:28.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:28.783+0000 7f7801d90640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f77fc0d20d0 con 0x7f77fc0d3880 2024-09-21T00:29:28.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:28.783+0000 7f7801d90640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f77fc0d23d0 con 0x7f77fc0d48a0 2024-09-21T00:29:28.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:28.783+0000 7f77fb7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77fc0d01b0 0x7f77fc161be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:28.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:28.783+0000 7f77fb7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77fc0d01b0 0x7f77fc161be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57822/0 (socket says 172.21.15.112:57822) 2024-09-21T00:29:28.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:28.783+0000 7f77fb7fe640 1 -- 172.21.15.112:0/3966189360 learned_addr learned my addr 172.21.15.112:0/3966189360 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:29:28.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:28.783+0000 7f77faffd640 1 --2- 172.21.15.112:0/3966189360 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f77fc0d3880 0x7f77fc1545b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:28.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:28.783+0000 7f77fbfff640 1 --2- 172.21.15.112:0/3966189360 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f77fc0d48a0 0x7f77fc0b3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:28.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:28.783+0000 7f77fb7fe640 1 -- 172.21.15.112:0/3966189360 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f77fc0d48a0 msgr2=0x7f77fc0b3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:28.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:28.783+0000 7f77fb7fe640 1 --2- 172.21.15.112:0/3966189360 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f77fc0d48a0 0x7f77fc0b3370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:28.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:28.783+0000 7f77fb7fe640 1 -- 172.21.15.112:0/3966189360 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f77fc0d3880 msgr2=0x7f77fc1545b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:28.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:28.783+0000 7f77fb7fe640 1 --2- 172.21.15.112:0/3966189360 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f77fc0d3880 0x7f77fc1545b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:28.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:28.783+0000 7f77fb7fe640 1 -- 172.21.15.112:0/3966189360 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77fc154af0 con 0x7f77fc0d01b0 2024-09-21T00:29:28.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:28.783+0000 7f77fb7fe640 1 --2- 172.21.15.112:0/3966189360 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77fc0d01b0 0x7f77fc161be0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f77ec0099c0 tx=0x7f77ec0038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:28.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:28.783+0000 7f77e3fff640 1 -- 172.21.15.112:0/3966189360 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77ec03e030 con 0x7f77fc0d01b0 2024-09-21T00:29:28.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:28.783+0000 7f77e3fff640 1 -- 172.21.15.112:0/3966189360 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f77ec003db0 con 0x7f77fc0d01b0 2024-09-21T00:29:28.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:28.783+0000 7f7801d90640 1 -- 172.21.15.112:0/3966189360 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77fc06dad0 con 0x7f77fc0d01b0 2024-09-21T00:29:28.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:28.783+0000 7f77e3fff640 1 -- 172.21.15.112:0/3966189360 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77ec035040 con 0x7f77fc0d01b0 2024-09-21T00:29:28.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:28.783+0000 7f7801d90640 1 -- 172.21.15.112:0/3966189360 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77fc06e090 con 0x7f77fc0d01b0 2024-09-21T00:29:28.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:28.784+0000 7f7801d90640 1 -- 172.21.15.112:0/3966189360 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77fc163580 con 0x7f77fc0d01b0 2024-09-21T00:29:28.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:28.785+0000 7f77e3fff640 1 -- 172.21.15.112:0/3966189360 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f77ec041030 con 0x7f77fc0d01b0 2024-09-21T00:29:28.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:28.786+0000 7f77e3fff640 1 --2- 172.21.15.112:0/3966189360 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f77e8078bc0 0x7f77e807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:28.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:28.786+0000 7f77e3fff640 1 -- 172.21.15.112:0/3966189360 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6188+0+0 (secure 0 0 0) 0x7f77ec0bba50 con 0x7f77fc0d01b0 2024-09-21T00:29:28.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:28.786+0000 7f77faffd640 1 --2- 172.21.15.112:0/3966189360 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f77e8078bc0 0x7f77e807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:28.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:28.786+0000 7f77faffd640 1 --2- 172.21.15.112:0/3966189360 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f77e8078bc0 0x7f77e807b0d0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f77fc1f93f0 tx=0x7f77f0005f90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:28.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:28.790+0000 7f77e3fff640 1 -- 172.21.15.112:0/3966189360 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f77ec083c30 con 0x7f77fc0d01b0 2024-09-21T00:29:29.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.041+0000 7f7801d90640 1 -- 172.21.15.112:0/3966189360 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f77fc0b6f90 con 0x7f77fc0d01b0 2024-09-21T00:29:29.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.044+0000 7f77e3fff640 1 -- 172.21.15.112:0/3966189360 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v20) ==== 190+0+147 (secure 0 0 0) 0x7f77ec083530 con 0x7f77fc0d01b0 2024-09-21T00:29:29.045 INFO:teuthology.orchestra.run.smithi112.stdout:[client.0] 2024-09-21T00:29:29.045 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQBPEu5m49NPHBAAl/rcjiE8xYI+XdWwvZ6FOw== 2024-09-21T00:29:29.045 INFO:teuthology.orchestra.run.smithi112.stdout: caps mds = "allow" 2024-09-21T00:29:29.046 INFO:teuthology.orchestra.run.smithi112.stdout: caps mon = "allow r" 2024-09-21T00:29:29.046 INFO:teuthology.orchestra.run.smithi112.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-21T00:29:29.046 INFO:teuthology.orchestra.run.smithi112.stderr:updated caps for client.0 2024-09-21T00:29:29.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.046+0000 7f7801d90640 1 -- 172.21.15.112:0/3966189360 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f77e8078bc0 msgr2=0x7f77e807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:29.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.046+0000 7f7801d90640 1 --2- 172.21.15.112:0/3966189360 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f77e8078bc0 0x7f77e807b0d0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f77fc1f93f0 tx=0x7f77f0005f90 comp rx=0 tx=0).stop 2024-09-21T00:29:29.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.046+0000 7f7801d90640 1 -- 172.21.15.112:0/3966189360 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77fc0d01b0 msgr2=0x7f77fc161be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:29.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.046+0000 7f7801d90640 1 --2- 172.21.15.112:0/3966189360 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77fc0d01b0 0x7f77fc161be0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f77ec0099c0 tx=0x7f77ec0038a0 comp rx=0 tx=0).stop 2024-09-21T00:29:29.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.046+0000 7f7801d90640 1 -- 172.21.15.112:0/3966189360 shutdown_connections 2024-09-21T00:29:29.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.046+0000 7f7801d90640 1 --2- 172.21.15.112:0/3966189360 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f77fc0d48a0 0x7f77fc0b3370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:29.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.046+0000 7f7801d90640 1 --2- 172.21.15.112:0/3966189360 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f77fc0d3880 0x7f77fc1545b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:29.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.046+0000 7f7801d90640 1 --2- 172.21.15.112:0/3966189360 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f77e8078bc0 0x7f77e807b0d0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:29.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.046+0000 7f7801d90640 1 --2- 172.21.15.112:0/3966189360 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77fc0d01b0 0x7f77fc161be0 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:29.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.046+0000 7f7801d90640 1 -- 172.21.15.112:0/3966189360 >> 172.21.15.112:0/3966189360 conn(0x7f77fc08bfe0 msgr2=0x7f77fc15ea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:29.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.046+0000 7f7801d90640 1 -- 172.21.15.112:0/3966189360 shutdown_connections 2024-09-21T00:29:29.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.046+0000 7f7801d90640 1 -- 172.21.15.112:0/3966189360 wait complete. 2024-09-21T00:29:29.059 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T00:29:29.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.170+0000 7ff36f84d640 1 -- 172.21.15.112:0/3497555946 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff3680ccae0 msgr2=0x7ff3680ccef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:29.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.170+0000 7ff36f84d640 1 --2- 172.21.15.112:0/3497555946 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff3680ccae0 0x7ff3680ccef0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7ff360005fd0 tx=0x7ff36002aa00 comp rx=0 tx=0).stop 2024-09-21T00:29:29.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.170+0000 7ff36f84d640 1 -- 172.21.15.112:0/3497555946 shutdown_connections 2024-09-21T00:29:29.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.170+0000 7ff36f84d640 1 --2- 172.21.15.112:0/3497555946 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff3680ccae0 0x7ff3680ccef0 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:29.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.170+0000 7ff36f84d640 1 --2- 172.21.15.112:0/3497555946 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff3680d2030 0x7ff36815d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:29.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.170+0000 7ff36f84d640 1 -- 172.21.15.112:0/3497555946 >> 172.21.15.112:0/3497555946 conn(0x7ff3680b43f0 msgr2=0x7ff3680b4830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:29.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.170+0000 7ff36f84d640 1 -- 172.21.15.112:0/3497555946 shutdown_connections 2024-09-21T00:29:29.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.170+0000 7ff36f84d640 1 -- 172.21.15.112:0/3497555946 wait complete. 2024-09-21T00:29:29.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.172+0000 7ff36f84d640 1 Processor -- start 2024-09-21T00:29:29.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.172+0000 7ff36f84d640 1 -- start start 2024-09-21T00:29:29.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.173+0000 7ff36f84d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff3680ccae0 0x7ff3681e6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:29.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.173+0000 7ff36f84d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff3680d0860 0x7ff3681f4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:29.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.173+0000 7ff36f84d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff3680d2030 0x7ff3681f6be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:29.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.173+0000 7ff36f84d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff3680b22a0 con 0x7ff3680d2030 2024-09-21T00:29:29.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.173+0000 7ff36f84d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff3680b2120 con 0x7ff3680ccae0 2024-09-21T00:29:29.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.173+0000 7ff36f84d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff3680b2420 con 0x7ff3680d0860 2024-09-21T00:29:29.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.173+0000 7ff36ddea640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff3680d2030 0x7ff3681f6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:29.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.173+0000 7ff36ddea640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff3680d2030 0x7ff3681f6be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57838/0 (socket says 172.21.15.112:57838) 2024-09-21T00:29:29.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.173+0000 7ff36ddea640 1 -- 172.21.15.112:0/870204130 learned_addr learned my addr 172.21.15.112:0/870204130 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:29:29.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.173+0000 7ff36cde8640 1 --2- 172.21.15.112:0/870204130 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff3680d0860 0x7ff3681f4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:29.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.173+0000 7ff36d5e9640 1 --2- 172.21.15.112:0/870204130 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff3680ccae0 0x7ff3681e6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:29.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.173+0000 7ff36ddea640 1 -- 172.21.15.112:0/870204130 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff3680d0860 msgr2=0x7ff3681f4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:29.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.173+0000 7ff36ddea640 1 --2- 172.21.15.112:0/870204130 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff3680d0860 0x7ff3681f4720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:29.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.173+0000 7ff36ddea640 1 -- 172.21.15.112:0/870204130 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff3680ccae0 msgr2=0x7ff3681e6cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:29.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.173+0000 7ff36ddea640 1 --2- 172.21.15.112:0/870204130 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff3680ccae0 0x7ff3681e6cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:29.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.173+0000 7ff36ddea640 1 -- 172.21.15.112:0/870204130 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3681e5c50 con 0x7ff3680d2030 2024-09-21T00:29:29.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.174+0000 7ff36ddea640 1 --2- 172.21.15.112:0/870204130 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff3680d2030 0x7ff3681f6be0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7ff360006010 tx=0x7ff360003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:29.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.174+0000 7ff35dffb640 1 -- 172.21.15.112:0/870204130 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff360046070 con 0x7ff3680d2030 2024-09-21T00:29:29.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.174+0000 7ff35dffb640 1 -- 172.21.15.112:0/870204130 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff360004540 con 0x7ff3680d2030 2024-09-21T00:29:29.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.174+0000 7ff35dffb640 1 -- 172.21.15.112:0/870204130 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff360033070 con 0x7ff3680d2030 2024-09-21T00:29:29.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.174+0000 7ff36f84d640 1 -- 172.21.15.112:0/870204130 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3681e5f30 con 0x7ff3680d2030 2024-09-21T00:29:29.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.174+0000 7ff36f84d640 1 -- 172.21.15.112:0/870204130 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3681e64f0 con 0x7ff3680d2030 2024-09-21T00:29:29.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.175+0000 7ff36f84d640 1 -- 172.21.15.112:0/870204130 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3680b2950 con 0x7ff3680d2030 2024-09-21T00:29:29.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.176+0000 7ff35dffb640 1 -- 172.21.15.112:0/870204130 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff360049070 con 0x7ff3680d2030 2024-09-21T00:29:29.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.176+0000 7ff35dffb640 1 --2- 172.21.15.112:0/870204130 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff34c078b90 0x7ff34c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:29.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.176+0000 7ff35dffb640 1 -- 172.21.15.112:0/870204130 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 6188+0+0 (secure 0 0 0) 0x7ff3600bc060 con 0x7ff3680d2030 2024-09-21T00:29:29.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.176+0000 7ff36d5e9640 1 --2- 172.21.15.112:0/870204130 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff34c078b90 0x7ff34c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:29.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.177+0000 7ff36d5e9640 1 --2- 172.21.15.112:0/870204130 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff34c078b90 0x7ff34c07b0a0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7ff3640098e0 tx=0x7ff364009060 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:29.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.181+0000 7ff35dffb640 1 -- 172.21.15.112:0/870204130 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ff36003e070 con 0x7ff3680d2030 2024-09-21T00:29:29.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.392+0000 7ff36f84d640 1 -- 172.21.15.112:0/870204130 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff36815e0d0 con 0x7ff3680d2030 2024-09-21T00:29:29.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.392+0000 7ff35dffb640 1 -- 172.21.15.112:0/870204130 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) ==== 94+0+5237 (secure 0 0 0) 0x7ff3680cd430 con 0x7ff3680d2030 2024-09-21T00:29:29.394 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:29:29.394 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":64,"btime":"2024-09-21T00:29:22:809921+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5804,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/3818262153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":3818262153},{"type":"v1","addr":"172.21.15.187:6835","nonce":3818262153}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59},{"gid":6214,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6837/1321768683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":1321768683},{"type":"v1","addr":"172.21.15.187:6837","nonce":1321768683}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59},{"gid":6801,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6834/4037012343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":4037012343},{"type":"v1","addr":"172.21.15.112:6834","nonce":4037012343}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T00:29:19.145794+0000","modified":"2024-09-21T00:29:22.172462+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6804},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6804":{"gid":6804,"name":"c","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6835/2797048793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":2797048793},{"type":"v1","addr":"172.21.15.112:6835","nonce":2797048793}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6804,"qdb_cluster":[6804]},"id":7}]} 2024-09-21T00:29:29.394 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 64 2024-09-21T00:29:29.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.394+0000 7ff36f84d640 1 -- 172.21.15.112:0/870204130 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff34c078b90 msgr2=0x7ff34c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:29.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.394+0000 7ff36f84d640 1 --2- 172.21.15.112:0/870204130 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff34c078b90 0x7ff34c07b0a0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7ff3640098e0 tx=0x7ff364009060 comp rx=0 tx=0).stop 2024-09-21T00:29:29.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.394+0000 7ff36f84d640 1 -- 172.21.15.112:0/870204130 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff3680d2030 msgr2=0x7ff3681f6be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:29.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.394+0000 7ff36f84d640 1 --2- 172.21.15.112:0/870204130 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff3680d2030 0x7ff3681f6be0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7ff360006010 tx=0x7ff360003f10 comp rx=0 tx=0).stop 2024-09-21T00:29:29.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.394+0000 7ff36f84d640 1 -- 172.21.15.112:0/870204130 shutdown_connections 2024-09-21T00:29:29.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.394+0000 7ff36f84d640 1 --2- 172.21.15.112:0/870204130 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff34c078b90 0x7ff34c07b0a0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:29.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.395+0000 7ff36f84d640 1 --2- 172.21.15.112:0/870204130 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff3680d2030 0x7ff3681f6be0 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:29.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.395+0000 7ff36f84d640 1 --2- 172.21.15.112:0/870204130 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff3680d0860 0x7ff3681f4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:29.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.395+0000 7ff36f84d640 1 --2- 172.21.15.112:0/870204130 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff3680ccae0 0x7ff3681e6cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:29.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.395+0000 7ff36f84d640 1 -- 172.21.15.112:0/870204130 >> 172.21.15.112:0/870204130 conn(0x7ff3680b43f0 msgr2=0x7ff36815f070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:29.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.395+0000 7ff36f84d640 1 -- 172.21.15.112:0/870204130 shutdown_connections 2024-09-21T00:29:29.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.395+0000 7ff36f84d640 1 -- 172.21.15.112:0/870204130 wait complete. 2024-09-21T00:29:29.811 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 64, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T00:29:19.145794+0000', 'modified': '2024-09-21T00:29:22.172462+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6804}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6804': {'gid': 6804, 'name': 'c', 'rank': 0, 'incarnation': 60, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.112:6835/2797048793', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6833', 'nonce': 2797048793}, {'type': 'v1', 'addr': '172.21.15.112:6835', 'nonce': 2797048793}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6804, 'qdb_cluster': [6804]} 2024-09-21T00:29:29.811 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-21T00:29:29.811 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-21T00:29:29.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.924+0000 7f77ee312640 1 -- 172.21.15.112:0/3867794479 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77e80ccbf0 msgr2=0x7f77e80cd000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:29.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.924+0000 7f77ee312640 1 --2- 172.21.15.112:0/3867794479 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77e80ccbf0 0x7f77e80cd000 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f77d8005fd0 tx=0x7f77d802a410 comp rx=0 tx=0).stop 2024-09-21T00:29:29.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.924+0000 7f77ee312640 1 -- 172.21.15.112:0/3867794479 shutdown_connections 2024-09-21T00:29:29.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.924+0000 7f77ee312640 1 --2- 172.21.15.112:0/3867794479 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f77e80c6a00 0x7f77e80b2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:29.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.924+0000 7f77ee312640 1 --2- 172.21.15.112:0/3867794479 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77e80ccbf0 0x7f77e80cd000 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:29.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.924+0000 7f77ee312640 1 -- 172.21.15.112:0/3867794479 >> 172.21.15.112:0/3867794479 conn(0x7f77e80b42e0 msgr2=0x7f77e80b4720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:29.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.924+0000 7f77ee312640 1 -- 172.21.15.112:0/3867794479 shutdown_connections 2024-09-21T00:29:29.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.924+0000 7f77ee312640 1 -- 172.21.15.112:0/3867794479 wait complete. 2024-09-21T00:29:29.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.926+0000 7f77ee312640 1 Processor -- start 2024-09-21T00:29:29.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.926+0000 7f77ee312640 1 -- start start 2024-09-21T00:29:29.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.927+0000 7f77ee312640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f77e80c5ee0 0x7f77e80c9d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:29.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.927+0000 7f77ee312640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f77e80c6a00 0x7f77e8156390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:29.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.927+0000 7f77ee312640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77e80ccbf0 0x7f77e80cbfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:29.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.927+0000 7f77ee312640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f77e80b3610 con 0x7f77e80ccbf0 2024-09-21T00:29:29.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.927+0000 7f77ee312640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f77e80b3490 con 0x7f77e80c6a00 2024-09-21T00:29:29.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.927+0000 7f77ee312640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f77e80b3790 con 0x7f77e80c5ee0 2024-09-21T00:29:29.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.927+0000 7f77ec8af640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77e80ccbf0 0x7f77e80cbfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:29.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.927+0000 7f77ec8af640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77e80ccbf0 0x7f77e80cbfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57844/0 (socket says 172.21.15.112:57844) 2024-09-21T00:29:29.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.927+0000 7f77ec8af640 1 -- 172.21.15.112:0/2480098479 learned_addr learned my addr 172.21.15.112:0/2480098479 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:29:29.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.927+0000 7f77e7fff640 1 --2- 172.21.15.112:0/2480098479 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f77e80c5ee0 0x7f77e80c9d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:29.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.927+0000 7f77e77fe640 1 --2- 172.21.15.112:0/2480098479 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f77e80c6a00 0x7f77e8156390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:29.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.928+0000 7f77ec8af640 1 -- 172.21.15.112:0/2480098479 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f77e80c5ee0 msgr2=0x7f77e80c9d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:29.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.928+0000 7f77ec8af640 1 --2- 172.21.15.112:0/2480098479 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f77e80c5ee0 0x7f77e80c9d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:29.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.928+0000 7f77ec8af640 1 -- 172.21.15.112:0/2480098479 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f77e80c6a00 msgr2=0x7f77e8156390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:29.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.928+0000 7f77ec8af640 1 --2- 172.21.15.112:0/2480098479 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f77e80c6a00 0x7f77e8156390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:29.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.928+0000 7f77ec8af640 1 -- 172.21.15.112:0/2480098479 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77e80b58f0 con 0x7f77e80ccbf0 2024-09-21T00:29:29.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.928+0000 7f77ec8af640 1 --2- 172.21.15.112:0/2480098479 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77e80ccbf0 0x7f77e80cbfc0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f77e000cab0 tx=0x7f77e000cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:29.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.928+0000 7f77e4ff9640 1 -- 172.21.15.112:0/2480098479 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77e0007d90 con 0x7f77e80ccbf0 2024-09-21T00:29:29.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.928+0000 7f77e4ff9640 1 -- 172.21.15.112:0/2480098479 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f77e000f540 con 0x7f77e80ccbf0 2024-09-21T00:29:29.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.928+0000 7f77e4ff9640 1 -- 172.21.15.112:0/2480098479 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77e000f990 con 0x7f77e80ccbf0 2024-09-21T00:29:29.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.928+0000 7f77ee312640 1 -- 172.21.15.112:0/2480098479 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77e80b5bd0 con 0x7f77e80ccbf0 2024-09-21T00:29:29.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.928+0000 7f77ee312640 1 -- 172.21.15.112:0/2480098479 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77e80cbaa0 con 0x7f77e80ccbf0 2024-09-21T00:29:29.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.930+0000 7f77e4ff9640 1 -- 172.21.15.112:0/2480098479 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f77e000fb30 con 0x7f77e80ccbf0 2024-09-21T00:29:29.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.930+0000 7f77e4ff9640 1 --2- 172.21.15.112:0/2480098479 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f77d4078b90 0x7f77d407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:29.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.930+0000 7f77e4ff9640 1 -- 172.21.15.112:0/2480098479 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 6188+0+0 (secure 0 0 0) 0x7f77e009b850 con 0x7f77e80ccbf0 2024-09-21T00:29:29.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.931+0000 7f77e7fff640 1 --2- 172.21.15.112:0/2480098479 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f77d4078b90 0x7f77d407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:29.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.931+0000 7f77f5279740 1 Processor -- start 2024-09-21T00:29:29.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.931+0000 7f77e7fff640 1 --2- 172.21.15.112:0/2480098479 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f77d4078b90 0x7f77d407b0a0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f77d8033000 tx=0x7f77d8004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:29.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.931+0000 7f77f5279740 1 -- start start 2024-09-21T00:29:29.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.933+0000 7f77f5279740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563a6bb48270 0x563a6bf19fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:29.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.933+0000 7f77f5279740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x563a6bb1f3b0 0x563a6bf1a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:29.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.933+0000 7f77f5279740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563a6bb27590 0x563a6bf373e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:29.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.933+0000 7f77f5279740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x563a6be62820 con 0x563a6bb48270 2024-09-21T00:29:29.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.933+0000 7f77f5279740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x563a6be51c50 con 0x563a6bb27590 2024-09-21T00:29:29.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.933+0000 7f77f5279740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x563a6be55a60 con 0x563a6bb1f3b0 2024-09-21T00:29:29.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.933+0000 7f77e7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563a6bb48270 0x563a6bf19fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:29.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.933+0000 7f77e7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563a6bb48270 0x563a6bf19fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57850/0 (socket says 172.21.15.112:57850) 2024-09-21T00:29:29.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.933+0000 7f77e7fff640 1 -- 172.21.15.112:0/1366500563 learned_addr learned my addr 172.21.15.112:0/1366500563 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:29:29.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.933+0000 7f77e77fe640 1 --2- 172.21.15.112:0/1366500563 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x563a6bb1f3b0 0x563a6bf1a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:29.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.933+0000 7f77ec8af640 1 --2- 172.21.15.112:0/1366500563 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563a6bb27590 0x563a6bf373e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:29.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.933+0000 7f77e7fff640 1 -- 172.21.15.112:0/1366500563 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x563a6bb1f3b0 msgr2=0x563a6bf1a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:29:29.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.933+0000 7f77e7fff640 1 --2- 172.21.15.112:0/1366500563 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x563a6bb1f3b0 0x563a6bf1a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:29.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.933+0000 7f77e7fff640 1 -- 172.21.15.112:0/1366500563 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563a6bb27590 msgr2=0x563a6bf373e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:29:29.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.933+0000 7f77e7fff640 1 --2- 172.21.15.112:0/1366500563 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563a6bb27590 0x563a6bf373e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:29.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.933+0000 7f77e7fff640 1 -- 172.21.15.112:0/1366500563 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a6bbb3850 con 0x563a6bb48270 2024-09-21T00:29:29.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.933+0000 7f77e7fff640 1 --2- 172.21.15.112:0/1366500563 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563a6bb48270 0x563a6bf19fa0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f77d8006010 tx=0x7f77d803d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad71e22469a6cb27 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:29.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.933+0000 7f77cdffb640 1 -- 172.21.15.112:0/1366500563 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77d8035040 con 0x563a6bb48270 2024-09-21T00:29:29.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.933+0000 7f77cdffb640 1 -- 172.21.15.112:0/1366500563 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f77d80035f0 con 0x563a6bb48270 2024-09-21T00:29:29.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.933+0000 7f77cdffb640 1 -- 172.21.15.112:0/1366500563 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77d8003800 con 0x563a6bb48270 2024-09-21T00:29:29.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.934+0000 7f77f5279740 1 -- 172.21.15.112:0/1366500563 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563a6bb48270 msgr2=0x563a6bf19fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:29.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.934+0000 7f77f5279740 1 --2- 172.21.15.112:0/1366500563 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563a6bb48270 0x563a6bf19fa0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f77d8006010 tx=0x7f77d803d000 comp rx=0 tx=0).stop 2024-09-21T00:29:29.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.934+0000 7f77f5279740 1 -- 172.21.15.112:0/1366500563 shutdown_connections 2024-09-21T00:29:29.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.934+0000 7f77f5279740 1 --2- 172.21.15.112:0/1366500563 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563a6bb27590 0x563a6bf373e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:29.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.934+0000 7f77f5279740 1 --2- 172.21.15.112:0/1366500563 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x563a6bb1f3b0 0x563a6bf1a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:29.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.934+0000 7f77f5279740 1 --2- 172.21.15.112:0/1366500563 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563a6bb48270 0x563a6bf19fa0 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:29.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.934+0000 7f77f5279740 1 -- 172.21.15.112:0/1366500563 >> 172.21.15.112:0/1366500563 conn(0x563a6bb3c3c0 msgr2=0x563a6bef0260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:29.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.934+0000 7f77f5279740 1 -- 172.21.15.112:0/1366500563 shutdown_connections 2024-09-21T00:29:29.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.934+0000 7f77f5279740 1 -- 172.21.15.112:0/1366500563 wait complete. 2024-09-21T00:29:29.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.935+0000 7f77f5279740 1 -- start start 2024-09-21T00:29:29.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.935+0000 7f77f5279740 1 Processor -- start 2024-09-21T00:29:29.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.937+0000 7f77f5279740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563a6bb1f3b0 0x563a6bf35460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:29.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.937+0000 7f77f5279740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563a6bb27590 0x563a6bf081f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:29.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.937+0000 7f77f5279740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x563a6bb48270 0x563a6bf37140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:29.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.937+0000 7f77f5279740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x563a6bbc2620 con 0x563a6bb27590 2024-09-21T00:29:29.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.937+0000 7f77f5279740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x563a6bdf75f0 con 0x563a6bb1f3b0 2024-09-21T00:29:29.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.937+0000 7f77f5279740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x563a6bf1aa20 con 0x563a6bb48270 2024-09-21T00:29:29.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.937+0000 7f77f5279740 10 client.0 _ll_register_callbacks cb 0x563a6bdfb0a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:29:29.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.937+0000 7f77e77fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563a6bb27590 0x563a6bf081f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:29.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.937+0000 7f77e77fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563a6bb27590 0x563a6bf081f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57858/0 (socket says 172.21.15.112:57858) 2024-09-21T00:29:29.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.937+0000 7f77e77fe640 1 -- 172.21.15.112:0/1351317146 learned_addr learned my addr 172.21.15.112:0/1351317146 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:29:29.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.937+0000 7f77e7fff640 1 --2- 172.21.15.112:0/1351317146 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563a6bb1f3b0 0x563a6bf35460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:29.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.937+0000 7f77ec8af640 1 --2- 172.21.15.112:0/1351317146 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x563a6bb48270 0x563a6bf37140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:29.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.937+0000 7f77e77fe640 1 -- 172.21.15.112:0/1351317146 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x563a6bb48270 msgr2=0x563a6bf37140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:29:29.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.937+0000 7f77e77fe640 1 --2- 172.21.15.112:0/1351317146 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x563a6bb48270 0x563a6bf37140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:29.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.937+0000 7f77e77fe640 1 -- 172.21.15.112:0/1351317146 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563a6bb1f3b0 msgr2=0x563a6bf35460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:29:29.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.937+0000 7f77e77fe640 1 --2- 172.21.15.112:0/1351317146 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563a6bb1f3b0 0x563a6bf35460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:29.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.937+0000 7f77e77fe640 1 -- 172.21.15.112:0/1351317146 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a6bc2a6b0 con 0x563a6bb27590 2024-09-21T00:29:29.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.937+0000 7f77e77fe640 1 --2- 172.21.15.112:0/1351317146 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563a6bb27590 0x563a6bf081f0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f77dc011570 tx=0x7f77dc011970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f77b6c8e323e1caf server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:29.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.937+0000 7f77c6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T00:29:29.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.937+0000 7f77c6ffd640 1 -- 172.21.15.112:0/1351317146 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77dc013670 con 0x563a6bb27590 2024-09-21T00:29:29.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.937+0000 7f77c6ffd640 1 -- 172.21.15.112:0/1351317146 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 6188+0+0 (secure 0 0 0) 0x7f77dc013810 con 0x563a6bb27590 2024-09-21T00:29:29.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.937+0000 7f77f5279740 1 -- 172.21.15.112:0/1351317146 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f77d80035f0 con 0x563a6bb27590 2024-09-21T00:29:29.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.938+0000 7f77c6ffd640 1 -- 172.21.15.112:0/1351317146 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f77dc013a10 con 0x563a6bb27590 2024-09-21T00:29:29.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.938+0000 7f77f5279740 10 client.6951 fetch_fsmap learned FSMap version 64 2024-09-21T00:29:29.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.938+0000 7f77f5279740 1 -- 172.21.15.112:0/1351317146 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=2+}) -- 0x563a6bf30350 con 0x563a6bb27590 2024-09-21T00:29:29.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.938+0000 7f77c6ffd640 1 -- 172.21.15.112:0/1351317146 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f77d80035f0 con 0x563a6bb27590 2024-09-21T00:29:29.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.938+0000 7f77c6ffd640 1 -- 172.21.15.112:0/1351317146 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 64) ==== 3502+0+0 (secure 0 0 0) 0x7f77dc028080 con 0x563a6bb27590 2024-09-21T00:29:29.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.938+0000 7f77f5279740 10 client.6951 fetch_fsmap finished waiting for FSMap version 64 2024-09-21T00:29:29.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.938+0000 7f77f5279740 10 client.6951 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T00:29:29.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.938+0000 7f77f5279740 20 client.6951 populate_metadata read hostname 'smithi112' 2024-09-21T00:29:29.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.938+0000 7f77f5279740 1 --2- 172.21.15.112:0/1351317146 >> [v2:172.21.15.112:6833/2797048793,v1:172.21.15.112:6835/2797048793] conn(0x563a6bb06e40 0x563a6bf47440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:29.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.938+0000 7f77f5279740 4 client.6951 mds_command: new command op to 6804 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T00:29:29.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.938+0000 7f77f5279740 1 -- 172.21.15.112:0/1351317146 --> [v2:172.21.15.112:6833/2797048793,v1:172.21.15.112:6835/2797048793] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563a6be64770 con 0x563a6bb06e40 2024-09-21T00:29:29.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.938+0000 7f77e7fff640 1 --2- 172.21.15.112:0/1351317146 >> [v2:172.21.15.112:6833/2797048793,v1:172.21.15.112:6835/2797048793] conn(0x563a6bb06e40 0x563a6bf47440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:29.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.938+0000 7f77c6ffd640 10 client.6951 ms_handle_connect on v2:172.21.15.112:6833/2797048793 2024-09-21T00:29:29.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.939+0000 7f77e7fff640 1 --2- 172.21.15.112:0/1351317146 >> [v2:172.21.15.112:6833/2797048793,v1:172.21.15.112:6835/2797048793] conn(0x563a6bb06e40 0x563a6bf47440 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c7c5fe3e107cff7 server_cookie=56819aecd3bf6ed7 in_seq=0 out_seq=0 2024-09-21T00:29:29.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.939+0000 7f77c6ffd640 1 -- 172.21.15.112:0/1351317146 <== mds.0 v2:172.21.15.112:6833/2797048793 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563a6be64770 con 0x563a6bb06e40 2024-09-21T00:29:29.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.939+0000 7f77c6ffd640 10 client.6951 handle_command_reply: tid=0 2024-09-21T00:29:29.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.939+0000 7f77f5279740 1 client.6951 shutdown 2024-09-21T00:29:29.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.939+0000 7f77f5279740 1 -- 172.21.15.112:0/1351317146 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563a6bb27590 msgr2=0x563a6bf081f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:29.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.939+0000 7f77f5279740 1 --2- 172.21.15.112:0/1351317146 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563a6bb27590 0x563a6bf081f0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f77dc011570 tx=0x7f77dc011970 comp rx=0 tx=0).stop 2024-09-21T00:29:29.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.939+0000 7f77f5279740 1 -- 172.21.15.112:0/1351317146 shutdown_connections 2024-09-21T00:29:29.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.939+0000 7f77f5279740 1 --2- 172.21.15.112:0/1351317146 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x563a6bb48270 0x563a6bf37140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:29.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.939+0000 7f77f5279740 1 --2- 172.21.15.112:0/1351317146 >> [v2:172.21.15.112:6833/2797048793,v1:172.21.15.112:6835/2797048793] conn(0x563a6bb06e40 0x563a6bf47440 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:29.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.940+0000 7f77f5279740 1 --2- 172.21.15.112:0/1351317146 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563a6bb27590 0x563a6bf081f0 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:29.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.940+0000 7f77f5279740 1 --2- 172.21.15.112:0/1351317146 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563a6bb1f3b0 0x563a6bf35460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:29.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.940+0000 7f77c6ffd640 0 client.6951 ms_handle_reset on v2:172.21.15.112:6833/2797048793 2024-09-21T00:29:29.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.940+0000 7f77f5279740 1 -- 172.21.15.112:0/1351317146 >> 172.21.15.112:0/1351317146 conn(0x563a6bb3c3c0 msgr2=0x563a6bf369b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:29.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.940+0000 7f77f5279740 1 -- 172.21.15.112:0/1351317146 shutdown_connections 2024-09-21T00:29:29.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.940+0000 7f77f5279740 1 -- 172.21.15.112:0/1351317146 wait complete. 2024-09-21T00:29:29.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.940+0000 7f77f5279740 20 client.6951 trim_cache size 0 max 16384 2024-09-21T00:29:29.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.958+0000 7f77f5279740 1 Processor -- start 2024-09-21T00:29:29.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.958+0000 7f77f5279740 1 -- start start 2024-09-21T00:29:29.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.959+0000 7f77f5279740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563a6bb1f3b0 0x563a6bf36a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:29.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.959+0000 7f77f5279740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563a6bb27590 0x563a6bf34ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:29.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.959+0000 7f77f5279740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x563a6bb48270 0x563a6bf35220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:29.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.959+0000 7f77f5279740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f77e8201ad0 con 0x563a6bb1f3b0 2024-09-21T00:29:29.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.959+0000 7f77f5279740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f77e81f9ab0 con 0x563a6bb27590 2024-09-21T00:29:29.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.959+0000 7f77f5279740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f77e8201480 con 0x563a6bb48270 2024-09-21T00:29:29.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.959+0000 7f77e7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563a6bb1f3b0 0x563a6bf36a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:29.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.959+0000 7f77e7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563a6bb1f3b0 0x563a6bf36a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57862/0 (socket says 172.21.15.112:57862) 2024-09-21T00:29:29.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.959+0000 7f77e7fff640 1 -- 172.21.15.112:0/27372423 learned_addr learned my addr 172.21.15.112:0/27372423 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:29:29.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.960+0000 7f77e7fff640 1 -- 172.21.15.112:0/27372423 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x563a6bb48270 msgr2=0x563a6bf35220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:29:29.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.960+0000 7f77e7fff640 1 --2- 172.21.15.112:0/27372423 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x563a6bb48270 0x563a6bf35220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:29.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.960+0000 7f77e7fff640 1 -- 172.21.15.112:0/27372423 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563a6bb27590 msgr2=0x563a6bf34ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:29:29.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.960+0000 7f77e7fff640 1 --2- 172.21.15.112:0/27372423 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563a6bb27590 0x563a6bf34ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:29.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.960+0000 7f77e7fff640 1 -- 172.21.15.112:0/27372423 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a6beeaa70 con 0x563a6bb1f3b0 2024-09-21T00:29:29.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.960+0000 7f77e7fff640 1 --2- 172.21.15.112:0/27372423 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563a6bb1f3b0 0x563a6bf36a80 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f77d802a450 tx=0x7f77d80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c175e57140a7773 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:29.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.960+0000 7f77cd7fa640 1 -- 172.21.15.112:0/27372423 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77d8040040 con 0x563a6bb1f3b0 2024-09-21T00:29:29.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.960+0000 7f77cd7fa640 1 -- 172.21.15.112:0/27372423 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f77d80039a0 con 0x563a6bb1f3b0 2024-09-21T00:29:29.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.960+0000 7f77cd7fa640 1 -- 172.21.15.112:0/27372423 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77d8003bb0 con 0x563a6bb1f3b0 2024-09-21T00:29:29.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.961+0000 7f77f5279740 1 -- 172.21.15.112:0/27372423 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563a6bb1f3b0 msgr2=0x563a6bf36a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:29.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.961+0000 7f77f5279740 1 --2- 172.21.15.112:0/27372423 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563a6bb1f3b0 0x563a6bf36a80 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f77d802a450 tx=0x7f77d80046c0 comp rx=0 tx=0).stop 2024-09-21T00:29:29.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.961+0000 7f77f5279740 1 -- 172.21.15.112:0/27372423 shutdown_connections 2024-09-21T00:29:29.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.961+0000 7f77f5279740 1 --2- 172.21.15.112:0/27372423 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x563a6bb48270 0x563a6bf35220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:29.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.961+0000 7f77f5279740 1 --2- 172.21.15.112:0/27372423 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563a6bb27590 0x563a6bf34ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:29.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.961+0000 7f77f5279740 1 --2- 172.21.15.112:0/27372423 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563a6bb1f3b0 0x563a6bf36a80 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:29.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.961+0000 7f77f5279740 1 -- 172.21.15.112:0/27372423 >> 172.21.15.112:0/27372423 conn(0x563a6bb3c3c0 msgr2=0x563a6bf09920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:29.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.961+0000 7f77f5279740 1 -- 172.21.15.112:0/27372423 shutdown_connections 2024-09-21T00:29:29.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.961+0000 7f77f5279740 1 -- 172.21.15.112:0/27372423 wait complete. 2024-09-21T00:29:29.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.962+0000 7f77f5279740 1 -- start start 2024-09-21T00:29:29.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.962+0000 7f77f5279740 1 Processor -- start 2024-09-21T00:29:29.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.963+0000 7f77f5279740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563a6bb1f3b0 0x563a6bf346b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:29.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.963+0000 7f77f5279740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563a6bb27590 0x563a6bf4c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:29.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.963+0000 7f77f5279740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x563a6bb48270 0x563a6bf40ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:29.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.963+0000 7f77f5279740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x563a6bf37860 con 0x563a6bb27590 2024-09-21T00:29:29.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.963+0000 7f77f5279740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x563a6bf44d50 con 0x563a6bb1f3b0 2024-09-21T00:29:29.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.963+0000 7f77f5279740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x563a6bf2f560 con 0x563a6bb48270 2024-09-21T00:29:29.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.963+0000 7f77f5279740 10 client.0 _ll_register_callbacks cb 0x7f77dc009d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:29:29.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.963+0000 7f77e77fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563a6bb27590 0x563a6bf4c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:29.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.963+0000 7f77e77fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563a6bb27590 0x563a6bf4c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57870/0 (socket says 172.21.15.112:57870) 2024-09-21T00:29:29.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.963+0000 7f77e77fe640 1 -- 172.21.15.112:0/4223116375 learned_addr learned my addr 172.21.15.112:0/4223116375 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:29:29.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.963+0000 7f77e77fe640 1 -- 172.21.15.112:0/4223116375 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x563a6bb48270 msgr2=0x563a6bf40ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:29:29.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.963+0000 7f77e77fe640 1 --2- 172.21.15.112:0/4223116375 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x563a6bb48270 0x563a6bf40ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:29.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.963+0000 7f77e7fff640 1 --2- 172.21.15.112:0/4223116375 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563a6bb1f3b0 0x563a6bf346b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:29.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.963+0000 7f77e77fe640 1 -- 172.21.15.112:0/4223116375 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563a6bb1f3b0 msgr2=0x563a6bf346b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:29:29.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.964+0000 7f77e77fe640 1 --2- 172.21.15.112:0/4223116375 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563a6bb1f3b0 0x563a6bf346b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:29.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.964+0000 7f77e77fe640 1 -- 172.21.15.112:0/4223116375 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a6bf2b2c0 con 0x563a6bb27590 2024-09-21T00:29:29.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.964+0000 7f77e77fe640 1 --2- 172.21.15.112:0/4223116375 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563a6bb27590 0x563a6bf4c9c0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f77dc014460 tx=0x7f77dc01d630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=248b1c29bfa00947 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:29.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.964+0000 7f77c6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T00:29:29.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.964+0000 7f77c6ffd640 1 -- 172.21.15.112:0/4223116375 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77dc02b060 con 0x563a6bb27590 2024-09-21T00:29:29.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.964+0000 7f77c6ffd640 1 -- 172.21.15.112:0/4223116375 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 6188+0+0 (secure 0 0 0) 0x7f77dc004440 con 0x563a6bb27590 2024-09-21T00:29:29.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.964+0000 7f77f5279740 1 -- 172.21.15.112:0/4223116375 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f77d80039a0 con 0x563a6bb27590 2024-09-21T00:29:29.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.964+0000 7f77c6ffd640 1 -- 172.21.15.112:0/4223116375 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f77dc031050 con 0x563a6bb27590 2024-09-21T00:29:29.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.964+0000 7f77f5279740 10 client.6957 fetch_fsmap learned FSMap version 64 2024-09-21T00:29:29.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.964+0000 7f77f5279740 1 -- 172.21.15.112:0/4223116375 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=2+}) -- 0x563a6beef270 con 0x563a6bb27590 2024-09-21T00:29:29.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.964+0000 7f77c6ffd640 1 -- 172.21.15.112:0/4223116375 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f77d80039a0 con 0x563a6bb27590 2024-09-21T00:29:29.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.964+0000 7f77c6ffd640 1 -- 172.21.15.112:0/4223116375 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 64) ==== 3502+0+0 (secure 0 0 0) 0x7f77dc034020 con 0x563a6bb27590 2024-09-21T00:29:29.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.964+0000 7f77f5279740 10 client.6957 fetch_fsmap finished waiting for FSMap version 64 2024-09-21T00:29:29.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.964+0000 7f77f5279740 10 client.6957 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T00:29:29.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.964+0000 7f77f5279740 20 client.6957 populate_metadata read hostname 'smithi112' 2024-09-21T00:29:29.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.964+0000 7f77f5279740 1 --2- 172.21.15.112:0/4223116375 >> [v2:172.21.15.112:6833/2797048793,v1:172.21.15.112:6835/2797048793] conn(0x563a6bfb7940 0x563a6bfb7d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:29.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.964+0000 7f77f5279740 4 client.6957 mds_command: new command op to 6804 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T00:29:29.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.964+0000 7f77f5279740 1 -- 172.21.15.112:0/4223116375 --> [v2:172.21.15.112:6833/2797048793,v1:172.21.15.112:6835/2797048793] -- command(tid 0: {"prefix": "status"}) -- 0x563a6be60220 con 0x563a6bfb7940 2024-09-21T00:29:29.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.964+0000 7f77e7fff640 1 --2- 172.21.15.112:0/4223116375 >> [v2:172.21.15.112:6833/2797048793,v1:172.21.15.112:6835/2797048793] conn(0x563a6bfb7940 0x563a6bfb7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:29.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.965+0000 7f77c6ffd640 10 client.6957 ms_handle_connect on v2:172.21.15.112:6833/2797048793 2024-09-21T00:29:29.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.965+0000 7f77e7fff640 1 --2- 172.21.15.112:0/4223116375 >> [v2:172.21.15.112:6833/2797048793,v1:172.21.15.112:6835/2797048793] conn(0x563a6bfb7940 0x563a6bfb7d50 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c32a2146f95b2c7 server_cookie=a2b3ee691b06e36 in_seq=0 out_seq=0 2024-09-21T00:29:29.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.965+0000 7f77c6ffd640 1 -- 172.21.15.112:0/4223116375 <== mds.0 v2:172.21.15.112:6833/2797048793 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x563a6be60220 con 0x563a6bfb7940 2024-09-21T00:29:29.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.965+0000 7f77c6ffd640 10 client.6957 handle_command_reply: tid=0 2024-09-21T00:29:29.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.965+0000 7f77f5279740 1 client.6957 shutdown 2024-09-21T00:29:29.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.965+0000 7f77f5279740 1 -- 172.21.15.112:0/4223116375 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563a6bb27590 msgr2=0x563a6bf4c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:29.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.965+0000 7f77f5279740 1 --2- 172.21.15.112:0/4223116375 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563a6bb27590 0x563a6bf4c9c0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f77dc014460 tx=0x7f77dc01d630 comp rx=0 tx=0).stop 2024-09-21T00:29:29.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.965+0000 7f77f5279740 1 -- 172.21.15.112:0/4223116375 shutdown_connections 2024-09-21T00:29:29.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.965+0000 7f77f5279740 1 --2- 172.21.15.112:0/4223116375 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x563a6bb48270 0x563a6bf40ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:29.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.965+0000 7f77f5279740 1 --2- 172.21.15.112:0/4223116375 >> [v2:172.21.15.112:6833/2797048793,v1:172.21.15.112:6835/2797048793] conn(0x563a6bfb7940 0x563a6bfb7d50 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:29.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.965+0000 7f77f5279740 1 --2- 172.21.15.112:0/4223116375 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563a6bb27590 0x563a6bf4c9c0 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:29.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.965+0000 7f77f5279740 1 --2- 172.21.15.112:0/4223116375 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563a6bb1f3b0 0x563a6bf346b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:29.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.965+0000 7f77c6ffd640 0 client.6957 ms_handle_reset on v2:172.21.15.112:6833/2797048793 2024-09-21T00:29:29.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.965+0000 7f77f5279740 1 -- 172.21.15.112:0/4223116375 >> 172.21.15.112:0/4223116375 conn(0x563a6bb3c3c0 msgr2=0x563a6bf09920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:29.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.965+0000 7f77f5279740 1 -- 172.21.15.112:0/4223116375 shutdown_connections 2024-09-21T00:29:29.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.965+0000 7f77f5279740 1 -- 172.21.15.112:0/4223116375 wait complete. 2024-09-21T00:29:29.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.966+0000 7f77f5279740 20 client.6957 trim_cache size 0 max 16384 2024-09-21T00:29:29.973 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-21T00:29:29.973 INFO:teuthology.orchestra.run.smithi112.stdout: "cluster_fsid": "8b34692b-dc65-49b2-a08f-26f707b96ac5", 2024-09-21T00:29:29.973 INFO:teuthology.orchestra.run.smithi112.stdout: "whoami": 0, 2024-09-21T00:29:29.973 INFO:teuthology.orchestra.run.smithi112.stdout: "id": 6804, 2024-09-21T00:29:29.973 INFO:teuthology.orchestra.run.smithi112.stdout: "want_state": "up:active", 2024-09-21T00:29:29.973 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "up:active", 2024-09-21T00:29:29.973 INFO:teuthology.orchestra.run.smithi112.stdout: "fs_name": "cephfs", 2024-09-21T00:29:29.973 INFO:teuthology.orchestra.run.smithi112.stdout: "rank_uptime": 10.168873937000001, 2024-09-21T00:29:29.973 INFO:teuthology.orchestra.run.smithi112.stdout: "mdsmap_epoch": 64, 2024-09-21T00:29:29.973 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch": 96, 2024-09-21T00:29:29.973 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch_barrier": 94, 2024-09-21T00:29:29.973 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 14.847007244 2024-09-21T00:29:29.973 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-21T00:29:29.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.966+0000 7f77ee312640 1 -- 172.21.15.112:0/2480098479 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f77d4078b90 msgr2=0x7f77d407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:29.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.966+0000 7f77ee312640 1 --2- 172.21.15.112:0/2480098479 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f77d4078b90 0x7f77d407b0a0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f77d8033000 tx=0x7f77d8004730 comp rx=0 tx=0).stop 2024-09-21T00:29:29.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.966+0000 7f77ee312640 1 -- 172.21.15.112:0/2480098479 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77e80ccbf0 msgr2=0x7f77e80cbfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:29.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.966+0000 7f77ee312640 1 --2- 172.21.15.112:0/2480098479 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77e80ccbf0 0x7f77e80cbfc0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f77e000cab0 tx=0x7f77e000cf70 comp rx=0 tx=0).stop 2024-09-21T00:29:29.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.966+0000 7f77ee312640 1 -- 172.21.15.112:0/2480098479 shutdown_connections 2024-09-21T00:29:29.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.966+0000 7f77ee312640 1 --2- 172.21.15.112:0/2480098479 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f77d4078b90 0x7f77d407b0a0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:29.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.966+0000 7f77ee312640 1 --2- 172.21.15.112:0/2480098479 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77e80ccbf0 0x7f77e80cbfc0 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:29.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.966+0000 7f77ee312640 1 --2- 172.21.15.112:0/2480098479 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f77e80c6a00 0x7f77e8156390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:29.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.966+0000 7f77ee312640 1 --2- 172.21.15.112:0/2480098479 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f77e80c5ee0 0x7f77e80c9d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:29.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.966+0000 7f77ee312640 1 -- 172.21.15.112:0/2480098479 >> 172.21.15.112:0/2480098479 conn(0x7f77e80b42e0 msgr2=0x7f77e8072010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:29.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.966+0000 7f77ee312640 1 -- 172.21.15.112:0/2480098479 shutdown_connections 2024-09-21T00:29:29.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:29.966+0000 7f77ee312640 1 -- 172.21.15.112:0/2480098479 wait complete. 2024-09-21T00:29:29.979 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T00:29:30.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:30.090+0000 7f803858b640 1 -- 172.21.15.112:0/543134338 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80300d3880 msgr2=0x7f8030163260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:30.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:30.090+0000 7f803858b640 1 --2- 172.21.15.112:0/543134338 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80300d3880 0x7f8030163260 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f80300d2050 tx=0x7f802002a5f0 comp rx=0 tx=0).stop 2024-09-21T00:29:30.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:30.090+0000 7f803858b640 1 -- 172.21.15.112:0/543134338 shutdown_connections 2024-09-21T00:29:30.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:30.090+0000 7f803858b640 1 --2- 172.21.15.112:0/543134338 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80300d3880 0x7f8030163260 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:30.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:30.090+0000 7f803858b640 1 --2- 172.21.15.112:0/543134338 >> v2:172.21.15.187:3300/0 conn(0x7f80300d48a0 0x7f80300d3dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:30.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:30.090+0000 7f803858b640 1 -- 172.21.15.112:0/543134338 >> 172.21.15.112:0/543134338 conn(0x7f803008bfe0 msgr2=0x7f803009d890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:30.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:30.090+0000 7f803858b640 1 -- 172.21.15.112:0/543134338 shutdown_connections 2024-09-21T00:29:30.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:30.090+0000 7f803858b640 1 -- 172.21.15.112:0/543134338 wait complete. 2024-09-21T00:29:30.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:30.092+0000 7f803858b640 1 Processor -- start 2024-09-21T00:29:30.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:30.092+0000 7f803858b640 1 -- start start 2024-09-21T00:29:30.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:30.093+0000 7f803858b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f80300d01b0 0x7f80300ce450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:30.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:30.093+0000 7f803858b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f80300d3880 0x7f80300cbdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:30.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:30.093+0000 7f803858b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80300d48a0 0x7f803006efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:30.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:30.093+0000 7f803858b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f80300d2460 con 0x7f80300d48a0 2024-09-21T00:29:30.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:30.093+0000 7f803858b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f80300d22e0 con 0x7f80300d01b0 2024-09-21T00:29:30.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:30.093+0000 7f803858b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f80300d25e0 con 0x7f80300d3880 2024-09-21T00:29:30.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:30.093+0000 7f8036b28640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80300d48a0 0x7f803006efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:30.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:30.093+0000 7f8036b28640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80300d48a0 0x7f803006efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57898/0 (socket says 172.21.15.112:57898) 2024-09-21T00:29:30.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:30.093+0000 7f8036b28640 1 -- 172.21.15.112:0/4136148313 learned_addr learned my addr 172.21.15.112:0/4136148313 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:29:30.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:30.093+0000 7f8036327640 1 --2- 172.21.15.112:0/4136148313 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f80300d01b0 0x7f80300ce450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:30.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:30.093+0000 7f8035b26640 1 --2- 172.21.15.112:0/4136148313 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f80300d3880 0x7f80300cbdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:30.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:30.094+0000 7f8036b28640 1 -- 172.21.15.112:0/4136148313 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f80300d3880 msgr2=0x7f80300cbdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:30.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:30.094+0000 7f8036b28640 1 --2- 172.21.15.112:0/4136148313 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f80300d3880 0x7f80300cbdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:30.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:30.094+0000 7f8036b28640 1 -- 172.21.15.112:0/4136148313 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f80300d01b0 msgr2=0x7f80300ce450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:30.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:30.094+0000 7f8036b28640 1 --2- 172.21.15.112:0/4136148313 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f80300d01b0 0x7f80300ce450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:30.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:30.094+0000 7f8036b28640 1 -- 172.21.15.112:0/4136148313 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8030160af0 con 0x7f80300d48a0 2024-09-21T00:29:30.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:30.094+0000 7f8036b28640 1 --2- 172.21.15.112:0/4136148313 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80300d48a0 0x7f803006efb0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f802800c6b0 tx=0x7f802800cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:30.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:30.094+0000 7f8026ffd640 1 -- 172.21.15.112:0/4136148313 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8028003b70 con 0x7f80300d48a0 2024-09-21T00:29:30.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:30.094+0000 7f8026ffd640 1 -- 172.21.15.112:0/4136148313 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8028003d30 con 0x7f80300d48a0 2024-09-21T00:29:30.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:30.094+0000 7f803858b640 1 -- 172.21.15.112:0/4136148313 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8030160870 con 0x7f80300d48a0 2024-09-21T00:29:30.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:30.094+0000 7f8026ffd640 1 -- 172.21.15.112:0/4136148313 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8028005330 con 0x7f80300d48a0 2024-09-21T00:29:30.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:30.094+0000 7f803858b640 1 -- 172.21.15.112:0/4136148313 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80301f5f60 con 0x7f80300d48a0 2024-09-21T00:29:30.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:30.095+0000 7f803858b640 1 -- 172.21.15.112:0/4136148313 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8030160cc0 con 0x7f80300d48a0 2024-09-21T00:29:30.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:30.096+0000 7f8026ffd640 1 -- 172.21.15.112:0/4136148313 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f80280054d0 con 0x7f80300d48a0 2024-09-21T00:29:30.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:30.096+0000 7f8026ffd640 1 --2- 172.21.15.112:0/4136148313 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f8014078b90 0x7f801407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:30.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:30.096+0000 7f8026ffd640 1 -- 172.21.15.112:0/4136148313 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 6188+0+0 (secure 0 0 0) 0x7f802809abc0 con 0x7f80300d48a0 2024-09-21T00:29:30.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:30.097+0000 7f8036327640 1 --2- 172.21.15.112:0/4136148313 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f8014078b90 0x7f801407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:30.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:30.097+0000 7f8036327640 1 --2- 172.21.15.112:0/4136148313 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f8014078b90 0x7f801407b0a0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f802c00bb50 tx=0x7f802c00b2e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:30.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:30.101+0000 7f8026ffd640 1 -- 172.21.15.112:0/4136148313 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f80280627d0 con 0x7f80300d48a0 2024-09-21T00:29:30.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:30.317+0000 7f803858b640 1 -- 172.21.15.112:0/4136148313 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f80300d2a90 con 0x7f80300d48a0 2024-09-21T00:29:30.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:30.317+0000 7f8026ffd640 1 -- 172.21.15.112:0/4136148313 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v64) ==== 53+0+73 (secure 0 0 0) 0x7f8028067200 con 0x7f80300d48a0 2024-09-21T00:29:30.319 INFO:teuthology.orchestra.run.smithi112.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T00:29:30.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:30.319+0000 7f803858b640 1 -- 172.21.15.112:0/4136148313 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f8014078b90 msgr2=0x7f801407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:30.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:30.319+0000 7f803858b640 1 --2- 172.21.15.112:0/4136148313 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f8014078b90 0x7f801407b0a0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f802c00bb50 tx=0x7f802c00b2e0 comp rx=0 tx=0).stop 2024-09-21T00:29:30.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:30.319+0000 7f803858b640 1 -- 172.21.15.112:0/4136148313 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80300d48a0 msgr2=0x7f803006efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:30.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:30.319+0000 7f803858b640 1 --2- 172.21.15.112:0/4136148313 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80300d48a0 0x7f803006efb0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f802800c6b0 tx=0x7f802800cb70 comp rx=0 tx=0).stop 2024-09-21T00:29:30.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:30.320+0000 7f803858b640 1 -- 172.21.15.112:0/4136148313 shutdown_connections 2024-09-21T00:29:30.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:30.320+0000 7f803858b640 1 --2- 172.21.15.112:0/4136148313 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f8014078b90 0x7f801407b0a0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:30.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:30.320+0000 7f803858b640 1 --2- 172.21.15.112:0/4136148313 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80300d48a0 0x7f803006efb0 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:30.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:30.320+0000 7f803858b640 1 --2- 172.21.15.112:0/4136148313 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f80300d3880 0x7f80300cbdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:30.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:30.320+0000 7f803858b640 1 --2- 172.21.15.112:0/4136148313 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f80300d01b0 0x7f80300ce450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:30.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:30.320+0000 7f803858b640 1 -- 172.21.15.112:0/4136148313 >> 172.21.15.112:0/4136148313 conn(0x7f803008bfe0 msgr2=0x7f80300d05c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:30.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:30.320+0000 7f803858b640 1 -- 172.21.15.112:0/4136148313 shutdown_connections 2024-09-21T00:29:30.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:30.320+0000 7f803858b640 1 -- 172.21.15.112:0/4136148313 wait complete. 2024-09-21T00:29:30.333 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T00:29:30.333 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T00:29:30.333 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi112.front.sepia.ceph.com 2024-09-21T00:29:30.333 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T00:29:30.333 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T00:29:30.333 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T00:29:30.333 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-21T00:29:30.333 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T00:29:30.333 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T00:29:30.333 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:29:30.333 DEBUG:teuthology.orchestra.run.smithi112:> ip addr 2024-09-21T00:29:30.349 INFO:teuthology.orchestra.run.smithi112.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T00:29:30.349 INFO:teuthology.orchestra.run.smithi112.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T00:29:30.349 INFO:teuthology.orchestra.run.smithi112.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T00:29:30.349 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:29:30.349 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 ::1/128 scope host 2024-09-21T00:29:30.349 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:29:30.349 INFO:teuthology.orchestra.run.smithi112.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T00:29:30.349 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:d8:a5:2c brd ff:ff:ff:ff:ff:ff 2024-09-21T00:29:30.349 INFO:teuthology.orchestra.run.smithi112.stdout: altname enp7s0f0 2024-09-21T00:29:30.349 INFO:teuthology.orchestra.run.smithi112.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T00:29:30.350 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:d8:a5:2d brd ff:ff:ff:ff:ff:ff 2024-09-21T00:29:30.350 INFO:teuthology.orchestra.run.smithi112.stdout: altname enp7s0f1 2024-09-21T00:29:30.350 INFO:teuthology.orchestra.run.smithi112.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T00:29:30.350 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:8f:cb:68 brd ff:ff:ff:ff:ff:ff 2024-09-21T00:29:30.350 INFO:teuthology.orchestra.run.smithi112.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T00:29:30.350 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:8f:cb:69 brd ff:ff:ff:ff:ff:ff 2024-09-21T00:29:30.350 INFO:teuthology.orchestra.run.smithi112.stdout: inet 172.21.15.112/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-21T00:29:30.350 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft 42552sec preferred_lft 42552sec 2024-09-21T00:29:30.350 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::ec4:7aff:fe8f:cb69/64 scope link noprefixroute 2024-09-21T00:29:30.350 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:29:30.350 INFO:teuthology.orchestra.run.smithi112.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T00:29:30.350 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 1a:39:64:e7:fe:d7 brd ff:ff:ff:ff:ff:ff 2024-09-21T00:29:30.350 INFO:teuthology.orchestra.run.smithi112.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T00:29:30.351 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:29:30.351 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::5858:3dff:feaf:1774/64 scope link 2024-09-21T00:29:30.351 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:29:30.351 INFO:teuthology.orchestra.run.smithi112.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T00:29:30.351 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 1a:39:64:e7:fe:d7 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T00:29:30.351 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::1839:64ff:fee7:fed7/64 scope link 2024-09-21T00:29:30.351 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:29:30.351 INFO:teuthology.orchestra.run.smithi112.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T00:29:30.351 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 86:c9:6c:ef:e4:e9 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T00:29:30.351 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::84c9:6cff:feef:e4e9/64 scope link 2024-09-21T00:29:30.351 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:29:30.351 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:29:30.352 DEBUG:teuthology.orchestra.run.smithi112:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T00:29:30.412 INFO:teuthology.orchestra.run.smithi112.stdout:1 2024-09-21T00:29:30.414 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:29:30.414 DEBUG:teuthology.orchestra.run.smithi112:> route 2024-09-21T00:29:30.470 INFO:teuthology.orchestra.run.smithi112.stdout:Kernel IP routing table 2024-09-21T00:29:30.470 INFO:teuthology.orchestra.run.smithi112.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T00:29:30.471 INFO:teuthology.orchestra.run.smithi112.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-21T00:29:30.471 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-21T00:29:30.471 INFO:teuthology.orchestra.run.smithi112.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T00:29:30.471 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T00:29:30.471 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2024-09-21T00:29:30.472 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-21T00:29:30.472 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-21T00:29:30.472 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-21T00:29:30.472 DEBUG:teuthology.orchestra.run.smithi112:> ') 2024-09-21T00:29:30.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:30 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T00:29:30.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T00:29:30.596 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:29:30.596 DEBUG:teuthology.orchestra.run.smithi112:> ip netns list 2024-09-21T00:29:30.611 INFO:teuthology.orchestra.run.smithi112.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-21T00:29:30.611 INFO:teuthology.orchestra.run.smithi112.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T00:29:30.611 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T00:29:30.611 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:29:30.612 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link set brx.0 up 2024-09-21T00:29:30.676 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T00:29:30.676 DEBUG:teuthology.orchestra.run.smithi112:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T00:29:30.731 INFO:teuthology.orchestra.run.smithi112.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T00:29:30.731 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T00:29:30.732 DEBUG:teuthology.orchestra.run.smithi112:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T00:29:30.786 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-21T00:29:30.786 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:29:30.786 DEBUG:teuthology.orchestra.run.smithi112:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T00:29:30.880 INFO:teuthology.orchestra.run.smithi112.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T00:29:30.880 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T00:29:30.881 INFO:teuthology.orchestra.run.smithi112.stdout:invalid new device string format 2024-09-21T00:29:30.891 INFO:teuthology.orchestra.run.smithi112.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T00:29:30.891 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T00:29:30.891 INFO:teuthology.orchestra.run.smithi112.stdout:invalid new device string format 2024-09-21T00:29:30.891 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: resolved to: "172.21.15.112:3300,172.21.15.187:3300,172.21.15.187:3301" 2024-09-21T00:29:30.891 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T00:29:30.898 INFO:teuthology.orchestra.run.smithi112.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T00:29:30.898 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T00:29:30.898 INFO:teuthology.orchestra.run.smithi112.stdout:invalid new device string format 2024-09-21T00:29:30.899 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: resolved to: "172.21.15.112:3300,172.21.15.187:3300,172.21.15.187:3301" 2024-09-21T00:29:30.899 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T00:29:30.899 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: resolved to: "172.21.15.112:6789,172.21.15.187:6789,172.21.15.187:6790" 2024-09-21T00:29:30.899 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.112:6789,172.21.15.187:6789,172.21.15.187:6790:/ 2024-09-21T00:29:30.899 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=8b34692b-dc65-49b2-a08f-26f707b96ac5" will pass to kernel 2024-09-21T00:29:30.899 INFO:teuthology.orchestra.run.smithi112.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-21T00:29:30.899 INFO:teuthology.orchestra.run.smithi112.stdout: You just mounted a file system that supports labels which does not 2024-09-21T00:29:30.900 INFO:teuthology.orchestra.run.smithi112.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T00:29:30.900 INFO:teuthology.orchestra.run.smithi112.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T00:29:30.900 INFO:teuthology.orchestra.run.smithi112.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T00:29:30.901 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T00:29:30.901 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:29:30.901 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T00:29:30.926 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T00:29:31.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:31 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T00:29:31.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T00:29:31.027 INFO:teuthology.orchestra.run.smithi112.stdout:client5888 2024-09-21T00:29:31.027 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T00:29:31.027 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:29:31.043 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T00:29:31.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:31.194+0000 7f04c17ba640 1 -- 172.21.15.112:0/3307208929 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04a4004db0 msgr2=0x7f04a4005280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:31.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:31.194+0000 7f04c17ba640 1 --2- 172.21.15.112:0/3307208929 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04a4004db0 0x7f04a4005280 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f04bc0b80f0 tx=0x7f04a4005830 comp rx=0 tx=0).stop 2024-09-21T00:29:31.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:31.194+0000 7f04bb7fe640 1 -- 172.21.15.112:0/3307208929 reap_dead start 2024-09-21T00:29:31.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:31.194+0000 7f04c17ba640 1 -- 172.21.15.112:0/3307208929 shutdown_connections 2024-09-21T00:29:31.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:31.194+0000 7f04c17ba640 1 -- 172.21.15.112:0/3307208929 >> 172.21.15.112:0/3307208929 conn(0x7f04bc0b4110 msgr2=0x7f04bc0b4520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:31.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:31.194+0000 7f04c17ba640 1 -- 172.21.15.112:0/3307208929 shutdown_connections 2024-09-21T00:29:31.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:31.194+0000 7f04c17ba640 1 -- 172.21.15.112:0/3307208929 wait complete. 2024-09-21T00:29:31.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:31.196+0000 7f04c17ba640 1 Processor -- start 2024-09-21T00:29:31.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:31.196+0000 7f04c17ba640 1 -- start start 2024-09-21T00:29:31.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:31.197+0000 7f04c17ba640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04a4004db0 0x7f04bc154260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:31.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:31.197+0000 7f04c17ba640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04bc150820 0x7f04bc0c7550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:31.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:31.197+0000 7f04c17ba640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04bc0c7a90 0x7f04bc151c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:31.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:31.197+0000 7f04c17ba640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f04bc15cce0 con 0x7f04bc150820 2024-09-21T00:29:31.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:31.197+0000 7f04c17ba640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f04bc15cb60 con 0x7f04a4004db0 2024-09-21T00:29:31.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:31.198+0000 7f04c17ba640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f04bc0b5230 con 0x7f04bc0c7a90 2024-09-21T00:29:31.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:31.198+0000 7f04ba7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04bc150820 0x7f04bc0c7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:31.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:31.198+0000 7f04ba7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04bc150820 0x7f04bc0c7550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57920/0 (socket says 172.21.15.112:57920) 2024-09-21T00:29:31.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:31.198+0000 7f04ba7fc640 1 -- 172.21.15.112:0/62826657 learned_addr learned my addr 172.21.15.112:0/62826657 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:29:31.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:31.198+0000 7f04bb7fe640 1 --2- 172.21.15.112:0/62826657 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04bc0c7a90 0x7f04bc151c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:31.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:31.198+0000 7f04ba7fc640 1 -- 172.21.15.112:0/62826657 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04bc0c7a90 msgr2=0x7f04bc151c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:31.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:31.198+0000 7f04ba7fc640 1 --2- 172.21.15.112:0/62826657 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04bc0c7a90 0x7f04bc151c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:31.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:31.198+0000 7f04baffd640 1 --2- 172.21.15.112:0/62826657 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04a4004db0 0x7f04bc154260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:31.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:31.198+0000 7f04ba7fc640 1 -- 172.21.15.112:0/62826657 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04a4004db0 msgr2=0x7f04bc154260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:31.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:31.198+0000 7f04ba7fc640 1 --2- 172.21.15.112:0/62826657 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04a4004db0 0x7f04bc154260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:31.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:31.198+0000 7f04ba7fc640 1 -- 172.21.15.112:0/62826657 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04bc0c7ea0 con 0x7f04bc150820 2024-09-21T00:29:31.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:31.198+0000 7f04ba7fc640 1 --2- 172.21.15.112:0/62826657 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04bc150820 0x7f04bc0c7550 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f04b000fa00 tx=0x7f04b000fec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:31.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:31.198+0000 7f04a37fe640 1 -- 172.21.15.112:0/62826657 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b0015070 con 0x7f04bc150820 2024-09-21T00:29:31.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:31.198+0000 7f04a37fe640 1 -- 172.21.15.112:0/62826657 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f04b000b960 con 0x7f04bc150820 2024-09-21T00:29:31.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:31.198+0000 7f04c17ba640 1 -- 172.21.15.112:0/62826657 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04bc0c9d60 con 0x7f04bc150820 2024-09-21T00:29:31.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:31.198+0000 7f04a37fe640 1 -- 172.21.15.112:0/62826657 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b000bd70 con 0x7f04bc150820 2024-09-21T00:29:31.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:31.198+0000 7f04c17ba640 1 -- 172.21.15.112:0/62826657 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04bc151470 con 0x7f04bc150820 2024-09-21T00:29:31.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:31.199+0000 7f04c17ba640 1 -- 172.21.15.112:0/62826657 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04bc151260 con 0x7f04bc150820 2024-09-21T00:29:31.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:31.200+0000 7f04a37fe640 1 -- 172.21.15.112:0/62826657 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f04b00035f0 con 0x7f04bc150820 2024-09-21T00:29:31.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:31.201+0000 7f04a37fe640 1 --2- 172.21.15.112:0/62826657 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f04a8078bc0 0x7f04a807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:31.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:31.201+0000 7f04a37fe640 1 -- 172.21.15.112:0/62826657 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 6188+0+0 (secure 0 0 0) 0x7f04b009baf0 con 0x7f04bc150820 2024-09-21T00:29:31.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:31.201+0000 7f04baffd640 1 --2- 172.21.15.112:0/62826657 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f04a8078bc0 0x7f04a807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:31.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:31.202+0000 7f04baffd640 1 --2- 172.21.15.112:0/62826657 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f04a8078bc0 0x7f04a807b0d0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f04bc0747d0 tx=0x7f04ac00b260 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:31.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:31.206+0000 7f04a37fe640 1 -- 172.21.15.112:0/62826657 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f04bc0c7ea0 con 0x7f04bc150820 2024-09-21T00:29:31.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:31.418+0000 7f04c17ba640 1 -- 172.21.15.112:0/62826657 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f04bc0d1c10 con 0x7f04bc150820 2024-09-21T00:29:31.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:31.419+0000 7f04a37fe640 1 -- 172.21.15.112:0/62826657 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v64) ==== 53+0+73 (secure 0 0 0) 0x7f04b0063de0 con 0x7f04bc150820 2024-09-21T00:29:31.420 INFO:teuthology.orchestra.run.smithi112.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T00:29:31.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:31.421+0000 7f04c17ba640 1 -- 172.21.15.112:0/62826657 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f04a8078bc0 msgr2=0x7f04a807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:31.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:31.421+0000 7f04c17ba640 1 --2- 172.21.15.112:0/62826657 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f04a8078bc0 0x7f04a807b0d0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f04bc0747d0 tx=0x7f04ac00b260 comp rx=0 tx=0).stop 2024-09-21T00:29:31.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:31.421+0000 7f04c17ba640 1 -- 172.21.15.112:0/62826657 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04bc150820 msgr2=0x7f04bc0c7550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:31.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:31.421+0000 7f04c17ba640 1 --2- 172.21.15.112:0/62826657 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04bc150820 0x7f04bc0c7550 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f04b000fa00 tx=0x7f04b000fec0 comp rx=0 tx=0).stop 2024-09-21T00:29:31.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:31.421+0000 7f04c17ba640 1 -- 172.21.15.112:0/62826657 shutdown_connections 2024-09-21T00:29:31.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:31.421+0000 7f04c17ba640 1 --2- 172.21.15.112:0/62826657 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04bc0c7a90 0x7f04bc151c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:31.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:31.421+0000 7f04c17ba640 1 --2- 172.21.15.112:0/62826657 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f04a8078bc0 0x7f04a807b0d0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:31.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:31.421+0000 7f04c17ba640 1 --2- 172.21.15.112:0/62826657 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04bc150820 0x7f04bc0c7550 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:31.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:31.421+0000 7f04c17ba640 1 --2- 172.21.15.112:0/62826657 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04a4004db0 0x7f04bc154260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:31.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:31.421+0000 7f04c17ba640 1 -- 172.21.15.112:0/62826657 >> 172.21.15.112:0/62826657 conn(0x7f04bc0b4110 msgr2=0x7f04bc157d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:31.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:31.421+0000 7f04c17ba640 1 -- 172.21.15.112:0/62826657 shutdown_connections 2024-09-21T00:29:31.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:31.421+0000 7f04c17ba640 1 -- 172.21.15.112:0/62826657 wait complete. 2024-09-21T00:29:31.433 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T00:29:31.434 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T00:29:31.434 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi187.front.sepia.ceph.com 2024-09-21T00:29:31.434 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-21T00:29:31.434 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-21T00:29:31.434 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T00:29:31.434 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-21T00:29:31.434 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T00:29:31.434 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-21T00:29:31.434 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:29:31.435 DEBUG:teuthology.orchestra.run.smithi187:> ip addr 2024-09-21T00:29:31.454 INFO:teuthology.orchestra.run.smithi187.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T00:29:31.454 INFO:teuthology.orchestra.run.smithi187.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T00:29:31.454 INFO:teuthology.orchestra.run.smithi187.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T00:29:31.454 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:29:31.454 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 ::1/128 scope host 2024-09-21T00:29:31.455 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:29:31.455 INFO:teuthology.orchestra.run.smithi187.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T00:29:31.455 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 0c:c4:7a:d9:36:78 brd ff:ff:ff:ff:ff:ff 2024-09-21T00:29:31.455 INFO:teuthology.orchestra.run.smithi187.stdout: altname enp7s0f0 2024-09-21T00:29:31.455 INFO:teuthology.orchestra.run.smithi187.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T00:29:31.455 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 0c:c4:7a:d9:36:79 brd ff:ff:ff:ff:ff:ff 2024-09-21T00:29:31.455 INFO:teuthology.orchestra.run.smithi187.stdout: altname enp7s0f1 2024-09-21T00:29:31.455 INFO:teuthology.orchestra.run.smithi187.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T00:29:31.455 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 0c:c4:7a:88:77:b0 brd ff:ff:ff:ff:ff:ff 2024-09-21T00:29:31.455 INFO:teuthology.orchestra.run.smithi187.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T00:29:31.455 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 0c:c4:7a:88:77:b1 brd ff:ff:ff:ff:ff:ff 2024-09-21T00:29:31.455 INFO:teuthology.orchestra.run.smithi187.stdout: inet 172.21.15.187/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-21T00:29:31.455 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft 42539sec preferred_lft 42539sec 2024-09-21T00:29:31.455 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 fe80::ec4:7aff:fe88:77b1/64 scope link noprefixroute 2024-09-21T00:29:31.455 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:29:31.456 INFO:teuthology.orchestra.run.smithi187.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T00:29:31.456 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 86:3b:f6:5b:1b:32 brd ff:ff:ff:ff:ff:ff 2024-09-21T00:29:31.456 INFO:teuthology.orchestra.run.smithi187.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T00:29:31.456 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:29:31.456 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 fe80::ac6f:97ff:fe09:b82e/64 scope link 2024-09-21T00:29:31.456 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:29:31.456 INFO:teuthology.orchestra.run.smithi187.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T00:29:31.456 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether ca:0b:a0:f5:2b:b5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T00:29:31.456 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 fe80::c80b:a0ff:fef5:2bb5/64 scope link 2024-09-21T00:29:31.456 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:29:31.456 INFO:teuthology.orchestra.run.smithi187.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T00:29:31.456 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 86:3b:f6:5b:1b:32 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-21T00:29:31.457 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 fe80::843b:f6ff:fe5b:1b32/64 scope link 2024-09-21T00:29:31.457 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:29:31.457 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:29:31.457 DEBUG:teuthology.orchestra.run.smithi187:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T00:29:31.522 INFO:teuthology.orchestra.run.smithi187.stdout:1 2024-09-21T00:29:31.523 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:29:31.523 DEBUG:teuthology.orchestra.run.smithi187:> route 2024-09-21T00:29:31.583 INFO:teuthology.orchestra.run.smithi187.stdout:Kernel IP routing table 2024-09-21T00:29:31.583 INFO:teuthology.orchestra.run.smithi187.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T00:29:31.583 INFO:teuthology.orchestra.run.smithi187.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-21T00:29:31.583 INFO:teuthology.orchestra.run.smithi187.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-21T00:29:31.584 INFO:teuthology.orchestra.run.smithi187.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T00:29:31.584 DEBUG:teuthology.orchestra.run.smithi187:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T00:29:31.584 DEBUG:teuthology.orchestra.run.smithi187:> set -e 2024-09-21T00:29:31.584 DEBUG:teuthology.orchestra.run.smithi187:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-21T00:29:31.584 DEBUG:teuthology.orchestra.run.smithi187:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-21T00:29:31.584 DEBUG:teuthology.orchestra.run.smithi187:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-21T00:29:31.584 DEBUG:teuthology.orchestra.run.smithi187:> ') 2024-09-21T00:29:31.677 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:29:31 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T00:29:31.711 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:29:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T00:29:31.717 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:29:31.718 DEBUG:teuthology.orchestra.run.smithi187:> ip netns list 2024-09-21T00:29:31.733 INFO:teuthology.orchestra.run.smithi187.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-21T00:29:31.734 INFO:teuthology.orchestra.run.smithi187.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-21T00:29:31.734 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-21T00:29:31.734 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:29:31.734 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip link set brx.0 up 2024-09-21T00:29:31.799 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T00:29:31.799 DEBUG:teuthology.orchestra.run.smithi187:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-21T00:29:31.855 INFO:teuthology.orchestra.run.smithi187.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-21T00:29:31.855 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T00:29:31.855 DEBUG:teuthology.orchestra.run.smithi187:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-21T00:29:31.910 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-21T00:29:31.911 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:29:31.911 DEBUG:teuthology.orchestra.run.smithi187:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-21T00:29:32.006 INFO:teuthology.orchestra.run.smithi187.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-21T00:29:32.006 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-21T00:29:32.006 INFO:teuthology.orchestra.run.smithi187.stdout:invalid new device string format 2024-09-21T00:29:32.017 INFO:teuthology.orchestra.run.smithi187.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-21T00:29:32.017 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-21T00:29:32.017 INFO:teuthology.orchestra.run.smithi187.stdout:invalid new device string format 2024-09-21T00:29:32.017 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: resolved to: "172.21.15.112:3300,172.21.15.187:3300,172.21.15.187:3301" 2024-09-21T00:29:32.017 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T00:29:32.026 INFO:teuthology.orchestra.run.smithi187.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-21T00:29:32.026 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-21T00:29:32.027 INFO:teuthology.orchestra.run.smithi187.stdout:invalid new device string format 2024-09-21T00:29:32.027 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: resolved to: "172.21.15.112:3300,172.21.15.187:3300,172.21.15.187:3301" 2024-09-21T00:29:32.027 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T00:29:32.027 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: resolved to: "172.21.15.112:6789,172.21.15.187:6789,172.21.15.187:6790" 2024-09-21T00:29:32.027 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.112:6789,172.21.15.187:6789,172.21.15.187:6790:/ 2024-09-21T00:29:32.027 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=8b34692b-dc65-49b2-a08f-26f707b96ac5" will pass to kernel 2024-09-21T00:29:32.027 INFO:teuthology.orchestra.run.smithi187.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-09-21T00:29:32.027 INFO:teuthology.orchestra.run.smithi187.stdout: You just mounted a file system that supports labels which does not 2024-09-21T00:29:32.028 INFO:teuthology.orchestra.run.smithi187.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T00:29:32.028 INFO:teuthology.orchestra.run.smithi187.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T00:29:32.028 INFO:teuthology.orchestra.run.smithi187.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T00:29:32.029 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T00:29:32.029 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:29:32.029 DEBUG:teuthology.orchestra.run.smithi187:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-21T00:29:32.053 DEBUG:teuthology.orchestra.run.smithi187:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T00:29:32.141 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:29:32 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T00:29:32.143 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:29:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T00:29:32.149 INFO:teuthology.orchestra.run.smithi187.stdout:client6334 2024-09-21T00:29:32.149 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-09-21T00:29:32.149 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:29:32.165 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T00:29:32.165 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:29:32.183 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T00:29:32.183 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:29:32.241 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T00:29:32.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:32.400+0000 7ff3982cc640 1 -- 172.21.15.112:0/3203139024 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff37c004800 msgr2=0x7ff37c004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:32.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:32.400+0000 7ff3982cc640 1 --2- 172.21.15.112:0/3203139024 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff37c004800 0x7ff37c004c10 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7ff3900b7490 tx=0x7ff37c005830 comp rx=0 tx=0).stop 2024-09-21T00:29:32.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:32.400+0000 7ff3982cc640 1 -- 172.21.15.112:0/3203139024 shutdown_connections 2024-09-21T00:29:32.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:32.400+0000 7ff3982cc640 1 --2- 172.21.15.112:0/3203139024 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff37c004800 0x7ff37c004c10 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:32.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:32.400+0000 7ff3982cc640 1 -- 172.21.15.112:0/3203139024 >> 172.21.15.112:0/3203139024 conn(0x7ff39009d8a0 msgr2=0x7ff39008bfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:32.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:32.400+0000 7ff3982cc640 1 -- 172.21.15.112:0/3203139024 shutdown_connections 2024-09-21T00:29:32.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:32.400+0000 7ff3982cc640 1 -- 172.21.15.112:0/3203139024 wait complete. 2024-09-21T00:29:32.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:32.401+0000 7ff3982cc640 1 Processor -- start 2024-09-21T00:29:32.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:32.402+0000 7ff3982cc640 1 -- start start 2024-09-21T00:29:32.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:32.403+0000 7ff3982cc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff37c004800 0x7ff3901537f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:32.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:32.403+0000 7ff3982cc640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff39006cde0 0x7ff39006d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:32.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:32.403+0000 7ff3982cc640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff39006d730 0x7ff3900cbc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:32.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:32.403+0000 7ff3982cc640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff3900705e0 con 0x7ff37c004800 2024-09-21T00:29:32.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:32.403+0000 7ff3982cc640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff390070460 con 0x7ff39006cde0 2024-09-21T00:29:32.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:32.403+0000 7ff3982cc640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff390070760 con 0x7ff39006d730 2024-09-21T00:29:32.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:32.403+0000 7ff396068640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff37c004800 0x7ff3901537f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:32.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:32.403+0000 7ff396068640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff37c004800 0x7ff3901537f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57942/0 (socket says 172.21.15.112:57942) 2024-09-21T00:29:32.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:32.403+0000 7ff396068640 1 -- 172.21.15.112:0/424644292 learned_addr learned my addr 172.21.15.112:0/424644292 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:29:32.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:32.403+0000 7ff396869640 1 --2- 172.21.15.112:0/424644292 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff39006d730 0x7ff3900cbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:32.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:32.403+0000 7ff395867640 1 --2- 172.21.15.112:0/424644292 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff39006cde0 0x7ff39006d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:32.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:32.403+0000 7ff396068640 1 -- 172.21.15.112:0/424644292 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff39006d730 msgr2=0x7ff3900cbc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:32.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:32.403+0000 7ff396068640 1 --2- 172.21.15.112:0/424644292 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff39006d730 0x7ff3900cbc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:32.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:32.403+0000 7ff396068640 1 -- 172.21.15.112:0/424644292 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff39006cde0 msgr2=0x7ff39006d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:32.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:32.403+0000 7ff396068640 1 --2- 172.21.15.112:0/424644292 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff39006cde0 0x7ff39006d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:32.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:32.403+0000 7ff396068640 1 -- 172.21.15.112:0/424644292 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff39006de90 con 0x7ff37c004800 2024-09-21T00:29:32.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:32.403+0000 7ff395867640 1 --2- 172.21.15.112:0/424644292 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff39006cde0 0x7ff39006d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:29:32.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:32.403+0000 7ff396068640 1 --2- 172.21.15.112:0/424644292 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff37c004800 0x7ff3901537f0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7ff38c001090 tx=0x7ff38c0039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:32.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:32.403+0000 7ff386ffd640 1 -- 172.21.15.112:0/424644292 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff38c015070 con 0x7ff37c004800 2024-09-21T00:29:32.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:32.403+0000 7ff386ffd640 1 -- 172.21.15.112:0/424644292 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff38c011900 con 0x7ff37c004800 2024-09-21T00:29:32.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:32.403+0000 7ff3982cc640 1 -- 172.21.15.112:0/424644292 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff39006e170 con 0x7ff37c004800 2024-09-21T00:29:32.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:32.403+0000 7ff386ffd640 1 -- 172.21.15.112:0/424644292 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff38c011d30 con 0x7ff37c004800 2024-09-21T00:29:32.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:32.404+0000 7ff3982cc640 1 -- 172.21.15.112:0/424644292 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3900ce0e0 con 0x7ff37c004800 2024-09-21T00:29:32.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:32.405+0000 7ff3982cc640 1 -- 172.21.15.112:0/424644292 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3900d4710 con 0x7ff37c004800 2024-09-21T00:29:32.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:32.405+0000 7ff386ffd640 1 -- 172.21.15.112:0/424644292 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff38c018070 con 0x7ff37c004800 2024-09-21T00:29:32.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:32.406+0000 7ff386ffd640 1 --2- 172.21.15.112:0/424644292 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff374078bc0 0x7ff37407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:32.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:32.406+0000 7ff386ffd640 1 -- 172.21.15.112:0/424644292 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 6188+0+0 (secure 0 0 0) 0x7ff38c09bb40 con 0x7ff37c004800 2024-09-21T00:29:32.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:32.406+0000 7ff395867640 1 --2- 172.21.15.112:0/424644292 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff374078bc0 0x7ff37407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:32.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:32.406+0000 7ff395867640 1 --2- 172.21.15.112:0/424644292 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff374078bc0 0x7ff37407b0d0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7ff380032000 tx=0x7ff380001190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:32.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:32.410+0000 7ff386ffd640 1 -- 172.21.15.112:0/424644292 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ff38c063760 con 0x7ff37c004800 2024-09-21T00:29:32.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:32.597+0000 7ff3982cc640 1 -- 172.21.15.112:0/424644292 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7ff390050f60 con 0x7ff37c004800 2024-09-21T00:29:32.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:32.597+0000 7ff386ffd640 1 -- 172.21.15.112:0/424644292 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v96) ==== 80+0+0 (secure 0 0 0) 0x7ff38c068190 con 0x7ff37c004800 2024-09-21T00:29:32.598 INFO:teuthology.orchestra.run.smithi112.stderr:listed 0 entries 2024-09-21T00:29:32.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:32.599+0000 7ff3982cc640 1 -- 172.21.15.112:0/424644292 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff374078bc0 msgr2=0x7ff37407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:32.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:32.600+0000 7ff3982cc640 1 --2- 172.21.15.112:0/424644292 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff374078bc0 0x7ff37407b0d0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7ff380032000 tx=0x7ff380001190 comp rx=0 tx=0).stop 2024-09-21T00:29:32.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:32.600+0000 7ff3982cc640 1 -- 172.21.15.112:0/424644292 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff37c004800 msgr2=0x7ff3901537f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:32.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:32.600+0000 7ff3982cc640 1 --2- 172.21.15.112:0/424644292 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff37c004800 0x7ff3901537f0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7ff38c001090 tx=0x7ff38c0039c0 comp rx=0 tx=0).stop 2024-09-21T00:29:32.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:32.600+0000 7ff3982cc640 1 -- 172.21.15.112:0/424644292 shutdown_connections 2024-09-21T00:29:32.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:32.600+0000 7ff3982cc640 1 --2- 172.21.15.112:0/424644292 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff39006d730 0x7ff3900cbc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:32.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:32.600+0000 7ff3982cc640 1 --2- 172.21.15.112:0/424644292 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff39006cde0 0x7ff39006d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:32.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:32.600+0000 7ff3982cc640 1 --2- 172.21.15.112:0/424644292 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff374078bc0 0x7ff37407b0d0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:32.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:32.600+0000 7ff3982cc640 1 --2- 172.21.15.112:0/424644292 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff37c004800 0x7ff3901537f0 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:32.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:32.600+0000 7ff3982cc640 1 -- 172.21.15.112:0/424644292 >> 172.21.15.112:0/424644292 conn(0x7ff39009d8a0 msgr2=0x7ff3900d2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:32.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:32.600+0000 7ff3982cc640 1 -- 172.21.15.112:0/424644292 shutdown_connections 2024-09-21T00:29:32.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:32.600+0000 7ff3982cc640 1 -- 172.21.15.112:0/424644292 wait complete. 2024-09-21T00:29:32.612 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-21T00:29:32.612 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:29:32.612 DEBUG:teuthology.orchestra.run.smithi112:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-21T00:29:32.640 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi112.front.sepia.ceph.com 2024-09-21T00:29:32.640 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:29:32.640 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T00:29:32.696 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-09-21T00:29:32.697 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:29:32.714 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-09-21T00:29:32.714 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:29:32.770 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T00:29:32.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:32.885+0000 7f6cd489b640 1 -- 172.21.15.112:0/291582679 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6cd00d01b0 msgr2=0x7f6cd00d05c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:32.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:32.885+0000 7f6cd489b640 1 --2- 172.21.15.112:0/291582679 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6cd00d01b0 0x7f6cd00d05c0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f6cc8005fd0 tx=0x7f6cc802aa00 comp rx=0 tx=0).stop 2024-09-21T00:29:32.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:32.885+0000 7f6cd489b640 1 -- 172.21.15.112:0/291582679 shutdown_connections 2024-09-21T00:29:32.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:32.885+0000 7f6cd489b640 1 --2- 172.21.15.112:0/291582679 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6cd00d01b0 0x7f6cd00d05c0 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:32.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:32.885+0000 7f6cd489b640 1 --2- 172.21.15.112:0/291582679 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6cd00d48a0 0x7f6cd00d3dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:32.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:32.885+0000 7f6cd489b640 1 -- 172.21.15.112:0/291582679 >> 172.21.15.112:0/291582679 conn(0x7f6cd008bfe0 msgr2=0x7f6cd009d890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:32.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:32.885+0000 7f6cd489b640 1 -- 172.21.15.112:0/291582679 shutdown_connections 2024-09-21T00:29:32.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:32.885+0000 7f6cd489b640 1 -- 172.21.15.112:0/291582679 wait complete. 2024-09-21T00:29:32.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:32.887+0000 7f6cd489b640 1 Processor -- start 2024-09-21T00:29:32.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:32.887+0000 7f6cd489b640 1 -- start start 2024-09-21T00:29:32.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:32.888+0000 7f6cd489b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6cd00d01b0 0x7f6cd006cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:32.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:32.888+0000 7f6cd489b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6cd00d3880 0x7f6cd015ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:32.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:32.888+0000 7f6cd489b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6cd00d48a0 0x7f6cd01f5f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:32.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:32.888+0000 7f6cd489b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6cd00d2460 con 0x7f6cd00d01b0 2024-09-21T00:29:32.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:32.888+0000 7f6cd489b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f6cd00d22e0 con 0x7f6cd00d48a0 2024-09-21T00:29:32.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:32.888+0000 7f6cd489b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f6cd00d25e0 con 0x7f6cd00d3880 2024-09-21T00:29:32.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:32.888+0000 7f6cce59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6cd00d01b0 0x7f6cd006cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:32.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:32.888+0000 7f6cce59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6cd00d01b0 0x7f6cd006cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57960/0 (socket says 172.21.15.112:57960) 2024-09-21T00:29:32.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:32.888+0000 7f6cce59c640 1 -- 172.21.15.112:0/2913449471 learned_addr learned my addr 172.21.15.112:0/2913449471 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:29:32.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:32.888+0000 7f6cced9d640 1 --2- 172.21.15.112:0/2913449471 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6cd00d48a0 0x7f6cd01f5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:32.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:32.888+0000 7f6ccdd9b640 1 --2- 172.21.15.112:0/2913449471 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6cd00d3880 0x7f6cd015ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:32.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:32.888+0000 7f6cce59c640 1 -- 172.21.15.112:0/2913449471 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6cd00d3880 msgr2=0x7f6cd015ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:32.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:32.888+0000 7f6cce59c640 1 --2- 172.21.15.112:0/2913449471 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6cd00d3880 0x7f6cd015ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:32.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:32.888+0000 7f6cce59c640 1 -- 172.21.15.112:0/2913449471 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6cd00d48a0 msgr2=0x7f6cd01f5f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:32.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:32.888+0000 7f6cce59c640 1 --2- 172.21.15.112:0/2913449471 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6cd00d48a0 0x7f6cd01f5f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:32.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:32.888+0000 7f6cce59c640 1 -- 172.21.15.112:0/2913449471 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cd015bfb0 con 0x7f6cd00d01b0 2024-09-21T00:29:32.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:32.889+0000 7f6cce59c640 1 --2- 172.21.15.112:0/2913449471 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6cd00d01b0 0x7f6cd006cde0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f6cc000b410 tx=0x7f6cc000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:32.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:32.889+0000 7f6cbeffd640 1 -- 172.21.15.112:0/2913449471 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cc0015020 con 0x7f6cd00d01b0 2024-09-21T00:29:32.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:32.889+0000 7f6cbeffd640 1 -- 172.21.15.112:0/2913449471 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6cc0011940 con 0x7f6cd00d01b0 2024-09-21T00:29:32.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:32.889+0000 7f6cbeffd640 1 -- 172.21.15.112:0/2913449471 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cc0011d90 con 0x7f6cd00d01b0 2024-09-21T00:29:32.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:32.889+0000 7f6cd489b640 1 -- 172.21.15.112:0/2913449471 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cd00caf50 con 0x7f6cd00d01b0 2024-09-21T00:29:32.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:32.889+0000 7f6cd489b640 1 -- 172.21.15.112:0/2913449471 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cd006fc60 con 0x7f6cd00d01b0 2024-09-21T00:29:32.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:32.890+0000 7f6cd489b640 1 -- 172.21.15.112:0/2913449471 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cd00d2b20 con 0x7f6cd00d01b0 2024-09-21T00:29:32.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:32.891+0000 7f6cbeffd640 1 -- 172.21.15.112:0/2913449471 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6cc0018020 con 0x7f6cd00d01b0 2024-09-21T00:29:32.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:32.891+0000 7f6cbeffd640 1 --2- 172.21.15.112:0/2913449471 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f6cb0078b90 0x7f6cb007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:32.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:32.891+0000 7f6cbeffd640 1 -- 172.21.15.112:0/2913449471 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 6188+0+0 (secure 0 0 0) 0x7f6cc009ae00 con 0x7f6cd00d01b0 2024-09-21T00:29:32.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:32.891+0000 7f6ccdd9b640 1 --2- 172.21.15.112:0/2913449471 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f6cb0078b90 0x7f6cb007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:32.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:32.892+0000 7f6ccdd9b640 1 --2- 172.21.15.112:0/2913449471 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f6cb0078b90 0x7f6cb007b0a0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f6cd01efbe0 tx=0x7f6cc4009060 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:32.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:32.896+0000 7f6cbeffd640 1 -- 172.21.15.112:0/2913449471 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f6cc001f020 con 0x7f6cd00d01b0 2024-09-21T00:29:33.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.081+0000 7f6cd489b640 1 -- 172.21.15.112:0/2913449471 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f6cd00b6f90 con 0x7f6cd00d01b0 2024-09-21T00:29:33.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.081+0000 7f6cbeffd640 1 -- 172.21.15.112:0/2913449471 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v96) ==== 80+0+0 (secure 0 0 0) 0x7f6cc0062a10 con 0x7f6cd00d01b0 2024-09-21T00:29:33.082 INFO:teuthology.orchestra.run.smithi112.stderr:listed 0 entries 2024-09-21T00:29:33.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.083+0000 7f6cd489b640 1 -- 172.21.15.112:0/2913449471 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f6cb0078b90 msgr2=0x7f6cb007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:33.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.083+0000 7f6cd489b640 1 --2- 172.21.15.112:0/2913449471 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f6cb0078b90 0x7f6cb007b0a0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f6cd01efbe0 tx=0x7f6cc4009060 comp rx=0 tx=0).stop 2024-09-21T00:29:33.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.083+0000 7f6cd489b640 1 -- 172.21.15.112:0/2913449471 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6cd00d01b0 msgr2=0x7f6cd006cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:33.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.083+0000 7f6cd489b640 1 --2- 172.21.15.112:0/2913449471 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6cd00d01b0 0x7f6cd006cde0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f6cc000b410 tx=0x7f6cc000b8d0 comp rx=0 tx=0).stop 2024-09-21T00:29:33.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.083+0000 7f6cd489b640 1 -- 172.21.15.112:0/2913449471 shutdown_connections 2024-09-21T00:29:33.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.083+0000 7f6cd489b640 1 --2- 172.21.15.112:0/2913449471 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6cd00d48a0 0x7f6cd01f5f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:33.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.083+0000 7f6cd489b640 1 --2- 172.21.15.112:0/2913449471 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6cd00d3880 0x7f6cd015ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:33.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.083+0000 7f6cd489b640 1 --2- 172.21.15.112:0/2913449471 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f6cb0078b90 0x7f6cb007b0a0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:33.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.083+0000 7f6cd489b640 1 --2- 172.21.15.112:0/2913449471 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6cd00d01b0 0x7f6cd006cde0 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:33.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.083+0000 7f6cd489b640 1 -- 172.21.15.112:0/2913449471 >> 172.21.15.112:0/2913449471 conn(0x7f6cd008bfe0 msgr2=0x7f6cd00b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:33.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.083+0000 7f6cd489b640 1 -- 172.21.15.112:0/2913449471 shutdown_connections 2024-09-21T00:29:33.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.083+0000 7f6cd489b640 1 -- 172.21.15.112:0/2913449471 wait complete. 2024-09-21T00:29:33.095 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-09-21T00:29:33.095 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:29:33.095 DEBUG:teuthology.orchestra.run.smithi187:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-09-21T00:29:33.125 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi187.front.sepia.ceph.com 2024-09-21T00:29:33.125 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:29:33.125 DEBUG:teuthology.orchestra.run.smithi187:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-21T00:29:33.181 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T00:29:33.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.293+0000 7fa8d247e640 1 -- 172.21.15.112:0/681224012 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8cc15ca50 msgr2=0x7fa8cc0b41e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:33.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.293+0000 7fa8d247e640 1 --2- 172.21.15.112:0/681224012 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8cc15ca50 0x7fa8cc0b41e0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc0ced40 tx=0x7fa8bc02a560 comp rx=0 tx=0).stop 2024-09-21T00:29:33.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.293+0000 7fa8d247e640 1 -- 172.21.15.112:0/681224012 shutdown_connections 2024-09-21T00:29:33.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.293+0000 7fa8d247e640 1 --2- 172.21.15.112:0/681224012 >> v2:172.21.15.187:3300/0 conn(0x7fa8cc0ccce0 0x7fa8cc0b4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:33.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.293+0000 7fa8d247e640 1 --2- 172.21.15.112:0/681224012 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8cc15ca50 0x7fa8cc0b41e0 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:33.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.293+0000 7fa8d247e640 1 -- 172.21.15.112:0/681224012 >> 172.21.15.112:0/681224012 conn(0x7fa8cc0cf220 msgr2=0x7fa8cc0cf630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:33.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.293+0000 7fa8d247e640 1 -- 172.21.15.112:0/681224012 shutdown_connections 2024-09-21T00:29:33.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.293+0000 7fa8d247e640 1 -- 172.21.15.112:0/681224012 wait complete. 2024-09-21T00:29:33.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.295+0000 7fa8d247e640 1 Processor -- start 2024-09-21T00:29:33.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.295+0000 7fa8d247e640 1 -- start start 2024-09-21T00:29:33.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.296+0000 7fa8d247e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa8cc0b1bc0 0x7fa8cc15f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:33.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.296+0000 7fa8d247e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa8cc0ccce0 0x7fa8cc15e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:33.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.296+0000 7fa8d247e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8cc15ca50 0x7fa8cc1f6f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:33.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.296+0000 7fa8d247e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa8cc0d2130 con 0x7fa8cc15ca50 2024-09-21T00:29:33.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.296+0000 7fa8d247e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa8cc0d1fb0 con 0x7fa8cc0ccce0 2024-09-21T00:29:33.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.296+0000 7fa8d247e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa8cc0b26e0 con 0x7fa8cc0b1bc0 2024-09-21T00:29:33.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.297+0000 7fa8d0a1b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8cc15ca50 0x7fa8cc1f6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:33.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.297+0000 7fa8d0a1b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8cc15ca50 0x7fa8cc1f6f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57966/0 (socket says 172.21.15.112:57966) 2024-09-21T00:29:33.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.297+0000 7fa8d0a1b640 1 -- 172.21.15.112:0/3356608291 learned_addr learned my addr 172.21.15.112:0/3356608291 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:29:33.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.297+0000 7fa8cb7fe640 1 --2- 172.21.15.112:0/3356608291 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa8cc0ccce0 0x7fa8cc15e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:33.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.297+0000 7fa8cbfff640 1 --2- 172.21.15.112:0/3356608291 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa8cc0b1bc0 0x7fa8cc15f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:33.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.297+0000 7fa8d0a1b640 1 -- 172.21.15.112:0/3356608291 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa8cc0b1bc0 msgr2=0x7fa8cc15f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:33.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.297+0000 7fa8d0a1b640 1 --2- 172.21.15.112:0/3356608291 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa8cc0b1bc0 0x7fa8cc15f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:33.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.297+0000 7fa8d0a1b640 1 -- 172.21.15.112:0/3356608291 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa8cc0ccce0 msgr2=0x7fa8cc15e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:33.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.297+0000 7fa8d0a1b640 1 --2- 172.21.15.112:0/3356608291 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa8cc0ccce0 0x7fa8cc15e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:33.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.297+0000 7fa8d0a1b640 1 -- 172.21.15.112:0/3356608291 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8cc15d200 con 0x7fa8cc15ca50 2024-09-21T00:29:33.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.297+0000 7fa8cbfff640 1 --2- 172.21.15.112:0/3356608291 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa8cc0b1bc0 0x7fa8cc15f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:29:33.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.297+0000 7fa8cb7fe640 1 --2- 172.21.15.112:0/3356608291 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa8cc0ccce0 0x7fa8cc15e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:29:33.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.297+0000 7fa8d0a1b640 1 --2- 172.21.15.112:0/3356608291 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8cc15ca50 0x7fa8cc1f6f70 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fa8c400b9e0 tx=0x7fa8c400bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:33.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.297+0000 7fa8c8ff9640 1 -- 172.21.15.112:0/3356608291 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8c4019070 con 0x7fa8cc15ca50 2024-09-21T00:29:33.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.297+0000 7fa8c8ff9640 1 -- 172.21.15.112:0/3356608291 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa8c400cc10 con 0x7fa8cc15ca50 2024-09-21T00:29:33.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.297+0000 7fa8c8ff9640 1 -- 172.21.15.112:0/3356608291 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8c40146a0 con 0x7fa8cc15ca50 2024-09-21T00:29:33.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.297+0000 7fa8d247e640 1 -- 172.21.15.112:0/3356608291 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8cc15d4e0 con 0x7fa8cc15ca50 2024-09-21T00:29:33.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.297+0000 7fa8d247e640 1 -- 172.21.15.112:0/3356608291 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8cc15daa0 con 0x7fa8cc15ca50 2024-09-21T00:29:33.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.298+0000 7fa8d247e640 1 -- 172.21.15.112:0/3356608291 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8cc0b1fd0 con 0x7fa8cc15ca50 2024-09-21T00:29:33.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.299+0000 7fa8c8ff9640 1 -- 172.21.15.112:0/3356608291 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa8c401c070 con 0x7fa8cc15ca50 2024-09-21T00:29:33.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.299+0000 7fa8c8ff9640 1 --2- 172.21.15.112:0/3356608291 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fa8b8078b90 0x7fa8b807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:33.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.300+0000 7fa8c8ff9640 1 -- 172.21.15.112:0/3356608291 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 6188+0+0 (secure 0 0 0) 0x7fa8c409ac30 con 0x7fa8cc15ca50 2024-09-21T00:29:33.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.300+0000 7fa8cbfff640 1 --2- 172.21.15.112:0/3356608291 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fa8b8078b90 0x7fa8b807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:33.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.300+0000 7fa8cbfff640 1 --2- 172.21.15.112:0/3356608291 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fa8b8078b90 0x7fa8b807b0a0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc033000 tx=0x7fa8bc004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:33.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.304+0000 7fa8c8ff9640 1 -- 172.21.15.112:0/3356608291 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa8c4062f20 con 0x7fa8cc15ca50 2024-09-21T00:29:33.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.516+0000 7fa8d247e640 1 -- 172.21.15.112:0/3356608291 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa8cc15e7e0 con 0x7fa8cc15ca50 2024-09-21T00:29:33.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.517+0000 7fa8c8ff9640 1 -- 172.21.15.112:0/3356608291 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) ==== 94+0+5237 (secure 0 0 0) 0x7fa8c4062820 con 0x7fa8cc15ca50 2024-09-21T00:29:33.518 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:29:33.518 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":64,"btime":"2024-09-21T00:29:22:809921+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5804,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/3818262153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":3818262153},{"type":"v1","addr":"172.21.15.187:6835","nonce":3818262153}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59},{"gid":6214,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6837/1321768683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":1321768683},{"type":"v1","addr":"172.21.15.187:6837","nonce":1321768683}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59},{"gid":6801,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6834/4037012343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":4037012343},{"type":"v1","addr":"172.21.15.112:6834","nonce":4037012343}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T00:29:19.145794+0000","modified":"2024-09-21T00:29:22.172462+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6804},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6804":{"gid":6804,"name":"c","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6835/2797048793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":2797048793},{"type":"v1","addr":"172.21.15.112:6835","nonce":2797048793}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6804,"qdb_cluster":[6804]},"id":7}]} 2024-09-21T00:29:33.518 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 64 2024-09-21T00:29:33.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.520+0000 7fa8d247e640 1 -- 172.21.15.112:0/3356608291 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fa8b8078b90 msgr2=0x7fa8b807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:33.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.520+0000 7fa8d247e640 1 --2- 172.21.15.112:0/3356608291 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fa8b8078b90 0x7fa8b807b0a0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc033000 tx=0x7fa8bc004730 comp rx=0 tx=0).stop 2024-09-21T00:29:33.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.520+0000 7fa8d247e640 1 -- 172.21.15.112:0/3356608291 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8cc15ca50 msgr2=0x7fa8cc1f6f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:33.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.520+0000 7fa8d247e640 1 --2- 172.21.15.112:0/3356608291 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8cc15ca50 0x7fa8cc1f6f70 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fa8c400b9e0 tx=0x7fa8c400bea0 comp rx=0 tx=0).stop 2024-09-21T00:29:33.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.520+0000 7fa8d247e640 1 -- 172.21.15.112:0/3356608291 shutdown_connections 2024-09-21T00:29:33.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.520+0000 7fa8d247e640 1 --2- 172.21.15.112:0/3356608291 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fa8b8078b90 0x7fa8b807b0a0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:33.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.520+0000 7fa8d247e640 1 --2- 172.21.15.112:0/3356608291 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8cc15ca50 0x7fa8cc1f6f70 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:33.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.520+0000 7fa8d247e640 1 --2- 172.21.15.112:0/3356608291 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa8cc0ccce0 0x7fa8cc15e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:33.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.520+0000 7fa8d247e640 1 --2- 172.21.15.112:0/3356608291 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa8cc0b1bc0 0x7fa8cc15f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:33.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.520+0000 7fa8d247e640 1 -- 172.21.15.112:0/3356608291 >> 172.21.15.112:0/3356608291 conn(0x7fa8cc0cf220 msgr2=0x7fa8cc159e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:33.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.520+0000 7fa8d247e640 1 -- 172.21.15.112:0/3356608291 shutdown_connections 2024-09-21T00:29:33.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.520+0000 7fa8d247e640 1 -- 172.21.15.112:0/3356608291 wait complete. 2024-09-21T00:29:33.531 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.7:0 session ls 2024-09-21T00:29:33.646 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.645+0000 7fe04722e640 1 -- 172.21.15.112:0/2113899755 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0400b1d30 msgr2=0x7fe0400b2140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:33.646 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.645+0000 7fe04722e640 1 --2- 172.21.15.112:0/2113899755 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0400b1d30 0x7fe0400b2140 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fe0400cde90 tx=0x7fe03402a4d0 comp rx=0 tx=0).stop 2024-09-21T00:29:33.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.645+0000 7fe04722e640 1 -- 172.21.15.112:0/2113899755 shutdown_connections 2024-09-21T00:29:33.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.645+0000 7fe04722e640 1 --2- 172.21.15.112:0/2113899755 >> v2:172.21.15.187:3300/0 conn(0x7fe0400b2680 0x7fe0400b2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:33.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.645+0000 7fe04722e640 1 --2- 172.21.15.112:0/2113899755 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0400b1d30 0x7fe0400b2140 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:33.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.645+0000 7fe04722e640 1 -- 172.21.15.112:0/2113899755 >> 172.21.15.112:0/2113899755 conn(0x7fe0400b42e0 msgr2=0x7fe0400b4720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:33.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.645+0000 7fe04722e640 1 -- 172.21.15.112:0/2113899755 shutdown_connections 2024-09-21T00:29:33.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.645+0000 7fe04722e640 1 -- 172.21.15.112:0/2113899755 wait complete. 2024-09-21T00:29:33.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.647+0000 7fe04722e640 1 Processor -- start 2024-09-21T00:29:33.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.647+0000 7fe04722e640 1 -- start start 2024-09-21T00:29:33.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.648+0000 7fe04722e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe0400b1d30 0x7fe040154170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:33.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.648+0000 7fe04722e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0400b2680 0x7fe0401f6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:33.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.648+0000 7fe04722e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe0400ccbf0 0x7fe040150250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:33.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.648+0000 7fe04722e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe0400b3420 con 0x7fe0400b2680 2024-09-21T00:29:33.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.648+0000 7fe04722e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe0400b32a0 con 0x7fe0400ccbf0 2024-09-21T00:29:33.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.648+0000 7fe04722e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe0400b35a0 con 0x7fe0400b1d30 2024-09-21T00:29:33.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.648+0000 7fe03ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0400b2680 0x7fe0401f6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:33.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.648+0000 7fe03ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0400b2680 0x7fe0401f6dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57994/0 (socket says 172.21.15.112:57994) 2024-09-21T00:29:33.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.648+0000 7fe03ffff640 1 -- 172.21.15.112:0/4147102032 learned_addr learned my addr 172.21.15.112:0/4147102032 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:29:33.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.648+0000 7fe0457cb640 1 --2- 172.21.15.112:0/4147102032 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe0400ccbf0 0x7fe040150250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:33.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.648+0000 7fe044fca640 1 --2- 172.21.15.112:0/4147102032 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe0400b1d30 0x7fe040154170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:33.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.648+0000 7fe03ffff640 1 -- 172.21.15.112:0/4147102032 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe0400b1d30 msgr2=0x7fe040154170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:33.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.648+0000 7fe03ffff640 1 --2- 172.21.15.112:0/4147102032 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe0400b1d30 0x7fe040154170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:33.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.648+0000 7fe03ffff640 1 -- 172.21.15.112:0/4147102032 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe0400ccbf0 msgr2=0x7fe040150250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:33.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.648+0000 7fe03ffff640 1 --2- 172.21.15.112:0/4147102032 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe0400ccbf0 0x7fe040150250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:33.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.648+0000 7fe03ffff640 1 -- 172.21.15.112:0/4147102032 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0400b6fe0 con 0x7fe0400b2680 2024-09-21T00:29:33.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.648+0000 7fe044fca640 1 --2- 172.21.15.112:0/4147102032 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe0400b1d30 0x7fe040154170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:29:33.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.649+0000 7fe03ffff640 1 --2- 172.21.15.112:0/4147102032 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0400b2680 0x7fe0401f6dc0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fe0340099c0 tx=0x7fe0340039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:33.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.649+0000 7fe03d7fa640 1 -- 172.21.15.112:0/4147102032 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe03403e030 con 0x7fe0400b2680 2024-09-21T00:29:33.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.649+0000 7fe03d7fa640 1 -- 172.21.15.112:0/4147102032 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe0340047a0 con 0x7fe0400b2680 2024-09-21T00:29:33.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.649+0000 7fe03d7fa640 1 -- 172.21.15.112:0/4147102032 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe034035040 con 0x7fe0400b2680 2024-09-21T00:29:33.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.649+0000 7fe04722e640 1 -- 172.21.15.112:0/4147102032 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0400b7280 con 0x7fe0400b2680 2024-09-21T00:29:33.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.649+0000 7fe04722e640 1 -- 172.21.15.112:0/4147102032 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0400b7840 con 0x7fe0400b2680 2024-09-21T00:29:33.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.651+0000 7fe03d7fa640 1 -- 172.21.15.112:0/4147102032 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe034041030 con 0x7fe0400b2680 2024-09-21T00:29:33.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.651+0000 7fe03d7fa640 1 --2- 172.21.15.112:0/4147102032 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fe02c078b90 0x7fe02c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:33.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.651+0000 7fe03d7fa640 1 -- 172.21.15.112:0/4147102032 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 6188+0+0 (secure 0 0 0) 0x7fe0340bb780 con 0x7fe0400b2680 2024-09-21T00:29:33.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.651+0000 7fe044fca640 1 --2- 172.21.15.112:0/4147102032 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fe02c078b90 0x7fe02c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:33.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.652+0000 7fe044fca640 1 --2- 172.21.15.112:0/4147102032 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fe02c078b90 0x7fe02c07b0a0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fe030007fd0 tx=0x7fe030009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:33.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.652+0000 7fe04e276740 1 Processor -- start 2024-09-21T00:29:33.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.652+0000 7fe04e276740 1 -- start start 2024-09-21T00:29:33.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.653+0000 7fe04e276740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560a2c835390 0x560a2cbf5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:33.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.653+0000 7fe04e276740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a2c80c4d0 0x560a2cc040a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:33.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.653+0000 7fe04e276740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560a2c8146b0 0x560a2cc045e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:33.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.653+0000 7fe04e276740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x560a2cb4f8e0 con 0x560a2c80c4d0 2024-09-21T00:29:33.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.653+0000 7fe04e276740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x560a2cb3ed10 con 0x560a2c8146b0 2024-09-21T00:29:33.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.653+0000 7fe04e276740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x560a2cb42b20 con 0x560a2c835390 2024-09-21T00:29:33.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.653+0000 7fe03ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a2c80c4d0 0x560a2cc040a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:33.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.653+0000 7fe03ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a2c80c4d0 0x560a2cc040a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58004/0 (socket says 172.21.15.112:58004) 2024-09-21T00:29:33.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.653+0000 7fe03ffff640 1 -- 172.21.15.112:0/2558717063 learned_addr learned my addr 172.21.15.112:0/2558717063 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:29:33.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.653+0000 7fe0457cb640 1 --2- 172.21.15.112:0/2558717063 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560a2c8146b0 0x560a2cc045e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:33.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.653+0000 7fe044fca640 1 --2- 172.21.15.112:0/2558717063 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560a2c835390 0x560a2cbf5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:33.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.654+0000 7fe03ffff640 1 -- 172.21.15.112:0/2558717063 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560a2c835390 msgr2=0x560a2cbf5ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:29:33.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.654+0000 7fe03ffff640 1 --2- 172.21.15.112:0/2558717063 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560a2c835390 0x560a2cbf5ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:33.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.654+0000 7fe03ffff640 1 -- 172.21.15.112:0/2558717063 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560a2c8146b0 msgr2=0x560a2cc045e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:29:33.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.654+0000 7fe03ffff640 1 --2- 172.21.15.112:0/2558717063 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560a2c8146b0 0x560a2cc045e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:33.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.654+0000 7fe03ffff640 1 -- 172.21.15.112:0/2558717063 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a2c8a0970 con 0x560a2c80c4d0 2024-09-21T00:29:33.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.654+0000 7fe03ffff640 1 --2- 172.21.15.112:0/2558717063 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a2c80c4d0 0x560a2cc040a0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7fe034083270 tx=0x7fe0340830b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd22d2d5fdf7430e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:33.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.654+0000 7fe0267fc640 1 -- 172.21.15.112:0/2558717063 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe034048040 con 0x560a2c80c4d0 2024-09-21T00:29:33.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.654+0000 7fe0267fc640 1 -- 172.21.15.112:0/2558717063 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe034005310 con 0x560a2c80c4d0 2024-09-21T00:29:33.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.654+0000 7fe0267fc640 1 -- 172.21.15.112:0/2558717063 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe03408ca70 con 0x560a2c80c4d0 2024-09-21T00:29:33.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.655+0000 7fe04e276740 1 -- 172.21.15.112:0/2558717063 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a2c80c4d0 msgr2=0x560a2cc040a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:33.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.655+0000 7fe04e276740 1 --2- 172.21.15.112:0/2558717063 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a2c80c4d0 0x560a2cc040a0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7fe034083270 tx=0x7fe0340830b0 comp rx=0 tx=0).stop 2024-09-21T00:29:33.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.655+0000 7fe04e276740 1 -- 172.21.15.112:0/2558717063 shutdown_connections 2024-09-21T00:29:33.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.655+0000 7fe04e276740 1 --2- 172.21.15.112:0/2558717063 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560a2c8146b0 0x560a2cc045e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:33.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.655+0000 7fe04e276740 1 --2- 172.21.15.112:0/2558717063 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a2c80c4d0 0x560a2cc040a0 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:33.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.655+0000 7fe04e276740 1 --2- 172.21.15.112:0/2558717063 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560a2c835390 0x560a2cbf5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:33.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.655+0000 7fe04e276740 1 -- 172.21.15.112:0/2558717063 >> 172.21.15.112:0/2558717063 conn(0x560a2c8294e0 msgr2=0x560a2cbdd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:33.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.655+0000 7fe04e276740 1 -- 172.21.15.112:0/2558717063 shutdown_connections 2024-09-21T00:29:33.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.655+0000 7fe04e276740 1 -- 172.21.15.112:0/2558717063 wait complete. 2024-09-21T00:29:33.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.656+0000 7fe04e276740 1 -- start start 2024-09-21T00:29:33.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.656+0000 7fe04e276740 1 Processor -- start 2024-09-21T00:29:33.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.657+0000 7fe04e276740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560a2c80c4d0 0x560a2cc18ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:33.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.657+0000 7fe04e276740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560a2c8146b0 0x560a2cc1a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:33.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.657+0000 7fe04e276740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a2c835390 0x560a2cc1d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:33.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.657+0000 7fe04e276740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x560a2c8af740 con 0x560a2c835390 2024-09-21T00:29:33.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.657+0000 7fe04e276740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x560a2cae47f0 con 0x560a2c80c4d0 2024-09-21T00:29:33.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.657+0000 7fe04e276740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x560a2cb4f8e0 con 0x560a2c8146b0 2024-09-21T00:29:33.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.657+0000 7fe04e276740 10 client.0 _ll_register_callbacks cb 0x560a2cae82a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:29:33.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.657+0000 7fe0457cb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a2c835390 0x560a2cc1d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:33.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.657+0000 7fe0457cb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a2c835390 0x560a2cc1d5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58020/0 (socket says 172.21.15.112:58020) 2024-09-21T00:29:33.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.657+0000 7fe0457cb640 1 -- 172.21.15.112:0/2754863943 learned_addr learned my addr 172.21.15.112:0/2754863943 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:29:33.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.657+0000 7fe03ffff640 1 --2- 172.21.15.112:0/2754863943 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560a2c8146b0 0x560a2cc1a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:33.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.657+0000 7fe044fca640 1 --2- 172.21.15.112:0/2754863943 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560a2c80c4d0 0x560a2cc18ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:33.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.657+0000 7fe0457cb640 1 -- 172.21.15.112:0/2754863943 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560a2c8146b0 msgr2=0x560a2cc1a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:29:33.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.658+0000 7fe0457cb640 1 --2- 172.21.15.112:0/2754863943 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560a2c8146b0 0x560a2cc1a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:33.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.658+0000 7fe0457cb640 1 -- 172.21.15.112:0/2754863943 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560a2c80c4d0 msgr2=0x560a2cc18ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:29:33.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.658+0000 7fe0457cb640 1 --2- 172.21.15.112:0/2754863943 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560a2c80c4d0 0x560a2cc18ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:33.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.658+0000 7fe0457cb640 1 -- 172.21.15.112:0/2754863943 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a2c9177d0 con 0x560a2c835390 2024-09-21T00:29:33.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.658+0000 7fe0457cb640 1 --2- 172.21.15.112:0/2754863943 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a2c835390 0x560a2cc1d5f0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fe0400ce190 tx=0x7fe038011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d882a8e4f2356fc5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:33.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.658+0000 7fe01f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T00:29:33.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.658+0000 7fe01f7fe640 1 -- 172.21.15.112:0/2754863943 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe038004b20 con 0x560a2c835390 2024-09-21T00:29:33.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.658+0000 7fe01f7fe640 1 -- 172.21.15.112:0/2754863943 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 6188+0+0 (secure 0 0 0) 0x7fe038004cc0 con 0x560a2c835390 2024-09-21T00:29:33.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.658+0000 7fe04e276740 1 -- 172.21.15.112:0/2754863943 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe034005310 con 0x560a2c835390 2024-09-21T00:29:33.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.658+0000 7fe01f7fe640 1 -- 172.21.15.112:0/2754863943 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7fe03800f910 con 0x560a2c835390 2024-09-21T00:29:33.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.658+0000 7fe04e276740 10 client.6999 fetch_fsmap learned FSMap version 64 2024-09-21T00:29:33.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.658+0000 7fe04e276740 1 -- 172.21.15.112:0/2754863943 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=2+}) -- 0x560a2cc1aef0 con 0x560a2c835390 2024-09-21T00:29:33.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.658+0000 7fe01f7fe640 1 -- 172.21.15.112:0/2754863943 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe034005310 con 0x560a2c835390 2024-09-21T00:29:33.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.658+0000 7fe01f7fe640 1 -- 172.21.15.112:0/2754863943 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 64) ==== 3502+0+0 (secure 0 0 0) 0x7fe038026080 con 0x560a2c835390 2024-09-21T00:29:33.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.658+0000 7fe04e276740 10 client.6999 fetch_fsmap finished waiting for FSMap version 64 2024-09-21T00:29:33.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.658+0000 7fe04e276740 10 client.6999 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-09-21T00:29:33.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.658+0000 7fe04e276740 20 client.6999 populate_metadata read hostname 'smithi112' 2024-09-21T00:29:33.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.658+0000 7fe04e276740 1 --2- 172.21.15.112:0/2754863943 >> [v2:172.21.15.112:6833/2797048793,v1:172.21.15.112:6835/2797048793] conn(0x560a2c7f3f60 0x560a2cbdcfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:33.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.658+0000 7fe04e276740 4 client.6999 mds_command: new command op to 6804 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T00:29:33.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.658+0000 7fe04e276740 1 -- 172.21.15.112:0/2754863943 --> [v2:172.21.15.112:6833/2797048793,v1:172.21.15.112:6835/2797048793] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560a2cb51830 con 0x560a2c7f3f60 2024-09-21T00:29:33.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.658+0000 7fe044fca640 1 --2- 172.21.15.112:0/2754863943 >> [v2:172.21.15.112:6833/2797048793,v1:172.21.15.112:6835/2797048793] conn(0x560a2c7f3f60 0x560a2cbdcfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:33.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.659+0000 7fe01f7fe640 10 client.6999 ms_handle_connect on v2:172.21.15.112:6833/2797048793 2024-09-21T00:29:33.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.659+0000 7fe044fca640 1 --2- 172.21.15.112:0/2754863943 >> [v2:172.21.15.112:6833/2797048793,v1:172.21.15.112:6835/2797048793] conn(0x560a2c7f3f60 0x560a2cbdcfc0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cbc974157ea5f03d server_cookie=8bbb8c86dd220ee6 in_seq=0 out_seq=0 2024-09-21T00:29:33.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.660+0000 7fe01f7fe640 1 -- 172.21.15.112:0/2754863943 <== mds.0 v2:172.21.15.112:6833/2797048793 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560a2cb51830 con 0x560a2c7f3f60 2024-09-21T00:29:33.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.660+0000 7fe01f7fe640 10 client.6999 handle_command_reply: tid=0 2024-09-21T00:29:33.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.660+0000 7fe04e276740 1 client.6999 shutdown 2024-09-21T00:29:33.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.660+0000 7fe04e276740 1 -- 172.21.15.112:0/2754863943 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a2c835390 msgr2=0x560a2cc1d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:33.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.660+0000 7fe04e276740 1 --2- 172.21.15.112:0/2754863943 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a2c835390 0x560a2cc1d5f0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fe0400ce190 tx=0x7fe038011b50 comp rx=0 tx=0).stop 2024-09-21T00:29:33.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.660+0000 7fe04e276740 1 -- 172.21.15.112:0/2754863943 shutdown_connections 2024-09-21T00:29:33.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.660+0000 7fe04e276740 1 --2- 172.21.15.112:0/2754863943 >> [v2:172.21.15.112:6833/2797048793,v1:172.21.15.112:6835/2797048793] conn(0x560a2c7f3f60 0x560a2cbdcfc0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:33.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.660+0000 7fe01f7fe640 0 client.6999 ms_handle_reset on v2:172.21.15.112:6833/2797048793 2024-09-21T00:29:33.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.660+0000 7fe04e276740 1 --2- 172.21.15.112:0/2754863943 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a2c835390 0x560a2cc1d5f0 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:33.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.660+0000 7fe04e276740 1 --2- 172.21.15.112:0/2754863943 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560a2c8146b0 0x560a2cc1a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:33.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.660+0000 7fe04e276740 1 --2- 172.21.15.112:0/2754863943 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560a2c80c4d0 0x560a2cc18ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:33.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.660+0000 7fe04e276740 1 -- 172.21.15.112:0/2754863943 >> 172.21.15.112:0/2754863943 conn(0x560a2c8294e0 msgr2=0x560a2cbf73e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:33.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.660+0000 7fe04e276740 1 -- 172.21.15.112:0/2754863943 shutdown_connections 2024-09-21T00:29:33.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.660+0000 7fe04e276740 1 -- 172.21.15.112:0/2754863943 wait complete. 2024-09-21T00:29:33.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.660+0000 7fe04e276740 20 client.6999 trim_cache size 0 max 16384 2024-09-21T00:29:33.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.678+0000 7fe04e276740 1 Processor -- start 2024-09-21T00:29:33.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.678+0000 7fe04e276740 1 -- start start 2024-09-21T00:29:33.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.679+0000 7fe04e276740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560a2c7f3f60 0x560a2cc1a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:33.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.679+0000 7fe04e276740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560a2c80c4d0 0x560a2cc2fb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:33.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.679+0000 7fe04e276740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a2c8146b0 0x560a2cbd8a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:33.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.679+0000 7fe04e276740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe0401fe5d0 con 0x560a2c8146b0 2024-09-21T00:29:33.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.679+0000 7fe04e276740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe040071470 con 0x560a2c80c4d0 2024-09-21T00:29:33.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.679+0000 7fe04e276740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe040155c80 con 0x560a2c7f3f60 2024-09-21T00:29:33.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.680+0000 7fe0457cb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a2c8146b0 0x560a2cbd8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:33.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.680+0000 7fe0457cb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a2c8146b0 0x560a2cbd8a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58022/0 (socket says 172.21.15.112:58022) 2024-09-21T00:29:33.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.680+0000 7fe0457cb640 1 -- 172.21.15.112:0/1546967163 learned_addr learned my addr 172.21.15.112:0/1546967163 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:29:33.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.680+0000 7fe044fca640 1 --2- 172.21.15.112:0/1546967163 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560a2c7f3f60 0x560a2cc1a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:33.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.680+0000 7fe03ffff640 1 --2- 172.21.15.112:0/1546967163 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560a2c80c4d0 0x560a2cc2fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:33.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.680+0000 7fe0457cb640 1 -- 172.21.15.112:0/1546967163 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560a2c7f3f60 msgr2=0x560a2cc1a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:29:33.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.680+0000 7fe0457cb640 1 --2- 172.21.15.112:0/1546967163 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560a2c7f3f60 0x560a2cc1a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:33.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.680+0000 7fe0457cb640 1 -- 172.21.15.112:0/1546967163 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560a2c80c4d0 msgr2=0x560a2cc2fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:29:33.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.680+0000 7fe0457cb640 1 --2- 172.21.15.112:0/1546967163 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560a2c80c4d0 0x560a2cc2fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:33.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.680+0000 7fe0457cb640 1 -- 172.21.15.112:0/1546967163 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a2cc1c3b0 con 0x560a2c8146b0 2024-09-21T00:29:33.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.680+0000 7fe044fca640 1 --2- 172.21.15.112:0/1546967163 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560a2c7f3f60 0x560a2cc1a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:29:33.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.680+0000 7fe0457cb640 1 --2- 172.21.15.112:0/1546967163 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a2c8146b0 0x560a2cbd8a60 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fe038000f80 tx=0x7fe038011f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26cf67f35a261059 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:33.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.680+0000 7fe025ffb640 1 -- 172.21.15.112:0/1546967163 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe038004b20 con 0x560a2c8146b0 2024-09-21T00:29:33.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.680+0000 7fe025ffb640 1 -- 172.21.15.112:0/1546967163 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe038004dd0 con 0x560a2c8146b0 2024-09-21T00:29:33.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.680+0000 7fe025ffb640 1 -- 172.21.15.112:0/1546967163 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe038005280 con 0x560a2c8146b0 2024-09-21T00:29:33.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.681+0000 7fe04e276740 1 -- 172.21.15.112:0/1546967163 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a2c8146b0 msgr2=0x560a2cbd8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:33.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.681+0000 7fe04e276740 1 --2- 172.21.15.112:0/1546967163 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a2c8146b0 0x560a2cbd8a60 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fe038000f80 tx=0x7fe038011f30 comp rx=0 tx=0).stop 2024-09-21T00:29:33.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.681+0000 7fe04e276740 1 -- 172.21.15.112:0/1546967163 shutdown_connections 2024-09-21T00:29:33.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.681+0000 7fe04e276740 1 --2- 172.21.15.112:0/1546967163 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a2c8146b0 0x560a2cbd8a60 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:33.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.681+0000 7fe04e276740 1 --2- 172.21.15.112:0/1546967163 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560a2c80c4d0 0x560a2cc2fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:33.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.681+0000 7fe04e276740 1 --2- 172.21.15.112:0/1546967163 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560a2c7f3f60 0x560a2cc1a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:33.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.681+0000 7fe04e276740 1 -- 172.21.15.112:0/1546967163 >> 172.21.15.112:0/1546967163 conn(0x560a2c8294e0 msgr2=0x560a2cbf7230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:33.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.681+0000 7fe04e276740 1 -- 172.21.15.112:0/1546967163 shutdown_connections 2024-09-21T00:29:33.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.681+0000 7fe04e276740 1 -- 172.21.15.112:0/1546967163 wait complete. 2024-09-21T00:29:33.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.682+0000 7fe04e276740 1 -- start start 2024-09-21T00:29:33.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.682+0000 7fe04e276740 1 Processor -- start 2024-09-21T00:29:33.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.683+0000 7fe04e276740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560a2c7f3f60 0x560a2cc496f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:33.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.683+0000 7fe04e276740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560a2c80c4d0 0x560a2cca51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:33.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.683+0000 7fe04e276740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a2c8146b0 0x560a2cbf5e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:33.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.683+0000 7fe04e276740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x560a2cbdc340 con 0x560a2c8146b0 2024-09-21T00:29:33.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.683+0000 7fe04e276740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe040070e20 con 0x560a2c80c4d0 2024-09-21T00:29:33.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.683+0000 7fe04e276740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x560a2cc249d0 con 0x560a2c7f3f60 2024-09-21T00:29:33.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.683+0000 7fe04e276740 10 client.0 _ll_register_callbacks cb 0x7fe0380095c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:29:33.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.683+0000 7fe0457cb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a2c8146b0 0x560a2cbf5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:33.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.684+0000 7fe0457cb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a2c8146b0 0x560a2cbf5e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58032/0 (socket says 172.21.15.112:58032) 2024-09-21T00:29:33.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.684+0000 7fe0457cb640 1 -- 172.21.15.112:0/2341013916 learned_addr learned my addr 172.21.15.112:0/2341013916 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:29:33.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.684+0000 7fe03ffff640 1 --2- 172.21.15.112:0/2341013916 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560a2c80c4d0 0x560a2cca51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:33.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.684+0000 7fe044fca640 1 --2- 172.21.15.112:0/2341013916 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560a2c7f3f60 0x560a2cc496f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:33.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.684+0000 7fe0457cb640 1 -- 172.21.15.112:0/2341013916 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560a2c7f3f60 msgr2=0x560a2cc496f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:29:33.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.684+0000 7fe0457cb640 1 --2- 172.21.15.112:0/2341013916 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560a2c7f3f60 0x560a2cc496f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:33.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.684+0000 7fe0457cb640 1 -- 172.21.15.112:0/2341013916 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560a2c80c4d0 msgr2=0x560a2cca51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:29:33.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.684+0000 7fe0457cb640 1 --2- 172.21.15.112:0/2341013916 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560a2c80c4d0 0x560a2cca51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:33.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.684+0000 7fe0457cb640 1 -- 172.21.15.112:0/2341013916 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a2cbf52d0 con 0x560a2c8146b0 2024-09-21T00:29:33.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.684+0000 7fe044fca640 1 --2- 172.21.15.112:0/2341013916 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560a2c7f3f60 0x560a2cc496f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:29:33.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.684+0000 7fe0457cb640 1 --2- 172.21.15.112:0/2341013916 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a2c8146b0 0x560a2cbf5e00 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fe038009de0 tx=0x7fe038009f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18c6716b356726a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:33.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.684+0000 7fe01f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T00:29:33.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.684+0000 7fe01f7fe640 1 -- 172.21.15.112:0/2341013916 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe038004b20 con 0x560a2c8146b0 2024-09-21T00:29:33.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.684+0000 7fe01f7fe640 1 -- 172.21.15.112:0/2341013916 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 6188+0+0 (secure 0 0 0) 0x7fe038005430 con 0x560a2c8146b0 2024-09-21T00:29:33.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.684+0000 7fe04e276740 1 -- 172.21.15.112:0/2341013916 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe038004dd0 con 0x560a2c8146b0 2024-09-21T00:29:33.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.684+0000 7fe01f7fe640 1 -- 172.21.15.112:0/2341013916 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7fe038044050 con 0x560a2c8146b0 2024-09-21T00:29:33.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.684+0000 7fe04e276740 10 client.7005 fetch_fsmap learned FSMap version 64 2024-09-21T00:29:33.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.684+0000 7fe04e276740 1 -- 172.21.15.112:0/2341013916 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=2+}) -- 0x560a2cca56f0 con 0x560a2c8146b0 2024-09-21T00:29:33.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.685+0000 7fe01f7fe640 1 -- 172.21.15.112:0/2341013916 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe038004dd0 con 0x560a2c8146b0 2024-09-21T00:29:33.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.685+0000 7fe01f7fe640 1 -- 172.21.15.112:0/2341013916 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 64) ==== 3502+0+0 (secure 0 0 0) 0x7fe038047020 con 0x560a2c8146b0 2024-09-21T00:29:33.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.685+0000 7fe04e276740 10 client.7005 fetch_fsmap finished waiting for FSMap version 64 2024-09-21T00:29:33.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.685+0000 7fe04e276740 10 client.7005 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-09-21T00:29:33.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.685+0000 7fe04e276740 20 client.7005 populate_metadata read hostname 'smithi112' 2024-09-21T00:29:33.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.685+0000 7fe04e276740 1 --2- 172.21.15.112:0/2341013916 >> [v2:172.21.15.112:6833/2797048793,v1:172.21.15.112:6835/2797048793] conn(0x560a2c7c93a0 0x560a2cbf6340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:33.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.685+0000 7fe04e276740 4 client.7005 mds_command: new command op to 6804 tid=0 multi_id=0 [{"prefix": "session ls"}] 2024-09-21T00:29:33.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.685+0000 7fe04e276740 1 -- 172.21.15.112:0/2341013916 --> [v2:172.21.15.112:6833/2797048793,v1:172.21.15.112:6835/2797048793] -- command(tid 0: {"prefix": "session ls"}) -- 0x560a2cb4d2e0 con 0x560a2c7c93a0 2024-09-21T00:29:33.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.685+0000 7fe044fca640 1 --2- 172.21.15.112:0/2341013916 >> [v2:172.21.15.112:6833/2797048793,v1:172.21.15.112:6835/2797048793] conn(0x560a2c7c93a0 0x560a2cbf6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:33.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.685+0000 7fe01f7fe640 10 client.7005 ms_handle_connect on v2:172.21.15.112:6833/2797048793 2024-09-21T00:29:33.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.685+0000 7fe044fca640 1 --2- 172.21.15.112:0/2341013916 >> [v2:172.21.15.112:6833/2797048793,v1:172.21.15.112:6835/2797048793] conn(0x560a2c7c93a0 0x560a2cbf6340 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60b23cbca89baf6b server_cookie=f27ac3ae69a2a6cd in_seq=0 out_seq=0 2024-09-21T00:29:33.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.686+0000 7fe01f7fe640 1 -- 172.21.15.112:0/2341013916 <== mds.0 v2:172.21.15.112:6833/2797048793 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560a2cb4d2e0 con 0x560a2c7c93a0 2024-09-21T00:29:33.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.686+0000 7fe01f7fe640 10 client.7005 handle_command_reply: tid=0 2024-09-21T00:29:33.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.686+0000 7fe04e276740 1 client.7005 shutdown 2024-09-21T00:29:33.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.686+0000 7fe04e276740 1 -- 172.21.15.112:0/2341013916 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a2c8146b0 msgr2=0x560a2cbf5e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:33.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.686+0000 7fe04e276740 1 --2- 172.21.15.112:0/2341013916 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a2c8146b0 0x560a2cbf5e00 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fe038009de0 tx=0x7fe038009f70 comp rx=0 tx=0).stop 2024-09-21T00:29:33.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.686+0000 7fe04e276740 1 -- 172.21.15.112:0/2341013916 shutdown_connections 2024-09-21T00:29:33.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.686+0000 7fe04e276740 1 --2- 172.21.15.112:0/2341013916 >> [v2:172.21.15.112:6833/2797048793,v1:172.21.15.112:6835/2797048793] conn(0x560a2c7c93a0 0x560a2cbf6340 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:33.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.686+0000 7fe04e276740 1 --2- 172.21.15.112:0/2341013916 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a2c8146b0 0x560a2cbf5e00 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:33.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.686+0000 7fe04e276740 1 --2- 172.21.15.112:0/2341013916 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560a2c80c4d0 0x560a2cca51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:33.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.686+0000 7fe01f7fe640 0 client.7005 ms_handle_reset on v2:172.21.15.112:6833/2797048793 2024-09-21T00:29:33.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.686+0000 7fe04e276740 1 --2- 172.21.15.112:0/2341013916 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560a2c7f3f60 0x560a2cc496f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:33.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.686+0000 7fe04e276740 1 -- 172.21.15.112:0/2341013916 >> 172.21.15.112:0/2341013916 conn(0x560a2c8294e0 msgr2=0x560a2cc18aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:33.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.686+0000 7fe04e276740 1 -- 172.21.15.112:0/2341013916 shutdown_connections 2024-09-21T00:29:33.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.686+0000 7fe04e276740 1 -- 172.21.15.112:0/2341013916 wait complete. 2024-09-21T00:29:33.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.686+0000 7fe04e276740 20 client.7005 trim_cache size 0 max 16384 2024-09-21T00:29:33.692 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-09-21T00:29:33.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.686+0000 7fe04722e640 1 -- 172.21.15.112:0/4147102032 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fe02c078b90 msgr2=0x7fe02c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:33.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.686+0000 7fe04722e640 1 --2- 172.21.15.112:0/4147102032 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fe02c078b90 0x7fe02c07b0a0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fe030007fd0 tx=0x7fe030009040 comp rx=0 tx=0).stop 2024-09-21T00:29:33.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.686+0000 7fe04722e640 1 -- 172.21.15.112:0/4147102032 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0400b2680 msgr2=0x7fe0401f6dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:33.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.686+0000 7fe04722e640 1 --2- 172.21.15.112:0/4147102032 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0400b2680 0x7fe0401f6dc0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fe0340099c0 tx=0x7fe0340039e0 comp rx=0 tx=0).stop 2024-09-21T00:29:33.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.687+0000 7fe04722e640 1 -- 172.21.15.112:0/4147102032 shutdown_connections 2024-09-21T00:29:33.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.687+0000 7fe04722e640 1 --2- 172.21.15.112:0/4147102032 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe0400ccbf0 0x7fe040150250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:33.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.687+0000 7fe04722e640 1 --2- 172.21.15.112:0/4147102032 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fe02c078b90 0x7fe02c07b0a0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:33.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.687+0000 7fe04722e640 1 --2- 172.21.15.112:0/4147102032 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0400b2680 0x7fe0401f6dc0 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:33.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.687+0000 7fe04722e640 1 --2- 172.21.15.112:0/4147102032 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe0400b1d30 0x7fe040154170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:33.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.687+0000 7fe04722e640 1 -- 172.21.15.112:0/4147102032 >> 172.21.15.112:0/4147102032 conn(0x7fe0400b42e0 msgr2=0x7fe0400cd000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:33.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.687+0000 7fe04722e640 1 -- 172.21.15.112:0/4147102032 shutdown_connections 2024-09-21T00:29:33.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:33.687+0000 7fe04722e640 1 -- 172.21.15.112:0/4147102032 wait complete. 2024-09-21T00:29:33.698 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2024-09-21T00:29:33.823 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-09-21T00:29:33.823 DEBUG:tasks.cephfs.filesystem:_json_asok output [] 2024-09-21T00:29:33.824 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T00:29:33.824 DEBUG:teuthology.orchestra.run.smithi112:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T00:29:33.874 DEBUG:teuthology.orchestra.run.smithi187:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T00:29:33.917 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T00:29:33.917 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T00:29:33.918 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T00:29:33.918 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:29:33.934 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056010,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1439 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248716k,nr_inodes=812179,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T00:29:33.934 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T00:29:33.934 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-09-21T00:29:33.934 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:29:33.975 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056013,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497440k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16835 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248720k,nr_inodes=812180,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497440k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T00:29:33.976 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T00:29:33.976 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T00:29:33.976 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:29:33.991 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056010,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1439 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248716k,nr_inodes=812179,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T00:29:33.991 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T00:29:33.992 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-09-21T00:29:33.992 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:29:34.032 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056013,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497440k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16835 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248720k,nr_inodes=812180,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497440k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T00:29:34.032 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T00:29:34.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.143+0000 7f2cbae4d640 1 -- 172.21.15.112:0/1667076450 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2cb40d48a0 msgr2=0x7f2cb40d3dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:34.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.143+0000 7f2cbae4d640 1 --2- 172.21.15.112:0/1667076450 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2cb40d48a0 0x7f2cb40d3dc0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f2ca4009830 tx=0x7f2ca402a510 comp rx=0 tx=0).stop 2024-09-21T00:29:34.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.143+0000 7f2cbae4d640 1 -- 172.21.15.112:0/1667076450 shutdown_connections 2024-09-21T00:29:34.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.143+0000 7f2cbae4d640 1 --2- 172.21.15.112:0/1667076450 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2cb40d48a0 0x7f2cb40d3dc0 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:34.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.143+0000 7f2cbae4d640 1 -- 172.21.15.112:0/1667076450 >> 172.21.15.112:0/1667076450 conn(0x7f2cb408bfe0 msgr2=0x7f2cb409d890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:34.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.143+0000 7f2cbae4d640 1 -- 172.21.15.112:0/1667076450 shutdown_connections 2024-09-21T00:29:34.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.143+0000 7f2cbae4d640 1 -- 172.21.15.112:0/1667076450 wait complete. 2024-09-21T00:29:34.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.145+0000 7f2cbae4d640 1 Processor -- start 2024-09-21T00:29:34.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.145+0000 7f2cbae4d640 1 -- start start 2024-09-21T00:29:34.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.146+0000 7f2cbae4d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2cb40d01b0 0x7f2cb406d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:34.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.146+0000 7f2cbae4d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2cb40d3880 0x7f2cb4154a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:34.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.146+0000 7f2cbae4d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2cb40d48a0 0x7f2cb406f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:34.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.146+0000 7f2cbae4d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2cb40d2650 con 0x7f2cb40d48a0 2024-09-21T00:29:34.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.146+0000 7f2cbae4d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f2cb40d24d0 con 0x7f2cb40d01b0 2024-09-21T00:29:34.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.146+0000 7f2cbae4d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f2cb40d27d0 con 0x7f2cb40d3880 2024-09-21T00:29:34.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.146+0000 7f2cb93ea640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2cb40d48a0 0x7f2cb406f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:34.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.146+0000 7f2cb93ea640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2cb40d48a0 0x7f2cb406f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58056/0 (socket says 172.21.15.112:58056) 2024-09-21T00:29:34.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.146+0000 7f2cb93ea640 1 -- 172.21.15.112:0/3467738463 learned_addr learned my addr 172.21.15.112:0/3467738463 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:29:34.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.146+0000 7f2cb8be9640 1 --2- 172.21.15.112:0/3467738463 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2cb40d01b0 0x7f2cb406d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:34.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.146+0000 7f2cb3fff640 1 --2- 172.21.15.112:0/3467738463 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2cb40d3880 0x7f2cb4154a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:34.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.147+0000 7f2cb93ea640 1 -- 172.21.15.112:0/3467738463 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2cb40d3880 msgr2=0x7f2cb4154a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:34.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.147+0000 7f2cb93ea640 1 --2- 172.21.15.112:0/3467738463 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2cb40d3880 0x7f2cb4154a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:34.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.147+0000 7f2cb93ea640 1 -- 172.21.15.112:0/3467738463 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2cb40d01b0 msgr2=0x7f2cb406d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:34.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.147+0000 7f2cb93ea640 1 --2- 172.21.15.112:0/3467738463 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2cb40d01b0 0x7f2cb406d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:34.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.147+0000 7f2cb93ea640 1 -- 172.21.15.112:0/3467738463 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cb4160ed0 con 0x7f2cb40d48a0 2024-09-21T00:29:34.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.147+0000 7f2cb93ea640 1 --2- 172.21.15.112:0/3467738463 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2cb40d48a0 0x7f2cb406f110 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f2cac00c760 tx=0x7f2cac00cb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:34.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.147+0000 7f2cb17fa640 1 -- 172.21.15.112:0/3467738463 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cac003b70 con 0x7f2cb40d48a0 2024-09-21T00:29:34.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.147+0000 7f2cb17fa640 1 -- 172.21.15.112:0/3467738463 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2cac003d10 con 0x7f2cb40d48a0 2024-09-21T00:29:34.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.147+0000 7f2cbae4d640 1 -- 172.21.15.112:0/3467738463 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cb4161140 con 0x7f2cb40d48a0 2024-09-21T00:29:34.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.147+0000 7f2cb17fa640 1 -- 172.21.15.112:0/3467738463 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cac0052d0 con 0x7f2cb40d48a0 2024-09-21T00:29:34.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.147+0000 7f2cbae4d640 1 -- 172.21.15.112:0/3467738463 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cb4161700 con 0x7f2cb40d48a0 2024-09-21T00:29:34.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.148+0000 7f2cbae4d640 1 -- 172.21.15.112:0/3467738463 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cb40d05c0 con 0x7f2cb40d48a0 2024-09-21T00:29:34.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.149+0000 7f2cb17fa640 1 -- 172.21.15.112:0/3467738463 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2cac005470 con 0x7f2cb40d48a0 2024-09-21T00:29:34.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.149+0000 7f2cb17fa640 1 --2- 172.21.15.112:0/3467738463 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2ca0078b90 0x7f2ca007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:34.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.149+0000 7f2cb17fa640 1 -- 172.21.15.112:0/3467738463 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 6188+0+0 (secure 0 0 0) 0x7f2cac09ab60 con 0x7f2cb40d48a0 2024-09-21T00:29:34.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.149+0000 7f2cb8be9640 1 --2- 172.21.15.112:0/3467738463 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2ca0078b90 0x7f2ca007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:34.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.150+0000 7f2cb8be9640 1 --2- 172.21.15.112:0/3467738463 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2ca0078b90 0x7f2ca007b0a0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f2ca4034000 tx=0x7f2ca4004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:34.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.153+0000 7f2cb17fa640 1 -- 172.21.15.112:0/3467738463 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2cac062770 con 0x7f2cb40d48a0 2024-09-21T00:29:34.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.365+0000 7f2cbae4d640 1 -- 172.21.15.112:0/3467738463 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2cb415edd0 con 0x7f2cb40d48a0 2024-09-21T00:29:34.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.365+0000 7f2cb17fa640 1 -- 172.21.15.112:0/3467738463 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) ==== 94+0+5237 (secure 0 0 0) 0x7f2cac0671a0 con 0x7f2cb40d48a0 2024-09-21T00:29:34.366 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:29:34.366 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":64,"btime":"2024-09-21T00:29:22:809921+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5804,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/3818262153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":3818262153},{"type":"v1","addr":"172.21.15.187:6835","nonce":3818262153}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59},{"gid":6214,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6837/1321768683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":1321768683},{"type":"v1","addr":"172.21.15.187:6837","nonce":1321768683}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59},{"gid":6801,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6834/4037012343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":4037012343},{"type":"v1","addr":"172.21.15.112:6834","nonce":4037012343}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T00:29:19.145794+0000","modified":"2024-09-21T00:29:22.172462+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6804},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6804":{"gid":6804,"name":"c","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6835/2797048793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":2797048793},{"type":"v1","addr":"172.21.15.112:6835","nonce":2797048793}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6804,"qdb_cluster":[6804]},"id":7}]} 2024-09-21T00:29:34.367 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 64 2024-09-21T00:29:34.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.367+0000 7f2cbae4d640 1 -- 172.21.15.112:0/3467738463 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2ca0078b90 msgr2=0x7f2ca007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:34.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.367+0000 7f2cbae4d640 1 --2- 172.21.15.112:0/3467738463 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2ca0078b90 0x7f2ca007b0a0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f2ca4034000 tx=0x7f2ca4004730 comp rx=0 tx=0).stop 2024-09-21T00:29:34.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.367+0000 7f2cbae4d640 1 -- 172.21.15.112:0/3467738463 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2cb40d48a0 msgr2=0x7f2cb406f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:34.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.367+0000 7f2cbae4d640 1 --2- 172.21.15.112:0/3467738463 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2cb40d48a0 0x7f2cb406f110 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f2cac00c760 tx=0x7f2cac00cb60 comp rx=0 tx=0).stop 2024-09-21T00:29:34.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.367+0000 7f2cbae4d640 1 -- 172.21.15.112:0/3467738463 shutdown_connections 2024-09-21T00:29:34.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.367+0000 7f2cbae4d640 1 --2- 172.21.15.112:0/3467738463 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2ca0078b90 0x7f2ca007b0a0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:34.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.367+0000 7f2cbae4d640 1 --2- 172.21.15.112:0/3467738463 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2cb40d48a0 0x7f2cb406f110 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:34.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.367+0000 7f2cbae4d640 1 --2- 172.21.15.112:0/3467738463 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2cb40d3880 0x7f2cb4154a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:34.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.367+0000 7f2cbae4d640 1 --2- 172.21.15.112:0/3467738463 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2cb40d01b0 0x7f2cb406d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:34.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.367+0000 7f2cbae4d640 1 -- 172.21.15.112:0/3467738463 >> 172.21.15.112:0/3467738463 conn(0x7f2cb408bfe0 msgr2=0x7f2cb40b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:34.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.368+0000 7f2cbae4d640 1 -- 172.21.15.112:0/3467738463 shutdown_connections 2024-09-21T00:29:34.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.368+0000 7f2cbae4d640 1 -- 172.21.15.112:0/3467738463 wait complete. 2024-09-21T00:29:34.379 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T00:29:34.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.490+0000 7f0c88d41640 1 -- 172.21.15.112:0/742651381 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c8415d870 msgr2=0x7f0c840d0860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:34.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.490+0000 7f0c88d41640 1 --2- 172.21.15.112:0/742651381 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c8415d870 0x7f0c840d0860 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c009830 tx=0x7f0c6c02a510 comp rx=0 tx=0).stop 2024-09-21T00:29:34.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.490+0000 7f0c88d41640 1 -- 172.21.15.112:0/742651381 shutdown_connections 2024-09-21T00:29:34.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.490+0000 7f0c88d41640 1 --2- 172.21.15.112:0/742651381 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0c840ccae0 0x7f0c840ccef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:34.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.490+0000 7f0c88d41640 1 --2- 172.21.15.112:0/742651381 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c8415d870 0x7f0c840d0860 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:34.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.490+0000 7f0c88d41640 1 -- 172.21.15.112:0/742651381 >> 172.21.15.112:0/742651381 conn(0x7f0c840b43f0 msgr2=0x7f0c840b4830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:34.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.490+0000 7f0c88d41640 1 -- 172.21.15.112:0/742651381 shutdown_connections 2024-09-21T00:29:34.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.490+0000 7f0c88d41640 1 -- 172.21.15.112:0/742651381 wait complete. 2024-09-21T00:29:34.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.493+0000 7f0c88d41640 1 Processor -- start 2024-09-21T00:29:34.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.493+0000 7f0c88d41640 1 -- start start 2024-09-21T00:29:34.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.494+0000 7f0c88d41640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0c840ccae0 0x7f0c8415ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:34.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.494+0000 7f0c88d41640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c840d2030 0x7f0c84155840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:34.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.494+0000 7f0c88d41640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0c8415d870 0x7f0c841f6d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:34.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.494+0000 7f0c88d41640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0c840b22e0 con 0x7f0c840d2030 2024-09-21T00:29:34.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.494+0000 7f0c88d41640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0c840b2160 con 0x7f0c8415d870 2024-09-21T00:29:34.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.494+0000 7f0c88d41640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f0c840b2460 con 0x7f0c840ccae0 2024-09-21T00:29:34.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.494+0000 7f0c79d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c840d2030 0x7f0c84155840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:34.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.494+0000 7f0c79d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c840d2030 0x7f0c84155840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58074/0 (socket says 172.21.15.112:58074) 2024-09-21T00:29:34.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.494+0000 7f0c79d9b640 1 -- 172.21.15.112:0/1340445939 learned_addr learned my addr 172.21.15.112:0/1340445939 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:29:34.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.494+0000 7f0c82d9d640 1 --2- 172.21.15.112:0/1340445939 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0c8415d870 0x7f0c841f6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:34.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.494+0000 7f0c8259c640 1 --2- 172.21.15.112:0/1340445939 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0c840ccae0 0x7f0c8415ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:34.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.494+0000 7f0c79d9b640 1 -- 172.21.15.112:0/1340445939 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0c840ccae0 msgr2=0x7f0c8415ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:34.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.494+0000 7f0c79d9b640 1 --2- 172.21.15.112:0/1340445939 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0c840ccae0 0x7f0c8415ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:34.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.494+0000 7f0c79d9b640 1 -- 172.21.15.112:0/1340445939 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0c8415d870 msgr2=0x7f0c841f6d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:34.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.494+0000 7f0c79d9b640 1 --2- 172.21.15.112:0/1340445939 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0c8415d870 0x7f0c841f6d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:34.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.494+0000 7f0c79d9b640 1 -- 172.21.15.112:0/1340445939 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c84157e30 con 0x7f0c840d2030 2024-09-21T00:29:34.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.495+0000 7f0c79d9b640 1 --2- 172.21.15.112:0/1340445939 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c840d2030 0x7f0c84155840 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c009950 tx=0x7f0c6c033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:34.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.495+0000 7f0c7b7fe640 1 -- 172.21.15.112:0/1340445939 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c6c005c90 con 0x7f0c840d2030 2024-09-21T00:29:34.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.495+0000 7f0c7b7fe640 1 -- 172.21.15.112:0/1340445939 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0c6c031e50 con 0x7f0c840d2030 2024-09-21T00:29:34.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.495+0000 7f0c7b7fe640 1 -- 172.21.15.112:0/1340445939 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c6c005260 con 0x7f0c840d2030 2024-09-21T00:29:34.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.495+0000 7f0c88d41640 1 -- 172.21.15.112:0/1340445939 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c841582a0 con 0x7f0c840d2030 2024-09-21T00:29:34.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.495+0000 7f0c88d41640 1 -- 172.21.15.112:0/1340445939 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c84158860 con 0x7f0c840d2030 2024-09-21T00:29:34.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.496+0000 7f0c88d41640 1 -- 172.21.15.112:0/1340445939 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c840b2980 con 0x7f0c840d2030 2024-09-21T00:29:34.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.497+0000 7f0c7b7fe640 1 -- 172.21.15.112:0/1340445939 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0c6c005400 con 0x7f0c840d2030 2024-09-21T00:29:34.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.497+0000 7f0c7b7fe640 1 --2- 172.21.15.112:0/1340445939 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f0c64078b90 0x7f0c6407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:34.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.497+0000 7f0c7b7fe640 1 -- 172.21.15.112:0/1340445939 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 6188+0+0 (secure 0 0 0) 0x7f0c6c0bbfc0 con 0x7f0c840d2030 2024-09-21T00:29:34.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.497+0000 7f0c8259c640 1 --2- 172.21.15.112:0/1340445939 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f0c64078b90 0x7f0c6407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:34.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.498+0000 7f0c8259c640 1 --2- 172.21.15.112:0/1340445939 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f0c64078b90 0x7f0c6407b0a0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f0c74007d60 tx=0x7f0c74007cf0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:34.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.502+0000 7f0c7b7fe640 1 -- 172.21.15.112:0/1340445939 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f0c6c03f030 con 0x7f0c840d2030 2024-09-21T00:29:34.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.713+0000 7f0c88d41640 1 -- 172.21.15.112:0/1340445939 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0c84158000 con 0x7f0c840d2030 2024-09-21T00:29:34.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.713+0000 7f0c7b7fe640 1 -- 172.21.15.112:0/1340445939 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) ==== 94+0+5237 (secure 0 0 0) 0x7f0c6c083be0 con 0x7f0c840d2030 2024-09-21T00:29:34.714 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:29:34.714 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":64,"btime":"2024-09-21T00:29:22:809921+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5804,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/3818262153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":3818262153},{"type":"v1","addr":"172.21.15.187:6835","nonce":3818262153}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59},{"gid":6214,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6837/1321768683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":1321768683},{"type":"v1","addr":"172.21.15.187:6837","nonce":1321768683}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59},{"gid":6801,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6834/4037012343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":4037012343},{"type":"v1","addr":"172.21.15.112:6834","nonce":4037012343}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T00:29:19.145794+0000","modified":"2024-09-21T00:29:22.172462+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6804},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6804":{"gid":6804,"name":"c","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6835/2797048793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":2797048793},{"type":"v1","addr":"172.21.15.112:6835","nonce":2797048793}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6804,"qdb_cluster":[6804]},"id":7}]} 2024-09-21T00:29:34.715 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 64 2024-09-21T00:29:34.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.715+0000 7f0c88d41640 1 -- 172.21.15.112:0/1340445939 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f0c64078b90 msgr2=0x7f0c6407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:34.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.715+0000 7f0c88d41640 1 --2- 172.21.15.112:0/1340445939 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f0c64078b90 0x7f0c6407b0a0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f0c74007d60 tx=0x7f0c74007cf0 comp rx=0 tx=0).stop 2024-09-21T00:29:34.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.715+0000 7f0c88d41640 1 -- 172.21.15.112:0/1340445939 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c840d2030 msgr2=0x7f0c84155840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:34.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.715+0000 7f0c88d41640 1 --2- 172.21.15.112:0/1340445939 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c840d2030 0x7f0c84155840 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c009950 tx=0x7f0c6c033c20 comp rx=0 tx=0).stop 2024-09-21T00:29:34.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.716+0000 7f0c88d41640 1 -- 172.21.15.112:0/1340445939 shutdown_connections 2024-09-21T00:29:34.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.716+0000 7f0c88d41640 1 --2- 172.21.15.112:0/1340445939 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0c8415d870 0x7f0c841f6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:34.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.716+0000 7f0c88d41640 1 --2- 172.21.15.112:0/1340445939 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f0c64078b90 0x7f0c6407b0a0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:34.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.716+0000 7f0c88d41640 1 --2- 172.21.15.112:0/1340445939 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c840d2030 0x7f0c84155840 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:34.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.716+0000 7f0c88d41640 1 --2- 172.21.15.112:0/1340445939 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0c840ccae0 0x7f0c8415ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:34.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.716+0000 7f0c88d41640 1 -- 172.21.15.112:0/1340445939 >> 172.21.15.112:0/1340445939 conn(0x7f0c840b43f0 msgr2=0x7f0c8415c160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:34.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.716+0000 7f0c88d41640 1 -- 172.21.15.112:0/1340445939 shutdown_connections 2024-09-21T00:29:34.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.716+0000 7f0c88d41640 1 -- 172.21.15.112:0/1340445939 wait complete. 2024-09-21T00:29:34.727 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T00:29:34.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.840+0000 7fa69b5c3640 1 -- 172.21.15.112:0/3368978703 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa6940b48f0 msgr2=0x7fa6940c5ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:34.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.840+0000 7fa69b5c3640 1 --2- 172.21.15.112:0/3368978703 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa6940b48f0 0x7fa6940c5ed0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fa684009830 tx=0x7fa68402a510 comp rx=0 tx=0).stop 2024-09-21T00:29:34.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.840+0000 7fa69b5c3640 1 -- 172.21.15.112:0/3368978703 shutdown_connections 2024-09-21T00:29:34.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.840+0000 7fa69b5c3640 1 --2- 172.21.15.112:0/3368978703 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa6940ccce0 0x7fa6940c6410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:34.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.840+0000 7fa69b5c3640 1 --2- 172.21.15.112:0/3368978703 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa6940b48f0 0x7fa6940c5ed0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:34.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.840+0000 7fa69b5c3640 1 -- 172.21.15.112:0/3368978703 >> 172.21.15.112:0/3368978703 conn(0x7fa6940cf220 msgr2=0x7fa6940cf630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:34.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.840+0000 7fa69b5c3640 1 -- 172.21.15.112:0/3368978703 shutdown_connections 2024-09-21T00:29:34.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.840+0000 7fa69b5c3640 1 -- 172.21.15.112:0/3368978703 wait complete. 2024-09-21T00:29:34.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.842+0000 7fa69b5c3640 1 Processor -- start 2024-09-21T00:29:34.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.842+0000 7fa69b5c3640 1 -- start start 2024-09-21T00:29:34.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.843+0000 7fa69b5c3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6940ccce0 0x7fa694154010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:34.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.843+0000 7fa69b5c3640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa69415ca50 0x7fa6940cb600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:34.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.843+0000 7fa69b5c3640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa694158610 0x7fa6940cbb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:34.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.843+0000 7fa69b5c3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa6940d2130 con 0x7fa6940ccce0 2024-09-21T00:29:34.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.843+0000 7fa69b5c3640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa6940d1fb0 con 0x7fa694158610 2024-09-21T00:29:34.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.843+0000 7fa69b5c3640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa6940c6950 con 0x7fa69415ca50 2024-09-21T00:29:34.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.843+0000 7fa69935f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6940ccce0 0x7fa694154010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:34.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.844+0000 7fa69935f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6940ccce0 0x7fa694154010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58076/0 (socket says 172.21.15.112:58076) 2024-09-21T00:29:34.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.844+0000 7fa69935f640 1 -- 172.21.15.112:0/1798108484 learned_addr learned my addr 172.21.15.112:0/1798108484 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:29:34.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.844+0000 7fa698b5e640 1 --2- 172.21.15.112:0/1798108484 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa69415ca50 0x7fa6940cb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:34.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.844+0000 7fa699b60640 1 --2- 172.21.15.112:0/1798108484 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa694158610 0x7fa6940cbb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:34.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.844+0000 7fa69935f640 1 -- 172.21.15.112:0/1798108484 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa69415ca50 msgr2=0x7fa6940cb600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:34.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.844+0000 7fa69935f640 1 --2- 172.21.15.112:0/1798108484 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa69415ca50 0x7fa6940cb600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:34.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.844+0000 7fa69935f640 1 -- 172.21.15.112:0/1798108484 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa694158610 msgr2=0x7fa6940cbb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:34.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.844+0000 7fa69935f640 1 --2- 172.21.15.112:0/1798108484 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa694158610 0x7fa6940cbb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:34.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.844+0000 7fa69935f640 1 -- 172.21.15.112:0/1798108484 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa69415ee10 con 0x7fa6940ccce0 2024-09-21T00:29:34.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.844+0000 7fa698b5e640 1 --2- 172.21.15.112:0/1798108484 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa69415ca50 0x7fa6940cb600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:29:34.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.844+0000 7fa69935f640 1 --2- 172.21.15.112:0/1798108484 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6940ccce0 0x7fa694154010 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fa69000c3a0 tx=0x7fa69000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:34.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.844+0000 7fa689ffb640 1 -- 172.21.15.112:0/1798108484 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa690011790 con 0x7fa6940ccce0 2024-09-21T00:29:34.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.844+0000 7fa689ffb640 1 -- 172.21.15.112:0/1798108484 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa690017070 con 0x7fa6940ccce0 2024-09-21T00:29:34.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.844+0000 7fa69b5c3640 1 -- 172.21.15.112:0/1798108484 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa69415f0f0 con 0x7fa6940ccce0 2024-09-21T00:29:34.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.844+0000 7fa689ffb640 1 -- 172.21.15.112:0/1798108484 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa690011cc0 con 0x7fa6940ccce0 2024-09-21T00:29:34.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.844+0000 7fa69b5c3640 1 -- 172.21.15.112:0/1798108484 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa694157e10 con 0x7fa6940ccce0 2024-09-21T00:29:34.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.846+0000 7fa69b5c3640 1 -- 172.21.15.112:0/1798108484 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6940c5ed0 con 0x7fa6940ccce0 2024-09-21T00:29:34.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.846+0000 7fa689ffb640 1 -- 172.21.15.112:0/1798108484 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa69001c460 con 0x7fa6940ccce0 2024-09-21T00:29:34.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.846+0000 7fa689ffb640 1 --2- 172.21.15.112:0/1798108484 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fa678078b90 0x7fa67807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:34.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.847+0000 7fa689ffb640 1 -- 172.21.15.112:0/1798108484 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 6188+0+0 (secure 0 0 0) 0x7fa69009aba0 con 0x7fa6940ccce0 2024-09-21T00:29:34.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.847+0000 7fa698b5e640 1 --2- 172.21.15.112:0/1798108484 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fa678078b90 0x7fa67807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:34.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.847+0000 7fa698b5e640 1 --2- 172.21.15.112:0/1798108484 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fa678078b90 0x7fa67807b0a0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fa684034000 tx=0x7fa684004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:34.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:34.851+0000 7fa689ffb640 1 -- 172.21.15.112:0/1798108484 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa690062ea0 con 0x7fa6940ccce0 2024-09-21T00:29:35.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.029+0000 7fa69b5c3640 1 -- 172.21.15.112:0/1798108484 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa69415bd00 con 0x7fa6940ccce0 2024-09-21T00:29:35.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.030+0000 7fa689ffb640 1 -- 172.21.15.112:0/1798108484 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) ==== 74+0+17102 (secure 0 0 0) 0x7fa6900627a0 con 0x7fa6940ccce0 2024-09-21T00:29:35.031 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:29:35.031 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":96,"fsid":"8b34692b-dc65-49b2-a08f-26f707b96ac5","created":"2024-09-21T00:25:22.788490+0000","modified":"2024-09-21T00:29:28.841606+0000","last_up_change":"2024-09-21T00:25:36.161835+0000","last_in_change":"2024-09-21T00:25:26.346500+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T00:25:37.105006+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-09-21T00:29:16.794043+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-09-21T00:29:18.114604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9c575086-67d2-4de2-b01b-d44dcc21d604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6809","nonce":755760621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6811","nonce":755760621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6815","nonce":755760621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6813","nonce":755760621}]},"public_addr":"172.21.15.112:6809/755760621","cluster_addr":"172.21.15.112:6811/755760621","heartbeat_back_addr":"172.21.15.112:6815/755760621","heartbeat_front_addr":"172.21.15.112:6813/755760621","state":["exists","up"]},{"osd":1,"uuid":"a983c227-a3ce-4bdd-82f1-438ba7e3ecd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6801","nonce":344932592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6803","nonce":344932592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6807","nonce":344932592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6805","nonce":344932592}]},"public_addr":"172.21.15.112:6801/344932592","cluster_addr":"172.21.15.112:6803/344932592","heartbeat_back_addr":"172.21.15.112:6807/344932592","heartbeat_front_addr":"172.21.15.112:6805/344932592","state":["exists","up"]},{"osd":2,"uuid":"1193c1d8-bfff-4a57-8200-5f69bf1ea3af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6825","nonce":407933616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6827","nonce":407933616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6831","nonce":407933616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6829","nonce":407933616}]},"public_addr":"172.21.15.112:6825/407933616","cluster_addr":"172.21.15.112:6827/407933616","heartbeat_back_addr":"172.21.15.112:6831/407933616","heartbeat_front_addr":"172.21.15.112:6829/407933616","state":["exists","up"]},{"osd":3,"uuid":"af04330a-36ad-40cc-a8dd-fa259b221de5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6817","nonce":4120164806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6819","nonce":4120164806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6823","nonce":4120164806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6821","nonce":4120164806}]},"public_addr":"172.21.15.112:6817/4120164806","cluster_addr":"172.21.15.112:6819/4120164806","heartbeat_back_addr":"172.21.15.112:6823/4120164806","heartbeat_front_addr":"172.21.15.112:6821/4120164806","state":["exists","up"]},{"osd":4,"uuid":"5b893111-292c-4c5a-b184-53ad38c7199e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6803","nonce":522077560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6805","nonce":522077560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6809","nonce":522077560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6807","nonce":522077560}]},"public_addr":"172.21.15.187:6803/522077560","cluster_addr":"172.21.15.187:6805/522077560","heartbeat_back_addr":"172.21.15.187:6809/522077560","heartbeat_front_addr":"172.21.15.187:6807/522077560","state":["exists","up"]},{"osd":5,"uuid":"c0b8e011-45fe-4c5e-925e-6be2a7f1868a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6827","nonce":3593144721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6829","nonce":3593144721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6833","nonce":3593144721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6831","nonce":3593144721}]},"public_addr":"172.21.15.187:6827/3593144721","cluster_addr":"172.21.15.187:6829/3593144721","heartbeat_back_addr":"172.21.15.187:6833/3593144721","heartbeat_front_addr":"172.21.15.187:6831/3593144721","state":["exists","up"]},{"osd":6,"uuid":"87d2d650-7489-4593-98a3-a1f433ce8dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6811","nonce":1494387657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6813","nonce":1494387657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6817","nonce":1494387657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6815","nonce":1494387657}]},"public_addr":"172.21.15.187:6811/1494387657","cluster_addr":"172.21.15.187:6813/1494387657","heartbeat_back_addr":"172.21.15.187:6817/1494387657","heartbeat_front_addr":"172.21.15.187:6815/1494387657","state":["exists","up"]},{"osd":7,"uuid":"c6b4f8e2-a1ee-4438-aa37-67ff4b6ef1f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6819","nonce":1624699842}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6821","nonce":1624699842}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6825","nonce":1624699842}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6823","nonce":1624699842}]},"public_addr":"172.21.15.187:6819/1624699842","cluster_addr":"172.21.15.187:6821/1624699842","heartbeat_back_addr":"172.21.15.187:6825/1624699842","heartbeat_front_addr":"172.21.15.187:6823/1624699842","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.873162+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.400506+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.664317+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.159791+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.202630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.113212+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.384786+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.448441+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.6","mappings":[{"from":6,"to":7}]},{"pgid":"16.13","mappings":[{"from":1,"to":4}]},{"pgid":"16.29","mappings":[{"from":5,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T00:29:35.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.032+0000 7fa69b5c3640 1 -- 172.21.15.112:0/1798108484 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fa678078b90 msgr2=0x7fa67807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:35.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.032+0000 7fa69b5c3640 1 --2- 172.21.15.112:0/1798108484 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fa678078b90 0x7fa67807b0a0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fa684034000 tx=0x7fa684004730 comp rx=0 tx=0).stop 2024-09-21T00:29:35.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.032+0000 7fa69b5c3640 1 -- 172.21.15.112:0/1798108484 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6940ccce0 msgr2=0x7fa694154010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:35.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.032+0000 7fa69b5c3640 1 --2- 172.21.15.112:0/1798108484 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6940ccce0 0x7fa694154010 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fa69000c3a0 tx=0x7fa69000c860 comp rx=0 tx=0).stop 2024-09-21T00:29:35.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.032+0000 7fa69b5c3640 1 -- 172.21.15.112:0/1798108484 shutdown_connections 2024-09-21T00:29:35.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.032+0000 7fa69b5c3640 1 --2- 172.21.15.112:0/1798108484 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa694158610 0x7fa6940cbb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:35.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.032+0000 7fa69b5c3640 1 --2- 172.21.15.112:0/1798108484 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa69415ca50 0x7fa6940cb600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:35.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.032+0000 7fa69b5c3640 1 --2- 172.21.15.112:0/1798108484 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fa678078b90 0x7fa67807b0a0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:35.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.032+0000 7fa69b5c3640 1 --2- 172.21.15.112:0/1798108484 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6940ccce0 0x7fa694154010 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:35.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.032+0000 7fa69b5c3640 1 -- 172.21.15.112:0/1798108484 >> 172.21.15.112:0/1798108484 conn(0x7fa6940cf220 msgr2=0x7fa6940cf630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:35.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.032+0000 7fa69b5c3640 1 -- 172.21.15.112:0/1798108484 shutdown_connections 2024-09-21T00:29:35.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.032+0000 7fa69b5c3640 1 -- 172.21.15.112:0/1798108484 wait complete. 2024-09-21T00:29:35.044 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-21T00:29:35.045 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T00:29:35.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.156+0000 7f2ac429b640 1 -- 172.21.15.112:0/2236664368 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2abc0b41c0 msgr2=0x7f2abc0b45d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:35.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.156+0000 7f2ac429b640 1 --2- 172.21.15.112:0/2236664368 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2abc0b41c0 0x7f2abc0b45d0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f2abc0c93e0 tx=0x7f2ab402a820 comp rx=0 tx=0).stop 2024-09-21T00:29:35.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.156+0000 7f2ac429b640 1 -- 172.21.15.112:0/2236664368 shutdown_connections 2024-09-21T00:29:35.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.156+0000 7f2ac429b640 1 --2- 172.21.15.112:0/2236664368 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2abc0b41c0 0x7f2abc0b45d0 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:35.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.156+0000 7f2ac429b640 1 --2- 172.21.15.112:0/2236664368 >> v2:172.21.15.187:3300/0 conn(0x7f2abc0d1a80 0x7f2abc15c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:35.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.156+0000 7f2ac429b640 1 -- 172.21.15.112:0/2236664368 >> 172.21.15.112:0/2236664368 conn(0x7f2abc0cf200 msgr2=0x7f2abc0cf610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:35.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.156+0000 7f2ac429b640 1 -- 172.21.15.112:0/2236664368 shutdown_connections 2024-09-21T00:29:35.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.156+0000 7f2ac429b640 1 -- 172.21.15.112:0/2236664368 wait complete. 2024-09-21T00:29:35.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.158+0000 7f2ac429b640 1 Processor -- start 2024-09-21T00:29:35.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.158+0000 7f2ac429b640 1 -- start start 2024-09-21T00:29:35.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.159+0000 7f2ac429b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2abc0b1c10 0x7f2abc0b0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:35.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.159+0000 7f2ac429b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2abc0b41c0 0x7f2abc147c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:35.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.159+0000 7f2ac429b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2abc0d1a80 0x7f2abc148150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:35.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.159+0000 7f2ac429b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2abc0d0d00 con 0x7f2abc0d1a80 2024-09-21T00:29:35.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.159+0000 7f2ac429b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f2abc0d0b80 con 0x7f2abc0b41c0 2024-09-21T00:29:35.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.160+0000 7f2ac429b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f2abc0d0e80 con 0x7f2abc0b1c10 2024-09-21T00:29:35.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.160+0000 7f2ac2838640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2abc0d1a80 0x7f2abc148150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:35.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.160+0000 7f2ac2838640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2abc0d1a80 0x7f2abc148150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58088/0 (socket says 172.21.15.112:58088) 2024-09-21T00:29:35.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.160+0000 7f2ac2838640 1 -- 172.21.15.112:0/4013082461 learned_addr learned my addr 172.21.15.112:0/4013082461 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:29:35.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.160+0000 7f2ac1836640 1 --2- 172.21.15.112:0/4013082461 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2abc0b41c0 0x7f2abc147c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:35.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.160+0000 7f2ac2037640 1 --2- 172.21.15.112:0/4013082461 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2abc0b1c10 0x7f2abc0b0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:35.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.160+0000 7f2ac2838640 1 -- 172.21.15.112:0/4013082461 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2abc0b1c10 msgr2=0x7f2abc0b0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:35.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.160+0000 7f2ac2838640 1 --2- 172.21.15.112:0/4013082461 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2abc0b1c10 0x7f2abc0b0ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:35.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.160+0000 7f2ac2838640 1 -- 172.21.15.112:0/4013082461 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2abc0b41c0 msgr2=0x7f2abc147c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:35.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.160+0000 7f2ac2838640 1 --2- 172.21.15.112:0/4013082461 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2abc0b41c0 0x7f2abc147c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:35.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.160+0000 7f2ac2838640 1 -- 172.21.15.112:0/4013082461 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2abc147450 con 0x7f2abc0d1a80 2024-09-21T00:29:35.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.160+0000 7f2ac2037640 1 --2- 172.21.15.112:0/4013082461 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2abc0b1c10 0x7f2abc0b0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:29:35.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.160+0000 7f2ac1836640 1 --2- 172.21.15.112:0/4013082461 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2abc0b41c0 0x7f2abc147c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:29:35.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.160+0000 7f2ac2838640 1 --2- 172.21.15.112:0/4013082461 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2abc0d1a80 0x7f2abc148150 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f2ab4009ca0 tx=0x7f2ab4032c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:35.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.160+0000 7f2ab2ffd640 1 -- 172.21.15.112:0/4013082461 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ab403f020 con 0x7f2abc0d1a80 2024-09-21T00:29:35.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.160+0000 7f2ab2ffd640 1 -- 172.21.15.112:0/4013082461 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2ab4004040 con 0x7f2abc0d1a80 2024-09-21T00:29:35.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.160+0000 7f2ac429b640 1 -- 172.21.15.112:0/4013082461 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2abc0cc090 con 0x7f2abc0d1a80 2024-09-21T00:29:35.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.160+0000 7f2ab2ffd640 1 -- 172.21.15.112:0/4013082461 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ab4037040 con 0x7f2abc0d1a80 2024-09-21T00:29:35.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.160+0000 7f2ac429b640 1 -- 172.21.15.112:0/4013082461 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2abc148c90 con 0x7f2abc0d1a80 2024-09-21T00:29:35.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.161+0000 7f2ac429b640 1 -- 172.21.15.112:0/4013082461 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2abc148ef0 con 0x7f2abc0d1a80 2024-09-21T00:29:35.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.162+0000 7f2ab2ffd640 1 -- 172.21.15.112:0/4013082461 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2ab4042020 con 0x7f2abc0d1a80 2024-09-21T00:29:35.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.163+0000 7f2ab2ffd640 1 --2- 172.21.15.112:0/4013082461 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2aa0078bc0 0x7f2aa007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:35.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.163+0000 7f2ab2ffd640 1 -- 172.21.15.112:0/4013082461 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 6188+0+0 (secure 0 0 0) 0x7f2ab40bb430 con 0x7f2abc0d1a80 2024-09-21T00:29:35.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.163+0000 7f2ac2037640 1 --2- 172.21.15.112:0/4013082461 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2aa0078bc0 0x7f2aa007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:35.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.163+0000 7f2ac2037640 1 --2- 172.21.15.112:0/4013082461 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2aa0078bc0 0x7f2aa007b0d0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f2abc0750a0 tx=0x7f2ab8005d10 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:35.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.167+0000 7f2ab2ffd640 1 -- 172.21.15.112:0/4013082461 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2ab4049070 con 0x7f2abc0d1a80 2024-09-21T00:29:35.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.379+0000 7f2ac429b640 1 -- 172.21.15.112:0/4013082461 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2abc0d12a0 con 0x7f2abc0d1a80 2024-09-21T00:29:35.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.379+0000 7f2ab2ffd640 1 -- 172.21.15.112:0/4013082461 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) ==== 94+0+5237 (secure 0 0 0) 0x7f2ab4083690 con 0x7f2abc0d1a80 2024-09-21T00:29:35.380 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:29:35.380 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":64,"btime":"2024-09-21T00:29:22:809921+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5804,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/3818262153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":3818262153},{"type":"v1","addr":"172.21.15.187:6835","nonce":3818262153}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59},{"gid":6214,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6837/1321768683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":1321768683},{"type":"v1","addr":"172.21.15.187:6837","nonce":1321768683}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59},{"gid":6801,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6834/4037012343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":4037012343},{"type":"v1","addr":"172.21.15.112:6834","nonce":4037012343}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T00:29:19.145794+0000","modified":"2024-09-21T00:29:22.172462+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6804},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6804":{"gid":6804,"name":"c","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6835/2797048793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":2797048793},{"type":"v1","addr":"172.21.15.112:6835","nonce":2797048793}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6804,"qdb_cluster":[6804]},"id":7}]} 2024-09-21T00:29:35.381 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 64 2024-09-21T00:29:35.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.381+0000 7f2ac429b640 1 -- 172.21.15.112:0/4013082461 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2aa0078bc0 msgr2=0x7f2aa007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:35.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.381+0000 7f2ac429b640 1 --2- 172.21.15.112:0/4013082461 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2aa0078bc0 0x7f2aa007b0d0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f2abc0750a0 tx=0x7f2ab8005d10 comp rx=0 tx=0).stop 2024-09-21T00:29:35.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.381+0000 7f2ac429b640 1 -- 172.21.15.112:0/4013082461 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2abc0d1a80 msgr2=0x7f2abc148150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:35.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.381+0000 7f2ac429b640 1 --2- 172.21.15.112:0/4013082461 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2abc0d1a80 0x7f2abc148150 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f2ab4009ca0 tx=0x7f2ab4032c90 comp rx=0 tx=0).stop 2024-09-21T00:29:35.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.382+0000 7f2ac429b640 1 -- 172.21.15.112:0/4013082461 shutdown_connections 2024-09-21T00:29:35.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.382+0000 7f2ac429b640 1 --2- 172.21.15.112:0/4013082461 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2aa0078bc0 0x7f2aa007b0d0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:35.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.382+0000 7f2ac429b640 1 --2- 172.21.15.112:0/4013082461 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2abc0d1a80 0x7f2abc148150 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:35.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.382+0000 7f2ac429b640 1 --2- 172.21.15.112:0/4013082461 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2abc0b41c0 0x7f2abc147c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:35.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.382+0000 7f2ac429b640 1 --2- 172.21.15.112:0/4013082461 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2abc0b1c10 0x7f2abc0b0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:35.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.382+0000 7f2ac429b640 1 -- 172.21.15.112:0/4013082461 >> 172.21.15.112:0/4013082461 conn(0x7f2abc0cf200 msgr2=0x7f2abc0cf610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:35.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.382+0000 7f2ac429b640 1 -- 172.21.15.112:0/4013082461 shutdown_connections 2024-09-21T00:29:35.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.382+0000 7f2ac429b640 1 -- 172.21.15.112:0/4013082461 wait complete. 2024-09-21T00:29:35.394 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T00:29:35.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.504+0000 7f15040f9640 1 -- 172.21.15.112:0/3600426911 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14fc0ce9e0 msgr2=0x7f14fc0caef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:35.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.504+0000 7f15040f9640 1 --2- 172.21.15.112:0/3600426911 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14fc0ce9e0 0x7f14fc0caef0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f14f4009830 tx=0x7f14f402a5a0 comp rx=0 tx=0).stop 2024-09-21T00:29:35.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.504+0000 7f15040f9640 1 -- 172.21.15.112:0/3600426911 shutdown_connections 2024-09-21T00:29:35.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.504+0000 7f15040f9640 1 --2- 172.21.15.112:0/3600426911 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14fc0ce9e0 0x7f14fc0caef0 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:35.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.504+0000 7f15040f9640 1 -- 172.21.15.112:0/3600426911 >> 172.21.15.112:0/3600426911 conn(0x7f14fc0b3580 msgr2=0x7f14fc0b3990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:35.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.504+0000 7f15040f9640 1 -- 172.21.15.112:0/3600426911 shutdown_connections 2024-09-21T00:29:35.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.504+0000 7f15040f9640 1 -- 172.21.15.112:0/3600426911 wait complete. 2024-09-21T00:29:35.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.506+0000 7f15040f9640 1 Processor -- start 2024-09-21T00:29:35.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.506+0000 7f15040f9640 1 -- start start 2024-09-21T00:29:35.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.507+0000 7f15040f9640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f14fc0cc270 0x7f14fc0afbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:35.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.507+0000 7f15040f9640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f14fc0ce9e0 0x7f14fc1e6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:35.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.507+0000 7f15040f9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14fc0b2940 0x7f14fc155c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:35.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.507+0000 7f15040f9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f14fc0cb5b0 con 0x7f14fc0b2940 2024-09-21T00:29:35.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.507+0000 7f15040f9640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f14fc0cb430 con 0x7f14fc0cc270 2024-09-21T00:29:35.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.507+0000 7f15040f9640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f14fc0cb730 con 0x7f14fc0ce9e0 2024-09-21T00:29:35.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.507+0000 7f1502696640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14fc0b2940 0x7f14fc155c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:35.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.507+0000 7f1502696640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14fc0b2940 0x7f14fc155c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58106/0 (socket says 172.21.15.112:58106) 2024-09-21T00:29:35.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.507+0000 7f1502696640 1 -- 172.21.15.112:0/2273238911 learned_addr learned my addr 172.21.15.112:0/2273238911 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:29:35.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.507+0000 7f1501e95640 1 --2- 172.21.15.112:0/2273238911 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f14fc0cc270 0x7f14fc0afbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:35.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.507+0000 7f1502696640 1 -- 172.21.15.112:0/2273238911 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f14fc0ce9e0 msgr2=0x7f14fc1e6310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:35.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.507+0000 7f1501694640 1 --2- 172.21.15.112:0/2273238911 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f14fc0ce9e0 0x7f14fc1e6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:35.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.507+0000 7f1502696640 1 --2- 172.21.15.112:0/2273238911 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f14fc0ce9e0 0x7f14fc1e6310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:35.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.507+0000 7f1502696640 1 -- 172.21.15.112:0/2273238911 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f14fc0cc270 msgr2=0x7f14fc0afbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:35.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.507+0000 7f1502696640 1 --2- 172.21.15.112:0/2273238911 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f14fc0cc270 0x7f14fc0afbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:35.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.507+0000 7f1502696640 1 -- 172.21.15.112:0/2273238911 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14fc0b04d0 con 0x7f14fc0b2940 2024-09-21T00:29:35.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.507+0000 7f1501e95640 1 --2- 172.21.15.112:0/2273238911 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f14fc0cc270 0x7f14fc0afbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:29:35.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.507+0000 7f1501694640 1 --2- 172.21.15.112:0/2273238911 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f14fc0ce9e0 0x7f14fc1e6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:29:35.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.508+0000 7f1502696640 1 --2- 172.21.15.112:0/2273238911 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14fc0b2940 0x7f14fc155c70 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f14f402ab90 tx=0x7f14f4033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:35.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.508+0000 7f14f27fc640 1 -- 172.21.15.112:0/2273238911 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14f4031d60 con 0x7f14fc0b2940 2024-09-21T00:29:35.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.508+0000 7f14f27fc640 1 -- 172.21.15.112:0/2273238911 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f14f4033e10 con 0x7f14fc0b2940 2024-09-21T00:29:35.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.508+0000 7f15040f9640 1 -- 172.21.15.112:0/2273238911 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14fc0b1e60 con 0x7f14fc0b2940 2024-09-21T00:29:35.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.508+0000 7f14f27fc640 1 -- 172.21.15.112:0/2273238911 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14f4005240 con 0x7f14fc0b2940 2024-09-21T00:29:35.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.508+0000 7f15040f9640 1 -- 172.21.15.112:0/2273238911 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14fc1e6850 con 0x7f14fc0b2940 2024-09-21T00:29:35.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.509+0000 7f15040f9640 1 -- 172.21.15.112:0/2273238911 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14fc0add30 con 0x7f14fc0b2940 2024-09-21T00:29:35.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.510+0000 7f14f27fc640 1 -- 172.21.15.112:0/2273238911 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f14f40053e0 con 0x7f14fc0b2940 2024-09-21T00:29:35.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.510+0000 7f14f27fc640 1 --2- 172.21.15.112:0/2273238911 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f14e0078b90 0x7f14e007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:35.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.510+0000 7f14f27fc640 1 -- 172.21.15.112:0/2273238911 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 6188+0+0 (secure 0 0 0) 0x7f14f40bc0a0 con 0x7f14fc0b2940 2024-09-21T00:29:35.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.511+0000 7f1501e95640 1 --2- 172.21.15.112:0/2273238911 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f14e0078b90 0x7f14e007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:35.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.511+0000 7f1501e95640 1 --2- 172.21.15.112:0/2273238911 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f14e0078b90 0x7f14e007b0a0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f14f8007e90 tx=0x7f14f80073a0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:35.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.515+0000 7f14f27fc640 1 -- 172.21.15.112:0/2273238911 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f14f403f030 con 0x7f14fc0b2940 2024-09-21T00:29:35.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.728+0000 7f15040f9640 1 -- 172.21.15.112:0/2273238911 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f14fc0ada50 con 0x7f14fc0b2940 2024-09-21T00:29:35.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.728+0000 7f14f27fc640 1 -- 172.21.15.112:0/2273238911 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) ==== 94+0+5237 (secure 0 0 0) 0x7f14f4084390 con 0x7f14fc0b2940 2024-09-21T00:29:35.729 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:29:35.729 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":64,"btime":"2024-09-21T00:29:22:809921+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5804,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/3818262153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":3818262153},{"type":"v1","addr":"172.21.15.187:6835","nonce":3818262153}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59},{"gid":6214,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6837/1321768683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":1321768683},{"type":"v1","addr":"172.21.15.187:6837","nonce":1321768683}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59},{"gid":6801,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6834/4037012343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":4037012343},{"type":"v1","addr":"172.21.15.112:6834","nonce":4037012343}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T00:29:19.145794+0000","modified":"2024-09-21T00:29:22.172462+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6804},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6804":{"gid":6804,"name":"c","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6835/2797048793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":2797048793},{"type":"v1","addr":"172.21.15.112:6835","nonce":2797048793}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6804,"qdb_cluster":[6804]},"id":7}]} 2024-09-21T00:29:35.729 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 64 2024-09-21T00:29:35.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.730+0000 7f15040f9640 1 -- 172.21.15.112:0/2273238911 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f14e0078b90 msgr2=0x7f14e007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:35.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.730+0000 7f15040f9640 1 --2- 172.21.15.112:0/2273238911 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f14e0078b90 0x7f14e007b0a0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f14f8007e90 tx=0x7f14f80073a0 comp rx=0 tx=0).stop 2024-09-21T00:29:35.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.730+0000 7f15040f9640 1 -- 172.21.15.112:0/2273238911 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14fc0b2940 msgr2=0x7f14fc155c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:35.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.730+0000 7f15040f9640 1 --2- 172.21.15.112:0/2273238911 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14fc0b2940 0x7f14fc155c70 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f14f402ab90 tx=0x7f14f4033c20 comp rx=0 tx=0).stop 2024-09-21T00:29:35.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.730+0000 7f15040f9640 1 -- 172.21.15.112:0/2273238911 shutdown_connections 2024-09-21T00:29:35.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.730+0000 7f15040f9640 1 --2- 172.21.15.112:0/2273238911 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f14e0078b90 0x7f14e007b0a0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:35.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.730+0000 7f15040f9640 1 --2- 172.21.15.112:0/2273238911 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14fc0b2940 0x7f14fc155c70 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:35.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.730+0000 7f15040f9640 1 --2- 172.21.15.112:0/2273238911 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f14fc0ce9e0 0x7f14fc1e6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:35.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.730+0000 7f15040f9640 1 --2- 172.21.15.112:0/2273238911 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f14fc0cc270 0x7f14fc0afbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:35.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.730+0000 7f15040f9640 1 -- 172.21.15.112:0/2273238911 >> 172.21.15.112:0/2273238911 conn(0x7f14fc0b3580 msgr2=0x7f14fc069100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:35.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.730+0000 7f15040f9640 1 -- 172.21.15.112:0/2273238911 shutdown_connections 2024-09-21T00:29:35.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.730+0000 7f15040f9640 1 -- 172.21.15.112:0/2273238911 wait complete. 2024-09-21T00:29:35.742 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T00:29:35.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.856+0000 7f498b470640 1 -- 172.21.15.112:0/1229823190 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49840b55f0 msgr2=0x7f498415c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:35.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.856+0000 7f498b470640 1 --2- 172.21.15.112:0/1229823190 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49840b55f0 0x7f498415c850 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f4980005fd0 tx=0x7f498002a670 comp rx=0 tx=0).stop 2024-09-21T00:29:35.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.856+0000 7f498b470640 1 -- 172.21.15.112:0/1229823190 shutdown_connections 2024-09-21T00:29:35.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.856+0000 7f498b470640 1 --2- 172.21.15.112:0/1229823190 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49840b55f0 0x7f498415c850 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:35.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.856+0000 7f498b470640 1 -- 172.21.15.112:0/1229823190 >> 172.21.15.112:0/1229823190 conn(0x7f4984073fe0 msgr2=0x7f49840743f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:35.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.856+0000 7f498b470640 1 -- 172.21.15.112:0/1229823190 shutdown_connections 2024-09-21T00:29:35.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.856+0000 7f498b470640 1 -- 172.21.15.112:0/1229823190 wait complete. 2024-09-21T00:29:35.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.858+0000 7f498b470640 1 Processor -- start 2024-09-21T00:29:35.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.858+0000 7f498b470640 1 -- start start 2024-09-21T00:29:35.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.859+0000 7f498b470640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f498406e0b0 0x7f4984141d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:35.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.859+0000 7f498b470640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49840b55f0 0x7f498415e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:35.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.859+0000 7f498b470640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f49840cce50 0x7f49841413d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:35.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.859+0000 7f498b470640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f49840b63e0 con 0x7f49840b55f0 2024-09-21T00:29:35.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.859+0000 7f498b470640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f49840cd3d0 con 0x7f498406e0b0 2024-09-21T00:29:35.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.859+0000 7f498b470640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f49840b6560 con 0x7f49840cce50 2024-09-21T00:29:35.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.859+0000 7f4988a0b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49840b55f0 0x7f498415e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:35.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.859+0000 7f4988a0b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49840b55f0 0x7f498415e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58118/0 (socket says 172.21.15.112:58118) 2024-09-21T00:29:35.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.859+0000 7f4988a0b640 1 -- 172.21.15.112:0/2457578271 learned_addr learned my addr 172.21.15.112:0/2457578271 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:29:35.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.859+0000 7f498920c640 1 --2- 172.21.15.112:0/2457578271 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f498406e0b0 0x7f4984141d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:35.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.859+0000 7f4989a0d640 1 --2- 172.21.15.112:0/2457578271 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f49840cce50 0x7f49841413d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:35.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.860+0000 7f4988a0b640 1 -- 172.21.15.112:0/2457578271 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f49840cce50 msgr2=0x7f49841413d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:35.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.860+0000 7f4988a0b640 1 --2- 172.21.15.112:0/2457578271 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f49840cce50 0x7f49841413d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:35.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.860+0000 7f4988a0b640 1 -- 172.21.15.112:0/2457578271 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f498406e0b0 msgr2=0x7f4984141d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:35.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.860+0000 7f4988a0b640 1 --2- 172.21.15.112:0/2457578271 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f498406e0b0 0x7f4984141d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:35.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.860+0000 7f4988a0b640 1 -- 172.21.15.112:0/2457578271 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f498415d5f0 con 0x7f49840b55f0 2024-09-21T00:29:35.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.860+0000 7f4988a0b640 1 --2- 172.21.15.112:0/2457578271 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49840b55f0 0x7f498415e690 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f497400c430 tx=0x7f497400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:35.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.860+0000 7f4979ffb640 1 -- 172.21.15.112:0/2457578271 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4974017070 con 0x7f49840b55f0 2024-09-21T00:29:35.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.860+0000 7f4979ffb640 1 -- 172.21.15.112:0/2457578271 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f49740053c0 con 0x7f49840b55f0 2024-09-21T00:29:35.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.860+0000 7f4979ffb640 1 -- 172.21.15.112:0/2457578271 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4974003850 con 0x7f49840b55f0 2024-09-21T00:29:35.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.860+0000 7f498b470640 1 -- 172.21.15.112:0/2457578271 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f498415d8d0 con 0x7f49840b55f0 2024-09-21T00:29:35.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.860+0000 7f498b470640 1 -- 172.21.15.112:0/2457578271 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f498415de90 con 0x7f49840b55f0 2024-09-21T00:29:35.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.861+0000 7f498b470640 1 -- 172.21.15.112:0/2457578271 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f498415c850 con 0x7f49840b55f0 2024-09-21T00:29:35.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.862+0000 7f4979ffb640 1 -- 172.21.15.112:0/2457578271 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f497401a070 con 0x7f49840b55f0 2024-09-21T00:29:35.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.862+0000 7f4979ffb640 1 --2- 172.21.15.112:0/2457578271 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4968078bc0 0x7f496807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:35.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.863+0000 7f4979ffb640 1 -- 172.21.15.112:0/2457578271 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 6188+0+0 (secure 0 0 0) 0x7f497409a170 con 0x7f49840b55f0 2024-09-21T00:29:35.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.863+0000 7f498920c640 1 --2- 172.21.15.112:0/2457578271 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4968078bc0 0x7f496807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:35.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.863+0000 7f498920c640 1 --2- 172.21.15.112:0/2457578271 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4968078bc0 0x7f496807b0d0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f4980033000 tx=0x7f4980004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:35.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:35.867+0000 7f4979ffb640 1 -- 172.21.15.112:0/2457578271 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f4974061e10 con 0x7f49840b55f0 2024-09-21T00:29:36.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:36.046+0000 7f498b470640 1 -- 172.21.15.112:0/2457578271 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f49841608f0 con 0x7f49840b55f0 2024-09-21T00:29:36.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:36.047+0000 7f4979ffb640 1 -- 172.21.15.112:0/2457578271 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) ==== 74+0+17102 (secure 0 0 0) 0x7f4974066840 con 0x7f49840b55f0 2024-09-21T00:29:36.048 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:29:36.048 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":96,"fsid":"8b34692b-dc65-49b2-a08f-26f707b96ac5","created":"2024-09-21T00:25:22.788490+0000","modified":"2024-09-21T00:29:28.841606+0000","last_up_change":"2024-09-21T00:25:36.161835+0000","last_in_change":"2024-09-21T00:25:26.346500+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T00:25:37.105006+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-09-21T00:29:16.794043+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-09-21T00:29:18.114604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9c575086-67d2-4de2-b01b-d44dcc21d604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6809","nonce":755760621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6811","nonce":755760621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6815","nonce":755760621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6813","nonce":755760621}]},"public_addr":"172.21.15.112:6809/755760621","cluster_addr":"172.21.15.112:6811/755760621","heartbeat_back_addr":"172.21.15.112:6815/755760621","heartbeat_front_addr":"172.21.15.112:6813/755760621","state":["exists","up"]},{"osd":1,"uuid":"a983c227-a3ce-4bdd-82f1-438ba7e3ecd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6801","nonce":344932592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6803","nonce":344932592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6807","nonce":344932592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6805","nonce":344932592}]},"public_addr":"172.21.15.112:6801/344932592","cluster_addr":"172.21.15.112:6803/344932592","heartbeat_back_addr":"172.21.15.112:6807/344932592","heartbeat_front_addr":"172.21.15.112:6805/344932592","state":["exists","up"]},{"osd":2,"uuid":"1193c1d8-bfff-4a57-8200-5f69bf1ea3af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6825","nonce":407933616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6827","nonce":407933616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6831","nonce":407933616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6829","nonce":407933616}]},"public_addr":"172.21.15.112:6825/407933616","cluster_addr":"172.21.15.112:6827/407933616","heartbeat_back_addr":"172.21.15.112:6831/407933616","heartbeat_front_addr":"172.21.15.112:6829/407933616","state":["exists","up"]},{"osd":3,"uuid":"af04330a-36ad-40cc-a8dd-fa259b221de5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6817","nonce":4120164806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6819","nonce":4120164806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6823","nonce":4120164806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6821","nonce":4120164806}]},"public_addr":"172.21.15.112:6817/4120164806","cluster_addr":"172.21.15.112:6819/4120164806","heartbeat_back_addr":"172.21.15.112:6823/4120164806","heartbeat_front_addr":"172.21.15.112:6821/4120164806","state":["exists","up"]},{"osd":4,"uuid":"5b893111-292c-4c5a-b184-53ad38c7199e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6803","nonce":522077560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6805","nonce":522077560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6809","nonce":522077560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6807","nonce":522077560}]},"public_addr":"172.21.15.187:6803/522077560","cluster_addr":"172.21.15.187:6805/522077560","heartbeat_back_addr":"172.21.15.187:6809/522077560","heartbeat_front_addr":"172.21.15.187:6807/522077560","state":["exists","up"]},{"osd":5,"uuid":"c0b8e011-45fe-4c5e-925e-6be2a7f1868a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6827","nonce":3593144721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6829","nonce":3593144721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6833","nonce":3593144721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6831","nonce":3593144721}]},"public_addr":"172.21.15.187:6827/3593144721","cluster_addr":"172.21.15.187:6829/3593144721","heartbeat_back_addr":"172.21.15.187:6833/3593144721","heartbeat_front_addr":"172.21.15.187:6831/3593144721","state":["exists","up"]},{"osd":6,"uuid":"87d2d650-7489-4593-98a3-a1f433ce8dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6811","nonce":1494387657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6813","nonce":1494387657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6817","nonce":1494387657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6815","nonce":1494387657}]},"public_addr":"172.21.15.187:6811/1494387657","cluster_addr":"172.21.15.187:6813/1494387657","heartbeat_back_addr":"172.21.15.187:6817/1494387657","heartbeat_front_addr":"172.21.15.187:6815/1494387657","state":["exists","up"]},{"osd":7,"uuid":"c6b4f8e2-a1ee-4438-aa37-67ff4b6ef1f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6819","nonce":1624699842}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6821","nonce":1624699842}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6825","nonce":1624699842}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6823","nonce":1624699842}]},"public_addr":"172.21.15.187:6819/1624699842","cluster_addr":"172.21.15.187:6821/1624699842","heartbeat_back_addr":"172.21.15.187:6825/1624699842","heartbeat_front_addr":"172.21.15.187:6823/1624699842","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.873162+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.400506+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.664317+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.159791+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.202630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.113212+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.384786+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.448441+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.6","mappings":[{"from":6,"to":7}]},{"pgid":"16.13","mappings":[{"from":1,"to":4}]},{"pgid":"16.29","mappings":[{"from":5,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T00:29:36.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:36.050+0000 7f498b470640 1 -- 172.21.15.112:0/2457578271 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4968078bc0 msgr2=0x7f496807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:36.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:36.050+0000 7f498b470640 1 --2- 172.21.15.112:0/2457578271 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4968078bc0 0x7f496807b0d0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f4980033000 tx=0x7f4980004730 comp rx=0 tx=0).stop 2024-09-21T00:29:36.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:36.050+0000 7f498b470640 1 -- 172.21.15.112:0/2457578271 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49840b55f0 msgr2=0x7f498415e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:36.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:36.050+0000 7f498b470640 1 --2- 172.21.15.112:0/2457578271 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49840b55f0 0x7f498415e690 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f497400c430 tx=0x7f497400c8f0 comp rx=0 tx=0).stop 2024-09-21T00:29:36.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:36.050+0000 7f498b470640 1 -- 172.21.15.112:0/2457578271 shutdown_connections 2024-09-21T00:29:36.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:36.050+0000 7f498b470640 1 --2- 172.21.15.112:0/2457578271 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f49840cce50 0x7f49841413d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:36.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:36.050+0000 7f498b470640 1 --2- 172.21.15.112:0/2457578271 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4968078bc0 0x7f496807b0d0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:36.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:36.050+0000 7f498b470640 1 --2- 172.21.15.112:0/2457578271 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49840b55f0 0x7f498415e690 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:36.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:36.050+0000 7f498b470640 1 --2- 172.21.15.112:0/2457578271 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f498406e0b0 0x7f4984141d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:36.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:36.050+0000 7f498b470640 1 -- 172.21.15.112:0/2457578271 >> 172.21.15.112:0/2457578271 conn(0x7f4984073fe0 msgr2=0x7f498415bee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:36.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:36.050+0000 7f498b470640 1 -- 172.21.15.112:0/2457578271 shutdown_connections 2024-09-21T00:29:36.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:36.050+0000 7f498b470640 1 -- 172.21.15.112:0/2457578271 wait complete. 2024-09-21T00:29:36.062 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T00:29:36.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:36.176+0000 7f0f67750640 1 -- 172.21.15.112:0/3506909666 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f480040b0 msgr2=0x7f0f48004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:36.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:36.176+0000 7f0f67750640 1 --2- 172.21.15.112:0/3506909666 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f480040b0 0x7f0f48004800 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f0f5400c290 tx=0x7f0f48005830 comp rx=0 tx=0).stop 2024-09-21T00:29:36.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:36.176+0000 7f0f65ced640 1 -- 172.21.15.112:0/3506909666 reap_dead start 2024-09-21T00:29:36.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:36.176+0000 7f0f67750640 1 -- 172.21.15.112:0/3506909666 shutdown_connections 2024-09-21T00:29:36.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:36.176+0000 7f0f67750640 1 -- 172.21.15.112:0/3506909666 >> 172.21.15.112:0/3506909666 conn(0x7f0f60071690 msgr2=0x7f0f60071aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:36.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:36.176+0000 7f0f67750640 1 -- 172.21.15.112:0/3506909666 shutdown_connections 2024-09-21T00:29:36.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:36.176+0000 7f0f67750640 1 -- 172.21.15.112:0/3506909666 wait complete. 2024-09-21T00:29:36.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:36.178+0000 7f0f67750640 1 Processor -- start 2024-09-21T00:29:36.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:36.178+0000 7f0f67750640 1 -- start start 2024-09-21T00:29:36.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:36.179+0000 7f0f67750640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f480040b0 0x7f0f6014cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:36.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:36.179+0000 7f0f67750640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0f60152b80 0x7f0f60155d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:36.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:36.179+0000 7f0f67750640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0f601562c0 0x7f0f6006d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:36.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:36.179+0000 7f0f67750640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0f600b4c30 con 0x7f0f480040b0 2024-09-21T00:29:36.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:36.179+0000 7f0f67750640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0f600cf420 con 0x7f0f60152b80 2024-09-21T00:29:36.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:36.179+0000 7f0f67750640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f0f600b4db0 con 0x7f0f601562c0 2024-09-21T00:29:36.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:36.179+0000 7f0f654ec640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f480040b0 0x7f0f6014cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:36.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:36.179+0000 7f0f654ec640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f480040b0 0x7f0f6014cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36248/0 (socket says 172.21.15.112:36248) 2024-09-21T00:29:36.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:36.179+0000 7f0f654ec640 1 -- 172.21.15.112:0/2141301496 learned_addr learned my addr 172.21.15.112:0/2141301496 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:29:36.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:36.179+0000 7f0f65ced640 1 --2- 172.21.15.112:0/2141301496 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0f601562c0 0x7f0f6006d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:36.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:36.179+0000 7f0f64ceb640 1 --2- 172.21.15.112:0/2141301496 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0f60152b80 0x7f0f60155d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:36.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:36.179+0000 7f0f654ec640 1 -- 172.21.15.112:0/2141301496 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0f601562c0 msgr2=0x7f0f6006d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:36.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:36.179+0000 7f0f654ec640 1 --2- 172.21.15.112:0/2141301496 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0f601562c0 0x7f0f6006d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:36.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:36.179+0000 7f0f654ec640 1 -- 172.21.15.112:0/2141301496 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0f60152b80 msgr2=0x7f0f60155d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:36.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:36.179+0000 7f0f654ec640 1 --2- 172.21.15.112:0/2141301496 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0f60152b80 0x7f0f60155d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:36.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:36.179+0000 7f0f654ec640 1 -- 172.21.15.112:0/2141301496 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f601566d0 con 0x7f0f480040b0 2024-09-21T00:29:36.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:36.179+0000 7f0f64ceb640 1 --2- 172.21.15.112:0/2141301496 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0f60152b80 0x7f0f60155d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:29:36.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:36.180+0000 7f0f654ec640 1 --2- 172.21.15.112:0/2141301496 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f480040b0 0x7f0f6014cbb0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f0f50010c10 tx=0x7f0f5000e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:36.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:36.180+0000 7f0f5dffb640 1 -- 172.21.15.112:0/2141301496 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f50003b70 con 0x7f0f480040b0 2024-09-21T00:29:36.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:36.180+0000 7f0f5dffb640 1 -- 172.21.15.112:0/2141301496 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0f50003e00 con 0x7f0f480040b0 2024-09-21T00:29:36.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:36.180+0000 7f0f67750640 1 -- 172.21.15.112:0/2141301496 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f601f8e20 con 0x7f0f480040b0 2024-09-21T00:29:36.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:36.180+0000 7f0f5dffb640 1 -- 172.21.15.112:0/2141301496 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f500042b0 con 0x7f0f480040b0 2024-09-21T00:29:36.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:36.180+0000 7f0f67750640 1 -- 172.21.15.112:0/2141301496 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f601f93e0 con 0x7f0f480040b0 2024-09-21T00:29:36.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:36.181+0000 7f0f67750640 1 -- 172.21.15.112:0/2141301496 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f6015de00 con 0x7f0f480040b0 2024-09-21T00:29:36.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:36.182+0000 7f0f5dffb640 1 -- 172.21.15.112:0/2141301496 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0f50004450 con 0x7f0f480040b0 2024-09-21T00:29:36.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:36.182+0000 7f0f5dffb640 1 --2- 172.21.15.112:0/2141301496 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f0f4c078b90 0x7f0f4c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:36.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:36.182+0000 7f0f5dffb640 1 -- 172.21.15.112:0/2141301496 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 6188+0+0 (secure 0 0 0) 0x7f0f5009bfa0 con 0x7f0f480040b0 2024-09-21T00:29:36.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:36.183+0000 7f0f64ceb640 1 --2- 172.21.15.112:0/2141301496 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f0f4c078b90 0x7f0f4c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:36.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:36.183+0000 7f0f64ceb640 1 --2- 172.21.15.112:0/2141301496 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f0f4c078b90 0x7f0f4c07b0a0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f0f54033000 tx=0x7f0f600cfe00 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:36.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:36.187+0000 7f0f5dffb640 1 -- 172.21.15.112:0/2141301496 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f0f601566d0 con 0x7f0f480040b0 2024-09-21T00:29:36.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:36.398+0000 7f0f67750640 1 -- 172.21.15.112:0/2141301496 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f0f6015db20 con 0x7f0f480040b0 2024-09-21T00:29:36.399 INFO:tasks.ceph.mon.a.smithi112.stderr:2024-09-21T00:29:36.399+0000 7f7656c13640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T00:29:36.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:36.400+0000 7f0f5dffb640 1 -- 172.21.15.112:0/2141301496 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v65) ==== 161+0+0 (secure 0 0 0) 0x7f0f50064210 con 0x7f0f480040b0 2024-09-21T00:29:36.401 INFO:teuthology.orchestra.run.smithi112.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T00:29:36.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:36.403+0000 7f0f67750640 1 -- 172.21.15.112:0/2141301496 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f0f4c078b90 msgr2=0x7f0f4c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:36.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:36.403+0000 7f0f67750640 1 --2- 172.21.15.112:0/2141301496 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f0f4c078b90 0x7f0f4c07b0a0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f0f54033000 tx=0x7f0f600cfe00 comp rx=0 tx=0).stop 2024-09-21T00:29:36.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:36.403+0000 7f0f67750640 1 -- 172.21.15.112:0/2141301496 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f480040b0 msgr2=0x7f0f6014cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:36.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:36.403+0000 7f0f67750640 1 --2- 172.21.15.112:0/2141301496 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f480040b0 0x7f0f6014cbb0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f0f50010c10 tx=0x7f0f5000e720 comp rx=0 tx=0).stop 2024-09-21T00:29:36.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:36.403+0000 7f0f67750640 1 -- 172.21.15.112:0/2141301496 shutdown_connections 2024-09-21T00:29:36.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:36.403+0000 7f0f67750640 1 --2- 172.21.15.112:0/2141301496 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0f601562c0 0x7f0f6006d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:36.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:36.403+0000 7f0f67750640 1 --2- 172.21.15.112:0/2141301496 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0f60152b80 0x7f0f60155d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:36.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:36.403+0000 7f0f67750640 1 --2- 172.21.15.112:0/2141301496 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f0f4c078b90 0x7f0f4c07b0a0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:36.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:36.403+0000 7f0f67750640 1 --2- 172.21.15.112:0/2141301496 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f480040b0 0x7f0f6014cbb0 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:36.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:36.403+0000 7f0f67750640 1 -- 172.21.15.112:0/2141301496 >> 172.21.15.112:0/2141301496 conn(0x7f0f60071690 msgr2=0x7f0f6006eab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:36.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:36.403+0000 7f0f67750640 1 -- 172.21.15.112:0/2141301496 shutdown_connections 2024-09-21T00:29:36.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:36.403+0000 7f0f67750640 1 -- 172.21.15.112:0/2141301496 wait complete. 2024-09-21T00:29:36.405 INFO:tasks.ceph.mds.c.smithi112.stderr: -2365> 2024-09-21T00:29:21.774+0000 7f07c0c0c640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T00:29:36.418 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-21T00:29:36.458 INFO:tasks.ceph.mds.c.smithi112.stdout:starting mds.c at 2024-09-21T00:29:36.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:36.534+0000 7fcfd838b640 1 -- 172.21.15.112:0/809091366 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcfd00d01b0 msgr2=0x7fcfd00d05c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:36.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:36.534+0000 7fcfd838b640 1 --2- 172.21.15.112:0/809091366 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcfd00d01b0 0x7fcfd00d05c0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fcfd00d2050 tx=0x7fcfc802aac0 comp rx=0 tx=0).stop 2024-09-21T00:29:36.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:36.534+0000 7fcfd838b640 1 -- 172.21.15.112:0/809091366 shutdown_connections 2024-09-21T00:29:36.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:36.534+0000 7fcfd838b640 1 --2- 172.21.15.112:0/809091366 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcfd00d01b0 0x7fcfd00d05c0 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:36.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:36.534+0000 7fcfd838b640 1 --2- 172.21.15.112:0/809091366 >> v2:172.21.15.187:3300/0 conn(0x7fcfd00d48a0 0x7fcfd00d3dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:36.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:36.534+0000 7fcfd838b640 1 -- 172.21.15.112:0/809091366 >> 172.21.15.112:0/809091366 conn(0x7fcfd008bfe0 msgr2=0x7fcfd009d890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:36.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:36.534+0000 7fcfd838b640 1 -- 172.21.15.112:0/809091366 shutdown_connections 2024-09-21T00:29:36.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:36.534+0000 7fcfd838b640 1 -- 172.21.15.112:0/809091366 wait complete. 2024-09-21T00:29:36.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:36.536+0000 7fcfd838b640 1 Processor -- start 2024-09-21T00:29:36.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:36.536+0000 7fcfd838b640 1 -- start start 2024-09-21T00:29:36.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:36.537+0000 7fcfd838b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcfd00d01b0 0x7fcfd006f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:36.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:36.537+0000 7fcfd838b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcfd00d3880 0x7fcfd01545b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:36.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:36.537+0000 7fcfd838b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcfd00d48a0 0x7fcfd00cb7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:36.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:36.537+0000 7fcfd838b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcfd00d2460 con 0x7fcfd00d01b0 2024-09-21T00:29:36.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:36.537+0000 7fcfd838b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fcfd00d22e0 con 0x7fcfd00d48a0 2024-09-21T00:29:36.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:36.537+0000 7fcfd838b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fcfd00d25e0 con 0x7fcfd00d3880 2024-09-21T00:29:36.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:36.537+0000 7fcfd6127640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcfd00d01b0 0x7fcfd006f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:36.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:36.537+0000 7fcfd6127640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcfd00d01b0 0x7fcfd006f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36290/0 (socket says 172.21.15.112:36290) 2024-09-21T00:29:36.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:36.537+0000 7fcfd6127640 1 -- 172.21.15.112:0/1054847518 learned_addr learned my addr 172.21.15.112:0/1054847518 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:29:36.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:36.538+0000 7fcfd5926640 1 --2- 172.21.15.112:0/1054847518 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcfd00d3880 0x7fcfd01545b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:36.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:36.538+0000 7fcfd6928640 1 --2- 172.21.15.112:0/1054847518 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcfd00d48a0 0x7fcfd00cb7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:36.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:36.538+0000 7fcfd6127640 1 -- 172.21.15.112:0/1054847518 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcfd00d3880 msgr2=0x7fcfd01545b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:36.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:36.538+0000 7fcfd6127640 1 --2- 172.21.15.112:0/1054847518 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcfd00d3880 0x7fcfd01545b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:36.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:36.538+0000 7fcfd6127640 1 -- 172.21.15.112:0/1054847518 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcfd00d48a0 msgr2=0x7fcfd00cb7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:36.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:36.538+0000 7fcfd6127640 1 --2- 172.21.15.112:0/1054847518 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcfd00d48a0 0x7fcfd00cb7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:36.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:36.538+0000 7fcfd6127640 1 -- 172.21.15.112:0/1054847518 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfd006ddd0 con 0x7fcfd00d01b0 2024-09-21T00:29:36.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:36.538+0000 7fcfd6127640 1 --2- 172.21.15.112:0/1054847518 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcfd00d01b0 0x7fcfd006f0b0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc00d780 tx=0x7fcfcc00dc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:36.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:36.538+0000 7fcfc6ffd640 1 -- 172.21.15.112:0/1054847518 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfcc017070 con 0x7fcfd00d01b0 2024-09-21T00:29:36.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:36.538+0000 7fcfc6ffd640 1 -- 172.21.15.112:0/1054847518 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcfcc004310 con 0x7fcfd00d01b0 2024-09-21T00:29:36.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:36.538+0000 7fcfd838b640 1 -- 172.21.15.112:0/1054847518 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfd0154af0 con 0x7fcfd00d01b0 2024-09-21T00:29:36.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:36.538+0000 7fcfc6ffd640 1 -- 172.21.15.112:0/1054847518 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfcc005280 con 0x7fcfd00d01b0 2024-09-21T00:29:36.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:36.538+0000 7fcfd838b640 1 -- 172.21.15.112:0/1054847518 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfd006e5d0 con 0x7fcfd00d01b0 2024-09-21T00:29:36.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:36.539+0000 7fcfd838b640 1 -- 172.21.15.112:0/1054847518 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfd00d2b00 con 0x7fcfd00d01b0 2024-09-21T00:29:36.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:36.540+0000 7fcfc6ffd640 1 -- 172.21.15.112:0/1054847518 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fcfcc005420 con 0x7fcfd00d01b0 2024-09-21T00:29:36.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:36.540+0000 7fcfc6ffd640 1 --2- 172.21.15.112:0/1054847518 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fcfb4078b90 0x7fcfb407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:36.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:36.541+0000 7fcfc6ffd640 1 -- 172.21.15.112:0/1054847518 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6274+0+0 (secure 0 0 0) 0x7fcfcc09b0e0 con 0x7fcfd00d01b0 2024-09-21T00:29:36.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:36.541+0000 7fcfd5926640 1 --2- 172.21.15.112:0/1054847518 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fcfb4078b90 0x7fcfb407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:36.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:36.541+0000 7fcfd5926640 1 --2- 172.21.15.112:0/1054847518 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fcfb4078b90 0x7fcfb407b0a0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fcfd00cddb0 tx=0x7fcfc0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:36.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:36.545+0000 7fcfc6ffd640 1 -- 172.21.15.112:0/1054847518 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fcfcc020020 con 0x7fcfd00d01b0 2024-09-21T00:29:36.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:36.759+0000 7fcfd838b640 1 -- 172.21.15.112:0/1054847518 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fcfd00b6f90 con 0x7fcfd00d01b0 2024-09-21T00:29:37.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:37.402+0000 7fcfc6ffd640 1 -- 172.21.15.112:0/1054847518 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v66) ==== 104+0+0 (secure 0 0 0) 0x7fcfcc062db0 con 0x7fcfd00d01b0 2024-09-21T00:29:37.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:37.405+0000 7fcfd838b640 1 -- 172.21.15.112:0/1054847518 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fcfb4078b90 msgr2=0x7fcfb407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:37.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:37.405+0000 7fcfd838b640 1 --2- 172.21.15.112:0/1054847518 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fcfb4078b90 0x7fcfb407b0a0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fcfd00cddb0 tx=0x7fcfc0009040 comp rx=0 tx=0).stop 2024-09-21T00:29:37.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:37.405+0000 7fcfd838b640 1 -- 172.21.15.112:0/1054847518 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcfd00d01b0 msgr2=0x7fcfd006f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:37.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:37.405+0000 7fcfd838b640 1 --2- 172.21.15.112:0/1054847518 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcfd00d01b0 0x7fcfd006f0b0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc00d780 tx=0x7fcfcc00dc40 comp rx=0 tx=0).stop 2024-09-21T00:29:37.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:37.405+0000 7fcfd838b640 1 -- 172.21.15.112:0/1054847518 shutdown_connections 2024-09-21T00:29:37.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:37.405+0000 7fcfd838b640 1 --2- 172.21.15.112:0/1054847518 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcfd00d48a0 0x7fcfd00cb7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:37.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:37.405+0000 7fcfd838b640 1 --2- 172.21.15.112:0/1054847518 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcfd00d3880 0x7fcfd01545b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:37.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:37.405+0000 7fcfd838b640 1 --2- 172.21.15.112:0/1054847518 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fcfb4078b90 0x7fcfb407b0a0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:37.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:37.405+0000 7fcfd838b640 1 --2- 172.21.15.112:0/1054847518 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcfd00d01b0 0x7fcfd006f0b0 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:37.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:37.405+0000 7fcfd838b640 1 -- 172.21.15.112:0/1054847518 >> 172.21.15.112:0/1054847518 conn(0x7fcfd008bfe0 msgr2=0x7fcfd00b4180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:37.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:37.405+0000 7fcfd838b640 1 -- 172.21.15.112:0/1054847518 shutdown_connections 2024-09-21T00:29:37.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:37.405+0000 7fcfd838b640 1 -- 172.21.15.112:0/1054847518 wait complete. 2024-09-21T00:29:37.420 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-21T00:29:37.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:37.584+0000 7fec62c94640 1 -- 172.21.15.112:0/417349072 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec5c0d10c0 msgr2=0x7fec5c0d14d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:37.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:37.584+0000 7fec62c94640 1 --2- 172.21.15.112:0/417349072 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec5c0d10c0 0x7fec5c0d14d0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fec5c06de40 tx=0x7fec4c02a560 comp rx=0 tx=0).stop 2024-09-21T00:29:37.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:37.584+0000 7fec62c94640 1 -- 172.21.15.112:0/417349072 shutdown_connections 2024-09-21T00:29:37.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:37.584+0000 7fec62c94640 1 --2- 172.21.15.112:0/417349072 >> v2:172.21.15.187:3300/0 conn(0x7fec5c15cce0 0x7fec5c15d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:37.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:37.584+0000 7fec62c94640 1 --2- 172.21.15.112:0/417349072 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec5c0d10c0 0x7fec5c0d14d0 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:37.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:37.584+0000 7fec62c94640 1 -- 172.21.15.112:0/417349072 >> 172.21.15.112:0/417349072 conn(0x7fec5c0cef60 msgr2=0x7fec5c0cf370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:37.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:37.584+0000 7fec62c94640 1 -- 172.21.15.112:0/417349072 shutdown_connections 2024-09-21T00:29:37.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:37.584+0000 7fec62c94640 1 -- 172.21.15.112:0/417349072 wait complete. 2024-09-21T00:29:37.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:37.586+0000 7fec62c94640 1 Processor -- start 2024-09-21T00:29:37.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:37.586+0000 7fec62c94640 1 -- start start 2024-09-21T00:29:37.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:37.587+0000 7fec62c94640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fec5c0b5f40 0x7fec5c159210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:37.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:37.587+0000 7fec62c94640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fec5c0d10c0 0x7fec5c1f63d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:37.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:37.587+0000 7fec62c94640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec5c15cce0 0x7fec5c1f71e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:37.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:37.587+0000 7fec62c94640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fec5c153220 con 0x7fec5c15cce0 2024-09-21T00:29:37.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:37.587+0000 7fec62c94640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fec5c1530a0 con 0x7fec5c0d10c0 2024-09-21T00:29:37.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:37.587+0000 7fec62c94640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fec5c1533a0 con 0x7fec5c0b5f40 2024-09-21T00:29:37.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:37.587+0000 7fec61231640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec5c15cce0 0x7fec5c1f71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:37.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:37.587+0000 7fec61231640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec5c15cce0 0x7fec5c1f71e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36316/0 (socket says 172.21.15.112:36316) 2024-09-21T00:29:37.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:37.588+0000 7fec61231640 1 -- 172.21.15.112:0/940347904 learned_addr learned my addr 172.21.15.112:0/940347904 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:29:37.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:37.588+0000 7fec53fff640 1 --2- 172.21.15.112:0/940347904 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fec5c0d10c0 0x7fec5c1f63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:37.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:37.588+0000 7fec60a30640 1 --2- 172.21.15.112:0/940347904 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fec5c0b5f40 0x7fec5c159210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:37.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:37.588+0000 7fec61231640 1 -- 172.21.15.112:0/940347904 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fec5c0b5f40 msgr2=0x7fec5c159210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:37.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:37.588+0000 7fec61231640 1 --2- 172.21.15.112:0/940347904 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fec5c0b5f40 0x7fec5c159210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:37.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:37.588+0000 7fec61231640 1 -- 172.21.15.112:0/940347904 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fec5c0d10c0 msgr2=0x7fec5c1f63d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:37.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:37.588+0000 7fec61231640 1 --2- 172.21.15.112:0/940347904 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fec5c0d10c0 0x7fec5c1f63d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:37.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:37.588+0000 7fec61231640 1 -- 172.21.15.112:0/940347904 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec5c155050 con 0x7fec5c15cce0 2024-09-21T00:29:37.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:37.588+0000 7fec60a30640 1 --2- 172.21.15.112:0/940347904 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fec5c0b5f40 0x7fec5c159210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:29:37.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:37.588+0000 7fec61231640 1 --2- 172.21.15.112:0/940347904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec5c15cce0 0x7fec5c1f71e0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fec5400dac0 tx=0x7fec5400df80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:37.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:37.588+0000 7fec517fa640 1 -- 172.21.15.112:0/940347904 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec54017070 con 0x7fec5c15cce0 2024-09-21T00:29:37.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:37.588+0000 7fec517fa640 1 -- 172.21.15.112:0/940347904 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fec54003970 con 0x7fec5c15cce0 2024-09-21T00:29:37.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:37.588+0000 7fec62c94640 1 -- 172.21.15.112:0/940347904 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec5c1547b0 con 0x7fec5c15cce0 2024-09-21T00:29:37.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:37.588+0000 7fec517fa640 1 -- 172.21.15.112:0/940347904 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec54003dc0 con 0x7fec5c15cce0 2024-09-21T00:29:37.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:37.588+0000 7fec62c94640 1 -- 172.21.15.112:0/940347904 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec5c154d70 con 0x7fec5c15cce0 2024-09-21T00:29:37.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:37.589+0000 7fec62c94640 1 -- 172.21.15.112:0/940347904 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec5c158d70 con 0x7fec5c15cce0 2024-09-21T00:29:37.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:37.590+0000 7fec517fa640 1 -- 172.21.15.112:0/940347904 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fec54003f60 con 0x7fec5c15cce0 2024-09-21T00:29:37.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:37.590+0000 7fec517fa640 1 --2- 172.21.15.112:0/940347904 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fec48078bc0 0x7fec4807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:37.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:37.591+0000 7fec517fa640 1 -- 172.21.15.112:0/940347904 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6274+0+0 (secure 0 0 0) 0x7fec5409b270 con 0x7fec5c15cce0 2024-09-21T00:29:37.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:37.591+0000 7fec60a30640 1 --2- 172.21.15.112:0/940347904 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fec48078bc0 0x7fec4807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:37.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:37.591+0000 7fec60a30640 1 --2- 172.21.15.112:0/940347904 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fec48078bc0 0x7fec4807b0d0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fec58007fd0 tx=0x7fec58009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:37.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:37.595+0000 7fec517fa640 1 -- 172.21.15.112:0/940347904 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fec5c155050 con 0x7fec5c15cce0 2024-09-21T00:29:37.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:37.772+0000 7fec62c94640 1 -- 172.21.15.112:0/940347904 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fec5c0d14d0 con 0x7fec5c15cce0 2024-09-21T00:29:38.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:38.403+0000 7fec517fa640 1 -- 172.21.15.112:0/940347904 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v98) ==== 181+0+0 (secure 0 0 0) 0x7fec54062eb0 con 0x7fec5c15cce0 2024-09-21T00:29:38.404 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'cephfs_metadata' removed 2024-09-21T00:29:38.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:38.407+0000 7fec62c94640 1 -- 172.21.15.112:0/940347904 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fec48078bc0 msgr2=0x7fec4807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:38.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:38.407+0000 7fec62c94640 1 --2- 172.21.15.112:0/940347904 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fec48078bc0 0x7fec4807b0d0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fec58007fd0 tx=0x7fec58009040 comp rx=0 tx=0).stop 2024-09-21T00:29:38.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:38.407+0000 7fec62c94640 1 -- 172.21.15.112:0/940347904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec5c15cce0 msgr2=0x7fec5c1f71e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:38.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:38.407+0000 7fec62c94640 1 --2- 172.21.15.112:0/940347904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec5c15cce0 0x7fec5c1f71e0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fec5400dac0 tx=0x7fec5400df80 comp rx=0 tx=0).stop 2024-09-21T00:29:38.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:38.407+0000 7fec62c94640 1 -- 172.21.15.112:0/940347904 shutdown_connections 2024-09-21T00:29:38.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:38.407+0000 7fec62c94640 1 --2- 172.21.15.112:0/940347904 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fec48078bc0 0x7fec4807b0d0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:38.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:38.407+0000 7fec62c94640 1 --2- 172.21.15.112:0/940347904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec5c15cce0 0x7fec5c1f71e0 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:38.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:38.407+0000 7fec62c94640 1 --2- 172.21.15.112:0/940347904 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fec5c0d10c0 0x7fec5c1f63d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:38.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:38.407+0000 7fec62c94640 1 --2- 172.21.15.112:0/940347904 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fec5c0b5f40 0x7fec5c159210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:38.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:38.407+0000 7fec62c94640 1 -- 172.21.15.112:0/940347904 >> 172.21.15.112:0/940347904 conn(0x7fec5c0cef60 msgr2=0x7fec5c15a890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:38.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:38.407+0000 7fec62c94640 1 -- 172.21.15.112:0/940347904 shutdown_connections 2024-09-21T00:29:38.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:38.407+0000 7fec62c94640 1 -- 172.21.15.112:0/940347904 wait complete. 2024-09-21T00:29:38.420 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-21T00:29:38.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:38.539+0000 7fd8a45e6640 1 -- 172.21.15.112:0/1962583828 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd89c152700 msgr2=0x7fd89c15f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:38.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:38.539+0000 7fd8a45e6640 1 --2- 172.21.15.112:0/1962583828 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd89c152700 0x7fd89c15f190 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7fd89c0b6320 tx=0x7fd88c02a530 comp rx=0 tx=0).stop 2024-09-21T00:29:38.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:38.539+0000 7fd8a45e6640 1 -- 172.21.15.112:0/1962583828 shutdown_connections 2024-09-21T00:29:38.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:38.539+0000 7fd8a45e6640 1 --2- 172.21.15.112:0/1962583828 >> v2:172.21.15.187:3300/0 conn(0x7fd89c0b5240 0x7fd89c0cfba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:38.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:38.539+0000 7fd8a45e6640 1 --2- 172.21.15.112:0/1962583828 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd89c152700 0x7fd89c15f190 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:38.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:38.539+0000 7fd8a45e6640 1 -- 172.21.15.112:0/1962583828 >> 172.21.15.112:0/1962583828 conn(0x7fd89c0d12d0 msgr2=0x7fd89c0d16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:38.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:38.539+0000 7fd8a45e6640 1 -- 172.21.15.112:0/1962583828 shutdown_connections 2024-09-21T00:29:38.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:38.539+0000 7fd8a45e6640 1 -- 172.21.15.112:0/1962583828 wait complete. 2024-09-21T00:29:38.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:38.541+0000 7fd8a45e6640 1 Processor -- start 2024-09-21T00:29:38.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:38.541+0000 7fd8a45e6640 1 -- start start 2024-09-21T00:29:38.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:38.542+0000 7fd8a45e6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd89c0b5240 0x7fd89c151510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:38.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:38.542+0000 7fd8a45e6640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd89c152700 0x7fd89c151a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:38.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:38.542+0000 7fd8a45e6640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd89c15f6d0 0x7fd89c0ca5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:38.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:38.542+0000 7fd8a45e6640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd89c0d0520 con 0x7fd89c0b5240 2024-09-21T00:29:38.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:38.542+0000 7fd8a45e6640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd89c0d03a0 con 0x7fd89c15f6d0 2024-09-21T00:29:38.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:38.542+0000 7fd8a45e6640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd89c0d06a0 con 0x7fd89c152700 2024-09-21T00:29:38.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:38.542+0000 7fd8a2382640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd89c0b5240 0x7fd89c151510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:38.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:38.542+0000 7fd8a2382640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd89c0b5240 0x7fd89c151510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36342/0 (socket says 172.21.15.112:36342) 2024-09-21T00:29:38.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:38.542+0000 7fd8a2382640 1 -- 172.21.15.112:0/3810964351 learned_addr learned my addr 172.21.15.112:0/3810964351 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:29:38.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:38.542+0000 7fd8a2b83640 1 --2- 172.21.15.112:0/3810964351 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd89c15f6d0 0x7fd89c0ca5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:38.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:38.542+0000 7fd8a1b81640 1 --2- 172.21.15.112:0/3810964351 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd89c152700 0x7fd89c151a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:38.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:38.543+0000 7fd8a2382640 1 -- 172.21.15.112:0/3810964351 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd89c152700 msgr2=0x7fd89c151a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:38.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:38.543+0000 7fd8a2382640 1 --2- 172.21.15.112:0/3810964351 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd89c152700 0x7fd89c151a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:38.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:38.543+0000 7fd8a2382640 1 -- 172.21.15.112:0/3810964351 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd89c15f6d0 msgr2=0x7fd89c0ca5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:38.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:38.543+0000 7fd8a2382640 1 --2- 172.21.15.112:0/3810964351 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd89c15f6d0 0x7fd89c0ca5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:38.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:38.543+0000 7fd8a2382640 1 -- 172.21.15.112:0/3810964351 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd89c1543b0 con 0x7fd89c0b5240 2024-09-21T00:29:38.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:38.543+0000 7fd8a2382640 1 --2- 172.21.15.112:0/3810964351 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd89c0b5240 0x7fd89c151510 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fd88c02ab20 tx=0x7fd88c033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:38.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:38.543+0000 7fd89affd640 1 -- 172.21.15.112:0/3810964351 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd88c031d60 con 0x7fd89c0b5240 2024-09-21T00:29:38.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:38.543+0000 7fd89affd640 1 -- 172.21.15.112:0/3810964351 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd88c033e10 con 0x7fd89c0b5240 2024-09-21T00:29:38.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:38.543+0000 7fd8a45e6640 1 -- 172.21.15.112:0/3810964351 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd89c154690 con 0x7fd89c0b5240 2024-09-21T00:29:38.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:38.543+0000 7fd89affd640 1 -- 172.21.15.112:0/3810964351 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd88c005260 con 0x7fd89c0b5240 2024-09-21T00:29:38.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:38.543+0000 7fd8a45e6640 1 -- 172.21.15.112:0/3810964351 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd89c154c50 con 0x7fd89c0b5240 2024-09-21T00:29:38.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:38.544+0000 7fd8a45e6640 1 -- 172.21.15.112:0/3810964351 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd89c0d0bc0 con 0x7fd89c0b5240 2024-09-21T00:29:38.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:38.545+0000 7fd89affd640 1 -- 172.21.15.112:0/3810964351 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd88c005400 con 0x7fd89c0b5240 2024-09-21T00:29:38.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:38.545+0000 7fd89affd640 1 --2- 172.21.15.112:0/3810964351 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fd888078b90 0x7fd88807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:38.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:38.546+0000 7fd89affd640 1 -- 172.21.15.112:0/3810964351 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5836+0+0 (secure 0 0 0) 0x7fd88c0bbe30 con 0x7fd89c0b5240 2024-09-21T00:29:38.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:38.546+0000 7fd8a1b81640 1 --2- 172.21.15.112:0/3810964351 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fd888078b90 0x7fd88807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:38.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:38.546+0000 7fd8a1b81640 1 --2- 172.21.15.112:0/3810964351 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fd888078b90 0x7fd88807b0a0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fd89c155c30 tx=0x7fd890009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:38.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:38.550+0000 7fd89affd640 1 -- 172.21.15.112:0/3810964351 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd88c03f030 con 0x7fd89c0b5240 2024-09-21T00:29:38.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:38.726+0000 7fd8a45e6640 1 -- 172.21.15.112:0/3810964351 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fd89c071070 con 0x7fd89c0b5240 2024-09-21T00:29:39.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:39.411+0000 7fd89affd640 1 -- 172.21.15.112:0/3810964351 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v99) ==== 169+0+0 (secure 0 0 0) 0x7fd88c005d00 con 0x7fd89c0b5240 2024-09-21T00:29:39.412 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'cephfs_data' removed 2024-09-21T00:29:39.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:39.416+0000 7fd8a45e6640 1 -- 172.21.15.112:0/3810964351 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fd888078b90 msgr2=0x7fd88807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:39.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:39.416+0000 7fd8a45e6640 1 --2- 172.21.15.112:0/3810964351 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fd888078b90 0x7fd88807b0a0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fd89c155c30 tx=0x7fd890009040 comp rx=0 tx=0).stop 2024-09-21T00:29:39.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:39.416+0000 7fd8a45e6640 1 -- 172.21.15.112:0/3810964351 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd89c0b5240 msgr2=0x7fd89c151510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:39.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:39.416+0000 7fd8a45e6640 1 --2- 172.21.15.112:0/3810964351 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd89c0b5240 0x7fd89c151510 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fd88c02ab20 tx=0x7fd88c033c20 comp rx=0 tx=0).stop 2024-09-21T00:29:39.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:39.416+0000 7fd8a45e6640 1 -- 172.21.15.112:0/3810964351 shutdown_connections 2024-09-21T00:29:39.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:39.416+0000 7fd8a45e6640 1 --2- 172.21.15.112:0/3810964351 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd89c15f6d0 0x7fd89c0ca5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:39.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:39.416+0000 7fd8a45e6640 1 --2- 172.21.15.112:0/3810964351 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd89c152700 0x7fd89c151a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:39.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:39.416+0000 7fd8a45e6640 1 --2- 172.21.15.112:0/3810964351 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fd888078b90 0x7fd88807b0a0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:39.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:39.416+0000 7fd8a45e6640 1 --2- 172.21.15.112:0/3810964351 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd89c0b5240 0x7fd89c151510 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:39.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:39.416+0000 7fd8a45e6640 1 -- 172.21.15.112:0/3810964351 >> 172.21.15.112:0/3810964351 conn(0x7fd89c0d12d0 msgr2=0x7fd89c15ece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:39.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:39.417+0000 7fd8a45e6640 1 -- 172.21.15.112:0/3810964351 shutdown_connections 2024-09-21T00:29:39.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:39.417+0000 7fd8a45e6640 1 -- 172.21.15.112:0/3810964351 wait complete. 2024-09-21T00:29:39.431 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 31 2024-09-21T00:29:39.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:39.546+0000 7f0e9a4ec640 1 -- 172.21.15.112:0/4134474024 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e940b3bb0 msgr2=0x7f0e940b6740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:39.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:39.546+0000 7f0e9a4ec640 1 --2- 172.21.15.112:0/4134474024 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e940b3bb0 0x7f0e940b6740 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f0e8c005fd0 tx=0x7f0e8c02aa10 comp rx=0 tx=0).stop 2024-09-21T00:29:39.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:39.546+0000 7f0e9a4ec640 1 -- 172.21.15.112:0/4134474024 shutdown_connections 2024-09-21T00:29:39.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:39.546+0000 7f0e9a4ec640 1 --2- 172.21.15.112:0/4134474024 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e940b3bb0 0x7f0e940b6740 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:39.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:39.546+0000 7f0e9a4ec640 1 --2- 172.21.15.112:0/4134474024 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0e9415c3c0 0x7f0e940b3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:39.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:39.546+0000 7f0e9a4ec640 1 -- 172.21.15.112:0/4134474024 >> 172.21.15.112:0/4134474024 conn(0x7f0e9408dcb0 msgr2=0x7f0e9409d8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:39.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:39.546+0000 7f0e9a4ec640 1 -- 172.21.15.112:0/4134474024 shutdown_connections 2024-09-21T00:29:39.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:39.546+0000 7f0e9a4ec640 1 -- 172.21.15.112:0/4134474024 wait complete. 2024-09-21T00:29:39.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:39.548+0000 7f0e9a4ec640 1 Processor -- start 2024-09-21T00:29:39.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:39.548+0000 7f0e9a4ec640 1 -- start start 2024-09-21T00:29:39.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:39.549+0000 7f0e9a4ec640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e940b37a0 0x7f0e94151b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:39.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:39.550+0000 7f0e9a4ec640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0e940b3bb0 0x7f0e9406cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:39.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:39.550+0000 7f0e9a4ec640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0e9415c3c0 0x7f0e940b44b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:39.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:39.550+0000 7f0e9a4ec640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0e94163180 con 0x7f0e940b37a0 2024-09-21T00:29:39.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:39.550+0000 7f0e9a4ec640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0e940d0d30 con 0x7f0e940b3bb0 2024-09-21T00:29:39.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:39.550+0000 7f0e9a4ec640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f0e94163300 con 0x7f0e9415c3c0 2024-09-21T00:29:39.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:39.550+0000 7f0e93fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e940b37a0 0x7f0e94151b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:39.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:39.550+0000 7f0e93fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e940b37a0 0x7f0e94151b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36368/0 (socket says 172.21.15.112:36368) 2024-09-21T00:29:39.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:39.550+0000 7f0e93fff640 1 -- 172.21.15.112:0/4223398041 learned_addr learned my addr 172.21.15.112:0/4223398041 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:29:39.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:39.550+0000 7f0e937fe640 1 --2- 172.21.15.112:0/4223398041 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0e940b3bb0 0x7f0e9406cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:39.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:39.550+0000 7f0e98a89640 1 --2- 172.21.15.112:0/4223398041 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0e9415c3c0 0x7f0e940b44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:39.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:39.550+0000 7f0e93fff640 1 -- 172.21.15.112:0/4223398041 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0e9415c3c0 msgr2=0x7f0e940b44b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:39.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:39.550+0000 7f0e93fff640 1 --2- 172.21.15.112:0/4223398041 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0e9415c3c0 0x7f0e940b44b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:39.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:39.550+0000 7f0e93fff640 1 -- 172.21.15.112:0/4223398041 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0e940b3bb0 msgr2=0x7f0e9406cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:39.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:39.550+0000 7f0e93fff640 1 --2- 172.21.15.112:0/4223398041 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0e940b3bb0 0x7f0e9406cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:39.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:39.550+0000 7f0e93fff640 1 -- 172.21.15.112:0/4223398041 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e9406d880 con 0x7f0e940b37a0 2024-09-21T00:29:39.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:39.550+0000 7f0e937fe640 1 --2- 172.21.15.112:0/4223398041 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0e940b3bb0 0x7f0e9406cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:29:39.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:39.550+0000 7f0e98a89640 1 --2- 172.21.15.112:0/4223398041 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0e9415c3c0 0x7f0e940b44b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:29:39.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:39.550+0000 7f0e93fff640 1 --2- 172.21.15.112:0/4223398041 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e940b37a0 0x7f0e94151b20 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f0e8400d720 tx=0x7f0e8400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:39.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:39.550+0000 7f0e90ff9640 1 -- 172.21.15.112:0/4223398041 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e84017070 con 0x7f0e940b37a0 2024-09-21T00:29:39.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:39.550+0000 7f0e90ff9640 1 -- 172.21.15.112:0/4223398041 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0e84009b30 con 0x7f0e940b37a0 2024-09-21T00:29:39.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:39.550+0000 7f0e9a4ec640 1 -- 172.21.15.112:0/4223398041 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e9406db60 con 0x7f0e940b37a0 2024-09-21T00:29:39.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:39.550+0000 7f0e90ff9640 1 -- 172.21.15.112:0/4223398041 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e84010460 con 0x7f0e940b37a0 2024-09-21T00:29:39.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:39.550+0000 7f0e9a4ec640 1 -- 172.21.15.112:0/4223398041 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e940a0e40 con 0x7f0e940b37a0 2024-09-21T00:29:39.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:39.551+0000 7f0e9a4ec640 1 -- 172.21.15.112:0/4223398041 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e940b3260 con 0x7f0e940b37a0 2024-09-21T00:29:39.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:39.552+0000 7f0e90ff9640 1 -- 172.21.15.112:0/4223398041 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0e8401a070 con 0x7f0e940b37a0 2024-09-21T00:29:39.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:39.553+0000 7f0e90ff9640 1 --2- 172.21.15.112:0/4223398041 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f0e80078b90 0x7f0e8007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:39.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:39.553+0000 7f0e90ff9640 1 -- 172.21.15.112:0/4223398041 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5335+0+0 (secure 0 0 0) 0x7f0e8409ac90 con 0x7f0e940b37a0 2024-09-21T00:29:39.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:39.553+0000 7f0e937fe640 1 --2- 172.21.15.112:0/4223398041 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f0e80078b90 0x7f0e8007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:39.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:39.554+0000 7f0e937fe640 1 --2- 172.21.15.112:0/4223398041 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f0e80078b90 0x7f0e8007b0a0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f0e941f9df0 tx=0x7f0e88009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:39.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:39.557+0000 7f0e90ff9640 1 -- 172.21.15.112:0/4223398041 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f0e840632d0 con 0x7f0e940b37a0 2024-09-21T00:29:39.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:39.734+0000 7f0e9a4ec640 1 -- 172.21.15.112:0/4223398041 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "config reset", "num": 31} v 0) -- 0x7f0e9406dd30 con 0x7f0e940b37a0 2024-09-21T00:29:39.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:39.738+0000 7f0e90ff9640 1 -- 172.21.15.112:0/4223398041 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 31}]=0 v46) ==== 71+0+0 (secure 0 0 0) 0x7f0e84062bd0 con 0x7f0e940b37a0 2024-09-21T00:29:39.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:39.740+0000 7f0e9a4ec640 1 -- 172.21.15.112:0/4223398041 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f0e80078b90 msgr2=0x7f0e8007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:39.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:39.740+0000 7f0e9a4ec640 1 --2- 172.21.15.112:0/4223398041 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f0e80078b90 0x7f0e8007b0a0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f0e941f9df0 tx=0x7f0e88009040 comp rx=0 tx=0).stop 2024-09-21T00:29:39.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:39.740+0000 7f0e9a4ec640 1 -- 172.21.15.112:0/4223398041 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e940b37a0 msgr2=0x7f0e94151b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:39.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:39.740+0000 7f0e9a4ec640 1 --2- 172.21.15.112:0/4223398041 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e940b37a0 0x7f0e94151b20 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f0e8400d720 tx=0x7f0e8400dbe0 comp rx=0 tx=0).stop 2024-09-21T00:29:39.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:39.740+0000 7f0e9a4ec640 1 -- 172.21.15.112:0/4223398041 shutdown_connections 2024-09-21T00:29:39.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:39.740+0000 7f0e9a4ec640 1 --2- 172.21.15.112:0/4223398041 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0e9415c3c0 0x7f0e940b44b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:39.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:39.740+0000 7f0e9a4ec640 1 --2- 172.21.15.112:0/4223398041 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0e940b3bb0 0x7f0e9406cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:39.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:39.740+0000 7f0e9a4ec640 1 --2- 172.21.15.112:0/4223398041 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f0e80078b90 0x7f0e8007b0a0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:39.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:39.740+0000 7f0e9a4ec640 1 --2- 172.21.15.112:0/4223398041 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e940b37a0 0x7f0e94151b20 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:39.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:39.740+0000 7f0e9a4ec640 1 -- 172.21.15.112:0/4223398041 >> 172.21.15.112:0/4223398041 conn(0x7f0e9408dcb0 msgr2=0x7f0e940b1240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:39.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:39.740+0000 7f0e9a4ec640 1 -- 172.21.15.112:0/4223398041 shutdown_connections 2024-09-21T00:29:39.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:39.740+0000 7f0e9a4ec640 1 -- 172.21.15.112:0/4223398041 wait complete. 2024-09-21T00:29:39.752 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_sessionmap.TestSessionMap.test_tell_session_drop' 2024-09-21T00:29:39.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:39.867+0000 7f1ac908b640 1 -- 172.21.15.112:0/547669061 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1ac40ccbf0 msgr2=0x7f1ac40cd000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:39.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:39.867+0000 7f1ac908b640 1 --2- 172.21.15.112:0/547669061 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1ac40ccbf0 0x7f1ac40cd000 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f1ab4009830 tx=0x7f1ab402a3f0 comp rx=0 tx=0).stop 2024-09-21T00:29:39.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:39.867+0000 7f1ac908b640 1 -- 172.21.15.112:0/547669061 shutdown_connections 2024-09-21T00:29:39.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:39.867+0000 7f1ac908b640 1 --2- 172.21.15.112:0/547669061 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1ac40c6a00 0x7f1ac40b2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:39.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:39.867+0000 7f1ac908b640 1 --2- 172.21.15.112:0/547669061 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1ac40ccbf0 0x7f1ac40cd000 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:39.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:39.867+0000 7f1ac908b640 1 -- 172.21.15.112:0/547669061 >> 172.21.15.112:0/547669061 conn(0x7f1ac40b42e0 msgr2=0x7f1ac40b4720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:39.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:39.867+0000 7f1ac908b640 1 -- 172.21.15.112:0/547669061 shutdown_connections 2024-09-21T00:29:39.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:39.867+0000 7f1ac908b640 1 -- 172.21.15.112:0/547669061 wait complete. 2024-09-21T00:29:39.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:39.869+0000 7f1ac908b640 1 Processor -- start 2024-09-21T00:29:39.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:39.869+0000 7f1ac908b640 1 -- start start 2024-09-21T00:29:39.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:39.870+0000 7f1ac908b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1ac40c5ee0 0x7f1ac41f7440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:39.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:39.870+0000 7f1ac908b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ac40c6a00 0x7f1ac41532c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:39.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:39.870+0000 7f1ac908b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1ac40ccbf0 0x7f1ac4154200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:39.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:39.870+0000 7f1ac908b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac40b3610 con 0x7f1ac40c6a00 2024-09-21T00:29:39.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:39.870+0000 7f1ac908b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac40b3490 con 0x7f1ac40c5ee0 2024-09-21T00:29:39.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:39.870+0000 7f1ac908b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f1ac40b3790 con 0x7f1ac40ccbf0 2024-09-21T00:29:39.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:39.870+0000 7f1abbfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ac40c6a00 0x7f1ac41532c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:39.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:39.870+0000 7f1abbfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ac40c6a00 0x7f1ac41532c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36382/0 (socket says 172.21.15.112:36382) 2024-09-21T00:29:39.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:39.870+0000 7f1abbfff640 1 -- 172.21.15.112:0/3682251124 learned_addr learned my addr 172.21.15.112:0/3682251124 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:29:39.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:39.870+0000 7f1ac2d9d640 1 --2- 172.21.15.112:0/3682251124 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1ac40c5ee0 0x7f1ac41f7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:39.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:39.870+0000 7f1ac359e640 1 --2- 172.21.15.112:0/3682251124 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1ac40ccbf0 0x7f1ac4154200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:39.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:39.870+0000 7f1abbfff640 1 -- 172.21.15.112:0/3682251124 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1ac40ccbf0 msgr2=0x7f1ac4154200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:39.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:39.870+0000 7f1abbfff640 1 --2- 172.21.15.112:0/3682251124 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1ac40ccbf0 0x7f1ac4154200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:39.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:39.870+0000 7f1abbfff640 1 -- 172.21.15.112:0/3682251124 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1ac40c5ee0 msgr2=0x7f1ac41f7440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:39.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:39.870+0000 7f1abbfff640 1 --2- 172.21.15.112:0/3682251124 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1ac40c5ee0 0x7f1ac41f7440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:39.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:39.870+0000 7f1abbfff640 1 -- 172.21.15.112:0/3682251124 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ac41f7980 con 0x7f1ac40c6a00 2024-09-21T00:29:39.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:39.871+0000 7f1abbfff640 1 --2- 172.21.15.112:0/3682251124 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ac40c6a00 0x7f1ac41532c0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f1aac001eb0 tx=0x7f1aac00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:39.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:39.871+0000 7f1abb7fe640 1 -- 172.21.15.112:0/3682251124 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aac017070 con 0x7f1ac40c6a00 2024-09-21T00:29:39.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:39.871+0000 7f1abb7fe640 1 -- 172.21.15.112:0/3682251124 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1aac0053c0 con 0x7f1ac40c6a00 2024-09-21T00:29:39.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:39.871+0000 7f1abb7fe640 1 -- 172.21.15.112:0/3682251124 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aac003830 con 0x7f1ac40c6a00 2024-09-21T00:29:39.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:39.871+0000 7f1ac908b640 1 -- 172.21.15.112:0/3682251124 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ac4153800 con 0x7f1ac40c6a00 2024-09-21T00:29:39.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:39.871+0000 7f1ac908b640 1 -- 172.21.15.112:0/3682251124 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ac41519b0 con 0x7f1ac40c6a00 2024-09-21T00:29:39.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:39.872+0000 7f1ac908b640 1 -- 172.21.15.112:0/3682251124 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ac4151720 con 0x7f1ac40c6a00 2024-09-21T00:29:39.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:39.873+0000 7f1abb7fe640 1 -- 172.21.15.112:0/3682251124 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1aac01a070 con 0x7f1ac40c6a00 2024-09-21T00:29:39.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:39.873+0000 7f1abb7fe640 1 --2- 172.21.15.112:0/3682251124 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1aa4078bc0 0x7f1aa407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:39.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:39.873+0000 7f1abb7fe640 1 -- 172.21.15.112:0/3682251124 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5335+0+0 (secure 0 0 0) 0x7f1aac099ea0 con 0x7f1ac40c6a00 2024-09-21T00:29:39.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:39.873+0000 7f1ac2d9d640 1 --2- 172.21.15.112:0/3682251124 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1aa4078bc0 0x7f1aa407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:39.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:39.874+0000 7f1ac2d9d640 1 --2- 172.21.15.112:0/3682251124 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1aa4078bc0 0x7f1aa407b0d0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f1ab4034000 tx=0x7f1ab4004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:39.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:39.877+0000 7f1abb7fe640 1 -- 172.21.15.112:0/3682251124 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1ac41f7980 con 0x7f1ac40c6a00 2024-09-21T00:29:40.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:40.123+0000 7f1ac908b640 1 -- 172.21.15.112:0/3682251124 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_sessionmap.TestSessionMap.test_tell_session_drop"]} v 0) -- 0x7f1ac40cd000 con 0x7f1ac40c6a00 2024-09-21T00:29:40.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:40.413+0000 7f1abb7fe640 1 -- 172.21.15.112:0/3682251124 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_sessionmap.TestSessionMap.test_tell_session_drop"]}]=0 v236) ==== 145+0+0 (secure 0 0 0) 0x7f1ac40cd540 con 0x7f1ac40c6a00 2024-09-21T00:29:40.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:40.416+0000 7f1ac908b640 1 -- 172.21.15.112:0/3682251124 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1aa4078bc0 msgr2=0x7f1aa407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:40.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:40.416+0000 7f1ac908b640 1 --2- 172.21.15.112:0/3682251124 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1aa4078bc0 0x7f1aa407b0d0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f1ab4034000 tx=0x7f1ab4004730 comp rx=0 tx=0).stop 2024-09-21T00:29:40.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:40.416+0000 7f1ac908b640 1 -- 172.21.15.112:0/3682251124 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ac40c6a00 msgr2=0x7f1ac41532c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:40.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:40.416+0000 7f1ac908b640 1 --2- 172.21.15.112:0/3682251124 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ac40c6a00 0x7f1ac41532c0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f1aac001eb0 tx=0x7f1aac00c890 comp rx=0 tx=0).stop 2024-09-21T00:29:40.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:40.416+0000 7f1ac908b640 1 -- 172.21.15.112:0/3682251124 shutdown_connections 2024-09-21T00:29:40.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:40.416+0000 7f1ac908b640 1 --2- 172.21.15.112:0/3682251124 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1ac40ccbf0 0x7f1ac4154200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:40.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:40.416+0000 7f1ac908b640 1 --2- 172.21.15.112:0/3682251124 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1aa4078bc0 0x7f1aa407b0d0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:40.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:40.416+0000 7f1ac908b640 1 --2- 172.21.15.112:0/3682251124 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ac40c6a00 0x7f1ac41532c0 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:40.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:40.416+0000 7f1ac908b640 1 --2- 172.21.15.112:0/3682251124 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1ac40c5ee0 0x7f1ac41f7440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:40.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:40.416+0000 7f1ac908b640 1 -- 172.21.15.112:0/3682251124 >> 172.21.15.112:0/3682251124 conn(0x7f1ac40b42e0 msgr2=0x7f1ac40711b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:40.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:40.416+0000 7f1ac908b640 1 -- 172.21.15.112:0/3682251124 shutdown_connections 2024-09-21T00:29:40.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:40.416+0000 7f1ac908b640 1 -- 172.21.15.112:0/3682251124 wait complete. 2024-09-21T00:29:40.428 INFO:tasks.cephfs_test_runner:That when a `tell` command is sent using the python CLI, ... ok 2024-09-21T00:29:40.428 INFO:tasks.cephfs_test_runner:Starting test: test_version_splitting (tasks.cephfs.test_sessionmap.TestSessionMap) That when many sessions are updated, they are correctly 2024-09-21T00:29:40.429 INFO:tasks.cephfs_test_runner:test_version_splitting (tasks.cephfs.test_sessionmap.TestSessionMap) 2024-09-21T00:29:40.429 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T00:29:40.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:40.541+0000 7f212fcb6640 1 -- 172.21.15.112:0/2237328090 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2114004f60 msgr2=0x7f211400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:40.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:40.541+0000 7f212fcb6640 1 --2- 172.21.15.112:0/2237328090 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2114004f60 0x7f211400b9c0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f212000c610 tx=0x7f2114005830 comp rx=0 tx=0).stop 2024-09-21T00:29:40.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:40.541+0000 7f212e253640 1 -- 172.21.15.112:0/2237328090 reap_dead start 2024-09-21T00:29:40.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:40.541+0000 7f212fcb6640 1 -- 172.21.15.112:0/2237328090 shutdown_connections 2024-09-21T00:29:40.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:40.541+0000 7f212fcb6640 1 -- 172.21.15.112:0/2237328090 >> 172.21.15.112:0/2237328090 conn(0x7f21280b3580 msgr2=0x7f21280b3990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:40.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:40.541+0000 7f212fcb6640 1 -- 172.21.15.112:0/2237328090 shutdown_connections 2024-09-21T00:29:40.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:40.541+0000 7f212fcb6640 1 -- 172.21.15.112:0/2237328090 wait complete. 2024-09-21T00:29:40.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:40.543+0000 7f212fcb6640 1 Processor -- start 2024-09-21T00:29:40.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:40.543+0000 7f212fcb6640 1 -- start start 2024-09-21T00:29:40.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:40.544+0000 7f212fcb6640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2114004f60 0x7f21281f6610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:40.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:40.544+0000 7f212fcb6640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2128099bc0 0x7f21280b1a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:40.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:40.544+0000 7f212fcb6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2128099670 0x7f21281e5f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:40.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:40.544+0000 7f212fcb6640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f212806db90 con 0x7f2128099670 2024-09-21T00:29:40.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:40.544+0000 7f212fcb6640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f212806da10 con 0x7f2128099bc0 2024-09-21T00:29:40.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:40.544+0000 7f212fcb6640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f212815d320 con 0x7f2114004f60 2024-09-21T00:29:40.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:40.544+0000 7f212e253640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2128099670 0x7f21281e5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:40.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:40.545+0000 7f212e253640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2128099670 0x7f21281e5f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36388/0 (socket says 172.21.15.112:36388) 2024-09-21T00:29:40.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:40.545+0000 7f212e253640 1 -- 172.21.15.112:0/2934048350 learned_addr learned my addr 172.21.15.112:0/2934048350 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:29:40.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:40.545+0000 7f212da52640 1 --2- 172.21.15.112:0/2934048350 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2114004f60 0x7f21281f6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:40.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:40.545+0000 7f212d251640 1 --2- 172.21.15.112:0/2934048350 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2128099bc0 0x7f21280b1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:40.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:40.545+0000 7f212e253640 1 -- 172.21.15.112:0/2934048350 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2114004f60 msgr2=0x7f21281f6610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:40.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:40.545+0000 7f212e253640 1 --2- 172.21.15.112:0/2934048350 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2114004f60 0x7f21281f6610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:40.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:40.545+0000 7f212e253640 1 -- 172.21.15.112:0/2934048350 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2128099bc0 msgr2=0x7f21280b1a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:40.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:40.545+0000 7f212e253640 1 --2- 172.21.15.112:0/2934048350 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2128099bc0 0x7f21280b1a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:40.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:40.545+0000 7f212e253640 1 -- 172.21.15.112:0/2934048350 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21280b1f70 con 0x7f2128099670 2024-09-21T00:29:40.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:40.545+0000 7f212e253640 1 --2- 172.21.15.112:0/2934048350 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2128099670 0x7f21281e5f90 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f212002b4b0 tx=0x7f2120004bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:40.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:40.545+0000 7f211e7fc640 1 -- 172.21.15.112:0/2934048350 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f212002b800 con 0x7f2128099670 2024-09-21T00:29:40.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:40.545+0000 7f211e7fc640 1 -- 172.21.15.112:0/2934048350 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f212002b9a0 con 0x7f2128099670 2024-09-21T00:29:40.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:40.545+0000 7f212fcb6640 1 -- 172.21.15.112:0/2934048350 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21281e89d0 con 0x7f2128099670 2024-09-21T00:29:40.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:40.545+0000 7f211e7fc640 1 -- 172.21.15.112:0/2934048350 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f212002bdf0 con 0x7f2128099670 2024-09-21T00:29:40.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:40.545+0000 7f212fcb6640 1 -- 172.21.15.112:0/2934048350 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21281f5e10 con 0x7f2128099670 2024-09-21T00:29:40.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:40.546+0000 7f212fcb6640 1 -- 172.21.15.112:0/2934048350 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f212815d880 con 0x7f2128099670 2024-09-21T00:29:40.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:40.547+0000 7f211e7fc640 1 -- 172.21.15.112:0/2934048350 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f212003d030 con 0x7f2128099670 2024-09-21T00:29:40.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:40.547+0000 7f211e7fc640 1 --2- 172.21.15.112:0/2934048350 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f210c078b90 0x7f210c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:40.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:40.548+0000 7f211e7fc640 1 -- 172.21.15.112:0/2934048350 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5335+0+0 (secure 0 0 0) 0x7f21200ba390 con 0x7f2128099670 2024-09-21T00:29:40.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:40.548+0000 7f212da52640 1 --2- 172.21.15.112:0/2934048350 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f210c078b90 0x7f210c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:40.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:40.548+0000 7f212da52640 1 --2- 172.21.15.112:0/2934048350 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f210c078b90 0x7f210c07b0a0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f2124004610 tx=0x7f212400b260 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:40.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:40.552+0000 7f211e7fc640 1 -- 172.21.15.112:0/2934048350 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f21200829d0 con 0x7f2128099670 2024-09-21T00:29:40.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:40.729+0000 7f212fcb6640 1 -- 172.21.15.112:0/2934048350 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f21280ccf50 con 0x7f2128099670 2024-09-21T00:29:40.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:40.730+0000 7f211e7fc640 1 -- 172.21.15.112:0/2934048350 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) ==== 74+0+13332 (secure 0 0 0) 0x7f21200822d0 con 0x7f2128099670 2024-09-21T00:29:40.731 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:29:40.731 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":99,"fsid":"8b34692b-dc65-49b2-a08f-26f707b96ac5","created":"2024-09-21T00:25:22.788490+0000","modified":"2024-09-21T00:29:39.404147+0000","last_up_change":"2024-09-21T00:25:36.161835+0000","last_in_change":"2024-09-21T00:25:26.346500+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T00:25:37.105006+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9c575086-67d2-4de2-b01b-d44dcc21d604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6809","nonce":755760621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6811","nonce":755760621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6815","nonce":755760621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6813","nonce":755760621}]},"public_addr":"172.21.15.112:6809/755760621","cluster_addr":"172.21.15.112:6811/755760621","heartbeat_back_addr":"172.21.15.112:6815/755760621","heartbeat_front_addr":"172.21.15.112:6813/755760621","state":["exists","up"]},{"osd":1,"uuid":"a983c227-a3ce-4bdd-82f1-438ba7e3ecd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6801","nonce":344932592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6803","nonce":344932592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6807","nonce":344932592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6805","nonce":344932592}]},"public_addr":"172.21.15.112:6801/344932592","cluster_addr":"172.21.15.112:6803/344932592","heartbeat_back_addr":"172.21.15.112:6807/344932592","heartbeat_front_addr":"172.21.15.112:6805/344932592","state":["exists","up"]},{"osd":2,"uuid":"1193c1d8-bfff-4a57-8200-5f69bf1ea3af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6825","nonce":407933616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6827","nonce":407933616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6831","nonce":407933616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6829","nonce":407933616}]},"public_addr":"172.21.15.112:6825/407933616","cluster_addr":"172.21.15.112:6827/407933616","heartbeat_back_addr":"172.21.15.112:6831/407933616","heartbeat_front_addr":"172.21.15.112:6829/407933616","state":["exists","up"]},{"osd":3,"uuid":"af04330a-36ad-40cc-a8dd-fa259b221de5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6817","nonce":4120164806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6819","nonce":4120164806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6823","nonce":4120164806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6821","nonce":4120164806}]},"public_addr":"172.21.15.112:6817/4120164806","cluster_addr":"172.21.15.112:6819/4120164806","heartbeat_back_addr":"172.21.15.112:6823/4120164806","heartbeat_front_addr":"172.21.15.112:6821/4120164806","state":["exists","up"]},{"osd":4,"uuid":"5b893111-292c-4c5a-b184-53ad38c7199e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6803","nonce":522077560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6805","nonce":522077560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6809","nonce":522077560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6807","nonce":522077560}]},"public_addr":"172.21.15.187:6803/522077560","cluster_addr":"172.21.15.187:6805/522077560","heartbeat_back_addr":"172.21.15.187:6809/522077560","heartbeat_front_addr":"172.21.15.187:6807/522077560","state":["exists","up"]},{"osd":5,"uuid":"c0b8e011-45fe-4c5e-925e-6be2a7f1868a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6827","nonce":3593144721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6829","nonce":3593144721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6833","nonce":3593144721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6831","nonce":3593144721}]},"public_addr":"172.21.15.187:6827/3593144721","cluster_addr":"172.21.15.187:6829/3593144721","heartbeat_back_addr":"172.21.15.187:6833/3593144721","heartbeat_front_addr":"172.21.15.187:6831/3593144721","state":["exists","up"]},{"osd":6,"uuid":"87d2d650-7489-4593-98a3-a1f433ce8dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6811","nonce":1494387657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6813","nonce":1494387657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6817","nonce":1494387657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6815","nonce":1494387657}]},"public_addr":"172.21.15.187:6811/1494387657","cluster_addr":"172.21.15.187:6813/1494387657","heartbeat_back_addr":"172.21.15.187:6817/1494387657","heartbeat_front_addr":"172.21.15.187:6815/1494387657","state":["exists","up"]},{"osd":7,"uuid":"c6b4f8e2-a1ee-4438-aa37-67ff4b6ef1f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6819","nonce":1624699842}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6821","nonce":1624699842}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6825","nonce":1624699842}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6823","nonce":1624699842}]},"public_addr":"172.21.15.187:6819/1624699842","cluster_addr":"172.21.15.187:6821/1624699842","heartbeat_back_addr":"172.21.15.187:6825/1624699842","heartbeat_front_addr":"172.21.15.187:6823/1624699842","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.873162+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.400506+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.664317+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.159791+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.202630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.113212+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.384786+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.448441+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:6835/2797048793":"2024-09-22T00:29:36.399445+0000","172.21.15.112:6833/2797048793":"2024-09-22T00:29:36.399445+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T00:29:40.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:40.732+0000 7f212fcb6640 1 -- 172.21.15.112:0/2934048350 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f210c078b90 msgr2=0x7f210c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:40.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:40.732+0000 7f212fcb6640 1 --2- 172.21.15.112:0/2934048350 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f210c078b90 0x7f210c07b0a0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f2124004610 tx=0x7f212400b260 comp rx=0 tx=0).stop 2024-09-21T00:29:40.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:40.732+0000 7f212fcb6640 1 -- 172.21.15.112:0/2934048350 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2128099670 msgr2=0x7f21281e5f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:40.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:40.732+0000 7f212fcb6640 1 --2- 172.21.15.112:0/2934048350 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2128099670 0x7f21281e5f90 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f212002b4b0 tx=0x7f2120004bb0 comp rx=0 tx=0).stop 2024-09-21T00:29:40.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:40.732+0000 7f212fcb6640 1 -- 172.21.15.112:0/2934048350 shutdown_connections 2024-09-21T00:29:40.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:40.732+0000 7f212fcb6640 1 --2- 172.21.15.112:0/2934048350 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f210c078b90 0x7f210c07b0a0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:40.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:40.732+0000 7f212fcb6640 1 --2- 172.21.15.112:0/2934048350 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2128099670 0x7f21281e5f90 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:40.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:40.732+0000 7f212fcb6640 1 --2- 172.21.15.112:0/2934048350 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2128099bc0 0x7f21280b1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:40.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:40.732+0000 7f212fcb6640 1 --2- 172.21.15.112:0/2934048350 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2114004f60 0x7f21281f6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:40.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:40.732+0000 7f212fcb6640 1 -- 172.21.15.112:0/2934048350 >> 172.21.15.112:0/2934048350 conn(0x7f21280b3580 msgr2=0x7f21280b21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:40.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:40.732+0000 7f212fcb6640 1 -- 172.21.15.112:0/2934048350 shutdown_connections 2024-09-21T00:29:40.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:40.732+0000 7f212fcb6640 1 -- 172.21.15.112:0/2934048350 wait complete. 2024-09-21T00:29:40.745 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T00:25:37.105006+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-21T00:29:40.745 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-21T00:29:40.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:40.861+0000 7fc474932640 1 -- 172.21.15.112:0/2692880618 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4700d3880 msgr2=0x7fc470163260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:40.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:40.861+0000 7fc474932640 1 --2- 172.21.15.112:0/2692880618 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4700d3880 0x7fc470163260 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fc4700cb560 tx=0x7fc46402a530 comp rx=0 tx=0).stop 2024-09-21T00:29:40.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:40.861+0000 7fc474932640 1 -- 172.21.15.112:0/2692880618 shutdown_connections 2024-09-21T00:29:40.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:40.861+0000 7fc474932640 1 --2- 172.21.15.112:0/2692880618 >> v2:172.21.15.187:3300/0 conn(0x7fc4700d01b0 0x7fc4700d05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:40.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:40.861+0000 7fc474932640 1 --2- 172.21.15.112:0/2692880618 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4700d3880 0x7fc470163260 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:40.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:40.861+0000 7fc474932640 1 -- 172.21.15.112:0/2692880618 >> 172.21.15.112:0/2692880618 conn(0x7fc47008bfe0 msgr2=0x7fc47009d890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:40.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:40.861+0000 7fc474932640 1 -- 172.21.15.112:0/2692880618 shutdown_connections 2024-09-21T00:29:40.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:40.861+0000 7fc474932640 1 -- 172.21.15.112:0/2692880618 wait complete. 2024-09-21T00:29:40.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:40.863+0000 7fc474932640 1 Processor -- start 2024-09-21T00:29:40.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:40.863+0000 7fc474932640 1 -- start start 2024-09-21T00:29:40.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:40.864+0000 7fc474932640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4700d01b0 0x7fc4701f5f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:40.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:40.864+0000 7fc474932640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc4700d3880 0x7fc4701f4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:40.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:40.864+0000 7fc474932640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc4700d48a0 0x7fc47015f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:40.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:40.864+0000 7fc474932640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc4700d2460 con 0x7fc4700d01b0 2024-09-21T00:29:40.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:40.864+0000 7fc474932640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc4700d22e0 con 0x7fc4700d48a0 2024-09-21T00:29:40.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:40.864+0000 7fc474932640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc4700d25e0 con 0x7fc4700d3880 2024-09-21T00:29:40.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:40.864+0000 7fc46e59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4700d01b0 0x7fc4701f5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:40.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:40.865+0000 7fc46e59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4700d01b0 0x7fc4701f5f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36416/0 (socket says 172.21.15.112:36416) 2024-09-21T00:29:40.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:40.865+0000 7fc46e59c640 1 -- 172.21.15.112:0/3618492024 learned_addr learned my addr 172.21.15.112:0/3618492024 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:29:40.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:40.865+0000 7fc46ed9d640 1 --2- 172.21.15.112:0/3618492024 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc4700d48a0 0x7fc47015f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:40.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:40.865+0000 7fc46dd9b640 1 --2- 172.21.15.112:0/3618492024 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc4700d3880 0x7fc4701f4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:40.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:40.865+0000 7fc46e59c640 1 -- 172.21.15.112:0/3618492024 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc4700d3880 msgr2=0x7fc4701f4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:40.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:40.865+0000 7fc46e59c640 1 --2- 172.21.15.112:0/3618492024 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc4700d3880 0x7fc4701f4cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:40.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:40.865+0000 7fc46e59c640 1 -- 172.21.15.112:0/3618492024 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc4700d48a0 msgr2=0x7fc47015f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:40.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:40.865+0000 7fc46e59c640 1 --2- 172.21.15.112:0/3618492024 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc4700d48a0 0x7fc47015f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:40.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:40.865+0000 7fc46e59c640 1 -- 172.21.15.112:0/3618492024 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4701617c0 con 0x7fc4700d01b0 2024-09-21T00:29:40.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:40.865+0000 7fc46e59c640 1 --2- 172.21.15.112:0/3618492024 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4700d01b0 0x7fc4701f5f30 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fc46000c430 tx=0x7fc46000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:40.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:40.865+0000 7fc45effd640 1 -- 172.21.15.112:0/3618492024 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc460011790 con 0x7fc4700d01b0 2024-09-21T00:29:40.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:40.865+0000 7fc45effd640 1 -- 172.21.15.112:0/3618492024 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc460017070 con 0x7fc4700d01b0 2024-09-21T00:29:40.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:40.865+0000 7fc45effd640 1 -- 172.21.15.112:0/3618492024 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc460011ce0 con 0x7fc4700d01b0 2024-09-21T00:29:40.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:40.865+0000 7fc474932640 1 -- 172.21.15.112:0/3618492024 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc470161aa0 con 0x7fc4700d01b0 2024-09-21T00:29:40.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:40.865+0000 7fc474932640 1 -- 172.21.15.112:0/3618492024 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc470162060 con 0x7fc4700d01b0 2024-09-21T00:29:40.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:40.866+0000 7fc474932640 1 -- 172.21.15.112:0/3618492024 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4700d2a70 con 0x7fc4700d01b0 2024-09-21T00:29:40.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:40.867+0000 7fc45effd640 1 -- 172.21.15.112:0/3618492024 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc46001c460 con 0x7fc4700d01b0 2024-09-21T00:29:40.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:40.867+0000 7fc45effd640 1 --2- 172.21.15.112:0/3618492024 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fc450078bc0 0x7fc45007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:40.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:40.868+0000 7fc45effd640 1 -- 172.21.15.112:0/3618492024 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5335+0+0 (secure 0 0 0) 0x7fc46009a930 con 0x7fc4700d01b0 2024-09-21T00:29:40.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:40.868+0000 7fc46dd9b640 1 --2- 172.21.15.112:0/3618492024 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fc450078bc0 0x7fc45007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:40.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:40.868+0000 7fc46dd9b640 1 --2- 172.21.15.112:0/3618492024 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fc450078bc0 0x7fc45007b0d0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fc464034000 tx=0x7fc464004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:40.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:40.872+0000 7fc45effd640 1 -- 172.21.15.112:0/3618492024 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc460062f70 con 0x7fc4700d01b0 2024-09-21T00:29:41.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:41.048+0000 7fc474932640 1 -- 172.21.15.112:0/3618492024 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc4700b6f90 con 0x7fc4700d01b0 2024-09-21T00:29:41.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:41.049+0000 7fc45effd640 1 -- 172.21.15.112:0/3618492024 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v99) ==== 93+0+10 (secure 0 0 0) 0x7fc460062870 con 0x7fc4700d01b0 2024-09-21T00:29:41.050 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2024-09-21T00:29:41.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:41.051+0000 7fc474932640 1 -- 172.21.15.112:0/3618492024 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fc450078bc0 msgr2=0x7fc45007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:41.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:41.051+0000 7fc474932640 1 --2- 172.21.15.112:0/3618492024 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fc450078bc0 0x7fc45007b0d0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fc464034000 tx=0x7fc464004730 comp rx=0 tx=0).stop 2024-09-21T00:29:41.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:41.051+0000 7fc474932640 1 -- 172.21.15.112:0/3618492024 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4700d01b0 msgr2=0x7fc4701f5f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:41.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:41.051+0000 7fc474932640 1 --2- 172.21.15.112:0/3618492024 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4700d01b0 0x7fc4701f5f30 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fc46000c430 tx=0x7fc46000c8f0 comp rx=0 tx=0).stop 2024-09-21T00:29:41.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:41.051+0000 7fc474932640 1 -- 172.21.15.112:0/3618492024 shutdown_connections 2024-09-21T00:29:41.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:41.051+0000 7fc474932640 1 --2- 172.21.15.112:0/3618492024 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc4700d48a0 0x7fc47015f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:41.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:41.051+0000 7fc474932640 1 --2- 172.21.15.112:0/3618492024 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc4700d3880 0x7fc4701f4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:41.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:41.051+0000 7fc474932640 1 --2- 172.21.15.112:0/3618492024 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fc450078bc0 0x7fc45007b0d0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:41.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:41.051+0000 7fc474932640 1 --2- 172.21.15.112:0/3618492024 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4700d01b0 0x7fc4701f5f30 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:41.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:41.051+0000 7fc474932640 1 -- 172.21.15.112:0/3618492024 >> 172.21.15.112:0/3618492024 conn(0x7fc47008bfe0 msgr2=0x7fc4700b12d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:41.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:41.051+0000 7fc474932640 1 -- 172.21.15.112:0/3618492024 shutdown_connections 2024-09-21T00:29:41.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:41.051+0000 7fc474932640 1 -- 172.21.15.112:0/3618492024 wait complete. 2024-09-21T00:29:41.063 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_sessionmap.TestSessionMap.test_version_splitting' 2024-09-21T00:29:41.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:41.175+0000 7ff7cf739640 1 -- 172.21.15.112:0/2114297400 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff7c80d2030 msgr2=0x7ff7c815d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:41.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:41.175+0000 7ff7cf739640 1 --2- 172.21.15.112:0/2114297400 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff7c80d2030 0x7ff7c815d870 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7ff7c4005fd0 tx=0x7ff7c402a530 comp rx=0 tx=0).stop 2024-09-21T00:29:41.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:41.175+0000 7ff7cf739640 1 -- 172.21.15.112:0/2114297400 shutdown_connections 2024-09-21T00:29:41.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:41.175+0000 7ff7cf739640 1 --2- 172.21.15.112:0/2114297400 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff7c80d0860 0x7ff7c815a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:41.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:41.175+0000 7ff7cf739640 1 --2- 172.21.15.112:0/2114297400 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff7c80d2030 0x7ff7c815d870 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:41.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:41.175+0000 7ff7cf739640 1 -- 172.21.15.112:0/2114297400 >> 172.21.15.112:0/2114297400 conn(0x7ff7c80b43f0 msgr2=0x7ff7c80b4830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:41.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:41.175+0000 7ff7cf739640 1 -- 172.21.15.112:0/2114297400 shutdown_connections 2024-09-21T00:29:41.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:41.175+0000 7ff7cf739640 1 -- 172.21.15.112:0/2114297400 wait complete. 2024-09-21T00:29:41.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:41.177+0000 7ff7cf739640 1 Processor -- start 2024-09-21T00:29:41.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:41.177+0000 7ff7cf739640 1 -- start start 2024-09-21T00:29:41.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:41.178+0000 7ff7cf739640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff7c80ccae0 0x7ff7c81576c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:41.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:41.178+0000 7ff7cf739640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff7c80d0860 0x7ff7c80b7e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:41.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:41.178+0000 7ff7cf739640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff7c80d2030 0x7ff7c81f6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:41.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:41.178+0000 7ff7cf739640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c80b22d0 con 0x7ff7c80d2030 2024-09-21T00:29:41.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:41.178+0000 7ff7cf739640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c80b2150 con 0x7ff7c80ccae0 2024-09-21T00:29:41.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:41.178+0000 7ff7cf739640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff7c80b2450 con 0x7ff7c80d0860 2024-09-21T00:29:41.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:41.178+0000 7ff7cdcd6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff7c80d2030 0x7ff7c81f6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:41.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:41.178+0000 7ff7cdcd6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff7c80d2030 0x7ff7c81f6fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36436/0 (socket says 172.21.15.112:36436) 2024-09-21T00:29:41.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:41.178+0000 7ff7cdcd6640 1 -- 172.21.15.112:0/1737402778 learned_addr learned my addr 172.21.15.112:0/1737402778 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:29:41.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:41.178+0000 7ff7cccd4640 1 --2- 172.21.15.112:0/1737402778 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff7c80d0860 0x7ff7c80b7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:41.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:41.178+0000 7ff7cd4d5640 1 --2- 172.21.15.112:0/1737402778 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff7c80ccae0 0x7ff7c81576c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:41.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:41.178+0000 7ff7cdcd6640 1 -- 172.21.15.112:0/1737402778 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff7c80d0860 msgr2=0x7ff7c80b7e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:41.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:41.178+0000 7ff7cdcd6640 1 --2- 172.21.15.112:0/1737402778 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff7c80d0860 0x7ff7c80b7e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:41.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:41.178+0000 7ff7cdcd6640 1 -- 172.21.15.112:0/1737402778 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff7c80ccae0 msgr2=0x7ff7c81576c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:41.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:41.178+0000 7ff7cdcd6640 1 --2- 172.21.15.112:0/1737402778 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff7c80ccae0 0x7ff7c81576c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:41.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:41.178+0000 7ff7cdcd6640 1 -- 172.21.15.112:0/1737402778 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7c81f7e70 con 0x7ff7c80d2030 2024-09-21T00:29:41.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:41.179+0000 7ff7cdcd6640 1 --2- 172.21.15.112:0/1737402778 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff7c80d2030 0x7ff7c81f6fa0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7ff7c000ba10 tx=0x7ff7c000bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:41.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:41.179+0000 7ff7bdffb640 1 -- 172.21.15.112:0/1737402778 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7c0019070 con 0x7ff7c80d2030 2024-09-21T00:29:41.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:41.179+0000 7ff7bdffb640 1 -- 172.21.15.112:0/1737402778 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff7c000cc90 con 0x7ff7c80d2030 2024-09-21T00:29:41.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:41.179+0000 7ff7cf739640 1 -- 172.21.15.112:0/1737402778 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7c80b7b20 con 0x7ff7c80d2030 2024-09-21T00:29:41.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:41.179+0000 7ff7bdffb640 1 -- 172.21.15.112:0/1737402778 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7c00146a0 con 0x7ff7c80d2030 2024-09-21T00:29:41.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:41.179+0000 7ff7cf739640 1 -- 172.21.15.112:0/1737402778 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7c80b1640 con 0x7ff7c80d2030 2024-09-21T00:29:41.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:41.180+0000 7ff7cf739640 1 -- 172.21.15.112:0/1737402778 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7c80b7890 con 0x7ff7c80d2030 2024-09-21T00:29:41.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:41.181+0000 7ff7bdffb640 1 -- 172.21.15.112:0/1737402778 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff7c001c070 con 0x7ff7c80d2030 2024-09-21T00:29:41.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:41.181+0000 7ff7bdffb640 1 --2- 172.21.15.112:0/1737402778 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff7ac078b90 0x7ff7ac07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:41.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:41.181+0000 7ff7bdffb640 1 -- 172.21.15.112:0/1737402778 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5335+0+0 (secure 0 0 0) 0x7ff7c009a8d0 con 0x7ff7c80d2030 2024-09-21T00:29:41.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:41.182+0000 7ff7cd4d5640 1 --2- 172.21.15.112:0/1737402778 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff7ac078b90 0x7ff7ac07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:41.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:41.182+0000 7ff7cd4d5640 1 --2- 172.21.15.112:0/1737402778 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff7ac078b90 0x7ff7ac07b0a0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7ff7c4033000 tx=0x7ff7c4004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:41.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:41.186+0000 7ff7bdffb640 1 -- 172.21.15.112:0/1737402778 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ff7c0062fa0 con 0x7ff7c80d2030 2024-09-21T00:29:41.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:41.431+0000 7ff7cf739640 1 -- 172.21.15.112:0/1737402778 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_sessionmap.TestSessionMap.test_version_splitting"]} v 0) -- 0x7ff7c80b2960 con 0x7ff7c80d2030 2024-09-21T00:29:42.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:42.419+0000 7ff7bdffb640 1 -- 172.21.15.112:0/1737402778 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_sessionmap.TestSessionMap.test_version_splitting"]}]=0 v238) ==== 148+0+0 (secure 0 0 0) 0x7ff7c00628a0 con 0x7ff7c80d2030 2024-09-21T00:29:42.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:42.422+0000 7ff7cf739640 1 -- 172.21.15.112:0/1737402778 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff7ac078b90 msgr2=0x7ff7ac07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:42.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:42.422+0000 7ff7cf739640 1 --2- 172.21.15.112:0/1737402778 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff7ac078b90 0x7ff7ac07b0a0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7ff7c4033000 tx=0x7ff7c4004730 comp rx=0 tx=0).stop 2024-09-21T00:29:42.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:42.422+0000 7ff7cf739640 1 -- 172.21.15.112:0/1737402778 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff7c80d2030 msgr2=0x7ff7c81f6fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:42.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:42.422+0000 7ff7cf739640 1 --2- 172.21.15.112:0/1737402778 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff7c80d2030 0x7ff7c81f6fa0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7ff7c000ba10 tx=0x7ff7c000bed0 comp rx=0 tx=0).stop 2024-09-21T00:29:42.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:42.422+0000 7ff7cf739640 1 -- 172.21.15.112:0/1737402778 shutdown_connections 2024-09-21T00:29:42.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:42.422+0000 7ff7cf739640 1 --2- 172.21.15.112:0/1737402778 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff7ac078b90 0x7ff7ac07b0a0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:42.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:42.422+0000 7ff7cf739640 1 --2- 172.21.15.112:0/1737402778 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff7c80d2030 0x7ff7c81f6fa0 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:42.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:42.422+0000 7ff7cf739640 1 --2- 172.21.15.112:0/1737402778 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff7c80d0860 0x7ff7c80b7e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:42.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:42.422+0000 7ff7cf739640 1 --2- 172.21.15.112:0/1737402778 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff7c80ccae0 0x7ff7c81576c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:42.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:42.422+0000 7ff7cf739640 1 -- 172.21.15.112:0/1737402778 >> 172.21.15.112:0/1737402778 conn(0x7ff7c80b43f0 msgr2=0x7ff7c815d110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:42.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:42.422+0000 7ff7cf739640 1 -- 172.21.15.112:0/1737402778 shutdown_connections 2024-09-21T00:29:42.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:42.422+0000 7ff7cf739640 1 -- 172.21.15.112:0/1737402778 wait complete. 2024-09-21T00:29:42.436 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-21T00:29:42.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:42.554+0000 7f23d715f640 1 -- 172.21.15.112:0/3120303519 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23d00ccce0 msgr2=0x7f23d00b4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:42.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:42.554+0000 7f23d715f640 1 --2- 172.21.15.112:0/3120303519 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23d00ccce0 0x7f23d00b4720 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f23d0074ba0 tx=0x7f23c002a530 comp rx=0 tx=0).stop 2024-09-21T00:29:42.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:42.554+0000 7f23d715f640 1 -- 172.21.15.112:0/3120303519 shutdown_connections 2024-09-21T00:29:42.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:42.554+0000 7f23d715f640 1 --2- 172.21.15.112:0/3120303519 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23d00ccce0 0x7f23d00b4720 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:42.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:42.554+0000 7f23d715f640 1 --2- 172.21.15.112:0/3120303519 >> v2:172.21.15.187:3300/0 conn(0x7f23d015ca50 0x7f23d00b41e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:42.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:42.554+0000 7f23d715f640 1 -- 172.21.15.112:0/3120303519 >> 172.21.15.112:0/3120303519 conn(0x7f23d00cf220 msgr2=0x7f23d00cf630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:42.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:42.554+0000 7f23d715f640 1 -- 172.21.15.112:0/3120303519 shutdown_connections 2024-09-21T00:29:42.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:42.554+0000 7f23d715f640 1 -- 172.21.15.112:0/3120303519 wait complete. 2024-09-21T00:29:42.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:42.556+0000 7f23d715f640 1 Processor -- start 2024-09-21T00:29:42.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:42.556+0000 7f23d715f640 1 -- start start 2024-09-21T00:29:42.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:42.557+0000 7f23d715f640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f23d00b1bc0 0x7f23d00cbfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:42.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:42.557+0000 7f23d715f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f23d00ccce0 0x7f23d01f6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:42.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:42.557+0000 7f23d715f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23d015ca50 0x7f23d01f70f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:42.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:42.557+0000 7f23d715f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f23d00d1fb0 con 0x7f23d015ca50 2024-09-21T00:29:42.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:42.557+0000 7f23d715f640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f23d00c92b0 con 0x7f23d00ccce0 2024-09-21T00:29:42.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:42.557+0000 7f23d715f640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f23d00d2130 con 0x7f23d00b1bc0 2024-09-21T00:29:42.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:42.557+0000 7f23d56fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23d015ca50 0x7f23d01f70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:42.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:42.558+0000 7f23d56fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23d015ca50 0x7f23d01f70f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36458/0 (socket says 172.21.15.112:36458) 2024-09-21T00:29:42.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:42.558+0000 7f23d56fc640 1 -- 172.21.15.112:0/2565712970 learned_addr learned my addr 172.21.15.112:0/2565712970 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:29:42.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:42.558+0000 7f23d4efb640 1 --2- 172.21.15.112:0/2565712970 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f23d00b1bc0 0x7f23d00cbfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:42.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:42.558+0000 7f23c7fff640 1 --2- 172.21.15.112:0/2565712970 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f23d00ccce0 0x7f23d01f6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:42.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:42.558+0000 7f23d56fc640 1 -- 172.21.15.112:0/2565712970 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f23d00b1bc0 msgr2=0x7f23d00cbfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:42.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:42.558+0000 7f23d56fc640 1 --2- 172.21.15.112:0/2565712970 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f23d00b1bc0 0x7f23d00cbfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:42.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:42.558+0000 7f23d56fc640 1 -- 172.21.15.112:0/2565712970 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f23d00ccce0 msgr2=0x7f23d01f6bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:42.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:42.558+0000 7f23d56fc640 1 --2- 172.21.15.112:0/2565712970 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f23d00ccce0 0x7f23d01f6bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:42.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:42.558+0000 7f23d56fc640 1 -- 172.21.15.112:0/2565712970 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23d0157520 con 0x7f23d015ca50 2024-09-21T00:29:42.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:42.558+0000 7f23d4efb640 1 --2- 172.21.15.112:0/2565712970 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f23d00b1bc0 0x7f23d00cbfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:29:42.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:42.558+0000 7f23d56fc640 1 --2- 172.21.15.112:0/2565712970 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23d015ca50 0x7f23d01f70f0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f23c800c730 tx=0x7f23c800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:42.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:42.558+0000 7f23c57fa640 1 -- 172.21.15.112:0/2565712970 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23c8003b70 con 0x7f23d015ca50 2024-09-21T00:29:42.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:42.558+0000 7f23c57fa640 1 -- 172.21.15.112:0/2565712970 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f23c8003d30 con 0x7f23d015ca50 2024-09-21T00:29:42.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:42.558+0000 7f23d715f640 1 -- 172.21.15.112:0/2565712970 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23d0159620 con 0x7f23d015ca50 2024-09-21T00:29:42.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:42.558+0000 7f23c57fa640 1 -- 172.21.15.112:0/2565712970 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23c80052d0 con 0x7f23d015ca50 2024-09-21T00:29:42.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:42.558+0000 7f23d715f640 1 -- 172.21.15.112:0/2565712970 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23d00b6170 con 0x7f23d015ca50 2024-09-21T00:29:42.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:42.559+0000 7f23d715f640 1 -- 172.21.15.112:0/2565712970 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23d00b2850 con 0x7f23d015ca50 2024-09-21T00:29:42.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:42.560+0000 7f23c57fa640 1 -- 172.21.15.112:0/2565712970 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f23c8005470 con 0x7f23d015ca50 2024-09-21T00:29:42.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:42.561+0000 7f23c57fa640 1 --2- 172.21.15.112:0/2565712970 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f23bc078b90 0x7f23bc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:42.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:42.561+0000 7f23c57fa640 1 -- 172.21.15.112:0/2565712970 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5335+0+0 (secure 0 0 0) 0x7f23c809a910 con 0x7f23d015ca50 2024-09-21T00:29:42.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:42.561+0000 7f23d4efb640 1 --2- 172.21.15.112:0/2565712970 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f23bc078b90 0x7f23bc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:42.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:42.561+0000 7f23d4efb640 1 --2- 172.21.15.112:0/2565712970 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f23bc078b90 0x7f23bc07b0a0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f23cc005fd0 tx=0x7f23cc00f360 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:42.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:42.565+0000 7f23c57fa640 1 -- 172.21.15.112:0/2565712970 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f23c8062ec0 con 0x7f23d015ca50 2024-09-21T00:29:42.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:42.741+0000 7f23d715f640 1 -- 172.21.15.112:0/2565712970 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f23d00b7dc0 con 0x7f23d015ca50 2024-09-21T00:29:42.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:42.742+0000 7f23c57fa640 1 -- 172.21.15.112:0/2565712970 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v46) ==== 59+0+2294 (secure 0 0 0) 0x7f23c80627c0 con 0x7f23d015ca50 2024-09-21T00:29:42.746 INFO:teuthology.orchestra.run.smithi112.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-21T00:29:42.746 INFO:teuthology.orchestra.run.smithi112.stdout:mon advanced mon_op_complaint_time 120 2024-09-21T00:29:42.746 INFO:teuthology.orchestra.run.smithi112.stdout:mgr advanced client_mount_timeout 30 2024-09-21T00:29:42.747 INFO:teuthology.orchestra.run.smithi112.stdout:mgr advanced debug_client 20/20 2024-09-21T00:29:42.747 INFO:teuthology.orchestra.run.smithi112.stdout:mgr advanced debug_mgr 20/20 2024-09-21T00:29:42.747 INFO:teuthology.orchestra.run.smithi112.stdout:mgr advanced debug_ms 1/1 2024-09-21T00:29:42.747 INFO:teuthology.orchestra.run.smithi112.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-21T00:29:42.747 INFO:teuthology.orchestra.run.smithi112.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-21T00:29:42.747 INFO:teuthology.orchestra.run.smithi112.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 5090.401743 2024-09-21T00:29:42.747 INFO:teuthology.orchestra.run.smithi112.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 7876.743127 2024-09-21T00:29:42.747 INFO:teuthology.orchestra.run.smithi112.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 4865.073913 2024-09-21T00:29:42.747 INFO:teuthology.orchestra.run.smithi112.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 5117.206240 2024-09-21T00:29:42.747 INFO:teuthology.orchestra.run.smithi112.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 4329.674604 2024-09-21T00:29:42.747 INFO:teuthology.orchestra.run.smithi112.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4322.652151 2024-09-21T00:29:42.747 INFO:teuthology.orchestra.run.smithi112.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 4244.674337 2024-09-21T00:29:42.747 INFO:teuthology.orchestra.run.smithi112.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 4472.316624 2024-09-21T00:29:42.747 INFO:teuthology.orchestra.run.smithi112.stdout:mds advanced debug_mds 20/20 2024-09-21T00:29:42.748 INFO:teuthology.orchestra.run.smithi112.stdout:mds advanced debug_mds_balancer 20/20 2024-09-21T00:29:42.748 INFO:teuthology.orchestra.run.smithi112.stdout:mds advanced debug_ms 2/2 2024-09-21T00:29:42.748 INFO:teuthology.orchestra.run.smithi112.stdout:mds dev mds_debug_frag true 2024-09-21T00:29:42.748 INFO:teuthology.orchestra.run.smithi112.stdout:mds dev mds_debug_scatterstat true 2024-09-21T00:29:42.748 INFO:teuthology.orchestra.run.smithi112.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-21T00:29:42.748 INFO:teuthology.orchestra.run.smithi112.stdout:mds dev mds_verify_scatter true 2024-09-21T00:29:42.748 INFO:teuthology.orchestra.run.smithi112.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-21T00:29:42.748 INFO:teuthology.orchestra.run.smithi112.stdout:mds advanced rados_mon_op_timeout 900 2024-09-21T00:29:42.748 INFO:teuthology.orchestra.run.smithi112.stdout:mds advanced rados_osd_op_timeout 900 2024-09-21T00:29:42.748 INFO:teuthology.orchestra.run.smithi112.stdout:client advanced client_mount_timeout 600 2024-09-21T00:29:42.748 INFO:teuthology.orchestra.run.smithi112.stdout:client advanced debug_client 20/20 2024-09-21T00:29:42.748 INFO:teuthology.orchestra.run.smithi112.stdout:client advanced debug_ms 1/1 2024-09-21T00:29:42.748 INFO:teuthology.orchestra.run.smithi112.stdout:client advanced rados_mon_op_timeout 900 2024-09-21T00:29:42.748 INFO:teuthology.orchestra.run.smithi112.stdout:client advanced rados_osd_op_timeout 900 2024-09-21T00:29:42.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:42.744+0000 7f23d715f640 1 -- 172.21.15.112:0/2565712970 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f23bc078b90 msgr2=0x7f23bc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:42.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:42.744+0000 7f23d715f640 1 --2- 172.21.15.112:0/2565712970 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f23bc078b90 0x7f23bc07b0a0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f23cc005fd0 tx=0x7f23cc00f360 comp rx=0 tx=0).stop 2024-09-21T00:29:42.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:42.744+0000 7f23d715f640 1 -- 172.21.15.112:0/2565712970 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23d015ca50 msgr2=0x7f23d01f70f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:42.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:42.744+0000 7f23d715f640 1 --2- 172.21.15.112:0/2565712970 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23d015ca50 0x7f23d01f70f0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f23c800c730 tx=0x7f23c800cbf0 comp rx=0 tx=0).stop 2024-09-21T00:29:42.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:42.744+0000 7f23d715f640 1 -- 172.21.15.112:0/2565712970 shutdown_connections 2024-09-21T00:29:42.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:42.744+0000 7f23d715f640 1 --2- 172.21.15.112:0/2565712970 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f23bc078b90 0x7f23bc07b0a0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:42.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:42.744+0000 7f23d715f640 1 --2- 172.21.15.112:0/2565712970 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23d015ca50 0x7f23d01f70f0 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:42.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:42.744+0000 7f23d715f640 1 --2- 172.21.15.112:0/2565712970 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f23d00ccce0 0x7f23d01f6bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:42.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:42.744+0000 7f23d715f640 1 --2- 172.21.15.112:0/2565712970 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f23d00b1bc0 0x7f23d00cbfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:42.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:42.744+0000 7f23d715f640 1 -- 172.21.15.112:0/2565712970 >> 172.21.15.112:0/2565712970 conn(0x7f23d00cf220 msgr2=0x7f23d00b1fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:42.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:42.744+0000 7f23d715f640 1 -- 172.21.15.112:0/2565712970 shutdown_connections 2024-09-21T00:29:42.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:42.744+0000 7f23d715f640 1 -- 172.21.15.112:0/2565712970 wait complete. 2024-09-21T00:29:42.756 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-21T00:29:42.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:42.870+0000 7ff9cc9f1640 1 -- 172.21.15.112:0/631254731 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff9c80ce9e0 msgr2=0x7ff9c80cad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:42.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:42.870+0000 7ff9cc9f1640 1 --2- 172.21.15.112:0/631254731 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff9c80ce9e0 0x7ff9c80cad20 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7ff9c0005fd0 tx=0x7ff9c002aa90 comp rx=0 tx=0).stop 2024-09-21T00:29:42.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:42.870+0000 7ff9cc9f1640 1 -- 172.21.15.112:0/631254731 shutdown_connections 2024-09-21T00:29:42.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:42.870+0000 7ff9cc9f1640 1 --2- 172.21.15.112:0/631254731 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff9c80ce9e0 0x7ff9c80cad20 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:42.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:42.870+0000 7ff9cc9f1640 1 --2- 172.21.15.112:0/631254731 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff9c80d0e10 0x7ff9c80ca7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:42.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:42.870+0000 7ff9cc9f1640 1 -- 172.21.15.112:0/631254731 >> 172.21.15.112:0/631254731 conn(0x7ff9c80b3580 msgr2=0x7ff9c80b3990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:42.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:42.870+0000 7ff9cc9f1640 1 -- 172.21.15.112:0/631254731 shutdown_connections 2024-09-21T00:29:42.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:42.870+0000 7ff9cc9f1640 1 -- 172.21.15.112:0/631254731 wait complete. 2024-09-21T00:29:42.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:42.872+0000 7ff9cc9f1640 1 Processor -- start 2024-09-21T00:29:42.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:42.872+0000 7ff9cc9f1640 1 -- start start 2024-09-21T00:29:42.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:42.874+0000 7ff9cc9f1640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff9c80cc270 0x7ff9c809b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:42.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:42.874+0000 7ff9cc9f1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff9c80ce9e0 0x7ff9c81f69c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:42.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:42.874+0000 7ff9cc9f1640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff9c80b1150 0x7ff9c80b1560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:42.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:42.874+0000 7ff9cc9f1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c80cb3e0 con 0x7ff9c80ce9e0 2024-09-21T00:29:42.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:42.874+0000 7ff9cc9f1640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c80cb260 con 0x7ff9c80cc270 2024-09-21T00:29:42.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:42.874+0000 7ff9cc9f1640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff9c80cb560 con 0x7ff9c80b1150 2024-09-21T00:29:42.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:42.874+0000 7ff9c5d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff9c80ce9e0 0x7ff9c81f69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:42.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:42.874+0000 7ff9c5d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff9c80ce9e0 0x7ff9c81f69c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36484/0 (socket says 172.21.15.112:36484) 2024-09-21T00:29:42.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:42.874+0000 7ff9c5d9b640 1 -- 172.21.15.112:0/3969281665 learned_addr learned my addr 172.21.15.112:0/3969281665 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:29:42.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:42.874+0000 7ff9c6d9d640 1 --2- 172.21.15.112:0/3969281665 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff9c80b1150 0x7ff9c80b1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:42.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:42.874+0000 7ff9c659c640 1 --2- 172.21.15.112:0/3969281665 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff9c80cc270 0x7ff9c809b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:42.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:42.874+0000 7ff9c5d9b640 1 -- 172.21.15.112:0/3969281665 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff9c80b1150 msgr2=0x7ff9c80b1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:42.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:42.874+0000 7ff9c5d9b640 1 --2- 172.21.15.112:0/3969281665 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff9c80b1150 0x7ff9c80b1560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:42.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:42.874+0000 7ff9c5d9b640 1 -- 172.21.15.112:0/3969281665 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff9c80cc270 msgr2=0x7ff9c809b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:42.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:42.874+0000 7ff9c5d9b640 1 --2- 172.21.15.112:0/3969281665 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff9c80cc270 0x7ff9c809b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:42.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:42.874+0000 7ff9c5d9b640 1 -- 172.21.15.112:0/3969281665 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9c80b1aa0 con 0x7ff9c80ce9e0 2024-09-21T00:29:42.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:42.874+0000 7ff9c5d9b640 1 --2- 172.21.15.112:0/3969281665 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff9c80ce9e0 0x7ff9c81f69c0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc00d6d0 tx=0x7ff9bc00db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:42.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:42.874+0000 7ff9b6ffd640 1 -- 172.21.15.112:0/3969281665 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9bc017070 con 0x7ff9c80ce9e0 2024-09-21T00:29:42.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:42.874+0000 7ff9b6ffd640 1 -- 172.21.15.112:0/3969281665 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff9bc005310 con 0x7ff9c80ce9e0 2024-09-21T00:29:42.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:42.874+0000 7ff9cc9f1640 1 -- 172.21.15.112:0/3969281665 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9c809a5a0 con 0x7ff9c80ce9e0 2024-09-21T00:29:42.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:42.874+0000 7ff9b6ffd640 1 -- 172.21.15.112:0/3969281665 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9bc009820 con 0x7ff9c80ce9e0 2024-09-21T00:29:42.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:42.875+0000 7ff9cc9f1640 1 -- 172.21.15.112:0/3969281665 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9c809ab60 con 0x7ff9c80ce9e0 2024-09-21T00:29:42.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:42.875+0000 7ff9cc9f1640 1 -- 172.21.15.112:0/3969281665 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9c80ca7e0 con 0x7ff9c80ce9e0 2024-09-21T00:29:42.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:42.876+0000 7ff9b6ffd640 1 -- 172.21.15.112:0/3969281665 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff9bc0099c0 con 0x7ff9c80ce9e0 2024-09-21T00:29:42.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:42.877+0000 7ff9b6ffd640 1 --2- 172.21.15.112:0/3969281665 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff9a8078b90 0x7ff9a807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:42.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:42.877+0000 7ff9b6ffd640 1 -- 172.21.15.112:0/3969281665 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5335+0+0 (secure 0 0 0) 0x7ff9bc09abe0 con 0x7ff9c80ce9e0 2024-09-21T00:29:42.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:42.877+0000 7ff9c659c640 1 --2- 172.21.15.112:0/3969281665 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff9a8078b90 0x7ff9a807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:42.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:42.877+0000 7ff9c659c640 1 --2- 172.21.15.112:0/3969281665 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff9a8078b90 0x7ff9a807b0a0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7ff9b8006fd0 tx=0x7ff9b8009060 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:42.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:42.881+0000 7ff9b6ffd640 1 -- 172.21.15.112:0/3969281665 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ff9c80b1aa0 con 0x7ff9c80ce9e0 2024-09-21T00:29:43.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:43.058+0000 7ff9cc9f1640 1 -- 172.21.15.112:0/3969281665 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7ff9c81e8b60 con 0x7ff9c80ce9e0 2024-09-21T00:29:43.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:43.062+0000 7ff9b6ffd640 1 -- 172.21.15.112:0/3969281665 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v47)=0 v47) ==== 122+0+0 (secure 0 0 0) 0x7ff9bc062bd0 con 0x7ff9c80ce9e0 2024-09-21T00:29:43.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:43.064+0000 7ff9cc9f1640 1 -- 172.21.15.112:0/3969281665 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff9a8078b90 msgr2=0x7ff9a807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:43.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:43.064+0000 7ff9cc9f1640 1 --2- 172.21.15.112:0/3969281665 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff9a8078b90 0x7ff9a807b0a0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7ff9b8006fd0 tx=0x7ff9b8009060 comp rx=0 tx=0).stop 2024-09-21T00:29:43.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:43.064+0000 7ff9cc9f1640 1 -- 172.21.15.112:0/3969281665 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff9c80ce9e0 msgr2=0x7ff9c81f69c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:43.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:43.064+0000 7ff9cc9f1640 1 --2- 172.21.15.112:0/3969281665 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff9c80ce9e0 0x7ff9c81f69c0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc00d6d0 tx=0x7ff9bc00db90 comp rx=0 tx=0).stop 2024-09-21T00:29:43.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:43.064+0000 7ff9cc9f1640 1 -- 172.21.15.112:0/3969281665 shutdown_connections 2024-09-21T00:29:43.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:43.064+0000 7ff9cc9f1640 1 --2- 172.21.15.112:0/3969281665 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff9c80b1150 0x7ff9c80b1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:43.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:43.064+0000 7ff9cc9f1640 1 --2- 172.21.15.112:0/3969281665 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff9a8078b90 0x7ff9a807b0a0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:43.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:43.064+0000 7ff9cc9f1640 1 --2- 172.21.15.112:0/3969281665 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff9c80ce9e0 0x7ff9c81f69c0 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:43.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:43.064+0000 7ff9cc9f1640 1 --2- 172.21.15.112:0/3969281665 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff9c80cc270 0x7ff9c809b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:43.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:43.064+0000 7ff9cc9f1640 1 -- 172.21.15.112:0/3969281665 >> 172.21.15.112:0/3969281665 conn(0x7ff9c80b3580 msgr2=0x7ff9c8074590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:43.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:43.064+0000 7ff9cc9f1640 1 -- 172.21.15.112:0/3969281665 shutdown_connections 2024-09-21T00:29:43.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:43.064+0000 7ff9cc9f1640 1 -- 172.21.15.112:0/3969281665 wait complete. 2024-09-21T00:29:43.076 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T00:29:43.076 DEBUG:teuthology.orchestra.run.smithi112:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T00:29:43.113 DEBUG:teuthology.orchestra.run.smithi187:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T00:29:43.159 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T00:29:43.159 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T00:29:43.159 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:29:43.174 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056010,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1439 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248716k,nr_inodes=812179,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T00:29:43.174 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-09-21T00:29:43.175 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:29:43.192 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056013,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497440k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16835 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248720k,nr_inodes=812180,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497440k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T00:29:43.192 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T00:29:43.192 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:29:43.230 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056010,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1439 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248716k,nr_inodes=812179,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T00:29:43.230 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-09-21T00:29:43.230 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:29:43.249 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056013,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497440k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16835 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248720k,nr_inodes=812180,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497440k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T00:29:43.249 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-21T00:29:43.249 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T00:29:43.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:43.381+0000 7fa083a94640 1 -- 172.21.15.112:0/3882093646 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa07c0ccbf0 msgr2=0x7fa07c0cd000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:43.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:43.381+0000 7fa083a94640 1 --2- 172.21.15.112:0/3882093646 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa07c0ccbf0 0x7fa07c0cd000 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fa078005fd0 tx=0x7fa07802a540 comp rx=0 tx=0).stop 2024-09-21T00:29:43.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:43.381+0000 7fa083a94640 1 -- 172.21.15.112:0/3882093646 shutdown_connections 2024-09-21T00:29:43.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:43.381+0000 7fa083a94640 1 --2- 172.21.15.112:0/3882093646 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa07c0c5ee0 0x7fa07c0c62f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:43.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:43.381+0000 7fa083a94640 1 --2- 172.21.15.112:0/3882093646 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa07c0ccbf0 0x7fa07c0cd000 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:43.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:43.381+0000 7fa083a94640 1 -- 172.21.15.112:0/3882093646 >> 172.21.15.112:0/3882093646 conn(0x7fa07c0b42e0 msgr2=0x7fa07c0b4720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:43.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:43.381+0000 7fa083a94640 1 -- 172.21.15.112:0/3882093646 shutdown_connections 2024-09-21T00:29:43.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:43.381+0000 7fa083a94640 1 -- 172.21.15.112:0/3882093646 wait complete. 2024-09-21T00:29:43.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:43.383+0000 7fa083a94640 1 Processor -- start 2024-09-21T00:29:43.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:43.383+0000 7fa083a94640 1 -- start start 2024-09-21T00:29:43.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:43.384+0000 7fa083a94640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa07c0c5ee0 0x7fa07c1f96c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:43.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:43.384+0000 7fa083a94640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa07c0c6830 0x7fa07c14c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:43.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:43.384+0000 7fa083a94640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa07c0ccbf0 0x7fa07c15b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:43.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:43.384+0000 7fa083a94640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa07c0b3450 con 0x7fa07c0c6830 2024-09-21T00:29:43.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:43.384+0000 7fa083a94640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa07c0c6f10 con 0x7fa07c0ccbf0 2024-09-21T00:29:43.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:43.384+0000 7fa083a94640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa07c0b35d0 con 0x7fa07c0c5ee0 2024-09-21T00:29:43.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:43.385+0000 7fa08102f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa07c0c6830 0x7fa07c14c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:43.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:43.385+0000 7fa08102f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa07c0c6830 0x7fa07c14c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36508/0 (socket says 172.21.15.112:36508) 2024-09-21T00:29:43.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:43.385+0000 7fa08102f640 1 -- 172.21.15.112:0/1381882826 learned_addr learned my addr 172.21.15.112:0/1381882826 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:29:43.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:43.385+0000 7fa081830640 1 --2- 172.21.15.112:0/1381882826 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa07c0c5ee0 0x7fa07c1f96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:43.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:43.385+0000 7fa082031640 1 --2- 172.21.15.112:0/1381882826 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa07c0ccbf0 0x7fa07c15b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:43.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:43.385+0000 7fa08102f640 1 -- 172.21.15.112:0/1381882826 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa07c0c5ee0 msgr2=0x7fa07c1f96c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:43.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:43.385+0000 7fa08102f640 1 --2- 172.21.15.112:0/1381882826 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa07c0c5ee0 0x7fa07c1f96c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:43.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:43.385+0000 7fa08102f640 1 -- 172.21.15.112:0/1381882826 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa07c0ccbf0 msgr2=0x7fa07c15b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:43.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:43.385+0000 7fa08102f640 1 --2- 172.21.15.112:0/1381882826 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa07c0ccbf0 0x7fa07c15b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:43.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:43.385+0000 7fa08102f640 1 -- 172.21.15.112:0/1381882826 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa07c0ada50 con 0x7fa07c0c6830 2024-09-21T00:29:43.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:43.385+0000 7fa08102f640 1 --2- 172.21.15.112:0/1381882826 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa07c0c6830 0x7fa07c14c8d0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fa06c00d6d0 tx=0x7fa06c00db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:43.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:43.385+0000 7fa0727fc640 1 -- 172.21.15.112:0/1381882826 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa06c017070 con 0x7fa07c0c6830 2024-09-21T00:29:43.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:43.385+0000 7fa0727fc640 1 -- 172.21.15.112:0/1381882826 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa06c005330 con 0x7fa07c0c6830 2024-09-21T00:29:43.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:43.385+0000 7fa083a94640 1 -- 172.21.15.112:0/1381882826 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa07c0add30 con 0x7fa07c0c6830 2024-09-21T00:29:43.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:43.385+0000 7fa0727fc640 1 -- 172.21.15.112:0/1381882826 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa06c009820 con 0x7fa07c0c6830 2024-09-21T00:29:43.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:43.385+0000 7fa083a94640 1 -- 172.21.15.112:0/1381882826 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa07c0ae010 con 0x7fa07c0c6830 2024-09-21T00:29:43.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:43.386+0000 7fa083a94640 1 -- 172.21.15.112:0/1381882826 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa07c0c62f0 con 0x7fa07c0c6830 2024-09-21T00:29:43.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:43.387+0000 7fa0727fc640 1 -- 172.21.15.112:0/1381882826 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa06c009bd0 con 0x7fa07c0c6830 2024-09-21T00:29:43.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:43.388+0000 7fa0727fc640 1 --2- 172.21.15.112:0/1381882826 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fa060078bc0 0x7fa06007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:43.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:43.388+0000 7fa0727fc640 1 -- 172.21.15.112:0/1381882826 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5335+0+0 (secure 0 0 0) 0x7fa06c09ad50 con 0x7fa07c0c6830 2024-09-21T00:29:43.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:43.388+0000 7fa081830640 1 --2- 172.21.15.112:0/1381882826 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fa060078bc0 0x7fa06007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:43.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:43.389+0000 7fa081830640 1 --2- 172.21.15.112:0/1381882826 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fa060078bc0 0x7fa06007b0d0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fa078033000 tx=0x7fa078004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:43.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:43.392+0000 7fa0727fc640 1 -- 172.21.15.112:0/1381882826 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa06c09e030 con 0x7fa07c0c6830 2024-09-21T00:29:43.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:43.605+0000 7fa083a94640 1 -- 172.21.15.112:0/1381882826 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa07c15c680 con 0x7fa07c0c6830 2024-09-21T00:29:43.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:43.606+0000 7fa0727fc640 1 -- 172.21.15.112:0/1381882826 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) ==== 94+0+3923 (secure 0 0 0) 0x7fa06c062dd0 con 0x7fa07c0c6830 2024-09-21T00:29:43.607 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:29:43.607 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":66,"btime":"2024-09-21T00:29:37:401172+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5804,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/3818262153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":3818262153},{"type":"v1","addr":"172.21.15.187:6835","nonce":3818262153}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59},{"gid":6214,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6837/1321768683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":1321768683},{"type":"v1","addr":"172.21.15.187:6837","nonce":1321768683}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59},{"gid":6801,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6834/4037012343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":4037012343},{"type":"v1","addr":"172.21.15.112:6834","nonce":4037012343}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58},{"gid":7068,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/4048643112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":4048643112},{"type":"v1","addr":"172.21.15.112:6835","nonce":4048643112}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66}],"filesystems":[]} 2024-09-21T00:29:43.607 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 66 2024-09-21T00:29:43.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:43.608+0000 7fa083a94640 1 -- 172.21.15.112:0/1381882826 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fa060078bc0 msgr2=0x7fa06007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:43.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:43.608+0000 7fa083a94640 1 --2- 172.21.15.112:0/1381882826 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fa060078bc0 0x7fa06007b0d0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fa078033000 tx=0x7fa078004730 comp rx=0 tx=0).stop 2024-09-21T00:29:43.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:43.608+0000 7fa083a94640 1 -- 172.21.15.112:0/1381882826 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa07c0c6830 msgr2=0x7fa07c14c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:43.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:43.608+0000 7fa083a94640 1 --2- 172.21.15.112:0/1381882826 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa07c0c6830 0x7fa07c14c8d0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fa06c00d6d0 tx=0x7fa06c00db90 comp rx=0 tx=0).stop 2024-09-21T00:29:43.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:43.608+0000 7fa083a94640 1 -- 172.21.15.112:0/1381882826 shutdown_connections 2024-09-21T00:29:43.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:43.608+0000 7fa083a94640 1 --2- 172.21.15.112:0/1381882826 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa07c0ccbf0 0x7fa07c15b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:43.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:43.608+0000 7fa083a94640 1 --2- 172.21.15.112:0/1381882826 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fa060078bc0 0x7fa06007b0d0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:43.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:43.608+0000 7fa083a94640 1 --2- 172.21.15.112:0/1381882826 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa07c0c6830 0x7fa07c14c8d0 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:43.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:43.608+0000 7fa083a94640 1 --2- 172.21.15.112:0/1381882826 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa07c0c5ee0 0x7fa07c1f96c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:43.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:43.608+0000 7fa083a94640 1 -- 172.21.15.112:0/1381882826 >> 172.21.15.112:0/1381882826 conn(0x7fa07c0b42e0 msgr2=0x7fa07c071050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:43.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:43.608+0000 7fa083a94640 1 -- 172.21.15.112:0/1381882826 shutdown_connections 2024-09-21T00:29:43.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:43.608+0000 7fa083a94640 1 -- 172.21.15.112:0/1381882826 wait complete. 2024-09-21T00:29:43.620 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-21T00:29:43.621 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-21T00:29:43.621 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-21T00:29:43.621 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T00:29:43.621 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-21T00:29:43.621 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-21T00:29:43.621 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-21T00:29:43.621 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T00:29:43.622 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-21T00:29:43.622 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-21T00:29:43.622 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-21T00:29:43.622 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T00:29:43.622 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-21T00:29:43.622 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-21T00:29:43.622 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-21T00:29:43.622 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T00:29:43.630 INFO:tasks.ceph.mds.a:Stopped 2024-09-21T00:29:43.630 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-21T00:29:43.635 INFO:tasks.ceph.mds.c:Stopped 2024-09-21T00:29:43.635 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-21T00:29:43.636 INFO:tasks.ceph.mds.d:Stopped 2024-09-21T00:29:43.636 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-21T00:29:43.636 INFO:tasks.ceph.mds.b:Stopped 2024-09-21T00:29:43.637 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-21T00:29:43.676 INFO:tasks.ceph.mds.a:Started 2024-09-21T00:29:43.677 DEBUG:teuthology.parallel:result is None 2024-09-21T00:29:43.678 INFO:tasks.ceph.mds.c:Started 2024-09-21T00:29:43.678 DEBUG:teuthology.parallel:result is None 2024-09-21T00:29:43.679 INFO:tasks.ceph.mds.d:Started 2024-09-21T00:29:43.679 DEBUG:teuthology.parallel:result is None 2024-09-21T00:29:43.680 INFO:tasks.ceph.mds.b:Started 2024-09-21T00:29:43.680 DEBUG:teuthology.parallel:result is None 2024-09-21T00:29:43.680 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-21T00:29:43.763 INFO:tasks.ceph.mds.a.smithi112.stdout:starting mds.a at 2024-09-21T00:29:43.763 INFO:tasks.ceph.mds.c.smithi112.stdout:starting mds.c at 2024-09-21T00:29:43.774 INFO:tasks.ceph.mds.d.smithi187.stdout:starting mds.d at 2024-09-21T00:29:43.775 INFO:tasks.ceph.mds.b.smithi187.stdout:starting mds.b at 2024-09-21T00:29:43.775 INFO:tasks.ceph.mds.d.smithi187.stderr:2024-09-21T00:29:43.773+0000 7f4fc63cd640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2024-09-21T00:29:43.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:43.799+0000 7f191c60e640 1 -- 172.21.15.112:0/730590222 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1914070210 msgr2=0x7f1914070620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:43.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:43.799+0000 7f191c60e640 1 --2- 172.21.15.112:0/730590222 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1914070210 0x7f1914070620 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f1914159760 tx=0x7f190402a4d0 comp rx=0 tx=0).stop 2024-09-21T00:29:43.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:43.799+0000 7f191c60e640 1 -- 172.21.15.112:0/730590222 shutdown_connections 2024-09-21T00:29:43.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:43.799+0000 7f191c60e640 1 --2- 172.21.15.112:0/730590222 >> v2:172.21.15.187:3300/0 conn(0x7f19140ccbb0 0x7f19140ccfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:43.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:43.799+0000 7f191c60e640 1 --2- 172.21.15.112:0/730590222 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1914070210 0x7f1914070620 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:43.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:43.799+0000 7f191c60e640 1 --2- 172.21.15.112:0/730590222 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f191415c5f0 0x7f1914155900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:43.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:43.799+0000 7f191c60e640 1 -- 172.21.15.112:0/730590222 >> 172.21.15.112:0/730590222 conn(0x7f1914073fe0 msgr2=0x7f19140743f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:43.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:43.799+0000 7f191c60e640 1 -- 172.21.15.112:0/730590222 shutdown_connections 2024-09-21T00:29:43.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:43.799+0000 7f191c60e640 1 -- 172.21.15.112:0/730590222 wait complete. 2024-09-21T00:29:43.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:43.801+0000 7f191c60e640 1 Processor -- start 2024-09-21T00:29:43.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:43.801+0000 7f191c60e640 1 -- start start 2024-09-21T00:29:43.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:43.802+0000 7f191c60e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1914070210 0x7f191414b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:43.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:43.802+0000 7f191c60e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f19140ccbb0 0x7f19141465e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:43.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:43.802+0000 7f191c60e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f191415c5f0 0x7f19141570f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:43.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:43.802+0000 7f191c60e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1914070fd0 con 0x7f1914070210 2024-09-21T00:29:43.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:43.802+0000 7f191c60e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f1914070c40 con 0x7f19140ccbb0 2024-09-21T00:29:43.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:43.802+0000 7f191c60e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f1914071150 con 0x7f191415c5f0 2024-09-21T00:29:43.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:43.802+0000 7f191a3aa640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1914070210 0x7f191414b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:43.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:43.802+0000 7f191a3aa640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1914070210 0x7f191414b070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36544/0 (socket says 172.21.15.112:36544) 2024-09-21T00:29:43.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:43.802+0000 7f191a3aa640 1 -- 172.21.15.112:0/4177761070 learned_addr learned my addr 172.21.15.112:0/4177761070 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:29:43.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:43.802+0000 7f1919ba9640 1 --2- 172.21.15.112:0/4177761070 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f19140ccbb0 0x7f19141465e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:43.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:43.802+0000 7f191abab640 1 --2- 172.21.15.112:0/4177761070 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f191415c5f0 0x7f19141570f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:43.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:43.803+0000 7f191a3aa640 1 -- 172.21.15.112:0/4177761070 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f191415c5f0 msgr2=0x7f19141570f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:43.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:43.803+0000 7f191a3aa640 1 --2- 172.21.15.112:0/4177761070 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f191415c5f0 0x7f19141570f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:43.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:43.803+0000 7f191a3aa640 1 -- 172.21.15.112:0/4177761070 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f19140ccbb0 msgr2=0x7f19141465e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:43.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:43.803+0000 7f191a3aa640 1 --2- 172.21.15.112:0/4177761070 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f19140ccbb0 0x7f19141465e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:43.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:43.803+0000 7f191a3aa640 1 -- 172.21.15.112:0/4177761070 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f191414cb40 con 0x7f1914070210 2024-09-21T00:29:43.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:43.803+0000 7f1919ba9640 1 --2- 172.21.15.112:0/4177761070 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f19140ccbb0 0x7f19141465e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:29:43.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:43.803+0000 7f191a3aa640 1 --2- 172.21.15.112:0/4177761070 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1914070210 0x7f191414b070 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f191000b440 tx=0x7f191000b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:43.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:43.803+0000 7f190affd640 1 -- 172.21.15.112:0/4177761070 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1910015020 con 0x7f1914070210 2024-09-21T00:29:43.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:43.803+0000 7f190affd640 1 -- 172.21.15.112:0/4177761070 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1910011980 con 0x7f1914070210 2024-09-21T00:29:43.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:43.803+0000 7f191c60e640 1 -- 172.21.15.112:0/4177761070 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f191414ce20 con 0x7f1914070210 2024-09-21T00:29:43.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:43.803+0000 7f190affd640 1 -- 172.21.15.112:0/4177761070 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1910011dd0 con 0x7f1914070210 2024-09-21T00:29:43.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:43.803+0000 7f191c60e640 1 -- 172.21.15.112:0/4177761070 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f191414d3e0 con 0x7f1914070210 2024-09-21T00:29:43.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:43.804+0000 7f191c60e640 1 -- 172.21.15.112:0/4177761070 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19140b0730 con 0x7f1914070210 2024-09-21T00:29:43.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:43.805+0000 7f190affd640 1 -- 172.21.15.112:0/4177761070 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1910018020 con 0x7f1914070210 2024-09-21T00:29:43.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:43.805+0000 7f190affd640 1 --2- 172.21.15.112:0/4177761070 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f18f8078bc0 0x7f18f807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:43.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:43.806+0000 7f190affd640 1 -- 172.21.15.112:0/4177761070 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5335+0+0 (secure 0 0 0) 0x7f191009ac10 con 0x7f1914070210 2024-09-21T00:29:43.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:43.806+0000 7f1919ba9640 1 --2- 172.21.15.112:0/4177761070 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f18f8078bc0 0x7f18f807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:43.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:43.806+0000 7f1919ba9640 1 --2- 172.21.15.112:0/4177761070 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f18f8078bc0 0x7f18f807b0d0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f1904033000 tx=0x7f1904004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:43.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:43.810+0000 7f190affd640 1 -- 172.21.15.112:0/4177761070 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f19140b0730 con 0x7f1914070210 2024-09-21T00:29:44.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:44.000+0000 7f191c60e640 1 -- 172.21.15.112:0/4177761070 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f1914156000 con 0x7f1914070210 2024-09-21T00:29:45.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.075+0000 7f190affd640 1 -- 172.21.15.112:0/4177761070 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v104) ==== 97+0+0 (secure 0 0 0) 0x7f191001f020 con 0x7f1914070210 2024-09-21T00:29:45.076 INFO:teuthology.orchestra.run.smithi112.stderr: removed all blocklist entries 2024-09-21T00:29:45.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.078+0000 7f191c60e640 1 -- 172.21.15.112:0/4177761070 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f18f8078bc0 msgr2=0x7f18f807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:45.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.078+0000 7f191c60e640 1 --2- 172.21.15.112:0/4177761070 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f18f8078bc0 0x7f18f807b0d0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f1904033000 tx=0x7f1904004730 comp rx=0 tx=0).stop 2024-09-21T00:29:45.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.078+0000 7f191c60e640 1 -- 172.21.15.112:0/4177761070 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1914070210 msgr2=0x7f191414b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:45.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.078+0000 7f191c60e640 1 --2- 172.21.15.112:0/4177761070 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1914070210 0x7f191414b070 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f191000b440 tx=0x7f191000b900 comp rx=0 tx=0).stop 2024-09-21T00:29:45.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.078+0000 7f191c60e640 1 -- 172.21.15.112:0/4177761070 shutdown_connections 2024-09-21T00:29:45.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.078+0000 7f191c60e640 1 --2- 172.21.15.112:0/4177761070 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f191415c5f0 0x7f19141570f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:45.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.078+0000 7f191c60e640 1 --2- 172.21.15.112:0/4177761070 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f19140ccbb0 0x7f19141465e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:45.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.078+0000 7f191c60e640 1 --2- 172.21.15.112:0/4177761070 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f18f8078bc0 0x7f18f807b0d0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:45.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.078+0000 7f191c60e640 1 --2- 172.21.15.112:0/4177761070 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1914070210 0x7f191414b070 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:45.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.078+0000 7f191c60e640 1 -- 172.21.15.112:0/4177761070 >> 172.21.15.112:0/4177761070 conn(0x7f1914073fe0 msgr2=0x7f191414b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:45.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.078+0000 7f191c60e640 1 -- 172.21.15.112:0/4177761070 shutdown_connections 2024-09-21T00:29:45.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.079+0000 7f191c60e640 1 -- 172.21.15.112:0/4177761070 wait complete. 2024-09-21T00:29:45.093 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-21T00:29:45.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.209+0000 7f867c962640 1 -- 172.21.15.112:0/3617641898 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8664003b30 msgr2=0x7f8664003fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:45.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.209+0000 7f867c962640 1 --2- 172.21.15.112:0/3617641898 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8664003b30 0x7f8664003fa0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f86680016c0 tx=0x7f8664005830 comp rx=0 tx=0).stop 2024-09-21T00:29:45.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.209+0000 7f867c962640 1 -- 172.21.15.112:0/3617641898 shutdown_connections 2024-09-21T00:29:45.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.209+0000 7f867c962640 1 --2- 172.21.15.112:0/3617641898 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8664003b30 0x7f8664003fa0 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:45.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.209+0000 7f867c962640 1 -- 172.21.15.112:0/3617641898 >> 172.21.15.112:0/3617641898 conn(0x7f86780b43f0 msgr2=0x7f86780b4830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:45.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.209+0000 7f867c962640 1 -- 172.21.15.112:0/3617641898 shutdown_connections 2024-09-21T00:29:45.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.209+0000 7f867c962640 1 -- 172.21.15.112:0/3617641898 wait complete. 2024-09-21T00:29:45.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.211+0000 7f867c962640 1 Processor -- start 2024-09-21T00:29:45.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.211+0000 7f867c962640 1 -- start start 2024-09-21T00:29:45.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.212+0000 7f867c962640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8664003b30 0x7f86780c8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:45.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.212+0000 7f867c962640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f86780c6cc0 0x7f86780c70d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:45.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.212+0000 7f867c962640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8678154970 0x7f8678154d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:45.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.212+0000 7f867c962640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f86780cd280 con 0x7f8664003b30 2024-09-21T00:29:45.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.212+0000 7f867c962640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f86780cd100 con 0x7f8678154970 2024-09-21T00:29:45.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.212+0000 7f867c962640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f86780cd400 con 0x7f86780c6cc0 2024-09-21T00:29:45.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.212+0000 7f867659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8664003b30 0x7f86780c8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:45.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.212+0000 7f867659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8664003b30 0x7f86780c8c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36558/0 (socket says 172.21.15.112:36558) 2024-09-21T00:29:45.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.212+0000 7f867659c640 1 -- 172.21.15.112:0/3611720830 learned_addr learned my addr 172.21.15.112:0/3611720830 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:29:45.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.213+0000 7f8676d9d640 1 --2- 172.21.15.112:0/3611720830 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8678154970 0x7f8678154d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:45.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.213+0000 7f866dd9b640 1 --2- 172.21.15.112:0/3611720830 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f86780c6cc0 0x7f86780c70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:45.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.213+0000 7f867659c640 1 -- 172.21.15.112:0/3611720830 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f86780c6cc0 msgr2=0x7f86780c70d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:45.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.213+0000 7f867659c640 1 --2- 172.21.15.112:0/3611720830 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f86780c6cc0 0x7f86780c70d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:45.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.213+0000 7f867659c640 1 -- 172.21.15.112:0/3611720830 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8678154970 msgr2=0x7f8678154d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:45.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.213+0000 7f867659c640 1 --2- 172.21.15.112:0/3611720830 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8678154970 0x7f8678154d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:45.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.213+0000 7f867659c640 1 -- 172.21.15.112:0/3611720830 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86781552c0 con 0x7f8664003b30 2024-09-21T00:29:45.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.213+0000 7f866dd9b640 1 --2- 172.21.15.112:0/3611720830 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f86780c6cc0 0x7f86780c70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:29:45.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.213+0000 7f867659c640 1 --2- 172.21.15.112:0/3611720830 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8664003b30 0x7f86780c8c00 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f86680017e0 tx=0x7f86680063a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:45.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.213+0000 7f866f7fe640 1 -- 172.21.15.112:0/3611720830 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8668042070 con 0x7f8664003b30 2024-09-21T00:29:45.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.213+0000 7f866f7fe640 1 -- 172.21.15.112:0/3611720830 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8668039070 con 0x7f8664003b30 2024-09-21T00:29:45.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.213+0000 7f867c962640 1 -- 172.21.15.112:0/3611720830 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f867806f650 con 0x7f8664003b30 2024-09-21T00:29:45.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.213+0000 7f866f7fe640 1 -- 172.21.15.112:0/3611720830 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8668003d90 con 0x7f8664003b30 2024-09-21T00:29:45.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.213+0000 7f867c962640 1 -- 172.21.15.112:0/3611720830 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f867806fc10 con 0x7f8664003b30 2024-09-21T00:29:45.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.214+0000 7f867c962640 1 -- 172.21.15.112:0/3611720830 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f867815a340 con 0x7f8664003b30 2024-09-21T00:29:45.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.215+0000 7f866f7fe640 1 -- 172.21.15.112:0/3611720830 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f866803b070 con 0x7f8664003b30 2024-09-21T00:29:45.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.215+0000 7f866f7fe640 1 --2- 172.21.15.112:0/3611720830 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f8658078b90 0x7f865807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:45.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.215+0000 7f866f7fe640 1 -- 172.21.15.112:0/3611720830 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5249+0+0 (secure 0 0 0) 0x7f86680ba090 con 0x7f8664003b30 2024-09-21T00:29:45.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.216+0000 7f866dd9b640 1 --2- 172.21.15.112:0/3611720830 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f8658078b90 0x7f865807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:45.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.216+0000 7f866dd9b640 1 --2- 172.21.15.112:0/3611720830 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f8658078b90 0x7f865807b0a0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f86780cd8c0 tx=0x7f8660005f90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:45.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.220+0000 7f866f7fe640 1 -- 172.21.15.112:0/3611720830 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f86781552c0 con 0x7f8664003b30 2024-09-21T00:29:45.420 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T00:29:45.422 DEBUG:teuthology.orchestra.run.smithi187:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T00:29:45.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.464+0000 7f867c962640 1 -- 172.21.15.112:0/3611720830 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f86780cbf40 con 0x7f8664003b30 2024-09-21T00:29:45.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.465+0000 7f866f7fe640 1 -- 172.21.15.112:0/3611720830 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v20) ==== 80+0+5304 (secure 0 0 0) 0x7f86680827b0 con 0x7f8664003b30 2024-09-21T00:29:45.466 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:29:45.466 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-21T00:29:45.466 INFO:teuthology.orchestra.run.smithi112.stdout: "auth_dump": [ 2024-09-21T00:29:45.466 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:29:45.466 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "mds.a", 2024-09-21T00:29:45.466 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBPEu5m3capAhAAczRM4L8ldX985NW4ep3w5Q==", 2024-09-21T00:29:45.466 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:29:45.466 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2024-09-21T00:29:45.466 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow *", 2024-09-21T00:29:45.466 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow *", 2024-09-21T00:29:45.467 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2024-09-21T00:29:45.467 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:29:45.467 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:29:45.467 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:29:45.467 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "mds.b", 2024-09-21T00:29:45.467 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBPEu5mxlDlDxAAtqngbTOhPqXDjFhJpSCLHg==", 2024-09-21T00:29:45.467 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:29:45.467 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2024-09-21T00:29:45.467 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow *", 2024-09-21T00:29:45.467 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow *", 2024-09-21T00:29:45.467 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2024-09-21T00:29:45.467 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:29:45.467 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:29:45.467 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:29:45.468 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "mds.c", 2024-09-21T00:29:45.468 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBPEu5mXNVRChAA1qsPbEybmY16jw5Jwk/v1A==", 2024-09-21T00:29:45.468 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:29:45.468 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2024-09-21T00:29:45.468 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow *", 2024-09-21T00:29:45.468 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow *", 2024-09-21T00:29:45.468 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2024-09-21T00:29:45.468 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:29:45.468 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:29:45.468 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:29:45.468 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "mds.d", 2024-09-21T00:29:45.468 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBPEu5mnzySFxAAX3VfTgGYGcDweI7rPf3xvw==", 2024-09-21T00:29:45.468 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:29:45.468 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2024-09-21T00:29:45.468 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow *", 2024-09-21T00:29:45.469 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow *", 2024-09-21T00:29:45.469 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2024-09-21T00:29:45.469 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:29:45.469 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:29:45.469 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:29:45.469 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.0", 2024-09-21T00:29:45.469 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBSEu5moQg4OBAAi8mzLx7tWnORlrL64/MWlw==", 2024-09-21T00:29:45.469 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:29:45.469 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2024-09-21T00:29:45.469 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2024-09-21T00:29:45.469 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2024-09-21T00:29:45.469 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:29:45.469 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:29:45.469 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:29:45.470 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.1", 2024-09-21T00:29:45.470 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBWEu5mxfobDhAAi7I6LYq9udB0W8bnYaBY9Q==", 2024-09-21T00:29:45.470 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:29:45.470 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2024-09-21T00:29:45.470 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2024-09-21T00:29:45.470 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2024-09-21T00:29:45.470 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:29:45.470 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:29:45.470 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:29:45.470 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.2", 2024-09-21T00:29:45.470 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBZEu5mDqC/EhAAwzTbj22ocR4xJ8DMJvN7Lg==", 2024-09-21T00:29:45.470 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:29:45.470 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2024-09-21T00:29:45.470 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2024-09-21T00:29:45.471 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2024-09-21T00:29:45.471 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:29:45.471 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:29:45.471 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:29:45.471 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.3", 2024-09-21T00:29:45.471 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBcEu5mwweGJBAAT3hZn5pto0aWt6BwGuaqcw==", 2024-09-21T00:29:45.471 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:29:45.471 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2024-09-21T00:29:45.471 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2024-09-21T00:29:45.471 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2024-09-21T00:29:45.471 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:29:45.471 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:29:45.471 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:29:45.471 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.4", 2024-09-21T00:29:45.472 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBjEu5mIPd3ARAAkxTYphn38uKh48UUVEB+Fg==", 2024-09-21T00:29:45.472 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:29:45.472 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2024-09-21T00:29:45.472 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2024-09-21T00:29:45.472 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2024-09-21T00:29:45.472 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:29:45.472 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:29:45.472 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:29:45.472 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.5", 2024-09-21T00:29:45.472 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBmEu5mP9IVBBAAftfK5V1K4vYRyFyJS2UzKQ==", 2024-09-21T00:29:45.472 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:29:45.472 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2024-09-21T00:29:45.472 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2024-09-21T00:29:45.472 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2024-09-21T00:29:45.472 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:29:45.472 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:29:45.473 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:29:45.473 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.6", 2024-09-21T00:29:45.473 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBpEu5mmjiyFRAAbiW2M8JE0jJOaNg1u+1kgw==", 2024-09-21T00:29:45.473 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:29:45.473 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2024-09-21T00:29:45.473 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2024-09-21T00:29:45.473 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2024-09-21T00:29:45.473 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:29:45.473 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:29:45.473 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:29:45.473 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.7", 2024-09-21T00:29:45.473 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBsEu5moklPKBAA6hzVvgvpmZMkPaFFce4mqw==", 2024-09-21T00:29:45.473 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:29:45.473 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2024-09-21T00:29:45.473 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2024-09-21T00:29:45.474 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2024-09-21T00:29:45.474 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:29:45.474 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:29:45.474 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:29:45.474 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "client.0", 2024-09-21T00:29:45.474 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBPEu5m49NPHBAAl/rcjiE8xYI+XdWwvZ6FOw==", 2024-09-21T00:29:45.474 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:29:45.474 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2024-09-21T00:29:45.474 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow r", 2024-09-21T00:29:45.474 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-21T00:29:45.474 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:29:45.474 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:29:45.474 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:29:45.474 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "client.1", 2024-09-21T00:29:45.475 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBPEu5mG33aIxAAirQhTlVyF/pRJx93T/0bIQ==", 2024-09-21T00:29:45.475 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:29:45.475 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2024-09-21T00:29:45.475 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow r", 2024-09-21T00:29:45.475 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow rw", 2024-09-21T00:29:45.475 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow rwx" 2024-09-21T00:29:45.475 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:29:45.475 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:29:45.475 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:29:45.475 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "client.2", 2024-09-21T00:29:45.475 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBPEu5mreUUIBAAELeN1N58srw6GqBrj8UOoQ==", 2024-09-21T00:29:45.475 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:29:45.475 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2024-09-21T00:29:45.475 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow r", 2024-09-21T00:29:45.475 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow rw", 2024-09-21T00:29:45.476 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow rwx" 2024-09-21T00:29:45.476 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:29:45.476 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:29:45.476 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:29:45.476 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "client.3", 2024-09-21T00:29:45.476 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBPEu5mcTmiJxAAY6bmTtow0R0heJHjx0ahDQ==", 2024-09-21T00:29:45.476 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:29:45.476 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2024-09-21T00:29:45.476 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow r", 2024-09-21T00:29:45.476 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow rw", 2024-09-21T00:29:45.476 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow rwx" 2024-09-21T00:29:45.476 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:29:45.476 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:29:45.476 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:29:45.476 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "client.admin", 2024-09-21T00:29:45.477 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBOEu5mTxhNIBAAlStXGsGd/okXhakY6WI/4w==", 2024-09-21T00:29:45.477 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:29:45.477 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow *", 2024-09-21T00:29:45.477 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow *", 2024-09-21T00:29:45.477 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow *", 2024-09-21T00:29:45.477 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2024-09-21T00:29:45.477 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:29:45.477 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:29:45.477 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:29:45.477 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "mgr.x", 2024-09-21T00:29:45.477 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBOEu5mjGpDOhAAJFudb2wZlOBjo8Ss65Q+eg==", 2024-09-21T00:29:45.477 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:29:45.477 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow *", 2024-09-21T00:29:45.477 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile mgr", 2024-09-21T00:29:45.477 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2024-09-21T00:29:45.477 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:29:45.478 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:29:45.478 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:29:45.478 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "mgr.y", 2024-09-21T00:29:45.478 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQBOEu5myGjKNRAA9cMDObZYYOXMGdv2mOXkJA==", 2024-09-21T00:29:45.478 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-09-21T00:29:45.478 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow *", 2024-09-21T00:29:45.478 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile mgr", 2024-09-21T00:29:45.478 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2024-09-21T00:29:45.478 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:29:45.478 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:29:45.478 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-09-21T00:29:45.478 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-21T00:29:45.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.467+0000 7f867c962640 1 -- 172.21.15.112:0/3611720830 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f8658078b90 msgr2=0x7f865807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:45.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.467+0000 7f867c962640 1 --2- 172.21.15.112:0/3611720830 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f8658078b90 0x7f865807b0a0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f86780cd8c0 tx=0x7f8660005f90 comp rx=0 tx=0).stop 2024-09-21T00:29:45.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.467+0000 7f867c962640 1 -- 172.21.15.112:0/3611720830 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8664003b30 msgr2=0x7f86780c8c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:45.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.467+0000 7f867c962640 1 --2- 172.21.15.112:0/3611720830 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8664003b30 0x7f86780c8c00 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f86680017e0 tx=0x7f86680063a0 comp rx=0 tx=0).stop 2024-09-21T00:29:45.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.467+0000 7f867c962640 1 -- 172.21.15.112:0/3611720830 shutdown_connections 2024-09-21T00:29:45.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.467+0000 7f867c962640 1 --2- 172.21.15.112:0/3611720830 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8678154970 0x7f8678154d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:45.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.467+0000 7f867c962640 1 --2- 172.21.15.112:0/3611720830 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f86780c6cc0 0x7f86780c70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:45.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.467+0000 7f867c962640 1 --2- 172.21.15.112:0/3611720830 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f8658078b90 0x7f865807b0a0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:45.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.467+0000 7f867c962640 1 --2- 172.21.15.112:0/3611720830 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8664003b30 0x7f86780c8c00 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:45.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.467+0000 7f867c962640 1 -- 172.21.15.112:0/3611720830 >> 172.21.15.112:0/3611720830 conn(0x7f86780b43f0 msgr2=0x7f867815b690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:45.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.467+0000 7f867c962640 1 -- 172.21.15.112:0/3611720830 shutdown_connections 2024-09-21T00:29:45.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.467+0000 7f867c962640 1 -- 172.21.15.112:0/3611720830 wait complete. 2024-09-21T00:29:45.481 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-21T00:29:45.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.593+0000 7ffadc9f1640 1 -- 172.21.15.112:0/3061915815 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffad80d4a90 msgr2=0x7ffad8070250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:45.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.593+0000 7ffadc9f1640 1 --2- 172.21.15.112:0/3061915815 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffad80d4a90 0x7ffad8070250 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7ffad80d5560 tx=0x7ffad002a820 comp rx=0 tx=0).stop 2024-09-21T00:29:45.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.593+0000 7ffadc9f1640 1 -- 172.21.15.112:0/3061915815 shutdown_connections 2024-09-21T00:29:45.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.593+0000 7ffadc9f1640 1 --2- 172.21.15.112:0/3061915815 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffad80d4a90 0x7ffad8070250 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:45.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.593+0000 7ffadc9f1640 1 --2- 172.21.15.112:0/3061915815 >> v2:172.21.15.187:3300/0 conn(0x7ffad80d0ae0 0x7ffad80d38c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:45.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.593+0000 7ffadc9f1640 1 -- 172.21.15.112:0/3061915815 >> 172.21.15.112:0/3061915815 conn(0x7ffad809d8a0 msgr2=0x7ffad808bfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:45.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.593+0000 7ffadc9f1640 1 -- 172.21.15.112:0/3061915815 shutdown_connections 2024-09-21T00:29:45.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.593+0000 7ffadc9f1640 1 -- 172.21.15.112:0/3061915815 wait complete. 2024-09-21T00:29:45.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.595+0000 7ffadc9f1640 1 Processor -- start 2024-09-21T00:29:45.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.595+0000 7ffadc9f1640 1 -- start start 2024-09-21T00:29:45.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.596+0000 7ffadc9f1640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffad80d0ae0 0x7ffad806d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:45.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.596+0000 7ffadc9f1640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffad80d4a90 0x7ffad815a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:45.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.596+0000 7ffadc9f1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffad815ea40 0x7ffad80ce180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:45.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.596+0000 7ffadc9f1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ffad8162ae0 con 0x7ffad815ea40 2024-09-21T00:29:45.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.596+0000 7ffadc9f1640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ffad8162960 con 0x7ffad80d4a90 2024-09-21T00:29:45.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.596+0000 7ffadc9f1640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ffad8162c60 con 0x7ffad80d0ae0 2024-09-21T00:29:45.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.596+0000 7ffad6d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffad815ea40 0x7ffad80ce180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:45.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.596+0000 7ffad6d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffad815ea40 0x7ffad80ce180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36582/0 (socket says 172.21.15.112:36582) 2024-09-21T00:29:45.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.596+0000 7ffad6d9d640 1 -- 172.21.15.112:0/3469225928 learned_addr learned my addr 172.21.15.112:0/3469225928 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:29:45.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.596+0000 7ffad659c640 1 --2- 172.21.15.112:0/3469225928 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffad80d0ae0 0x7ffad806d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:45.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.596+0000 7ffad5d9b640 1 --2- 172.21.15.112:0/3469225928 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffad80d4a90 0x7ffad815a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:45.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.596+0000 7ffad6d9d640 1 -- 172.21.15.112:0/3469225928 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffad80d0ae0 msgr2=0x7ffad806d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:45.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.596+0000 7ffad6d9d640 1 --2- 172.21.15.112:0/3469225928 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffad80d0ae0 0x7ffad806d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:45.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.596+0000 7ffad6d9d640 1 -- 172.21.15.112:0/3469225928 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffad80d4a90 msgr2=0x7ffad815a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:45.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.596+0000 7ffad6d9d640 1 --2- 172.21.15.112:0/3469225928 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffad80d4a90 0x7ffad815a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:45.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.596+0000 7ffad6d9d640 1 -- 172.21.15.112:0/3469225928 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffad806f520 con 0x7ffad815ea40 2024-09-21T00:29:45.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.596+0000 7ffad5d9b640 1 --2- 172.21.15.112:0/3469225928 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffad80d4a90 0x7ffad815a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:29:45.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.597+0000 7ffad6d9d640 1 --2- 172.21.15.112:0/3469225928 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffad815ea40 0x7ffad80ce180 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7ffad0009b30 tx=0x7ffad0032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:45.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.597+0000 7ffac6ffd640 1 -- 172.21.15.112:0/3469225928 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffad0037040 con 0x7ffad815ea40 2024-09-21T00:29:45.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.597+0000 7ffac6ffd640 1 -- 172.21.15.112:0/3469225928 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffad0042070 con 0x7ffad815ea40 2024-09-21T00:29:45.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.597+0000 7ffadc9f1640 1 -- 172.21.15.112:0/3469225928 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffad806f800 con 0x7ffad815ea40 2024-09-21T00:29:45.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.597+0000 7ffac6ffd640 1 -- 172.21.15.112:0/3469225928 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffad00042a0 con 0x7ffad815ea40 2024-09-21T00:29:45.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.597+0000 7ffadc9f1640 1 -- 172.21.15.112:0/3469225928 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffad806fdc0 con 0x7ffad815ea40 2024-09-21T00:29:45.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.598+0000 7ffadc9f1640 1 -- 172.21.15.112:0/3469225928 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffad806de40 con 0x7ffad815ea40 2024-09-21T00:29:45.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.598+0000 7ffac6ffd640 1 -- 172.21.15.112:0/3469225928 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ffad0004440 con 0x7ffad815ea40 2024-09-21T00:29:45.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.599+0000 7ffac6ffd640 1 --2- 172.21.15.112:0/3469225928 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ffab8078bc0 0x7ffab807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:45.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.599+0000 7ffac6ffd640 1 -- 172.21.15.112:0/3469225928 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5249+0+0 (secure 0 0 0) 0x7ffad00bb920 con 0x7ffad815ea40 2024-09-21T00:29:45.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.599+0000 7ffad659c640 1 --2- 172.21.15.112:0/3469225928 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ffab8078bc0 0x7ffab807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:45.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.600+0000 7ffad659c640 1 --2- 172.21.15.112:0/3469225928 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ffab8078bc0 0x7ffab807b0d0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7ffac800bb30 tx=0x7ffac800b2e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:45.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.604+0000 7ffac6ffd640 1 -- 172.21.15.112:0/3469225928 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ffad0083fb0 con 0x7ffad815ea40 2024-09-21T00:29:45.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.782+0000 7ffadc9f1640 1 -- 172.21.15.112:0/3469225928 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7ffad806ffc0 con 0x7ffad815ea40 2024-09-21T00:29:45.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.782+0000 7ffac6ffd640 1 -- 172.21.15.112:0/3469225928 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v104) ==== 84+0+65 (secure 0 0 0) 0x7ffad806ffc0 con 0x7ffad815ea40 2024-09-21T00:29:45.783 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:29:45.783 INFO:teuthology.orchestra.run.smithi112.stdout:[ 2024-09-21T00:29:45.783 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:29:45.783 INFO:teuthology.orchestra.run.smithi112.stdout: "poolnum": 1, 2024-09-21T00:29:45.783 INFO:teuthology.orchestra.run.smithi112.stdout: "poolname": ".mgr" 2024-09-21T00:29:45.783 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:29:45.784 INFO:teuthology.orchestra.run.smithi112.stdout:] 2024-09-21T00:29:45.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.784+0000 7ffadc9f1640 1 -- 172.21.15.112:0/3469225928 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ffab8078bc0 msgr2=0x7ffab807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:45.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.784+0000 7ffadc9f1640 1 --2- 172.21.15.112:0/3469225928 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ffab8078bc0 0x7ffab807b0d0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7ffac800bb30 tx=0x7ffac800b2e0 comp rx=0 tx=0).stop 2024-09-21T00:29:45.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.784+0000 7ffadc9f1640 1 -- 172.21.15.112:0/3469225928 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffad815ea40 msgr2=0x7ffad80ce180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:45.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.784+0000 7ffadc9f1640 1 --2- 172.21.15.112:0/3469225928 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffad815ea40 0x7ffad80ce180 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7ffad0009b30 tx=0x7ffad0032d10 comp rx=0 tx=0).stop 2024-09-21T00:29:45.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.784+0000 7ffadc9f1640 1 -- 172.21.15.112:0/3469225928 shutdown_connections 2024-09-21T00:29:45.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.784+0000 7ffadc9f1640 1 --2- 172.21.15.112:0/3469225928 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ffab8078bc0 0x7ffab807b0d0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:45.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.784+0000 7ffadc9f1640 1 --2- 172.21.15.112:0/3469225928 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffad815ea40 0x7ffad80ce180 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:45.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.784+0000 7ffadc9f1640 1 --2- 172.21.15.112:0/3469225928 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffad80d4a90 0x7ffad815a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:45.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.784+0000 7ffadc9f1640 1 --2- 172.21.15.112:0/3469225928 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffad80d0ae0 0x7ffad806d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:45.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.784+0000 7ffadc9f1640 1 -- 172.21.15.112:0/3469225928 >> 172.21.15.112:0/3469225928 conn(0x7ffad809d8a0 msgr2=0x7ffad815f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:45.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.784+0000 7ffadc9f1640 1 -- 172.21.15.112:0/3469225928 shutdown_connections 2024-09-21T00:29:45.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.784+0000 7ffadc9f1640 1 -- 172.21.15.112:0/3469225928 wait complete. 2024-09-21T00:29:45.796 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-21T00:29:45.797 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-21T00:29:45.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.912+0000 7fb2eadeb640 1 -- 172.21.15.112:0/3766903109 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2e4070bc0 msgr2=0x7fb2e413f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:45.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.912+0000 7fb2eadeb640 1 --2- 172.21.15.112:0/3766903109 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2e4070bc0 0x7fb2e413f5d0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fb2d4009830 tx=0x7fb2d402a480 comp rx=0 tx=0).stop 2024-09-21T00:29:45.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.912+0000 7fb2eadeb640 1 -- 172.21.15.112:0/3766903109 shutdown_connections 2024-09-21T00:29:45.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.912+0000 7fb2eadeb640 1 --2- 172.21.15.112:0/3766903109 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2e4070bc0 0x7fb2e413f5d0 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:45.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.912+0000 7fb2eadeb640 1 -- 172.21.15.112:0/3766903109 >> 172.21.15.112:0/3766903109 conn(0x7fb2e40d12d0 msgr2=0x7fb2e40d16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:45.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.912+0000 7fb2eadeb640 1 -- 172.21.15.112:0/3766903109 shutdown_connections 2024-09-21T00:29:45.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.912+0000 7fb2eadeb640 1 -- 172.21.15.112:0/3766903109 wait complete. 2024-09-21T00:29:45.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.914+0000 7fb2eadeb640 1 Processor -- start 2024-09-21T00:29:45.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.914+0000 7fb2eadeb640 1 -- start start 2024-09-21T00:29:45.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.916+0000 7fb2eadeb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2e4070bc0 0x7fb2e40aff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:45.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.916+0000 7fb2eadeb640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb2e40d07a0 0x7fb2e41f5c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:45.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.916+0000 7fb2eadeb640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb2e4141f30 0x7fb2e41588a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:45.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.916+0000 7fb2eadeb640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e406dc50 con 0x7fb2e4070bc0 2024-09-21T00:29:45.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.916+0000 7fb2eadeb640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e406dad0 con 0x7fb2e4141f30 2024-09-21T00:29:45.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.916+0000 7fb2eadeb640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fb2e406ddd0 con 0x7fb2e40d07a0 2024-09-21T00:29:45.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.916+0000 7fb2e8b87640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2e4070bc0 0x7fb2e40aff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:45.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.916+0000 7fb2e8b87640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2e4070bc0 0x7fb2e40aff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36606/0 (socket says 172.21.15.112:36606) 2024-09-21T00:29:45.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.916+0000 7fb2e8b87640 1 -- 172.21.15.112:0/2858292320 learned_addr learned my addr 172.21.15.112:0/2858292320 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:29:45.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.916+0000 7fb2dbfff640 1 --2- 172.21.15.112:0/2858292320 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb2e40d07a0 0x7fb2e41f5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:45.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.916+0000 7fb2e9388640 1 --2- 172.21.15.112:0/2858292320 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb2e4141f30 0x7fb2e41588a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:45.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.916+0000 7fb2e8b87640 1 -- 172.21.15.112:0/2858292320 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb2e40d07a0 msgr2=0x7fb2e41f5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:45.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.916+0000 7fb2e8b87640 1 --2- 172.21.15.112:0/2858292320 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb2e40d07a0 0x7fb2e41f5c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:45.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.916+0000 7fb2e8b87640 1 -- 172.21.15.112:0/2858292320 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb2e4141f30 msgr2=0x7fb2e41588a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:45.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.916+0000 7fb2e8b87640 1 --2- 172.21.15.112:0/2858292320 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb2e4141f30 0x7fb2e41588a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:45.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.916+0000 7fb2e8b87640 1 -- 172.21.15.112:0/2858292320 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2e41f3840 con 0x7fb2e4070bc0 2024-09-21T00:29:45.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.916+0000 7fb2e9388640 1 --2- 172.21.15.112:0/2858292320 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb2e4141f30 0x7fb2e41588a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:29:45.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.916+0000 7fb2e8b87640 1 --2- 172.21.15.112:0/2858292320 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2e4070bc0 0x7fb2e40aff60 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7fb2e000c3f0 tx=0x7fb2e000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:45.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.916+0000 7fb2d97fa640 1 -- 172.21.15.112:0/2858292320 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e0017070 con 0x7fb2e4070bc0 2024-09-21T00:29:45.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.916+0000 7fb2d97fa640 1 -- 172.21.15.112:0/2858292320 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb2e00035f0 con 0x7fb2e4070bc0 2024-09-21T00:29:45.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.916+0000 7fb2eadeb640 1 -- 172.21.15.112:0/2858292320 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2e41f3b20 con 0x7fb2e4070bc0 2024-09-21T00:29:45.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.916+0000 7fb2d97fa640 1 -- 172.21.15.112:0/2858292320 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e0003a40 con 0x7fb2e4070bc0 2024-09-21T00:29:45.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.916+0000 7fb2eadeb640 1 -- 172.21.15.112:0/2858292320 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2e40a95e0 con 0x7fb2e4070bc0 2024-09-21T00:29:45.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.917+0000 7fb2eadeb640 1 -- 172.21.15.112:0/2858292320 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2e413f5d0 con 0x7fb2e4070bc0 2024-09-21T00:29:45.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.918+0000 7fb2d97fa640 1 -- 172.21.15.112:0/2858292320 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb2e001a070 con 0x7fb2e4070bc0 2024-09-21T00:29:45.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.919+0000 7fb2d97fa640 1 --2- 172.21.15.112:0/2858292320 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fb2d0078b90 0x7fb2d007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:45.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.919+0000 7fb2d97fa640 1 -- 172.21.15.112:0/2858292320 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5249+0+0 (secure 0 0 0) 0x7fb2e0099e20 con 0x7fb2e4070bc0 2024-09-21T00:29:45.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.919+0000 7fb2dbfff640 1 --2- 172.21.15.112:0/2858292320 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fb2d0078b90 0x7fb2d007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:45.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.919+0000 7fb2dbfff640 1 --2- 172.21.15.112:0/2858292320 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fb2d0078b90 0x7fb2d007b0a0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fb2d4034000 tx=0x7fb2d4004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:45.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:45.923+0000 7fb2d97fa640 1 -- 172.21.15.112:0/2858292320 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb2e00624b0 con 0x7fb2e4070bc0 2024-09-21T00:29:46.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:46.099+0000 7fb2eadeb640 1 -- 172.21.15.112:0/2858292320 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fb2e41424e0 con 0x7fb2e4070bc0 2024-09-21T00:29:47.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:47.079+0000 7fb2d97fa640 1 -- 172.21.15.112:0/2858292320 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v105) ==== 138+0+0 (secure 0 0 0) 0x7fb2e0061db0 con 0x7fb2e4070bc0 2024-09-21T00:29:47.080 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'cephfs_metadata' created 2024-09-21T00:29:47.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:47.082+0000 7fb2eadeb640 1 -- 172.21.15.112:0/2858292320 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fb2d0078b90 msgr2=0x7fb2d007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:47.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:47.082+0000 7fb2eadeb640 1 --2- 172.21.15.112:0/2858292320 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fb2d0078b90 0x7fb2d007b0a0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fb2d4034000 tx=0x7fb2d4004730 comp rx=0 tx=0).stop 2024-09-21T00:29:47.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:47.082+0000 7fb2eadeb640 1 -- 172.21.15.112:0/2858292320 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2e4070bc0 msgr2=0x7fb2e40aff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:47.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:47.082+0000 7fb2eadeb640 1 --2- 172.21.15.112:0/2858292320 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2e4070bc0 0x7fb2e40aff60 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7fb2e000c3f0 tx=0x7fb2e000c8b0 comp rx=0 tx=0).stop 2024-09-21T00:29:47.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:47.083+0000 7fb2eadeb640 1 -- 172.21.15.112:0/2858292320 shutdown_connections 2024-09-21T00:29:47.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:47.083+0000 7fb2eadeb640 1 --2- 172.21.15.112:0/2858292320 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb2e4141f30 0x7fb2e41588a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:47.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:47.083+0000 7fb2eadeb640 1 --2- 172.21.15.112:0/2858292320 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb2e40d07a0 0x7fb2e41f5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:47.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:47.083+0000 7fb2eadeb640 1 --2- 172.21.15.112:0/2858292320 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fb2d0078b90 0x7fb2d007b0a0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:47.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:47.083+0000 7fb2eadeb640 1 --2- 172.21.15.112:0/2858292320 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2e4070bc0 0x7fb2e40aff60 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:47.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:47.083+0000 7fb2eadeb640 1 -- 172.21.15.112:0/2858292320 >> 172.21.15.112:0/2858292320 conn(0x7fb2e40d12d0 msgr2=0x7fb2e4141070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:47.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:47.084+0000 7fb2eadeb640 1 -- 172.21.15.112:0/2858292320 shutdown_connections 2024-09-21T00:29:47.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:47.084+0000 7fb2eadeb640 1 -- 172.21.15.112:0/2858292320 wait complete. 2024-09-21T00:29:47.101 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-21T00:29:47.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:47.226+0000 7f58c7807640 1 -- 172.21.15.112:0/2308314756 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f58c00ccae0 msgr2=0x7f58c00ccef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:47.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:47.226+0000 7f58c7807640 1 --2- 172.21.15.112:0/2308314756 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f58c00ccae0 0x7f58c00ccef0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f58b8005fd0 tx=0x7f58b802a8b0 comp rx=0 tx=0).stop 2024-09-21T00:29:47.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:47.226+0000 7f58c7807640 1 -- 172.21.15.112:0/2308314756 shutdown_connections 2024-09-21T00:29:47.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:47.226+0000 7f58c7807640 1 --2- 172.21.15.112:0/2308314756 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f58c00ccae0 0x7f58c00ccef0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:47.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:47.226+0000 7f58c7807640 1 --2- 172.21.15.112:0/2308314756 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f58c015d870 0x7f58c00d0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:47.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:47.226+0000 7f58c7807640 1 -- 172.21.15.112:0/2308314756 >> 172.21.15.112:0/2308314756 conn(0x7f58c00b43f0 msgr2=0x7f58c00b4830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:47.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:47.226+0000 7f58c7807640 1 -- 172.21.15.112:0/2308314756 shutdown_connections 2024-09-21T00:29:47.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:47.226+0000 7f58c7807640 1 -- 172.21.15.112:0/2308314756 wait complete. 2024-09-21T00:29:47.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:47.228+0000 7f58c7807640 1 Processor -- start 2024-09-21T00:29:47.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:47.228+0000 7f58c7807640 1 -- start start 2024-09-21T00:29:47.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:47.229+0000 7f58c7807640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f58c00ccae0 0x7f58c00c8de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:47.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:47.230+0000 7f58c7807640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f58c00d2030 0x7f58c01f3450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:47.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:47.230+0000 7f58c7807640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f58c015d870 0x7f58c00c7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:47.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:47.230+0000 7f58c7807640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f58c00b22d0 con 0x7f58c00d2030 2024-09-21T00:29:47.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:47.230+0000 7f58c7807640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f58c00b2150 con 0x7f58c015d870 2024-09-21T00:29:47.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:47.230+0000 7f58c7807640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f58c00b2450 con 0x7f58c00ccae0 2024-09-21T00:29:47.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:47.230+0000 7f58c4da2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f58c00d2030 0x7f58c01f3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:47.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:47.230+0000 7f58c4da2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f58c00d2030 0x7f58c01f3450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60928/0 (socket says 172.21.15.112:60928) 2024-09-21T00:29:47.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:47.230+0000 7f58c4da2640 1 -- 172.21.15.112:0/1290314857 learned_addr learned my addr 172.21.15.112:0/1290314857 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:29:47.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:47.230+0000 7f58c5da4640 1 --2- 172.21.15.112:0/1290314857 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f58c015d870 0x7f58c00c7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:47.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:47.230+0000 7f58c55a3640 1 --2- 172.21.15.112:0/1290314857 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f58c00ccae0 0x7f58c00c8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:47.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:47.230+0000 7f58c4da2640 1 -- 172.21.15.112:0/1290314857 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f58c00ccae0 msgr2=0x7f58c00c8de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:47.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:47.230+0000 7f58c4da2640 1 --2- 172.21.15.112:0/1290314857 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f58c00ccae0 0x7f58c00c8de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:47.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:47.230+0000 7f58c4da2640 1 -- 172.21.15.112:0/1290314857 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f58c015d870 msgr2=0x7f58c00c7ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:47.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:47.230+0000 7f58c4da2640 1 --2- 172.21.15.112:0/1290314857 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f58c015d870 0x7f58c00c7ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:47.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:47.230+0000 7f58c4da2640 1 -- 172.21.15.112:0/1290314857 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58c0147f60 con 0x7f58c00d2030 2024-09-21T00:29:47.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:47.230+0000 7f58c4da2640 1 --2- 172.21.15.112:0/1290314857 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f58c00d2030 0x7f58c01f3450 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f58b000e7c0 tx=0x7f58b000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:47.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:47.230+0000 7f58b5ffb640 1 -- 172.21.15.112:0/1290314857 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58b000cc60 con 0x7f58c00d2030 2024-09-21T00:29:47.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:47.230+0000 7f58b5ffb640 1 -- 172.21.15.112:0/1290314857 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f58b0010430 con 0x7f58c00d2030 2024-09-21T00:29:47.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:47.230+0000 7f58c7807640 1 -- 172.21.15.112:0/1290314857 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58c0153460 con 0x7f58c00d2030 2024-09-21T00:29:47.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:47.230+0000 7f58b5ffb640 1 -- 172.21.15.112:0/1290314857 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58b0010860 con 0x7f58c00d2030 2024-09-21T00:29:47.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:47.231+0000 7f58c7807640 1 -- 172.21.15.112:0/1290314857 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58c0153a20 con 0x7f58c00d2030 2024-09-21T00:29:47.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:47.231+0000 7f58c7807640 1 -- 172.21.15.112:0/1290314857 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58c00b2970 con 0x7f58c00d2030 2024-09-21T00:29:47.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:47.232+0000 7f58b5ffb640 1 -- 172.21.15.112:0/1290314857 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f58b0010a00 con 0x7f58c00d2030 2024-09-21T00:29:47.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:47.233+0000 7f58b5ffb640 1 --2- 172.21.15.112:0/1290314857 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f58a4078bc0 0x7f58a407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:47.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:47.233+0000 7f58b5ffb640 1 -- 172.21.15.112:0/1290314857 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5619+0+0 (secure 0 0 0) 0x7f58b009b490 con 0x7f58c00d2030 2024-09-21T00:29:47.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:47.233+0000 7f58c55a3640 1 --2- 172.21.15.112:0/1290314857 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f58a4078bc0 0x7f58a407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:47.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:47.234+0000 7f58c55a3640 1 --2- 172.21.15.112:0/1290314857 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f58a4078bc0 0x7f58a407b0d0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f58bc001720 tx=0x7f58bc007e90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:47.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:47.238+0000 7f58b5ffb640 1 -- 172.21.15.112:0/1290314857 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f58b00633f0 con 0x7f58c00d2030 2024-09-21T00:29:47.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:47.415+0000 7f58c7807640 1 -- 172.21.15.112:0/1290314857 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f58c00b4a50 con 0x7f58c00d2030 2024-09-21T00:29:48.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:48.086+0000 7f58b5ffb640 1 -- 172.21.15.112:0/1290314857 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v106) ==== 170+0+0 (secure 0 0 0) 0x7f58b0067e20 con 0x7f58c00d2030 2024-09-21T00:29:48.088 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'cephfs_data' created 2024-09-21T00:29:48.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:48.089+0000 7f58c7807640 1 -- 172.21.15.112:0/1290314857 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f58a4078bc0 msgr2=0x7f58a407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:48.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:48.089+0000 7f58c7807640 1 --2- 172.21.15.112:0/1290314857 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f58a4078bc0 0x7f58a407b0d0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f58bc001720 tx=0x7f58bc007e90 comp rx=0 tx=0).stop 2024-09-21T00:29:48.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:48.089+0000 7f58c7807640 1 -- 172.21.15.112:0/1290314857 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f58c00d2030 msgr2=0x7f58c01f3450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:48.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:48.089+0000 7f58c7807640 1 --2- 172.21.15.112:0/1290314857 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f58c00d2030 0x7f58c01f3450 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f58b000e7c0 tx=0x7f58b000ec80 comp rx=0 tx=0).stop 2024-09-21T00:29:48.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:48.089+0000 7f58c7807640 1 -- 172.21.15.112:0/1290314857 shutdown_connections 2024-09-21T00:29:48.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:48.089+0000 7f58c7807640 1 --2- 172.21.15.112:0/1290314857 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f58c015d870 0x7f58c00c7ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:48.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:48.089+0000 7f58c7807640 1 --2- 172.21.15.112:0/1290314857 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f58a4078bc0 0x7f58a407b0d0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:48.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:48.089+0000 7f58c7807640 1 --2- 172.21.15.112:0/1290314857 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f58c00d2030 0x7f58c01f3450 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:48.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:48.089+0000 7f58c7807640 1 --2- 172.21.15.112:0/1290314857 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f58c00ccae0 0x7f58c00c8de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:48.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:48.089+0000 7f58c7807640 1 -- 172.21.15.112:0/1290314857 >> 172.21.15.112:0/1290314857 conn(0x7f58c00b43f0 msgr2=0x7f58c015c160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:48.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:48.090+0000 7f58c7807640 1 -- 172.21.15.112:0/1290314857 shutdown_connections 2024-09-21T00:29:48.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:48.090+0000 7f58c7807640 1 -- 172.21.15.112:0/1290314857 wait complete. 2024-09-21T00:29:48.111 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-21T00:29:48.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:48.227+0000 7f9553225640 1 -- 172.21.15.112:0/1088722366 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f953c003d10 msgr2=0x7f953c004160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:48.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:48.227+0000 7f9553225640 1 --2- 172.21.15.112:0/1088722366 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f953c003d10 0x7f953c004160 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f9540006f90 tx=0x7f953c005830 comp rx=0 tx=0).stop 2024-09-21T00:29:48.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:48.227+0000 7f9553225640 1 -- 172.21.15.112:0/1088722366 shutdown_connections 2024-09-21T00:29:48.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:48.227+0000 7f9553225640 1 --2- 172.21.15.112:0/1088722366 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f953c003d10 0x7f953c004160 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:48.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:48.227+0000 7f9553225640 1 -- 172.21.15.112:0/1088722366 >> 172.21.15.112:0/1088722366 conn(0x7f954c0b43f0 msgr2=0x7f954c0b4830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:48.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:48.227+0000 7f9553225640 1 -- 172.21.15.112:0/1088722366 shutdown_connections 2024-09-21T00:29:48.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:48.227+0000 7f9553225640 1 -- 172.21.15.112:0/1088722366 wait complete. 2024-09-21T00:29:48.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:48.228+0000 7f9553225640 1 Processor -- start 2024-09-21T00:29:48.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:48.229+0000 7f9553225640 1 -- start start 2024-09-21T00:29:48.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:48.230+0000 7f9553225640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f953c003d10 0x7f954c15bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:48.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:48.230+0000 7f9553225640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f954c15c040 0x7f954c06fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:48.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:48.230+0000 7f9553225640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f954c070250 0x7f954c150730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:48.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:48.230+0000 7f9553225640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f954c15a450 con 0x7f954c15c040 2024-09-21T00:29:48.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:48.230+0000 7f9553225640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f954c15a2d0 con 0x7f953c003d10 2024-09-21T00:29:48.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:48.230+0000 7f9553225640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f954c15a5d0 con 0x7f954c070250 2024-09-21T00:29:48.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:48.230+0000 7f954bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f954c15c040 0x7f954c06fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:48.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:48.230+0000 7f954bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f954c15c040 0x7f954c06fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60938/0 (socket says 172.21.15.112:60938) 2024-09-21T00:29:48.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:48.230+0000 7f954bfff640 1 -- 172.21.15.112:0/1307252664 learned_addr learned my addr 172.21.15.112:0/1307252664 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:29:48.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:48.230+0000 7f95517c2640 1 --2- 172.21.15.112:0/1307252664 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f954c070250 0x7f954c150730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:48.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:48.230+0000 7f9550fc1640 1 --2- 172.21.15.112:0/1307252664 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f953c003d10 0x7f954c15bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:48.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:48.230+0000 7f954bfff640 1 -- 172.21.15.112:0/1307252664 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f954c070250 msgr2=0x7f954c150730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:48.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:48.230+0000 7f954bfff640 1 --2- 172.21.15.112:0/1307252664 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f954c070250 0x7f954c150730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:48.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:48.230+0000 7f954bfff640 1 -- 172.21.15.112:0/1307252664 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f953c003d10 msgr2=0x7f954c15bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:48.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:48.230+0000 7f954bfff640 1 --2- 172.21.15.112:0/1307252664 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f953c003d10 0x7f954c15bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:48.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:48.230+0000 7f954bfff640 1 -- 172.21.15.112:0/1307252664 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f954c06e990 con 0x7f954c15c040 2024-09-21T00:29:48.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:48.230+0000 7f95517c2640 1 --2- 172.21.15.112:0/1307252664 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f954c070250 0x7f954c150730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:29:48.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:48.230+0000 7f9550fc1640 1 --2- 172.21.15.112:0/1307252664 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f953c003d10 0x7f954c15bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:29:48.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:48.231+0000 7f954bfff640 1 --2- 172.21.15.112:0/1307252664 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f954c15c040 0x7f954c06fd10 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f9538001050 tx=0x7f9538001800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:48.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:48.231+0000 7f95497fa640 1 -- 172.21.15.112:0/1307252664 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9538011640 con 0x7f954c15c040 2024-09-21T00:29:48.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:48.231+0000 7f95497fa640 1 -- 172.21.15.112:0/1307252664 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9538016020 con 0x7f954c15c040 2024-09-21T00:29:48.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:48.231+0000 7f9553225640 1 -- 172.21.15.112:0/1307252664 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f954c06ec70 con 0x7f954c15c040 2024-09-21T00:29:48.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:48.231+0000 7f95497fa640 1 -- 172.21.15.112:0/1307252664 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9538011aa0 con 0x7f954c15c040 2024-09-21T00:29:48.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:48.231+0000 7f9553225640 1 -- 172.21.15.112:0/1307252664 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f954c06f230 con 0x7f954c15c040 2024-09-21T00:29:48.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:48.232+0000 7f9553225640 1 -- 172.21.15.112:0/1307252664 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f954c06e700 con 0x7f954c15c040 2024-09-21T00:29:48.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:48.232+0000 7f95497fa640 1 -- 172.21.15.112:0/1307252664 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9538011c40 con 0x7f954c15c040 2024-09-21T00:29:48.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:48.233+0000 7f95497fa640 1 --2- 172.21.15.112:0/1307252664 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f9530078bc0 0x7f953007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:48.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:48.233+0000 7f95497fa640 1 -- 172.21.15.112:0/1307252664 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6001+0+0 (secure 0 0 0) 0x7f953809ba70 con 0x7f954c15c040 2024-09-21T00:29:48.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:48.233+0000 7f9550fc1640 1 --2- 172.21.15.112:0/1307252664 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f9530078bc0 0x7f953007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:48.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:48.234+0000 7f9550fc1640 1 --2- 172.21.15.112:0/1307252664 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f9530078bc0 0x7f953007b0d0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f9540033000 tx=0x7f954c0ce330 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:48.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:48.238+0000 7f95497fa640 1 -- 172.21.15.112:0/1307252664 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f9538020020 con 0x7f954c15c040 2024-09-21T00:29:48.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:48.454+0000 7f9553225640 1 -- 172.21.15.112:0/1307252664 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f954c06f610 con 0x7f954c15c040 2024-09-21T00:29:49.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.088+0000 7f95497fa640 1 -- 172.21.15.112:0/1307252664 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 17 and data pool 18 v68) ==== 174+0+0 (secure 0 0 0) 0x7f954c06f610 con 0x7f954c15c040 2024-09-21T00:29:49.090 INFO:teuthology.orchestra.run.smithi112.stderr:new fs with metadata pool 17 and data pool 18 2024-09-21T00:29:49.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.090+0000 7f9553225640 1 -- 172.21.15.112:0/1307252664 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f9530078bc0 msgr2=0x7f953007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:49.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.090+0000 7f9553225640 1 --2- 172.21.15.112:0/1307252664 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f9530078bc0 0x7f953007b0d0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f9540033000 tx=0x7f954c0ce330 comp rx=0 tx=0).stop 2024-09-21T00:29:49.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.090+0000 7f9553225640 1 -- 172.21.15.112:0/1307252664 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f954c15c040 msgr2=0x7f954c06fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:49.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.090+0000 7f9553225640 1 --2- 172.21.15.112:0/1307252664 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f954c15c040 0x7f954c06fd10 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f9538001050 tx=0x7f9538001800 comp rx=0 tx=0).stop 2024-09-21T00:29:49.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.091+0000 7f9553225640 1 -- 172.21.15.112:0/1307252664 shutdown_connections 2024-09-21T00:29:49.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.091+0000 7f9553225640 1 --2- 172.21.15.112:0/1307252664 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f954c070250 0x7f954c150730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:49.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.091+0000 7f9553225640 1 --2- 172.21.15.112:0/1307252664 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f9530078bc0 0x7f953007b0d0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:49.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.091+0000 7f9553225640 1 --2- 172.21.15.112:0/1307252664 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f954c15c040 0x7f954c06fd10 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:49.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.091+0000 7f9553225640 1 --2- 172.21.15.112:0/1307252664 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f953c003d10 0x7f954c15bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:49.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.091+0000 7f9553225640 1 -- 172.21.15.112:0/1307252664 >> 172.21.15.112:0/1307252664 conn(0x7f954c0b43f0 msgr2=0x7f954c15cdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:49.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.091+0000 7f9553225640 1 -- 172.21.15.112:0/1307252664 shutdown_connections 2024-09-21T00:29:49.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.091+0000 7f9553225640 1 -- 172.21.15.112:0/1307252664 wait complete. 2024-09-21T00:29:49.120 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T00:29:49.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.234+0000 7f09a4a40640 1 -- 172.21.15.112:0/2197798020 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f09a0144dd0 msgr2=0x7f09a015f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:49.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.234+0000 7f09a4a40640 1 --2- 172.21.15.112:0/2197798020 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f09a0144dd0 0x7f09a015f0e0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f09a0072ae0 tx=0x7f098802a560 comp rx=0 tx=0).stop 2024-09-21T00:29:49.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.234+0000 7f09a4a40640 1 -- 172.21.15.112:0/2197798020 shutdown_connections 2024-09-21T00:29:49.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.234+0000 7f09a4a40640 1 --2- 172.21.15.112:0/2197798020 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f09a0144dd0 0x7f09a015f0e0 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:49.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.234+0000 7f09a4a40640 1 --2- 172.21.15.112:0/2197798020 >> v2:172.21.15.187:3300/0 conn(0x7f09a00b5ab0 0x7f09a00b0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:49.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.234+0000 7f09a4a40640 1 -- 172.21.15.112:0/2197798020 >> 172.21.15.112:0/2197798020 conn(0x7f09a0073fe0 msgr2=0x7f09a00743f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:49.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.234+0000 7f09a4a40640 1 -- 172.21.15.112:0/2197798020 shutdown_connections 2024-09-21T00:29:49.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.234+0000 7f09a4a40640 1 -- 172.21.15.112:0/2197798020 wait complete. 2024-09-21T00:29:49.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.236+0000 7f09a4a40640 1 Processor -- start 2024-09-21T00:29:49.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.236+0000 7f09a4a40640 1 -- start start 2024-09-21T00:29:49.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.237+0000 7f09a4a40640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f09a0144dd0 0x7f09a00cd320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:49.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.237+0000 7f09a4a40640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f09a015f620 0x7f09a01f2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:49.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.237+0000 7f09a4a40640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f09a00cd860 0x7f09a01f28c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:49.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.237+0000 7f09a4a40640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f09a006ee90 con 0x7f09a0144dd0 2024-09-21T00:29:49.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.237+0000 7f09a4a40640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f09a00d04d0 con 0x7f09a015f620 2024-09-21T00:29:49.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.237+0000 7f09a4a40640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f09a006f010 con 0x7f09a00cd860 2024-09-21T00:29:49.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.237+0000 7f099e59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f09a0144dd0 0x7f09a00cd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:49.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.237+0000 7f099e59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f09a0144dd0 0x7f09a00cd320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60946/0 (socket says 172.21.15.112:60946) 2024-09-21T00:29:49.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.237+0000 7f099e59c640 1 -- 172.21.15.112:0/2247543746 learned_addr learned my addr 172.21.15.112:0/2247543746 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:29:49.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.237+0000 7f0995d9b640 1 --2- 172.21.15.112:0/2247543746 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f09a015f620 0x7f09a01f2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:49.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.237+0000 7f099ed9d640 1 --2- 172.21.15.112:0/2247543746 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f09a00cd860 0x7f09a01f28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:49.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.238+0000 7f099e59c640 1 -- 172.21.15.112:0/2247543746 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f09a00cd860 msgr2=0x7f09a01f28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:49.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.238+0000 7f099e59c640 1 --2- 172.21.15.112:0/2247543746 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f09a00cd860 0x7f09a01f28c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:49.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.238+0000 7f099e59c640 1 -- 172.21.15.112:0/2247543746 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f09a015f620 msgr2=0x7f09a01f2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:49.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.238+0000 7f099e59c640 1 --2- 172.21.15.112:0/2247543746 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f09a015f620 0x7f09a01f2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:49.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.238+0000 7f099e59c640 1 -- 172.21.15.112:0/2247543746 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09a01f3140 con 0x7f09a0144dd0 2024-09-21T00:29:49.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.238+0000 7f099e59c640 1 --2- 172.21.15.112:0/2247543746 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f09a0144dd0 0x7f09a00cd320 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f0990009a10 tx=0x7f0990009ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:49.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.238+0000 7f09977fe640 1 -- 172.21.15.112:0/2247543746 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f099001c070 con 0x7f09a0144dd0 2024-09-21T00:29:49.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.238+0000 7f09977fe640 1 -- 172.21.15.112:0/2247543746 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f099000ba50 con 0x7f09a0144dd0 2024-09-21T00:29:49.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.238+0000 7f09977fe640 1 -- 172.21.15.112:0/2247543746 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f099000be60 con 0x7f09a0144dd0 2024-09-21T00:29:49.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.238+0000 7f09a4a40640 1 -- 172.21.15.112:0/2247543746 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09a00a94b0 con 0x7f09a0144dd0 2024-09-21T00:29:49.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.238+0000 7f09a4a40640 1 -- 172.21.15.112:0/2247543746 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09a00a9a70 con 0x7f09a0144dd0 2024-09-21T00:29:49.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.239+0000 7f09a4a40640 1 -- 172.21.15.112:0/2247543746 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09a006f530 con 0x7f09a0144dd0 2024-09-21T00:29:49.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.240+0000 7f09977fe640 1 -- 172.21.15.112:0/2247543746 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f099001f070 con 0x7f09a0144dd0 2024-09-21T00:29:49.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.240+0000 7f09977fe640 1 --2- 172.21.15.112:0/2247543746 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f0980078b90 0x7f098007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:49.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.240+0000 7f09977fe640 1 -- 172.21.15.112:0/2247543746 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 6101+0+0 (secure 0 0 0) 0x7f099009c9a0 con 0x7f09a0144dd0 2024-09-21T00:29:49.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.240+0000 7f0995d9b640 1 --2- 172.21.15.112:0/2247543746 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f0980078b90 0x7f098007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:49.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.241+0000 7f0995d9b640 1 --2- 172.21.15.112:0/2247543746 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f0980078b90 0x7f098007b0a0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f0988033000 tx=0x7f0988004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:49.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.245+0000 7f09977fe640 1 -- 172.21.15.112:0/2247543746 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f0990064600 con 0x7f09a0144dd0 2024-09-21T00:29:49.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.423+0000 7f09a4a40640 1 -- 172.21.15.112:0/2247543746 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f09a00b5cf0 con 0x7f09a0144dd0 2024-09-21T00:29:49.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.424+0000 7f09977fe640 1 -- 172.21.15.112:0/2247543746 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v107) ==== 74+0+17049 (secure 0 0 0) 0x7f0990069030 con 0x7f09a0144dd0 2024-09-21T00:29:49.426 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:29:49.426 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":107,"fsid":"8b34692b-dc65-49b2-a08f-26f707b96ac5","created":"2024-09-21T00:25:22.788490+0000","modified":"2024-09-21T00:29:48.456723+0000","last_up_change":"2024-09-21T00:25:36.161835+0000","last_in_change":"2024-09-21T00:25:26.346500+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T00:25:37.105006+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-09-21T00:29:46.103570+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-09-21T00:29:47.420294+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9c575086-67d2-4de2-b01b-d44dcc21d604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6809","nonce":755760621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6811","nonce":755760621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6815","nonce":755760621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6813","nonce":755760621}]},"public_addr":"172.21.15.112:6809/755760621","cluster_addr":"172.21.15.112:6811/755760621","heartbeat_back_addr":"172.21.15.112:6815/755760621","heartbeat_front_addr":"172.21.15.112:6813/755760621","state":["exists","up"]},{"osd":1,"uuid":"a983c227-a3ce-4bdd-82f1-438ba7e3ecd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6801","nonce":344932592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6803","nonce":344932592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6807","nonce":344932592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6805","nonce":344932592}]},"public_addr":"172.21.15.112:6801/344932592","cluster_addr":"172.21.15.112:6803/344932592","heartbeat_back_addr":"172.21.15.112:6807/344932592","heartbeat_front_addr":"172.21.15.112:6805/344932592","state":["exists","up"]},{"osd":2,"uuid":"1193c1d8-bfff-4a57-8200-5f69bf1ea3af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6825","nonce":407933616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6827","nonce":407933616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6831","nonce":407933616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6829","nonce":407933616}]},"public_addr":"172.21.15.112:6825/407933616","cluster_addr":"172.21.15.112:6827/407933616","heartbeat_back_addr":"172.21.15.112:6831/407933616","heartbeat_front_addr":"172.21.15.112:6829/407933616","state":["exists","up"]},{"osd":3,"uuid":"af04330a-36ad-40cc-a8dd-fa259b221de5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6817","nonce":4120164806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6819","nonce":4120164806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6823","nonce":4120164806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6821","nonce":4120164806}]},"public_addr":"172.21.15.112:6817/4120164806","cluster_addr":"172.21.15.112:6819/4120164806","heartbeat_back_addr":"172.21.15.112:6823/4120164806","heartbeat_front_addr":"172.21.15.112:6821/4120164806","state":["exists","up"]},{"osd":4,"uuid":"5b893111-292c-4c5a-b184-53ad38c7199e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6803","nonce":522077560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6805","nonce":522077560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6809","nonce":522077560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6807","nonce":522077560}]},"public_addr":"172.21.15.187:6803/522077560","cluster_addr":"172.21.15.187:6805/522077560","heartbeat_back_addr":"172.21.15.187:6809/522077560","heartbeat_front_addr":"172.21.15.187:6807/522077560","state":["exists","up"]},{"osd":5,"uuid":"c0b8e011-45fe-4c5e-925e-6be2a7f1868a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6827","nonce":3593144721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6829","nonce":3593144721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6833","nonce":3593144721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6831","nonce":3593144721}]},"public_addr":"172.21.15.187:6827/3593144721","cluster_addr":"172.21.15.187:6829/3593144721","heartbeat_back_addr":"172.21.15.187:6833/3593144721","heartbeat_front_addr":"172.21.15.187:6831/3593144721","state":["exists","up"]},{"osd":6,"uuid":"87d2d650-7489-4593-98a3-a1f433ce8dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6811","nonce":1494387657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6813","nonce":1494387657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6817","nonce":1494387657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6815","nonce":1494387657}]},"public_addr":"172.21.15.187:6811/1494387657","cluster_addr":"172.21.15.187:6813/1494387657","heartbeat_back_addr":"172.21.15.187:6817/1494387657","heartbeat_front_addr":"172.21.15.187:6815/1494387657","state":["exists","up"]},{"osd":7,"uuid":"c6b4f8e2-a1ee-4438-aa37-67ff4b6ef1f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6819","nonce":1624699842}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6821","nonce":1624699842}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6825","nonce":1624699842}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6823","nonce":1624699842}]},"public_addr":"172.21.15.187:6819/1624699842","cluster_addr":"172.21.15.187:6821/1624699842","heartbeat_back_addr":"172.21.15.187:6825/1624699842","heartbeat_front_addr":"172.21.15.187:6823/1624699842","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.873162+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.400506+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.664317+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.159791+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.202630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.113212+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.384786+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.448441+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T00:29:49.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.426+0000 7f09a4a40640 1 -- 172.21.15.112:0/2247543746 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f0980078b90 msgr2=0x7f098007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:49.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.426+0000 7f09a4a40640 1 --2- 172.21.15.112:0/2247543746 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f0980078b90 0x7f098007b0a0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f0988033000 tx=0x7f0988004730 comp rx=0 tx=0).stop 2024-09-21T00:29:49.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.426+0000 7f09a4a40640 1 -- 172.21.15.112:0/2247543746 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f09a0144dd0 msgr2=0x7f09a00cd320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:49.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.426+0000 7f09a4a40640 1 --2- 172.21.15.112:0/2247543746 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f09a0144dd0 0x7f09a00cd320 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f0990009a10 tx=0x7f0990009ed0 comp rx=0 tx=0).stop 2024-09-21T00:29:49.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.426+0000 7f09a4a40640 1 -- 172.21.15.112:0/2247543746 shutdown_connections 2024-09-21T00:29:49.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.426+0000 7f09a4a40640 1 --2- 172.21.15.112:0/2247543746 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f09a00cd860 0x7f09a01f28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:49.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.426+0000 7f09a4a40640 1 --2- 172.21.15.112:0/2247543746 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f09a015f620 0x7f09a01f2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:49.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.426+0000 7f09a4a40640 1 --2- 172.21.15.112:0/2247543746 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f0980078b90 0x7f098007b0a0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:49.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.426+0000 7f09a4a40640 1 --2- 172.21.15.112:0/2247543746 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f09a0144dd0 0x7f09a00cd320 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:49.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.426+0000 7f09a4a40640 1 -- 172.21.15.112:0/2247543746 >> 172.21.15.112:0/2247543746 conn(0x7f09a0073fe0 msgr2=0x7f09a0143700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:49.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.426+0000 7f09a4a40640 1 -- 172.21.15.112:0/2247543746 shutdown_connections 2024-09-21T00:29:49.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.426+0000 7f09a4a40640 1 -- 172.21.15.112:0/2247543746 wait complete. 2024-09-21T00:29:49.440 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T00:29:49.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.552+0000 7fef1a3ad640 1 -- 172.21.15.112:0/3624398895 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feefc003ae0 msgr2=0x7feefc003f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:49.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.552+0000 7fef1a3ad640 1 --2- 172.21.15.112:0/3624398895 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feefc003ae0 0x7feefc003f50 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fef0400c260 tx=0x7feefc005830 comp rx=0 tx=0).stop 2024-09-21T00:29:49.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.552+0000 7fef1894a640 1 -- 172.21.15.112:0/3624398895 reap_dead start 2024-09-21T00:29:49.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.552+0000 7fef1a3ad640 1 -- 172.21.15.112:0/3624398895 shutdown_connections 2024-09-21T00:29:49.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.552+0000 7fef1a3ad640 1 -- 172.21.15.112:0/3624398895 >> 172.21.15.112:0/3624398895 conn(0x7fef1409d8a0 msgr2=0x7fef1408bfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:49.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.552+0000 7fef1a3ad640 1 -- 172.21.15.112:0/3624398895 shutdown_connections 2024-09-21T00:29:49.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.552+0000 7fef1a3ad640 1 -- 172.21.15.112:0/3624398895 wait complete. 2024-09-21T00:29:49.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.553+0000 7fef1a3ad640 1 Processor -- start 2024-09-21T00:29:49.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.554+0000 7fef1a3ad640 1 -- start start 2024-09-21T00:29:49.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.555+0000 7fef1a3ad640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feefc003ae0 0x7fef141e7a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:49.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.555+0000 7fef1a3ad640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fef14155ab0 0x7fef141e7f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:49.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.555+0000 7fef1a3ad640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef14155fd0 0x7fef140b4850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:49.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.555+0000 7fef1a3ad640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fef14162630 con 0x7fef14155fd0 2024-09-21T00:29:49.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.555+0000 7fef1a3ad640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fef141624b0 con 0x7fef14155ab0 2024-09-21T00:29:49.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.555+0000 7fef1a3ad640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fef141627b0 con 0x7feefc003ae0 2024-09-21T00:29:49.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.555+0000 7fef1894a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef14155fd0 0x7fef140b4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:49.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.555+0000 7fef1894a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef14155fd0 0x7fef140b4850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60966/0 (socket says 172.21.15.112:60966) 2024-09-21T00:29:49.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.555+0000 7fef1894a640 1 -- 172.21.15.112:0/2806183219 learned_addr learned my addr 172.21.15.112:0/2806183219 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:29:49.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.555+0000 7fef13fff640 1 --2- 172.21.15.112:0/2806183219 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feefc003ae0 0x7fef141e7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:49.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.555+0000 7fef137fe640 1 --2- 172.21.15.112:0/2806183219 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fef14155ab0 0x7fef141e7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:49.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.555+0000 7fef1894a640 1 -- 172.21.15.112:0/2806183219 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feefc003ae0 msgr2=0x7fef141e7a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:49.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.555+0000 7fef1894a640 1 --2- 172.21.15.112:0/2806183219 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feefc003ae0 0x7fef141e7a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:49.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.555+0000 7fef1894a640 1 -- 172.21.15.112:0/2806183219 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fef14155ab0 msgr2=0x7fef141e7f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:49.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.555+0000 7fef1894a640 1 --2- 172.21.15.112:0/2806183219 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fef14155ab0 0x7fef141e7f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:49.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.555+0000 7fef1894a640 1 -- 172.21.15.112:0/2806183219 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef140b58d0 con 0x7fef14155fd0 2024-09-21T00:29:49.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.556+0000 7fef1894a640 1 --2- 172.21.15.112:0/2806183219 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef14155fd0 0x7fef140b4850 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fef1408dc00 tx=0x7fef0c011e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:49.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.556+0000 7fef10ff9640 1 -- 172.21.15.112:0/2806183219 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef0c003e40 con 0x7fef14155fd0 2024-09-21T00:29:49.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.556+0000 7fef10ff9640 1 -- 172.21.15.112:0/2806183219 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fef0c004b20 con 0x7fef14155fd0 2024-09-21T00:29:49.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.556+0000 7fef1a3ad640 1 -- 172.21.15.112:0/2806183219 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef141e9ab0 con 0x7fef14155fd0 2024-09-21T00:29:49.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.556+0000 7fef10ff9640 1 -- 172.21.15.112:0/2806183219 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef0c00f9b0 con 0x7fef14155fd0 2024-09-21T00:29:49.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.556+0000 7fef1a3ad640 1 -- 172.21.15.112:0/2806183219 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef140b4050 con 0x7fef14155fd0 2024-09-21T00:29:49.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.557+0000 7fef1a3ad640 1 -- 172.21.15.112:0/2806183219 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef140d38c0 con 0x7fef14155fd0 2024-09-21T00:29:49.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.558+0000 7fef10ff9640 1 -- 172.21.15.112:0/2806183219 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fef0c00fb50 con 0x7fef14155fd0 2024-09-21T00:29:49.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.559+0000 7fef10ff9640 1 --2- 172.21.15.112:0/2806183219 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fef00078bc0 0x7fef0007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:49.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.559+0000 7fef10ff9640 1 -- 172.21.15.112:0/2806183219 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 6101+0+0 (secure 0 0 0) 0x7fef0c09b280 con 0x7fef14155fd0 2024-09-21T00:29:49.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.559+0000 7fef13fff640 1 --2- 172.21.15.112:0/2806183219 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fef00078bc0 0x7fef0007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:49.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.559+0000 7fef13fff640 1 --2- 172.21.15.112:0/2806183219 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fef00078bc0 0x7fef0007b0d0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fef04033000 tx=0x7fef140d4100 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:49.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.563+0000 7fef10ff9640 1 -- 172.21.15.112:0/2806183219 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fef0c062e40 con 0x7fef14155fd0 2024-09-21T00:29:49.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.746+0000 7fef1a3ad640 1 -- 172.21.15.112:0/2806183219 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fef140d4560 con 0x7fef14155fd0 2024-09-21T00:29:49.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.747+0000 7fef10ff9640 1 -- 172.21.15.112:0/2806183219 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v107) ==== 74+0+17049 (secure 0 0 0) 0x7fef0c067870 con 0x7fef14155fd0 2024-09-21T00:29:49.751 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:29:49.751 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":107,"fsid":"8b34692b-dc65-49b2-a08f-26f707b96ac5","created":"2024-09-21T00:25:22.788490+0000","modified":"2024-09-21T00:29:48.456723+0000","last_up_change":"2024-09-21T00:25:36.161835+0000","last_in_change":"2024-09-21T00:25:26.346500+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T00:25:37.105006+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-09-21T00:29:46.103570+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-09-21T00:29:47.420294+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9c575086-67d2-4de2-b01b-d44dcc21d604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6809","nonce":755760621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6811","nonce":755760621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6815","nonce":755760621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6813","nonce":755760621}]},"public_addr":"172.21.15.112:6809/755760621","cluster_addr":"172.21.15.112:6811/755760621","heartbeat_back_addr":"172.21.15.112:6815/755760621","heartbeat_front_addr":"172.21.15.112:6813/755760621","state":["exists","up"]},{"osd":1,"uuid":"a983c227-a3ce-4bdd-82f1-438ba7e3ecd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6801","nonce":344932592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6803","nonce":344932592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6807","nonce":344932592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6805","nonce":344932592}]},"public_addr":"172.21.15.112:6801/344932592","cluster_addr":"172.21.15.112:6803/344932592","heartbeat_back_addr":"172.21.15.112:6807/344932592","heartbeat_front_addr":"172.21.15.112:6805/344932592","state":["exists","up"]},{"osd":2,"uuid":"1193c1d8-bfff-4a57-8200-5f69bf1ea3af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6825","nonce":407933616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6827","nonce":407933616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6831","nonce":407933616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6829","nonce":407933616}]},"public_addr":"172.21.15.112:6825/407933616","cluster_addr":"172.21.15.112:6827/407933616","heartbeat_back_addr":"172.21.15.112:6831/407933616","heartbeat_front_addr":"172.21.15.112:6829/407933616","state":["exists","up"]},{"osd":3,"uuid":"af04330a-36ad-40cc-a8dd-fa259b221de5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6817","nonce":4120164806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6819","nonce":4120164806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6823","nonce":4120164806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6821","nonce":4120164806}]},"public_addr":"172.21.15.112:6817/4120164806","cluster_addr":"172.21.15.112:6819/4120164806","heartbeat_back_addr":"172.21.15.112:6823/4120164806","heartbeat_front_addr":"172.21.15.112:6821/4120164806","state":["exists","up"]},{"osd":4,"uuid":"5b893111-292c-4c5a-b184-53ad38c7199e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6803","nonce":522077560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6805","nonce":522077560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6809","nonce":522077560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6807","nonce":522077560}]},"public_addr":"172.21.15.187:6803/522077560","cluster_addr":"172.21.15.187:6805/522077560","heartbeat_back_addr":"172.21.15.187:6809/522077560","heartbeat_front_addr":"172.21.15.187:6807/522077560","state":["exists","up"]},{"osd":5,"uuid":"c0b8e011-45fe-4c5e-925e-6be2a7f1868a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6827","nonce":3593144721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6829","nonce":3593144721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6833","nonce":3593144721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6831","nonce":3593144721}]},"public_addr":"172.21.15.187:6827/3593144721","cluster_addr":"172.21.15.187:6829/3593144721","heartbeat_back_addr":"172.21.15.187:6833/3593144721","heartbeat_front_addr":"172.21.15.187:6831/3593144721","state":["exists","up"]},{"osd":6,"uuid":"87d2d650-7489-4593-98a3-a1f433ce8dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6811","nonce":1494387657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6813","nonce":1494387657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6817","nonce":1494387657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6815","nonce":1494387657}]},"public_addr":"172.21.15.187:6811/1494387657","cluster_addr":"172.21.15.187:6813/1494387657","heartbeat_back_addr":"172.21.15.187:6817/1494387657","heartbeat_front_addr":"172.21.15.187:6815/1494387657","state":["exists","up"]},{"osd":7,"uuid":"c6b4f8e2-a1ee-4438-aa37-67ff4b6ef1f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6819","nonce":1624699842}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6821","nonce":1624699842}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6825","nonce":1624699842}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6823","nonce":1624699842}]},"public_addr":"172.21.15.187:6819/1624699842","cluster_addr":"172.21.15.187:6821/1624699842","heartbeat_back_addr":"172.21.15.187:6825/1624699842","heartbeat_front_addr":"172.21.15.187:6823/1624699842","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.873162+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.400506+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.664317+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.159791+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.202630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.113212+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.384786+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.448441+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T00:29:49.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.750+0000 7fef1a3ad640 1 -- 172.21.15.112:0/2806183219 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fef00078bc0 msgr2=0x7fef0007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:49.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.750+0000 7fef1a3ad640 1 --2- 172.21.15.112:0/2806183219 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fef00078bc0 0x7fef0007b0d0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fef04033000 tx=0x7fef140d4100 comp rx=0 tx=0).stop 2024-09-21T00:29:49.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.750+0000 7fef1a3ad640 1 -- 172.21.15.112:0/2806183219 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef14155fd0 msgr2=0x7fef140b4850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:49.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.750+0000 7fef1a3ad640 1 --2- 172.21.15.112:0/2806183219 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef14155fd0 0x7fef140b4850 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fef1408dc00 tx=0x7fef0c011e80 comp rx=0 tx=0).stop 2024-09-21T00:29:49.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.750+0000 7fef1a3ad640 1 -- 172.21.15.112:0/2806183219 shutdown_connections 2024-09-21T00:29:49.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.750+0000 7fef1a3ad640 1 --2- 172.21.15.112:0/2806183219 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fef00078bc0 0x7fef0007b0d0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:49.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.750+0000 7fef1a3ad640 1 --2- 172.21.15.112:0/2806183219 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef14155fd0 0x7fef140b4850 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:49.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.750+0000 7fef1a3ad640 1 --2- 172.21.15.112:0/2806183219 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fef14155ab0 0x7fef141e7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:49.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.750+0000 7fef1a3ad640 1 --2- 172.21.15.112:0/2806183219 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feefc003ae0 0x7fef141e7a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:49.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.750+0000 7fef1a3ad640 1 -- 172.21.15.112:0/2806183219 >> 172.21.15.112:0/2806183219 conn(0x7fef1409d8a0 msgr2=0x7fef14160220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:49.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.750+0000 7fef1a3ad640 1 -- 172.21.15.112:0/2806183219 shutdown_connections 2024-09-21T00:29:49.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.750+0000 7fef1a3ad640 1 -- 172.21.15.112:0/2806183219 wait complete. 2024-09-21T00:29:49.764 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-21T00:29:49.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.877+0000 7f576cc2e640 1 -- 172.21.15.112:0/3415315192 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f576815d870 msgr2=0x7f57680d0860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:49.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.877+0000 7f576cc2e640 1 --2- 172.21.15.112:0/3415315192 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f576815d870 0x7f57680d0860 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f57680cda60 tx=0x7f575002a5f0 comp rx=0 tx=0).stop 2024-09-21T00:29:49.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.877+0000 7f576cc2e640 1 -- 172.21.15.112:0/3415315192 shutdown_connections 2024-09-21T00:29:49.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.877+0000 7f576cc2e640 1 --2- 172.21.15.112:0/3415315192 >> v2:172.21.15.187:3300/0 conn(0x7f57680ccae0 0x7f57680ccef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:49.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.877+0000 7f576cc2e640 1 --2- 172.21.15.112:0/3415315192 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f576815d870 0x7f57680d0860 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:49.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.877+0000 7f576cc2e640 1 -- 172.21.15.112:0/3415315192 >> 172.21.15.112:0/3415315192 conn(0x7f57680b43f0 msgr2=0x7f57680b4830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:49.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.877+0000 7f576cc2e640 1 -- 172.21.15.112:0/3415315192 shutdown_connections 2024-09-21T00:29:49.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.877+0000 7f576cc2e640 1 -- 172.21.15.112:0/3415315192 wait complete. 2024-09-21T00:29:49.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.879+0000 7f576cc2e640 1 Processor -- start 2024-09-21T00:29:49.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.879+0000 7f576cc2e640 1 -- start start 2024-09-21T00:29:49.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.880+0000 7f576cc2e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f57680ccae0 0x7f57680c4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:49.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.880+0000 7f576cc2e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f57680d2030 0x7f576806f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:49.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.880+0000 7f576cc2e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f576815d870 0x7f576806f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:49.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.880+0000 7f576cc2e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f57680b2280 con 0x7f57680d2030 2024-09-21T00:29:49.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.880+0000 7f576cc2e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f57680b2100 con 0x7f576815d870 2024-09-21T00:29:49.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.880+0000 7f576cc2e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f57680b2400 con 0x7f57680ccae0 2024-09-21T00:29:49.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.880+0000 7f575dd9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f57680d2030 0x7f576806f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:49.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.880+0000 7f575dd9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f57680d2030 0x7f576806f220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60984/0 (socket says 172.21.15.112:60984) 2024-09-21T00:29:49.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.880+0000 7f575dd9b640 1 -- 172.21.15.112:0/3826677996 learned_addr learned my addr 172.21.15.112:0/3826677996 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:29:49.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.880+0000 7f576659c640 1 --2- 172.21.15.112:0/3826677996 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f57680ccae0 0x7f57680c4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:49.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.880+0000 7f5766d9d640 1 --2- 172.21.15.112:0/3826677996 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f576815d870 0x7f576806f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:49.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.881+0000 7f575dd9b640 1 -- 172.21.15.112:0/3826677996 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f57680ccae0 msgr2=0x7f57680c4f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:49.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.881+0000 7f575dd9b640 1 --2- 172.21.15.112:0/3826677996 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f57680ccae0 0x7f57680c4f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:49.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.881+0000 7f575dd9b640 1 -- 172.21.15.112:0/3826677996 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f576815d870 msgr2=0x7f576806f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:49.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.881+0000 7f575dd9b640 1 --2- 172.21.15.112:0/3826677996 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f576815d870 0x7f576806f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:49.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.881+0000 7f575dd9b640 1 -- 172.21.15.112:0/3826677996 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f576806ffe0 con 0x7f57680d2030 2024-09-21T00:29:49.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.881+0000 7f575dd9b640 1 --2- 172.21.15.112:0/3826677996 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f57680d2030 0x7f576806f220 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f5750000c00 tx=0x7f5750003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:49.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.881+0000 7f575f7fe640 1 -- 172.21.15.112:0/3826677996 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f575003e030 con 0x7f57680d2030 2024-09-21T00:29:49.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.881+0000 7f575f7fe640 1 -- 172.21.15.112:0/3826677996 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5750003e00 con 0x7f57680d2030 2024-09-21T00:29:49.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.881+0000 7f576cc2e640 1 -- 172.21.15.112:0/3826677996 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57681fb2b0 con 0x7f57680d2030 2024-09-21T00:29:49.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.881+0000 7f575f7fe640 1 -- 172.21.15.112:0/3826677996 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5750035040 con 0x7f57680d2030 2024-09-21T00:29:49.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.881+0000 7f576cc2e640 1 -- 172.21.15.112:0/3826677996 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57681fb870 con 0x7f57680d2030 2024-09-21T00:29:49.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.882+0000 7f576cc2e640 1 -- 172.21.15.112:0/3826677996 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57680b2920 con 0x7f57680d2030 2024-09-21T00:29:49.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.883+0000 7f575f7fe640 1 -- 172.21.15.112:0/3826677996 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5750041030 con 0x7f57680d2030 2024-09-21T00:29:49.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.883+0000 7f575f7fe640 1 --2- 172.21.15.112:0/3826677996 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f5748078b90 0x7f574807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:49.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.883+0000 7f575f7fe640 1 -- 172.21.15.112:0/3826677996 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 6101+0+0 (secure 0 0 0) 0x7f57500bb5c0 con 0x7f57680d2030 2024-09-21T00:29:49.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.883+0000 7f576659c640 1 --2- 172.21.15.112:0/3826677996 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f5748078b90 0x7f574807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:49.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.884+0000 7f576659c640 1 --2- 172.21.15.112:0/3826677996 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f5748078b90 0x7f574807b0a0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f5758001720 tx=0x7f5758009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:49.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:49.888+0000 7f575f7fe640 1 -- 172.21.15.112:0/3826677996 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f57500832c0 con 0x7f57680d2030 2024-09-21T00:29:50.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:50.102+0000 7f576cc2e640 1 -- 172.21.15.112:0/3826677996 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f576806fd70 con 0x7f57680d2030 2024-09-21T00:29:50.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:50.113+0000 7f575f7fe640 1 -- 172.21.15.112:0/3826677996 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v69) ==== 118+0+0 (secure 0 0 0) 0x7f5750087cf0 con 0x7f57680d2030 2024-09-21T00:29:50.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:50.115+0000 7f576cc2e640 1 -- 172.21.15.112:0/3826677996 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f5748078b90 msgr2=0x7f574807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:50.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:50.115+0000 7f576cc2e640 1 --2- 172.21.15.112:0/3826677996 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f5748078b90 0x7f574807b0a0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f5758001720 tx=0x7f5758009040 comp rx=0 tx=0).stop 2024-09-21T00:29:50.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:50.116+0000 7f576cc2e640 1 -- 172.21.15.112:0/3826677996 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f57680d2030 msgr2=0x7f576806f220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:50.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:50.116+0000 7f576cc2e640 1 --2- 172.21.15.112:0/3826677996 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f57680d2030 0x7f576806f220 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f5750000c00 tx=0x7f5750003910 comp rx=0 tx=0).stop 2024-09-21T00:29:50.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:50.116+0000 7f576cc2e640 1 -- 172.21.15.112:0/3826677996 shutdown_connections 2024-09-21T00:29:50.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:50.116+0000 7f576cc2e640 1 --2- 172.21.15.112:0/3826677996 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f576815d870 0x7f576806f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:50.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:50.116+0000 7f576cc2e640 1 --2- 172.21.15.112:0/3826677996 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f5748078b90 0x7f574807b0a0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:50.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:50.116+0000 7f576cc2e640 1 --2- 172.21.15.112:0/3826677996 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f57680d2030 0x7f576806f220 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:50.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:50.116+0000 7f576cc2e640 1 --2- 172.21.15.112:0/3826677996 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f57680ccae0 0x7f57680c4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:50.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:50.116+0000 7f576cc2e640 1 -- 172.21.15.112:0/3826677996 >> 172.21.15.112:0/3826677996 conn(0x7f57680b43f0 msgr2=0x7f57680b4800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:50.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:50.116+0000 7f576cc2e640 1 -- 172.21.15.112:0/3826677996 shutdown_connections 2024-09-21T00:29:50.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:50.116+0000 7f576cc2e640 1 -- 172.21.15.112:0/3826677996 wait complete. 2024-09-21T00:29:50.129 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-21T00:29:50.129 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-21T00:29:50.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:50.243+0000 7f9ff67a9640 1 -- 172.21.15.112:0/4181836442 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9fd8004800 msgr2=0x7f9fd8004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:50.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:50.243+0000 7f9ff67a9640 1 --2- 172.21.15.112:0/4181836442 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9fd8004800 0x7f9fd8004c10 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f9ff00b7490 tx=0x7f9fd8005830 comp rx=0 tx=0).stop 2024-09-21T00:29:50.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:50.243+0000 7f9ff67a9640 1 -- 172.21.15.112:0/4181836442 shutdown_connections 2024-09-21T00:29:50.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:50.243+0000 7f9ff67a9640 1 --2- 172.21.15.112:0/4181836442 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9fd8004800 0x7f9fd8004c10 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:50.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:50.243+0000 7f9ff67a9640 1 -- 172.21.15.112:0/4181836442 >> 172.21.15.112:0/4181836442 conn(0x7f9ff009d8a0 msgr2=0x7f9ff008bfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:50.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:50.243+0000 7f9ff67a9640 1 -- 172.21.15.112:0/4181836442 shutdown_connections 2024-09-21T00:29:50.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:50.243+0000 7f9ff67a9640 1 -- 172.21.15.112:0/4181836442 wait complete. 2024-09-21T00:29:50.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:50.245+0000 7f9ff67a9640 1 Processor -- start 2024-09-21T00:29:50.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:50.245+0000 7f9ff67a9640 1 -- start start 2024-09-21T00:29:50.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:50.246+0000 7f9ff67a9640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9fd8004800 0x7f9ff006e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:50.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:50.246+0000 7f9ff67a9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ff006eb90 0x7f9ff01f5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:50.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:50.246+0000 7f9ff67a9640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9ff01f5ec0 0x7f9ff00cb640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:50.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:50.246+0000 7f9ff67a9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff00705e0 con 0x7f9ff006eb90 2024-09-21T00:29:50.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:50.246+0000 7f9ff67a9640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff0070460 con 0x7f9ff01f5ec0 2024-09-21T00:29:50.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:50.246+0000 7f9ff67a9640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f9ff0070760 con 0x7f9fd8004800 2024-09-21T00:29:50.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:50.246+0000 7f9fef7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ff006eb90 0x7f9ff01f5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:50.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:50.246+0000 7f9fef7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ff006eb90 0x7f9ff01f5980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:32770/0 (socket says 172.21.15.112:32770) 2024-09-21T00:29:50.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:50.246+0000 7f9fef7fe640 1 -- 172.21.15.112:0/218871732 learned_addr learned my addr 172.21.15.112:0/218871732 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:29:50.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:50.246+0000 7f9feffff640 1 --2- 172.21.15.112:0/218871732 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9fd8004800 0x7f9ff006e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:50.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:50.246+0000 7f9ff4d46640 1 --2- 172.21.15.112:0/218871732 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9ff01f5ec0 0x7f9ff00cb640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:50.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:50.246+0000 7f9fef7fe640 1 -- 172.21.15.112:0/218871732 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9fd8004800 msgr2=0x7f9ff006e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:50.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:50.246+0000 7f9fef7fe640 1 --2- 172.21.15.112:0/218871732 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9fd8004800 0x7f9ff006e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:50.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:50.246+0000 7f9fef7fe640 1 -- 172.21.15.112:0/218871732 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9ff01f5ec0 msgr2=0x7f9ff00cb640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:50.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:50.246+0000 7f9fef7fe640 1 --2- 172.21.15.112:0/218871732 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9ff01f5ec0 0x7f9ff00cb640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:50.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:50.246+0000 7f9fef7fe640 1 -- 172.21.15.112:0/218871732 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ff01f66b0 con 0x7f9ff006eb90 2024-09-21T00:29:50.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:50.247+0000 7f9fef7fe640 1 --2- 172.21.15.112:0/218871732 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ff006eb90 0x7f9ff01f5980 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f9fe4009c10 tx=0x7f9fe4033670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:50.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:50.247+0000 7f9fecff9640 1 -- 172.21.15.112:0/218871732 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fe4038040 con 0x7f9ff006eb90 2024-09-21T00:29:50.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:50.247+0000 7f9fecff9640 1 -- 172.21.15.112:0/218871732 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9fe4033860 con 0x7f9ff006eb90 2024-09-21T00:29:50.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:50.247+0000 7f9ff67a9640 1 -- 172.21.15.112:0/218871732 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ff006ce80 con 0x7f9ff006eb90 2024-09-21T00:29:50.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:50.247+0000 7f9fecff9640 1 -- 172.21.15.112:0/218871732 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fe4033cb0 con 0x7f9ff006eb90 2024-09-21T00:29:50.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:50.247+0000 7f9ff67a9640 1 -- 172.21.15.112:0/218871732 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ff006d440 con 0x7f9ff006eb90 2024-09-21T00:29:50.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:50.248+0000 7f9ff67a9640 1 -- 172.21.15.112:0/218871732 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ff01f6420 con 0x7f9ff006eb90 2024-09-21T00:29:50.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:50.249+0000 7f9fecff9640 1 -- 172.21.15.112:0/218871732 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9fe4043020 con 0x7f9ff006eb90 2024-09-21T00:29:50.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:50.249+0000 7f9fecff9640 1 --2- 172.21.15.112:0/218871732 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f9fdc078b90 0x7f9fdc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:50.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:50.249+0000 7f9fecff9640 1 -- 172.21.15.112:0/218871732 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 6101+0+0 (secure 0 0 0) 0x7f9fe40bb0a0 con 0x7f9ff006eb90 2024-09-21T00:29:50.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:50.249+0000 7f9feffff640 1 --2- 172.21.15.112:0/218871732 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f9fdc078b90 0x7f9fdc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:50.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:50.250+0000 7f9feffff640 1 --2- 172.21.15.112:0/218871732 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f9fdc078b90 0x7f9fdc07b0a0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f9fe0004710 tx=0x7f9fe0006e30 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:50.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:50.254+0000 7f9fecff9640 1 -- 172.21.15.112:0/218871732 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f9fe404a020 con 0x7f9ff006eb90 2024-09-21T00:29:50.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:50.466+0000 7f9ff67a9640 1 -- 172.21.15.112:0/218871732 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f9ff00d4620 con 0x7f9ff006eb90 2024-09-21T00:29:51.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.120+0000 7f9fecff9640 1 -- 172.21.15.112:0/218871732 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v70) ==== 122+0+0 (secure 0 0 0) 0x7f9fe4082d00 con 0x7f9ff006eb90 2024-09-21T00:29:51.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.122+0000 7f9ff67a9640 1 -- 172.21.15.112:0/218871732 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f9fdc078b90 msgr2=0x7f9fdc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:51.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.122+0000 7f9ff67a9640 1 --2- 172.21.15.112:0/218871732 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f9fdc078b90 0x7f9fdc07b0a0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f9fe0004710 tx=0x7f9fe0006e30 comp rx=0 tx=0).stop 2024-09-21T00:29:51.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.122+0000 7f9ff67a9640 1 -- 172.21.15.112:0/218871732 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ff006eb90 msgr2=0x7f9ff01f5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:51.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.122+0000 7f9ff67a9640 1 --2- 172.21.15.112:0/218871732 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ff006eb90 0x7f9ff01f5980 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f9fe4009c10 tx=0x7f9fe4033670 comp rx=0 tx=0).stop 2024-09-21T00:29:51.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.122+0000 7f9ff67a9640 1 -- 172.21.15.112:0/218871732 shutdown_connections 2024-09-21T00:29:51.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.122+0000 7f9ff67a9640 1 --2- 172.21.15.112:0/218871732 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9ff01f5ec0 0x7f9ff00cb640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:51.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.122+0000 7f9ff67a9640 1 --2- 172.21.15.112:0/218871732 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f9fdc078b90 0x7f9fdc07b0a0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:51.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.122+0000 7f9ff67a9640 1 --2- 172.21.15.112:0/218871732 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ff006eb90 0x7f9ff01f5980 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:51.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.122+0000 7f9ff67a9640 1 --2- 172.21.15.112:0/218871732 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9fd8004800 0x7f9ff006e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:51.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.122+0000 7f9ff67a9640 1 -- 172.21.15.112:0/218871732 >> 172.21.15.112:0/218871732 conn(0x7f9ff009d8a0 msgr2=0x7f9ff015fa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:51.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.123+0000 7f9ff67a9640 1 -- 172.21.15.112:0/218871732 shutdown_connections 2024-09-21T00:29:51.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.123+0000 7f9ff67a9640 1 -- 172.21.15.112:0/218871732 wait complete. 2024-09-21T00:29:51.138 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-21T00:29:51.139 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T00:29:51.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.251+0000 7fcc6d486640 1 -- 172.21.15.112:0/2920438968 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc50005330 msgr2=0x7fcc5000b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:51.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.251+0000 7fcc6d486640 1 --2- 172.21.15.112:0/2920438968 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc50005330 0x7fcc5000b9e0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7fcc60009b50 tx=0x7fcc50005830 comp rx=0 tx=0).stop 2024-09-21T00:29:51.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.251+0000 7fcc6d486640 1 -- 172.21.15.112:0/2920438968 shutdown_connections 2024-09-21T00:29:51.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.251+0000 7fcc6d486640 1 --2- 172.21.15.112:0/2920438968 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc50005330 0x7fcc5000b9e0 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:51.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.251+0000 7fcc6d486640 1 -- 172.21.15.112:0/2920438968 >> 172.21.15.112:0/2920438968 conn(0x7fcc6808bfe0 msgr2=0x7fcc6809d890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:51.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.251+0000 7fcc6d486640 1 -- 172.21.15.112:0/2920438968 shutdown_connections 2024-09-21T00:29:51.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.251+0000 7fcc6d486640 1 -- 172.21.15.112:0/2920438968 wait complete. 2024-09-21T00:29:51.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.253+0000 7fcc6d486640 1 Processor -- start 2024-09-21T00:29:51.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.253+0000 7fcc6d486640 1 -- start start 2024-09-21T00:29:51.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.254+0000 7fcc6d486640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcc50005330 0x7fcc680c9810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:51.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.254+0000 7fcc6d486640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc6806fea0 0x7fcc6815da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:51.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.254+0000 7fcc6d486640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcc680cdce0 0x7fcc680ce0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:51.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.254+0000 7fcc6d486640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcc681633e0 con 0x7fcc6806fea0 2024-09-21T00:29:51.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.254+0000 7fcc6d486640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fcc68163260 con 0x7fcc680cdce0 2024-09-21T00:29:51.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.254+0000 7fcc6d486640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fcc68163560 con 0x7fcc50005330 2024-09-21T00:29:51.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.255+0000 7fcc667fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc6806fea0 0x7fcc6815da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:51.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.255+0000 7fcc667fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc6806fea0 0x7fcc6815da90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:32792/0 (socket says 172.21.15.112:32792) 2024-09-21T00:29:51.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.255+0000 7fcc667fc640 1 -- 172.21.15.112:0/117922178 learned_addr learned my addr 172.21.15.112:0/117922178 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:29:51.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.255+0000 7fcc66ffd640 1 --2- 172.21.15.112:0/117922178 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcc50005330 0x7fcc680c9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:51.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.255+0000 7fcc677fe640 1 --2- 172.21.15.112:0/117922178 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcc680cdce0 0x7fcc680ce0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:51.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.255+0000 7fcc667fc640 1 -- 172.21.15.112:0/117922178 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcc50005330 msgr2=0x7fcc680c9810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:51.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.255+0000 7fcc667fc640 1 --2- 172.21.15.112:0/117922178 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcc50005330 0x7fcc680c9810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:51.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.255+0000 7fcc667fc640 1 -- 172.21.15.112:0/117922178 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcc680cdce0 msgr2=0x7fcc680ce0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:51.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.255+0000 7fcc667fc640 1 --2- 172.21.15.112:0/117922178 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcc680cdce0 0x7fcc680ce0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:51.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.255+0000 7fcc667fc640 1 -- 172.21.15.112:0/117922178 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc680ce630 con 0x7fcc6806fea0 2024-09-21T00:29:51.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.255+0000 7fcc667fc640 1 --2- 172.21.15.112:0/117922178 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc6806fea0 0x7fcc6815da90 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c00e570 tx=0x7fcc5c00ea30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:51.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.255+0000 7fcc4f7fe640 1 -- 172.21.15.112:0/117922178 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc5c015020 con 0x7fcc6806fea0 2024-09-21T00:29:51.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.255+0000 7fcc4f7fe640 1 -- 172.21.15.112:0/117922178 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcc5c010980 con 0x7fcc6806fea0 2024-09-21T00:29:51.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.255+0000 7fcc6d486640 1 -- 172.21.15.112:0/117922178 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc680ca180 con 0x7fcc6806fea0 2024-09-21T00:29:51.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.255+0000 7fcc4f7fe640 1 -- 172.21.15.112:0/117922178 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc5c010d70 con 0x7fcc6806fea0 2024-09-21T00:29:51.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.255+0000 7fcc6d486640 1 -- 172.21.15.112:0/117922178 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc680ca740 con 0x7fcc6806fea0 2024-09-21T00:29:51.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.256+0000 7fcc6d486640 1 -- 172.21.15.112:0/117922178 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc680d05c0 con 0x7fcc6806fea0 2024-09-21T00:29:51.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.257+0000 7fcc4f7fe640 1 -- 172.21.15.112:0/117922178 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fcc5c018020 con 0x7fcc6806fea0 2024-09-21T00:29:51.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.258+0000 7fcc4f7fe640 1 --2- 172.21.15.112:0/117922178 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fcc54078b90 0x7fcc5407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:51.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.258+0000 7fcc4f7fe640 1 -- 172.21.15.112:0/117922178 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 6101+0+0 (secure 0 0 0) 0x7fcc5c09b370 con 0x7fcc6806fea0 2024-09-21T00:29:51.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.258+0000 7fcc66ffd640 1 --2- 172.21.15.112:0/117922178 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fcc54078b90 0x7fcc5407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:51.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.259+0000 7fcc66ffd640 1 --2- 172.21.15.112:0/117922178 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fcc54078b90 0x7fcc5407b0a0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fcc58000f80 tx=0x7fcc58004a00 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:51.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.263+0000 7fcc4f7fe640 1 -- 172.21.15.112:0/117922178 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fcc5c01f020 con 0x7fcc6806fea0 2024-09-21T00:29:51.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.475+0000 7fcc6d486640 1 -- 172.21.15.112:0/117922178 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcc680c9ed0 con 0x7fcc6806fea0 2024-09-21T00:29:51.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.475+0000 7fcc4f7fe640 1 -- 172.21.15.112:0/117922178 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) ==== 94+0+5237 (secure 0 0 0) 0x7fcc680ce630 con 0x7fcc6806fea0 2024-09-21T00:29:51.477 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:29:51.477 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":70,"btime":"2024-09-21T00:29:51:115169+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6472,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6836/2828207894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":2828207894},{"type":"v1","addr":"172.21.15.187:6836","nonce":2828207894}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":67},{"gid":7152,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/2711765184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":2711765184},{"type":"v1","addr":"172.21.15.112:6833","nonce":2711765184}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":67},{"gid":7155,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/1863087214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":1863087214},{"type":"v1","addr":"172.21.15.112:6835","nonce":1863087214}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":70,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T00:29:48.456672+0000","modified":"2024-09-21T00:29:50.468508+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7167},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7167":{"gid":7167,"name":"d","rank":0,"incarnation":68,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6837/2071643267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6835","nonce":2071643267},{"type":"v1","addr":"172.21.15.187:6837","nonce":2071643267}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7167,"qdb_cluster":[7167]},"id":8}]} 2024-09-21T00:29:51.477 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 70 2024-09-21T00:29:51.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.477+0000 7fcc6d486640 1 -- 172.21.15.112:0/117922178 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fcc54078b90 msgr2=0x7fcc5407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:51.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.477+0000 7fcc6d486640 1 --2- 172.21.15.112:0/117922178 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fcc54078b90 0x7fcc5407b0a0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fcc58000f80 tx=0x7fcc58004a00 comp rx=0 tx=0).stop 2024-09-21T00:29:51.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.477+0000 7fcc6d486640 1 -- 172.21.15.112:0/117922178 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc6806fea0 msgr2=0x7fcc6815da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:51.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.477+0000 7fcc6d486640 1 --2- 172.21.15.112:0/117922178 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc6806fea0 0x7fcc6815da90 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c00e570 tx=0x7fcc5c00ea30 comp rx=0 tx=0).stop 2024-09-21T00:29:51.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.477+0000 7fcc6d486640 1 -- 172.21.15.112:0/117922178 shutdown_connections 2024-09-21T00:29:51.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.477+0000 7fcc6d486640 1 --2- 172.21.15.112:0/117922178 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcc680cdce0 0x7fcc680ce0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:51.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.477+0000 7fcc6d486640 1 --2- 172.21.15.112:0/117922178 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fcc54078b90 0x7fcc5407b0a0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:51.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.477+0000 7fcc6d486640 1 --2- 172.21.15.112:0/117922178 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc6806fea0 0x7fcc6815da90 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:51.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.477+0000 7fcc6d486640 1 --2- 172.21.15.112:0/117922178 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcc50005330 0x7fcc680c9810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:51.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.477+0000 7fcc6d486640 1 -- 172.21.15.112:0/117922178 >> 172.21.15.112:0/117922178 conn(0x7fcc6808bfe0 msgr2=0x7fcc681613b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:51.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.477+0000 7fcc6d486640 1 -- 172.21.15.112:0/117922178 shutdown_connections 2024-09-21T00:29:51.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.477+0000 7fcc6d486640 1 -- 172.21.15.112:0/117922178 wait complete. 2024-09-21T00:29:51.491 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T00:29:51.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.600+0000 7ffbb16a2640 1 -- 172.21.15.112:0/2739350495 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffbac0ccbb0 msgr2=0x7ffbac0ccfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:51.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.600+0000 7ffbb16a2640 1 --2- 172.21.15.112:0/2739350495 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffbac0ccbb0 0x7ffbac0ccfc0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7ffba4005fd0 tx=0x7ffba402aa10 comp rx=0 tx=0).stop 2024-09-21T00:29:51.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.600+0000 7ffbb16a2640 1 -- 172.21.15.112:0/2739350495 shutdown_connections 2024-09-21T00:29:51.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.600+0000 7ffbb16a2640 1 --2- 172.21.15.112:0/2739350495 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffbac0ccbb0 0x7ffbac0ccfc0 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:51.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.600+0000 7ffbb16a2640 1 --2- 172.21.15.112:0/2739350495 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffbac070210 0x7ffbac070620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:51.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.600+0000 7ffbb16a2640 1 -- 172.21.15.112:0/2739350495 >> 172.21.15.112:0/2739350495 conn(0x7ffbac073fe0 msgr2=0x7ffbac0743f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:51.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.600+0000 7ffbb16a2640 1 -- 172.21.15.112:0/2739350495 shutdown_connections 2024-09-21T00:29:51.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.600+0000 7ffbb16a2640 1 -- 172.21.15.112:0/2739350495 wait complete. 2024-09-21T00:29:51.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.602+0000 7ffbb16a2640 1 Processor -- start 2024-09-21T00:29:51.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.602+0000 7ffbb16a2640 1 -- start start 2024-09-21T00:29:51.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.603+0000 7ffbb16a2640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffbac070210 0x7ffbac06dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:51.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.603+0000 7ffbb16a2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffbac0ccbb0 0x7ffbac0cb020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:51.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.603+0000 7ffbb16a2640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffbac15c5f0 0x7ffbac158210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:51.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.603+0000 7ffbb16a2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ffbac15ca00 con 0x7ffbac0ccbb0 2024-09-21T00:29:51.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.603+0000 7ffbb16a2640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ffbac070b60 con 0x7ffbac070210 2024-09-21T00:29:51.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.603+0000 7ffbb16a2640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ffbac156840 con 0x7ffbac15c5f0 2024-09-21T00:29:51.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.603+0000 7ffba27fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffbac0ccbb0 0x7ffbac0cb020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:51.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.604+0000 7ffba27fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffbac0ccbb0 0x7ffbac0cb020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:32802/0 (socket says 172.21.15.112:32802) 2024-09-21T00:29:51.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.604+0000 7ffba27fc640 1 -- 172.21.15.112:0/1768550581 learned_addr learned my addr 172.21.15.112:0/1768550581 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:29:51.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.604+0000 7ffbab7fe640 1 --2- 172.21.15.112:0/1768550581 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffbac15c5f0 0x7ffbac158210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:51.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.604+0000 7ffbaaffd640 1 --2- 172.21.15.112:0/1768550581 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffbac070210 0x7ffbac06dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:51.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.604+0000 7ffba27fc640 1 -- 172.21.15.112:0/1768550581 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffbac15c5f0 msgr2=0x7ffbac158210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:51.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.604+0000 7ffba27fc640 1 --2- 172.21.15.112:0/1768550581 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffbac15c5f0 0x7ffbac158210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:51.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.604+0000 7ffba27fc640 1 -- 172.21.15.112:0/1768550581 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffbac070210 msgr2=0x7ffbac06dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:51.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.604+0000 7ffba27fc640 1 --2- 172.21.15.112:0/1768550581 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffbac070210 0x7ffbac06dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:51.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.604+0000 7ffba27fc640 1 -- 172.21.15.112:0/1768550581 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbac1f55f0 con 0x7ffbac0ccbb0 2024-09-21T00:29:51.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.604+0000 7ffba27fc640 1 --2- 172.21.15.112:0/1768550581 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffbac0ccbb0 0x7ffbac0cb020 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7ffb94009fd0 tx=0x7ffb9400ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:51.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.604+0000 7ffba3fff640 1 -- 172.21.15.112:0/1768550581 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb9400ccc0 con 0x7ffbac0ccbb0 2024-09-21T00:29:51.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.604+0000 7ffba3fff640 1 -- 172.21.15.112:0/1768550581 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffb9400ee70 con 0x7ffbac0ccbb0 2024-09-21T00:29:51.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.604+0000 7ffba3fff640 1 -- 172.21.15.112:0/1768550581 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb9400ce60 con 0x7ffbac0ccbb0 2024-09-21T00:29:51.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.604+0000 7ffbb16a2640 1 -- 172.21.15.112:0/1768550581 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbac1f9110 con 0x7ffbac0ccbb0 2024-09-21T00:29:51.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.604+0000 7ffbb16a2640 1 -- 172.21.15.112:0/1768550581 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbac1f96d0 con 0x7ffbac0ccbb0 2024-09-21T00:29:51.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.605+0000 7ffbb16a2640 1 -- 172.21.15.112:0/1768550581 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffbac070620 con 0x7ffbac0ccbb0 2024-09-21T00:29:51.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.606+0000 7ffba3fff640 1 -- 172.21.15.112:0/1768550581 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ffb94004280 con 0x7ffbac0ccbb0 2024-09-21T00:29:51.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.607+0000 7ffba3fff640 1 --2- 172.21.15.112:0/1768550581 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ffb8c078b90 0x7ffb8c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:51.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.607+0000 7ffba3fff640 1 -- 172.21.15.112:0/1768550581 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 6101+0+0 (secure 0 0 0) 0x7ffb9409b650 con 0x7ffbac0ccbb0 2024-09-21T00:29:51.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.607+0000 7ffbaaffd640 1 --2- 172.21.15.112:0/1768550581 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ffb8c078b90 0x7ffb8c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:51.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.608+0000 7ffbaaffd640 1 --2- 172.21.15.112:0/1768550581 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ffb8c078b90 0x7ffb8c07b0a0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c001720 tx=0x7ffb9c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:51.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.611+0000 7ffba3fff640 1 -- 172.21.15.112:0/1768550581 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ffb94063340 con 0x7ffbac0ccbb0 2024-09-21T00:29:51.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.790+0000 7ffbb16a2640 1 -- 172.21.15.112:0/1768550581 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ffbac074640 con 0x7ffbac0ccbb0 2024-09-21T00:29:51.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.791+0000 7ffba3fff640 1 -- 172.21.15.112:0/1768550581 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v108) ==== 74+0+17036 (secure 0 0 0) 0x7ffb94067d70 con 0x7ffbac0ccbb0 2024-09-21T00:29:51.793 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:29:51.793 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":108,"fsid":"8b34692b-dc65-49b2-a08f-26f707b96ac5","created":"2024-09-21T00:25:22.788490+0000","modified":"2024-09-21T00:29:50.089526+0000","last_up_change":"2024-09-21T00:25:36.161835+0000","last_in_change":"2024-09-21T00:25:26.346500+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T00:25:37.105006+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-09-21T00:29:46.103570+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-09-21T00:29:47.420294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9c575086-67d2-4de2-b01b-d44dcc21d604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6809","nonce":755760621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6811","nonce":755760621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6815","nonce":755760621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6813","nonce":755760621}]},"public_addr":"172.21.15.112:6809/755760621","cluster_addr":"172.21.15.112:6811/755760621","heartbeat_back_addr":"172.21.15.112:6815/755760621","heartbeat_front_addr":"172.21.15.112:6813/755760621","state":["exists","up"]},{"osd":1,"uuid":"a983c227-a3ce-4bdd-82f1-438ba7e3ecd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6801","nonce":344932592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6803","nonce":344932592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6807","nonce":344932592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6805","nonce":344932592}]},"public_addr":"172.21.15.112:6801/344932592","cluster_addr":"172.21.15.112:6803/344932592","heartbeat_back_addr":"172.21.15.112:6807/344932592","heartbeat_front_addr":"172.21.15.112:6805/344932592","state":["exists","up"]},{"osd":2,"uuid":"1193c1d8-bfff-4a57-8200-5f69bf1ea3af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6825","nonce":407933616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6827","nonce":407933616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6831","nonce":407933616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6829","nonce":407933616}]},"public_addr":"172.21.15.112:6825/407933616","cluster_addr":"172.21.15.112:6827/407933616","heartbeat_back_addr":"172.21.15.112:6831/407933616","heartbeat_front_addr":"172.21.15.112:6829/407933616","state":["exists","up"]},{"osd":3,"uuid":"af04330a-36ad-40cc-a8dd-fa259b221de5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6817","nonce":4120164806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6819","nonce":4120164806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6823","nonce":4120164806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6821","nonce":4120164806}]},"public_addr":"172.21.15.112:6817/4120164806","cluster_addr":"172.21.15.112:6819/4120164806","heartbeat_back_addr":"172.21.15.112:6823/4120164806","heartbeat_front_addr":"172.21.15.112:6821/4120164806","state":["exists","up"]},{"osd":4,"uuid":"5b893111-292c-4c5a-b184-53ad38c7199e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6803","nonce":522077560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6805","nonce":522077560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6809","nonce":522077560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6807","nonce":522077560}]},"public_addr":"172.21.15.187:6803/522077560","cluster_addr":"172.21.15.187:6805/522077560","heartbeat_back_addr":"172.21.15.187:6809/522077560","heartbeat_front_addr":"172.21.15.187:6807/522077560","state":["exists","up"]},{"osd":5,"uuid":"c0b8e011-45fe-4c5e-925e-6be2a7f1868a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6827","nonce":3593144721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6829","nonce":3593144721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6833","nonce":3593144721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6831","nonce":3593144721}]},"public_addr":"172.21.15.187:6827/3593144721","cluster_addr":"172.21.15.187:6829/3593144721","heartbeat_back_addr":"172.21.15.187:6833/3593144721","heartbeat_front_addr":"172.21.15.187:6831/3593144721","state":["exists","up"]},{"osd":6,"uuid":"87d2d650-7489-4593-98a3-a1f433ce8dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6811","nonce":1494387657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6813","nonce":1494387657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6817","nonce":1494387657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6815","nonce":1494387657}]},"public_addr":"172.21.15.187:6811/1494387657","cluster_addr":"172.21.15.187:6813/1494387657","heartbeat_back_addr":"172.21.15.187:6817/1494387657","heartbeat_front_addr":"172.21.15.187:6815/1494387657","state":["exists","up"]},{"osd":7,"uuid":"c6b4f8e2-a1ee-4438-aa37-67ff4b6ef1f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6819","nonce":1624699842}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6821","nonce":1624699842}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6825","nonce":1624699842}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6823","nonce":1624699842}]},"public_addr":"172.21.15.187:6819/1624699842","cluster_addr":"172.21.15.187:6821/1624699842","heartbeat_back_addr":"172.21.15.187:6825/1624699842","heartbeat_front_addr":"172.21.15.187:6823/1624699842","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.873162+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.400506+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.664317+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.159791+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.202630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.113212+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.384786+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.448441+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T00:29:51.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.793+0000 7ffbb16a2640 1 -- 172.21.15.112:0/1768550581 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ffb8c078b90 msgr2=0x7ffb8c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:51.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.793+0000 7ffbb16a2640 1 --2- 172.21.15.112:0/1768550581 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ffb8c078b90 0x7ffb8c07b0a0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c001720 tx=0x7ffb9c009040 comp rx=0 tx=0).stop 2024-09-21T00:29:51.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.793+0000 7ffbb16a2640 1 -- 172.21.15.112:0/1768550581 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffbac0ccbb0 msgr2=0x7ffbac0cb020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:51.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.793+0000 7ffbb16a2640 1 --2- 172.21.15.112:0/1768550581 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffbac0ccbb0 0x7ffbac0cb020 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7ffb94009fd0 tx=0x7ffb9400ec20 comp rx=0 tx=0).stop 2024-09-21T00:29:51.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.793+0000 7ffbb16a2640 1 -- 172.21.15.112:0/1768550581 shutdown_connections 2024-09-21T00:29:51.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.793+0000 7ffbb16a2640 1 --2- 172.21.15.112:0/1768550581 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffbac15c5f0 0x7ffbac158210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:51.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.793+0000 7ffbb16a2640 1 --2- 172.21.15.112:0/1768550581 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ffb8c078b90 0x7ffb8c07b0a0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:51.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.793+0000 7ffbb16a2640 1 --2- 172.21.15.112:0/1768550581 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffbac0ccbb0 0x7ffbac0cb020 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:51.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.793+0000 7ffbb16a2640 1 --2- 172.21.15.112:0/1768550581 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffbac070210 0x7ffbac06dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:51.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.793+0000 7ffbb16a2640 1 -- 172.21.15.112:0/1768550581 >> 172.21.15.112:0/1768550581 conn(0x7ffbac073fe0 msgr2=0x7ffbac0cf820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:51.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.793+0000 7ffbb16a2640 1 -- 172.21.15.112:0/1768550581 shutdown_connections 2024-09-21T00:29:51.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.793+0000 7ffbb16a2640 1 -- 172.21.15.112:0/1768550581 wait complete. 2024-09-21T00:29:51.807 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T00:29:51.807 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T00:29:51.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.918+0000 7f2451894640 1 -- 172.21.15.112:0/2883273627 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f244c15c3c0 msgr2=0x7f244c0b3260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:51.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.918+0000 7f2451894640 1 --2- 172.21.15.112:0/2883273627 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f244c15c3c0 0x7f244c0b3260 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f244c0d17f0 tx=0x7f243c02a560 comp rx=0 tx=0).stop 2024-09-21T00:29:51.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.918+0000 7f2451894640 1 -- 172.21.15.112:0/2883273627 shutdown_connections 2024-09-21T00:29:51.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.918+0000 7f2451894640 1 --2- 172.21.15.112:0/2883273627 >> v2:172.21.15.187:3300/0 conn(0x7f244c0b37a0 0x7f244c0b6030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:51.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.918+0000 7f2451894640 1 --2- 172.21.15.112:0/2883273627 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f244c15c3c0 0x7f244c0b3260 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:51.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.918+0000 7f2451894640 1 -- 172.21.15.112:0/2883273627 >> 172.21.15.112:0/2883273627 conn(0x7f244c08dcb0 msgr2=0x7f244c09d8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:51.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.918+0000 7f2451894640 1 -- 172.21.15.112:0/2883273627 shutdown_connections 2024-09-21T00:29:51.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.918+0000 7f2451894640 1 -- 172.21.15.112:0/2883273627 wait complete. 2024-09-21T00:29:51.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.919+0000 7f2451894640 1 Processor -- start 2024-09-21T00:29:51.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.920+0000 7f2451894640 1 -- start start 2024-09-21T00:29:51.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.921+0000 7f2451894640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f244c0b37a0 0x7f244c0ce210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:51.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.921+0000 7f2451894640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f244c0b3bb0 0x7f244c1f5c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:51.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.921+0000 7f2451894640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f244c15c3c0 0x7f244c0b5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:51.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.921+0000 7f2451894640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f244c163180 con 0x7f244c0b37a0 2024-09-21T00:29:51.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.921+0000 7f2451894640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f244c0d0d30 con 0x7f244c0b3bb0 2024-09-21T00:29:51.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.921+0000 7f2451894640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f244c163300 con 0x7f244c15c3c0 2024-09-21T00:29:51.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.921+0000 7f244affd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f244c0b37a0 0x7f244c0ce210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:51.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.921+0000 7f244affd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f244c0b37a0 0x7f244c0ce210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:32810/0 (socket says 172.21.15.112:32810) 2024-09-21T00:29:51.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.921+0000 7f244affd640 1 -- 172.21.15.112:0/799175575 learned_addr learned my addr 172.21.15.112:0/799175575 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:29:51.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.921+0000 7f244a7fc640 1 --2- 172.21.15.112:0/799175575 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f244c0b3bb0 0x7f244c1f5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:51.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.921+0000 7f244b7fe640 1 --2- 172.21.15.112:0/799175575 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f244c15c3c0 0x7f244c0b5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:51.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.921+0000 7f244affd640 1 -- 172.21.15.112:0/799175575 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f244c15c3c0 msgr2=0x7f244c0b5fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:51.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.921+0000 7f244affd640 1 --2- 172.21.15.112:0/799175575 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f244c15c3c0 0x7f244c0b5fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:51.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.921+0000 7f244affd640 1 -- 172.21.15.112:0/799175575 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f244c0b3bb0 msgr2=0x7f244c1f5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:51.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.921+0000 7f244affd640 1 --2- 172.21.15.112:0/799175575 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f244c0b3bb0 0x7f244c1f5c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:51.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.921+0000 7f244affd640 1 -- 172.21.15.112:0/799175575 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f244c1f6990 con 0x7f244c0b37a0 2024-09-21T00:29:51.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.921+0000 7f244b7fe640 1 --2- 172.21.15.112:0/799175575 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f244c15c3c0 0x7f244c0b5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:29:51.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.921+0000 7f244affd640 1 --2- 172.21.15.112:0/799175575 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f244c0b37a0 0x7f244c0ce210 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f243c02ab50 tx=0x7f243c0052f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:51.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.922+0000 7f2433fff640 1 -- 172.21.15.112:0/799175575 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f243c005cf0 con 0x7f244c0b37a0 2024-09-21T00:29:51.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.922+0000 7f2433fff640 1 -- 172.21.15.112:0/799175575 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f243c031c40 con 0x7f244c0b37a0 2024-09-21T00:29:51.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.922+0000 7f2451894640 1 -- 172.21.15.112:0/799175575 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f244c14cef0 con 0x7f244c0b37a0 2024-09-21T00:29:51.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.922+0000 7f2433fff640 1 -- 172.21.15.112:0/799175575 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f243c033330 con 0x7f244c0b37a0 2024-09-21T00:29:51.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.922+0000 7f2451894640 1 -- 172.21.15.112:0/799175575 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f244c1f9e60 con 0x7f244c0b37a0 2024-09-21T00:29:51.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.923+0000 7f2451894640 1 -- 172.21.15.112:0/799175575 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f244c0b3260 con 0x7f244c0b37a0 2024-09-21T00:29:51.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.923+0000 7f2433fff640 1 -- 172.21.15.112:0/799175575 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f243c0334d0 con 0x7f244c0b37a0 2024-09-21T00:29:51.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.924+0000 7f2433fff640 1 --2- 172.21.15.112:0/799175575 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2438078b90 0x7f243807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:51.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.924+0000 7f2433fff640 1 -- 172.21.15.112:0/799175575 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 6101+0+0 (secure 0 0 0) 0x7f243c0bc1b0 con 0x7f244c0b37a0 2024-09-21T00:29:51.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.924+0000 7f244a7fc640 1 --2- 172.21.15.112:0/799175575 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2438078b90 0x7f243807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:51.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.925+0000 7f244a7fc640 1 --2- 172.21.15.112:0/799175575 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2438078b90 0x7f243807b0a0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f2440005fd0 tx=0x7f2440005850 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:51.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:51.929+0000 7f2433fff640 1 -- 172.21.15.112:0/799175575 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f243c03f030 con 0x7f244c0b37a0 2024-09-21T00:29:52.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.103+0000 7f2451894640 1 -- 172.21.15.112:0/799175575 --> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f244c0d0220 con 0x7f2438078b90 2024-09-21T00:29:52.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.107+0000 7f2433fff640 1 -- 172.21.15.112:0/799175575 <== mgr.4103 v2:172.21.15.187:6800/2921522906 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+238398 (secure 0 0 0) 0x7f244c0d0220 con 0x7f2438078b90 2024-09-21T00:29:52.111 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:29:52.112 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2024-09-21T00:29:52.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.110+0000 7f2451894640 1 -- 172.21.15.112:0/799175575 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2438078b90 msgr2=0x7f243807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:52.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.110+0000 7f2451894640 1 --2- 172.21.15.112:0/799175575 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2438078b90 0x7f243807b0a0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f2440005fd0 tx=0x7f2440005850 comp rx=0 tx=0).stop 2024-09-21T00:29:52.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.110+0000 7f2451894640 1 -- 172.21.15.112:0/799175575 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f244c0b37a0 msgr2=0x7f244c0ce210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:52.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.110+0000 7f2451894640 1 --2- 172.21.15.112:0/799175575 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f244c0b37a0 0x7f244c0ce210 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f243c02ab50 tx=0x7f243c0052f0 comp rx=0 tx=0).stop 2024-09-21T00:29:52.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.110+0000 7f2451894640 1 -- 172.21.15.112:0/799175575 shutdown_connections 2024-09-21T00:29:52.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.110+0000 7f2451894640 1 --2- 172.21.15.112:0/799175575 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f244c15c3c0 0x7f244c0b5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:52.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.110+0000 7f2451894640 1 --2- 172.21.15.112:0/799175575 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f244c0b3bb0 0x7f244c1f5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:52.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.110+0000 7f2451894640 1 --2- 172.21.15.112:0/799175575 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2438078b90 0x7f243807b0a0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:52.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.110+0000 7f2451894640 1 --2- 172.21.15.112:0/799175575 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f244c0b37a0 0x7f244c0ce210 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:52.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.110+0000 7f2451894640 1 -- 172.21.15.112:0/799175575 >> 172.21.15.112:0/799175575 conn(0x7f244c08dcb0 msgr2=0x7f244c0d43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:52.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.110+0000 7f2451894640 1 -- 172.21.15.112:0/799175575 shutdown_connections 2024-09-21T00:29:52.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.110+0000 7f2451894640 1 -- 172.21.15.112:0/799175575 wait complete. 2024-09-21T00:29:52.125 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":230,"stamp":"2024-09-21T00:29:51.152365+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":3,"kb":754974720,"kb_used":260828,"kb_used_data":11264,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754713892,"statfs":{"total":773094113280,"available":772827025408,"internally_reserved":0,"allocated":11534336,"data_stored":5875548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.071150"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-09-21T00:29:48.098942+0000","last_change":"2024-09-21T00:29:48.098942+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.098942+0000","last_undegraded":"2024-09-21T00:29:48.098942+0000","last_fullsized":"2024-09-21T00:29:48.098942+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-09-21T00:29:48.099873+0000","last_change":"2024-09-21T00:29:48.099873+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.099873+0000","last_undegraded":"2024-09-21T00:29:48.099873+0000","last_fullsized":"2024-09-21T00:29:48.099873+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-09-21T00:29:48.097589+0000","last_change":"2024-09-21T00:29:48.097589+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.097589+0000","last_undegraded":"2024-09-21T00:29:48.097589+0000","last_fullsized":"2024-09-21T00:29:48.097589+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-09-21T00:29:48.097051+0000","last_change":"2024-09-21T00:29:48.097051+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.097051+0000","last_undegraded":"2024-09-21T00:29:48.097051+0000","last_fullsized":"2024-09-21T00:29:48.097051+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:47.077435+0000","last_change":"2024-09-21T00:29:47.077435+0000","last_active":"2024-09-21T00:29:47.077435+0000","last_peered":"2024-09-21T00:29:47.077435+0000","last_clean":"2024-09-21T00:29:47.077435+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:47.077435+0000","last_undegraded":"2024-09-21T00:29:47.077435+0000","last_fullsized":"2024-09-21T00:29:47.077435+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-21T00:29:48.105694+0000","last_change":"2024-09-21T00:29:48.102532+0000","last_active":"2024-09-21T00:29:48.105694+0000","last_peered":"2024-09-21T00:29:48.105694+0000","last_clean":"2024-09-21T00:29:48.105694+0000","last_became_active":"2024-09-21T00:29:48.102195+0000","last_became_peered":"2024-09-21T00:29:48.102195+0000","last_unstale":"2024-09-21T00:29:48.105694+0000","last_undegraded":"2024-09-21T00:29:48.105694+0000","last_fullsized":"2024-09-21T00:29:48.105694+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:41:20.768969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:47.077435+0000","last_change":"2024-09-21T00:29:47.077435+0000","last_active":"2024-09-21T00:29:47.077435+0000","last_peered":"2024-09-21T00:29:47.077435+0000","last_clean":"2024-09-21T00:29:47.077435+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:47.077435+0000","last_undegraded":"2024-09-21T00:29:47.077435+0000","last_fullsized":"2024-09-21T00:29:47.077435+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-21T00:29:48.107959+0000","last_change":"2024-09-21T00:29:48.103570+0000","last_active":"2024-09-21T00:29:48.107959+0000","last_peered":"2024-09-21T00:29:48.107959+0000","last_clean":"2024-09-21T00:29:48.107959+0000","last_became_active":"2024-09-21T00:29:48.103262+0000","last_became_peered":"2024-09-21T00:29:48.103262+0000","last_unstale":"2024-09-21T00:29:48.107959+0000","last_undegraded":"2024-09-21T00:29:48.107959+0000","last_fullsized":"2024-09-21T00:29:48.107959+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:19:55.232168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-09-21T00:29:48.094970+0000","last_change":"2024-09-21T00:29:48.094970+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.094970+0000","last_undegraded":"2024-09-21T00:29:48.094970+0000","last_fullsized":"2024-09-21T00:29:48.094970+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:47.077435+0000","last_change":"2024-09-21T00:29:47.077435+0000","last_active":"2024-09-21T00:29:47.077435+0000","last_peered":"2024-09-21T00:29:47.077435+0000","last_clean":"2024-09-21T00:29:47.077435+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:47.077435+0000","last_undegraded":"2024-09-21T00:29:47.077435+0000","last_fullsized":"2024-09-21T00:29:47.077435+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:47.077435+0000","last_change":"2024-09-21T00:29:47.077435+0000","last_active":"2024-09-21T00:29:47.077435+0000","last_peered":"2024-09-21T00:29:47.077435+0000","last_clean":"2024-09-21T00:29:47.077435+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:47.077435+0000","last_undegraded":"2024-09-21T00:29:47.077435+0000","last_fullsized":"2024-09-21T00:29:47.077435+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:47.077435+0000","last_change":"2024-09-21T00:29:47.077435+0000","last_active":"2024-09-21T00:29:47.077435+0000","last_peered":"2024-09-21T00:29:47.077435+0000","last_clean":"2024-09-21T00:29:47.077435+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:47.077435+0000","last_undegraded":"2024-09-21T00:29:47.077435+0000","last_fullsized":"2024-09-21T00:29:47.077435+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-09-21T00:29:48.096433+0000","last_change":"2024-09-21T00:29:48.096433+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.096433+0000","last_undegraded":"2024-09-21T00:29:48.096433+0000","last_fullsized":"2024-09-21T00:29:48.096433+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:47.077435+0000","last_change":"2024-09-21T00:29:47.077435+0000","last_active":"2024-09-21T00:29:47.077435+0000","last_peered":"2024-09-21T00:29:47.077435+0000","last_clean":"2024-09-21T00:29:47.077435+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:47.077435+0000","last_undegraded":"2024-09-21T00:29:47.077435+0000","last_fullsized":"2024-09-21T00:29:47.077435+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:47.077435+0000","last_change":"2024-09-21T00:29:47.077435+0000","last_active":"2024-09-21T00:29:47.077435+0000","last_peered":"2024-09-21T00:29:47.077435+0000","last_clean":"2024-09-21T00:29:47.077435+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:47.077435+0000","last_undegraded":"2024-09-21T00:29:47.077435+0000","last_fullsized":"2024-09-21T00:29:47.077435+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:47.077435+0000","last_change":"2024-09-21T00:29:47.077435+0000","last_active":"2024-09-21T00:29:47.077435+0000","last_peered":"2024-09-21T00:29:47.077435+0000","last_clean":"2024-09-21T00:29:47.077435+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:47.077435+0000","last_undegraded":"2024-09-21T00:29:47.077435+0000","last_fullsized":"2024-09-21T00:29:47.077435+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:47.077435+0000","last_change":"2024-09-21T00:29:47.077435+0000","last_active":"2024-09-21T00:29:47.077435+0000","last_peered":"2024-09-21T00:29:47.077435+0000","last_clean":"2024-09-21T00:29:47.077435+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:47.077435+0000","last_undegraded":"2024-09-21T00:29:47.077435+0000","last_fullsized":"2024-09-21T00:29:47.077435+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:47.077435+0000","last_change":"2024-09-21T00:29:47.077435+0000","last_active":"2024-09-21T00:29:47.077435+0000","last_peered":"2024-09-21T00:29:47.077435+0000","last_clean":"2024-09-21T00:29:47.077435+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:47.077435+0000","last_undegraded":"2024-09-21T00:29:47.077435+0000","last_fullsized":"2024-09-21T00:29:47.077435+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:47.077435+0000","last_change":"2024-09-21T00:29:47.077435+0000","last_active":"2024-09-21T00:29:47.077435+0000","last_peered":"2024-09-21T00:29:47.077435+0000","last_clean":"2024-09-21T00:29:47.077435+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:47.077435+0000","last_undegraded":"2024-09-21T00:29:47.077435+0000","last_fullsized":"2024-09-21T00:29:47.077435+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-09-21T00:29:48.095354+0000","last_change":"2024-09-21T00:29:48.095354+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.095354+0000","last_undegraded":"2024-09-21T00:29:48.095354+0000","last_fullsized":"2024-09-21T00:29:48.095354+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:47.077435+0000","last_change":"2024-09-21T00:29:47.077435+0000","last_active":"2024-09-21T00:29:47.077435+0000","last_peered":"2024-09-21T00:29:47.077435+0000","last_clean":"2024-09-21T00:29:47.077435+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:47.077435+0000","last_undegraded":"2024-09-21T00:29:47.077435+0000","last_fullsized":"2024-09-21T00:29:47.077435+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:47.077435+0000","last_change":"2024-09-21T00:29:47.077435+0000","last_active":"2024-09-21T00:29:47.077435+0000","last_peered":"2024-09-21T00:29:47.077435+0000","last_clean":"2024-09-21T00:29:47.077435+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:47.077435+0000","last_undegraded":"2024-09-21T00:29:47.077435+0000","last_fullsized":"2024-09-21T00:29:47.077435+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-21T00:29:48.108007+0000","last_change":"2024-09-21T00:29:48.104276+0000","last_active":"2024-09-21T00:29:48.108007+0000","last_peered":"2024-09-21T00:29:48.108007+0000","last_clean":"2024-09-21T00:29:48.108007+0000","last_became_active":"2024-09-21T00:29:48.103972+0000","last_became_peered":"2024-09-21T00:29:48.103972+0000","last_unstale":"2024-09-21T00:29:48.108007+0000","last_undegraded":"2024-09-21T00:29:48.108007+0000","last_fullsized":"2024-09-21T00:29:48.108007+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:48:50.178516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:47.077435+0000","last_change":"2024-09-21T00:29:47.077435+0000","last_active":"2024-09-21T00:29:47.077435+0000","last_peered":"2024-09-21T00:29:47.077435+0000","last_clean":"2024-09-21T00:29:47.077435+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:47.077435+0000","last_undegraded":"2024-09-21T00:29:47.077435+0000","last_fullsized":"2024-09-21T00:29:47.077435+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:47.077435+0000","last_change":"2024-09-21T00:29:47.077435+0000","last_active":"2024-09-21T00:29:47.077435+0000","last_peered":"2024-09-21T00:29:47.077435+0000","last_clean":"2024-09-21T00:29:47.077435+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:47.077435+0000","last_undegraded":"2024-09-21T00:29:47.077435+0000","last_fullsized":"2024-09-21T00:29:47.077435+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:47.077435+0000","last_change":"2024-09-21T00:29:47.077435+0000","last_active":"2024-09-21T00:29:47.077435+0000","last_peered":"2024-09-21T00:29:47.077435+0000","last_clean":"2024-09-21T00:29:47.077435+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:47.077435+0000","last_undegraded":"2024-09-21T00:29:47.077435+0000","last_fullsized":"2024-09-21T00:29:47.077435+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:47.077435+0000","last_change":"2024-09-21T00:29:47.077435+0000","last_active":"2024-09-21T00:29:47.077435+0000","last_peered":"2024-09-21T00:29:47.077435+0000","last_clean":"2024-09-21T00:29:47.077435+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:47.077435+0000","last_undegraded":"2024-09-21T00:29:47.077435+0000","last_fullsized":"2024-09-21T00:29:47.077435+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:47.077435+0000","last_change":"2024-09-21T00:29:47.077435+0000","last_active":"2024-09-21T00:29:47.077435+0000","last_peered":"2024-09-21T00:29:47.077435+0000","last_clean":"2024-09-21T00:29:47.077435+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:47.077435+0000","last_undegraded":"2024-09-21T00:29:47.077435+0000","last_fullsized":"2024-09-21T00:29:47.077435+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:47.077435+0000","last_change":"2024-09-21T00:29:47.077435+0000","last_active":"2024-09-21T00:29:47.077435+0000","last_peered":"2024-09-21T00:29:47.077435+0000","last_clean":"2024-09-21T00:29:47.077435+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:47.077435+0000","last_undegraded":"2024-09-21T00:29:47.077435+0000","last_fullsized":"2024-09-21T00:29:47.077435+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:47.077435+0000","last_change":"2024-09-21T00:29:47.077435+0000","last_active":"2024-09-21T00:29:47.077435+0000","last_peered":"2024-09-21T00:29:47.077435+0000","last_clean":"2024-09-21T00:29:47.077435+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:47.077435+0000","last_undegraded":"2024-09-21T00:29:47.077435+0000","last_fullsized":"2024-09-21T00:29:47.077435+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:47.077435+0000","last_change":"2024-09-21T00:29:47.077435+0000","last_active":"2024-09-21T00:29:47.077435+0000","last_peered":"2024-09-21T00:29:47.077435+0000","last_clean":"2024-09-21T00:29:47.077435+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:47.077435+0000","last_undegraded":"2024-09-21T00:29:47.077435+0000","last_fullsized":"2024-09-21T00:29:47.077435+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-09-21T00:29:48.095849+0000","last_change":"2024-09-21T00:29:48.095849+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.095849+0000","last_undegraded":"2024-09-21T00:29:48.095849+0000","last_fullsized":"2024-09-21T00:29:48.095849+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:47.077435+0000","last_change":"2024-09-21T00:29:47.077435+0000","last_active":"2024-09-21T00:29:47.077435+0000","last_peered":"2024-09-21T00:29:47.077435+0000","last_clean":"2024-09-21T00:29:47.077435+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:47.077435+0000","last_undegraded":"2024-09-21T00:29:47.077435+0000","last_fullsized":"2024-09-21T00:29:47.077435+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:47.077435+0000","last_change":"2024-09-21T00:29:47.077435+0000","last_active":"2024-09-21T00:29:47.077435+0000","last_peered":"2024-09-21T00:29:47.077435+0000","last_clean":"2024-09-21T00:29:47.077435+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:47.077435+0000","last_undegraded":"2024-09-21T00:29:47.077435+0000","last_fullsized":"2024-09-21T00:29:47.077435+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:47.077435+0000","last_change":"2024-09-21T00:29:47.077435+0000","last_active":"2024-09-21T00:29:47.077435+0000","last_peered":"2024-09-21T00:29:47.077435+0000","last_clean":"2024-09-21T00:29:47.077435+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:47.077435+0000","last_undegraded":"2024-09-21T00:29:47.077435+0000","last_fullsized":"2024-09-21T00:29:47.077435+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:47.077435+0000","last_change":"2024-09-21T00:29:47.077435+0000","last_active":"2024-09-21T00:29:47.077435+0000","last_peered":"2024-09-21T00:29:47.077435+0000","last_clean":"2024-09-21T00:29:47.077435+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:47.077435+0000","last_undegraded":"2024-09-21T00:29:47.077435+0000","last_fullsized":"2024-09-21T00:29:47.077435+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:47.077435+0000","last_change":"2024-09-21T00:29:47.077435+0000","last_active":"2024-09-21T00:29:47.077435+0000","last_peered":"2024-09-21T00:29:47.077435+0000","last_clean":"2024-09-21T00:29:47.077435+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:47.077435+0000","last_undegraded":"2024-09-21T00:29:47.077435+0000","last_fullsized":"2024-09-21T00:29:47.077435+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:47.077435+0000","last_change":"2024-09-21T00:29:47.077435+0000","last_active":"2024-09-21T00:29:47.077435+0000","last_peered":"2024-09-21T00:29:47.077435+0000","last_clean":"2024-09-21T00:29:47.077435+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:47.077435+0000","last_undegraded":"2024-09-21T00:29:47.077435+0000","last_fullsized":"2024-09-21T00:29:47.077435+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-09-21T00:29:48.095876+0000","last_change":"2024-09-21T00:29:48.095876+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.095876+0000","last_undegraded":"2024-09-21T00:29:48.095876+0000","last_fullsized":"2024-09-21T00:29:48.095876+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:47.077435+0000","last_change":"2024-09-21T00:29:47.077435+0000","last_active":"2024-09-21T00:29:47.077435+0000","last_peered":"2024-09-21T00:29:47.077435+0000","last_clean":"2024-09-21T00:29:47.077435+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:47.077435+0000","last_undegraded":"2024-09-21T00:29:47.077435+0000","last_fullsized":"2024-09-21T00:29:47.077435+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:47.077435+0000","last_change":"2024-09-21T00:29:47.077435+0000","last_active":"2024-09-21T00:29:47.077435+0000","last_peered":"2024-09-21T00:29:47.077435+0000","last_clean":"2024-09-21T00:29:47.077435+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:47.077435+0000","last_undegraded":"2024-09-21T00:29:47.077435+0000","last_fullsized":"2024-09-21T00:29:47.077435+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":263,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T00:29:45.425002+0000","last_change":"2024-09-21T00:25:38.725792+0000","last_active":"2024-09-21T00:29:45.425002+0000","last_peered":"2024-09-21T00:29:45.425002+0000","last_clean":"2024-09-21T00:29:45.425002+0000","last_became_active":"2024-09-21T00:25:38.724668+0000","last_became_peered":"2024-09-21T00:25:38.724668+0000","last_unstale":"2024-09-21T00:29:45.425002+0000","last_undegraded":"2024-09-21T00:29:45.425002+0000","last_fullsized":"2024-09-21T00:29:45.425002+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:37.163216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:37.163216+0000","last_clean_scrub_stamp":"2024-09-21T00:25:37.163216+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:14:36.588660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":18,"acting":18,"num_store_stats":1},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":6,"acting":6,"num_store_stats":1},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509493,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33032,"kb_used_data":1840,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338808,"statfs":{"total":96636764160,"available":96602939392,"internally_reserved":0,"allocated":1884160,"data_stored":1175610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509494,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32452,"kb_used_data":1260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339388,"statfs":{"total":96636764160,"available":96603533312,"internally_reserved":0,"allocated":1290240,"data_stored":585242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509493,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32460,"kb_used_data":1260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339380,"statfs":{"total":96636764160,"available":96603525120,"internally_reserved":0,"allocated":1290240,"data_stored":585242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509492,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32464,"kb_used_data":1260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339376,"statfs":{"total":96636764160,"available":96603521024,"internally_reserved":0,"allocated":1290240,"data_stored":585242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509494,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32456,"kb_used_data":1260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339384,"statfs":{"total":96636764160,"available":96603529216,"internally_reserved":0,"allocated":1290240,"data_stored":585242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509494,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32452,"kb_used_data":1260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339388,"statfs":{"total":96636764160,"available":96603533312,"internally_reserved":0,"allocated":1290240,"data_stored":585242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509494,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33036,"kb_used_data":1840,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338804,"statfs":{"total":96636764160,"available":96602935296,"internally_reserved":0,"allocated":1884160,"data_stored":1175610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574902,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32476,"kb_used_data":1284,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339364,"statfs":{"total":96636764160,"available":96603508736,"internally_reserved":0,"allocated":1314816,"data_stored":598118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T00:29:52.131 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T00:29:52.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.274+0000 7f0456194640 1 -- 172.21.15.112:0/3094902966 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04500b1d30 msgr2=0x7f04500b2140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:52.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.274+0000 7f0456194640 1 --2- 172.21.15.112:0/3094902966 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04500b1d30 0x7f04500b2140 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f0444009830 tx=0x7f044402a3f0 comp rx=0 tx=0).stop 2024-09-21T00:29:52.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.274+0000 7f0456194640 1 -- 172.21.15.112:0/3094902966 shutdown_connections 2024-09-21T00:29:52.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.274+0000 7f0456194640 1 --2- 172.21.15.112:0/3094902966 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04500b1d30 0x7f04500b2140 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:52.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.274+0000 7f0456194640 1 -- 172.21.15.112:0/3094902966 >> 172.21.15.112:0/3094902966 conn(0x7f04500b42e0 msgr2=0x7f04500b4720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:52.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.274+0000 7f0456194640 1 -- 172.21.15.112:0/3094902966 shutdown_connections 2024-09-21T00:29:52.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.274+0000 7f0456194640 1 -- 172.21.15.112:0/3094902966 wait complete. 2024-09-21T00:29:52.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.275+0000 7f0456194640 1 Processor -- start 2024-09-21T00:29:52.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.276+0000 7f0456194640 1 -- start start 2024-09-21T00:29:52.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.277+0000 7f0456194640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04500b1d30 0x7f045015b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:52.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.277+0000 7f0456194640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04500b2680 0x7f045015b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:52.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.277+0000 7f0456194640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04500ccbf0 0x7f04501f5db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:52.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.277+0000 7f0456194640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f04500b3670 con 0x7f04500b2680 2024-09-21T00:29:52.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.277+0000 7f0456194640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f04500b34f0 con 0x7f04500b1d30 2024-09-21T00:29:52.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.277+0000 7f0456194640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f0450167ed0 con 0x7f04500ccbf0 2024-09-21T00:29:52.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.277+0000 7f044effd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04500b2680 0x7f045015b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:52.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.277+0000 7f044effd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04500b2680 0x7f045015b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:32826/0 (socket says 172.21.15.112:32826) 2024-09-21T00:29:52.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.277+0000 7f044effd640 1 -- 172.21.15.112:0/2642995627 learned_addr learned my addr 172.21.15.112:0/2642995627 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:29:52.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.277+0000 7f044ffff640 1 --2- 172.21.15.112:0/2642995627 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04500ccbf0 0x7f04501f5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:52.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.277+0000 7f044f7fe640 1 --2- 172.21.15.112:0/2642995627 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04500b1d30 0x7f045015b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:52.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.277+0000 7f044effd640 1 -- 172.21.15.112:0/2642995627 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04500ccbf0 msgr2=0x7f04501f5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:52.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.277+0000 7f044effd640 1 --2- 172.21.15.112:0/2642995627 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04500ccbf0 0x7f04501f5db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:52.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.277+0000 7f044effd640 1 -- 172.21.15.112:0/2642995627 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04500b1d30 msgr2=0x7f045015b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:52.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.277+0000 7f044effd640 1 --2- 172.21.15.112:0/2642995627 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04500b1d30 0x7f045015b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:52.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.277+0000 7f044effd640 1 -- 172.21.15.112:0/2642995627 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0450155bc0 con 0x7f04500b2680 2024-09-21T00:29:52.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.277+0000 7f044ffff640 1 --2- 172.21.15.112:0/2642995627 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04500ccbf0 0x7f04501f5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:29:52.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.277+0000 7f044f7fe640 1 --2- 172.21.15.112:0/2642995627 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04500b1d30 0x7f045015b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:29:52.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.278+0000 7f044effd640 1 --2- 172.21.15.112:0/2642995627 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04500b2680 0x7f045015b9f0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f0444009950 tx=0x7f0444033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:52.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.278+0000 7f0437fff640 1 -- 172.21.15.112:0/2642995627 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0444031d60 con 0x7f04500b2680 2024-09-21T00:29:52.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.278+0000 7f0437fff640 1 -- 172.21.15.112:0/2642995627 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0444033e10 con 0x7f04500b2680 2024-09-21T00:29:52.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.278+0000 7f0456194640 1 -- 172.21.15.112:0/2642995627 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0450155600 con 0x7f04500b2680 2024-09-21T00:29:52.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.278+0000 7f0437fff640 1 -- 172.21.15.112:0/2642995627 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0444005260 con 0x7f04500b2680 2024-09-21T00:29:52.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.278+0000 7f0456194640 1 -- 172.21.15.112:0/2642995627 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f045006eca0 con 0x7f04500b2680 2024-09-21T00:29:52.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.279+0000 7f0456194640 1 -- 172.21.15.112:0/2642995627 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04500b2140 con 0x7f04500b2680 2024-09-21T00:29:52.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.280+0000 7f0437fff640 1 -- 172.21.15.112:0/2642995627 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0444005400 con 0x7f04500b2680 2024-09-21T00:29:52.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.280+0000 7f0437fff640 1 --2- 172.21.15.112:0/2642995627 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f043c078bc0 0x7f043c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:52.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.280+0000 7f0437fff640 1 -- 172.21.15.112:0/2642995627 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 6101+0+0 (secure 0 0 0) 0x7f04440bc560 con 0x7f04500b2680 2024-09-21T00:29:52.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.280+0000 7f044f7fe640 1 --2- 172.21.15.112:0/2642995627 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f043c078bc0 0x7f043c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:52.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.281+0000 7f044f7fe640 1 --2- 172.21.15.112:0/2642995627 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f043c078bc0 0x7f043c07b0d0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f044000a830 tx=0x7f0440008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:52.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.284+0000 7f0437fff640 1 -- 172.21.15.112:0/2642995627 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f044403f030 con 0x7f04500b2680 2024-09-21T00:29:52.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.457+0000 7f0456194640 1 -- 172.21.15.112:0/2642995627 --> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0450072770 con 0x7f043c078bc0 2024-09-21T00:29:52.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.461+0000 7f0437fff640 1 -- 172.21.15.112:0/2642995627 <== mgr.4103 v2:172.21.15.187:6800/2921522906 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+238398 (secure 0 0 0) 0x7f0450072770 con 0x7f043c078bc0 2024-09-21T00:29:52.465 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:29:52.466 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2024-09-21T00:29:52.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.464+0000 7f0456194640 1 -- 172.21.15.112:0/2642995627 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f043c078bc0 msgr2=0x7f043c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:52.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.464+0000 7f0456194640 1 --2- 172.21.15.112:0/2642995627 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f043c078bc0 0x7f043c07b0d0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f044000a830 tx=0x7f0440008040 comp rx=0 tx=0).stop 2024-09-21T00:29:52.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.464+0000 7f0456194640 1 -- 172.21.15.112:0/2642995627 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04500b2680 msgr2=0x7f045015b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:52.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.464+0000 7f0456194640 1 --2- 172.21.15.112:0/2642995627 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04500b2680 0x7f045015b9f0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f0444009950 tx=0x7f0444033c20 comp rx=0 tx=0).stop 2024-09-21T00:29:52.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.464+0000 7f0456194640 1 -- 172.21.15.112:0/2642995627 shutdown_connections 2024-09-21T00:29:52.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.464+0000 7f0456194640 1 --2- 172.21.15.112:0/2642995627 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04500ccbf0 0x7f04501f5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:52.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.464+0000 7f0456194640 1 --2- 172.21.15.112:0/2642995627 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f043c078bc0 0x7f043c07b0d0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:52.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.465+0000 7f0456194640 1 --2- 172.21.15.112:0/2642995627 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04500b2680 0x7f045015b9f0 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:52.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.465+0000 7f0456194640 1 --2- 172.21.15.112:0/2642995627 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04500b1d30 0x7f045015b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:52.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.465+0000 7f0456194640 1 -- 172.21.15.112:0/2642995627 >> 172.21.15.112:0/2642995627 conn(0x7f04500b42e0 msgr2=0x7f04500b54a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:52.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.465+0000 7f0456194640 1 -- 172.21.15.112:0/2642995627 shutdown_connections 2024-09-21T00:29:52.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.465+0000 7f0456194640 1 -- 172.21.15.112:0/2642995627 wait complete. 2024-09-21T00:29:52.477 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":230,"stamp":"2024-09-21T00:29:51.152365+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":3,"kb":754974720,"kb_used":260828,"kb_used_data":11264,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754713892,"statfs":{"total":773094113280,"available":772827025408,"internally_reserved":0,"allocated":11534336,"data_stored":5875548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.071150"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-09-21T00:29:48.098942+0000","last_change":"2024-09-21T00:29:48.098942+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.098942+0000","last_undegraded":"2024-09-21T00:29:48.098942+0000","last_fullsized":"2024-09-21T00:29:48.098942+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-09-21T00:29:48.099873+0000","last_change":"2024-09-21T00:29:48.099873+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.099873+0000","last_undegraded":"2024-09-21T00:29:48.099873+0000","last_fullsized":"2024-09-21T00:29:48.099873+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-09-21T00:29:48.097589+0000","last_change":"2024-09-21T00:29:48.097589+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.097589+0000","last_undegraded":"2024-09-21T00:29:48.097589+0000","last_fullsized":"2024-09-21T00:29:48.097589+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-09-21T00:29:48.097051+0000","last_change":"2024-09-21T00:29:48.097051+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.097051+0000","last_undegraded":"2024-09-21T00:29:48.097051+0000","last_fullsized":"2024-09-21T00:29:48.097051+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:47.077435+0000","last_change":"2024-09-21T00:29:47.077435+0000","last_active":"2024-09-21T00:29:47.077435+0000","last_peered":"2024-09-21T00:29:47.077435+0000","last_clean":"2024-09-21T00:29:47.077435+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:47.077435+0000","last_undegraded":"2024-09-21T00:29:47.077435+0000","last_fullsized":"2024-09-21T00:29:47.077435+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-21T00:29:48.105694+0000","last_change":"2024-09-21T00:29:48.102532+0000","last_active":"2024-09-21T00:29:48.105694+0000","last_peered":"2024-09-21T00:29:48.105694+0000","last_clean":"2024-09-21T00:29:48.105694+0000","last_became_active":"2024-09-21T00:29:48.102195+0000","last_became_peered":"2024-09-21T00:29:48.102195+0000","last_unstale":"2024-09-21T00:29:48.105694+0000","last_undegraded":"2024-09-21T00:29:48.105694+0000","last_fullsized":"2024-09-21T00:29:48.105694+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:41:20.768969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:47.077435+0000","last_change":"2024-09-21T00:29:47.077435+0000","last_active":"2024-09-21T00:29:47.077435+0000","last_peered":"2024-09-21T00:29:47.077435+0000","last_clean":"2024-09-21T00:29:47.077435+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:47.077435+0000","last_undegraded":"2024-09-21T00:29:47.077435+0000","last_fullsized":"2024-09-21T00:29:47.077435+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-21T00:29:48.107959+0000","last_change":"2024-09-21T00:29:48.103570+0000","last_active":"2024-09-21T00:29:48.107959+0000","last_peered":"2024-09-21T00:29:48.107959+0000","last_clean":"2024-09-21T00:29:48.107959+0000","last_became_active":"2024-09-21T00:29:48.103262+0000","last_became_peered":"2024-09-21T00:29:48.103262+0000","last_unstale":"2024-09-21T00:29:48.107959+0000","last_undegraded":"2024-09-21T00:29:48.107959+0000","last_fullsized":"2024-09-21T00:29:48.107959+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:19:55.232168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-09-21T00:29:48.094970+0000","last_change":"2024-09-21T00:29:48.094970+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.094970+0000","last_undegraded":"2024-09-21T00:29:48.094970+0000","last_fullsized":"2024-09-21T00:29:48.094970+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:47.077435+0000","last_change":"2024-09-21T00:29:47.077435+0000","last_active":"2024-09-21T00:29:47.077435+0000","last_peered":"2024-09-21T00:29:47.077435+0000","last_clean":"2024-09-21T00:29:47.077435+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:47.077435+0000","last_undegraded":"2024-09-21T00:29:47.077435+0000","last_fullsized":"2024-09-21T00:29:47.077435+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:47.077435+0000","last_change":"2024-09-21T00:29:47.077435+0000","last_active":"2024-09-21T00:29:47.077435+0000","last_peered":"2024-09-21T00:29:47.077435+0000","last_clean":"2024-09-21T00:29:47.077435+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:47.077435+0000","last_undegraded":"2024-09-21T00:29:47.077435+0000","last_fullsized":"2024-09-21T00:29:47.077435+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:47.077435+0000","last_change":"2024-09-21T00:29:47.077435+0000","last_active":"2024-09-21T00:29:47.077435+0000","last_peered":"2024-09-21T00:29:47.077435+0000","last_clean":"2024-09-21T00:29:47.077435+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:47.077435+0000","last_undegraded":"2024-09-21T00:29:47.077435+0000","last_fullsized":"2024-09-21T00:29:47.077435+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-09-21T00:29:48.096433+0000","last_change":"2024-09-21T00:29:48.096433+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.096433+0000","last_undegraded":"2024-09-21T00:29:48.096433+0000","last_fullsized":"2024-09-21T00:29:48.096433+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:47.077435+0000","last_change":"2024-09-21T00:29:47.077435+0000","last_active":"2024-09-21T00:29:47.077435+0000","last_peered":"2024-09-21T00:29:47.077435+0000","last_clean":"2024-09-21T00:29:47.077435+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:47.077435+0000","last_undegraded":"2024-09-21T00:29:47.077435+0000","last_fullsized":"2024-09-21T00:29:47.077435+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:47.077435+0000","last_change":"2024-09-21T00:29:47.077435+0000","last_active":"2024-09-21T00:29:47.077435+0000","last_peered":"2024-09-21T00:29:47.077435+0000","last_clean":"2024-09-21T00:29:47.077435+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:47.077435+0000","last_undegraded":"2024-09-21T00:29:47.077435+0000","last_fullsized":"2024-09-21T00:29:47.077435+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:47.077435+0000","last_change":"2024-09-21T00:29:47.077435+0000","last_active":"2024-09-21T00:29:47.077435+0000","last_peered":"2024-09-21T00:29:47.077435+0000","last_clean":"2024-09-21T00:29:47.077435+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:47.077435+0000","last_undegraded":"2024-09-21T00:29:47.077435+0000","last_fullsized":"2024-09-21T00:29:47.077435+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:47.077435+0000","last_change":"2024-09-21T00:29:47.077435+0000","last_active":"2024-09-21T00:29:47.077435+0000","last_peered":"2024-09-21T00:29:47.077435+0000","last_clean":"2024-09-21T00:29:47.077435+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:47.077435+0000","last_undegraded":"2024-09-21T00:29:47.077435+0000","last_fullsized":"2024-09-21T00:29:47.077435+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:47.077435+0000","last_change":"2024-09-21T00:29:47.077435+0000","last_active":"2024-09-21T00:29:47.077435+0000","last_peered":"2024-09-21T00:29:47.077435+0000","last_clean":"2024-09-21T00:29:47.077435+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:47.077435+0000","last_undegraded":"2024-09-21T00:29:47.077435+0000","last_fullsized":"2024-09-21T00:29:47.077435+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:47.077435+0000","last_change":"2024-09-21T00:29:47.077435+0000","last_active":"2024-09-21T00:29:47.077435+0000","last_peered":"2024-09-21T00:29:47.077435+0000","last_clean":"2024-09-21T00:29:47.077435+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:47.077435+0000","last_undegraded":"2024-09-21T00:29:47.077435+0000","last_fullsized":"2024-09-21T00:29:47.077435+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-09-21T00:29:48.095354+0000","last_change":"2024-09-21T00:29:48.095354+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.095354+0000","last_undegraded":"2024-09-21T00:29:48.095354+0000","last_fullsized":"2024-09-21T00:29:48.095354+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:47.077435+0000","last_change":"2024-09-21T00:29:47.077435+0000","last_active":"2024-09-21T00:29:47.077435+0000","last_peered":"2024-09-21T00:29:47.077435+0000","last_clean":"2024-09-21T00:29:47.077435+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:47.077435+0000","last_undegraded":"2024-09-21T00:29:47.077435+0000","last_fullsized":"2024-09-21T00:29:47.077435+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:47.077435+0000","last_change":"2024-09-21T00:29:47.077435+0000","last_active":"2024-09-21T00:29:47.077435+0000","last_peered":"2024-09-21T00:29:47.077435+0000","last_clean":"2024-09-21T00:29:47.077435+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:47.077435+0000","last_undegraded":"2024-09-21T00:29:47.077435+0000","last_fullsized":"2024-09-21T00:29:47.077435+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-21T00:29:48.108007+0000","last_change":"2024-09-21T00:29:48.104276+0000","last_active":"2024-09-21T00:29:48.108007+0000","last_peered":"2024-09-21T00:29:48.108007+0000","last_clean":"2024-09-21T00:29:48.108007+0000","last_became_active":"2024-09-21T00:29:48.103972+0000","last_became_peered":"2024-09-21T00:29:48.103972+0000","last_unstale":"2024-09-21T00:29:48.108007+0000","last_undegraded":"2024-09-21T00:29:48.108007+0000","last_fullsized":"2024-09-21T00:29:48.108007+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:48:50.178516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:47.077435+0000","last_change":"2024-09-21T00:29:47.077435+0000","last_active":"2024-09-21T00:29:47.077435+0000","last_peered":"2024-09-21T00:29:47.077435+0000","last_clean":"2024-09-21T00:29:47.077435+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:47.077435+0000","last_undegraded":"2024-09-21T00:29:47.077435+0000","last_fullsized":"2024-09-21T00:29:47.077435+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:47.077435+0000","last_change":"2024-09-21T00:29:47.077435+0000","last_active":"2024-09-21T00:29:47.077435+0000","last_peered":"2024-09-21T00:29:47.077435+0000","last_clean":"2024-09-21T00:29:47.077435+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:47.077435+0000","last_undegraded":"2024-09-21T00:29:47.077435+0000","last_fullsized":"2024-09-21T00:29:47.077435+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:47.077435+0000","last_change":"2024-09-21T00:29:47.077435+0000","last_active":"2024-09-21T00:29:47.077435+0000","last_peered":"2024-09-21T00:29:47.077435+0000","last_clean":"2024-09-21T00:29:47.077435+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:47.077435+0000","last_undegraded":"2024-09-21T00:29:47.077435+0000","last_fullsized":"2024-09-21T00:29:47.077435+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:47.077435+0000","last_change":"2024-09-21T00:29:47.077435+0000","last_active":"2024-09-21T00:29:47.077435+0000","last_peered":"2024-09-21T00:29:47.077435+0000","last_clean":"2024-09-21T00:29:47.077435+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:47.077435+0000","last_undegraded":"2024-09-21T00:29:47.077435+0000","last_fullsized":"2024-09-21T00:29:47.077435+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:47.077435+0000","last_change":"2024-09-21T00:29:47.077435+0000","last_active":"2024-09-21T00:29:47.077435+0000","last_peered":"2024-09-21T00:29:47.077435+0000","last_clean":"2024-09-21T00:29:47.077435+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:47.077435+0000","last_undegraded":"2024-09-21T00:29:47.077435+0000","last_fullsized":"2024-09-21T00:29:47.077435+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:47.077435+0000","last_change":"2024-09-21T00:29:47.077435+0000","last_active":"2024-09-21T00:29:47.077435+0000","last_peered":"2024-09-21T00:29:47.077435+0000","last_clean":"2024-09-21T00:29:47.077435+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:47.077435+0000","last_undegraded":"2024-09-21T00:29:47.077435+0000","last_fullsized":"2024-09-21T00:29:47.077435+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:47.077435+0000","last_change":"2024-09-21T00:29:47.077435+0000","last_active":"2024-09-21T00:29:47.077435+0000","last_peered":"2024-09-21T00:29:47.077435+0000","last_clean":"2024-09-21T00:29:47.077435+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:47.077435+0000","last_undegraded":"2024-09-21T00:29:47.077435+0000","last_fullsized":"2024-09-21T00:29:47.077435+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:47.077435+0000","last_change":"2024-09-21T00:29:47.077435+0000","last_active":"2024-09-21T00:29:47.077435+0000","last_peered":"2024-09-21T00:29:47.077435+0000","last_clean":"2024-09-21T00:29:47.077435+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:47.077435+0000","last_undegraded":"2024-09-21T00:29:47.077435+0000","last_fullsized":"2024-09-21T00:29:47.077435+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-09-21T00:29:48.095849+0000","last_change":"2024-09-21T00:29:48.095849+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.095849+0000","last_undegraded":"2024-09-21T00:29:48.095849+0000","last_fullsized":"2024-09-21T00:29:48.095849+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:47.077435+0000","last_change":"2024-09-21T00:29:47.077435+0000","last_active":"2024-09-21T00:29:47.077435+0000","last_peered":"2024-09-21T00:29:47.077435+0000","last_clean":"2024-09-21T00:29:47.077435+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:47.077435+0000","last_undegraded":"2024-09-21T00:29:47.077435+0000","last_fullsized":"2024-09-21T00:29:47.077435+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:47.077435+0000","last_change":"2024-09-21T00:29:47.077435+0000","last_active":"2024-09-21T00:29:47.077435+0000","last_peered":"2024-09-21T00:29:47.077435+0000","last_clean":"2024-09-21T00:29:47.077435+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:47.077435+0000","last_undegraded":"2024-09-21T00:29:47.077435+0000","last_fullsized":"2024-09-21T00:29:47.077435+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:47.077435+0000","last_change":"2024-09-21T00:29:47.077435+0000","last_active":"2024-09-21T00:29:47.077435+0000","last_peered":"2024-09-21T00:29:47.077435+0000","last_clean":"2024-09-21T00:29:47.077435+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:47.077435+0000","last_undegraded":"2024-09-21T00:29:47.077435+0000","last_fullsized":"2024-09-21T00:29:47.077435+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:47.077435+0000","last_change":"2024-09-21T00:29:47.077435+0000","last_active":"2024-09-21T00:29:47.077435+0000","last_peered":"2024-09-21T00:29:47.077435+0000","last_clean":"2024-09-21T00:29:47.077435+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:47.077435+0000","last_undegraded":"2024-09-21T00:29:47.077435+0000","last_fullsized":"2024-09-21T00:29:47.077435+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:47.077435+0000","last_change":"2024-09-21T00:29:47.077435+0000","last_active":"2024-09-21T00:29:47.077435+0000","last_peered":"2024-09-21T00:29:47.077435+0000","last_clean":"2024-09-21T00:29:47.077435+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:47.077435+0000","last_undegraded":"2024-09-21T00:29:47.077435+0000","last_fullsized":"2024-09-21T00:29:47.077435+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:47.077435+0000","last_change":"2024-09-21T00:29:47.077435+0000","last_active":"2024-09-21T00:29:47.077435+0000","last_peered":"2024-09-21T00:29:47.077435+0000","last_clean":"2024-09-21T00:29:47.077435+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:47.077435+0000","last_undegraded":"2024-09-21T00:29:47.077435+0000","last_fullsized":"2024-09-21T00:29:47.077435+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-09-21T00:29:48.095876+0000","last_change":"2024-09-21T00:29:48.095876+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.095876+0000","last_undegraded":"2024-09-21T00:29:48.095876+0000","last_fullsized":"2024-09-21T00:29:48.095876+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:47.077435+0000","last_change":"2024-09-21T00:29:47.077435+0000","last_active":"2024-09-21T00:29:47.077435+0000","last_peered":"2024-09-21T00:29:47.077435+0000","last_clean":"2024-09-21T00:29:47.077435+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:47.077435+0000","last_undegraded":"2024-09-21T00:29:47.077435+0000","last_fullsized":"2024-09-21T00:29:47.077435+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:47.077435+0000","last_change":"2024-09-21T00:29:47.077435+0000","last_active":"2024-09-21T00:29:47.077435+0000","last_peered":"2024-09-21T00:29:47.077435+0000","last_clean":"2024-09-21T00:29:47.077435+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:47.077435+0000","last_undegraded":"2024-09-21T00:29:47.077435+0000","last_fullsized":"2024-09-21T00:29:47.077435+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":263,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T00:29:45.425002+0000","last_change":"2024-09-21T00:25:38.725792+0000","last_active":"2024-09-21T00:29:45.425002+0000","last_peered":"2024-09-21T00:29:45.425002+0000","last_clean":"2024-09-21T00:29:45.425002+0000","last_became_active":"2024-09-21T00:25:38.724668+0000","last_became_peered":"2024-09-21T00:25:38.724668+0000","last_unstale":"2024-09-21T00:29:45.425002+0000","last_undegraded":"2024-09-21T00:29:45.425002+0000","last_fullsized":"2024-09-21T00:29:45.425002+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:37.163216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:37.163216+0000","last_clean_scrub_stamp":"2024-09-21T00:25:37.163216+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:14:36.588660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":18,"acting":18,"num_store_stats":1},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":6,"acting":6,"num_store_stats":1},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509493,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33032,"kb_used_data":1840,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338808,"statfs":{"total":96636764160,"available":96602939392,"internally_reserved":0,"allocated":1884160,"data_stored":1175610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509494,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32452,"kb_used_data":1260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339388,"statfs":{"total":96636764160,"available":96603533312,"internally_reserved":0,"allocated":1290240,"data_stored":585242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509493,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32460,"kb_used_data":1260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339380,"statfs":{"total":96636764160,"available":96603525120,"internally_reserved":0,"allocated":1290240,"data_stored":585242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509492,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32464,"kb_used_data":1260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339376,"statfs":{"total":96636764160,"available":96603521024,"internally_reserved":0,"allocated":1290240,"data_stored":585242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509494,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32456,"kb_used_data":1260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339384,"statfs":{"total":96636764160,"available":96603529216,"internally_reserved":0,"allocated":1290240,"data_stored":585242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509494,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32452,"kb_used_data":1260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339388,"statfs":{"total":96636764160,"available":96603533312,"internally_reserved":0,"allocated":1290240,"data_stored":585242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509494,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33036,"kb_used_data":1840,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338804,"statfs":{"total":96636764160,"available":96602935296,"internally_reserved":0,"allocated":1884160,"data_stored":1175610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574902,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32476,"kb_used_data":1284,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339364,"statfs":{"total":96636764160,"available":96603508736,"internally_reserved":0,"allocated":1314816,"data_stored":598118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T00:29:52.481 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T00:29:52.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.628+0000 7efee8027640 1 -- 172.21.15.112:0/1556161179 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efee00b2870 msgr2=0x7efee006fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:52.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.628+0000 7efee8027640 1 --2- 172.21.15.112:0/1556161179 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efee00b2870 0x7efee006fc50 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7efed0009890 tx=0x7efed002a4f0 comp rx=0 tx=0).stop 2024-09-21T00:29:52.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.628+0000 7efee8027640 1 -- 172.21.15.112:0/1556161179 shutdown_connections 2024-09-21T00:29:52.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.628+0000 7efee8027640 1 --2- 172.21.15.112:0/1556161179 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efee00cf2f0 0x7efee00cf700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:52.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.628+0000 7efee8027640 1 --2- 172.21.15.112:0/1556161179 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efee00b2870 0x7efee006fc50 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:52.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.628+0000 7efee8027640 1 -- 172.21.15.112:0/1556161179 >> 172.21.15.112:0/1556161179 conn(0x7efee00b42e0 msgr2=0x7efee00b4720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:52.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.629+0000 7efee8027640 1 -- 172.21.15.112:0/1556161179 shutdown_connections 2024-09-21T00:29:52.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.629+0000 7efee8027640 1 -- 172.21.15.112:0/1556161179 wait complete. 2024-09-21T00:29:52.632 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.630+0000 7efee8027640 1 Processor -- start 2024-09-21T00:29:52.632 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.630+0000 7efee8027640 1 -- start start 2024-09-21T00:29:52.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.631+0000 7efee8027640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efee00b2870 0x7efee015b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:52.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.631+0000 7efee8027640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efee00cf2f0 0x7efee015a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:52.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.632+0000 7efee8027640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efee015fa10 0x7efee015aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:52.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.632+0000 7efee8027640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7efee006ecd0 con 0x7efee00cf2f0 2024-09-21T00:29:52.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.632+0000 7efee8027640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7efee006eb50 con 0x7efee00b2870 2024-09-21T00:29:52.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.632+0000 7efee8027640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7efee006ee50 con 0x7efee015fa10 2024-09-21T00:29:52.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.632+0000 7efee55c2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efee00cf2f0 0x7efee015a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:52.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.632+0000 7efee55c2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efee00cf2f0 0x7efee015a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:32844/0 (socket says 172.21.15.112:32844) 2024-09-21T00:29:52.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.632+0000 7efee55c2640 1 -- 172.21.15.112:0/1099653024 learned_addr learned my addr 172.21.15.112:0/1099653024 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:29:52.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.632+0000 7efee65c4640 1 --2- 172.21.15.112:0/1099653024 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efee015fa10 0x7efee015aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:52.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.632+0000 7efee55c2640 1 -- 172.21.15.112:0/1099653024 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efee015fa10 msgr2=0x7efee015aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:52.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.632+0000 7efee5dc3640 1 --2- 172.21.15.112:0/1099653024 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efee00b2870 0x7efee015b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:52.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.632+0000 7efee55c2640 1 --2- 172.21.15.112:0/1099653024 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efee015fa10 0x7efee015aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:52.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.632+0000 7efee55c2640 1 -- 172.21.15.112:0/1099653024 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efee00b2870 msgr2=0x7efee015b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:52.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.632+0000 7efee55c2640 1 --2- 172.21.15.112:0/1099653024 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efee00b2870 0x7efee015b9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:52.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.632+0000 7efee55c2640 1 -- 172.21.15.112:0/1099653024 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efee01520a0 con 0x7efee00cf2f0 2024-09-21T00:29:52.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.632+0000 7efee55c2640 1 --2- 172.21.15.112:0/1099653024 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efee00cf2f0 0x7efee015a590 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7efed00099b0 tx=0x7efed0033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:52.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.632+0000 7efed67fc640 1 -- 172.21.15.112:0/1099653024 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed0031d60 con 0x7efee00cf2f0 2024-09-21T00:29:52.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.632+0000 7efed67fc640 1 -- 172.21.15.112:0/1099653024 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efed0033e10 con 0x7efee00cf2f0 2024-09-21T00:29:52.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.632+0000 7efed67fc640 1 -- 172.21.15.112:0/1099653024 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed0005260 con 0x7efee00cf2f0 2024-09-21T00:29:52.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.632+0000 7efee8027640 1 -- 172.21.15.112:0/1099653024 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efee00c60f0 con 0x7efee00cf2f0 2024-09-21T00:29:52.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.632+0000 7efee8027640 1 -- 172.21.15.112:0/1099653024 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efee015c6f0 con 0x7efee00cf2f0 2024-09-21T00:29:52.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.633+0000 7efee8027640 1 -- 172.21.15.112:0/1099653024 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efee00c6a90 con 0x7efee00cf2f0 2024-09-21T00:29:52.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.634+0000 7efed67fc640 1 -- 172.21.15.112:0/1099653024 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7efed0005400 con 0x7efee00cf2f0 2024-09-21T00:29:52.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.634+0000 7efed67fc640 1 --2- 172.21.15.112:0/1099653024 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7efec4078b90 0x7efec407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:52.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.635+0000 7efed67fc640 1 -- 172.21.15.112:0/1099653024 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 6101+0+0 (secure 0 0 0) 0x7efed00bc100 con 0x7efee00cf2f0 2024-09-21T00:29:52.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.635+0000 7efee5dc3640 1 --2- 172.21.15.112:0/1099653024 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7efec4078b90 0x7efec407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:52.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.635+0000 7efee5dc3640 1 --2- 172.21.15.112:0/1099653024 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7efec4078b90 0x7efec407b0a0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7efedc001540 tx=0x7efedc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:52.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.639+0000 7efed67fc640 1 -- 172.21.15.112:0/1099653024 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7efed003f030 con 0x7efee00cf2f0 2024-09-21T00:29:52.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.812+0000 7efee8027640 1 -- 172.21.15.112:0/1099653024 --> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7efee00c5ea0 con 0x7efec4078b90 2024-09-21T00:29:52.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.816+0000 7efed67fc640 1 -- 172.21.15.112:0/1099653024 <== mgr.4103 v2:172.21.15.187:6800/2921522906 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+238398 (secure 0 0 0) 0x7efee00c5ea0 con 0x7efec4078b90 2024-09-21T00:29:52.821 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:29:52.822 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2024-09-21T00:29:52.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.820+0000 7efee8027640 1 -- 172.21.15.112:0/1099653024 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7efec4078b90 msgr2=0x7efec407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:52.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.820+0000 7efee8027640 1 --2- 172.21.15.112:0/1099653024 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7efec4078b90 0x7efec407b0a0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7efedc001540 tx=0x7efedc009040 comp rx=0 tx=0).stop 2024-09-21T00:29:52.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.820+0000 7efee8027640 1 -- 172.21.15.112:0/1099653024 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efee00cf2f0 msgr2=0x7efee015a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:52.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.820+0000 7efee8027640 1 --2- 172.21.15.112:0/1099653024 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efee00cf2f0 0x7efee015a590 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7efed00099b0 tx=0x7efed0033c20 comp rx=0 tx=0).stop 2024-09-21T00:29:52.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.820+0000 7efee8027640 1 -- 172.21.15.112:0/1099653024 shutdown_connections 2024-09-21T00:29:52.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.820+0000 7efee8027640 1 --2- 172.21.15.112:0/1099653024 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efee015fa10 0x7efee015aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:52.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.820+0000 7efee8027640 1 --2- 172.21.15.112:0/1099653024 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7efec4078b90 0x7efec407b0a0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:52.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.820+0000 7efee8027640 1 --2- 172.21.15.112:0/1099653024 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efee00cf2f0 0x7efee015a590 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:52.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.820+0000 7efee8027640 1 --2- 172.21.15.112:0/1099653024 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efee00b2870 0x7efee015b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:52.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.820+0000 7efee8027640 1 -- 172.21.15.112:0/1099653024 >> 172.21.15.112:0/1099653024 conn(0x7efee00b42e0 msgr2=0x7efee00b61a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:52.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.820+0000 7efee8027640 1 -- 172.21.15.112:0/1099653024 shutdown_connections 2024-09-21T00:29:52.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.820+0000 7efee8027640 1 -- 172.21.15.112:0/1099653024 wait complete. 2024-09-21T00:29:52.833 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":230,"stamp":"2024-09-21T00:29:51.152365+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":3,"kb":754974720,"kb_used":260828,"kb_used_data":11264,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754713892,"statfs":{"total":773094113280,"available":772827025408,"internally_reserved":0,"allocated":11534336,"data_stored":5875548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.071150"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-09-21T00:29:48.098942+0000","last_change":"2024-09-21T00:29:48.098942+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.098942+0000","last_undegraded":"2024-09-21T00:29:48.098942+0000","last_fullsized":"2024-09-21T00:29:48.098942+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-09-21T00:29:48.099873+0000","last_change":"2024-09-21T00:29:48.099873+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.099873+0000","last_undegraded":"2024-09-21T00:29:48.099873+0000","last_fullsized":"2024-09-21T00:29:48.099873+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-09-21T00:29:48.097589+0000","last_change":"2024-09-21T00:29:48.097589+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.097589+0000","last_undegraded":"2024-09-21T00:29:48.097589+0000","last_fullsized":"2024-09-21T00:29:48.097589+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-09-21T00:29:48.097051+0000","last_change":"2024-09-21T00:29:48.097051+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.097051+0000","last_undegraded":"2024-09-21T00:29:48.097051+0000","last_fullsized":"2024-09-21T00:29:48.097051+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:47.077435+0000","last_change":"2024-09-21T00:29:47.077435+0000","last_active":"2024-09-21T00:29:47.077435+0000","last_peered":"2024-09-21T00:29:47.077435+0000","last_clean":"2024-09-21T00:29:47.077435+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:47.077435+0000","last_undegraded":"2024-09-21T00:29:47.077435+0000","last_fullsized":"2024-09-21T00:29:47.077435+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-21T00:29:48.105694+0000","last_change":"2024-09-21T00:29:48.102532+0000","last_active":"2024-09-21T00:29:48.105694+0000","last_peered":"2024-09-21T00:29:48.105694+0000","last_clean":"2024-09-21T00:29:48.105694+0000","last_became_active":"2024-09-21T00:29:48.102195+0000","last_became_peered":"2024-09-21T00:29:48.102195+0000","last_unstale":"2024-09-21T00:29:48.105694+0000","last_undegraded":"2024-09-21T00:29:48.105694+0000","last_fullsized":"2024-09-21T00:29:48.105694+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:41:20.768969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:47.077435+0000","last_change":"2024-09-21T00:29:47.077435+0000","last_active":"2024-09-21T00:29:47.077435+0000","last_peered":"2024-09-21T00:29:47.077435+0000","last_clean":"2024-09-21T00:29:47.077435+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:47.077435+0000","last_undegraded":"2024-09-21T00:29:47.077435+0000","last_fullsized":"2024-09-21T00:29:47.077435+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-21T00:29:48.107959+0000","last_change":"2024-09-21T00:29:48.103570+0000","last_active":"2024-09-21T00:29:48.107959+0000","last_peered":"2024-09-21T00:29:48.107959+0000","last_clean":"2024-09-21T00:29:48.107959+0000","last_became_active":"2024-09-21T00:29:48.103262+0000","last_became_peered":"2024-09-21T00:29:48.103262+0000","last_unstale":"2024-09-21T00:29:48.107959+0000","last_undegraded":"2024-09-21T00:29:48.107959+0000","last_fullsized":"2024-09-21T00:29:48.107959+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:19:55.232168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-09-21T00:29:48.094970+0000","last_change":"2024-09-21T00:29:48.094970+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.094970+0000","last_undegraded":"2024-09-21T00:29:48.094970+0000","last_fullsized":"2024-09-21T00:29:48.094970+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:47.077435+0000","last_change":"2024-09-21T00:29:47.077435+0000","last_active":"2024-09-21T00:29:47.077435+0000","last_peered":"2024-09-21T00:29:47.077435+0000","last_clean":"2024-09-21T00:29:47.077435+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:47.077435+0000","last_undegraded":"2024-09-21T00:29:47.077435+0000","last_fullsized":"2024-09-21T00:29:47.077435+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:47.077435+0000","last_change":"2024-09-21T00:29:47.077435+0000","last_active":"2024-09-21T00:29:47.077435+0000","last_peered":"2024-09-21T00:29:47.077435+0000","last_clean":"2024-09-21T00:29:47.077435+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:47.077435+0000","last_undegraded":"2024-09-21T00:29:47.077435+0000","last_fullsized":"2024-09-21T00:29:47.077435+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:47.077435+0000","last_change":"2024-09-21T00:29:47.077435+0000","last_active":"2024-09-21T00:29:47.077435+0000","last_peered":"2024-09-21T00:29:47.077435+0000","last_clean":"2024-09-21T00:29:47.077435+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:47.077435+0000","last_undegraded":"2024-09-21T00:29:47.077435+0000","last_fullsized":"2024-09-21T00:29:47.077435+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-09-21T00:29:48.096433+0000","last_change":"2024-09-21T00:29:48.096433+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.096433+0000","last_undegraded":"2024-09-21T00:29:48.096433+0000","last_fullsized":"2024-09-21T00:29:48.096433+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:47.077435+0000","last_change":"2024-09-21T00:29:47.077435+0000","last_active":"2024-09-21T00:29:47.077435+0000","last_peered":"2024-09-21T00:29:47.077435+0000","last_clean":"2024-09-21T00:29:47.077435+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:47.077435+0000","last_undegraded":"2024-09-21T00:29:47.077435+0000","last_fullsized":"2024-09-21T00:29:47.077435+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:47.077435+0000","last_change":"2024-09-21T00:29:47.077435+0000","last_active":"2024-09-21T00:29:47.077435+0000","last_peered":"2024-09-21T00:29:47.077435+0000","last_clean":"2024-09-21T00:29:47.077435+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:47.077435+0000","last_undegraded":"2024-09-21T00:29:47.077435+0000","last_fullsized":"2024-09-21T00:29:47.077435+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:47.077435+0000","last_change":"2024-09-21T00:29:47.077435+0000","last_active":"2024-09-21T00:29:47.077435+0000","last_peered":"2024-09-21T00:29:47.077435+0000","last_clean":"2024-09-21T00:29:47.077435+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:47.077435+0000","last_undegraded":"2024-09-21T00:29:47.077435+0000","last_fullsized":"2024-09-21T00:29:47.077435+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:47.077435+0000","last_change":"2024-09-21T00:29:47.077435+0000","last_active":"2024-09-21T00:29:47.077435+0000","last_peered":"2024-09-21T00:29:47.077435+0000","last_clean":"2024-09-21T00:29:47.077435+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:47.077435+0000","last_undegraded":"2024-09-21T00:29:47.077435+0000","last_fullsized":"2024-09-21T00:29:47.077435+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:47.077435+0000","last_change":"2024-09-21T00:29:47.077435+0000","last_active":"2024-09-21T00:29:47.077435+0000","last_peered":"2024-09-21T00:29:47.077435+0000","last_clean":"2024-09-21T00:29:47.077435+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:47.077435+0000","last_undegraded":"2024-09-21T00:29:47.077435+0000","last_fullsized":"2024-09-21T00:29:47.077435+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:47.077435+0000","last_change":"2024-09-21T00:29:47.077435+0000","last_active":"2024-09-21T00:29:47.077435+0000","last_peered":"2024-09-21T00:29:47.077435+0000","last_clean":"2024-09-21T00:29:47.077435+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:47.077435+0000","last_undegraded":"2024-09-21T00:29:47.077435+0000","last_fullsized":"2024-09-21T00:29:47.077435+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-09-21T00:29:48.095354+0000","last_change":"2024-09-21T00:29:48.095354+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.095354+0000","last_undegraded":"2024-09-21T00:29:48.095354+0000","last_fullsized":"2024-09-21T00:29:48.095354+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:47.077435+0000","last_change":"2024-09-21T00:29:47.077435+0000","last_active":"2024-09-21T00:29:47.077435+0000","last_peered":"2024-09-21T00:29:47.077435+0000","last_clean":"2024-09-21T00:29:47.077435+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:47.077435+0000","last_undegraded":"2024-09-21T00:29:47.077435+0000","last_fullsized":"2024-09-21T00:29:47.077435+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:47.077435+0000","last_change":"2024-09-21T00:29:47.077435+0000","last_active":"2024-09-21T00:29:47.077435+0000","last_peered":"2024-09-21T00:29:47.077435+0000","last_clean":"2024-09-21T00:29:47.077435+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:47.077435+0000","last_undegraded":"2024-09-21T00:29:47.077435+0000","last_fullsized":"2024-09-21T00:29:47.077435+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-21T00:29:48.108007+0000","last_change":"2024-09-21T00:29:48.104276+0000","last_active":"2024-09-21T00:29:48.108007+0000","last_peered":"2024-09-21T00:29:48.108007+0000","last_clean":"2024-09-21T00:29:48.108007+0000","last_became_active":"2024-09-21T00:29:48.103972+0000","last_became_peered":"2024-09-21T00:29:48.103972+0000","last_unstale":"2024-09-21T00:29:48.108007+0000","last_undegraded":"2024-09-21T00:29:48.108007+0000","last_fullsized":"2024-09-21T00:29:48.108007+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:48:50.178516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:47.077435+0000","last_change":"2024-09-21T00:29:47.077435+0000","last_active":"2024-09-21T00:29:47.077435+0000","last_peered":"2024-09-21T00:29:47.077435+0000","last_clean":"2024-09-21T00:29:47.077435+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:47.077435+0000","last_undegraded":"2024-09-21T00:29:47.077435+0000","last_fullsized":"2024-09-21T00:29:47.077435+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:47.077435+0000","last_change":"2024-09-21T00:29:47.077435+0000","last_active":"2024-09-21T00:29:47.077435+0000","last_peered":"2024-09-21T00:29:47.077435+0000","last_clean":"2024-09-21T00:29:47.077435+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:47.077435+0000","last_undegraded":"2024-09-21T00:29:47.077435+0000","last_fullsized":"2024-09-21T00:29:47.077435+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:47.077435+0000","last_change":"2024-09-21T00:29:47.077435+0000","last_active":"2024-09-21T00:29:47.077435+0000","last_peered":"2024-09-21T00:29:47.077435+0000","last_clean":"2024-09-21T00:29:47.077435+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:47.077435+0000","last_undegraded":"2024-09-21T00:29:47.077435+0000","last_fullsized":"2024-09-21T00:29:47.077435+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:47.077435+0000","last_change":"2024-09-21T00:29:47.077435+0000","last_active":"2024-09-21T00:29:47.077435+0000","last_peered":"2024-09-21T00:29:47.077435+0000","last_clean":"2024-09-21T00:29:47.077435+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:47.077435+0000","last_undegraded":"2024-09-21T00:29:47.077435+0000","last_fullsized":"2024-09-21T00:29:47.077435+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:47.077435+0000","last_change":"2024-09-21T00:29:47.077435+0000","last_active":"2024-09-21T00:29:47.077435+0000","last_peered":"2024-09-21T00:29:47.077435+0000","last_clean":"2024-09-21T00:29:47.077435+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:47.077435+0000","last_undegraded":"2024-09-21T00:29:47.077435+0000","last_fullsized":"2024-09-21T00:29:47.077435+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:47.077435+0000","last_change":"2024-09-21T00:29:47.077435+0000","last_active":"2024-09-21T00:29:47.077435+0000","last_peered":"2024-09-21T00:29:47.077435+0000","last_clean":"2024-09-21T00:29:47.077435+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:47.077435+0000","last_undegraded":"2024-09-21T00:29:47.077435+0000","last_fullsized":"2024-09-21T00:29:47.077435+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:47.077435+0000","last_change":"2024-09-21T00:29:47.077435+0000","last_active":"2024-09-21T00:29:47.077435+0000","last_peered":"2024-09-21T00:29:47.077435+0000","last_clean":"2024-09-21T00:29:47.077435+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:47.077435+0000","last_undegraded":"2024-09-21T00:29:47.077435+0000","last_fullsized":"2024-09-21T00:29:47.077435+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:47.077435+0000","last_change":"2024-09-21T00:29:47.077435+0000","last_active":"2024-09-21T00:29:47.077435+0000","last_peered":"2024-09-21T00:29:47.077435+0000","last_clean":"2024-09-21T00:29:47.077435+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:47.077435+0000","last_undegraded":"2024-09-21T00:29:47.077435+0000","last_fullsized":"2024-09-21T00:29:47.077435+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-09-21T00:29:48.095849+0000","last_change":"2024-09-21T00:29:48.095849+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.095849+0000","last_undegraded":"2024-09-21T00:29:48.095849+0000","last_fullsized":"2024-09-21T00:29:48.095849+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:47.077435+0000","last_change":"2024-09-21T00:29:47.077435+0000","last_active":"2024-09-21T00:29:47.077435+0000","last_peered":"2024-09-21T00:29:47.077435+0000","last_clean":"2024-09-21T00:29:47.077435+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:47.077435+0000","last_undegraded":"2024-09-21T00:29:47.077435+0000","last_fullsized":"2024-09-21T00:29:47.077435+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:47.077435+0000","last_change":"2024-09-21T00:29:47.077435+0000","last_active":"2024-09-21T00:29:47.077435+0000","last_peered":"2024-09-21T00:29:47.077435+0000","last_clean":"2024-09-21T00:29:47.077435+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:47.077435+0000","last_undegraded":"2024-09-21T00:29:47.077435+0000","last_fullsized":"2024-09-21T00:29:47.077435+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:47.077435+0000","last_change":"2024-09-21T00:29:47.077435+0000","last_active":"2024-09-21T00:29:47.077435+0000","last_peered":"2024-09-21T00:29:47.077435+0000","last_clean":"2024-09-21T00:29:47.077435+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:47.077435+0000","last_undegraded":"2024-09-21T00:29:47.077435+0000","last_fullsized":"2024-09-21T00:29:47.077435+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:47.077435+0000","last_change":"2024-09-21T00:29:47.077435+0000","last_active":"2024-09-21T00:29:47.077435+0000","last_peered":"2024-09-21T00:29:47.077435+0000","last_clean":"2024-09-21T00:29:47.077435+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:47.077435+0000","last_undegraded":"2024-09-21T00:29:47.077435+0000","last_fullsized":"2024-09-21T00:29:47.077435+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:47.077435+0000","last_change":"2024-09-21T00:29:47.077435+0000","last_active":"2024-09-21T00:29:47.077435+0000","last_peered":"2024-09-21T00:29:47.077435+0000","last_clean":"2024-09-21T00:29:47.077435+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:47.077435+0000","last_undegraded":"2024-09-21T00:29:47.077435+0000","last_fullsized":"2024-09-21T00:29:47.077435+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:47.077435+0000","last_change":"2024-09-21T00:29:47.077435+0000","last_active":"2024-09-21T00:29:47.077435+0000","last_peered":"2024-09-21T00:29:47.077435+0000","last_clean":"2024-09-21T00:29:47.077435+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:47.077435+0000","last_undegraded":"2024-09-21T00:29:47.077435+0000","last_fullsized":"2024-09-21T00:29:47.077435+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-09-21T00:29:48.095876+0000","last_change":"2024-09-21T00:29:48.095876+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.095876+0000","last_undegraded":"2024-09-21T00:29:48.095876+0000","last_fullsized":"2024-09-21T00:29:48.095876+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:47.077435+0000","last_change":"2024-09-21T00:29:47.077435+0000","last_active":"2024-09-21T00:29:47.077435+0000","last_peered":"2024-09-21T00:29:47.077435+0000","last_clean":"2024-09-21T00:29:47.077435+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:47.077435+0000","last_undegraded":"2024-09-21T00:29:47.077435+0000","last_fullsized":"2024-09-21T00:29:47.077435+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:48.079691+0000","last_change":"2024-09-21T00:29:48.079691+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.079691+0000","last_undegraded":"2024-09-21T00:29:48.079691+0000","last_fullsized":"2024-09-21T00:29:48.079691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T00:29:47.077435+0000","last_change":"2024-09-21T00:29:47.077435+0000","last_active":"2024-09-21T00:29:47.077435+0000","last_peered":"2024-09-21T00:29:47.077435+0000","last_clean":"2024-09-21T00:29:47.077435+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:47.077435+0000","last_undegraded":"2024-09-21T00:29:47.077435+0000","last_fullsized":"2024-09-21T00:29:47.077435+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":263,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T00:29:45.425002+0000","last_change":"2024-09-21T00:25:38.725792+0000","last_active":"2024-09-21T00:29:45.425002+0000","last_peered":"2024-09-21T00:29:45.425002+0000","last_clean":"2024-09-21T00:29:45.425002+0000","last_became_active":"2024-09-21T00:25:38.724668+0000","last_became_peered":"2024-09-21T00:25:38.724668+0000","last_unstale":"2024-09-21T00:29:45.425002+0000","last_undegraded":"2024-09-21T00:29:45.425002+0000","last_fullsized":"2024-09-21T00:29:45.425002+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:37.163216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:37.163216+0000","last_clean_scrub_stamp":"2024-09-21T00:25:37.163216+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:14:36.588660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":18,"acting":18,"num_store_stats":1},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":6,"acting":6,"num_store_stats":1},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509493,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33032,"kb_used_data":1840,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338808,"statfs":{"total":96636764160,"available":96602939392,"internally_reserved":0,"allocated":1884160,"data_stored":1175610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509494,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32452,"kb_used_data":1260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339388,"statfs":{"total":96636764160,"available":96603533312,"internally_reserved":0,"allocated":1290240,"data_stored":585242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509493,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32460,"kb_used_data":1260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339380,"statfs":{"total":96636764160,"available":96603525120,"internally_reserved":0,"allocated":1290240,"data_stored":585242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509492,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32464,"kb_used_data":1260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339376,"statfs":{"total":96636764160,"available":96603521024,"internally_reserved":0,"allocated":1290240,"data_stored":585242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509494,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32456,"kb_used_data":1260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339384,"statfs":{"total":96636764160,"available":96603529216,"internally_reserved":0,"allocated":1290240,"data_stored":585242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509494,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32452,"kb_used_data":1260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339388,"statfs":{"total":96636764160,"available":96603533312,"internally_reserved":0,"allocated":1290240,"data_stored":585242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509494,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33036,"kb_used_data":1840,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338804,"statfs":{"total":96636764160,"available":96602935296,"internally_reserved":0,"allocated":1884160,"data_stored":1175610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574902,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32476,"kb_used_data":1284,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339364,"statfs":{"total":96636764160,"available":96603508736,"internally_reserved":0,"allocated":1314816,"data_stored":598118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T00:29:52.837 INFO:tasks.ceph.ceph_manager.ceph:PG 18.3f is not active+clean 2024-09-21T00:29:52.837 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:29:48.079691+0000', 'last_change': '2024-09-21T00:29:48.079691+0000', 'last_active': '2024-09-21T00:29:48.079691+0000', 'last_peered': '2024-09-21T00:29:48.079691+0000', 'last_clean': '2024-09-21T00:29:48.079691+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:29:48.079691+0000', 'last_undegraded': '2024-09-21T00:29:48.079691+0000', 'last_fullsized': '2024-09-21T00:29:48.079691+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'last_clean_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:29:52.838 INFO:tasks.ceph.ceph_manager.ceph:PG 18.3e is not active+clean 2024-09-21T00:29:52.838 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:29:48.079691+0000', 'last_change': '2024-09-21T00:29:48.079691+0000', 'last_active': '2024-09-21T00:29:48.079691+0000', 'last_peered': '2024-09-21T00:29:48.079691+0000', 'last_clean': '2024-09-21T00:29:48.079691+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:29:48.079691+0000', 'last_undegraded': '2024-09-21T00:29:48.079691+0000', 'last_fullsized': '2024-09-21T00:29:48.079691+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'last_clean_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:29:52.838 INFO:tasks.ceph.ceph_manager.ceph:PG 18.3d is not active+clean 2024-09-21T00:29:52.838 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:29:48.079691+0000', 'last_change': '2024-09-21T00:29:48.079691+0000', 'last_active': '2024-09-21T00:29:48.079691+0000', 'last_peered': '2024-09-21T00:29:48.079691+0000', 'last_clean': '2024-09-21T00:29:48.079691+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:29:48.079691+0000', 'last_undegraded': '2024-09-21T00:29:48.079691+0000', 'last_fullsized': '2024-09-21T00:29:48.079691+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'last_clean_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:29:52.838 INFO:tasks.ceph.ceph_manager.ceph:PG 18.3c is not active+clean 2024-09-21T00:29:52.838 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:29:48.079691+0000', 'last_change': '2024-09-21T00:29:48.079691+0000', 'last_active': '2024-09-21T00:29:48.079691+0000', 'last_peered': '2024-09-21T00:29:48.079691+0000', 'last_clean': '2024-09-21T00:29:48.079691+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:29:48.079691+0000', 'last_undegraded': '2024-09-21T00:29:48.079691+0000', 'last_fullsized': '2024-09-21T00:29:48.079691+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'last_clean_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:29:52.838 INFO:tasks.ceph.ceph_manager.ceph:PG 18.3b is not active+clean 2024-09-21T00:29:52.838 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:29:48.079691+0000', 'last_change': '2024-09-21T00:29:48.079691+0000', 'last_active': '2024-09-21T00:29:48.079691+0000', 'last_peered': '2024-09-21T00:29:48.079691+0000', 'last_clean': '2024-09-21T00:29:48.079691+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:29:48.079691+0000', 'last_undegraded': '2024-09-21T00:29:48.079691+0000', 'last_fullsized': '2024-09-21T00:29:48.079691+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'last_clean_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:29:52.838 INFO:tasks.ceph.ceph_manager.ceph:PG 18.3a is not active+clean 2024-09-21T00:29:52.838 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:29:48.079691+0000', 'last_change': '2024-09-21T00:29:48.079691+0000', 'last_active': '2024-09-21T00:29:48.079691+0000', 'last_peered': '2024-09-21T00:29:48.079691+0000', 'last_clean': '2024-09-21T00:29:48.079691+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:29:48.079691+0000', 'last_undegraded': '2024-09-21T00:29:48.079691+0000', 'last_fullsized': '2024-09-21T00:29:48.079691+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'last_clean_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:29:52.839 INFO:tasks.ceph.ceph_manager.ceph:PG 18.39 is not active+clean 2024-09-21T00:29:52.839 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:29:48.079691+0000', 'last_change': '2024-09-21T00:29:48.079691+0000', 'last_active': '2024-09-21T00:29:48.079691+0000', 'last_peered': '2024-09-21T00:29:48.079691+0000', 'last_clean': '2024-09-21T00:29:48.079691+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:29:48.079691+0000', 'last_undegraded': '2024-09-21T00:29:48.079691+0000', 'last_fullsized': '2024-09-21T00:29:48.079691+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'last_clean_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:29:52.839 INFO:tasks.ceph.ceph_manager.ceph:PG 18.38 is not active+clean 2024-09-21T00:29:52.839 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:29:48.079691+0000', 'last_change': '2024-09-21T00:29:48.079691+0000', 'last_active': '2024-09-21T00:29:48.079691+0000', 'last_peered': '2024-09-21T00:29:48.079691+0000', 'last_clean': '2024-09-21T00:29:48.079691+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:29:48.079691+0000', 'last_undegraded': '2024-09-21T00:29:48.079691+0000', 'last_fullsized': '2024-09-21T00:29:48.079691+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'last_clean_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:29:52.839 INFO:tasks.ceph.ceph_manager.ceph:PG 18.37 is not active+clean 2024-09-21T00:29:52.839 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:29:48.079691+0000', 'last_change': '2024-09-21T00:29:48.079691+0000', 'last_active': '2024-09-21T00:29:48.079691+0000', 'last_peered': '2024-09-21T00:29:48.079691+0000', 'last_clean': '2024-09-21T00:29:48.079691+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:29:48.079691+0000', 'last_undegraded': '2024-09-21T00:29:48.079691+0000', 'last_fullsized': '2024-09-21T00:29:48.079691+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'last_clean_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:29:52.839 INFO:tasks.ceph.ceph_manager.ceph:PG 18.36 is not active+clean 2024-09-21T00:29:52.839 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 106, 'state': 'creating+peering', 'last_fresh': '2024-09-21T00:29:48.098942+0000', 'last_change': '2024-09-21T00:29:48.098942+0000', 'last_active': '2024-09-21T00:29:48.079691+0000', 'last_peered': '2024-09-21T00:29:48.079691+0000', 'last_clean': '2024-09-21T00:29:48.079691+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:29:48.098942+0000', 'last_undegraded': '2024-09-21T00:29:48.098942+0000', 'last_fullsized': '2024-09-21T00:29:48.098942+0000', 'mapping_epoch': 106, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 106, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'last_clean_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-21T00:29:52.839 INFO:tasks.ceph.ceph_manager.ceph:PG 18.35 is not active+clean 2024-09-21T00:29:52.839 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:29:48.079691+0000', 'last_change': '2024-09-21T00:29:48.079691+0000', 'last_active': '2024-09-21T00:29:48.079691+0000', 'last_peered': '2024-09-21T00:29:48.079691+0000', 'last_clean': '2024-09-21T00:29:48.079691+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:29:48.079691+0000', 'last_undegraded': '2024-09-21T00:29:48.079691+0000', 'last_fullsized': '2024-09-21T00:29:48.079691+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'last_clean_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:29:52.839 INFO:tasks.ceph.ceph_manager.ceph:PG 18.34 is not active+clean 2024-09-21T00:29:52.839 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:29:48.079691+0000', 'last_change': '2024-09-21T00:29:48.079691+0000', 'last_active': '2024-09-21T00:29:48.079691+0000', 'last_peered': '2024-09-21T00:29:48.079691+0000', 'last_clean': '2024-09-21T00:29:48.079691+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:29:48.079691+0000', 'last_undegraded': '2024-09-21T00:29:48.079691+0000', 'last_fullsized': '2024-09-21T00:29:48.079691+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'last_clean_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:29:52.840 INFO:tasks.ceph.ceph_manager.ceph:PG 18.33 is not active+clean 2024-09-21T00:29:52.840 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 106, 'state': 'creating+peering', 'last_fresh': '2024-09-21T00:29:48.099873+0000', 'last_change': '2024-09-21T00:29:48.099873+0000', 'last_active': '2024-09-21T00:29:48.079691+0000', 'last_peered': '2024-09-21T00:29:48.079691+0000', 'last_clean': '2024-09-21T00:29:48.079691+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:29:48.099873+0000', 'last_undegraded': '2024-09-21T00:29:48.099873+0000', 'last_fullsized': '2024-09-21T00:29:48.099873+0000', 'mapping_epoch': 106, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 106, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'last_clean_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-21T00:29:52.840 INFO:tasks.ceph.ceph_manager.ceph:PG 18.32 is not active+clean 2024-09-21T00:29:52.840 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:29:48.079691+0000', 'last_change': '2024-09-21T00:29:48.079691+0000', 'last_active': '2024-09-21T00:29:48.079691+0000', 'last_peered': '2024-09-21T00:29:48.079691+0000', 'last_clean': '2024-09-21T00:29:48.079691+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:29:48.079691+0000', 'last_undegraded': '2024-09-21T00:29:48.079691+0000', 'last_fullsized': '2024-09-21T00:29:48.079691+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'last_clean_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:29:52.840 INFO:tasks.ceph.ceph_manager.ceph:PG 18.31 is not active+clean 2024-09-21T00:29:52.840 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:29:48.079691+0000', 'last_change': '2024-09-21T00:29:48.079691+0000', 'last_active': '2024-09-21T00:29:48.079691+0000', 'last_peered': '2024-09-21T00:29:48.079691+0000', 'last_clean': '2024-09-21T00:29:48.079691+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:29:48.079691+0000', 'last_undegraded': '2024-09-21T00:29:48.079691+0000', 'last_fullsized': '2024-09-21T00:29:48.079691+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'last_clean_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:29:52.840 INFO:tasks.ceph.ceph_manager.ceph:PG 18.30 is not active+clean 2024-09-21T00:29:52.840 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:29:48.079691+0000', 'last_change': '2024-09-21T00:29:48.079691+0000', 'last_active': '2024-09-21T00:29:48.079691+0000', 'last_peered': '2024-09-21T00:29:48.079691+0000', 'last_clean': '2024-09-21T00:29:48.079691+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:29:48.079691+0000', 'last_undegraded': '2024-09-21T00:29:48.079691+0000', 'last_fullsized': '2024-09-21T00:29:48.079691+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'last_clean_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:29:52.840 INFO:tasks.ceph.ceph_manager.ceph:PG 18.2f is not active+clean 2024-09-21T00:29:52.840 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 106, 'state': 'creating+peering', 'last_fresh': '2024-09-21T00:29:48.097589+0000', 'last_change': '2024-09-21T00:29:48.097589+0000', 'last_active': '2024-09-21T00:29:48.079691+0000', 'last_peered': '2024-09-21T00:29:48.079691+0000', 'last_clean': '2024-09-21T00:29:48.079691+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:29:48.097589+0000', 'last_undegraded': '2024-09-21T00:29:48.097589+0000', 'last_fullsized': '2024-09-21T00:29:48.097589+0000', 'mapping_epoch': 106, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 106, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'last_clean_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-21T00:29:52.840 INFO:tasks.ceph.ceph_manager.ceph:PG 18.2e is not active+clean 2024-09-21T00:29:52.841 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:29:48.079691+0000', 'last_change': '2024-09-21T00:29:48.079691+0000', 'last_active': '2024-09-21T00:29:48.079691+0000', 'last_peered': '2024-09-21T00:29:48.079691+0000', 'last_clean': '2024-09-21T00:29:48.079691+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:29:48.079691+0000', 'last_undegraded': '2024-09-21T00:29:48.079691+0000', 'last_fullsized': '2024-09-21T00:29:48.079691+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'last_clean_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:29:52.841 INFO:tasks.ceph.ceph_manager.ceph:PG 18.2d is not active+clean 2024-09-21T00:29:52.841 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:29:48.079691+0000', 'last_change': '2024-09-21T00:29:48.079691+0000', 'last_active': '2024-09-21T00:29:48.079691+0000', 'last_peered': '2024-09-21T00:29:48.079691+0000', 'last_clean': '2024-09-21T00:29:48.079691+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:29:48.079691+0000', 'last_undegraded': '2024-09-21T00:29:48.079691+0000', 'last_fullsized': '2024-09-21T00:29:48.079691+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'last_clean_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:29:52.841 INFO:tasks.ceph.ceph_manager.ceph:PG 18.2c is not active+clean 2024-09-21T00:29:52.841 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:29:48.079691+0000', 'last_change': '2024-09-21T00:29:48.079691+0000', 'last_active': '2024-09-21T00:29:48.079691+0000', 'last_peered': '2024-09-21T00:29:48.079691+0000', 'last_clean': '2024-09-21T00:29:48.079691+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:29:48.079691+0000', 'last_undegraded': '2024-09-21T00:29:48.079691+0000', 'last_fullsized': '2024-09-21T00:29:48.079691+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'last_clean_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:29:52.841 INFO:tasks.ceph.ceph_manager.ceph:PG 18.2b is not active+clean 2024-09-21T00:29:52.841 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:29:48.079691+0000', 'last_change': '2024-09-21T00:29:48.079691+0000', 'last_active': '2024-09-21T00:29:48.079691+0000', 'last_peered': '2024-09-21T00:29:48.079691+0000', 'last_clean': '2024-09-21T00:29:48.079691+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:29:48.079691+0000', 'last_undegraded': '2024-09-21T00:29:48.079691+0000', 'last_fullsized': '2024-09-21T00:29:48.079691+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'last_clean_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:29:52.841 INFO:tasks.ceph.ceph_manager.ceph:PG 18.2a is not active+clean 2024-09-21T00:29:52.841 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:29:48.079691+0000', 'last_change': '2024-09-21T00:29:48.079691+0000', 'last_active': '2024-09-21T00:29:48.079691+0000', 'last_peered': '2024-09-21T00:29:48.079691+0000', 'last_clean': '2024-09-21T00:29:48.079691+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:29:48.079691+0000', 'last_undegraded': '2024-09-21T00:29:48.079691+0000', 'last_fullsized': '2024-09-21T00:29:48.079691+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'last_clean_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:29:52.841 INFO:tasks.ceph.ceph_manager.ceph:PG 18.29 is not active+clean 2024-09-21T00:29:52.841 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:29:48.079691+0000', 'last_change': '2024-09-21T00:29:48.079691+0000', 'last_active': '2024-09-21T00:29:48.079691+0000', 'last_peered': '2024-09-21T00:29:48.079691+0000', 'last_clean': '2024-09-21T00:29:48.079691+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:29:48.079691+0000', 'last_undegraded': '2024-09-21T00:29:48.079691+0000', 'last_fullsized': '2024-09-21T00:29:48.079691+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'last_clean_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:29:52.842 INFO:tasks.ceph.ceph_manager.ceph:PG 18.28 is not active+clean 2024-09-21T00:29:52.842 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:29:48.079691+0000', 'last_change': '2024-09-21T00:29:48.079691+0000', 'last_active': '2024-09-21T00:29:48.079691+0000', 'last_peered': '2024-09-21T00:29:48.079691+0000', 'last_clean': '2024-09-21T00:29:48.079691+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:29:48.079691+0000', 'last_undegraded': '2024-09-21T00:29:48.079691+0000', 'last_fullsized': '2024-09-21T00:29:48.079691+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'last_clean_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:29:52.842 INFO:tasks.ceph.ceph_manager.ceph:PG 18.27 is not active+clean 2024-09-21T00:29:52.842 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:29:48.079691+0000', 'last_change': '2024-09-21T00:29:48.079691+0000', 'last_active': '2024-09-21T00:29:48.079691+0000', 'last_peered': '2024-09-21T00:29:48.079691+0000', 'last_clean': '2024-09-21T00:29:48.079691+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:29:48.079691+0000', 'last_undegraded': '2024-09-21T00:29:48.079691+0000', 'last_fullsized': '2024-09-21T00:29:48.079691+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'last_clean_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:29:52.842 INFO:tasks.ceph.ceph_manager.ceph:PG 18.26 is not active+clean 2024-09-21T00:29:52.842 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:29:48.079691+0000', 'last_change': '2024-09-21T00:29:48.079691+0000', 'last_active': '2024-09-21T00:29:48.079691+0000', 'last_peered': '2024-09-21T00:29:48.079691+0000', 'last_clean': '2024-09-21T00:29:48.079691+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:29:48.079691+0000', 'last_undegraded': '2024-09-21T00:29:48.079691+0000', 'last_fullsized': '2024-09-21T00:29:48.079691+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'last_clean_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:29:52.842 INFO:tasks.ceph.ceph_manager.ceph:PG 18.25 is not active+clean 2024-09-21T00:29:52.842 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:29:48.079691+0000', 'last_change': '2024-09-21T00:29:48.079691+0000', 'last_active': '2024-09-21T00:29:48.079691+0000', 'last_peered': '2024-09-21T00:29:48.079691+0000', 'last_clean': '2024-09-21T00:29:48.079691+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:29:48.079691+0000', 'last_undegraded': '2024-09-21T00:29:48.079691+0000', 'last_fullsized': '2024-09-21T00:29:48.079691+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'last_clean_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:29:52.842 INFO:tasks.ceph.ceph_manager.ceph:PG 18.24 is not active+clean 2024-09-21T00:29:52.842 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:29:48.079691+0000', 'last_change': '2024-09-21T00:29:48.079691+0000', 'last_active': '2024-09-21T00:29:48.079691+0000', 'last_peered': '2024-09-21T00:29:48.079691+0000', 'last_clean': '2024-09-21T00:29:48.079691+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:29:48.079691+0000', 'last_undegraded': '2024-09-21T00:29:48.079691+0000', 'last_fullsized': '2024-09-21T00:29:48.079691+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'last_clean_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:29:52.842 INFO:tasks.ceph.ceph_manager.ceph:PG 18.23 is not active+clean 2024-09-21T00:29:52.842 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:29:48.079691+0000', 'last_change': '2024-09-21T00:29:48.079691+0000', 'last_active': '2024-09-21T00:29:48.079691+0000', 'last_peered': '2024-09-21T00:29:48.079691+0000', 'last_clean': '2024-09-21T00:29:48.079691+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:29:48.079691+0000', 'last_undegraded': '2024-09-21T00:29:48.079691+0000', 'last_fullsized': '2024-09-21T00:29:48.079691+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'last_clean_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:29:52.843 INFO:tasks.ceph.ceph_manager.ceph:PG 18.22 is not active+clean 2024-09-21T00:29:52.843 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:29:48.079691+0000', 'last_change': '2024-09-21T00:29:48.079691+0000', 'last_active': '2024-09-21T00:29:48.079691+0000', 'last_peered': '2024-09-21T00:29:48.079691+0000', 'last_clean': '2024-09-21T00:29:48.079691+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:29:48.079691+0000', 'last_undegraded': '2024-09-21T00:29:48.079691+0000', 'last_fullsized': '2024-09-21T00:29:48.079691+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'last_clean_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:29:52.843 INFO:tasks.ceph.ceph_manager.ceph:PG 18.21 is not active+clean 2024-09-21T00:29:52.843 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:29:48.079691+0000', 'last_change': '2024-09-21T00:29:48.079691+0000', 'last_active': '2024-09-21T00:29:48.079691+0000', 'last_peered': '2024-09-21T00:29:48.079691+0000', 'last_clean': '2024-09-21T00:29:48.079691+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:29:48.079691+0000', 'last_undegraded': '2024-09-21T00:29:48.079691+0000', 'last_fullsized': '2024-09-21T00:29:48.079691+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'last_clean_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:29:52.843 INFO:tasks.ceph.ceph_manager.ceph:PG 18.20 is not active+clean 2024-09-21T00:29:52.843 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:29:48.079691+0000', 'last_change': '2024-09-21T00:29:48.079691+0000', 'last_active': '2024-09-21T00:29:48.079691+0000', 'last_peered': '2024-09-21T00:29:48.079691+0000', 'last_clean': '2024-09-21T00:29:48.079691+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:29:48.079691+0000', 'last_undegraded': '2024-09-21T00:29:48.079691+0000', 'last_fullsized': '2024-09-21T00:29:48.079691+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'last_clean_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:29:52.843 INFO:tasks.ceph.ceph_manager.ceph:PG 18.1c is not active+clean 2024-09-21T00:29:52.843 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 106, 'state': 'creating+peering', 'last_fresh': '2024-09-21T00:29:48.097051+0000', 'last_change': '2024-09-21T00:29:48.097051+0000', 'last_active': '2024-09-21T00:29:48.079691+0000', 'last_peered': '2024-09-21T00:29:48.079691+0000', 'last_clean': '2024-09-21T00:29:48.079691+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:29:48.097051+0000', 'last_undegraded': '2024-09-21T00:29:48.097051+0000', 'last_fullsized': '2024-09-21T00:29:48.097051+0000', 'mapping_epoch': 106, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 106, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'last_clean_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-21T00:29:52.843 INFO:tasks.ceph.ceph_manager.ceph:PG 17.1f is not active+clean 2024-09-21T00:29:52.843 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:29:47.077435+0000', 'last_change': '2024-09-21T00:29:47.077435+0000', 'last_active': '2024-09-21T00:29:47.077435+0000', 'last_peered': '2024-09-21T00:29:47.077435+0000', 'last_clean': '2024-09-21T00:29:47.077435+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:29:47.077435+0000', 'last_undegraded': '2024-09-21T00:29:47.077435+0000', 'last_fullsized': '2024-09-21T00:29:47.077435+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:29:47.077435+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:29:47.077435+0000', 'last_clean_scrub_stamp': '2024-09-21T00:29:47.077435+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:29:52.843 INFO:tasks.ceph.ceph_manager.ceph:PG 18.1d is not active+clean 2024-09-21T00:29:52.844 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:29:48.079691+0000', 'last_change': '2024-09-21T00:29:48.079691+0000', 'last_active': '2024-09-21T00:29:48.079691+0000', 'last_peered': '2024-09-21T00:29:48.079691+0000', 'last_clean': '2024-09-21T00:29:48.079691+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:29:48.079691+0000', 'last_undegraded': '2024-09-21T00:29:48.079691+0000', 'last_fullsized': '2024-09-21T00:29:48.079691+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'last_clean_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:29:52.844 INFO:tasks.ceph.ceph_manager.ceph:PG 18.1e is not active+clean 2024-09-21T00:29:52.844 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:29:48.079691+0000', 'last_change': '2024-09-21T00:29:48.079691+0000', 'last_active': '2024-09-21T00:29:48.079691+0000', 'last_peered': '2024-09-21T00:29:48.079691+0000', 'last_clean': '2024-09-21T00:29:48.079691+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:29:48.079691+0000', 'last_undegraded': '2024-09-21T00:29:48.079691+0000', 'last_fullsized': '2024-09-21T00:29:48.079691+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'last_clean_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:29:52.844 INFO:tasks.ceph.ceph_manager.ceph:PG 17.1d is not active+clean 2024-09-21T00:29:52.844 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:29:47.077435+0000', 'last_change': '2024-09-21T00:29:47.077435+0000', 'last_active': '2024-09-21T00:29:47.077435+0000', 'last_peered': '2024-09-21T00:29:47.077435+0000', 'last_clean': '2024-09-21T00:29:47.077435+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:29:47.077435+0000', 'last_undegraded': '2024-09-21T00:29:47.077435+0000', 'last_fullsized': '2024-09-21T00:29:47.077435+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:29:47.077435+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:29:47.077435+0000', 'last_clean_scrub_stamp': '2024-09-21T00:29:47.077435+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:29:52.844 INFO:tasks.ceph.ceph_manager.ceph:PG 18.1f is not active+clean 2024-09-21T00:29:52.844 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:29:48.079691+0000', 'last_change': '2024-09-21T00:29:48.079691+0000', 'last_active': '2024-09-21T00:29:48.079691+0000', 'last_peered': '2024-09-21T00:29:48.079691+0000', 'last_clean': '2024-09-21T00:29:48.079691+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:29:48.079691+0000', 'last_undegraded': '2024-09-21T00:29:48.079691+0000', 'last_fullsized': '2024-09-21T00:29:48.079691+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'last_clean_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:29:52.844 INFO:tasks.ceph.ceph_manager.ceph:PG 18.18 is not active+clean 2024-09-21T00:29:52.844 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 106, 'state': 'creating+peering', 'last_fresh': '2024-09-21T00:29:48.094970+0000', 'last_change': '2024-09-21T00:29:48.094970+0000', 'last_active': '2024-09-21T00:29:48.079691+0000', 'last_peered': '2024-09-21T00:29:48.079691+0000', 'last_clean': '2024-09-21T00:29:48.079691+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:29:48.094970+0000', 'last_undegraded': '2024-09-21T00:29:48.094970+0000', 'last_fullsized': '2024-09-21T00:29:48.094970+0000', 'mapping_epoch': 106, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 106, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'last_clean_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-21T00:29:52.844 INFO:tasks.ceph.ceph_manager.ceph:PG 17.1b is not active+clean 2024-09-21T00:29:52.844 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:29:47.077435+0000', 'last_change': '2024-09-21T00:29:47.077435+0000', 'last_active': '2024-09-21T00:29:47.077435+0000', 'last_peered': '2024-09-21T00:29:47.077435+0000', 'last_clean': '2024-09-21T00:29:47.077435+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:29:47.077435+0000', 'last_undegraded': '2024-09-21T00:29:47.077435+0000', 'last_fullsized': '2024-09-21T00:29:47.077435+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:29:47.077435+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:29:47.077435+0000', 'last_clean_scrub_stamp': '2024-09-21T00:29:47.077435+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:29:52.844 INFO:tasks.ceph.ceph_manager.ceph:PG 18.19 is not active+clean 2024-09-21T00:29:52.845 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:29:48.079691+0000', 'last_change': '2024-09-21T00:29:48.079691+0000', 'last_active': '2024-09-21T00:29:48.079691+0000', 'last_peered': '2024-09-21T00:29:48.079691+0000', 'last_clean': '2024-09-21T00:29:48.079691+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:29:48.079691+0000', 'last_undegraded': '2024-09-21T00:29:48.079691+0000', 'last_fullsized': '2024-09-21T00:29:48.079691+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'last_clean_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:29:52.845 INFO:tasks.ceph.ceph_manager.ceph:PG 17.1a is not active+clean 2024-09-21T00:29:52.845 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:29:47.077435+0000', 'last_change': '2024-09-21T00:29:47.077435+0000', 'last_active': '2024-09-21T00:29:47.077435+0000', 'last_peered': '2024-09-21T00:29:47.077435+0000', 'last_clean': '2024-09-21T00:29:47.077435+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:29:47.077435+0000', 'last_undegraded': '2024-09-21T00:29:47.077435+0000', 'last_fullsized': '2024-09-21T00:29:47.077435+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:29:47.077435+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:29:47.077435+0000', 'last_clean_scrub_stamp': '2024-09-21T00:29:47.077435+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:29:52.845 INFO:tasks.ceph.ceph_manager.ceph:PG 18.1a is not active+clean 2024-09-21T00:29:52.845 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:29:48.079691+0000', 'last_change': '2024-09-21T00:29:48.079691+0000', 'last_active': '2024-09-21T00:29:48.079691+0000', 'last_peered': '2024-09-21T00:29:48.079691+0000', 'last_clean': '2024-09-21T00:29:48.079691+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:29:48.079691+0000', 'last_undegraded': '2024-09-21T00:29:48.079691+0000', 'last_fullsized': '2024-09-21T00:29:48.079691+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'last_clean_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:29:52.845 INFO:tasks.ceph.ceph_manager.ceph:PG 17.19 is not active+clean 2024-09-21T00:29:52.845 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:29:47.077435+0000', 'last_change': '2024-09-21T00:29:47.077435+0000', 'last_active': '2024-09-21T00:29:47.077435+0000', 'last_peered': '2024-09-21T00:29:47.077435+0000', 'last_clean': '2024-09-21T00:29:47.077435+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:29:47.077435+0000', 'last_undegraded': '2024-09-21T00:29:47.077435+0000', 'last_fullsized': '2024-09-21T00:29:47.077435+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:29:47.077435+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:29:47.077435+0000', 'last_clean_scrub_stamp': '2024-09-21T00:29:47.077435+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:29:52.845 INFO:tasks.ceph.ceph_manager.ceph:PG 18.1b is not active+clean 2024-09-21T00:29:52.845 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 106, 'state': 'creating+peering', 'last_fresh': '2024-09-21T00:29:48.096433+0000', 'last_change': '2024-09-21T00:29:48.096433+0000', 'last_active': '2024-09-21T00:29:48.079691+0000', 'last_peered': '2024-09-21T00:29:48.079691+0000', 'last_clean': '2024-09-21T00:29:48.079691+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:29:48.096433+0000', 'last_undegraded': '2024-09-21T00:29:48.096433+0000', 'last_fullsized': '2024-09-21T00:29:48.096433+0000', 'mapping_epoch': 106, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 106, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'last_clean_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-21T00:29:52.845 INFO:tasks.ceph.ceph_manager.ceph:PG 17.18 is not active+clean 2024-09-21T00:29:52.846 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:29:47.077435+0000', 'last_change': '2024-09-21T00:29:47.077435+0000', 'last_active': '2024-09-21T00:29:47.077435+0000', 'last_peered': '2024-09-21T00:29:47.077435+0000', 'last_clean': '2024-09-21T00:29:47.077435+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:29:47.077435+0000', 'last_undegraded': '2024-09-21T00:29:47.077435+0000', 'last_fullsized': '2024-09-21T00:29:47.077435+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:29:47.077435+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:29:47.077435+0000', 'last_clean_scrub_stamp': '2024-09-21T00:29:47.077435+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:29:52.846 INFO:tasks.ceph.ceph_manager.ceph:PG 18.14 is not active+clean 2024-09-21T00:29:52.846 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:29:48.079691+0000', 'last_change': '2024-09-21T00:29:48.079691+0000', 'last_active': '2024-09-21T00:29:48.079691+0000', 'last_peered': '2024-09-21T00:29:48.079691+0000', 'last_clean': '2024-09-21T00:29:48.079691+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:29:48.079691+0000', 'last_undegraded': '2024-09-21T00:29:48.079691+0000', 'last_fullsized': '2024-09-21T00:29:48.079691+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'last_clean_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:29:52.846 INFO:tasks.ceph.ceph_manager.ceph:PG 17.17 is not active+clean 2024-09-21T00:29:52.846 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:29:47.077435+0000', 'last_change': '2024-09-21T00:29:47.077435+0000', 'last_active': '2024-09-21T00:29:47.077435+0000', 'last_peered': '2024-09-21T00:29:47.077435+0000', 'last_clean': '2024-09-21T00:29:47.077435+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:29:47.077435+0000', 'last_undegraded': '2024-09-21T00:29:47.077435+0000', 'last_fullsized': '2024-09-21T00:29:47.077435+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:29:47.077435+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:29:47.077435+0000', 'last_clean_scrub_stamp': '2024-09-21T00:29:47.077435+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:29:52.846 INFO:tasks.ceph.ceph_manager.ceph:PG 18.15 is not active+clean 2024-09-21T00:29:52.846 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:29:48.079691+0000', 'last_change': '2024-09-21T00:29:48.079691+0000', 'last_active': '2024-09-21T00:29:48.079691+0000', 'last_peered': '2024-09-21T00:29:48.079691+0000', 'last_clean': '2024-09-21T00:29:48.079691+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:29:48.079691+0000', 'last_undegraded': '2024-09-21T00:29:48.079691+0000', 'last_fullsized': '2024-09-21T00:29:48.079691+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'last_clean_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:29:52.846 INFO:tasks.ceph.ceph_manager.ceph:PG 17.16 is not active+clean 2024-09-21T00:29:52.846 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:29:47.077435+0000', 'last_change': '2024-09-21T00:29:47.077435+0000', 'last_active': '2024-09-21T00:29:47.077435+0000', 'last_peered': '2024-09-21T00:29:47.077435+0000', 'last_clean': '2024-09-21T00:29:47.077435+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:29:47.077435+0000', 'last_undegraded': '2024-09-21T00:29:47.077435+0000', 'last_fullsized': '2024-09-21T00:29:47.077435+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:29:47.077435+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:29:47.077435+0000', 'last_clean_scrub_stamp': '2024-09-21T00:29:47.077435+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:29:52.846 INFO:tasks.ceph.ceph_manager.ceph:PG 18.16 is not active+clean 2024-09-21T00:29:52.846 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:29:48.079691+0000', 'last_change': '2024-09-21T00:29:48.079691+0000', 'last_active': '2024-09-21T00:29:48.079691+0000', 'last_peered': '2024-09-21T00:29:48.079691+0000', 'last_clean': '2024-09-21T00:29:48.079691+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:29:48.079691+0000', 'last_undegraded': '2024-09-21T00:29:48.079691+0000', 'last_fullsized': '2024-09-21T00:29:48.079691+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'last_clean_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:29:52.846 INFO:tasks.ceph.ceph_manager.ceph:PG 17.15 is not active+clean 2024-09-21T00:29:52.847 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:29:47.077435+0000', 'last_change': '2024-09-21T00:29:47.077435+0000', 'last_active': '2024-09-21T00:29:47.077435+0000', 'last_peered': '2024-09-21T00:29:47.077435+0000', 'last_clean': '2024-09-21T00:29:47.077435+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:29:47.077435+0000', 'last_undegraded': '2024-09-21T00:29:47.077435+0000', 'last_fullsized': '2024-09-21T00:29:47.077435+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:29:47.077435+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:29:47.077435+0000', 'last_clean_scrub_stamp': '2024-09-21T00:29:47.077435+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:29:52.847 INFO:tasks.ceph.ceph_manager.ceph:PG 18.17 is not active+clean 2024-09-21T00:29:52.847 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:29:48.079691+0000', 'last_change': '2024-09-21T00:29:48.079691+0000', 'last_active': '2024-09-21T00:29:48.079691+0000', 'last_peered': '2024-09-21T00:29:48.079691+0000', 'last_clean': '2024-09-21T00:29:48.079691+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:29:48.079691+0000', 'last_undegraded': '2024-09-21T00:29:48.079691+0000', 'last_fullsized': '2024-09-21T00:29:48.079691+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'last_clean_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:29:52.847 INFO:tasks.ceph.ceph_manager.ceph:PG 17.14 is not active+clean 2024-09-21T00:29:52.847 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:29:47.077435+0000', 'last_change': '2024-09-21T00:29:47.077435+0000', 'last_active': '2024-09-21T00:29:47.077435+0000', 'last_peered': '2024-09-21T00:29:47.077435+0000', 'last_clean': '2024-09-21T00:29:47.077435+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:29:47.077435+0000', 'last_undegraded': '2024-09-21T00:29:47.077435+0000', 'last_fullsized': '2024-09-21T00:29:47.077435+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:29:47.077435+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:29:47.077435+0000', 'last_clean_scrub_stamp': '2024-09-21T00:29:47.077435+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:29:52.847 INFO:tasks.ceph.ceph_manager.ceph:PG 18.10 is not active+clean 2024-09-21T00:29:52.847 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:29:48.079691+0000', 'last_change': '2024-09-21T00:29:48.079691+0000', 'last_active': '2024-09-21T00:29:48.079691+0000', 'last_peered': '2024-09-21T00:29:48.079691+0000', 'last_clean': '2024-09-21T00:29:48.079691+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:29:48.079691+0000', 'last_undegraded': '2024-09-21T00:29:48.079691+0000', 'last_fullsized': '2024-09-21T00:29:48.079691+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'last_clean_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:29:52.847 INFO:tasks.ceph.ceph_manager.ceph:PG 17.13 is not active+clean 2024-09-21T00:29:52.847 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:29:47.077435+0000', 'last_change': '2024-09-21T00:29:47.077435+0000', 'last_active': '2024-09-21T00:29:47.077435+0000', 'last_peered': '2024-09-21T00:29:47.077435+0000', 'last_clean': '2024-09-21T00:29:47.077435+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:29:47.077435+0000', 'last_undegraded': '2024-09-21T00:29:47.077435+0000', 'last_fullsized': '2024-09-21T00:29:47.077435+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:29:47.077435+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:29:47.077435+0000', 'last_clean_scrub_stamp': '2024-09-21T00:29:47.077435+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:29:52.847 INFO:tasks.ceph.ceph_manager.ceph:PG 18.11 is not active+clean 2024-09-21T00:29:52.847 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 106, 'state': 'creating+peering', 'last_fresh': '2024-09-21T00:29:48.095354+0000', 'last_change': '2024-09-21T00:29:48.095354+0000', 'last_active': '2024-09-21T00:29:48.079691+0000', 'last_peered': '2024-09-21T00:29:48.079691+0000', 'last_clean': '2024-09-21T00:29:48.079691+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:29:48.095354+0000', 'last_undegraded': '2024-09-21T00:29:48.095354+0000', 'last_fullsized': '2024-09-21T00:29:48.095354+0000', 'mapping_epoch': 106, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 106, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'last_clean_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-21T00:29:52.847 INFO:tasks.ceph.ceph_manager.ceph:PG 17.12 is not active+clean 2024-09-21T00:29:52.848 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:29:47.077435+0000', 'last_change': '2024-09-21T00:29:47.077435+0000', 'last_active': '2024-09-21T00:29:47.077435+0000', 'last_peered': '2024-09-21T00:29:47.077435+0000', 'last_clean': '2024-09-21T00:29:47.077435+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:29:47.077435+0000', 'last_undegraded': '2024-09-21T00:29:47.077435+0000', 'last_fullsized': '2024-09-21T00:29:47.077435+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:29:47.077435+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:29:47.077435+0000', 'last_clean_scrub_stamp': '2024-09-21T00:29:47.077435+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:29:52.848 INFO:tasks.ceph.ceph_manager.ceph:PG 18.12 is not active+clean 2024-09-21T00:29:52.848 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:29:48.079691+0000', 'last_change': '2024-09-21T00:29:48.079691+0000', 'last_active': '2024-09-21T00:29:48.079691+0000', 'last_peered': '2024-09-21T00:29:48.079691+0000', 'last_clean': '2024-09-21T00:29:48.079691+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:29:48.079691+0000', 'last_undegraded': '2024-09-21T00:29:48.079691+0000', 'last_fullsized': '2024-09-21T00:29:48.079691+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'last_clean_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:29:52.848 INFO:tasks.ceph.ceph_manager.ceph:PG 17.11 is not active+clean 2024-09-21T00:29:52.848 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:29:47.077435+0000', 'last_change': '2024-09-21T00:29:47.077435+0000', 'last_active': '2024-09-21T00:29:47.077435+0000', 'last_peered': '2024-09-21T00:29:47.077435+0000', 'last_clean': '2024-09-21T00:29:47.077435+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:29:47.077435+0000', 'last_undegraded': '2024-09-21T00:29:47.077435+0000', 'last_fullsized': '2024-09-21T00:29:47.077435+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:29:47.077435+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:29:47.077435+0000', 'last_clean_scrub_stamp': '2024-09-21T00:29:47.077435+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:29:52.848 INFO:tasks.ceph.ceph_manager.ceph:PG 18.c is not active+clean 2024-09-21T00:29:52.848 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:29:48.079691+0000', 'last_change': '2024-09-21T00:29:48.079691+0000', 'last_active': '2024-09-21T00:29:48.079691+0000', 'last_peered': '2024-09-21T00:29:48.079691+0000', 'last_clean': '2024-09-21T00:29:48.079691+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:29:48.079691+0000', 'last_undegraded': '2024-09-21T00:29:48.079691+0000', 'last_fullsized': '2024-09-21T00:29:48.079691+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'last_clean_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:29:52.848 INFO:tasks.ceph.ceph_manager.ceph:PG 18.d is not active+clean 2024-09-21T00:29:52.848 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:29:48.079691+0000', 'last_change': '2024-09-21T00:29:48.079691+0000', 'last_active': '2024-09-21T00:29:48.079691+0000', 'last_peered': '2024-09-21T00:29:48.079691+0000', 'last_clean': '2024-09-21T00:29:48.079691+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:29:48.079691+0000', 'last_undegraded': '2024-09-21T00:29:48.079691+0000', 'last_fullsized': '2024-09-21T00:29:48.079691+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'last_clean_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:29:52.848 INFO:tasks.ceph.ceph_manager.ceph:PG 17.e is not active+clean 2024-09-21T00:29:52.848 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:29:47.077435+0000', 'last_change': '2024-09-21T00:29:47.077435+0000', 'last_active': '2024-09-21T00:29:47.077435+0000', 'last_peered': '2024-09-21T00:29:47.077435+0000', 'last_clean': '2024-09-21T00:29:47.077435+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:29:47.077435+0000', 'last_undegraded': '2024-09-21T00:29:47.077435+0000', 'last_fullsized': '2024-09-21T00:29:47.077435+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:29:47.077435+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:29:47.077435+0000', 'last_clean_scrub_stamp': '2024-09-21T00:29:47.077435+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:29:52.849 INFO:tasks.ceph.ceph_manager.ceph:PG 18.e is not active+clean 2024-09-21T00:29:52.849 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:29:48.079691+0000', 'last_change': '2024-09-21T00:29:48.079691+0000', 'last_active': '2024-09-21T00:29:48.079691+0000', 'last_peered': '2024-09-21T00:29:48.079691+0000', 'last_clean': '2024-09-21T00:29:48.079691+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:29:48.079691+0000', 'last_undegraded': '2024-09-21T00:29:48.079691+0000', 'last_fullsized': '2024-09-21T00:29:48.079691+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'last_clean_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:29:52.849 INFO:tasks.ceph.ceph_manager.ceph:PG 17.d is not active+clean 2024-09-21T00:29:52.849 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:29:47.077435+0000', 'last_change': '2024-09-21T00:29:47.077435+0000', 'last_active': '2024-09-21T00:29:47.077435+0000', 'last_peered': '2024-09-21T00:29:47.077435+0000', 'last_clean': '2024-09-21T00:29:47.077435+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:29:47.077435+0000', 'last_undegraded': '2024-09-21T00:29:47.077435+0000', 'last_fullsized': '2024-09-21T00:29:47.077435+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:29:47.077435+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:29:47.077435+0000', 'last_clean_scrub_stamp': '2024-09-21T00:29:47.077435+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:29:52.849 INFO:tasks.ceph.ceph_manager.ceph:PG 18.f is not active+clean 2024-09-21T00:29:52.849 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:29:48.079691+0000', 'last_change': '2024-09-21T00:29:48.079691+0000', 'last_active': '2024-09-21T00:29:48.079691+0000', 'last_peered': '2024-09-21T00:29:48.079691+0000', 'last_clean': '2024-09-21T00:29:48.079691+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:29:48.079691+0000', 'last_undegraded': '2024-09-21T00:29:48.079691+0000', 'last_fullsized': '2024-09-21T00:29:48.079691+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'last_clean_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:29:52.849 INFO:tasks.ceph.ceph_manager.ceph:PG 17.c is not active+clean 2024-09-21T00:29:52.849 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:29:47.077435+0000', 'last_change': '2024-09-21T00:29:47.077435+0000', 'last_active': '2024-09-21T00:29:47.077435+0000', 'last_peered': '2024-09-21T00:29:47.077435+0000', 'last_clean': '2024-09-21T00:29:47.077435+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:29:47.077435+0000', 'last_undegraded': '2024-09-21T00:29:47.077435+0000', 'last_fullsized': '2024-09-21T00:29:47.077435+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:29:47.077435+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:29:47.077435+0000', 'last_clean_scrub_stamp': '2024-09-21T00:29:47.077435+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:29:52.849 INFO:tasks.ceph.ceph_manager.ceph:PG 18.8 is not active+clean 2024-09-21T00:29:52.849 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:29:48.079691+0000', 'last_change': '2024-09-21T00:29:48.079691+0000', 'last_active': '2024-09-21T00:29:48.079691+0000', 'last_peered': '2024-09-21T00:29:48.079691+0000', 'last_clean': '2024-09-21T00:29:48.079691+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:29:48.079691+0000', 'last_undegraded': '2024-09-21T00:29:48.079691+0000', 'last_fullsized': '2024-09-21T00:29:48.079691+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'last_clean_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:29:52.849 INFO:tasks.ceph.ceph_manager.ceph:PG 17.b is not active+clean 2024-09-21T00:29:52.849 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:29:47.077435+0000', 'last_change': '2024-09-21T00:29:47.077435+0000', 'last_active': '2024-09-21T00:29:47.077435+0000', 'last_peered': '2024-09-21T00:29:47.077435+0000', 'last_clean': '2024-09-21T00:29:47.077435+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:29:47.077435+0000', 'last_undegraded': '2024-09-21T00:29:47.077435+0000', 'last_fullsized': '2024-09-21T00:29:47.077435+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:29:47.077435+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:29:47.077435+0000', 'last_clean_scrub_stamp': '2024-09-21T00:29:47.077435+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:29:52.850 INFO:tasks.ceph.ceph_manager.ceph:PG 18.9 is not active+clean 2024-09-21T00:29:52.850 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:29:48.079691+0000', 'last_change': '2024-09-21T00:29:48.079691+0000', 'last_active': '2024-09-21T00:29:48.079691+0000', 'last_peered': '2024-09-21T00:29:48.079691+0000', 'last_clean': '2024-09-21T00:29:48.079691+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:29:48.079691+0000', 'last_undegraded': '2024-09-21T00:29:48.079691+0000', 'last_fullsized': '2024-09-21T00:29:48.079691+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'last_clean_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:29:52.850 INFO:tasks.ceph.ceph_manager.ceph:PG 17.a is not active+clean 2024-09-21T00:29:52.850 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:29:47.077435+0000', 'last_change': '2024-09-21T00:29:47.077435+0000', 'last_active': '2024-09-21T00:29:47.077435+0000', 'last_peered': '2024-09-21T00:29:47.077435+0000', 'last_clean': '2024-09-21T00:29:47.077435+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:29:47.077435+0000', 'last_undegraded': '2024-09-21T00:29:47.077435+0000', 'last_fullsized': '2024-09-21T00:29:47.077435+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:29:47.077435+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:29:47.077435+0000', 'last_clean_scrub_stamp': '2024-09-21T00:29:47.077435+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:29:52.850 INFO:tasks.ceph.ceph_manager.ceph:PG 18.a is not active+clean 2024-09-21T00:29:52.850 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:29:48.079691+0000', 'last_change': '2024-09-21T00:29:48.079691+0000', 'last_active': '2024-09-21T00:29:48.079691+0000', 'last_peered': '2024-09-21T00:29:48.079691+0000', 'last_clean': '2024-09-21T00:29:48.079691+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:29:48.079691+0000', 'last_undegraded': '2024-09-21T00:29:48.079691+0000', 'last_fullsized': '2024-09-21T00:29:48.079691+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'last_clean_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:29:52.850 INFO:tasks.ceph.ceph_manager.ceph:PG 17.9 is not active+clean 2024-09-21T00:29:52.850 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:29:47.077435+0000', 'last_change': '2024-09-21T00:29:47.077435+0000', 'last_active': '2024-09-21T00:29:47.077435+0000', 'last_peered': '2024-09-21T00:29:47.077435+0000', 'last_clean': '2024-09-21T00:29:47.077435+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:29:47.077435+0000', 'last_undegraded': '2024-09-21T00:29:47.077435+0000', 'last_fullsized': '2024-09-21T00:29:47.077435+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:29:47.077435+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:29:47.077435+0000', 'last_clean_scrub_stamp': '2024-09-21T00:29:47.077435+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:29:52.850 INFO:tasks.ceph.ceph_manager.ceph:PG 18.b is not active+clean 2024-09-21T00:29:52.850 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:29:48.079691+0000', 'last_change': '2024-09-21T00:29:48.079691+0000', 'last_active': '2024-09-21T00:29:48.079691+0000', 'last_peered': '2024-09-21T00:29:48.079691+0000', 'last_clean': '2024-09-21T00:29:48.079691+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:29:48.079691+0000', 'last_undegraded': '2024-09-21T00:29:48.079691+0000', 'last_fullsized': '2024-09-21T00:29:48.079691+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'last_clean_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:29:52.850 INFO:tasks.ceph.ceph_manager.ceph:PG 17.8 is not active+clean 2024-09-21T00:29:52.850 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:29:47.077435+0000', 'last_change': '2024-09-21T00:29:47.077435+0000', 'last_active': '2024-09-21T00:29:47.077435+0000', 'last_peered': '2024-09-21T00:29:47.077435+0000', 'last_clean': '2024-09-21T00:29:47.077435+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:29:47.077435+0000', 'last_undegraded': '2024-09-21T00:29:47.077435+0000', 'last_fullsized': '2024-09-21T00:29:47.077435+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:29:47.077435+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:29:47.077435+0000', 'last_clean_scrub_stamp': '2024-09-21T00:29:47.077435+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:29:52.851 INFO:tasks.ceph.ceph_manager.ceph:PG 18.4 is not active+clean 2024-09-21T00:29:52.851 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:29:48.079691+0000', 'last_change': '2024-09-21T00:29:48.079691+0000', 'last_active': '2024-09-21T00:29:48.079691+0000', 'last_peered': '2024-09-21T00:29:48.079691+0000', 'last_clean': '2024-09-21T00:29:48.079691+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:29:48.079691+0000', 'last_undegraded': '2024-09-21T00:29:48.079691+0000', 'last_fullsized': '2024-09-21T00:29:48.079691+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'last_clean_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:29:52.851 INFO:tasks.ceph.ceph_manager.ceph:PG 17.7 is not active+clean 2024-09-21T00:29:52.851 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:29:47.077435+0000', 'last_change': '2024-09-21T00:29:47.077435+0000', 'last_active': '2024-09-21T00:29:47.077435+0000', 'last_peered': '2024-09-21T00:29:47.077435+0000', 'last_clean': '2024-09-21T00:29:47.077435+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:29:47.077435+0000', 'last_undegraded': '2024-09-21T00:29:47.077435+0000', 'last_fullsized': '2024-09-21T00:29:47.077435+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:29:47.077435+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:29:47.077435+0000', 'last_clean_scrub_stamp': '2024-09-21T00:29:47.077435+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:29:52.851 INFO:tasks.ceph.ceph_manager.ceph:PG 18.5 is not active+clean 2024-09-21T00:29:52.851 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 106, 'state': 'creating+peering', 'last_fresh': '2024-09-21T00:29:48.095849+0000', 'last_change': '2024-09-21T00:29:48.095849+0000', 'last_active': '2024-09-21T00:29:48.079691+0000', 'last_peered': '2024-09-21T00:29:48.079691+0000', 'last_clean': '2024-09-21T00:29:48.079691+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:29:48.095849+0000', 'last_undegraded': '2024-09-21T00:29:48.095849+0000', 'last_fullsized': '2024-09-21T00:29:48.095849+0000', 'mapping_epoch': 106, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 106, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'last_clean_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-21T00:29:52.851 INFO:tasks.ceph.ceph_manager.ceph:PG 17.6 is not active+clean 2024-09-21T00:29:52.851 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:29:47.077435+0000', 'last_change': '2024-09-21T00:29:47.077435+0000', 'last_active': '2024-09-21T00:29:47.077435+0000', 'last_peered': '2024-09-21T00:29:47.077435+0000', 'last_clean': '2024-09-21T00:29:47.077435+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:29:47.077435+0000', 'last_undegraded': '2024-09-21T00:29:47.077435+0000', 'last_fullsized': '2024-09-21T00:29:47.077435+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:29:47.077435+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:29:47.077435+0000', 'last_clean_scrub_stamp': '2024-09-21T00:29:47.077435+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:29:52.851 INFO:tasks.ceph.ceph_manager.ceph:PG 18.6 is not active+clean 2024-09-21T00:29:52.851 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:29:48.079691+0000', 'last_change': '2024-09-21T00:29:48.079691+0000', 'last_active': '2024-09-21T00:29:48.079691+0000', 'last_peered': '2024-09-21T00:29:48.079691+0000', 'last_clean': '2024-09-21T00:29:48.079691+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:29:48.079691+0000', 'last_undegraded': '2024-09-21T00:29:48.079691+0000', 'last_fullsized': '2024-09-21T00:29:48.079691+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'last_clean_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:29:52.851 INFO:tasks.ceph.ceph_manager.ceph:PG 17.5 is not active+clean 2024-09-21T00:29:52.851 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:29:47.077435+0000', 'last_change': '2024-09-21T00:29:47.077435+0000', 'last_active': '2024-09-21T00:29:47.077435+0000', 'last_peered': '2024-09-21T00:29:47.077435+0000', 'last_clean': '2024-09-21T00:29:47.077435+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:29:47.077435+0000', 'last_undegraded': '2024-09-21T00:29:47.077435+0000', 'last_fullsized': '2024-09-21T00:29:47.077435+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:29:47.077435+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:29:47.077435+0000', 'last_clean_scrub_stamp': '2024-09-21T00:29:47.077435+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:29:52.852 INFO:tasks.ceph.ceph_manager.ceph:PG 18.7 is not active+clean 2024-09-21T00:29:52.852 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:29:48.079691+0000', 'last_change': '2024-09-21T00:29:48.079691+0000', 'last_active': '2024-09-21T00:29:48.079691+0000', 'last_peered': '2024-09-21T00:29:48.079691+0000', 'last_clean': '2024-09-21T00:29:48.079691+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:29:48.079691+0000', 'last_undegraded': '2024-09-21T00:29:48.079691+0000', 'last_fullsized': '2024-09-21T00:29:48.079691+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'last_clean_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:29:52.852 INFO:tasks.ceph.ceph_manager.ceph:PG 17.4 is not active+clean 2024-09-21T00:29:52.852 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:29:47.077435+0000', 'last_change': '2024-09-21T00:29:47.077435+0000', 'last_active': '2024-09-21T00:29:47.077435+0000', 'last_peered': '2024-09-21T00:29:47.077435+0000', 'last_clean': '2024-09-21T00:29:47.077435+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:29:47.077435+0000', 'last_undegraded': '2024-09-21T00:29:47.077435+0000', 'last_fullsized': '2024-09-21T00:29:47.077435+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:29:47.077435+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:29:47.077435+0000', 'last_clean_scrub_stamp': '2024-09-21T00:29:47.077435+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:29:52.852 INFO:tasks.ceph.ceph_manager.ceph:PG 18.0 is not active+clean 2024-09-21T00:29:52.852 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:29:48.079691+0000', 'last_change': '2024-09-21T00:29:48.079691+0000', 'last_active': '2024-09-21T00:29:48.079691+0000', 'last_peered': '2024-09-21T00:29:48.079691+0000', 'last_clean': '2024-09-21T00:29:48.079691+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:29:48.079691+0000', 'last_undegraded': '2024-09-21T00:29:48.079691+0000', 'last_fullsized': '2024-09-21T00:29:48.079691+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'last_clean_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:29:52.852 INFO:tasks.ceph.ceph_manager.ceph:PG 17.3 is not active+clean 2024-09-21T00:29:52.852 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:29:47.077435+0000', 'last_change': '2024-09-21T00:29:47.077435+0000', 'last_active': '2024-09-21T00:29:47.077435+0000', 'last_peered': '2024-09-21T00:29:47.077435+0000', 'last_clean': '2024-09-21T00:29:47.077435+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:29:47.077435+0000', 'last_undegraded': '2024-09-21T00:29:47.077435+0000', 'last_fullsized': '2024-09-21T00:29:47.077435+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:29:47.077435+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:29:47.077435+0000', 'last_clean_scrub_stamp': '2024-09-21T00:29:47.077435+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:29:52.852 INFO:tasks.ceph.ceph_manager.ceph:PG 18.1 is not active+clean 2024-09-21T00:29:52.852 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:29:48.079691+0000', 'last_change': '2024-09-21T00:29:48.079691+0000', 'last_active': '2024-09-21T00:29:48.079691+0000', 'last_peered': '2024-09-21T00:29:48.079691+0000', 'last_clean': '2024-09-21T00:29:48.079691+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:29:48.079691+0000', 'last_undegraded': '2024-09-21T00:29:48.079691+0000', 'last_fullsized': '2024-09-21T00:29:48.079691+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'last_clean_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:29:52.852 INFO:tasks.ceph.ceph_manager.ceph:PG 17.2 is not active+clean 2024-09-21T00:29:52.852 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:29:47.077435+0000', 'last_change': '2024-09-21T00:29:47.077435+0000', 'last_active': '2024-09-21T00:29:47.077435+0000', 'last_peered': '2024-09-21T00:29:47.077435+0000', 'last_clean': '2024-09-21T00:29:47.077435+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:29:47.077435+0000', 'last_undegraded': '2024-09-21T00:29:47.077435+0000', 'last_fullsized': '2024-09-21T00:29:47.077435+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:29:47.077435+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:29:47.077435+0000', 'last_clean_scrub_stamp': '2024-09-21T00:29:47.077435+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:29:52.853 INFO:tasks.ceph.ceph_manager.ceph:PG 18.2 is not active+clean 2024-09-21T00:29:52.853 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:29:48.079691+0000', 'last_change': '2024-09-21T00:29:48.079691+0000', 'last_active': '2024-09-21T00:29:48.079691+0000', 'last_peered': '2024-09-21T00:29:48.079691+0000', 'last_clean': '2024-09-21T00:29:48.079691+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:29:48.079691+0000', 'last_undegraded': '2024-09-21T00:29:48.079691+0000', 'last_fullsized': '2024-09-21T00:29:48.079691+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'last_clean_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:29:52.853 INFO:tasks.ceph.ceph_manager.ceph:PG 17.1 is not active+clean 2024-09-21T00:29:52.853 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:29:47.077435+0000', 'last_change': '2024-09-21T00:29:47.077435+0000', 'last_active': '2024-09-21T00:29:47.077435+0000', 'last_peered': '2024-09-21T00:29:47.077435+0000', 'last_clean': '2024-09-21T00:29:47.077435+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:29:47.077435+0000', 'last_undegraded': '2024-09-21T00:29:47.077435+0000', 'last_fullsized': '2024-09-21T00:29:47.077435+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:29:47.077435+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:29:47.077435+0000', 'last_clean_scrub_stamp': '2024-09-21T00:29:47.077435+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:29:52.853 INFO:tasks.ceph.ceph_manager.ceph:PG 18.3 is not active+clean 2024-09-21T00:29:52.853 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 106, 'state': 'creating+peering', 'last_fresh': '2024-09-21T00:29:48.095876+0000', 'last_change': '2024-09-21T00:29:48.095876+0000', 'last_active': '2024-09-21T00:29:48.079691+0000', 'last_peered': '2024-09-21T00:29:48.079691+0000', 'last_clean': '2024-09-21T00:29:48.079691+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:29:48.095876+0000', 'last_undegraded': '2024-09-21T00:29:48.095876+0000', 'last_fullsized': '2024-09-21T00:29:48.095876+0000', 'mapping_epoch': 106, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 106, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'last_clean_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-21T00:29:52.853 INFO:tasks.ceph.ceph_manager.ceph:PG 17.0 is not active+clean 2024-09-21T00:29:52.853 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:29:47.077435+0000', 'last_change': '2024-09-21T00:29:47.077435+0000', 'last_active': '2024-09-21T00:29:47.077435+0000', 'last_peered': '2024-09-21T00:29:47.077435+0000', 'last_clean': '2024-09-21T00:29:47.077435+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:29:47.077435+0000', 'last_undegraded': '2024-09-21T00:29:47.077435+0000', 'last_fullsized': '2024-09-21T00:29:47.077435+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:29:47.077435+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:29:47.077435+0000', 'last_clean_scrub_stamp': '2024-09-21T00:29:47.077435+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:29:52.853 INFO:tasks.ceph.ceph_manager.ceph:PG 18.13 is not active+clean 2024-09-21T00:29:52.853 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:29:48.079691+0000', 'last_change': '2024-09-21T00:29:48.079691+0000', 'last_active': '2024-09-21T00:29:48.079691+0000', 'last_peered': '2024-09-21T00:29:48.079691+0000', 'last_clean': '2024-09-21T00:29:48.079691+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:29:48.079691+0000', 'last_undegraded': '2024-09-21T00:29:48.079691+0000', 'last_fullsized': '2024-09-21T00:29:48.079691+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'last_clean_scrub_stamp': '2024-09-21T00:29:48.079691+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:29:52.853 INFO:tasks.ceph.ceph_manager.ceph:PG 17.10 is not active+clean 2024-09-21T00:29:52.853 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T00:29:47.077435+0000', 'last_change': '2024-09-21T00:29:47.077435+0000', 'last_active': '2024-09-21T00:29:47.077435+0000', 'last_peered': '2024-09-21T00:29:47.077435+0000', 'last_clean': '2024-09-21T00:29:47.077435+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T00:29:47.077435+0000', 'last_undegraded': '2024-09-21T00:29:47.077435+0000', 'last_fullsized': '2024-09-21T00:29:47.077435+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T00:29:47.077435+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T00:29:47.077435+0000', 'last_clean_scrub_stamp': '2024-09-21T00:29:47.077435+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T00:29:52.854 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-21T00:29:52.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.982+0000 7fda64ed5640 1 -- 172.21.15.112:0/2946103079 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda60164eb0 msgr2=0x7fda601672d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:52.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.982+0000 7fda64ed5640 1 --2- 172.21.15.112:0/2946103079 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda60164eb0 0x7fda601672d0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fda58005fd0 tx=0x7fda5802a980 comp rx=0 tx=0).stop 2024-09-21T00:29:52.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.982+0000 7fda64ed5640 1 -- 172.21.15.112:0/2946103079 shutdown_connections 2024-09-21T00:29:52.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.982+0000 7fda64ed5640 1 --2- 172.21.15.112:0/2946103079 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda60164eb0 0x7fda601672d0 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:52.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.982+0000 7fda64ed5640 1 -- 172.21.15.112:0/2946103079 >> 172.21.15.112:0/2946103079 conn(0x7fda600b42e0 msgr2=0x7fda600b4720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:52.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.982+0000 7fda64ed5640 1 -- 172.21.15.112:0/2946103079 shutdown_connections 2024-09-21T00:29:52.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.982+0000 7fda64ed5640 1 -- 172.21.15.112:0/2946103079 wait complete. 2024-09-21T00:29:52.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.984+0000 7fda64ed5640 1 Processor -- start 2024-09-21T00:29:52.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.984+0000 7fda64ed5640 1 -- start start 2024-09-21T00:29:52.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.985+0000 7fda64ed5640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fda600b1d30 0x7fda60145760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:52.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.985+0000 7fda64ed5640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fda600ccbf0 0x7fda601550e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:52.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.985+0000 7fda64ed5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda60164eb0 0x7fda60155620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:52.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.985+0000 7fda64ed5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fda60167bd0 con 0x7fda60164eb0 2024-09-21T00:29:52.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.985+0000 7fda64ed5640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fda60167a50 con 0x7fda600ccbf0 2024-09-21T00:29:52.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.985+0000 7fda64ed5640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fda60167d50 con 0x7fda600b1d30 2024-09-21T00:29:52.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.985+0000 7fda5ed9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda60164eb0 0x7fda60155620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:52.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.985+0000 7fda5ed9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda60164eb0 0x7fda60155620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:32858/0 (socket says 172.21.15.112:32858) 2024-09-21T00:29:52.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.985+0000 7fda5ed9d640 1 -- 172.21.15.112:0/1918765048 learned_addr learned my addr 172.21.15.112:0/1918765048 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:29:52.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.985+0000 7fda5e59c640 1 --2- 172.21.15.112:0/1918765048 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fda600b1d30 0x7fda60145760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:52.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.985+0000 7fda55d9b640 1 --2- 172.21.15.112:0/1918765048 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fda600ccbf0 0x7fda601550e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:52.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.985+0000 7fda5ed9d640 1 -- 172.21.15.112:0/1918765048 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fda600b1d30 msgr2=0x7fda60145760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:52.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.985+0000 7fda5ed9d640 1 --2- 172.21.15.112:0/1918765048 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fda600b1d30 0x7fda60145760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:52.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.985+0000 7fda5ed9d640 1 -- 172.21.15.112:0/1918765048 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fda600ccbf0 msgr2=0x7fda601550e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:52.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.985+0000 7fda5ed9d640 1 --2- 172.21.15.112:0/1918765048 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fda600ccbf0 0x7fda601550e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:52.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.985+0000 7fda5ed9d640 1 -- 172.21.15.112:0/1918765048 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda60155b60 con 0x7fda60164eb0 2024-09-21T00:29:52.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.986+0000 7fda5ed9d640 1 --2- 172.21.15.112:0/1918765048 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda60164eb0 0x7fda60155620 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fda58009d90 tx=0x7fda58003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:52.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.986+0000 7fda577fe640 1 -- 172.21.15.112:0/1918765048 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda58034040 con 0x7fda60164eb0 2024-09-21T00:29:52.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.986+0000 7fda64ed5640 1 -- 172.21.15.112:0/1918765048 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda60154320 con 0x7fda60164eb0 2024-09-21T00:29:52.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.986+0000 7fda577fe640 1 -- 172.21.15.112:0/1918765048 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fda580045e0 con 0x7fda60164eb0 2024-09-21T00:29:52.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.986+0000 7fda577fe640 1 -- 172.21.15.112:0/1918765048 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda58045480 con 0x7fda60164eb0 2024-09-21T00:29:52.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.986+0000 7fda64ed5640 1 -- 172.21.15.112:0/1918765048 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda601548e0 con 0x7fda60164eb0 2024-09-21T00:29:52.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.987+0000 7fda64ed5640 1 -- 172.21.15.112:0/1918765048 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda600cd000 con 0x7fda60164eb0 2024-09-21T00:29:52.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.988+0000 7fda577fe640 1 -- 172.21.15.112:0/1918765048 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fda58037020 con 0x7fda60164eb0 2024-09-21T00:29:52.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.988+0000 7fda577fe640 1 --2- 172.21.15.112:0/1918765048 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fda40078b80 0x7fda4007b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:52.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.988+0000 7fda577fe640 1 -- 172.21.15.112:0/1918765048 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 6101+0+0 (secure 0 0 0) 0x7fda580bc750 con 0x7fda60164eb0 2024-09-21T00:29:52.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.988+0000 7fda5e59c640 1 --2- 172.21.15.112:0/1918765048 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fda40078b80 0x7fda4007b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:52.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.989+0000 7fda5e59c640 1 --2- 172.21.15.112:0/1918765048 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fda40078b80 0x7fda4007b090 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fda50001720 tx=0x7fda50009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:52.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:52.993+0000 7fda577fe640 1 -- 172.21.15.112:0/1918765048 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fda58084450 con 0x7fda60164eb0 2024-09-21T00:29:53.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:53.239+0000 7fda64ed5640 1 -- 172.21.15.112:0/1918765048 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fda60156e70 con 0x7fda60164eb0 2024-09-21T00:29:53.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:53.239+0000 7fda577fe640 1 -- 172.21.15.112:0/1918765048 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1571 (secure 0 0 0) 0x7fda58088e80 con 0x7fda60164eb0 2024-09-21T00:29:53.241 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:29:53.242 INFO:teuthology.orchestra.run.smithi112.stdout:{"fsid":"8b34692b-dc65-49b2-a08f-26f707b96ac5","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":270,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":108,"num_osds":8,"num_up_osds":8,"osd_up_since":1726878336,"num_in_osds":8,"osd_in_since":1726878326,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":84},{"state_name":"creating+peering","count":9},{"state_name":"active+clean","count":4}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":267087872,"bytes_avail":772827025408,"bytes_total":773094113280,"unknown_pgs_ratio":0.86597937345504761,"inactive_pgs_ratio":0.092783503234386444},"fsmap":{"epoch":70,"btime":"2024-09-21T00:29:51:115169+0000","id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"d","status":"up:active","gid":7167}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-21T00:29:37.149171+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-21T00:29:53.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:53.242+0000 7fda64ed5640 1 -- 172.21.15.112:0/1918765048 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fda40078b80 msgr2=0x7fda4007b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:53.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:53.242+0000 7fda64ed5640 1 --2- 172.21.15.112:0/1918765048 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fda40078b80 0x7fda4007b090 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fda50001720 tx=0x7fda50009040 comp rx=0 tx=0).stop 2024-09-21T00:29:53.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:53.242+0000 7fda64ed5640 1 -- 172.21.15.112:0/1918765048 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda60164eb0 msgr2=0x7fda60155620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:53.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:53.242+0000 7fda64ed5640 1 --2- 172.21.15.112:0/1918765048 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda60164eb0 0x7fda60155620 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fda58009d90 tx=0x7fda58003f90 comp rx=0 tx=0).stop 2024-09-21T00:29:53.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:53.242+0000 7fda64ed5640 1 -- 172.21.15.112:0/1918765048 shutdown_connections 2024-09-21T00:29:53.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:53.242+0000 7fda64ed5640 1 --2- 172.21.15.112:0/1918765048 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fda40078b80 0x7fda4007b090 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:53.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:53.242+0000 7fda64ed5640 1 --2- 172.21.15.112:0/1918765048 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda60164eb0 0x7fda60155620 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:53.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:53.242+0000 7fda64ed5640 1 --2- 172.21.15.112:0/1918765048 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fda600ccbf0 0x7fda601550e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:53.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:53.242+0000 7fda64ed5640 1 --2- 172.21.15.112:0/1918765048 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fda600b1d30 0x7fda60145760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:53.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:53.242+0000 7fda64ed5640 1 -- 172.21.15.112:0/1918765048 >> 172.21.15.112:0/1918765048 conn(0x7fda600b42e0 msgr2=0x7fda60070990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:53.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:53.242+0000 7fda64ed5640 1 -- 172.21.15.112:0/1918765048 shutdown_connections 2024-09-21T00:29:53.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:53.242+0000 7fda64ed5640 1 -- 172.21.15.112:0/1918765048 wait complete. 2024-09-21T00:29:53.255 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-21T00:29:53.255 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T00:29:53.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:53.366+0000 7f932e027640 1 -- 172.21.15.112:0/4145102913 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9310003ae0 msgr2=0x7f9310003f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:53.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:53.366+0000 7f932e027640 1 --2- 172.21.15.112:0/4145102913 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9310003ae0 0x7f9310003f50 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f931800c2e0 tx=0x7f9310005830 comp rx=0 tx=0).stop 2024-09-21T00:29:53.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:53.366+0000 7f9327fff640 1 -- 172.21.15.112:0/4145102913 reap_dead start 2024-09-21T00:29:53.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:53.366+0000 7f932e027640 1 -- 172.21.15.112:0/4145102913 shutdown_connections 2024-09-21T00:29:53.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:53.366+0000 7f932e027640 1 -- 172.21.15.112:0/4145102913 >> 172.21.15.112:0/4145102913 conn(0x7f932809d8a0 msgr2=0x7f932808bfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:53.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:53.366+0000 7f932e027640 1 -- 172.21.15.112:0/4145102913 shutdown_connections 2024-09-21T00:29:53.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:53.366+0000 7f932e027640 1 -- 172.21.15.112:0/4145102913 wait complete. 2024-09-21T00:29:53.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:53.367+0000 7f932e027640 1 Processor -- start 2024-09-21T00:29:53.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:53.368+0000 7f932e027640 1 -- start start 2024-09-21T00:29:53.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:53.369+0000 7f932e027640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9310003ae0 0x7f932815baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:53.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:53.369+0000 7f932e027640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f93280cc0a0 0x7f93280cc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:53.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:53.369+0000 7f932e027640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f932815c1c0 0x7f932815c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:53.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:53.369+0000 7f932e027640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9328162630 con 0x7f9310003ae0 2024-09-21T00:29:53.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:53.369+0000 7f932e027640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f93281624b0 con 0x7f93280cc0a0 2024-09-21T00:29:53.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:53.369+0000 7f932e027640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f93281627b0 con 0x7f932815c1c0 2024-09-21T00:29:53.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:53.369+0000 7f93277fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9310003ae0 0x7f932815baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:53.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:53.369+0000 7f93277fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9310003ae0 0x7f932815baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:32874/0 (socket says 172.21.15.112:32874) 2024-09-21T00:29:53.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:53.369+0000 7f93277fe640 1 -- 172.21.15.112:0/1712139845 learned_addr learned my addr 172.21.15.112:0/1712139845 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:29:53.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:53.369+0000 7f9327fff640 1 --2- 172.21.15.112:0/1712139845 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f932815c1c0 0x7f932815c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:53.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:53.369+0000 7f9326ffd640 1 --2- 172.21.15.112:0/1712139845 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f93280cc0a0 0x7f93280cc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:53.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:53.369+0000 7f93277fe640 1 -- 172.21.15.112:0/1712139845 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f932815c1c0 msgr2=0x7f932815c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:53.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:53.369+0000 7f93277fe640 1 --2- 172.21.15.112:0/1712139845 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f932815c1c0 0x7f932815c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:53.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:53.369+0000 7f93277fe640 1 -- 172.21.15.112:0/1712139845 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f93280cc0a0 msgr2=0x7f93280cc4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:53.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:53.369+0000 7f93277fe640 1 --2- 172.21.15.112:0/1712139845 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f93280cc0a0 0x7f93280cc4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:53.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:53.369+0000 7f93277fe640 1 -- 172.21.15.112:0/1712139845 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f932815cb10 con 0x7f9310003ae0 2024-09-21T00:29:53.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:53.369+0000 7f9327fff640 1 --2- 172.21.15.112:0/1712139845 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f932815c1c0 0x7f932815c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:29:53.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:53.369+0000 7f9326ffd640 1 --2- 172.21.15.112:0/1712139845 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f93280cc0a0 0x7f93280cc4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:29:53.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:53.369+0000 7f93277fe640 1 --2- 172.21.15.112:0/1712139845 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9310003ae0 0x7f932815baf0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f931802b0e0 tx=0x7f9318006d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:53.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:53.369+0000 7f930ffff640 1 -- 172.21.15.112:0/1712139845 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9318031070 con 0x7f9310003ae0 2024-09-21T00:29:53.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:53.370+0000 7f930ffff640 1 -- 172.21.15.112:0/1712139845 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9318003610 con 0x7f9310003ae0 2024-09-21T00:29:53.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:53.370+0000 7f930ffff640 1 -- 172.21.15.112:0/1712139845 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9318001840 con 0x7f9310003ae0 2024-09-21T00:29:53.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:53.370+0000 7f932e027640 1 -- 172.21.15.112:0/1712139845 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f932806de80 con 0x7f9310003ae0 2024-09-21T00:29:53.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:53.370+0000 7f932e027640 1 -- 172.21.15.112:0/1712139845 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f932806d0c0 con 0x7f9310003ae0 2024-09-21T00:29:53.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:53.371+0000 7f932e027640 1 -- 172.21.15.112:0/1712139845 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93280b1aa0 con 0x7f9310003ae0 2024-09-21T00:29:53.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:53.371+0000 7f930ffff640 1 -- 172.21.15.112:0/1712139845 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9318003a60 con 0x7f9310003ae0 2024-09-21T00:29:53.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:53.372+0000 7f930ffff640 1 --2- 172.21.15.112:0/1712139845 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f9314078b90 0x7f931407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:53.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:53.372+0000 7f930ffff640 1 -- 172.21.15.112:0/1712139845 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 6101+0+0 (secure 0 0 0) 0x7f93180bab80 con 0x7f9310003ae0 2024-09-21T00:29:53.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:53.372+0000 7f9326ffd640 1 --2- 172.21.15.112:0/1712139845 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f9314078b90 0x7f931407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:53.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:53.373+0000 7f9326ffd640 1 --2- 172.21.15.112:0/1712139845 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f9314078b90 0x7f931407b0a0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f932815d0a0 tx=0x7f931c00e4c0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:53.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:53.376+0000 7f930ffff640 1 -- 172.21.15.112:0/1712139845 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f9318046030 con 0x7f9310003ae0 2024-09-21T00:29:53.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:53.549+0000 7f932e027640 1 -- 172.21.15.112:0/1712139845 --> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f93280b17c0 con 0x7f9314078b90 2024-09-21T00:29:53.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:53.554+0000 7f930ffff640 1 -- 172.21.15.112:0/1712139845 <== mgr.4103 v2:172.21.15.187:6800/2921522906 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251727 (secure 0 0 0) 0x7f93280b17c0 con 0x7f9314078b90 2024-09-21T00:29:53.558 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:29:53.559 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2024-09-21T00:29:53.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:53.557+0000 7f932e027640 1 -- 172.21.15.112:0/1712139845 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f9314078b90 msgr2=0x7f931407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:53.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:53.557+0000 7f932e027640 1 --2- 172.21.15.112:0/1712139845 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f9314078b90 0x7f931407b0a0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f932815d0a0 tx=0x7f931c00e4c0 comp rx=0 tx=0).stop 2024-09-21T00:29:53.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:53.557+0000 7f932e027640 1 -- 172.21.15.112:0/1712139845 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9310003ae0 msgr2=0x7f932815baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:53.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:53.557+0000 7f932e027640 1 --2- 172.21.15.112:0/1712139845 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9310003ae0 0x7f932815baf0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f931802b0e0 tx=0x7f9318006d50 comp rx=0 tx=0).stop 2024-09-21T00:29:53.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:53.557+0000 7f932e027640 1 -- 172.21.15.112:0/1712139845 shutdown_connections 2024-09-21T00:29:53.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:53.557+0000 7f932e027640 1 --2- 172.21.15.112:0/1712139845 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f932815c1c0 0x7f932815c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:53.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:53.557+0000 7f932e027640 1 --2- 172.21.15.112:0/1712139845 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f93280cc0a0 0x7f93280cc4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:53.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:53.557+0000 7f932e027640 1 --2- 172.21.15.112:0/1712139845 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f9314078b90 0x7f931407b0a0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:53.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:53.557+0000 7f932e027640 1 --2- 172.21.15.112:0/1712139845 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9310003ae0 0x7f932815baf0 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:53.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:53.557+0000 7f932e027640 1 -- 172.21.15.112:0/1712139845 >> 172.21.15.112:0/1712139845 conn(0x7f932809d8a0 msgr2=0x7f9328161730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:53.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:53.557+0000 7f932e027640 1 -- 172.21.15.112:0/1712139845 shutdown_connections 2024-09-21T00:29:53.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:53.557+0000 7f932e027640 1 -- 172.21.15.112:0/1712139845 wait complete. 2024-09-21T00:29:53.570 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":231,"stamp":"2024-09-21T00:29:53.152809+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592756,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":142,"num_write_kb":1641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":107,"ondisk_log_size":107,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":154,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":261168,"kb_used_data":11636,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754713552,"statfs":{"total":773094113280,"available":772826677248,"internally_reserved":0,"allocated":11915264,"data_stored":6054107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2388,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001337"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:50.094565+0000","last_change":"2024-09-21T00:29:49.108008+0000","last_active":"2024-09-21T00:29:50.094565+0000","last_peered":"2024-09-21T00:29:50.094565+0000","last_clean":"2024-09-21T00:29:50.094565+0000","last_became_active":"2024-09-21T00:29:49.107047+0000","last_became_peered":"2024-09-21T00:29:49.107047+0000","last_unstale":"2024-09-21T00:29:50.094565+0000","last_undegraded":"2024-09-21T00:29:50.094565+0000","last_fullsized":"2024-09-21T00:29:50.094565+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:53:16.984494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:51.164613+0000","last_change":"2024-09-21T00:29:49.128276+0000","last_active":"2024-09-21T00:29:51.164613+0000","last_peered":"2024-09-21T00:29:51.164613+0000","last_clean":"2024-09-21T00:29:51.164613+0000","last_became_active":"2024-09-21T00:29:49.127942+0000","last_became_peered":"2024-09-21T00:29:49.127942+0000","last_unstale":"2024-09-21T00:29:51.164613+0000","last_undegraded":"2024-09-21T00:29:51.164613+0000","last_fullsized":"2024-09-21T00:29:51.164613+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:46:55.941733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:51.164642+0000","last_change":"2024-09-21T00:29:49.121824+0000","last_active":"2024-09-21T00:29:51.164642+0000","last_peered":"2024-09-21T00:29:51.164642+0000","last_clean":"2024-09-21T00:29:51.164642+0000","last_became_active":"2024-09-21T00:29:49.121486+0000","last_became_peered":"2024-09-21T00:29:49.121486+0000","last_unstale":"2024-09-21T00:29:51.164642+0000","last_undegraded":"2024-09-21T00:29:51.164642+0000","last_fullsized":"2024-09-21T00:29:51.164642+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:21:34.883179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:51.164354+0000","last_change":"2024-09-21T00:29:49.128936+0000","last_active":"2024-09-21T00:29:51.164354+0000","last_peered":"2024-09-21T00:29:51.164354+0000","last_clean":"2024-09-21T00:29:51.164354+0000","last_became_active":"2024-09-21T00:29:49.128662+0000","last_became_peered":"2024-09-21T00:29:49.128662+0000","last_unstale":"2024-09-21T00:29:51.164354+0000","last_undegraded":"2024-09-21T00:29:51.164354+0000","last_fullsized":"2024-09-21T00:29:51.164354+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:16:09.264216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:50.093904+0000","last_change":"2024-09-21T00:29:49.108365+0000","last_active":"2024-09-21T00:29:50.093904+0000","last_peered":"2024-09-21T00:29:50.093904+0000","last_clean":"2024-09-21T00:29:50.093904+0000","last_became_active":"2024-09-21T00:29:49.108004+0000","last_became_peered":"2024-09-21T00:29:49.108004+0000","last_unstale":"2024-09-21T00:29:50.093904+0000","last_undegraded":"2024-09-21T00:29:50.093904+0000","last_fullsized":"2024-09-21T00:29:50.093904+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:39:43.972280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:51.101083+0000","last_change":"2024-09-21T00:29:49.106784+0000","last_active":"2024-09-21T00:29:51.101083+0000","last_peered":"2024-09-21T00:29:51.101083+0000","last_clean":"2024-09-21T00:29:51.101083+0000","last_became_active":"2024-09-21T00:29:49.106502+0000","last_became_peered":"2024-09-21T00:29:49.106502+0000","last_unstale":"2024-09-21T00:29:51.101083+0000","last_undegraded":"2024-09-21T00:29:51.101083+0000","last_fullsized":"2024-09-21T00:29:51.101083+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:27:41.443648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:51.101031+0000","last_change":"2024-09-21T00:29:49.119608+0000","last_active":"2024-09-21T00:29:51.101031+0000","last_peered":"2024-09-21T00:29:51.101031+0000","last_clean":"2024-09-21T00:29:51.101031+0000","last_became_active":"2024-09-21T00:29:49.119217+0000","last_became_peered":"2024-09-21T00:29:49.119217+0000","last_unstale":"2024-09-21T00:29:51.101031+0000","last_undegraded":"2024-09-21T00:29:51.101031+0000","last_fullsized":"2024-09-21T00:29:51.101031+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:54:13.945567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:51.163936+0000","last_change":"2024-09-21T00:29:49.122526+0000","last_active":"2024-09-21T00:29:51.163936+0000","last_peered":"2024-09-21T00:29:51.163936+0000","last_clean":"2024-09-21T00:29:51.163936+0000","last_became_active":"2024-09-21T00:29:49.122215+0000","last_became_peered":"2024-09-21T00:29:49.122215+0000","last_unstale":"2024-09-21T00:29:51.163936+0000","last_undegraded":"2024-09-21T00:29:51.163936+0000","last_fullsized":"2024-09-21T00:29:51.163936+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:16:09.264216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:51.164751+0000","last_change":"2024-09-21T00:29:49.126353+0000","last_active":"2024-09-21T00:29:51.164751+0000","last_peered":"2024-09-21T00:29:51.164751+0000","last_clean":"2024-09-21T00:29:51.164751+0000","last_became_active":"2024-09-21T00:29:49.126053+0000","last_became_peered":"2024-09-21T00:29:49.126053+0000","last_unstale":"2024-09-21T00:29:51.164751+0000","last_undegraded":"2024-09-21T00:29:51.164751+0000","last_fullsized":"2024-09-21T00:29:51.164751+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:28:58.560933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-09-21T00:29:48.098942+0000","last_change":"2024-09-21T00:29:48.098942+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.098942+0000","last_undegraded":"2024-09-21T00:29:48.098942+0000","last_fullsized":"2024-09-21T00:29:48.098942+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:50.094412+0000","last_change":"2024-09-21T00:29:49.112574+0000","last_active":"2024-09-21T00:29:50.094412+0000","last_peered":"2024-09-21T00:29:50.094412+0000","last_clean":"2024-09-21T00:29:50.094412+0000","last_became_active":"2024-09-21T00:29:49.112267+0000","last_became_peered":"2024-09-21T00:29:49.112267+0000","last_unstale":"2024-09-21T00:29:50.094412+0000","last_undegraded":"2024-09-21T00:29:50.094412+0000","last_fullsized":"2024-09-21T00:29:50.094412+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:36:10.110340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:51.101475+0000","last_change":"2024-09-21T00:29:49.105771+0000","last_active":"2024-09-21T00:29:51.101475+0000","last_peered":"2024-09-21T00:29:51.101475+0000","last_clean":"2024-09-21T00:29:51.101475+0000","last_became_active":"2024-09-21T00:29:49.105062+0000","last_became_peered":"2024-09-21T00:29:49.105062+0000","last_unstale":"2024-09-21T00:29:51.101475+0000","last_undegraded":"2024-09-21T00:29:51.101475+0000","last_fullsized":"2024-09-21T00:29:51.101475+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:27:23.331150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-09-21T00:29:48.099873+0000","last_change":"2024-09-21T00:29:48.099873+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.099873+0000","last_undegraded":"2024-09-21T00:29:48.099873+0000","last_fullsized":"2024-09-21T00:29:48.099873+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:51.164695+0000","last_change":"2024-09-21T00:29:49.123712+0000","last_active":"2024-09-21T00:29:51.164695+0000","last_peered":"2024-09-21T00:29:51.164695+0000","last_clean":"2024-09-21T00:29:51.164695+0000","last_became_active":"2024-09-21T00:29:49.121885+0000","last_became_peered":"2024-09-21T00:29:49.121885+0000","last_unstale":"2024-09-21T00:29:51.164695+0000","last_undegraded":"2024-09-21T00:29:51.164695+0000","last_fullsized":"2024-09-21T00:29:51.164695+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:54:11.086218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:50.094237+0000","last_change":"2024-09-21T00:29:49.112561+0000","last_active":"2024-09-21T00:29:50.094237+0000","last_peered":"2024-09-21T00:29:50.094237+0000","last_clean":"2024-09-21T00:29:50.094237+0000","last_became_active":"2024-09-21T00:29:49.112203+0000","last_became_peered":"2024-09-21T00:29:49.112203+0000","last_unstale":"2024-09-21T00:29:50.094237+0000","last_undegraded":"2024-09-21T00:29:50.094237+0000","last_fullsized":"2024-09-21T00:29:50.094237+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:59:27.769358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:51.101009+0000","last_change":"2024-09-21T00:29:49.115569+0000","last_active":"2024-09-21T00:29:51.101009+0000","last_peered":"2024-09-21T00:29:51.101009+0000","last_clean":"2024-09-21T00:29:51.101009+0000","last_became_active":"2024-09-21T00:29:49.115141+0000","last_became_peered":"2024-09-21T00:29:49.115141+0000","last_unstale":"2024-09-21T00:29:51.101009+0000","last_undegraded":"2024-09-21T00:29:51.101009+0000","last_fullsized":"2024-09-21T00:29:51.101009+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:03:51.843446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-09-21T00:29:48.097589+0000","last_change":"2024-09-21T00:29:48.097589+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.097589+0000","last_undegraded":"2024-09-21T00:29:48.097589+0000","last_fullsized":"2024-09-21T00:29:48.097589+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:51.164845+0000","last_change":"2024-09-21T00:29:49.122788+0000","last_active":"2024-09-21T00:29:51.164845+0000","last_peered":"2024-09-21T00:29:51.164845+0000","last_clean":"2024-09-21T00:29:51.164845+0000","last_became_active":"2024-09-21T00:29:49.122455+0000","last_became_peered":"2024-09-21T00:29:49.122455+0000","last_unstale":"2024-09-21T00:29:51.164845+0000","last_undegraded":"2024-09-21T00:29:51.164845+0000","last_fullsized":"2024-09-21T00:29:51.164845+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:54:08.552661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:51.098900+0000","last_change":"2024-09-21T00:29:49.124316+0000","last_active":"2024-09-21T00:29:51.098900+0000","last_peered":"2024-09-21T00:29:51.098900+0000","last_clean":"2024-09-21T00:29:51.098900+0000","last_became_active":"2024-09-21T00:29:49.124097+0000","last_became_peered":"2024-09-21T00:29:49.124097+0000","last_unstale":"2024-09-21T00:29:51.098900+0000","last_undegraded":"2024-09-21T00:29:51.098900+0000","last_fullsized":"2024-09-21T00:29:51.098900+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:24:49.582188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:50.095080+0000","last_change":"2024-09-21T00:29:49.110158+0000","last_active":"2024-09-21T00:29:50.095080+0000","last_peered":"2024-09-21T00:29:50.095080+0000","last_clean":"2024-09-21T00:29:50.095080+0000","last_became_active":"2024-09-21T00:29:49.109796+0000","last_became_peered":"2024-09-21T00:29:49.109796+0000","last_unstale":"2024-09-21T00:29:50.095080+0000","last_undegraded":"2024-09-21T00:29:50.095080+0000","last_fullsized":"2024-09-21T00:29:50.095080+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:09:25.991935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:51.164320+0000","last_change":"2024-09-21T00:29:49.118758+0000","last_active":"2024-09-21T00:29:51.164320+0000","last_peered":"2024-09-21T00:29:51.164320+0000","last_clean":"2024-09-21T00:29:51.164320+0000","last_became_active":"2024-09-21T00:29:49.117990+0000","last_became_peered":"2024-09-21T00:29:49.117990+0000","last_unstale":"2024-09-21T00:29:51.164320+0000","last_undegraded":"2024-09-21T00:29:51.164320+0000","last_fullsized":"2024-09-21T00:29:51.164320+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T00:52:40.359414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:50.094687+0000","last_change":"2024-09-21T00:29:49.109190+0000","last_active":"2024-09-21T00:29:50.094687+0000","last_peered":"2024-09-21T00:29:50.094687+0000","last_clean":"2024-09-21T00:29:50.094687+0000","last_became_active":"2024-09-21T00:29:49.108841+0000","last_became_peered":"2024-09-21T00:29:49.108841+0000","last_unstale":"2024-09-21T00:29:50.094687+0000","last_undegraded":"2024-09-21T00:29:50.094687+0000","last_fullsized":"2024-09-21T00:29:50.094687+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:18:13.099451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:51.164337+0000","last_change":"2024-09-21T00:29:49.126344+0000","last_active":"2024-09-21T00:29:51.164337+0000","last_peered":"2024-09-21T00:29:51.164337+0000","last_clean":"2024-09-21T00:29:51.164337+0000","last_became_active":"2024-09-21T00:29:49.126013+0000","last_became_peered":"2024-09-21T00:29:49.126013+0000","last_unstale":"2024-09-21T00:29:51.164337+0000","last_undegraded":"2024-09-21T00:29:51.164337+0000","last_fullsized":"2024-09-21T00:29:51.164337+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:34:59.188136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:50.094378+0000","last_change":"2024-09-21T00:29:49.116501+0000","last_active":"2024-09-21T00:29:50.094378+0000","last_peered":"2024-09-21T00:29:50.094378+0000","last_clean":"2024-09-21T00:29:50.094378+0000","last_became_active":"2024-09-21T00:29:49.114983+0000","last_became_peered":"2024-09-21T00:29:49.114983+0000","last_unstale":"2024-09-21T00:29:50.094378+0000","last_undegraded":"2024-09-21T00:29:50.094378+0000","last_fullsized":"2024-09-21T00:29:50.094378+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:37:20.001351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:50.094978+0000","last_change":"2024-09-21T00:29:49.106766+0000","last_active":"2024-09-21T00:29:50.094978+0000","last_peered":"2024-09-21T00:29:50.094978+0000","last_clean":"2024-09-21T00:29:50.094978+0000","last_became_active":"2024-09-21T00:29:49.106450+0000","last_became_peered":"2024-09-21T00:29:49.106450+0000","last_unstale":"2024-09-21T00:29:50.094978+0000","last_undegraded":"2024-09-21T00:29:50.094978+0000","last_fullsized":"2024-09-21T00:29:50.094978+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:39:54.320073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:50.095416+0000","last_change":"2024-09-21T00:29:49.112842+0000","last_active":"2024-09-21T00:29:50.095416+0000","last_peered":"2024-09-21T00:29:50.095416+0000","last_clean":"2024-09-21T00:29:50.095416+0000","last_became_active":"2024-09-21T00:29:49.112592+0000","last_became_peered":"2024-09-21T00:29:49.112592+0000","last_unstale":"2024-09-21T00:29:50.095416+0000","last_undegraded":"2024-09-21T00:29:50.095416+0000","last_fullsized":"2024-09-21T00:29:50.095416+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:05:23.748224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:51.098980+0000","last_change":"2024-09-21T00:29:49.122017+0000","last_active":"2024-09-21T00:29:51.098980+0000","last_peered":"2024-09-21T00:29:51.098980+0000","last_clean":"2024-09-21T00:29:51.098980+0000","last_became_active":"2024-09-21T00:29:49.121686+0000","last_became_peered":"2024-09-21T00:29:49.121686+0000","last_unstale":"2024-09-21T00:29:51.098980+0000","last_undegraded":"2024-09-21T00:29:51.098980+0000","last_fullsized":"2024-09-21T00:29:51.098980+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:19:04.671994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:51.101566+0000","last_change":"2024-09-21T00:29:49.106112+0000","last_active":"2024-09-21T00:29:51.101566+0000","last_peered":"2024-09-21T00:29:51.101566+0000","last_clean":"2024-09-21T00:29:51.101566+0000","last_became_active":"2024-09-21T00:29:49.104746+0000","last_became_peered":"2024-09-21T00:29:49.104746+0000","last_unstale":"2024-09-21T00:29:51.101566+0000","last_undegraded":"2024-09-21T00:29:51.101566+0000","last_fullsized":"2024-09-21T00:29:51.101566+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:00:47.300212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:50.095413+0000","last_change":"2024-09-21T00:29:49.124680+0000","last_active":"2024-09-21T00:29:50.095413+0000","last_peered":"2024-09-21T00:29:50.095413+0000","last_clean":"2024-09-21T00:29:50.095413+0000","last_became_active":"2024-09-21T00:29:49.124314+0000","last_became_peered":"2024-09-21T00:29:49.124314+0000","last_unstale":"2024-09-21T00:29:50.095413+0000","last_undegraded":"2024-09-21T00:29:50.095413+0000","last_fullsized":"2024-09-21T00:29:50.095413+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:46:01.470514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:51.164915+0000","last_change":"2024-09-21T00:29:49.112291+0000","last_active":"2024-09-21T00:29:51.164915+0000","last_peered":"2024-09-21T00:29:51.164915+0000","last_clean":"2024-09-21T00:29:51.164915+0000","last_became_active":"2024-09-21T00:29:49.111111+0000","last_became_peered":"2024-09-21T00:29:49.111111+0000","last_unstale":"2024-09-21T00:29:51.164915+0000","last_undegraded":"2024-09-21T00:29:51.164915+0000","last_fullsized":"2024-09-21T00:29:51.164915+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:14:08.961080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:50.095161+0000","last_change":"2024-09-21T00:29:49.106140+0000","last_active":"2024-09-21T00:29:50.095161+0000","last_peered":"2024-09-21T00:29:50.095161+0000","last_clean":"2024-09-21T00:29:50.095161+0000","last_became_active":"2024-09-21T00:29:49.105806+0000","last_became_peered":"2024-09-21T00:29:49.105806+0000","last_unstale":"2024-09-21T00:29:50.095161+0000","last_undegraded":"2024-09-21T00:29:50.095161+0000","last_fullsized":"2024-09-21T00:29:50.095161+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:55:51.631707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:50.094589+0000","last_change":"2024-09-21T00:29:49.114756+0000","last_active":"2024-09-21T00:29:50.094589+0000","last_peered":"2024-09-21T00:29:50.094589+0000","last_clean":"2024-09-21T00:29:50.094589+0000","last_became_active":"2024-09-21T00:29:49.114271+0000","last_became_peered":"2024-09-21T00:29:49.114271+0000","last_unstale":"2024-09-21T00:29:50.094589+0000","last_undegraded":"2024-09-21T00:29:50.094589+0000","last_fullsized":"2024-09-21T00:29:50.094589+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:02:09.408653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-09-21T00:29:48.097051+0000","last_change":"2024-09-21T00:29:48.097051+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.097051+0000","last_undegraded":"2024-09-21T00:29:48.097051+0000","last_fullsized":"2024-09-21T00:29:48.097051+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1f","version":"107'4","reported_seq":19,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:51.098944+0000","last_change":"2024-09-21T00:29:48.115204+0000","last_active":"2024-09-21T00:29:51.098944+0000","last_peered":"2024-09-21T00:29:51.098944+0000","last_clean":"2024-09-21T00:29:51.098944+0000","last_became_active":"2024-09-21T00:29:48.114873+0000","last_became_peered":"2024-09-21T00:29:48.114873+0000","last_unstale":"2024-09-21T00:29:51.098944+0000","last_undegraded":"2024-09-21T00:29:51.098944+0000","last_fullsized":"2024-09-21T00:29:51.098944+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:43:59.212358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:51.101294+0000","last_change":"2024-09-21T00:29:49.109428+0000","last_active":"2024-09-21T00:29:51.101294+0000","last_peered":"2024-09-21T00:29:51.101294+0000","last_clean":"2024-09-21T00:29:51.101294+0000","last_became_active":"2024-09-21T00:29:49.109005+0000","last_became_peered":"2024-09-21T00:29:49.109005+0000","last_unstale":"2024-09-21T00:29:51.101294+0000","last_undegraded":"2024-09-21T00:29:51.101294+0000","last_fullsized":"2024-09-21T00:29:51.101294+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:21:09.408372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-21T00:29:48.105694+0000","last_change":"2024-09-21T00:29:48.102532+0000","last_active":"2024-09-21T00:29:48.105694+0000","last_peered":"2024-09-21T00:29:48.105694+0000","last_clean":"2024-09-21T00:29:48.105694+0000","last_became_active":"2024-09-21T00:29:48.102195+0000","last_became_peered":"2024-09-21T00:29:48.102195+0000","last_unstale":"2024-09-21T00:29:48.105694+0000","last_undegraded":"2024-09-21T00:29:48.105694+0000","last_fullsized":"2024-09-21T00:29:48.105694+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:41:20.768969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:50.095686+0000","last_change":"2024-09-21T00:29:49.110911+0000","last_active":"2024-09-21T00:29:50.095686+0000","last_peered":"2024-09-21T00:29:50.095686+0000","last_clean":"2024-09-21T00:29:50.095686+0000","last_became_active":"2024-09-21T00:29:49.110574+0000","last_became_peered":"2024-09-21T00:29:49.110574+0000","last_unstale":"2024-09-21T00:29:50.095686+0000","last_undegraded":"2024-09-21T00:29:50.095686+0000","last_fullsized":"2024-09-21T00:29:50.095686+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:34:40.420590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1d","version":"107'2","reported_seq":17,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:51.164459+0000","last_change":"2024-09-21T00:29:48.110005+0000","last_active":"2024-09-21T00:29:51.164459+0000","last_peered":"2024-09-21T00:29:51.164459+0000","last_clean":"2024-09-21T00:29:51.164459+0000","last_became_active":"2024-09-21T00:29:48.109719+0000","last_became_peered":"2024-09-21T00:29:48.109719+0000","last_unstale":"2024-09-21T00:29:51.164459+0000","last_undegraded":"2024-09-21T00:29:51.164459+0000","last_fullsized":"2024-09-21T00:29:51.164459+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:04:02.357288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:51.164779+0000","last_change":"2024-09-21T00:29:49.117451+0000","last_active":"2024-09-21T00:29:51.164779+0000","last_peered":"2024-09-21T00:29:51.164779+0000","last_clean":"2024-09-21T00:29:51.164779+0000","last_became_active":"2024-09-21T00:29:49.117151+0000","last_became_peered":"2024-09-21T00:29:49.117151+0000","last_unstale":"2024-09-21T00:29:51.164779+0000","last_undegraded":"2024-09-21T00:29:51.164779+0000","last_fullsized":"2024-09-21T00:29:51.164779+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:08:12.844670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-21T00:29:48.107959+0000","last_change":"2024-09-21T00:29:48.103570+0000","last_active":"2024-09-21T00:29:48.107959+0000","last_peered":"2024-09-21T00:29:48.107959+0000","last_clean":"2024-09-21T00:29:48.107959+0000","last_became_active":"2024-09-21T00:29:48.103262+0000","last_became_peered":"2024-09-21T00:29:48.103262+0000","last_unstale":"2024-09-21T00:29:48.107959+0000","last_undegraded":"2024-09-21T00:29:48.107959+0000","last_fullsized":"2024-09-21T00:29:48.107959+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:19:55.232168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-09-21T00:29:48.094970+0000","last_change":"2024-09-21T00:29:48.094970+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.094970+0000","last_undegraded":"2024-09-21T00:29:48.094970+0000","last_fullsized":"2024-09-21T00:29:48.094970+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"107'1","reported_seq":16,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:50.094061+0000","last_change":"2024-09-21T00:29:48.114417+0000","last_active":"2024-09-21T00:29:50.094061+0000","last_peered":"2024-09-21T00:29:50.094061+0000","last_clean":"2024-09-21T00:29:50.094061+0000","last_became_active":"2024-09-21T00:29:48.114114+0000","last_became_peered":"2024-09-21T00:29:48.114114+0000","last_unstale":"2024-09-21T00:29:50.094061+0000","last_undegraded":"2024-09-21T00:29:50.094061+0000","last_fullsized":"2024-09-21T00:29:50.094061+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:30:30.206828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:50.094568+0000","last_change":"2024-09-21T00:29:49.123289+0000","last_active":"2024-09-21T00:29:50.094568+0000","last_peered":"2024-09-21T00:29:50.094568+0000","last_clean":"2024-09-21T00:29:50.094568+0000","last_became_active":"2024-09-21T00:29:49.122910+0000","last_became_peered":"2024-09-21T00:29:49.122910+0000","last_unstale":"2024-09-21T00:29:50.094568+0000","last_undegraded":"2024-09-21T00:29:50.094568+0000","last_fullsized":"2024-09-21T00:29:50.094568+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:35:02.314410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"107'1","reported_seq":16,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:51.101164+0000","last_change":"2024-09-21T00:29:48.109506+0000","last_active":"2024-09-21T00:29:51.101164+0000","last_peered":"2024-09-21T00:29:51.101164+0000","last_clean":"2024-09-21T00:29:51.101164+0000","last_became_active":"2024-09-21T00:29:48.109216+0000","last_became_peered":"2024-09-21T00:29:48.109216+0000","last_unstale":"2024-09-21T00:29:51.101164+0000","last_undegraded":"2024-09-21T00:29:51.101164+0000","last_fullsized":"2024-09-21T00:29:51.101164+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:28:37.053291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:51.165058+0000","last_change":"2024-09-21T00:29:49.123189+0000","last_active":"2024-09-21T00:29:51.165058+0000","last_peered":"2024-09-21T00:29:51.165058+0000","last_clean":"2024-09-21T00:29:51.165058+0000","last_became_active":"2024-09-21T00:29:49.122005+0000","last_became_peered":"2024-09-21T00:29:49.122005+0000","last_unstale":"2024-09-21T00:29:51.165058+0000","last_undegraded":"2024-09-21T00:29:51.165058+0000","last_fullsized":"2024-09-21T00:29:51.165058+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:09:03.700916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":15,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:50.095251+0000","last_change":"2024-09-21T00:29:48.115406+0000","last_active":"2024-09-21T00:29:50.095251+0000","last_peered":"2024-09-21T00:29:50.095251+0000","last_clean":"2024-09-21T00:29:50.095251+0000","last_became_active":"2024-09-21T00:29:48.114774+0000","last_became_peered":"2024-09-21T00:29:48.114774+0000","last_unstale":"2024-09-21T00:29:50.095251+0000","last_undegraded":"2024-09-21T00:29:50.095251+0000","last_fullsized":"2024-09-21T00:29:50.095251+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:33:51.336627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-09-21T00:29:48.096433+0000","last_change":"2024-09-21T00:29:48.096433+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.096433+0000","last_undegraded":"2024-09-21T00:29:48.096433+0000","last_fullsized":"2024-09-21T00:29:48.096433+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":15,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:51.101410+0000","last_change":"2024-09-21T00:29:48.109429+0000","last_active":"2024-09-21T00:29:51.101410+0000","last_peered":"2024-09-21T00:29:51.101410+0000","last_clean":"2024-09-21T00:29:51.101410+0000","last_became_active":"2024-09-21T00:29:48.109135+0000","last_became_peered":"2024-09-21T00:29:48.109135+0000","last_unstale":"2024-09-21T00:29:51.101410+0000","last_undegraded":"2024-09-21T00:29:51.101410+0000","last_fullsized":"2024-09-21T00:29:51.101410+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:52:45.185741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:51.098909+0000","last_change":"2024-09-21T00:29:49.124510+0000","last_active":"2024-09-21T00:29:51.098909+0000","last_peered":"2024-09-21T00:29:51.098909+0000","last_clean":"2024-09-21T00:29:51.098909+0000","last_became_active":"2024-09-21T00:29:49.124143+0000","last_became_peered":"2024-09-21T00:29:49.124143+0000","last_unstale":"2024-09-21T00:29:51.098909+0000","last_undegraded":"2024-09-21T00:29:51.098909+0000","last_fullsized":"2024-09-21T00:29:51.098909+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:52:27.753303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":15,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:51.098867+0000","last_change":"2024-09-21T00:29:48.110534+0000","last_active":"2024-09-21T00:29:51.098867+0000","last_peered":"2024-09-21T00:29:51.098867+0000","last_clean":"2024-09-21T00:29:51.098867+0000","last_became_active":"2024-09-21T00:29:48.110188+0000","last_became_peered":"2024-09-21T00:29:48.110188+0000","last_unstale":"2024-09-21T00:29:51.098867+0000","last_undegraded":"2024-09-21T00:29:51.098867+0000","last_fullsized":"2024-09-21T00:29:51.098867+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:57:08.835632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:50.095510+0000","last_change":"2024-09-21T00:29:49.110941+0000","last_active":"2024-09-21T00:29:50.095510+0000","last_peered":"2024-09-21T00:29:50.095510+0000","last_clean":"2024-09-21T00:29:50.095510+0000","last_became_active":"2024-09-21T00:29:49.110626+0000","last_became_peered":"2024-09-21T00:29:49.110626+0000","last_unstale":"2024-09-21T00:29:50.095510+0000","last_undegraded":"2024-09-21T00:29:50.095510+0000","last_fullsized":"2024-09-21T00:29:50.095510+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:17:42.952108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":15,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:50.095610+0000","last_change":"2024-09-21T00:29:48.100585+0000","last_active":"2024-09-21T00:29:50.095610+0000","last_peered":"2024-09-21T00:29:50.095610+0000","last_clean":"2024-09-21T00:29:50.095610+0000","last_became_active":"2024-09-21T00:29:48.100260+0000","last_became_peered":"2024-09-21T00:29:48.100260+0000","last_unstale":"2024-09-21T00:29:50.095610+0000","last_undegraded":"2024-09-21T00:29:50.095610+0000","last_fullsized":"2024-09-21T00:29:50.095610+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:35:04.548036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:51.164870+0000","last_change":"2024-09-21T00:29:49.121983+0000","last_active":"2024-09-21T00:29:51.164870+0000","last_peered":"2024-09-21T00:29:51.164870+0000","last_clean":"2024-09-21T00:29:51.164870+0000","last_became_active":"2024-09-21T00:29:49.121653+0000","last_became_peered":"2024-09-21T00:29:49.121653+0000","last_unstale":"2024-09-21T00:29:51.164870+0000","last_undegraded":"2024-09-21T00:29:51.164870+0000","last_fullsized":"2024-09-21T00:29:51.164870+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:30:21.800274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.15","version":"107'3","reported_seq":18,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:50.095482+0000","last_change":"2024-09-21T00:29:48.107282+0000","last_active":"2024-09-21T00:29:50.095482+0000","last_peered":"2024-09-21T00:29:50.095482+0000","last_clean":"2024-09-21T00:29:50.095482+0000","last_became_active":"2024-09-21T00:29:48.106624+0000","last_became_peered":"2024-09-21T00:29:48.106624+0000","last_unstale":"2024-09-21T00:29:50.095482+0000","last_undegraded":"2024-09-21T00:29:50.095482+0000","last_fullsized":"2024-09-21T00:29:50.095482+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:54:25.882509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:50.095213+0000","last_change":"2024-09-21T00:29:49.105627+0000","last_active":"2024-09-21T00:29:50.095213+0000","last_peered":"2024-09-21T00:29:50.095213+0000","last_clean":"2024-09-21T00:29:50.095213+0000","last_became_active":"2024-09-21T00:29:49.105260+0000","last_became_peered":"2024-09-21T00:29:49.105260+0000","last_unstale":"2024-09-21T00:29:50.095213+0000","last_undegraded":"2024-09-21T00:29:50.095213+0000","last_fullsized":"2024-09-21T00:29:50.095213+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:47:44.730787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.14","version":"107'6","reported_seq":21,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:51.164236+0000","last_change":"2024-09-21T00:29:48.114730+0000","last_active":"2024-09-21T00:29:51.164236+0000","last_peered":"2024-09-21T00:29:51.164236+0000","last_clean":"2024-09-21T00:29:51.164236+0000","last_became_active":"2024-09-21T00:29:48.114389+0000","last_became_peered":"2024-09-21T00:29:48.114389+0000","last_unstale":"2024-09-21T00:29:51.164236+0000","last_undegraded":"2024-09-21T00:29:51.164236+0000","last_fullsized":"2024-09-21T00:29:51.164236+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:53:15.982238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:51.099041+0000","last_change":"2024-09-21T00:29:49.109745+0000","last_active":"2024-09-21T00:29:51.099041+0000","last_peered":"2024-09-21T00:29:51.099041+0000","last_clean":"2024-09-21T00:29:51.099041+0000","last_became_active":"2024-09-21T00:29:49.109453+0000","last_became_peered":"2024-09-21T00:29:49.109453+0000","last_unstale":"2024-09-21T00:29:51.099041+0000","last_undegraded":"2024-09-21T00:29:51.099041+0000","last_fullsized":"2024-09-21T00:29:51.099041+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:55:39.386900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.13","version":"107'2","reported_seq":17,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:51.164082+0000","last_change":"2024-09-21T00:29:48.109904+0000","last_active":"2024-09-21T00:29:51.164082+0000","last_peered":"2024-09-21T00:29:51.164082+0000","last_clean":"2024-09-21T00:29:51.164082+0000","last_became_active":"2024-09-21T00:29:48.109587+0000","last_became_peered":"2024-09-21T00:29:48.109587+0000","last_unstale":"2024-09-21T00:29:51.164082+0000","last_undegraded":"2024-09-21T00:29:51.164082+0000","last_fullsized":"2024-09-21T00:29:51.164082+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:59:26.767103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-09-21T00:29:48.095354+0000","last_change":"2024-09-21T00:29:48.095354+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.095354+0000","last_undegraded":"2024-09-21T00:29:48.095354+0000","last_fullsized":"2024-09-21T00:29:48.095354+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":15,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:51.165135+0000","last_change":"2024-09-21T00:29:48.106128+0000","last_active":"2024-09-21T00:29:51.165135+0000","last_peered":"2024-09-21T00:29:51.165135+0000","last_clean":"2024-09-21T00:29:51.165135+0000","last_became_active":"2024-09-21T00:29:48.105831+0000","last_became_peered":"2024-09-21T00:29:48.105831+0000","last_unstale":"2024-09-21T00:29:51.165135+0000","last_undegraded":"2024-09-21T00:29:51.165135+0000","last_fullsized":"2024-09-21T00:29:51.165135+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:18:12.097196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:50.095064+0000","last_change":"2024-09-21T00:29:49.114501+0000","last_active":"2024-09-21T00:29:50.095064+0000","last_peered":"2024-09-21T00:29:50.095064+0000","last_clean":"2024-09-21T00:29:50.095064+0000","last_became_active":"2024-09-21T00:29:49.114159+0000","last_became_peered":"2024-09-21T00:29:49.114159+0000","last_unstale":"2024-09-21T00:29:50.095064+0000","last_undegraded":"2024-09-21T00:29:50.095064+0000","last_fullsized":"2024-09-21T00:29:50.095064+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:52:25.673726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":15,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:51.164102+0000","last_change":"2024-09-21T00:29:48.112258+0000","last_active":"2024-09-21T00:29:51.164102+0000","last_peered":"2024-09-21T00:29:51.164102+0000","last_clean":"2024-09-21T00:29:51.164102+0000","last_became_active":"2024-09-21T00:29:48.111976+0000","last_became_peered":"2024-09-21T00:29:48.111976+0000","last_unstale":"2024-09-21T00:29:51.164102+0000","last_undegraded":"2024-09-21T00:29:51.164102+0000","last_fullsized":"2024-09-21T00:29:51.164102+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:35:35.720354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:50.094527+0000","last_change":"2024-09-21T00:29:49.111503+0000","last_active":"2024-09-21T00:29:50.094527+0000","last_peered":"2024-09-21T00:29:50.094527+0000","last_clean":"2024-09-21T00:29:50.094527+0000","last_became_active":"2024-09-21T00:29:49.111197+0000","last_became_peered":"2024-09-21T00:29:49.111197+0000","last_unstale":"2024-09-21T00:29:50.094527+0000","last_undegraded":"2024-09-21T00:29:50.094527+0000","last_fullsized":"2024-09-21T00:29:50.094527+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:51:04.982608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-21T00:29:48.108007+0000","last_change":"2024-09-21T00:29:48.104276+0000","last_active":"2024-09-21T00:29:48.108007+0000","last_peered":"2024-09-21T00:29:48.108007+0000","last_clean":"2024-09-21T00:29:48.108007+0000","last_became_active":"2024-09-21T00:29:48.103972+0000","last_became_peered":"2024-09-21T00:29:48.103972+0000","last_unstale":"2024-09-21T00:29:48.108007+0000","last_undegraded":"2024-09-21T00:29:48.108007+0000","last_fullsized":"2024-09-21T00:29:48.108007+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:48:50.178516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:50.094637+0000","last_change":"2024-09-21T00:29:49.110046+0000","last_active":"2024-09-21T00:29:50.094637+0000","last_peered":"2024-09-21T00:29:50.094637+0000","last_clean":"2024-09-21T00:29:50.094637+0000","last_became_active":"2024-09-21T00:29:49.109727+0000","last_became_peered":"2024-09-21T00:29:49.109727+0000","last_unstale":"2024-09-21T00:29:50.094637+0000","last_undegraded":"2024-09-21T00:29:50.094637+0000","last_fullsized":"2024-09-21T00:29:50.094637+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:27:53.080062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.e","version":"107'2","reported_seq":17,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:50.095384+0000","last_change":"2024-09-21T00:29:48.103336+0000","last_active":"2024-09-21T00:29:50.095384+0000","last_peered":"2024-09-21T00:29:50.095384+0000","last_clean":"2024-09-21T00:29:50.095384+0000","last_became_active":"2024-09-21T00:29:48.103043+0000","last_became_peered":"2024-09-21T00:29:48.103043+0000","last_unstale":"2024-09-21T00:29:50.095384+0000","last_undegraded":"2024-09-21T00:29:50.095384+0000","last_fullsized":"2024-09-21T00:29:50.095384+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:59:23.042894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:51.101521+0000","last_change":"2024-09-21T00:29:49.108911+0000","last_active":"2024-09-21T00:29:51.101521+0000","last_peered":"2024-09-21T00:29:51.101521+0000","last_clean":"2024-09-21T00:29:51.101521+0000","last_became_active":"2024-09-21T00:29:49.108470+0000","last_became_peered":"2024-09-21T00:29:49.108470+0000","last_unstale":"2024-09-21T00:29:51.101521+0000","last_undegraded":"2024-09-21T00:29:51.101521+0000","last_fullsized":"2024-09-21T00:29:51.101521+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:56:55.321362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"107'1","reported_seq":16,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:51.164501+0000","last_change":"2024-09-21T00:29:48.108936+0000","last_active":"2024-09-21T00:29:51.164501+0000","last_peered":"2024-09-21T00:29:51.164501+0000","last_clean":"2024-09-21T00:29:51.164501+0000","last_became_active":"2024-09-21T00:29:48.108614+0000","last_became_peered":"2024-09-21T00:29:48.108614+0000","last_unstale":"2024-09-21T00:29:51.164501+0000","last_undegraded":"2024-09-21T00:29:51.164501+0000","last_fullsized":"2024-09-21T00:29:51.164501+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:18:12.097196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:50.095444+0000","last_change":"2024-09-21T00:29:49.117960+0000","last_active":"2024-09-21T00:29:50.095444+0000","last_peered":"2024-09-21T00:29:50.095444+0000","last_clean":"2024-09-21T00:29:50.095444+0000","last_became_active":"2024-09-21T00:29:49.117642+0000","last_became_peered":"2024-09-21T00:29:49.117642+0000","last_unstale":"2024-09-21T00:29:50.095444+0000","last_undegraded":"2024-09-21T00:29:50.095444+0000","last_fullsized":"2024-09-21T00:29:50.095444+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:11:38.972017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":15,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:51.164239+0000","last_change":"2024-09-21T00:29:48.111869+0000","last_active":"2024-09-21T00:29:51.164239+0000","last_peered":"2024-09-21T00:29:51.164239+0000","last_clean":"2024-09-21T00:29:51.164239+0000","last_became_active":"2024-09-21T00:29:48.111497+0000","last_became_peered":"2024-09-21T00:29:48.111497+0000","last_unstale":"2024-09-21T00:29:51.164239+0000","last_undegraded":"2024-09-21T00:29:51.164239+0000","last_fullsized":"2024-09-21T00:29:51.164239+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:27:52.077806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:51.101233+0000","last_change":"2024-09-21T00:29:49.116325+0000","last_active":"2024-09-21T00:29:51.101233+0000","last_peered":"2024-09-21T00:29:51.101233+0000","last_clean":"2024-09-21T00:29:51.101233+0000","last_became_active":"2024-09-21T00:29:49.115969+0000","last_became_peered":"2024-09-21T00:29:49.115969+0000","last_unstale":"2024-09-21T00:29:51.101233+0000","last_undegraded":"2024-09-21T00:29:51.101233+0000","last_fullsized":"2024-09-21T00:29:51.101233+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:02:14.985306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.b","version":"107'2","reported_seq":17,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:51.101065+0000","last_change":"2024-09-21T00:29:48.108744+0000","last_active":"2024-09-21T00:29:51.101065+0000","last_peered":"2024-09-21T00:29:51.101065+0000","last_clean":"2024-09-21T00:29:51.101065+0000","last_became_active":"2024-09-21T00:29:48.108377+0000","last_became_peered":"2024-09-21T00:29:48.108377+0000","last_unstale":"2024-09-21T00:29:51.101065+0000","last_undegraded":"2024-09-21T00:29:51.101065+0000","last_fullsized":"2024-09-21T00:29:51.101065+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:08:14.403928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:51.164206+0000","last_change":"2024-09-21T00:29:49.112211+0000","last_active":"2024-09-21T00:29:51.164206+0000","last_peered":"2024-09-21T00:29:51.164206+0000","last_clean":"2024-09-21T00:29:51.164206+0000","last_became_active":"2024-09-21T00:29:49.111893+0000","last_became_peered":"2024-09-21T00:29:49.111893+0000","last_unstale":"2024-09-21T00:29:51.164206+0000","last_undegraded":"2024-09-21T00:29:51.164206+0000","last_fullsized":"2024-09-21T00:29:51.164206+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:21:26.639292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":15,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:51.164017+0000","last_change":"2024-09-21T00:29:48.114856+0000","last_active":"2024-09-21T00:29:51.164017+0000","last_peered":"2024-09-21T00:29:51.164017+0000","last_clean":"2024-09-21T00:29:51.164017+0000","last_became_active":"2024-09-21T00:29:48.114394+0000","last_became_peered":"2024-09-21T00:29:48.114394+0000","last_unstale":"2024-09-21T00:29:51.164017+0000","last_undegraded":"2024-09-21T00:29:51.164017+0000","last_fullsized":"2024-09-21T00:29:51.164017+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:35:35.720354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:51.165086+0000","last_change":"2024-09-21T00:29:49.122315+0000","last_active":"2024-09-21T00:29:51.165086+0000","last_peered":"2024-09-21T00:29:51.165086+0000","last_clean":"2024-09-21T00:29:51.165086+0000","last_became_active":"2024-09-21T00:29:49.121467+0000","last_became_peered":"2024-09-21T00:29:49.121467+0000","last_unstale":"2024-09-21T00:29:51.165086+0000","last_undegraded":"2024-09-21T00:29:51.165086+0000","last_fullsized":"2024-09-21T00:29:51.165086+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:05:41.315339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":15,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:51.164325+0000","last_change":"2024-09-21T00:29:48.106397+0000","last_active":"2024-09-21T00:29:51.164325+0000","last_peered":"2024-09-21T00:29:51.164325+0000","last_clean":"2024-09-21T00:29:51.164325+0000","last_became_active":"2024-09-21T00:29:48.105468+0000","last_became_peered":"2024-09-21T00:29:48.105468+0000","last_unstale":"2024-09-21T00:29:51.164325+0000","last_undegraded":"2024-09-21T00:29:51.164325+0000","last_fullsized":"2024-09-21T00:29:51.164325+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:27:52.077806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:50.094870+0000","last_change":"2024-09-21T00:29:49.108718+0000","last_active":"2024-09-21T00:29:50.094870+0000","last_peered":"2024-09-21T00:29:50.094870+0000","last_clean":"2024-09-21T00:29:50.094870+0000","last_became_active":"2024-09-21T00:29:49.108401+0000","last_became_peered":"2024-09-21T00:29:49.108401+0000","last_unstale":"2024-09-21T00:29:50.094870+0000","last_undegraded":"2024-09-21T00:29:50.094870+0000","last_fullsized":"2024-09-21T00:29:50.094870+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:37:08.281841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":15,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:51.163904+0000","last_change":"2024-09-21T00:29:48.105932+0000","last_active":"2024-09-21T00:29:51.163904+0000","last_peered":"2024-09-21T00:29:51.163904+0000","last_clean":"2024-09-21T00:29:51.163904+0000","last_became_active":"2024-09-21T00:29:48.105570+0000","last_became_peered":"2024-09-21T00:29:48.105570+0000","last_unstale":"2024-09-21T00:29:51.163904+0000","last_undegraded":"2024-09-21T00:29:51.163904+0000","last_fullsized":"2024-09-21T00:29:51.163904+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:36:09.108085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:51.101609+0000","last_change":"2024-09-21T00:29:49.106013+0000","last_active":"2024-09-21T00:29:51.101609+0000","last_peered":"2024-09-21T00:29:51.101609+0000","last_clean":"2024-09-21T00:29:51.101609+0000","last_became_active":"2024-09-21T00:29:49.105459+0000","last_became_peered":"2024-09-21T00:29:49.105459+0000","last_unstale":"2024-09-21T00:29:51.101609+0000","last_undegraded":"2024-09-21T00:29:51.101609+0000","last_fullsized":"2024-09-21T00:29:51.101609+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:19:16.290142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.7","version":"107'2","reported_seq":17,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:51.101488+0000","last_change":"2024-09-21T00:29:48.114760+0000","last_active":"2024-09-21T00:29:51.101488+0000","last_peered":"2024-09-21T00:29:51.101488+0000","last_clean":"2024-09-21T00:29:51.101488+0000","last_became_active":"2024-09-21T00:29:48.114326+0000","last_became_peered":"2024-09-21T00:29:48.114326+0000","last_unstale":"2024-09-21T00:29:51.101488+0000","last_undegraded":"2024-09-21T00:29:51.101488+0000","last_fullsized":"2024-09-21T00:29:51.101488+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:04:09.518964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-09-21T00:29:48.095849+0000","last_change":"2024-09-21T00:29:48.095849+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.095849+0000","last_undegraded":"2024-09-21T00:29:48.095849+0000","last_fullsized":"2024-09-21T00:29:48.095849+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.6","version":"108'1","reported_seq":16,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:51.100577+0000","last_change":"2024-09-21T00:29:48.111642+0000","last_active":"2024-09-21T00:29:51.100577+0000","last_peered":"2024-09-21T00:29:51.100577+0000","last_clean":"2024-09-21T00:29:51.100577+0000","last_became_active":"2024-09-21T00:29:48.111306+0000","last_became_peered":"2024-09-21T00:29:48.111306+0000","last_unstale":"2024-09-21T00:29:51.100577+0000","last_undegraded":"2024-09-21T00:29:51.100577+0000","last_fullsized":"2024-09-21T00:29:51.100577+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:22:52.316356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:51.164692+0000","last_change":"2024-09-21T00:29:49.128963+0000","last_active":"2024-09-21T00:29:51.164692+0000","last_peered":"2024-09-21T00:29:51.164692+0000","last_clean":"2024-09-21T00:29:51.164692+0000","last_became_active":"2024-09-21T00:29:49.128634+0000","last_became_peered":"2024-09-21T00:29:49.128634+0000","last_unstale":"2024-09-21T00:29:51.164692+0000","last_undegraded":"2024-09-21T00:29:51.164692+0000","last_fullsized":"2024-09-21T00:29:51.164692+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:47:19.603347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":15,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:51.099046+0000","last_change":"2024-09-21T00:29:48.112285+0000","last_active":"2024-09-21T00:29:51.099046+0000","last_peered":"2024-09-21T00:29:51.099046+0000","last_clean":"2024-09-21T00:29:51.099046+0000","last_became_active":"2024-09-21T00:29:48.111945+0000","last_became_peered":"2024-09-21T00:29:48.111945+0000","last_unstale":"2024-09-21T00:29:51.099046+0000","last_undegraded":"2024-09-21T00:29:51.099046+0000","last_fullsized":"2024-09-21T00:29:51.099046+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:24:40.002666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:50.095275+0000","last_change":"2024-09-21T00:29:49.111611+0000","last_active":"2024-09-21T00:29:50.095275+0000","last_peered":"2024-09-21T00:29:50.095275+0000","last_clean":"2024-09-21T00:29:50.095275+0000","last_became_active":"2024-09-21T00:29:49.110763+0000","last_became_peered":"2024-09-21T00:29:49.110763+0000","last_unstale":"2024-09-21T00:29:50.095275+0000","last_undegraded":"2024-09-21T00:29:50.095275+0000","last_fullsized":"2024-09-21T00:29:50.095275+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:33:10.243092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":15,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:51.101638+0000","last_change":"2024-09-21T00:29:48.110571+0000","last_active":"2024-09-21T00:29:51.101638+0000","last_peered":"2024-09-21T00:29:51.101638+0000","last_clean":"2024-09-21T00:29:51.101638+0000","last_became_active":"2024-09-21T00:29:48.110252+0000","last_became_peered":"2024-09-21T00:29:48.110252+0000","last_unstale":"2024-09-21T00:29:51.101638+0000","last_undegraded":"2024-09-21T00:29:51.101638+0000","last_fullsized":"2024-09-21T00:29:51.101638+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:04:20.619113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:51.101237+0000","last_change":"2024-09-21T00:29:49.103676+0000","last_active":"2024-09-21T00:29:51.101237+0000","last_peered":"2024-09-21T00:29:51.101237+0000","last_clean":"2024-09-21T00:29:51.101237+0000","last_became_active":"2024-09-21T00:29:49.103338+0000","last_became_peered":"2024-09-21T00:29:49.103338+0000","last_unstale":"2024-09-21T00:29:51.101237+0000","last_undegraded":"2024-09-21T00:29:51.101237+0000","last_fullsized":"2024-09-21T00:29:51.101237+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:39:22.826869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":15,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:50.094972+0000","last_change":"2024-09-21T00:29:48.111343+0000","last_active":"2024-09-21T00:29:50.094972+0000","last_peered":"2024-09-21T00:29:50.094972+0000","last_clean":"2024-09-21T00:29:50.094972+0000","last_became_active":"2024-09-21T00:29:48.111032+0000","last_became_peered":"2024-09-21T00:29:48.111032+0000","last_unstale":"2024-09-21T00:29:50.094972+0000","last_undegraded":"2024-09-21T00:29:50.094972+0000","last_fullsized":"2024-09-21T00:29:50.094972+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:39:13.998725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:51.163972+0000","last_change":"2024-09-21T00:29:49.121015+0000","last_active":"2024-09-21T00:29:51.163972+0000","last_peered":"2024-09-21T00:29:51.163972+0000","last_clean":"2024-09-21T00:29:51.163972+0000","last_became_active":"2024-09-21T00:29:49.120710+0000","last_became_peered":"2024-09-21T00:29:49.120710+0000","last_unstale":"2024-09-21T00:29:51.163972+0000","last_undegraded":"2024-09-21T00:29:51.163972+0000","last_fullsized":"2024-09-21T00:29:51.163972+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:40:47.831055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":15,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:50.094242+0000","last_change":"2024-09-21T00:29:48.111988+0000","last_active":"2024-09-21T00:29:50.094242+0000","last_peered":"2024-09-21T00:29:50.094242+0000","last_clean":"2024-09-21T00:29:50.094242+0000","last_became_active":"2024-09-21T00:29:48.110924+0000","last_became_peered":"2024-09-21T00:29:48.110924+0000","last_unstale":"2024-09-21T00:29:50.094242+0000","last_undegraded":"2024-09-21T00:29:50.094242+0000","last_fullsized":"2024-09-21T00:29:50.094242+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:02:38.960612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:51.164214+0000","last_change":"2024-09-21T00:29:49.115962+0000","last_active":"2024-09-21T00:29:51.164214+0000","last_peered":"2024-09-21T00:29:51.164214+0000","last_clean":"2024-09-21T00:29:51.164214+0000","last_became_active":"2024-09-21T00:29:49.115602+0000","last_became_peered":"2024-09-21T00:29:49.115602+0000","last_unstale":"2024-09-21T00:29:51.164214+0000","last_undegraded":"2024-09-21T00:29:51.164214+0000","last_fullsized":"2024-09-21T00:29:51.164214+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:11:45.373594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":15,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:51.101044+0000","last_change":"2024-09-21T00:29:48.110619+0000","last_active":"2024-09-21T00:29:51.101044+0000","last_peered":"2024-09-21T00:29:51.101044+0000","last_clean":"2024-09-21T00:29:51.101044+0000","last_became_active":"2024-09-21T00:29:48.110292+0000","last_became_peered":"2024-09-21T00:29:48.110292+0000","last_unstale":"2024-09-21T00:29:51.101044+0000","last_undegraded":"2024-09-21T00:29:51.101044+0000","last_fullsized":"2024-09-21T00:29:51.101044+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:36:50.249803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-09-21T00:29:48.095876+0000","last_change":"2024-09-21T00:29:48.095876+0000","last_active":"2024-09-21T00:29:48.079691+0000","last_peered":"2024-09-21T00:29:48.079691+0000","last_clean":"2024-09-21T00:29:48.079691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T00:29:48.095876+0000","last_undegraded":"2024-09-21T00:29:48.095876+0000","last_fullsized":"2024-09-21T00:29:48.095876+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":15,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:51.163893+0000","last_change":"2024-09-21T00:29:48.107972+0000","last_active":"2024-09-21T00:29:51.163893+0000","last_peered":"2024-09-21T00:29:51.163893+0000","last_clean":"2024-09-21T00:29:51.163893+0000","last_became_active":"2024-09-21T00:29:48.107553+0000","last_became_peered":"2024-09-21T00:29:48.107553+0000","last_unstale":"2024-09-21T00:29:51.163893+0000","last_undegraded":"2024-09-21T00:29:51.163893+0000","last_fullsized":"2024-09-21T00:29:51.163893+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:53:15.982238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:51.164466+0000","last_change":"2024-09-21T00:29:49.118874+0000","last_active":"2024-09-21T00:29:51.164466+0000","last_peered":"2024-09-21T00:29:51.164466+0000","last_clean":"2024-09-21T00:29:51.164466+0000","last_became_active":"2024-09-21T00:29:49.118310+0000","last_became_peered":"2024-09-21T00:29:49.118310+0000","last_unstale":"2024-09-21T00:29:51.164466+0000","last_undegraded":"2024-09-21T00:29:51.164466+0000","last_fullsized":"2024-09-21T00:29:51.164466+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:58:30.921993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.10","version":"107'8","reported_seq":23,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:51.164071+0000","last_change":"2024-09-21T00:29:48.107294+0000","last_active":"2024-09-21T00:29:51.164071+0000","last_peered":"2024-09-21T00:29:51.164071+0000","last_clean":"2024-09-21T00:29:51.164071+0000","last_became_active":"2024-09-21T00:29:48.107003+0000","last_became_peered":"2024-09-21T00:29:48.107003+0000","last_unstale":"2024-09-21T00:29:51.164071+0000","last_undegraded":"2024-09-21T00:29:51.164071+0000","last_fullsized":"2024-09-21T00:29:51.164071+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:36:09.108085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":271,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:51.164055+0000","last_change":"2024-09-21T00:25:38.725792+0000","last_active":"2024-09-21T00:29:51.164055+0000","last_peered":"2024-09-21T00:29:51.164055+0000","last_clean":"2024-09-21T00:29:51.164055+0000","last_became_active":"2024-09-21T00:25:38.724668+0000","last_became_peered":"2024-09-21T00:25:38.724668+0000","last_unstale":"2024-09-21T00:29:51.164055+0000","last_undegraded":"2024-09-21T00:29:51.164055+0000","last_fullsized":"2024-09-21T00:29:51.164055+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:37.163216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:37.163216+0000","last_clean_scrub_stamp":"2024-09-21T00:25:37.163216+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:14:36.588660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":2388,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":35,"ondisk_log_size":35,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509494,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33092,"kb_used_data":1904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338748,"statfs":{"total":96636764160,"available":96602877952,"internally_reserved":0,"allocated":1949696,"data_stored":1203121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509495,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32500,"kb_used_data":1312,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339340,"statfs":{"total":96636764160,"available":96603484160,"internally_reserved":0,"allocated":1343488,"data_stored":612559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509494,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32504,"kb_used_data":1316,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339336,"statfs":{"total":96636764160,"available":96603480064,"internally_reserved":0,"allocated":1347584,"data_stored":612137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509493,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32508,"kb_used_data":1320,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339332,"statfs":{"total":96636764160,"available":96603475968,"internally_reserved":0,"allocated":1351680,"data_stored":613589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509495,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32508,"kb_used_data":1312,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339332,"statfs":{"total":96636764160,"available":96603475968,"internally_reserved":0,"allocated":1343488,"data_stored":606931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509495,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32504,"kb_used_data":1308,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339336,"statfs":{"total":96636764160,"available":96603480064,"internally_reserved":0,"allocated":1339392,"data_stored":611977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509495,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33076,"kb_used_data":1880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338764,"statfs":{"total":96636764160,"available":96602894336,"internally_reserved":0,"allocated":1925120,"data_stored":1195675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574902,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32476,"kb_used_data":1284,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339364,"statfs":{"total":96636764160,"available":96603508736,"internally_reserved":0,"allocated":1314816,"data_stored":598118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T00:29:54.434 INFO:tasks.ceph.mds.d.smithi187.stderr:2024-09-21T00:29:54.433+0000 7f4fc03c1640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T00:29:56.575 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T00:29:56.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:56.700+0000 7fbb3996a640 1 -- 172.21.15.112:0/3120572238 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbb3415c3c0 msgr2=0x7fbb340b3260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:56.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:56.700+0000 7fbb3996a640 1 --2- 172.21.15.112:0/3120572238 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbb3415c3c0 0x7fbb340b3260 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7fbb240098e0 tx=0x7fbb2402a530 comp rx=0 tx=0).stop 2024-09-21T00:29:56.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:56.700+0000 7fbb3996a640 1 -- 172.21.15.112:0/3120572238 shutdown_connections 2024-09-21T00:29:56.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:56.700+0000 7fbb3996a640 1 --2- 172.21.15.112:0/3120572238 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbb340b3bb0 0x7fbb340b6740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:56.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:56.700+0000 7fbb3996a640 1 --2- 172.21.15.112:0/3120572238 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbb3415c3c0 0x7fbb340b3260 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:56.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:56.700+0000 7fbb3996a640 1 -- 172.21.15.112:0/3120572238 >> 172.21.15.112:0/3120572238 conn(0x7fbb3408dcb0 msgr2=0x7fbb3409d8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:56.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:56.700+0000 7fbb3996a640 1 -- 172.21.15.112:0/3120572238 shutdown_connections 2024-09-21T00:29:56.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:56.700+0000 7fbb3996a640 1 -- 172.21.15.112:0/3120572238 wait complete. 2024-09-21T00:29:56.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:56.702+0000 7fbb3996a640 1 Processor -- start 2024-09-21T00:29:56.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:56.702+0000 7fbb3996a640 1 -- start start 2024-09-21T00:29:56.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:56.703+0000 7fbb3996a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbb340b37a0 0x7fbb3415f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:56.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:56.703+0000 7fbb3996a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbb340b3bb0 0x7fbb341f8d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:56.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:56.703+0000 7fbb3996a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbb3415c3c0 0x7fbb341f9240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:56.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:56.703+0000 7fbb3996a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbb34163180 con 0x7fbb340b37a0 2024-09-21T00:29:56.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:56.703+0000 7fbb3996a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fbb340d0d30 con 0x7fbb3415c3c0 2024-09-21T00:29:56.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:56.703+0000 7fbb3996a640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fbb34163300 con 0x7fbb340b3bb0 2024-09-21T00:29:56.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:56.703+0000 7fbb32ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbb340b37a0 0x7fbb3415f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:56.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:56.703+0000 7fbb32ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbb340b37a0 0x7fbb3415f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40708/0 (socket says 172.21.15.112:40708) 2024-09-21T00:29:56.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:56.703+0000 7fbb32ffd640 1 -- 172.21.15.112:0/653789856 learned_addr learned my addr 172.21.15.112:0/653789856 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:29:56.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:56.704+0000 7fbb327fc640 1 --2- 172.21.15.112:0/653789856 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbb340b3bb0 0x7fbb341f8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:56.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:56.704+0000 7fbb337fe640 1 --2- 172.21.15.112:0/653789856 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbb3415c3c0 0x7fbb341f9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:56.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:56.704+0000 7fbb32ffd640 1 -- 172.21.15.112:0/653789856 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbb340b3bb0 msgr2=0x7fbb341f8d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:56.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:56.704+0000 7fbb32ffd640 1 --2- 172.21.15.112:0/653789856 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbb340b3bb0 0x7fbb341f8d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:56.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:56.704+0000 7fbb32ffd640 1 -- 172.21.15.112:0/653789856 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbb3415c3c0 msgr2=0x7fbb341f9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:56.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:56.704+0000 7fbb32ffd640 1 --2- 172.21.15.112:0/653789856 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbb3415c3c0 0x7fbb341f9240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:56.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:56.704+0000 7fbb32ffd640 1 -- 172.21.15.112:0/653789856 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb3414e760 con 0x7fbb340b37a0 2024-09-21T00:29:56.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:56.704+0000 7fbb32ffd640 1 --2- 172.21.15.112:0/653789856 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbb340b37a0 0x7fbb3415f080 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7fbb24009a00 tx=0x7fbb240052f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:56.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:56.704+0000 7fbb1bfff640 1 -- 172.21.15.112:0/653789856 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb2403f070 con 0x7fbb340b37a0 2024-09-21T00:29:56.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:56.704+0000 7fbb1bfff640 1 -- 172.21.15.112:0/653789856 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbb24004340 con 0x7fbb340b37a0 2024-09-21T00:29:56.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:56.704+0000 7fbb3996a640 1 -- 172.21.15.112:0/653789856 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb340cbbf0 con 0x7fbb340b37a0 2024-09-21T00:29:56.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:56.704+0000 7fbb1bfff640 1 -- 172.21.15.112:0/653789856 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb24005cf0 con 0x7fbb340b37a0 2024-09-21T00:29:56.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:56.704+0000 7fbb3996a640 1 -- 172.21.15.112:0/653789856 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb340cbe90 con 0x7fbb340b37a0 2024-09-21T00:29:56.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:56.705+0000 7fbb3996a640 1 -- 172.21.15.112:0/653789856 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb340b3260 con 0x7fbb340b37a0 2024-09-21T00:29:56.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:56.706+0000 7fbb1bfff640 1 -- 172.21.15.112:0/653789856 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fbb24042070 con 0x7fbb340b37a0 2024-09-21T00:29:56.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:56.706+0000 7fbb1bfff640 1 --2- 172.21.15.112:0/653789856 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fbb20078bc0 0x7fbb2007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:56.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:56.706+0000 7fbb1bfff640 1 -- 172.21.15.112:0/653789856 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 6101+0+0 (secure 0 0 0) 0x7fbb240bbc80 con 0x7fbb340b37a0 2024-09-21T00:29:56.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:56.707+0000 7fbb327fc640 1 --2- 172.21.15.112:0/653789856 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fbb20078bc0 0x7fbb2007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:56.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:56.707+0000 7fbb327fc640 1 --2- 172.21.15.112:0/653789856 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fbb20078bc0 0x7fbb2007b0d0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fbb3415fb10 tx=0x7fbb28009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:56.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:56.711+0000 7fbb1bfff640 1 -- 172.21.15.112:0/653789856 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fbb24031770 con 0x7fbb340b37a0 2024-09-21T00:29:56.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:56.884+0000 7fbb3996a640 1 -- 172.21.15.112:0/653789856 --> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbb340cc230 con 0x7fbb20078bc0 2024-09-21T00:29:56.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:56.889+0000 7fbb1bfff640 1 -- 172.21.15.112:0/653789856 <== mgr.4103 v2:172.21.15.187:6800/2921522906 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252477 (secure 0 0 0) 0x7fbb340cc230 con 0x7fbb20078bc0 2024-09-21T00:29:56.891 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:29:56.899 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2024-09-21T00:29:56.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:56.892+0000 7fbb3996a640 1 -- 172.21.15.112:0/653789856 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fbb20078bc0 msgr2=0x7fbb2007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:56.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:56.892+0000 7fbb3996a640 1 --2- 172.21.15.112:0/653789856 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fbb20078bc0 0x7fbb2007b0d0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fbb3415fb10 tx=0x7fbb28009040 comp rx=0 tx=0).stop 2024-09-21T00:29:56.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:56.892+0000 7fbb3996a640 1 -- 172.21.15.112:0/653789856 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbb340b37a0 msgr2=0x7fbb3415f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:56.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:56.892+0000 7fbb3996a640 1 --2- 172.21.15.112:0/653789856 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbb340b37a0 0x7fbb3415f080 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7fbb24009a00 tx=0x7fbb240052f0 comp rx=0 tx=0).stop 2024-09-21T00:29:56.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:56.892+0000 7fbb3996a640 1 -- 172.21.15.112:0/653789856 shutdown_connections 2024-09-21T00:29:56.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:56.892+0000 7fbb3996a640 1 --2- 172.21.15.112:0/653789856 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbb3415c3c0 0x7fbb341f9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:56.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:56.892+0000 7fbb3996a640 1 --2- 172.21.15.112:0/653789856 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbb340b3bb0 0x7fbb341f8d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:56.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:56.892+0000 7fbb3996a640 1 --2- 172.21.15.112:0/653789856 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fbb20078bc0 0x7fbb2007b0d0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:56.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:56.892+0000 7fbb3996a640 1 --2- 172.21.15.112:0/653789856 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbb340b37a0 0x7fbb3415f080 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:56.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:56.892+0000 7fbb3996a640 1 -- 172.21.15.112:0/653789856 >> 172.21.15.112:0/653789856 conn(0x7fbb3408dcb0 msgr2=0x7fbb34160060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:56.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:56.893+0000 7fbb3996a640 1 -- 172.21.15.112:0/653789856 shutdown_connections 2024-09-21T00:29:56.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:56.893+0000 7fbb3996a640 1 -- 172.21.15.112:0/653789856 wait complete. 2024-09-21T00:29:56.905 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":232,"stamp":"2024-09-21T00:29:55.153556+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":174,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":261196,"kb_used_data":11664,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754713524,"statfs":{"total":773094113280,"available":772826648576,"internally_reserved":0,"allocated":11943936,"data_stored":6067988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.060364"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:50.094565+0000","last_change":"2024-09-21T00:29:49.108008+0000","last_active":"2024-09-21T00:29:50.094565+0000","last_peered":"2024-09-21T00:29:50.094565+0000","last_clean":"2024-09-21T00:29:50.094565+0000","last_became_active":"2024-09-21T00:29:49.107047+0000","last_became_peered":"2024-09-21T00:29:49.107047+0000","last_unstale":"2024-09-21T00:29:50.094565+0000","last_undegraded":"2024-09-21T00:29:50.094565+0000","last_fullsized":"2024-09-21T00:29:50.094565+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:53:16.984494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:51.164613+0000","last_change":"2024-09-21T00:29:49.128276+0000","last_active":"2024-09-21T00:29:51.164613+0000","last_peered":"2024-09-21T00:29:51.164613+0000","last_clean":"2024-09-21T00:29:51.164613+0000","last_became_active":"2024-09-21T00:29:49.127942+0000","last_became_peered":"2024-09-21T00:29:49.127942+0000","last_unstale":"2024-09-21T00:29:51.164613+0000","last_undegraded":"2024-09-21T00:29:51.164613+0000","last_fullsized":"2024-09-21T00:29:51.164613+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:46:55.941733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:51.164642+0000","last_change":"2024-09-21T00:29:49.121824+0000","last_active":"2024-09-21T00:29:51.164642+0000","last_peered":"2024-09-21T00:29:51.164642+0000","last_clean":"2024-09-21T00:29:51.164642+0000","last_became_active":"2024-09-21T00:29:49.121486+0000","last_became_peered":"2024-09-21T00:29:49.121486+0000","last_unstale":"2024-09-21T00:29:51.164642+0000","last_undegraded":"2024-09-21T00:29:51.164642+0000","last_fullsized":"2024-09-21T00:29:51.164642+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:21:34.883179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:51.164354+0000","last_change":"2024-09-21T00:29:49.128936+0000","last_active":"2024-09-21T00:29:51.164354+0000","last_peered":"2024-09-21T00:29:51.164354+0000","last_clean":"2024-09-21T00:29:51.164354+0000","last_became_active":"2024-09-21T00:29:49.128662+0000","last_became_peered":"2024-09-21T00:29:49.128662+0000","last_unstale":"2024-09-21T00:29:51.164354+0000","last_undegraded":"2024-09-21T00:29:51.164354+0000","last_fullsized":"2024-09-21T00:29:51.164354+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:16:09.264216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:50.093904+0000","last_change":"2024-09-21T00:29:49.108365+0000","last_active":"2024-09-21T00:29:50.093904+0000","last_peered":"2024-09-21T00:29:50.093904+0000","last_clean":"2024-09-21T00:29:50.093904+0000","last_became_active":"2024-09-21T00:29:49.108004+0000","last_became_peered":"2024-09-21T00:29:49.108004+0000","last_unstale":"2024-09-21T00:29:50.093904+0000","last_undegraded":"2024-09-21T00:29:50.093904+0000","last_fullsized":"2024-09-21T00:29:50.093904+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:39:43.972280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:51.101083+0000","last_change":"2024-09-21T00:29:49.106784+0000","last_active":"2024-09-21T00:29:51.101083+0000","last_peered":"2024-09-21T00:29:51.101083+0000","last_clean":"2024-09-21T00:29:51.101083+0000","last_became_active":"2024-09-21T00:29:49.106502+0000","last_became_peered":"2024-09-21T00:29:49.106502+0000","last_unstale":"2024-09-21T00:29:51.101083+0000","last_undegraded":"2024-09-21T00:29:51.101083+0000","last_fullsized":"2024-09-21T00:29:51.101083+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:27:41.443648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:51.101031+0000","last_change":"2024-09-21T00:29:49.119608+0000","last_active":"2024-09-21T00:29:51.101031+0000","last_peered":"2024-09-21T00:29:51.101031+0000","last_clean":"2024-09-21T00:29:51.101031+0000","last_became_active":"2024-09-21T00:29:49.119217+0000","last_became_peered":"2024-09-21T00:29:49.119217+0000","last_unstale":"2024-09-21T00:29:51.101031+0000","last_undegraded":"2024-09-21T00:29:51.101031+0000","last_fullsized":"2024-09-21T00:29:51.101031+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:54:13.945567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:51.163936+0000","last_change":"2024-09-21T00:29:49.122526+0000","last_active":"2024-09-21T00:29:51.163936+0000","last_peered":"2024-09-21T00:29:51.163936+0000","last_clean":"2024-09-21T00:29:51.163936+0000","last_became_active":"2024-09-21T00:29:49.122215+0000","last_became_peered":"2024-09-21T00:29:49.122215+0000","last_unstale":"2024-09-21T00:29:51.163936+0000","last_undegraded":"2024-09-21T00:29:51.163936+0000","last_fullsized":"2024-09-21T00:29:51.163936+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:16:09.264216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:51.164751+0000","last_change":"2024-09-21T00:29:49.126353+0000","last_active":"2024-09-21T00:29:51.164751+0000","last_peered":"2024-09-21T00:29:51.164751+0000","last_clean":"2024-09-21T00:29:51.164751+0000","last_became_active":"2024-09-21T00:29:49.126053+0000","last_became_peered":"2024-09-21T00:29:49.126053+0000","last_unstale":"2024-09-21T00:29:51.164751+0000","last_undegraded":"2024-09-21T00:29:51.164751+0000","last_fullsized":"2024-09-21T00:29:51.164751+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:28:58.560933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:51.163971+0000","last_change":"2024-09-21T00:29:49.109476+0000","last_active":"2024-09-21T00:29:51.163971+0000","last_peered":"2024-09-21T00:29:51.163971+0000","last_clean":"2024-09-21T00:29:51.163971+0000","last_became_active":"2024-09-21T00:29:49.109156+0000","last_became_peered":"2024-09-21T00:29:49.109156+0000","last_unstale":"2024-09-21T00:29:51.163971+0000","last_undegraded":"2024-09-21T00:29:51.163971+0000","last_fullsized":"2024-09-21T00:29:51.163971+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:09:54.207029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:50.094412+0000","last_change":"2024-09-21T00:29:49.112574+0000","last_active":"2024-09-21T00:29:50.094412+0000","last_peered":"2024-09-21T00:29:50.094412+0000","last_clean":"2024-09-21T00:29:50.094412+0000","last_became_active":"2024-09-21T00:29:49.112267+0000","last_became_peered":"2024-09-21T00:29:49.112267+0000","last_unstale":"2024-09-21T00:29:50.094412+0000","last_undegraded":"2024-09-21T00:29:50.094412+0000","last_fullsized":"2024-09-21T00:29:50.094412+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:36:10.110340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:51.101475+0000","last_change":"2024-09-21T00:29:49.105771+0000","last_active":"2024-09-21T00:29:51.101475+0000","last_peered":"2024-09-21T00:29:51.101475+0000","last_clean":"2024-09-21T00:29:51.101475+0000","last_became_active":"2024-09-21T00:29:49.105062+0000","last_became_peered":"2024-09-21T00:29:49.105062+0000","last_unstale":"2024-09-21T00:29:51.101475+0000","last_undegraded":"2024-09-21T00:29:51.101475+0000","last_fullsized":"2024-09-21T00:29:51.101475+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:27:23.331150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:51.164473+0000","last_change":"2024-09-21T00:29:49.116392+0000","last_active":"2024-09-21T00:29:51.164473+0000","last_peered":"2024-09-21T00:29:51.164473+0000","last_clean":"2024-09-21T00:29:51.164473+0000","last_became_active":"2024-09-21T00:29:49.116013+0000","last_became_peered":"2024-09-21T00:29:49.116013+0000","last_unstale":"2024-09-21T00:29:51.164473+0000","last_undegraded":"2024-09-21T00:29:51.164473+0000","last_fullsized":"2024-09-21T00:29:51.164473+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:29:48.268483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:51.164695+0000","last_change":"2024-09-21T00:29:49.123712+0000","last_active":"2024-09-21T00:29:51.164695+0000","last_peered":"2024-09-21T00:29:51.164695+0000","last_clean":"2024-09-21T00:29:51.164695+0000","last_became_active":"2024-09-21T00:29:49.121885+0000","last_became_peered":"2024-09-21T00:29:49.121885+0000","last_unstale":"2024-09-21T00:29:51.164695+0000","last_undegraded":"2024-09-21T00:29:51.164695+0000","last_fullsized":"2024-09-21T00:29:51.164695+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:54:11.086218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:50.094237+0000","last_change":"2024-09-21T00:29:49.112561+0000","last_active":"2024-09-21T00:29:50.094237+0000","last_peered":"2024-09-21T00:29:50.094237+0000","last_clean":"2024-09-21T00:29:50.094237+0000","last_became_active":"2024-09-21T00:29:49.112203+0000","last_became_peered":"2024-09-21T00:29:49.112203+0000","last_unstale":"2024-09-21T00:29:50.094237+0000","last_undegraded":"2024-09-21T00:29:50.094237+0000","last_fullsized":"2024-09-21T00:29:50.094237+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:59:27.769358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:51.101009+0000","last_change":"2024-09-21T00:29:49.115569+0000","last_active":"2024-09-21T00:29:51.101009+0000","last_peered":"2024-09-21T00:29:51.101009+0000","last_clean":"2024-09-21T00:29:51.101009+0000","last_became_active":"2024-09-21T00:29:49.115141+0000","last_became_peered":"2024-09-21T00:29:49.115141+0000","last_unstale":"2024-09-21T00:29:51.101009+0000","last_undegraded":"2024-09-21T00:29:51.101009+0000","last_fullsized":"2024-09-21T00:29:51.101009+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:03:51.843446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:51.164316+0000","last_change":"2024-09-21T00:29:49.115711+0000","last_active":"2024-09-21T00:29:51.164316+0000","last_peered":"2024-09-21T00:29:51.164316+0000","last_clean":"2024-09-21T00:29:51.164316+0000","last_became_active":"2024-09-21T00:29:49.115141+0000","last_became_peered":"2024-09-21T00:29:49.115141+0000","last_unstale":"2024-09-21T00:29:51.164316+0000","last_undegraded":"2024-09-21T00:29:51.164316+0000","last_fullsized":"2024-09-21T00:29:51.164316+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:48:14.375384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:51.164845+0000","last_change":"2024-09-21T00:29:49.122788+0000","last_active":"2024-09-21T00:29:51.164845+0000","last_peered":"2024-09-21T00:29:51.164845+0000","last_clean":"2024-09-21T00:29:51.164845+0000","last_became_active":"2024-09-21T00:29:49.122455+0000","last_became_peered":"2024-09-21T00:29:49.122455+0000","last_unstale":"2024-09-21T00:29:51.164845+0000","last_undegraded":"2024-09-21T00:29:51.164845+0000","last_fullsized":"2024-09-21T00:29:51.164845+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:54:08.552661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:51.098900+0000","last_change":"2024-09-21T00:29:49.124316+0000","last_active":"2024-09-21T00:29:51.098900+0000","last_peered":"2024-09-21T00:29:51.098900+0000","last_clean":"2024-09-21T00:29:51.098900+0000","last_became_active":"2024-09-21T00:29:49.124097+0000","last_became_peered":"2024-09-21T00:29:49.124097+0000","last_unstale":"2024-09-21T00:29:51.098900+0000","last_undegraded":"2024-09-21T00:29:51.098900+0000","last_fullsized":"2024-09-21T00:29:51.098900+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:24:49.582188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:50.095080+0000","last_change":"2024-09-21T00:29:49.110158+0000","last_active":"2024-09-21T00:29:50.095080+0000","last_peered":"2024-09-21T00:29:50.095080+0000","last_clean":"2024-09-21T00:29:50.095080+0000","last_became_active":"2024-09-21T00:29:49.109796+0000","last_became_peered":"2024-09-21T00:29:49.109796+0000","last_unstale":"2024-09-21T00:29:50.095080+0000","last_undegraded":"2024-09-21T00:29:50.095080+0000","last_fullsized":"2024-09-21T00:29:50.095080+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:09:25.991935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:51.164320+0000","last_change":"2024-09-21T00:29:49.118758+0000","last_active":"2024-09-21T00:29:51.164320+0000","last_peered":"2024-09-21T00:29:51.164320+0000","last_clean":"2024-09-21T00:29:51.164320+0000","last_became_active":"2024-09-21T00:29:49.117990+0000","last_became_peered":"2024-09-21T00:29:49.117990+0000","last_unstale":"2024-09-21T00:29:51.164320+0000","last_undegraded":"2024-09-21T00:29:51.164320+0000","last_fullsized":"2024-09-21T00:29:51.164320+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T00:52:40.359414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:50.094687+0000","last_change":"2024-09-21T00:29:49.109190+0000","last_active":"2024-09-21T00:29:50.094687+0000","last_peered":"2024-09-21T00:29:50.094687+0000","last_clean":"2024-09-21T00:29:50.094687+0000","last_became_active":"2024-09-21T00:29:49.108841+0000","last_became_peered":"2024-09-21T00:29:49.108841+0000","last_unstale":"2024-09-21T00:29:50.094687+0000","last_undegraded":"2024-09-21T00:29:50.094687+0000","last_fullsized":"2024-09-21T00:29:50.094687+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:18:13.099451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:51.164337+0000","last_change":"2024-09-21T00:29:49.126344+0000","last_active":"2024-09-21T00:29:51.164337+0000","last_peered":"2024-09-21T00:29:51.164337+0000","last_clean":"2024-09-21T00:29:51.164337+0000","last_became_active":"2024-09-21T00:29:49.126013+0000","last_became_peered":"2024-09-21T00:29:49.126013+0000","last_unstale":"2024-09-21T00:29:51.164337+0000","last_undegraded":"2024-09-21T00:29:51.164337+0000","last_fullsized":"2024-09-21T00:29:51.164337+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:34:59.188136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:50.094378+0000","last_change":"2024-09-21T00:29:49.116501+0000","last_active":"2024-09-21T00:29:50.094378+0000","last_peered":"2024-09-21T00:29:50.094378+0000","last_clean":"2024-09-21T00:29:50.094378+0000","last_became_active":"2024-09-21T00:29:49.114983+0000","last_became_peered":"2024-09-21T00:29:49.114983+0000","last_unstale":"2024-09-21T00:29:50.094378+0000","last_undegraded":"2024-09-21T00:29:50.094378+0000","last_fullsized":"2024-09-21T00:29:50.094378+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:37:20.001351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:50.094978+0000","last_change":"2024-09-21T00:29:49.106766+0000","last_active":"2024-09-21T00:29:50.094978+0000","last_peered":"2024-09-21T00:29:50.094978+0000","last_clean":"2024-09-21T00:29:50.094978+0000","last_became_active":"2024-09-21T00:29:49.106450+0000","last_became_peered":"2024-09-21T00:29:49.106450+0000","last_unstale":"2024-09-21T00:29:50.094978+0000","last_undegraded":"2024-09-21T00:29:50.094978+0000","last_fullsized":"2024-09-21T00:29:50.094978+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:39:54.320073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:50.095416+0000","last_change":"2024-09-21T00:29:49.112842+0000","last_active":"2024-09-21T00:29:50.095416+0000","last_peered":"2024-09-21T00:29:50.095416+0000","last_clean":"2024-09-21T00:29:50.095416+0000","last_became_active":"2024-09-21T00:29:49.112592+0000","last_became_peered":"2024-09-21T00:29:49.112592+0000","last_unstale":"2024-09-21T00:29:50.095416+0000","last_undegraded":"2024-09-21T00:29:50.095416+0000","last_fullsized":"2024-09-21T00:29:50.095416+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:05:23.748224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:51.098980+0000","last_change":"2024-09-21T00:29:49.122017+0000","last_active":"2024-09-21T00:29:51.098980+0000","last_peered":"2024-09-21T00:29:51.098980+0000","last_clean":"2024-09-21T00:29:51.098980+0000","last_became_active":"2024-09-21T00:29:49.121686+0000","last_became_peered":"2024-09-21T00:29:49.121686+0000","last_unstale":"2024-09-21T00:29:51.098980+0000","last_undegraded":"2024-09-21T00:29:51.098980+0000","last_fullsized":"2024-09-21T00:29:51.098980+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:19:04.671994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:51.101566+0000","last_change":"2024-09-21T00:29:49.106112+0000","last_active":"2024-09-21T00:29:51.101566+0000","last_peered":"2024-09-21T00:29:51.101566+0000","last_clean":"2024-09-21T00:29:51.101566+0000","last_became_active":"2024-09-21T00:29:49.104746+0000","last_became_peered":"2024-09-21T00:29:49.104746+0000","last_unstale":"2024-09-21T00:29:51.101566+0000","last_undegraded":"2024-09-21T00:29:51.101566+0000","last_fullsized":"2024-09-21T00:29:51.101566+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:00:47.300212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:50.095413+0000","last_change":"2024-09-21T00:29:49.124680+0000","last_active":"2024-09-21T00:29:50.095413+0000","last_peered":"2024-09-21T00:29:50.095413+0000","last_clean":"2024-09-21T00:29:50.095413+0000","last_became_active":"2024-09-21T00:29:49.124314+0000","last_became_peered":"2024-09-21T00:29:49.124314+0000","last_unstale":"2024-09-21T00:29:50.095413+0000","last_undegraded":"2024-09-21T00:29:50.095413+0000","last_fullsized":"2024-09-21T00:29:50.095413+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:46:01.470514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:51.164915+0000","last_change":"2024-09-21T00:29:49.112291+0000","last_active":"2024-09-21T00:29:51.164915+0000","last_peered":"2024-09-21T00:29:51.164915+0000","last_clean":"2024-09-21T00:29:51.164915+0000","last_became_active":"2024-09-21T00:29:49.111111+0000","last_became_peered":"2024-09-21T00:29:49.111111+0000","last_unstale":"2024-09-21T00:29:51.164915+0000","last_undegraded":"2024-09-21T00:29:51.164915+0000","last_fullsized":"2024-09-21T00:29:51.164915+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:14:08.961080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:50.095161+0000","last_change":"2024-09-21T00:29:49.106140+0000","last_active":"2024-09-21T00:29:50.095161+0000","last_peered":"2024-09-21T00:29:50.095161+0000","last_clean":"2024-09-21T00:29:50.095161+0000","last_became_active":"2024-09-21T00:29:49.105806+0000","last_became_peered":"2024-09-21T00:29:49.105806+0000","last_unstale":"2024-09-21T00:29:50.095161+0000","last_undegraded":"2024-09-21T00:29:50.095161+0000","last_fullsized":"2024-09-21T00:29:50.095161+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:55:51.631707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:50.094589+0000","last_change":"2024-09-21T00:29:49.114756+0000","last_active":"2024-09-21T00:29:50.094589+0000","last_peered":"2024-09-21T00:29:50.094589+0000","last_clean":"2024-09-21T00:29:50.094589+0000","last_became_active":"2024-09-21T00:29:49.114271+0000","last_became_peered":"2024-09-21T00:29:49.114271+0000","last_unstale":"2024-09-21T00:29:50.094589+0000","last_undegraded":"2024-09-21T00:29:50.094589+0000","last_fullsized":"2024-09-21T00:29:50.094589+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:02:09.408653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:51.163936+0000","last_change":"2024-09-21T00:29:49.110001+0000","last_active":"2024-09-21T00:29:51.163936+0000","last_peered":"2024-09-21T00:29:51.163936+0000","last_clean":"2024-09-21T00:29:51.163936+0000","last_became_active":"2024-09-21T00:29:49.109223+0000","last_became_peered":"2024-09-21T00:29:49.109223+0000","last_unstale":"2024-09-21T00:29:51.163936+0000","last_undegraded":"2024-09-21T00:29:51.163936+0000","last_fullsized":"2024-09-21T00:29:51.163936+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:49:07.675325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1f","version":"107'4","reported_seq":19,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:51.098944+0000","last_change":"2024-09-21T00:29:48.115204+0000","last_active":"2024-09-21T00:29:51.098944+0000","last_peered":"2024-09-21T00:29:51.098944+0000","last_clean":"2024-09-21T00:29:51.098944+0000","last_became_active":"2024-09-21T00:29:48.114873+0000","last_became_peered":"2024-09-21T00:29:48.114873+0000","last_unstale":"2024-09-21T00:29:51.098944+0000","last_undegraded":"2024-09-21T00:29:51.098944+0000","last_fullsized":"2024-09-21T00:29:51.098944+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:43:59.212358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:51.101294+0000","last_change":"2024-09-21T00:29:49.109428+0000","last_active":"2024-09-21T00:29:51.101294+0000","last_peered":"2024-09-21T00:29:51.101294+0000","last_clean":"2024-09-21T00:29:51.101294+0000","last_became_active":"2024-09-21T00:29:49.109005+0000","last_became_peered":"2024-09-21T00:29:49.109005+0000","last_unstale":"2024-09-21T00:29:51.101294+0000","last_undegraded":"2024-09-21T00:29:51.101294+0000","last_fullsized":"2024-09-21T00:29:51.101294+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:21:09.408372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1e","version":"107'1","reported_seq":16,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:51.164063+0000","last_change":"2024-09-21T00:29:48.102532+0000","last_active":"2024-09-21T00:29:51.164063+0000","last_peered":"2024-09-21T00:29:51.164063+0000","last_clean":"2024-09-21T00:29:51.164063+0000","last_became_active":"2024-09-21T00:29:48.102195+0000","last_became_peered":"2024-09-21T00:29:48.102195+0000","last_unstale":"2024-09-21T00:29:51.164063+0000","last_undegraded":"2024-09-21T00:29:51.164063+0000","last_fullsized":"2024-09-21T00:29:51.164063+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:05:20.540283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:50.095686+0000","last_change":"2024-09-21T00:29:49.110911+0000","last_active":"2024-09-21T00:29:50.095686+0000","last_peered":"2024-09-21T00:29:50.095686+0000","last_clean":"2024-09-21T00:29:50.095686+0000","last_became_active":"2024-09-21T00:29:49.110574+0000","last_became_peered":"2024-09-21T00:29:49.110574+0000","last_unstale":"2024-09-21T00:29:50.095686+0000","last_undegraded":"2024-09-21T00:29:50.095686+0000","last_fullsized":"2024-09-21T00:29:50.095686+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:34:40.420590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1d","version":"107'2","reported_seq":17,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:51.164459+0000","last_change":"2024-09-21T00:29:48.110005+0000","last_active":"2024-09-21T00:29:51.164459+0000","last_peered":"2024-09-21T00:29:51.164459+0000","last_clean":"2024-09-21T00:29:51.164459+0000","last_became_active":"2024-09-21T00:29:48.109719+0000","last_became_peered":"2024-09-21T00:29:48.109719+0000","last_unstale":"2024-09-21T00:29:51.164459+0000","last_undegraded":"2024-09-21T00:29:51.164459+0000","last_fullsized":"2024-09-21T00:29:51.164459+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:04:02.357288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:51.164779+0000","last_change":"2024-09-21T00:29:49.117451+0000","last_active":"2024-09-21T00:29:51.164779+0000","last_peered":"2024-09-21T00:29:51.164779+0000","last_clean":"2024-09-21T00:29:51.164779+0000","last_became_active":"2024-09-21T00:29:49.117151+0000","last_became_peered":"2024-09-21T00:29:49.117151+0000","last_unstale":"2024-09-21T00:29:51.164779+0000","last_undegraded":"2024-09-21T00:29:51.164779+0000","last_fullsized":"2024-09-21T00:29:51.164779+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:08:12.844670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"107'2","reported_seq":17,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:51.164122+0000","last_change":"2024-09-21T00:29:48.103570+0000","last_active":"2024-09-21T00:29:51.164122+0000","last_peered":"2024-09-21T00:29:51.164122+0000","last_clean":"2024-09-21T00:29:51.164122+0000","last_became_active":"2024-09-21T00:29:48.103262+0000","last_became_peered":"2024-09-21T00:29:48.103262+0000","last_unstale":"2024-09-21T00:29:51.164122+0000","last_undegraded":"2024-09-21T00:29:51.164122+0000","last_fullsized":"2024-09-21T00:29:51.164122+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:11:49.988457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:51.163510+0000","last_change":"2024-09-21T00:29:49.115475+0000","last_active":"2024-09-21T00:29:51.163510+0000","last_peered":"2024-09-21T00:29:51.163510+0000","last_clean":"2024-09-21T00:29:51.163510+0000","last_became_active":"2024-09-21T00:29:49.115168+0000","last_became_peered":"2024-09-21T00:29:49.115168+0000","last_unstale":"2024-09-21T00:29:51.163510+0000","last_undegraded":"2024-09-21T00:29:51.163510+0000","last_fullsized":"2024-09-21T00:29:51.163510+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:55:39.386900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"107'1","reported_seq":16,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:50.094061+0000","last_change":"2024-09-21T00:29:48.114417+0000","last_active":"2024-09-21T00:29:50.094061+0000","last_peered":"2024-09-21T00:29:50.094061+0000","last_clean":"2024-09-21T00:29:50.094061+0000","last_became_active":"2024-09-21T00:29:48.114114+0000","last_became_peered":"2024-09-21T00:29:48.114114+0000","last_unstale":"2024-09-21T00:29:50.094061+0000","last_undegraded":"2024-09-21T00:29:50.094061+0000","last_fullsized":"2024-09-21T00:29:50.094061+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:30:30.206828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:50.094568+0000","last_change":"2024-09-21T00:29:49.123289+0000","last_active":"2024-09-21T00:29:50.094568+0000","last_peered":"2024-09-21T00:29:50.094568+0000","last_clean":"2024-09-21T00:29:50.094568+0000","last_became_active":"2024-09-21T00:29:49.122910+0000","last_became_peered":"2024-09-21T00:29:49.122910+0000","last_unstale":"2024-09-21T00:29:50.094568+0000","last_undegraded":"2024-09-21T00:29:50.094568+0000","last_fullsized":"2024-09-21T00:29:50.094568+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:35:02.314410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"107'1","reported_seq":16,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:51.101164+0000","last_change":"2024-09-21T00:29:48.109506+0000","last_active":"2024-09-21T00:29:51.101164+0000","last_peered":"2024-09-21T00:29:51.101164+0000","last_clean":"2024-09-21T00:29:51.101164+0000","last_became_active":"2024-09-21T00:29:48.109216+0000","last_became_peered":"2024-09-21T00:29:48.109216+0000","last_unstale":"2024-09-21T00:29:51.101164+0000","last_undegraded":"2024-09-21T00:29:51.101164+0000","last_fullsized":"2024-09-21T00:29:51.101164+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:28:37.053291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:51.165058+0000","last_change":"2024-09-21T00:29:49.123189+0000","last_active":"2024-09-21T00:29:51.165058+0000","last_peered":"2024-09-21T00:29:51.165058+0000","last_clean":"2024-09-21T00:29:51.165058+0000","last_became_active":"2024-09-21T00:29:49.122005+0000","last_became_peered":"2024-09-21T00:29:49.122005+0000","last_unstale":"2024-09-21T00:29:51.165058+0000","last_undegraded":"2024-09-21T00:29:51.165058+0000","last_fullsized":"2024-09-21T00:29:51.165058+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:09:03.700916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":15,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:50.095251+0000","last_change":"2024-09-21T00:29:48.115406+0000","last_active":"2024-09-21T00:29:50.095251+0000","last_peered":"2024-09-21T00:29:50.095251+0000","last_clean":"2024-09-21T00:29:50.095251+0000","last_became_active":"2024-09-21T00:29:48.114774+0000","last_became_peered":"2024-09-21T00:29:48.114774+0000","last_unstale":"2024-09-21T00:29:50.095251+0000","last_undegraded":"2024-09-21T00:29:50.095251+0000","last_fullsized":"2024-09-21T00:29:50.095251+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:33:51.336627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:51.164615+0000","last_change":"2024-09-21T00:29:49.124019+0000","last_active":"2024-09-21T00:29:51.164615+0000","last_peered":"2024-09-21T00:29:51.164615+0000","last_clean":"2024-09-21T00:29:51.164615+0000","last_became_active":"2024-09-21T00:29:49.123744+0000","last_became_peered":"2024-09-21T00:29:49.123744+0000","last_unstale":"2024-09-21T00:29:51.164615+0000","last_undegraded":"2024-09-21T00:29:51.164615+0000","last_fullsized":"2024-09-21T00:29:51.164615+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:39:26.009491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":15,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:51.101410+0000","last_change":"2024-09-21T00:29:48.109429+0000","last_active":"2024-09-21T00:29:51.101410+0000","last_peered":"2024-09-21T00:29:51.101410+0000","last_clean":"2024-09-21T00:29:51.101410+0000","last_became_active":"2024-09-21T00:29:48.109135+0000","last_became_peered":"2024-09-21T00:29:48.109135+0000","last_unstale":"2024-09-21T00:29:51.101410+0000","last_undegraded":"2024-09-21T00:29:51.101410+0000","last_fullsized":"2024-09-21T00:29:51.101410+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:52:45.185741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:51.098909+0000","last_change":"2024-09-21T00:29:49.124510+0000","last_active":"2024-09-21T00:29:51.098909+0000","last_peered":"2024-09-21T00:29:51.098909+0000","last_clean":"2024-09-21T00:29:51.098909+0000","last_became_active":"2024-09-21T00:29:49.124143+0000","last_became_peered":"2024-09-21T00:29:49.124143+0000","last_unstale":"2024-09-21T00:29:51.098909+0000","last_undegraded":"2024-09-21T00:29:51.098909+0000","last_fullsized":"2024-09-21T00:29:51.098909+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:52:27.753303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":15,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:51.098867+0000","last_change":"2024-09-21T00:29:48.110534+0000","last_active":"2024-09-21T00:29:51.098867+0000","last_peered":"2024-09-21T00:29:51.098867+0000","last_clean":"2024-09-21T00:29:51.098867+0000","last_became_active":"2024-09-21T00:29:48.110188+0000","last_became_peered":"2024-09-21T00:29:48.110188+0000","last_unstale":"2024-09-21T00:29:51.098867+0000","last_undegraded":"2024-09-21T00:29:51.098867+0000","last_fullsized":"2024-09-21T00:29:51.098867+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:57:08.835632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:50.095510+0000","last_change":"2024-09-21T00:29:49.110941+0000","last_active":"2024-09-21T00:29:50.095510+0000","last_peered":"2024-09-21T00:29:50.095510+0000","last_clean":"2024-09-21T00:29:50.095510+0000","last_became_active":"2024-09-21T00:29:49.110626+0000","last_became_peered":"2024-09-21T00:29:49.110626+0000","last_unstale":"2024-09-21T00:29:50.095510+0000","last_undegraded":"2024-09-21T00:29:50.095510+0000","last_fullsized":"2024-09-21T00:29:50.095510+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:17:42.952108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":15,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:50.095610+0000","last_change":"2024-09-21T00:29:48.100585+0000","last_active":"2024-09-21T00:29:50.095610+0000","last_peered":"2024-09-21T00:29:50.095610+0000","last_clean":"2024-09-21T00:29:50.095610+0000","last_became_active":"2024-09-21T00:29:48.100260+0000","last_became_peered":"2024-09-21T00:29:48.100260+0000","last_unstale":"2024-09-21T00:29:50.095610+0000","last_undegraded":"2024-09-21T00:29:50.095610+0000","last_fullsized":"2024-09-21T00:29:50.095610+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:35:04.548036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:51.164870+0000","last_change":"2024-09-21T00:29:49.121983+0000","last_active":"2024-09-21T00:29:51.164870+0000","last_peered":"2024-09-21T00:29:51.164870+0000","last_clean":"2024-09-21T00:29:51.164870+0000","last_became_active":"2024-09-21T00:29:49.121653+0000","last_became_peered":"2024-09-21T00:29:49.121653+0000","last_unstale":"2024-09-21T00:29:51.164870+0000","last_undegraded":"2024-09-21T00:29:51.164870+0000","last_fullsized":"2024-09-21T00:29:51.164870+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:30:21.800274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.15","version":"107'3","reported_seq":18,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:50.095482+0000","last_change":"2024-09-21T00:29:48.107282+0000","last_active":"2024-09-21T00:29:50.095482+0000","last_peered":"2024-09-21T00:29:50.095482+0000","last_clean":"2024-09-21T00:29:50.095482+0000","last_became_active":"2024-09-21T00:29:48.106624+0000","last_became_peered":"2024-09-21T00:29:48.106624+0000","last_unstale":"2024-09-21T00:29:50.095482+0000","last_undegraded":"2024-09-21T00:29:50.095482+0000","last_fullsized":"2024-09-21T00:29:50.095482+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:54:25.882509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:50.095213+0000","last_change":"2024-09-21T00:29:49.105627+0000","last_active":"2024-09-21T00:29:50.095213+0000","last_peered":"2024-09-21T00:29:50.095213+0000","last_clean":"2024-09-21T00:29:50.095213+0000","last_became_active":"2024-09-21T00:29:49.105260+0000","last_became_peered":"2024-09-21T00:29:49.105260+0000","last_unstale":"2024-09-21T00:29:50.095213+0000","last_undegraded":"2024-09-21T00:29:50.095213+0000","last_fullsized":"2024-09-21T00:29:50.095213+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:47:44.730787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.14","version":"107'6","reported_seq":21,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:51.164236+0000","last_change":"2024-09-21T00:29:48.114730+0000","last_active":"2024-09-21T00:29:51.164236+0000","last_peered":"2024-09-21T00:29:51.164236+0000","last_clean":"2024-09-21T00:29:51.164236+0000","last_became_active":"2024-09-21T00:29:48.114389+0000","last_became_peered":"2024-09-21T00:29:48.114389+0000","last_unstale":"2024-09-21T00:29:51.164236+0000","last_undegraded":"2024-09-21T00:29:51.164236+0000","last_fullsized":"2024-09-21T00:29:51.164236+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:53:15.982238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:51.099041+0000","last_change":"2024-09-21T00:29:49.109745+0000","last_active":"2024-09-21T00:29:51.099041+0000","last_peered":"2024-09-21T00:29:51.099041+0000","last_clean":"2024-09-21T00:29:51.099041+0000","last_became_active":"2024-09-21T00:29:49.109453+0000","last_became_peered":"2024-09-21T00:29:49.109453+0000","last_unstale":"2024-09-21T00:29:51.099041+0000","last_undegraded":"2024-09-21T00:29:51.099041+0000","last_fullsized":"2024-09-21T00:29:51.099041+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:55:39.386900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.13","version":"107'2","reported_seq":17,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:51.164082+0000","last_change":"2024-09-21T00:29:48.109904+0000","last_active":"2024-09-21T00:29:51.164082+0000","last_peered":"2024-09-21T00:29:51.164082+0000","last_clean":"2024-09-21T00:29:51.164082+0000","last_became_active":"2024-09-21T00:29:48.109587+0000","last_became_peered":"2024-09-21T00:29:48.109587+0000","last_unstale":"2024-09-21T00:29:51.164082+0000","last_undegraded":"2024-09-21T00:29:51.164082+0000","last_fullsized":"2024-09-21T00:29:51.164082+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:59:26.767103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:51.163645+0000","last_change":"2024-09-21T00:29:49.117095+0000","last_active":"2024-09-21T00:29:51.163645+0000","last_peered":"2024-09-21T00:29:51.163645+0000","last_clean":"2024-09-21T00:29:51.163645+0000","last_became_active":"2024-09-21T00:29:49.116809+0000","last_became_peered":"2024-09-21T00:29:49.116809+0000","last_unstale":"2024-09-21T00:29:51.163645+0000","last_undegraded":"2024-09-21T00:29:51.163645+0000","last_fullsized":"2024-09-21T00:29:51.163645+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:13:34.212477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":15,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:51.165135+0000","last_change":"2024-09-21T00:29:48.106128+0000","last_active":"2024-09-21T00:29:51.165135+0000","last_peered":"2024-09-21T00:29:51.165135+0000","last_clean":"2024-09-21T00:29:51.165135+0000","last_became_active":"2024-09-21T00:29:48.105831+0000","last_became_peered":"2024-09-21T00:29:48.105831+0000","last_unstale":"2024-09-21T00:29:51.165135+0000","last_undegraded":"2024-09-21T00:29:51.165135+0000","last_fullsized":"2024-09-21T00:29:51.165135+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:18:12.097196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:50.095064+0000","last_change":"2024-09-21T00:29:49.114501+0000","last_active":"2024-09-21T00:29:50.095064+0000","last_peered":"2024-09-21T00:29:50.095064+0000","last_clean":"2024-09-21T00:29:50.095064+0000","last_became_active":"2024-09-21T00:29:49.114159+0000","last_became_peered":"2024-09-21T00:29:49.114159+0000","last_unstale":"2024-09-21T00:29:50.095064+0000","last_undegraded":"2024-09-21T00:29:50.095064+0000","last_fullsized":"2024-09-21T00:29:50.095064+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:52:25.673726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":15,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:51.164102+0000","last_change":"2024-09-21T00:29:48.112258+0000","last_active":"2024-09-21T00:29:51.164102+0000","last_peered":"2024-09-21T00:29:51.164102+0000","last_clean":"2024-09-21T00:29:51.164102+0000","last_became_active":"2024-09-21T00:29:48.111976+0000","last_became_peered":"2024-09-21T00:29:48.111976+0000","last_unstale":"2024-09-21T00:29:51.164102+0000","last_undegraded":"2024-09-21T00:29:51.164102+0000","last_fullsized":"2024-09-21T00:29:51.164102+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:35:35.720354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:50.094527+0000","last_change":"2024-09-21T00:29:49.111503+0000","last_active":"2024-09-21T00:29:50.094527+0000","last_peered":"2024-09-21T00:29:50.094527+0000","last_clean":"2024-09-21T00:29:50.094527+0000","last_became_active":"2024-09-21T00:29:49.111197+0000","last_became_peered":"2024-09-21T00:29:49.111197+0000","last_unstale":"2024-09-21T00:29:50.094527+0000","last_undegraded":"2024-09-21T00:29:50.094527+0000","last_fullsized":"2024-09-21T00:29:50.094527+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:51:04.982608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"107'1","reported_seq":16,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:51.164489+0000","last_change":"2024-09-21T00:29:48.104276+0000","last_active":"2024-09-21T00:29:51.164489+0000","last_peered":"2024-09-21T00:29:51.164489+0000","last_clean":"2024-09-21T00:29:51.164489+0000","last_became_active":"2024-09-21T00:29:48.103972+0000","last_became_peered":"2024-09-21T00:29:48.103972+0000","last_unstale":"2024-09-21T00:29:51.164489+0000","last_undegraded":"2024-09-21T00:29:51.164489+0000","last_fullsized":"2024-09-21T00:29:51.164489+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:55:01.307862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:50.094637+0000","last_change":"2024-09-21T00:29:49.110046+0000","last_active":"2024-09-21T00:29:50.094637+0000","last_peered":"2024-09-21T00:29:50.094637+0000","last_clean":"2024-09-21T00:29:50.094637+0000","last_became_active":"2024-09-21T00:29:49.109727+0000","last_became_peered":"2024-09-21T00:29:49.109727+0000","last_unstale":"2024-09-21T00:29:50.094637+0000","last_undegraded":"2024-09-21T00:29:50.094637+0000","last_fullsized":"2024-09-21T00:29:50.094637+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:27:53.080062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.e","version":"107'2","reported_seq":17,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:50.095384+0000","last_change":"2024-09-21T00:29:48.103336+0000","last_active":"2024-09-21T00:29:50.095384+0000","last_peered":"2024-09-21T00:29:50.095384+0000","last_clean":"2024-09-21T00:29:50.095384+0000","last_became_active":"2024-09-21T00:29:48.103043+0000","last_became_peered":"2024-09-21T00:29:48.103043+0000","last_unstale":"2024-09-21T00:29:50.095384+0000","last_undegraded":"2024-09-21T00:29:50.095384+0000","last_fullsized":"2024-09-21T00:29:50.095384+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:59:23.042894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:51.101521+0000","last_change":"2024-09-21T00:29:49.108911+0000","last_active":"2024-09-21T00:29:51.101521+0000","last_peered":"2024-09-21T00:29:51.101521+0000","last_clean":"2024-09-21T00:29:51.101521+0000","last_became_active":"2024-09-21T00:29:49.108470+0000","last_became_peered":"2024-09-21T00:29:49.108470+0000","last_unstale":"2024-09-21T00:29:51.101521+0000","last_undegraded":"2024-09-21T00:29:51.101521+0000","last_fullsized":"2024-09-21T00:29:51.101521+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:56:55.321362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"107'1","reported_seq":16,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:51.164501+0000","last_change":"2024-09-21T00:29:48.108936+0000","last_active":"2024-09-21T00:29:51.164501+0000","last_peered":"2024-09-21T00:29:51.164501+0000","last_clean":"2024-09-21T00:29:51.164501+0000","last_became_active":"2024-09-21T00:29:48.108614+0000","last_became_peered":"2024-09-21T00:29:48.108614+0000","last_unstale":"2024-09-21T00:29:51.164501+0000","last_undegraded":"2024-09-21T00:29:51.164501+0000","last_fullsized":"2024-09-21T00:29:51.164501+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:18:12.097196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:50.095444+0000","last_change":"2024-09-21T00:29:49.117960+0000","last_active":"2024-09-21T00:29:50.095444+0000","last_peered":"2024-09-21T00:29:50.095444+0000","last_clean":"2024-09-21T00:29:50.095444+0000","last_became_active":"2024-09-21T00:29:49.117642+0000","last_became_peered":"2024-09-21T00:29:49.117642+0000","last_unstale":"2024-09-21T00:29:50.095444+0000","last_undegraded":"2024-09-21T00:29:50.095444+0000","last_fullsized":"2024-09-21T00:29:50.095444+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:11:38.972017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":15,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:51.164239+0000","last_change":"2024-09-21T00:29:48.111869+0000","last_active":"2024-09-21T00:29:51.164239+0000","last_peered":"2024-09-21T00:29:51.164239+0000","last_clean":"2024-09-21T00:29:51.164239+0000","last_became_active":"2024-09-21T00:29:48.111497+0000","last_became_peered":"2024-09-21T00:29:48.111497+0000","last_unstale":"2024-09-21T00:29:51.164239+0000","last_undegraded":"2024-09-21T00:29:51.164239+0000","last_fullsized":"2024-09-21T00:29:51.164239+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:27:52.077806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:51.101233+0000","last_change":"2024-09-21T00:29:49.116325+0000","last_active":"2024-09-21T00:29:51.101233+0000","last_peered":"2024-09-21T00:29:51.101233+0000","last_clean":"2024-09-21T00:29:51.101233+0000","last_became_active":"2024-09-21T00:29:49.115969+0000","last_became_peered":"2024-09-21T00:29:49.115969+0000","last_unstale":"2024-09-21T00:29:51.101233+0000","last_undegraded":"2024-09-21T00:29:51.101233+0000","last_fullsized":"2024-09-21T00:29:51.101233+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:02:14.985306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.b","version":"107'2","reported_seq":17,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:51.101065+0000","last_change":"2024-09-21T00:29:48.108744+0000","last_active":"2024-09-21T00:29:51.101065+0000","last_peered":"2024-09-21T00:29:51.101065+0000","last_clean":"2024-09-21T00:29:51.101065+0000","last_became_active":"2024-09-21T00:29:48.108377+0000","last_became_peered":"2024-09-21T00:29:48.108377+0000","last_unstale":"2024-09-21T00:29:51.101065+0000","last_undegraded":"2024-09-21T00:29:51.101065+0000","last_fullsized":"2024-09-21T00:29:51.101065+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:08:14.403928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:51.164206+0000","last_change":"2024-09-21T00:29:49.112211+0000","last_active":"2024-09-21T00:29:51.164206+0000","last_peered":"2024-09-21T00:29:51.164206+0000","last_clean":"2024-09-21T00:29:51.164206+0000","last_became_active":"2024-09-21T00:29:49.111893+0000","last_became_peered":"2024-09-21T00:29:49.111893+0000","last_unstale":"2024-09-21T00:29:51.164206+0000","last_undegraded":"2024-09-21T00:29:51.164206+0000","last_fullsized":"2024-09-21T00:29:51.164206+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:21:26.639292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":15,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:51.164017+0000","last_change":"2024-09-21T00:29:48.114856+0000","last_active":"2024-09-21T00:29:51.164017+0000","last_peered":"2024-09-21T00:29:51.164017+0000","last_clean":"2024-09-21T00:29:51.164017+0000","last_became_active":"2024-09-21T00:29:48.114394+0000","last_became_peered":"2024-09-21T00:29:48.114394+0000","last_unstale":"2024-09-21T00:29:51.164017+0000","last_undegraded":"2024-09-21T00:29:51.164017+0000","last_fullsized":"2024-09-21T00:29:51.164017+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:35:35.720354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:51.165086+0000","last_change":"2024-09-21T00:29:49.122315+0000","last_active":"2024-09-21T00:29:51.165086+0000","last_peered":"2024-09-21T00:29:51.165086+0000","last_clean":"2024-09-21T00:29:51.165086+0000","last_became_active":"2024-09-21T00:29:49.121467+0000","last_became_peered":"2024-09-21T00:29:49.121467+0000","last_unstale":"2024-09-21T00:29:51.165086+0000","last_undegraded":"2024-09-21T00:29:51.165086+0000","last_fullsized":"2024-09-21T00:29:51.165086+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:05:41.315339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":15,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:51.164325+0000","last_change":"2024-09-21T00:29:48.106397+0000","last_active":"2024-09-21T00:29:51.164325+0000","last_peered":"2024-09-21T00:29:51.164325+0000","last_clean":"2024-09-21T00:29:51.164325+0000","last_became_active":"2024-09-21T00:29:48.105468+0000","last_became_peered":"2024-09-21T00:29:48.105468+0000","last_unstale":"2024-09-21T00:29:51.164325+0000","last_undegraded":"2024-09-21T00:29:51.164325+0000","last_fullsized":"2024-09-21T00:29:51.164325+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:27:52.077806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:50.094870+0000","last_change":"2024-09-21T00:29:49.108718+0000","last_active":"2024-09-21T00:29:50.094870+0000","last_peered":"2024-09-21T00:29:50.094870+0000","last_clean":"2024-09-21T00:29:50.094870+0000","last_became_active":"2024-09-21T00:29:49.108401+0000","last_became_peered":"2024-09-21T00:29:49.108401+0000","last_unstale":"2024-09-21T00:29:50.094870+0000","last_undegraded":"2024-09-21T00:29:50.094870+0000","last_fullsized":"2024-09-21T00:29:50.094870+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:37:08.281841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":15,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:51.163904+0000","last_change":"2024-09-21T00:29:48.105932+0000","last_active":"2024-09-21T00:29:51.163904+0000","last_peered":"2024-09-21T00:29:51.163904+0000","last_clean":"2024-09-21T00:29:51.163904+0000","last_became_active":"2024-09-21T00:29:48.105570+0000","last_became_peered":"2024-09-21T00:29:48.105570+0000","last_unstale":"2024-09-21T00:29:51.163904+0000","last_undegraded":"2024-09-21T00:29:51.163904+0000","last_fullsized":"2024-09-21T00:29:51.163904+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:36:09.108085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:51.101609+0000","last_change":"2024-09-21T00:29:49.106013+0000","last_active":"2024-09-21T00:29:51.101609+0000","last_peered":"2024-09-21T00:29:51.101609+0000","last_clean":"2024-09-21T00:29:51.101609+0000","last_became_active":"2024-09-21T00:29:49.105459+0000","last_became_peered":"2024-09-21T00:29:49.105459+0000","last_unstale":"2024-09-21T00:29:51.101609+0000","last_undegraded":"2024-09-21T00:29:51.101609+0000","last_fullsized":"2024-09-21T00:29:51.101609+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:19:16.290142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.7","version":"107'2","reported_seq":17,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:51.101488+0000","last_change":"2024-09-21T00:29:48.114760+0000","last_active":"2024-09-21T00:29:51.101488+0000","last_peered":"2024-09-21T00:29:51.101488+0000","last_clean":"2024-09-21T00:29:51.101488+0000","last_became_active":"2024-09-21T00:29:48.114326+0000","last_became_peered":"2024-09-21T00:29:48.114326+0000","last_unstale":"2024-09-21T00:29:51.101488+0000","last_undegraded":"2024-09-21T00:29:51.101488+0000","last_fullsized":"2024-09-21T00:29:51.101488+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:04:09.518964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:51.163635+0000","last_change":"2024-09-21T00:29:49.114688+0000","last_active":"2024-09-21T00:29:51.163635+0000","last_peered":"2024-09-21T00:29:51.163635+0000","last_clean":"2024-09-21T00:29:51.163635+0000","last_became_active":"2024-09-21T00:29:49.114431+0000","last_became_peered":"2024-09-21T00:29:49.114431+0000","last_unstale":"2024-09-21T00:29:51.163635+0000","last_undegraded":"2024-09-21T00:29:51.163635+0000","last_fullsized":"2024-09-21T00:29:51.163635+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:57:44.978896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.6","version":"108'1","reported_seq":16,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:51.100577+0000","last_change":"2024-09-21T00:29:48.111642+0000","last_active":"2024-09-21T00:29:51.100577+0000","last_peered":"2024-09-21T00:29:51.100577+0000","last_clean":"2024-09-21T00:29:51.100577+0000","last_became_active":"2024-09-21T00:29:48.111306+0000","last_became_peered":"2024-09-21T00:29:48.111306+0000","last_unstale":"2024-09-21T00:29:51.100577+0000","last_undegraded":"2024-09-21T00:29:51.100577+0000","last_fullsized":"2024-09-21T00:29:51.100577+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:22:52.316356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:51.164692+0000","last_change":"2024-09-21T00:29:49.128963+0000","last_active":"2024-09-21T00:29:51.164692+0000","last_peered":"2024-09-21T00:29:51.164692+0000","last_clean":"2024-09-21T00:29:51.164692+0000","last_became_active":"2024-09-21T00:29:49.128634+0000","last_became_peered":"2024-09-21T00:29:49.128634+0000","last_unstale":"2024-09-21T00:29:51.164692+0000","last_undegraded":"2024-09-21T00:29:51.164692+0000","last_fullsized":"2024-09-21T00:29:51.164692+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:47:19.603347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":15,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:51.099046+0000","last_change":"2024-09-21T00:29:48.112285+0000","last_active":"2024-09-21T00:29:51.099046+0000","last_peered":"2024-09-21T00:29:51.099046+0000","last_clean":"2024-09-21T00:29:51.099046+0000","last_became_active":"2024-09-21T00:29:48.111945+0000","last_became_peered":"2024-09-21T00:29:48.111945+0000","last_unstale":"2024-09-21T00:29:51.099046+0000","last_undegraded":"2024-09-21T00:29:51.099046+0000","last_fullsized":"2024-09-21T00:29:51.099046+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:24:40.002666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:50.095275+0000","last_change":"2024-09-21T00:29:49.111611+0000","last_active":"2024-09-21T00:29:50.095275+0000","last_peered":"2024-09-21T00:29:50.095275+0000","last_clean":"2024-09-21T00:29:50.095275+0000","last_became_active":"2024-09-21T00:29:49.110763+0000","last_became_peered":"2024-09-21T00:29:49.110763+0000","last_unstale":"2024-09-21T00:29:50.095275+0000","last_undegraded":"2024-09-21T00:29:50.095275+0000","last_fullsized":"2024-09-21T00:29:50.095275+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:33:10.243092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":15,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:51.101638+0000","last_change":"2024-09-21T00:29:48.110571+0000","last_active":"2024-09-21T00:29:51.101638+0000","last_peered":"2024-09-21T00:29:51.101638+0000","last_clean":"2024-09-21T00:29:51.101638+0000","last_became_active":"2024-09-21T00:29:48.110252+0000","last_became_peered":"2024-09-21T00:29:48.110252+0000","last_unstale":"2024-09-21T00:29:51.101638+0000","last_undegraded":"2024-09-21T00:29:51.101638+0000","last_fullsized":"2024-09-21T00:29:51.101638+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:04:20.619113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:51.101237+0000","last_change":"2024-09-21T00:29:49.103676+0000","last_active":"2024-09-21T00:29:51.101237+0000","last_peered":"2024-09-21T00:29:51.101237+0000","last_clean":"2024-09-21T00:29:51.101237+0000","last_became_active":"2024-09-21T00:29:49.103338+0000","last_became_peered":"2024-09-21T00:29:49.103338+0000","last_unstale":"2024-09-21T00:29:51.101237+0000","last_undegraded":"2024-09-21T00:29:51.101237+0000","last_fullsized":"2024-09-21T00:29:51.101237+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:39:22.826869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":15,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:50.094972+0000","last_change":"2024-09-21T00:29:48.111343+0000","last_active":"2024-09-21T00:29:50.094972+0000","last_peered":"2024-09-21T00:29:50.094972+0000","last_clean":"2024-09-21T00:29:50.094972+0000","last_became_active":"2024-09-21T00:29:48.111032+0000","last_became_peered":"2024-09-21T00:29:48.111032+0000","last_unstale":"2024-09-21T00:29:50.094972+0000","last_undegraded":"2024-09-21T00:29:50.094972+0000","last_fullsized":"2024-09-21T00:29:50.094972+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:39:13.998725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:51.163972+0000","last_change":"2024-09-21T00:29:49.121015+0000","last_active":"2024-09-21T00:29:51.163972+0000","last_peered":"2024-09-21T00:29:51.163972+0000","last_clean":"2024-09-21T00:29:51.163972+0000","last_became_active":"2024-09-21T00:29:49.120710+0000","last_became_peered":"2024-09-21T00:29:49.120710+0000","last_unstale":"2024-09-21T00:29:51.163972+0000","last_undegraded":"2024-09-21T00:29:51.163972+0000","last_fullsized":"2024-09-21T00:29:51.163972+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:40:47.831055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":15,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:50.094242+0000","last_change":"2024-09-21T00:29:48.111988+0000","last_active":"2024-09-21T00:29:50.094242+0000","last_peered":"2024-09-21T00:29:50.094242+0000","last_clean":"2024-09-21T00:29:50.094242+0000","last_became_active":"2024-09-21T00:29:48.110924+0000","last_became_peered":"2024-09-21T00:29:48.110924+0000","last_unstale":"2024-09-21T00:29:50.094242+0000","last_undegraded":"2024-09-21T00:29:50.094242+0000","last_fullsized":"2024-09-21T00:29:50.094242+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:02:38.960612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:51.164214+0000","last_change":"2024-09-21T00:29:49.115962+0000","last_active":"2024-09-21T00:29:51.164214+0000","last_peered":"2024-09-21T00:29:51.164214+0000","last_clean":"2024-09-21T00:29:51.164214+0000","last_became_active":"2024-09-21T00:29:49.115602+0000","last_became_peered":"2024-09-21T00:29:49.115602+0000","last_unstale":"2024-09-21T00:29:51.164214+0000","last_undegraded":"2024-09-21T00:29:51.164214+0000","last_fullsized":"2024-09-21T00:29:51.164214+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:11:45.373594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":15,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:51.101044+0000","last_change":"2024-09-21T00:29:48.110619+0000","last_active":"2024-09-21T00:29:51.101044+0000","last_peered":"2024-09-21T00:29:51.101044+0000","last_clean":"2024-09-21T00:29:51.101044+0000","last_became_active":"2024-09-21T00:29:48.110292+0000","last_became_peered":"2024-09-21T00:29:48.110292+0000","last_unstale":"2024-09-21T00:29:51.101044+0000","last_undegraded":"2024-09-21T00:29:51.101044+0000","last_fullsized":"2024-09-21T00:29:51.101044+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:36:50.249803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:51.164833+0000","last_change":"2024-09-21T00:29:49.120355+0000","last_active":"2024-09-21T00:29:51.164833+0000","last_peered":"2024-09-21T00:29:51.164833+0000","last_clean":"2024-09-21T00:29:51.164833+0000","last_became_active":"2024-09-21T00:29:49.119687+0000","last_became_peered":"2024-09-21T00:29:49.119687+0000","last_unstale":"2024-09-21T00:29:51.164833+0000","last_undegraded":"2024-09-21T00:29:51.164833+0000","last_fullsized":"2024-09-21T00:29:51.164833+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:11:07.377438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":15,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:51.163893+0000","last_change":"2024-09-21T00:29:48.107972+0000","last_active":"2024-09-21T00:29:51.163893+0000","last_peered":"2024-09-21T00:29:51.163893+0000","last_clean":"2024-09-21T00:29:51.163893+0000","last_became_active":"2024-09-21T00:29:48.107553+0000","last_became_peered":"2024-09-21T00:29:48.107553+0000","last_unstale":"2024-09-21T00:29:51.163893+0000","last_undegraded":"2024-09-21T00:29:51.163893+0000","last_fullsized":"2024-09-21T00:29:51.163893+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:53:15.982238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:51.164466+0000","last_change":"2024-09-21T00:29:49.118874+0000","last_active":"2024-09-21T00:29:51.164466+0000","last_peered":"2024-09-21T00:29:51.164466+0000","last_clean":"2024-09-21T00:29:51.164466+0000","last_became_active":"2024-09-21T00:29:49.118310+0000","last_became_peered":"2024-09-21T00:29:49.118310+0000","last_unstale":"2024-09-21T00:29:51.164466+0000","last_undegraded":"2024-09-21T00:29:51.164466+0000","last_fullsized":"2024-09-21T00:29:51.164466+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:48.079691+0000","last_clean_scrub_stamp":"2024-09-21T00:29:48.079691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:58:30.921993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.10","version":"107'8","reported_seq":23,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:51.164071+0000","last_change":"2024-09-21T00:29:48.107294+0000","last_active":"2024-09-21T00:29:51.164071+0000","last_peered":"2024-09-21T00:29:51.164071+0000","last_clean":"2024-09-21T00:29:51.164071+0000","last_became_active":"2024-09-21T00:29:48.107003+0000","last_became_peered":"2024-09-21T00:29:48.107003+0000","last_unstale":"2024-09-21T00:29:51.164071+0000","last_undegraded":"2024-09-21T00:29:51.164071+0000","last_fullsized":"2024-09-21T00:29:51.164071+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:29:47.077435+0000","last_clean_scrub_stamp":"2024-09-21T00:29:47.077435+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:36:09.108085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":271,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T00:29:51.164055+0000","last_change":"2024-09-21T00:25:38.725792+0000","last_active":"2024-09-21T00:29:51.164055+0000","last_peered":"2024-09-21T00:29:51.164055+0000","last_clean":"2024-09-21T00:29:51.164055+0000","last_became_active":"2024-09-21T00:25:38.724668+0000","last_became_peered":"2024-09-21T00:25:38.724668+0000","last_unstale":"2024-09-21T00:29:51.164055+0000","last_undegraded":"2024-09-21T00:29:51.164055+0000","last_fullsized":"2024-09-21T00:29:51.164055+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:37.163216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:37.163216+0000","last_clean_scrub_stamp":"2024-09-21T00:25:37.163216+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:14:36.588660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509494,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33092,"kb_used_data":1904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338748,"statfs":{"total":96636764160,"available":96602877952,"internally_reserved":0,"allocated":1949696,"data_stored":1203121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509495,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32500,"kb_used_data":1312,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339340,"statfs":{"total":96636764160,"available":96603484160,"internally_reserved":0,"allocated":1343488,"data_stored":612559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509494,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32504,"kb_used_data":1316,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339336,"statfs":{"total":96636764160,"available":96603480064,"internally_reserved":0,"allocated":1347584,"data_stored":612137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509493,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32508,"kb_used_data":1320,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339332,"statfs":{"total":96636764160,"available":96603475968,"internally_reserved":0,"allocated":1351680,"data_stored":613589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509495,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32508,"kb_used_data":1312,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339332,"statfs":{"total":96636764160,"available":96603475968,"internally_reserved":0,"allocated":1343488,"data_stored":606931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509495,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32504,"kb_used_data":1308,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339336,"statfs":{"total":96636764160,"available":96603480064,"internally_reserved":0,"allocated":1339392,"data_stored":611977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509495,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33076,"kb_used_data":1880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338764,"statfs":{"total":96636764160,"available":96602894336,"internally_reserved":0,"allocated":1925120,"data_stored":1195675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574903,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32504,"kb_used_data":1312,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339336,"statfs":{"total":96636764160,"available":96603480064,"internally_reserved":0,"allocated":1343488,"data_stored":611999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T00:29:56.909 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T00:29:56.909 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-21T00:29:57.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:57.068+0000 7f984e913640 1 -- 172.21.15.112:0/1477721280 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f984815c5f0 msgr2=0x7f9848155900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:57.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:57.068+0000 7f984e913640 1 --2- 172.21.15.112:0/1477721280 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f984815c5f0 0x7f9848155900 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f98480cefe0 tx=0x7f983802a420 comp rx=0 tx=0).stop 2024-09-21T00:29:57.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:57.068+0000 7f984e913640 1 -- 172.21.15.112:0/1477721280 shutdown_connections 2024-09-21T00:29:57.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:57.068+0000 7f984e913640 1 --2- 172.21.15.112:0/1477721280 >> v2:172.21.15.187:3300/0 conn(0x7f98480ccbb0 0x7f98480ccfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:57.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:57.068+0000 7f984e913640 1 --2- 172.21.15.112:0/1477721280 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f984815c5f0 0x7f9848155900 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:57.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:57.068+0000 7f984e913640 1 -- 172.21.15.112:0/1477721280 >> 172.21.15.112:0/1477721280 conn(0x7f9848073fe0 msgr2=0x7f98480743f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:57.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:57.068+0000 7f984e913640 1 -- 172.21.15.112:0/1477721280 shutdown_connections 2024-09-21T00:29:57.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:57.068+0000 7f984e913640 1 -- 172.21.15.112:0/1477721280 wait complete. 2024-09-21T00:29:57.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:57.070+0000 7f984e913640 1 Processor -- start 2024-09-21T00:29:57.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:57.070+0000 7f984e913640 1 -- start start 2024-09-21T00:29:57.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:57.071+0000 7f984e913640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9848070210 0x7f98481f2450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:57.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:57.071+0000 7f984e913640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f98480ccbb0 0x7f98481e5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:57.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:57.071+0000 7f984e913640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f984815c5f0 0x7f98481f6020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:57.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:57.071+0000 7f984e913640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9848071150 con 0x7f9848070210 2024-09-21T00:29:57.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:57.071+0000 7f984e913640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9848070fd0 con 0x7f984815c5f0 2024-09-21T00:29:57.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:57.071+0000 7f984e913640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f9848155e40 con 0x7f98480ccbb0 2024-09-21T00:29:57.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:57.071+0000 7f9847fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9848070210 0x7f98481f2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:57.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:57.071+0000 7f9847fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9848070210 0x7f98481f2450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40716/0 (socket says 172.21.15.112:40716) 2024-09-21T00:29:57.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:57.071+0000 7f9847fff640 1 -- 172.21.15.112:0/182107697 learned_addr learned my addr 172.21.15.112:0/182107697 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:29:57.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:57.071+0000 7f983ffff640 1 --2- 172.21.15.112:0/182107697 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f98480ccbb0 0x7f98481e5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:57.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:57.071+0000 7f984ceb0640 1 --2- 172.21.15.112:0/182107697 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f984815c5f0 0x7f98481f6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:57.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:57.071+0000 7f9847fff640 1 -- 172.21.15.112:0/182107697 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f98480ccbb0 msgr2=0x7f98481e5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:57.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:57.071+0000 7f9847fff640 1 --2- 172.21.15.112:0/182107697 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f98480ccbb0 0x7f98481e5180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:57.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:57.071+0000 7f9847fff640 1 -- 172.21.15.112:0/182107697 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f984815c5f0 msgr2=0x7f98481f6020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:57.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:57.071+0000 7f9847fff640 1 --2- 172.21.15.112:0/182107697 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f984815c5f0 0x7f98481f6020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:57.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:57.071+0000 7f9847fff640 1 -- 172.21.15.112:0/182107697 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98480b5800 con 0x7f9848070210 2024-09-21T00:29:57.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:57.071+0000 7f983ffff640 1 --2- 172.21.15.112:0/182107697 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f98480ccbb0 0x7f98481e5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:29:57.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:57.072+0000 7f9847fff640 1 --2- 172.21.15.112:0/182107697 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9848070210 0x7f98481f2450 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f9838005fd0 tx=0x7f98380039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:57.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:57.072+0000 7f98457fa640 1 -- 172.21.15.112:0/182107697 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f983803e030 con 0x7f9848070210 2024-09-21T00:29:57.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:57.072+0000 7f98457fa640 1 -- 172.21.15.112:0/182107697 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f98380047a0 con 0x7f9848070210 2024-09-21T00:29:57.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:57.072+0000 7f984e913640 1 -- 172.21.15.112:0/182107697 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98480b5ae0 con 0x7f9848070210 2024-09-21T00:29:57.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:57.072+0000 7f98457fa640 1 -- 172.21.15.112:0/182107697 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9838035040 con 0x7f9848070210 2024-09-21T00:29:57.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:57.072+0000 7f984e913640 1 -- 172.21.15.112:0/182107697 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98481f5b00 con 0x7f9848070210 2024-09-21T00:29:57.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:57.073+0000 7f984e913640 1 -- 172.21.15.112:0/182107697 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9848155900 con 0x7f9848070210 2024-09-21T00:29:57.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:57.073+0000 7f98457fa640 1 -- 172.21.15.112:0/182107697 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9838041030 con 0x7f9848070210 2024-09-21T00:29:57.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:57.074+0000 7f98457fa640 1 --2- 172.21.15.112:0/182107697 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f9828078b90 0x7f982807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:57.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:57.074+0000 7f98457fa640 1 -- 172.21.15.112:0/182107697 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 6101+0+0 (secure 0 0 0) 0x7f98380bb5f0 con 0x7f9848070210 2024-09-21T00:29:57.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:57.074+0000 7f983ffff640 1 --2- 172.21.15.112:0/182107697 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f9828078b90 0x7f982807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:57.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:57.075+0000 7f983ffff640 1 --2- 172.21.15.112:0/182107697 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f9828078b90 0x7f982807b0a0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f9830006fd0 tx=0x7f9830008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:57.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:57.079+0000 7f98457fa640 1 -- 172.21.15.112:0/182107697 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f9838083940 con 0x7f9848070210 2024-09-21T00:29:57.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:57.331+0000 7f984e913640 1 -- 172.21.15.112:0/182107697 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f98481e56c0 con 0x7f9848070210 2024-09-21T00:29:57.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:57.334+0000 7f98457fa640 1 -- 172.21.15.112:0/182107697 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v21) ==== 190+0+147 (secure 0 0 0) 0x7f9838083240 con 0x7f9848070210 2024-09-21T00:29:57.336 INFO:teuthology.orchestra.run.smithi112.stdout:[client.0] 2024-09-21T00:29:57.336 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQBPEu5m49NPHBAAl/rcjiE8xYI+XdWwvZ6FOw== 2024-09-21T00:29:57.336 INFO:teuthology.orchestra.run.smithi112.stdout: caps mds = "allow" 2024-09-21T00:29:57.336 INFO:teuthology.orchestra.run.smithi112.stdout: caps mon = "allow r" 2024-09-21T00:29:57.336 INFO:teuthology.orchestra.run.smithi112.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-21T00:29:57.337 INFO:teuthology.orchestra.run.smithi112.stderr:updated caps for client.0 2024-09-21T00:29:57.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:57.336+0000 7f984e913640 1 -- 172.21.15.112:0/182107697 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f9828078b90 msgr2=0x7f982807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:57.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:57.336+0000 7f984e913640 1 --2- 172.21.15.112:0/182107697 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f9828078b90 0x7f982807b0a0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f9830006fd0 tx=0x7f9830008040 comp rx=0 tx=0).stop 2024-09-21T00:29:57.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:57.336+0000 7f984e913640 1 -- 172.21.15.112:0/182107697 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9848070210 msgr2=0x7f98481f2450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:57.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:57.336+0000 7f984e913640 1 --2- 172.21.15.112:0/182107697 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9848070210 0x7f98481f2450 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f9838005fd0 tx=0x7f98380039e0 comp rx=0 tx=0).stop 2024-09-21T00:29:57.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:57.336+0000 7f984e913640 1 -- 172.21.15.112:0/182107697 shutdown_connections 2024-09-21T00:29:57.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:57.336+0000 7f984e913640 1 --2- 172.21.15.112:0/182107697 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f984815c5f0 0x7f98481f6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:57.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:57.336+0000 7f984e913640 1 --2- 172.21.15.112:0/182107697 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f98480ccbb0 0x7f98481e5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:57.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:57.336+0000 7f984e913640 1 --2- 172.21.15.112:0/182107697 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f9828078b90 0x7f982807b0a0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:57.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:57.336+0000 7f984e913640 1 --2- 172.21.15.112:0/182107697 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9848070210 0x7f98481f2450 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:57.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:57.336+0000 7f984e913640 1 -- 172.21.15.112:0/182107697 >> 172.21.15.112:0/182107697 conn(0x7f9848073fe0 msgr2=0x7f98480743f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:57.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:57.337+0000 7f984e913640 1 -- 172.21.15.112:0/182107697 shutdown_connections 2024-09-21T00:29:57.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:57.337+0000 7f984e913640 1 -- 172.21.15.112:0/182107697 wait complete. 2024-09-21T00:29:57.350 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T00:29:57.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:57.460+0000 7f0c71da8640 1 -- 172.21.15.112:0/3180527141 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c6c0d4a90 msgr2=0x7f0c6c0d38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:57.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:57.460+0000 7f0c71da8640 1 --2- 172.21.15.112:0/3180527141 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c6c0d4a90 0x7f0c6c0d38c0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f0c60009830 tx=0x7f0c6002a480 comp rx=0 tx=0).stop 2024-09-21T00:29:57.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:57.460+0000 7f0c71da8640 1 -- 172.21.15.112:0/3180527141 shutdown_connections 2024-09-21T00:29:57.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:57.460+0000 7f0c71da8640 1 --2- 172.21.15.112:0/3180527141 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c6c0d4a90 0x7f0c6c0d38c0 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:57.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:57.460+0000 7f0c71da8640 1 -- 172.21.15.112:0/3180527141 >> 172.21.15.112:0/3180527141 conn(0x7f0c6c09d8a0 msgr2=0x7f0c6c08bfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:57.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:57.460+0000 7f0c71da8640 1 -- 172.21.15.112:0/3180527141 shutdown_connections 2024-09-21T00:29:57.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:57.460+0000 7f0c71da8640 1 -- 172.21.15.112:0/3180527141 wait complete. 2024-09-21T00:29:57.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:57.462+0000 7f0c71da8640 1 Processor -- start 2024-09-21T00:29:57.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:57.462+0000 7f0c71da8640 1 -- start start 2024-09-21T00:29:57.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:57.463+0000 7f0c71da8640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0c6c0d0ae0 0x7f0c6c0c84e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:57.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:57.463+0000 7f0c71da8640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0c6c0d4a90 0x7f0c6c0c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:57.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:57.463+0000 7f0c71da8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c6c15ea40 0x7f0c6c0c8f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:57.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:57.463+0000 7f0c71da8640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0c6c162f50 con 0x7f0c6c15ea40 2024-09-21T00:29:57.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:57.463+0000 7f0c71da8640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0c6c162dd0 con 0x7f0c6c0d0ae0 2024-09-21T00:29:57.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:57.463+0000 7f0c71da8640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f0c6c1630d0 con 0x7f0c6c0d4a90 2024-09-21T00:29:57.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:57.463+0000 7f0c6bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c6c15ea40 0x7f0c6c0c8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:57.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:57.463+0000 7f0c6bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c6c15ea40 0x7f0c6c0c8f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40734/0 (socket says 172.21.15.112:40734) 2024-09-21T00:29:57.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:57.463+0000 7f0c6bfff640 1 -- 172.21.15.112:0/3466065608 learned_addr learned my addr 172.21.15.112:0/3466065608 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:29:57.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:57.463+0000 7f0c6affd640 1 --2- 172.21.15.112:0/3466065608 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0c6c0d4a90 0x7f0c6c0c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:57.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:57.463+0000 7f0c6b7fe640 1 --2- 172.21.15.112:0/3466065608 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0c6c0d0ae0 0x7f0c6c0c84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:57.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:57.463+0000 7f0c6bfff640 1 -- 172.21.15.112:0/3466065608 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0c6c0d4a90 msgr2=0x7f0c6c0c8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:57.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:57.463+0000 7f0c6bfff640 1 --2- 172.21.15.112:0/3466065608 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0c6c0d4a90 0x7f0c6c0c8a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:57.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:57.463+0000 7f0c6bfff640 1 -- 172.21.15.112:0/3466065608 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0c6c0d0ae0 msgr2=0x7f0c6c0c84e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:57.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:57.463+0000 7f0c6bfff640 1 --2- 172.21.15.112:0/3466065608 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0c6c0d0ae0 0x7f0c6c0c84e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:57.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:57.463+0000 7f0c6bfff640 1 -- 172.21.15.112:0/3466065608 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c6c0ce290 con 0x7f0c6c15ea40 2024-09-21T00:29:57.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:57.464+0000 7f0c6bfff640 1 --2- 172.21.15.112:0/3466065608 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c6c15ea40 0x7f0c6c0c8f60 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f0c6400ba10 tx=0x7f0c6400bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:57.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:57.464+0000 7f0c53fff640 1 -- 172.21.15.112:0/3466065608 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c64019070 con 0x7f0c6c15ea40 2024-09-21T00:29:57.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:57.464+0000 7f0c53fff640 1 -- 172.21.15.112:0/3466065608 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0c6400cc30 con 0x7f0c6c15ea40 2024-09-21T00:29:57.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:57.464+0000 7f0c71da8640 1 -- 172.21.15.112:0/3466065608 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c6c06e330 con 0x7f0c6c15ea40 2024-09-21T00:29:57.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:57.464+0000 7f0c53fff640 1 -- 172.21.15.112:0/3466065608 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c640146a0 con 0x7f0c6c15ea40 2024-09-21T00:29:57.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:57.464+0000 7f0c71da8640 1 -- 172.21.15.112:0/3466065608 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c6c0ce570 con 0x7f0c6c15ea40 2024-09-21T00:29:57.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:57.465+0000 7f0c71da8640 1 -- 172.21.15.112:0/3466065608 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c6c0d38c0 con 0x7f0c6c15ea40 2024-09-21T00:29:57.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:57.466+0000 7f0c53fff640 1 -- 172.21.15.112:0/3466065608 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0c6401c070 con 0x7f0c6c15ea40 2024-09-21T00:29:57.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:57.466+0000 7f0c53fff640 1 --2- 172.21.15.112:0/3466065608 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f0c58078b90 0x7f0c5807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:57.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:57.466+0000 7f0c53fff640 1 -- 172.21.15.112:0/3466065608 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 6101+0+0 (secure 0 0 0) 0x7f0c6409aae0 con 0x7f0c6c15ea40 2024-09-21T00:29:57.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:57.467+0000 7f0c6b7fe640 1 --2- 172.21.15.112:0/3466065608 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f0c58078b90 0x7f0c5807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:57.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:57.467+0000 7f0c6b7fe640 1 --2- 172.21.15.112:0/3466065608 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f0c58078b90 0x7f0c5807b0a0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c0cff30 tx=0x7f0c5c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:57.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:57.471+0000 7f0c53fff640 1 -- 172.21.15.112:0/3466065608 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f0c6400ce20 con 0x7f0c6c15ea40 2024-09-21T00:29:57.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:57.685+0000 7f0c71da8640 1 -- 172.21.15.112:0/3466065608 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0c6c0cdfb0 con 0x7f0c6c15ea40 2024-09-21T00:29:57.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:57.686+0000 7f0c53fff640 1 -- 172.21.15.112:0/3466065608 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) ==== 94+0+5237 (secure 0 0 0) 0x7f0c64062d90 con 0x7f0c6c15ea40 2024-09-21T00:29:57.687 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:29:57.688 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":70,"btime":"2024-09-21T00:29:51:115169+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6472,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6836/2828207894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":2828207894},{"type":"v1","addr":"172.21.15.187:6836","nonce":2828207894}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":67},{"gid":7152,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/2711765184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":2711765184},{"type":"v1","addr":"172.21.15.112:6833","nonce":2711765184}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":67},{"gid":7155,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/1863087214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":1863087214},{"type":"v1","addr":"172.21.15.112:6835","nonce":1863087214}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":70,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T00:29:48.456672+0000","modified":"2024-09-21T00:29:50.468508+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7167},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7167":{"gid":7167,"name":"d","rank":0,"incarnation":68,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6837/2071643267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6835","nonce":2071643267},{"type":"v1","addr":"172.21.15.187:6837","nonce":2071643267}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7167,"qdb_cluster":[7167]},"id":8}]} 2024-09-21T00:29:57.688 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 70 2024-09-21T00:29:57.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:57.688+0000 7f0c71da8640 1 -- 172.21.15.112:0/3466065608 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f0c58078b90 msgr2=0x7f0c5807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:57.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:57.688+0000 7f0c71da8640 1 --2- 172.21.15.112:0/3466065608 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f0c58078b90 0x7f0c5807b0a0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c0cff30 tx=0x7f0c5c009040 comp rx=0 tx=0).stop 2024-09-21T00:29:57.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:57.688+0000 7f0c71da8640 1 -- 172.21.15.112:0/3466065608 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c6c15ea40 msgr2=0x7f0c6c0c8f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:57.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:57.688+0000 7f0c71da8640 1 --2- 172.21.15.112:0/3466065608 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c6c15ea40 0x7f0c6c0c8f60 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f0c6400ba10 tx=0x7f0c6400bed0 comp rx=0 tx=0).stop 2024-09-21T00:29:57.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:57.688+0000 7f0c71da8640 1 -- 172.21.15.112:0/3466065608 shutdown_connections 2024-09-21T00:29:57.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:57.688+0000 7f0c71da8640 1 --2- 172.21.15.112:0/3466065608 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f0c58078b90 0x7f0c5807b0a0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:57.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:57.688+0000 7f0c71da8640 1 --2- 172.21.15.112:0/3466065608 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c6c15ea40 0x7f0c6c0c8f60 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:57.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:57.688+0000 7f0c71da8640 1 --2- 172.21.15.112:0/3466065608 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0c6c0d4a90 0x7f0c6c0c8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:57.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:57.688+0000 7f0c71da8640 1 --2- 172.21.15.112:0/3466065608 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0c6c0d0ae0 0x7f0c6c0c84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:57.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:57.688+0000 7f0c71da8640 1 -- 172.21.15.112:0/3466065608 >> 172.21.15.112:0/3466065608 conn(0x7f0c6c09d8a0 msgr2=0x7f0c6c15f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:57.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:57.688+0000 7f0c71da8640 1 -- 172.21.15.112:0/3466065608 shutdown_connections 2024-09-21T00:29:57.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:57.688+0000 7f0c71da8640 1 -- 172.21.15.112:0/3466065608 wait complete. 2024-09-21T00:29:58.104 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 70, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T00:29:48.456672+0000', 'modified': '2024-09-21T00:29:50.468508+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7167}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7167': {'gid': 7167, 'name': 'd', 'rank': 0, 'incarnation': 68, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.187:6837/2071643267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.187:6835', 'nonce': 2071643267}, {'type': 'v1', 'addr': '172.21.15.187:6837', 'nonce': 2071643267}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7167, 'qdb_cluster': [7167]} 2024-09-21T00:29:58.104 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-21T00:29:58.104 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-09-21T00:29:58.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.218+0000 7f3f32cf9640 1 -- 172.21.15.112:0/1449977174 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f2c15f5e0 msgr2=0x7f3f2c15bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:58.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.218+0000 7f3f32cf9640 1 --2- 172.21.15.112:0/1449977174 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f2c15f5e0 0x7f3f2c15bab0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f3f2c0ce2c0 tx=0x7f3f2402a4d0 comp rx=0 tx=0).stop 2024-09-21T00:29:58.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.218+0000 7f3f32cf9640 1 -- 172.21.15.112:0/1449977174 shutdown_connections 2024-09-21T00:29:58.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.218+0000 7f3f32cf9640 1 --2- 172.21.15.112:0/1449977174 >> v2:172.21.15.187:3300/0 conn(0x7f3f2c0cf5f0 0x7f3f2c15f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:58.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.218+0000 7f3f32cf9640 1 --2- 172.21.15.112:0/1449977174 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f2c15f5e0 0x7f3f2c15bab0 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:58.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.218+0000 7f3f32cf9640 1 --2- 172.21.15.112:0/1449977174 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f2c070540 0x7f3f2c0cf0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:58.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.218+0000 7f3f32cf9640 1 -- 172.21.15.112:0/1449977174 >> 172.21.15.112:0/1449977174 conn(0x7f3f2c071690 msgr2=0x7f3f2c071aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:58.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.218+0000 7f3f32cf9640 1 -- 172.21.15.112:0/1449977174 shutdown_connections 2024-09-21T00:29:58.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.218+0000 7f3f32cf9640 1 -- 172.21.15.112:0/1449977174 wait complete. 2024-09-21T00:29:58.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.220+0000 7f3f32cf9640 1 Processor -- start 2024-09-21T00:29:58.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.220+0000 7f3f32cf9640 1 -- start start 2024-09-21T00:29:58.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.221+0000 7f3f32cf9640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3f2c070540 0x7f3f2c149cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:58.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.221+0000 7f3f32cf9640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f2c0cf5f0 0x7f3f2c0c6f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:58.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.221+0000 7f3f32cf9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f2c15f5e0 0x7f3f2c0b46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:58.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.221+0000 7f3f32cf9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3f2c15c400 con 0x7f3f2c15f5e0 2024-09-21T00:29:58.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.221+0000 7f3f32cf9640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3f2c15c280 con 0x7f3f2c070540 2024-09-21T00:29:58.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.221+0000 7f3f32cf9640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3f2c15c580 con 0x7f3f2c0cf5f0 2024-09-21T00:29:58.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.222+0000 7f3f31296640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f2c15f5e0 0x7f3f2c0b46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:58.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.222+0000 7f3f31296640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f2c15f5e0 0x7f3f2c0b46f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40758/0 (socket says 172.21.15.112:40758) 2024-09-21T00:29:58.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.222+0000 7f3f31296640 1 -- 172.21.15.112:0/3106284641 learned_addr learned my addr 172.21.15.112:0/3106284641 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:29:58.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.222+0000 7f3f30a95640 1 --2- 172.21.15.112:0/3106284641 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3f2c070540 0x7f3f2c149cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:58.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.222+0000 7f3f2bfff640 1 --2- 172.21.15.112:0/3106284641 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f2c0cf5f0 0x7f3f2c0c6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:58.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.222+0000 7f3f31296640 1 -- 172.21.15.112:0/3106284641 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f2c0cf5f0 msgr2=0x7f3f2c0c6f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:58.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.222+0000 7f3f31296640 1 --2- 172.21.15.112:0/3106284641 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f2c0cf5f0 0x7f3f2c0c6f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:58.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.222+0000 7f3f31296640 1 -- 172.21.15.112:0/3106284641 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3f2c070540 msgr2=0x7f3f2c149cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:58.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.222+0000 7f3f31296640 1 --2- 172.21.15.112:0/3106284641 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3f2c070540 0x7f3f2c149cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:58.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.222+0000 7f3f31296640 1 -- 172.21.15.112:0/3106284641 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f2c14a230 con 0x7f3f2c15f5e0 2024-09-21T00:29:58.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.222+0000 7f3f2bfff640 1 --2- 172.21.15.112:0/3106284641 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f2c0cf5f0 0x7f3f2c0c6f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:29:58.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.222+0000 7f3f31296640 1 --2- 172.21.15.112:0/3106284641 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f2c15f5e0 0x7f3f2c0b46f0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f3f24009230 tx=0x7f3f24003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:58.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.222+0000 7f3f297fa640 1 -- 172.21.15.112:0/3106284641 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f2403e030 con 0x7f3f2c15f5e0 2024-09-21T00:29:58.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.222+0000 7f3f297fa640 1 -- 172.21.15.112:0/3106284641 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3f24003c20 con 0x7f3f2c15f5e0 2024-09-21T00:29:58.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.222+0000 7f3f32cf9640 1 -- 172.21.15.112:0/3106284641 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f2c0c7450 con 0x7f3f2c15f5e0 2024-09-21T00:29:58.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.222+0000 7f3f297fa640 1 -- 172.21.15.112:0/3106284641 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f24035040 con 0x7f3f2c15f5e0 2024-09-21T00:29:58.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.222+0000 7f3f32cf9640 1 -- 172.21.15.112:0/3106284641 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f2c1556d0 con 0x7f3f2c15f5e0 2024-09-21T00:29:58.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.225+0000 7f3f297fa640 1 -- 172.21.15.112:0/3106284641 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3f24041030 con 0x7f3f2c15f5e0 2024-09-21T00:29:58.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.225+0000 7f3f297fa640 1 --2- 172.21.15.112:0/3106284641 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f3f18078b90 0x7f3f1807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:58.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.225+0000 7f3f297fa640 1 -- 172.21.15.112:0/3106284641 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 6101+0+0 (secure 0 0 0) 0x7f3f240bb4e0 con 0x7f3f2c15f5e0 2024-09-21T00:29:58.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.225+0000 7f3f30a95640 1 --2- 172.21.15.112:0/3106284641 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f3f18078b90 0x7f3f1807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:58.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.226+0000 7f3f39c67740 1 Processor -- start 2024-09-21T00:29:58.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.226+0000 7f3f39c67740 1 -- start start 2024-09-21T00:29:58.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.226+0000 7f3f30a95640 1 --2- 172.21.15.112:0/3106284641 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f3f18078b90 0x7f3f1807b0a0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f3f1c009ab0 tx=0x7f3f1c005f90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:58.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.227+0000 7f3f39c67740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a94afa9270 0x55a94b38e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:58.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.227+0000 7f3f39c67740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a94af803b0 0x55a94b38e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:58.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.227+0000 7f3f39c67740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a94af88590 0x55a94b37cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:58.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.227+0000 7f3f39c67740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a94b2c3820 con 0x55a94af803b0 2024-09-21T00:29:58.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.227+0000 7f3f39c67740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55a94b2b2c50 con 0x55a94afa9270 2024-09-21T00:29:58.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.227+0000 7f3f39c67740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x55a94b2b6a60 con 0x55a94af88590 2024-09-21T00:29:58.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.227+0000 7f3f2bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a94af803b0 0x55a94b38e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:58.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.227+0000 7f3f2bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a94af803b0 0x55a94b38e9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40770/0 (socket says 172.21.15.112:40770) 2024-09-21T00:29:58.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.227+0000 7f3f2bfff640 1 -- 172.21.15.112:0/1807059272 learned_addr learned my addr 172.21.15.112:0/1807059272 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:29:58.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.227+0000 7f3f31296640 1 --2- 172.21.15.112:0/1807059272 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a94af88590 0x55a94b37cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:58.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.227+0000 7f3f30a95640 1 --2- 172.21.15.112:0/1807059272 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a94afa9270 0x55a94b38e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:58.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.227+0000 7f3f2bfff640 1 -- 172.21.15.112:0/1807059272 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a94af88590 msgr2=0x55a94b37cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:29:58.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.227+0000 7f3f2bfff640 1 --2- 172.21.15.112:0/1807059272 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a94af88590 0x55a94b37cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:58.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.227+0000 7f3f2bfff640 1 -- 172.21.15.112:0/1807059272 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a94afa9270 msgr2=0x55a94b38e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:29:58.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.227+0000 7f3f2bfff640 1 --2- 172.21.15.112:0/1807059272 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a94afa9270 0x55a94b38e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:58.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.227+0000 7f3f2bfff640 1 -- 172.21.15.112:0/1807059272 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a94b014850 con 0x55a94af803b0 2024-09-21T00:29:58.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.227+0000 7f3f30a95640 1 --2- 172.21.15.112:0/1807059272 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a94afa9270 0x55a94b38e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:29:58.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.228+0000 7f3f2bfff640 1 --2- 172.21.15.112:0/1807059272 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a94af803b0 0x55a94b38e9d0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f3f20009fd0 tx=0x7f3f20010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb773276af4d1b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:58.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.228+0000 7f3f127fc640 1 -- 172.21.15.112:0/1807059272 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f20011700 con 0x55a94af803b0 2024-09-21T00:29:58.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.228+0000 7f3f127fc640 1 -- 172.21.15.112:0/1807059272 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3f20011990 con 0x55a94af803b0 2024-09-21T00:29:58.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.228+0000 7f3f127fc640 1 -- 172.21.15.112:0/1807059272 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f20011da0 con 0x55a94af803b0 2024-09-21T00:29:58.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.229+0000 7f3f39c67740 1 -- 172.21.15.112:0/1807059272 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a94af803b0 msgr2=0x55a94b38e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:58.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.229+0000 7f3f39c67740 1 --2- 172.21.15.112:0/1807059272 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a94af803b0 0x55a94b38e9d0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f3f20009fd0 tx=0x7f3f20010860 comp rx=0 tx=0).stop 2024-09-21T00:29:58.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.229+0000 7f3f39c67740 1 -- 172.21.15.112:0/1807059272 shutdown_connections 2024-09-21T00:29:58.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.229+0000 7f3f39c67740 1 --2- 172.21.15.112:0/1807059272 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a94af88590 0x55a94b37cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:58.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.229+0000 7f3f39c67740 1 --2- 172.21.15.112:0/1807059272 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a94af803b0 0x55a94b38e9d0 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:58.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.229+0000 7f3f39c67740 1 --2- 172.21.15.112:0/1807059272 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a94afa9270 0x55a94b38e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:58.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.229+0000 7f3f39c67740 1 -- 172.21.15.112:0/1807059272 >> 172.21.15.112:0/1807059272 conn(0x55a94af9d3c0 msgr2=0x55a94b353470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:58.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.229+0000 7f3f39c67740 1 -- 172.21.15.112:0/1807059272 shutdown_connections 2024-09-21T00:29:58.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.229+0000 7f3f39c67740 1 -- 172.21.15.112:0/1807059272 wait complete. 2024-09-21T00:29:58.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.230+0000 7f3f39c67740 1 -- start start 2024-09-21T00:29:58.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.230+0000 7f3f39c67740 1 Processor -- start 2024-09-21T00:29:58.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.231+0000 7f3f39c67740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a94af803b0 0x55a94b36b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:58.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.231+0000 7f3f39c67740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a94af88590 0x55a94b390e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:58.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.231+0000 7f3f39c67740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a94afa9270 0x55a94b38ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:58.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.231+0000 7f3f39c67740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a94b023620 con 0x55a94af803b0 2024-09-21T00:29:58.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.231+0000 7f3f39c67740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55a94b2585f0 con 0x55a94afa9270 2024-09-21T00:29:58.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.231+0000 7f3f39c67740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x55a94b37b900 con 0x55a94af88590 2024-09-21T00:29:58.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.231+0000 7f3f39c67740 10 client.0 _ll_register_callbacks cb 0x55a94b25c0a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:29:58.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.231+0000 7f3f30a95640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a94af803b0 0x55a94b36b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:58.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.231+0000 7f3f30a95640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a94af803b0 0x55a94b36b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40784/0 (socket says 172.21.15.112:40784) 2024-09-21T00:29:58.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.231+0000 7f3f30a95640 1 -- 172.21.15.112:0/2005473866 learned_addr learned my addr 172.21.15.112:0/2005473866 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:29:58.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.231+0000 7f3f2bfff640 1 --2- 172.21.15.112:0/2005473866 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a94af88590 0x55a94b390e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:58.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.231+0000 7f3f31296640 1 --2- 172.21.15.112:0/2005473866 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a94afa9270 0x55a94b38ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:58.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.232+0000 7f3f30a95640 1 -- 172.21.15.112:0/2005473866 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a94af88590 msgr2=0x55a94b390e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:29:58.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.232+0000 7f3f30a95640 1 --2- 172.21.15.112:0/2005473866 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a94af88590 0x55a94b390e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:58.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.232+0000 7f3f30a95640 1 -- 172.21.15.112:0/2005473866 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a94afa9270 msgr2=0x55a94b38ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:29:58.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.232+0000 7f3f30a95640 1 --2- 172.21.15.112:0/2005473866 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a94afa9270 0x55a94b38ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:58.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.232+0000 7f3f30a95640 1 -- 172.21.15.112:0/2005473866 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a94b08b6b0 con 0x55a94af803b0 2024-09-21T00:29:58.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.232+0000 7f3f30a95640 1 --2- 172.21.15.112:0/2005473866 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a94af803b0 0x55a94b36b810 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f3f1c004570 tx=0x7f3f1c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96c0e1bf4748a609 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:58.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.232+0000 7f3f0b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T00:29:58.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.232+0000 7f3f0b7fe640 1 -- 172.21.15.112:0/2005473866 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f1c01f840 con 0x55a94af803b0 2024-09-21T00:29:58.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.232+0000 7f3f0b7fe640 1 -- 172.21.15.112:0/2005473866 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 6101+0+0 (secure 0 0 0) 0x7f3f1c01f9e0 con 0x55a94af803b0 2024-09-21T00:29:58.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.232+0000 7f3f39c67740 1 -- 172.21.15.112:0/2005473866 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f20011990 con 0x55a94af803b0 2024-09-21T00:29:58.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.232+0000 7f3f0b7fe640 1 -- 172.21.15.112:0/2005473866 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7f3f1c01fbe0 con 0x55a94af803b0 2024-09-21T00:29:58.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.232+0000 7f3f39c67740 10 client.7326 fetch_fsmap learned FSMap version 70 2024-09-21T00:29:58.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.232+0000 7f3f39c67740 1 -- 172.21.15.112:0/2005473866 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=2+}) -- 0x55a94b3913c0 con 0x55a94af803b0 2024-09-21T00:29:58.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.232+0000 7f3f0b7fe640 1 -- 172.21.15.112:0/2005473866 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3f20011990 con 0x55a94af803b0 2024-09-21T00:29:58.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.232+0000 7f3f0b7fe640 1 -- 172.21.15.112:0/2005473866 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 70) ==== 3502+0+0 (secure 0 0 0) 0x7f3f1c032080 con 0x55a94af803b0 2024-09-21T00:29:58.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.232+0000 7f3f39c67740 10 client.7326 fetch_fsmap finished waiting for FSMap version 70 2024-09-21T00:29:58.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.232+0000 7f3f39c67740 10 client.7326 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-21T00:29:58.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.232+0000 7f3f39c67740 20 client.7326 populate_metadata read hostname 'smithi112' 2024-09-21T00:29:58.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.232+0000 7f3f39c67740 1 --2- 172.21.15.112:0/2005473866 >> [v2:172.21.15.187:6835/2071643267,v1:172.21.15.187:6837/2071643267] conn(0x55a94af67e40 0x55a94b3ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:58.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.232+0000 7f3f39c67740 4 client.7326 mds_command: new command op to 7167 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T00:29:58.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.232+0000 7f3f39c67740 1 -- 172.21.15.112:0/2005473866 --> [v2:172.21.15.187:6835/2071643267,v1:172.21.15.187:6837/2071643267] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a94b2c5770 con 0x55a94af67e40 2024-09-21T00:29:58.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.233+0000 7f3f2bfff640 1 --2- 172.21.15.112:0/2005473866 >> [v2:172.21.15.187:6835/2071643267,v1:172.21.15.187:6837/2071643267] conn(0x55a94af67e40 0x55a94b3ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:58.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.233+0000 7f3f0b7fe640 10 client.7326 ms_handle_connect on v2:172.21.15.187:6835/2071643267 2024-09-21T00:29:58.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.233+0000 7f3f2bfff640 1 --2- 172.21.15.112:0/2005473866 >> [v2:172.21.15.187:6835/2071643267,v1:172.21.15.187:6837/2071643267] conn(0x55a94af67e40 0x55a94b3ac8e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55c2ffe49351441e server_cookie=3c1c71daee7cbb43 in_seq=0 out_seq=0 2024-09-21T00:29:58.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.234+0000 7f3f0b7fe640 1 -- 172.21.15.112:0/2005473866 <== mds.0 v2:172.21.15.187:6835/2071643267 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a94b2c5770 con 0x55a94af67e40 2024-09-21T00:29:58.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.234+0000 7f3f0b7fe640 10 client.7326 handle_command_reply: tid=0 2024-09-21T00:29:58.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.234+0000 7f3f39c67740 1 client.7326 shutdown 2024-09-21T00:29:58.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.234+0000 7f3f39c67740 1 -- 172.21.15.112:0/2005473866 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a94af803b0 msgr2=0x55a94b36b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:58.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.234+0000 7f3f39c67740 1 --2- 172.21.15.112:0/2005473866 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a94af803b0 0x55a94b36b810 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f3f1c004570 tx=0x7f3f1c004730 comp rx=0 tx=0).stop 2024-09-21T00:29:58.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.234+0000 7f3f39c67740 1 -- 172.21.15.112:0/2005473866 shutdown_connections 2024-09-21T00:29:58.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.234+0000 7f3f39c67740 1 --2- 172.21.15.112:0/2005473866 >> [v2:172.21.15.187:6835/2071643267,v1:172.21.15.187:6837/2071643267] conn(0x55a94af67e40 0x55a94b3ac8e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:58.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.234+0000 7f3f39c67740 1 --2- 172.21.15.112:0/2005473866 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a94afa9270 0x55a94b38ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:58.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.234+0000 7f3f39c67740 1 --2- 172.21.15.112:0/2005473866 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a94af88590 0x55a94b390e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:58.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.234+0000 7f3f0b7fe640 0 client.7326 ms_handle_reset on v2:172.21.15.187:6835/2071643267 2024-09-21T00:29:58.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.234+0000 7f3f39c67740 1 --2- 172.21.15.112:0/2005473866 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a94af803b0 0x55a94b36b810 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:58.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.234+0000 7f3f39c67740 1 -- 172.21.15.112:0/2005473866 >> 172.21.15.112:0/2005473866 conn(0x55a94af9d3c0 msgr2=0x55a94b352ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:58.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.234+0000 7f3f39c67740 1 -- 172.21.15.112:0/2005473866 shutdown_connections 2024-09-21T00:29:58.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.234+0000 7f3f39c67740 1 -- 172.21.15.112:0/2005473866 wait complete. 2024-09-21T00:29:58.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.235+0000 7f3f39c67740 20 client.7326 trim_cache size 0 max 16384 2024-09-21T00:29:58.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.252+0000 7f3f39c67740 1 Processor -- start 2024-09-21T00:29:58.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.252+0000 7f3f39c67740 1 -- start start 2024-09-21T00:29:58.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.254+0000 7f3f39c67740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a94af803b0 0x55a94b346920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:58.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.254+0000 7f3f39c67740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a94af88590 0x55a94b3b3180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:58.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.254+0000 7f3f39c67740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a94afa9270 0x55a94b352ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:58.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.254+0000 7f3f39c67740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3f2c06e110 con 0x55a94af88590 2024-09-21T00:29:58.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.254+0000 7f3f39c67740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3f2c1f6510 con 0x55a94af803b0 2024-09-21T00:29:58.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.254+0000 7f3f39c67740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3f2c06dac0 con 0x55a94afa9270 2024-09-21T00:29:58.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.254+0000 7f3f2bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a94af88590 0x55a94b3b3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:58.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.254+0000 7f3f2bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a94af88590 0x55a94b3b3180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40796/0 (socket says 172.21.15.112:40796) 2024-09-21T00:29:58.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.254+0000 7f3f2bfff640 1 -- 172.21.15.112:0/3019003225 learned_addr learned my addr 172.21.15.112:0/3019003225 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:29:58.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.254+0000 7f3f31296640 1 --2- 172.21.15.112:0/3019003225 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a94afa9270 0x55a94b352ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:58.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.254+0000 7f3f30a95640 1 --2- 172.21.15.112:0/3019003225 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a94af803b0 0x55a94b346920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:58.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.254+0000 7f3f2bfff640 1 -- 172.21.15.112:0/3019003225 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a94afa9270 msgr2=0x55a94b352ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:29:58.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.254+0000 7f3f2bfff640 1 --2- 172.21.15.112:0/3019003225 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a94afa9270 0x55a94b352ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:58.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.254+0000 7f3f2bfff640 1 -- 172.21.15.112:0/3019003225 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a94af803b0 msgr2=0x55a94b346920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:29:58.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.254+0000 7f3f2bfff640 1 --2- 172.21.15.112:0/3019003225 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a94af803b0 0x55a94b346920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:58.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.254+0000 7f3f2bfff640 1 -- 172.21.15.112:0/3019003225 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a94b3aa0e0 con 0x55a94af88590 2024-09-21T00:29:58.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.254+0000 7f3f2bfff640 1 --2- 172.21.15.112:0/3019003225 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a94af88590 0x55a94b3b3180 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f3f20018000 tx=0x7f3f20009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=567074c60844727f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:58.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.255+0000 7f3f11ffb640 1 -- 172.21.15.112:0/3019003225 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f20033070 con 0x55a94af88590 2024-09-21T00:29:58.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.255+0000 7f3f11ffb640 1 -- 172.21.15.112:0/3019003225 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3f20011990 con 0x55a94af88590 2024-09-21T00:29:58.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.255+0000 7f3f11ffb640 1 -- 172.21.15.112:0/3019003225 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f20011de0 con 0x55a94af88590 2024-09-21T00:29:58.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.255+0000 7f3f39c67740 1 -- 172.21.15.112:0/3019003225 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a94af88590 msgr2=0x55a94b3b3180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:58.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.255+0000 7f3f39c67740 1 --2- 172.21.15.112:0/3019003225 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a94af88590 0x55a94b3b3180 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f3f20018000 tx=0x7f3f20009500 comp rx=0 tx=0).stop 2024-09-21T00:29:58.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.255+0000 7f3f39c67740 1 -- 172.21.15.112:0/3019003225 shutdown_connections 2024-09-21T00:29:58.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.255+0000 7f3f39c67740 1 --2- 172.21.15.112:0/3019003225 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a94afa9270 0x55a94b352ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:58.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.255+0000 7f3f39c67740 1 --2- 172.21.15.112:0/3019003225 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a94af88590 0x55a94b3b3180 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:58.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.255+0000 7f3f39c67740 1 --2- 172.21.15.112:0/3019003225 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a94af803b0 0x55a94b346920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:58.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.255+0000 7f3f39c67740 1 -- 172.21.15.112:0/3019003225 >> 172.21.15.112:0/3019003225 conn(0x55a94af9d3c0 msgr2=0x55a94b36b120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:58.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.255+0000 7f3f39c67740 1 -- 172.21.15.112:0/3019003225 shutdown_connections 2024-09-21T00:29:58.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.255+0000 7f3f39c67740 1 -- 172.21.15.112:0/3019003225 wait complete. 2024-09-21T00:29:58.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.256+0000 7f3f39c67740 1 -- start start 2024-09-21T00:29:58.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.256+0000 7f3f39c67740 1 Processor -- start 2024-09-21T00:29:58.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.258+0000 7f3f39c67740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a94af803b0 0x55a94b4101a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:58.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.258+0000 7f3f39c67740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a94af88590 0x55a94b352ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:58.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.258+0000 7f3f39c67740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a94afa9270 0x55a94b353420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:58.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.258+0000 7f3f39c67740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a94b37b370 con 0x55a94af803b0 2024-09-21T00:29:58.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.258+0000 7f3f39c67740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55a94b38ff90 con 0x55a94afa9270 2024-09-21T00:29:58.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.258+0000 7f3f39c67740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x55a94b37d270 con 0x55a94af88590 2024-09-21T00:29:58.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.258+0000 7f3f39c67740 10 client.0 _ll_register_callbacks cb 0x7f3f1c009720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:29:58.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.258+0000 7f3f30a95640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a94af803b0 0x55a94b4101a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:58.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.258+0000 7f3f30a95640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a94af803b0 0x55a94b4101a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40798/0 (socket says 172.21.15.112:40798) 2024-09-21T00:29:58.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.258+0000 7f3f30a95640 1 -- 172.21.15.112:0/2673041911 learned_addr learned my addr 172.21.15.112:0/2673041911 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:29:58.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.258+0000 7f3f2bfff640 1 --2- 172.21.15.112:0/2673041911 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a94af88590 0x55a94b352ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:58.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.258+0000 7f3f31296640 1 --2- 172.21.15.112:0/2673041911 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a94afa9270 0x55a94b353420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:58.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.258+0000 7f3f30a95640 1 -- 172.21.15.112:0/2673041911 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a94af88590 msgr2=0x55a94b352ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:29:58.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.258+0000 7f3f30a95640 1 --2- 172.21.15.112:0/2673041911 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a94af88590 0x55a94b352ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:58.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.258+0000 7f3f30a95640 1 -- 172.21.15.112:0/2673041911 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a94afa9270 msgr2=0x55a94b353420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:29:58.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.258+0000 7f3f30a95640 1 --2- 172.21.15.112:0/2673041911 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a94afa9270 0x55a94b353420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:58.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.258+0000 7f3f30a95640 1 -- 172.21.15.112:0/2673041911 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a94b41a380 con 0x55a94af803b0 2024-09-21T00:29:58.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.258+0000 7f3f31296640 1 --2- 172.21.15.112:0/2673041911 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a94afa9270 0x55a94b353420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:29:58.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.258+0000 7f3f30a95640 1 --2- 172.21.15.112:0/2673041911 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a94af803b0 0x55a94b4101a0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f3f1c01fe40 tx=0x7f3f1c01f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=684b27149abb000 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:58.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.258+0000 7f3f0b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T00:29:58.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.258+0000 7f3f0b7fe640 1 -- 172.21.15.112:0/2673041911 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f1c03b020 con 0x55a94af803b0 2024-09-21T00:29:58.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.258+0000 7f3f0b7fe640 1 -- 172.21.15.112:0/2673041911 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 6101+0+0 (secure 0 0 0) 0x7f3f1c034470 con 0x55a94af803b0 2024-09-21T00:29:58.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.258+0000 7f3f39c67740 1 -- 172.21.15.112:0/2673041911 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f20011990 con 0x55a94af803b0 2024-09-21T00:29:58.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.259+0000 7f3f0b7fe640 1 -- 172.21.15.112:0/2673041911 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7f3f1c034670 con 0x55a94af803b0 2024-09-21T00:29:58.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.259+0000 7f3f39c67740 10 client.7332 fetch_fsmap learned FSMap version 70 2024-09-21T00:29:58.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.259+0000 7f3f39c67740 1 -- 172.21.15.112:0/2673041911 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=2+}) -- 0x55a94b4106e0 con 0x55a94af803b0 2024-09-21T00:29:58.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.259+0000 7f3f0b7fe640 1 -- 172.21.15.112:0/2673041911 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3f20011990 con 0x55a94af803b0 2024-09-21T00:29:58.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.259+0000 7f3f0b7fe640 1 -- 172.21.15.112:0/2673041911 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 70) ==== 3502+0+0 (secure 0 0 0) 0x7f3f1c032040 con 0x55a94af803b0 2024-09-21T00:29:58.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.259+0000 7f3f39c67740 10 client.7332 fetch_fsmap finished waiting for FSMap version 70 2024-09-21T00:29:58.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.259+0000 7f3f39c67740 10 client.7332 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-21T00:29:58.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.259+0000 7f3f39c67740 20 client.7332 populate_metadata read hostname 'smithi112' 2024-09-21T00:29:58.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.259+0000 7f3f39c67740 1 --2- 172.21.15.112:0/2673041911 >> [v2:172.21.15.187:6835/2071643267,v1:172.21.15.187:6837/2071643267] conn(0x55a94b34c9f0 0x55a94b41ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:58.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.259+0000 7f3f39c67740 4 client.7332 mds_command: new command op to 7167 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T00:29:58.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.259+0000 7f3f39c67740 1 -- 172.21.15.112:0/2673041911 --> [v2:172.21.15.187:6835/2071643267,v1:172.21.15.187:6837/2071643267] -- command(tid 0: {"prefix": "status"}) -- 0x55a94b2c1220 con 0x55a94b34c9f0 2024-09-21T00:29:58.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.259+0000 7f3f2bfff640 1 --2- 172.21.15.112:0/2673041911 >> [v2:172.21.15.187:6835/2071643267,v1:172.21.15.187:6837/2071643267] conn(0x55a94b34c9f0 0x55a94b41ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:58.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.260+0000 7f3f0b7fe640 10 client.7332 ms_handle_connect on v2:172.21.15.187:6835/2071643267 2024-09-21T00:29:58.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.260+0000 7f3f2bfff640 1 --2- 172.21.15.112:0/2673041911 >> [v2:172.21.15.187:6835/2071643267,v1:172.21.15.187:6837/2071643267] conn(0x55a94b34c9f0 0x55a94b41ce50 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=faeac6c82b99e2f9 server_cookie=b6f88b41131d4ca9 in_seq=0 out_seq=0 2024-09-21T00:29:58.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.260+0000 7f3f0b7fe640 1 -- 172.21.15.112:0/2673041911 <== mds.0 v2:172.21.15.187:6835/2071643267 1 ==== command_reply(tid 0: 0 ) ==== 8+0+333 (crc 0 0 0) 0x55a94b2c1220 con 0x55a94b34c9f0 2024-09-21T00:29:58.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.260+0000 7f3f0b7fe640 10 client.7332 handle_command_reply: tid=0 2024-09-21T00:29:58.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.260+0000 7f3f39c67740 1 client.7332 shutdown 2024-09-21T00:29:58.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.260+0000 7f3f39c67740 1 -- 172.21.15.112:0/2673041911 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a94af803b0 msgr2=0x55a94b4101a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:58.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.260+0000 7f3f39c67740 1 --2- 172.21.15.112:0/2673041911 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a94af803b0 0x55a94b4101a0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f3f1c01fe40 tx=0x7f3f1c01f730 comp rx=0 tx=0).stop 2024-09-21T00:29:58.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.260+0000 7f3f39c67740 1 -- 172.21.15.112:0/2673041911 shutdown_connections 2024-09-21T00:29:58.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.260+0000 7f3f39c67740 1 --2- 172.21.15.112:0/2673041911 >> [v2:172.21.15.187:6835/2071643267,v1:172.21.15.187:6837/2071643267] conn(0x55a94b34c9f0 0x55a94b41ce50 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:58.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.260+0000 7f3f39c67740 1 --2- 172.21.15.112:0/2673041911 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a94afa9270 0x55a94b353420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:58.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.260+0000 7f3f39c67740 1 --2- 172.21.15.112:0/2673041911 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a94af88590 0x55a94b352ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:58.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.260+0000 7f3f39c67740 1 --2- 172.21.15.112:0/2673041911 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a94af803b0 0x55a94b4101a0 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:58.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.260+0000 7f3f0b7fe640 0 client.7332 ms_handle_reset on v2:172.21.15.187:6835/2071643267 2024-09-21T00:29:58.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.260+0000 7f3f39c67740 1 -- 172.21.15.112:0/2673041911 >> 172.21.15.112:0/2673041911 conn(0x55a94af9d3c0 msgr2=0x55a94b346920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:58.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.260+0000 7f3f39c67740 1 -- 172.21.15.112:0/2673041911 shutdown_connections 2024-09-21T00:29:58.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.260+0000 7f3f39c67740 1 -- 172.21.15.112:0/2673041911 wait complete. 2024-09-21T00:29:58.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.261+0000 7f3f39c67740 20 client.7332 trim_cache size 0 max 16384 2024-09-21T00:29:58.267 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-21T00:29:58.267 INFO:teuthology.orchestra.run.smithi112.stdout: "cluster_fsid": "8b34692b-dc65-49b2-a08f-26f707b96ac5", 2024-09-21T00:29:58.267 INFO:teuthology.orchestra.run.smithi112.stdout: "whoami": 0, 2024-09-21T00:29:58.267 INFO:teuthology.orchestra.run.smithi112.stdout: "id": 7167, 2024-09-21T00:29:58.268 INFO:teuthology.orchestra.run.smithi112.stdout: "want_state": "up:active", 2024-09-21T00:29:58.268 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "up:active", 2024-09-21T00:29:58.268 INFO:teuthology.orchestra.run.smithi112.stdout: "fs_name": "cephfs", 2024-09-21T00:29:58.268 INFO:teuthology.orchestra.run.smithi112.stdout: "rank_uptime": 9.1711895069999994, 2024-09-21T00:29:58.268 INFO:teuthology.orchestra.run.smithi112.stdout: "mdsmap_epoch": 70, 2024-09-21T00:29:58.268 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch": 108, 2024-09-21T00:29:58.268 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch_barrier": 108, 2024-09-21T00:29:58.269 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 14.486297026000001 2024-09-21T00:29:58.269 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-21T00:29:58.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.261+0000 7f3f32cf9640 1 -- 172.21.15.112:0/3106284641 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f3f18078b90 msgr2=0x7f3f1807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:58.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.261+0000 7f3f32cf9640 1 --2- 172.21.15.112:0/3106284641 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f3f18078b90 0x7f3f1807b0a0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f3f1c009ab0 tx=0x7f3f1c005f90 comp rx=0 tx=0).stop 2024-09-21T00:29:58.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.261+0000 7f3f32cf9640 1 -- 172.21.15.112:0/3106284641 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f2c15f5e0 msgr2=0x7f3f2c0b46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:58.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.261+0000 7f3f32cf9640 1 --2- 172.21.15.112:0/3106284641 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f2c15f5e0 0x7f3f2c0b46f0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f3f24009230 tx=0x7f3f24003820 comp rx=0 tx=0).stop 2024-09-21T00:29:58.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.261+0000 7f3f32cf9640 1 -- 172.21.15.112:0/3106284641 shutdown_connections 2024-09-21T00:29:58.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.261+0000 7f3f32cf9640 1 --2- 172.21.15.112:0/3106284641 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f3f18078b90 0x7f3f1807b0a0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:58.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.261+0000 7f3f32cf9640 1 --2- 172.21.15.112:0/3106284641 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f2c15f5e0 0x7f3f2c0b46f0 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:58.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.261+0000 7f3f32cf9640 1 --2- 172.21.15.112:0/3106284641 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f2c0cf5f0 0x7f3f2c0c6f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:58.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.261+0000 7f3f32cf9640 1 --2- 172.21.15.112:0/3106284641 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3f2c070540 0x7f3f2c149cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:58.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.261+0000 7f3f32cf9640 1 -- 172.21.15.112:0/3106284641 >> 172.21.15.112:0/3106284641 conn(0x7f3f2c071690 msgr2=0x7f3f2c15e810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:58.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.261+0000 7f3f32cf9640 1 -- 172.21.15.112:0/3106284641 shutdown_connections 2024-09-21T00:29:58.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.261+0000 7f3f32cf9640 1 -- 172.21.15.112:0/3106284641 wait complete. 2024-09-21T00:29:58.275 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T00:29:58.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.388+0000 7f74d864a640 1 -- 172.21.15.112:0/115137714 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74bc0040b0 msgr2=0x7f74bc004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:58.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.388+0000 7f74d864a640 1 --2- 172.21.15.112:0/115137714 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74bc0040b0 0x7f74bc004800 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f74c000c260 tx=0x7f74bc005830 comp rx=0 tx=0).stop 2024-09-21T00:29:58.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.388+0000 7f74d6be7640 1 -- 172.21.15.112:0/115137714 reap_dead start 2024-09-21T00:29:58.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.388+0000 7f74d864a640 1 -- 172.21.15.112:0/115137714 shutdown_connections 2024-09-21T00:29:58.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.388+0000 7f74d864a640 1 -- 172.21.15.112:0/115137714 >> 172.21.15.112:0/115137714 conn(0x7f74d008dcb0 msgr2=0x7f74d009d8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:58.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.388+0000 7f74d864a640 1 -- 172.21.15.112:0/115137714 shutdown_connections 2024-09-21T00:29:58.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.388+0000 7f74d864a640 1 -- 172.21.15.112:0/115137714 wait complete. 2024-09-21T00:29:58.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.390+0000 7f74d864a640 1 Processor -- start 2024-09-21T00:29:58.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.390+0000 7f74d864a640 1 -- start start 2024-09-21T00:29:58.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.391+0000 7f74d864a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f74bc0040b0 0x7f74d00a1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:58.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.391+0000 7f74d864a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74d009f7f0 0x7f74d006fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:58.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.391+0000 7f74d864a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f74d00700b0 0x7f74d01f5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:58.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.391+0000 7f74d864a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f74d00d0d70 con 0x7f74d009f7f0 2024-09-21T00:29:58.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.391+0000 7f74d864a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f74d0163390 con 0x7f74bc0040b0 2024-09-21T00:29:58.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.391+0000 7f74d864a640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f74d00b6740 con 0x7f74d00700b0 2024-09-21T00:29:58.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.391+0000 7f74d5be5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74d009f7f0 0x7f74d006fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:58.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.391+0000 7f74d5be5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74d009f7f0 0x7f74d006fb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40812/0 (socket says 172.21.15.112:40812) 2024-09-21T00:29:58.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.391+0000 7f74d5be5640 1 -- 172.21.15.112:0/2179325044 learned_addr learned my addr 172.21.15.112:0/2179325044 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:29:58.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.391+0000 7f74d63e6640 1 --2- 172.21.15.112:0/2179325044 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f74bc0040b0 0x7f74d00a1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:58.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.391+0000 7f74d6be7640 1 --2- 172.21.15.112:0/2179325044 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f74d00700b0 0x7f74d01f5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:58.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.392+0000 7f74d5be5640 1 -- 172.21.15.112:0/2179325044 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f74d00700b0 msgr2=0x7f74d01f5360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:58.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.392+0000 7f74d5be5640 1 --2- 172.21.15.112:0/2179325044 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f74d00700b0 0x7f74d01f5360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:58.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.392+0000 7f74d5be5640 1 -- 172.21.15.112:0/2179325044 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f74bc0040b0 msgr2=0x7f74d00a1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:58.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.392+0000 7f74d5be5640 1 --2- 172.21.15.112:0/2179325044 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f74bc0040b0 0x7f74d00a1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:58.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.392+0000 7f74d5be5640 1 -- 172.21.15.112:0/2179325044 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74d009e250 con 0x7f74d009f7f0 2024-09-21T00:29:58.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.392+0000 7f74d5be5640 1 --2- 172.21.15.112:0/2179325044 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74d009f7f0 0x7f74d006fb70 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f74c002b110 tx=0x7f74c0006c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:58.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.392+0000 7f74c6ffd640 1 -- 172.21.15.112:0/2179325044 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c0031040 con 0x7f74d009f7f0 2024-09-21T00:29:58.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.392+0000 7f74c6ffd640 1 -- 172.21.15.112:0/2179325044 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f74c00035f0 con 0x7f74d009f7f0 2024-09-21T00:29:58.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.392+0000 7f74d864a640 1 -- 172.21.15.112:0/2179325044 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74d009e750 con 0x7f74d009f7f0 2024-09-21T00:29:58.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.392+0000 7f74c6ffd640 1 -- 172.21.15.112:0/2179325044 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c0003a40 con 0x7f74d009f7f0 2024-09-21T00:29:58.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.392+0000 7f74d864a640 1 -- 172.21.15.112:0/2179325044 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74d009ed10 con 0x7f74d009f7f0 2024-09-21T00:29:58.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.393+0000 7f74d864a640 1 -- 172.21.15.112:0/2179325044 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74d00d1140 con 0x7f74d009f7f0 2024-09-21T00:29:58.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.394+0000 7f74c6ffd640 1 -- 172.21.15.112:0/2179325044 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f74c0034070 con 0x7f74d009f7f0 2024-09-21T00:29:58.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.394+0000 7f74c6ffd640 1 --2- 172.21.15.112:0/2179325044 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f74b4078b90 0x7f74b407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:58.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.394+0000 7f74c6ffd640 1 -- 172.21.15.112:0/2179325044 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 6101+0+0 (secure 0 0 0) 0x7f74c00bb060 con 0x7f74d009f7f0 2024-09-21T00:29:58.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.395+0000 7f74d63e6640 1 --2- 172.21.15.112:0/2179325044 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f74b4078b90 0x7f74b407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:58.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.395+0000 7f74d63e6640 1 --2- 172.21.15.112:0/2179325044 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f74b4078b90 0x7f74b407b0a0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f74cc009cc0 tx=0x7f74cc005f30 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:58.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.399+0000 7f74c6ffd640 1 -- 172.21.15.112:0/2179325044 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f74d00d1140 con 0x7f74d009f7f0 2024-09-21T00:29:58.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.614+0000 7f74d864a640 1 -- 172.21.15.112:0/2179325044 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f74d00b6c30 con 0x7f74d009f7f0 2024-09-21T00:29:58.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.615+0000 7f74c6ffd640 1 -- 172.21.15.112:0/2179325044 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v70) ==== 53+0+73 (secure 0 0 0) 0x7f74c00834b0 con 0x7f74d009f7f0 2024-09-21T00:29:58.618 INFO:teuthology.orchestra.run.smithi112.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T00:29:58.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.617+0000 7f74d864a640 1 -- 172.21.15.112:0/2179325044 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f74b4078b90 msgr2=0x7f74b407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:58.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.617+0000 7f74d864a640 1 --2- 172.21.15.112:0/2179325044 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f74b4078b90 0x7f74b407b0a0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f74cc009cc0 tx=0x7f74cc005f30 comp rx=0 tx=0).stop 2024-09-21T00:29:58.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.617+0000 7f74d864a640 1 -- 172.21.15.112:0/2179325044 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74d009f7f0 msgr2=0x7f74d006fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:58.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.617+0000 7f74d864a640 1 --2- 172.21.15.112:0/2179325044 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74d009f7f0 0x7f74d006fb70 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f74c002b110 tx=0x7f74c0006c50 comp rx=0 tx=0).stop 2024-09-21T00:29:58.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.617+0000 7f74d864a640 1 -- 172.21.15.112:0/2179325044 shutdown_connections 2024-09-21T00:29:58.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.617+0000 7f74d864a640 1 --2- 172.21.15.112:0/2179325044 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f74d00700b0 0x7f74d01f5360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:58.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.617+0000 7f74d864a640 1 --2- 172.21.15.112:0/2179325044 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f74b4078b90 0x7f74b407b0a0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:58.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.617+0000 7f74d864a640 1 --2- 172.21.15.112:0/2179325044 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74d009f7f0 0x7f74d006fb70 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:58.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.617+0000 7f74d864a640 1 --2- 172.21.15.112:0/2179325044 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f74bc0040b0 0x7f74d00a1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:58.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.617+0000 7f74d864a640 1 -- 172.21.15.112:0/2179325044 >> 172.21.15.112:0/2179325044 conn(0x7f74d008dcb0 msgr2=0x7f74d00b5cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:58.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.617+0000 7f74d864a640 1 -- 172.21.15.112:0/2179325044 shutdown_connections 2024-09-21T00:29:58.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58.617+0000 7f74d864a640 1 -- 172.21.15.112:0/2179325044 wait complete. 2024-09-21T00:29:58.630 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T00:29:58.630 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T00:29:58.630 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi112.front.sepia.ceph.com 2024-09-21T00:29:58.630 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T00:29:58.630 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T00:29:58.631 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T00:29:58.631 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-21T00:29:58.631 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T00:29:58.631 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T00:29:58.631 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:29:58.631 DEBUG:teuthology.orchestra.run.smithi112:> ip addr 2024-09-21T00:29:58.647 INFO:teuthology.orchestra.run.smithi112.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T00:29:58.647 INFO:teuthology.orchestra.run.smithi112.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T00:29:58.647 INFO:teuthology.orchestra.run.smithi112.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T00:29:58.647 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:29:58.647 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 ::1/128 scope host 2024-09-21T00:29:58.648 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:29:58.648 INFO:teuthology.orchestra.run.smithi112.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T00:29:58.648 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:d8:a5:2c brd ff:ff:ff:ff:ff:ff 2024-09-21T00:29:58.648 INFO:teuthology.orchestra.run.smithi112.stdout: altname enp7s0f0 2024-09-21T00:29:58.648 INFO:teuthology.orchestra.run.smithi112.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T00:29:58.648 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:d8:a5:2d brd ff:ff:ff:ff:ff:ff 2024-09-21T00:29:58.648 INFO:teuthology.orchestra.run.smithi112.stdout: altname enp7s0f1 2024-09-21T00:29:58.648 INFO:teuthology.orchestra.run.smithi112.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T00:29:58.648 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:8f:cb:68 brd ff:ff:ff:ff:ff:ff 2024-09-21T00:29:58.648 INFO:teuthology.orchestra.run.smithi112.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T00:29:58.648 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:8f:cb:69 brd ff:ff:ff:ff:ff:ff 2024-09-21T00:29:58.648 INFO:teuthology.orchestra.run.smithi112.stdout: inet 172.21.15.112/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-21T00:29:58.649 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft 42524sec preferred_lft 42524sec 2024-09-21T00:29:58.649 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::ec4:7aff:fe8f:cb69/64 scope link noprefixroute 2024-09-21T00:29:58.649 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:29:58.649 INFO:teuthology.orchestra.run.smithi112.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T00:29:58.649 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 1a:39:64:e7:fe:d7 brd ff:ff:ff:ff:ff:ff 2024-09-21T00:29:58.649 INFO:teuthology.orchestra.run.smithi112.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T00:29:58.649 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:29:58.649 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::5858:3dff:feaf:1774/64 scope link 2024-09-21T00:29:58.649 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:29:58.649 INFO:teuthology.orchestra.run.smithi112.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T00:29:58.649 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 1a:39:64:e7:fe:d7 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T00:29:58.649 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::1839:64ff:fee7:fed7/64 scope link 2024-09-21T00:29:58.649 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:29:58.649 INFO:teuthology.orchestra.run.smithi112.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T00:29:58.649 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 86:c9:6c:ef:e4:e9 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T00:29:58.650 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::84c9:6cff:feef:e4e9/64 scope link 2024-09-21T00:29:58.650 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:29:58.650 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:29:58.650 DEBUG:teuthology.orchestra.run.smithi112:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T00:29:58.710 INFO:teuthology.orchestra.run.smithi112.stdout:1 2024-09-21T00:29:58.712 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:29:58.712 DEBUG:teuthology.orchestra.run.smithi112:> route 2024-09-21T00:29:58.768 INFO:teuthology.orchestra.run.smithi112.stdout:Kernel IP routing table 2024-09-21T00:29:58.769 INFO:teuthology.orchestra.run.smithi112.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T00:29:58.769 INFO:teuthology.orchestra.run.smithi112.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-21T00:29:58.769 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-21T00:29:58.769 INFO:teuthology.orchestra.run.smithi112.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T00:29:58.769 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T00:29:58.769 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2024-09-21T00:29:58.769 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-21T00:29:58.769 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-21T00:29:58.769 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-21T00:29:58.769 DEBUG:teuthology.orchestra.run.smithi112:> ') 2024-09-21T00:29:58.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T00:29:58.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T00:29:58.894 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:29:58.894 DEBUG:teuthology.orchestra.run.smithi112:> ip netns list 2024-09-21T00:29:58.909 INFO:teuthology.orchestra.run.smithi112.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-21T00:29:58.909 INFO:teuthology.orchestra.run.smithi112.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T00:29:58.909 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T00:29:58.910 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:29:58.910 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link set brx.0 up 2024-09-21T00:29:58.974 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T00:29:58.975 DEBUG:teuthology.orchestra.run.smithi112:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T00:29:59.030 INFO:teuthology.orchestra.run.smithi112.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T00:29:59.030 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T00:29:59.030 DEBUG:teuthology.orchestra.run.smithi112:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T00:29:59.087 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-21T00:29:59.087 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:29:59.087 DEBUG:teuthology.orchestra.run.smithi112:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T00:29:59.180 INFO:teuthology.orchestra.run.smithi112.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T00:29:59.180 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T00:29:59.181 INFO:teuthology.orchestra.run.smithi112.stdout:invalid new device string format 2024-09-21T00:29:59.191 INFO:teuthology.orchestra.run.smithi112.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T00:29:59.191 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T00:29:59.191 INFO:teuthology.orchestra.run.smithi112.stdout:invalid new device string format 2024-09-21T00:29:59.191 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: resolved to: "172.21.15.112:3300,172.21.15.187:3300,172.21.15.187:3301" 2024-09-21T00:29:59.191 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T00:29:59.197 INFO:teuthology.orchestra.run.smithi112.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T00:29:59.198 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T00:29:59.198 INFO:teuthology.orchestra.run.smithi112.stdout:invalid new device string format 2024-09-21T00:29:59.198 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: resolved to: "172.21.15.112:3300,172.21.15.187:3300,172.21.15.187:3301" 2024-09-21T00:29:59.198 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T00:29:59.198 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: resolved to: "172.21.15.112:6789,172.21.15.187:6789,172.21.15.187:6790" 2024-09-21T00:29:59.198 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.112:6789,172.21.15.187:6789,172.21.15.187:6790:/ 2024-09-21T00:29:59.198 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=8b34692b-dc65-49b2-a08f-26f707b96ac5" will pass to kernel 2024-09-21T00:29:59.198 INFO:teuthology.orchestra.run.smithi112.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-21T00:29:59.198 INFO:teuthology.orchestra.run.smithi112.stdout: You just mounted a file system that supports labels which does not 2024-09-21T00:29:59.199 INFO:teuthology.orchestra.run.smithi112.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T00:29:59.199 INFO:teuthology.orchestra.run.smithi112.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T00:29:59.199 INFO:teuthology.orchestra.run.smithi112.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T00:29:59.200 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T00:29:59.200 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:29:59.200 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T00:29:59.227 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T00:29:59.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:59 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T00:29:59.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T00:29:59.335 INFO:teuthology.orchestra.run.smithi112.stdout:client7341 2024-09-21T00:29:59.336 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T00:29:59.336 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:29:59.352 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T00:29:59.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:59.502+0000 7fda794b6640 1 -- 172.21.15.112:0/3385782342 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda740b5d70 msgr2=0x7fda7406fb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:59.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:59.502+0000 7fda794b6640 1 --2- 172.21.15.112:0/3385782342 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda740b5d70 0x7fda7406fb00 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7fda6c009830 tx=0x7fda6c02a510 comp rx=0 tx=0).stop 2024-09-21T00:29:59.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:59.502+0000 7fda794b6640 1 -- 172.21.15.112:0/3385782342 shutdown_connections 2024-09-21T00:29:59.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:59.502+0000 7fda794b6640 1 --2- 172.21.15.112:0/3385782342 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda740b5d70 0x7fda7406fb00 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:59.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:59.502+0000 7fda794b6640 1 --2- 172.21.15.112:0/3385782342 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fda74144480 0x7fda740b4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:59.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:59.502+0000 7fda794b6640 1 -- 172.21.15.112:0/3385782342 >> 172.21.15.112:0/3385782342 conn(0x7fda740cdd30 msgr2=0x7fda740ce140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:59.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:59.502+0000 7fda794b6640 1 -- 172.21.15.112:0/3385782342 shutdown_connections 2024-09-21T00:29:59.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:59.502+0000 7fda794b6640 1 -- 172.21.15.112:0/3385782342 wait complete. 2024-09-21T00:29:59.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:59.504+0000 7fda794b6640 1 Processor -- start 2024-09-21T00:29:59.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:59.504+0000 7fda794b6640 1 -- start start 2024-09-21T00:29:59.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:59.505+0000 7fda794b6640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fda74144480 0x7fda741f3310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:59.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:59.505+0000 7fda794b6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda741448d0 0x7fda7415f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:59.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:59.505+0000 7fda794b6640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fda741430c0 0x7fda741434d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:59.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:59.505+0000 7fda794b6640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fda740705f0 con 0x7fda741448d0 2024-09-21T00:29:59.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:59.505+0000 7fda794b6640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fda74070470 con 0x7fda741430c0 2024-09-21T00:29:59.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:59.505+0000 7fda794b6640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fda74070770 con 0x7fda74144480 2024-09-21T00:29:59.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:59.505+0000 7fda727fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda741448d0 0x7fda7415f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:59.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:59.505+0000 7fda727fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda741448d0 0x7fda7415f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40832/0 (socket says 172.21.15.112:40832) 2024-09-21T00:29:59.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:59.505+0000 7fda727fc640 1 -- 172.21.15.112:0/209391769 learned_addr learned my addr 172.21.15.112:0/209391769 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:29:59.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:59.506+0000 7fda72ffd640 1 --2- 172.21.15.112:0/209391769 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fda74144480 0x7fda741f3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:59.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:59.506+0000 7fda737fe640 1 --2- 172.21.15.112:0/209391769 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fda741430c0 0x7fda741434d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:59.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:59.506+0000 7fda727fc640 1 -- 172.21.15.112:0/209391769 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fda74144480 msgr2=0x7fda741f3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:59.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:59.506+0000 7fda727fc640 1 --2- 172.21.15.112:0/209391769 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fda74144480 0x7fda741f3310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:59.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:59.506+0000 7fda727fc640 1 -- 172.21.15.112:0/209391769 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fda741430c0 msgr2=0x7fda741434d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:59.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:59.506+0000 7fda727fc640 1 --2- 172.21.15.112:0/209391769 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fda741430c0 0x7fda741434d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:59.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:59.506+0000 7fda727fc640 1 -- 172.21.15.112:0/209391769 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda74143a10 con 0x7fda741448d0 2024-09-21T00:29:59.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:59.506+0000 7fda727fc640 1 --2- 172.21.15.112:0/209391769 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda741448d0 0x7fda7415f780 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7fda5c00c370 tx=0x7fda5c00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:59.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:59.506+0000 7fda6b7fe640 1 -- 172.21.15.112:0/209391769 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda5c011790 con 0x7fda741448d0 2024-09-21T00:29:59.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:59.506+0000 7fda6b7fe640 1 -- 172.21.15.112:0/209391769 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fda5c017070 con 0x7fda741448d0 2024-09-21T00:29:59.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:59.506+0000 7fda794b6640 1 -- 172.21.15.112:0/209391769 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda741f3cf0 con 0x7fda741448d0 2024-09-21T00:29:59.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:59.506+0000 7fda6b7fe640 1 -- 172.21.15.112:0/209391769 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda5c011cc0 con 0x7fda741448d0 2024-09-21T00:29:59.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:59.506+0000 7fda794b6640 1 -- 172.21.15.112:0/209391769 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda741f42b0 con 0x7fda741448d0 2024-09-21T00:29:59.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:59.507+0000 7fda794b6640 1 -- 172.21.15.112:0/209391769 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda740b6350 con 0x7fda741448d0 2024-09-21T00:29:59.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:59.508+0000 7fda6b7fe640 1 -- 172.21.15.112:0/209391769 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fda5c01c460 con 0x7fda741448d0 2024-09-21T00:29:59.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:59.508+0000 7fda6b7fe640 1 --2- 172.21.15.112:0/209391769 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fda54078bc0 0x7fda5407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:29:59.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:59.509+0000 7fda6b7fe640 1 -- 172.21.15.112:0/209391769 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 6101+0+0 (secure 0 0 0) 0x7fda5c09aa70 con 0x7fda741448d0 2024-09-21T00:29:59.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:59.509+0000 7fda72ffd640 1 --2- 172.21.15.112:0/209391769 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fda54078bc0 0x7fda5407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:29:59.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:59.509+0000 7fda72ffd640 1 --2- 172.21.15.112:0/209391769 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fda54078bc0 0x7fda5407b0d0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fda64005fd0 tx=0x7fda640095e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:29:59.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:59.513+0000 7fda6b7fe640 1 -- 172.21.15.112:0/209391769 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fda740b6350 con 0x7fda741448d0 2024-09-21T00:29:59.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:59.727+0000 7fda794b6640 1 -- 172.21.15.112:0/209391769 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fda74070c90 con 0x7fda741448d0 2024-09-21T00:29:59.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:59.727+0000 7fda6b7fe640 1 -- 172.21.15.112:0/209391769 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v70) ==== 53+0+73 (secure 0 0 0) 0x7fda74143a10 con 0x7fda741448d0 2024-09-21T00:29:59.729 INFO:teuthology.orchestra.run.smithi112.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T00:29:59.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:59.729+0000 7fda794b6640 1 -- 172.21.15.112:0/209391769 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fda54078bc0 msgr2=0x7fda5407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:59.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:59.729+0000 7fda794b6640 1 --2- 172.21.15.112:0/209391769 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fda54078bc0 0x7fda5407b0d0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fda64005fd0 tx=0x7fda640095e0 comp rx=0 tx=0).stop 2024-09-21T00:29:59.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:59.729+0000 7fda794b6640 1 -- 172.21.15.112:0/209391769 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda741448d0 msgr2=0x7fda7415f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:29:59.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:59.729+0000 7fda794b6640 1 --2- 172.21.15.112:0/209391769 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda741448d0 0x7fda7415f780 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7fda5c00c370 tx=0x7fda5c00c830 comp rx=0 tx=0).stop 2024-09-21T00:29:59.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:59.729+0000 7fda794b6640 1 -- 172.21.15.112:0/209391769 shutdown_connections 2024-09-21T00:29:59.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:59.729+0000 7fda794b6640 1 --2- 172.21.15.112:0/209391769 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fda741430c0 0x7fda741434d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:59.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:59.729+0000 7fda794b6640 1 --2- 172.21.15.112:0/209391769 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fda54078bc0 0x7fda5407b0d0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:59.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:59.729+0000 7fda794b6640 1 --2- 172.21.15.112:0/209391769 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda741448d0 0x7fda7415f780 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:59.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:59.729+0000 7fda794b6640 1 --2- 172.21.15.112:0/209391769 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fda74144480 0x7fda741f3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:29:59.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:59.729+0000 7fda794b6640 1 -- 172.21.15.112:0/209391769 >> 172.21.15.112:0/209391769 conn(0x7fda740cdd30 msgr2=0x7fda74140910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:29:59.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:59.730+0000 7fda794b6640 1 -- 172.21.15.112:0/209391769 shutdown_connections 2024-09-21T00:29:59.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:29:59.730+0000 7fda794b6640 1 -- 172.21.15.112:0/209391769 wait complete. 2024-09-21T00:29:59.743 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T00:29:59.743 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T00:29:59.743 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi187.front.sepia.ceph.com 2024-09-21T00:29:59.743 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-21T00:29:59.743 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-21T00:29:59.743 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T00:29:59.743 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-21T00:29:59.743 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T00:29:59.743 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-21T00:29:59.743 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:29:59.743 DEBUG:teuthology.orchestra.run.smithi187:> ip addr 2024-09-21T00:29:59.764 INFO:teuthology.orchestra.run.smithi187.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T00:29:59.764 INFO:teuthology.orchestra.run.smithi187.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T00:29:59.764 INFO:teuthology.orchestra.run.smithi187.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T00:29:59.764 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:29:59.764 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 ::1/128 scope host 2024-09-21T00:29:59.764 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:29:59.765 INFO:teuthology.orchestra.run.smithi187.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T00:29:59.765 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 0c:c4:7a:d9:36:78 brd ff:ff:ff:ff:ff:ff 2024-09-21T00:29:59.765 INFO:teuthology.orchestra.run.smithi187.stdout: altname enp7s0f0 2024-09-21T00:29:59.765 INFO:teuthology.orchestra.run.smithi187.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T00:29:59.765 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 0c:c4:7a:d9:36:79 brd ff:ff:ff:ff:ff:ff 2024-09-21T00:29:59.765 INFO:teuthology.orchestra.run.smithi187.stdout: altname enp7s0f1 2024-09-21T00:29:59.765 INFO:teuthology.orchestra.run.smithi187.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T00:29:59.765 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 0c:c4:7a:88:77:b0 brd ff:ff:ff:ff:ff:ff 2024-09-21T00:29:59.765 INFO:teuthology.orchestra.run.smithi187.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T00:29:59.766 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 0c:c4:7a:88:77:b1 brd ff:ff:ff:ff:ff:ff 2024-09-21T00:29:59.766 INFO:teuthology.orchestra.run.smithi187.stdout: inet 172.21.15.187/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-21T00:29:59.766 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft 42511sec preferred_lft 42511sec 2024-09-21T00:29:59.766 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 fe80::ec4:7aff:fe88:77b1/64 scope link noprefixroute 2024-09-21T00:29:59.766 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:29:59.766 INFO:teuthology.orchestra.run.smithi187.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T00:29:59.766 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 86:3b:f6:5b:1b:32 brd ff:ff:ff:ff:ff:ff 2024-09-21T00:29:59.766 INFO:teuthology.orchestra.run.smithi187.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T00:29:59.766 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:29:59.766 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 fe80::ac6f:97ff:fe09:b82e/64 scope link 2024-09-21T00:29:59.766 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:29:59.767 INFO:teuthology.orchestra.run.smithi187.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T00:29:59.767 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether ca:0b:a0:f5:2b:b5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T00:29:59.767 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 fe80::c80b:a0ff:fef5:2bb5/64 scope link 2024-09-21T00:29:59.767 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:29:59.767 INFO:teuthology.orchestra.run.smithi187.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T00:29:59.767 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 86:3b:f6:5b:1b:32 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-21T00:29:59.767 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 fe80::843b:f6ff:fe5b:1b32/64 scope link 2024-09-21T00:29:59.767 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:29:59.768 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:29:59.768 DEBUG:teuthology.orchestra.run.smithi187:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T00:29:59.832 INFO:teuthology.orchestra.run.smithi187.stdout:1 2024-09-21T00:29:59.834 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:29:59.834 DEBUG:teuthology.orchestra.run.smithi187:> route 2024-09-21T00:29:59.893 INFO:teuthology.orchestra.run.smithi187.stdout:Kernel IP routing table 2024-09-21T00:29:59.893 INFO:teuthology.orchestra.run.smithi187.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T00:29:59.893 INFO:teuthology.orchestra.run.smithi187.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-21T00:29:59.893 INFO:teuthology.orchestra.run.smithi187.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-21T00:29:59.894 INFO:teuthology.orchestra.run.smithi187.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T00:29:59.894 DEBUG:teuthology.orchestra.run.smithi187:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T00:29:59.894 DEBUG:teuthology.orchestra.run.smithi187:> set -e 2024-09-21T00:29:59.894 DEBUG:teuthology.orchestra.run.smithi187:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-21T00:29:59.894 DEBUG:teuthology.orchestra.run.smithi187:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-21T00:29:59.894 DEBUG:teuthology.orchestra.run.smithi187:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-21T00:29:59.894 DEBUG:teuthology.orchestra.run.smithi187:> ') 2024-09-21T00:29:59.986 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:29:59 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T00:30:00.022 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:30:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T00:30:00.029 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:30:00.029 DEBUG:teuthology.orchestra.run.smithi187:> ip netns list 2024-09-21T00:30:00.045 INFO:teuthology.orchestra.run.smithi187.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-21T00:30:00.045 INFO:teuthology.orchestra.run.smithi187.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-21T00:30:00.045 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-21T00:30:00.045 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:30:00.045 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip link set brx.0 up 2024-09-21T00:30:00.110 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T00:30:00.110 DEBUG:teuthology.orchestra.run.smithi187:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-21T00:30:00.166 INFO:teuthology.orchestra.run.smithi187.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-21T00:30:00.166 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T00:30:00.166 DEBUG:teuthology.orchestra.run.smithi187:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-21T00:30:00.221 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-21T00:30:00.221 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:30:00.221 DEBUG:teuthology.orchestra.run.smithi187:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-21T00:30:00.316 INFO:teuthology.orchestra.run.smithi187.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-21T00:30:00.316 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-21T00:30:00.316 INFO:teuthology.orchestra.run.smithi187.stdout:invalid new device string format 2024-09-21T00:30:00.326 INFO:teuthology.orchestra.run.smithi187.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-21T00:30:00.327 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-21T00:30:00.327 INFO:teuthology.orchestra.run.smithi187.stdout:invalid new device string format 2024-09-21T00:30:00.327 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: resolved to: "172.21.15.112:3300,172.21.15.187:3300,172.21.15.187:3301" 2024-09-21T00:30:00.327 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T00:30:00.336 INFO:teuthology.orchestra.run.smithi187.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-21T00:30:00.336 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-21T00:30:00.336 INFO:teuthology.orchestra.run.smithi187.stdout:invalid new device string format 2024-09-21T00:30:00.336 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: resolved to: "172.21.15.112:3300,172.21.15.187:3300,172.21.15.187:3301" 2024-09-21T00:30:00.336 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T00:30:00.336 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: resolved to: "172.21.15.112:6789,172.21.15.187:6789,172.21.15.187:6790" 2024-09-21T00:30:00.336 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.112:6789,172.21.15.187:6789,172.21.15.187:6790:/ 2024-09-21T00:30:00.336 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=8b34692b-dc65-49b2-a08f-26f707b96ac5" will pass to kernel 2024-09-21T00:30:00.337 INFO:teuthology.orchestra.run.smithi187.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-09-21T00:30:00.337 INFO:teuthology.orchestra.run.smithi187.stdout: You just mounted a file system that supports labels which does not 2024-09-21T00:30:00.337 INFO:teuthology.orchestra.run.smithi187.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T00:30:00.337 INFO:teuthology.orchestra.run.smithi187.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T00:30:00.337 INFO:teuthology.orchestra.run.smithi187.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T00:30:00.338 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T00:30:00.338 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:30:00.338 DEBUG:teuthology.orchestra.run.smithi187:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-21T00:30:00.363 DEBUG:teuthology.orchestra.run.smithi187:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T00:30:00.451 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:30:00 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T00:30:00.453 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:30:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T00:30:00.458 INFO:teuthology.orchestra.run.smithi187.stdout:client7353 2024-09-21T00:30:00.458 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-09-21T00:30:00.459 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:30:00.474 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T00:30:00.474 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:30:00.493 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T00:30:00.493 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:30:00.552 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T00:30:00.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:00.713+0000 7ff0ef39b640 1 -- 172.21.15.112:0/1889157414 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0e806cde0 msgr2=0x7ff0e806d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:00.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:00.713+0000 7ff0ef39b640 1 --2- 172.21.15.112:0/1889157414 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0e806cde0 0x7ff0e806d1f0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7ff0e8072960 tx=0x7ff0e002aac0 comp rx=0 tx=0).stop 2024-09-21T00:30:00.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:00.713+0000 7ff0ef39b640 1 -- 172.21.15.112:0/1889157414 shutdown_connections 2024-09-21T00:30:00.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:00.713+0000 7ff0ef39b640 1 --2- 172.21.15.112:0/1889157414 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0e806cde0 0x7ff0e806d1f0 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:00.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:00.713+0000 7ff0ef39b640 1 --2- 172.21.15.112:0/1889157414 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff0e80ce9e0 0x7ff0e80d0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:00.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:00.713+0000 7ff0ef39b640 1 --2- 172.21.15.112:0/1889157414 >> v2:172.21.15.187:3300/0 conn(0x7ff0e80cc270 0x7ff0e80d0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:00.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:00.713+0000 7ff0ef39b640 1 -- 172.21.15.112:0/1889157414 >> 172.21.15.112:0/1889157414 conn(0x7ff0e80b3580 msgr2=0x7ff0e80b3990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:00.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:00.713+0000 7ff0ef39b640 1 -- 172.21.15.112:0/1889157414 shutdown_connections 2024-09-21T00:30:00.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:00.713+0000 7ff0ef39b640 1 -- 172.21.15.112:0/1889157414 wait complete. 2024-09-21T00:30:00.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:00.715+0000 7ff0ef39b640 1 Processor -- start 2024-09-21T00:30:00.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:00.715+0000 7ff0ef39b640 1 -- start start 2024-09-21T00:30:00.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:00.716+0000 7ff0ef39b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff0e806cde0 0x7ff0e815aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:00.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:00.716+0000 7ff0ef39b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff0e80cc270 0x7ff0e815afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:00.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:00.716+0000 7ff0ef39b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0e80ce9e0 0x7ff0e8156050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:00.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:00.716+0000 7ff0ef39b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff0e806d730 con 0x7ff0e80ce9e0 2024-09-21T00:30:00.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:00.716+0000 7ff0ef39b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff0e806d8b0 con 0x7ff0e806cde0 2024-09-21T00:30:00.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:00.716+0000 7ff0ef39b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff0e806da30 con 0x7ff0e80cc270 2024-09-21T00:30:00.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:00.716+0000 7ff0ed938640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0e80ce9e0 0x7ff0e8156050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:00.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:00.716+0000 7ff0ed938640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0e80ce9e0 0x7ff0e8156050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40846/0 (socket says 172.21.15.112:40846) 2024-09-21T00:30:00.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:00.716+0000 7ff0ed938640 1 -- 172.21.15.112:0/3066527590 learned_addr learned my addr 172.21.15.112:0/3066527590 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:30:00.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:00.716+0000 7ff0ed137640 1 --2- 172.21.15.112:0/3066527590 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff0e806cde0 0x7ff0e815aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:00.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:00.716+0000 7ff0ec936640 1 --2- 172.21.15.112:0/3066527590 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff0e80cc270 0x7ff0e815afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:00.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:00.716+0000 7ff0ed938640 1 -- 172.21.15.112:0/3066527590 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff0e80cc270 msgr2=0x7ff0e815afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:00.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:00.716+0000 7ff0ed938640 1 --2- 172.21.15.112:0/3066527590 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff0e80cc270 0x7ff0e815afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:00.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:00.716+0000 7ff0ed938640 1 -- 172.21.15.112:0/3066527590 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff0e806cde0 msgr2=0x7ff0e815aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:00.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:00.716+0000 7ff0ed938640 1 --2- 172.21.15.112:0/3066527590 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff0e806cde0 0x7ff0e815aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:00.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:00.716+0000 7ff0ed938640 1 -- 172.21.15.112:0/3066527590 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0e815d150 con 0x7ff0e80ce9e0 2024-09-21T00:30:00.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:00.717+0000 7ff0ed938640 1 --2- 172.21.15.112:0/3066527590 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0e80ce9e0 0x7ff0e8156050 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7ff0e002b0b0 tx=0x7ff0e0003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:00.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:00.717+0000 7ff0ddffb640 1 -- 172.21.15.112:0/3066527590 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0e0004040 con 0x7ff0e80ce9e0 2024-09-21T00:30:00.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:00.717+0000 7ff0ddffb640 1 -- 172.21.15.112:0/3066527590 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff0e00041e0 con 0x7ff0e80ce9e0 2024-09-21T00:30:00.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:00.717+0000 7ff0ef39b640 1 -- 172.21.15.112:0/3066527590 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0e815d430 con 0x7ff0e80ce9e0 2024-09-21T00:30:00.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:00.717+0000 7ff0ddffb640 1 -- 172.21.15.112:0/3066527590 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0e0004610 con 0x7ff0e80ce9e0 2024-09-21T00:30:00.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:00.717+0000 7ff0ef39b640 1 -- 172.21.15.112:0/3066527590 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0e80c5770 con 0x7ff0e80ce9e0 2024-09-21T00:30:00.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:00.718+0000 7ff0ef39b640 1 -- 172.21.15.112:0/3066527590 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0e806d1f0 con 0x7ff0e80ce9e0 2024-09-21T00:30:00.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:00.719+0000 7ff0ddffb640 1 -- 172.21.15.112:0/3066527590 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff0e0049430 con 0x7ff0e80ce9e0 2024-09-21T00:30:00.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:00.719+0000 7ff0ddffb640 1 --2- 172.21.15.112:0/3066527590 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff0cc078bc0 0x7ff0cc07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:00.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:00.719+0000 7ff0ddffb640 1 -- 172.21.15.112:0/3066527590 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 6101+0+0 (secure 0 0 0) 0x7ff0e00bb9a0 con 0x7ff0e80ce9e0 2024-09-21T00:30:00.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:00.720+0000 7ff0ed137640 1 --2- 172.21.15.112:0/3066527590 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff0cc078bc0 0x7ff0cc07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:00.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:00.720+0000 7ff0ed137640 1 --2- 172.21.15.112:0/3066527590 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff0cc078bc0 0x7ff0cc07b0d0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7ff0e4009ab0 tx=0x7ff0e4005f90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:00.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:00.724+0000 7ff0ddffb640 1 -- 172.21.15.112:0/3066527590 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ff0e0032030 con 0x7ff0e80ce9e0 2024-09-21T00:30:00.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:00.913+0000 7ff0ef39b640 1 -- 172.21.15.112:0/3066527590 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7ff0e81f7af0 con 0x7ff0e80ce9e0 2024-09-21T00:30:00.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:00.913+0000 7ff0ddffb640 1 -- 172.21.15.112:0/3066527590 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v108) ==== 80+0+0 (secure 0 0 0) 0x7ff0e0083690 con 0x7ff0e80ce9e0 2024-09-21T00:30:00.915 INFO:teuthology.orchestra.run.smithi112.stderr:listed 0 entries 2024-09-21T00:30:00.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:00.915+0000 7ff0ef39b640 1 -- 172.21.15.112:0/3066527590 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff0cc078bc0 msgr2=0x7ff0cc07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:00.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:00.915+0000 7ff0ef39b640 1 --2- 172.21.15.112:0/3066527590 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff0cc078bc0 0x7ff0cc07b0d0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7ff0e4009ab0 tx=0x7ff0e4005f90 comp rx=0 tx=0).stop 2024-09-21T00:30:00.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:00.915+0000 7ff0ef39b640 1 -- 172.21.15.112:0/3066527590 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0e80ce9e0 msgr2=0x7ff0e8156050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:00.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:00.915+0000 7ff0ef39b640 1 --2- 172.21.15.112:0/3066527590 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0e80ce9e0 0x7ff0e8156050 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7ff0e002b0b0 tx=0x7ff0e0003c10 comp rx=0 tx=0).stop 2024-09-21T00:30:00.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:00.915+0000 7ff0ef39b640 1 -- 172.21.15.112:0/3066527590 shutdown_connections 2024-09-21T00:30:00.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:00.915+0000 7ff0ef39b640 1 --2- 172.21.15.112:0/3066527590 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff0cc078bc0 0x7ff0cc07b0d0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:00.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:00.915+0000 7ff0ef39b640 1 --2- 172.21.15.112:0/3066527590 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0e80ce9e0 0x7ff0e8156050 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:00.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:00.915+0000 7ff0ef39b640 1 --2- 172.21.15.112:0/3066527590 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff0e80cc270 0x7ff0e815afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:00.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:00.915+0000 7ff0ef39b640 1 --2- 172.21.15.112:0/3066527590 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff0e806cde0 0x7ff0e815aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:00.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:00.915+0000 7ff0ef39b640 1 -- 172.21.15.112:0/3066527590 >> 172.21.15.112:0/3066527590 conn(0x7ff0e80b3580 msgr2=0x7ff0e80d09e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:00.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:00.915+0000 7ff0ef39b640 1 -- 172.21.15.112:0/3066527590 shutdown_connections 2024-09-21T00:30:00.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:00.915+0000 7ff0ef39b640 1 -- 172.21.15.112:0/3066527590 wait complete. 2024-09-21T00:30:00.928 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-21T00:30:00.928 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:30:00.928 DEBUG:teuthology.orchestra.run.smithi112:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-21T00:30:00.957 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi112.front.sepia.ceph.com 2024-09-21T00:30:00.958 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:30:00.958 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T00:30:01.014 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-09-21T00:30:01.014 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:30:01.032 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-09-21T00:30:01.032 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:30:01.088 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T00:30:01.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.201+0000 7f97c2430640 1 -- 172.21.15.112:0/3808833158 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97a40040b0 msgr2=0x7f97a4004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:01.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.201+0000 7f97c2430640 1 --2- 172.21.15.112:0/3808833158 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97a40040b0 0x7f97a4004800 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f97b000c290 tx=0x7f97a4005830 comp rx=0 tx=0).stop 2024-09-21T00:30:01.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.201+0000 7f97c09cd640 1 -- 172.21.15.112:0/3808833158 reap_dead start 2024-09-21T00:30:01.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.201+0000 7f97c2430640 1 -- 172.21.15.112:0/3808833158 shutdown_connections 2024-09-21T00:30:01.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.201+0000 7f97c2430640 1 -- 172.21.15.112:0/3808833158 >> 172.21.15.112:0/3808833158 conn(0x7f97bc0cd5c0 msgr2=0x7f97bc0cd9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:01.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.201+0000 7f97c2430640 1 -- 172.21.15.112:0/3808833158 shutdown_connections 2024-09-21T00:30:01.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.201+0000 7f97c2430640 1 -- 172.21.15.112:0/3808833158 wait complete. 2024-09-21T00:30:01.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.203+0000 7f97c2430640 1 Processor -- start 2024-09-21T00:30:01.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.203+0000 7f97c2430640 1 -- start start 2024-09-21T00:30:01.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.205+0000 7f97c2430640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f97a40040b0 0x7f97bc1f2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:01.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.205+0000 7f97c2430640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97bc15e720 0x7f97bc152ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:01.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.205+0000 7f97c2430640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f97bc1506d0 0x7f97bc155540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:01.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.205+0000 7f97c2430640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f97bc0cf6e0 con 0x7f97bc15e720 2024-09-21T00:30:01.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.205+0000 7f97c2430640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f97bc0cf560 con 0x7f97a40040b0 2024-09-21T00:30:01.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.205+0000 7f97c2430640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f97bc15c810 con 0x7f97bc1506d0 2024-09-21T00:30:01.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.205+0000 7f97bb7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97bc15e720 0x7f97bc152ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:01.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.205+0000 7f97bb7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97bc15e720 0x7f97bc152ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40862/0 (socket says 172.21.15.112:40862) 2024-09-21T00:30:01.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.205+0000 7f97bb7fe640 1 -- 172.21.15.112:0/1058437403 learned_addr learned my addr 172.21.15.112:0/1058437403 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:30:01.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.205+0000 7f97c09cd640 1 --2- 172.21.15.112:0/1058437403 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f97bc1506d0 0x7f97bc155540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:01.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.205+0000 7f97bbfff640 1 --2- 172.21.15.112:0/1058437403 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f97a40040b0 0x7f97bc1f2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:01.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.205+0000 7f97bb7fe640 1 -- 172.21.15.112:0/1058437403 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f97bc1506d0 msgr2=0x7f97bc155540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:01.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.205+0000 7f97bb7fe640 1 --2- 172.21.15.112:0/1058437403 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f97bc1506d0 0x7f97bc155540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:01.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.205+0000 7f97bb7fe640 1 -- 172.21.15.112:0/1058437403 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f97a40040b0 msgr2=0x7f97bc1f2ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:01.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.205+0000 7f97bb7fe640 1 --2- 172.21.15.112:0/1058437403 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f97a40040b0 0x7f97bc1f2ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:01.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.205+0000 7f97bb7fe640 1 -- 172.21.15.112:0/1058437403 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97bc153400 con 0x7f97bc15e720 2024-09-21T00:30:01.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.205+0000 7f97bb7fe640 1 --2- 172.21.15.112:0/1058437403 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97bc15e720 0x7f97bc152ec0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f97b000c380 tx=0x7f97b0006c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:01.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.205+0000 7f97b8ff9640 1 -- 172.21.15.112:0/1058437403 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97b0031040 con 0x7f97bc15e720 2024-09-21T00:30:01.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.205+0000 7f97b8ff9640 1 -- 172.21.15.112:0/1058437403 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f97b00035f0 con 0x7f97bc15e720 2024-09-21T00:30:01.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.205+0000 7f97c2430640 1 -- 172.21.15.112:0/1058437403 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97bc15ee50 con 0x7f97bc15e720 2024-09-21T00:30:01.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.205+0000 7f97b8ff9640 1 -- 172.21.15.112:0/1058437403 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97b0003a40 con 0x7f97bc15e720 2024-09-21T00:30:01.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.205+0000 7f97c2430640 1 -- 172.21.15.112:0/1058437403 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97bc15cee0 con 0x7f97bc15e720 2024-09-21T00:30:01.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.206+0000 7f97c2430640 1 -- 172.21.15.112:0/1058437403 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97bc070850 con 0x7f97bc15e720 2024-09-21T00:30:01.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.207+0000 7f97b8ff9640 1 -- 172.21.15.112:0/1058437403 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f97b0034070 con 0x7f97bc15e720 2024-09-21T00:30:01.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.208+0000 7f97b8ff9640 1 --2- 172.21.15.112:0/1058437403 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f97a8078bc0 0x7f97a807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:01.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.208+0000 7f97b8ff9640 1 -- 172.21.15.112:0/1058437403 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 6101+0+0 (secure 0 0 0) 0x7f97b00bb1d0 con 0x7f97bc15e720 2024-09-21T00:30:01.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.208+0000 7f97bbfff640 1 --2- 172.21.15.112:0/1058437403 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f97a8078bc0 0x7f97a807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:01.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.208+0000 7f97bbfff640 1 --2- 172.21.15.112:0/1058437403 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f97a8078bc0 0x7f97a807b0d0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f97bc072d50 tx=0x7f97ac005f90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:01.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.212+0000 7f97b8ff9640 1 -- 172.21.15.112:0/1058437403 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f97b0083560 con 0x7f97bc15e720 2024-09-21T00:30:01.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.402+0000 7f97c2430640 1 -- 172.21.15.112:0/1058437403 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f97bc15ebe0 con 0x7f97bc15e720 2024-09-21T00:30:01.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.402+0000 7f97b8ff9640 1 -- 172.21.15.112:0/1058437403 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v108) ==== 80+0+0 (secure 0 0 0) 0x7f97b0082e60 con 0x7f97bc15e720 2024-09-21T00:30:01.404 INFO:teuthology.orchestra.run.smithi112.stderr:listed 0 entries 2024-09-21T00:30:01.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.405+0000 7f97c2430640 1 -- 172.21.15.112:0/1058437403 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f97a8078bc0 msgr2=0x7f97a807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:01.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.405+0000 7f97c2430640 1 --2- 172.21.15.112:0/1058437403 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f97a8078bc0 0x7f97a807b0d0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f97bc072d50 tx=0x7f97ac005f90 comp rx=0 tx=0).stop 2024-09-21T00:30:01.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.405+0000 7f97c2430640 1 -- 172.21.15.112:0/1058437403 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97bc15e720 msgr2=0x7f97bc152ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:01.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.405+0000 7f97c2430640 1 --2- 172.21.15.112:0/1058437403 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97bc15e720 0x7f97bc152ec0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f97b000c380 tx=0x7f97b0006c50 comp rx=0 tx=0).stop 2024-09-21T00:30:01.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.405+0000 7f97c2430640 1 -- 172.21.15.112:0/1058437403 shutdown_connections 2024-09-21T00:30:01.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.405+0000 7f97c2430640 1 --2- 172.21.15.112:0/1058437403 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f97bc1506d0 0x7f97bc155540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:01.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.405+0000 7f97c2430640 1 --2- 172.21.15.112:0/1058437403 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f97a8078bc0 0x7f97a807b0d0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:01.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.405+0000 7f97c2430640 1 --2- 172.21.15.112:0/1058437403 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97bc15e720 0x7f97bc152ec0 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:01.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.405+0000 7f97c2430640 1 --2- 172.21.15.112:0/1058437403 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f97a40040b0 0x7f97bc1f2ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:01.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.405+0000 7f97c2430640 1 -- 172.21.15.112:0/1058437403 >> 172.21.15.112:0/1058437403 conn(0x7f97bc0cd5c0 msgr2=0x7f97bc1526c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:01.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.405+0000 7f97c2430640 1 -- 172.21.15.112:0/1058437403 shutdown_connections 2024-09-21T00:30:01.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.405+0000 7f97c2430640 1 -- 172.21.15.112:0/1058437403 wait complete. 2024-09-21T00:30:01.417 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-09-21T00:30:01.418 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:30:01.418 DEBUG:teuthology.orchestra.run.smithi187:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-09-21T00:30:01.449 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi187.front.sepia.ceph.com 2024-09-21T00:30:01.449 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:30:01.449 DEBUG:teuthology.orchestra.run.smithi187:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-21T00:30:01.507 DEBUG:teuthology.orchestra.run.smithi112:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-21T00:30:01.510 DEBUG:teuthology.orchestra.run.smithi187:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-21T00:30:01.550 INFO:teuthology.orchestra.run.smithi112.stdout:[global] 2024-09-21T00:30:01.550 INFO:teuthology.orchestra.run.smithi112.stdout: chdir = "" 2024-09-21T00:30:01.550 INFO:teuthology.orchestra.run.smithi112.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-21T00:30:01.550 INFO:teuthology.orchestra.run.smithi112.stdout: auth supported = cephx 2024-09-21T00:30:01.551 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:30:01.551 INFO:teuthology.orchestra.run.smithi112.stdout: filestore xattr use omap = true 2024-09-21T00:30:01.551 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:30:01.551 INFO:teuthology.orchestra.run.smithi112.stdout: mon clock drift allowed = 1.000 2024-09-21T00:30:01.551 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:30:01.551 INFO:teuthology.orchestra.run.smithi112.stdout: osd crush chooseleaf type = 0 2024-09-21T00:30:01.551 INFO:teuthology.orchestra.run.smithi112.stdout: auth debug = true 2024-09-21T00:30:01.551 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:30:01.551 INFO:teuthology.orchestra.run.smithi112.stdout: ms die on old message = true 2024-09-21T00:30:01.551 INFO:teuthology.orchestra.run.smithi112.stdout: ms die on bug = true 2024-09-21T00:30:01.551 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:30:01.551 INFO:teuthology.orchestra.run.smithi112.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-21T00:30:01.551 INFO:teuthology.orchestra.run.smithi112.stdout: mon pg warn max object skew = 0 2024-09-21T00:30:01.551 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:30:01.551 INFO:teuthology.orchestra.run.smithi112.stdout: # disable pg_autoscaler by default for new pools 2024-09-21T00:30:01.552 INFO:teuthology.orchestra.run.smithi112.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-21T00:30:01.552 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:30:01.552 INFO:teuthology.orchestra.run.smithi112.stdout: osd pool default size = 2 2024-09-21T00:30:01.552 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:30:01.552 INFO:teuthology.orchestra.run.smithi112.stdout: mon osd allow primary affinity = true 2024-09-21T00:30:01.552 INFO:teuthology.orchestra.run.smithi112.stdout: mon osd allow pg remap = true 2024-09-21T00:30:01.552 INFO:teuthology.orchestra.run.smithi112.stdout: mon warn on legacy crush tunables = false 2024-09-21T00:30:01.552 INFO:teuthology.orchestra.run.smithi112.stdout: mon warn on crush straw calc version zero = false 2024-09-21T00:30:01.552 INFO:teuthology.orchestra.run.smithi112.stdout: mon warn on no sortbitwise = false 2024-09-21T00:30:01.552 INFO:teuthology.orchestra.run.smithi112.stdout: mon warn on osd down out interval zero = false 2024-09-21T00:30:01.552 INFO:teuthology.orchestra.run.smithi112.stdout: mon warn on too few osds = false 2024-09-21T00:30:01.552 INFO:teuthology.orchestra.run.smithi112.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-21T00:30:01.552 INFO:teuthology.orchestra.run.smithi112.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-21T00:30:01.552 INFO:teuthology.orchestra.run.smithi112.stdout: mon_allow_pool_size_one = true 2024-09-21T00:30:01.553 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:30:01.553 INFO:teuthology.orchestra.run.smithi112.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-21T00:30:01.553 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:30:01.553 INFO:teuthology.orchestra.run.smithi112.stdout: osd default data pool replay window = 5 2024-09-21T00:30:01.553 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:30:01.553 INFO:teuthology.orchestra.run.smithi112.stdout: mon allow pool delete = true 2024-09-21T00:30:01.553 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:30:01.553 INFO:teuthology.orchestra.run.smithi112.stdout: mon cluster log file level = debug 2024-09-21T00:30:01.553 INFO:teuthology.orchestra.run.smithi112.stdout: debug asserts on shutdown = true 2024-09-21T00:30:01.553 INFO:teuthology.orchestra.run.smithi112.stdout: mon health detail to clog = false 2024-09-21T00:30:01.553 INFO:teuthology.orchestra.run.smithi112.stdout: mon host = "172.21.15.112,172.21.15.187,[v2:172.21.15.187:3301,v1:172.21.15.187:6790]" 2024-09-21T00:30:01.553 INFO:teuthology.orchestra.run.smithi112.stdout: ms die on skipped message = False 2024-09-21T00:30:01.553 INFO:teuthology.orchestra.run.smithi112.stdout: fsid = 8b34692b-dc65-49b2-a08f-26f707b96ac5 2024-09-21T00:30:01.553 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:30:01.553 INFO:teuthology.orchestra.run.smithi112.stdout:[osd] 2024-09-21T00:30:01.554 INFO:teuthology.orchestra.run.smithi112.stdout: osd journal size = 100 2024-09-21T00:30:01.554 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:30:01.554 INFO:teuthology.orchestra.run.smithi112.stdout: osd scrub load threshold = 5.0 2024-09-21T00:30:01.554 INFO:teuthology.orchestra.run.smithi112.stdout: osd scrub max interval = 600 2024-09-21T00:30:01.554 INFO:teuthology.orchestra.run.smithi112.stdout: osd mclock profile = high_recovery_ops 2024-09-21T00:30:01.554 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:30:01.554 INFO:teuthology.orchestra.run.smithi112.stdout: osd recover clone overlap = true 2024-09-21T00:30:01.554 INFO:teuthology.orchestra.run.smithi112.stdout: osd recovery max chunk = 1048576 2024-09-21T00:30:01.554 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:30:01.554 INFO:teuthology.orchestra.run.smithi112.stdout: osd debug shutdown = true 2024-09-21T00:30:01.554 INFO:teuthology.orchestra.run.smithi112.stdout: osd debug op order = true 2024-09-21T00:30:01.554 INFO:teuthology.orchestra.run.smithi112.stdout: osd debug verify stray on activate = true 2024-09-21T00:30:01.554 INFO:teuthology.orchestra.run.smithi112.stdout: osd debug trim objects = true 2024-09-21T00:30:01.554 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:30:01.555 INFO:teuthology.orchestra.run.smithi112.stdout: osd open classes on start = true 2024-09-21T00:30:01.555 INFO:teuthology.orchestra.run.smithi112.stdout: osd debug pg log writeout = true 2024-09-21T00:30:01.555 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:30:01.555 INFO:teuthology.orchestra.run.smithi112.stdout: osd deep scrub update digest min age = 30 2024-09-21T00:30:01.555 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:30:01.555 INFO:teuthology.orchestra.run.smithi112.stdout: osd map max advance = 10 2024-09-21T00:30:01.555 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:30:01.555 INFO:teuthology.orchestra.run.smithi112.stdout: journal zero on create = true 2024-09-21T00:30:01.555 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:30:01.555 INFO:teuthology.orchestra.run.smithi112.stdout: filestore ondisk finisher threads = 3 2024-09-21T00:30:01.555 INFO:teuthology.orchestra.run.smithi112.stdout: filestore apply finisher threads = 3 2024-09-21T00:30:01.555 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:30:01.555 INFO:teuthology.orchestra.run.smithi112.stdout: bdev debug aio = true 2024-09-21T00:30:01.556 INFO:teuthology.orchestra.run.smithi112.stdout: osd debug misdirected ops = true 2024-09-21T00:30:01.556 INFO:teuthology.orchestra.run.smithi112.stdout: bluestore block size = 96636764160 2024-09-21T00:30:01.556 INFO:teuthology.orchestra.run.smithi112.stdout: bluestore fsck on mount = True 2024-09-21T00:30:01.556 INFO:teuthology.orchestra.run.smithi112.stdout: debug bluefs = 20 2024-09-21T00:30:01.556 INFO:teuthology.orchestra.run.smithi112.stdout: debug bluestore = 20 2024-09-21T00:30:01.556 INFO:teuthology.orchestra.run.smithi112.stdout: debug ms = 1 2024-09-21T00:30:01.556 INFO:teuthology.orchestra.run.smithi112.stdout: debug osd = 20 2024-09-21T00:30:01.556 INFO:teuthology.orchestra.run.smithi112.stdout: debug rocksdb = 10 2024-09-21T00:30:01.556 INFO:teuthology.orchestra.run.smithi112.stdout: mon osd backfillfull_ratio = 0.85 2024-09-21T00:30:01.556 INFO:teuthology.orchestra.run.smithi112.stdout: mon osd full ratio = 0.9 2024-09-21T00:30:01.556 INFO:teuthology.orchestra.run.smithi112.stdout: mon osd nearfull ratio = 0.8 2024-09-21T00:30:01.556 INFO:teuthology.orchestra.run.smithi112.stdout: osd failsafe full ratio = 0.95 2024-09-21T00:30:01.556 INFO:teuthology.orchestra.run.smithi112.stdout: osd objectstore = bluestore 2024-09-21T00:30:01.556 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:30:01.556 INFO:teuthology.orchestra.run.smithi112.stdout:[mgr] 2024-09-21T00:30:01.557 INFO:teuthology.orchestra.run.smithi112.stdout: debug ms = 1 2024-09-21T00:30:01.557 INFO:teuthology.orchestra.run.smithi112.stdout: debug mgr = 20 2024-09-21T00:30:01.557 INFO:teuthology.orchestra.run.smithi112.stdout: debug mon = 20 2024-09-21T00:30:01.557 INFO:teuthology.orchestra.run.smithi112.stdout: debug auth = 20 2024-09-21T00:30:01.557 INFO:teuthology.orchestra.run.smithi112.stdout: mon reweight min pgs per osd = 4 2024-09-21T00:30:01.557 INFO:teuthology.orchestra.run.smithi112.stdout: mon reweight min bytes per osd = 10 2024-09-21T00:30:01.557 INFO:teuthology.orchestra.run.smithi112.stdout: mgr/telemetry/nag = false 2024-09-21T00:30:01.557 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:30:01.557 INFO:teuthology.orchestra.run.smithi112.stdout:[mon] 2024-09-21T00:30:01.557 INFO:teuthology.orchestra.run.smithi112.stdout: debug ms = 1 2024-09-21T00:30:01.557 INFO:teuthology.orchestra.run.smithi112.stdout: debug mon = 20 2024-09-21T00:30:01.557 INFO:teuthology.orchestra.run.smithi112.stdout: debug paxos = 20 2024-09-21T00:30:01.557 INFO:teuthology.orchestra.run.smithi112.stdout: debug auth = 20 2024-09-21T00:30:01.557 INFO:teuthology.orchestra.run.smithi112.stdout: mon data avail warn = 5 2024-09-21T00:30:01.558 INFO:teuthology.orchestra.run.smithi112.stdout: mon mgr mkfs grace = 240 2024-09-21T00:30:01.558 INFO:teuthology.orchestra.run.smithi112.stdout: mon reweight min pgs per osd = 4 2024-09-21T00:30:01.558 INFO:teuthology.orchestra.run.smithi112.stdout: mon osd reporter subtree level = osd 2024-09-21T00:30:01.558 INFO:teuthology.orchestra.run.smithi112.stdout: mon osd prime pg temp = true 2024-09-21T00:30:01.558 INFO:teuthology.orchestra.run.smithi112.stdout: mon reweight min bytes per osd = 10 2024-09-21T00:30:01.558 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:30:01.558 INFO:teuthology.orchestra.run.smithi112.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-21T00:30:01.558 INFO:teuthology.orchestra.run.smithi112.stdout: auth mon ticket ttl = 660 # 11m 2024-09-21T00:30:01.558 INFO:teuthology.orchestra.run.smithi112.stdout: auth service ticket ttl = 240 # 4m 2024-09-21T00:30:01.558 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:30:01.558 INFO:teuthology.orchestra.run.smithi112.stdout: # don't complain about insecure global_id in the test suite 2024-09-21T00:30:01.558 INFO:teuthology.orchestra.run.smithi112.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-21T00:30:01.558 INFO:teuthology.orchestra.run.smithi112.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-21T00:30:01.558 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:30:01.558 INFO:teuthology.orchestra.run.smithi112.stdout: # 1m isn't quite enough 2024-09-21T00:30:01.559 INFO:teuthology.orchestra.run.smithi112.stdout: mon_down_mkfs_grace = 2m 2024-09-21T00:30:01.559 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:30:01.559 INFO:teuthology.orchestra.run.smithi112.stdout: mon_warn_on_filestore_osds = false 2024-09-21T00:30:01.559 INFO:teuthology.orchestra.run.smithi112.stdout: mon down mkfs grace = 300 2024-09-21T00:30:01.559 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:30:01.559 INFO:teuthology.orchestra.run.smithi112.stdout:[client] 2024-09-21T00:30:01.559 INFO:teuthology.orchestra.run.smithi112.stdout: rgw cache enabled = true 2024-09-21T00:30:01.559 INFO:teuthology.orchestra.run.smithi112.stdout: rgw enable ops log = true 2024-09-21T00:30:01.559 INFO:teuthology.orchestra.run.smithi112.stdout: rgw enable usage log = true 2024-09-21T00:30:01.559 INFO:teuthology.orchestra.run.smithi112.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-21T00:30:01.559 INFO:teuthology.orchestra.run.smithi112.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-21T00:30:01.559 INFO:teuthology.orchestra.run.smithi112.stdout: pid file = "" 2024-09-21T00:30:01.559 INFO:teuthology.orchestra.run.smithi112.stdout: client reconnect stale = True 2024-09-21T00:30:01.559 INFO:teuthology.orchestra.run.smithi112.stdout:[mon.a] 2024-09-21T00:30:01.559 INFO:teuthology.orchestra.run.smithi112.stdout:[mon.b] 2024-09-21T00:30:01.560 INFO:teuthology.orchestra.run.smithi112.stdout:[mon.c] 2024-09-21T00:30:01.560 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.a] 2024-09-21T00:30:01.560 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.c] 2024-09-21T00:30:01.560 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.b] 2024-09-21T00:30:01.560 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.d] 2024-09-21T00:30:01.560 INFO:teuthology.orchestra.run.smithi112.stdout:[mds] 2024-09-21T00:30:01.560 INFO:teuthology.orchestra.run.smithi112.stdout: mds_sessionmap_keys_per_op = 1 2024-09-21T00:30:01.591 INFO:teuthology.orchestra.run.smithi187.stdout:[global] 2024-09-21T00:30:01.591 INFO:teuthology.orchestra.run.smithi187.stdout: chdir = "" 2024-09-21T00:30:01.591 INFO:teuthology.orchestra.run.smithi187.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-21T00:30:01.591 INFO:teuthology.orchestra.run.smithi187.stdout: auth supported = cephx 2024-09-21T00:30:01.591 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:30:01.591 INFO:teuthology.orchestra.run.smithi187.stdout: filestore xattr use omap = true 2024-09-21T00:30:01.591 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:30:01.591 INFO:teuthology.orchestra.run.smithi187.stdout: mon clock drift allowed = 1.000 2024-09-21T00:30:01.591 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:30:01.591 INFO:teuthology.orchestra.run.smithi187.stdout: osd crush chooseleaf type = 0 2024-09-21T00:30:01.592 INFO:teuthology.orchestra.run.smithi187.stdout: auth debug = true 2024-09-21T00:30:01.592 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:30:01.592 INFO:teuthology.orchestra.run.smithi187.stdout: ms die on old message = true 2024-09-21T00:30:01.592 INFO:teuthology.orchestra.run.smithi187.stdout: ms die on bug = true 2024-09-21T00:30:01.592 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:30:01.592 INFO:teuthology.orchestra.run.smithi187.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-21T00:30:01.592 INFO:teuthology.orchestra.run.smithi187.stdout: mon pg warn max object skew = 0 2024-09-21T00:30:01.592 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:30:01.592 INFO:teuthology.orchestra.run.smithi187.stdout: # disable pg_autoscaler by default for new pools 2024-09-21T00:30:01.592 INFO:teuthology.orchestra.run.smithi187.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-21T00:30:01.592 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:30:01.592 INFO:teuthology.orchestra.run.smithi187.stdout: osd pool default size = 2 2024-09-21T00:30:01.592 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:30:01.592 INFO:teuthology.orchestra.run.smithi187.stdout: mon osd allow primary affinity = true 2024-09-21T00:30:01.592 INFO:teuthology.orchestra.run.smithi187.stdout: mon osd allow pg remap = true 2024-09-21T00:30:01.593 INFO:teuthology.orchestra.run.smithi187.stdout: mon warn on legacy crush tunables = false 2024-09-21T00:30:01.593 INFO:teuthology.orchestra.run.smithi187.stdout: mon warn on crush straw calc version zero = false 2024-09-21T00:30:01.593 INFO:teuthology.orchestra.run.smithi187.stdout: mon warn on no sortbitwise = false 2024-09-21T00:30:01.593 INFO:teuthology.orchestra.run.smithi187.stdout: mon warn on osd down out interval zero = false 2024-09-21T00:30:01.593 INFO:teuthology.orchestra.run.smithi187.stdout: mon warn on too few osds = false 2024-09-21T00:30:01.593 INFO:teuthology.orchestra.run.smithi187.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-21T00:30:01.593 INFO:teuthology.orchestra.run.smithi187.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-21T00:30:01.593 INFO:teuthology.orchestra.run.smithi187.stdout: mon_allow_pool_size_one = true 2024-09-21T00:30:01.593 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:30:01.593 INFO:teuthology.orchestra.run.smithi187.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-21T00:30:01.593 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:30:01.593 INFO:teuthology.orchestra.run.smithi187.stdout: osd default data pool replay window = 5 2024-09-21T00:30:01.593 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:30:01.593 INFO:teuthology.orchestra.run.smithi187.stdout: mon allow pool delete = true 2024-09-21T00:30:01.594 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:30:01.594 INFO:teuthology.orchestra.run.smithi187.stdout: mon cluster log file level = debug 2024-09-21T00:30:01.594 INFO:teuthology.orchestra.run.smithi187.stdout: debug asserts on shutdown = true 2024-09-21T00:30:01.594 INFO:teuthology.orchestra.run.smithi187.stdout: mon health detail to clog = false 2024-09-21T00:30:01.594 INFO:teuthology.orchestra.run.smithi187.stdout: mon host = "172.21.15.112,172.21.15.187,[v2:172.21.15.187:3301,v1:172.21.15.187:6790]" 2024-09-21T00:30:01.594 INFO:teuthology.orchestra.run.smithi187.stdout: ms die on skipped message = False 2024-09-21T00:30:01.594 INFO:teuthology.orchestra.run.smithi187.stdout: fsid = 8b34692b-dc65-49b2-a08f-26f707b96ac5 2024-09-21T00:30:01.594 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:30:01.594 INFO:teuthology.orchestra.run.smithi187.stdout:[osd] 2024-09-21T00:30:01.594 INFO:teuthology.orchestra.run.smithi187.stdout: osd journal size = 100 2024-09-21T00:30:01.594 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:30:01.594 INFO:teuthology.orchestra.run.smithi187.stdout: osd scrub load threshold = 5.0 2024-09-21T00:30:01.594 INFO:teuthology.orchestra.run.smithi187.stdout: osd scrub max interval = 600 2024-09-21T00:30:01.595 INFO:teuthology.orchestra.run.smithi187.stdout: osd mclock profile = high_recovery_ops 2024-09-21T00:30:01.595 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:30:01.595 INFO:teuthology.orchestra.run.smithi187.stdout: osd recover clone overlap = true 2024-09-21T00:30:01.595 INFO:teuthology.orchestra.run.smithi187.stdout: osd recovery max chunk = 1048576 2024-09-21T00:30:01.595 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:30:01.595 INFO:teuthology.orchestra.run.smithi187.stdout: osd debug shutdown = true 2024-09-21T00:30:01.595 INFO:teuthology.orchestra.run.smithi187.stdout: osd debug op order = true 2024-09-21T00:30:01.595 INFO:teuthology.orchestra.run.smithi187.stdout: osd debug verify stray on activate = true 2024-09-21T00:30:01.595 INFO:teuthology.orchestra.run.smithi187.stdout: osd debug trim objects = true 2024-09-21T00:30:01.595 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:30:01.595 INFO:teuthology.orchestra.run.smithi187.stdout: osd open classes on start = true 2024-09-21T00:30:01.595 INFO:teuthology.orchestra.run.smithi187.stdout: osd debug pg log writeout = true 2024-09-21T00:30:01.595 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:30:01.595 INFO:teuthology.orchestra.run.smithi187.stdout: osd deep scrub update digest min age = 30 2024-09-21T00:30:01.595 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:30:01.596 INFO:teuthology.orchestra.run.smithi187.stdout: osd map max advance = 10 2024-09-21T00:30:01.596 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:30:01.596 INFO:teuthology.orchestra.run.smithi187.stdout: journal zero on create = true 2024-09-21T00:30:01.596 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:30:01.596 INFO:teuthology.orchestra.run.smithi187.stdout: filestore ondisk finisher threads = 3 2024-09-21T00:30:01.596 INFO:teuthology.orchestra.run.smithi187.stdout: filestore apply finisher threads = 3 2024-09-21T00:30:01.596 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:30:01.596 INFO:teuthology.orchestra.run.smithi187.stdout: bdev debug aio = true 2024-09-21T00:30:01.596 INFO:teuthology.orchestra.run.smithi187.stdout: osd debug misdirected ops = true 2024-09-21T00:30:01.596 INFO:teuthology.orchestra.run.smithi187.stdout: bluestore block size = 96636764160 2024-09-21T00:30:01.596 INFO:teuthology.orchestra.run.smithi187.stdout: bluestore fsck on mount = True 2024-09-21T00:30:01.596 INFO:teuthology.orchestra.run.smithi187.stdout: debug bluefs = 20 2024-09-21T00:30:01.596 INFO:teuthology.orchestra.run.smithi187.stdout: debug bluestore = 20 2024-09-21T00:30:01.596 INFO:teuthology.orchestra.run.smithi187.stdout: debug ms = 1 2024-09-21T00:30:01.597 INFO:teuthology.orchestra.run.smithi187.stdout: debug osd = 20 2024-09-21T00:30:01.597 INFO:teuthology.orchestra.run.smithi187.stdout: debug rocksdb = 10 2024-09-21T00:30:01.597 INFO:teuthology.orchestra.run.smithi187.stdout: mon osd backfillfull_ratio = 0.85 2024-09-21T00:30:01.597 INFO:teuthology.orchestra.run.smithi187.stdout: mon osd full ratio = 0.9 2024-09-21T00:30:01.597 INFO:teuthology.orchestra.run.smithi187.stdout: mon osd nearfull ratio = 0.8 2024-09-21T00:30:01.597 INFO:teuthology.orchestra.run.smithi187.stdout: osd failsafe full ratio = 0.95 2024-09-21T00:30:01.597 INFO:teuthology.orchestra.run.smithi187.stdout: osd objectstore = bluestore 2024-09-21T00:30:01.597 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:30:01.597 INFO:teuthology.orchestra.run.smithi187.stdout:[mgr] 2024-09-21T00:30:01.597 INFO:teuthology.orchestra.run.smithi187.stdout: debug ms = 1 2024-09-21T00:30:01.597 INFO:teuthology.orchestra.run.smithi187.stdout: debug mgr = 20 2024-09-21T00:30:01.597 INFO:teuthology.orchestra.run.smithi187.stdout: debug mon = 20 2024-09-21T00:30:01.597 INFO:teuthology.orchestra.run.smithi187.stdout: debug auth = 20 2024-09-21T00:30:01.597 INFO:teuthology.orchestra.run.smithi187.stdout: mon reweight min pgs per osd = 4 2024-09-21T00:30:01.598 INFO:teuthology.orchestra.run.smithi187.stdout: mon reweight min bytes per osd = 10 2024-09-21T00:30:01.598 INFO:teuthology.orchestra.run.smithi187.stdout: mgr/telemetry/nag = false 2024-09-21T00:30:01.598 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:30:01.598 INFO:teuthology.orchestra.run.smithi187.stdout:[mon] 2024-09-21T00:30:01.598 INFO:teuthology.orchestra.run.smithi187.stdout: debug ms = 1 2024-09-21T00:30:01.598 INFO:teuthology.orchestra.run.smithi187.stdout: debug mon = 20 2024-09-21T00:30:01.598 INFO:teuthology.orchestra.run.smithi187.stdout: debug paxos = 20 2024-09-21T00:30:01.598 INFO:teuthology.orchestra.run.smithi187.stdout: debug auth = 20 2024-09-21T00:30:01.598 INFO:teuthology.orchestra.run.smithi187.stdout: mon data avail warn = 5 2024-09-21T00:30:01.598 INFO:teuthology.orchestra.run.smithi187.stdout: mon mgr mkfs grace = 240 2024-09-21T00:30:01.598 INFO:teuthology.orchestra.run.smithi187.stdout: mon reweight min pgs per osd = 4 2024-09-21T00:30:01.598 INFO:teuthology.orchestra.run.smithi187.stdout: mon osd reporter subtree level = osd 2024-09-21T00:30:01.598 INFO:teuthology.orchestra.run.smithi187.stdout: mon osd prime pg temp = true 2024-09-21T00:30:01.598 INFO:teuthology.orchestra.run.smithi187.stdout: mon reweight min bytes per osd = 10 2024-09-21T00:30:01.599 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:30:01.599 INFO:teuthology.orchestra.run.smithi187.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-21T00:30:01.599 INFO:teuthology.orchestra.run.smithi187.stdout: auth mon ticket ttl = 660 # 11m 2024-09-21T00:30:01.599 INFO:teuthology.orchestra.run.smithi187.stdout: auth service ticket ttl = 240 # 4m 2024-09-21T00:30:01.599 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:30:01.599 INFO:teuthology.orchestra.run.smithi187.stdout: # don't complain about insecure global_id in the test suite 2024-09-21T00:30:01.599 INFO:teuthology.orchestra.run.smithi187.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-21T00:30:01.599 INFO:teuthology.orchestra.run.smithi187.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-21T00:30:01.599 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:30:01.599 INFO:teuthology.orchestra.run.smithi187.stdout: # 1m isn't quite enough 2024-09-21T00:30:01.599 INFO:teuthology.orchestra.run.smithi187.stdout: mon_down_mkfs_grace = 2m 2024-09-21T00:30:01.599 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:30:01.599 INFO:teuthology.orchestra.run.smithi187.stdout: mon_warn_on_filestore_osds = false 2024-09-21T00:30:01.599 INFO:teuthology.orchestra.run.smithi187.stdout: mon down mkfs grace = 300 2024-09-21T00:30:01.599 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:30:01.600 INFO:teuthology.orchestra.run.smithi187.stdout:[client] 2024-09-21T00:30:01.600 INFO:teuthology.orchestra.run.smithi187.stdout: rgw cache enabled = true 2024-09-21T00:30:01.600 INFO:teuthology.orchestra.run.smithi187.stdout: rgw enable ops log = true 2024-09-21T00:30:01.600 INFO:teuthology.orchestra.run.smithi187.stdout: rgw enable usage log = true 2024-09-21T00:30:01.600 INFO:teuthology.orchestra.run.smithi187.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-21T00:30:01.600 INFO:teuthology.orchestra.run.smithi187.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-21T00:30:01.600 INFO:teuthology.orchestra.run.smithi187.stdout: pid file = "" 2024-09-21T00:30:01.600 INFO:teuthology.orchestra.run.smithi187.stdout: client reconnect stale = True 2024-09-21T00:30:01.600 INFO:teuthology.orchestra.run.smithi187.stdout:[mon.a] 2024-09-21T00:30:01.600 INFO:teuthology.orchestra.run.smithi187.stdout:[mon.b] 2024-09-21T00:30:01.600 INFO:teuthology.orchestra.run.smithi187.stdout:[mon.c] 2024-09-21T00:30:01.600 INFO:teuthology.orchestra.run.smithi187.stdout:[mds.a] 2024-09-21T00:30:01.600 INFO:teuthology.orchestra.run.smithi187.stdout:[mds.c] 2024-09-21T00:30:01.600 INFO:teuthology.orchestra.run.smithi187.stdout:[mds.b] 2024-09-21T00:30:01.601 INFO:teuthology.orchestra.run.smithi187.stdout:[mds.d] 2024-09-21T00:30:01.601 INFO:teuthology.orchestra.run.smithi187.stdout:[mds] 2024-09-21T00:30:01.601 INFO:teuthology.orchestra.run.smithi187.stdout: mds_sessionmap_keys_per_op = 1 2024-09-21T00:30:01.602 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-21T00:30:01.602 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T00:30:01.603 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-21T00:30:01.603 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T00:30:01.603 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-21T00:30:01.603 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T00:30:01.603 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-21T00:30:01.603 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T00:30:01.612 INFO:tasks.ceph.mds.a:Stopped 2024-09-21T00:30:01.612 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail a 2024-09-21T00:30:01.613 INFO:tasks.ceph.mds.c:Stopped 2024-09-21T00:30:01.613 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail c 2024-09-21T00:30:01.614 INFO:tasks.ceph.mds.b:Stopped 2024-09-21T00:30:01.614 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail b 2024-09-21T00:30:01.615 INFO:tasks.ceph.mds.d:Stopped 2024-09-21T00:30:01.615 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail d 2024-09-21T00:30:01.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.775+0000 7f382db77640 1 -- 172.21.15.112:0/1470351890 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38280cfda0 msgr2=0x7f38280d1ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:02.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.775+0000 7f382db77640 1 --2- 172.21.15.112:0/1470351890 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38280cfda0 0x7f38280d1ef0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f381c005fd0 tx=0x7f381c02a5f0 comp rx=0 tx=0).stop 2024-09-21T00:30:02.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.775+0000 7f382db77640 1 -- 172.21.15.112:0/1470351890 shutdown_connections 2024-09-21T00:30:02.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.775+0000 7f382db77640 1 --2- 172.21.15.112:0/1470351890 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38280cfda0 0x7f38280d1ef0 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:02.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.775+0000 7f382db77640 1 -- 172.21.15.112:0/1470351890 >> 172.21.15.112:0/1470351890 conn(0x7f38280d13d0 msgr2=0x7f38280d17e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:02.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.775+0000 7f382db77640 1 -- 172.21.15.112:0/1470351890 shutdown_connections 2024-09-21T00:30:02.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.775+0000 7f382db77640 1 -- 172.21.15.112:0/1470351890 wait complete. 2024-09-21T00:30:02.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.775+0000 7f79de4f2640 1 -- 172.21.15.112:0/2311978264 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f79d80b3cc0 msgr2=0x7f79d80cfc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:02.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.775+0000 7f79de4f2640 1 --2- 172.21.15.112:0/2311978264 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f79d80b3cc0 0x7f79d80cfc20 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f79d0005fd0 tx=0x7f79d002a930 comp rx=0 tx=0).stop 2024-09-21T00:30:02.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.775+0000 7f79de4f2640 1 -- 172.21.15.112:0/2311978264 shutdown_connections 2024-09-21T00:30:02.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.775+0000 7f79de4f2640 1 --2- 172.21.15.112:0/2311978264 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f79d80b3cc0 0x7f79d80cfc20 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:02.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.775+0000 7f79de4f2640 1 --2- 172.21.15.112:0/2311978264 >> v2:172.21.15.112:3300/0 conn(0x7f79d80d0fc0 0x7f79d815baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:02.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.776+0000 7f79de4f2640 1 -- 172.21.15.112:0/2311978264 >> 172.21.15.112:0/2311978264 conn(0x7f79d806d4c0 msgr2=0x7f79d80d02c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:02.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.776+0000 7f79de4f2640 1 -- 172.21.15.112:0/2311978264 shutdown_connections 2024-09-21T00:30:02.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.776+0000 7f79de4f2640 1 -- 172.21.15.112:0/2311978264 wait complete. 2024-09-21T00:30:02.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.776+0000 7fce0e951640 1 -- 172.21.15.112:0/613612055 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fce0815f100 msgr2=0x7fce081519b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:02.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.776+0000 7fce0e951640 1 --2- 172.21.15.112:0/613612055 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fce0815f100 0x7fce081519b0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fcdf80082a0 tx=0x7fcdf802a380 comp rx=0 tx=0).stop 2024-09-21T00:30:02.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.776+0000 7fce0e951640 1 -- 172.21.15.112:0/613612055 shutdown_connections 2024-09-21T00:30:02.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.776+0000 7fce0e951640 1 --2- 172.21.15.112:0/613612055 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fce0815f100 0x7fce081519b0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:02.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.776+0000 7fce0e951640 1 --2- 172.21.15.112:0/613612055 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fce080cfd80 0x7fce0815ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:02.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.776+0000 7fce0e951640 1 -- 172.21.15.112:0/613612055 >> 172.21.15.112:0/613612055 conn(0x7fce0806d4c0 msgr2=0x7fce080d02c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:02.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.776+0000 7f0f0a43f640 1 -- 172.21.15.112:0/1957522701 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f040cf2d0 msgr2=0x7f0f040cf6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:02.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.776+0000 7fce0e951640 1 -- 172.21.15.112:0/613612055 shutdown_connections 2024-09-21T00:30:02.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.776+0000 7f0f0a43f640 1 --2- 172.21.15.112:0/1957522701 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f040cf2d0 0x7f0f040cf6e0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f0ef8009900 tx=0x7f0ef802a5f0 comp rx=0 tx=0).stop 2024-09-21T00:30:02.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.776+0000 7fce0e951640 1 -- 172.21.15.112:0/613612055 wait complete. 2024-09-21T00:30:02.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.776+0000 7f0f0a43f640 1 -- 172.21.15.112:0/1957522701 shutdown_connections 2024-09-21T00:30:02.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.776+0000 7f0f0a43f640 1 --2- 172.21.15.112:0/1957522701 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f040cf2d0 0x7f0f040cf6e0 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:02.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.776+0000 7f0f0a43f640 1 --2- 172.21.15.112:0/1957522701 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0f040b64e0 0x7f0f040b3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:02.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.776+0000 7f0f0a43f640 1 -- 172.21.15.112:0/1957522701 >> 172.21.15.112:0/1957522701 conn(0x7f0f040b2fa0 msgr2=0x7f0f040b33b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:02.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.776+0000 7f0f0a43f640 1 -- 172.21.15.112:0/1957522701 shutdown_connections 2024-09-21T00:30:02.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.776+0000 7f0f0a43f640 1 -- 172.21.15.112:0/1957522701 wait complete. 2024-09-21T00:30:02.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.777+0000 7f382db77640 1 Processor -- start 2024-09-21T00:30:02.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.777+0000 7f382db77640 1 -- start start 2024-09-21T00:30:02.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.778+0000 7fce0e951640 1 Processor -- start 2024-09-21T00:30:02.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.778+0000 7f79de4f2640 1 Processor -- start 2024-09-21T00:30:02.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.778+0000 7f79de4f2640 1 -- start start 2024-09-21T00:30:02.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.778+0000 7f0f0a43f640 1 Processor -- start 2024-09-21T00:30:02.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.778+0000 7fce0e951640 1 -- start start 2024-09-21T00:30:02.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.778+0000 7f0f0a43f640 1 -- start start 2024-09-21T00:30:02.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.779+0000 7f382db77640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38280b4bc0 0x7f38281f4020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:02.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.779+0000 7f382db77640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f38280b6700 0x7f38281555f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:02.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.779+0000 7f382db77640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f38281626b0 0x7f3828162ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:02.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.779+0000 7f382db77640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f38280b5dd0 con 0x7f38280b4bc0 2024-09-21T00:30:02.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.779+0000 7f382db77640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f38280b5c50 con 0x7f38280b6700 2024-09-21T00:30:02.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.779+0000 7f382db77640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f38281670b0 con 0x7f38281626b0 2024-09-21T00:30:02.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.779+0000 7f38277fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38280b4bc0 0x7f38281f4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:02.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.779+0000 7f38277fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38280b4bc0 0x7f38281f4020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40910/0 (socket says 172.21.15.112:40910) 2024-09-21T00:30:02.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.779+0000 7f38277fe640 1 -- 172.21.15.112:0/2731314693 learned_addr learned my addr 172.21.15.112:0/2731314693 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:30:02.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.779+0000 7f3826ffd640 1 --2- 172.21.15.112:0/2731314693 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f38280b6700 0x7f38281555f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:02.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.779+0000 7f3827fff640 1 --2- 172.21.15.112:0/2731314693 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f38281626b0 0x7f3828162ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:02.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.779+0000 7f38277fe640 1 -- 172.21.15.112:0/2731314693 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f38281626b0 msgr2=0x7f3828162ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:02.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.779+0000 7f79de4f2640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f79d80cc450 0x7f79d81f4870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:02.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.779+0000 7f79de4f2640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f79d80d0fc0 0x7f79d80b07d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:02.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.779+0000 7f38277fe640 1 --2- 172.21.15.112:0/2731314693 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f38281626b0 0x7f3828162ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:02.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.779+0000 7f38277fe640 1 -- 172.21.15.112:0/2731314693 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f38280b6700 msgr2=0x7f38281555f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:02.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.779+0000 7f38277fe640 1 --2- 172.21.15.112:0/2731314693 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f38280b6700 0x7f38281555f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:02.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.779+0000 7f38277fe640 1 -- 172.21.15.112:0/2731314693 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3828163000 con 0x7f38280b4bc0 2024-09-21T00:30:02.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.779+0000 7f3826ffd640 1 --2- 172.21.15.112:0/2731314693 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f38280b6700 0x7f38281555f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T00:30:02.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.780+0000 7f79de4f2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f79d80b2550 0x7f79d8073ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:02.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.780+0000 7f38277fe640 1 --2- 172.21.15.112:0/2731314693 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38280b4bc0 0x7f38281f4020 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f381800c3c0 tx=0x7f381800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:02.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.780+0000 7f79d77fe640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f79d80d0fc0 0x7f79d80b07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:02.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.780+0000 7f79de4f2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f79d806f630 con 0x7f79d80b2550 2024-09-21T00:30:02.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.780+0000 7f79de4f2640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f79d806f4b0 con 0x7f79d80cc450 2024-09-21T00:30:02.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.780+0000 7f79de4f2640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f79d806f7b0 con 0x7f79d80d0fc0 2024-09-21T00:30:02.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.780+0000 7fce0e951640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce080cf970 0x7fce08158480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:02.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.780+0000 7f79d77fe640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f79d80d0fc0 0x7f79d80b07d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:55580/0 (socket says 172.21.15.112:55580) 2024-09-21T00:30:02.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.780+0000 7f79d77fe640 1 -- 172.21.15.112:0/3744253193 learned_addr learned my addr 172.21.15.112:0/3744253193 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:30:02.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.780+0000 7f79d7fff640 1 --2- 172.21.15.112:0/3744253193 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f79d80cc450 0x7f79d81f4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:02.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.780+0000 7f0f0a43f640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0f040b64e0 0x7f0f041f9f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:02.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.780+0000 7f0f0a43f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f040cdd00 0x7f0f04165520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:02.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.780+0000 7f380ffff640 1 -- 172.21.15.112:0/2731314693 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3818017070 con 0x7f38280b4bc0 2024-09-21T00:30:02.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.780+0000 7f79dca8f640 1 --2- 172.21.15.112:0/3744253193 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f79d80b2550 0x7f79d8073ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:02.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.780+0000 7f382db77640 1 -- 172.21.15.112:0/2731314693 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38281f6e80 con 0x7f38280b4bc0 2024-09-21T00:30:02.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.780+0000 7fce0e951640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fce080cfd80 0x7fce081f4b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:02.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.780+0000 7f0f0a43f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0f040cf2d0 0x7f0f0415f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:02.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.780+0000 7fce0e951640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fce0815f100 0x7fce081f50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:02.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.780+0000 7f0f0a43f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0f040ce820 con 0x7f0f040cdd00 2024-09-21T00:30:02.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.780+0000 7f0f0a43f640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0f040b54c0 con 0x7f0f040cf2d0 2024-09-21T00:30:02.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.780+0000 7f0f0a43f640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f0f040ce9a0 con 0x7f0f040b64e0 2024-09-21T00:30:02.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.780+0000 7f0f03fff640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0f040b64e0 0x7f0f041f9f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:02.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.780+0000 7f79d77fe640 1 -- 172.21.15.112:0/3744253193 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f79d80cc450 msgr2=0x7f79d81f4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:02.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.780+0000 7f79d77fe640 1 --2- 172.21.15.112:0/3744253193 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f79d80cc450 0x7f79d81f4870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:02.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.780+0000 7f79d77fe640 1 -- 172.21.15.112:0/3744253193 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f79d80b2550 msgr2=0x7f79d8073ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:02.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.780+0000 7f79d77fe640 1 --2- 172.21.15.112:0/3744253193 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f79d80b2550 0x7f79d8073ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:02.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.780+0000 7f0f03fff640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0f040b64e0 0x7f0f041f9f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:55588/0 (socket says 172.21.15.112:55588) 2024-09-21T00:30:02.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.780+0000 7f380ffff640 1 -- 172.21.15.112:0/2731314693 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f38180035f0 con 0x7f38280b4bc0 2024-09-21T00:30:02.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.780+0000 7f382db77640 1 -- 172.21.15.112:0/2731314693 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38281f7440 con 0x7f38280b4bc0 2024-09-21T00:30:02.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.780+0000 7f380ffff640 1 -- 172.21.15.112:0/2731314693 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3818003a20 con 0x7f38280b4bc0 2024-09-21T00:30:02.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.780+0000 7f79d77fe640 1 -- 172.21.15.112:0/3744253193 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79d81f4db0 con 0x7f79d80d0fc0 2024-09-21T00:30:02.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.780+0000 7f0f03fff640 1 -- 172.21.15.112:0/1320105834 learned_addr learned my addr 172.21.15.112:0/1320105834 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:30:02.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.780+0000 7f79dca8f640 1 --2- 172.21.15.112:0/3744253193 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f79d80b2550 0x7f79d8073ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:30:02.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.780+0000 7f0f089dc640 1 --2- 172.21.15.112:0/1320105834 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0f040cf2d0 0x7f0f0415f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:02.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.781+0000 7f0f037fe640 1 --2- 172.21.15.112:0/1320105834 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f040cdd00 0x7f0f04165520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:02.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.781+0000 7f79d77fe640 1 --2- 172.21.15.112:0/3744253193 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f79d80d0fc0 0x7f79d80b07d0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f79cc00c460 tx=0x7f79cc00c920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:02.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.781+0000 7f0f089dc640 1 -- 172.21.15.112:0/1320105834 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0f040b64e0 msgr2=0x7f0f041f9f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:02.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.781+0000 7f0f089dc640 1 --2- 172.21.15.112:0/1320105834 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0f040b64e0 0x7f0f041f9f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:02.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.781+0000 7f0f089dc640 1 -- 172.21.15.112:0/1320105834 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f040cdd00 msgr2=0x7f0f04165520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:02.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.781+0000 7f0f089dc640 1 --2- 172.21.15.112:0/1320105834 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f040cdd00 0x7f0f04165520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:02.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.781+0000 7f0f089dc640 1 -- 172.21.15.112:0/1320105834 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f041f64a0 con 0x7f0f040cf2d0 2024-09-21T00:30:02.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.781+0000 7f79d4ff9640 1 -- 172.21.15.112:0/3744253193 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79cc016020 con 0x7f79d80d0fc0 2024-09-21T00:30:02.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.780+0000 7fce07fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce080cf970 0x7fce08158480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:02.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.780+0000 7fce07fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce080cf970 0x7fce08158480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40928/0 (socket says 172.21.15.112:40928) 2024-09-21T00:30:02.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.780+0000 7fce07fff640 1 -- 172.21.15.112:0/123440946 learned_addr learned my addr 172.21.15.112:0/123440946 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:30:02.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.780+0000 7fce0e951640 1 -- 172.21.15.112:0/123440946 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fce08152100 con 0x7fce080cf970 2024-09-21T00:30:02.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.780+0000 7fce0e951640 1 -- 172.21.15.112:0/123440946 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fce08070ba0 con 0x7fce0815f100 2024-09-21T00:30:02.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.780+0000 7fce0e951640 1 -- 172.21.15.112:0/123440946 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fce08152280 con 0x7fce080cfd80 2024-09-21T00:30:02.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.780+0000 7fce0ceee640 1 --2- 172.21.15.112:0/123440946 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fce0815f100 0x7fce081f50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:02.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.781+0000 7fce077fe640 1 --2- 172.21.15.112:0/123440946 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fce080cfd80 0x7fce081f4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:02.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.781+0000 7fce07fff640 1 -- 172.21.15.112:0/123440946 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fce080cfd80 msgr2=0x7fce081f4b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:02.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.781+0000 7fce07fff640 1 --2- 172.21.15.112:0/123440946 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fce080cfd80 0x7fce081f4b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:02.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.781+0000 7fce07fff640 1 -- 172.21.15.112:0/123440946 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fce0815f100 msgr2=0x7fce081f50c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:02.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.781+0000 7fce07fff640 1 --2- 172.21.15.112:0/123440946 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fce0815f100 0x7fce081f50c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:02.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.781+0000 7fce07fff640 1 -- 172.21.15.112:0/123440946 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce080b1d00 con 0x7fce080cf970 2024-09-21T00:30:02.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.781+0000 7f79de4f2640 1 -- 172.21.15.112:0/3744253193 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79d80afca0 con 0x7f79d80d0fc0 2024-09-21T00:30:02.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.781+0000 7f79d4ff9640 1 -- 172.21.15.112:0/3744253193 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f79cc012940 con 0x7f79d80d0fc0 2024-09-21T00:30:02.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.781+0000 7f0f089dc640 1 --2- 172.21.15.112:0/1320105834 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0f040cf2d0 0x7f0f0415f1e0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f0f040cb9d0 tx=0x7f0efc009ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:02.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.781+0000 7fce07fff640 1 --2- 172.21.15.112:0/123440946 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce080cf970 0x7fce08158480 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fce0000c530 tx=0x7fce0000c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:02.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.781+0000 7f79de4f2640 1 -- 172.21.15.112:0/3744253193 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f79d80c75c0 con 0x7f79d80d0fc0 2024-09-21T00:30:02.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.781+0000 7f0f00ff9640 1 -- 172.21.15.112:0/1320105834 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0efc019070 con 0x7f0f040cf2d0 2024-09-21T00:30:02.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.781+0000 7f0f00ff9640 1 -- 172.21.15.112:0/1320105834 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0efc014440 con 0x7f0f040cf2d0 2024-09-21T00:30:02.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.781+0000 7f0f00ff9640 1 -- 172.21.15.112:0/1320105834 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0efc0148d0 con 0x7f0f040cf2d0 2024-09-21T00:30:02.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.781+0000 7f0f0a43f640 1 -- 172.21.15.112:0/1320105834 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f041f6780 con 0x7f0f040cf2d0 2024-09-21T00:30:02.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.781+0000 7fce04ff9640 1 -- 172.21.15.112:0/123440946 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce00027020 con 0x7fce080cf970 2024-09-21T00:30:02.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.781+0000 7fce04ff9640 1 -- 172.21.15.112:0/123440946 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fce00023730 con 0x7fce080cf970 2024-09-21T00:30:02.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.782+0000 7fce04ff9640 1 -- 172.21.15.112:0/123440946 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce00023b40 con 0x7fce080cf970 2024-09-21T00:30:02.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.781+0000 7f79d4ff9640 1 -- 172.21.15.112:0/3744253193 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79cc012d90 con 0x7f79d80d0fc0 2024-09-21T00:30:02.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.782+0000 7fce0e951640 1 -- 172.21.15.112:0/123440946 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce080b1fe0 con 0x7fce080cf970 2024-09-21T00:30:02.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.782+0000 7f0f0a43f640 1 -- 172.21.15.112:0/1320105834 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f041f6d40 con 0x7f0f040cf2d0 2024-09-21T00:30:02.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.782+0000 7f382db77640 1 -- 172.21.15.112:0/2731314693 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3808005290 con 0x7f38280b4bc0 2024-09-21T00:30:02.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.782+0000 7fce0e951640 1 -- 172.21.15.112:0/123440946 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce080b22c0 con 0x7fce080cf970 2024-09-21T00:30:02.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.783+0000 7fce0e951640 1 -- 172.21.15.112:0/123440946 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce0815ebc0 con 0x7fce080cf970 2024-09-21T00:30:02.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.784+0000 7f0f0a43f640 1 -- 172.21.15.112:0/1320105834 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f040cf6e0 con 0x7f0f040cf2d0 2024-09-21T00:30:02.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.784+0000 7f79de4f2640 1 -- 172.21.15.112:0/3744253193 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79b8005290 con 0x7f79d80d0fc0 2024-09-21T00:30:02.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.784+0000 7f79d4ff9640 1 -- 172.21.15.112:0/3744253193 <== mon.2 v2:172.21.15.187:3301/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f79cc019020 con 0x7f79d80d0fc0 2024-09-21T00:30:02.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.784+0000 7f380ffff640 1 -- 172.21.15.112:0/2731314693 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f381801a070 con 0x7f38280b4bc0 2024-09-21T00:30:02.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.784+0000 7f0f00ff9640 1 -- 172.21.15.112:0/1320105834 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0efc01c070 con 0x7f0f040cf2d0 2024-09-21T00:30:02.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.784+0000 7fce04ff9640 1 -- 172.21.15.112:0/123440946 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fce0002a020 con 0x7fce080cf970 2024-09-21T00:30:02.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.784+0000 7f79d4ff9640 1 --2- 172.21.15.112:0/3744253193 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f79c4078b90 0x7f79c407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:02.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.784+0000 7f380ffff640 1 --2- 172.21.15.112:0/2731314693 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f3814078b90 0x7f381407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:02.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.785+0000 7f79d4ff9640 1 -- 172.21.15.112:0/3744253193 <== mon.2 v2:172.21.15.187:3301/0 5 ==== osd_map(108..108 src has 1..108) ==== 6101+0+0 (secure 0 0 0) 0x7f79cc09aad0 con 0x7f79d80d0fc0 2024-09-21T00:30:02.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.785+0000 7f79d7fff640 1 --2- 172.21.15.112:0/3744253193 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f79c4078b90 0x7f79c407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:02.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.785+0000 7f0f00ff9640 1 --2- 172.21.15.112:0/1320105834 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f0ef0078c10 0x7f0ef007b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:02.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.785+0000 7f3826ffd640 1 --2- 172.21.15.112:0/2731314693 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f3814078b90 0x7f381407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:02.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.785+0000 7f0f03fff640 1 --2- 172.21.15.112:0/1320105834 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f0ef0078c10 0x7f0ef007b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:02.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.785+0000 7f0f00ff9640 1 -- 172.21.15.112:0/1320105834 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 6101+0+0 (secure 0 0 0) 0x7f0efc09ad90 con 0x7f0f040cf2d0 2024-09-21T00:30:02.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.785+0000 7f380ffff640 1 -- 172.21.15.112:0/2731314693 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 6101+0+0 (secure 0 0 0) 0x7f3818099fc0 con 0x7f38280b4bc0 2024-09-21T00:30:02.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.785+0000 7fce04ff9640 1 --2- 172.21.15.112:0/123440946 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fcdec078be0 0x7fcdec07b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:02.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.785+0000 7fce04ff9640 1 -- 172.21.15.112:0/123440946 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 6101+0+0 (secure 0 0 0) 0x7fce000aba60 con 0x7fce080cf970 2024-09-21T00:30:02.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.785+0000 7f79d7fff640 1 --2- 172.21.15.112:0/3744253193 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f79c4078b90 0x7f79c407b0a0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f79c8009ac0 tx=0x7f79c8005f90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:02.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.785+0000 7f3826ffd640 1 --2- 172.21.15.112:0/2731314693 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f3814078b90 0x7f381407b0a0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f381c033000 tx=0x7f381c004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:02.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.785+0000 7fce077fe640 1 --2- 172.21.15.112:0/123440946 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fcdec078be0 0x7fcdec07b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:02.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.785+0000 7f0f03fff640 1 --2- 172.21.15.112:0/1320105834 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f0ef0078c10 0x7f0ef007b120 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f0ef4009670 tx=0x7f0ef4005f90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:02.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.786+0000 7fce077fe640 1 --2- 172.21.15.112:0/123440946 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fcdec078be0 0x7fcdec07b0f0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fce080adb00 tx=0x7fcdf000d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:02.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.789+0000 7f79d4ff9640 1 -- 172.21.15.112:0/3744253193 <== mon.2 v2:172.21.15.187:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f79cc020030 con 0x7f79d80d0fc0 2024-09-21T00:30:02.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.789+0000 7f380ffff640 1 -- 172.21.15.112:0/2731314693 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3828163000 con 0x7f38280b4bc0 2024-09-21T00:30:02.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.790+0000 7f0f00ff9640 1 -- 172.21.15.112:0/1320105834 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f0efc062b60 con 0x7f0f040cf2d0 2024-09-21T00:30:02.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:01.798+0000 7fce04ff9640 1 -- 172.21.15.112:0/123440946 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fce00031020 con 0x7fce080cf970 2024-09-21T00:30:02.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:02.053+0000 7f382db77640 1 -- 172.21.15.112:0/2731314693 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "b"} v 0) -- 0x7f3808005850 con 0x7f38280b4bc0 2024-09-21T00:30:02.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:02.059+0000 7f79de4f2640 1 -- 172.21.15.112:0/3744253193 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "a"} v 0) -- 0x7f79b8005580 con 0x7f79d80d0fc0 2024-09-21T00:30:02.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:02.087+0000 7f0f0a43f640 1 -- 172.21.15.112:0/1320105834 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "c"} v 0) -- 0x7f0f040ce360 con 0x7f0f040cf2d0 2024-09-21T00:30:02.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:02.095+0000 7fce0e951640 1 -- 172.21.15.112:0/123440946 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "d"} v 0) -- 0x7fce080705e0 con 0x7fce080cf970 2024-09-21T00:30:02.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:02.227+0000 7f380ffff640 1 -- 172.21.15.112:0/2731314693 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "b"}]=0 failed mds gid 6472 v71) ==== 95+0+0 (secure 0 0 0) 0x7f38280ce7a0 con 0x7f38280b4bc0 2024-09-21T00:30:02.233 INFO:teuthology.orchestra.run.smithi112.stderr:failed mds gid 6472 2024-09-21T00:30:02.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:02.229+0000 7f382db77640 1 -- 172.21.15.112:0/2731314693 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f3814078b90 msgr2=0x7f381407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:02.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:02.229+0000 7f382db77640 1 --2- 172.21.15.112:0/2731314693 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f3814078b90 0x7f381407b0a0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f381c033000 tx=0x7f381c004730 comp rx=0 tx=0).stop 2024-09-21T00:30:02.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:02.229+0000 7f382db77640 1 -- 172.21.15.112:0/2731314693 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38280b4bc0 msgr2=0x7f38281f4020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:02.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:02.229+0000 7f382db77640 1 --2- 172.21.15.112:0/2731314693 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38280b4bc0 0x7f38281f4020 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f381800c3c0 tx=0x7f381800c880 comp rx=0 tx=0).stop 2024-09-21T00:30:02.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:02.229+0000 7f382db77640 1 -- 172.21.15.112:0/2731314693 shutdown_connections 2024-09-21T00:30:02.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:02.229+0000 7f382db77640 1 --2- 172.21.15.112:0/2731314693 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f38281626b0 0x7f3828162ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:02.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:02.229+0000 7f382db77640 1 --2- 172.21.15.112:0/2731314693 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f38280b6700 0x7f38281555f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:02.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:02.229+0000 7f382db77640 1 --2- 172.21.15.112:0/2731314693 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f3814078b90 0x7f381407b0a0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:02.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:02.229+0000 7f382db77640 1 --2- 172.21.15.112:0/2731314693 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38280b4bc0 0x7f38281f4020 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:02.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:02.229+0000 7f382db77640 1 -- 172.21.15.112:0/2731314693 >> 172.21.15.112:0/2731314693 conn(0x7f38280d13d0 msgr2=0x7f38280b4fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:02.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:02.229+0000 7f382db77640 1 -- 172.21.15.112:0/2731314693 shutdown_connections 2024-09-21T00:30:02.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:02.229+0000 7f382db77640 1 -- 172.21.15.112:0/2731314693 wait complete. 2024-09-21T00:30:02.243 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-21T00:30:02.244 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-21T00:30:02.244 INFO:tasks.ceph.mon.a.smithi112.stderr:2024-09-21T00:30:02.242+0000 7f765440e640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T00:30:02.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:02.246+0000 7fce04ff9640 1 -- 172.21.15.112:0/123440946 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "d"}]=0 failed mds gid 7167 v72) ==== 95+0+0 (secure 0 0 0) 0x7fce00073f10 con 0x7fce080cf970 2024-09-21T00:30:02.248 INFO:teuthology.orchestra.run.smithi112.stderr:failed mds gid 7167 2024-09-21T00:30:02.248 INFO:tasks.ceph.mds.b:Started 2024-09-21T00:30:02.249 DEBUG:teuthology.parallel:result is None 2024-09-21T00:30:02.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:02.248+0000 7fce0e951640 1 -- 172.21.15.112:0/123440946 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fcdec078be0 msgr2=0x7fcdec07b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:02.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:02.248+0000 7fce0e951640 1 --2- 172.21.15.112:0/123440946 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fcdec078be0 0x7fcdec07b0f0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fce080adb00 tx=0x7fcdf000d040 comp rx=0 tx=0).stop 2024-09-21T00:30:02.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:02.248+0000 7fce0e951640 1 -- 172.21.15.112:0/123440946 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce080cf970 msgr2=0x7fce08158480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:02.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:02.248+0000 7fce0e951640 1 --2- 172.21.15.112:0/123440946 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce080cf970 0x7fce08158480 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fce0000c530 tx=0x7fce0000c9f0 comp rx=0 tx=0).stop 2024-09-21T00:30:02.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:02.248+0000 7fce0e951640 1 -- 172.21.15.112:0/123440946 shutdown_connections 2024-09-21T00:30:02.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:02.248+0000 7fce0e951640 1 --2- 172.21.15.112:0/123440946 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fce0815f100 0x7fce081f50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:02.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:02.248+0000 7fce0e951640 1 --2- 172.21.15.112:0/123440946 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fce080cfd80 0x7fce081f4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:02.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:02.248+0000 7fce0e951640 1 --2- 172.21.15.112:0/123440946 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fcdec078be0 0x7fcdec07b0f0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:02.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:02.248+0000 7fce0e951640 1 --2- 172.21.15.112:0/123440946 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce080cf970 0x7fce08158480 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:02.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:02.248+0000 7fce0e951640 1 -- 172.21.15.112:0/123440946 >> 172.21.15.112:0/123440946 conn(0x7fce0806d4c0 msgr2=0x7fce0809d9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:02.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:02.248+0000 7fce0e951640 1 -- 172.21.15.112:0/123440946 shutdown_connections 2024-09-21T00:30:02.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:02.248+0000 7fce0e951640 1 -- 172.21.15.112:0/123440946 wait complete. 2024-09-21T00:30:02.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:02.249+0000 7f79d4ff9640 1 -- 172.21.15.112:0/3744253193 <== mon.2 v2:172.21.15.187:3301/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "a"}]=0 failed mds gid 7152 v72) ==== 95+0+0 (secure 0 0 0) 0x7f79d81f4db0 con 0x7f79d80d0fc0 2024-09-21T00:30:02.253 INFO:teuthology.orchestra.run.smithi112.stderr:failed mds gid 7152 2024-09-21T00:30:02.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:02.250+0000 7f0f00ff9640 1 -- 172.21.15.112:0/1320105834 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "c"}]=0 failed mds gid 7155 v72) ==== 95+0+0 (secure 0 0 0) 0x7f0efc067590 con 0x7f0f040cf2d0 2024-09-21T00:30:02.254 INFO:teuthology.orchestra.run.smithi112.stderr:failed mds gid 7155 2024-09-21T00:30:02.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:02.251+0000 7f79de4f2640 1 -- 172.21.15.112:0/3744253193 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f79c4078b90 msgr2=0x7f79c407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:02.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:02.251+0000 7f79de4f2640 1 --2- 172.21.15.112:0/3744253193 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f79c4078b90 0x7f79c407b0a0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f79c8009ac0 tx=0x7f79c8005f90 comp rx=0 tx=0).stop 2024-09-21T00:30:02.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:02.251+0000 7f79de4f2640 1 -- 172.21.15.112:0/3744253193 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f79d80d0fc0 msgr2=0x7f79d80b07d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:02.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:02.251+0000 7f79de4f2640 1 --2- 172.21.15.112:0/3744253193 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f79d80d0fc0 0x7f79d80b07d0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f79cc00c460 tx=0x7f79cc00c920 comp rx=0 tx=0).stop 2024-09-21T00:30:02.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:02.252+0000 7f79de4f2640 1 -- 172.21.15.112:0/3744253193 shutdown_connections 2024-09-21T00:30:02.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:02.252+0000 7f79de4f2640 1 --2- 172.21.15.112:0/3744253193 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f79c4078b90 0x7f79c407b0a0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:02.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:02.252+0000 7f79de4f2640 1 --2- 172.21.15.112:0/3744253193 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f79d80b2550 0x7f79d8073ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:02.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:02.252+0000 7f79de4f2640 1 --2- 172.21.15.112:0/3744253193 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f79d80d0fc0 0x7f79d80b07d0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:02.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:02.252+0000 7f79de4f2640 1 --2- 172.21.15.112:0/3744253193 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f79d80cc450 0x7f79d81f4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:02.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:02.252+0000 7f79de4f2640 1 -- 172.21.15.112:0/3744253193 >> 172.21.15.112:0/3744253193 conn(0x7f79d806d4c0 msgr2=0x7f79d8074800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:02.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:02.252+0000 7f79de4f2640 1 -- 172.21.15.112:0/3744253193 shutdown_connections 2024-09-21T00:30:02.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:02.252+0000 7f79de4f2640 1 -- 172.21.15.112:0/3744253193 wait complete. 2024-09-21T00:30:02.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:02.252+0000 7f0f0a43f640 1 -- 172.21.15.112:0/1320105834 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f0ef0078c10 msgr2=0x7f0ef007b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:02.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:02.252+0000 7f0f0a43f640 1 --2- 172.21.15.112:0/1320105834 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f0ef0078c10 0x7f0ef007b120 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f0ef4009670 tx=0x7f0ef4005f90 comp rx=0 tx=0).stop 2024-09-21T00:30:02.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:02.252+0000 7f0f0a43f640 1 -- 172.21.15.112:0/1320105834 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0f040cf2d0 msgr2=0x7f0f0415f1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:02.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:02.252+0000 7f0f0a43f640 1 --2- 172.21.15.112:0/1320105834 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0f040cf2d0 0x7f0f0415f1e0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f0f040cb9d0 tx=0x7f0efc009ed0 comp rx=0 tx=0).stop 2024-09-21T00:30:02.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:02.252+0000 7f0f0a43f640 1 -- 172.21.15.112:0/1320105834 shutdown_connections 2024-09-21T00:30:02.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:02.252+0000 7f0f0a43f640 1 --2- 172.21.15.112:0/1320105834 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0f040cf2d0 0x7f0f0415f1e0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:02.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:02.252+0000 7f0f0a43f640 1 --2- 172.21.15.112:0/1320105834 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f0ef0078c10 0x7f0ef007b120 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:02.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:02.252+0000 7f0f0a43f640 1 --2- 172.21.15.112:0/1320105834 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f040cdd00 0x7f0f04165520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:02.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:02.252+0000 7f0f0a43f640 1 --2- 172.21.15.112:0/1320105834 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0f040b64e0 0x7f0f041f9f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:02.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:02.252+0000 7f0f0a43f640 1 -- 172.21.15.112:0/1320105834 >> 172.21.15.112:0/1320105834 conn(0x7f0f040b2fa0 msgr2=0x7f0f04159a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:02.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:02.253+0000 7f0f0a43f640 1 -- 172.21.15.112:0/1320105834 shutdown_connections 2024-09-21T00:30:02.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:02.253+0000 7f0f0a43f640 1 -- 172.21.15.112:0/1320105834 wait complete. 2024-09-21T00:30:02.264 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-21T00:30:02.264 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-21T00:30:02.266 INFO:tasks.ceph.mds.d:Started 2024-09-21T00:30:02.267 DEBUG:teuthology.parallel:result is None 2024-09-21T00:30:02.268 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-21T00:30:02.268 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-21T00:30:02.269 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-21T00:30:02.269 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-21T00:30:02.311 INFO:tasks.ceph.mds.a:Started 2024-09-21T00:30:02.312 DEBUG:teuthology.parallel:result is None 2024-09-21T00:30:02.312 INFO:tasks.ceph.mds.c:Started 2024-09-21T00:30:02.312 DEBUG:teuthology.parallel:result is None 2024-09-21T00:30:02.313 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T00:30:02.337 INFO:tasks.ceph.mds.b.smithi187.stdout:starting mds.b at 2024-09-21T00:30:02.354 INFO:tasks.ceph.mds.d.smithi187.stdout:starting mds.d at 2024-09-21T00:30:02.400 INFO:tasks.ceph.mds.a.smithi112.stdout:starting mds.a at 2024-09-21T00:30:02.400 INFO:tasks.ceph.mds.c.smithi112.stdout:starting mds.c at 2024-09-21T00:30:02.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:02.430+0000 7f621132b640 1 -- 172.21.15.112:0/1881678474 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f620c0b64e0 msgr2=0x7f620c0b3ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:02.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:02.430+0000 7f621132b640 1 --2- 172.21.15.112:0/1881678474 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f620c0b64e0 0x7f620c0b3ac0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f61fc009860 tx=0x7f61fc02a5d0 comp rx=0 tx=0).stop 2024-09-21T00:30:02.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:02.430+0000 7f621132b640 1 -- 172.21.15.112:0/1881678474 shutdown_connections 2024-09-21T00:30:02.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:02.430+0000 7f621132b640 1 --2- 172.21.15.112:0/1881678474 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f620c0cf2d0 0x7f620c0cf6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:02.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:02.430+0000 7f621132b640 1 --2- 172.21.15.112:0/1881678474 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f620c0b64e0 0x7f620c0b3ac0 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:02.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:02.430+0000 7f621132b640 1 -- 172.21.15.112:0/1881678474 >> 172.21.15.112:0/1881678474 conn(0x7f620c0b2fa0 msgr2=0x7f620c0b33b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:02.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:02.430+0000 7f621132b640 1 -- 172.21.15.112:0/1881678474 shutdown_connections 2024-09-21T00:30:02.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:02.430+0000 7f621132b640 1 -- 172.21.15.112:0/1881678474 wait complete. 2024-09-21T00:30:02.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:02.432+0000 7f621132b640 1 Processor -- start 2024-09-21T00:30:02.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:02.432+0000 7f621132b640 1 -- start start 2024-09-21T00:30:02.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:02.433+0000 7f621132b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f620c0b64e0 0x7f620c0afba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:02.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:02.433+0000 7f621132b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f620c0cdd00 0x7f620c1525c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:02.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:02.433+0000 7f621132b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f620c0cf2d0 0x7f620c153480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:02.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:02.433+0000 7f621132b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f620c0ce820 con 0x7f620c0cdd00 2024-09-21T00:30:02.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:02.433+0000 7f621132b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f620c0b54c0 con 0x7f620c0cf2d0 2024-09-21T00:30:02.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:02.433+0000 7f621132b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f620c0ce9a0 con 0x7f620c0b64e0 2024-09-21T00:30:02.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:02.433+0000 7f620a7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f620c0cdd00 0x7f620c1525c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:02.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:02.433+0000 7f620a7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f620c0cdd00 0x7f620c1525c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40970/0 (socket says 172.21.15.112:40970) 2024-09-21T00:30:02.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:02.433+0000 7f620a7fc640 1 -- 172.21.15.112:0/2631508890 learned_addr learned my addr 172.21.15.112:0/2631508890 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:30:02.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:02.433+0000 7f620affd640 1 --2- 172.21.15.112:0/2631508890 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f620c0b64e0 0x7f620c0afba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:02.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:02.433+0000 7f620b7fe640 1 --2- 172.21.15.112:0/2631508890 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f620c0cf2d0 0x7f620c153480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:02.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:02.434+0000 7f620a7fc640 1 -- 172.21.15.112:0/2631508890 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f620c0b64e0 msgr2=0x7f620c0afba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:02.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:02.434+0000 7f620a7fc640 1 --2- 172.21.15.112:0/2631508890 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f620c0b64e0 0x7f620c0afba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:02.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:02.434+0000 7f620a7fc640 1 -- 172.21.15.112:0/2631508890 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f620c0cf2d0 msgr2=0x7f620c153480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:02.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:02.434+0000 7f620a7fc640 1 --2- 172.21.15.112:0/2631508890 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f620c0cf2d0 0x7f620c153480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:02.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:02.434+0000 7f620a7fc640 1 -- 172.21.15.112:0/2631508890 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f620c158840 con 0x7f620c0cdd00 2024-09-21T00:30:02.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:02.434+0000 7f620affd640 1 --2- 172.21.15.112:0/2631508890 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f620c0b64e0 0x7f620c0afba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:30:02.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:02.434+0000 7f620a7fc640 1 --2- 172.21.15.112:0/2631508890 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f620c0cdd00 0x7f620c1525c0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f620000e6d0 tx=0x7f620000eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:02.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:02.434+0000 7f61f37fe640 1 -- 172.21.15.112:0/2631508890 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6200004220 con 0x7f620c0cdd00 2024-09-21T00:30:02.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:02.434+0000 7f61f37fe640 1 -- 172.21.15.112:0/2631508890 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f62000043c0 con 0x7f620c0cdd00 2024-09-21T00:30:02.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:02.434+0000 7f61f37fe640 1 -- 172.21.15.112:0/2631508890 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f620000cb10 con 0x7f620c0cdd00 2024-09-21T00:30:02.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:02.435+0000 7f621132b640 1 -- 172.21.15.112:0/2631508890 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f620c158b20 con 0x7f620c0cdd00 2024-09-21T00:30:02.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:02.435+0000 7f621132b640 1 -- 172.21.15.112:0/2631508890 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f620c1590e0 con 0x7f620c0cdd00 2024-09-21T00:30:02.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:02.436+0000 7f621132b640 1 -- 172.21.15.112:0/2631508890 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f620c0cf6e0 con 0x7f620c0cdd00 2024-09-21T00:30:02.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:02.436+0000 7f61f37fe640 1 -- 172.21.15.112:0/2631508890 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f620000ccb0 con 0x7f620c0cdd00 2024-09-21T00:30:02.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:02.437+0000 7f61f37fe640 1 --2- 172.21.15.112:0/2631508890 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f61f8078bc0 0x7f61f807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:02.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:02.437+0000 7f61f37fe640 1 -- 172.21.15.112:0/2631508890 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 6187+0+0 (secure 0 0 0) 0x7f620009b790 con 0x7f620c0cdd00 2024-09-21T00:30:02.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:02.437+0000 7f620affd640 1 --2- 172.21.15.112:0/2631508890 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f61f8078bc0 0x7f61f807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:02.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:02.437+0000 7f620affd640 1 --2- 172.21.15.112:0/2631508890 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f61f8078bc0 0x7f61f807b0d0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f61fc034000 tx=0x7f61fc004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:02.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:02.441+0000 7f61f37fe640 1 -- 172.21.15.112:0/2631508890 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f6200063970 con 0x7f620c0cdd00 2024-09-21T00:30:02.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:02.661+0000 7f621132b640 1 -- 172.21.15.112:0/2631508890 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f620c15a770 con 0x7f620c0cdd00 2024-09-21T00:30:02.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:02.662+0000 7f61f37fe640 1 -- 172.21.15.112:0/2631508890 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) ==== 94+0+5229 (secure 0 0 0) 0x7f6200063270 con 0x7f620c0cdd00 2024-09-21T00:30:02.664 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:30:02.664 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":73,"btime":"2024-09-21T00:30:02:608297+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6113,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/3217596137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":3217596137},{"type":"v1","addr":"172.21.15.187:6835","nonce":3217596137}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":73},{"gid":6649,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6837/776068904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":776068904},{"type":"v1","addr":"172.21.15.187:6837","nonce":776068904}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":73},{"gid":7419,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6834/3291181711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":3291181711},{"type":"v1","addr":"172.21.15.112:6834","nonce":3291181711}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":73}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T00:29:48.456672+0000","modified":"2024-09-21T00:30:02.608280+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":112,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7422":{"gid":7422,"name":"c","rank":0,"incarnation":73,"state":"up:replay","state_seq":1,"addr":"172.21.15.112:6835/3392867613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":3392867613},{"type":"v1","addr":"172.21.15.112:6835","nonce":3392867613}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":8}]} 2024-09-21T00:30:02.664 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 73 2024-09-21T00:30:02.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:02.664+0000 7f621132b640 1 -- 172.21.15.112:0/2631508890 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f61f8078bc0 msgr2=0x7f61f807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:02.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:02.664+0000 7f621132b640 1 --2- 172.21.15.112:0/2631508890 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f61f8078bc0 0x7f61f807b0d0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f61fc034000 tx=0x7f61fc004730 comp rx=0 tx=0).stop 2024-09-21T00:30:02.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:02.664+0000 7f621132b640 1 -- 172.21.15.112:0/2631508890 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f620c0cdd00 msgr2=0x7f620c1525c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:02.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:02.664+0000 7f621132b640 1 --2- 172.21.15.112:0/2631508890 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f620c0cdd00 0x7f620c1525c0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f620000e6d0 tx=0x7f620000eb90 comp rx=0 tx=0).stop 2024-09-21T00:30:02.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:02.664+0000 7f621132b640 1 -- 172.21.15.112:0/2631508890 shutdown_connections 2024-09-21T00:30:02.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:02.664+0000 7f621132b640 1 --2- 172.21.15.112:0/2631508890 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f620c0cf2d0 0x7f620c153480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:02.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:02.664+0000 7f621132b640 1 --2- 172.21.15.112:0/2631508890 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f61f8078bc0 0x7f61f807b0d0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:02.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:02.664+0000 7f621132b640 1 --2- 172.21.15.112:0/2631508890 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f620c0cdd00 0x7f620c1525c0 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:02.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:02.664+0000 7f621132b640 1 --2- 172.21.15.112:0/2631508890 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f620c0b64e0 0x7f620c0afba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:02.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:02.664+0000 7f621132b640 1 -- 172.21.15.112:0/2631508890 >> 172.21.15.112:0/2631508890 conn(0x7f620c0b2fa0 msgr2=0x7f620c0ce110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:02.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:02.664+0000 7f621132b640 1 -- 172.21.15.112:0/2631508890 shutdown_connections 2024-09-21T00:30:02.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:02.664+0000 7f621132b640 1 -- 172.21.15.112:0/2631508890 wait complete. 2024-09-21T00:30:03.080 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 73, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T00:29:48.456672+0000', 'modified': '2024-09-21T00:30:02.608280+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 112, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7422}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7422': {'gid': 7422, 'name': 'c', 'rank': 0, 'incarnation': 73, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.112:6835/3392867613', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6833', 'nonce': 3392867613}, {'type': 'v1', 'addr': '172.21.15.112:6835', 'nonce': 3392867613}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-21T00:30:03.080 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7422:up:replay 2024-09-21T00:30:04.081 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T00:30:04.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:04.207+0000 7f41b4ade640 1 -- 172.21.15.112:0/472251379 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f419c004800 msgr2=0x7f419c004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:04.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:04.207+0000 7f41b4ade640 1 --2- 172.21.15.112:0/472251379 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f419c004800 0x7f419c004c10 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f41b00b6f30 tx=0x7f419c005830 comp rx=0 tx=0).stop 2024-09-21T00:30:04.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:04.207+0000 7f41aed9d640 1 -- 172.21.15.112:0/472251379 reap_dead start 2024-09-21T00:30:04.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:04.207+0000 7f41b4ade640 1 -- 172.21.15.112:0/472251379 shutdown_connections 2024-09-21T00:30:04.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:04.207+0000 7f41b4ade640 1 -- 172.21.15.112:0/472251379 >> 172.21.15.112:0/472251379 conn(0x7f41b00b2fa0 msgr2=0x7f41b00b33b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:04.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:04.207+0000 7f41b4ade640 1 -- 172.21.15.112:0/472251379 shutdown_connections 2024-09-21T00:30:04.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:04.207+0000 7f41b4ade640 1 -- 172.21.15.112:0/472251379 wait complete. 2024-09-21T00:30:04.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:04.209+0000 7f41b4ade640 1 Processor -- start 2024-09-21T00:30:04.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:04.209+0000 7f41b4ade640 1 -- start start 2024-09-21T00:30:04.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:04.210+0000 7f41b4ade640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f419c004800 0x7f41b009db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:04.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:04.210+0000 7f41b4ade640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41b009ecc0 0x7f41b015ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:04.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:04.210+0000 7f41b4ade640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f41b006cde0 0x7f41b006d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:04.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:04.210+0000 7f41b4ade640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f41b00b3e50 con 0x7f41b009ecc0 2024-09-21T00:30:04.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:04.210+0000 7f41b4ade640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f41b00b3cd0 con 0x7f419c004800 2024-09-21T00:30:04.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:04.210+0000 7f41b4ade640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f41b00cfc00 con 0x7f41b006cde0 2024-09-21T00:30:04.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:04.211+0000 7f41a5d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41b009ecc0 0x7f41b015ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:04.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:04.211+0000 7f41a5d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41b009ecc0 0x7f41b015ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40988/0 (socket says 172.21.15.112:40988) 2024-09-21T00:30:04.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:04.211+0000 7f41a5d9b640 1 -- 172.21.15.112:0/1783035331 learned_addr learned my addr 172.21.15.112:0/1783035331 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:30:04.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:04.211+0000 7f41ae59c640 1 --2- 172.21.15.112:0/1783035331 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f419c004800 0x7f41b009db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:04.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:04.211+0000 7f41aed9d640 1 --2- 172.21.15.112:0/1783035331 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f41b006cde0 0x7f41b006d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:04.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:04.211+0000 7f41a5d9b640 1 -- 172.21.15.112:0/1783035331 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f41b006cde0 msgr2=0x7f41b006d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:04.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:04.211+0000 7f41a5d9b640 1 --2- 172.21.15.112:0/1783035331 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f41b006cde0 0x7f41b006d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:04.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:04.211+0000 7f41a5d9b640 1 -- 172.21.15.112:0/1783035331 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f419c004800 msgr2=0x7f41b009db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:04.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:04.211+0000 7f41a5d9b640 1 --2- 172.21.15.112:0/1783035331 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f419c004800 0x7f41b009db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:04.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:04.211+0000 7f41a5d9b640 1 -- 172.21.15.112:0/1783035331 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41b006d730 con 0x7f41b009ecc0 2024-09-21T00:30:04.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:04.211+0000 7f41a5d9b640 1 --2- 172.21.15.112:0/1783035331 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41b009ecc0 0x7f41b015ab00 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f4198036040 tx=0x7f4198033670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:04.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:04.211+0000 7f41a77fe640 1 -- 172.21.15.112:0/1783035331 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4198038040 con 0x7f41b009ecc0 2024-09-21T00:30:04.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:04.211+0000 7f41a77fe640 1 -- 172.21.15.112:0/1783035331 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4198033860 con 0x7f41b009ecc0 2024-09-21T00:30:04.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:04.211+0000 7f41b4ade640 1 -- 172.21.15.112:0/1783035331 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41b009e350 con 0x7f41b009ecc0 2024-09-21T00:30:04.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:04.211+0000 7f41a77fe640 1 -- 172.21.15.112:0/1783035331 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4198033cb0 con 0x7f41b009ecc0 2024-09-21T00:30:04.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:04.211+0000 7f41b4ade640 1 -- 172.21.15.112:0/1783035331 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41b009e910 con 0x7f41b009ecc0 2024-09-21T00:30:04.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:04.213+0000 7f41b4ade640 1 -- 172.21.15.112:0/1783035331 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41b00b5150 con 0x7f41b009ecc0 2024-09-21T00:30:04.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:04.213+0000 7f41a77fe640 1 -- 172.21.15.112:0/1783035331 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f4198043020 con 0x7f41b009ecc0 2024-09-21T00:30:04.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:04.213+0000 7f41a77fe640 1 --2- 172.21.15.112:0/1783035331 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4190078b90 0x7f419007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:04.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:04.214+0000 7f41a77fe640 1 -- 172.21.15.112:0/1783035331 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 6187+0+0 (secure 0 0 0) 0x7f41980bb050 con 0x7f41b009ecc0 2024-09-21T00:30:04.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:04.214+0000 7f41ae59c640 1 --2- 172.21.15.112:0/1783035331 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4190078b90 0x7f419007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:04.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:04.214+0000 7f41ae59c640 1 --2- 172.21.15.112:0/1783035331 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4190078b90 0x7f419007b0a0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f41a0000fb0 tx=0x7f41a0006040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:04.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:04.218+0000 7f41a77fe640 1 -- 172.21.15.112:0/1783035331 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f419804a020 con 0x7f41b009ecc0 2024-09-21T00:30:04.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:04.432+0000 7f41b4ade640 1 -- 172.21.15.112:0/1783035331 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f41b015b040 con 0x7f41b009ecc0 2024-09-21T00:30:04.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:04.433+0000 7f41a77fe640 1 -- 172.21.15.112:0/1783035331 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) ==== 94+0+5232 (secure 0 0 0) 0x7f41b006d730 con 0x7f41b009ecc0 2024-09-21T00:30:04.435 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:30:04.435 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":74,"btime":"2024-09-21T00:30:03:610089+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6113,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/3217596137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":3217596137},{"type":"v1","addr":"172.21.15.187:6835","nonce":3217596137}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":73},{"gid":6649,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6837/776068904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":776068904},{"type":"v1","addr":"172.21.15.187:6837","nonce":776068904}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":73},{"gid":7419,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6834/3291181711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":3291181711},{"type":"v1","addr":"172.21.15.112:6834","nonce":3291181711}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":73}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T00:29:48.456672+0000","modified":"2024-09-21T00:30:02.623840+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":112,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7422":{"gid":7422,"name":"c","rank":0,"incarnation":73,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.112:6835/3392867613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":3392867613},{"type":"v1","addr":"172.21.15.112:6835","nonce":3392867613}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":8}]} 2024-09-21T00:30:04.435 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 74 2024-09-21T00:30:04.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:04.435+0000 7f41b4ade640 1 -- 172.21.15.112:0/1783035331 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4190078b90 msgr2=0x7f419007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:04.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:04.435+0000 7f41b4ade640 1 --2- 172.21.15.112:0/1783035331 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4190078b90 0x7f419007b0a0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f41a0000fb0 tx=0x7f41a0006040 comp rx=0 tx=0).stop 2024-09-21T00:30:04.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:04.435+0000 7f41b4ade640 1 -- 172.21.15.112:0/1783035331 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41b009ecc0 msgr2=0x7f41b015ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:04.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:04.435+0000 7f41b4ade640 1 --2- 172.21.15.112:0/1783035331 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41b009ecc0 0x7f41b015ab00 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f4198036040 tx=0x7f4198033670 comp rx=0 tx=0).stop 2024-09-21T00:30:04.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:04.435+0000 7f41b4ade640 1 -- 172.21.15.112:0/1783035331 shutdown_connections 2024-09-21T00:30:04.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:04.435+0000 7f41b4ade640 1 --2- 172.21.15.112:0/1783035331 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f41b006cde0 0x7f41b006d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:04.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:04.435+0000 7f41b4ade640 1 --2- 172.21.15.112:0/1783035331 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4190078b90 0x7f419007b0a0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:04.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:04.435+0000 7f41b4ade640 1 --2- 172.21.15.112:0/1783035331 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41b009ecc0 0x7f41b015ab00 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:04.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:04.435+0000 7f41b4ade640 1 --2- 172.21.15.112:0/1783035331 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f419c004800 0x7f41b009db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:04.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:04.435+0000 7f41b4ade640 1 -- 172.21.15.112:0/1783035331 >> 172.21.15.112:0/1783035331 conn(0x7f41b00b2fa0 msgr2=0x7f41b015b600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:04.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:04.435+0000 7f41b4ade640 1 -- 172.21.15.112:0/1783035331 shutdown_connections 2024-09-21T00:30:04.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:04.435+0000 7f41b4ade640 1 -- 172.21.15.112:0/1783035331 wait complete. 2024-09-21T00:30:04.852 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 74, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T00:29:48.456672+0000', 'modified': '2024-09-21T00:30:02.623840+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 112, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7422}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7422': {'gid': 7422, 'name': 'c', 'rank': 0, 'incarnation': 73, 'state': 'up:reconnect', 'state_seq': 2, 'addr': '172.21.15.112:6835/3392867613', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6833', 'nonce': 3392867613}, {'type': 'v1', 'addr': '172.21.15.112:6835', 'nonce': 3392867613}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-21T00:30:04.852 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7422:up:reconnect 2024-09-21T00:30:05.853 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T00:30:05.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:05.978+0000 7fab396fa640 1 -- 172.21.15.112:0/1204972310 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fab20004df0 msgr2=0x7fab200052c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:05.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:05.978+0000 7fab396fa640 1 --2- 172.21.15.112:0/1204972310 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fab20004df0 0x7fab200052c0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fab340a2460 tx=0x7fab20005830 comp rx=0 tx=0).stop 2024-09-21T00:30:05.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:05.978+0000 7fab337fe640 1 -- 172.21.15.112:0/1204972310 reap_dead start 2024-09-21T00:30:05.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:05.978+0000 7fab396fa640 1 -- 172.21.15.112:0/1204972310 shutdown_connections 2024-09-21T00:30:05.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:05.978+0000 7fab396fa640 1 -- 172.21.15.112:0/1204972310 >> 172.21.15.112:0/1204972310 conn(0x7fab340d45e0 msgr2=0x7fab340d49f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:05.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:05.978+0000 7fab396fa640 1 -- 172.21.15.112:0/1204972310 shutdown_connections 2024-09-21T00:30:05.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:05.978+0000 7fab396fa640 1 -- 172.21.15.112:0/1204972310 wait complete. 2024-09-21T00:30:05.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:05.980+0000 7fab396fa640 1 Processor -- start 2024-09-21T00:30:05.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:05.980+0000 7fab396fa640 1 -- start start 2024-09-21T00:30:05.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:05.981+0000 7fab396fa640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fab20004df0 0x7fab341e5e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:05.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:05.981+0000 7fab396fa640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fab341e8c20 0x7fab3414d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:05.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:05.981+0000 7fab396fa640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fab340cd7b0 0x7fab340cdbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:05.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:05.981+0000 7fab396fa640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fab3406f080 con 0x7fab20004df0 2024-09-21T00:30:05.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:05.981+0000 7fab396fa640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fab3406ef00 con 0x7fab341e8c20 2024-09-21T00:30:05.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:05.981+0000 7fab396fa640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fab3406f200 con 0x7fab340cd7b0 2024-09-21T00:30:05.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:05.981+0000 7fab32ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fab20004df0 0x7fab341e5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:05.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:05.981+0000 7fab32ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fab20004df0 0x7fab341e5e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41006/0 (socket says 172.21.15.112:41006) 2024-09-21T00:30:05.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:05.981+0000 7fab32ffd640 1 -- 172.21.15.112:0/2487984924 learned_addr learned my addr 172.21.15.112:0/2487984924 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:30:05.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:05.982+0000 7fab2bfff640 1 --2- 172.21.15.112:0/2487984924 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fab341e8c20 0x7fab3414d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:05.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:05.982+0000 7fab337fe640 1 --2- 172.21.15.112:0/2487984924 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fab340cd7b0 0x7fab340cdbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:05.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:05.982+0000 7fab32ffd640 1 -- 172.21.15.112:0/2487984924 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fab340cd7b0 msgr2=0x7fab340cdbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:05.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:05.982+0000 7fab32ffd640 1 --2- 172.21.15.112:0/2487984924 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fab340cd7b0 0x7fab340cdbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:05.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:05.982+0000 7fab32ffd640 1 -- 172.21.15.112:0/2487984924 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fab341e8c20 msgr2=0x7fab3414d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:05.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:05.982+0000 7fab32ffd640 1 --2- 172.21.15.112:0/2487984924 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fab341e8c20 0x7fab3414d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:05.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:05.982+0000 7fab32ffd640 1 -- 172.21.15.112:0/2487984924 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab340cc710 con 0x7fab20004df0 2024-09-21T00:30:05.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:05.982+0000 7fab337fe640 1 --2- 172.21.15.112:0/2487984924 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fab340cd7b0 0x7fab340cdbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:30:05.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:05.982+0000 7fab32ffd640 1 --2- 172.21.15.112:0/2487984924 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fab20004df0 0x7fab341e5e60 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7fab24011ae0 tx=0x7fab24009e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:05.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:05.982+0000 7fab2b7fe640 1 -- 172.21.15.112:0/2487984924 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab2401e070 con 0x7fab20004df0 2024-09-21T00:30:05.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:05.982+0000 7fab2b7fe640 1 -- 172.21.15.112:0/2487984924 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fab2400fe70 con 0x7fab20004df0 2024-09-21T00:30:05.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:05.982+0000 7fab2b7fe640 1 -- 172.21.15.112:0/2487984924 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab24004290 con 0x7fab20004df0 2024-09-21T00:30:05.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:05.982+0000 7fab396fa640 1 -- 172.21.15.112:0/2487984924 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab340cc9f0 con 0x7fab20004df0 2024-09-21T00:30:05.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:05.982+0000 7fab396fa640 1 -- 172.21.15.112:0/2487984924 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab340ccfb0 con 0x7fab20004df0 2024-09-21T00:30:05.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:05.983+0000 7fab396fa640 1 -- 172.21.15.112:0/2487984924 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab3406f790 con 0x7fab20004df0 2024-09-21T00:30:05.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:05.984+0000 7fab2b7fe640 1 -- 172.21.15.112:0/2487984924 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fab24021070 con 0x7fab20004df0 2024-09-21T00:30:05.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:05.984+0000 7fab2b7fe640 1 --2- 172.21.15.112:0/2487984924 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fab14078bc0 0x7fab1407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:05.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:05.985+0000 7fab2b7fe640 1 -- 172.21.15.112:0/2487984924 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 6187+0+0 (secure 0 0 0) 0x7fab2409b8a0 con 0x7fab20004df0 2024-09-21T00:30:05.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:05.985+0000 7fab2bfff640 1 --2- 172.21.15.112:0/2487984924 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fab14078bc0 0x7fab1407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:05.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:05.985+0000 7fab2bfff640 1 --2- 172.21.15.112:0/2487984924 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fab14078bc0 0x7fab1407b0d0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fab1c001990 tx=0x7fab1c006040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:05.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:05.989+0000 7fab2b7fe640 1 -- 172.21.15.112:0/2487984924 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fab24063600 con 0x7fab20004df0 2024-09-21T00:30:06.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.201+0000 7fab396fa640 1 -- 172.21.15.112:0/2487984924 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fab340d1390 con 0x7fab20004df0 2024-09-21T00:30:06.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.202+0000 7fab2b7fe640 1 -- 172.21.15.112:0/2487984924 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) ==== 94+0+5236 (secure 0 0 0) 0x7fab24068030 con 0x7fab20004df0 2024-09-21T00:30:06.204 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:30:06.204 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":76,"btime":"2024-09-21T00:30:05:624778+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6113,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/3217596137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":3217596137},{"type":"v1","addr":"172.21.15.187:6835","nonce":3217596137}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":73},{"gid":6649,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6837/776068904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":776068904},{"type":"v1","addr":"172.21.15.187:6837","nonce":776068904}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":73},{"gid":7419,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6834/3291181711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":3291181711},{"type":"v1","addr":"172.21.15.112:6834","nonce":3291181711}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":73}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T00:29:48.456672+0000","modified":"2024-09-21T00:30:05.624767+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":112,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7422":{"gid":7422,"name":"c","rank":0,"incarnation":73,"state":"up:active","state_seq":4,"addr":"172.21.15.112:6835/3392867613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":3392867613},{"type":"v1","addr":"172.21.15.112:6835","nonce":3392867613}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7422,"qdb_cluster":[7422]},"id":8}]} 2024-09-21T00:30:06.204 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 76 2024-09-21T00:30:06.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.204+0000 7fab396fa640 1 -- 172.21.15.112:0/2487984924 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fab14078bc0 msgr2=0x7fab1407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:06.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.204+0000 7fab396fa640 1 --2- 172.21.15.112:0/2487984924 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fab14078bc0 0x7fab1407b0d0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fab1c001990 tx=0x7fab1c006040 comp rx=0 tx=0).stop 2024-09-21T00:30:06.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.204+0000 7fab396fa640 1 -- 172.21.15.112:0/2487984924 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fab20004df0 msgr2=0x7fab341e5e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:06.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.204+0000 7fab396fa640 1 --2- 172.21.15.112:0/2487984924 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fab20004df0 0x7fab341e5e60 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7fab24011ae0 tx=0x7fab24009e90 comp rx=0 tx=0).stop 2024-09-21T00:30:06.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.205+0000 7fab396fa640 1 -- 172.21.15.112:0/2487984924 shutdown_connections 2024-09-21T00:30:06.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.205+0000 7fab396fa640 1 --2- 172.21.15.112:0/2487984924 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fab340cd7b0 0x7fab340cdbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:06.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.205+0000 7fab396fa640 1 --2- 172.21.15.112:0/2487984924 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fab341e8c20 0x7fab3414d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:06.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.205+0000 7fab396fa640 1 --2- 172.21.15.112:0/2487984924 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fab14078bc0 0x7fab1407b0d0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:06.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.205+0000 7fab396fa640 1 --2- 172.21.15.112:0/2487984924 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fab20004df0 0x7fab341e5e60 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:06.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.205+0000 7fab396fa640 1 -- 172.21.15.112:0/2487984924 >> 172.21.15.112:0/2487984924 conn(0x7fab340d45e0 msgr2=0x7fab340b2c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:06.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.205+0000 7fab396fa640 1 -- 172.21.15.112:0/2487984924 shutdown_connections 2024-09-21T00:30:06.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.205+0000 7fab396fa640 1 -- 172.21.15.112:0/2487984924 wait complete. 2024-09-21T00:30:06.620 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 76, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T00:29:48.456672+0000', 'modified': '2024-09-21T00:30:05.624767+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 112, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7422}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7422': {'gid': 7422, 'name': 'c', 'rank': 0, 'incarnation': 73, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.112:6835/3392867613', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6833', 'nonce': 3392867613}, {'type': 'v1', 'addr': '172.21.15.112:6835', 'nonce': 3392867613}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7422, 'qdb_cluster': [7422]} 2024-09-21T00:30:06.620 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-21T00:30:06.621 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-21T00:30:06.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.734+0000 7fde262b6640 1 -- 172.21.15.112:0/3248209207 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde08005350 msgr2=0x7fde0800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:06.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.734+0000 7fde262b6640 1 --2- 172.21.15.112:0/3248209207 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde08005350 0x7fde0800b9c0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7fde18005fd0 tx=0x7fde08005830 comp rx=0 tx=0).stop 2024-09-21T00:30:06.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.734+0000 7fde262b6640 1 -- 172.21.15.112:0/3248209207 shutdown_connections 2024-09-21T00:30:06.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.734+0000 7fde262b6640 1 --2- 172.21.15.112:0/3248209207 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde08005350 0x7fde0800b9c0 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:06.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.734+0000 7fde262b6640 1 -- 172.21.15.112:0/3248209207 >> 172.21.15.112:0/3248209207 conn(0x7fde200d13d0 msgr2=0x7fde200d17e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:06.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.734+0000 7fde262b6640 1 -- 172.21.15.112:0/3248209207 shutdown_connections 2024-09-21T00:30:06.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.734+0000 7fde262b6640 1 -- 172.21.15.112:0/3248209207 wait complete. 2024-09-21T00:30:06.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.736+0000 7fde262b6640 1 Processor -- start 2024-09-21T00:30:06.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.736+0000 7fde262b6640 1 -- start start 2024-09-21T00:30:06.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.738+0000 7fde262b6640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fde08005350 0x7fde201f7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:06.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.738+0000 7fde262b6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde201e54d0 0x7fde201d1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:06.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.738+0000 7fde262b6640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fde201f6cd0 0x7fde201f70e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:06.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.738+0000 7fde262b6640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fde200cfb30 con 0x7fde201e54d0 2024-09-21T00:30:06.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.738+0000 7fde262b6640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fde200cf9b0 con 0x7fde201f6cd0 2024-09-21T00:30:06.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.738+0000 7fde262b6640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fde200cfcb0 con 0x7fde08005350 2024-09-21T00:30:06.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.738+0000 7fde1f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde201e54d0 0x7fde201d1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:06.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.738+0000 7fde1f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde201e54d0 0x7fde201d1cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35536/0 (socket says 172.21.15.112:35536) 2024-09-21T00:30:06.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.738+0000 7fde1f7fe640 1 -- 172.21.15.112:0/1910666249 learned_addr learned my addr 172.21.15.112:0/1910666249 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:30:06.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.738+0000 7fde24853640 1 --2- 172.21.15.112:0/1910666249 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fde201f6cd0 0x7fde201f70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:06.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.738+0000 7fde1ffff640 1 --2- 172.21.15.112:0/1910666249 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fde08005350 0x7fde201f7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:06.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.738+0000 7fde1f7fe640 1 -- 172.21.15.112:0/1910666249 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fde08005350 msgr2=0x7fde201f7870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:06.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.738+0000 7fde1f7fe640 1 --2- 172.21.15.112:0/1910666249 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fde08005350 0x7fde201f7870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:06.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.738+0000 7fde1f7fe640 1 -- 172.21.15.112:0/1910666249 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fde201f6cd0 msgr2=0x7fde201f70e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:06.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.738+0000 7fde1f7fe640 1 --2- 172.21.15.112:0/1910666249 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fde201f6cd0 0x7fde201f70e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:06.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.738+0000 7fde1f7fe640 1 -- 172.21.15.112:0/1910666249 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde201d21f0 con 0x7fde201e54d0 2024-09-21T00:30:06.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.738+0000 7fde1ffff640 1 --2- 172.21.15.112:0/1910666249 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fde08005350 0x7fde201f7870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:30:06.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.738+0000 7fde1f7fe640 1 --2- 172.21.15.112:0/1910666249 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde201e54d0 0x7fde201d1cb0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7fde14004d90 tx=0x7fde14001630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:06.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.738+0000 7fde1cff9640 1 -- 172.21.15.112:0/1910666249 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde14001d40 con 0x7fde201e54d0 2024-09-21T00:30:06.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.738+0000 7fde1cff9640 1 -- 172.21.15.112:0/1910666249 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fde14016070 con 0x7fde201e54d0 2024-09-21T00:30:06.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.738+0000 7fde262b6640 1 -- 172.21.15.112:0/1910666249 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde201f7620 con 0x7fde201e54d0 2024-09-21T00:30:06.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.738+0000 7fde1cff9640 1 -- 172.21.15.112:0/1910666249 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde14010b40 con 0x7fde201e54d0 2024-09-21T00:30:06.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.738+0000 7fde262b6640 1 -- 172.21.15.112:0/1910666249 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde201e83c0 con 0x7fde201e54d0 2024-09-21T00:30:06.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.740+0000 7fde1cff9640 1 -- 172.21.15.112:0/1910666249 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fde14010ce0 con 0x7fde201e54d0 2024-09-21T00:30:06.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.741+0000 7fde1cff9640 1 --2- 172.21.15.112:0/1910666249 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fde0c078b90 0x7fde0c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:06.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.741+0000 7fde1cff9640 1 -- 172.21.15.112:0/1910666249 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 6187+0+0 (secure 0 0 0) 0x7fde1409b6f0 con 0x7fde201e54d0 2024-09-21T00:30:06.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.741+0000 7fde1ffff640 1 --2- 172.21.15.112:0/1910666249 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fde0c078b90 0x7fde0c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:06.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.742+0000 7fde2d222740 1 Processor -- start 2024-09-21T00:30:06.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.742+0000 7fde1ffff640 1 --2- 172.21.15.112:0/1910666249 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fde0c078b90 0x7fde0c07b0a0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fde10001eb0 tx=0x7fde10001000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:06.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.742+0000 7fde2d222740 1 -- start start 2024-09-21T00:30:06.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.743+0000 7fde2d222740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d218a63270 0x55d218e34fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:06.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.743+0000 7fde2d222740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d218a3a3b0 0x55d218e35500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:06.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.743+0000 7fde2d222740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d218a42590 0x55d218e52400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:06.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.743+0000 7fde2d222740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d218d7d820 con 0x55d218a42590 2024-09-21T00:30:06.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.743+0000 7fde2d222740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55d218d6cc50 con 0x55d218a3a3b0 2024-09-21T00:30:06.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.743+0000 7fde2d222740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x55d218d70a60 con 0x55d218a63270 2024-09-21T00:30:06.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.743+0000 7fde24853640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d218a42590 0x55d218e52400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:06.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.743+0000 7fde24853640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d218a42590 0x55d218e52400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35540/0 (socket says 172.21.15.112:35540) 2024-09-21T00:30:06.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.743+0000 7fde24853640 1 -- 172.21.15.112:0/3017774345 learned_addr learned my addr 172.21.15.112:0/3017774345 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:30:06.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.743+0000 7fde1ffff640 1 --2- 172.21.15.112:0/3017774345 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d218a63270 0x55d218e34fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:06.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.743+0000 7fde1f7fe640 1 --2- 172.21.15.112:0/3017774345 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d218a3a3b0 0x55d218e35500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:06.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.743+0000 7fde24853640 1 -- 172.21.15.112:0/3017774345 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d218a63270 msgr2=0x55d218e34fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:30:06.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.743+0000 7fde24853640 1 --2- 172.21.15.112:0/3017774345 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d218a63270 0x55d218e34fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:06.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.743+0000 7fde24853640 1 -- 172.21.15.112:0/3017774345 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d218a3a3b0 msgr2=0x55d218e35500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:30:06.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.743+0000 7fde24853640 1 --2- 172.21.15.112:0/3017774345 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d218a3a3b0 0x55d218e35500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:06.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.743+0000 7fde24853640 1 -- 172.21.15.112:0/3017774345 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d218ace850 con 0x55d218a42590 2024-09-21T00:30:06.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.743+0000 7fde24853640 1 --2- 172.21.15.112:0/3017774345 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d218a42590 0x55d218e52400 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7fde1802e020 tx=0x7fde18004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2821cd6265df7270 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:06.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.744+0000 7fde05ffb640 1 -- 172.21.15.112:0/3017774345 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde18003a30 con 0x55d218a42590 2024-09-21T00:30:06.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.744+0000 7fde05ffb640 1 -- 172.21.15.112:0/3017774345 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fde18040070 con 0x55d218a42590 2024-09-21T00:30:06.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.744+0000 7fde05ffb640 1 -- 172.21.15.112:0/3017774345 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde18003dd0 con 0x55d218a42590 2024-09-21T00:30:06.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.744+0000 7fde2d222740 1 -- 172.21.15.112:0/3017774345 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d218a42590 msgr2=0x55d218e52400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:06.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.744+0000 7fde2d222740 1 --2- 172.21.15.112:0/3017774345 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d218a42590 0x55d218e52400 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7fde1802e020 tx=0x7fde18004ab0 comp rx=0 tx=0).stop 2024-09-21T00:30:06.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.744+0000 7fde2d222740 1 -- 172.21.15.112:0/3017774345 shutdown_connections 2024-09-21T00:30:06.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.744+0000 7fde2d222740 1 --2- 172.21.15.112:0/3017774345 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d218a42590 0x55d218e52400 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:06.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.744+0000 7fde2d222740 1 --2- 172.21.15.112:0/3017774345 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d218a3a3b0 0x55d218e35500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:06.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.744+0000 7fde2d222740 1 --2- 172.21.15.112:0/3017774345 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d218a63270 0x55d218e34fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:06.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.744+0000 7fde2d222740 1 -- 172.21.15.112:0/3017774345 >> 172.21.15.112:0/3017774345 conn(0x55d218a573c0 msgr2=0x55d218e0b280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:06.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.744+0000 7fde2d222740 1 -- 172.21.15.112:0/3017774345 shutdown_connections 2024-09-21T00:30:06.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.744+0000 7fde2d222740 1 -- 172.21.15.112:0/3017774345 wait complete. 2024-09-21T00:30:06.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.745+0000 7fde2d222740 1 -- start start 2024-09-21T00:30:06.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.746+0000 7fde2d222740 1 Processor -- start 2024-09-21T00:30:06.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.747+0000 7fde2d222740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d218a3a3b0 0x55d218e48ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:06.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.747+0000 7fde2d222740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d218a42590 0x55d218e23490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:06.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.747+0000 7fde2d222740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d218a63270 0x55d218e4a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:06.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.747+0000 7fde2d222740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d218add620 con 0x55d218a3a3b0 2024-09-21T00:30:06.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.747+0000 7fde2d222740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55d218d125f0 con 0x55d218a63270 2024-09-21T00:30:06.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.747+0000 7fde2d222740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x55d218e4ae50 con 0x55d218a42590 2024-09-21T00:30:06.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.747+0000 7fde2d222740 10 client.0 _ll_register_callbacks cb 0x55d218d160a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:30:06.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.747+0000 7fde1ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d218a3a3b0 0x55d218e48ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:06.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.747+0000 7fde1ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d218a3a3b0 0x55d218e48ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35556/0 (socket says 172.21.15.112:35556) 2024-09-21T00:30:06.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.747+0000 7fde1ffff640 1 -- 172.21.15.112:0/2000489537 learned_addr learned my addr 172.21.15.112:0/2000489537 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:30:06.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.747+0000 7fde24853640 1 --2- 172.21.15.112:0/2000489537 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d218a63270 0x55d218e4a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:06.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.747+0000 7fde1f7fe640 1 --2- 172.21.15.112:0/2000489537 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d218a42590 0x55d218e23490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:06.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.747+0000 7fde1ffff640 1 -- 172.21.15.112:0/2000489537 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d218a42590 msgr2=0x55d218e23490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:30:06.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.747+0000 7fde1ffff640 1 --2- 172.21.15.112:0/2000489537 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d218a42590 0x55d218e23490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:06.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.747+0000 7fde1ffff640 1 -- 172.21.15.112:0/2000489537 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d218a63270 msgr2=0x55d218e4a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:30:06.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.747+0000 7fde1ffff640 1 --2- 172.21.15.112:0/2000489537 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d218a63270 0x55d218e4a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:06.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.747+0000 7fde1ffff640 1 -- 172.21.15.112:0/2000489537 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d218b456b0 con 0x55d218a3a3b0 2024-09-21T00:30:06.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.747+0000 7fde1ffff640 1 --2- 172.21.15.112:0/2000489537 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d218a3a3b0 0x55d218e48ff0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7fde10003de0 tx=0x7fde200b2ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=243f18a6c3db3dd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:06.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.747+0000 7fddfeffd640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T00:30:06.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.748+0000 7fddfeffd640 1 -- 172.21.15.112:0/2000489537 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde10003e10 con 0x55d218a3a3b0 2024-09-21T00:30:06.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.748+0000 7fddfeffd640 1 -- 172.21.15.112:0/2000489537 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 6187+0+0 (secure 0 0 0) 0x7fde1000a040 con 0x55d218a3a3b0 2024-09-21T00:30:06.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.748+0000 7fde2d222740 1 -- 172.21.15.112:0/2000489537 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde18040070 con 0x55d218a3a3b0 2024-09-21T00:30:06.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.748+0000 7fddfeffd640 1 -- 172.21.15.112:0/2000489537 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7fde10020650 con 0x55d218a3a3b0 2024-09-21T00:30:06.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.748+0000 7fde2d222740 10 client.7455 fetch_fsmap learned FSMap version 76 2024-09-21T00:30:06.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.748+0000 7fde2d222740 1 -- 172.21.15.112:0/2000489537 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=2+}) -- 0x55d218e5f8f0 con 0x55d218a3a3b0 2024-09-21T00:30:06.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.748+0000 7fddfeffd640 1 -- 172.21.15.112:0/2000489537 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fde18040070 con 0x55d218a3a3b0 2024-09-21T00:30:06.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.748+0000 7fddfeffd640 1 -- 172.21.15.112:0/2000489537 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 76) ==== 3502+0+0 (secure 0 0 0) 0x7fde10010040 con 0x55d218a3a3b0 2024-09-21T00:30:06.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.748+0000 7fde2d222740 10 client.7455 fetch_fsmap finished waiting for FSMap version 76 2024-09-21T00:30:06.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.748+0000 7fde2d222740 10 client.7455 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T00:30:06.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.748+0000 7fde2d222740 20 client.7455 populate_metadata read hostname 'smithi112' 2024-09-21T00:30:06.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.748+0000 7fde2d222740 1 --2- 172.21.15.112:0/2000489537 >> [v2:172.21.15.112:6833/3392867613,v1:172.21.15.112:6835/3392867613] conn(0x55d218a21e40 0x55d218e094b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:06.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.748+0000 7fde2d222740 4 client.7455 mds_command: new command op to 7422 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T00:30:06.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.748+0000 7fde2d222740 1 -- 172.21.15.112:0/2000489537 --> [v2:172.21.15.112:6833/3392867613,v1:172.21.15.112:6835/3392867613] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d218d7f770 con 0x55d218a21e40 2024-09-21T00:30:06.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.748+0000 7fde1f7fe640 1 --2- 172.21.15.112:0/2000489537 >> [v2:172.21.15.112:6833/3392867613,v1:172.21.15.112:6835/3392867613] conn(0x55d218a21e40 0x55d218e094b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:06.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.749+0000 7fddfeffd640 10 client.7455 ms_handle_connect on v2:172.21.15.112:6833/3392867613 2024-09-21T00:30:06.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.749+0000 7fde1f7fe640 1 --2- 172.21.15.112:0/2000489537 >> [v2:172.21.15.112:6833/3392867613,v1:172.21.15.112:6835/3392867613] conn(0x55d218a21e40 0x55d218e094b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e9a1c79551f1ab9 server_cookie=e46af060bfee99d4 in_seq=0 out_seq=0 2024-09-21T00:30:06.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.749+0000 7fddfeffd640 1 -- 172.21.15.112:0/2000489537 <== mds.0 v2:172.21.15.112:6833/3392867613 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d218d7f770 con 0x55d218a21e40 2024-09-21T00:30:06.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.749+0000 7fddfeffd640 10 client.7455 handle_command_reply: tid=0 2024-09-21T00:30:06.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.749+0000 7fde2d222740 1 client.7455 shutdown 2024-09-21T00:30:06.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.750+0000 7fde2d222740 1 -- 172.21.15.112:0/2000489537 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d218a3a3b0 msgr2=0x55d218e48ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:06.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.750+0000 7fde2d222740 1 --2- 172.21.15.112:0/2000489537 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d218a3a3b0 0x55d218e48ff0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7fde10003de0 tx=0x7fde200b2ca0 comp rx=0 tx=0).stop 2024-09-21T00:30:06.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.750+0000 7fde2d222740 1 -- 172.21.15.112:0/2000489537 shutdown_connections 2024-09-21T00:30:06.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.750+0000 7fde2d222740 1 --2- 172.21.15.112:0/2000489537 >> [v2:172.21.15.112:6833/3392867613,v1:172.21.15.112:6835/3392867613] conn(0x55d218a21e40 0x55d218e094b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:06.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.750+0000 7fde2d222740 1 --2- 172.21.15.112:0/2000489537 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d218a63270 0x55d218e4a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:06.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.750+0000 7fde2d222740 1 --2- 172.21.15.112:0/2000489537 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d218a42590 0x55d218e23490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:06.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.750+0000 7fddfeffd640 0 client.7455 ms_handle_reset on v2:172.21.15.112:6833/3392867613 2024-09-21T00:30:06.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.750+0000 7fde2d222740 1 --2- 172.21.15.112:0/2000489537 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d218a3a3b0 0x55d218e48ff0 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:06.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.750+0000 7fde2d222740 1 -- 172.21.15.112:0/2000489537 >> 172.21.15.112:0/2000489537 conn(0x55d218a573c0 msgr2=0x55d218e0b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:06.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.750+0000 7fde2d222740 1 -- 172.21.15.112:0/2000489537 shutdown_connections 2024-09-21T00:30:06.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.750+0000 7fde2d222740 1 -- 172.21.15.112:0/2000489537 wait complete. 2024-09-21T00:30:06.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.750+0000 7fde2d222740 20 client.7455 trim_cache size 0 max 16384 2024-09-21T00:30:06.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.768+0000 7fde2d222740 1 Processor -- start 2024-09-21T00:30:06.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.768+0000 7fde2d222740 1 -- start start 2024-09-21T00:30:06.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.769+0000 7fde2d222740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d218a3a3b0 0x55d218e45f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:06.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.769+0000 7fde2d222740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d218a42590 0x55d218e728e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:06.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.769+0000 7fde2d222740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d218a63270 0x55d218e72e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:06.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.769+0000 7fde2d222740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fde201ffcf0 con 0x55d218a63270 2024-09-21T00:30:06.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.769+0000 7fde2d222740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fde201ff0b0 con 0x55d218a42590 2024-09-21T00:30:06.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.769+0000 7fde2d222740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fde201ff6a0 con 0x55d218a3a3b0 2024-09-21T00:30:06.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.769+0000 7fde24853640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d218a63270 0x55d218e72e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:06.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.769+0000 7fde24853640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d218a63270 0x55d218e72e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35560/0 (socket says 172.21.15.112:35560) 2024-09-21T00:30:06.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.769+0000 7fde24853640 1 -- 172.21.15.112:0/1757925857 learned_addr learned my addr 172.21.15.112:0/1757925857 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:30:06.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.770+0000 7fde1f7fe640 1 --2- 172.21.15.112:0/1757925857 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d218a42590 0x55d218e728e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:06.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.770+0000 7fde1ffff640 1 --2- 172.21.15.112:0/1757925857 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d218a3a3b0 0x55d218e45f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:06.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.770+0000 7fde24853640 1 -- 172.21.15.112:0/1757925857 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d218a3a3b0 msgr2=0x55d218e45f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:30:06.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.770+0000 7fde24853640 1 --2- 172.21.15.112:0/1757925857 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d218a3a3b0 0x55d218e45f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:06.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.770+0000 7fde24853640 1 -- 172.21.15.112:0/1757925857 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d218a42590 msgr2=0x55d218e728e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:30:06.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.770+0000 7fde24853640 1 --2- 172.21.15.112:0/1757925857 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d218a42590 0x55d218e728e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:06.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.770+0000 7fde24853640 1 -- 172.21.15.112:0/1757925857 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d218e473c0 con 0x55d218a63270 2024-09-21T00:30:06.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.770+0000 7fde24853640 1 --2- 172.21.15.112:0/1757925857 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d218a63270 0x55d218e72e20 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7fde1802aec0 tx=0x7fde08005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e6dbbd2f2d7b20b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:06.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.770+0000 7fde057fa640 1 -- 172.21.15.112:0/1757925857 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde1804b070 con 0x55d218a63270 2024-09-21T00:30:06.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.770+0000 7fde057fa640 1 -- 172.21.15.112:0/1757925857 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fde18040040 con 0x55d218a63270 2024-09-21T00:30:06.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.770+0000 7fde057fa640 1 -- 172.21.15.112:0/1757925857 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde18003c70 con 0x55d218a63270 2024-09-21T00:30:06.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.771+0000 7fde2d222740 1 -- 172.21.15.112:0/1757925857 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d218a63270 msgr2=0x55d218e72e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:06.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.771+0000 7fde2d222740 1 --2- 172.21.15.112:0/1757925857 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d218a63270 0x55d218e72e20 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7fde1802aec0 tx=0x7fde08005830 comp rx=0 tx=0).stop 2024-09-21T00:30:06.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.771+0000 7fde2d222740 1 -- 172.21.15.112:0/1757925857 shutdown_connections 2024-09-21T00:30:06.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.771+0000 7fde2d222740 1 --2- 172.21.15.112:0/1757925857 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d218a63270 0x55d218e72e20 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:06.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.771+0000 7fde2d222740 1 --2- 172.21.15.112:0/1757925857 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d218a42590 0x55d218e728e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:06.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.771+0000 7fde2d222740 1 --2- 172.21.15.112:0/1757925857 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d218a3a3b0 0x55d218e45f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:06.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.771+0000 7fde2d222740 1 -- 172.21.15.112:0/1757925857 >> 172.21.15.112:0/1757925857 conn(0x55d218a573c0 msgr2=0x55d218e22f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:06.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.771+0000 7fde2d222740 1 -- 172.21.15.112:0/1757925857 shutdown_connections 2024-09-21T00:30:06.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.771+0000 7fde2d222740 1 -- 172.21.15.112:0/1757925857 wait complete. 2024-09-21T00:30:06.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.772+0000 7fde2d222740 1 -- start start 2024-09-21T00:30:06.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.772+0000 7fde2d222740 1 Processor -- start 2024-09-21T00:30:06.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.773+0000 7fde2d222740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d218a3a3b0 0x55d218ece990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:06.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.773+0000 7fde2d222740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d218a42590 0x55d218eceed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:06.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.773+0000 7fde2d222740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d218a63270 0x55d218e09750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:06.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.773+0000 7fde2d222740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d218e5f3e0 con 0x55d218a3a3b0 2024-09-21T00:30:06.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.773+0000 7fde2d222740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55d218e4ab00 con 0x55d218a42590 2024-09-21T00:30:06.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.773+0000 7fde2d222740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x55d218e47fb0 con 0x55d218a63270 2024-09-21T00:30:06.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.773+0000 7fde2d222740 10 client.0 _ll_register_callbacks cb 0x7fde10016040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:30:06.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.773+0000 7fde1ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d218a3a3b0 0x55d218ece990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:06.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.773+0000 7fde1ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d218a3a3b0 0x55d218ece990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35570/0 (socket says 172.21.15.112:35570) 2024-09-21T00:30:06.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.773+0000 7fde1ffff640 1 -- 172.21.15.112:0/4188814431 learned_addr learned my addr 172.21.15.112:0/4188814431 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:30:06.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.773+0000 7fde1f7fe640 1 --2- 172.21.15.112:0/4188814431 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d218a42590 0x55d218eceed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:06.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.773+0000 7fde24853640 1 --2- 172.21.15.112:0/4188814431 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d218a63270 0x55d218e09750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:06.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.774+0000 7fde1ffff640 1 -- 172.21.15.112:0/4188814431 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d218a63270 msgr2=0x55d218e09750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:30:06.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.774+0000 7fde1ffff640 1 --2- 172.21.15.112:0/4188814431 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d218a63270 0x55d218e09750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:06.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.774+0000 7fde1ffff640 1 -- 172.21.15.112:0/4188814431 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d218a42590 msgr2=0x55d218eceed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:30:06.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.774+0000 7fde1ffff640 1 --2- 172.21.15.112:0/4188814431 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d218a42590 0x55d218eceed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:06.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.774+0000 7fde1ffff640 1 -- 172.21.15.112:0/4188814431 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d218e5e410 con 0x55d218a3a3b0 2024-09-21T00:30:06.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.774+0000 7fde1ffff640 1 --2- 172.21.15.112:0/4188814431 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d218a3a3b0 0x55d218ece990 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7fde10020e90 tx=0x7fde10012e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c55b9ebceed4a00 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:06.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.774+0000 7fddfeffd640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T00:30:06.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.774+0000 7fddfeffd640 1 -- 172.21.15.112:0/4188814431 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde10024030 con 0x55d218a3a3b0 2024-09-21T00:30:06.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.774+0000 7fddfeffd640 1 -- 172.21.15.112:0/4188814431 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 6187+0+0 (secure 0 0 0) 0x7fde10026030 con 0x55d218a3a3b0 2024-09-21T00:30:06.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.774+0000 7fde2d222740 1 -- 172.21.15.112:0/4188814431 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde18040040 con 0x55d218a3a3b0 2024-09-21T00:30:06.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.774+0000 7fddfeffd640 1 -- 172.21.15.112:0/4188814431 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7fde10011660 con 0x55d218a3a3b0 2024-09-21T00:30:06.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.774+0000 7fde2d222740 10 client.7461 fetch_fsmap learned FSMap version 76 2024-09-21T00:30:06.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.774+0000 7fde2d222740 1 -- 172.21.15.112:0/4188814431 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=2+}) -- 0x55d218ecf4a0 con 0x55d218a3a3b0 2024-09-21T00:30:06.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.774+0000 7fddfeffd640 1 -- 172.21.15.112:0/4188814431 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fde18040040 con 0x55d218a3a3b0 2024-09-21T00:30:06.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.774+0000 7fddfeffd640 1 -- 172.21.15.112:0/4188814431 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 76) ==== 3502+0+0 (secure 0 0 0) 0x7fde10010070 con 0x55d218a3a3b0 2024-09-21T00:30:06.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.774+0000 7fde2d222740 10 client.7461 fetch_fsmap finished waiting for FSMap version 76 2024-09-21T00:30:06.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.774+0000 7fde2d222740 10 client.7461 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T00:30:06.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.774+0000 7fde2d222740 20 client.7461 populate_metadata read hostname 'smithi112' 2024-09-21T00:30:06.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.774+0000 7fde2d222740 1 --2- 172.21.15.112:0/4188814431 >> [v2:172.21.15.112:6833/3392867613,v1:172.21.15.112:6835/3392867613] conn(0x55d218ed3220 0x55d218e30770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:06.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.774+0000 7fde2d222740 4 client.7461 mds_command: new command op to 7422 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T00:30:06.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.774+0000 7fde2d222740 1 -- 172.21.15.112:0/4188814431 --> [v2:172.21.15.112:6833/3392867613,v1:172.21.15.112:6835/3392867613] -- command(tid 0: {"prefix": "status"}) -- 0x55d218d7b220 con 0x55d218ed3220 2024-09-21T00:30:06.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.775+0000 7fde1f7fe640 1 --2- 172.21.15.112:0/4188814431 >> [v2:172.21.15.112:6833/3392867613,v1:172.21.15.112:6835/3392867613] conn(0x55d218ed3220 0x55d218e30770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:06.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.775+0000 7fddfeffd640 10 client.7461 ms_handle_connect on v2:172.21.15.112:6833/3392867613 2024-09-21T00:30:06.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.775+0000 7fde1f7fe640 1 --2- 172.21.15.112:0/4188814431 >> [v2:172.21.15.112:6833/3392867613,v1:172.21.15.112:6835/3392867613] conn(0x55d218ed3220 0x55d218e30770 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ffb9c5c7fbec5c42 server_cookie=2c505868ec77fa1b in_seq=0 out_seq=0 2024-09-21T00:30:06.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.775+0000 7fddfeffd640 1 -- 172.21.15.112:0/4188814431 <== mds.0 v2:172.21.15.112:6833/3392867613 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x55d218d7b220 con 0x55d218ed3220 2024-09-21T00:30:06.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.775+0000 7fddfeffd640 10 client.7461 handle_command_reply: tid=0 2024-09-21T00:30:06.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.775+0000 7fde2d222740 1 client.7461 shutdown 2024-09-21T00:30:06.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.775+0000 7fde2d222740 1 -- 172.21.15.112:0/4188814431 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d218a3a3b0 msgr2=0x55d218ece990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:06.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.775+0000 7fde2d222740 1 --2- 172.21.15.112:0/4188814431 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d218a3a3b0 0x55d218ece990 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7fde10020e90 tx=0x7fde10012e10 comp rx=0 tx=0).stop 2024-09-21T00:30:06.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.775+0000 7fde2d222740 1 -- 172.21.15.112:0/4188814431 shutdown_connections 2024-09-21T00:30:06.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.775+0000 7fde2d222740 1 --2- 172.21.15.112:0/4188814431 >> [v2:172.21.15.112:6833/3392867613,v1:172.21.15.112:6835/3392867613] conn(0x55d218ed3220 0x55d218e30770 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:06.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.775+0000 7fde2d222740 1 --2- 172.21.15.112:0/4188814431 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55d218a63270 0x55d218e09750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:06.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.775+0000 7fde2d222740 1 --2- 172.21.15.112:0/4188814431 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d218a42590 0x55d218eceed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:06.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.775+0000 7fddfeffd640 0 client.7461 ms_handle_reset on v2:172.21.15.112:6833/3392867613 2024-09-21T00:30:06.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.775+0000 7fde2d222740 1 --2- 172.21.15.112:0/4188814431 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d218a3a3b0 0x55d218ece990 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:06.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.775+0000 7fde2d222740 1 -- 172.21.15.112:0/4188814431 >> 172.21.15.112:0/4188814431 conn(0x55d218a573c0 msgr2=0x55d218e4a160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:06.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.776+0000 7fde2d222740 1 -- 172.21.15.112:0/4188814431 shutdown_connections 2024-09-21T00:30:06.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.776+0000 7fde2d222740 1 -- 172.21.15.112:0/4188814431 wait complete. 2024-09-21T00:30:06.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.776+0000 7fde2d222740 20 client.7461 trim_cache size 0 max 16384 2024-09-21T00:30:06.783 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-21T00:30:06.783 INFO:teuthology.orchestra.run.smithi112.stdout: "cluster_fsid": "8b34692b-dc65-49b2-a08f-26f707b96ac5", 2024-09-21T00:30:06.783 INFO:teuthology.orchestra.run.smithi112.stdout: "whoami": 0, 2024-09-21T00:30:06.783 INFO:teuthology.orchestra.run.smithi112.stdout: "id": 7422, 2024-09-21T00:30:06.783 INFO:teuthology.orchestra.run.smithi112.stdout: "want_state": "up:active", 2024-09-21T00:30:06.784 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "up:active", 2024-09-21T00:30:06.784 INFO:teuthology.orchestra.run.smithi112.stdout: "fs_name": "cephfs", 2024-09-21T00:30:06.784 INFO:teuthology.orchestra.run.smithi112.stdout: "rank_uptime": 4.1668384019999998, 2024-09-21T00:30:06.784 INFO:teuthology.orchestra.run.smithi112.stdout: "mdsmap_epoch": 76, 2024-09-21T00:30:06.784 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch": 112, 2024-09-21T00:30:06.784 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch_barrier": 112, 2024-09-21T00:30:06.784 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 4.375526282 2024-09-21T00:30:06.784 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-21T00:30:06.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.776+0000 7fde262b6640 1 -- 172.21.15.112:0/1910666249 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fde0c078b90 msgr2=0x7fde0c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:06.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.776+0000 7fde262b6640 1 --2- 172.21.15.112:0/1910666249 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fde0c078b90 0x7fde0c07b0a0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fde10001eb0 tx=0x7fde10001000 comp rx=0 tx=0).stop 2024-09-21T00:30:06.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.776+0000 7fde262b6640 1 -- 172.21.15.112:0/1910666249 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde201e54d0 msgr2=0x7fde201d1cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:06.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.776+0000 7fde262b6640 1 --2- 172.21.15.112:0/1910666249 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde201e54d0 0x7fde201d1cb0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7fde14004d90 tx=0x7fde14001630 comp rx=0 tx=0).stop 2024-09-21T00:30:06.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.776+0000 7fde262b6640 1 -- 172.21.15.112:0/1910666249 shutdown_connections 2024-09-21T00:30:06.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.776+0000 7fde262b6640 1 --2- 172.21.15.112:0/1910666249 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fde201f6cd0 0x7fde201f70e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:06.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.776+0000 7fde262b6640 1 --2- 172.21.15.112:0/1910666249 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fde0c078b90 0x7fde0c07b0a0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:06.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.776+0000 7fde262b6640 1 --2- 172.21.15.112:0/1910666249 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde201e54d0 0x7fde201d1cb0 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:06.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.776+0000 7fde262b6640 1 --2- 172.21.15.112:0/1910666249 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fde08005350 0x7fde201f7870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:06.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.776+0000 7fde262b6640 1 -- 172.21.15.112:0/1910666249 >> 172.21.15.112:0/1910666249 conn(0x7fde200d13d0 msgr2=0x7fde201e4bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:06.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.776+0000 7fde262b6640 1 -- 172.21.15.112:0/1910666249 shutdown_connections 2024-09-21T00:30:06.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.776+0000 7fde262b6640 1 -- 172.21.15.112:0/1910666249 wait complete. 2024-09-21T00:30:06.789 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T00:30:06.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.900+0000 7f2d12120640 1 -- 172.21.15.112:0/2528449638 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2cf40051a0 msgr2=0x7f2cf400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:06.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.900+0000 7f2d12120640 1 --2- 172.21.15.112:0/2528449638 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2cf40051a0 0x7f2cf400b9c0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f2d0400c610 tx=0x7f2cf4005830 comp rx=0 tx=0).stop 2024-09-21T00:30:06.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.900+0000 7f2d0bfff640 1 -- 172.21.15.112:0/2528449638 reap_dead start 2024-09-21T00:30:06.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.900+0000 7f2d12120640 1 -- 172.21.15.112:0/2528449638 shutdown_connections 2024-09-21T00:30:06.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.900+0000 7f2d12120640 1 -- 172.21.15.112:0/2528449638 >> 172.21.15.112:0/2528449638 conn(0x7f2d0c08bf90 msgr2=0x7f2d0c0d4650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:06.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.900+0000 7f2d12120640 1 -- 172.21.15.112:0/2528449638 shutdown_connections 2024-09-21T00:30:06.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.900+0000 7f2d12120640 1 -- 172.21.15.112:0/2528449638 wait complete. 2024-09-21T00:30:06.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.902+0000 7f2d12120640 1 Processor -- start 2024-09-21T00:30:06.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.902+0000 7f2d12120640 1 -- start start 2024-09-21T00:30:06.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.904+0000 7f2d12120640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2cf40051a0 0x7f2d0c0a0910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:06.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.904+0000 7f2d12120640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2d0c1f54f0 0x7f2d0c158aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:06.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.904+0000 7f2d12120640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2d0c09e9c0 0x7f2d0c09edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:06.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.904+0000 7f2d12120640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2d0c0ba020 con 0x7f2d0c09e9c0 2024-09-21T00:30:06.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.904+0000 7f2d12120640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f2d0c0b9ea0 con 0x7f2d0c1f54f0 2024-09-21T00:30:06.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.904+0000 7f2d12120640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f2d0c06e8d0 con 0x7f2cf40051a0 2024-09-21T00:30:06.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.904+0000 7f2d0bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2d0c09e9c0 0x7f2d0c09edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:06.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.904+0000 7f2d0bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2d0c09e9c0 0x7f2d0c09edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35586/0 (socket says 172.21.15.112:35586) 2024-09-21T00:30:06.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.904+0000 7f2d0bfff640 1 -- 172.21.15.112:0/386230251 learned_addr learned my addr 172.21.15.112:0/386230251 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:30:06.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.904+0000 7f2d0affd640 1 --2- 172.21.15.112:0/386230251 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2d0c1f54f0 0x7f2d0c158aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:06.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.904+0000 7f2d0b7fe640 1 --2- 172.21.15.112:0/386230251 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2cf40051a0 0x7f2d0c0a0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:06.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.904+0000 7f2d0bfff640 1 -- 172.21.15.112:0/386230251 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2cf40051a0 msgr2=0x7f2d0c0a0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:06.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.904+0000 7f2d0bfff640 1 --2- 172.21.15.112:0/386230251 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2cf40051a0 0x7f2d0c0a0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:06.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.904+0000 7f2d0bfff640 1 -- 172.21.15.112:0/386230251 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2d0c1f54f0 msgr2=0x7f2d0c158aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:06.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.904+0000 7f2d0bfff640 1 --2- 172.21.15.112:0/386230251 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2d0c1f54f0 0x7f2d0c158aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:06.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.904+0000 7f2d0bfff640 1 -- 172.21.15.112:0/386230251 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d0c0a21e0 con 0x7f2d0c09e9c0 2024-09-21T00:30:06.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.904+0000 7f2d0bfff640 1 --2- 172.21.15.112:0/386230251 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2d0c09e9c0 0x7f2d0c09edd0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f2d0402b3a0 tx=0x7f2d0400c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:06.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.904+0000 7f2cf3fff640 1 -- 172.21.15.112:0/386230251 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d04039070 con 0x7f2d0c09e9c0 2024-09-21T00:30:06.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.904+0000 7f2cf3fff640 1 -- 172.21.15.112:0/386230251 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2d0402b6f0 con 0x7f2d0c09e9c0 2024-09-21T00:30:06.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.904+0000 7f2d12120640 1 -- 172.21.15.112:0/386230251 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d0c0a1140 con 0x7f2d0c09e9c0 2024-09-21T00:30:06.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.904+0000 7f2cf3fff640 1 -- 172.21.15.112:0/386230251 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d0402bae0 con 0x7f2d0c09e9c0 2024-09-21T00:30:06.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.904+0000 7f2d12120640 1 -- 172.21.15.112:0/386230251 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d0c0a1700 con 0x7f2d0c09e9c0 2024-09-21T00:30:06.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.905+0000 7f2d12120640 1 -- 172.21.15.112:0/386230251 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d0c0b60d0 con 0x7f2d0c09e9c0 2024-09-21T00:30:06.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.906+0000 7f2cf3fff640 1 -- 172.21.15.112:0/386230251 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2d0403c070 con 0x7f2d0c09e9c0 2024-09-21T00:30:06.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.906+0000 7f2cf3fff640 1 --2- 172.21.15.112:0/386230251 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2cf8078b90 0x7f2cf807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:06.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.907+0000 7f2cf3fff640 1 -- 172.21.15.112:0/386230251 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 6187+0+0 (secure 0 0 0) 0x7f2d040ba4d0 con 0x7f2d0c09e9c0 2024-09-21T00:30:06.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.907+0000 7f2d0b7fe640 1 --2- 172.21.15.112:0/386230251 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2cf8078b90 0x7f2cf807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:06.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.907+0000 7f2d0b7fe640 1 --2- 172.21.15.112:0/386230251 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2cf8078b90 0x7f2cf807b0a0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc006fd0 tx=0x7f2cfc006b60 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:06.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:06.911+0000 7f2cf3fff640 1 -- 172.21.15.112:0/386230251 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2d040827c0 con 0x7f2d0c09e9c0 2024-09-21T00:30:07.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:07.125+0000 7f2d12120640 1 -- 172.21.15.112:0/386230251 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f2d0c08c4a0 con 0x7f2d0c09e9c0 2024-09-21T00:30:07.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:07.125+0000 7f2cf3fff640 1 -- 172.21.15.112:0/386230251 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v76) ==== 53+0+73 (secure 0 0 0) 0x7f2d040820c0 con 0x7f2d0c09e9c0 2024-09-21T00:30:07.127 INFO:teuthology.orchestra.run.smithi112.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T00:30:07.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:07.128+0000 7f2d12120640 1 -- 172.21.15.112:0/386230251 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2cf8078b90 msgr2=0x7f2cf807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:07.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:07.128+0000 7f2d12120640 1 --2- 172.21.15.112:0/386230251 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2cf8078b90 0x7f2cf807b0a0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc006fd0 tx=0x7f2cfc006b60 comp rx=0 tx=0).stop 2024-09-21T00:30:07.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:07.128+0000 7f2d12120640 1 -- 172.21.15.112:0/386230251 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2d0c09e9c0 msgr2=0x7f2d0c09edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:07.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:07.128+0000 7f2d12120640 1 --2- 172.21.15.112:0/386230251 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2d0c09e9c0 0x7f2d0c09edd0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f2d0402b3a0 tx=0x7f2d0400c260 comp rx=0 tx=0).stop 2024-09-21T00:30:07.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:07.128+0000 7f2d12120640 1 -- 172.21.15.112:0/386230251 shutdown_connections 2024-09-21T00:30:07.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:07.128+0000 7f2d12120640 1 --2- 172.21.15.112:0/386230251 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2cf8078b90 0x7f2cf807b0a0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:07.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:07.128+0000 7f2d12120640 1 --2- 172.21.15.112:0/386230251 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2d0c09e9c0 0x7f2d0c09edd0 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:07.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:07.128+0000 7f2d12120640 1 --2- 172.21.15.112:0/386230251 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2d0c1f54f0 0x7f2d0c158aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:07.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:07.128+0000 7f2d12120640 1 --2- 172.21.15.112:0/386230251 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2cf40051a0 0x7f2d0c0a0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:07.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:07.128+0000 7f2d12120640 1 -- 172.21.15.112:0/386230251 >> 172.21.15.112:0/386230251 conn(0x7f2d0c08bf90 msgr2=0x7f2d0c161c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:07.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:07.128+0000 7f2d12120640 1 -- 172.21.15.112:0/386230251 shutdown_connections 2024-09-21T00:30:07.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:07.128+0000 7f2d12120640 1 -- 172.21.15.112:0/386230251 wait complete. 2024-09-21T00:30:07.141 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T00:30:07.141 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T00:30:07.141 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi112.front.sepia.ceph.com 2024-09-21T00:30:07.141 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T00:30:07.141 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T00:30:07.141 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T00:30:07.141 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-21T00:30:07.141 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T00:30:07.141 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T00:30:07.141 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:30:07.141 DEBUG:teuthology.orchestra.run.smithi112:> ip addr 2024-09-21T00:30:07.157 INFO:teuthology.orchestra.run.smithi112.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T00:30:07.158 INFO:teuthology.orchestra.run.smithi112.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T00:30:07.158 INFO:teuthology.orchestra.run.smithi112.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T00:30:07.158 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:30:07.158 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 ::1/128 scope host 2024-09-21T00:30:07.158 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:30:07.158 INFO:teuthology.orchestra.run.smithi112.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T00:30:07.158 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:d8:a5:2c brd ff:ff:ff:ff:ff:ff 2024-09-21T00:30:07.158 INFO:teuthology.orchestra.run.smithi112.stdout: altname enp7s0f0 2024-09-21T00:30:07.158 INFO:teuthology.orchestra.run.smithi112.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T00:30:07.158 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:d8:a5:2d brd ff:ff:ff:ff:ff:ff 2024-09-21T00:30:07.158 INFO:teuthology.orchestra.run.smithi112.stdout: altname enp7s0f1 2024-09-21T00:30:07.158 INFO:teuthology.orchestra.run.smithi112.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T00:30:07.158 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:8f:cb:68 brd ff:ff:ff:ff:ff:ff 2024-09-21T00:30:07.158 INFO:teuthology.orchestra.run.smithi112.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T00:30:07.159 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:8f:cb:69 brd ff:ff:ff:ff:ff:ff 2024-09-21T00:30:07.159 INFO:teuthology.orchestra.run.smithi112.stdout: inet 172.21.15.112/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-21T00:30:07.159 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft 42515sec preferred_lft 42515sec 2024-09-21T00:30:07.159 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::ec4:7aff:fe8f:cb69/64 scope link noprefixroute 2024-09-21T00:30:07.159 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:30:07.159 INFO:teuthology.orchestra.run.smithi112.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T00:30:07.159 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 1a:39:64:e7:fe:d7 brd ff:ff:ff:ff:ff:ff 2024-09-21T00:30:07.159 INFO:teuthology.orchestra.run.smithi112.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T00:30:07.159 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:30:07.159 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::5858:3dff:feaf:1774/64 scope link 2024-09-21T00:30:07.159 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:30:07.159 INFO:teuthology.orchestra.run.smithi112.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T00:30:07.159 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 1a:39:64:e7:fe:d7 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T00:30:07.159 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::1839:64ff:fee7:fed7/64 scope link 2024-09-21T00:30:07.159 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:30:07.160 INFO:teuthology.orchestra.run.smithi112.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T00:30:07.160 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 86:c9:6c:ef:e4:e9 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T00:30:07.160 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::84c9:6cff:feef:e4e9/64 scope link 2024-09-21T00:30:07.160 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:30:07.160 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:30:07.160 DEBUG:teuthology.orchestra.run.smithi112:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T00:30:07.220 INFO:teuthology.orchestra.run.smithi112.stdout:1 2024-09-21T00:30:07.222 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:30:07.222 DEBUG:teuthology.orchestra.run.smithi112:> route 2024-09-21T00:30:07.278 INFO:teuthology.orchestra.run.smithi112.stdout:Kernel IP routing table 2024-09-21T00:30:07.278 INFO:teuthology.orchestra.run.smithi112.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T00:30:07.278 INFO:teuthology.orchestra.run.smithi112.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-21T00:30:07.278 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-21T00:30:07.278 INFO:teuthology.orchestra.run.smithi112.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T00:30:07.278 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T00:30:07.278 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2024-09-21T00:30:07.278 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-21T00:30:07.279 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-21T00:30:07.279 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-21T00:30:07.279 DEBUG:teuthology.orchestra.run.smithi112:> ') 2024-09-21T00:30:07.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T00:30:07.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T00:30:07.404 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:30:07.404 DEBUG:teuthology.orchestra.run.smithi112:> ip netns list 2024-09-21T00:30:07.418 INFO:teuthology.orchestra.run.smithi112.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-21T00:30:07.419 INFO:teuthology.orchestra.run.smithi112.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T00:30:07.419 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T00:30:07.419 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:30:07.419 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link set brx.0 up 2024-09-21T00:30:07.483 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T00:30:07.483 DEBUG:teuthology.orchestra.run.smithi112:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T00:30:07.539 INFO:teuthology.orchestra.run.smithi112.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T00:30:07.539 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T00:30:07.539 DEBUG:teuthology.orchestra.run.smithi112:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T00:30:07.594 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-21T00:30:07.594 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:30:07.595 DEBUG:teuthology.orchestra.run.smithi112:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T00:30:07.617 INFO:tasks.ceph.mds.c.smithi112.stderr:2024-09-21T00:30:07.616+0000 7fb1438b0640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T00:30:07.671 INFO:teuthology.orchestra.run.smithi112.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T00:30:07.671 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T00:30:07.671 INFO:teuthology.orchestra.run.smithi112.stdout:invalid new device string format 2024-09-21T00:30:07.682 INFO:teuthology.orchestra.run.smithi112.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T00:30:07.682 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T00:30:07.682 INFO:teuthology.orchestra.run.smithi112.stdout:invalid new device string format 2024-09-21T00:30:07.682 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: resolved to: "172.21.15.112:3300,172.21.15.187:3300,172.21.15.187:3301" 2024-09-21T00:30:07.682 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T00:30:07.688 INFO:teuthology.orchestra.run.smithi112.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T00:30:07.688 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T00:30:07.688 INFO:teuthology.orchestra.run.smithi112.stdout:invalid new device string format 2024-09-21T00:30:07.688 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: resolved to: "172.21.15.112:3300,172.21.15.187:3300,172.21.15.187:3301" 2024-09-21T00:30:07.688 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T00:30:07.688 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: resolved to: "172.21.15.112:6789,172.21.15.187:6789,172.21.15.187:6790" 2024-09-21T00:30:07.689 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.112:6789,172.21.15.187:6789,172.21.15.187:6790:/ 2024-09-21T00:30:07.689 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=8b34692b-dc65-49b2-a08f-26f707b96ac5" will pass to kernel 2024-09-21T00:30:07.689 INFO:teuthology.orchestra.run.smithi112.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-21T00:30:07.689 INFO:teuthology.orchestra.run.smithi112.stdout: You just mounted a file system that supports labels which does not 2024-09-21T00:30:07.689 INFO:teuthology.orchestra.run.smithi112.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T00:30:07.689 INFO:teuthology.orchestra.run.smithi112.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T00:30:07.689 INFO:teuthology.orchestra.run.smithi112.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T00:30:07.691 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T00:30:07.691 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:30:07.691 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T00:30:07.717 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T00:30:07.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:07 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T00:30:07.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T00:30:07.811 INFO:teuthology.orchestra.run.smithi112.stdout:client7470 2024-09-21T00:30:07.811 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T00:30:07.811 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:30:07.827 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T00:30:07.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:07.978+0000 7f1b6a686640 1 -- 172.21.15.112:0/1494613287 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b4c004800 msgr2=0x7f1b4c004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:07.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:07.978+0000 7f1b6a686640 1 --2- 172.21.15.112:0/1494613287 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b4c004800 0x7f1b4c004c10 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f1b58001690 tx=0x7f1b4c005830 comp rx=0 tx=0).stop 2024-09-21T00:30:07.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:07.978+0000 7f1b6a686640 1 -- 172.21.15.112:0/1494613287 shutdown_connections 2024-09-21T00:30:07.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:07.978+0000 7f1b6a686640 1 --2- 172.21.15.112:0/1494613287 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b4c004800 0x7f1b4c004c10 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:07.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:07.978+0000 7f1b6a686640 1 -- 172.21.15.112:0/1494613287 >> 172.21.15.112:0/1494613287 conn(0x7f1b6408bf90 msgr2=0x7f1b640d4650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:07.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:07.978+0000 7f1b6a686640 1 -- 172.21.15.112:0/1494613287 shutdown_connections 2024-09-21T00:30:07.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:07.978+0000 7f1b6a686640 1 -- 172.21.15.112:0/1494613287 wait complete. 2024-09-21T00:30:07.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:07.980+0000 7f1b6a686640 1 Processor -- start 2024-09-21T00:30:07.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:07.980+0000 7f1b6a686640 1 -- start start 2024-09-21T00:30:07.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:07.981+0000 7f1b6a686640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1b4c004800 0x7f1b6414d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:07.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:07.982+0000 7f1b6a686640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1b6415bb70 0x7f1b641f97e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:07.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:07.982+0000 7f1b6a686640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b6406e910 0x7f1b6406ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:07.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:07.982+0000 7f1b6a686640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1b640b6660 con 0x7f1b6406e910 2024-09-21T00:30:07.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:07.982+0000 7f1b6a686640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f1b640b64e0 con 0x7f1b4c004800 2024-09-21T00:30:07.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:07.982+0000 7f1b6a686640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f1b640b67e0 con 0x7f1b6415bb70 2024-09-21T00:30:07.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:07.982+0000 7f1b68c23640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b6406e910 0x7f1b6406ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:07.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:07.982+0000 7f1b68c23640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b6406e910 0x7f1b6406ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35604/0 (socket says 172.21.15.112:35604) 2024-09-21T00:30:07.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:07.982+0000 7f1b68c23640 1 -- 172.21.15.112:0/4180147444 learned_addr learned my addr 172.21.15.112:0/4180147444 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:30:07.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:07.982+0000 7f1b637fe640 1 --2- 172.21.15.112:0/4180147444 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1b6415bb70 0x7f1b641f97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:07.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:07.982+0000 7f1b63fff640 1 --2- 172.21.15.112:0/4180147444 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1b4c004800 0x7f1b6414d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:07.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:07.982+0000 7f1b68c23640 1 -- 172.21.15.112:0/4180147444 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1b6415bb70 msgr2=0x7f1b641f97e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:07.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:07.982+0000 7f1b68c23640 1 --2- 172.21.15.112:0/4180147444 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1b6415bb70 0x7f1b641f97e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:07.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:07.982+0000 7f1b68c23640 1 -- 172.21.15.112:0/4180147444 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1b4c004800 msgr2=0x7f1b6414d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:07.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:07.982+0000 7f1b68c23640 1 --2- 172.21.15.112:0/4180147444 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1b4c004800 0x7f1b6414d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:07.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:07.982+0000 7f1b68c23640 1 -- 172.21.15.112:0/4180147444 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b6406f260 con 0x7f1b6406e910 2024-09-21T00:30:07.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:07.982+0000 7f1b68c23640 1 --2- 172.21.15.112:0/4180147444 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b6406e910 0x7f1b6406ed20 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f1b640d43e0 tx=0x7f1b5c00ea20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:07.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:07.982+0000 7f1b60ff9640 1 -- 172.21.15.112:0/4180147444 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b5c010890 con 0x7f1b6406e910 2024-09-21T00:30:07.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:07.982+0000 7f1b60ff9640 1 -- 172.21.15.112:0/4180147444 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1b5c018070 con 0x7f1b6406e910 2024-09-21T00:30:07.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:07.982+0000 7f1b6a686640 1 -- 172.21.15.112:0/4180147444 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b6414fda0 con 0x7f1b6406e910 2024-09-21T00:30:07.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:07.982+0000 7f1b60ff9640 1 -- 172.21.15.112:0/4180147444 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b5c010dc0 con 0x7f1b6406e910 2024-09-21T00:30:07.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:07.982+0000 7f1b6a686640 1 -- 172.21.15.112:0/4180147444 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b64150360 con 0x7f1b6406e910 2024-09-21T00:30:07.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:07.983+0000 7f1b6a686640 1 -- 172.21.15.112:0/4180147444 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b640d2480 con 0x7f1b6406e910 2024-09-21T00:30:07.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:07.984+0000 7f1b60ff9640 1 -- 172.21.15.112:0/4180147444 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1b5c01d460 con 0x7f1b6406e910 2024-09-21T00:30:07.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:07.985+0000 7f1b60ff9640 1 --2- 172.21.15.112:0/4180147444 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1b50078b90 0x7f1b5007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:07.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:07.985+0000 7f1b60ff9640 1 -- 172.21.15.112:0/4180147444 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 6187+0+0 (secure 0 0 0) 0x7f1b5c09b1a0 con 0x7f1b6406e910 2024-09-21T00:30:07.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:07.985+0000 7f1b63fff640 1 --2- 172.21.15.112:0/4180147444 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1b50078b90 0x7f1b5007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:07.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:07.985+0000 7f1b63fff640 1 --2- 172.21.15.112:0/4180147444 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1b50078b90 0x7f1b5007b0a0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f1b54009d60 tx=0x7f1b54005f30 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:07.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:07.989+0000 7f1b60ff9640 1 -- 172.21.15.112:0/4180147444 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1b6406f260 con 0x7f1b6406e910 2024-09-21T00:30:08.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:08.203+0000 7f1b6a686640 1 -- 172.21.15.112:0/4180147444 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f1b6414fb30 con 0x7f1b6406e910 2024-09-21T00:30:08.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:08.203+0000 7f1b60ff9640 1 -- 172.21.15.112:0/4180147444 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v76) ==== 53+0+73 (secure 0 0 0) 0x7f1b5c063490 con 0x7f1b6406e910 2024-09-21T00:30:08.205 INFO:teuthology.orchestra.run.smithi112.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T00:30:08.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:08.205+0000 7f1b6a686640 1 -- 172.21.15.112:0/4180147444 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1b50078b90 msgr2=0x7f1b5007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:08.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:08.205+0000 7f1b6a686640 1 --2- 172.21.15.112:0/4180147444 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1b50078b90 0x7f1b5007b0a0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f1b54009d60 tx=0x7f1b54005f30 comp rx=0 tx=0).stop 2024-09-21T00:30:08.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:08.205+0000 7f1b6a686640 1 -- 172.21.15.112:0/4180147444 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b6406e910 msgr2=0x7f1b6406ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:08.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:08.205+0000 7f1b6a686640 1 --2- 172.21.15.112:0/4180147444 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b6406e910 0x7f1b6406ed20 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f1b640d43e0 tx=0x7f1b5c00ea20 comp rx=0 tx=0).stop 2024-09-21T00:30:08.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:08.205+0000 7f1b6a686640 1 -- 172.21.15.112:0/4180147444 shutdown_connections 2024-09-21T00:30:08.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:08.205+0000 7f1b6a686640 1 --2- 172.21.15.112:0/4180147444 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1b50078b90 0x7f1b5007b0a0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:08.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:08.205+0000 7f1b6a686640 1 --2- 172.21.15.112:0/4180147444 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b6406e910 0x7f1b6406ed20 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:08.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:08.205+0000 7f1b6a686640 1 --2- 172.21.15.112:0/4180147444 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1b6415bb70 0x7f1b641f97e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:08.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:08.205+0000 7f1b6a686640 1 --2- 172.21.15.112:0/4180147444 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1b4c004800 0x7f1b6414d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:08.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:08.205+0000 7f1b6a686640 1 -- 172.21.15.112:0/4180147444 >> 172.21.15.112:0/4180147444 conn(0x7f1b6408bf90 msgr2=0x7f1b641604e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:08.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:08.205+0000 7f1b6a686640 1 -- 172.21.15.112:0/4180147444 shutdown_connections 2024-09-21T00:30:08.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:08.205+0000 7f1b6a686640 1 -- 172.21.15.112:0/4180147444 wait complete. 2024-09-21T00:30:08.218 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T00:30:08.218 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T00:30:08.218 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi187.front.sepia.ceph.com 2024-09-21T00:30:08.218 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-21T00:30:08.218 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-21T00:30:08.218 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T00:30:08.218 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-21T00:30:08.218 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T00:30:08.218 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-21T00:30:08.219 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:30:08.219 DEBUG:teuthology.orchestra.run.smithi187:> ip addr 2024-09-21T00:30:08.238 INFO:teuthology.orchestra.run.smithi187.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T00:30:08.238 INFO:teuthology.orchestra.run.smithi187.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T00:30:08.238 INFO:teuthology.orchestra.run.smithi187.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T00:30:08.238 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:30:08.238 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 ::1/128 scope host 2024-09-21T00:30:08.239 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:30:08.239 INFO:teuthology.orchestra.run.smithi187.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T00:30:08.239 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 0c:c4:7a:d9:36:78 brd ff:ff:ff:ff:ff:ff 2024-09-21T00:30:08.239 INFO:teuthology.orchestra.run.smithi187.stdout: altname enp7s0f0 2024-09-21T00:30:08.239 INFO:teuthology.orchestra.run.smithi187.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T00:30:08.239 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 0c:c4:7a:d9:36:79 brd ff:ff:ff:ff:ff:ff 2024-09-21T00:30:08.239 INFO:teuthology.orchestra.run.smithi187.stdout: altname enp7s0f1 2024-09-21T00:30:08.239 INFO:teuthology.orchestra.run.smithi187.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T00:30:08.239 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 0c:c4:7a:88:77:b0 brd ff:ff:ff:ff:ff:ff 2024-09-21T00:30:08.239 INFO:teuthology.orchestra.run.smithi187.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T00:30:08.239 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 0c:c4:7a:88:77:b1 brd ff:ff:ff:ff:ff:ff 2024-09-21T00:30:08.239 INFO:teuthology.orchestra.run.smithi187.stdout: inet 172.21.15.187/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-21T00:30:08.239 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft 42503sec preferred_lft 42503sec 2024-09-21T00:30:08.240 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 fe80::ec4:7aff:fe88:77b1/64 scope link noprefixroute 2024-09-21T00:30:08.240 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:30:08.240 INFO:teuthology.orchestra.run.smithi187.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T00:30:08.240 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 86:3b:f6:5b:1b:32 brd ff:ff:ff:ff:ff:ff 2024-09-21T00:30:08.240 INFO:teuthology.orchestra.run.smithi187.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T00:30:08.240 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:30:08.240 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 fe80::ac6f:97ff:fe09:b82e/64 scope link 2024-09-21T00:30:08.240 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:30:08.240 INFO:teuthology.orchestra.run.smithi187.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T00:30:08.240 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether ca:0b:a0:f5:2b:b5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T00:30:08.240 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 fe80::c80b:a0ff:fef5:2bb5/64 scope link 2024-09-21T00:30:08.240 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:30:08.240 INFO:teuthology.orchestra.run.smithi187.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T00:30:08.240 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 86:3b:f6:5b:1b:32 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-21T00:30:08.240 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 fe80::843b:f6ff:fe5b:1b32/64 scope link 2024-09-21T00:30:08.241 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-09-21T00:30:08.241 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:30:08.241 DEBUG:teuthology.orchestra.run.smithi187:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T00:30:08.306 INFO:teuthology.orchestra.run.smithi187.stdout:1 2024-09-21T00:30:08.308 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:30:08.308 DEBUG:teuthology.orchestra.run.smithi187:> route 2024-09-21T00:30:08.367 INFO:teuthology.orchestra.run.smithi187.stdout:Kernel IP routing table 2024-09-21T00:30:08.367 INFO:teuthology.orchestra.run.smithi187.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T00:30:08.368 INFO:teuthology.orchestra.run.smithi187.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-21T00:30:08.368 INFO:teuthology.orchestra.run.smithi187.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-21T00:30:08.368 INFO:teuthology.orchestra.run.smithi187.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T00:30:08.368 DEBUG:teuthology.orchestra.run.smithi187:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T00:30:08.368 DEBUG:teuthology.orchestra.run.smithi187:> set -e 2024-09-21T00:30:08.368 DEBUG:teuthology.orchestra.run.smithi187:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-21T00:30:08.368 DEBUG:teuthology.orchestra.run.smithi187:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-21T00:30:08.368 DEBUG:teuthology.orchestra.run.smithi187:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-21T00:30:08.368 DEBUG:teuthology.orchestra.run.smithi187:> ') 2024-09-21T00:30:08.463 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:30:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T00:30:08.497 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:30:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T00:30:08.504 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:30:08.504 DEBUG:teuthology.orchestra.run.smithi187:> ip netns list 2024-09-21T00:30:08.520 INFO:teuthology.orchestra.run.smithi187.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-21T00:30:08.520 INFO:teuthology.orchestra.run.smithi187.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-21T00:30:08.520 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-21T00:30:08.520 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:30:08.520 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip link set brx.0 up 2024-09-21T00:30:08.585 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T00:30:08.586 DEBUG:teuthology.orchestra.run.smithi187:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-21T00:30:08.642 INFO:teuthology.orchestra.run.smithi187.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-21T00:30:08.642 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T00:30:08.642 DEBUG:teuthology.orchestra.run.smithi187:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-21T00:30:08.697 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-21T00:30:08.697 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:30:08.697 DEBUG:teuthology.orchestra.run.smithi187:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-21T00:30:08.791 INFO:teuthology.orchestra.run.smithi187.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-21T00:30:08.792 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-21T00:30:08.792 INFO:teuthology.orchestra.run.smithi187.stdout:invalid new device string format 2024-09-21T00:30:08.802 INFO:teuthology.orchestra.run.smithi187.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-21T00:30:08.802 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-21T00:30:08.802 INFO:teuthology.orchestra.run.smithi187.stdout:invalid new device string format 2024-09-21T00:30:08.802 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: resolved to: "172.21.15.112:3300,172.21.15.187:3300,172.21.15.187:3301" 2024-09-21T00:30:08.803 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T00:30:08.808 INFO:teuthology.orchestra.run.smithi187.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-21T00:30:08.808 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-21T00:30:08.808 INFO:teuthology.orchestra.run.smithi187.stdout:invalid new device string format 2024-09-21T00:30:08.808 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: resolved to: "172.21.15.112:3300,172.21.15.187:3300,172.21.15.187:3301" 2024-09-21T00:30:08.808 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T00:30:08.808 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: resolved to: "172.21.15.112:6789,172.21.15.187:6789,172.21.15.187:6790" 2024-09-21T00:30:08.808 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.112:6789,172.21.15.187:6789,172.21.15.187:6790:/ 2024-09-21T00:30:08.808 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=8b34692b-dc65-49b2-a08f-26f707b96ac5" will pass to kernel 2024-09-21T00:30:08.810 INFO:teuthology.orchestra.run.smithi187.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-09-21T00:30:08.810 INFO:teuthology.orchestra.run.smithi187.stdout: You just mounted a file system that supports labels which does not 2024-09-21T00:30:08.811 INFO:teuthology.orchestra.run.smithi187.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T00:30:08.811 INFO:teuthology.orchestra.run.smithi187.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T00:30:08.811 INFO:teuthology.orchestra.run.smithi187.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T00:30:08.811 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T00:30:08.811 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:30:08.811 DEBUG:teuthology.orchestra.run.smithi187:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-21T00:30:08.841 DEBUG:teuthology.orchestra.run.smithi187:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T00:30:08.929 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:30:08 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T00:30:08.931 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T00:30:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T00:30:08.937 INFO:teuthology.orchestra.run.smithi187.stdout:client7479 2024-09-21T00:30:08.937 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-09-21T00:30:08.937 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:30:08.953 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2024-09-21T00:30:09.084 INFO:teuthology.orchestra.run.smithi112.stdout:[{"id":7479,"entity":{"name":{"type":"client","num":7479},"addr":{"type":"v1","addr":"172.21.15.187:0","nonce":579439030}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":3,"uptime":0.26999347800000001,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":0.99938099513942213,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":4,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.7479 v1:172.21.15.187:0/579439030","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x000000000003fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"1","hostname":"smithi187","kernel_version":"5.14.0-508.el9.x86_64","root":"/"}},{"id":7470,"entity":{"name":{"type":"client","num":7470},"addr":{"type":"v1","addr":"192.168.0.1:0","nonce":3384400602}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":3,"uptime":1.389966424,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":0.99679825306934633,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":4,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.7470 v1:192.168.0.1:0/3384400602","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x000000000003fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"0","hostname":"smithi112","kernel_version":"5.14.0-508.el9.x86_64","root":"/"}}] 2024-09-21T00:30:09.085 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000003fbff" }, "entity_id": "1", "hostname": "smithi187", "kernel_version": "5.14.0-508.el9.x86_64", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "172.21.15.187:0", "nonce": 579439030, "type": "v1" }, "name": { "num": 7479, "type": "client" } }, "id": 7479, "inst": "client.7479 v1:172.21.15.187:0/579439030", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 4, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 3, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 0.9993809951394221 }, "state": "open", "uptime": 0.269993478 }, { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000003fbff" }, "entity_id": "0", "hostname": "smithi112", "kernel_version": "5.14.0-508.el9.x86_64", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 3384400602, "type": "v1" }, "name": { "num": 7470, "type": "client" } }, "id": 7470, "inst": "client.7470 v1:192.168.0.1:0/3384400602", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 4, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 3, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 0.9967982530693463 }, "state": "open", "uptime": 1.389966424 } ] 2024-09-21T00:30:09.085 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-09-21T00:30:09.223 INFO:teuthology.orchestra.run.smithi112.stdout:{"message":"","return_code":0} 2024-09-21T00:30:09.223 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-09-21T00:30:09.224 DEBUG:teuthology.orchestra.run.smithi112:> cephfs-table-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 0 show session 2024-09-21T00:30:09.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.275+0000 7fb2471df280 1 -- start start 2024-09-21T00:30:09.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.275+0000 7fb2471df280 1 Processor -- start 2024-09-21T00:30:09.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.276+0000 7fb2471df280 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5623a1869180 0x5623a185eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:09.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.276+0000 7fb2471df280 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5623a1866480 0x5623a185e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:09.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.276+0000 7fb2471df280 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5623a1866900 0x5623a185e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:09.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.276+0000 7fb2471df280 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5623a186c000 con 0x5623a1866900 2024-09-21T00:30:09.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.276+0000 7fb2471df280 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5623a186c180 con 0x5623a1866480 2024-09-21T00:30:09.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.276+0000 7fb2471df280 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x5623a186c300 con 0x5623a1869180 2024-09-21T00:30:09.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.276+0000 7fb246185640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5623a1866900 0x5623a185e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:09.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.276+0000 7fb246185640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5623a1866900 0x5623a185e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35622/0 (socket says 172.21.15.112:35622) 2024-09-21T00:30:09.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.276+0000 7fb246185640 1 -- 172.21.15.112:0/3312473909 learned_addr learned my addr 172.21.15.112:0/3312473909 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:30:09.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.276+0000 7fb245984640 1 --2- 172.21.15.112:0/3312473909 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5623a1869180 0x5623a185eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:09.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.276+0000 7fb245183640 1 --2- 172.21.15.112:0/3312473909 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5623a1866480 0x5623a185e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:09.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.276+0000 7fb246185640 1 -- 172.21.15.112:0/3312473909 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5623a1869180 msgr2=0x5623a185eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:30:09.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.276+0000 7fb246185640 1 --2- 172.21.15.112:0/3312473909 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5623a1869180 0x5623a185eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:09.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.276+0000 7fb246185640 1 -- 172.21.15.112:0/3312473909 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5623a1866480 msgr2=0x5623a185e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:30:09.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.276+0000 7fb246185640 1 --2- 172.21.15.112:0/3312473909 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5623a1866480 0x5623a185e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:09.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.276+0000 7fb246185640 1 -- 172.21.15.112:0/3312473909 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623a0ac4780 con 0x5623a1866900 2024-09-21T00:30:09.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.276+0000 7fb246185640 1 --2- 172.21.15.112:0/3312473909 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5623a1866900 0x5623a185e000 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x5623a18cb8f0 tx=0x5623a186e060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ee24660576d7361 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:09.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.277+0000 7fb24317f640 1 -- 172.21.15.112:0/3312473909 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x56239ca7cb60 con 0x5623a1866900 2024-09-21T00:30:09.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.277+0000 7fb24317f640 1 -- 172.21.15.112:0/3312473909 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x56239ca82540 con 0x5623a1866900 2024-09-21T00:30:09.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.277+0000 7fb24317f640 1 -- 172.21.15.112:0/3312473909 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x56239ca7c340 con 0x5623a1866900 2024-09-21T00:30:09.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.277+0000 7fb2471df280 1 -- 172.21.15.112:0/3312473909 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623a0ac43c0 con 0x5623a1866900 2024-09-21T00:30:09.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.277+0000 7fb2471df280 1 -- 172.21.15.112:0/3312473909 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({fsmap=0}) -- 0x5623a0ac5680 con 0x5623a1866900 2024-09-21T00:30:09.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.277+0000 7fb2471df280 4 waiting for MDS map... 2024-09-21T00:30:09.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.277+0000 7fb24317f640 1 -- 172.21.15.112:0/3312473909 <== mon.0 v2:172.21.15.112:3300/0 4 ==== osd_map(112..112 src has 1..112) ==== 6187+0+0 (secure 0 0 0) 0x5623a1976600 con 0x5623a1866900 2024-09-21T00:30:09.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.277+0000 7fb24317f640 1 -- 172.21.15.112:0/3312473909 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 76) ==== 3502+0+0 (secure 0 0 0) 0x5623a0ae1180 con 0x5623a1866900 2024-09-21T00:30:09.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.277+0000 7fb2471df280 4 Got MDS map 76 2024-09-21T00:30:09.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.277+0000 7fb2471df280 10 main: main 2024-09-21T00:30:09.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.277+0000 7fb2471df280 4 main: connecting to RADOS... 2024-09-21T00:30:09.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.278+0000 7fb2471df280 1 Processor -- start 2024-09-21T00:30:09.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.278+0000 7fb2471df280 1 -- start start 2024-09-21T00:30:09.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.279+0000 7fb2471df280 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5623a1867200 0x5623a185f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:09.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.279+0000 7fb2471df280 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5623a1867680 0x5623a185fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:09.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.279+0000 7fb2471df280 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5623a1867b00 0x5623a1860100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:09.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.279+0000 7fb2471df280 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5623a186c480 con 0x5623a1867200 2024-09-21T00:30:09.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.279+0000 7fb2471df280 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5623a186c600 con 0x5623a1867b00 2024-09-21T00:30:09.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.279+0000 7fb2471df280 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x5623a186c780 con 0x5623a1867680 2024-09-21T00:30:09.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.279+0000 7fb245183640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5623a1867200 0x5623a185f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:09.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.279+0000 7fb245183640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5623a1867200 0x5623a185f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35624/0 (socket says 172.21.15.112:35624) 2024-09-21T00:30:09.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.279+0000 7fb245183640 1 -- 172.21.15.112:0/1490593449 learned_addr learned my addr 172.21.15.112:0/1490593449 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:30:09.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.279+0000 7fb245984640 1 --2- 172.21.15.112:0/1490593449 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5623a1867680 0x5623a185fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:09.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.279+0000 7fb245183640 1 --2- 172.21.15.112:0/1490593449 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5623a1867b00 0x5623a1860100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:09.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.279+0000 7fb245183640 1 -- 172.21.15.112:0/1490593449 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5623a1867680 msgr2=0x5623a185fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:30:09.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.279+0000 7fb245183640 1 --2- 172.21.15.112:0/1490593449 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5623a1867680 0x5623a185fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:09.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.279+0000 7fb245183640 1 -- 172.21.15.112:0/1490593449 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5623a1867b00 msgr2=0x5623a1860100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:30:09.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.279+0000 7fb245183640 1 --2- 172.21.15.112:0/1490593449 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5623a1867b00 0x5623a1860100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:09.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.280+0000 7fb245183640 1 -- 172.21.15.112:0/1490593449 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623a0ac5860 con 0x5623a1867200 2024-09-21T00:30:09.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.280+0000 7fb245183640 1 --2- 172.21.15.112:0/1490593449 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5623a1867200 0x5623a185f600 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x5623a18cae40 tx=0x5623a1994c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6aad5e330d530969 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:09.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.280+0000 7fb24117b640 1 -- 172.21.15.112:0/1490593449 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x56239ca7d380 con 0x5623a1867200 2024-09-21T00:30:09.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.280+0000 7fb24117b640 1 -- 172.21.15.112:0/1490593449 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x56239ca82fc0 con 0x5623a1867200 2024-09-21T00:30:09.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.280+0000 7fb24117b640 1 -- 172.21.15.112:0/1490593449 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x56239ca7d1e0 con 0x5623a1867200 2024-09-21T00:30:09.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.280+0000 7fb2471df280 1 -- 172.21.15.112:0/1490593449 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5623a1867200 msgr2=0x5623a185f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:09.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.280+0000 7fb2471df280 1 --2- 172.21.15.112:0/1490593449 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5623a1867200 0x5623a185f600 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x5623a18cae40 tx=0x5623a1994c60 comp rx=0 tx=0).stop 2024-09-21T00:30:09.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.280+0000 7fb2471df280 1 -- 172.21.15.112:0/1490593449 shutdown_connections 2024-09-21T00:30:09.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.280+0000 7fb2471df280 1 --2- 172.21.15.112:0/1490593449 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5623a1867b00 0x5623a1860100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:09.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.280+0000 7fb2471df280 1 --2- 172.21.15.112:0/1490593449 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5623a1867680 0x5623a185fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:09.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.280+0000 7fb2471df280 1 --2- 172.21.15.112:0/1490593449 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5623a1867200 0x5623a185f600 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:09.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.280+0000 7fb2471df280 1 -- 172.21.15.112:0/1490593449 >> 172.21.15.112:0/1490593449 conn(0x5623a1866d80 msgr2=0x5623a0b71800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:09.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.280+0000 7fb2471df280 1 -- 172.21.15.112:0/1490593449 shutdown_connections 2024-09-21T00:30:09.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.280+0000 7fb2471df280 1 -- 172.21.15.112:0/1490593449 wait complete. 2024-09-21T00:30:09.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.281+0000 7fb2471df280 1 Processor -- start 2024-09-21T00:30:09.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.281+0000 7fb2471df280 1 -- start start 2024-09-21T00:30:09.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.282+0000 7fb2471df280 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5623a1867200 0x5623a185f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:09.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.282+0000 7fb2471df280 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5623a1867680 0x5623a185fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:09.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.282+0000 7fb2471df280 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5623a1867b00 0x5623a1860100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:09.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.282+0000 7fb2471df280 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5623a186de00 con 0x5623a1867b00 2024-09-21T00:30:09.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.282+0000 7fb2471df280 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5623a186cd80 con 0x5623a1867200 2024-09-21T00:30:09.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.282+0000 7fb2471df280 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x5623a186cf00 con 0x5623a1867680 2024-09-21T00:30:09.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.282+0000 7fb245183640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5623a1867b00 0x5623a1860100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:09.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.282+0000 7fb245183640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5623a1867b00 0x5623a1860100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35632/0 (socket says 172.21.15.112:35632) 2024-09-21T00:30:09.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.282+0000 7fb245183640 1 -- 172.21.15.112:0/4000488620 learned_addr learned my addr 172.21.15.112:0/4000488620 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:30:09.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.283+0000 7fb245183640 1 -- 172.21.15.112:0/4000488620 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5623a1867680 msgr2=0x5623a185fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:09.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.283+0000 7fb245183640 1 --2- 172.21.15.112:0/4000488620 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5623a1867680 0x5623a185fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:09.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.283+0000 7fb245183640 1 -- 172.21.15.112:0/4000488620 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5623a1867200 msgr2=0x5623a185f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:09.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.283+0000 7fb245183640 1 --2- 172.21.15.112:0/4000488620 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5623a1867200 0x5623a185f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:09.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.283+0000 7fb245183640 1 -- 172.21.15.112:0/4000488620 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623a0ac54a0 con 0x5623a1867b00 2024-09-21T00:30:09.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.283+0000 7fb245183640 1 --2- 172.21.15.112:0/4000488620 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5623a1867b00 0x5623a1860100 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x5623a198f0b0 tx=0x5623a186e180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:09.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.283+0000 7fb23f978640 1 -- 172.21.15.112:0/4000488620 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x56239ca7d380 con 0x5623a1867b00 2024-09-21T00:30:09.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.283+0000 7fb23f978640 1 -- 172.21.15.112:0/4000488620 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x56239ca82e00 con 0x5623a1867b00 2024-09-21T00:30:09.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.283+0000 7fb2471df280 1 -- 172.21.15.112:0/4000488620 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5623a0ac4960 con 0x5623a1867b00 2024-09-21T00:30:09.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.283+0000 7fb23f978640 1 -- 172.21.15.112:0/4000488620 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x56239ca7d1e0 con 0x5623a1867b00 2024-09-21T00:30:09.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.283+0000 7fb2471df280 1 -- 172.21.15.112:0/4000488620 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623a0ac4000 con 0x5623a1867b00 2024-09-21T00:30:09.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.283+0000 7fb2471df280 4 main: resolving pool 17 2024-09-21T00:30:09.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.284+0000 7fb23f978640 1 -- 172.21.15.112:0/4000488620 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x5623a19ecc00 con 0x5623a1867b00 2024-09-21T00:30:09.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.285+0000 7fb23f978640 1 --2- 172.21.15.112:0/4000488620 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x5623a1868d00 0x5623a1860680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:09.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.285+0000 7fb23f978640 1 -- 172.21.15.112:0/4000488620 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 6187+0+0 (secure 0 0 0) 0x5623a19f2400 con 0x5623a1867b00 2024-09-21T00:30:09.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.285+0000 7fb2471df280 4 main: creating IoCtx.. 2024-09-21T00:30:09.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.285+0000 7fb2471df280 1 --2- 172.21.15.112:0/4000488620 >> [v2:172.21.15.112:6808/755760621,v1:172.21.15.112:6809/755760621] conn(0x5623a1867f80 0x5623a1860c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:09.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.285+0000 7fb2471df280 1 -- 172.21.15.112:0/4000488620 --> [v2:172.21.15.112:6808/755760621,v1:172.21.15.112:6809/755760621] -- osd_op(unknown.0.0:1 17.b 17:d0630e4c:::mds0_sessionmap:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112) -- 0x5623a0ea2000 con 0x5623a1867f80 2024-09-21T00:30:09.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.285+0000 7fb245984640 1 --2- 172.21.15.112:0/4000488620 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x5623a1868d00 0x5623a1860680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:09.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.285+0000 7fb245183640 1 --2- 172.21.15.112:0/4000488620 >> [v2:172.21.15.112:6808/755760621,v1:172.21.15.112:6809/755760621] conn(0x5623a1867f80 0x5623a1860c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:09.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.285+0000 7fb245984640 1 --2- 172.21.15.112:0/4000488620 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x5623a1868d00 0x5623a1860680 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x5623a0b4cc60 tx=0x5623a0ea4600 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:09.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.285+0000 7fb245183640 1 --2- 172.21.15.112:0/4000488620 >> [v2:172.21.15.112:6808/755760621,v1:172.21.15.112:6809/755760621] conn(0x5623a1867f80 0x5623a1860c00 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:09.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.286+0000 7fb245183640 1 -- 172.21.15.112:0/4000488620 <== osd.0 v2:172.21.15.112:6808/755760621 1 ==== osd_op_reply(1 mds0_sessionmap [omap-get-header out=14b] v0'0 uv5 ondisk = 0) ==== 159+0+14 (crc 0 0 0) 0x5623a1854000 con 0x5623a1867f80 2024-09-21T00:30:09.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.286+0000 7fb2471df280 1 -- 172.21.15.112:0/4000488620 --> [v2:172.21.15.112:6808/755760621,v1:172.21.15.112:6809/755760621] -- osd_op(unknown.0.0:2 17.b 17:d0630e4c:::mds0_sessionmap:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112) -- 0x5623a0ea2400 con 0x5623a1867f80 2024-09-21T00:30:09.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.286+0000 7fb245183640 1 -- 172.21.15.112:0/4000488620 <== osd.0 v2:172.21.15.112:6808/755760621 2 ==== osd_op_reply(2 mds0_sessionmap [omap-get-vals out=471b] v0'0 uv5 ondisk = 0) ==== 159+0+471 (crc 0 0 0) 0x5623a1854000 con 0x5623a1867f80 2024-09-21T00:30:09.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.286+0000 7fb2471df280 1 -- 172.21.15.112:0/4000488620 --> [v2:172.21.15.112:6808/755760621,v1:172.21.15.112:6809/755760621] -- osd_op(unknown.0.0:3 17.b 17:d0630e4c:::mds0_sessionmap:head [omap-get-vals in=27b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112) -- 0x5623a0ea3400 con 0x5623a1867f80 2024-09-21T00:30:09.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.287+0000 7fb245183640 1 -- 172.21.15.112:0/4000488620 <== osd.0 v2:172.21.15.112:6808/755760621 3 ==== osd_op_reply(3 mds0_sessionmap [omap-get-vals out=5b] v0'0 uv5 ondisk = 0) ==== 159+0+5 (crc 0 0 0) 0x5623a1854000 con 0x5623a1867f80 2024-09-21T00:30:09.294 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-21T00:30:09.294 INFO:teuthology.orchestra.run.smithi112.stdout: "0": { 2024-09-21T00:30:09.294 INFO:teuthology.orchestra.run.smithi112.stdout: "data": { 2024-09-21T00:30:09.294 INFO:teuthology.orchestra.run.smithi112.stdout: "sessions": [ 2024-09-21T00:30:09.294 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:30:09.294 INFO:teuthology.orchestra.run.smithi112.stdout: "id": 7479, 2024-09-21T00:30:09.294 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": { 2024-09-21T00:30:09.295 INFO:teuthology.orchestra.run.smithi112.stdout: "name": { 2024-09-21T00:30:09.295 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "client", 2024-09-21T00:30:09.295 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 7479 2024-09-21T00:30:09.295 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:30:09.295 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": { 2024-09-21T00:30:09.295 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-09-21T00:30:09.295 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.187:0", 2024-09-21T00:30:09.295 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 579439030 2024-09-21T00:30:09.295 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:30:09.295 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:30:09.295 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "open", 2024-09-21T00:30:09.295 INFO:teuthology.orchestra.run.smithi112.stdout: "num_leases": 0, 2024-09-21T00:30:09.295 INFO:teuthology.orchestra.run.smithi112.stdout: "num_caps": 0, 2024-09-21T00:30:09.295 INFO:teuthology.orchestra.run.smithi112.stdout: "request_load_avg": 0, 2024-09-21T00:30:09.295 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 0.00099997600000000008, 2024-09-21T00:30:09.296 INFO:teuthology.orchestra.run.smithi112.stdout: "requests_in_flight": 0, 2024-09-21T00:30:09.296 INFO:teuthology.orchestra.run.smithi112.stdout: "num_completed_requests": 0, 2024-09-21T00:30:09.296 INFO:teuthology.orchestra.run.smithi112.stdout: "num_completed_flushes": 0, 2024-09-21T00:30:09.296 INFO:teuthology.orchestra.run.smithi112.stdout: "reconnecting": false, 2024-09-21T00:30:09.296 INFO:teuthology.orchestra.run.smithi112.stdout: "recall_caps": { 2024-09-21T00:30:09.296 INFO:teuthology.orchestra.run.smithi112.stdout: "value": 0, 2024-09-21T00:30:09.296 INFO:teuthology.orchestra.run.smithi112.stdout: "halflife": 60 2024-09-21T00:30:09.296 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:30:09.296 INFO:teuthology.orchestra.run.smithi112.stdout: "release_caps": { 2024-09-21T00:30:09.296 INFO:teuthology.orchestra.run.smithi112.stdout: "value": 0, 2024-09-21T00:30:09.296 INFO:teuthology.orchestra.run.smithi112.stdout: "halflife": 60 2024-09-21T00:30:09.296 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:30:09.296 INFO:teuthology.orchestra.run.smithi112.stdout: "recall_caps_throttle": { 2024-09-21T00:30:09.296 INFO:teuthology.orchestra.run.smithi112.stdout: "value": 0, 2024-09-21T00:30:09.297 INFO:teuthology.orchestra.run.smithi112.stdout: "halflife": 1.5 2024-09-21T00:30:09.297 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:30:09.297 INFO:teuthology.orchestra.run.smithi112.stdout: "recall_caps_throttle2o": { 2024-09-21T00:30:09.297 INFO:teuthology.orchestra.run.smithi112.stdout: "value": 0, 2024-09-21T00:30:09.297 INFO:teuthology.orchestra.run.smithi112.stdout: "halflife": 0.5 2024-09-21T00:30:09.297 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:30:09.297 INFO:teuthology.orchestra.run.smithi112.stdout: "session_cache_liveness": { 2024-09-21T00:30:09.297 INFO:teuthology.orchestra.run.smithi112.stdout: "value": 0, 2024-09-21T00:30:09.297 INFO:teuthology.orchestra.run.smithi112.stdout: "halflife": 300 2024-09-21T00:30:09.297 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:30:09.297 INFO:teuthology.orchestra.run.smithi112.stdout: "cap_acquisition": { 2024-09-21T00:30:09.297 INFO:teuthology.orchestra.run.smithi112.stdout: "value": 0, 2024-09-21T00:30:09.297 INFO:teuthology.orchestra.run.smithi112.stdout: "halflife": 30 2024-09-21T00:30:09.297 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:30:09.297 INFO:teuthology.orchestra.run.smithi112.stdout: "last_trim_completed_requests_tid": 0, 2024-09-21T00:30:09.298 INFO:teuthology.orchestra.run.smithi112.stdout: "last_trim_completed_flushes_tid": 0, 2024-09-21T00:30:09.298 INFO:teuthology.orchestra.run.smithi112.stdout: "delegated_inos": [], 2024-09-21T00:30:09.298 INFO:teuthology.orchestra.run.smithi112.stdout: "inst": "client.7479 v1:172.21.15.187:0/579439030", 2024-09-21T00:30:09.298 INFO:teuthology.orchestra.run.smithi112.stdout: "completed_requests": [], 2024-09-21T00:30:09.298 INFO:teuthology.orchestra.run.smithi112.stdout: "prealloc_inos": [], 2024-09-21T00:30:09.298 INFO:teuthology.orchestra.run.smithi112.stdout: "client_metadata": { 2024-09-21T00:30:09.298 INFO:teuthology.orchestra.run.smithi112.stdout: "client_features": { 2024-09-21T00:30:09.298 INFO:teuthology.orchestra.run.smithi112.stdout: "feature_bits": "0x000000000003fbff" 2024-09-21T00:30:09.298 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:30:09.298 INFO:teuthology.orchestra.run.smithi112.stdout: "metric_spec": { 2024-09-21T00:30:09.298 INFO:teuthology.orchestra.run.smithi112.stdout: "metric_flags": { 2024-09-21T00:30:09.298 INFO:teuthology.orchestra.run.smithi112.stdout: "feature_bits": "0x000000000080ffff" 2024-09-21T00:30:09.298 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:30:09.298 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:30:09.299 INFO:teuthology.orchestra.run.smithi112.stdout: "entity_id": "1", 2024-09-21T00:30:09.299 INFO:teuthology.orchestra.run.smithi112.stdout: "hostname": "smithi187", 2024-09-21T00:30:09.299 INFO:teuthology.orchestra.run.smithi112.stdout: "kernel_version": "5.14.0-508.el9.x86_64", 2024-09-21T00:30:09.299 INFO:teuthology.orchestra.run.smithi112.stdout: "root": "/" 2024-09-21T00:30:09.299 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:30:09.299 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:30:09.299 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T00:30:09.299 INFO:teuthology.orchestra.run.smithi112.stdout: "id": 7470, 2024-09-21T00:30:09.299 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": { 2024-09-21T00:30:09.299 INFO:teuthology.orchestra.run.smithi112.stdout: "name": { 2024-09-21T00:30:09.299 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "client", 2024-09-21T00:30:09.299 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 7470 2024-09-21T00:30:09.299 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:30:09.299 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": { 2024-09-21T00:30:09.299 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-09-21T00:30:09.300 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "192.168.0.1:0", 2024-09-21T00:30:09.300 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 3384400602 2024-09-21T00:30:09.300 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:30:09.300 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:30:09.300 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "open", 2024-09-21T00:30:09.300 INFO:teuthology.orchestra.run.smithi112.stdout: "num_leases": 0, 2024-09-21T00:30:09.300 INFO:teuthology.orchestra.run.smithi112.stdout: "num_caps": 0, 2024-09-21T00:30:09.300 INFO:teuthology.orchestra.run.smithi112.stdout: "request_load_avg": 0, 2024-09-21T00:30:09.300 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 0.00099997600000000008, 2024-09-21T00:30:09.300 INFO:teuthology.orchestra.run.smithi112.stdout: "requests_in_flight": 0, 2024-09-21T00:30:09.300 INFO:teuthology.orchestra.run.smithi112.stdout: "num_completed_requests": 0, 2024-09-21T00:30:09.300 INFO:teuthology.orchestra.run.smithi112.stdout: "num_completed_flushes": 0, 2024-09-21T00:30:09.300 INFO:teuthology.orchestra.run.smithi112.stdout: "reconnecting": false, 2024-09-21T00:30:09.300 INFO:teuthology.orchestra.run.smithi112.stdout: "recall_caps": { 2024-09-21T00:30:09.300 INFO:teuthology.orchestra.run.smithi112.stdout: "value": 0, 2024-09-21T00:30:09.301 INFO:teuthology.orchestra.run.smithi112.stdout: "halflife": 60 2024-09-21T00:30:09.301 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:30:09.301 INFO:teuthology.orchestra.run.smithi112.stdout: "release_caps": { 2024-09-21T00:30:09.301 INFO:teuthology.orchestra.run.smithi112.stdout: "value": 0, 2024-09-21T00:30:09.301 INFO:teuthology.orchestra.run.smithi112.stdout: "halflife": 60 2024-09-21T00:30:09.301 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:30:09.301 INFO:teuthology.orchestra.run.smithi112.stdout: "recall_caps_throttle": { 2024-09-21T00:30:09.301 INFO:teuthology.orchestra.run.smithi112.stdout: "value": 0, 2024-09-21T00:30:09.301 INFO:teuthology.orchestra.run.smithi112.stdout: "halflife": 1.5 2024-09-21T00:30:09.301 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:30:09.301 INFO:teuthology.orchestra.run.smithi112.stdout: "recall_caps_throttle2o": { 2024-09-21T00:30:09.301 INFO:teuthology.orchestra.run.smithi112.stdout: "value": 0, 2024-09-21T00:30:09.301 INFO:teuthology.orchestra.run.smithi112.stdout: "halflife": 0.5 2024-09-21T00:30:09.301 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:30:09.301 INFO:teuthology.orchestra.run.smithi112.stdout: "session_cache_liveness": { 2024-09-21T00:30:09.302 INFO:teuthology.orchestra.run.smithi112.stdout: "value": 0, 2024-09-21T00:30:09.302 INFO:teuthology.orchestra.run.smithi112.stdout: "halflife": 300 2024-09-21T00:30:09.302 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:30:09.302 INFO:teuthology.orchestra.run.smithi112.stdout: "cap_acquisition": { 2024-09-21T00:30:09.302 INFO:teuthology.orchestra.run.smithi112.stdout: "value": 0, 2024-09-21T00:30:09.302 INFO:teuthology.orchestra.run.smithi112.stdout: "halflife": 30 2024-09-21T00:30:09.302 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:30:09.302 INFO:teuthology.orchestra.run.smithi112.stdout: "last_trim_completed_requests_tid": 0, 2024-09-21T00:30:09.302 INFO:teuthology.orchestra.run.smithi112.stdout: "last_trim_completed_flushes_tid": 0, 2024-09-21T00:30:09.302 INFO:teuthology.orchestra.run.smithi112.stdout: "delegated_inos": [], 2024-09-21T00:30:09.302 INFO:teuthology.orchestra.run.smithi112.stdout: "inst": "client.7470 v1:192.168.0.1:0/3384400602", 2024-09-21T00:30:09.302 INFO:teuthology.orchestra.run.smithi112.stdout: "completed_requests": [], 2024-09-21T00:30:09.302 INFO:teuthology.orchestra.run.smithi112.stdout: "prealloc_inos": [], 2024-09-21T00:30:09.303 INFO:teuthology.orchestra.run.smithi112.stdout: "client_metadata": { 2024-09-21T00:30:09.303 INFO:teuthology.orchestra.run.smithi112.stdout: "client_features": { 2024-09-21T00:30:09.303 INFO:teuthology.orchestra.run.smithi112.stdout: "feature_bits": "0x000000000003fbff" 2024-09-21T00:30:09.303 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:30:09.303 INFO:teuthology.orchestra.run.smithi112.stdout: "metric_spec": { 2024-09-21T00:30:09.303 INFO:teuthology.orchestra.run.smithi112.stdout: "metric_flags": { 2024-09-21T00:30:09.303 INFO:teuthology.orchestra.run.smithi112.stdout: "feature_bits": "0x000000000080ffff" 2024-09-21T00:30:09.303 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:30:09.303 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:30:09.303 INFO:teuthology.orchestra.run.smithi112.stdout: "entity_id": "0", 2024-09-21T00:30:09.303 INFO:teuthology.orchestra.run.smithi112.stdout: "hostname": "smithi112", 2024-09-21T00:30:09.303 INFO:teuthology.orchestra.run.smithi112.stdout: "kernel_version": "5.14.0-508.el9.x86_64", 2024-09-21T00:30:09.303 INFO:teuthology.orchestra.run.smithi112.stdout: "root": "/" 2024-09-21T00:30:09.304 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:30:09.304 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:30:09.304 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-09-21T00:30:09.304 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:30:09.304 INFO:teuthology.orchestra.run.smithi112.stdout: "result": 0 2024-09-21T00:30:09.304 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:30:09.304 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-21T00:30:09.304 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:30:09.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.287+0000 7fb2471df280 1 -- 172.21.15.112:0/4000488620 >> [v2:172.21.15.112:6808/755760621,v1:172.21.15.112:6809/755760621] conn(0x5623a1867f80 msgr2=0x5623a1860c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:09.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.287+0000 7fb2471df280 1 --2- 172.21.15.112:0/4000488620 >> [v2:172.21.15.112:6808/755760621,v1:172.21.15.112:6809/755760621] conn(0x5623a1867f80 0x5623a1860c00 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:09.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.287+0000 7fb2471df280 1 -- 172.21.15.112:0/4000488620 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x5623a1868d00 msgr2=0x5623a1860680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:09.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.287+0000 7fb2471df280 1 --2- 172.21.15.112:0/4000488620 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x5623a1868d00 0x5623a1860680 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x5623a0b4cc60 tx=0x5623a0ea4600 comp rx=0 tx=0).stop 2024-09-21T00:30:09.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.287+0000 7fb2471df280 1 -- 172.21.15.112:0/4000488620 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5623a1867b00 msgr2=0x5623a1860100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:09.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.287+0000 7fb2471df280 1 --2- 172.21.15.112:0/4000488620 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5623a1867b00 0x5623a1860100 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x5623a198f0b0 tx=0x5623a186e180 comp rx=0 tx=0).stop 2024-09-21T00:30:09.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.287+0000 7fb245984640 1 -- 172.21.15.112:0/4000488620 reap_dead start 2024-09-21T00:30:09.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.287+0000 7fb2471df280 1 -- 172.21.15.112:0/4000488620 shutdown_connections 2024-09-21T00:30:09.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.287+0000 7fb2471df280 1 -- 172.21.15.112:0/4000488620 >> 172.21.15.112:0/4000488620 conn(0x5623a1866d80 msgr2=0x5623a0b71800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:09.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.287+0000 7fb2471df280 1 -- 172.21.15.112:0/4000488620 shutdown_connections 2024-09-21T00:30:09.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.287+0000 7fb2471df280 1 -- 172.21.15.112:0/4000488620 wait complete. 2024-09-21T00:30:09.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.287+0000 7fb2471df280 1 -- 172.21.15.112:0/3312473909 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5623a1866900 msgr2=0x5623a185e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:09.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.287+0000 7fb2471df280 1 --2- 172.21.15.112:0/3312473909 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5623a1866900 0x5623a185e000 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x5623a18cb8f0 tx=0x5623a186e060 comp rx=0 tx=0).stop 2024-09-21T00:30:09.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.287+0000 7fb2471df280 1 -- 172.21.15.112:0/3312473909 shutdown_connections 2024-09-21T00:30:09.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.287+0000 7fb2471df280 1 --2- 172.21.15.112:0/3312473909 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5623a1866900 0x5623a185e000 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:09.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.287+0000 7fb2471df280 1 --2- 172.21.15.112:0/3312473909 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5623a1866480 0x5623a185e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:09.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.287+0000 7fb2471df280 1 --2- 172.21.15.112:0/3312473909 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5623a1869180 0x5623a185eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:09.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.287+0000 7fb2471df280 1 -- 172.21.15.112:0/3312473909 >> 172.21.15.112:0/3312473909 conn(0x5623a0c1b600 msgr2=0x5623a0b71000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:09.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.287+0000 7fb2471df280 1 -- 172.21.15.112:0/3312473909 shutdown_connections 2024-09-21T00:30:09.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.287+0000 7fb2471df280 1 -- 172.21.15.112:0/3312473909 wait complete. 2024-09-21T00:30:09.308 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-table-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '0', 'show', 'session'] in time 0:00:00.083995, result: { "0": { "data": { "sessions": [ { "id": 7479, "entity": { "name": { "type": "client", "num": 7479 }, "addr": { "type": "v1", "addr": "172.21.15.187:0", "nonce": 579439030 } }, "state": "open", "num_leases": 0, "num_caps": 0, "request_load_avg": 0, "uptime": 0.00099997600000000008, "requests_in_flight": 0, "num_completed_requests": 0, "num_completed_flushes": 0, "reconnecting": false, "recall_caps": { "value": 0, "halflife": 60 }, "release_caps": { "value": 0, "halflife": 60 }, "recall_caps_throttle": { "value": 0, "halflife": 1.5 }, "recall_caps_throttle2o": { "value": 0, "halflife": 0.5 }, "session_cache_liveness": { "value": 0, "halflife": 300 }, "cap_acquisition": { "value": 0, "halflife": 30 }, "last_trim_completed_requests_tid": 0, "last_trim_completed_flushes_tid": 0, "delegated_inos": [], "inst": "client.7479 v1:172.21.15.187:0/579439030", "completed_requests": [], "prealloc_inos": [], "client_metadata": { "client_features": { "feature_bits": "0x000000000003fbff" }, "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "entity_id": "1", "hostname": "smithi187", "kernel_version": "5.14.0-508.el9.x86_64", "root": "/" } }, { "id": 7470, "entity": { "name": { "type": "client", "num": 7470 }, "addr": { "type": "v1", "addr": "192.168.0.1:0", "nonce": 3384400602 } }, "state": "open", "num_leases": 0, "num_caps": 0, "request_load_avg": 0, "uptime": 0.00099997600000000008, "requests_in_flight": 0, "num_completed_requests": 0, "num_completed_flushes": 0, "reconnecting": false, "recall_caps": { "value": 0, "halflife": 60 }, "release_caps": { "value": 0, "halflife": 60 }, "recall_caps_throttle": { "value": 0, "halflife": 1.5 }, "recall_caps_throttle2o": { "value": 0, "halflife": 0.5 }, "session_cache_liveness": { "value": 0, "halflife": 300 }, "cap_acquisition": { "value": 0, "halflife": 30 }, "last_trim_completed_requests_tid": 0, "last_trim_completed_flushes_tid": 0, "delegated_inos": [], "inst": "client.7470 v1:192.168.0.1:0/3384400602", "completed_requests": [], "prealloc_inos": [], "client_metadata": { "client_features": { "feature_bits": "0x000000000003fbff" }, "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "entity_id": "0", "hostname": "smithi112", "kernel_version": "5.14.0-508.el9.x86_64", "root": "/" } } ] }, "result": 0 } } 2024-09-21T00:30:09.308 INFO:tasks.cephfs.test_sessionmap:SessionMap: { "0": { "data": { "sessions": [ { "id": 7479, "entity": { "name": { "type": "client", "num": 7479 }, "addr": { "type": "v1", "addr": "172.21.15.187:0", "nonce": 579439030 } }, "state": "open", "num_leases": 0, "num_caps": 0, "request_load_avg": 0, "uptime": 0.000999976, "requests_in_flight": 0, "num_completed_requests": 0, "num_completed_flushes": 0, "reconnecting": false, "recall_caps": { "value": 0, "halflife": 60 }, "release_caps": { "value": 0, "halflife": 60 }, "recall_caps_throttle": { "value": 0, "halflife": 1.5 }, "recall_caps_throttle2o": { "value": 0, "halflife": 0.5 }, "session_cache_liveness": { "value": 0, "halflife": 300 }, "cap_acquisition": { "value": 0, "halflife": 30 }, "last_trim_completed_requests_tid": 0, "last_trim_completed_flushes_tid": 0, "delegated_inos": [], "inst": "client.7479 v1:172.21.15.187:0/579439030", "completed_requests": [], "prealloc_inos": [], "client_metadata": { "client_features": { "feature_bits": "0x000000000003fbff" }, "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "entity_id": "1", "hostname": "smithi187", "kernel_version": "5.14.0-508.el9.x86_64", "root": "/" } }, { "id": 7470, "entity": { "name": { "type": "client", "num": 7470 }, "addr": { "type": "v1", "addr": "192.168.0.1:0", "nonce": 3384400602 } }, "state": "open", "num_leases": 0, "num_caps": 0, "request_load_avg": 0, "uptime": 0.000999976, "requests_in_flight": 0, "num_completed_requests": 0, "num_completed_flushes": 0, "reconnecting": false, "recall_caps": { "value": 0, "halflife": 60 }, "release_caps": { "value": 0, "halflife": 60 }, "recall_caps_throttle": { "value": 0, "halflife": 1.5 }, "recall_caps_throttle2o": { "value": 0, "halflife": 0.5 }, "session_cache_liveness": { "value": 0, "halflife": 300 }, "cap_acquisition": { "value": 0, "halflife": 30 }, "last_trim_completed_requests_tid": 0, "last_trim_completed_flushes_tid": 0, "delegated_inos": [], "inst": "client.7470 v1:192.168.0.1:0/3384400602", "completed_requests": [], "prealloc_inos": [], "client_metadata": { "client_features": { "feature_bits": "0x000000000003fbff" }, "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "entity_id": "0", "hostname": "smithi112", "kernel_version": "5.14.0-508.el9.x86_64", "root": "/" } } ] }, "result": 0 } } 2024-09-21T00:30:09.308 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:30:09.309 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir bravo) 2024-09-21T00:30:09.328 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:30:09.328 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch bravo/file_a) 2024-09-21T00:30:09.385 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:30:09.385 DEBUG:teuthology.orchestra.run.smithi187:> (cd /home/ubuntu/cephtest/mnt.1 && exec touch bravo/file_b) 2024-09-21T00:30:09.408 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 2 2024-09-21T00:30:09.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.539+0000 7ff59941f640 1 -- 172.21.15.112:0/2620231086 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5940d24a0 msgr2=0x7ff5940d28b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:09.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.539+0000 7ff59941f640 1 --2- 172.21.15.112:0/2620231086 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5940d24a0 0x7ff5940d28b0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7ff59408d590 tx=0x7ff58802a4a0 comp rx=0 tx=0).stop 2024-09-21T00:30:09.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.539+0000 7ff59941f640 1 -- 172.21.15.112:0/2620231086 shutdown_connections 2024-09-21T00:30:09.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.539+0000 7ff59941f640 1 --2- 172.21.15.112:0/2620231086 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5940d24a0 0x7ff5940d28b0 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:09.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.539+0000 7ff59941f640 1 --2- 172.21.15.112:0/2620231086 >> v2:172.21.15.187:3300/0 conn(0x7ff5940cfd90 0x7ff5940b9d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:09.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.539+0000 7ff59941f640 1 -- 172.21.15.112:0/2620231086 >> 172.21.15.112:0/2620231086 conn(0x7ff5940d4650 msgr2=0x7ff5940d4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:09.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.539+0000 7ff59941f640 1 -- 172.21.15.112:0/2620231086 shutdown_connections 2024-09-21T00:30:09.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.539+0000 7ff59941f640 1 -- 172.21.15.112:0/2620231086 wait complete. 2024-09-21T00:30:09.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.541+0000 7ff59941f640 1 Processor -- start 2024-09-21T00:30:09.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.541+0000 7ff59941f640 1 -- start start 2024-09-21T00:30:09.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.542+0000 7ff59941f640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff5940cfd90 0x7ff5940cdc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:09.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.542+0000 7ff59941f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff5940d24a0 0x7ff5940ce180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:09.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.542+0000 7ff59941f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5940d2df0 0x7ff5941f9050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:09.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.542+0000 7ff59941f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff594162f30 con 0x7ff5940d2df0 2024-09-21T00:30:09.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.542+0000 7ff59941f640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff5940b2d60 con 0x7ff5940d24a0 2024-09-21T00:30:09.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.542+0000 7ff59941f640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff5941630b0 con 0x7ff5940cfd90 2024-09-21T00:30:09.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.542+0000 7ff5937fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5940d2df0 0x7ff5941f9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:09.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.542+0000 7ff5937fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5940d2df0 0x7ff5941f9050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35650/0 (socket says 172.21.15.112:35650) 2024-09-21T00:30:09.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.542+0000 7ff5937fe640 1 -- 172.21.15.112:0/167931993 learned_addr learned my addr 172.21.15.112:0/167931993 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:30:09.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.542+0000 7ff5927fc640 1 --2- 172.21.15.112:0/167931993 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff5940d24a0 0x7ff5940ce180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:09.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.542+0000 7ff592ffd640 1 --2- 172.21.15.112:0/167931993 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff5940cfd90 0x7ff5940cdc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:09.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.542+0000 7ff5937fe640 1 -- 172.21.15.112:0/167931993 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff5940cfd90 msgr2=0x7ff5940cdc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:09.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.542+0000 7ff5937fe640 1 --2- 172.21.15.112:0/167931993 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff5940cfd90 0x7ff5940cdc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:09.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.542+0000 7ff5937fe640 1 -- 172.21.15.112:0/167931993 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff5940d24a0 msgr2=0x7ff5940ce180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:09.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.542+0000 7ff5937fe640 1 --2- 172.21.15.112:0/167931993 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff5940d24a0 0x7ff5940ce180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:09.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.542+0000 7ff5937fe640 1 -- 172.21.15.112:0/167931993 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5940b3b90 con 0x7ff5940d2df0 2024-09-21T00:30:09.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.543+0000 7ff5937fe640 1 --2- 172.21.15.112:0/167931993 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5940d2df0 0x7ff5941f9050 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7ff58c00c3c0 tx=0x7ff58c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:09.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.543+0000 7ff57b7fe640 1 -- 172.21.15.112:0/167931993 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff58c017070 con 0x7ff5940d2df0 2024-09-21T00:30:09.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.543+0000 7ff57b7fe640 1 -- 172.21.15.112:0/167931993 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff58c0035f0 con 0x7ff5940d2df0 2024-09-21T00:30:09.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.543+0000 7ff59941f640 1 -- 172.21.15.112:0/167931993 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5940a1600 con 0x7ff5940d2df0 2024-09-21T00:30:09.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.543+0000 7ff57b7fe640 1 -- 172.21.15.112:0/167931993 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff58c003a40 con 0x7ff5940d2df0 2024-09-21T00:30:09.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.543+0000 7ff59941f640 1 -- 172.21.15.112:0/167931993 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5940a1bc0 con 0x7ff5940d2df0 2024-09-21T00:30:09.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.544+0000 7ff59941f640 1 -- 172.21.15.112:0/167931993 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff594163560 con 0x7ff5940d2df0 2024-09-21T00:30:09.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.544+0000 7ff57b7fe640 1 -- 172.21.15.112:0/167931993 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff58c01a070 con 0x7ff5940d2df0 2024-09-21T00:30:09.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.545+0000 7ff57b7fe640 1 --2- 172.21.15.112:0/167931993 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff580078b90 0x7ff58007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:09.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.545+0000 7ff57b7fe640 1 -- 172.21.15.112:0/167931993 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 6187+0+0 (secure 0 0 0) 0x7ff58c099f10 con 0x7ff5940d2df0 2024-09-21T00:30:09.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.545+0000 7ff592ffd640 1 --2- 172.21.15.112:0/167931993 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff580078b90 0x7ff58007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:09.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.546+0000 7ff592ffd640 1 --2- 172.21.15.112:0/167931993 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff580078b90 0x7ff58007b0a0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7ff584005fd0 tx=0x7ff584009630 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:09.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.549+0000 7ff57b7fe640 1 -- 172.21.15.112:0/167931993 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ff58c061d00 con 0x7ff5940d2df0 2024-09-21T00:30:09.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:09.770+0000 7ff59941f640 1 -- 172.21.15.112:0/167931993 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"} v 0) -- 0x7ff5940d5200 con 0x7ff5940d2df0 2024-09-21T00:30:10.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:10.645+0000 7ff57b7fe640 1 -- 172.21.15.112:0/167931993 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]=0 v77) ==== 105+0+0 (secure 0 0 0) 0x7ff58c066730 con 0x7ff5940d2df0 2024-09-21T00:30:10.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:10.650+0000 7ff59941f640 1 -- 172.21.15.112:0/167931993 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff580078b90 msgr2=0x7ff58007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:10.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:10.650+0000 7ff59941f640 1 --2- 172.21.15.112:0/167931993 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff580078b90 0x7ff58007b0a0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7ff584005fd0 tx=0x7ff584009630 comp rx=0 tx=0).stop 2024-09-21T00:30:10.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:10.650+0000 7ff59941f640 1 -- 172.21.15.112:0/167931993 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5940d2df0 msgr2=0x7ff5941f9050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:10.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:10.650+0000 7ff59941f640 1 --2- 172.21.15.112:0/167931993 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5940d2df0 0x7ff5941f9050 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7ff58c00c3c0 tx=0x7ff58c00c880 comp rx=0 tx=0).stop 2024-09-21T00:30:10.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:10.650+0000 7ff59941f640 1 -- 172.21.15.112:0/167931993 shutdown_connections 2024-09-21T00:30:10.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:10.650+0000 7ff59941f640 1 --2- 172.21.15.112:0/167931993 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff580078b90 0x7ff58007b0a0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:10.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:10.650+0000 7ff59941f640 1 --2- 172.21.15.112:0/167931993 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5940d2df0 0x7ff5941f9050 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:10.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:10.650+0000 7ff59941f640 1 --2- 172.21.15.112:0/167931993 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff5940d24a0 0x7ff5940ce180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:10.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:10.650+0000 7ff59941f640 1 --2- 172.21.15.112:0/167931993 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff5940cfd90 0x7ff5940cdc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:10.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:10.650+0000 7ff59941f640 1 -- 172.21.15.112:0/167931993 >> 172.21.15.112:0/167931993 conn(0x7ff5940d4650 msgr2=0x7ff5940d0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:10.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:10.650+0000 7ff59941f640 1 -- 172.21.15.112:0/167931993 shutdown_connections 2024-09-21T00:30:10.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:10.650+0000 7ff59941f640 1 -- 172.21.15.112:0/167931993 wait complete. 2024-09-21T00:30:10.671 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T00:30:10.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:10.788+0000 7f5e6787f640 1 -- 172.21.15.112:0/2098959862 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e600b6450 msgr2=0x7f5e600ce450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:10.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:10.788+0000 7f5e6787f640 1 --2- 172.21.15.112:0/2098959862 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e600b6450 0x7f5e600ce450 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f5e60073c50 tx=0x7f5e5802a950 comp rx=0 tx=0).stop 2024-09-21T00:30:10.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:10.788+0000 7f5e6787f640 1 -- 172.21.15.112:0/2098959862 shutdown_connections 2024-09-21T00:30:10.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:10.788+0000 7f5e6787f640 1 --2- 172.21.15.112:0/2098959862 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e600b6450 0x7f5e600ce450 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:10.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:10.788+0000 7f5e6787f640 1 --2- 172.21.15.112:0/2098959862 >> v2:172.21.15.187:3300/0 conn(0x7f5e6009baa0 0x7f5e600b53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:10.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:10.788+0000 7f5e6787f640 1 -- 172.21.15.112:0/2098959862 >> 172.21.15.112:0/2098959862 conn(0x7f5e600cef20 msgr2=0x7f5e600cf330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:10.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:10.788+0000 7f5e6787f640 1 -- 172.21.15.112:0/2098959862 shutdown_connections 2024-09-21T00:30:10.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:10.788+0000 7f5e6787f640 1 -- 172.21.15.112:0/2098959862 wait complete. 2024-09-21T00:30:10.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:10.790+0000 7f5e6787f640 1 Processor -- start 2024-09-21T00:30:10.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:10.790+0000 7f5e6787f640 1 -- start start 2024-09-21T00:30:10.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:10.791+0000 7f5e6787f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e6009baa0 0x7f5e601499b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:10.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:10.791+0000 7f5e6787f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5e600b5930 0x7f5e60161890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:10.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:10.791+0000 7f5e6787f640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5e600b6450 0x7f5e6015f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:10.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:10.791+0000 7f5e6787f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5e600ce990 con 0x7f5e6009baa0 2024-09-21T00:30:10.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:10.791+0000 7f5e6787f640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5e600d33b0 con 0x7f5e600b5930 2024-09-21T00:30:10.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:10.791+0000 7f5e6787f640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f5e600ceb10 con 0x7f5e600b6450 2024-09-21T00:30:10.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:10.791+0000 7f5e6561b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e6009baa0 0x7f5e601499b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:10.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:10.791+0000 7f5e6561b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e6009baa0 0x7f5e601499b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35666/0 (socket says 172.21.15.112:35666) 2024-09-21T00:30:10.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:10.791+0000 7f5e6561b640 1 -- 172.21.15.112:0/3294533085 learned_addr learned my addr 172.21.15.112:0/3294533085 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:30:10.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:10.791+0000 7f5e64e1a640 1 --2- 172.21.15.112:0/3294533085 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5e600b5930 0x7f5e60161890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:10.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:10.791+0000 7f5e65e1c640 1 --2- 172.21.15.112:0/3294533085 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5e600b6450 0x7f5e6015f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:10.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:10.791+0000 7f5e6561b640 1 -- 172.21.15.112:0/3294533085 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5e600b6450 msgr2=0x7f5e6015f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:10.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:10.791+0000 7f5e6561b640 1 --2- 172.21.15.112:0/3294533085 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5e600b6450 0x7f5e6015f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:10.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:10.791+0000 7f5e6561b640 1 -- 172.21.15.112:0/3294533085 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5e600b5930 msgr2=0x7f5e60161890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:10.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:10.791+0000 7f5e6561b640 1 --2- 172.21.15.112:0/3294533085 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5e600b5930 0x7f5e60161890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:10.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:10.791+0000 7f5e6561b640 1 -- 172.21.15.112:0/3294533085 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e601f3af0 con 0x7f5e6009baa0 2024-09-21T00:30:10.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:10.792+0000 7f5e6561b640 1 --2- 172.21.15.112:0/3294533085 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e6009baa0 0x7f5e601499b0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f5e5c00d740 tx=0x7f5e5c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:10.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:10.792+0000 7f5e55ffb640 1 -- 172.21.15.112:0/3294533085 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e5c017070 con 0x7f5e6009baa0 2024-09-21T00:30:10.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:10.792+0000 7f5e55ffb640 1 -- 172.21.15.112:0/3294533085 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5e5c009d20 con 0x7f5e6009baa0 2024-09-21T00:30:10.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:10.792+0000 7f5e55ffb640 1 -- 172.21.15.112:0/3294533085 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e5c0106c0 con 0x7f5e6009baa0 2024-09-21T00:30:10.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:10.792+0000 7f5e6787f640 1 -- 172.21.15.112:0/3294533085 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e601f3dd0 con 0x7f5e6009baa0 2024-09-21T00:30:10.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:10.792+0000 7f5e6787f640 1 -- 172.21.15.112:0/3294533085 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e6015bea0 con 0x7f5e6009baa0 2024-09-21T00:30:10.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:10.793+0000 7f5e6787f640 1 -- 172.21.15.112:0/3294533085 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e6015c100 con 0x7f5e6009baa0 2024-09-21T00:30:10.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:10.794+0000 7f5e55ffb640 1 -- 172.21.15.112:0/3294533085 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5e5c01a070 con 0x7f5e6009baa0 2024-09-21T00:30:10.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:10.794+0000 7f5e55ffb640 1 --2- 172.21.15.112:0/3294533085 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f5e44078b90 0x7f5e4407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:10.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:10.794+0000 7f5e55ffb640 1 -- 172.21.15.112:0/3294533085 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 6187+0+0 (secure 0 0 0) 0x7f5e5c09b010 con 0x7f5e6009baa0 2024-09-21T00:30:10.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:10.795+0000 7f5e64e1a640 1 --2- 172.21.15.112:0/3294533085 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f5e44078b90 0x7f5e4407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:10.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:10.795+0000 7f5e64e1a640 1 --2- 172.21.15.112:0/3294533085 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f5e44078b90 0x7f5e4407b0a0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f5e6014d420 tx=0x7f5e50009060 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:10.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:10.799+0000 7f5e55ffb640 1 -- 172.21.15.112:0/3294533085 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5e5c062c20 con 0x7f5e6009baa0 2024-09-21T00:30:11.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:11.013+0000 7f5e6787f640 1 -- 172.21.15.112:0/3294533085 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5e600b53f0 con 0x7f5e6009baa0 2024-09-21T00:30:11.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:11.014+0000 7f5e55ffb640 1 -- 172.21.15.112:0/3294533085 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) ==== 94+0+5252 (secure 0 0 0) 0x7f5e5c067650 con 0x7f5e6009baa0 2024-09-21T00:30:11.016 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:30:11.016 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":78,"btime":"2024-09-21T00:30:10:651164+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6113,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/3217596137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":3217596137},{"type":"v1","addr":"172.21.15.187:6835","nonce":3217596137}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":73},{"gid":6649,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6837/776068904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":776068904},{"type":"v1","addr":"172.21.15.187:6837","nonce":776068904}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":73}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T00:29:48.456672+0000","modified":"2024-09-21T00:30:10.651121+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":112,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7422,"mds_1":7419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7419":{"gid":7419,"name":"a","rank":1,"incarnation":78,"state":"up:creating","state_seq":1,"addr":"172.21.15.112:6834/3291181711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":3291181711},{"type":"v1","addr":"172.21.15.112:6834","nonce":3291181711}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7422":{"gid":7422,"name":"c","rank":0,"incarnation":73,"state":"up:active","state_seq":4,"addr":"172.21.15.112:6835/3392867613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":3392867613},{"type":"v1","addr":"172.21.15.112:6835","nonce":3392867613}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7422,"qdb_cluster":[7422]},"id":8}]} 2024-09-21T00:30:11.016 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 78 2024-09-21T00:30:11.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:11.016+0000 7f5e6787f640 1 -- 172.21.15.112:0/3294533085 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f5e44078b90 msgr2=0x7f5e4407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:11.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:11.016+0000 7f5e6787f640 1 --2- 172.21.15.112:0/3294533085 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f5e44078b90 0x7f5e4407b0a0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f5e6014d420 tx=0x7f5e50009060 comp rx=0 tx=0).stop 2024-09-21T00:30:11.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:11.016+0000 7f5e6787f640 1 -- 172.21.15.112:0/3294533085 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e6009baa0 msgr2=0x7f5e601499b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:11.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:11.016+0000 7f5e6787f640 1 --2- 172.21.15.112:0/3294533085 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e6009baa0 0x7f5e601499b0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f5e5c00d740 tx=0x7f5e5c00dc00 comp rx=0 tx=0).stop 2024-09-21T00:30:11.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:11.016+0000 7f5e6787f640 1 -- 172.21.15.112:0/3294533085 shutdown_connections 2024-09-21T00:30:11.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:11.016+0000 7f5e6787f640 1 --2- 172.21.15.112:0/3294533085 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5e600b6450 0x7f5e6015f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:11.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:11.016+0000 7f5e6787f640 1 --2- 172.21.15.112:0/3294533085 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5e600b5930 0x7f5e60161890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:11.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:11.016+0000 7f5e6787f640 1 --2- 172.21.15.112:0/3294533085 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f5e44078b90 0x7f5e4407b0a0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:11.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:11.016+0000 7f5e6787f640 1 --2- 172.21.15.112:0/3294533085 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e6009baa0 0x7f5e601499b0 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:11.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:11.016+0000 7f5e6787f640 1 -- 172.21.15.112:0/3294533085 >> 172.21.15.112:0/3294533085 conn(0x7f5e600cef20 msgr2=0x7f5e600b5d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:11.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:11.016+0000 7f5e6787f640 1 -- 172.21.15.112:0/3294533085 shutdown_connections 2024-09-21T00:30:11.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:11.016+0000 7f5e6787f640 1 -- 172.21.15.112:0/3294533085 wait complete. 2024-09-21T00:30:11.432 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 78, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T00:29:48.456672+0000', 'modified': '2024-09-21T00:30:10.651121+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 112, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 2, 'in': [0, 1], 'up': {'mds_0': 7422, 'mds_1': 7419}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7419': {'gid': 7419, 'name': 'a', 'rank': 1, 'incarnation': 78, 'state': 'up:creating', 'state_seq': 1, 'addr': '172.21.15.112:6834/3291181711', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6832', 'nonce': 3291181711}, {'type': 'v1', 'addr': '172.21.15.112:6834', 'nonce': 3291181711}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_7422': {'gid': 7422, 'name': 'c', 'rank': 0, 'incarnation': 73, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.112:6835/3392867613', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6833', 'nonce': 3392867613}, {'type': 'v1', 'addr': '172.21.15.112:6835', 'nonce': 3392867613}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7422, 'qdb_cluster': [7422]} 2024-09-21T00:30:11.432 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7419:up:creating 2024-09-21T00:30:12.433 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T00:30:12.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:12.566+0000 7fc3020f2640 1 -- 172.21.15.112:0/1295236421 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2fc0b9dc0 msgr2=0x7fc2fc0d25c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:12.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:12.566+0000 7fc3020f2640 1 --2- 172.21.15.112:0/1295236421 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2fc0b9dc0 0x7fc2fc0d25c0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec009830 tx=0x7fc2ec02a3f0 comp rx=0 tx=0).stop 2024-09-21T00:30:12.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:12.566+0000 7fc3020f2640 1 -- 172.21.15.112:0/1295236421 shutdown_connections 2024-09-21T00:30:12.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:12.566+0000 7fc3020f2640 1 --2- 172.21.15.112:0/1295236421 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc2fc0b6770 0x7fc2fc06e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:12.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:12.566+0000 7fc3020f2640 1 --2- 172.21.15.112:0/1295236421 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2fc0b9dc0 0x7fc2fc0d25c0 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:12.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:12.566+0000 7fc3020f2640 1 -- 172.21.15.112:0/1295236421 >> 172.21.15.112:0/1295236421 conn(0x7fc2fc0d45e0 msgr2=0x7fc2fc0d49f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:12.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:12.566+0000 7fc3020f2640 1 -- 172.21.15.112:0/1295236421 shutdown_connections 2024-09-21T00:30:12.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:12.566+0000 7fc3020f2640 1 -- 172.21.15.112:0/1295236421 wait complete. 2024-09-21T00:30:12.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:12.568+0000 7fc3020f2640 1 Processor -- start 2024-09-21T00:30:12.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:12.568+0000 7fc3020f2640 1 -- start start 2024-09-21T00:30:12.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:12.569+0000 7fc3020f2640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc2fc0b3b40 0x7fc2fc15af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:12.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:12.569+0000 7fc3020f2640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc2fc0b9dc0 0x7fc2fc15b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:12.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:12.569+0000 7fc3020f2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2fc1e6620 0x7fc2fc159b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:12.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:12.569+0000 7fc3020f2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc2fc06f280 con 0x7fc2fc1e6620 2024-09-21T00:30:12.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:12.569+0000 7fc3020f2640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc2fc06f100 con 0x7fc2fc0b9dc0 2024-09-21T00:30:12.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:12.569+0000 7fc3020f2640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc2fc06f400 con 0x7fc2fc0b3b40 2024-09-21T00:30:12.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:12.569+0000 7fc2fbfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2fc1e6620 0x7fc2fc159b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:12.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:12.569+0000 7fc2fbfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2fc1e6620 0x7fc2fc159b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35684/0 (socket says 172.21.15.112:35684) 2024-09-21T00:30:12.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:12.569+0000 7fc2fbfff640 1 -- 172.21.15.112:0/4073321553 learned_addr learned my addr 172.21.15.112:0/4073321553 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:30:12.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:12.569+0000 7fc2fb7fe640 1 --2- 172.21.15.112:0/4073321553 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc2fc0b3b40 0x7fc2fc15af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:12.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:12.569+0000 7fc2faffd640 1 --2- 172.21.15.112:0/4073321553 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc2fc0b9dc0 0x7fc2fc15b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:12.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:12.570+0000 7fc2fbfff640 1 -- 172.21.15.112:0/4073321553 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc2fc0b3b40 msgr2=0x7fc2fc15af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:12.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:12.570+0000 7fc2fbfff640 1 --2- 172.21.15.112:0/4073321553 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc2fc0b3b40 0x7fc2fc15af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:12.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:12.570+0000 7fc2fbfff640 1 -- 172.21.15.112:0/4073321553 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc2fc0b9dc0 msgr2=0x7fc2fc15b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:12.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:12.570+0000 7fc2fbfff640 1 --2- 172.21.15.112:0/4073321553 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc2fc0b9dc0 0x7fc2fc15b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:12.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:12.570+0000 7fc2fbfff640 1 -- 172.21.15.112:0/4073321553 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2fc1f67d0 con 0x7fc2fc1e6620 2024-09-21T00:30:12.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:12.570+0000 7fc2fbfff640 1 --2- 172.21.15.112:0/4073321553 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2fc1e6620 0x7fc2fc159b60 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7fc2f400cae0 tx=0x7fc2f40071c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:12.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:12.570+0000 7fc2e3fff640 1 -- 172.21.15.112:0/4073321553 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f4007da0 con 0x7fc2fc1e6620 2024-09-21T00:30:12.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:12.570+0000 7fc2e3fff640 1 -- 172.21.15.112:0/4073321553 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc2f4003990 con 0x7fc2fc1e6620 2024-09-21T00:30:12.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:12.570+0000 7fc3020f2640 1 -- 172.21.15.112:0/4073321553 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2fc1e40b0 con 0x7fc2fc1e6620 2024-09-21T00:30:12.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:12.570+0000 7fc2e3fff640 1 -- 172.21.15.112:0/4073321553 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f4003e00 con 0x7fc2fc1e6620 2024-09-21T00:30:12.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:12.570+0000 7fc3020f2640 1 -- 172.21.15.112:0/4073321553 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2fc1e4630 con 0x7fc2fc1e6620 2024-09-21T00:30:12.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:12.571+0000 7fc3020f2640 1 -- 172.21.15.112:0/4073321553 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2fc06f920 con 0x7fc2fc1e6620 2024-09-21T00:30:12.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:12.572+0000 7fc2e3fff640 1 -- 172.21.15.112:0/4073321553 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc2f4004030 con 0x7fc2fc1e6620 2024-09-21T00:30:12.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:12.572+0000 7fc2e3fff640 1 --2- 172.21.15.112:0/4073321553 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fc2e8078bc0 0x7fc2e807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:12.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:12.572+0000 7fc2e3fff640 1 -- 172.21.15.112:0/4073321553 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 6187+0+0 (secure 0 0 0) 0x7fc2f409b6a0 con 0x7fc2fc1e6620 2024-09-21T00:30:12.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:12.573+0000 7fc2fb7fe640 1 --2- 172.21.15.112:0/4073321553 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fc2e8078bc0 0x7fc2e807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:12.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:12.573+0000 7fc2fb7fe640 1 --2- 172.21.15.112:0/4073321553 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fc2e8078bc0 0x7fc2e807b0d0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec034000 tx=0x7fc2ec004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:12.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:12.577+0000 7fc2e3fff640 1 -- 172.21.15.112:0/4073321553 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc2f4014070 con 0x7fc2fc1e6620 2024-09-21T00:30:12.617 INFO:tasks.ceph.mds.c.smithi112.stderr:2024-09-21T00:30:12.616+0000 7fb1438b0640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-09-21T00:30:12.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:12.791+0000 7fc3020f2640 1 -- 172.21.15.112:0/4073321553 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc2fc0b2180 con 0x7fc2fc1e6620 2024-09-21T00:30:12.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:12.792+0000 7fc2e3fff640 1 -- 172.21.15.112:0/4073321553 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) ==== 94+0+5255 (secure 0 0 0) 0x7fc2f4063990 con 0x7fc2fc1e6620 2024-09-21T00:30:12.794 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:30:12.794 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":79,"btime":"2024-09-21T00:30:11:653023+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6113,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/3217596137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":3217596137},{"type":"v1","addr":"172.21.15.187:6835","nonce":3217596137}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":73},{"gid":6649,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6837/776068904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":776068904},{"type":"v1","addr":"172.21.15.187:6837","nonce":776068904}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":73}],"filesystems":[{"mdsmap":{"epoch":79,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T00:29:48.456672+0000","modified":"2024-09-21T00:30:11.653011+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":112,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7422,"mds_1":7419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7419":{"gid":7419,"name":"a","rank":1,"incarnation":78,"state":"up:active","state_seq":4,"addr":"172.21.15.112:6834/3291181711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":3291181711},{"type":"v1","addr":"172.21.15.112:6834","nonce":3291181711}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7422":{"gid":7422,"name":"c","rank":0,"incarnation":73,"state":"up:active","state_seq":4,"addr":"172.21.15.112:6835/3392867613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":3392867613},{"type":"v1","addr":"172.21.15.112:6835","nonce":3392867613}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7422,"qdb_cluster":[7419,7422]},"id":8}]} 2024-09-21T00:30:12.794 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 79 2024-09-21T00:30:12.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:12.794+0000 7fc3020f2640 1 -- 172.21.15.112:0/4073321553 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fc2e8078bc0 msgr2=0x7fc2e807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:12.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:12.794+0000 7fc3020f2640 1 --2- 172.21.15.112:0/4073321553 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fc2e8078bc0 0x7fc2e807b0d0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec034000 tx=0x7fc2ec004730 comp rx=0 tx=0).stop 2024-09-21T00:30:12.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:12.794+0000 7fc3020f2640 1 -- 172.21.15.112:0/4073321553 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2fc1e6620 msgr2=0x7fc2fc159b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:12.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:12.794+0000 7fc3020f2640 1 --2- 172.21.15.112:0/4073321553 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2fc1e6620 0x7fc2fc159b60 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7fc2f400cae0 tx=0x7fc2f40071c0 comp rx=0 tx=0).stop 2024-09-21T00:30:12.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:12.794+0000 7fc3020f2640 1 -- 172.21.15.112:0/4073321553 shutdown_connections 2024-09-21T00:30:12.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:12.794+0000 7fc3020f2640 1 --2- 172.21.15.112:0/4073321553 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fc2e8078bc0 0x7fc2e807b0d0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:12.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:12.794+0000 7fc3020f2640 1 --2- 172.21.15.112:0/4073321553 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2fc1e6620 0x7fc2fc159b60 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:12.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:12.794+0000 7fc3020f2640 1 --2- 172.21.15.112:0/4073321553 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc2fc0b9dc0 0x7fc2fc15b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:12.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:12.794+0000 7fc3020f2640 1 --2- 172.21.15.112:0/4073321553 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc2fc0b3b40 0x7fc2fc15af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:12.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:12.794+0000 7fc3020f2640 1 -- 172.21.15.112:0/4073321553 >> 172.21.15.112:0/4073321553 conn(0x7fc2fc0d45e0 msgr2=0x7fc2fc0b7c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:12.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:12.794+0000 7fc3020f2640 1 -- 172.21.15.112:0/4073321553 shutdown_connections 2024-09-21T00:30:12.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:12.794+0000 7fc3020f2640 1 -- 172.21.15.112:0/4073321553 wait complete. 2024-09-21T00:30:13.211 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 79, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T00:29:48.456672+0000', 'modified': '2024-09-21T00:30:11.653011+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 112, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 2, 'in': [0, 1], 'up': {'mds_0': 7422, 'mds_1': 7419}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7419': {'gid': 7419, 'name': 'a', 'rank': 1, 'incarnation': 78, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.112:6834/3291181711', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6832', 'nonce': 3291181711}, {'type': 'v1', 'addr': '172.21.15.112:6834', 'nonce': 3291181711}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_7422': {'gid': 7422, 'name': 'c', 'rank': 0, 'incarnation': 73, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.112:6835/3392867613', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6833', 'nonce': 3392867613}, {'type': 'v1', 'addr': '172.21.15.112:6835', 'nonce': 3392867613}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7422, 'qdb_cluster': [7419, 7422]} 2024-09-21T00:30:13.211 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 2/2 2024-09-21T00:30:13.211 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-21T00:30:13.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.327+0000 7f14cada8640 1 -- 172.21.15.112:0/2964519001 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14c40b6280 msgr2=0x7f14c40cd440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:13.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.327+0000 7f14cada8640 1 --2- 172.21.15.112:0/2964519001 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14c40b6280 0x7f14c40cd440 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f14c4073fc0 tx=0x7f14bc02a980 comp rx=0 tx=0).stop 2024-09-21T00:30:13.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.327+0000 7f14cada8640 1 -- 172.21.15.112:0/2964519001 shutdown_connections 2024-09-21T00:30:13.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.327+0000 7f14cada8640 1 --2- 172.21.15.112:0/2964519001 >> v2:172.21.15.187:3300/0 conn(0x7f14c40b5930 0x7f14c40b5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:13.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.327+0000 7f14cada8640 1 --2- 172.21.15.112:0/2964519001 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14c40b6280 0x7f14c40cd440 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:13.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.327+0000 7f14cada8640 1 --2- 172.21.15.112:0/2964519001 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f14c409baa0 0x7f14c40b53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:13.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.327+0000 7f14cada8640 1 -- 172.21.15.112:0/2964519001 >> 172.21.15.112:0/2964519001 conn(0x7f14c40cef20 msgr2=0x7f14c40cf330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:13.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.328+0000 7f14cada8640 1 -- 172.21.15.112:0/2964519001 shutdown_connections 2024-09-21T00:30:13.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.328+0000 7f14cada8640 1 -- 172.21.15.112:0/2964519001 wait complete. 2024-09-21T00:30:13.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.329+0000 7f14cada8640 1 Processor -- start 2024-09-21T00:30:13.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.329+0000 7f14cada8640 1 -- start start 2024-09-21T00:30:13.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.331+0000 7f14cada8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14c409baa0 0x7f14c41e51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:13.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.331+0000 7f14cada8640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f14c40b5930 0x7f14c41dfa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:13.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.331+0000 7f14cada8640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f14c40b6280 0x7f14c41f6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:13.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.331+0000 7f14cada8640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f14c40b78f0 con 0x7f14c409baa0 2024-09-21T00:30:13.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.331+0000 7f14cada8640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f14c40d0750 con 0x7f14c40b6280 2024-09-21T00:30:13.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.331+0000 7f14cada8640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f14c40b7a70 con 0x7f14c40b5930 2024-09-21T00:30:13.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.331+0000 7f14c8b44640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14c409baa0 0x7f14c41e51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:13.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.331+0000 7f14c8b44640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14c409baa0 0x7f14c41e51c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35702/0 (socket says 172.21.15.112:35702) 2024-09-21T00:30:13.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.331+0000 7f14c8b44640 1 -- 172.21.15.112:0/2863206668 learned_addr learned my addr 172.21.15.112:0/2863206668 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:30:13.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.331+0000 7f14bbfff640 1 --2- 172.21.15.112:0/2863206668 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f14c40b5930 0x7f14c41dfa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:13.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.331+0000 7f14c9345640 1 --2- 172.21.15.112:0/2863206668 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f14c40b6280 0x7f14c41f6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:13.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.331+0000 7f14c8b44640 1 -- 172.21.15.112:0/2863206668 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f14c40b5930 msgr2=0x7f14c41dfa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:13.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.331+0000 7f14c8b44640 1 --2- 172.21.15.112:0/2863206668 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f14c40b5930 0x7f14c41dfa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:13.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.331+0000 7f14c8b44640 1 -- 172.21.15.112:0/2863206668 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f14c40b6280 msgr2=0x7f14c41f6bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:13.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.331+0000 7f14c8b44640 1 --2- 172.21.15.112:0/2863206668 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f14c40b6280 0x7f14c41f6bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:13.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.331+0000 7f14c8b44640 1 -- 172.21.15.112:0/2863206668 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14c40c9090 con 0x7f14c409baa0 2024-09-21T00:30:13.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.332+0000 7f14c8b44640 1 --2- 172.21.15.112:0/2863206668 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14c409baa0 0x7f14c41e51c0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f14c000d690 tx=0x7f14c000db50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:13.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.332+0000 7f14b97fa640 1 -- 172.21.15.112:0/2863206668 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c0017070 con 0x7f14c409baa0 2024-09-21T00:30:13.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.332+0000 7f14b97fa640 1 -- 172.21.15.112:0/2863206668 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f14c0005310 con 0x7f14c409baa0 2024-09-21T00:30:13.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.332+0000 7f14cada8640 1 -- 172.21.15.112:0/2863206668 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14c40c8db0 con 0x7f14c409baa0 2024-09-21T00:30:13.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.332+0000 7f14b97fa640 1 -- 172.21.15.112:0/2863206668 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c0009820 con 0x7f14c409baa0 2024-09-21T00:30:13.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.332+0000 7f14cada8640 1 -- 172.21.15.112:0/2863206668 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14c40cc0b0 con 0x7f14c409baa0 2024-09-21T00:30:13.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.334+0000 7f14b97fa640 1 -- 172.21.15.112:0/2863206668 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f14c00099c0 con 0x7f14c409baa0 2024-09-21T00:30:13.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.334+0000 7f14b97fa640 1 --2- 172.21.15.112:0/2863206668 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f14b0078bc0 0x7f14b007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:13.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.334+0000 7f14b97fa640 1 -- 172.21.15.112:0/2863206668 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 6187+0+0 (secure 0 0 0) 0x7f14c009af10 con 0x7f14c409baa0 2024-09-21T00:30:13.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.334+0000 7f14bbfff640 1 --2- 172.21.15.112:0/2863206668 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f14b0078bc0 0x7f14b007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:13.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.335+0000 7f14d1df1740 1 Processor -- start 2024-09-21T00:30:13.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.335+0000 7f14d1df1740 1 -- start start 2024-09-21T00:30:13.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.335+0000 7f14bbfff640 1 --2- 172.21.15.112:0/2863206668 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f14b0078bc0 0x7f14b007b0d0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f14c41e24f0 tx=0x7f14b400b2e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:13.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.336+0000 7f14d1df1740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555746b04270 0x555746ec4970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:13.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.336+0000 7f14d1df1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555746adb3b0 0x555746ed2e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:13.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.336+0000 7f14d1df1740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x555746ae3590 0x555746ed33b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:13.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.336+0000 7f14d1df1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x555746e1e820 con 0x555746adb3b0 2024-09-21T00:30:13.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.336+0000 7f14d1df1740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x555746e0dc50 con 0x555746b04270 2024-09-21T00:30:13.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.336+0000 7f14d1df1740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x555746e11a60 con 0x555746ae3590 2024-09-21T00:30:13.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.336+0000 7f14bbfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555746adb3b0 0x555746ed2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:13.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.336+0000 7f14bbfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555746adb3b0 0x555746ed2e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35704/0 (socket says 172.21.15.112:35704) 2024-09-21T00:30:13.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.336+0000 7f14bbfff640 1 -- 172.21.15.112:0/2286404765 learned_addr learned my addr 172.21.15.112:0/2286404765 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:30:13.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.336+0000 7f14c8b44640 1 --2- 172.21.15.112:0/2286404765 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555746b04270 0x555746ec4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:13.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.336+0000 7f14c9345640 1 --2- 172.21.15.112:0/2286404765 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x555746ae3590 0x555746ed33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:13.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.337+0000 7f14bbfff640 1 -- 172.21.15.112:0/2286404765 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x555746ae3590 msgr2=0x555746ed33b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:30:13.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.337+0000 7f14bbfff640 1 --2- 172.21.15.112:0/2286404765 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x555746ae3590 0x555746ed33b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:13.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.337+0000 7f14bbfff640 1 -- 172.21.15.112:0/2286404765 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555746b04270 msgr2=0x555746ec4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:30:13.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.337+0000 7f14bbfff640 1 --2- 172.21.15.112:0/2286404765 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555746b04270 0x555746ec4970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:13.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.337+0000 7f14bbfff640 1 -- 172.21.15.112:0/2286404765 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555746b6f850 con 0x555746adb3b0 2024-09-21T00:30:13.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.337+0000 7f14c8b44640 1 --2- 172.21.15.112:0/2286404765 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555746b04270 0x555746ec4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:30:13.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.337+0000 7f14bbfff640 1 --2- 172.21.15.112:0/2286404765 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555746adb3b0 0x555746ed2e70 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f14b4010f50 tx=0x7f14b4013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e70382de92ff6eea server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:13.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.337+0000 7f14aa7fc640 1 -- 172.21.15.112:0/2286404765 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14b401e620 con 0x555746adb3b0 2024-09-21T00:30:13.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.337+0000 7f14aa7fc640 1 -- 172.21.15.112:0/2286404765 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f14b4011e50 con 0x555746adb3b0 2024-09-21T00:30:13.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.337+0000 7f14aa7fc640 1 -- 172.21.15.112:0/2286404765 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14b401ead0 con 0x555746adb3b0 2024-09-21T00:30:13.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.338+0000 7f14d1df1740 1 -- 172.21.15.112:0/2286404765 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555746adb3b0 msgr2=0x555746ed2e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:13.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.338+0000 7f14d1df1740 1 --2- 172.21.15.112:0/2286404765 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555746adb3b0 0x555746ed2e70 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f14b4010f50 tx=0x7f14b4013000 comp rx=0 tx=0).stop 2024-09-21T00:30:13.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.338+0000 7f14d1df1740 1 -- 172.21.15.112:0/2286404765 shutdown_connections 2024-09-21T00:30:13.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.338+0000 7f14d1df1740 1 --2- 172.21.15.112:0/2286404765 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x555746ae3590 0x555746ed33b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:13.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.338+0000 7f14d1df1740 1 --2- 172.21.15.112:0/2286404765 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555746adb3b0 0x555746ed2e70 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:13.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.338+0000 7f14d1df1740 1 --2- 172.21.15.112:0/2286404765 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555746b04270 0x555746ec4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:13.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.338+0000 7f14d1df1740 1 -- 172.21.15.112:0/2286404765 >> 172.21.15.112:0/2286404765 conn(0x555746af83c0 msgr2=0x555746eac260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:13.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.338+0000 7f14d1df1740 1 -- 172.21.15.112:0/2286404765 shutdown_connections 2024-09-21T00:30:13.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.338+0000 7f14d1df1740 1 -- 172.21.15.112:0/2286404765 wait complete. 2024-09-21T00:30:13.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.339+0000 7f14d1df1740 1 -- start start 2024-09-21T00:30:13.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.339+0000 7f14d1df1740 1 Processor -- start 2024-09-21T00:30:13.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.340+0000 7f14d1df1740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x555746adb3b0 0x555746eec380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:13.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.340+0000 7f14d1df1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555746ae3590 0x555746eedd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:13.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.340+0000 7f14d1df1740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555746b04270 0x555746eaba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:13.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.340+0000 7f14d1df1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x555746b7e620 con 0x555746ae3590 2024-09-21T00:30:13.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.340+0000 7f14d1df1740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x555746db35f0 con 0x555746b04270 2024-09-21T00:30:13.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.340+0000 7f14d1df1740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x555746eea3c0 con 0x555746adb3b0 2024-09-21T00:30:13.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.340+0000 7f14d1df1740 10 client.0 _ll_register_callbacks cb 0x555746db70a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:30:13.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.340+0000 7f14bbfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555746ae3590 0x555746eedd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:13.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.340+0000 7f14bbfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555746ae3590 0x555746eedd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35712/0 (socket says 172.21.15.112:35712) 2024-09-21T00:30:13.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.340+0000 7f14bbfff640 1 -- 172.21.15.112:0/1748965844 learned_addr learned my addr 172.21.15.112:0/1748965844 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:30:13.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.340+0000 7f14c8b44640 1 --2- 172.21.15.112:0/1748965844 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x555746adb3b0 0x555746eec380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:13.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.340+0000 7f14c9345640 1 --2- 172.21.15.112:0/1748965844 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555746b04270 0x555746eaba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:13.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.340+0000 7f14bbfff640 1 -- 172.21.15.112:0/1748965844 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x555746adb3b0 msgr2=0x555746eec380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:30:13.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.340+0000 7f14bbfff640 1 --2- 172.21.15.112:0/1748965844 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x555746adb3b0 0x555746eec380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:13.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.341+0000 7f14bbfff640 1 -- 172.21.15.112:0/1748965844 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555746b04270 msgr2=0x555746eaba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:30:13.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.341+0000 7f14bbfff640 1 --2- 172.21.15.112:0/1748965844 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555746b04270 0x555746eaba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:13.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.341+0000 7f14bbfff640 1 -- 172.21.15.112:0/1748965844 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555746be66b0 con 0x555746ae3590 2024-09-21T00:30:13.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.341+0000 7f14bbfff640 1 --2- 172.21.15.112:0/1748965844 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555746ae3590 0x555746eedd90 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f14b401e650 tx=0x7f14b401e680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec218dbfed3ffdd9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:13.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.341+0000 7f14a37fe640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T00:30:13.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.341+0000 7f14a37fe640 1 -- 172.21.15.112:0/1748965844 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14b4034060 con 0x555746ae3590 2024-09-21T00:30:13.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.341+0000 7f14a37fe640 1 -- 172.21.15.112:0/1748965844 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 6187+0+0 (secure 0 0 0) 0x7f14b401ec00 con 0x555746ae3590 2024-09-21T00:30:13.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.341+0000 7f14d1df1740 1 -- 172.21.15.112:0/1748965844 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14b4011e50 con 0x555746ae3590 2024-09-21T00:30:13.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.341+0000 7f14a37fe640 1 -- 172.21.15.112:0/1748965844 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=79) ==== 24+0+0 (secure 0 0 0) 0x7f14b403a050 con 0x555746ae3590 2024-09-21T00:30:13.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.341+0000 7f14d1df1740 10 client.7527 fetch_fsmap learned FSMap version 79 2024-09-21T00:30:13.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.341+0000 7f14d1df1740 1 -- 172.21.15.112:0/1748965844 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=79,monmap=2+}) -- 0x555746f08ad0 con 0x555746ae3590 2024-09-21T00:30:13.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.341+0000 7f14a37fe640 1 -- 172.21.15.112:0/1748965844 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f14b4011e50 con 0x555746ae3590 2024-09-21T00:30:13.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.341+0000 7f14a37fe640 1 -- 172.21.15.112:0/1748965844 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 79) ==== 3522+0+0 (secure 0 0 0) 0x7f14b403d020 con 0x555746ae3590 2024-09-21T00:30:13.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.341+0000 7f14d1df1740 10 client.7527 fetch_fsmap finished waiting for FSMap version 79 2024-09-21T00:30:13.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.341+0000 7f14d1df1740 10 client.7527 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-21T00:30:13.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.341+0000 7f14d1df1740 20 client.7527 populate_metadata read hostname 'smithi112' 2024-09-21T00:30:13.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.341+0000 7f14d1df1740 1 --2- 172.21.15.112:0/1748965844 >> [v2:172.21.15.112:6832/3291181711,v1:172.21.15.112:6834/3291181711] conn(0x555746ac2e40 0x555746ee7c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:13.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.341+0000 7f14d1df1740 4 client.7527 mds_command: new command op to 7419 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T00:30:13.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.341+0000 7f14d1df1740 1 -- 172.21.15.112:0/1748965844 --> [v2:172.21.15.112:6832/3291181711,v1:172.21.15.112:6834/3291181711] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555746e20770 con 0x555746ac2e40 2024-09-21T00:30:13.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.341+0000 7f14c8b44640 1 --2- 172.21.15.112:0/1748965844 >> [v2:172.21.15.112:6832/3291181711,v1:172.21.15.112:6834/3291181711] conn(0x555746ac2e40 0x555746ee7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:13.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.342+0000 7f14a37fe640 10 client.7527 ms_handle_connect on v2:172.21.15.112:6832/3291181711 2024-09-21T00:30:13.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.342+0000 7f14c8b44640 1 --2- 172.21.15.112:0/1748965844 >> [v2:172.21.15.112:6832/3291181711,v1:172.21.15.112:6834/3291181711] conn(0x555746ac2e40 0x555746ee7c90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3ebe45a600a4d7a4 server_cookie=a4f9814a5d4af3cd in_seq=0 out_seq=0 2024-09-21T00:30:13.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.343+0000 7f14a37fe640 1 -- 172.21.15.112:0/1748965844 <== mds.1 v2:172.21.15.112:6832/3291181711 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x555746e20770 con 0x555746ac2e40 2024-09-21T00:30:13.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.343+0000 7f14a37fe640 10 client.7527 handle_command_reply: tid=0 2024-09-21T00:30:13.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.343+0000 7f14d1df1740 1 client.7527 shutdown 2024-09-21T00:30:13.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.343+0000 7f14d1df1740 1 -- 172.21.15.112:0/1748965844 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555746ae3590 msgr2=0x555746eedd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:13.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.343+0000 7f14d1df1740 1 --2- 172.21.15.112:0/1748965844 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555746ae3590 0x555746eedd90 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f14b401e650 tx=0x7f14b401e680 comp rx=0 tx=0).stop 2024-09-21T00:30:13.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.343+0000 7f14d1df1740 1 -- 172.21.15.112:0/1748965844 shutdown_connections 2024-09-21T00:30:13.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.343+0000 7f14d1df1740 1 --2- 172.21.15.112:0/1748965844 >> [v2:172.21.15.112:6832/3291181711,v1:172.21.15.112:6834/3291181711] conn(0x555746ac2e40 0x555746ee7c90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:13.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.343+0000 7f14d1df1740 1 --2- 172.21.15.112:0/1748965844 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555746b04270 0x555746eaba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:13.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.343+0000 7f14a37fe640 0 client.7527 ms_handle_reset on v2:172.21.15.112:6832/3291181711 2024-09-21T00:30:13.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.343+0000 7f14d1df1740 1 --2- 172.21.15.112:0/1748965844 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555746ae3590 0x555746eedd90 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:13.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.343+0000 7f14d1df1740 1 --2- 172.21.15.112:0/1748965844 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x555746adb3b0 0x555746eec380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:13.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.343+0000 7f14d1df1740 1 -- 172.21.15.112:0/1748965844 >> 172.21.15.112:0/1748965844 conn(0x555746af83c0 msgr2=0x555746ed3450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:13.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.343+0000 7f14d1df1740 1 -- 172.21.15.112:0/1748965844 shutdown_connections 2024-09-21T00:30:13.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.343+0000 7f14d1df1740 1 -- 172.21.15.112:0/1748965844 wait complete. 2024-09-21T00:30:13.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.343+0000 7f14d1df1740 20 client.7527 trim_cache size 0 max 16384 2024-09-21T00:30:13.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.361+0000 7f14d1df1740 1 Processor -- start 2024-09-21T00:30:13.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.361+0000 7f14d1df1740 1 -- start start 2024-09-21T00:30:13.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.362+0000 7f14d1df1740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x555746adb3b0 0x555746ea77e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:13.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.362+0000 7f14d1df1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555746ae3590 0x555746ea7d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:13.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.362+0000 7f14d1df1740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555746b04270 0x555746eea550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:13.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.362+0000 7f14d1df1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f14c41fe5e0 con 0x555746ae3590 2024-09-21T00:30:13.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.362+0000 7f14d1df1740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f14c41e8110 con 0x555746b04270 2024-09-21T00:30:13.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.362+0000 7f14d1df1740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f14c406f370 con 0x555746adb3b0 2024-09-21T00:30:13.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.362+0000 7f14bbfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555746ae3590 0x555746ea7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:13.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.362+0000 7f14bbfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555746ae3590 0x555746ea7d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35718/0 (socket says 172.21.15.112:35718) 2024-09-21T00:30:13.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.362+0000 7f14bbfff640 1 -- 172.21.15.112:0/1186522475 learned_addr learned my addr 172.21.15.112:0/1186522475 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:30:13.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.363+0000 7f14c8b44640 1 --2- 172.21.15.112:0/1186522475 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x555746adb3b0 0x555746ea77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:13.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.363+0000 7f14c9345640 1 --2- 172.21.15.112:0/1186522475 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555746b04270 0x555746eea550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:13.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.363+0000 7f14bbfff640 1 -- 172.21.15.112:0/1186522475 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x555746adb3b0 msgr2=0x555746ea77e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:30:13.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.363+0000 7f14bbfff640 1 --2- 172.21.15.112:0/1186522475 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x555746adb3b0 0x555746ea77e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:13.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.363+0000 7f14bbfff640 1 -- 172.21.15.112:0/1186522475 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555746b04270 msgr2=0x555746eea550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:30:13.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.363+0000 7f14bbfff640 1 --2- 172.21.15.112:0/1186522475 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555746b04270 0x555746eea550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:13.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.363+0000 7f14bbfff640 1 -- 172.21.15.112:0/1186522475 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555746ea95d0 con 0x555746ae3590 2024-09-21T00:30:13.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.363+0000 7f14bbfff640 1 --2- 172.21.15.112:0/1186522475 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555746ae3590 0x555746ea7d20 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f14b402f430 tx=0x7f14b400b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23f2d2cfa6159f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:13.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.363+0000 7f14a9ffb640 1 -- 172.21.15.112:0/1186522475 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14b402a020 con 0x555746ae3590 2024-09-21T00:30:13.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.363+0000 7f14a9ffb640 1 -- 172.21.15.112:0/1186522475 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f14b4011be0 con 0x555746ae3590 2024-09-21T00:30:13.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.363+0000 7f14a9ffb640 1 -- 172.21.15.112:0/1186522475 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14b402f550 con 0x555746ae3590 2024-09-21T00:30:13.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.364+0000 7f14d1df1740 1 -- 172.21.15.112:0/1186522475 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555746ae3590 msgr2=0x555746ea7d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:13.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.364+0000 7f14d1df1740 1 --2- 172.21.15.112:0/1186522475 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555746ae3590 0x555746ea7d20 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f14b402f430 tx=0x7f14b400b880 comp rx=0 tx=0).stop 2024-09-21T00:30:13.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.364+0000 7f14d1df1740 1 -- 172.21.15.112:0/1186522475 shutdown_connections 2024-09-21T00:30:13.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.364+0000 7f14d1df1740 1 --2- 172.21.15.112:0/1186522475 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555746b04270 0x555746eea550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:13.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.364+0000 7f14d1df1740 1 --2- 172.21.15.112:0/1186522475 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555746ae3590 0x555746ea7d20 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:13.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.364+0000 7f14d1df1740 1 --2- 172.21.15.112:0/1186522475 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x555746adb3b0 0x555746ea77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:13.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.364+0000 7f14d1df1740 1 -- 172.21.15.112:0/1186522475 >> 172.21.15.112:0/1186522475 conn(0x555746af83c0 msgr2=0x555746eedb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:13.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.364+0000 7f14d1df1740 1 -- 172.21.15.112:0/1186522475 shutdown_connections 2024-09-21T00:30:13.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.364+0000 7f14d1df1740 1 -- 172.21.15.112:0/1186522475 wait complete. 2024-09-21T00:30:13.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.365+0000 7f14d1df1740 1 -- start start 2024-09-21T00:30:13.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.365+0000 7f14d1df1740 1 Processor -- start 2024-09-21T00:30:13.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.366+0000 7f14d1df1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555746adb3b0 0x555746f78810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:13.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.366+0000 7f14d1df1740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555746ae3590 0x555746ed5020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:13.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.366+0000 7f14d1df1740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x555746b04270 0x555746ed5560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:13.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.366+0000 7f14d1df1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x555746ee9ad0 con 0x555746adb3b0 2024-09-21T00:30:13.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.366+0000 7f14d1df1740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x555746eabf50 con 0x555746ae3590 2024-09-21T00:30:13.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.366+0000 7f14d1df1740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x555746ef0b20 con 0x555746b04270 2024-09-21T00:30:13.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.366+0000 7f14d1df1740 10 client.0 _ll_register_callbacks cb 0x7f14b4010e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:30:13.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.366+0000 7f14c8b44640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555746adb3b0 0x555746f78810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:13.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.366+0000 7f14c8b44640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555746adb3b0 0x555746f78810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35730/0 (socket says 172.21.15.112:35730) 2024-09-21T00:30:13.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.366+0000 7f14c8b44640 1 -- 172.21.15.112:0/3607651047 learned_addr learned my addr 172.21.15.112:0/3607651047 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:30:13.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.366+0000 7f14bbfff640 1 --2- 172.21.15.112:0/3607651047 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555746ae3590 0x555746ed5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:13.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.366+0000 7f14c9345640 1 --2- 172.21.15.112:0/3607651047 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x555746b04270 0x555746ed5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:13.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.366+0000 7f14c8b44640 1 -- 172.21.15.112:0/3607651047 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x555746b04270 msgr2=0x555746ed5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:30:13.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.366+0000 7f14c8b44640 1 --2- 172.21.15.112:0/3607651047 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x555746b04270 0x555746ed5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:13.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.367+0000 7f14c8b44640 1 -- 172.21.15.112:0/3607651047 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555746ae3590 msgr2=0x555746ed5020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:30:13.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.367+0000 7f14c8b44640 1 --2- 172.21.15.112:0/3607651047 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555746ae3590 0x555746ed5020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:13.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.367+0000 7f14c8b44640 1 -- 172.21.15.112:0/3607651047 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555746ea5450 con 0x555746adb3b0 2024-09-21T00:30:13.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.367+0000 7f14c8b44640 1 --2- 172.21.15.112:0/3607651047 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555746adb3b0 0x555746f78810 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f14c0009610 tx=0x7f14c0065040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff9d9bd60a51a7d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:13.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.367+0000 7f14a37fe640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T00:30:13.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.367+0000 7f14a37fe640 1 -- 172.21.15.112:0/3607651047 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c0020040 con 0x555746adb3b0 2024-09-21T00:30:13.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.367+0000 7f14a37fe640 1 -- 172.21.15.112:0/3607651047 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 6187+0+0 (secure 0 0 0) 0x7f14c0061d80 con 0x555746adb3b0 2024-09-21T00:30:13.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.367+0000 7f14d1df1740 1 -- 172.21.15.112:0/3607651047 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14b4011be0 con 0x555746adb3b0 2024-09-21T00:30:13.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.367+0000 7f14a37fe640 1 -- 172.21.15.112:0/3607651047 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=79) ==== 24+0+0 (secure 0 0 0) 0x7f14c0067550 con 0x555746adb3b0 2024-09-21T00:30:13.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.367+0000 7f14d1df1740 10 client.7533 fetch_fsmap learned FSMap version 79 2024-09-21T00:30:13.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.367+0000 7f14d1df1740 1 -- 172.21.15.112:0/3607651047 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=79,monmap=2+}) -- 0x555746f77d10 con 0x555746adb3b0 2024-09-21T00:30:13.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.367+0000 7f14a37fe640 1 -- 172.21.15.112:0/3607651047 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f14b4011be0 con 0x555746adb3b0 2024-09-21T00:30:13.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.367+0000 7f14a37fe640 1 -- 172.21.15.112:0/3607651047 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 79) ==== 3522+0+0 (secure 0 0 0) 0x7f14c005e470 con 0x555746adb3b0 2024-09-21T00:30:13.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.367+0000 7f14d1df1740 10 client.7533 fetch_fsmap finished waiting for FSMap version 79 2024-09-21T00:30:13.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.367+0000 7f14d1df1740 10 client.7533 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-21T00:30:13.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.367+0000 7f14d1df1740 20 client.7533 populate_metadata read hostname 'smithi112' 2024-09-21T00:30:13.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.367+0000 7f14d1df1740 1 --2- 172.21.15.112:0/3607651047 >> [v2:172.21.15.112:6832/3291181711,v1:172.21.15.112:6834/3291181711] conn(0x555746eeacb0 0x555746ef1500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:13.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.367+0000 7f14d1df1740 4 client.7533 mds_command: new command op to 7419 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T00:30:13.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.367+0000 7f14d1df1740 1 -- 172.21.15.112:0/3607651047 --> [v2:172.21.15.112:6832/3291181711,v1:172.21.15.112:6834/3291181711] -- command(tid 0: {"prefix": "status"}) -- 0x555746e1c220 con 0x555746eeacb0 2024-09-21T00:30:13.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.367+0000 7f14bbfff640 1 --2- 172.21.15.112:0/3607651047 >> [v2:172.21.15.112:6832/3291181711,v1:172.21.15.112:6834/3291181711] conn(0x555746eeacb0 0x555746ef1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:13.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.368+0000 7f14a37fe640 10 client.7533 ms_handle_connect on v2:172.21.15.112:6832/3291181711 2024-09-21T00:30:13.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.368+0000 7f14bbfff640 1 --2- 172.21.15.112:0/3607651047 >> [v2:172.21.15.112:6832/3291181711,v1:172.21.15.112:6834/3291181711] conn(0x555746eeacb0 0x555746ef1500 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1762fb3a692be80d server_cookie=a5c8df92367b0d72 in_seq=0 out_seq=0 2024-09-21T00:30:13.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.368+0000 7f14a37fe640 1 -- 172.21.15.112:0/3607651047 <== mds.1 v2:172.21.15.112:6832/3291181711 1 ==== command_reply(tid 0: 0 ) ==== 8+0+333 (crc 0 0 0) 0x555746e1c220 con 0x555746eeacb0 2024-09-21T00:30:13.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.368+0000 7f14a37fe640 10 client.7533 handle_command_reply: tid=0 2024-09-21T00:30:13.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.368+0000 7f14d1df1740 1 client.7533 shutdown 2024-09-21T00:30:13.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.368+0000 7f14d1df1740 1 -- 172.21.15.112:0/3607651047 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555746adb3b0 msgr2=0x555746f78810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:13.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.368+0000 7f14d1df1740 1 --2- 172.21.15.112:0/3607651047 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555746adb3b0 0x555746f78810 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f14c0009610 tx=0x7f14c0065040 comp rx=0 tx=0).stop 2024-09-21T00:30:13.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.368+0000 7f14d1df1740 1 -- 172.21.15.112:0/3607651047 shutdown_connections 2024-09-21T00:30:13.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.368+0000 7f14d1df1740 1 --2- 172.21.15.112:0/3607651047 >> [v2:172.21.15.112:6832/3291181711,v1:172.21.15.112:6834/3291181711] conn(0x555746eeacb0 0x555746ef1500 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:13.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.368+0000 7f14d1df1740 1 --2- 172.21.15.112:0/3607651047 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x555746b04270 0x555746ed5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:13.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.368+0000 7f14d1df1740 1 --2- 172.21.15.112:0/3607651047 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555746ae3590 0x555746ed5020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:13.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.368+0000 7f14a37fe640 0 client.7533 ms_handle_reset on v2:172.21.15.112:6832/3291181711 2024-09-21T00:30:13.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.368+0000 7f14d1df1740 1 --2- 172.21.15.112:0/3607651047 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555746adb3b0 0x555746f78810 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:13.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.368+0000 7f14d1df1740 1 -- 172.21.15.112:0/3607651047 >> 172.21.15.112:0/3607651047 conn(0x555746af83c0 msgr2=0x555746ea1920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:13.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.368+0000 7f14d1df1740 1 -- 172.21.15.112:0/3607651047 shutdown_connections 2024-09-21T00:30:13.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.368+0000 7f14d1df1740 1 -- 172.21.15.112:0/3607651047 wait complete. 2024-09-21T00:30:13.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.368+0000 7f14d1df1740 20 client.7533 trim_cache size 0 max 16384 2024-09-21T00:30:13.383 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-21T00:30:13.383 INFO:teuthology.orchestra.run.smithi112.stdout: "cluster_fsid": "8b34692b-dc65-49b2-a08f-26f707b96ac5", 2024-09-21T00:30:13.384 INFO:teuthology.orchestra.run.smithi112.stdout: "whoami": 1, 2024-09-21T00:30:13.384 INFO:teuthology.orchestra.run.smithi112.stdout: "id": 7419, 2024-09-21T00:30:13.384 INFO:teuthology.orchestra.run.smithi112.stdout: "want_state": "up:active", 2024-09-21T00:30:13.384 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "up:active", 2024-09-21T00:30:13.384 INFO:teuthology.orchestra.run.smithi112.stdout: "fs_name": "cephfs", 2024-09-21T00:30:13.384 INFO:teuthology.orchestra.run.smithi112.stdout: "rank_uptime": 2.7165182840000002, 2024-09-21T00:30:13.384 INFO:teuthology.orchestra.run.smithi112.stdout: "mdsmap_epoch": 79, 2024-09-21T00:30:13.384 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch": 112, 2024-09-21T00:30:13.384 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch_barrier": 112, 2024-09-21T00:30:13.385 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 10.968642110999999 2024-09-21T00:30:13.385 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-21T00:30:13.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.369+0000 7f14cada8640 1 -- 172.21.15.112:0/2863206668 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f14b0078bc0 msgr2=0x7f14b007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:13.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.369+0000 7f14cada8640 1 --2- 172.21.15.112:0/2863206668 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f14b0078bc0 0x7f14b007b0d0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f14c41e24f0 tx=0x7f14b400b2e0 comp rx=0 tx=0).stop 2024-09-21T00:30:13.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.369+0000 7f14cada8640 1 -- 172.21.15.112:0/2863206668 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14c409baa0 msgr2=0x7f14c41e51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:13.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.369+0000 7f14cada8640 1 --2- 172.21.15.112:0/2863206668 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14c409baa0 0x7f14c41e51c0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f14c000d690 tx=0x7f14c000db50 comp rx=0 tx=0).stop 2024-09-21T00:30:13.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.369+0000 7f14cada8640 1 -- 172.21.15.112:0/2863206668 shutdown_connections 2024-09-21T00:30:13.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.369+0000 7f14cada8640 1 --2- 172.21.15.112:0/2863206668 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f14c40b6280 0x7f14c41f6bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:13.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.369+0000 7f14cada8640 1 --2- 172.21.15.112:0/2863206668 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f14c40b5930 0x7f14c41dfa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:13.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.369+0000 7f14cada8640 1 --2- 172.21.15.112:0/2863206668 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f14b0078bc0 0x7f14b007b0d0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:13.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.369+0000 7f14cada8640 1 --2- 172.21.15.112:0/2863206668 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14c409baa0 0x7f14c41e51c0 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:13.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.369+0000 7f14cada8640 1 -- 172.21.15.112:0/2863206668 >> 172.21.15.112:0/2863206668 conn(0x7f14c40cef20 msgr2=0x7f14c40cdcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:13.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.369+0000 7f14cada8640 1 -- 172.21.15.112:0/2863206668 shutdown_connections 2024-09-21T00:30:13.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.369+0000 7f14cada8640 1 -- 172.21.15.112:0/2863206668 wait complete. 2024-09-21T00:30:13.388 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-21T00:30:13.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.502+0000 7f5424630640 1 -- 172.21.15.112:0/1216196478 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5408004d90 msgr2=0x7f5408005220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:13.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.502+0000 7f5424630640 1 --2- 172.21.15.112:0/1216196478 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5408004d90 0x7f5408005220 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f5414005fd0 tx=0x7f5408005830 comp rx=0 tx=0).stop 2024-09-21T00:30:13.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.502+0000 7f5422bcd640 1 -- 172.21.15.112:0/1216196478 reap_dead start 2024-09-21T00:30:13.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.502+0000 7f5424630640 1 -- 172.21.15.112:0/1216196478 shutdown_connections 2024-09-21T00:30:13.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.502+0000 7f5424630640 1 -- 172.21.15.112:0/1216196478 >> 172.21.15.112:0/1216196478 conn(0x7f541c08bf90 msgr2=0x7f541c0d4650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:13.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.502+0000 7f5424630640 1 -- 172.21.15.112:0/1216196478 shutdown_connections 2024-09-21T00:30:13.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.502+0000 7f5424630640 1 -- 172.21.15.112:0/1216196478 wait complete. 2024-09-21T00:30:13.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.504+0000 7f5424630640 1 Processor -- start 2024-09-21T00:30:13.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.504+0000 7f5424630640 1 -- start start 2024-09-21T00:30:13.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.505+0000 7f5424630640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5408004d90 0x7f541c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:13.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.505+0000 7f5424630640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f541c1588a0 0x7f541c15a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:13.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.505+0000 7f5424630640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f541c15ab40 0x7f541c0b18c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:13.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.505+0000 7f5424630640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f541c06ea50 con 0x7f541c1588a0 2024-09-21T00:30:13.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.505+0000 7f5424630640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f541c06e8d0 con 0x7f541c15ab40 2024-09-21T00:30:13.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.505+0000 7f5424630640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f541c06ebd0 con 0x7f5408004d90 2024-09-21T00:30:13.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.505+0000 7f5421bcb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f541c1588a0 0x7f541c15a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:13.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.505+0000 7f5421bcb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f541c1588a0 0x7f541c15a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35738/0 (socket says 172.21.15.112:35738) 2024-09-21T00:30:13.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.505+0000 7f5421bcb640 1 -- 172.21.15.112:0/167552500 learned_addr learned my addr 172.21.15.112:0/167552500 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:30:13.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.505+0000 7f54223cc640 1 --2- 172.21.15.112:0/167552500 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5408004d90 0x7f541c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:13.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.505+0000 7f5422bcd640 1 --2- 172.21.15.112:0/167552500 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f541c15ab40 0x7f541c0b18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:13.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.505+0000 7f5421bcb640 1 -- 172.21.15.112:0/167552500 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5408004d90 msgr2=0x7f541c0b4dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:13.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.505+0000 7f5421bcb640 1 --2- 172.21.15.112:0/167552500 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5408004d90 0x7f541c0b4dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:13.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.505+0000 7f5421bcb640 1 -- 172.21.15.112:0/167552500 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f541c15ab40 msgr2=0x7f541c0b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:13.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.505+0000 7f5421bcb640 1 --2- 172.21.15.112:0/167552500 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f541c15ab40 0x7f541c0b18c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:13.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.506+0000 7f5421bcb640 1 -- 172.21.15.112:0/167552500 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f541c0b4000 con 0x7f541c1588a0 2024-09-21T00:30:13.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.506+0000 7f5421bcb640 1 --2- 172.21.15.112:0/167552500 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f541c1588a0 0x7f541c15a600 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f540c003ef0 tx=0x7f540c001800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:13.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.506+0000 7f5412ffd640 1 -- 172.21.15.112:0/167552500 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f540c015070 con 0x7f541c1588a0 2024-09-21T00:30:13.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.506+0000 7f5412ffd640 1 -- 172.21.15.112:0/167552500 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f540c011900 con 0x7f541c1588a0 2024-09-21T00:30:13.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.506+0000 7f5424630640 1 -- 172.21.15.112:0/167552500 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f541c0b42e0 con 0x7f541c1588a0 2024-09-21T00:30:13.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.506+0000 7f5412ffd640 1 -- 172.21.15.112:0/167552500 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f540c011d50 con 0x7f541c1588a0 2024-09-21T00:30:13.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.506+0000 7f5424630640 1 -- 172.21.15.112:0/167552500 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f541c15b060 con 0x7f541c1588a0 2024-09-21T00:30:13.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.508+0000 7f5412ffd640 1 -- 172.21.15.112:0/167552500 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f540c010420 con 0x7f541c1588a0 2024-09-21T00:30:13.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.508+0000 7f5412ffd640 1 --2- 172.21.15.112:0/167552500 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f5400078bc0 0x7f540007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:13.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.508+0000 7f5412ffd640 1 -- 172.21.15.112:0/167552500 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 6187+0+0 (secure 0 0 0) 0x7f540c09ba70 con 0x7f541c1588a0 2024-09-21T00:30:13.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.509+0000 7f54223cc640 1 --2- 172.21.15.112:0/167552500 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f5400078bc0 0x7f540007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:13.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.509+0000 7f54223cc640 1 --2- 172.21.15.112:0/167552500 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f5400078bc0 0x7f540007b0d0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f5418001370 tx=0x7f54180066c0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:13.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.509+0000 7f542b67f740 1 Processor -- start 2024-09-21T00:30:13.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.509+0000 7f542b67f740 1 -- start start 2024-09-21T00:30:13.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.511+0000 7f542b67f740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557911113270 0x5579114e4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:13.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.511+0000 7f542b67f740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5579110ea3b0 0x5579114e54e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:13.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.511+0000 7f542b67f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5579110f2590 0x5579115023e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:13.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.511+0000 7f542b67f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55791142d820 con 0x5579110f2590 2024-09-21T00:30:13.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.511+0000 7f542b67f740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55791141cc50 con 0x557911113270 2024-09-21T00:30:13.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.511+0000 7f542b67f740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x557911420a60 con 0x5579110ea3b0 2024-09-21T00:30:13.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.511+0000 7f5422bcd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5579110f2590 0x5579115023e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:13.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.511+0000 7f5422bcd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5579110f2590 0x5579115023e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35752/0 (socket says 172.21.15.112:35752) 2024-09-21T00:30:13.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.511+0000 7f5422bcd640 1 -- 172.21.15.112:0/3997434250 learned_addr learned my addr 172.21.15.112:0/3997434250 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:30:13.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.511+0000 7f5421bcb640 1 --2- 172.21.15.112:0/3997434250 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5579110ea3b0 0x5579114e54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:13.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.511+0000 7f54223cc640 1 --2- 172.21.15.112:0/3997434250 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557911113270 0x5579114e4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:13.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.511+0000 7f5422bcd640 1 -- 172.21.15.112:0/3997434250 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5579110ea3b0 msgr2=0x5579114e54e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:30:13.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.511+0000 7f5422bcd640 1 --2- 172.21.15.112:0/3997434250 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5579110ea3b0 0x5579114e54e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:13.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.511+0000 7f5422bcd640 1 -- 172.21.15.112:0/3997434250 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557911113270 msgr2=0x5579114e4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:30:13.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.511+0000 7f5422bcd640 1 --2- 172.21.15.112:0/3997434250 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557911113270 0x5579114e4fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:13.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.511+0000 7f5422bcd640 1 -- 172.21.15.112:0/3997434250 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55791117e850 con 0x5579110f2590 2024-09-21T00:30:13.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.511+0000 7f5421bcb640 1 --2- 172.21.15.112:0/3997434250 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5579110ea3b0 0x5579114e54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:30:13.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.511+0000 7f54223cc640 1 --2- 172.21.15.112:0/3997434250 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557911113270 0x5579114e4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:30:13.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.511+0000 7f5422bcd640 1 --2- 172.21.15.112:0/3997434250 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5579110f2590 0x5579115023e0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f5414002230 tx=0x7f5414004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f306a66ea733112 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:13.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.511+0000 7f53fffff640 1 -- 172.21.15.112:0/3997434250 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5414038040 con 0x5579110f2590 2024-09-21T00:30:13.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.512+0000 7f53fffff640 1 -- 172.21.15.112:0/3997434250 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5414044070 con 0x5579110f2590 2024-09-21T00:30:13.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.512+0000 7f53fffff640 1 -- 172.21.15.112:0/3997434250 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f541403d650 con 0x5579110f2590 2024-09-21T00:30:13.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.512+0000 7f542b67f740 1 -- 172.21.15.112:0/3997434250 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5579110f2590 msgr2=0x5579115023e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:13.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.512+0000 7f542b67f740 1 --2- 172.21.15.112:0/3997434250 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5579110f2590 0x5579115023e0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f5414002230 tx=0x7f5414004ab0 comp rx=0 tx=0).stop 2024-09-21T00:30:13.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.512+0000 7f542b67f740 1 -- 172.21.15.112:0/3997434250 shutdown_connections 2024-09-21T00:30:13.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.512+0000 7f542b67f740 1 --2- 172.21.15.112:0/3997434250 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5579110f2590 0x5579115023e0 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:13.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.512+0000 7f542b67f740 1 --2- 172.21.15.112:0/3997434250 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5579110ea3b0 0x5579114e54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:13.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.512+0000 7f542b67f740 1 --2- 172.21.15.112:0/3997434250 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557911113270 0x5579114e4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:13.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.512+0000 7f542b67f740 1 -- 172.21.15.112:0/3997434250 >> 172.21.15.112:0/3997434250 conn(0x5579111073c0 msgr2=0x5579114bb260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:13.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.513+0000 7f542b67f740 1 -- 172.21.15.112:0/3997434250 shutdown_connections 2024-09-21T00:30:13.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.513+0000 7f542b67f740 1 -- 172.21.15.112:0/3997434250 wait complete. 2024-09-21T00:30:13.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.513+0000 7f542b67f740 1 -- start start 2024-09-21T00:30:13.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.513+0000 7f542b67f740 1 Processor -- start 2024-09-21T00:30:13.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.515+0000 7f542b67f740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5579110ea3b0 0x5579114f5670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:13.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.515+0000 7f542b67f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5579110f2590 0x5579114d27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:13.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.515+0000 7f542b67f740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557911113270 0x5579114d2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:13.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.515+0000 7f542b67f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55791118d620 con 0x5579110f2590 2024-09-21T00:30:13.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.515+0000 7f542b67f740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5579113c25f0 con 0x557911113270 2024-09-21T00:30:13.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.515+0000 7f542b67f740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x5579114d1f80 con 0x5579110ea3b0 2024-09-21T00:30:13.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.515+0000 7f542b67f740 10 client.0 _ll_register_callbacks cb 0x5579113c60a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:30:13.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.515+0000 7f5421bcb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5579110f2590 0x5579114d27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:13.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.515+0000 7f5421bcb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5579110f2590 0x5579114d27a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35762/0 (socket says 172.21.15.112:35762) 2024-09-21T00:30:13.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.515+0000 7f5421bcb640 1 -- 172.21.15.112:0/719462542 learned_addr learned my addr 172.21.15.112:0/719462542 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:30:13.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.515+0000 7f54223cc640 1 --2- 172.21.15.112:0/719462542 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5579110ea3b0 0x5579114f5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:13.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.515+0000 7f5422bcd640 1 --2- 172.21.15.112:0/719462542 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557911113270 0x5579114d2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:13.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.515+0000 7f5421bcb640 1 -- 172.21.15.112:0/719462542 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5579110ea3b0 msgr2=0x5579114f5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:30:13.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.515+0000 7f5421bcb640 1 --2- 172.21.15.112:0/719462542 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5579110ea3b0 0x5579114f5670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:13.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.515+0000 7f5421bcb640 1 -- 172.21.15.112:0/719462542 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557911113270 msgr2=0x5579114d2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:30:13.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.515+0000 7f5421bcb640 1 --2- 172.21.15.112:0/719462542 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557911113270 0x5579114d2ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:13.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.515+0000 7f5421bcb640 1 -- 172.21.15.112:0/719462542 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5579111f56b0 con 0x5579110f2590 2024-09-21T00:30:13.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.516+0000 7f5421bcb640 1 --2- 172.21.15.112:0/719462542 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5579110f2590 0x5579114d27a0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f540c001eb0 tx=0x7f540c062f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de594f54f96ce222 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:13.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.516+0000 7f53fd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T00:30:13.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.516+0000 7f53fd7fa640 1 -- 172.21.15.112:0/719462542 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f540c089020 con 0x5579110f2590 2024-09-21T00:30:13.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.516+0000 7f53fd7fa640 1 -- 172.21.15.112:0/719462542 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 6187+0+0 (secure 0 0 0) 0x7f540c06a360 con 0x5579110f2590 2024-09-21T00:30:13.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.516+0000 7f542b67f740 1 -- 172.21.15.112:0/719462542 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5414044070 con 0x5579110f2590 2024-09-21T00:30:13.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.516+0000 7f53fd7fa640 1 -- 172.21.15.112:0/719462542 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=79) ==== 24+0+0 (secure 0 0 0) 0x7f540c020070 con 0x5579110f2590 2024-09-21T00:30:13.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.516+0000 7f542b67f740 10 client.7545 fetch_fsmap learned FSMap version 79 2024-09-21T00:30:13.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.516+0000 7f542b67f740 1 -- 172.21.15.112:0/719462542 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=79,monmap=2+}) -- 0x5579114cdc80 con 0x5579110f2590 2024-09-21T00:30:13.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.516+0000 7f53fd7fa640 1 -- 172.21.15.112:0/719462542 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5414044070 con 0x5579110f2590 2024-09-21T00:30:13.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.516+0000 7f53fd7fa640 1 -- 172.21.15.112:0/719462542 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 79) ==== 3522+0+0 (secure 0 0 0) 0x7f540c07db30 con 0x5579110f2590 2024-09-21T00:30:13.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.516+0000 7f542b67f740 10 client.7545 fetch_fsmap finished waiting for FSMap version 79 2024-09-21T00:30:13.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.516+0000 7f542b67f740 10 client.7545 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T00:30:13.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.516+0000 7f542b67f740 20 client.7545 populate_metadata read hostname 'smithi112' 2024-09-21T00:30:13.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.516+0000 7f542b67f740 1 --2- 172.21.15.112:0/719462542 >> [v2:172.21.15.112:6833/3392867613,v1:172.21.15.112:6835/3392867613] conn(0x5579110d1e40 0x5579114cb6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:13.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.516+0000 7f542b67f740 4 client.7545 mds_command: new command op to 7422 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T00:30:13.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.516+0000 7f542b67f740 1 -- 172.21.15.112:0/719462542 --> [v2:172.21.15.112:6833/3392867613,v1:172.21.15.112:6835/3392867613] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55791142f770 con 0x5579110d1e40 2024-09-21T00:30:13.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.516+0000 7f54223cc640 1 --2- 172.21.15.112:0/719462542 >> [v2:172.21.15.112:6833/3392867613,v1:172.21.15.112:6835/3392867613] conn(0x5579110d1e40 0x5579114cb6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:13.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.517+0000 7f53fd7fa640 10 client.7545 ms_handle_connect on v2:172.21.15.112:6833/3392867613 2024-09-21T00:30:13.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.517+0000 7f54223cc640 1 --2- 172.21.15.112:0/719462542 >> [v2:172.21.15.112:6833/3392867613,v1:172.21.15.112:6835/3392867613] conn(0x5579110d1e40 0x5579114cb6f0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f08d5c88df3aa37c server_cookie=9be691118ce44f04 in_seq=0 out_seq=0 2024-09-21T00:30:13.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.517+0000 7f53fd7fa640 1 -- 172.21.15.112:0/719462542 <== mds.0 v2:172.21.15.112:6833/3392867613 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55791142f770 con 0x5579110d1e40 2024-09-21T00:30:13.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.517+0000 7f53fd7fa640 10 client.7545 handle_command_reply: tid=0 2024-09-21T00:30:13.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.517+0000 7f542b67f740 1 client.7545 shutdown 2024-09-21T00:30:13.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.518+0000 7f542b67f740 1 -- 172.21.15.112:0/719462542 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5579110f2590 msgr2=0x5579114d27a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:13.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.518+0000 7f542b67f740 1 --2- 172.21.15.112:0/719462542 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5579110f2590 0x5579114d27a0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f540c001eb0 tx=0x7f540c062f10 comp rx=0 tx=0).stop 2024-09-21T00:30:13.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.518+0000 7f542b67f740 1 -- 172.21.15.112:0/719462542 shutdown_connections 2024-09-21T00:30:13.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.518+0000 7f542b67f740 1 --2- 172.21.15.112:0/719462542 >> [v2:172.21.15.112:6833/3392867613,v1:172.21.15.112:6835/3392867613] conn(0x5579110d1e40 0x5579114cb6f0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:13.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.518+0000 7f542b67f740 1 --2- 172.21.15.112:0/719462542 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557911113270 0x5579114d2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:13.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.518+0000 7f542b67f740 1 --2- 172.21.15.112:0/719462542 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5579110f2590 0x5579114d27a0 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:13.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.518+0000 7f53fd7fa640 0 client.7545 ms_handle_reset on v2:172.21.15.112:6833/3392867613 2024-09-21T00:30:13.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.518+0000 7f542b67f740 1 --2- 172.21.15.112:0/719462542 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5579110ea3b0 0x5579114f5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:13.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.518+0000 7f542b67f740 1 -- 172.21.15.112:0/719462542 >> 172.21.15.112:0/719462542 conn(0x5579111073c0 msgr2=0x5579114b5b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:13.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.518+0000 7f542b67f740 1 -- 172.21.15.112:0/719462542 shutdown_connections 2024-09-21T00:30:13.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.518+0000 7f542b67f740 1 -- 172.21.15.112:0/719462542 wait complete. 2024-09-21T00:30:13.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.518+0000 7f542b67f740 20 client.7545 trim_cache size 0 max 16384 2024-09-21T00:30:13.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.536+0000 7f542b67f740 1 Processor -- start 2024-09-21T00:30:13.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.536+0000 7f542b67f740 1 -- start start 2024-09-21T00:30:13.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.537+0000 7f542b67f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5579110ea3b0 0x5579114bb0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:13.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.537+0000 7f542b67f740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5579110f2590 0x5579114bb630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:13.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.537+0000 7f542b67f740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x557911113270 0x5579114f4d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:13.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.537+0000 7f542b67f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f541c202cc0 con 0x5579110ea3b0 2024-09-21T00:30:13.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.537+0000 7f542b67f740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f541c0cada0 con 0x5579110f2590 2024-09-21T00:30:13.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.537+0000 7f542b67f740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f541c2016f0 con 0x557911113270 2024-09-21T00:30:13.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.537+0000 7f54223cc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5579110ea3b0 0x5579114bb0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:13.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.537+0000 7f54223cc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5579110ea3b0 0x5579114bb0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35766/0 (socket says 172.21.15.112:35766) 2024-09-21T00:30:13.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.537+0000 7f54223cc640 1 -- 172.21.15.112:0/2741677592 learned_addr learned my addr 172.21.15.112:0/2741677592 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:30:13.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.537+0000 7f54223cc640 1 -- 172.21.15.112:0/2741677592 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x557911113270 msgr2=0x5579114f4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:30:13.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.537+0000 7f54223cc640 1 --2- 172.21.15.112:0/2741677592 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x557911113270 0x5579114f4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:13.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.537+0000 7f54223cc640 1 -- 172.21.15.112:0/2741677592 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5579110f2590 msgr2=0x5579114bb630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:30:13.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.537+0000 7f54223cc640 1 --2- 172.21.15.112:0/2741677592 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5579110f2590 0x5579114bb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:13.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.538+0000 7f54223cc640 1 -- 172.21.15.112:0/2741677592 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579114e5850 con 0x5579110ea3b0 2024-09-21T00:30:13.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.538+0000 7f54223cc640 1 --2- 172.21.15.112:0/2741677592 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5579110ea3b0 0x5579114bb0f0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f5418001eb0 tx=0x7f5418015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=544765c0e5b91a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:13.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.538+0000 7f53ff7fe640 1 -- 172.21.15.112:0/2741677592 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5418012810 con 0x5579110ea3b0 2024-09-21T00:30:13.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.538+0000 7f53ff7fe640 1 -- 172.21.15.112:0/2741677592 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f54180129b0 con 0x5579110ea3b0 2024-09-21T00:30:13.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.538+0000 7f53ff7fe640 1 -- 172.21.15.112:0/2741677592 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5418012bc0 con 0x5579110ea3b0 2024-09-21T00:30:13.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.538+0000 7f542b67f740 1 -- 172.21.15.112:0/2741677592 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5579110ea3b0 msgr2=0x5579114bb0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:13.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.538+0000 7f542b67f740 1 --2- 172.21.15.112:0/2741677592 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5579110ea3b0 0x5579114bb0f0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f5418001eb0 tx=0x7f5418015000 comp rx=0 tx=0).stop 2024-09-21T00:30:13.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.539+0000 7f542b67f740 1 -- 172.21.15.112:0/2741677592 shutdown_connections 2024-09-21T00:30:13.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.539+0000 7f542b67f740 1 --2- 172.21.15.112:0/2741677592 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x557911113270 0x5579114f4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:13.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.539+0000 7f542b67f740 1 --2- 172.21.15.112:0/2741677592 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5579110f2590 0x5579114bb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:13.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.539+0000 7f542b67f740 1 --2- 172.21.15.112:0/2741677592 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5579110ea3b0 0x5579114bb0f0 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:13.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.539+0000 7f542b67f740 1 -- 172.21.15.112:0/2741677592 >> 172.21.15.112:0/2741677592 conn(0x5579111073c0 msgr2=0x5579114b0980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:13.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.539+0000 7f542b67f740 1 -- 172.21.15.112:0/2741677592 shutdown_connections 2024-09-21T00:30:13.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.539+0000 7f542b67f740 1 -- 172.21.15.112:0/2741677592 wait complete. 2024-09-21T00:30:13.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.539+0000 7f542b67f740 1 -- start start 2024-09-21T00:30:13.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.540+0000 7f542b67f740 1 Processor -- start 2024-09-21T00:30:13.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.541+0000 7f542b67f740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5579110ea3b0 0x55791157e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:13.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.541+0000 7f542b67f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5579110f2590 0x5579114bb0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:13.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.541+0000 7f542b67f740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557911113270 0x5579114bb630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:13.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.541+0000 7f542b67f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5579114cd4b0 con 0x5579110f2590 2024-09-21T00:30:13.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.541+0000 7f542b67f740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55791150f820 con 0x557911113270 2024-09-21T00:30:13.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.541+0000 7f542b67f740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x55791150eb10 con 0x5579110ea3b0 2024-09-21T00:30:13.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.541+0000 7f542b67f740 10 client.0 _ll_register_callbacks cb 0x7f5418000f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T00:30:13.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.541+0000 7f5421bcb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5579110f2590 0x5579114bb0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:13.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.541+0000 7f5421bcb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5579110f2590 0x5579114bb0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35770/0 (socket says 172.21.15.112:35770) 2024-09-21T00:30:13.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.541+0000 7f5421bcb640 1 -- 172.21.15.112:0/3703187206 learned_addr learned my addr 172.21.15.112:0/3703187206 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:30:13.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.541+0000 7f5422bcd640 1 --2- 172.21.15.112:0/3703187206 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557911113270 0x5579114bb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:13.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.541+0000 7f54223cc640 1 --2- 172.21.15.112:0/3703187206 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5579110ea3b0 0x55791157e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:13.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.541+0000 7f5421bcb640 1 -- 172.21.15.112:0/3703187206 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5579110ea3b0 msgr2=0x55791157e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:30:13.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.541+0000 7f5421bcb640 1 --2- 172.21.15.112:0/3703187206 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5579110ea3b0 0x55791157e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:13.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.541+0000 7f5421bcb640 1 -- 172.21.15.112:0/3703187206 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557911113270 msgr2=0x5579114bb630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:30:13.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.541+0000 7f5421bcb640 1 --2- 172.21.15.112:0/3703187206 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557911113270 0x5579114bb630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:13.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.541+0000 7f5421bcb640 1 -- 172.21.15.112:0/3703187206 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5579114cd2e0 con 0x5579110f2590 2024-09-21T00:30:13.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.541+0000 7f5422bcd640 1 --2- 172.21.15.112:0/3703187206 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557911113270 0x5579114bb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:30:13.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.541+0000 7f54223cc640 1 --2- 172.21.15.112:0/3703187206 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5579110ea3b0 0x55791157e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:30:13.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.542+0000 7f5421bcb640 1 --2- 172.21.15.112:0/3703187206 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5579110f2590 0x5579114bb0f0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f540c001d00 tx=0x7f540c05fdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=682413e4109d1ef9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:13.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.542+0000 7f53fd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T00:30:13.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.542+0000 7f53fd7fa640 1 -- 172.21.15.112:0/3703187206 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f540c089070 con 0x5579110f2590 2024-09-21T00:30:13.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.542+0000 7f53fd7fa640 1 -- 172.21.15.112:0/3703187206 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 6187+0+0 (secure 0 0 0) 0x7f540c07db30 con 0x5579110f2590 2024-09-21T00:30:13.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.542+0000 7f542b67f740 1 -- 172.21.15.112:0/3703187206 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54180129b0 con 0x5579110f2590 2024-09-21T00:30:13.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.542+0000 7f53fd7fa640 1 -- 172.21.15.112:0/3703187206 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=79) ==== 24+0+0 (secure 0 0 0) 0x7f540c085e60 con 0x5579110f2590 2024-09-21T00:30:13.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.542+0000 7f542b67f740 10 client.7551 fetch_fsmap learned FSMap version 79 2024-09-21T00:30:13.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.542+0000 7f542b67f740 1 -- 172.21.15.112:0/3703187206 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=79,monmap=2+}) -- 0x5579114b8630 con 0x5579110f2590 2024-09-21T00:30:13.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.542+0000 7f53fd7fa640 1 -- 172.21.15.112:0/3703187206 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f54180129b0 con 0x5579110f2590 2024-09-21T00:30:13.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.542+0000 7f53fd7fa640 1 -- 172.21.15.112:0/3703187206 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 79) ==== 3522+0+0 (secure 0 0 0) 0x7f540c06a360 con 0x5579110f2590 2024-09-21T00:30:13.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.542+0000 7f542b67f740 10 client.7551 fetch_fsmap finished waiting for FSMap version 79 2024-09-21T00:30:13.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.542+0000 7f542b67f740 10 client.7551 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T00:30:13.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.542+0000 7f542b67f740 20 client.7551 populate_metadata read hostname 'smithi112' 2024-09-21T00:30:13.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.542+0000 7f542b67f740 1 --2- 172.21.15.112:0/3703187206 >> [v2:172.21.15.112:6833/3392867613,v1:172.21.15.112:6835/3392867613] conn(0x5579114f54a0 0x5579114febd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:13.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.542+0000 7f542b67f740 4 client.7551 mds_command: new command op to 7422 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T00:30:13.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.542+0000 7f542b67f740 1 -- 172.21.15.112:0/3703187206 --> [v2:172.21.15.112:6833/3392867613,v1:172.21.15.112:6835/3392867613] -- command(tid 0: {"prefix": "status"}) -- 0x5579114d15f0 con 0x5579114f54a0 2024-09-21T00:30:13.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.542+0000 7f54223cc640 1 --2- 172.21.15.112:0/3703187206 >> [v2:172.21.15.112:6833/3392867613,v1:172.21.15.112:6835/3392867613] conn(0x5579114f54a0 0x5579114febd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:13.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.543+0000 7f53fd7fa640 10 client.7551 ms_handle_connect on v2:172.21.15.112:6833/3392867613 2024-09-21T00:30:13.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.543+0000 7f54223cc640 1 --2- 172.21.15.112:0/3703187206 >> [v2:172.21.15.112:6833/3392867613,v1:172.21.15.112:6835/3392867613] conn(0x5579114f54a0 0x5579114febd0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6a8e225d88deb9a server_cookie=dbbab9c0910aa505 in_seq=0 out_seq=0 2024-09-21T00:30:13.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.543+0000 7f53fd7fa640 1 -- 172.21.15.112:0/3703187206 <== mds.0 v2:172.21.15.112:6833/3392867613 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5579114d15f0 con 0x5579114f54a0 2024-09-21T00:30:13.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.543+0000 7f53fd7fa640 10 client.7551 handle_command_reply: tid=0 2024-09-21T00:30:13.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.543+0000 7f542b67f740 1 client.7551 shutdown 2024-09-21T00:30:13.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.543+0000 7f542b67f740 1 -- 172.21.15.112:0/3703187206 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5579110f2590 msgr2=0x5579114bb0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:13.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.543+0000 7f542b67f740 1 --2- 172.21.15.112:0/3703187206 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5579110f2590 0x5579114bb0f0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f540c001d00 tx=0x7f540c05fdf0 comp rx=0 tx=0).stop 2024-09-21T00:30:13.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.543+0000 7f542b67f740 1 -- 172.21.15.112:0/3703187206 shutdown_connections 2024-09-21T00:30:13.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.543+0000 7f542b67f740 1 --2- 172.21.15.112:0/3703187206 >> [v2:172.21.15.112:6833/3392867613,v1:172.21.15.112:6835/3392867613] conn(0x5579114f54a0 0x5579114febd0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:13.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.543+0000 7f542b67f740 1 --2- 172.21.15.112:0/3703187206 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557911113270 0x5579114bb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:13.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.543+0000 7f53fd7fa640 0 client.7551 ms_handle_reset on v2:172.21.15.112:6833/3392867613 2024-09-21T00:30:13.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.543+0000 7f542b67f740 1 --2- 172.21.15.112:0/3703187206 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5579110f2590 0x5579114bb0f0 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:13.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.543+0000 7f542b67f740 1 --2- 172.21.15.112:0/3703187206 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5579110ea3b0 0x55791157e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:13.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.543+0000 7f542b67f740 1 -- 172.21.15.112:0/3703187206 >> 172.21.15.112:0/3703187206 conn(0x5579111073c0 msgr2=0x5579114bc390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:13.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.543+0000 7f542b67f740 1 -- 172.21.15.112:0/3703187206 shutdown_connections 2024-09-21T00:30:13.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.543+0000 7f542b67f740 1 -- 172.21.15.112:0/3703187206 wait complete. 2024-09-21T00:30:13.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.543+0000 7f542b67f740 20 client.7551 trim_cache size 0 max 16384 2024-09-21T00:30:13.554 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-21T00:30:13.554 INFO:teuthology.orchestra.run.smithi112.stdout: "cluster_fsid": "8b34692b-dc65-49b2-a08f-26f707b96ac5", 2024-09-21T00:30:13.555 INFO:teuthology.orchestra.run.smithi112.stdout: "whoami": 0, 2024-09-21T00:30:13.555 INFO:teuthology.orchestra.run.smithi112.stdout: "id": 7422, 2024-09-21T00:30:13.555 INFO:teuthology.orchestra.run.smithi112.stdout: "want_state": "up:active", 2024-09-21T00:30:13.555 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "up:active", 2024-09-21T00:30:13.555 INFO:teuthology.orchestra.run.smithi112.stdout: "fs_name": "cephfs", 2024-09-21T00:30:13.555 INFO:teuthology.orchestra.run.smithi112.stdout: "rank_uptime": 10.933574402, 2024-09-21T00:30:13.555 INFO:teuthology.orchestra.run.smithi112.stdout: "mdsmap_epoch": 79, 2024-09-21T00:30:13.555 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch": 112, 2024-09-21T00:30:13.555 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch_barrier": 112, 2024-09-21T00:30:13.555 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 11.142263484000001 2024-09-21T00:30:13.556 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-21T00:30:13.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.544+0000 7f5424630640 1 -- 172.21.15.112:0/167552500 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f5400078bc0 msgr2=0x7f540007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:13.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.544+0000 7f5424630640 1 --2- 172.21.15.112:0/167552500 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f5400078bc0 0x7f540007b0d0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f5418001370 tx=0x7f54180066c0 comp rx=0 tx=0).stop 2024-09-21T00:30:13.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.544+0000 7f5424630640 1 -- 172.21.15.112:0/167552500 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f541c1588a0 msgr2=0x7f541c15a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:13.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.544+0000 7f5424630640 1 --2- 172.21.15.112:0/167552500 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f541c1588a0 0x7f541c15a600 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f540c003ef0 tx=0x7f540c001800 comp rx=0 tx=0).stop 2024-09-21T00:30:13.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.544+0000 7f5424630640 1 -- 172.21.15.112:0/167552500 shutdown_connections 2024-09-21T00:30:13.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.544+0000 7f5424630640 1 --2- 172.21.15.112:0/167552500 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f541c15ab40 0x7f541c0b18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:13.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.544+0000 7f5424630640 1 --2- 172.21.15.112:0/167552500 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f5400078bc0 0x7f540007b0d0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:13.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.544+0000 7f5424630640 1 --2- 172.21.15.112:0/167552500 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f541c1588a0 0x7f541c15a600 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:13.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.544+0000 7f5424630640 1 --2- 172.21.15.112:0/167552500 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5408004d90 0x7f541c0b4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:13.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.544+0000 7f5424630640 1 -- 172.21.15.112:0/167552500 >> 172.21.15.112:0/167552500 conn(0x7f541c08bf90 msgr2=0x7f541c0b7650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:13.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.544+0000 7f5424630640 1 -- 172.21.15.112:0/167552500 shutdown_connections 2024-09-21T00:30:13.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:13.544+0000 7f5424630640 1 -- 172.21.15.112:0/167552500 wait complete. 2024-09-21T00:30:13.559 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2024-09-21T00:30:13.691 INFO:teuthology.orchestra.run.smithi112.stdout:{"message":"","return_code":0} 2024-09-21T00:30:13.691 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-09-21T00:30:13.692 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json perf dump objecter 2024-09-21T00:30:13.816 INFO:teuthology.orchestra.run.smithi112.stdout:{"objecter":{"op_active":0,"op_laggy":0,"op_send":39,"op_send_bytes":11472,"op_resend":33,"op_reply":39,"op_latency":{"avgcount":39,"sum":0.279993238,"avgtime":0.007179313},"op_inflight":0,"oplen_avg":{"avgcount":39,"sum":61},"op":39,"op_r":0,"op_w":39,"op_rmw":0,"op_pg":0,"osdop_stat":0,"osdop_create":10,"osdop_read":0,"osdop_write":2,"osdop_writefull":7,"osdop_writesame":0,"osdop_append":0,"osdop_zero":0,"osdop_truncate":0,"osdop_delete":6,"osdop_mapext":0,"osdop_sparse_read":0,"osdop_clonerange":0,"osdop_getxattr":0,"osdop_setxattr":20,"osdop_cmpxattr":0,"osdop_rmxattr":0,"osdop_resetxattrs":0,"osdop_call":0,"osdop_watch":0,"osdop_notify":0,"osdop_src_cmpxattr":0,"osdop_pgls":0,"osdop_pgls_filter":0,"osdop_other":0,"linger_active":0,"linger_send":0,"linger_resend":0,"linger_ping":0,"poolop_active":0,"poolop_send":0,"poolop_resend":0,"poolstat_active":0,"poolstat_send":0,"poolstat_resend":0,"statfs_active":0,"statfs_send":0,"statfs_resend":0,"command_active":0,"command_send":0,"command_resend":0,"map_epoch":0,"map_full":0,"map_inc":0,"osd_sessions":6,"osd_session_open":6,"osd_session_close":0,"osd_laggy":0,"omap_wr":15,"omap_rd":0,"omap_del":1}} 2024-09-21T00:30:13.817 DEBUG:tasks.cephfs.filesystem:_json_asok output { "objecter": { "command_active": 0, "command_resend": 0, "command_send": 0, "linger_active": 0, "linger_ping": 0, "linger_resend": 0, "linger_send": 0, "map_epoch": 0, "map_full": 0, "map_inc": 0, "omap_del": 1, "omap_rd": 0, "omap_wr": 15, "op": 39, "op_Infinitylight": 0, "op_active": 0, "op_laggy": 0, "op_latency": { "avgcount": 39, "avgtime": 0.007179313, "sum": 0.279993238 }, "op_pg": 0, "op_r": 0, "op_reply": 39, "op_resend": 33, "op_rmw": 0, "op_send": 39, "op_send_bytes": 11472, "op_w": 39, "oplen_avg": { "avgcount": 39, "sum": 61 }, "osd_laggy": 0, "osd_session_close": 0, "osd_session_open": 6, "osd_sessions": 6, "osdop_append": 0, "osdop_call": 0, "osdop_clonerange": 0, "osdop_cmpxattr": 0, "osdop_create": 10, "osdop_delete": 6, "osdop_getxattr": 0, "osdop_mapext": 0, "osdop_notify": 0, "osdop_other": 0, "osdop_pgls": 0, "osdop_pgls_filter": 0, "osdop_read": 0, "osdop_resetxattrs": 0, "osdop_rmxattr": 0, "osdop_setxattr": 20, "osdop_sparse_read": 0, "osdop_src_cmpxattr": 0, "osdop_stat": 0, "osdop_truncate": 0, "osdop_watch": 0, "osdop_write": 2, "osdop_writefull": 7, "osdop_writesame": 0, "osdop_zero": 0, "poolop_active": 0, "poolop_resend": 0, "poolop_send": 0, "poolstat_active": 0, "poolstat_resend": 0, "poolstat_send": 0, "statfs_active": 0, "statfs_resend": 0, "statfs_send": 0 } } 2024-09-21T00:30:13.818 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json export dir /bravo 1 2024-09-21T00:30:13.941 INFO:teuthology.orchestra.run.smithi112.stdout:{"return_code":0} 2024-09-21T00:30:13.941 DEBUG:tasks.cephfs.filesystem:_json_asok output { "return_code": 0 } 2024-09-21T00:30:13.942 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json perf dump objecter 2024-09-21T00:30:14.065 INFO:teuthology.orchestra.run.smithi112.stdout:{"objecter":{"op_active":0,"op_laggy":0,"op_send":39,"op_send_bytes":11472,"op_resend":33,"op_reply":39,"op_latency":{"avgcount":39,"sum":0.279993238,"avgtime":0.007179313},"op_inflight":0,"oplen_avg":{"avgcount":39,"sum":61},"op":39,"op_r":0,"op_w":39,"op_rmw":0,"op_pg":0,"osdop_stat":0,"osdop_create":10,"osdop_read":0,"osdop_write":2,"osdop_writefull":7,"osdop_writesame":0,"osdop_append":0,"osdop_zero":0,"osdop_truncate":0,"osdop_delete":6,"osdop_mapext":0,"osdop_sparse_read":0,"osdop_clonerange":0,"osdop_getxattr":0,"osdop_setxattr":20,"osdop_cmpxattr":0,"osdop_rmxattr":0,"osdop_resetxattrs":0,"osdop_call":0,"osdop_watch":0,"osdop_notify":0,"osdop_src_cmpxattr":0,"osdop_pgls":0,"osdop_pgls_filter":0,"osdop_other":0,"linger_active":0,"linger_send":0,"linger_resend":0,"linger_ping":0,"poolop_active":0,"poolop_send":0,"poolop_resend":0,"poolstat_active":0,"poolstat_send":0,"poolstat_resend":0,"statfs_active":0,"statfs_send":0,"statfs_resend":0,"command_active":0,"command_send":0,"command_resend":0,"map_epoch":0,"map_full":0,"map_inc":0,"osd_sessions":6,"osd_session_open":6,"osd_session_close":0,"osd_laggy":0,"omap_wr":15,"omap_rd":0,"omap_del":1}} 2024-09-21T00:30:14.066 DEBUG:tasks.cephfs.filesystem:_json_asok output { "objecter": { "command_active": 0, "command_resend": 0, "command_send": 0, "linger_active": 0, "linger_ping": 0, "linger_resend": 0, "linger_send": 0, "map_epoch": 0, "map_full": 0, "map_inc": 0, "omap_del": 1, "omap_rd": 0, "omap_wr": 15, "op": 39, "op_Infinitylight": 0, "op_active": 0, "op_laggy": 0, "op_latency": { "avgcount": 39, "avgtime": 0.007179313, "sum": 0.279993238 }, "op_pg": 0, "op_r": 0, "op_reply": 39, "op_resend": 33, "op_rmw": 0, "op_send": 39, "op_send_bytes": 11472, "op_w": 39, "oplen_avg": { "avgcount": 39, "sum": 61 }, "osd_laggy": 0, "osd_session_close": 0, "osd_session_open": 6, "osd_sessions": 6, "osdop_append": 0, "osdop_call": 0, "osdop_clonerange": 0, "osdop_cmpxattr": 0, "osdop_create": 10, "osdop_delete": 6, "osdop_getxattr": 0, "osdop_mapext": 0, "osdop_notify": 0, "osdop_other": 0, "osdop_pgls": 0, "osdop_pgls_filter": 0, "osdop_read": 0, "osdop_resetxattrs": 0, "osdop_rmxattr": 0, "osdop_setxattr": 20, "osdop_sparse_read": 0, "osdop_src_cmpxattr": 0, "osdop_stat": 0, "osdop_truncate": 0, "osdop_watch": 0, "osdop_write": 2, "osdop_writefull": 7, "osdop_writesame": 0, "osdop_zero": 0, "poolop_active": 0, "poolop_resend": 0, "poolop_send": 0, "poolstat_active": 0, "poolstat_resend": 0, "poolstat_send": 0, "statfs_active": 0, "statfs_resend": 0, "statfs_send": 0 } } 2024-09-21T00:30:14.066 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=30 retry_count=0)... 2024-09-21T00:30:15.567 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T00:30:15.569 DEBUG:teuthology.orchestra.run.smithi187:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T00:30:19.068 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json perf dump objecter 2024-09-21T00:30:19.206 INFO:teuthology.orchestra.run.smithi112.stdout:{"objecter":{"op_active":0,"op_laggy":0,"op_send":42,"op_send_bytes":15058,"op_resend":33,"op_reply":42,"op_latency":{"avgcount":42,"sum":0.285993094,"avgtime":0.006809359},"op_inflight":0,"oplen_avg":{"avgcount":42,"sum":66},"op":42,"op_r":0,"op_w":42,"op_rmw":0,"op_pg":0,"osdop_stat":0,"osdop_create":10,"osdop_read":0,"osdop_write":3,"osdop_writefull":7,"osdop_writesame":0,"osdop_append":0,"osdop_zero":0,"osdop_truncate":0,"osdop_delete":6,"osdop_mapext":0,"osdop_sparse_read":0,"osdop_clonerange":0,"osdop_getxattr":0,"osdop_setxattr":20,"osdop_cmpxattr":0,"osdop_rmxattr":0,"osdop_resetxattrs":0,"osdop_call":0,"osdop_watch":0,"osdop_notify":0,"osdop_src_cmpxattr":0,"osdop_pgls":0,"osdop_pgls_filter":0,"osdop_other":0,"linger_active":0,"linger_send":0,"linger_resend":0,"linger_ping":0,"poolop_active":0,"poolop_send":0,"poolop_resend":0,"poolstat_active":0,"poolstat_send":0,"poolstat_resend":0,"statfs_active":0,"statfs_send":0,"statfs_resend":0,"command_active":0,"command_send":0,"command_resend":0,"map_epoch":0,"map_full":0,"map_inc":0,"osd_sessions":6,"osd_session_open":6,"osd_session_close":0,"osd_laggy":0,"omap_wr":19,"omap_rd":0,"omap_del":1}} 2024-09-21T00:30:19.207 DEBUG:tasks.cephfs.filesystem:_json_asok output { "objecter": { "command_active": 0, "command_resend": 0, "command_send": 0, "linger_active": 0, "linger_ping": 0, "linger_resend": 0, "linger_send": 0, "map_epoch": 0, "map_full": 0, "map_inc": 0, "omap_del": 1, "omap_rd": 0, "omap_wr": 19, "op": 42, "op_Infinitylight": 0, "op_active": 0, "op_laggy": 0, "op_latency": { "avgcount": 42, "avgtime": 0.006809359, "sum": 0.285993094 }, "op_pg": 0, "op_r": 0, "op_reply": 42, "op_resend": 33, "op_rmw": 0, "op_send": 42, "op_send_bytes": 15058, "op_w": 42, "oplen_avg": { "avgcount": 42, "sum": 66 }, "osd_laggy": 0, "osd_session_close": 0, "osd_session_open": 6, "osd_sessions": 6, "osdop_append": 0, "osdop_call": 0, "osdop_clonerange": 0, "osdop_cmpxattr": 0, "osdop_create": 10, "osdop_delete": 6, "osdop_getxattr": 0, "osdop_mapext": 0, "osdop_notify": 0, "osdop_other": 0, "osdop_pgls": 0, "osdop_pgls_filter": 0, "osdop_read": 0, "osdop_resetxattrs": 0, "osdop_rmxattr": 0, "osdop_setxattr": 20, "osdop_sparse_read": 0, "osdop_src_cmpxattr": 0, "osdop_stat": 0, "osdop_truncate": 0, "osdop_watch": 0, "osdop_write": 3, "osdop_writefull": 7, "osdop_writesame": 0, "osdop_zero": 0, "poolop_active": 0, "poolop_resend": 0, "poolop_send": 0, "poolstat_active": 0, "poolstat_resend": 0, "poolstat_send": 0, "statfs_active": 0, "statfs_resend": 0, "statfs_send": 0 } } 2024-09-21T00:30:19.207 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-09-21T00:30:19.207 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T00:30:19.207 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:30:19.223 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T00:30:19.223 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:30:19.280 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T00:30:19.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:19.432+0000 7f9f59169640 1 -- 172.21.15.112:0/3933056531 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f5406e500 msgr2=0x7f9f540cf370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:19.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:19.432+0000 7f9f59169640 1 --2- 172.21.15.112:0/3933056531 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f5406e500 0x7f9f540cf370 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c005fd0 tx=0x7f9f4c02a9f0 comp rx=0 tx=0).stop 2024-09-21T00:30:19.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:19.432+0000 7f9f59169640 1 -- 172.21.15.112:0/3933056531 shutdown_connections 2024-09-21T00:30:19.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:19.432+0000 7f9f59169640 1 --2- 172.21.15.112:0/3933056531 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f5406e500 0x7f9f540cf370 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:19.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:19.432+0000 7f9f59169640 1 --2- 172.21.15.112:0/3933056531 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9f541526d0 0x7f9f5415c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:19.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:19.432+0000 7f9f59169640 1 -- 172.21.15.112:0/3933056531 >> 172.21.15.112:0/3933056531 conn(0x7f9f540738b0 msgr2=0x7f9f54073cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:19.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:19.432+0000 7f9f59169640 1 -- 172.21.15.112:0/3933056531 shutdown_connections 2024-09-21T00:30:19.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:19.432+0000 7f9f59169640 1 -- 172.21.15.112:0/3933056531 wait complete. 2024-09-21T00:30:19.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:19.434+0000 7f9f59169640 1 Processor -- start 2024-09-21T00:30:19.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:19.434+0000 7f9f59169640 1 -- start start 2024-09-21T00:30:19.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:19.436+0000 7f9f59169640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9f54071020 0x7f9f54157140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:19.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:19.436+0000 7f9f59169640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f541526d0 0x7f9f54156250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:19.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:19.436+0000 7f9f59169640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9f54156790 0x7f9f540b5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:19.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:19.436+0000 7f9f59169640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9f540cfdb0 con 0x7f9f541526d0 2024-09-21T00:30:19.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:19.436+0000 7f9f59169640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9f540cfc30 con 0x7f9f54156790 2024-09-21T00:30:19.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:19.436+0000 7f9f59169640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f9f540cff30 con 0x7f9f54071020 2024-09-21T00:30:19.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:19.436+0000 7f9f5259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f541526d0 0x7f9f54156250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:19.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:19.436+0000 7f9f5259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f541526d0 0x7f9f54156250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49496/0 (socket says 172.21.15.112:49496) 2024-09-21T00:30:19.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:19.436+0000 7f9f5259c640 1 -- 172.21.15.112:0/2887553321 learned_addr learned my addr 172.21.15.112:0/2887553321 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:30:19.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:19.436+0000 7f9f5359e640 1 --2- 172.21.15.112:0/2887553321 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9f54156790 0x7f9f540b5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:19.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:19.436+0000 7f9f52d9d640 1 --2- 172.21.15.112:0/2887553321 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9f54071020 0x7f9f54157140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:19.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:19.436+0000 7f9f5259c640 1 -- 172.21.15.112:0/2887553321 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9f54071020 msgr2=0x7f9f54157140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:19.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:19.436+0000 7f9f5259c640 1 --2- 172.21.15.112:0/2887553321 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9f54071020 0x7f9f54157140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:19.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:19.436+0000 7f9f5259c640 1 -- 172.21.15.112:0/2887553321 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9f54156790 msgr2=0x7f9f540b5210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:19.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:19.436+0000 7f9f5259c640 1 --2- 172.21.15.112:0/2887553321 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9f54156790 0x7f9f540b5210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:19.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:19.436+0000 7f9f5259c640 1 -- 172.21.15.112:0/2887553321 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f541e4110 con 0x7f9f541526d0 2024-09-21T00:30:19.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:19.436+0000 7f9f5359e640 1 --2- 172.21.15.112:0/2887553321 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9f54156790 0x7f9f540b5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:30:19.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:19.436+0000 7f9f5259c640 1 --2- 172.21.15.112:0/2887553321 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f541526d0 0x7f9f54156250 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f9f4800d9c0 tx=0x7f9f4800de80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:19.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:19.436+0000 7f9f3b7fe640 1 -- 172.21.15.112:0/2887553321 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f4801b070 con 0x7f9f541526d0 2024-09-21T00:30:19.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:19.436+0000 7f9f3b7fe640 1 -- 172.21.15.112:0/2887553321 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9f4800ee30 con 0x7f9f541526d0 2024-09-21T00:30:19.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:19.436+0000 7f9f59169640 1 -- 172.21.15.112:0/2887553321 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f541e43f0 con 0x7f9f541526d0 2024-09-21T00:30:19.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:19.436+0000 7f9f3b7fe640 1 -- 172.21.15.112:0/2887553321 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f48016680 con 0x7f9f541526d0 2024-09-21T00:30:19.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:19.436+0000 7f9f59169640 1 -- 172.21.15.112:0/2887553321 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f541e49b0 con 0x7f9f541526d0 2024-09-21T00:30:19.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:19.438+0000 7f9f59169640 1 -- 172.21.15.112:0/2887553321 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f541e3e70 con 0x7f9f541526d0 2024-09-21T00:30:19.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:19.438+0000 7f9f3b7fe640 1 -- 172.21.15.112:0/2887553321 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9f4801e070 con 0x7f9f541526d0 2024-09-21T00:30:19.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:19.438+0000 7f9f3b7fe640 1 --2- 172.21.15.112:0/2887553321 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f9f40078b90 0x7f9f4007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:19.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:19.439+0000 7f9f3b7fe640 1 -- 172.21.15.112:0/2887553321 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 6187+0+0 (secure 0 0 0) 0x7f9f4809ce90 con 0x7f9f541526d0 2024-09-21T00:30:19.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:19.439+0000 7f9f52d9d640 1 --2- 172.21.15.112:0/2887553321 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f9f40078b90 0x7f9f4007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:19.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:19.439+0000 7f9f52d9d640 1 --2- 172.21.15.112:0/2887553321 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f9f40078b90 0x7f9f4007b0a0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f9f4400a770 tx=0x7f9f44008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:19.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:19.443+0000 7f9f3b7fe640 1 -- 172.21.15.112:0/2887553321 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f9f541e3e70 con 0x7f9f541526d0 2024-09-21T00:30:19.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:19.631+0000 7f9f59169640 1 -- 172.21.15.112:0/2887553321 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f9f540d0460 con 0x7f9f541526d0 2024-09-21T00:30:19.632 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:19.631+0000 7f9f3b7fe640 1 -- 172.21.15.112:0/2887553321 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v112) ==== 80+0+124 (secure 0 0 0) 0x7f9f48065070 con 0x7f9f541526d0 2024-09-21T00:30:19.632 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.187:6837/2071643267 2024-09-22T00:30:02.243502+0000 2024-09-21T00:30:19.632 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.187:6835/2071643267 2024-09-22T00:30:02.243502+0000 2024-09-21T00:30:19.632 INFO:teuthology.orchestra.run.smithi112.stderr:listed 2 entries 2024-09-21T00:30:19.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:19.633+0000 7f9f59169640 1 -- 172.21.15.112:0/2887553321 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f9f40078b90 msgr2=0x7f9f4007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:19.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:19.633+0000 7f9f59169640 1 --2- 172.21.15.112:0/2887553321 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f9f40078b90 0x7f9f4007b0a0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f9f4400a770 tx=0x7f9f44008040 comp rx=0 tx=0).stop 2024-09-21T00:30:19.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:19.633+0000 7f9f59169640 1 -- 172.21.15.112:0/2887553321 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f541526d0 msgr2=0x7f9f54156250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:19.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:19.633+0000 7f9f59169640 1 --2- 172.21.15.112:0/2887553321 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f541526d0 0x7f9f54156250 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f9f4800d9c0 tx=0x7f9f4800de80 comp rx=0 tx=0).stop 2024-09-21T00:30:19.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:19.633+0000 7f9f59169640 1 -- 172.21.15.112:0/2887553321 shutdown_connections 2024-09-21T00:30:19.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:19.633+0000 7f9f59169640 1 --2- 172.21.15.112:0/2887553321 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9f54156790 0x7f9f540b5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:19.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:19.633+0000 7f9f59169640 1 --2- 172.21.15.112:0/2887553321 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f9f40078b90 0x7f9f4007b0a0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:19.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:19.633+0000 7f9f59169640 1 --2- 172.21.15.112:0/2887553321 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f541526d0 0x7f9f54156250 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:19.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:19.633+0000 7f9f59169640 1 --2- 172.21.15.112:0/2887553321 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9f54071020 0x7f9f54157140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:19.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:19.633+0000 7f9f59169640 1 -- 172.21.15.112:0/2887553321 >> 172.21.15.112:0/2887553321 conn(0x7f9f540738b0 msgr2=0x7f9f540b4240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:19.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:19.633+0000 7f9f59169640 1 -- 172.21.15.112:0/2887553321 shutdown_connections 2024-09-21T00:30:19.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:19.633+0000 7f9f59169640 1 -- 172.21.15.112:0/2887553321 wait complete. 2024-09-21T00:30:19.645 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-21T00:30:19.645 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:30:19.646 DEBUG:teuthology.orchestra.run.smithi112:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-21T00:30:19.701 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi112.front.sepia.ceph.com 2024-09-21T00:30:19.701 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:30:19.701 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T00:30:19.730 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-09-21T00:30:19.731 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:30:19.747 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-09-21T00:30:19.747 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:30:19.804 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T00:30:19.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:19.917+0000 7f9819aa4640 1 -- 172.21.15.112:0/3442146323 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97fc003fd0 msgr2=0x7f97fc004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:19.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:19.917+0000 7f9819aa4640 1 --2- 172.21.15.112:0/3442146323 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97fc003fd0 0x7f97fc004800 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f980800c470 tx=0x7f97fc005830 comp rx=0 tx=0).stop 2024-09-21T00:30:19.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:19.917+0000 7f9813fff640 1 -- 172.21.15.112:0/3442146323 reap_dead start 2024-09-21T00:30:19.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:19.918+0000 7f9819aa4640 1 -- 172.21.15.112:0/3442146323 shutdown_connections 2024-09-21T00:30:19.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:19.918+0000 7f9819aa4640 1 -- 172.21.15.112:0/3442146323 >> 172.21.15.112:0/3442146323 conn(0x7f98140cee60 msgr2=0x7f98140cf270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:19.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:19.918+0000 7f9819aa4640 1 -- 172.21.15.112:0/3442146323 shutdown_connections 2024-09-21T00:30:19.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:19.918+0000 7f9819aa4640 1 -- 172.21.15.112:0/3442146323 wait complete. 2024-09-21T00:30:19.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:19.919+0000 7f9819aa4640 1 Processor -- start 2024-09-21T00:30:19.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:19.920+0000 7f9819aa4640 1 -- start start 2024-09-21T00:30:19.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:19.921+0000 7f9819aa4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97fc003fd0 0x7f9814154680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:19.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:19.921+0000 7f9819aa4640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f981415f920 0x7f98141e94c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:19.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:19.921+0000 7f9819aa4640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f98141e9a00 0x7f9814145b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:19.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:19.921+0000 7f9819aa4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f98140cd210 con 0x7f97fc003fd0 2024-09-21T00:30:19.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:19.921+0000 7f9819aa4640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f98140cd780 con 0x7f981415f920 2024-09-21T00:30:19.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:19.921+0000 7f9819aa4640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f9814073770 con 0x7f98141e9a00 2024-09-21T00:30:19.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:19.921+0000 7f98137fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97fc003fd0 0x7f9814154680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:19.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:19.921+0000 7f98137fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97fc003fd0 0x7f9814154680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49518/0 (socket says 172.21.15.112:49518) 2024-09-21T00:30:19.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:19.921+0000 7f98137fe640 1 -- 172.21.15.112:0/260286550 learned_addr learned my addr 172.21.15.112:0/260286550 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:30:19.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:19.921+0000 7f9812ffd640 1 --2- 172.21.15.112:0/260286550 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f981415f920 0x7f98141e94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:19.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:19.921+0000 7f9813fff640 1 --2- 172.21.15.112:0/260286550 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f98141e9a00 0x7f9814145b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:19.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:19.921+0000 7f98137fe640 1 -- 172.21.15.112:0/260286550 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f98141e9a00 msgr2=0x7f9814145b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:19.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:19.921+0000 7f98137fe640 1 --2- 172.21.15.112:0/260286550 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f98141e9a00 0x7f9814145b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:19.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:19.921+0000 7f98137fe640 1 -- 172.21.15.112:0/260286550 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f981415f920 msgr2=0x7f98141e94c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:19.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:19.921+0000 7f98137fe640 1 --2- 172.21.15.112:0/260286550 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f981415f920 0x7f98141e94c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:19.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:19.921+0000 7f98137fe640 1 -- 172.21.15.112:0/260286550 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98141e9e10 con 0x7f97fc003fd0 2024-09-21T00:30:19.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:19.921+0000 7f9812ffd640 1 --2- 172.21.15.112:0/260286550 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f981415f920 0x7f98141e94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:30:19.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:19.921+0000 7f98137fe640 1 --2- 172.21.15.112:0/260286550 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97fc003fd0 0x7f9814154680 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f9804011a50 tx=0x7f9804011f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:19.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:19.921+0000 7f97fbfff640 1 -- 172.21.15.112:0/260286550 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f980401f070 con 0x7f97fc003fd0 2024-09-21T00:30:19.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:19.921+0000 7f97fbfff640 1 -- 172.21.15.112:0/260286550 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f980400fe80 con 0x7f97fc003fd0 2024-09-21T00:30:19.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:19.921+0000 7f97fbfff640 1 -- 172.21.15.112:0/260286550 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9804012690 con 0x7f97fc003fd0 2024-09-21T00:30:19.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:19.921+0000 7f9819aa4640 1 -- 172.21.15.112:0/260286550 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98141437e0 con 0x7f97fc003fd0 2024-09-21T00:30:19.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:19.922+0000 7f9819aa4640 1 -- 172.21.15.112:0/260286550 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98141f90b0 con 0x7f97fc003fd0 2024-09-21T00:30:19.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:19.923+0000 7f9819aa4640 1 -- 172.21.15.112:0/260286550 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9814154bc0 con 0x7f97fc003fd0 2024-09-21T00:30:19.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:19.923+0000 7f97fbfff640 1 -- 172.21.15.112:0/260286550 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9804022070 con 0x7f97fc003fd0 2024-09-21T00:30:19.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:19.924+0000 7f97fbfff640 1 --2- 172.21.15.112:0/260286550 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f9800078b90 0x7f980007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:19.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:19.924+0000 7f97fbfff640 1 -- 172.21.15.112:0/260286550 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 6187+0+0 (secure 0 0 0) 0x7f980409b900 con 0x7f97fc003fd0 2024-09-21T00:30:19.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:19.924+0000 7f9812ffd640 1 --2- 172.21.15.112:0/260286550 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f9800078b90 0x7f980007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:19.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:19.925+0000 7f9812ffd640 1 --2- 172.21.15.112:0/260286550 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f9800078b90 0x7f980007b0a0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f9808033000 tx=0x7f98140d2020 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:19.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:19.928+0000 7f97fbfff640 1 -- 172.21.15.112:0/260286550 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f98141e9e10 con 0x7f97fc003fd0 2024-09-21T00:30:20.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.119+0000 7f9819aa4640 1 -- 172.21.15.112:0/260286550 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f98141599f0 con 0x7f97fc003fd0 2024-09-21T00:30:20.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.120+0000 7f97fbfff640 1 -- 172.21.15.112:0/260286550 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v112) ==== 80+0+124 (secure 0 0 0) 0x7f98040635a0 con 0x7f97fc003fd0 2024-09-21T00:30:20.120 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.187:6837/2071643267 2024-09-22T00:30:02.243502+0000 2024-09-21T00:30:20.121 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.187:6835/2071643267 2024-09-22T00:30:02.243502+0000 2024-09-21T00:30:20.121 INFO:teuthology.orchestra.run.smithi112.stderr:listed 2 entries 2024-09-21T00:30:20.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.122+0000 7f9819aa4640 1 -- 172.21.15.112:0/260286550 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f9800078b90 msgr2=0x7f980007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:20.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.122+0000 7f9819aa4640 1 --2- 172.21.15.112:0/260286550 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f9800078b90 0x7f980007b0a0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f9808033000 tx=0x7f98140d2020 comp rx=0 tx=0).stop 2024-09-21T00:30:20.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.122+0000 7f9819aa4640 1 -- 172.21.15.112:0/260286550 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97fc003fd0 msgr2=0x7f9814154680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:20.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.122+0000 7f9819aa4640 1 --2- 172.21.15.112:0/260286550 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97fc003fd0 0x7f9814154680 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f9804011a50 tx=0x7f9804011f10 comp rx=0 tx=0).stop 2024-09-21T00:30:20.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.122+0000 7f9819aa4640 1 -- 172.21.15.112:0/260286550 shutdown_connections 2024-09-21T00:30:20.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.122+0000 7f9819aa4640 1 --2- 172.21.15.112:0/260286550 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f98141e9a00 0x7f9814145b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:20.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.122+0000 7f9819aa4640 1 --2- 172.21.15.112:0/260286550 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f981415f920 0x7f98141e94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:20.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.122+0000 7f9819aa4640 1 --2- 172.21.15.112:0/260286550 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f9800078b90 0x7f980007b0a0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:20.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.122+0000 7f9819aa4640 1 --2- 172.21.15.112:0/260286550 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97fc003fd0 0x7f9814154680 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:20.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.122+0000 7f9819aa4640 1 -- 172.21.15.112:0/260286550 >> 172.21.15.112:0/260286550 conn(0x7f98140cee60 msgr2=0x7f98140b6ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:20.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.122+0000 7f9819aa4640 1 -- 172.21.15.112:0/260286550 shutdown_connections 2024-09-21T00:30:20.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.122+0000 7f9819aa4640 1 -- 172.21.15.112:0/260286550 wait complete. 2024-09-21T00:30:20.133 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-09-21T00:30:20.133 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:30:20.134 DEBUG:teuthology.orchestra.run.smithi187:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-09-21T00:30:20.166 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi187.front.sepia.ceph.com 2024-09-21T00:30:20.166 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:30:20.166 DEBUG:teuthology.orchestra.run.smithi187:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-21T00:30:20.222 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2024-09-21T00:30:20.347 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-09-21T00:30:20.347 DEBUG:tasks.cephfs.filesystem:_json_asok output [] 2024-09-21T00:30:20.348 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-09-21T00:30:20.480 INFO:teuthology.orchestra.run.smithi112.stdout:{"message":"","return_code":0} 2024-09-21T00:30:20.480 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-09-21T00:30:20.480 DEBUG:teuthology.orchestra.run.smithi112:> cephfs-table-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 0 show session 2024-09-21T00:30:20.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.531+0000 7f28b7226280 1 -- start start 2024-09-21T00:30:20.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.531+0000 7f28b7226280 1 Processor -- start 2024-09-21T00:30:20.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.532+0000 7f28b7226280 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x564c6abbcd00 0x564c6abbe580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:20.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.532+0000 7f28b7226280 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c6abba480 0x564c6abbf080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:20.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.532+0000 7f28b7226280 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564c6abba900 0x564c6abbeb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:20.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.532+0000 7f28b7226280 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x564c6abfc000 con 0x564c6abba480 2024-09-21T00:30:20.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.532+0000 7f28b7226280 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x564c6abfc180 con 0x564c6abba900 2024-09-21T00:30:20.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.532+0000 7f28b7226280 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x564c6abfc300 con 0x564c6abbcd00 2024-09-21T00:30:20.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.532+0000 7f28b4523640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c6abba480 0x564c6abbf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:20.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.532+0000 7f28b4523640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c6abba480 0x564c6abbf080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49532/0 (socket says 172.21.15.112:49532) 2024-09-21T00:30:20.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.532+0000 7f28b4523640 1 -- 172.21.15.112:0/1121880372 learned_addr learned my addr 172.21.15.112:0/1121880372 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:30:20.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.532+0000 7f28b4d24640 1 --2- 172.21.15.112:0/1121880372 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x564c6abbcd00 0x564c6abbe580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:20.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.532+0000 7f28b5525640 1 --2- 172.21.15.112:0/1121880372 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564c6abba900 0x564c6abbeb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:20.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.532+0000 7f28b4523640 1 -- 172.21.15.112:0/1121880372 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x564c6abbcd00 msgr2=0x564c6abbe580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:30:20.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.532+0000 7f28b4523640 1 --2- 172.21.15.112:0/1121880372 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x564c6abbcd00 0x564c6abbe580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:20.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.532+0000 7f28b4523640 1 -- 172.21.15.112:0/1121880372 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564c6abba900 msgr2=0x564c6abbeb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:30:20.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.532+0000 7f28b4523640 1 --2- 172.21.15.112:0/1121880372 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564c6abba900 0x564c6abbeb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:20.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.532+0000 7f28b4523640 1 -- 172.21.15.112:0/1121880372 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c69e1c780 con 0x564c6abba480 2024-09-21T00:30:20.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.532+0000 7f28b5525640 1 --2- 172.21.15.112:0/1121880372 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564c6abba900 0x564c6abbeb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:30:20.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.532+0000 7f28b4523640 1 --2- 172.21.15.112:0/1121880372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c6abba480 0x564c6abbf080 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x564c6ac1fb90 tx=0x564c6abca000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=316738fa19ff1c3d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:20.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.532+0000 7f28b251f640 1 -- 172.21.15.112:0/1121880372 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x564c65dd4340 con 0x564c6abba480 2024-09-21T00:30:20.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.532+0000 7f28b251f640 1 -- 172.21.15.112:0/1121880372 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x564c65dda380 con 0x564c6abba480 2024-09-21T00:30:20.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.532+0000 7f28b251f640 1 -- 172.21.15.112:0/1121880372 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x564c65dd5380 con 0x564c6abba480 2024-09-21T00:30:20.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.532+0000 7f28b7226280 1 -- 172.21.15.112:0/1121880372 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c69e1c000 con 0x564c6abba480 2024-09-21T00:30:20.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.532+0000 7f28b7226280 1 -- 172.21.15.112:0/1121880372 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({fsmap=0}) -- 0x564c69e1da40 con 0x564c6abba480 2024-09-21T00:30:20.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.532+0000 7f28b7226280 4 waiting for MDS map... 2024-09-21T00:30:20.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.533+0000 7f28b251f640 1 -- 172.21.15.112:0/1121880372 <== mon.0 v2:172.21.15.112:3300/0 4 ==== osd_map(112..112 src has 1..112) ==== 6187+0+0 (secure 0 0 0) 0x564c6acd8400 con 0x564c6abba480 2024-09-21T00:30:20.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.533+0000 7f28b251f640 1 -- 172.21.15.112:0/1121880372 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 80) ==== 3526+0+0 (secure 0 0 0) 0x564c69e39180 con 0x564c6abba480 2024-09-21T00:30:20.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.533+0000 7f28b7226280 4 Got MDS map 80 2024-09-21T00:30:20.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.533+0000 7f28b7226280 10 main: main 2024-09-21T00:30:20.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.533+0000 7f28b7226280 4 main: connecting to RADOS... 2024-09-21T00:30:20.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.534+0000 7f28b7226280 1 Processor -- start 2024-09-21T00:30:20.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.534+0000 7f28b7226280 1 -- start start 2024-09-21T00:30:20.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.535+0000 7f28b7226280 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x564c6abbb200 0x564c6abbf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:20.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.535+0000 7f28b7226280 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564c6abbb680 0x564c6abbfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:20.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.535+0000 7f28b7226280 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c6abbbb00 0x564c6abc0100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:20.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.535+0000 7f28b7226280 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x564c6abfc480 con 0x564c6abbbb00 2024-09-21T00:30:20.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.535+0000 7f28b7226280 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x564c6abfc600 con 0x564c6abbb680 2024-09-21T00:30:20.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.535+0000 7f28b7226280 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x564c6abfc780 con 0x564c6abbb200 2024-09-21T00:30:20.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.535+0000 7f28b4523640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c6abbbb00 0x564c6abc0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:20.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.535+0000 7f28b4523640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c6abbbb00 0x564c6abc0100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49534/0 (socket says 172.21.15.112:49534) 2024-09-21T00:30:20.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.535+0000 7f28b4523640 1 -- 172.21.15.112:0/2563533733 learned_addr learned my addr 172.21.15.112:0/2563533733 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:30:20.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.535+0000 7f28b5525640 1 --2- 172.21.15.112:0/2563533733 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x564c6abbb200 0x564c6abbf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:20.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.535+0000 7f28b4d24640 1 --2- 172.21.15.112:0/2563533733 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564c6abbb680 0x564c6abbfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:20.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.535+0000 7f28b4523640 1 -- 172.21.15.112:0/2563533733 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x564c6abbb200 msgr2=0x564c6abbf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:30:20.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.535+0000 7f28b4523640 1 --2- 172.21.15.112:0/2563533733 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x564c6abbb200 0x564c6abbf600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:20.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.535+0000 7f28b4523640 1 -- 172.21.15.112:0/2563533733 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564c6abbb680 msgr2=0x564c6abbfb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T00:30:20.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.535+0000 7f28b4523640 1 --2- 172.21.15.112:0/2563533733 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564c6abbb680 0x564c6abbfb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:20.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.535+0000 7f28b4523640 1 -- 172.21.15.112:0/2563533733 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c69e1d860 con 0x564c6abbbb00 2024-09-21T00:30:20.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.535+0000 7f28b5525640 1 --2- 172.21.15.112:0/2563533733 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x564c6abbb200 0x564c6abbf600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:30:20.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.535+0000 7f28b4d24640 1 --2- 172.21.15.112:0/2563533733 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564c6abbb680 0x564c6abbfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:30:20.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.535+0000 7f28b4523640 1 --2- 172.21.15.112:0/2563533733 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c6abbbb00 0x564c6abc0100 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x564c6acc4f30 tx=0x564c6abcaa20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d17c1cd390a3f701 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:20.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.536+0000 7f28b051b640 1 -- 172.21.15.112:0/2563533733 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x564c65dd4340 con 0x564c6abbbb00 2024-09-21T00:30:20.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.536+0000 7f28b051b640 1 -- 172.21.15.112:0/2563533733 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x564c65ddafc0 con 0x564c6abbbb00 2024-09-21T00:30:20.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.536+0000 7f28b051b640 1 -- 172.21.15.112:0/2563533733 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x564c65dd5380 con 0x564c6abbbb00 2024-09-21T00:30:20.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.536+0000 7f28b7226280 1 -- 172.21.15.112:0/2563533733 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c6abbbb00 msgr2=0x564c6abc0100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:20.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.536+0000 7f28b7226280 1 --2- 172.21.15.112:0/2563533733 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c6abbbb00 0x564c6abc0100 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x564c6acc4f30 tx=0x564c6abcaa20 comp rx=0 tx=0).stop 2024-09-21T00:30:20.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.536+0000 7f28b7226280 1 -- 172.21.15.112:0/2563533733 shutdown_connections 2024-09-21T00:30:20.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.536+0000 7f28b7226280 1 --2- 172.21.15.112:0/2563533733 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c6abbbb00 0x564c6abc0100 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:20.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.536+0000 7f28b7226280 1 --2- 172.21.15.112:0/2563533733 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564c6abbb680 0x564c6abbfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:20.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.536+0000 7f28b7226280 1 --2- 172.21.15.112:0/2563533733 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x564c6abbb200 0x564c6abbf600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:20.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.536+0000 7f28b7226280 1 -- 172.21.15.112:0/2563533733 >> 172.21.15.112:0/2563533733 conn(0x564c6abbad80 msgr2=0x564c69ec9800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:20.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.536+0000 7f28b7226280 1 -- 172.21.15.112:0/2563533733 shutdown_connections 2024-09-21T00:30:20.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.536+0000 7f28b7226280 1 -- 172.21.15.112:0/2563533733 wait complete. 2024-09-21T00:30:20.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.537+0000 7f28b7226280 1 Processor -- start 2024-09-21T00:30:20.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.537+0000 7f28b7226280 1 -- start start 2024-09-21T00:30:20.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.538+0000 7f28b7226280 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x564c6abbb200 0x564c6abbf600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:20.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.538+0000 7f28b7226280 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564c6abbb680 0x564c6abbfb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:20.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.538+0000 7f28b7226280 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c6abbbb00 0x564c6abc0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:20.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.538+0000 7f28b7226280 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x564c6abfca80 con 0x564c6abbbb00 2024-09-21T00:30:20.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.538+0000 7f28b7226280 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x564c6abfc900 con 0x564c6abbb680 2024-09-21T00:30:20.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.538+0000 7f28b7226280 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x564c6abfdc80 con 0x564c6abbb200 2024-09-21T00:30:20.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.538+0000 7f28b4523640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c6abbbb00 0x564c6abc0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:20.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.538+0000 7f28b4523640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c6abbbb00 0x564c6abc0100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49546/0 (socket says 172.21.15.112:49546) 2024-09-21T00:30:20.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.538+0000 7f28b4523640 1 -- 172.21.15.112:0/411492511 learned_addr learned my addr 172.21.15.112:0/411492511 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:30:20.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.538+0000 7f28b4d24640 1 --2- 172.21.15.112:0/411492511 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564c6abbb680 0x564c6abbfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:20.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.539+0000 7f28b5525640 1 --2- 172.21.15.112:0/411492511 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x564c6abbb200 0x564c6abbf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:20.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.539+0000 7f28b4523640 1 -- 172.21.15.112:0/411492511 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x564c6abbb200 msgr2=0x564c6abbf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:20.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.539+0000 7f28b4523640 1 --2- 172.21.15.112:0/411492511 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x564c6abbb200 0x564c6abbf600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:20.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.539+0000 7f28b4523640 1 -- 172.21.15.112:0/411492511 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564c6abbb680 msgr2=0x564c6abbfb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:20.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.539+0000 7f28b4523640 1 --2- 172.21.15.112:0/411492511 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564c6abbb680 0x564c6abbfb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:20.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.539+0000 7f28b4523640 1 -- 172.21.15.112:0/411492511 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c69e1d4a0 con 0x564c6abbbb00 2024-09-21T00:30:20.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.539+0000 7f28b5525640 1 --2- 172.21.15.112:0/411492511 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x564c6abbb200 0x564c6abbf600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:30:20.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.539+0000 7f28b4d24640 1 --2- 172.21.15.112:0/411492511 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564c6abbb680 0x564c6abbfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:30:20.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.539+0000 7f28b4523640 1 --2- 172.21.15.112:0/411492511 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c6abbbb00 0x564c6abc0100 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x564c6acc51a0 tx=0x564c6acf0cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:20.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.539+0000 7f28aed18640 1 -- 172.21.15.112:0/411492511 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x564c65dd51e0 con 0x564c6abbbb00 2024-09-21T00:30:20.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.539+0000 7f28aed18640 1 -- 172.21.15.112:0/411492511 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x564c65ddae00 con 0x564c6abbbb00 2024-09-21T00:30:20.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.539+0000 7f28b7226280 1 -- 172.21.15.112:0/411492511 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x564c69e1c960 con 0x564c6abbbb00 2024-09-21T00:30:20.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.539+0000 7f28aed18640 1 -- 172.21.15.112:0/411492511 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x564c65dd4340 con 0x564c6abbbb00 2024-09-21T00:30:20.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.539+0000 7f28b7226280 1 -- 172.21.15.112:0/411492511 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c69e1c780 con 0x564c6abbbb00 2024-09-21T00:30:20.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.539+0000 7f28b7226280 4 main: resolving pool 17 2024-09-21T00:30:20.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.540+0000 7f28aed18640 1 -- 172.21.15.112:0/411492511 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x564c6ad42c00 con 0x564c6abbbb00 2024-09-21T00:30:20.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.541+0000 7f28aed18640 1 --2- 172.21.15.112:0/411492511 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x564c69f73180 0x564c6abc0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:20.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.541+0000 7f28aed18640 1 -- 172.21.15.112:0/411492511 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 6187+0+0 (secure 0 0 0) 0x564c6ad40a00 con 0x564c6abbbb00 2024-09-21T00:30:20.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.541+0000 7f28b7226280 4 main: creating IoCtx.. 2024-09-21T00:30:20.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.541+0000 7f28b7226280 1 --2- 172.21.15.112:0/411492511 >> [v2:172.21.15.112:6808/755760621,v1:172.21.15.112:6809/755760621] conn(0x564c6abbbf80 0x564c6abc0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:20.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.541+0000 7f28b7226280 1 -- 172.21.15.112:0/411492511 --> [v2:172.21.15.112:6808/755760621,v1:172.21.15.112:6809/755760621] -- osd_op(unknown.0.0:1 17.b 17:d0630e4c:::mds0_sessionmap:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112) -- 0x564c6a1f6000 con 0x564c6abbbf80 2024-09-21T00:30:20.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.541+0000 7f28b5525640 1 --2- 172.21.15.112:0/411492511 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x564c69f73180 0x564c6abc0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:20.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.541+0000 7f28b4d24640 1 --2- 172.21.15.112:0/411492511 >> [v2:172.21.15.112:6808/755760621,v1:172.21.15.112:6809/755760621] conn(0x564c6abbbf80 0x564c6abc0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:20.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.541+0000 7f28b4d24640 1 --2- 172.21.15.112:0/411492511 >> [v2:172.21.15.112:6808/755760621,v1:172.21.15.112:6809/755760621] conn(0x564c6abbbf80 0x564c6abc0c00 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:20.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.541+0000 7f28b5525640 1 --2- 172.21.15.112:0/411492511 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x564c69f73180 0x564c6abc0680 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x564c69ea4b70 tx=0x564c6a1ba3c0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:20.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.542+0000 7f28b4d24640 1 -- 172.21.15.112:0/411492511 <== osd.0 v2:172.21.15.112:6808/755760621 1 ==== osd_op_reply(1 mds0_sessionmap [omap-get-header out=14b] v0'0 uv10 ondisk = 0) ==== 159+0+14 (crc 0 0 0) 0x564c6abac000 con 0x564c6abbbf80 2024-09-21T00:30:20.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.542+0000 7f28b7226280 1 -- 172.21.15.112:0/411492511 --> [v2:172.21.15.112:6808/755760621,v1:172.21.15.112:6809/755760621] -- osd_op(unknown.0.0:2 17.b 17:d0630e4c:::mds0_sessionmap:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112) -- 0x564c6a1f6400 con 0x564c6abbbf80 2024-09-21T00:30:20.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.542+0000 7f28b4d24640 1 -- 172.21.15.112:0/411492511 <== osd.0 v2:172.21.15.112:6808/755760621 2 ==== osd_op_reply(2 mds0_sessionmap [omap-get-vals out=5b] v0'0 uv10 ondisk = 0) ==== 159+0+5 (crc 0 0 0) 0x564c6abac000 con 0x564c6abbbf80 2024-09-21T00:30:20.554 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-21T00:30:20.554 INFO:teuthology.orchestra.run.smithi112.stdout: "0": { 2024-09-21T00:30:20.554 INFO:teuthology.orchestra.run.smithi112.stdout: "data": { 2024-09-21T00:30:20.554 INFO:teuthology.orchestra.run.smithi112.stdout: "sessions": [] 2024-09-21T00:30:20.554 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T00:30:20.555 INFO:teuthology.orchestra.run.smithi112.stdout: "result": 0 2024-09-21T00:30:20.555 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T00:30:20.555 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-21T00:30:20.555 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:30:20.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.542+0000 7f28b7226280 1 -- 172.21.15.112:0/411492511 >> [v2:172.21.15.112:6808/755760621,v1:172.21.15.112:6809/755760621] conn(0x564c6abbbf80 msgr2=0x564c6abc0c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:20.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.542+0000 7f28b7226280 1 --2- 172.21.15.112:0/411492511 >> [v2:172.21.15.112:6808/755760621,v1:172.21.15.112:6809/755760621] conn(0x564c6abbbf80 0x564c6abc0c00 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:20.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.542+0000 7f28b7226280 1 -- 172.21.15.112:0/411492511 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x564c69f73180 msgr2=0x564c6abc0680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:20.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.542+0000 7f28b7226280 1 --2- 172.21.15.112:0/411492511 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x564c69f73180 0x564c6abc0680 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x564c69ea4b70 tx=0x564c6a1ba3c0 comp rx=0 tx=0).stop 2024-09-21T00:30:20.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.542+0000 7f28b7226280 1 -- 172.21.15.112:0/411492511 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c6abbbb00 msgr2=0x564c6abc0100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:20.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.542+0000 7f28b7226280 1 --2- 172.21.15.112:0/411492511 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c6abbbb00 0x564c6abc0100 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x564c6acc51a0 tx=0x564c6acf0cc0 comp rx=0 tx=0).stop 2024-09-21T00:30:20.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.542+0000 7f28b4d24640 1 -- 172.21.15.112:0/411492511 reap_dead start 2024-09-21T00:30:20.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.542+0000 7f28b7226280 1 -- 172.21.15.112:0/411492511 shutdown_connections 2024-09-21T00:30:20.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.542+0000 7f28b7226280 1 -- 172.21.15.112:0/411492511 >> 172.21.15.112:0/411492511 conn(0x564c6abbad80 msgr2=0x564c69ec9800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:20.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.542+0000 7f28b7226280 1 -- 172.21.15.112:0/411492511 shutdown_connections 2024-09-21T00:30:20.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.542+0000 7f28b7226280 1 -- 172.21.15.112:0/411492511 wait complete. 2024-09-21T00:30:20.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.542+0000 7f28b7226280 1 -- 172.21.15.112:0/1121880372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c6abba480 msgr2=0x564c6abbf080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:20.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.542+0000 7f28b7226280 1 --2- 172.21.15.112:0/1121880372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c6abba480 0x564c6abbf080 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x564c6ac1fb90 tx=0x564c6abca000 comp rx=0 tx=0).stop 2024-09-21T00:30:20.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.543+0000 7f28b7226280 1 -- 172.21.15.112:0/1121880372 shutdown_connections 2024-09-21T00:30:20.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.543+0000 7f28b7226280 1 --2- 172.21.15.112:0/1121880372 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564c6abba900 0x564c6abbeb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:20.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.543+0000 7f28b7226280 1 --2- 172.21.15.112:0/1121880372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c6abba480 0x564c6abbf080 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:20.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.543+0000 7f28b7226280 1 --2- 172.21.15.112:0/1121880372 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x564c6abbcd00 0x564c6abbe580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:20.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.543+0000 7f28b7226280 1 -- 172.21.15.112:0/1121880372 >> 172.21.15.112:0/1121880372 conn(0x564c69f73600 msgr2=0x564c69ec9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:20.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.543+0000 7f28b7226280 1 -- 172.21.15.112:0/1121880372 shutdown_connections 2024-09-21T00:30:20.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.543+0000 7f28b7226280 1 -- 172.21.15.112:0/1121880372 wait complete. 2024-09-21T00:30:20.562 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-table-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '0', 'show', 'session'] in time 0:00:00.081959, result: { "0": { "data": { "sessions": [] }, "result": 0 } } 2024-09-21T00:30:20.562 INFO:tasks.cephfs.test_sessionmap:SessionMap: { "0": { "data": { "sessions": [] }, "result": 0 } } 2024-09-21T00:30:20.563 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T00:30:20.563 DEBUG:teuthology.orchestra.run.smithi112:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T00:30:20.607 DEBUG:teuthology.orchestra.run.smithi187:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T00:30:20.647 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T00:30:20.648 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T00:30:20.648 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T00:30:20.648 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:30:20.663 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056010,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1439 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248716k,nr_inodes=812179,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T00:30:20.663 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T00:30:20.664 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-09-21T00:30:20.664 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:30:20.705 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056013,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497440k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16835 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248720k,nr_inodes=812180,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497440k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T00:30:20.705 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T00:30:20.705 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T00:30:20.705 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:30:20.721 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056010,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1439 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248716k,nr_inodes=812179,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T00:30:20.721 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T00:30:20.722 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-09-21T00:30:20.722 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:30:20.761 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056013,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497440k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16835 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248720k,nr_inodes=812180,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497440k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T00:30:20.761 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T00:30:20.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.872+0000 7f09b4e7e640 1 -- 172.21.15.112:0/1885690630 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f09b0071020 msgr2=0x7f09b006ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:20.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.872+0000 7f09b4e7e640 1 --2- 172.21.15.112:0/1885690630 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f09b0071020 0x7f09b006ddf0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f0998005fd0 tx=0x7f099802a520 comp rx=0 tx=0).stop 2024-09-21T00:30:20.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.872+0000 7f09b4e7e640 1 -- 172.21.15.112:0/1885690630 shutdown_connections 2024-09-21T00:30:20.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.872+0000 7f09b4e7e640 1 --2- 172.21.15.112:0/1885690630 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f09b0071020 0x7f09b006ddf0 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:20.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.872+0000 7f09b4e7e640 1 --2- 172.21.15.112:0/1885690630 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f09b01526d0 0x7f09b015c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:20.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.872+0000 7f09b4e7e640 1 -- 172.21.15.112:0/1885690630 >> 172.21.15.112:0/1885690630 conn(0x7f09b00738b0 msgr2=0x7f09b0073cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:20.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.872+0000 7f09b4e7e640 1 -- 172.21.15.112:0/1885690630 shutdown_connections 2024-09-21T00:30:20.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.872+0000 7f09b4e7e640 1 -- 172.21.15.112:0/1885690630 wait complete. 2024-09-21T00:30:20.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.874+0000 7f09b4e7e640 1 Processor -- start 2024-09-21T00:30:20.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.874+0000 7f09b4e7e640 1 -- start start 2024-09-21T00:30:20.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.875+0000 7f09b4e7e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f09b006e330 0x7f09b013a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:20.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.875+0000 7f09b4e7e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f09b0071020 0x7f09b013d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:20.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.875+0000 7f09b4e7e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f09b01526d0 0x7f09b0150760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:20.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.875+0000 7f09b4e7e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f09b015f920 con 0x7f09b006e330 2024-09-21T00:30:20.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.875+0000 7f09b4e7e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f09b00d0590 con 0x7f09b01526d0 2024-09-21T00:30:20.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.875+0000 7f09b4e7e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f09b015faa0 con 0x7f09b0071020 2024-09-21T00:30:20.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.875+0000 7f09ae59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f09b006e330 0x7f09b013a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:20.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.875+0000 7f09ae59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f09b006e330 0x7f09b013a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49578/0 (socket says 172.21.15.112:49578) 2024-09-21T00:30:20.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.875+0000 7f09ae59c640 1 -- 172.21.15.112:0/3538889019 learned_addr learned my addr 172.21.15.112:0/3538889019 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:30:20.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.876+0000 7f09ae59c640 1 -- 172.21.15.112:0/3538889019 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f09b0071020 msgr2=0x7f09b013d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:20.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.876+0000 7f09ae59c640 1 --2- 172.21.15.112:0/3538889019 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f09b0071020 0x7f09b013d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:20.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.876+0000 7f09ae59c640 1 -- 172.21.15.112:0/3538889019 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f09b01526d0 msgr2=0x7f09b0150760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:20.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.876+0000 7f09ae59c640 1 --2- 172.21.15.112:0/3538889019 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f09b01526d0 0x7f09b0150760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:20.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.876+0000 7f09ae59c640 1 -- 172.21.15.112:0/3538889019 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09b013d650 con 0x7f09b006e330 2024-09-21T00:30:20.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.876+0000 7f09ae59c640 1 --2- 172.21.15.112:0/3538889019 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f09b006e330 0x7f09b013a050 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f09a000cce0 tx=0x7f09a00075f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:20.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.876+0000 7f09a6ffd640 1 -- 172.21.15.112:0/3538889019 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09a0004b20 con 0x7f09b006e330 2024-09-21T00:30:20.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.876+0000 7f09a6ffd640 1 -- 172.21.15.112:0/3538889019 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f09a0004db0 con 0x7f09b006e330 2024-09-21T00:30:20.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.876+0000 7f09a6ffd640 1 -- 172.21.15.112:0/3538889019 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09a0005280 con 0x7f09b006e330 2024-09-21T00:30:20.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.876+0000 7f09b4e7e640 1 -- 172.21.15.112:0/3538889019 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09b0153100 con 0x7f09b006e330 2024-09-21T00:30:20.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.876+0000 7f09b4e7e640 1 -- 172.21.15.112:0/3538889019 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09b0154e10 con 0x7f09b006e330 2024-09-21T00:30:20.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.877+0000 7f09b4e7e640 1 -- 172.21.15.112:0/3538889019 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09b00757c0 con 0x7f09b006e330 2024-09-21T00:30:20.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.878+0000 7f09a6ffd640 1 -- 172.21.15.112:0/3538889019 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f09a0005420 con 0x7f09b006e330 2024-09-21T00:30:20.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.878+0000 7f09a6ffd640 1 --2- 172.21.15.112:0/3538889019 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f0990078b90 0x7f099007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:20.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.878+0000 7f09a6ffd640 1 -- 172.21.15.112:0/3538889019 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 6187+0+0 (secure 0 0 0) 0x7f09a009b820 con 0x7f09b006e330 2024-09-21T00:30:20.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.879+0000 7f09add9b640 1 --2- 172.21.15.112:0/3538889019 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f0990078b90 0x7f099007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:20.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.879+0000 7f09add9b640 1 --2- 172.21.15.112:0/3538889019 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f0990078b90 0x7f099007b0a0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f0998033000 tx=0x7f09980046c0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:20.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:20.883+0000 7f09a6ffd640 1 -- 172.21.15.112:0/3538889019 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f09a0014030 con 0x7f09b006e330 2024-09-21T00:30:21.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.097+0000 7f09b4e7e640 1 -- 172.21.15.112:0/3538889019 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f09b0160000 con 0x7f09b006e330 2024-09-21T00:30:21.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.097+0000 7f09a6ffd640 1 -- 172.21.15.112:0/3538889019 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) ==== 94+0+5256 (secure 0 0 0) 0x7f09a0063a90 con 0x7f09b006e330 2024-09-21T00:30:21.098 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:30:21.098 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":80,"btime":"2024-09-21T00:30:17:668412+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6113,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/3217596137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":3217596137},{"type":"v1","addr":"172.21.15.187:6835","nonce":3217596137}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":73},{"gid":6649,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6837/776068904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":776068904},{"type":"v1","addr":"172.21.15.187:6837","nonce":776068904}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":73}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T00:29:48.456672+0000","modified":"2024-09-21T00:30:17.404923+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":112,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7422,"mds_1":7419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7419":{"gid":7419,"name":"a","rank":1,"incarnation":78,"state":"up:active","state_seq":4,"addr":"172.21.15.112:6834/3291181711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":3291181711},{"type":"v1","addr":"172.21.15.112:6834","nonce":3291181711}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7422":{"gid":7422,"name":"c","rank":0,"incarnation":73,"state":"up:active","state_seq":4,"addr":"172.21.15.112:6835/3392867613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":3392867613},{"type":"v1","addr":"172.21.15.112:6835","nonce":3392867613}]},"join_fscid":-1,"export_targets":[1],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7422,"qdb_cluster":[7422,7419]},"id":8}]} 2024-09-21T00:30:21.099 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 80 2024-09-21T00:30:21.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.100+0000 7f09b4e7e640 1 -- 172.21.15.112:0/3538889019 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f0990078b90 msgr2=0x7f099007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:21.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.100+0000 7f09b4e7e640 1 --2- 172.21.15.112:0/3538889019 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f0990078b90 0x7f099007b0a0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f0998033000 tx=0x7f09980046c0 comp rx=0 tx=0).stop 2024-09-21T00:30:21.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.100+0000 7f09b4e7e640 1 -- 172.21.15.112:0/3538889019 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f09b006e330 msgr2=0x7f09b013a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:21.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.100+0000 7f09b4e7e640 1 --2- 172.21.15.112:0/3538889019 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f09b006e330 0x7f09b013a050 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f09a000cce0 tx=0x7f09a00075f0 comp rx=0 tx=0).stop 2024-09-21T00:30:21.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.100+0000 7f09b4e7e640 1 -- 172.21.15.112:0/3538889019 shutdown_connections 2024-09-21T00:30:21.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.100+0000 7f09b4e7e640 1 --2- 172.21.15.112:0/3538889019 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f09b01526d0 0x7f09b0150760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:21.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.100+0000 7f09b4e7e640 1 --2- 172.21.15.112:0/3538889019 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f09b0071020 0x7f09b013d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:21.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.100+0000 7f09b4e7e640 1 --2- 172.21.15.112:0/3538889019 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f0990078b90 0x7f099007b0a0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:21.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.100+0000 7f09b4e7e640 1 --2- 172.21.15.112:0/3538889019 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f09b006e330 0x7f09b013a050 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:21.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.100+0000 7f09b4e7e640 1 -- 172.21.15.112:0/3538889019 >> 172.21.15.112:0/3538889019 conn(0x7f09b00738b0 msgr2=0x7f09b00b1b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:21.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.100+0000 7f09b4e7e640 1 -- 172.21.15.112:0/3538889019 shutdown_connections 2024-09-21T00:30:21.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.100+0000 7f09b4e7e640 1 -- 172.21.15.112:0/3538889019 wait complete. 2024-09-21T00:30:21.113 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T00:30:21.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.224+0000 7f496878b640 1 -- 172.21.15.112:0/2400456700 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f494c003a50 msgr2=0x7f494c003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:21.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.224+0000 7f496878b640 1 --2- 172.21.15.112:0/2400456700 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f494c003a50 0x7f494c003ec0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f495c001710 tx=0x7f494c005830 comp rx=0 tx=0).stop 2024-09-21T00:30:21.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.224+0000 7f496878b640 1 -- 172.21.15.112:0/2400456700 shutdown_connections 2024-09-21T00:30:21.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.224+0000 7f496878b640 1 --2- 172.21.15.112:0/2400456700 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f494c003a50 0x7f494c003ec0 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:21.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.225+0000 7f496878b640 1 -- 172.21.15.112:0/2400456700 >> 172.21.15.112:0/2400456700 conn(0x7f49600ceed0 msgr2=0x7f49600cf2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:21.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.225+0000 7f496878b640 1 -- 172.21.15.112:0/2400456700 shutdown_connections 2024-09-21T00:30:21.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.225+0000 7f496878b640 1 -- 172.21.15.112:0/2400456700 wait complete. 2024-09-21T00:30:21.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.226+0000 7f496878b640 1 Processor -- start 2024-09-21T00:30:21.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.226+0000 7f496878b640 1 -- start start 2024-09-21T00:30:21.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.228+0000 7f496878b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f494c003a50 0x7f49601528f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:21.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.228+0000 7f496878b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49600c9530 0x7f496009cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:21.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.228+0000 7f496878b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f496009e130 0x7f496009d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:21.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.228+0000 7f496878b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f49600b61d0 con 0x7f49600c9530 2024-09-21T00:30:21.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.228+0000 7f496878b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f49600c5ae0 con 0x7f494c003a50 2024-09-21T00:30:21.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.228+0000 7f496878b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f49600b6350 con 0x7f496009e130 2024-09-21T00:30:21.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.228+0000 7f4965d26640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49600c9530 0x7f496009cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:21.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.228+0000 7f4965d26640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49600c9530 0x7f496009cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49594/0 (socket says 172.21.15.112:49594) 2024-09-21T00:30:21.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.228+0000 7f4965d26640 1 -- 172.21.15.112:0/1840740544 learned_addr learned my addr 172.21.15.112:0/1840740544 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:30:21.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.228+0000 7f4965d26640 1 -- 172.21.15.112:0/1840740544 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f496009e130 msgr2=0x7f496009d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:21.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.228+0000 7f4966527640 1 --2- 172.21.15.112:0/1840740544 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f494c003a50 0x7f49601528f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:21.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.228+0000 7f4965d26640 1 --2- 172.21.15.112:0/1840740544 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f496009e130 0x7f496009d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:21.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.228+0000 7f4965d26640 1 -- 172.21.15.112:0/1840740544 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f494c003a50 msgr2=0x7f49601528f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:21.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.228+0000 7f4965d26640 1 --2- 172.21.15.112:0/1840740544 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f494c003a50 0x7f49601528f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:21.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.228+0000 7f4965d26640 1 -- 172.21.15.112:0/1840740544 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f496009d9a0 con 0x7f49600c9530 2024-09-21T00:30:21.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.228+0000 7f4965d26640 1 --2- 172.21.15.112:0/1840740544 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49600c9530 0x7f496009cf20 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f495000e540 tx=0x7f495000ea00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:21.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.228+0000 7f4956ffd640 1 -- 172.21.15.112:0/1840740544 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4950015070 con 0x7f49600c9530 2024-09-21T00:30:21.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.228+0000 7f4956ffd640 1 -- 172.21.15.112:0/1840740544 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4950010980 con 0x7f49600c9530 2024-09-21T00:30:21.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.228+0000 7f496878b640 1 -- 172.21.15.112:0/1840740544 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f496009e860 con 0x7f49600c9530 2024-09-21T00:30:21.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.228+0000 7f4956ffd640 1 -- 172.21.15.112:0/1840740544 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4950010db0 con 0x7f49600c9530 2024-09-21T00:30:21.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.228+0000 7f496878b640 1 -- 172.21.15.112:0/1840740544 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f496009c720 con 0x7f49600c9530 2024-09-21T00:30:21.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.229+0000 7f496878b640 1 -- 172.21.15.112:0/1840740544 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f496009e5d0 con 0x7f49600c9530 2024-09-21T00:30:21.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.230+0000 7f4956ffd640 1 -- 172.21.15.112:0/1840740544 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f4950018070 con 0x7f49600c9530 2024-09-21T00:30:21.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.230+0000 7f4956ffd640 1 --2- 172.21.15.112:0/1840740544 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4944078b90 0x7f494407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:21.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.231+0000 7f4956ffd640 1 -- 172.21.15.112:0/1840740544 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 6187+0+0 (secure 0 0 0) 0x7f495009b240 con 0x7f49600c9530 2024-09-21T00:30:21.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.231+0000 7f4966527640 1 --2- 172.21.15.112:0/1840740544 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4944078b90 0x7f494407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:21.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.231+0000 7f4966527640 1 --2- 172.21.15.112:0/1840740544 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4944078b90 0x7f494407b0a0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f495c0016e0 tx=0x7f495c0011e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:21.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.235+0000 7f4956ffd640 1 -- 172.21.15.112:0/1840740544 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f4950063530 con 0x7f49600c9530 2024-09-21T00:30:21.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.449+0000 7f496878b640 1 -- 172.21.15.112:0/1840740544 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f49600d0430 con 0x7f49600c9530 2024-09-21T00:30:21.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.449+0000 7f4956ffd640 1 -- 172.21.15.112:0/1840740544 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) ==== 94+0+5256 (secure 0 0 0) 0x7f4950062e30 con 0x7f49600c9530 2024-09-21T00:30:21.450 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:30:21.450 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":80,"btime":"2024-09-21T00:30:17:668412+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6113,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/3217596137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":3217596137},{"type":"v1","addr":"172.21.15.187:6835","nonce":3217596137}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":73},{"gid":6649,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6837/776068904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":776068904},{"type":"v1","addr":"172.21.15.187:6837","nonce":776068904}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":73}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T00:29:48.456672+0000","modified":"2024-09-21T00:30:17.404923+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":112,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7422,"mds_1":7419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7419":{"gid":7419,"name":"a","rank":1,"incarnation":78,"state":"up:active","state_seq":4,"addr":"172.21.15.112:6834/3291181711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":3291181711},{"type":"v1","addr":"172.21.15.112:6834","nonce":3291181711}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7422":{"gid":7422,"name":"c","rank":0,"incarnation":73,"state":"up:active","state_seq":4,"addr":"172.21.15.112:6835/3392867613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":3392867613},{"type":"v1","addr":"172.21.15.112:6835","nonce":3392867613}]},"join_fscid":-1,"export_targets":[1],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7422,"qdb_cluster":[7422,7419]},"id":8}]} 2024-09-21T00:30:21.450 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 80 2024-09-21T00:30:21.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.451+0000 7f496878b640 1 -- 172.21.15.112:0/1840740544 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4944078b90 msgr2=0x7f494407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:21.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.451+0000 7f496878b640 1 --2- 172.21.15.112:0/1840740544 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4944078b90 0x7f494407b0a0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f495c0016e0 tx=0x7f495c0011e0 comp rx=0 tx=0).stop 2024-09-21T00:30:21.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.451+0000 7f496878b640 1 -- 172.21.15.112:0/1840740544 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49600c9530 msgr2=0x7f496009cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:21.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.451+0000 7f496878b640 1 --2- 172.21.15.112:0/1840740544 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49600c9530 0x7f496009cf20 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f495000e540 tx=0x7f495000ea00 comp rx=0 tx=0).stop 2024-09-21T00:30:21.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.451+0000 7f496878b640 1 -- 172.21.15.112:0/1840740544 shutdown_connections 2024-09-21T00:30:21.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.451+0000 7f496878b640 1 --2- 172.21.15.112:0/1840740544 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f496009e130 0x7f496009d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:21.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.451+0000 7f496878b640 1 --2- 172.21.15.112:0/1840740544 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4944078b90 0x7f494407b0a0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:21.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.451+0000 7f496878b640 1 --2- 172.21.15.112:0/1840740544 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49600c9530 0x7f496009cf20 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:21.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.451+0000 7f496878b640 1 --2- 172.21.15.112:0/1840740544 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f494c003a50 0x7f49601528f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:21.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.451+0000 7f496878b640 1 -- 172.21.15.112:0/1840740544 >> 172.21.15.112:0/1840740544 conn(0x7f49600ceed0 msgr2=0x7f49600712e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:21.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.451+0000 7f496878b640 1 -- 172.21.15.112:0/1840740544 shutdown_connections 2024-09-21T00:30:21.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.451+0000 7f496878b640 1 -- 172.21.15.112:0/1840740544 wait complete. 2024-09-21T00:30:21.463 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T00:30:21.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.576+0000 7f2a91cf9640 1 -- 172.21.15.112:0/47852619 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2a74003c10 msgr2=0x7f2a74004060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:21.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.576+0000 7f2a91cf9640 1 --2- 172.21.15.112:0/47852619 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2a74003c10 0x7f2a74004060 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c006fa0 tx=0x7f2a74005830 comp rx=0 tx=0).stop 2024-09-21T00:30:21.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.576+0000 7f2a8bfff640 1 -- 172.21.15.112:0/47852619 reap_dead start 2024-09-21T00:30:21.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.576+0000 7f2a91cf9640 1 -- 172.21.15.112:0/47852619 shutdown_connections 2024-09-21T00:30:21.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.576+0000 7f2a91cf9640 1 -- 172.21.15.112:0/47852619 >> 172.21.15.112:0/47852619 conn(0x7f2a8c0cef30 msgr2=0x7f2a8c0cf340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:21.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.576+0000 7f2a91cf9640 1 -- 172.21.15.112:0/47852619 shutdown_connections 2024-09-21T00:30:21.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.576+0000 7f2a91cf9640 1 -- 172.21.15.112:0/47852619 wait complete. 2024-09-21T00:30:21.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.578+0000 7f2a91cf9640 1 Processor -- start 2024-09-21T00:30:21.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.578+0000 7f2a91cf9640 1 -- start start 2024-09-21T00:30:21.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.579+0000 7f2a91cf9640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2a74003c10 0x7f2a8c0cbb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:21.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.579+0000 7f2a91cf9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2a8c09f670 0x7f2a8c15e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:21.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.579+0000 7f2a91cf9640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2a8c1f6bb0 0x7f2a8c1f6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:21.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.579+0000 7f2a91cf9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2a8c0b3de0 con 0x7f2a8c09f670 2024-09-21T00:30:21.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.579+0000 7f2a91cf9640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f2a8c0b3c60 con 0x7f2a74003c10 2024-09-21T00:30:21.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.579+0000 7f2a91cf9640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f2a8c0cfc60 con 0x7f2a8c1f6bb0 2024-09-21T00:30:21.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.579+0000 7f2a8affd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2a8c09f670 0x7f2a8c15e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:21.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.579+0000 7f2a8affd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2a8c09f670 0x7f2a8c15e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49618/0 (socket says 172.21.15.112:49618) 2024-09-21T00:30:21.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.579+0000 7f2a8affd640 1 -- 172.21.15.112:0/3810713996 learned_addr learned my addr 172.21.15.112:0/3810713996 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:30:21.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.580+0000 7f2a8b7fe640 1 --2- 172.21.15.112:0/3810713996 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2a74003c10 0x7f2a8c0cbb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:21.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.580+0000 7f2a8bfff640 1 --2- 172.21.15.112:0/3810713996 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2a8c1f6bb0 0x7f2a8c1f6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:21.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.580+0000 7f2a8affd640 1 -- 172.21.15.112:0/3810713996 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2a8c1f6bb0 msgr2=0x7f2a8c1f6fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:21.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.580+0000 7f2a8affd640 1 --2- 172.21.15.112:0/3810713996 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2a8c1f6bb0 0x7f2a8c1f6fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:21.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.580+0000 7f2a8affd640 1 -- 172.21.15.112:0/3810713996 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2a74003c10 msgr2=0x7f2a8c0cbb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:21.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.580+0000 7f2a8affd640 1 --2- 172.21.15.112:0/3810713996 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2a74003c10 0x7f2a8c0cbb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:21.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.580+0000 7f2a8affd640 1 -- 172.21.15.112:0/3810713996 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a8c1f7500 con 0x7f2a8c09f670 2024-09-21T00:30:21.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.580+0000 7f2a8affd640 1 --2- 172.21.15.112:0/3810713996 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2a8c09f670 0x7f2a8c15e110 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f2a80011820 tx=0x7f2a80011ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:21.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.580+0000 7f2a73fff640 1 -- 172.21.15.112:0/3810713996 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a80004460 con 0x7f2a8c09f670 2024-09-21T00:30:21.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.580+0000 7f2a73fff640 1 -- 172.21.15.112:0/3810713996 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2a80016070 con 0x7f2a8c09f670 2024-09-21T00:30:21.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.580+0000 7f2a73fff640 1 -- 172.21.15.112:0/3810713996 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a8000fb80 con 0x7f2a8c09f670 2024-09-21T00:30:21.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.580+0000 7f2a91cf9640 1 -- 172.21.15.112:0/3810713996 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a8c0caa70 con 0x7f2a8c09f670 2024-09-21T00:30:21.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.580+0000 7f2a91cf9640 1 -- 172.21.15.112:0/3810713996 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a8c0cb030 con 0x7f2a8c09f670 2024-09-21T00:30:21.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.581+0000 7f2a91cf9640 1 -- 172.21.15.112:0/3810713996 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a8c0ca7e0 con 0x7f2a8c09f670 2024-09-21T00:30:21.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.582+0000 7f2a73fff640 1 -- 172.21.15.112:0/3810713996 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2a80018030 con 0x7f2a8c09f670 2024-09-21T00:30:21.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.582+0000 7f2a73fff640 1 --2- 172.21.15.112:0/3810713996 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2a78078b90 0x7f2a7807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:21.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.582+0000 7f2a73fff640 1 -- 172.21.15.112:0/3810713996 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 6187+0+0 (secure 0 0 0) 0x7f2a8009b760 con 0x7f2a8c09f670 2024-09-21T00:30:21.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.582+0000 7f2a8b7fe640 1 --2- 172.21.15.112:0/3810713996 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2a78078b90 0x7f2a7807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:21.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.583+0000 7f2a8b7fe640 1 --2- 172.21.15.112:0/3810713996 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2a78078b90 0x7f2a7807b0a0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c033000 tx=0x7f2a7c02d000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:21.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.587+0000 7f2a73fff640 1 -- 172.21.15.112:0/3810713996 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2a8c1f7500 con 0x7f2a8c09f670 2024-09-21T00:30:21.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.766+0000 7f2a91cf9640 1 -- 172.21.15.112:0/3810713996 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2a8c0b5880 con 0x7f2a8c09f670 2024-09-21T00:30:21.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.767+0000 7f2a73fff640 1 -- 172.21.15.112:0/3810713996 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v112) ==== 74+0+17167 (secure 0 0 0) 0x7f2a800639c0 con 0x7f2a8c09f670 2024-09-21T00:30:21.768 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:30:21.768 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":112,"fsid":"8b34692b-dc65-49b2-a08f-26f707b96ac5","created":"2024-09-21T00:25:22.788490+0000","modified":"2024-09-21T00:30:02.243534+0000","last_up_change":"2024-09-21T00:25:36.161835+0000","last_in_change":"2024-09-21T00:25:26.346500+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T00:25:37.105006+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-09-21T00:29:46.103570+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-09-21T00:29:47.420294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9c575086-67d2-4de2-b01b-d44dcc21d604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6809","nonce":755760621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6811","nonce":755760621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6815","nonce":755760621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6813","nonce":755760621}]},"public_addr":"172.21.15.112:6809/755760621","cluster_addr":"172.21.15.112:6811/755760621","heartbeat_back_addr":"172.21.15.112:6815/755760621","heartbeat_front_addr":"172.21.15.112:6813/755760621","state":["exists","up"]},{"osd":1,"uuid":"a983c227-a3ce-4bdd-82f1-438ba7e3ecd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6801","nonce":344932592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6803","nonce":344932592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6807","nonce":344932592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6805","nonce":344932592}]},"public_addr":"172.21.15.112:6801/344932592","cluster_addr":"172.21.15.112:6803/344932592","heartbeat_back_addr":"172.21.15.112:6807/344932592","heartbeat_front_addr":"172.21.15.112:6805/344932592","state":["exists","up"]},{"osd":2,"uuid":"1193c1d8-bfff-4a57-8200-5f69bf1ea3af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6825","nonce":407933616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6827","nonce":407933616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6831","nonce":407933616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6829","nonce":407933616}]},"public_addr":"172.21.15.112:6825/407933616","cluster_addr":"172.21.15.112:6827/407933616","heartbeat_back_addr":"172.21.15.112:6831/407933616","heartbeat_front_addr":"172.21.15.112:6829/407933616","state":["exists","up"]},{"osd":3,"uuid":"af04330a-36ad-40cc-a8dd-fa259b221de5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6817","nonce":4120164806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6819","nonce":4120164806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6823","nonce":4120164806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6821","nonce":4120164806}]},"public_addr":"172.21.15.112:6817/4120164806","cluster_addr":"172.21.15.112:6819/4120164806","heartbeat_back_addr":"172.21.15.112:6823/4120164806","heartbeat_front_addr":"172.21.15.112:6821/4120164806","state":["exists","up"]},{"osd":4,"uuid":"5b893111-292c-4c5a-b184-53ad38c7199e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6803","nonce":522077560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6805","nonce":522077560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6809","nonce":522077560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6807","nonce":522077560}]},"public_addr":"172.21.15.187:6803/522077560","cluster_addr":"172.21.15.187:6805/522077560","heartbeat_back_addr":"172.21.15.187:6809/522077560","heartbeat_front_addr":"172.21.15.187:6807/522077560","state":["exists","up"]},{"osd":5,"uuid":"c0b8e011-45fe-4c5e-925e-6be2a7f1868a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6827","nonce":3593144721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6829","nonce":3593144721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6833","nonce":3593144721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6831","nonce":3593144721}]},"public_addr":"172.21.15.187:6827/3593144721","cluster_addr":"172.21.15.187:6829/3593144721","heartbeat_back_addr":"172.21.15.187:6833/3593144721","heartbeat_front_addr":"172.21.15.187:6831/3593144721","state":["exists","up"]},{"osd":6,"uuid":"87d2d650-7489-4593-98a3-a1f433ce8dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6811","nonce":1494387657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6813","nonce":1494387657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6817","nonce":1494387657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6815","nonce":1494387657}]},"public_addr":"172.21.15.187:6811/1494387657","cluster_addr":"172.21.15.187:6813/1494387657","heartbeat_back_addr":"172.21.15.187:6817/1494387657","heartbeat_front_addr":"172.21.15.187:6815/1494387657","state":["exists","up"]},{"osd":7,"uuid":"c6b4f8e2-a1ee-4438-aa37-67ff4b6ef1f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6819","nonce":1624699842}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6821","nonce":1624699842}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6825","nonce":1624699842}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6823","nonce":1624699842}]},"public_addr":"172.21.15.187:6819/1624699842","cluster_addr":"172.21.15.187:6821/1624699842","heartbeat_back_addr":"172.21.15.187:6825/1624699842","heartbeat_front_addr":"172.21.15.187:6823/1624699842","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.873162+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.400506+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.664317+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.159791+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.202630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.113212+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.384786+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.448441+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.187:6837/2071643267":"2024-09-22T00:30:02.243502+0000","172.21.15.187:6835/2071643267":"2024-09-22T00:30:02.243502+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T00:30:21.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.769+0000 7f2a91cf9640 1 -- 172.21.15.112:0/3810713996 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2a78078b90 msgr2=0x7f2a7807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:21.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.769+0000 7f2a91cf9640 1 --2- 172.21.15.112:0/3810713996 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2a78078b90 0x7f2a7807b0a0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c033000 tx=0x7f2a7c02d000 comp rx=0 tx=0).stop 2024-09-21T00:30:21.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.769+0000 7f2a91cf9640 1 -- 172.21.15.112:0/3810713996 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2a8c09f670 msgr2=0x7f2a8c15e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:21.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.769+0000 7f2a91cf9640 1 --2- 172.21.15.112:0/3810713996 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2a8c09f670 0x7f2a8c15e110 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f2a80011820 tx=0x7f2a80011ce0 comp rx=0 tx=0).stop 2024-09-21T00:30:21.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.769+0000 7f2a91cf9640 1 -- 172.21.15.112:0/3810713996 shutdown_connections 2024-09-21T00:30:21.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.769+0000 7f2a91cf9640 1 --2- 172.21.15.112:0/3810713996 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2a8c1f6bb0 0x7f2a8c1f6fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:21.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.769+0000 7f2a91cf9640 1 --2- 172.21.15.112:0/3810713996 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2a78078b90 0x7f2a7807b0a0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:21.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.769+0000 7f2a91cf9640 1 --2- 172.21.15.112:0/3810713996 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2a8c09f670 0x7f2a8c15e110 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:21.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.769+0000 7f2a91cf9640 1 --2- 172.21.15.112:0/3810713996 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2a74003c10 0x7f2a8c0cbb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:21.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.769+0000 7f2a91cf9640 1 -- 172.21.15.112:0/3810713996 >> 172.21.15.112:0/3810713996 conn(0x7f2a8c0cef30 msgr2=0x7f2a8c072790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:21.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.769+0000 7f2a91cf9640 1 -- 172.21.15.112:0/3810713996 shutdown_connections 2024-09-21T00:30:21.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.769+0000 7f2a91cf9640 1 -- 172.21.15.112:0/3810713996 wait complete. 2024-09-21T00:30:21.781 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-21T00:30:21.781 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T00:30:21.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.892+0000 7f59152cc640 1 -- 172.21.15.112:0/698158514 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59100cfd80 msgr2=0x7f591015ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:21.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.892+0000 7f59152cc640 1 --2- 172.21.15.112:0/698158514 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59100cfd80 0x7f591015ebc0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f58f800a970 tx=0x7f58f802b7f0 comp rx=0 tx=0).stop 2024-09-21T00:30:21.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.892+0000 7f59152cc640 1 -- 172.21.15.112:0/698158514 shutdown_connections 2024-09-21T00:30:21.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.892+0000 7f59152cc640 1 --2- 172.21.15.112:0/698158514 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59100cfd80 0x7f591015ebc0 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:21.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.892+0000 7f59152cc640 1 -- 172.21.15.112:0/698158514 >> 172.21.15.112:0/698158514 conn(0x7f591006d4c0 msgr2=0x7f59100d02c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:21.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.892+0000 7f59152cc640 1 -- 172.21.15.112:0/698158514 shutdown_connections 2024-09-21T00:30:21.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.892+0000 7f59152cc640 1 -- 172.21.15.112:0/698158514 wait complete. 2024-09-21T00:30:21.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.894+0000 7f59152cc640 1 Processor -- start 2024-09-21T00:30:21.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.894+0000 7f59152cc640 1 -- start start 2024-09-21T00:30:21.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.895+0000 7f59152cc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59100cf970 0x7f59101f3390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:21.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.895+0000 7f59152cc640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f59100cfd80 0x7f59100ccb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:21.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.895+0000 7f59152cc640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f591015f100 0x7f59100cd040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:21.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.895+0000 7f59152cc640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f59101520c0 con 0x7f59100cf970 2024-09-21T00:30:21.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.895+0000 7f59152cc640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f59100cf4e0 con 0x7f591015f100 2024-09-21T00:30:21.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.895+0000 7f59152cc640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f5910152240 con 0x7f59100cfd80 2024-09-21T00:30:21.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.895+0000 7f590effd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59100cf970 0x7f59101f3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:21.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.895+0000 7f590effd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59100cf970 0x7f59101f3390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49636/0 (socket says 172.21.15.112:49636) 2024-09-21T00:30:21.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.895+0000 7f590effd640 1 -- 172.21.15.112:0/292377776 learned_addr learned my addr 172.21.15.112:0/292377776 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:30:21.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.895+0000 7f59067fc640 1 --2- 172.21.15.112:0/292377776 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f59100cfd80 0x7f59100ccb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:21.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.895+0000 7f590f7fe640 1 --2- 172.21.15.112:0/292377776 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f591015f100 0x7f59100cd040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:21.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.895+0000 7f590effd640 1 -- 172.21.15.112:0/292377776 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f59100cfd80 msgr2=0x7f59100ccb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:21.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.895+0000 7f590effd640 1 --2- 172.21.15.112:0/292377776 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f59100cfd80 0x7f59100ccb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:21.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.895+0000 7f590effd640 1 -- 172.21.15.112:0/292377776 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f591015f100 msgr2=0x7f59100cd040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:21.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.896+0000 7f590effd640 1 --2- 172.21.15.112:0/292377776 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f591015f100 0x7f59100cd040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:21.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.896+0000 7f590effd640 1 -- 172.21.15.112:0/292377776 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59101581f0 con 0x7f59100cf970 2024-09-21T00:30:21.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.896+0000 7f590effd640 1 --2- 172.21.15.112:0/292377776 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59100cf970 0x7f59101f3390 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f590000b420 tx=0x7f590000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:21.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.896+0000 7f5907fff640 1 -- 172.21.15.112:0/292377776 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5900011830 con 0x7f59100cf970 2024-09-21T00:30:21.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.896+0000 7f5907fff640 1 -- 172.21.15.112:0/292377776 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5900016070 con 0x7f59100cf970 2024-09-21T00:30:21.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.896+0000 7f59152cc640 1 -- 172.21.15.112:0/292377776 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59100ad950 con 0x7f59100cf970 2024-09-21T00:30:21.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.896+0000 7f5907fff640 1 -- 172.21.15.112:0/292377776 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5900011d60 con 0x7f59100cf970 2024-09-21T00:30:21.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.896+0000 7f59152cc640 1 -- 172.21.15.112:0/292377776 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5910157f10 con 0x7f59100cf970 2024-09-21T00:30:21.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.897+0000 7f59152cc640 1 -- 172.21.15.112:0/292377776 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f591015ebc0 con 0x7f59100cf970 2024-09-21T00:30:21.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.898+0000 7f5907fff640 1 -- 172.21.15.112:0/292377776 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f590001b460 con 0x7f59100cf970 2024-09-21T00:30:21.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.898+0000 7f5907fff640 1 --2- 172.21.15.112:0/292377776 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f58f0078b90 0x7f58f007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:21.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.898+0000 7f5907fff640 1 -- 172.21.15.112:0/292377776 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 6187+0+0 (secure 0 0 0) 0x7f590009a990 con 0x7f59100cf970 2024-09-21T00:30:21.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.898+0000 7f59067fc640 1 --2- 172.21.15.112:0/292377776 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f58f0078b90 0x7f58f007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:21.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.899+0000 7f59067fc640 1 --2- 172.21.15.112:0/292377776 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f58f0078b90 0x7f58f007b0a0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f59100b3bd0 tx=0x7f58f8004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:21.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:21.903+0000 7f5907fff640 1 -- 172.21.15.112:0/292377776 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5900020020 con 0x7f59100cf970 2024-09-21T00:30:22.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.114+0000 7f59152cc640 1 -- 172.21.15.112:0/292377776 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f59100b3470 con 0x7f59100cf970 2024-09-21T00:30:22.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.115+0000 7f5907fff640 1 -- 172.21.15.112:0/292377776 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) ==== 94+0+5256 (secure 0 0 0) 0x7f5900062c80 con 0x7f59100cf970 2024-09-21T00:30:22.116 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:30:22.116 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":80,"btime":"2024-09-21T00:30:17:668412+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6113,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/3217596137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":3217596137},{"type":"v1","addr":"172.21.15.187:6835","nonce":3217596137}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":73},{"gid":6649,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6837/776068904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":776068904},{"type":"v1","addr":"172.21.15.187:6837","nonce":776068904}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":73}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T00:29:48.456672+0000","modified":"2024-09-21T00:30:17.404923+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":112,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7422,"mds_1":7419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7419":{"gid":7419,"name":"a","rank":1,"incarnation":78,"state":"up:active","state_seq":4,"addr":"172.21.15.112:6834/3291181711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":3291181711},{"type":"v1","addr":"172.21.15.112:6834","nonce":3291181711}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7422":{"gid":7422,"name":"c","rank":0,"incarnation":73,"state":"up:active","state_seq":4,"addr":"172.21.15.112:6835/3392867613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":3392867613},{"type":"v1","addr":"172.21.15.112:6835","nonce":3392867613}]},"join_fscid":-1,"export_targets":[1],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7422,"qdb_cluster":[7422,7419]},"id":8}]} 2024-09-21T00:30:22.117 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 80 2024-09-21T00:30:22.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.117+0000 7f59152cc640 1 -- 172.21.15.112:0/292377776 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f58f0078b90 msgr2=0x7f58f007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:22.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.117+0000 7f59152cc640 1 --2- 172.21.15.112:0/292377776 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f58f0078b90 0x7f58f007b0a0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f59100b3bd0 tx=0x7f58f8004730 comp rx=0 tx=0).stop 2024-09-21T00:30:22.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.117+0000 7f59152cc640 1 -- 172.21.15.112:0/292377776 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59100cf970 msgr2=0x7f59101f3390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:22.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.117+0000 7f59152cc640 1 --2- 172.21.15.112:0/292377776 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59100cf970 0x7f59101f3390 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f590000b420 tx=0x7f590000b8e0 comp rx=0 tx=0).stop 2024-09-21T00:30:22.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.117+0000 7f59152cc640 1 -- 172.21.15.112:0/292377776 shutdown_connections 2024-09-21T00:30:22.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.117+0000 7f59152cc640 1 --2- 172.21.15.112:0/292377776 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f591015f100 0x7f59100cd040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:22.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.117+0000 7f59152cc640 1 --2- 172.21.15.112:0/292377776 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f59100cfd80 0x7f59100ccb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:22.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.117+0000 7f59152cc640 1 --2- 172.21.15.112:0/292377776 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f58f0078b90 0x7f58f007b0a0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:22.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.117+0000 7f59152cc640 1 --2- 172.21.15.112:0/292377776 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59100cf970 0x7f59101f3390 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:22.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.117+0000 7f59152cc640 1 -- 172.21.15.112:0/292377776 >> 172.21.15.112:0/292377776 conn(0x7f591006d4c0 msgr2=0x7f591015c730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:22.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.117+0000 7f59152cc640 1 -- 172.21.15.112:0/292377776 shutdown_connections 2024-09-21T00:30:22.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.117+0000 7f59152cc640 1 -- 172.21.15.112:0/292377776 wait complete. 2024-09-21T00:30:22.129 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T00:30:22.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.240+0000 7f679d1ad640 1 -- 172.21.15.112:0/1539475321 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f679809baa0 msgr2=0x7f67980b53f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:22.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.240+0000 7f679d1ad640 1 --2- 172.21.15.112:0/1539475321 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f679809baa0 0x7f67980b53f0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f67980c5d50 tx=0x7f678802a4d0 comp rx=0 tx=0).stop 2024-09-21T00:30:22.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.241+0000 7f679d1ad640 1 -- 172.21.15.112:0/1539475321 shutdown_connections 2024-09-21T00:30:22.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.241+0000 7f679d1ad640 1 --2- 172.21.15.112:0/1539475321 >> v2:172.21.15.187:3300/0 conn(0x7f67980b6280 0x7f67980cd440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:22.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.241+0000 7f679d1ad640 1 --2- 172.21.15.112:0/1539475321 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f67980b5930 0x7f67980b5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:22.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.241+0000 7f679d1ad640 1 --2- 172.21.15.112:0/1539475321 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f679809baa0 0x7f67980b53f0 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:22.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.241+0000 7f679d1ad640 1 -- 172.21.15.112:0/1539475321 >> 172.21.15.112:0/1539475321 conn(0x7f67980cef20 msgr2=0x7f67980cf330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:22.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.241+0000 7f679d1ad640 1 -- 172.21.15.112:0/1539475321 shutdown_connections 2024-09-21T00:30:22.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.241+0000 7f679d1ad640 1 -- 172.21.15.112:0/1539475321 wait complete. 2024-09-21T00:30:22.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.243+0000 7f679d1ad640 1 Processor -- start 2024-09-21T00:30:22.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.243+0000 7f679d1ad640 1 -- start start 2024-09-21T00:30:22.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.244+0000 7f679d1ad640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f679809baa0 0x7f67980c7020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:22.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.244+0000 7f679d1ad640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f67980b5930 0x7f67981e0800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:22.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.244+0000 7f679d1ad640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f67980b6280 0x7f67981f49b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:22.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.244+0000 7f679d1ad640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f67980b78f0 con 0x7f679809baa0 2024-09-21T00:30:22.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.244+0000 7f679d1ad640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f67980d0750 con 0x7f67980b6280 2024-09-21T00:30:22.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.244+0000 7f679d1ad640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f67980b7a70 con 0x7f67980b5930 2024-09-21T00:30:22.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.244+0000 7f6796d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f679809baa0 0x7f67980c7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:22.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.244+0000 7f6796d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f679809baa0 0x7f67980c7020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49644/0 (socket says 172.21.15.112:49644) 2024-09-21T00:30:22.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.244+0000 7f6796d9d640 1 -- 172.21.15.112:0/3955394000 learned_addr learned my addr 172.21.15.112:0/3955394000 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:30:22.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.244+0000 7f678e59c640 1 --2- 172.21.15.112:0/3955394000 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f67980b5930 0x7f67981e0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:22.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.244+0000 7f679759e640 1 --2- 172.21.15.112:0/3955394000 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f67980b6280 0x7f67981f49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:22.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.245+0000 7f6796d9d640 1 -- 172.21.15.112:0/3955394000 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f67980b5930 msgr2=0x7f67981e0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:22.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.245+0000 7f6796d9d640 1 --2- 172.21.15.112:0/3955394000 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f67980b5930 0x7f67981e0800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:22.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.245+0000 7f6796d9d640 1 -- 172.21.15.112:0/3955394000 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f67980b6280 msgr2=0x7f67981f49b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:22.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.245+0000 7f6796d9d640 1 --2- 172.21.15.112:0/3955394000 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f67980b6280 0x7f67981f49b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:22.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.245+0000 7f6796d9d640 1 -- 172.21.15.112:0/3955394000 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67980c77e0 con 0x7f679809baa0 2024-09-21T00:30:22.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.245+0000 7f6796d9d640 1 --2- 172.21.15.112:0/3955394000 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f679809baa0 0x7f67980c7020 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f67880099c0 tx=0x7f67880039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:22.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.245+0000 7f678ffff640 1 -- 172.21.15.112:0/3955394000 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f678803e030 con 0x7f679809baa0 2024-09-21T00:30:22.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.245+0000 7f678ffff640 1 -- 172.21.15.112:0/3955394000 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f67880047a0 con 0x7f679809baa0 2024-09-21T00:30:22.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.245+0000 7f679d1ad640 1 -- 172.21.15.112:0/3955394000 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67980c7ac0 con 0x7f679809baa0 2024-09-21T00:30:22.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.245+0000 7f678ffff640 1 -- 172.21.15.112:0/3955394000 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6788035040 con 0x7f679809baa0 2024-09-21T00:30:22.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.245+0000 7f679d1ad640 1 -- 172.21.15.112:0/3955394000 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67980c8080 con 0x7f679809baa0 2024-09-21T00:30:22.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.246+0000 7f679d1ad640 1 -- 172.21.15.112:0/3955394000 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67980b5d40 con 0x7f679809baa0 2024-09-21T00:30:22.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.247+0000 7f678ffff640 1 -- 172.21.15.112:0/3955394000 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6788041030 con 0x7f679809baa0 2024-09-21T00:30:22.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.247+0000 7f678ffff640 1 --2- 172.21.15.112:0/3955394000 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f6778078b90 0x7f677807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:22.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.247+0000 7f678ffff640 1 -- 172.21.15.112:0/3955394000 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 6187+0+0 (secure 0 0 0) 0x7f67880bbb40 con 0x7f679809baa0 2024-09-21T00:30:22.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.248+0000 7f678e59c640 1 --2- 172.21.15.112:0/3955394000 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f6778078b90 0x7f677807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:22.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.248+0000 7f678e59c640 1 --2- 172.21.15.112:0/3955394000 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f6778078b90 0x7f677807b0a0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f67981e05f0 tx=0x7f6780005ed0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:22.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.251+0000 7f678ffff640 1 -- 172.21.15.112:0/3955394000 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f67880837e0 con 0x7f679809baa0 2024-09-21T00:30:22.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.463+0000 7f679d1ad640 1 -- 172.21.15.112:0/3955394000 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f67980cacf0 con 0x7f679809baa0 2024-09-21T00:30:22.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.464+0000 7f678ffff640 1 -- 172.21.15.112:0/3955394000 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) ==== 94+0+5256 (secure 0 0 0) 0x7f6788088210 con 0x7f679809baa0 2024-09-21T00:30:22.464 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:30:22.464 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":80,"btime":"2024-09-21T00:30:17:668412+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6113,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/3217596137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":3217596137},{"type":"v1","addr":"172.21.15.187:6835","nonce":3217596137}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":73},{"gid":6649,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6837/776068904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":776068904},{"type":"v1","addr":"172.21.15.187:6837","nonce":776068904}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":73}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T00:29:48.456672+0000","modified":"2024-09-21T00:30:17.404923+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":112,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7422,"mds_1":7419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7419":{"gid":7419,"name":"a","rank":1,"incarnation":78,"state":"up:active","state_seq":4,"addr":"172.21.15.112:6834/3291181711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":3291181711},{"type":"v1","addr":"172.21.15.112:6834","nonce":3291181711}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7422":{"gid":7422,"name":"c","rank":0,"incarnation":73,"state":"up:active","state_seq":4,"addr":"172.21.15.112:6835/3392867613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6833","nonce":3392867613},{"type":"v1","addr":"172.21.15.112:6835","nonce":3392867613}]},"join_fscid":-1,"export_targets":[1],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7422,"qdb_cluster":[7422,7419]},"id":8}]} 2024-09-21T00:30:22.465 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 80 2024-09-21T00:30:22.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.466+0000 7f679d1ad640 1 -- 172.21.15.112:0/3955394000 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f6778078b90 msgr2=0x7f677807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:22.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.466+0000 7f679d1ad640 1 --2- 172.21.15.112:0/3955394000 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f6778078b90 0x7f677807b0a0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f67981e05f0 tx=0x7f6780005ed0 comp rx=0 tx=0).stop 2024-09-21T00:30:22.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.466+0000 7f679d1ad640 1 -- 172.21.15.112:0/3955394000 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f679809baa0 msgr2=0x7f67980c7020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:22.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.466+0000 7f679d1ad640 1 --2- 172.21.15.112:0/3955394000 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f679809baa0 0x7f67980c7020 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f67880099c0 tx=0x7f67880039e0 comp rx=0 tx=0).stop 2024-09-21T00:30:22.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.466+0000 7f679d1ad640 1 -- 172.21.15.112:0/3955394000 shutdown_connections 2024-09-21T00:30:22.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.466+0000 7f679d1ad640 1 --2- 172.21.15.112:0/3955394000 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f67980b6280 0x7f67981f49b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:22.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.466+0000 7f679d1ad640 1 --2- 172.21.15.112:0/3955394000 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f67980b5930 0x7f67981e0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:22.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.466+0000 7f679d1ad640 1 --2- 172.21.15.112:0/3955394000 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f6778078b90 0x7f677807b0a0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:22.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.466+0000 7f679d1ad640 1 --2- 172.21.15.112:0/3955394000 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f679809baa0 0x7f67980c7020 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:22.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.466+0000 7f679d1ad640 1 -- 172.21.15.112:0/3955394000 >> 172.21.15.112:0/3955394000 conn(0x7f67980cef20 msgr2=0x7f67980cdcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:22.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.466+0000 7f679d1ad640 1 -- 172.21.15.112:0/3955394000 shutdown_connections 2024-09-21T00:30:22.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.466+0000 7f679d1ad640 1 -- 172.21.15.112:0/3955394000 wait complete. 2024-09-21T00:30:22.478 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T00:30:22.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.588+0000 7fc2e7c8c640 1 -- 172.21.15.112:0/392744930 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2e00b3cc0 msgr2=0x7fc2e015baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:22.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.588+0000 7fc2e7c8c640 1 --2- 172.21.15.112:0/392744930 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2e00b3cc0 0x7fc2e015baa0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7fc2e00d0c70 tx=0x7fc2d402a610 comp rx=0 tx=0).stop 2024-09-21T00:30:22.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.588+0000 7fc2e7c8c640 1 -- 172.21.15.112:0/392744930 shutdown_connections 2024-09-21T00:30:22.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.588+0000 7fc2e7c8c640 1 --2- 172.21.15.112:0/392744930 >> v2:172.21.15.187:3300/0 conn(0x7fc2e00cb9d0 0x7fc2e00cfc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:22.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.588+0000 7fc2e7c8c640 1 --2- 172.21.15.112:0/392744930 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2e00b3cc0 0x7fc2e015baa0 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:22.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.588+0000 7fc2e7c8c640 1 -- 172.21.15.112:0/392744930 >> 172.21.15.112:0/392744930 conn(0x7fc2e006d4c0 msgr2=0x7fc2e00d02c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:22.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.588+0000 7fc2e7c8c640 1 -- 172.21.15.112:0/392744930 shutdown_connections 2024-09-21T00:30:22.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.588+0000 7fc2e7c8c640 1 -- 172.21.15.112:0/392744930 wait complete. 2024-09-21T00:30:22.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.590+0000 7fc2e7c8c640 1 Processor -- start 2024-09-21T00:30:22.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.590+0000 7fc2e7c8c640 1 -- start start 2024-09-21T00:30:22.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.591+0000 7fc2e7c8c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc2e00cb9d0 0x7fc2e00c6820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:22.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.591+0000 7fc2e7c8c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2e00d0fc0 0x7fc2e015e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:22.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.591+0000 7fc2e7c8c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc2e01f39f0 0x7fc2e01f3e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:22.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.591+0000 7fc2e7c8c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc2e006f630 con 0x7fc2e00d0fc0 2024-09-21T00:30:22.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.591+0000 7fc2e7c8c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc2e006f4b0 con 0x7fc2e00cb9d0 2024-09-21T00:30:22.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.591+0000 7fc2e7c8c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc2e006f7b0 con 0x7fc2e01f39f0 2024-09-21T00:30:22.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.591+0000 7fc2e5227640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2e00d0fc0 0x7fc2e015e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:22.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.591+0000 7fc2e5227640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2e00d0fc0 0x7fc2e015e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49658/0 (socket says 172.21.15.112:49658) 2024-09-21T00:30:22.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.591+0000 7fc2e5227640 1 -- 172.21.15.112:0/3329129199 learned_addr learned my addr 172.21.15.112:0/3329129199 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:30:22.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.591+0000 7fc2e6229640 1 --2- 172.21.15.112:0/3329129199 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc2e01f39f0 0x7fc2e01f3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:22.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.591+0000 7fc2e5a28640 1 --2- 172.21.15.112:0/3329129199 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc2e00cb9d0 0x7fc2e00c6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:22.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.591+0000 7fc2e5227640 1 -- 172.21.15.112:0/3329129199 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc2e01f39f0 msgr2=0x7fc2e01f3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:22.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.591+0000 7fc2e5227640 1 --2- 172.21.15.112:0/3329129199 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc2e01f39f0 0x7fc2e01f3e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:22.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.591+0000 7fc2e5227640 1 -- 172.21.15.112:0/3329129199 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc2e00cb9d0 msgr2=0x7fc2e00c6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:22.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.591+0000 7fc2e5227640 1 --2- 172.21.15.112:0/3329129199 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc2e00cb9d0 0x7fc2e00c6820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:22.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.591+0000 7fc2e5227640 1 -- 172.21.15.112:0/3329129199 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2e00c5780 con 0x7fc2e00d0fc0 2024-09-21T00:30:22.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.592+0000 7fc2e5227640 1 --2- 172.21.15.112:0/3329129199 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2e00d0fc0 0x7fc2e015e890 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7fc2d4034040 tx=0x7fc2d4003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:22.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.592+0000 7fc2de7fc640 1 -- 172.21.15.112:0/3329129199 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2d4035040 con 0x7fc2e00d0fc0 2024-09-21T00:30:22.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.592+0000 7fc2de7fc640 1 -- 172.21.15.112:0/3329129199 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc2d4004890 con 0x7fc2e00d0fc0 2024-09-21T00:30:22.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.592+0000 7fc2e7c8c640 1 -- 172.21.15.112:0/3329129199 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2e00c5a60 con 0x7fc2e00d0fc0 2024-09-21T00:30:22.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.592+0000 7fc2de7fc640 1 -- 172.21.15.112:0/3329129199 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2d4004c80 con 0x7fc2e00d0fc0 2024-09-21T00:30:22.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.592+0000 7fc2e7c8c640 1 -- 172.21.15.112:0/3329129199 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2e00c6020 con 0x7fc2e00d0fc0 2024-09-21T00:30:22.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.593+0000 7fc2e7c8c640 1 -- 172.21.15.112:0/3329129199 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2e006fce0 con 0x7fc2e00d0fc0 2024-09-21T00:30:22.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.593+0000 7fc2de7fc640 1 -- 172.21.15.112:0/3329129199 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc2d4004e20 con 0x7fc2e00d0fc0 2024-09-21T00:30:22.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.594+0000 7fc2de7fc640 1 --2- 172.21.15.112:0/3329129199 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fc2cc078bc0 0x7fc2cc07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:22.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.594+0000 7fc2de7fc640 1 -- 172.21.15.112:0/3329129199 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 6187+0+0 (secure 0 0 0) 0x7fc2d40bd910 con 0x7fc2e00d0fc0 2024-09-21T00:30:22.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.594+0000 7fc2e5a28640 1 --2- 172.21.15.112:0/3329129199 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fc2cc078bc0 0x7fc2cc07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:22.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.595+0000 7fc2e5a28640 1 --2- 172.21.15.112:0/3329129199 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fc2cc078bc0 0x7fc2cc07b0d0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fc2d0009f60 tx=0x7fc2d00094f0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:22.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.599+0000 7fc2de7fc640 1 -- 172.21.15.112:0/3329129199 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc2d4048060 con 0x7fc2e00d0fc0 2024-09-21T00:30:22.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.780+0000 7fc2e7c8c640 1 -- 172.21.15.112:0/3329129199 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc2e006fce0 con 0x7fc2e00d0fc0 2024-09-21T00:30:22.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.782+0000 7fc2de7fc640 1 -- 172.21.15.112:0/3329129199 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v112) ==== 74+0+17167 (secure 0 0 0) 0x7fc2d4085b70 con 0x7fc2e00d0fc0 2024-09-21T00:30:22.783 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:30:22.783 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":112,"fsid":"8b34692b-dc65-49b2-a08f-26f707b96ac5","created":"2024-09-21T00:25:22.788490+0000","modified":"2024-09-21T00:30:02.243534+0000","last_up_change":"2024-09-21T00:25:36.161835+0000","last_in_change":"2024-09-21T00:25:26.346500+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T00:25:37.105006+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-09-21T00:29:46.103570+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-09-21T00:29:47.420294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9c575086-67d2-4de2-b01b-d44dcc21d604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6809","nonce":755760621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6811","nonce":755760621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6815","nonce":755760621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6813","nonce":755760621}]},"public_addr":"172.21.15.112:6809/755760621","cluster_addr":"172.21.15.112:6811/755760621","heartbeat_back_addr":"172.21.15.112:6815/755760621","heartbeat_front_addr":"172.21.15.112:6813/755760621","state":["exists","up"]},{"osd":1,"uuid":"a983c227-a3ce-4bdd-82f1-438ba7e3ecd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6801","nonce":344932592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6803","nonce":344932592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6807","nonce":344932592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6805","nonce":344932592}]},"public_addr":"172.21.15.112:6801/344932592","cluster_addr":"172.21.15.112:6803/344932592","heartbeat_back_addr":"172.21.15.112:6807/344932592","heartbeat_front_addr":"172.21.15.112:6805/344932592","state":["exists","up"]},{"osd":2,"uuid":"1193c1d8-bfff-4a57-8200-5f69bf1ea3af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6825","nonce":407933616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6827","nonce":407933616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6831","nonce":407933616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6829","nonce":407933616}]},"public_addr":"172.21.15.112:6825/407933616","cluster_addr":"172.21.15.112:6827/407933616","heartbeat_back_addr":"172.21.15.112:6831/407933616","heartbeat_front_addr":"172.21.15.112:6829/407933616","state":["exists","up"]},{"osd":3,"uuid":"af04330a-36ad-40cc-a8dd-fa259b221de5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6817","nonce":4120164806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6819","nonce":4120164806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6823","nonce":4120164806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6821","nonce":4120164806}]},"public_addr":"172.21.15.112:6817/4120164806","cluster_addr":"172.21.15.112:6819/4120164806","heartbeat_back_addr":"172.21.15.112:6823/4120164806","heartbeat_front_addr":"172.21.15.112:6821/4120164806","state":["exists","up"]},{"osd":4,"uuid":"5b893111-292c-4c5a-b184-53ad38c7199e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6803","nonce":522077560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6805","nonce":522077560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6809","nonce":522077560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6807","nonce":522077560}]},"public_addr":"172.21.15.187:6803/522077560","cluster_addr":"172.21.15.187:6805/522077560","heartbeat_back_addr":"172.21.15.187:6809/522077560","heartbeat_front_addr":"172.21.15.187:6807/522077560","state":["exists","up"]},{"osd":5,"uuid":"c0b8e011-45fe-4c5e-925e-6be2a7f1868a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6827","nonce":3593144721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6829","nonce":3593144721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6833","nonce":3593144721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6831","nonce":3593144721}]},"public_addr":"172.21.15.187:6827/3593144721","cluster_addr":"172.21.15.187:6829/3593144721","heartbeat_back_addr":"172.21.15.187:6833/3593144721","heartbeat_front_addr":"172.21.15.187:6831/3593144721","state":["exists","up"]},{"osd":6,"uuid":"87d2d650-7489-4593-98a3-a1f433ce8dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6811","nonce":1494387657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6813","nonce":1494387657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6817","nonce":1494387657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6815","nonce":1494387657}]},"public_addr":"172.21.15.187:6811/1494387657","cluster_addr":"172.21.15.187:6813/1494387657","heartbeat_back_addr":"172.21.15.187:6817/1494387657","heartbeat_front_addr":"172.21.15.187:6815/1494387657","state":["exists","up"]},{"osd":7,"uuid":"c6b4f8e2-a1ee-4438-aa37-67ff4b6ef1f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6819","nonce":1624699842}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6821","nonce":1624699842}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6825","nonce":1624699842}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6823","nonce":1624699842}]},"public_addr":"172.21.15.187:6819/1624699842","cluster_addr":"172.21.15.187:6821/1624699842","heartbeat_back_addr":"172.21.15.187:6825/1624699842","heartbeat_front_addr":"172.21.15.187:6823/1624699842","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.873162+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.400506+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.664317+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.159791+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.202630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.113212+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.384786+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.448441+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.187:6837/2071643267":"2024-09-22T00:30:02.243502+0000","172.21.15.187:6835/2071643267":"2024-09-22T00:30:02.243502+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T00:30:22.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.784+0000 7fc2e7c8c640 1 -- 172.21.15.112:0/3329129199 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fc2cc078bc0 msgr2=0x7fc2cc07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:22.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.784+0000 7fc2e7c8c640 1 --2- 172.21.15.112:0/3329129199 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fc2cc078bc0 0x7fc2cc07b0d0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fc2d0009f60 tx=0x7fc2d00094f0 comp rx=0 tx=0).stop 2024-09-21T00:30:22.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.784+0000 7fc2e7c8c640 1 -- 172.21.15.112:0/3329129199 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2e00d0fc0 msgr2=0x7fc2e015e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:22.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.784+0000 7fc2e7c8c640 1 --2- 172.21.15.112:0/3329129199 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2e00d0fc0 0x7fc2e015e890 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7fc2d4034040 tx=0x7fc2d4003b90 comp rx=0 tx=0).stop 2024-09-21T00:30:22.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.784+0000 7fc2e7c8c640 1 -- 172.21.15.112:0/3329129199 shutdown_connections 2024-09-21T00:30:22.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.784+0000 7fc2e7c8c640 1 --2- 172.21.15.112:0/3329129199 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc2e01f39f0 0x7fc2e01f3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:22.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.784+0000 7fc2e7c8c640 1 --2- 172.21.15.112:0/3329129199 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fc2cc078bc0 0x7fc2cc07b0d0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:22.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.784+0000 7fc2e7c8c640 1 --2- 172.21.15.112:0/3329129199 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2e00d0fc0 0x7fc2e015e890 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:22.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.784+0000 7fc2e7c8c640 1 --2- 172.21.15.112:0/3329129199 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc2e00cb9d0 0x7fc2e00c6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:22.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.784+0000 7fc2e7c8c640 1 -- 172.21.15.112:0/3329129199 >> 172.21.15.112:0/3329129199 conn(0x7fc2e006d4c0 msgr2=0x7fc2e0070500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:22.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.784+0000 7fc2e7c8c640 1 -- 172.21.15.112:0/3329129199 shutdown_connections 2024-09-21T00:30:22.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.784+0000 7fc2e7c8c640 1 -- 172.21.15.112:0/3329129199 wait complete. 2024-09-21T00:30:22.796 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T00:30:22.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.909+0000 7f1e24cbd640 1 -- 172.21.15.112:0/2225627788 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e200b9b40 msgr2=0x7f1e2006df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:22.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.909+0000 7f1e24cbd640 1 --2- 172.21.15.112:0/2225627788 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e200b9b40 0x7f1e2006df80 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f1e10005fd0 tx=0x7f1e1002a5d0 comp rx=0 tx=0).stop 2024-09-21T00:30:22.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.909+0000 7f1e24cbd640 1 -- 172.21.15.112:0/2225627788 shutdown_connections 2024-09-21T00:30:22.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.909+0000 7f1e24cbd640 1 --2- 172.21.15.112:0/2225627788 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1e200d17f0 0x7f1e200cfcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:22.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.909+0000 7f1e24cbd640 1 --2- 172.21.15.112:0/2225627788 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e200b9b40 0x7f1e2006df80 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:22.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.909+0000 7f1e24cbd640 1 -- 172.21.15.112:0/2225627788 >> 172.21.15.112:0/2225627788 conn(0x7f1e2008bf90 msgr2=0x7f1e200d4780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:22.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.909+0000 7f1e24cbd640 1 -- 172.21.15.112:0/2225627788 shutdown_connections 2024-09-21T00:30:22.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.909+0000 7f1e24cbd640 1 -- 172.21.15.112:0/2225627788 wait complete. 2024-09-21T00:30:22.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.911+0000 7f1e24cbd640 1 Processor -- start 2024-09-21T00:30:22.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.911+0000 7f1e24cbd640 1 -- start start 2024-09-21T00:30:22.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.912+0000 7f1e24cbd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e200b9b40 0x7f1e201e85d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:22.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.912+0000 7f1e24cbd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1e200d53a0 0x7f1e201f87f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:22.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.912+0000 7f1e24cbd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1e201f8d30 0x7f1e20150890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:22.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.912+0000 7f1e24cbd640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1e200b3d40 con 0x7f1e200b9b40 2024-09-21T00:30:22.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.912+0000 7f1e24cbd640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f1e200b3bc0 con 0x7f1e200d53a0 2024-09-21T00:30:22.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.912+0000 7f1e24cbd640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f1e200d0200 con 0x7f1e201f8d30 2024-09-21T00:30:22.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.912+0000 7f1e1e59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e200b9b40 0x7f1e201e85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:22.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.913+0000 7f1e1e59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e200b9b40 0x7f1e201e85d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49674/0 (socket says 172.21.15.112:49674) 2024-09-21T00:30:22.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.913+0000 7f1e1e59c640 1 -- 172.21.15.112:0/653537637 learned_addr learned my addr 172.21.15.112:0/653537637 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:30:22.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.913+0000 7f1e1ed9d640 1 --2- 172.21.15.112:0/653537637 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1e201f8d30 0x7f1e20150890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:22.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.913+0000 7f1e1dd9b640 1 --2- 172.21.15.112:0/653537637 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1e200d53a0 0x7f1e201f87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:22.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.913+0000 7f1e1e59c640 1 -- 172.21.15.112:0/653537637 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1e201f8d30 msgr2=0x7f1e20150890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:22.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.913+0000 7f1e1e59c640 1 --2- 172.21.15.112:0/653537637 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1e201f8d30 0x7f1e20150890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:22.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.913+0000 7f1e1e59c640 1 -- 172.21.15.112:0/653537637 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1e200d53a0 msgr2=0x7f1e201f87f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:22.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.913+0000 7f1e1e59c640 1 --2- 172.21.15.112:0/653537637 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1e200d53a0 0x7f1e201f87f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:22.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.913+0000 7f1e1e59c640 1 -- 172.21.15.112:0/653537637 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e201f9140 con 0x7f1e200b9b40 2024-09-21T00:30:22.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.913+0000 7f1e1e59c640 1 --2- 172.21.15.112:0/653537637 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e200b9b40 0x7f1e201e85d0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f1e10034040 tx=0x7f1e100038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:22.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.913+0000 7f1e0effd640 1 -- 172.21.15.112:0/653537637 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e1003e030 con 0x7f1e200b9b40 2024-09-21T00:30:22.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.913+0000 7f1e0effd640 1 -- 172.21.15.112:0/653537637 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1e10003ca0 con 0x7f1e200b9b40 2024-09-21T00:30:22.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.913+0000 7f1e24cbd640 1 -- 172.21.15.112:0/653537637 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e201e9ec0 con 0x7f1e200b9b40 2024-09-21T00:30:22.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.913+0000 7f1e0effd640 1 -- 172.21.15.112:0/653537637 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e10035040 con 0x7f1e200b9b40 2024-09-21T00:30:22.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.913+0000 7f1e24cbd640 1 -- 172.21.15.112:0/653537637 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e201e9100 con 0x7f1e200b9b40 2024-09-21T00:30:22.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.914+0000 7f1e24cbd640 1 -- 172.21.15.112:0/653537637 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e2006df80 con 0x7f1e200b9b40 2024-09-21T00:30:22.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.915+0000 7f1e0effd640 1 -- 172.21.15.112:0/653537637 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1e10041030 con 0x7f1e200b9b40 2024-09-21T00:30:22.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.915+0000 7f1e0effd640 1 --2- 172.21.15.112:0/653537637 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1e00078b90 0x7f1e0007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:22.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.916+0000 7f1e0effd640 1 -- 172.21.15.112:0/653537637 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 6187+0+0 (secure 0 0 0) 0x7f1e100bb550 con 0x7f1e200b9b40 2024-09-21T00:30:22.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.916+0000 7f1e1dd9b640 1 --2- 172.21.15.112:0/653537637 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1e00078b90 0x7f1e0007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:22.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.916+0000 7f1e1dd9b640 1 --2- 172.21.15.112:0/653537637 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1e00078b90 0x7f1e0007b0a0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f1e200b6f10 tx=0x7f1e14009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:22.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:22.920+0000 7f1e0effd640 1 -- 172.21.15.112:0/653537637 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1e201f9140 con 0x7f1e200b9b40 2024-09-21T00:30:23.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:23.135+0000 7f1e24cbd640 1 -- 172.21.15.112:0/653537637 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f1e200b20c0 con 0x7f1e200b9b40 2024-09-21T00:30:23.136 INFO:tasks.ceph.mon.a.smithi112.stderr:2024-09-21T00:30:23.136+0000 7f7656c13640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T00:30:23.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:23.138+0000 7f1e0effd640 1 -- 172.21.15.112:0/653537637 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v81) ==== 161+0+0 (secure 0 0 0) 0x7f1e10083160 con 0x7f1e200b9b40 2024-09-21T00:30:23.138 INFO:teuthology.orchestra.run.smithi112.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T00:30:23.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:23.141+0000 7f1e24cbd640 1 -- 172.21.15.112:0/653537637 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1e00078b90 msgr2=0x7f1e0007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:23.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:23.141+0000 7f1e24cbd640 1 --2- 172.21.15.112:0/653537637 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1e00078b90 0x7f1e0007b0a0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f1e200b6f10 tx=0x7f1e14009040 comp rx=0 tx=0).stop 2024-09-21T00:30:23.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:23.141+0000 7f1e24cbd640 1 -- 172.21.15.112:0/653537637 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e200b9b40 msgr2=0x7f1e201e85d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:23.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:23.141+0000 7f1e24cbd640 1 --2- 172.21.15.112:0/653537637 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e200b9b40 0x7f1e201e85d0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f1e10034040 tx=0x7f1e100038a0 comp rx=0 tx=0).stop 2024-09-21T00:30:23.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:23.141+0000 7f1e24cbd640 1 -- 172.21.15.112:0/653537637 shutdown_connections 2024-09-21T00:30:23.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:23.141+0000 7f1e24cbd640 1 --2- 172.21.15.112:0/653537637 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1e201f8d30 0x7f1e20150890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:23.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:23.141+0000 7f1e24cbd640 1 --2- 172.21.15.112:0/653537637 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1e200d53a0 0x7f1e201f87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:23.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:23.141+0000 7f1e24cbd640 1 --2- 172.21.15.112:0/653537637 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1e00078b90 0x7f1e0007b0a0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:23.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:23.141+0000 7f1e24cbd640 1 --2- 172.21.15.112:0/653537637 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e200b9b40 0x7f1e201e85d0 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:23.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:23.141+0000 7f1e24cbd640 1 -- 172.21.15.112:0/653537637 >> 172.21.15.112:0/653537637 conn(0x7f1e2008bf90 msgr2=0x7f1e20160c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:23.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:23.141+0000 7f1e24cbd640 1 -- 172.21.15.112:0/653537637 shutdown_connections 2024-09-21T00:30:23.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:23.141+0000 7f1e24cbd640 1 -- 172.21.15.112:0/653537637 wait complete. 2024-09-21T00:30:23.146 INFO:tasks.ceph.mds.c.smithi112.stderr: -5180> 2024-09-21T00:30:07.616+0000 7fb1438b0640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T00:30:23.151 INFO:tasks.ceph.mds.c.smithi112.stderr: -2907> 2024-09-21T00:30:12.616+0000 7fb1438b0640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-09-21T00:30:23.158 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-21T00:30:23.196 INFO:tasks.ceph.mds.a.smithi112.stdout:starting mds.a at 2024-09-21T00:30:23.207 INFO:tasks.ceph.mds.c.smithi112.stdout:starting mds.c at 2024-09-21T00:30:23.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:23.276+0000 7fe84cb3a640 1 -- 172.21.15.112:0/3649176217 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe8480b42a0 msgr2=0x7fe8480b46b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:23.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:23.276+0000 7fe84cb3a640 1 --2- 172.21.15.112:0/3649176217 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe8480b42a0 0x7fe8480b46b0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7fe840009900 tx=0x7fe84002a690 comp rx=0 tx=0).stop 2024-09-21T00:30:23.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:23.276+0000 7fe84cb3a640 1 -- 172.21.15.112:0/3649176217 shutdown_connections 2024-09-21T00:30:23.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:23.276+0000 7fe84cb3a640 1 --2- 172.21.15.112:0/3649176217 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe8480b42a0 0x7fe8480b46b0 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:23.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:23.276+0000 7fe84cb3a640 1 --2- 172.21.15.112:0/3649176217 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe8480b64e0 0x7fe8480b3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:23.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:23.276+0000 7fe84cb3a640 1 -- 172.21.15.112:0/3649176217 >> 172.21.15.112:0/3649176217 conn(0x7fe8480b2fa0 msgr2=0x7fe8480b33b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:23.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:23.276+0000 7fe84cb3a640 1 -- 172.21.15.112:0/3649176217 shutdown_connections 2024-09-21T00:30:23.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:23.276+0000 7fe84cb3a640 1 -- 172.21.15.112:0/3649176217 wait complete. 2024-09-21T00:30:23.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:23.278+0000 7fe84cb3a640 1 Processor -- start 2024-09-21T00:30:23.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:23.278+0000 7fe84cb3a640 1 -- start start 2024-09-21T00:30:23.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:23.279+0000 7fe84cb3a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe8480b42a0 0x7fe84809e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:23.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:23.279+0000 7fe84cb3a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe8480b64e0 0x7fe848150020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:23.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:23.280+0000 7fe84cb3a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe8480cf2d0 0x7fe848150560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:23.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:23.280+0000 7fe84cb3a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe8480b5140 con 0x7fe8480b64e0 2024-09-21T00:30:23.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:23.280+0000 7fe84cb3a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe8480b4fc0 con 0x7fe8480b42a0 2024-09-21T00:30:23.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:23.280+0000 7fe84cb3a640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe8480b52c0 con 0x7fe8480cf2d0 2024-09-21T00:30:23.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:23.280+0000 7fe845d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe8480b64e0 0x7fe848150020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:23.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:23.280+0000 7fe845d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe8480b64e0 0x7fe848150020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49722/0 (socket says 172.21.15.112:49722) 2024-09-21T00:30:23.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:23.280+0000 7fe845d9b640 1 -- 172.21.15.112:0/2513234911 learned_addr learned my addr 172.21.15.112:0/2513234911 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:30:23.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:23.280+0000 7fe845d9b640 1 -- 172.21.15.112:0/2513234911 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe8480cf2d0 msgr2=0x7fe848150560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:23.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:23.280+0000 7fe846d9d640 1 --2- 172.21.15.112:0/2513234911 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe8480cf2d0 0x7fe848150560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:23.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:23.280+0000 7fe84659c640 1 --2- 172.21.15.112:0/2513234911 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe8480b42a0 0x7fe84809e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:23.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:23.280+0000 7fe845d9b640 1 --2- 172.21.15.112:0/2513234911 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe8480cf2d0 0x7fe848150560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:23.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:23.280+0000 7fe845d9b640 1 -- 172.21.15.112:0/2513234911 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe8480b42a0 msgr2=0x7fe84809e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:23.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:23.280+0000 7fe845d9b640 1 --2- 172.21.15.112:0/2513234911 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe8480b42a0 0x7fe84809e200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:23.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:23.280+0000 7fe845d9b640 1 -- 172.21.15.112:0/2513234911 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8481f4400 con 0x7fe8480b64e0 2024-09-21T00:30:23.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:23.280+0000 7fe845d9b640 1 --2- 172.21.15.112:0/2513234911 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe8480b64e0 0x7fe848150020 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7fe83c00c340 tx=0x7fe83c00c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:23.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:23.280+0000 7fe836ffd640 1 -- 172.21.15.112:0/2513234911 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe83c017070 con 0x7fe8480b64e0 2024-09-21T00:30:23.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:23.280+0000 7fe836ffd640 1 -- 172.21.15.112:0/2513234911 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe83c0053c0 con 0x7fe8480b64e0 2024-09-21T00:30:23.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:23.280+0000 7fe84cb3a640 1 -- 172.21.15.112:0/2513234911 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8481f46e0 con 0x7fe8480b64e0 2024-09-21T00:30:23.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:23.280+0000 7fe836ffd640 1 -- 172.21.15.112:0/2513234911 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe83c003850 con 0x7fe8480b64e0 2024-09-21T00:30:23.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:23.280+0000 7fe84cb3a640 1 -- 172.21.15.112:0/2513234911 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8481f4ca0 con 0x7fe8480b64e0 2024-09-21T00:30:23.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:23.281+0000 7fe84cb3a640 1 -- 172.21.15.112:0/2513234911 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8480b46b0 con 0x7fe8480b64e0 2024-09-21T00:30:23.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:23.282+0000 7fe836ffd640 1 -- 172.21.15.112:0/2513234911 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe83c01a070 con 0x7fe8480b64e0 2024-09-21T00:30:23.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:23.283+0000 7fe836ffd640 1 --2- 172.21.15.112:0/2513234911 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fe828078b90 0x7fe82807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:23.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:23.283+0000 7fe836ffd640 1 -- 172.21.15.112:0/2513234911 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 6359+0+0 (secure 0 0 0) 0x7fe83c09a340 con 0x7fe8480b64e0 2024-09-21T00:30:23.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:23.283+0000 7fe84659c640 1 --2- 172.21.15.112:0/2513234911 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fe828078b90 0x7fe82807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:23.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:23.283+0000 7fe84659c640 1 --2- 172.21.15.112:0/2513234911 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fe828078b90 0x7fe82807b0a0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fe838005c40 tx=0x7fe838005bd0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:23.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:23.287+0000 7fe836ffd640 1 -- 172.21.15.112:0/2513234911 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe83c0624f0 con 0x7fe8480b64e0 2024-09-21T00:30:23.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:23.502+0000 7fe84cb3a640 1 -- 172.21.15.112:0/2513234911 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fe848072910 con 0x7fe8480b64e0 2024-09-21T00:30:24.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:24.143+0000 7fe836ffd640 1 -- 172.21.15.112:0/2513234911 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v82) ==== 104+0+0 (secure 0 0 0) 0x7fe83c061df0 con 0x7fe8480b64e0 2024-09-21T00:30:24.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:24.145+0000 7fe84cb3a640 1 -- 172.21.15.112:0/2513234911 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fe828078b90 msgr2=0x7fe82807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:24.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:24.145+0000 7fe84cb3a640 1 --2- 172.21.15.112:0/2513234911 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fe828078b90 0x7fe82807b0a0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fe838005c40 tx=0x7fe838005bd0 comp rx=0 tx=0).stop 2024-09-21T00:30:24.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:24.145+0000 7fe84cb3a640 1 -- 172.21.15.112:0/2513234911 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe8480b64e0 msgr2=0x7fe848150020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:24.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:24.146+0000 7fe84cb3a640 1 --2- 172.21.15.112:0/2513234911 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe8480b64e0 0x7fe848150020 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7fe83c00c340 tx=0x7fe83c00c800 comp rx=0 tx=0).stop 2024-09-21T00:30:24.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:24.146+0000 7fe84cb3a640 1 -- 172.21.15.112:0/2513234911 shutdown_connections 2024-09-21T00:30:24.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:24.146+0000 7fe84cb3a640 1 --2- 172.21.15.112:0/2513234911 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe8480cf2d0 0x7fe848150560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:24.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:24.146+0000 7fe84cb3a640 1 --2- 172.21.15.112:0/2513234911 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fe828078b90 0x7fe82807b0a0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:24.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:24.146+0000 7fe84cb3a640 1 --2- 172.21.15.112:0/2513234911 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe8480b64e0 0x7fe848150020 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:24.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:24.146+0000 7fe84cb3a640 1 --2- 172.21.15.112:0/2513234911 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe8480b42a0 0x7fe84809e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:24.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:24.146+0000 7fe84cb3a640 1 -- 172.21.15.112:0/2513234911 >> 172.21.15.112:0/2513234911 conn(0x7fe8480b2fa0 msgr2=0x7fe8480cdaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:24.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:24.146+0000 7fe84cb3a640 1 -- 172.21.15.112:0/2513234911 shutdown_connections 2024-09-21T00:30:24.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:24.146+0000 7fe84cb3a640 1 -- 172.21.15.112:0/2513234911 wait complete. 2024-09-21T00:30:24.160 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-21T00:30:24.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:24.318+0000 7f174ade4640 1 -- 172.21.15.112:0/291263098 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f174406e480 msgr2=0x7f174406e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:24.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:24.318+0000 7f174ade4640 1 --2- 172.21.15.112:0/291263098 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f174406e480 0x7f174406e890 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f173c009830 tx=0x7f173c02a480 comp rx=0 tx=0).stop 2024-09-21T00:30:24.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:24.318+0000 7f174ade4640 1 -- 172.21.15.112:0/291263098 shutdown_connections 2024-09-21T00:30:24.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:24.318+0000 7f174ade4640 1 --2- 172.21.15.112:0/291263098 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f174406e480 0x7f174406e890 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:24.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:24.318+0000 7f174ade4640 1 --2- 172.21.15.112:0/291263098 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f17440b60f0 0x7f17440b6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:24.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:24.318+0000 7f174ade4640 1 -- 172.21.15.112:0/291263098 >> 172.21.15.112:0/291263098 conn(0x7f17440d4670 msgr2=0x7f17440d4a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:24.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:24.318+0000 7f174ade4640 1 -- 172.21.15.112:0/291263098 shutdown_connections 2024-09-21T00:30:24.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:24.318+0000 7f174ade4640 1 -- 172.21.15.112:0/291263098 wait complete. 2024-09-21T00:30:24.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:24.320+0000 7f174ade4640 1 Processor -- start 2024-09-21T00:30:24.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:24.320+0000 7f174ade4640 1 -- start start 2024-09-21T00:30:24.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:24.321+0000 7f174ade4640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f174406e480 0x7f1744160470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:24.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:24.322+0000 7f174ade4640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f17440b60f0 0x7f174415d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:24.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:24.322+0000 7f174ade4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17440b9d40 0x7f174409e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:24.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:24.322+0000 7f174ade4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f174406ef50 con 0x7f17440b9d40 2024-09-21T00:30:24.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:24.322+0000 7f174ade4640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f174406edd0 con 0x7f174406e480 2024-09-21T00:30:24.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:24.322+0000 7f174ade4640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f174406f0d0 con 0x7f17440b60f0 2024-09-21T00:30:24.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:24.322+0000 7f1749381640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17440b9d40 0x7f174409e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:24.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:24.322+0000 7f1749381640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17440b9d40 0x7f174409e380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49734/0 (socket says 172.21.15.112:49734) 2024-09-21T00:30:24.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:24.322+0000 7f1749381640 1 -- 172.21.15.112:0/3642465128 learned_addr learned my addr 172.21.15.112:0/3642465128 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:30:24.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:24.322+0000 7f1743fff640 1 --2- 172.21.15.112:0/3642465128 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f17440b60f0 0x7f174415d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:24.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:24.322+0000 7f1748b80640 1 --2- 172.21.15.112:0/3642465128 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f174406e480 0x7f1744160470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:24.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:24.322+0000 7f1749381640 1 -- 172.21.15.112:0/3642465128 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f17440b60f0 msgr2=0x7f174415d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:24.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:24.322+0000 7f1749381640 1 --2- 172.21.15.112:0/3642465128 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f17440b60f0 0x7f174415d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:24.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:24.322+0000 7f1749381640 1 -- 172.21.15.112:0/3642465128 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f174406e480 msgr2=0x7f1744160470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:24.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:24.322+0000 7f1749381640 1 --2- 172.21.15.112:0/3642465128 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f174406e480 0x7f1744160470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:24.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:24.322+0000 7f1749381640 1 -- 172.21.15.112:0/3642465128 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1744160f30 con 0x7f17440b9d40 2024-09-21T00:30:24.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:24.322+0000 7f1743fff640 1 --2- 172.21.15.112:0/3642465128 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f17440b60f0 0x7f174415d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:30:24.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:24.322+0000 7f1749381640 1 --2- 172.21.15.112:0/3642465128 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17440b9d40 0x7f174409e380 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f173c009920 tx=0x7f173c033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:24.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:24.322+0000 7f17417fa640 1 -- 172.21.15.112:0/3642465128 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f173c005c90 con 0x7f17440b9d40 2024-09-21T00:30:24.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:24.322+0000 7f17417fa640 1 -- 172.21.15.112:0/3642465128 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f173c031e50 con 0x7f17440b9d40 2024-09-21T00:30:24.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:24.322+0000 7f174ade4640 1 -- 172.21.15.112:0/3642465128 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1744161210 con 0x7f17440b9d40 2024-09-21T00:30:24.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:24.322+0000 7f17417fa640 1 -- 172.21.15.112:0/3642465128 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f173c005220 con 0x7f17440b9d40 2024-09-21T00:30:24.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:24.322+0000 7f174ade4640 1 -- 172.21.15.112:0/3642465128 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17441617d0 con 0x7f17440b9d40 2024-09-21T00:30:24.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:24.324+0000 7f174ade4640 1 -- 172.21.15.112:0/3642465128 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1744160ca0 con 0x7f17440b9d40 2024-09-21T00:30:24.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:24.324+0000 7f17417fa640 1 -- 172.21.15.112:0/3642465128 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f173c0053c0 con 0x7f17440b9d40 2024-09-21T00:30:24.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:24.325+0000 7f17417fa640 1 --2- 172.21.15.112:0/3642465128 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1724078bc0 0x7f172407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:24.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:24.325+0000 7f17417fa640 1 -- 172.21.15.112:0/3642465128 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 6359+0+0 (secure 0 0 0) 0x7f173c0bc060 con 0x7f17440b9d40 2024-09-21T00:30:24.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:24.325+0000 7f1748b80640 1 --2- 172.21.15.112:0/3642465128 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1724078bc0 0x7f172407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:24.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:24.326+0000 7f1748b80640 1 --2- 172.21.15.112:0/3642465128 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1724078bc0 0x7f172407b0d0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f172c0046b0 tx=0x7f172c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:24.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:24.330+0000 7f17417fa640 1 -- 172.21.15.112:0/3642465128 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f173c03f030 con 0x7f17440b9d40 2024-09-21T00:30:24.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:24.509+0000 7f174ade4640 1 -- 172.21.15.112:0/3642465128 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f17440b3bf0 con 0x7f17440b9d40 2024-09-21T00:30:25.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:25.144+0000 7f17417fa640 1 -- 172.21.15.112:0/3642465128 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v114) ==== 181+0+0 (secure 0 0 0) 0x7f173c084210 con 0x7f17440b9d40 2024-09-21T00:30:25.145 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'cephfs_metadata' removed 2024-09-21T00:30:25.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:25.148+0000 7f174ade4640 1 -- 172.21.15.112:0/3642465128 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1724078bc0 msgr2=0x7f172407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:25.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:25.148+0000 7f174ade4640 1 --2- 172.21.15.112:0/3642465128 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1724078bc0 0x7f172407b0d0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f172c0046b0 tx=0x7f172c009040 comp rx=0 tx=0).stop 2024-09-21T00:30:25.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:25.148+0000 7f174ade4640 1 -- 172.21.15.112:0/3642465128 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17440b9d40 msgr2=0x7f174409e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:25.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:25.148+0000 7f174ade4640 1 --2- 172.21.15.112:0/3642465128 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17440b9d40 0x7f174409e380 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f173c009920 tx=0x7f173c033c20 comp rx=0 tx=0).stop 2024-09-21T00:30:25.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:25.148+0000 7f174ade4640 1 -- 172.21.15.112:0/3642465128 shutdown_connections 2024-09-21T00:30:25.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:25.148+0000 7f174ade4640 1 --2- 172.21.15.112:0/3642465128 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f1724078bc0 0x7f172407b0d0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:25.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:25.148+0000 7f174ade4640 1 --2- 172.21.15.112:0/3642465128 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17440b9d40 0x7f174409e380 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:25.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:25.148+0000 7f174ade4640 1 --2- 172.21.15.112:0/3642465128 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f17440b60f0 0x7f174415d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:25.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:25.148+0000 7f174ade4640 1 --2- 172.21.15.112:0/3642465128 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f174406e480 0x7f1744160470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:25.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:25.148+0000 7f174ade4640 1 -- 172.21.15.112:0/3642465128 >> 172.21.15.112:0/3642465128 conn(0x7f17440d4670 msgr2=0x7f17440b3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:25.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:25.148+0000 7f174ade4640 1 -- 172.21.15.112:0/3642465128 shutdown_connections 2024-09-21T00:30:25.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:25.148+0000 7f174ade4640 1 -- 172.21.15.112:0/3642465128 wait complete. 2024-09-21T00:30:25.163 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-21T00:30:25.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:25.284+0000 7fa6f96a9640 1 -- 172.21.15.112:0/2473910766 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6f40ce1c0 msgr2=0x7fa6f40ce5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:25.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:25.284+0000 7fa6f96a9640 1 --2- 172.21.15.112:0/2473910766 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6f40ce1c0 0x7fa6f40ce5d0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec005fd0 tx=0x7fa6ec02a820 comp rx=0 tx=0).stop 2024-09-21T00:30:25.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:25.284+0000 7fa6f96a9640 1 -- 172.21.15.112:0/2473910766 shutdown_connections 2024-09-21T00:30:25.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:25.284+0000 7fa6f96a9640 1 --2- 172.21.15.112:0/2473910766 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6f40ce1c0 0x7fa6f40ce5d0 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:25.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:25.284+0000 7fa6f96a9640 1 -- 172.21.15.112:0/2473910766 >> 172.21.15.112:0/2473910766 conn(0x7fa6f40cef70 msgr2=0x7fa6f40cf380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:25.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:25.284+0000 7fa6f96a9640 1 -- 172.21.15.112:0/2473910766 shutdown_connections 2024-09-21T00:30:25.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:25.284+0000 7fa6f96a9640 1 -- 172.21.15.112:0/2473910766 wait complete. 2024-09-21T00:30:25.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:25.286+0000 7fa6f96a9640 1 Processor -- start 2024-09-21T00:30:25.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:25.286+0000 7fa6f96a9640 1 -- start start 2024-09-21T00:30:25.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:25.287+0000 7fa6f96a9640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa6f40b16c0 0x7fa6f41f5b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:25.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:25.287+0000 7fa6f96a9640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa6f40b7d40 0x7fa6f415c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:25.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:25.287+0000 7fa6f96a9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6f40ce1c0 0x7fa6f415f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:25.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:25.287+0000 7fa6f96a9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa6f409bb90 con 0x7fa6f40ce1c0 2024-09-21T00:30:25.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:25.287+0000 7fa6f96a9640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa6f409ba10 con 0x7fa6f40b7d40 2024-09-21T00:30:25.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:25.287+0000 7fa6f96a9640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa6f40b1c10 con 0x7fa6f40b16c0 2024-09-21T00:30:25.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:25.287+0000 7fa6f37fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6f40ce1c0 0x7fa6f415f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:25.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:25.287+0000 7fa6f37fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6f40ce1c0 0x7fa6f415f890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49754/0 (socket says 172.21.15.112:49754) 2024-09-21T00:30:25.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:25.287+0000 7fa6f37fe640 1 -- 172.21.15.112:0/195660720 learned_addr learned my addr 172.21.15.112:0/195660720 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:30:25.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:25.288+0000 7fa6f27fc640 1 --2- 172.21.15.112:0/195660720 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa6f40b7d40 0x7fa6f415c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:25.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:25.288+0000 7fa6f2ffd640 1 --2- 172.21.15.112:0/195660720 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa6f40b16c0 0x7fa6f41f5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:25.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:25.288+0000 7fa6f37fe640 1 -- 172.21.15.112:0/195660720 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa6f40b16c0 msgr2=0x7fa6f41f5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:25.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:25.288+0000 7fa6f37fe640 1 --2- 172.21.15.112:0/195660720 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa6f40b16c0 0x7fa6f41f5b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:25.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:25.288+0000 7fa6f37fe640 1 -- 172.21.15.112:0/195660720 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa6f40b7d40 msgr2=0x7fa6f415c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:25.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:25.288+0000 7fa6f37fe640 1 --2- 172.21.15.112:0/195660720 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa6f40b7d40 0x7fa6f415c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:25.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:25.288+0000 7fa6f37fe640 1 -- 172.21.15.112:0/195660720 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6f40ca4e0 con 0x7fa6f40ce1c0 2024-09-21T00:30:25.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:25.288+0000 7fa6f37fe640 1 --2- 172.21.15.112:0/195660720 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6f40ce1c0 0x7fa6f415f890 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec009cd0 tx=0x7fa6ec032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:25.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:25.288+0000 7fa6db7fe640 1 -- 172.21.15.112:0/195660720 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6ec037040 con 0x7fa6f40ce1c0 2024-09-21T00:30:25.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:25.288+0000 7fa6db7fe640 1 -- 172.21.15.112:0/195660720 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa6ec042070 con 0x7fa6f40ce1c0 2024-09-21T00:30:25.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:25.288+0000 7fa6f96a9640 1 -- 172.21.15.112:0/195660720 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6f41f6450 con 0x7fa6f40ce1c0 2024-09-21T00:30:25.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:25.288+0000 7fa6db7fe640 1 -- 172.21.15.112:0/195660720 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6ec004260 con 0x7fa6f40ce1c0 2024-09-21T00:30:25.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:25.288+0000 7fa6f96a9640 1 -- 172.21.15.112:0/195660720 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6f41f6a10 con 0x7fa6f40ce1c0 2024-09-21T00:30:25.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:25.289+0000 7fa6f96a9640 1 -- 172.21.15.112:0/195660720 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6f40ce5d0 con 0x7fa6f40ce1c0 2024-09-21T00:30:25.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:25.290+0000 7fa6db7fe640 1 -- 172.21.15.112:0/195660720 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa6ec004400 con 0x7fa6f40ce1c0 2024-09-21T00:30:25.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:25.290+0000 7fa6db7fe640 1 --2- 172.21.15.112:0/195660720 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fa6e0078bc0 0x7fa6e007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:25.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:25.290+0000 7fa6db7fe640 1 -- 172.21.15.112:0/195660720 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5921+0+0 (secure 0 0 0) 0x7fa6ec0bbd00 con 0x7fa6f40ce1c0 2024-09-21T00:30:25.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:25.291+0000 7fa6f2ffd640 1 --2- 172.21.15.112:0/195660720 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fa6e0078bc0 0x7fa6e007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:25.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:25.291+0000 7fa6f2ffd640 1 --2- 172.21.15.112:0/195660720 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fa6e0078bc0 0x7fa6e007b0d0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fa6e4007fd0 tx=0x7fa6e4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:25.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:25.295+0000 7fa6db7fe640 1 -- 172.21.15.112:0/195660720 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa6ec084060 con 0x7fa6f40ce1c0 2024-09-21T00:30:25.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:25.473+0000 7fa6f96a9640 1 -- 172.21.15.112:0/195660720 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fa6f40b5de0 con 0x7fa6f40ce1c0 2024-09-21T00:30:26.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:26.150+0000 7fa6db7fe640 1 -- 172.21.15.112:0/195660720 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v115) ==== 169+0+0 (secure 0 0 0) 0x7fa6ec083960 con 0x7fa6f40ce1c0 2024-09-21T00:30:26.151 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'cephfs_data' removed 2024-09-21T00:30:26.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:26.153+0000 7fa6f96a9640 1 -- 172.21.15.112:0/195660720 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fa6e0078bc0 msgr2=0x7fa6e007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:26.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:26.153+0000 7fa6f96a9640 1 --2- 172.21.15.112:0/195660720 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fa6e0078bc0 0x7fa6e007b0d0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fa6e4007fd0 tx=0x7fa6e4009040 comp rx=0 tx=0).stop 2024-09-21T00:30:26.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:26.153+0000 7fa6f96a9640 1 -- 172.21.15.112:0/195660720 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6f40ce1c0 msgr2=0x7fa6f415f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:26.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:26.153+0000 7fa6f96a9640 1 --2- 172.21.15.112:0/195660720 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6f40ce1c0 0x7fa6f415f890 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec009cd0 tx=0x7fa6ec032d10 comp rx=0 tx=0).stop 2024-09-21T00:30:26.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:26.154+0000 7fa6f96a9640 1 -- 172.21.15.112:0/195660720 shutdown_connections 2024-09-21T00:30:26.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:26.154+0000 7fa6f96a9640 1 --2- 172.21.15.112:0/195660720 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fa6e0078bc0 0x7fa6e007b0d0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:26.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:26.154+0000 7fa6f96a9640 1 --2- 172.21.15.112:0/195660720 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6f40ce1c0 0x7fa6f415f890 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:26.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:26.154+0000 7fa6f96a9640 1 --2- 172.21.15.112:0/195660720 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa6f40b7d40 0x7fa6f415c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:26.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:26.154+0000 7fa6f96a9640 1 --2- 172.21.15.112:0/195660720 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa6f40b16c0 0x7fa6f41f5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:26.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:26.154+0000 7fa6f96a9640 1 -- 172.21.15.112:0/195660720 >> 172.21.15.112:0/195660720 conn(0x7fa6f40cef70 msgr2=0x7fa6f40b6300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:26.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:26.154+0000 7fa6f96a9640 1 -- 172.21.15.112:0/195660720 shutdown_connections 2024-09-21T00:30:26.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:26.154+0000 7fa6f96a9640 1 -- 172.21.15.112:0/195660720 wait complete. 2024-09-21T00:30:26.168 DEBUG:teuthology.orchestra.run.smithi112:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-21T00:30:26.170 DEBUG:teuthology.orchestra.run.smithi187:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-21T00:30:26.219 INFO:teuthology.orchestra.run.smithi112.stdout:[global] 2024-09-21T00:30:26.219 INFO:teuthology.orchestra.run.smithi112.stdout: chdir = "" 2024-09-21T00:30:26.219 INFO:teuthology.orchestra.run.smithi112.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-21T00:30:26.219 INFO:teuthology.orchestra.run.smithi112.stdout: auth supported = cephx 2024-09-21T00:30:26.219 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:30:26.219 INFO:teuthology.orchestra.run.smithi112.stdout: filestore xattr use omap = true 2024-09-21T00:30:26.219 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:30:26.219 INFO:teuthology.orchestra.run.smithi112.stdout: mon clock drift allowed = 1.000 2024-09-21T00:30:26.219 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:30:26.219 INFO:teuthology.orchestra.run.smithi112.stdout: osd crush chooseleaf type = 0 2024-09-21T00:30:26.219 INFO:teuthology.orchestra.run.smithi112.stdout: auth debug = true 2024-09-21T00:30:26.219 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:30:26.219 INFO:teuthology.orchestra.run.smithi112.stdout: ms die on old message = true 2024-09-21T00:30:26.220 INFO:teuthology.orchestra.run.smithi112.stdout: ms die on bug = true 2024-09-21T00:30:26.220 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:30:26.220 INFO:teuthology.orchestra.run.smithi112.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-21T00:30:26.220 INFO:teuthology.orchestra.run.smithi112.stdout: mon pg warn max object skew = 0 2024-09-21T00:30:26.220 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:30:26.220 INFO:teuthology.orchestra.run.smithi112.stdout: # disable pg_autoscaler by default for new pools 2024-09-21T00:30:26.220 INFO:teuthology.orchestra.run.smithi112.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-21T00:30:26.220 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:30:26.220 INFO:teuthology.orchestra.run.smithi112.stdout: osd pool default size = 2 2024-09-21T00:30:26.220 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:30:26.220 INFO:teuthology.orchestra.run.smithi112.stdout: mon osd allow primary affinity = true 2024-09-21T00:30:26.220 INFO:teuthology.orchestra.run.smithi112.stdout: mon osd allow pg remap = true 2024-09-21T00:30:26.220 INFO:teuthology.orchestra.run.smithi112.stdout: mon warn on legacy crush tunables = false 2024-09-21T00:30:26.220 INFO:teuthology.orchestra.run.smithi112.stdout: mon warn on crush straw calc version zero = false 2024-09-21T00:30:26.220 INFO:teuthology.orchestra.run.smithi112.stdout: mon warn on no sortbitwise = false 2024-09-21T00:30:26.221 INFO:teuthology.orchestra.run.smithi112.stdout: mon warn on osd down out interval zero = false 2024-09-21T00:30:26.221 INFO:teuthology.orchestra.run.smithi112.stdout: mon warn on too few osds = false 2024-09-21T00:30:26.221 INFO:teuthology.orchestra.run.smithi112.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-21T00:30:26.221 INFO:teuthology.orchestra.run.smithi112.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-21T00:30:26.221 INFO:teuthology.orchestra.run.smithi112.stdout: mon_allow_pool_size_one = true 2024-09-21T00:30:26.221 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:30:26.221 INFO:teuthology.orchestra.run.smithi112.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-21T00:30:26.221 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:30:26.221 INFO:teuthology.orchestra.run.smithi112.stdout: osd default data pool replay window = 5 2024-09-21T00:30:26.221 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:30:26.221 INFO:teuthology.orchestra.run.smithi112.stdout: mon allow pool delete = true 2024-09-21T00:30:26.221 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:30:26.221 INFO:teuthology.orchestra.run.smithi112.stdout: mon cluster log file level = debug 2024-09-21T00:30:26.221 INFO:teuthology.orchestra.run.smithi112.stdout: debug asserts on shutdown = true 2024-09-21T00:30:26.221 INFO:teuthology.orchestra.run.smithi112.stdout: mon health detail to clog = false 2024-09-21T00:30:26.222 INFO:teuthology.orchestra.run.smithi112.stdout: mon host = "172.21.15.112,172.21.15.187,[v2:172.21.15.187:3301,v1:172.21.15.187:6790]" 2024-09-21T00:30:26.222 INFO:teuthology.orchestra.run.smithi112.stdout: ms die on skipped message = False 2024-09-21T00:30:26.222 INFO:teuthology.orchestra.run.smithi112.stdout: fsid = 8b34692b-dc65-49b2-a08f-26f707b96ac5 2024-09-21T00:30:26.222 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:30:26.222 INFO:teuthology.orchestra.run.smithi112.stdout:[osd] 2024-09-21T00:30:26.222 INFO:teuthology.orchestra.run.smithi112.stdout: osd journal size = 100 2024-09-21T00:30:26.222 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:30:26.222 INFO:teuthology.orchestra.run.smithi112.stdout: osd scrub load threshold = 5.0 2024-09-21T00:30:26.222 INFO:teuthology.orchestra.run.smithi112.stdout: osd scrub max interval = 600 2024-09-21T00:30:26.222 INFO:teuthology.orchestra.run.smithi112.stdout: osd mclock profile = high_recovery_ops 2024-09-21T00:30:26.222 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:30:26.222 INFO:teuthology.orchestra.run.smithi112.stdout: osd recover clone overlap = true 2024-09-21T00:30:26.222 INFO:teuthology.orchestra.run.smithi112.stdout: osd recovery max chunk = 1048576 2024-09-21T00:30:26.222 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:30:26.222 INFO:teuthology.orchestra.run.smithi112.stdout: osd debug shutdown = true 2024-09-21T00:30:26.223 INFO:teuthology.orchestra.run.smithi112.stdout: osd debug op order = true 2024-09-21T00:30:26.223 INFO:teuthology.orchestra.run.smithi112.stdout: osd debug verify stray on activate = true 2024-09-21T00:30:26.223 INFO:teuthology.orchestra.run.smithi112.stdout: osd debug trim objects = true 2024-09-21T00:30:26.223 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:30:26.223 INFO:teuthology.orchestra.run.smithi112.stdout: osd open classes on start = true 2024-09-21T00:30:26.223 INFO:teuthology.orchestra.run.smithi112.stdout: osd debug pg log writeout = true 2024-09-21T00:30:26.223 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:30:26.223 INFO:teuthology.orchestra.run.smithi112.stdout: osd deep scrub update digest min age = 30 2024-09-21T00:30:26.223 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:30:26.223 INFO:teuthology.orchestra.run.smithi112.stdout: osd map max advance = 10 2024-09-21T00:30:26.223 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:30:26.223 INFO:teuthology.orchestra.run.smithi112.stdout: journal zero on create = true 2024-09-21T00:30:26.223 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:30:26.223 INFO:teuthology.orchestra.run.smithi112.stdout: filestore ondisk finisher threads = 3 2024-09-21T00:30:26.223 INFO:teuthology.orchestra.run.smithi112.stdout: filestore apply finisher threads = 3 2024-09-21T00:30:26.224 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:30:26.224 INFO:teuthology.orchestra.run.smithi112.stdout: bdev debug aio = true 2024-09-21T00:30:26.224 INFO:teuthology.orchestra.run.smithi112.stdout: osd debug misdirected ops = true 2024-09-21T00:30:26.224 INFO:teuthology.orchestra.run.smithi112.stdout: bluestore block size = 96636764160 2024-09-21T00:30:26.224 INFO:teuthology.orchestra.run.smithi112.stdout: bluestore fsck on mount = True 2024-09-21T00:30:26.224 INFO:teuthology.orchestra.run.smithi112.stdout: debug bluefs = 20 2024-09-21T00:30:26.224 INFO:teuthology.orchestra.run.smithi112.stdout: debug bluestore = 20 2024-09-21T00:30:26.224 INFO:teuthology.orchestra.run.smithi112.stdout: debug ms = 1 2024-09-21T00:30:26.224 INFO:teuthology.orchestra.run.smithi112.stdout: debug osd = 20 2024-09-21T00:30:26.224 INFO:teuthology.orchestra.run.smithi112.stdout: debug rocksdb = 10 2024-09-21T00:30:26.224 INFO:teuthology.orchestra.run.smithi112.stdout: mon osd backfillfull_ratio = 0.85 2024-09-21T00:30:26.224 INFO:teuthology.orchestra.run.smithi112.stdout: mon osd full ratio = 0.9 2024-09-21T00:30:26.224 INFO:teuthology.orchestra.run.smithi112.stdout: mon osd nearfull ratio = 0.8 2024-09-21T00:30:26.224 INFO:teuthology.orchestra.run.smithi112.stdout: osd failsafe full ratio = 0.95 2024-09-21T00:30:26.224 INFO:teuthology.orchestra.run.smithi112.stdout: osd objectstore = bluestore 2024-09-21T00:30:26.224 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:30:26.225 INFO:teuthology.orchestra.run.smithi112.stdout:[mgr] 2024-09-21T00:30:26.225 INFO:teuthology.orchestra.run.smithi112.stdout: debug ms = 1 2024-09-21T00:30:26.225 INFO:teuthology.orchestra.run.smithi112.stdout: debug mgr = 20 2024-09-21T00:30:26.225 INFO:teuthology.orchestra.run.smithi112.stdout: debug mon = 20 2024-09-21T00:30:26.225 INFO:teuthology.orchestra.run.smithi112.stdout: debug auth = 20 2024-09-21T00:30:26.225 INFO:teuthology.orchestra.run.smithi112.stdout: mon reweight min pgs per osd = 4 2024-09-21T00:30:26.225 INFO:teuthology.orchestra.run.smithi112.stdout: mon reweight min bytes per osd = 10 2024-09-21T00:30:26.225 INFO:teuthology.orchestra.run.smithi112.stdout: mgr/telemetry/nag = false 2024-09-21T00:30:26.225 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:30:26.225 INFO:teuthology.orchestra.run.smithi112.stdout:[mon] 2024-09-21T00:30:26.225 INFO:teuthology.orchestra.run.smithi112.stdout: debug ms = 1 2024-09-21T00:30:26.225 INFO:teuthology.orchestra.run.smithi112.stdout: debug mon = 20 2024-09-21T00:30:26.225 INFO:teuthology.orchestra.run.smithi112.stdout: debug paxos = 20 2024-09-21T00:30:26.225 INFO:teuthology.orchestra.run.smithi112.stdout: debug auth = 20 2024-09-21T00:30:26.225 INFO:teuthology.orchestra.run.smithi112.stdout: mon data avail warn = 5 2024-09-21T00:30:26.226 INFO:teuthology.orchestra.run.smithi112.stdout: mon mgr mkfs grace = 240 2024-09-21T00:30:26.226 INFO:teuthology.orchestra.run.smithi112.stdout: mon reweight min pgs per osd = 4 2024-09-21T00:30:26.226 INFO:teuthology.orchestra.run.smithi112.stdout: mon osd reporter subtree level = osd 2024-09-21T00:30:26.226 INFO:teuthology.orchestra.run.smithi112.stdout: mon osd prime pg temp = true 2024-09-21T00:30:26.226 INFO:teuthology.orchestra.run.smithi112.stdout: mon reweight min bytes per osd = 10 2024-09-21T00:30:26.226 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:30:26.226 INFO:teuthology.orchestra.run.smithi112.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-21T00:30:26.226 INFO:teuthology.orchestra.run.smithi112.stdout: auth mon ticket ttl = 660 # 11m 2024-09-21T00:30:26.226 INFO:teuthology.orchestra.run.smithi112.stdout: auth service ticket ttl = 240 # 4m 2024-09-21T00:30:26.226 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:30:26.226 INFO:teuthology.orchestra.run.smithi112.stdout: # don't complain about insecure global_id in the test suite 2024-09-21T00:30:26.226 INFO:teuthology.orchestra.run.smithi112.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-21T00:30:26.226 INFO:teuthology.orchestra.run.smithi112.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-21T00:30:26.226 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:30:26.226 INFO:teuthology.orchestra.run.smithi112.stdout: # 1m isn't quite enough 2024-09-21T00:30:26.227 INFO:teuthology.orchestra.run.smithi112.stdout: mon_down_mkfs_grace = 2m 2024-09-21T00:30:26.227 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:30:26.227 INFO:teuthology.orchestra.run.smithi112.stdout: mon_warn_on_filestore_osds = false 2024-09-21T00:30:26.227 INFO:teuthology.orchestra.run.smithi112.stdout: mon down mkfs grace = 300 2024-09-21T00:30:26.227 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:30:26.227 INFO:teuthology.orchestra.run.smithi112.stdout:[client] 2024-09-21T00:30:26.227 INFO:teuthology.orchestra.run.smithi112.stdout: rgw cache enabled = true 2024-09-21T00:30:26.227 INFO:teuthology.orchestra.run.smithi112.stdout: rgw enable ops log = true 2024-09-21T00:30:26.227 INFO:teuthology.orchestra.run.smithi112.stdout: rgw enable usage log = true 2024-09-21T00:30:26.227 INFO:teuthology.orchestra.run.smithi112.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-21T00:30:26.227 INFO:teuthology.orchestra.run.smithi112.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-21T00:30:26.227 INFO:teuthology.orchestra.run.smithi112.stdout: pid file = "" 2024-09-21T00:30:26.227 INFO:teuthology.orchestra.run.smithi112.stdout: client reconnect stale = True 2024-09-21T00:30:26.227 INFO:teuthology.orchestra.run.smithi112.stdout:[mon.a] 2024-09-21T00:30:26.227 INFO:teuthology.orchestra.run.smithi112.stdout:[mon.b] 2024-09-21T00:30:26.228 INFO:teuthology.orchestra.run.smithi112.stdout:[mon.c] 2024-09-21T00:30:26.228 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.a] 2024-09-21T00:30:26.228 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.c] 2024-09-21T00:30:26.228 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.b] 2024-09-21T00:30:26.228 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.d] 2024-09-21T00:30:26.228 INFO:teuthology.orchestra.run.smithi112.stdout:[mds] 2024-09-21T00:30:26.228 INFO:teuthology.orchestra.run.smithi187.stdout:[global] 2024-09-21T00:30:26.228 INFO:teuthology.orchestra.run.smithi187.stdout: chdir = "" 2024-09-21T00:30:26.228 INFO:teuthology.orchestra.run.smithi187.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-21T00:30:26.228 INFO:teuthology.orchestra.run.smithi187.stdout: auth supported = cephx 2024-09-21T00:30:26.228 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:30:26.229 INFO:teuthology.orchestra.run.smithi187.stdout: filestore xattr use omap = true 2024-09-21T00:30:26.229 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:30:26.229 INFO:teuthology.orchestra.run.smithi187.stdout: mon clock drift allowed = 1.000 2024-09-21T00:30:26.229 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:30:26.229 INFO:teuthology.orchestra.run.smithi187.stdout: osd crush chooseleaf type = 0 2024-09-21T00:30:26.229 INFO:teuthology.orchestra.run.smithi187.stdout: auth debug = true 2024-09-21T00:30:26.229 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:30:26.229 INFO:teuthology.orchestra.run.smithi187.stdout: ms die on old message = true 2024-09-21T00:30:26.229 INFO:teuthology.orchestra.run.smithi187.stdout: ms die on bug = true 2024-09-21T00:30:26.229 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:30:26.229 INFO:teuthology.orchestra.run.smithi187.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-21T00:30:26.229 INFO:teuthology.orchestra.run.smithi187.stdout: mon pg warn max object skew = 0 2024-09-21T00:30:26.229 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:30:26.229 INFO:teuthology.orchestra.run.smithi187.stdout: # disable pg_autoscaler by default for new pools 2024-09-21T00:30:26.230 INFO:teuthology.orchestra.run.smithi187.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-21T00:30:26.230 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:30:26.230 INFO:teuthology.orchestra.run.smithi187.stdout: osd pool default size = 2 2024-09-21T00:30:26.230 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:30:26.230 INFO:teuthology.orchestra.run.smithi187.stdout: mon osd allow primary affinity = true 2024-09-21T00:30:26.230 INFO:teuthology.orchestra.run.smithi187.stdout: mon osd allow pg remap = true 2024-09-21T00:30:26.230 INFO:teuthology.orchestra.run.smithi187.stdout: mon warn on legacy crush tunables = false 2024-09-21T00:30:26.230 INFO:teuthology.orchestra.run.smithi187.stdout: mon warn on crush straw calc version zero = false 2024-09-21T00:30:26.230 INFO:teuthology.orchestra.run.smithi187.stdout: mon warn on no sortbitwise = false 2024-09-21T00:30:26.230 INFO:teuthology.orchestra.run.smithi187.stdout: mon warn on osd down out interval zero = false 2024-09-21T00:30:26.230 INFO:teuthology.orchestra.run.smithi187.stdout: mon warn on too few osds = false 2024-09-21T00:30:26.230 INFO:teuthology.orchestra.run.smithi187.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-21T00:30:26.230 INFO:teuthology.orchestra.run.smithi187.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-21T00:30:26.230 INFO:teuthology.orchestra.run.smithi187.stdout: mon_allow_pool_size_one = true 2024-09-21T00:30:26.231 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:30:26.231 INFO:teuthology.orchestra.run.smithi187.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-21T00:30:26.231 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:30:26.231 INFO:teuthology.orchestra.run.smithi187.stdout: osd default data pool replay window = 5 2024-09-21T00:30:26.231 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:30:26.231 INFO:teuthology.orchestra.run.smithi187.stdout: mon allow pool delete = true 2024-09-21T00:30:26.231 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:30:26.231 INFO:teuthology.orchestra.run.smithi187.stdout: mon cluster log file level = debug 2024-09-21T00:30:26.231 INFO:teuthology.orchestra.run.smithi187.stdout: debug asserts on shutdown = true 2024-09-21T00:30:26.231 INFO:teuthology.orchestra.run.smithi187.stdout: mon health detail to clog = false 2024-09-21T00:30:26.231 INFO:teuthology.orchestra.run.smithi187.stdout: mon host = "172.21.15.112,172.21.15.187,[v2:172.21.15.187:3301,v1:172.21.15.187:6790]" 2024-09-21T00:30:26.231 INFO:teuthology.orchestra.run.smithi187.stdout: ms die on skipped message = False 2024-09-21T00:30:26.231 INFO:teuthology.orchestra.run.smithi187.stdout: fsid = 8b34692b-dc65-49b2-a08f-26f707b96ac5 2024-09-21T00:30:26.231 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:30:26.232 INFO:teuthology.orchestra.run.smithi187.stdout:[osd] 2024-09-21T00:30:26.232 INFO:teuthology.orchestra.run.smithi187.stdout: osd journal size = 100 2024-09-21T00:30:26.232 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:30:26.232 INFO:teuthology.orchestra.run.smithi187.stdout: osd scrub load threshold = 5.0 2024-09-21T00:30:26.232 INFO:teuthology.orchestra.run.smithi187.stdout: osd scrub max interval = 600 2024-09-21T00:30:26.232 INFO:teuthology.orchestra.run.smithi187.stdout: osd mclock profile = high_recovery_ops 2024-09-21T00:30:26.232 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:30:26.232 INFO:teuthology.orchestra.run.smithi187.stdout: osd recover clone overlap = true 2024-09-21T00:30:26.232 INFO:teuthology.orchestra.run.smithi187.stdout: osd recovery max chunk = 1048576 2024-09-21T00:30:26.232 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:30:26.232 INFO:teuthology.orchestra.run.smithi187.stdout: osd debug shutdown = true 2024-09-21T00:30:26.232 INFO:teuthology.orchestra.run.smithi187.stdout: osd debug op order = true 2024-09-21T00:30:26.232 INFO:teuthology.orchestra.run.smithi187.stdout: osd debug verify stray on activate = true 2024-09-21T00:30:26.232 INFO:teuthology.orchestra.run.smithi187.stdout: osd debug trim objects = true 2024-09-21T00:30:26.232 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:30:26.232 INFO:teuthology.orchestra.run.smithi187.stdout: osd open classes on start = true 2024-09-21T00:30:26.233 INFO:teuthology.orchestra.run.smithi187.stdout: osd debug pg log writeout = true 2024-09-21T00:30:26.233 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:30:26.233 INFO:teuthology.orchestra.run.smithi187.stdout: osd deep scrub update digest min age = 30 2024-09-21T00:30:26.233 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:30:26.233 INFO:teuthology.orchestra.run.smithi187.stdout: osd map max advance = 10 2024-09-21T00:30:26.233 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:30:26.233 INFO:teuthology.orchestra.run.smithi187.stdout: journal zero on create = true 2024-09-21T00:30:26.233 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:30:26.233 INFO:teuthology.orchestra.run.smithi187.stdout: filestore ondisk finisher threads = 3 2024-09-21T00:30:26.233 INFO:teuthology.orchestra.run.smithi187.stdout: filestore apply finisher threads = 3 2024-09-21T00:30:26.233 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:30:26.233 INFO:teuthology.orchestra.run.smithi187.stdout: bdev debug aio = true 2024-09-21T00:30:26.233 INFO:teuthology.orchestra.run.smithi187.stdout: osd debug misdirected ops = true 2024-09-21T00:30:26.233 INFO:teuthology.orchestra.run.smithi187.stdout: bluestore block size = 96636764160 2024-09-21T00:30:26.233 INFO:teuthology.orchestra.run.smithi187.stdout: bluestore fsck on mount = True 2024-09-21T00:30:26.234 INFO:teuthology.orchestra.run.smithi187.stdout: debug bluefs = 20 2024-09-21T00:30:26.234 INFO:teuthology.orchestra.run.smithi187.stdout: debug bluestore = 20 2024-09-21T00:30:26.234 INFO:teuthology.orchestra.run.smithi187.stdout: debug ms = 1 2024-09-21T00:30:26.234 INFO:teuthology.orchestra.run.smithi187.stdout: debug osd = 20 2024-09-21T00:30:26.234 INFO:teuthology.orchestra.run.smithi187.stdout: debug rocksdb = 10 2024-09-21T00:30:26.234 INFO:teuthology.orchestra.run.smithi187.stdout: mon osd backfillfull_ratio = 0.85 2024-09-21T00:30:26.234 INFO:teuthology.orchestra.run.smithi187.stdout: mon osd full ratio = 0.9 2024-09-21T00:30:26.234 INFO:teuthology.orchestra.run.smithi187.stdout: mon osd nearfull ratio = 0.8 2024-09-21T00:30:26.234 INFO:teuthology.orchestra.run.smithi187.stdout: osd failsafe full ratio = 0.95 2024-09-21T00:30:26.234 INFO:teuthology.orchestra.run.smithi187.stdout: osd objectstore = bluestore 2024-09-21T00:30:26.234 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:30:26.234 INFO:teuthology.orchestra.run.smithi187.stdout:[mgr] 2024-09-21T00:30:26.234 INFO:teuthology.orchestra.run.smithi187.stdout: debug ms = 1 2024-09-21T00:30:26.235 INFO:teuthology.orchestra.run.smithi187.stdout: debug mgr = 20 2024-09-21T00:30:26.235 INFO:teuthology.orchestra.run.smithi187.stdout: debug mon = 20 2024-09-21T00:30:26.235 INFO:teuthology.orchestra.run.smithi187.stdout: debug auth = 20 2024-09-21T00:30:26.235 INFO:teuthology.orchestra.run.smithi187.stdout: mon reweight min pgs per osd = 4 2024-09-21T00:30:26.235 INFO:teuthology.orchestra.run.smithi187.stdout: mon reweight min bytes per osd = 10 2024-09-21T00:30:26.235 INFO:teuthology.orchestra.run.smithi187.stdout: mgr/telemetry/nag = false 2024-09-21T00:30:26.235 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:30:26.235 INFO:teuthology.orchestra.run.smithi187.stdout:[mon] 2024-09-21T00:30:26.235 INFO:teuthology.orchestra.run.smithi187.stdout: debug ms = 1 2024-09-21T00:30:26.235 INFO:teuthology.orchestra.run.smithi187.stdout: debug mon = 20 2024-09-21T00:30:26.235 INFO:teuthology.orchestra.run.smithi187.stdout: debug paxos = 20 2024-09-21T00:30:26.235 INFO:teuthology.orchestra.run.smithi187.stdout: debug auth = 20 2024-09-21T00:30:26.235 INFO:teuthology.orchestra.run.smithi187.stdout: mon data avail warn = 5 2024-09-21T00:30:26.235 INFO:teuthology.orchestra.run.smithi187.stdout: mon mgr mkfs grace = 240 2024-09-21T00:30:26.235 INFO:teuthology.orchestra.run.smithi187.stdout: mon reweight min pgs per osd = 4 2024-09-21T00:30:26.235 INFO:teuthology.orchestra.run.smithi187.stdout: mon osd reporter subtree level = osd 2024-09-21T00:30:26.236 INFO:teuthology.orchestra.run.smithi187.stdout: mon osd prime pg temp = true 2024-09-21T00:30:26.236 INFO:teuthology.orchestra.run.smithi187.stdout: mon reweight min bytes per osd = 10 2024-09-21T00:30:26.236 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:30:26.236 INFO:teuthology.orchestra.run.smithi187.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-21T00:30:26.236 INFO:teuthology.orchestra.run.smithi187.stdout: auth mon ticket ttl = 660 # 11m 2024-09-21T00:30:26.236 INFO:teuthology.orchestra.run.smithi187.stdout: auth service ticket ttl = 240 # 4m 2024-09-21T00:30:26.236 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:30:26.236 INFO:teuthology.orchestra.run.smithi187.stdout: # don't complain about insecure global_id in the test suite 2024-09-21T00:30:26.236 INFO:teuthology.orchestra.run.smithi187.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-21T00:30:26.236 INFO:teuthology.orchestra.run.smithi187.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-21T00:30:26.236 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:30:26.236 INFO:teuthology.orchestra.run.smithi187.stdout: # 1m isn't quite enough 2024-09-21T00:30:26.236 INFO:teuthology.orchestra.run.smithi187.stdout: mon_down_mkfs_grace = 2m 2024-09-21T00:30:26.236 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:30:26.236 INFO:teuthology.orchestra.run.smithi187.stdout: mon_warn_on_filestore_osds = false 2024-09-21T00:30:26.237 INFO:teuthology.orchestra.run.smithi187.stdout: mon down mkfs grace = 300 2024-09-21T00:30:26.237 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T00:30:26.237 INFO:teuthology.orchestra.run.smithi187.stdout:[client] 2024-09-21T00:30:26.237 INFO:teuthology.orchestra.run.smithi187.stdout: rgw cache enabled = true 2024-09-21T00:30:26.237 INFO:teuthology.orchestra.run.smithi187.stdout: rgw enable ops log = true 2024-09-21T00:30:26.237 INFO:teuthology.orchestra.run.smithi187.stdout: rgw enable usage log = true 2024-09-21T00:30:26.237 INFO:teuthology.orchestra.run.smithi187.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-21T00:30:26.237 INFO:teuthology.orchestra.run.smithi187.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-21T00:30:26.237 INFO:teuthology.orchestra.run.smithi187.stdout: pid file = "" 2024-09-21T00:30:26.237 INFO:teuthology.orchestra.run.smithi187.stdout: client reconnect stale = True 2024-09-21T00:30:26.237 INFO:teuthology.orchestra.run.smithi187.stdout:[mon.a] 2024-09-21T00:30:26.237 INFO:teuthology.orchestra.run.smithi187.stdout:[mon.b] 2024-09-21T00:30:26.237 INFO:teuthology.orchestra.run.smithi187.stdout:[mon.c] 2024-09-21T00:30:26.237 INFO:teuthology.orchestra.run.smithi187.stdout:[mds.a] 2024-09-21T00:30:26.237 INFO:teuthology.orchestra.run.smithi187.stdout:[mds.c] 2024-09-21T00:30:26.237 INFO:teuthology.orchestra.run.smithi187.stdout:[mds.b] 2024-09-21T00:30:26.238 INFO:teuthology.orchestra.run.smithi187.stdout:[mds.d] 2024-09-21T00:30:26.238 INFO:teuthology.orchestra.run.smithi187.stdout:[mds] 2024-09-21T00:30:26.239 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 31 2024-09-21T00:30:26.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:26.351+0000 7f9ffb09b640 1 -- 172.21.15.112:0/1841807709 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ff406db10 msgr2=0x7f9ff40ca770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:26.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:26.351+0000 7f9ffb09b640 1 --2- 172.21.15.112:0/1841807709 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ff406db10 0x7f9ff40ca770 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f9fe4009830 tx=0x7f9fe402a510 comp rx=0 tx=0).stop 2024-09-21T00:30:26.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:26.351+0000 7f9ffb09b640 1 -- 172.21.15.112:0/1841807709 shutdown_connections 2024-09-21T00:30:26.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:26.351+0000 7f9ffb09b640 1 --2- 172.21.15.112:0/1841807709 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ff406db10 0x7f9ff40ca770 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:26.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:26.351+0000 7f9ffb09b640 1 --2- 172.21.15.112:0/1841807709 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9ff4158950 0x7f9ff406d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:26.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:26.351+0000 7f9ffb09b640 1 -- 172.21.15.112:0/1841807709 >> 172.21.15.112:0/1841807709 conn(0x7f9ff40b1c10 msgr2=0x7f9ff40b2020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:26.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:26.351+0000 7f9ffb09b640 1 -- 172.21.15.112:0/1841807709 shutdown_connections 2024-09-21T00:30:26.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:26.351+0000 7f9ffb09b640 1 -- 172.21.15.112:0/1841807709 wait complete. 2024-09-21T00:30:26.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:26.353+0000 7f9ffb09b640 1 Processor -- start 2024-09-21T00:30:26.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:26.353+0000 7f9ffb09b640 1 -- start start 2024-09-21T00:30:26.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:26.354+0000 7f9ffb09b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9ff406db10 0x7f9ff40c99e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:26.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:26.354+0000 7f9ffb09b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9ff40cacb0 0x7f9ff414f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:26.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:26.354+0000 7f9ffb09b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ff4158950 0x7f9ff40b4540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:26.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:26.354+0000 7f9ffb09b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff415f020 con 0x7f9ff4158950 2024-09-21T00:30:26.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:26.354+0000 7f9ffb09b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff4074740 con 0x7f9ff40cacb0 2024-09-21T00:30:26.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:26.354+0000 7f9ffb09b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f9ff415f1a0 con 0x7f9ff406db10 2024-09-21T00:30:26.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:26.354+0000 7f9ff9638640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ff4158950 0x7f9ff40b4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:26.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:26.354+0000 7f9ff9638640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ff4158950 0x7f9ff40b4540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56476/0 (socket says 172.21.15.112:56476) 2024-09-21T00:30:26.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:26.354+0000 7f9ff9638640 1 -- 172.21.15.112:0/1938816972 learned_addr learned my addr 172.21.15.112:0/1938816972 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:30:26.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:26.355+0000 7f9ff8e37640 1 --2- 172.21.15.112:0/1938816972 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9ff406db10 0x7f9ff40c99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:26.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:26.355+0000 7f9febfff640 1 --2- 172.21.15.112:0/1938816972 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9ff40cacb0 0x7f9ff414f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:26.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:26.355+0000 7f9ff9638640 1 -- 172.21.15.112:0/1938816972 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9ff406db10 msgr2=0x7f9ff40c99e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:26.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:26.355+0000 7f9ff9638640 1 --2- 172.21.15.112:0/1938816972 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9ff406db10 0x7f9ff40c99e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:26.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:26.355+0000 7f9ff9638640 1 -- 172.21.15.112:0/1938816972 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9ff40cacb0 msgr2=0x7f9ff414f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:26.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:26.355+0000 7f9ff9638640 1 --2- 172.21.15.112:0/1938816972 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9ff40cacb0 0x7f9ff414f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:26.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:26.355+0000 7f9ff9638640 1 -- 172.21.15.112:0/1938816972 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ff41539b0 con 0x7f9ff4158950 2024-09-21T00:30:26.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:26.355+0000 7f9febfff640 1 --2- 172.21.15.112:0/1938816972 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9ff40cacb0 0x7f9ff414f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:30:26.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:26.355+0000 7f9ff9638640 1 --2- 172.21.15.112:0/1938816972 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ff4158950 0x7f9ff40b4540 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f9fec00c730 tx=0x7f9fec00cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:26.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:26.355+0000 7f9fe97fa640 1 -- 172.21.15.112:0/1938816972 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fec003b70 con 0x7f9ff4158950 2024-09-21T00:30:26.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:26.355+0000 7f9fe97fa640 1 -- 172.21.15.112:0/1938816972 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9fec003d10 con 0x7f9ff4158950 2024-09-21T00:30:26.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:26.355+0000 7f9ffb09b640 1 -- 172.21.15.112:0/1938816972 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ff4153c90 con 0x7f9ff4158950 2024-09-21T00:30:26.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:26.355+0000 7f9fe97fa640 1 -- 172.21.15.112:0/1938816972 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fec0052d0 con 0x7f9ff4158950 2024-09-21T00:30:26.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:26.355+0000 7f9ffb09b640 1 -- 172.21.15.112:0/1938816972 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ff4154250 con 0x7f9ff4158950 2024-09-21T00:30:26.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:26.356+0000 7f9ffb09b640 1 -- 172.21.15.112:0/1938816972 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ff415f660 con 0x7f9ff4158950 2024-09-21T00:30:26.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:26.357+0000 7f9fe97fa640 1 -- 172.21.15.112:0/1938816972 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9fec005470 con 0x7f9ff4158950 2024-09-21T00:30:26.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:26.357+0000 7f9fe97fa640 1 --2- 172.21.15.112:0/1938816972 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f9fe0078bc0 0x7f9fe007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:26.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:26.358+0000 7f9fe97fa640 1 -- 172.21.15.112:0/1938816972 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5507+0+0 (secure 0 0 0) 0x7f9fec09a9b0 con 0x7f9ff4158950 2024-09-21T00:30:26.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:26.358+0000 7f9ff8e37640 1 --2- 172.21.15.112:0/1938816972 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f9fe0078bc0 0x7f9fe007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:26.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:26.358+0000 7f9ff8e37640 1 --2- 172.21.15.112:0/1938816972 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f9fe0078bc0 0x7f9fe007b0d0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f9ff40b2c40 tx=0x7f9ff000b2e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:26.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:26.362+0000 7f9fe97fa640 1 -- 172.21.15.112:0/1938816972 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f9ff406df20 con 0x7f9ff4158950 2024-09-21T00:30:26.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:26.541+0000 7f9ffb09b640 1 -- 172.21.15.112:0/1938816972 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "config reset", "num": 31} v 0) -- 0x7f9ff4069370 con 0x7f9ff4158950 2024-09-21T00:30:26.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:26.545+0000 7f9fe97fa640 1 -- 172.21.15.112:0/1938816972 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 31}]=0 v48) ==== 71+0+0 (secure 0 0 0) 0x7f9fec062ec0 con 0x7f9ff4158950 2024-09-21T00:30:26.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:26.547+0000 7f9ffb09b640 1 -- 172.21.15.112:0/1938816972 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f9fe0078bc0 msgr2=0x7f9fe007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:26.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:26.547+0000 7f9ffb09b640 1 --2- 172.21.15.112:0/1938816972 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f9fe0078bc0 0x7f9fe007b0d0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f9ff40b2c40 tx=0x7f9ff000b2e0 comp rx=0 tx=0).stop 2024-09-21T00:30:26.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:26.547+0000 7f9ffb09b640 1 -- 172.21.15.112:0/1938816972 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ff4158950 msgr2=0x7f9ff40b4540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:26.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:26.547+0000 7f9ffb09b640 1 --2- 172.21.15.112:0/1938816972 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ff4158950 0x7f9ff40b4540 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f9fec00c730 tx=0x7f9fec00cb30 comp rx=0 tx=0).stop 2024-09-21T00:30:26.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:26.547+0000 7f9ffb09b640 1 -- 172.21.15.112:0/1938816972 shutdown_connections 2024-09-21T00:30:26.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:26.547+0000 7f9ffb09b640 1 --2- 172.21.15.112:0/1938816972 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f9fe0078bc0 0x7f9fe007b0d0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:26.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:26.547+0000 7f9ffb09b640 1 --2- 172.21.15.112:0/1938816972 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ff4158950 0x7f9ff40b4540 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:26.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:26.547+0000 7f9ffb09b640 1 --2- 172.21.15.112:0/1938816972 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9ff40cacb0 0x7f9ff414f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:26.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:26.547+0000 7f9ffb09b640 1 --2- 172.21.15.112:0/1938816972 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9ff406db10 0x7f9ff40c99e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:26.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:26.548+0000 7f9ffb09b640 1 -- 172.21.15.112:0/1938816972 >> 172.21.15.112:0/1938816972 conn(0x7f9ff40b1c10 msgr2=0x7f9ff4155940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:26.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:26.548+0000 7f9ffb09b640 1 -- 172.21.15.112:0/1938816972 shutdown_connections 2024-09-21T00:30:26.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:26.548+0000 7f9ffb09b640 1 -- 172.21.15.112:0/1938816972 wait complete. 2024-09-21T00:30:26.559 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_sessionmap.TestSessionMap.test_version_splitting' 2024-09-21T00:30:26.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:26.671+0000 7f3809f0b640 1 -- 172.21.15.112:0/1584198380 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f37ec004800 msgr2=0x7f37ec004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:26.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:26.671+0000 7f3809f0b640 1 --2- 172.21.15.112:0/1584198380 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f37ec004800 0x7f37ec004c10 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f37f800c4a0 tx=0x7f37ec005830 comp rx=0 tx=0).stop 2024-09-21T00:30:26.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:26.671+0000 7f3809f0b640 1 -- 172.21.15.112:0/1584198380 shutdown_connections 2024-09-21T00:30:26.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:26.671+0000 7f3809f0b640 1 --2- 172.21.15.112:0/1584198380 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f37ec004800 0x7f37ec004c10 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:26.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:26.671+0000 7f3809f0b640 1 -- 172.21.15.112:0/1584198380 >> 172.21.15.112:0/1584198380 conn(0x7f38040cf400 msgr2=0x7f38040d2350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:26.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:26.671+0000 7f3809f0b640 1 -- 172.21.15.112:0/1584198380 shutdown_connections 2024-09-21T00:30:26.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:26.671+0000 7f3809f0b640 1 -- 172.21.15.112:0/1584198380 wait complete. 2024-09-21T00:30:26.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:26.673+0000 7f3809f0b640 1 Processor -- start 2024-09-21T00:30:26.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:26.673+0000 7f3809f0b640 1 -- start start 2024-09-21T00:30:26.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:26.674+0000 7f3809f0b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f37ec004800 0x7f38040b6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:26.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:26.674+0000 7f3809f0b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f38041f79a0 0x7f38041f6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:26.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:26.674+0000 7f3809f0b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38041f7400 0x7f380414ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:26.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:26.674+0000 7f3809f0b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3804160330 con 0x7f38041f7400 2024-09-21T00:30:26.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:26.674+0000 7f3809f0b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f38040ccbb0 con 0x7f37ec004800 2024-09-21T00:30:26.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:26.674+0000 7f3809f0b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f380414c790 con 0x7f38041f79a0 2024-09-21T00:30:26.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:26.674+0000 7f3803fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38041f7400 0x7f380414ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:26.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:26.674+0000 7f3803fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38041f7400 0x7f380414ad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56498/0 (socket says 172.21.15.112:56498) 2024-09-21T00:30:26.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:26.674+0000 7f3803fff640 1 -- 172.21.15.112:0/3784457116 learned_addr learned my addr 172.21.15.112:0/3784457116 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:30:26.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:26.675+0000 7f3802ffd640 1 --2- 172.21.15.112:0/3784457116 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f38041f79a0 0x7f38041f6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:26.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:26.675+0000 7f38037fe640 1 --2- 172.21.15.112:0/3784457116 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f37ec004800 0x7f38040b6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:26.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:26.675+0000 7f3803fff640 1 -- 172.21.15.112:0/3784457116 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f38041f79a0 msgr2=0x7f38041f6ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:26.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:26.675+0000 7f3803fff640 1 --2- 172.21.15.112:0/3784457116 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f38041f79a0 0x7f38041f6ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:26.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:26.675+0000 7f3803fff640 1 -- 172.21.15.112:0/3784457116 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f37ec004800 msgr2=0x7f38040b6310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:26.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:26.675+0000 7f3803fff640 1 --2- 172.21.15.112:0/3784457116 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f37ec004800 0x7f38040b6310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:26.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:26.675+0000 7f3803fff640 1 -- 172.21.15.112:0/3784457116 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38040b6c40 con 0x7f38041f7400 2024-09-21T00:30:26.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:26.675+0000 7f3803fff640 1 --2- 172.21.15.112:0/3784457116 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38041f7400 0x7f380414ad60 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f38040ced60 tx=0x7f37fc001b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:26.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:26.675+0000 7f37ebfff640 1 -- 172.21.15.112:0/3784457116 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37fc011870 con 0x7f38041f7400 2024-09-21T00:30:26.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:26.675+0000 7f37ebfff640 1 -- 172.21.15.112:0/3784457116 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f37fc017070 con 0x7f38041f7400 2024-09-21T00:30:26.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:26.675+0000 7f37ebfff640 1 -- 172.21.15.112:0/3784457116 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37fc011d80 con 0x7f38041f7400 2024-09-21T00:30:26.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:26.675+0000 7f3809f0b640 1 -- 172.21.15.112:0/3784457116 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38040b6f20 con 0x7f38041f7400 2024-09-21T00:30:26.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:26.675+0000 7f3809f0b640 1 -- 172.21.15.112:0/3784457116 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38040b74e0 con 0x7f38041f7400 2024-09-21T00:30:26.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:26.677+0000 7f3809f0b640 1 -- 172.21.15.112:0/3784457116 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38040d0680 con 0x7f38041f7400 2024-09-21T00:30:26.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:26.677+0000 7f37ebfff640 1 -- 172.21.15.112:0/3784457116 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f37fc01c460 con 0x7f38041f7400 2024-09-21T00:30:26.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:26.677+0000 7f37ebfff640 1 --2- 172.21.15.112:0/3784457116 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f37f0078b90 0x7f37f007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:26.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:26.678+0000 7f37ebfff640 1 -- 172.21.15.112:0/3784457116 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5507+0+0 (secure 0 0 0) 0x7f37fc09b560 con 0x7f38041f7400 2024-09-21T00:30:26.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:26.678+0000 7f38037fe640 1 --2- 172.21.15.112:0/3784457116 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f37f0078b90 0x7f37f007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:26.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:26.678+0000 7f38037fe640 1 --2- 172.21.15.112:0/3784457116 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f37f0078b90 0x7f37f007b0a0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f37f4005eb0 tx=0x7f37f4005f90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:26.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:26.682+0000 7f37ebfff640 1 -- 172.21.15.112:0/3784457116 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f37fc063a60 con 0x7f38041f7400 2024-09-21T00:30:26.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:26.925+0000 7f3809f0b640 1 -- 172.21.15.112:0/3784457116 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_sessionmap.TestSessionMap.test_version_splitting"]} v 0) -- 0x7f38040b69d0 con 0x7f38041f7400 2024-09-21T00:30:27.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:27.152+0000 7f37ebfff640 1 -- 172.21.15.112:0/3784457116 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_sessionmap.TestSessionMap.test_version_splitting"]}]=0 v276) ==== 145+0+0 (secure 0 0 0) 0x7f37fc063360 con 0x7f38041f7400 2024-09-21T00:30:27.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:27.154+0000 7f3809f0b640 1 -- 172.21.15.112:0/3784457116 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f37f0078b90 msgr2=0x7f37f007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:27.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:27.154+0000 7f3809f0b640 1 --2- 172.21.15.112:0/3784457116 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f37f0078b90 0x7f37f007b0a0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f37f4005eb0 tx=0x7f37f4005f90 comp rx=0 tx=0).stop 2024-09-21T00:30:27.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:27.154+0000 7f3809f0b640 1 -- 172.21.15.112:0/3784457116 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38041f7400 msgr2=0x7f380414ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:27.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:27.154+0000 7f3809f0b640 1 --2- 172.21.15.112:0/3784457116 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38041f7400 0x7f380414ad60 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f38040ced60 tx=0x7f37fc001b80 comp rx=0 tx=0).stop 2024-09-21T00:30:27.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:27.154+0000 7f3809f0b640 1 -- 172.21.15.112:0/3784457116 shutdown_connections 2024-09-21T00:30:27.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:27.154+0000 7f3809f0b640 1 --2- 172.21.15.112:0/3784457116 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f37f0078b90 0x7f37f007b0a0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:27.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:27.154+0000 7f3809f0b640 1 --2- 172.21.15.112:0/3784457116 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38041f7400 0x7f380414ad60 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:27.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:27.154+0000 7f3809f0b640 1 --2- 172.21.15.112:0/3784457116 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f38041f79a0 0x7f38041f6ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:27.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:27.154+0000 7f3809f0b640 1 --2- 172.21.15.112:0/3784457116 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f37ec004800 0x7f38040b6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:27.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:27.154+0000 7f3809f0b640 1 -- 172.21.15.112:0/3784457116 >> 172.21.15.112:0/3784457116 conn(0x7f38040cf400 msgr2=0x7f38040c9400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:27.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:27.154+0000 7f3809f0b640 1 -- 172.21.15.112:0/3784457116 shutdown_connections 2024-09-21T00:30:27.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:27.154+0000 7f3809f0b640 1 -- 172.21.15.112:0/3784457116 wait complete. 2024-09-21T00:30:27.166 INFO:tasks.cephfs_test_runner:That when many sessions are updated, they are correctly ... ok 2024-09-21T00:30:27.166 INFO:tasks.cephfs_test_runner: 2024-09-21T00:30:27.167 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-09-21T00:30:27.167 INFO:tasks.cephfs_test_runner:Ran 7 tests in 249.548s 2024-09-21T00:30:27.167 INFO:tasks.cephfs_test_runner: 2024-09-21T00:30:27.167 INFO:tasks.cephfs_test_runner:OK (skipped=2) 2024-09-21T00:30:27.167 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-09-21T00:30:27.183 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-09-21T00:30:27.209 INFO:tasks.kclient:Unmounting kernel clients... 2024-09-21T00:30:27.209 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T00:30:27.209 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:30:27.225 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056010,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1439 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248716k,nr_inodes=812179,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T00:30:27.225 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T00:30:27.225 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:30:27.281 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056010,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1439 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248716k,nr_inodes=812179,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T00:30:27.281 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-09-21T00:30:27.281 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:30:27.301 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056013,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497440k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16835 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248720k,nr_inodes=812180,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497440k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T00:30:27.301 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-09-21T00:30:27.301 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T00:30:27.360 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056013,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497440k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16835 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248720k,nr_inodes=812180,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497440k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T00:30:27.360 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:30:27.360 DEBUG:teuthology.orchestra.run.smithi112:> ip netns list 2024-09-21T00:30:27.375 INFO:teuthology.orchestra.run.smithi112.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-21T00:30:27.376 INFO:teuthology.orchestra.run.smithi112.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T00:30:27.376 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:30:27.376 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T00:30:27.440 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:30:27.441 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T00:30:27.505 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:30:27.505 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link delete ceph-brx 2024-09-21T00:30:27.607 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:30:27.608 DEBUG:teuthology.orchestra.run.smithi112:> ip netns list 2024-09-21T00:30:27.623 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:30:27.623 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link delete ceph-brx 2024-09-21T00:30:27.691 INFO:teuthology.orchestra.run.smithi112.stderr:Cannot find device "ceph-brx" 2024-09-21T00:30:27.692 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T00:30:27.692 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:30:27.692 DEBUG:teuthology.orchestra.run.smithi187:> ip netns list 2024-09-21T00:30:27.712 INFO:teuthology.orchestra.run.smithi187.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-21T00:30:27.712 INFO:teuthology.orchestra.run.smithi187.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-21T00:30:27.712 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:30:27.712 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-21T00:30:27.783 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:30:27.783 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T00:30:27.853 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:30:27.853 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip link delete ceph-brx 2024-09-21T00:30:27.959 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:30:27.959 DEBUG:teuthology.orchestra.run.smithi187:> ip netns list 2024-09-21T00:30:27.974 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T00:30:27.974 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip link delete ceph-brx 2024-09-21T00:30:28.043 INFO:teuthology.orchestra.run.smithi187.stderr:Cannot find device "ceph-brx" 2024-09-21T00:30:28.045 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T00:30:28.045 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-21T00:30:28.060 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T00:30:28.060 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T00:30:28.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.175+0000 7fcb399c6640 1 -- 172.21.15.112:0/3320377699 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb340cae80 msgr2=0x7fcb340743c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:28.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.175+0000 7fcb399c6640 1 --2- 172.21.15.112:0/3320377699 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb340cae80 0x7fcb340743c0 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c009830 tx=0x7fcb2c02a510 comp rx=0 tx=0).stop 2024-09-21T00:30:28.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.175+0000 7fcb399c6640 1 -- 172.21.15.112:0/3320377699 shutdown_connections 2024-09-21T00:30:28.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.175+0000 7fcb399c6640 1 --2- 172.21.15.112:0/3320377699 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb340cae80 0x7fcb340743c0 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:28.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.175+0000 7fcb399c6640 1 -- 172.21.15.112:0/3320377699 >> 172.21.15.112:0/3320377699 conn(0x7fcb340b1c10 msgr2=0x7fcb340b2020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:28.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.175+0000 7fcb399c6640 1 -- 172.21.15.112:0/3320377699 shutdown_connections 2024-09-21T00:30:28.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.175+0000 7fcb399c6640 1 -- 172.21.15.112:0/3320377699 wait complete. 2024-09-21T00:30:28.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.176+0000 7fcb399c6640 1 Processor -- start 2024-09-21T00:30:28.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.177+0000 7fcb399c6640 1 -- start start 2024-09-21T00:30:28.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.178+0000 7fcb399c6640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcb340cae80 0x7fcb3414f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:28.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.178+0000 7fcb399c6640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcb34158950 0x7fcb3415a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:28.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.178+0000 7fcb399c6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb34146390 0x7fcb3415a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:28.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.178+0000 7fcb399c6640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcb34166360 con 0x7fcb34146390 2024-09-21T00:30:28.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.178+0000 7fcb399c6640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fcb341661e0 con 0x7fcb340cae80 2024-09-21T00:30:28.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.178+0000 7fcb399c6640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fcb341664e0 con 0x7fcb34158950 2024-09-21T00:30:28.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.178+0000 7fcb337fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb34146390 0x7fcb3415a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:28.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.178+0000 7fcb337fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb34146390 0x7fcb3415a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56516/0 (socket says 172.21.15.112:56516) 2024-09-21T00:30:28.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.178+0000 7fcb337fe640 1 -- 172.21.15.112:0/3759040234 learned_addr learned my addr 172.21.15.112:0/3759040234 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:30:28.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.178+0000 7fcb32ffd640 1 --2- 172.21.15.112:0/3759040234 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcb340cae80 0x7fcb3414f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:28.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.178+0000 7fcb327fc640 1 --2- 172.21.15.112:0/3759040234 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcb34158950 0x7fcb3415a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:28.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.178+0000 7fcb337fe640 1 -- 172.21.15.112:0/3759040234 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcb34158950 msgr2=0x7fcb3415a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:28.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.178+0000 7fcb337fe640 1 --2- 172.21.15.112:0/3759040234 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcb34158950 0x7fcb3415a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:28.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.178+0000 7fcb337fe640 1 -- 172.21.15.112:0/3759040234 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcb340cae80 msgr2=0x7fcb3414f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:28.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.178+0000 7fcb337fe640 1 --2- 172.21.15.112:0/3759040234 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcb340cae80 0x7fcb3414f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:28.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.178+0000 7fcb337fe640 1 -- 172.21.15.112:0/3759040234 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb3414f7a0 con 0x7fcb34146390 2024-09-21T00:30:28.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.178+0000 7fcb327fc640 1 --2- 172.21.15.112:0/3759040234 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcb34158950 0x7fcb3415a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:30:28.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.178+0000 7fcb32ffd640 1 --2- 172.21.15.112:0/3759040234 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcb340cae80 0x7fcb3414f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:30:28.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.178+0000 7fcb337fe640 1 --2- 172.21.15.112:0/3759040234 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb34146390 0x7fcb3415a650 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c02ab00 tx=0x7fcb2c033c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:28.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.179+0000 7fcb1bfff640 1 -- 172.21.15.112:0/3759040234 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb2c031d60 con 0x7fcb34146390 2024-09-21T00:30:28.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.179+0000 7fcb1bfff640 1 -- 172.21.15.112:0/3759040234 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcb2c0330d0 con 0x7fcb34146390 2024-09-21T00:30:28.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.179+0000 7fcb399c6640 1 -- 172.21.15.112:0/3759040234 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb341f8ef0 con 0x7fcb34146390 2024-09-21T00:30:28.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.179+0000 7fcb1bfff640 1 -- 172.21.15.112:0/3759040234 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb2c033500 con 0x7fcb34146390 2024-09-21T00:30:28.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.179+0000 7fcb399c6640 1 -- 172.21.15.112:0/3759040234 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb341f91b0 con 0x7fcb34146390 2024-09-21T00:30:28.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.180+0000 7fcb399c6640 1 -- 172.21.15.112:0/3759040234 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb341f6480 con 0x7fcb34146390 2024-09-21T00:30:28.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.180+0000 7fcb1bfff640 1 -- 172.21.15.112:0/3759040234 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fcb2c0035f0 con 0x7fcb34146390 2024-09-21T00:30:28.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.181+0000 7fcb1bfff640 1 --2- 172.21.15.112:0/3759040234 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fcb20078bc0 0x7fcb2007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:28.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.181+0000 7fcb1bfff640 1 -- 172.21.15.112:0/3759040234 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5507+0+0 (secure 0 0 0) 0x7fcb2c0bc140 con 0x7fcb34146390 2024-09-21T00:30:28.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.181+0000 7fcb32ffd640 1 --2- 172.21.15.112:0/3759040234 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fcb20078bc0 0x7fcb2007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:28.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.181+0000 7fcb32ffd640 1 --2- 172.21.15.112:0/3759040234 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fcb20078bc0 0x7fcb2007b0d0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fcb340b2c40 tx=0x7fcb24009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:28.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.185+0000 7fcb1bfff640 1 -- 172.21.15.112:0/3759040234 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fcb3414f7a0 con 0x7fcb34146390 2024-09-21T00:30:28.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.360+0000 7fcb399c6640 1 -- 172.21.15.112:0/3759040234 --> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fcb340ca770 con 0x7fcb20078bc0 2024-09-21T00:30:28.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.361+0000 7fcb1bfff640 1 -- 172.21.15.112:0/3759040234 <== mgr.4103 v2:172.21.15.187:6800/2921522906 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13499 (secure 0 0 0) 0x7fcb340ca770 con 0x7fcb20078bc0 2024-09-21T00:30:28.362 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:30:28.362 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2024-09-21T00:30:28.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.363+0000 7fcb399c6640 1 -- 172.21.15.112:0/3759040234 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fcb20078bc0 msgr2=0x7fcb2007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:28.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.363+0000 7fcb399c6640 1 --2- 172.21.15.112:0/3759040234 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fcb20078bc0 0x7fcb2007b0d0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fcb340b2c40 tx=0x7fcb24009040 comp rx=0 tx=0).stop 2024-09-21T00:30:28.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.363+0000 7fcb399c6640 1 -- 172.21.15.112:0/3759040234 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb34146390 msgr2=0x7fcb3415a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:28.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.363+0000 7fcb399c6640 1 --2- 172.21.15.112:0/3759040234 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb34146390 0x7fcb3415a650 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c02ab00 tx=0x7fcb2c033c90 comp rx=0 tx=0).stop 2024-09-21T00:30:28.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.363+0000 7fcb399c6640 1 -- 172.21.15.112:0/3759040234 shutdown_connections 2024-09-21T00:30:28.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.363+0000 7fcb399c6640 1 --2- 172.21.15.112:0/3759040234 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fcb20078bc0 0x7fcb2007b0d0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:28.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.363+0000 7fcb399c6640 1 --2- 172.21.15.112:0/3759040234 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb34146390 0x7fcb3415a650 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:28.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.363+0000 7fcb399c6640 1 --2- 172.21.15.112:0/3759040234 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcb34158950 0x7fcb3415a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:28.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.363+0000 7fcb399c6640 1 --2- 172.21.15.112:0/3759040234 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcb340cae80 0x7fcb3414f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:28.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.363+0000 7fcb399c6640 1 -- 172.21.15.112:0/3759040234 >> 172.21.15.112:0/3759040234 conn(0x7fcb340b1c10 msgr2=0x7fcb3414ff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:28.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.363+0000 7fcb399c6640 1 -- 172.21.15.112:0/3759040234 shutdown_connections 2024-09-21T00:30:28.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.363+0000 7fcb399c6640 1 -- 172.21.15.112:0/3759040234 wait complete. 2024-09-21T00:30:28.375 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":255,"stamp":"2024-09-21T00:30:27.161772+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":170,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":3,"kb":754974720,"kb_used":261780,"kb_used_data":12252,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754712940,"statfs":{"total":773094113280,"available":772826050560,"internally_reserved":0,"allocated":12546048,"data_stored":6385429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-14137,"num_objects":-41,"num_object_clones":0,"num_object_copies":-82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-41,"num_whiteouts":0,"num_read":-92,"num_read_kb":-87,"num_write":-126,"num_write_kb":-54,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001280"},"pg_stats":[{"pgid":"1.0","version":"17'72","reported_seq":283,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-21T00:30:26.177047+0000","last_change":"2024-09-21T00:25:38.725792+0000","last_active":"2024-09-21T00:30:26.177047+0000","last_peered":"2024-09-21T00:30:26.177047+0000","last_clean":"2024-09-21T00:30:26.177047+0000","last_became_active":"2024-09-21T00:25:38.724668+0000","last_became_peered":"2024-09-21T00:25:38.724668+0000","last_unstale":"2024-09-21T00:30:26.177047+0000","last_undegraded":"2024-09-21T00:30:26.177047+0000","last_fullsized":"2024-09-21T00:30:26.177047+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:37.163216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:37.163216+0000","last_clean_scrub_stamp":"2024-09-21T00:25:37.163216+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:14:36.588660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509501,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33176,"kb_used_data":1980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338664,"statfs":{"total":96636764160,"available":96602791936,"internally_reserved":0,"allocated":2027520,"data_stored":1247148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509502,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32576,"kb_used_data":1384,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339264,"statfs":{"total":96636764160,"available":96603406336,"internally_reserved":0,"allocated":1417216,"data_stored":650651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509501,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32576,"kb_used_data":1384,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339264,"statfs":{"total":96636764160,"available":96603406336,"internally_reserved":0,"allocated":1417216,"data_stored":650651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509500,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32576,"kb_used_data":1384,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339264,"statfs":{"total":96636764160,"available":96603406336,"internally_reserved":0,"allocated":1417216,"data_stored":650651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509502,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32576,"kb_used_data":1384,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339264,"statfs":{"total":96636764160,"available":96603406336,"internally_reserved":0,"allocated":1417216,"data_stored":650651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509502,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32592,"kb_used_data":1400,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339248,"statfs":{"total":96636764160,"available":96603389952,"internally_reserved":0,"allocated":1433600,"data_stored":656780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509502,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33152,"kb_used_data":1964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338688,"statfs":{"total":96636764160,"available":96602816512,"internally_reserved":0,"allocated":2011136,"data_stored":1234676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574909,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32556,"kb_used_data":1372,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339284,"statfs":{"total":96636764160,"available":96603426816,"internally_reserved":0,"allocated":1404928,"data_stored":644221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T00:30:28.376 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T00:30:28.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.487+0000 7f3f17932640 1 -- 172.21.15.112:0/1032352420 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f100b37f0 msgr2=0x7f3f10162ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:28.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.487+0000 7f3f17932640 1 --2- 172.21.15.112:0/1032352420 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f100b37f0 0x7f3f10162ef0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f3f100b6e90 tx=0x7f3f0c02a4a0 comp rx=0 tx=0).stop 2024-09-21T00:30:28.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.487+0000 7f3f17932640 1 -- 172.21.15.112:0/1032352420 shutdown_connections 2024-09-21T00:30:28.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.487+0000 7f3f17932640 1 --2- 172.21.15.112:0/1032352420 >> v2:172.21.15.187:3300/0 conn(0x7f3f1015f900 0x7f3f1015e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:28.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.487+0000 7f3f17932640 1 --2- 172.21.15.112:0/1032352420 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f100b37f0 0x7f3f10162ef0 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:28.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.487+0000 7f3f17932640 1 -- 172.21.15.112:0/1032352420 >> 172.21.15.112:0/1032352420 conn(0x7f3f100b6050 msgr2=0x7f3f100b6460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:28.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.487+0000 7f3f17932640 1 -- 172.21.15.112:0/1032352420 shutdown_connections 2024-09-21T00:30:28.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.487+0000 7f3f17932640 1 -- 172.21.15.112:0/1032352420 wait complete. 2024-09-21T00:30:28.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.489+0000 7f3f17932640 1 Processor -- start 2024-09-21T00:30:28.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.489+0000 7f3f17932640 1 -- start start 2024-09-21T00:30:28.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.490+0000 7f3f17932640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f100b37f0 0x7f3f10150f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:28.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.490+0000 7f3f17932640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f100cfbc0 0x7f3f100b2db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:28.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.490+0000 7f3f17932640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3f1015f900 0x7f3f101f8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:28.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.490+0000 7f3f17932640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3f1006cec0 con 0x7f3f100cfbc0 2024-09-21T00:30:28.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.490+0000 7f3f17932640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3f1006cd40 con 0x7f3f1015f900 2024-09-21T00:30:28.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.490+0000 7f3f17932640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3f1006d040 con 0x7f3f100b37f0 2024-09-21T00:30:28.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.490+0000 7f3f14ecd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f100cfbc0 0x7f3f100b2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:28.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.490+0000 7f3f14ecd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f100cfbc0 0x7f3f100b2db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56532/0 (socket says 172.21.15.112:56532) 2024-09-21T00:30:28.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.490+0000 7f3f14ecd640 1 -- 172.21.15.112:0/2528268884 learned_addr learned my addr 172.21.15.112:0/2528268884 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:30:28.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.490+0000 7f3f15ecf640 1 --2- 172.21.15.112:0/2528268884 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3f1015f900 0x7f3f101f8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:28.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.490+0000 7f3f156ce640 1 --2- 172.21.15.112:0/2528268884 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f100b37f0 0x7f3f10150f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:28.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.490+0000 7f3f14ecd640 1 -- 172.21.15.112:0/2528268884 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f100b37f0 msgr2=0x7f3f10150f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:28.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.490+0000 7f3f14ecd640 1 --2- 172.21.15.112:0/2528268884 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f100b37f0 0x7f3f10150f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:28.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.490+0000 7f3f14ecd640 1 -- 172.21.15.112:0/2528268884 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3f1015f900 msgr2=0x7f3f101f8990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:28.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.490+0000 7f3f14ecd640 1 --2- 172.21.15.112:0/2528268884 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3f1015f900 0x7f3f101f8990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:28.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.490+0000 7f3f14ecd640 1 -- 172.21.15.112:0/2528268884 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f10150500 con 0x7f3f100cfbc0 2024-09-21T00:30:28.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.490+0000 7f3f156ce640 1 --2- 172.21.15.112:0/2528268884 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f100b37f0 0x7f3f10150f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:30:28.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.490+0000 7f3f15ecf640 1 --2- 172.21.15.112:0/2528268884 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3f1015f900 0x7f3f101f8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:30:28.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.491+0000 7f3f14ecd640 1 --2- 172.21.15.112:0/2528268884 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f100cfbc0 0x7f3f100b2db0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f3f0000e790 tx=0x7f3f0000ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:28.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.491+0000 7f3f05ffb640 1 -- 172.21.15.112:0/2528268884 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f0000eea0 con 0x7f3f100cfbc0 2024-09-21T00:30:28.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.491+0000 7f3f05ffb640 1 -- 172.21.15.112:0/2528268884 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3f0000cdb0 con 0x7f3f100cfbc0 2024-09-21T00:30:28.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.491+0000 7f3f17932640 1 -- 172.21.15.112:0/2528268884 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f101f9430 con 0x7f3f100cfbc0 2024-09-21T00:30:28.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.491+0000 7f3f05ffb640 1 -- 172.21.15.112:0/2528268884 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f00010690 con 0x7f3f100cfbc0 2024-09-21T00:30:28.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.491+0000 7f3f17932640 1 -- 172.21.15.112:0/2528268884 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f10150790 con 0x7f3f100cfbc0 2024-09-21T00:30:28.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.492+0000 7f3f17932640 1 -- 172.21.15.112:0/2528268884 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f10162ef0 con 0x7f3f100cfbc0 2024-09-21T00:30:28.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.493+0000 7f3f05ffb640 1 -- 172.21.15.112:0/2528268884 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3f00010830 con 0x7f3f100cfbc0 2024-09-21T00:30:28.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.493+0000 7f3f05ffb640 1 --2- 172.21.15.112:0/2528268884 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f3ef4078bc0 0x7f3ef407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:28.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.493+0000 7f3f05ffb640 1 -- 172.21.15.112:0/2528268884 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5507+0+0 (secure 0 0 0) 0x7f3f0009b5f0 con 0x7f3f100cfbc0 2024-09-21T00:30:28.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.493+0000 7f3f156ce640 1 --2- 172.21.15.112:0/2528268884 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f3ef4078bc0 0x7f3ef407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:28.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.494+0000 7f3f156ce640 1 --2- 172.21.15.112:0/2528268884 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f3ef4078bc0 0x7f3ef407b0d0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c034000 tx=0x7f3f0c004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:28.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.498+0000 7f3f05ffb640 1 -- 172.21.15.112:0/2528268884 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3f00063b80 con 0x7f3f100cfbc0 2024-09-21T00:30:28.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.670+0000 7f3f17932640 1 -- 172.21.15.112:0/2528268884 --> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3f10075a50 con 0x7f3ef4078bc0 2024-09-21T00:30:28.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.672+0000 7f3f05ffb640 1 -- 172.21.15.112:0/2528268884 <== mgr.4103 v2:172.21.15.187:6800/2921522906 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13499 (secure 0 0 0) 0x7f3f10075a50 con 0x7f3ef4078bc0 2024-09-21T00:30:28.673 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:30:28.674 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2024-09-21T00:30:28.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.674+0000 7f3f17932640 1 -- 172.21.15.112:0/2528268884 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f3ef4078bc0 msgr2=0x7f3ef407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:28.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.674+0000 7f3f17932640 1 --2- 172.21.15.112:0/2528268884 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f3ef4078bc0 0x7f3ef407b0d0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c034000 tx=0x7f3f0c004730 comp rx=0 tx=0).stop 2024-09-21T00:30:28.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.674+0000 7f3f17932640 1 -- 172.21.15.112:0/2528268884 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f100cfbc0 msgr2=0x7f3f100b2db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:28.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.674+0000 7f3f17932640 1 --2- 172.21.15.112:0/2528268884 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f100cfbc0 0x7f3f100b2db0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f3f0000e790 tx=0x7f3f0000ec50 comp rx=0 tx=0).stop 2024-09-21T00:30:28.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.674+0000 7f3f17932640 1 -- 172.21.15.112:0/2528268884 shutdown_connections 2024-09-21T00:30:28.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.674+0000 7f3f17932640 1 --2- 172.21.15.112:0/2528268884 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3f1015f900 0x7f3f101f8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:28.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.674+0000 7f3f17932640 1 --2- 172.21.15.112:0/2528268884 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f3ef4078bc0 0x7f3ef407b0d0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:28.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.674+0000 7f3f17932640 1 --2- 172.21.15.112:0/2528268884 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f100cfbc0 0x7f3f100b2db0 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:28.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.674+0000 7f3f17932640 1 --2- 172.21.15.112:0/2528268884 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f100b37f0 0x7f3f10150f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:28.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.674+0000 7f3f17932640 1 -- 172.21.15.112:0/2528268884 >> 172.21.15.112:0/2528268884 conn(0x7f3f100b6050 msgr2=0x7f3f100d2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:28.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.674+0000 7f3f17932640 1 -- 172.21.15.112:0/2528268884 shutdown_connections 2024-09-21T00:30:28.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.674+0000 7f3f17932640 1 -- 172.21.15.112:0/2528268884 wait complete. 2024-09-21T00:30:28.686 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":255,"stamp":"2024-09-21T00:30:27.161772+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":170,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":3,"kb":754974720,"kb_used":261780,"kb_used_data":12252,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754712940,"statfs":{"total":773094113280,"available":772826050560,"internally_reserved":0,"allocated":12546048,"data_stored":6385429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-14137,"num_objects":-41,"num_object_clones":0,"num_object_copies":-82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-41,"num_whiteouts":0,"num_read":-92,"num_read_kb":-87,"num_write":-126,"num_write_kb":-54,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001280"},"pg_stats":[{"pgid":"1.0","version":"17'72","reported_seq":283,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-21T00:30:26.177047+0000","last_change":"2024-09-21T00:25:38.725792+0000","last_active":"2024-09-21T00:30:26.177047+0000","last_peered":"2024-09-21T00:30:26.177047+0000","last_clean":"2024-09-21T00:30:26.177047+0000","last_became_active":"2024-09-21T00:25:38.724668+0000","last_became_peered":"2024-09-21T00:25:38.724668+0000","last_unstale":"2024-09-21T00:30:26.177047+0000","last_undegraded":"2024-09-21T00:30:26.177047+0000","last_fullsized":"2024-09-21T00:30:26.177047+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:37.163216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:37.163216+0000","last_clean_scrub_stamp":"2024-09-21T00:25:37.163216+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:14:36.588660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509501,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33176,"kb_used_data":1980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338664,"statfs":{"total":96636764160,"available":96602791936,"internally_reserved":0,"allocated":2027520,"data_stored":1247148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509502,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32576,"kb_used_data":1384,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339264,"statfs":{"total":96636764160,"available":96603406336,"internally_reserved":0,"allocated":1417216,"data_stored":650651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509501,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32576,"kb_used_data":1384,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339264,"statfs":{"total":96636764160,"available":96603406336,"internally_reserved":0,"allocated":1417216,"data_stored":650651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509500,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32576,"kb_used_data":1384,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339264,"statfs":{"total":96636764160,"available":96603406336,"internally_reserved":0,"allocated":1417216,"data_stored":650651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509502,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32576,"kb_used_data":1384,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339264,"statfs":{"total":96636764160,"available":96603406336,"internally_reserved":0,"allocated":1417216,"data_stored":650651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509502,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32592,"kb_used_data":1400,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339248,"statfs":{"total":96636764160,"available":96603389952,"internally_reserved":0,"allocated":1433600,"data_stored":656780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509502,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33152,"kb_used_data":1964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338688,"statfs":{"total":96636764160,"available":96602816512,"internally_reserved":0,"allocated":2011136,"data_stored":1234676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574909,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32556,"kb_used_data":1372,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339284,"statfs":{"total":96636764160,"available":96603426816,"internally_reserved":0,"allocated":1404928,"data_stored":644221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T00:30:28.687 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T00:30:28.687 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T00:30:28.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.796+0000 7f12a107e640 1 -- 172.21.15.112:0/2040812993 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f129c0ce600 msgr2=0x7f129c155a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:28.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.796+0000 7f12a107e640 1 --2- 172.21.15.112:0/2040812993 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f129c0ce600 0x7f129c155a20 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f129c0ce5b0 tx=0x7f128c02a530 comp rx=0 tx=0).stop 2024-09-21T00:30:28.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.797+0000 7f12a107e640 1 -- 172.21.15.112:0/2040812993 shutdown_connections 2024-09-21T00:30:28.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.797+0000 7f12a107e640 1 --2- 172.21.15.112:0/2040812993 >> v2:172.21.15.187:3300/0 conn(0x7f129c0cb080 0x7f129c0af520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:28.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.797+0000 7f12a107e640 1 --2- 172.21.15.112:0/2040812993 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f129c0ce600 0x7f129c155a20 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:28.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.797+0000 7f12a107e640 1 -- 172.21.15.112:0/2040812993 >> 172.21.15.112:0/2040812993 conn(0x7f129c0cd4b0 msgr2=0x7f129c0cd8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:28.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.797+0000 7f12a107e640 1 -- 172.21.15.112:0/2040812993 shutdown_connections 2024-09-21T00:30:28.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.797+0000 7f12a107e640 1 -- 172.21.15.112:0/2040812993 wait complete. 2024-09-21T00:30:28.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.798+0000 7f12a107e640 1 Processor -- start 2024-09-21T00:30:28.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.799+0000 7f12a107e640 1 -- start start 2024-09-21T00:30:28.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.800+0000 7f12a107e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f129c0afa60 0x7f129c1544a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:28.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.800+0000 7f12a107e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f129c0cb080 0x7f129c06f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:28.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.800+0000 7f12a107e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f129c0ce600 0x7f129c06fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:28.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.800+0000 7f12a107e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f129c15c320 con 0x7f129c0cb080 2024-09-21T00:30:28.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.800+0000 7f12a107e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f129c0b01f0 con 0x7f129c0ce600 2024-09-21T00:30:28.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.800+0000 7f12a107e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f129c15c4a0 con 0x7f129c0afa60 2024-09-21T00:30:28.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.800+0000 7f129259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f129c0cb080 0x7f129c06f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:28.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.800+0000 7f129259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f129c0cb080 0x7f129c06f4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56548/0 (socket says 172.21.15.112:56548) 2024-09-21T00:30:28.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.800+0000 7f129259c640 1 -- 172.21.15.112:0/1297862767 learned_addr learned my addr 172.21.15.112:0/1297862767 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:30:28.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.800+0000 7f129ad9d640 1 --2- 172.21.15.112:0/1297862767 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f129c0afa60 0x7f129c1544a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:28.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.800+0000 7f129b59e640 1 --2- 172.21.15.112:0/1297862767 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f129c0ce600 0x7f129c06fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:28.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.800+0000 7f129259c640 1 -- 172.21.15.112:0/1297862767 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f129c0afa60 msgr2=0x7f129c1544a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:28.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.800+0000 7f129259c640 1 --2- 172.21.15.112:0/1297862767 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f129c0afa60 0x7f129c1544a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:28.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.800+0000 7f129259c640 1 -- 172.21.15.112:0/1297862767 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f129c0ce600 msgr2=0x7f129c06fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:28.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.800+0000 7f129259c640 1 --2- 172.21.15.112:0/1297862767 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f129c0ce600 0x7f129c06fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:28.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.800+0000 7f129259c640 1 -- 172.21.15.112:0/1297862767 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f129c1f7730 con 0x7f129c0cb080 2024-09-21T00:30:28.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.800+0000 7f129b59e640 1 --2- 172.21.15.112:0/1297862767 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f129c0ce600 0x7f129c06fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:30:28.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.800+0000 7f129259c640 1 --2- 172.21.15.112:0/1297862767 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f129c0cb080 0x7f129c06f4e0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f128400d740 tx=0x7f128400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:28.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.801+0000 7f1293fff640 1 -- 172.21.15.112:0/1297862767 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1284017070 con 0x7f129c0cb080 2024-09-21T00:30:28.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.801+0000 7f1293fff640 1 -- 172.21.15.112:0/1297862767 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1284009d20 con 0x7f129c0cb080 2024-09-21T00:30:28.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.801+0000 7f12a107e640 1 -- 172.21.15.112:0/1297862767 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f129c0c8c10 con 0x7f129c0cb080 2024-09-21T00:30:28.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.801+0000 7f1293fff640 1 -- 172.21.15.112:0/1297862767 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12840106c0 con 0x7f129c0cb080 2024-09-21T00:30:28.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.801+0000 7f12a107e640 1 -- 172.21.15.112:0/1297862767 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f129c0c8eb0 con 0x7f129c0cb080 2024-09-21T00:30:28.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.802+0000 7f12a107e640 1 -- 172.21.15.112:0/1297862767 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f129c15db40 con 0x7f129c0cb080 2024-09-21T00:30:28.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.803+0000 7f1293fff640 1 -- 172.21.15.112:0/1297862767 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f128401a070 con 0x7f129c0cb080 2024-09-21T00:30:28.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.803+0000 7f1293fff640 1 --2- 172.21.15.112:0/1297862767 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f127c078bc0 0x7f127c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:28.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.803+0000 7f1293fff640 1 -- 172.21.15.112:0/1297862767 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5507+0+0 (secure 0 0 0) 0x7f128409ace0 con 0x7f129c0cb080 2024-09-21T00:30:28.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.803+0000 7f129ad9d640 1 --2- 172.21.15.112:0/1297862767 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f127c078bc0 0x7f127c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:28.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.804+0000 7f129ad9d640 1 --2- 172.21.15.112:0/1297862767 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f127c078bc0 0x7f127c07b0d0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f128c034000 tx=0x7f128c004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:28.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.807+0000 7f1293fff640 1 -- 172.21.15.112:0/1297862767 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1284062c20 con 0x7f129c0cb080 2024-09-21T00:30:28.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.981+0000 7f12a107e640 1 -- 172.21.15.112:0/1297862767 --> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f129c1f7200 con 0x7f127c078bc0 2024-09-21T00:30:28.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.982+0000 7f1293fff640 1 -- 172.21.15.112:0/1297862767 <== mgr.4103 v2:172.21.15.187:6800/2921522906 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13499 (secure 0 0 0) 0x7f129c1f7200 con 0x7f127c078bc0 2024-09-21T00:30:28.984 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:30:28.984 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2024-09-21T00:30:28.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.984+0000 7f12a107e640 1 -- 172.21.15.112:0/1297862767 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f127c078bc0 msgr2=0x7f127c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:28.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.984+0000 7f12a107e640 1 --2- 172.21.15.112:0/1297862767 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f127c078bc0 0x7f127c07b0d0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f128c034000 tx=0x7f128c004730 comp rx=0 tx=0).stop 2024-09-21T00:30:28.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.984+0000 7f12a107e640 1 -- 172.21.15.112:0/1297862767 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f129c0cb080 msgr2=0x7f129c06f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:28.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.984+0000 7f12a107e640 1 --2- 172.21.15.112:0/1297862767 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f129c0cb080 0x7f129c06f4e0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f128400d740 tx=0x7f128400dc00 comp rx=0 tx=0).stop 2024-09-21T00:30:28.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.984+0000 7f12a107e640 1 -- 172.21.15.112:0/1297862767 shutdown_connections 2024-09-21T00:30:28.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.984+0000 7f12a107e640 1 --2- 172.21.15.112:0/1297862767 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f129c0ce600 0x7f129c06fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:28.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.984+0000 7f12a107e640 1 --2- 172.21.15.112:0/1297862767 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f127c078bc0 0x7f127c07b0d0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:28.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.984+0000 7f12a107e640 1 --2- 172.21.15.112:0/1297862767 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f129c0cb080 0x7f129c06f4e0 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:28.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.984+0000 7f12a107e640 1 --2- 172.21.15.112:0/1297862767 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f129c0afa60 0x7f129c1544a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:28.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.984+0000 7f12a107e640 1 -- 172.21.15.112:0/1297862767 >> 172.21.15.112:0/1297862767 conn(0x7f129c0cd4b0 msgr2=0x7f129c0b4ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:28.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.984+0000 7f12a107e640 1 -- 172.21.15.112:0/1297862767 shutdown_connections 2024-09-21T00:30:28.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:28.984+0000 7f12a107e640 1 -- 172.21.15.112:0/1297862767 wait complete. 2024-09-21T00:30:28.997 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":255,"stamp":"2024-09-21T00:30:27.161772+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":170,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":3,"kb":754974720,"kb_used":261780,"kb_used_data":12252,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754712940,"statfs":{"total":773094113280,"available":772826050560,"internally_reserved":0,"allocated":12546048,"data_stored":6385429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-14137,"num_objects":-41,"num_object_clones":0,"num_object_copies":-82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-41,"num_whiteouts":0,"num_read":-92,"num_read_kb":-87,"num_write":-126,"num_write_kb":-54,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001280"},"pg_stats":[{"pgid":"1.0","version":"17'72","reported_seq":283,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-21T00:30:26.177047+0000","last_change":"2024-09-21T00:25:38.725792+0000","last_active":"2024-09-21T00:30:26.177047+0000","last_peered":"2024-09-21T00:30:26.177047+0000","last_clean":"2024-09-21T00:30:26.177047+0000","last_became_active":"2024-09-21T00:25:38.724668+0000","last_became_peered":"2024-09-21T00:25:38.724668+0000","last_unstale":"2024-09-21T00:30:26.177047+0000","last_undegraded":"2024-09-21T00:30:26.177047+0000","last_fullsized":"2024-09-21T00:30:26.177047+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:37.163216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:37.163216+0000","last_clean_scrub_stamp":"2024-09-21T00:25:37.163216+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:14:36.588660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509501,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33176,"kb_used_data":1980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338664,"statfs":{"total":96636764160,"available":96602791936,"internally_reserved":0,"allocated":2027520,"data_stored":1247148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509502,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32576,"kb_used_data":1384,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339264,"statfs":{"total":96636764160,"available":96603406336,"internally_reserved":0,"allocated":1417216,"data_stored":650651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509501,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32576,"kb_used_data":1384,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339264,"statfs":{"total":96636764160,"available":96603406336,"internally_reserved":0,"allocated":1417216,"data_stored":650651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509500,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32576,"kb_used_data":1384,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339264,"statfs":{"total":96636764160,"available":96603406336,"internally_reserved":0,"allocated":1417216,"data_stored":650651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509502,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32576,"kb_used_data":1384,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339264,"statfs":{"total":96636764160,"available":96603406336,"internally_reserved":0,"allocated":1417216,"data_stored":650651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509502,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32592,"kb_used_data":1400,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339248,"statfs":{"total":96636764160,"available":96603389952,"internally_reserved":0,"allocated":1433600,"data_stored":656780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509502,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33152,"kb_used_data":1964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338688,"statfs":{"total":96636764160,"available":96602816512,"internally_reserved":0,"allocated":2011136,"data_stored":1234676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574909,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32556,"kb_used_data":1372,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339284,"statfs":{"total":96636764160,"available":96603426816,"internally_reserved":0,"allocated":1404928,"data_stored":644221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T00:30:28.997 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T00:30:29.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:29.109+0000 7f27875f1640 1 -- 172.21.15.112:0/2778799603 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f276c004800 msgr2=0x7f276c004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:29.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:29.109+0000 7f27875f1640 1 --2- 172.21.15.112:0/2778799603 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f276c004800 0x7f276c004c10 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f277000c3f0 tx=0x7f276c005830 comp rx=0 tx=0).stop 2024-09-21T00:30:29.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:29.109+0000 7f27875f1640 1 -- 172.21.15.112:0/2778799603 shutdown_connections 2024-09-21T00:30:29.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:29.109+0000 7f27875f1640 1 --2- 172.21.15.112:0/2778799603 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f276c004800 0x7f276c004c10 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:29.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:29.109+0000 7f27875f1640 1 -- 172.21.15.112:0/2778799603 >> 172.21.15.112:0/2778799603 conn(0x7f27800cf400 msgr2=0x7f27800d2350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:29.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:29.109+0000 7f27875f1640 1 -- 172.21.15.112:0/2778799603 shutdown_connections 2024-09-21T00:30:29.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:29.109+0000 7f27875f1640 1 -- 172.21.15.112:0/2778799603 wait complete. 2024-09-21T00:30:29.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:29.111+0000 7f27875f1640 1 Processor -- start 2024-09-21T00:30:29.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:29.111+0000 7f27875f1640 1 -- start start 2024-09-21T00:30:29.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:29.112+0000 7f27875f1640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f276c004800 0x7f27800cb630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:29.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:29.112+0000 7f27875f1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f27800cbb70 0x7f27801f6e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:29.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:29.112+0000 7f27875f1640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f27801f73b0 0x7f27801f77c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:29.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:29.112+0000 7f27875f1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f27800ccbb0 con 0x7f27800cbb70 2024-09-21T00:30:29.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:29.112+0000 7f27875f1640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f2780160330 con 0x7f27801f73b0 2024-09-21T00:30:29.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:29.112+0000 7f27875f1640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f27800ccd30 con 0x7f276c004800 2024-09-21T00:30:29.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:29.112+0000 7f2784b8c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f27800cbb70 0x7f27801f6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:29.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:29.112+0000 7f2784b8c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f27800cbb70 0x7f27801f6e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56552/0 (socket says 172.21.15.112:56552) 2024-09-21T00:30:29.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:29.112+0000 7f2784b8c640 1 -- 172.21.15.112:0/2305172208 learned_addr learned my addr 172.21.15.112:0/2305172208 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:30:29.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:29.112+0000 7f278538d640 1 --2- 172.21.15.112:0/2305172208 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f276c004800 0x7f27800cb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:29.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:29.112+0000 7f2785b8e640 1 --2- 172.21.15.112:0/2305172208 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f27801f73b0 0x7f27801f77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:29.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:29.113+0000 7f2784b8c640 1 -- 172.21.15.112:0/2305172208 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f276c004800 msgr2=0x7f27800cb630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:29.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:29.113+0000 7f2784b8c640 1 --2- 172.21.15.112:0/2305172208 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f276c004800 0x7f27800cb630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:29.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:29.113+0000 7f2784b8c640 1 -- 172.21.15.112:0/2305172208 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f27801f73b0 msgr2=0x7f27801f77c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:29.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:29.113+0000 7f2784b8c640 1 --2- 172.21.15.112:0/2305172208 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f27801f73b0 0x7f27801f77c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:29.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:29.113+0000 7f2784b8c640 1 -- 172.21.15.112:0/2305172208 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27800cbf80 con 0x7f27800cbb70 2024-09-21T00:30:29.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:29.113+0000 7f2784b8c640 1 --2- 172.21.15.112:0/2305172208 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f27800cbb70 0x7f27801f6e70 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f277000c4e0 tx=0x7f2770006060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:29.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:29.113+0000 7f2775ffb640 1 -- 172.21.15.112:0/2305172208 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2770039070 con 0x7f27800cbb70 2024-09-21T00:30:29.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:29.113+0000 7f2775ffb640 1 -- 172.21.15.112:0/2305172208 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f277002b5f0 con 0x7f27800cbb70 2024-09-21T00:30:29.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:29.113+0000 7f27875f1640 1 -- 172.21.15.112:0/2305172208 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f278014eab0 con 0x7f27800cbb70 2024-09-21T00:30:29.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:29.113+0000 7f2775ffb640 1 -- 172.21.15.112:0/2305172208 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f277002ba20 con 0x7f27800cbb70 2024-09-21T00:30:29.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:29.113+0000 7f27875f1640 1 -- 172.21.15.112:0/2305172208 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f278014f070 con 0x7f27800cbb70 2024-09-21T00:30:29.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:29.114+0000 7f27875f1640 1 -- 172.21.15.112:0/2305172208 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f278014e820 con 0x7f27800cbb70 2024-09-21T00:30:29.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:29.115+0000 7f2775ffb640 1 -- 172.21.15.112:0/2305172208 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2770044020 con 0x7f27800cbb70 2024-09-21T00:30:29.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:29.115+0000 7f2775ffb640 1 --2- 172.21.15.112:0/2305172208 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2764078b90 0x7f276407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:29.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:29.116+0000 7f2775ffb640 1 -- 172.21.15.112:0/2305172208 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5507+0+0 (secure 0 0 0) 0x7f27700ba590 con 0x7f27800cbb70 2024-09-21T00:30:29.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:29.116+0000 7f278538d640 1 --2- 172.21.15.112:0/2305172208 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2764078b90 0x7f276407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:29.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:29.116+0000 7f278538d640 1 --2- 172.21.15.112:0/2305172208 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2764078b90 0x7f276407b0a0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f277c009cd0 tx=0x7f277c005f90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:29.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:29.120+0000 7f2775ffb640 1 -- 172.21.15.112:0/2305172208 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f27800cbf80 con 0x7f27800cbb70 2024-09-21T00:30:29.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:29.299+0000 7f27875f1640 1 -- 172.21.15.112:0/2305172208 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f27800d06d0 con 0x7f27800cbb70 2024-09-21T00:30:29.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:29.300+0000 7f2775ffb640 1 -- 172.21.15.112:0/2305172208 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v115) ==== 74+0+13605 (secure 0 0 0) 0x7f27800cd500 con 0x7f27800cbb70 2024-09-21T00:30:29.302 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:30:29.302 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":115,"fsid":"8b34692b-dc65-49b2-a08f-26f707b96ac5","created":"2024-09-21T00:25:22.788490+0000","modified":"2024-09-21T00:30:26.145621+0000","last_up_change":"2024-09-21T00:25:36.161835+0000","last_in_change":"2024-09-21T00:25:26.346500+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T00:25:37.105006+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9c575086-67d2-4de2-b01b-d44dcc21d604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6809","nonce":755760621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6811","nonce":755760621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6815","nonce":755760621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":755760621},{"type":"v1","addr":"172.21.15.112:6813","nonce":755760621}]},"public_addr":"172.21.15.112:6809/755760621","cluster_addr":"172.21.15.112:6811/755760621","heartbeat_back_addr":"172.21.15.112:6815/755760621","heartbeat_front_addr":"172.21.15.112:6813/755760621","state":["exists","up"]},{"osd":1,"uuid":"a983c227-a3ce-4bdd-82f1-438ba7e3ecd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6801","nonce":344932592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6803","nonce":344932592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6807","nonce":344932592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":344932592},{"type":"v1","addr":"172.21.15.112:6805","nonce":344932592}]},"public_addr":"172.21.15.112:6801/344932592","cluster_addr":"172.21.15.112:6803/344932592","heartbeat_back_addr":"172.21.15.112:6807/344932592","heartbeat_front_addr":"172.21.15.112:6805/344932592","state":["exists","up"]},{"osd":2,"uuid":"1193c1d8-bfff-4a57-8200-5f69bf1ea3af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6825","nonce":407933616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6827","nonce":407933616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6831","nonce":407933616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":407933616},{"type":"v1","addr":"172.21.15.112:6829","nonce":407933616}]},"public_addr":"172.21.15.112:6825/407933616","cluster_addr":"172.21.15.112:6827/407933616","heartbeat_back_addr":"172.21.15.112:6831/407933616","heartbeat_front_addr":"172.21.15.112:6829/407933616","state":["exists","up"]},{"osd":3,"uuid":"af04330a-36ad-40cc-a8dd-fa259b221de5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6817","nonce":4120164806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6819","nonce":4120164806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6823","nonce":4120164806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":4120164806},{"type":"v1","addr":"172.21.15.112:6821","nonce":4120164806}]},"public_addr":"172.21.15.112:6817/4120164806","cluster_addr":"172.21.15.112:6819/4120164806","heartbeat_back_addr":"172.21.15.112:6823/4120164806","heartbeat_front_addr":"172.21.15.112:6821/4120164806","state":["exists","up"]},{"osd":4,"uuid":"5b893111-292c-4c5a-b184-53ad38c7199e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6803","nonce":522077560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6805","nonce":522077560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6809","nonce":522077560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":522077560},{"type":"v1","addr":"172.21.15.187:6807","nonce":522077560}]},"public_addr":"172.21.15.187:6803/522077560","cluster_addr":"172.21.15.187:6805/522077560","heartbeat_back_addr":"172.21.15.187:6809/522077560","heartbeat_front_addr":"172.21.15.187:6807/522077560","state":["exists","up"]},{"osd":5,"uuid":"c0b8e011-45fe-4c5e-925e-6be2a7f1868a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6827","nonce":3593144721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6829","nonce":3593144721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6833","nonce":3593144721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":3593144721},{"type":"v1","addr":"172.21.15.187:6831","nonce":3593144721}]},"public_addr":"172.21.15.187:6827/3593144721","cluster_addr":"172.21.15.187:6829/3593144721","heartbeat_back_addr":"172.21.15.187:6833/3593144721","heartbeat_front_addr":"172.21.15.187:6831/3593144721","state":["exists","up"]},{"osd":6,"uuid":"87d2d650-7489-4593-98a3-a1f433ce8dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6811","nonce":1494387657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6813","nonce":1494387657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6817","nonce":1494387657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":1494387657},{"type":"v1","addr":"172.21.15.187:6815","nonce":1494387657}]},"public_addr":"172.21.15.187:6811/1494387657","cluster_addr":"172.21.15.187:6813/1494387657","heartbeat_back_addr":"172.21.15.187:6817/1494387657","heartbeat_front_addr":"172.21.15.187:6815/1494387657","state":["exists","up"]},{"osd":7,"uuid":"c6b4f8e2-a1ee-4438-aa37-67ff4b6ef1f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6819","nonce":1624699842}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6821","nonce":1624699842}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6825","nonce":1624699842}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":1624699842},{"type":"v1","addr":"172.21.15.187:6823","nonce":1624699842}]},"public_addr":"172.21.15.187:6819/1624699842","cluster_addr":"172.21.15.187:6821/1624699842","heartbeat_back_addr":"172.21.15.187:6825/1624699842","heartbeat_front_addr":"172.21.15.187:6823/1624699842","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.873162+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.400506+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.664317+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.159791+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.202630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.113212+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:32.384786+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T00:25:33.448441+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:6834/3291181711":"2024-09-22T00:30:23.136571+0000","172.21.15.112:6832/3291181711":"2024-09-22T00:30:23.136571+0000","172.21.15.112:6835/3392867613":"2024-09-22T00:30:23.136588+0000","172.21.15.112:6833/3392867613":"2024-09-22T00:30:23.136588+0000","172.21.15.187:6837/2071643267":"2024-09-22T00:30:02.243502+0000","172.21.15.187:6835/2071643267":"2024-09-22T00:30:02.243502+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T00:30:29.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:29.302+0000 7f27875f1640 1 -- 172.21.15.112:0/2305172208 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2764078b90 msgr2=0x7f276407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:29.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:29.302+0000 7f27875f1640 1 --2- 172.21.15.112:0/2305172208 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2764078b90 0x7f276407b0a0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f277c009cd0 tx=0x7f277c005f90 comp rx=0 tx=0).stop 2024-09-21T00:30:29.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:29.302+0000 7f27875f1640 1 -- 172.21.15.112:0/2305172208 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f27800cbb70 msgr2=0x7f27801f6e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:29.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:29.302+0000 7f27875f1640 1 --2- 172.21.15.112:0/2305172208 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f27800cbb70 0x7f27801f6e70 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f277000c4e0 tx=0x7f2770006060 comp rx=0 tx=0).stop 2024-09-21T00:30:29.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:29.302+0000 7f27875f1640 1 -- 172.21.15.112:0/2305172208 shutdown_connections 2024-09-21T00:30:29.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:29.302+0000 7f27875f1640 1 --2- 172.21.15.112:0/2305172208 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f27801f73b0 0x7f27801f77c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:29.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:29.302+0000 7f27875f1640 1 --2- 172.21.15.112:0/2305172208 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f2764078b90 0x7f276407b0a0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:29.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:29.302+0000 7f27875f1640 1 --2- 172.21.15.112:0/2305172208 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f27800cbb70 0x7f27801f6e70 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:29.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:29.302+0000 7f27875f1640 1 --2- 172.21.15.112:0/2305172208 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f276c004800 0x7f27800cb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:29.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:29.302+0000 7f27875f1640 1 -- 172.21.15.112:0/2305172208 >> 172.21.15.112:0/2305172208 conn(0x7f27800cf400 msgr2=0x7f27800ca410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:29.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:29.302+0000 7f27875f1640 1 -- 172.21.15.112:0/2305172208 shutdown_connections 2024-09-21T00:30:29.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:29.302+0000 7f27875f1640 1 -- 172.21.15.112:0/2305172208 wait complete. 2024-09-21T00:30:30.315 INFO:tasks.ceph:Scrubbing osd.0 2024-09-21T00:30:30.316 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-21T00:30:30.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.444+0000 7f5debb20640 1 -- 172.21.15.112:0/3923324670 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5de40d17b0 msgr2=0x7f5de40cd070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:30.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.444+0000 7f5debb20640 1 --2- 172.21.15.112:0/3923324670 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5de40d17b0 0x7f5de40cd070 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f5de40b38a0 tx=0x7f5dd402a5f0 comp rx=0 tx=0).stop 2024-09-21T00:30:30.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.444+0000 7f5debb20640 1 -- 172.21.15.112:0/3923324670 shutdown_connections 2024-09-21T00:30:30.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.444+0000 7f5debb20640 1 --2- 172.21.15.112:0/3923324670 >> v2:172.21.15.187:3300/0 conn(0x7f5de40b7ea0 0x7f5de4159f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:30.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.444+0000 7f5debb20640 1 --2- 172.21.15.112:0/3923324670 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5de40d17b0 0x7f5de40cd070 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:30.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.444+0000 7f5debb20640 1 --2- 172.21.15.112:0/3923324670 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5de415da90 0x7f5de40ccb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:30.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.444+0000 7f5debb20640 1 -- 172.21.15.112:0/3923324670 >> 172.21.15.112:0/3923324670 conn(0x7f5de40d29f0 msgr2=0x7f5de40cede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:30.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.444+0000 7f5debb20640 1 -- 172.21.15.112:0/3923324670 shutdown_connections 2024-09-21T00:30:30.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.444+0000 7f5debb20640 1 -- 172.21.15.112:0/3923324670 wait complete. 2024-09-21T00:30:30.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.446+0000 7f5debb20640 1 Processor -- start 2024-09-21T00:30:30.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.446+0000 7f5debb20640 1 -- start start 2024-09-21T00:30:30.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.447+0000 7f5debb20640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5de40b7ea0 0x7f5de41f58d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:30.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.447+0000 7f5debb20640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5de41537b0 0x7f5de4153100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:30.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.447+0000 7f5debb20640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5de40ca170 0x7f5de40ca580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:30.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.447+0000 7f5debb20640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5de415a4d0 con 0x7f5de40b7ea0 2024-09-21T00:30:30.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.447+0000 7f5debb20640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5de40d04c0 con 0x7f5de40ca170 2024-09-21T00:30:30.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.447+0000 7f5debb20640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f5de415a650 con 0x7f5de41537b0 2024-09-21T00:30:30.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.447+0000 7f5de98bc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5de40b7ea0 0x7f5de41f58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:30.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.447+0000 7f5de98bc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5de40b7ea0 0x7f5de41f58d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56560/0 (socket says 172.21.15.112:56560) 2024-09-21T00:30:30.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.447+0000 7f5de98bc640 1 -- 172.21.15.112:0/2232124294 learned_addr learned my addr 172.21.15.112:0/2232124294 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:30:30.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.447+0000 7f5de90bb640 1 --2- 172.21.15.112:0/2232124294 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5de41537b0 0x7f5de4153100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:30.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.447+0000 7f5dea0bd640 1 --2- 172.21.15.112:0/2232124294 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5de40ca170 0x7f5de40ca580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:30.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.447+0000 7f5de98bc640 1 -- 172.21.15.112:0/2232124294 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5de41537b0 msgr2=0x7f5de4153100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:30.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.447+0000 7f5de98bc640 1 --2- 172.21.15.112:0/2232124294 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5de41537b0 0x7f5de4153100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:30.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.447+0000 7f5de98bc640 1 -- 172.21.15.112:0/2232124294 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5de40ca170 msgr2=0x7f5de40ca580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:30.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.447+0000 7f5de98bc640 1 --2- 172.21.15.112:0/2232124294 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5de40ca170 0x7f5de40ca580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:30.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.447+0000 7f5de98bc640 1 -- 172.21.15.112:0/2232124294 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5de40caac0 con 0x7f5de40b7ea0 2024-09-21T00:30:30.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.448+0000 7f5de98bc640 1 --2- 172.21.15.112:0/2232124294 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5de40b7ea0 0x7f5de41f58d0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f5de000d700 tx=0x7f5de000dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:30.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.448+0000 7f5dda7fc640 1 -- 172.21.15.112:0/2232124294 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de0017070 con 0x7f5de40b7ea0 2024-09-21T00:30:30.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.448+0000 7f5dda7fc640 1 -- 172.21.15.112:0/2232124294 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5de0009d20 con 0x7f5de40b7ea0 2024-09-21T00:30:30.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.448+0000 7f5debb20640 1 -- 172.21.15.112:0/2232124294 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5de40cc300 con 0x7f5de40b7ea0 2024-09-21T00:30:30.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.448+0000 7f5dda7fc640 1 -- 172.21.15.112:0/2232124294 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de00106c0 con 0x7f5de40b7ea0 2024-09-21T00:30:30.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.448+0000 7f5debb20640 1 -- 172.21.15.112:0/2232124294 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5de40b6ac0 con 0x7f5de40b7ea0 2024-09-21T00:30:30.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.449+0000 7f5debb20640 1 -- 172.21.15.112:0/2232124294 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5de40d17b0 con 0x7f5de40b7ea0 2024-09-21T00:30:30.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.449+0000 7f5dda7fc640 1 -- 172.21.15.112:0/2232124294 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5de001a070 con 0x7f5de40b7ea0 2024-09-21T00:30:30.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.450+0000 7f5dda7fc640 1 --2- 172.21.15.112:0/2232124294 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f5dc8078b90 0x7f5dc807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:30.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.450+0000 7f5dda7fc640 1 -- 172.21.15.112:0/2232124294 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5507+0+0 (secure 0 0 0) 0x7f5de009b320 con 0x7f5de40b7ea0 2024-09-21T00:30:30.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.450+0000 7f5de90bb640 1 --2- 172.21.15.112:0/2232124294 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f5dc8078b90 0x7f5dc807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:30.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.450+0000 7f5dda7fc640 1 --2- 172.21.15.112:0/2232124294 >> [v2:172.21.15.112:6808/755760621,v1:172.21.15.112:6809/755760621] conn(0x7f5dc807f1c0 0x7f5dc8081650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:30.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.450+0000 7f5dda7fc640 1 -- 172.21.15.112:0/2232124294 --> [v2:172.21.15.112:6808/755760621,v1:172.21.15.112:6809/755760621] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5de0026e50 con 0x7f5dc807f1c0 2024-09-21T00:30:30.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.450+0000 7f5dda7fc640 1 -- 172.21.15.112:0/2232124294 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_get_version_reply(handle=1 version=115) ==== 24+0+0 (secure 0 0 0) 0x7f5de009b7b0 con 0x7f5de40b7ea0 2024-09-21T00:30:30.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.450+0000 7f5dea0bd640 1 --2- 172.21.15.112:0/2232124294 >> [v2:172.21.15.112:6808/755760621,v1:172.21.15.112:6809/755760621] conn(0x7f5dc807f1c0 0x7f5dc8081650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:30.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.451+0000 7f5dea0bd640 1 --2- 172.21.15.112:0/2232124294 >> [v2:172.21.15.112:6808/755760621,v1:172.21.15.112:6809/755760621] conn(0x7f5dc807f1c0 0x7f5dc8081650 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:30.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.451+0000 7f5de90bb640 1 --2- 172.21.15.112:0/2232124294 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f5dc8078b90 0x7f5dc807b0a0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f5dd4033000 tx=0x7f5dd4004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:30.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.451+0000 7f5dda7fc640 1 -- 172.21.15.112:0/2232124294 <== osd.0 v2:172.21.15.112:6808/755760621 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5de0026e50 con 0x7f5dc807f1c0 2024-09-21T00:30:30.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.464+0000 7f5debb20640 1 -- 172.21.15.112:0/2232124294 --> [v2:172.21.15.112:6808/755760621,v1:172.21.15.112:6809/755760621] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f5de40b82e0 con 0x7f5dc807f1c0 2024-09-21T00:30:30.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.464+0000 7f5dda7fc640 1 -- 172.21.15.112:0/2232124294 <== osd.0 v2:172.21.15.112:6808/755760621 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f5de40b82e0 con 0x7f5dc807f1c0 2024-09-21T00:30:30.466 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-21T00:30:30.466 INFO:teuthology.orchestra.run.smithi112.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T00:30:30.466 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-21T00:30:30.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.464+0000 7f5debb20640 1 -- 172.21.15.112:0/2232124294 >> [v2:172.21.15.112:6808/755760621,v1:172.21.15.112:6809/755760621] conn(0x7f5dc807f1c0 msgr2=0x7f5dc8081650 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:30.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.464+0000 7f5debb20640 1 --2- 172.21.15.112:0/2232124294 >> [v2:172.21.15.112:6808/755760621,v1:172.21.15.112:6809/755760621] conn(0x7f5dc807f1c0 0x7f5dc8081650 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:30.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.465+0000 7f5debb20640 1 -- 172.21.15.112:0/2232124294 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f5dc8078b90 msgr2=0x7f5dc807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:30.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.465+0000 7f5debb20640 1 --2- 172.21.15.112:0/2232124294 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f5dc8078b90 0x7f5dc807b0a0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f5dd4033000 tx=0x7f5dd4004730 comp rx=0 tx=0).stop 2024-09-21T00:30:30.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.465+0000 7f5debb20640 1 -- 172.21.15.112:0/2232124294 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5de40b7ea0 msgr2=0x7f5de41f58d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:30.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.465+0000 7f5debb20640 1 --2- 172.21.15.112:0/2232124294 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5de40b7ea0 0x7f5de41f58d0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f5de000d700 tx=0x7f5de000dbc0 comp rx=0 tx=0).stop 2024-09-21T00:30:30.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.465+0000 7f5dea0bd640 1 -- 172.21.15.112:0/2232124294 reap_dead start 2024-09-21T00:30:30.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.465+0000 7f5debb20640 1 -- 172.21.15.112:0/2232124294 shutdown_connections 2024-09-21T00:30:30.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.465+0000 7f5debb20640 1 -- 172.21.15.112:0/2232124294 >> 172.21.15.112:0/2232124294 conn(0x7f5de40d29f0 msgr2=0x7f5de41598b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:30.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.465+0000 7f5debb20640 1 -- 172.21.15.112:0/2232124294 shutdown_connections 2024-09-21T00:30:30.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.465+0000 7f5debb20640 1 -- 172.21.15.112:0/2232124294 wait complete. 2024-09-21T00:30:30.477 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-21T00:30:30.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.588+0000 7fa1f6f8b640 1 -- 172.21.15.112:0/4137095863 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1f00cfbc0 msgr2=0x7fa1f00cffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:30.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.588+0000 7fa1f6f8b640 1 --2- 172.21.15.112:0/4137095863 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1f00cfbc0 0x7fa1f00cffd0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7fa1f00b6e90 tx=0x7fa1e802a980 comp rx=0 tx=0).stop 2024-09-21T00:30:30.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.588+0000 7fa1f6f8b640 1 -- 172.21.15.112:0/4137095863 shutdown_connections 2024-09-21T00:30:30.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.588+0000 7fa1f6f8b640 1 --2- 172.21.15.112:0/4137095863 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1f00cfbc0 0x7fa1f00cffd0 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:30.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.588+0000 7fa1f6f8b640 1 --2- 172.21.15.112:0/4137095863 >> v2:172.21.15.187:3300/0 conn(0x7fa1f015f900 0x7fa1f015e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:30.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.588+0000 7fa1f6f8b640 1 -- 172.21.15.112:0/4137095863 >> 172.21.15.112:0/4137095863 conn(0x7fa1f00b6050 msgr2=0x7fa1f00b6460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:30.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.588+0000 7fa1f6f8b640 1 -- 172.21.15.112:0/4137095863 shutdown_connections 2024-09-21T00:30:30.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.588+0000 7fa1f6f8b640 1 -- 172.21.15.112:0/4137095863 wait complete. 2024-09-21T00:30:30.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.590+0000 7fa1f6f8b640 1 Processor -- start 2024-09-21T00:30:30.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.590+0000 7fa1f6f8b640 1 -- start start 2024-09-21T00:30:30.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.591+0000 7fa1f6f8b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa1f00b37f0 0x7fa1f00cd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:30.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.591+0000 7fa1f6f8b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa1f00cfbc0 0x7fa1f0149980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:30.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.591+0000 7fa1f6f8b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1f015f900 0x7fa1f0149ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:30.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.591+0000 7fa1f6f8b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa1f006cec0 con 0x7fa1f015f900 2024-09-21T00:30:30.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.591+0000 7fa1f6f8b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa1f006cd40 con 0x7fa1f00cfbc0 2024-09-21T00:30:30.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.591+0000 7fa1f6f8b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa1f006d040 con 0x7fa1f00b37f0 2024-09-21T00:30:30.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.591+0000 7fa1f5528640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1f015f900 0x7fa1f0149ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:30.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.592+0000 7fa1f5528640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1f015f900 0x7fa1f0149ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56578/0 (socket says 172.21.15.112:56578) 2024-09-21T00:30:30.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.592+0000 7fa1f5528640 1 -- 172.21.15.112:0/2677993891 learned_addr learned my addr 172.21.15.112:0/2677993891 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:30:30.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.592+0000 7fa1e7fff640 1 --2- 172.21.15.112:0/2677993891 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa1f00cfbc0 0x7fa1f0149980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:30.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.592+0000 7fa1f4d27640 1 --2- 172.21.15.112:0/2677993891 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa1f00b37f0 0x7fa1f00cd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:30.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.592+0000 7fa1f5528640 1 -- 172.21.15.112:0/2677993891 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa1f00b37f0 msgr2=0x7fa1f00cd3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:30.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.592+0000 7fa1f5528640 1 --2- 172.21.15.112:0/2677993891 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa1f00b37f0 0x7fa1f00cd3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:30.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.592+0000 7fa1f5528640 1 -- 172.21.15.112:0/2677993891 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa1f00cfbc0 msgr2=0x7fa1f0149980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:30.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.592+0000 7fa1f5528640 1 --2- 172.21.15.112:0/2677993891 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa1f00cfbc0 0x7fa1f0149980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:30.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.592+0000 7fa1f5528640 1 -- 172.21.15.112:0/2677993891 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1f00cd8f0 con 0x7fa1f015f900 2024-09-21T00:30:30.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.592+0000 7fa1f5528640 1 --2- 172.21.15.112:0/2677993891 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1f015f900 0x7fa1f0149ec0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fa1e8009d60 tx=0x7fa1e8003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:30.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.592+0000 7fa1e57fa640 1 -- 172.21.15.112:0/2677993891 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e8034040 con 0x7fa1f015f900 2024-09-21T00:30:30.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.592+0000 7fa1e57fa640 1 -- 172.21.15.112:0/2677993891 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa1e8004540 con 0x7fa1f015f900 2024-09-21T00:30:30.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.592+0000 7fa1f6f8b640 1 -- 172.21.15.112:0/2677993891 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1f0147ad0 con 0x7fa1f015f900 2024-09-21T00:30:30.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.592+0000 7fa1e57fa640 1 -- 172.21.15.112:0/2677993891 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e8045480 con 0x7fa1f015f900 2024-09-21T00:30:30.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.592+0000 7fa1f6f8b640 1 -- 172.21.15.112:0/2677993891 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1f015c8c0 con 0x7fa1f015f900 2024-09-21T00:30:30.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.593+0000 7fa1f6f8b640 1 -- 172.21.15.112:0/2677993891 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1f00cffd0 con 0x7fa1f015f900 2024-09-21T00:30:30.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.594+0000 7fa1e57fa640 1 -- 172.21.15.112:0/2677993891 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa1e8037020 con 0x7fa1f015f900 2024-09-21T00:30:30.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.594+0000 7fa1e57fa640 1 --2- 172.21.15.112:0/2677993891 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fa1dc078b90 0x7fa1dc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:30.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.595+0000 7fa1e57fa640 1 -- 172.21.15.112:0/2677993891 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5507+0+0 (secure 0 0 0) 0x7fa1e80bc450 con 0x7fa1f015f900 2024-09-21T00:30:30.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.595+0000 7fa1f4d27640 1 --2- 172.21.15.112:0/2677993891 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fa1dc078b90 0x7fa1dc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:30.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.596+0000 7fa1f4d27640 1 --2- 172.21.15.112:0/2677993891 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fa1dc078b90 0x7fa1dc07b0a0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec006fd0 tx=0x7fa1ec009060 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:30.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.599+0000 7fa1e57fa640 1 -- 172.21.15.112:0/2677993891 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa1f00d0510 con 0x7fa1f015f900 2024-09-21T00:30:30.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.775+0000 7fa1f6f8b640 1 -- 172.21.15.112:0/2677993891 --> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7fa1f015cb00 con 0x7fa1dc078b90 2024-09-21T00:30:30.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.776+0000 7fa1e57fa640 1 -- 172.21.15.112:0/2677993891 <== mgr.4103 v2:172.21.15.187:6800/2921522906 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fa1f015cb00 con 0x7fa1dc078b90 2024-09-21T00:30:30.778 INFO:teuthology.orchestra.run.smithi112.stderr:instructed osd(s) 0 to deep-scrub 2024-09-21T00:30:30.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.778+0000 7fa1f6f8b640 1 -- 172.21.15.112:0/2677993891 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fa1dc078b90 msgr2=0x7fa1dc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:30.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.778+0000 7fa1f6f8b640 1 --2- 172.21.15.112:0/2677993891 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fa1dc078b90 0x7fa1dc07b0a0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec006fd0 tx=0x7fa1ec009060 comp rx=0 tx=0).stop 2024-09-21T00:30:30.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.778+0000 7fa1f6f8b640 1 -- 172.21.15.112:0/2677993891 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1f015f900 msgr2=0x7fa1f0149ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:30.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.778+0000 7fa1f6f8b640 1 --2- 172.21.15.112:0/2677993891 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1f015f900 0x7fa1f0149ec0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fa1e8009d60 tx=0x7fa1e8003f90 comp rx=0 tx=0).stop 2024-09-21T00:30:30.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.779+0000 7fa1f6f8b640 1 -- 172.21.15.112:0/2677993891 shutdown_connections 2024-09-21T00:30:30.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.779+0000 7fa1f6f8b640 1 --2- 172.21.15.112:0/2677993891 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fa1dc078b90 0x7fa1dc07b0a0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:30.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.779+0000 7fa1f6f8b640 1 --2- 172.21.15.112:0/2677993891 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1f015f900 0x7fa1f0149ec0 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:30.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.779+0000 7fa1f6f8b640 1 --2- 172.21.15.112:0/2677993891 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa1f00cfbc0 0x7fa1f0149980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:30.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.779+0000 7fa1f6f8b640 1 --2- 172.21.15.112:0/2677993891 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa1f00b37f0 0x7fa1f00cd3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:30.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.779+0000 7fa1f6f8b640 1 -- 172.21.15.112:0/2677993891 >> 172.21.15.112:0/2677993891 conn(0x7fa1f00b6050 msgr2=0x7fa1f00d2080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:30.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.779+0000 7fa1f6f8b640 1 -- 172.21.15.112:0/2677993891 shutdown_connections 2024-09-21T00:30:30.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.779+0000 7fa1f6f8b640 1 -- 172.21.15.112:0/2677993891 wait complete. 2024-09-21T00:30:30.791 INFO:tasks.ceph:Scrubbing osd.1 2024-09-21T00:30:30.791 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-21T00:30:30.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.906+0000 7f5f7d99d640 1 -- 172.21.15.112:0/2185653104 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5f780d0620 msgr2=0x7f5f780d2fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:30.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.906+0000 7f5f7d99d640 1 --2- 172.21.15.112:0/2185653104 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5f780d0620 0x7f5f780d2fd0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c005fd0 tx=0x7f5f6c02a820 comp rx=0 tx=0).stop 2024-09-21T00:30:30.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.906+0000 7f5f7d99d640 1 -- 172.21.15.112:0/2185653104 shutdown_connections 2024-09-21T00:30:30.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.906+0000 7f5f7d99d640 1 --2- 172.21.15.112:0/2185653104 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5f780d0620 0x7f5f780d2fd0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:30.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.906+0000 7f5f7d99d640 1 --2- 172.21.15.112:0/2185653104 >> v2:172.21.15.187:3300/0 conn(0x7f5f7815e9a0 0x7f5f780b7900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:30.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.906+0000 7f5f7d99d640 1 -- 172.21.15.112:0/2185653104 >> 172.21.15.112:0/2185653104 conn(0x7f5f780d4a20 msgr2=0x7f5f780b5ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:30.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.906+0000 7f5f7d99d640 1 -- 172.21.15.112:0/2185653104 shutdown_connections 2024-09-21T00:30:30.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.906+0000 7f5f7d99d640 1 -- 172.21.15.112:0/2185653104 wait complete. 2024-09-21T00:30:30.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.908+0000 7f5f7d99d640 1 Processor -- start 2024-09-21T00:30:30.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.908+0000 7f5f7d99d640 1 -- start start 2024-09-21T00:30:30.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.909+0000 7f5f7d99d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5f780d0620 0x7f5f781f6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:30.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.909+0000 7f5f7d99d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5f780d3510 0x7f5f780c7f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:30.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.909+0000 7f5f7d99d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f7815e9a0 0x7f5f781507a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:30.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.909+0000 7f5f7d99d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5f7806d7c0 con 0x7f5f7815e9a0 2024-09-21T00:30:30.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.909+0000 7f5f7d99d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5f780ceb80 con 0x7f5f780d3510 2024-09-21T00:30:30.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.909+0000 7f5f7d99d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f5f7806d940 con 0x7f5f780d0620 2024-09-21T00:30:30.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.910+0000 7f5f7d19c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f7815e9a0 0x7f5f781507a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:30.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.910+0000 7f5f7d19c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f7815e9a0 0x7f5f781507a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56590/0 (socket says 172.21.15.112:56590) 2024-09-21T00:30:30.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.910+0000 7f5f7d19c640 1 -- 172.21.15.112:0/3352460079 learned_addr learned my addr 172.21.15.112:0/3352460079 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:30:30.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.910+0000 7f5f7c99b640 1 --2- 172.21.15.112:0/3352460079 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5f780d0620 0x7f5f781f6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:30.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.910+0000 7f5f77fff640 1 --2- 172.21.15.112:0/3352460079 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5f780d3510 0x7f5f780c7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:30.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.910+0000 7f5f7d19c640 1 -- 172.21.15.112:0/3352460079 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5f780d0620 msgr2=0x7f5f781f6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:30.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.910+0000 7f5f7d19c640 1 --2- 172.21.15.112:0/3352460079 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5f780d0620 0x7f5f781f6060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:30.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.910+0000 7f5f7d19c640 1 -- 172.21.15.112:0/3352460079 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5f780d3510 msgr2=0x7f5f780c7f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:30.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.910+0000 7f5f7d19c640 1 --2- 172.21.15.112:0/3352460079 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5f780d3510 0x7f5f780c7f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:30.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.910+0000 7f5f7d19c640 1 -- 172.21.15.112:0/3352460079 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f780c8750 con 0x7f5f7815e9a0 2024-09-21T00:30:30.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.910+0000 7f5f7d19c640 1 --2- 172.21.15.112:0/3352460079 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f7815e9a0 0x7f5f781507a0 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f5f7000c3c0 tx=0x7f5f7000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:30.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.910+0000 7f5f757fa640 1 -- 172.21.15.112:0/3352460079 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f70017070 con 0x7f5f7815e9a0 2024-09-21T00:30:30.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.910+0000 7f5f757fa640 1 -- 172.21.15.112:0/3352460079 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5f70006d90 con 0x7f5f7815e9a0 2024-09-21T00:30:30.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.910+0000 7f5f7d99d640 1 -- 172.21.15.112:0/3352460079 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f780c8a30 con 0x7f5f7815e9a0 2024-09-21T00:30:30.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.910+0000 7f5f757fa640 1 -- 172.21.15.112:0/3352460079 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f70007290 con 0x7f5f7815e9a0 2024-09-21T00:30:30.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.910+0000 7f5f7d99d640 1 -- 172.21.15.112:0/3352460079 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f780c8ff0 con 0x7f5f7815e9a0 2024-09-21T00:30:30.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.911+0000 7f5f7d99d640 1 -- 172.21.15.112:0/3352460079 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5f780d2fd0 con 0x7f5f7815e9a0 2024-09-21T00:30:30.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.912+0000 7f5f757fa640 1 -- 172.21.15.112:0/3352460079 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5f7001a070 con 0x7f5f7815e9a0 2024-09-21T00:30:30.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.912+0000 7f5f757fa640 1 --2- 172.21.15.112:0/3352460079 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f5f64078b90 0x7f5f6407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:30.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.913+0000 7f5f757fa640 1 -- 172.21.15.112:0/3352460079 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5507+0+0 (secure 0 0 0) 0x7f5f7009a550 con 0x7f5f7815e9a0 2024-09-21T00:30:30.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.913+0000 7f5f757fa640 1 --2- 172.21.15.112:0/3352460079 >> [v2:172.21.15.112:6800/344932592,v1:172.21.15.112:6801/344932592] conn(0x7f5f6407f1c0 0x7f5f64081650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:30.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.913+0000 7f5f7c99b640 1 --2- 172.21.15.112:0/3352460079 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f5f64078b90 0x7f5f6407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:30.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.913+0000 7f5f757fa640 1 -- 172.21.15.112:0/3352460079 --> [v2:172.21.15.112:6800/344932592,v1:172.21.15.112:6801/344932592] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5f64081d20 con 0x7f5f6407f1c0 2024-09-21T00:30:30.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.913+0000 7f5f757fa640 1 -- 172.21.15.112:0/3352460079 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_get_version_reply(handle=1 version=115) ==== 24+0+0 (secure 0 0 0) 0x7f5f7009a9e0 con 0x7f5f7815e9a0 2024-09-21T00:30:30.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.913+0000 7f5f77fff640 1 --2- 172.21.15.112:0/3352460079 >> [v2:172.21.15.112:6800/344932592,v1:172.21.15.112:6801/344932592] conn(0x7f5f6407f1c0 0x7f5f64081650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:30.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.913+0000 7f5f77fff640 1 --2- 172.21.15.112:0/3352460079 >> [v2:172.21.15.112:6800/344932592,v1:172.21.15.112:6801/344932592] conn(0x7f5f6407f1c0 0x7f5f64081650 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:30.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.913+0000 7f5f7c99b640 1 --2- 172.21.15.112:0/3352460079 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f5f64078b90 0x7f5f6407b0a0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f5f68009590 tx=0x7f5f68005f30 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:30.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.914+0000 7f5f757fa640 1 -- 172.21.15.112:0/3352460079 <== osd.1 v2:172.21.15.112:6800/344932592 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5f64081d20 con 0x7f5f6407f1c0 2024-09-21T00:30:30.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.926+0000 7f5f7d99d640 1 -- 172.21.15.112:0/3352460079 --> [v2:172.21.15.112:6800/344932592,v1:172.21.15.112:6801/344932592] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f5f780c8500 con 0x7f5f6407f1c0 2024-09-21T00:30:30.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.926+0000 7f5f757fa640 1 -- 172.21.15.112:0/3352460079 <== osd.1 v2:172.21.15.112:6800/344932592 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f5f780c8500 con 0x7f5f6407f1c0 2024-09-21T00:30:30.928 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-21T00:30:30.928 INFO:teuthology.orchestra.run.smithi112.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T00:30:30.928 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-21T00:30:30.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.927+0000 7f5f7d99d640 1 -- 172.21.15.112:0/3352460079 >> [v2:172.21.15.112:6800/344932592,v1:172.21.15.112:6801/344932592] conn(0x7f5f6407f1c0 msgr2=0x7f5f64081650 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:30.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.927+0000 7f5f7d99d640 1 --2- 172.21.15.112:0/3352460079 >> [v2:172.21.15.112:6800/344932592,v1:172.21.15.112:6801/344932592] conn(0x7f5f6407f1c0 0x7f5f64081650 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:30.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.927+0000 7f5f7d99d640 1 -- 172.21.15.112:0/3352460079 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f5f64078b90 msgr2=0x7f5f6407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:30.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.927+0000 7f5f7d99d640 1 --2- 172.21.15.112:0/3352460079 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f5f64078b90 0x7f5f6407b0a0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f5f68009590 tx=0x7f5f68005f30 comp rx=0 tx=0).stop 2024-09-21T00:30:30.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.927+0000 7f5f7d99d640 1 -- 172.21.15.112:0/3352460079 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f7815e9a0 msgr2=0x7f5f781507a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:30.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.927+0000 7f5f7d99d640 1 --2- 172.21.15.112:0/3352460079 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f7815e9a0 0x7f5f781507a0 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f5f7000c3c0 tx=0x7f5f7000c880 comp rx=0 tx=0).stop 2024-09-21T00:30:30.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.927+0000 7f5f7d19c640 1 -- 172.21.15.112:0/3352460079 reap_dead start 2024-09-21T00:30:30.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.927+0000 7f5f7d99d640 1 -- 172.21.15.112:0/3352460079 shutdown_connections 2024-09-21T00:30:30.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.927+0000 7f5f7d99d640 1 -- 172.21.15.112:0/3352460079 >> 172.21.15.112:0/3352460079 conn(0x7f5f780d4a20 msgr2=0x7f5f780b47f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:30.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.927+0000 7f5f7d99d640 1 -- 172.21.15.112:0/3352460079 shutdown_connections 2024-09-21T00:30:30.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:30.927+0000 7f5f7d99d640 1 -- 172.21.15.112:0/3352460079 wait complete. 2024-09-21T00:30:30.939 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-21T00:30:31.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.051+0000 7f5061d90640 1 -- 172.21.15.112:0/2618075161 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5044004800 msgr2=0x7f5044004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:31.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.051+0000 7f5061d90640 1 --2- 172.21.15.112:0/2618075161 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5044004800 0x7f5044004c10 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f5050001830 tx=0x7f5044005830 comp rx=0 tx=0).stop 2024-09-21T00:30:31.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.051+0000 7f505bfff640 1 -- 172.21.15.112:0/2618075161 reap_dead start 2024-09-21T00:30:31.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.051+0000 7f5061d90640 1 -- 172.21.15.112:0/2618075161 shutdown_connections 2024-09-21T00:30:31.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.051+0000 7f5061d90640 1 -- 172.21.15.112:0/2618075161 >> 172.21.15.112:0/2618075161 conn(0x7f505c073d30 msgr2=0x7f505c074140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:31.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.051+0000 7f5061d90640 1 -- 172.21.15.112:0/2618075161 shutdown_connections 2024-09-21T00:30:31.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.051+0000 7f5061d90640 1 -- 172.21.15.112:0/2618075161 wait complete. 2024-09-21T00:30:31.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.053+0000 7f5061d90640 1 Processor -- start 2024-09-21T00:30:31.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.053+0000 7f5061d90640 1 -- start start 2024-09-21T00:30:31.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.054+0000 7f5061d90640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5044004800 0x7f505c0c6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:31.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.054+0000 7f5061d90640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f505c1572b0 0x7f505c1f5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:31.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.054+0000 7f5061d90640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f505c1576c0 0x7f505c14dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:31.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.054+0000 7f5061d90640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f505c15d5e0 con 0x7f505c1572b0 2024-09-21T00:30:31.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.054+0000 7f5061d90640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f505c15d460 con 0x7f5044004800 2024-09-21T00:30:31.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.054+0000 7f5061d90640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f505c15d760 con 0x7f505c1576c0 2024-09-21T00:30:31.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.054+0000 7f505affd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f505c1572b0 0x7f505c1f5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:31.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.054+0000 7f505affd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f505c1572b0 0x7f505c1f5200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56616/0 (socket says 172.21.15.112:56616) 2024-09-21T00:30:31.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.054+0000 7f505affd640 1 -- 172.21.15.112:0/3860865677 learned_addr learned my addr 172.21.15.112:0/3860865677 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:30:31.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.054+0000 7f505b7fe640 1 --2- 172.21.15.112:0/3860865677 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5044004800 0x7f505c0c6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:31.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.054+0000 7f505bfff640 1 --2- 172.21.15.112:0/3860865677 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f505c1576c0 0x7f505c14dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:31.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.055+0000 7f505affd640 1 -- 172.21.15.112:0/3860865677 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f505c1576c0 msgr2=0x7f505c14dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:31.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.055+0000 7f505affd640 1 --2- 172.21.15.112:0/3860865677 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f505c1576c0 0x7f505c14dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:31.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.055+0000 7f505affd640 1 -- 172.21.15.112:0/3860865677 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5044004800 msgr2=0x7f505c0c6220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:31.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.055+0000 7f505affd640 1 --2- 172.21.15.112:0/3860865677 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5044004800 0x7f505c0c6220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:31.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.055+0000 7f505affd640 1 -- 172.21.15.112:0/3860865677 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f505c1f9720 con 0x7f505c1572b0 2024-09-21T00:30:31.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.055+0000 7f505affd640 1 --2- 172.21.15.112:0/3860865677 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f505c1572b0 0x7f505c1f5200 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f5050001950 tx=0x7f50500060e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:31.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.055+0000 7f5043fff640 1 -- 172.21.15.112:0/3860865677 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5050044070 con 0x7f505c1572b0 2024-09-21T00:30:31.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.055+0000 7f5043fff640 1 -- 172.21.15.112:0/3860865677 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5050003a00 con 0x7f505c1572b0 2024-09-21T00:30:31.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.055+0000 7f5061d90640 1 -- 172.21.15.112:0/3860865677 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f505c1f8b20 con 0x7f505c1572b0 2024-09-21T00:30:31.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.055+0000 7f5043fff640 1 -- 172.21.15.112:0/3860865677 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5050003e50 con 0x7f505c1572b0 2024-09-21T00:30:31.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.055+0000 7f5061d90640 1 -- 172.21.15.112:0/3860865677 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f505c1567d0 con 0x7f505c1572b0 2024-09-21T00:30:31.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.056+0000 7f5061d90640 1 -- 172.21.15.112:0/3860865677 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f505c0b7180 con 0x7f505c1572b0 2024-09-21T00:30:31.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.057+0000 7f5043fff640 1 -- 172.21.15.112:0/3860865677 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5050033480 con 0x7f505c1572b0 2024-09-21T00:30:31.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.057+0000 7f5043fff640 1 --2- 172.21.15.112:0/3860865677 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f5048078d30 0x7f504807b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:31.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.057+0000 7f5043fff640 1 -- 172.21.15.112:0/3860865677 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5507+0+0 (secure 0 0 0) 0x7f50500baab0 con 0x7f505c1572b0 2024-09-21T00:30:31.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.057+0000 7f505b7fe640 1 --2- 172.21.15.112:0/3860865677 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f5048078d30 0x7f504807b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:31.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.058+0000 7f505b7fe640 1 --2- 172.21.15.112:0/3860865677 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f5048078d30 0x7f504807b240 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f504c001860 tx=0x7f504c001670 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:31.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.062+0000 7f5043fff640 1 -- 172.21.15.112:0/3860865677 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f505c1f9720 con 0x7f505c1572b0 2024-09-21T00:30:31.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.244+0000 7f5061d90640 1 -- 172.21.15.112:0/3860865677 --> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7f505c0b4760 con 0x7f5048078d30 2024-09-21T00:30:31.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.245+0000 7f5043fff640 1 -- 172.21.15.112:0/3860865677 <== mgr.4103 v2:172.21.15.187:6800/2921522906 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f505c0b4760 con 0x7f5048078d30 2024-09-21T00:30:31.246 INFO:teuthology.orchestra.run.smithi112.stderr:instructed osd(s) 1 to deep-scrub 2024-09-21T00:30:31.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.247+0000 7f5061d90640 1 -- 172.21.15.112:0/3860865677 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f5048078d30 msgr2=0x7f504807b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:31.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.247+0000 7f5061d90640 1 --2- 172.21.15.112:0/3860865677 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f5048078d30 0x7f504807b240 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f504c001860 tx=0x7f504c001670 comp rx=0 tx=0).stop 2024-09-21T00:30:31.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.247+0000 7f5061d90640 1 -- 172.21.15.112:0/3860865677 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f505c1572b0 msgr2=0x7f505c1f5200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:31.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.247+0000 7f5061d90640 1 --2- 172.21.15.112:0/3860865677 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f505c1572b0 0x7f505c1f5200 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f5050001950 tx=0x7f50500060e0 comp rx=0 tx=0).stop 2024-09-21T00:30:31.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.247+0000 7f5061d90640 1 -- 172.21.15.112:0/3860865677 shutdown_connections 2024-09-21T00:30:31.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.247+0000 7f5061d90640 1 --2- 172.21.15.112:0/3860865677 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f505c1576c0 0x7f505c14dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:31.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.247+0000 7f5061d90640 1 --2- 172.21.15.112:0/3860865677 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f5048078d30 0x7f504807b240 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:31.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.247+0000 7f5061d90640 1 --2- 172.21.15.112:0/3860865677 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f505c1572b0 0x7f505c1f5200 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:31.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.247+0000 7f5061d90640 1 --2- 172.21.15.112:0/3860865677 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5044004800 0x7f505c0c6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:31.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.247+0000 7f5061d90640 1 -- 172.21.15.112:0/3860865677 >> 172.21.15.112:0/3860865677 conn(0x7f505c073d30 msgr2=0x7f505c0b2460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:31.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.247+0000 7f5061d90640 1 -- 172.21.15.112:0/3860865677 shutdown_connections 2024-09-21T00:30:31.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.247+0000 7f5061d90640 1 -- 172.21.15.112:0/3860865677 wait complete. 2024-09-21T00:30:31.259 INFO:tasks.ceph:Scrubbing osd.2 2024-09-21T00:30:31.259 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-21T00:30:31.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.374+0000 7fc8217c6640 1 -- 172.21.15.112:0/2039386966 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc81c0d4b20 msgr2=0x7fc81c15e200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:31.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.374+0000 7fc8217c6640 1 --2- 172.21.15.112:0/2039386966 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc81c0d4b20 0x7fc81c15e200 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7fc814005fd0 tx=0x7fc81402a540 comp rx=0 tx=0).stop 2024-09-21T00:30:31.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.374+0000 7fc8217c6640 1 -- 172.21.15.112:0/2039386966 shutdown_connections 2024-09-21T00:30:31.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.374+0000 7fc8217c6640 1 --2- 172.21.15.112:0/2039386966 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc81c0d4b20 0x7fc81c15e200 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:31.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.374+0000 7fc8217c6640 1 --2- 172.21.15.112:0/2039386966 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc81c162f70 0x7fc81c0b3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:31.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.374+0000 7fc8217c6640 1 -- 172.21.15.112:0/2039386966 >> 172.21.15.112:0/2039386966 conn(0x7fc81c08bfe0 msgr2=0x7fc81c09d890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:31.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.374+0000 7fc8217c6640 1 -- 172.21.15.112:0/2039386966 shutdown_connections 2024-09-21T00:30:31.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.374+0000 7fc8217c6640 1 -- 172.21.15.112:0/2039386966 wait complete. 2024-09-21T00:30:31.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.376+0000 7fc8217c6640 1 Processor -- start 2024-09-21T00:30:31.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.376+0000 7fc8217c6640 1 -- start start 2024-09-21T00:30:31.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.377+0000 7fc8217c6640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc81c0b6030 0x7fc81c06fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:31.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.377+0000 7fc8217c6640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc81c0d4b20 0x7fc81c14c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:31.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.377+0000 7fc8217c6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc81c162f70 0x7fc81c0b1d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:31.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.377+0000 7fc8217c6640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc81c0b67d0 con 0x7fc81c162f70 2024-09-21T00:30:31.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.377+0000 7fc8217c6640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc81c0b6650 con 0x7fc81c0d4b20 2024-09-21T00:30:31.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.377+0000 7fc8217c6640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc81c15e740 con 0x7fc81c0b6030 2024-09-21T00:30:31.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.377+0000 7fc81b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc81c162f70 0x7fc81c0b1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:31.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.377+0000 7fc81b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc81c162f70 0x7fc81c0b1d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56638/0 (socket says 172.21.15.112:56638) 2024-09-21T00:30:31.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.377+0000 7fc81b7fe640 1 -- 172.21.15.112:0/2653310585 learned_addr learned my addr 172.21.15.112:0/2653310585 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:30:31.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.377+0000 7fc81affd640 1 --2- 172.21.15.112:0/2653310585 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc81c0b6030 0x7fc81c06fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:31.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.377+0000 7fc81a7fc640 1 --2- 172.21.15.112:0/2653310585 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc81c0d4b20 0x7fc81c14c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:31.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.377+0000 7fc81b7fe640 1 -- 172.21.15.112:0/2653310585 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc81c0b6030 msgr2=0x7fc81c06fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:31.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.377+0000 7fc81b7fe640 1 --2- 172.21.15.112:0/2653310585 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc81c0b6030 0x7fc81c06fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:31.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.377+0000 7fc81b7fe640 1 -- 172.21.15.112:0/2653310585 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc81c0d4b20 msgr2=0x7fc81c14c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:31.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.377+0000 7fc81b7fe640 1 --2- 172.21.15.112:0/2653310585 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc81c0d4b20 0x7fc81c14c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:31.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.377+0000 7fc81b7fe640 1 -- 172.21.15.112:0/2653310585 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc81c1f5a00 con 0x7fc81c162f70 2024-09-21T00:30:31.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.378+0000 7fc81b7fe640 1 --2- 172.21.15.112:0/2653310585 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc81c162f70 0x7fc81c0b1d10 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7fc814006010 tx=0x7fc814004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:31.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.378+0000 7fc8137fe640 1 -- 172.21.15.112:0/2653310585 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc81403e030 con 0x7fc81c162f70 2024-09-21T00:30:31.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.378+0000 7fc8137fe640 1 -- 172.21.15.112:0/2653310585 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc8140047a0 con 0x7fc81c162f70 2024-09-21T00:30:31.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.378+0000 7fc8217c6640 1 -- 172.21.15.112:0/2653310585 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc81c1f5ce0 con 0x7fc81c162f70 2024-09-21T00:30:31.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.378+0000 7fc8137fe640 1 -- 172.21.15.112:0/2653310585 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc814035040 con 0x7fc81c162f70 2024-09-21T00:30:31.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.378+0000 7fc8217c6640 1 -- 172.21.15.112:0/2653310585 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc81c1f62a0 con 0x7fc81c162f70 2024-09-21T00:30:31.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.379+0000 7fc8217c6640 1 -- 172.21.15.112:0/2653310585 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc81c0d3190 con 0x7fc81c162f70 2024-09-21T00:30:31.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.380+0000 7fc8137fe640 1 -- 172.21.15.112:0/2653310585 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc814041030 con 0x7fc81c162f70 2024-09-21T00:30:31.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.380+0000 7fc8137fe640 1 --2- 172.21.15.112:0/2653310585 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fc7fc078bc0 0x7fc7fc07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:31.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.380+0000 7fc8137fe640 1 -- 172.21.15.112:0/2653310585 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5507+0+0 (secure 0 0 0) 0x7fc8140bb880 con 0x7fc81c162f70 2024-09-21T00:30:31.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.380+0000 7fc8137fe640 1 --2- 172.21.15.112:0/2653310585 >> [v2:172.21.15.112:6824/407933616,v1:172.21.15.112:6825/407933616] conn(0x7fc7fc07f1f0 0x7fc7fc081680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:31.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.380+0000 7fc8137fe640 1 -- 172.21.15.112:0/2653310585 --> [v2:172.21.15.112:6824/407933616,v1:172.21.15.112:6825/407933616] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc81404ee50 con 0x7fc7fc07f1f0 2024-09-21T00:30:31.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.380+0000 7fc8137fe640 1 -- 172.21.15.112:0/2653310585 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_get_version_reply(handle=1 version=115) ==== 24+0+0 (secure 0 0 0) 0x7fc8140bbd10 con 0x7fc81c162f70 2024-09-21T00:30:31.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.380+0000 7fc81affd640 1 --2- 172.21.15.112:0/2653310585 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fc7fc078bc0 0x7fc7fc07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:31.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.380+0000 7fc81a7fc640 1 --2- 172.21.15.112:0/2653310585 >> [v2:172.21.15.112:6824/407933616,v1:172.21.15.112:6825/407933616] conn(0x7fc7fc07f1f0 0x7fc7fc081680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:31.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.381+0000 7fc81a7fc640 1 --2- 172.21.15.112:0/2653310585 >> [v2:172.21.15.112:6824/407933616,v1:172.21.15.112:6825/407933616] conn(0x7fc7fc07f1f0 0x7fc7fc081680 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:31.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.381+0000 7fc81affd640 1 --2- 172.21.15.112:0/2653310585 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fc7fc078bc0 0x7fc7fc07b0d0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fc80c005fd0 tx=0x7fc80c009630 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:31.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.381+0000 7fc8137fe640 1 -- 172.21.15.112:0/2653310585 <== osd.2 v2:172.21.15.112:6824/407933616 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc81404ee50 con 0x7fc7fc07f1f0 2024-09-21T00:30:31.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.394+0000 7fc8217c6640 1 -- 172.21.15.112:0/2653310585 --> [v2:172.21.15.112:6824/407933616,v1:172.21.15.112:6825/407933616] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fc81c1f57b0 con 0x7fc7fc07f1f0 2024-09-21T00:30:31.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.394+0000 7fc8137fe640 1 -- 172.21.15.112:0/2653310585 <== osd.2 v2:172.21.15.112:6824/407933616 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fc81c1f57b0 con 0x7fc7fc07f1f0 2024-09-21T00:30:31.396 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-21T00:30:31.396 INFO:teuthology.orchestra.run.smithi112.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T00:30:31.396 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-21T00:30:31.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.394+0000 7fc8217c6640 1 -- 172.21.15.112:0/2653310585 >> [v2:172.21.15.112:6824/407933616,v1:172.21.15.112:6825/407933616] conn(0x7fc7fc07f1f0 msgr2=0x7fc7fc081680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:31.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.394+0000 7fc8217c6640 1 --2- 172.21.15.112:0/2653310585 >> [v2:172.21.15.112:6824/407933616,v1:172.21.15.112:6825/407933616] conn(0x7fc7fc07f1f0 0x7fc7fc081680 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:31.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.394+0000 7fc8217c6640 1 -- 172.21.15.112:0/2653310585 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fc7fc078bc0 msgr2=0x7fc7fc07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:31.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.394+0000 7fc8217c6640 1 --2- 172.21.15.112:0/2653310585 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fc7fc078bc0 0x7fc7fc07b0d0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fc80c005fd0 tx=0x7fc80c009630 comp rx=0 tx=0).stop 2024-09-21T00:30:31.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.394+0000 7fc8217c6640 1 -- 172.21.15.112:0/2653310585 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc81c162f70 msgr2=0x7fc81c0b1d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:31.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.394+0000 7fc8217c6640 1 --2- 172.21.15.112:0/2653310585 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc81c162f70 0x7fc81c0b1d10 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7fc814006010 tx=0x7fc814004c80 comp rx=0 tx=0).stop 2024-09-21T00:30:31.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.394+0000 7fc81b7fe640 1 -- 172.21.15.112:0/2653310585 reap_dead start 2024-09-21T00:30:31.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.394+0000 7fc8217c6640 1 -- 172.21.15.112:0/2653310585 shutdown_connections 2024-09-21T00:30:31.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.394+0000 7fc8217c6640 1 -- 172.21.15.112:0/2653310585 >> 172.21.15.112:0/2653310585 conn(0x7fc81c08bfe0 msgr2=0x7fc81c0b40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:31.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.394+0000 7fc8217c6640 1 -- 172.21.15.112:0/2653310585 shutdown_connections 2024-09-21T00:30:31.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.394+0000 7fc8217c6640 1 -- 172.21.15.112:0/2653310585 wait complete. 2024-09-21T00:30:31.407 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-21T00:30:31.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.518+0000 7f4721c30640 1 -- 172.21.15.112:0/1048138628 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f471c1602e0 msgr2=0x7f471c1606f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:31.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.518+0000 7f4721c30640 1 --2- 172.21.15.112:0/1048138628 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f471c1602e0 0x7f471c1606f0 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f4714005fd0 tx=0x7f471402a5d0 comp rx=0 tx=0).stop 2024-09-21T00:30:31.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.519+0000 7f4721c30640 1 -- 172.21.15.112:0/1048138628 shutdown_connections 2024-09-21T00:30:31.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.519+0000 7f4721c30640 1 --2- 172.21.15.112:0/1048138628 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f471c1602e0 0x7f471c1606f0 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:31.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.519+0000 7f4721c30640 1 --2- 172.21.15.112:0/1048138628 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f471c0b4530 0x7f471c0b3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:31.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.519+0000 7f4721c30640 1 -- 172.21.15.112:0/1048138628 >> 172.21.15.112:0/1048138628 conn(0x7f471c0cf020 msgr2=0x7f471c0cf430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:31.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.519+0000 7f4721c30640 1 -- 172.21.15.112:0/1048138628 shutdown_connections 2024-09-21T00:30:31.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.519+0000 7f4721c30640 1 -- 172.21.15.112:0/1048138628 wait complete. 2024-09-21T00:30:31.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.520+0000 7f4721c30640 1 Processor -- start 2024-09-21T00:30:31.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.521+0000 7f4721c30640 1 -- start start 2024-09-21T00:30:31.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.522+0000 7f4721c30640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f471c0b4530 0x7f471c1f5680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:31.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.522+0000 7f4721c30640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f471c1602e0 0x7f471c0b6030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:31.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.522+0000 7f4721c30640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f471c070460 0x7f471c15b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:31.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.522+0000 7f4721c30640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f471c0d0f70 con 0x7f471c1602e0 2024-09-21T00:30:31.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.522+0000 7f4721c30640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f471c0d0df0 con 0x7f471c070460 2024-09-21T00:30:31.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.522+0000 7f4721c30640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f471c0d10f0 con 0x7f471c0b4530 2024-09-21T00:30:31.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.522+0000 7f471affd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f471c1602e0 0x7f471c0b6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:31.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.522+0000 7f471affd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f471c1602e0 0x7f471c0b6030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56656/0 (socket says 172.21.15.112:56656) 2024-09-21T00:30:31.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.522+0000 7f471affd640 1 -- 172.21.15.112:0/4111469492 learned_addr learned my addr 172.21.15.112:0/4111469492 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:30:31.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.522+0000 7f471b7fe640 1 --2- 172.21.15.112:0/4111469492 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f471c0b4530 0x7f471c1f5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:31.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.522+0000 7f471bfff640 1 --2- 172.21.15.112:0/4111469492 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f471c070460 0x7f471c15b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:31.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.522+0000 7f471affd640 1 -- 172.21.15.112:0/4111469492 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f471c0b4530 msgr2=0x7f471c1f5680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:31.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.522+0000 7f471affd640 1 --2- 172.21.15.112:0/4111469492 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f471c0b4530 0x7f471c1f5680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:31.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.522+0000 7f471affd640 1 -- 172.21.15.112:0/4111469492 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f471c070460 msgr2=0x7f471c15b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:31.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.522+0000 7f471affd640 1 --2- 172.21.15.112:0/4111469492 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f471c070460 0x7f471c15b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:31.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.522+0000 7f471affd640 1 -- 172.21.15.112:0/4111469492 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f471c0b6b50 con 0x7f471c1602e0 2024-09-21T00:30:31.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.522+0000 7f471bfff640 1 --2- 172.21.15.112:0/4111469492 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f471c070460 0x7f471c15b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:30:31.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.522+0000 7f471b7fe640 1 --2- 172.21.15.112:0/4111469492 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f471c0b4530 0x7f471c1f5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:30:31.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.522+0000 7f471affd640 1 --2- 172.21.15.112:0/4111469492 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f471c1602e0 0x7f471c0b6030 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f471000d6a0 tx=0x7f471000db60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:31.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.523+0000 7f4703fff640 1 -- 172.21.15.112:0/4111469492 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4710017070 con 0x7f471c1602e0 2024-09-21T00:30:31.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.523+0000 7f4703fff640 1 -- 172.21.15.112:0/4111469492 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4710004310 con 0x7f471c1602e0 2024-09-21T00:30:31.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.523+0000 7f4721c30640 1 -- 172.21.15.112:0/4111469492 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f471c1f5fb0 con 0x7f471c1602e0 2024-09-21T00:30:31.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.523+0000 7f4703fff640 1 -- 172.21.15.112:0/4111469492 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4710005280 con 0x7f471c1602e0 2024-09-21T00:30:31.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.523+0000 7f4721c30640 1 -- 172.21.15.112:0/4111469492 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f471c0b6dc0 con 0x7f471c1602e0 2024-09-21T00:30:31.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.524+0000 7f4721c30640 1 -- 172.21.15.112:0/4111469492 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f471c0ce260 con 0x7f471c1602e0 2024-09-21T00:30:31.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.525+0000 7f4703fff640 1 -- 172.21.15.112:0/4111469492 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f4710005420 con 0x7f471c1602e0 2024-09-21T00:30:31.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.525+0000 7f4703fff640 1 --2- 172.21.15.112:0/4111469492 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4708078b90 0x7f470807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:31.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.525+0000 7f4703fff640 1 -- 172.21.15.112:0/4111469492 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5507+0+0 (secure 0 0 0) 0x7f471009b4e0 con 0x7f471c1602e0 2024-09-21T00:30:31.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.525+0000 7f471b7fe640 1 --2- 172.21.15.112:0/4111469492 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4708078b90 0x7f470807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:31.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.526+0000 7f471b7fe640 1 --2- 172.21.15.112:0/4111469492 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4708078b90 0x7f470807b0a0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f470c001dd0 tx=0x7f470c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:31.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.529+0000 7f4703fff640 1 -- 172.21.15.112:0/4111469492 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f4710020020 con 0x7f471c1602e0 2024-09-21T00:30:31.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.709+0000 7f4721c30640 1 -- 172.21.15.112:0/4111469492 --> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7f471c0d1610 con 0x7f4708078b90 2024-09-21T00:30:31.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.710+0000 7f4703fff640 1 -- 172.21.15.112:0/4111469492 <== mgr.4103 v2:172.21.15.187:6800/2921522906 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f471c0d1610 con 0x7f4708078b90 2024-09-21T00:30:31.711 INFO:teuthology.orchestra.run.smithi112.stderr:instructed osd(s) 2 to deep-scrub 2024-09-21T00:30:31.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.712+0000 7f4721c30640 1 -- 172.21.15.112:0/4111469492 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4708078b90 msgr2=0x7f470807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:31.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.712+0000 7f4721c30640 1 --2- 172.21.15.112:0/4111469492 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4708078b90 0x7f470807b0a0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f470c001dd0 tx=0x7f470c009040 comp rx=0 tx=0).stop 2024-09-21T00:30:31.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.712+0000 7f4721c30640 1 -- 172.21.15.112:0/4111469492 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f471c1602e0 msgr2=0x7f471c0b6030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:31.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.712+0000 7f4721c30640 1 --2- 172.21.15.112:0/4111469492 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f471c1602e0 0x7f471c0b6030 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f471000d6a0 tx=0x7f471000db60 comp rx=0 tx=0).stop 2024-09-21T00:30:31.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.712+0000 7f4721c30640 1 -- 172.21.15.112:0/4111469492 shutdown_connections 2024-09-21T00:30:31.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.712+0000 7f4721c30640 1 --2- 172.21.15.112:0/4111469492 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f471c070460 0x7f471c15b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:31.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.712+0000 7f4721c30640 1 --2- 172.21.15.112:0/4111469492 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4708078b90 0x7f470807b0a0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:31.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.712+0000 7f4721c30640 1 --2- 172.21.15.112:0/4111469492 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f471c1602e0 0x7f471c0b6030 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:31.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.712+0000 7f4721c30640 1 --2- 172.21.15.112:0/4111469492 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f471c0b4530 0x7f471c1f5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:31.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.712+0000 7f4721c30640 1 -- 172.21.15.112:0/4111469492 >> 172.21.15.112:0/4111469492 conn(0x7f471c0cf020 msgr2=0x7f471c0c7ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:31.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.712+0000 7f4721c30640 1 -- 172.21.15.112:0/4111469492 shutdown_connections 2024-09-21T00:30:31.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.712+0000 7f4721c30640 1 -- 172.21.15.112:0/4111469492 wait complete. 2024-09-21T00:30:31.725 INFO:tasks.ceph:Scrubbing osd.3 2024-09-21T00:30:31.725 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-21T00:30:31.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.839+0000 7ffbb7539640 1 -- 172.21.15.112:0/685500380 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffbb006ff20 msgr2=0x7ffbb015eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:31.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.839+0000 7ffbb7539640 1 --2- 172.21.15.112:0/685500380 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffbb006ff20 0x7ffbb015eba0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7ffbb00b4090 tx=0x7ffba002a4d0 comp rx=0 tx=0).stop 2024-09-21T00:30:31.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.839+0000 7ffbb7539640 1 -- 172.21.15.112:0/685500380 shutdown_connections 2024-09-21T00:30:31.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.839+0000 7ffbb7539640 1 --2- 172.21.15.112:0/685500380 >> v2:172.21.15.187:3300/0 conn(0x7ffbb00d1200 0x7ffbb00d3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:31.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.839+0000 7ffbb7539640 1 --2- 172.21.15.112:0/685500380 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffbb006ff20 0x7ffbb015eba0 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:31.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.839+0000 7ffbb7539640 1 -- 172.21.15.112:0/685500380 >> 172.21.15.112:0/685500380 conn(0x7ffbb008bfe0 msgr2=0x7ffbb009d910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:31.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.839+0000 7ffbb7539640 1 -- 172.21.15.112:0/685500380 shutdown_connections 2024-09-21T00:30:31.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.839+0000 7ffbb7539640 1 -- 172.21.15.112:0/685500380 wait complete. 2024-09-21T00:30:31.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.841+0000 7ffbb7539640 1 Processor -- start 2024-09-21T00:30:31.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.841+0000 7ffbb7539640 1 -- start start 2024-09-21T00:30:31.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.842+0000 7ffbb7539640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffbb006ff20 0x7ffbb01f9900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:31.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.842+0000 7ffbb7539640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffbb00b93a0 0x7ffbb015d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:31.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.843+0000 7ffbb7539640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffbb00d1200 0x7ffbb01eb410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:31.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.843+0000 7ffbb7539640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ffbb00d2e00 con 0x7ffbb00b93a0 2024-09-21T00:30:31.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.843+0000 7ffbb7539640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ffbb00d2c80 con 0x7ffbb006ff20 2024-09-21T00:30:31.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.843+0000 7ffbb7539640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ffbb00d2f80 con 0x7ffbb00d1200 2024-09-21T00:30:31.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.843+0000 7ffbb4ad4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffbb00b93a0 0x7ffbb015d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:31.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.843+0000 7ffbb4ad4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffbb00b93a0 0x7ffbb015d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56664/0 (socket says 172.21.15.112:56664) 2024-09-21T00:30:31.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.843+0000 7ffbb4ad4640 1 -- 172.21.15.112:0/4115100952 learned_addr learned my addr 172.21.15.112:0/4115100952 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:30:31.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.843+0000 7ffbb52d5640 1 --2- 172.21.15.112:0/4115100952 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffbb006ff20 0x7ffbb01f9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:31.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.843+0000 7ffbb5ad6640 1 --2- 172.21.15.112:0/4115100952 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffbb00d1200 0x7ffbb01eb410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:31.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.843+0000 7ffbb4ad4640 1 -- 172.21.15.112:0/4115100952 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffbb00d1200 msgr2=0x7ffbb01eb410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:31.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.843+0000 7ffbb4ad4640 1 --2- 172.21.15.112:0/4115100952 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffbb00d1200 0x7ffbb01eb410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:31.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.843+0000 7ffbb4ad4640 1 -- 172.21.15.112:0/4115100952 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffbb006ff20 msgr2=0x7ffbb01f9900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:31.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.843+0000 7ffbb4ad4640 1 --2- 172.21.15.112:0/4115100952 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffbb006ff20 0x7ffbb01f9900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:31.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.843+0000 7ffbb4ad4640 1 -- 172.21.15.112:0/4115100952 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbb01e3b00 con 0x7ffbb00b93a0 2024-09-21T00:30:31.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.843+0000 7ffbb4ad4640 1 --2- 172.21.15.112:0/4115100952 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffbb00b93a0 0x7ffbb015d460 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7ffbac00c970 tx=0x7ffbac00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:31.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.843+0000 7ffba5ffb640 1 -- 172.21.15.112:0/4115100952 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbac007bb0 con 0x7ffbb00b93a0 2024-09-21T00:30:31.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.843+0000 7ffba5ffb640 1 -- 172.21.15.112:0/4115100952 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffbac005020 con 0x7ffbb00b93a0 2024-09-21T00:30:31.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.843+0000 7ffbb7539640 1 -- 172.21.15.112:0/4115100952 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbb01e3de0 con 0x7ffbb00b93a0 2024-09-21T00:30:31.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.843+0000 7ffba5ffb640 1 -- 172.21.15.112:0/4115100952 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbac007e60 con 0x7ffbb00b93a0 2024-09-21T00:30:31.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.844+0000 7ffbb7539640 1 -- 172.21.15.112:0/4115100952 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbb00a01f0 con 0x7ffbb00b93a0 2024-09-21T00:30:31.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.844+0000 7ffbb7539640 1 -- 172.21.15.112:0/4115100952 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ffbb00d3470 con 0x7ffbb00b93a0 2024-09-21T00:30:31.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.845+0000 7ffba5ffb640 1 -- 172.21.15.112:0/4115100952 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ffbac005470 con 0x7ffbb00b93a0 2024-09-21T00:30:31.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.846+0000 7ffba5ffb640 1 --2- 172.21.15.112:0/4115100952 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ffb94078b90 0x7ffb9407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:31.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.846+0000 7ffba5ffb640 1 -- 172.21.15.112:0/4115100952 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5507+0+0 (secure 0 0 0) 0x7ffbac09b0e0 con 0x7ffbb00b93a0 2024-09-21T00:30:31.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.846+0000 7ffba5ffb640 1 --2- 172.21.15.112:0/4115100952 >> [v2:172.21.15.112:6816/4120164806,v1:172.21.15.112:6817/4120164806] conn(0x7ffb9407f1c0 0x7ffb94081650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:31.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.846+0000 7ffba5ffb640 1 -- 172.21.15.112:0/4115100952 --> [v2:172.21.15.112:6816/4120164806,v1:172.21.15.112:6817/4120164806] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ffbac004490 con 0x7ffb9407f1c0 2024-09-21T00:30:31.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.846+0000 7ffba5ffb640 1 -- 172.21.15.112:0/4115100952 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_get_version_reply(handle=1 version=115) ==== 24+0+0 (secure 0 0 0) 0x7ffbac0a2050 con 0x7ffbb00b93a0 2024-09-21T00:30:31.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.846+0000 7ffbb52d5640 1 --2- 172.21.15.112:0/4115100952 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ffb94078b90 0x7ffb9407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:31.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.846+0000 7ffbb5ad6640 1 --2- 172.21.15.112:0/4115100952 >> [v2:172.21.15.112:6816/4120164806,v1:172.21.15.112:6817/4120164806] conn(0x7ffb9407f1c0 0x7ffb94081650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:31.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.846+0000 7ffbb5ad6640 1 --2- 172.21.15.112:0/4115100952 >> [v2:172.21.15.112:6816/4120164806,v1:172.21.15.112:6817/4120164806] conn(0x7ffb9407f1c0 0x7ffb94081650 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:31.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.846+0000 7ffbb52d5640 1 --2- 172.21.15.112:0/4115100952 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ffb94078b90 0x7ffb9407b0a0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7ffba0033000 tx=0x7ffba0004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:31.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.847+0000 7ffba5ffb640 1 -- 172.21.15.112:0/4115100952 <== osd.3 v2:172.21.15.112:6816/4120164806 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ffbac004490 con 0x7ffb9407f1c0 2024-09-21T00:30:31.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.859+0000 7ffbb7539640 1 -- 172.21.15.112:0/4115100952 --> [v2:172.21.15.112:6816/4120164806,v1:172.21.15.112:6817/4120164806] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7ffbb01e4680 con 0x7ffb9407f1c0 2024-09-21T00:30:31.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.859+0000 7ffba5ffb640 1 -- 172.21.15.112:0/4115100952 <== osd.3 v2:172.21.15.112:6816/4120164806 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7ffbb01e4680 con 0x7ffb9407f1c0 2024-09-21T00:30:31.861 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-21T00:30:31.861 INFO:teuthology.orchestra.run.smithi112.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T00:30:31.861 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-21T00:30:31.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.860+0000 7ffbb7539640 1 -- 172.21.15.112:0/4115100952 >> [v2:172.21.15.112:6816/4120164806,v1:172.21.15.112:6817/4120164806] conn(0x7ffb9407f1c0 msgr2=0x7ffb94081650 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:31.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.860+0000 7ffbb7539640 1 --2- 172.21.15.112:0/4115100952 >> [v2:172.21.15.112:6816/4120164806,v1:172.21.15.112:6817/4120164806] conn(0x7ffb9407f1c0 0x7ffb94081650 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:31.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.860+0000 7ffbb7539640 1 -- 172.21.15.112:0/4115100952 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ffb94078b90 msgr2=0x7ffb9407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:31.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.860+0000 7ffbb7539640 1 --2- 172.21.15.112:0/4115100952 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ffb94078b90 0x7ffb9407b0a0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7ffba0033000 tx=0x7ffba0004730 comp rx=0 tx=0).stop 2024-09-21T00:30:31.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.860+0000 7ffbb7539640 1 -- 172.21.15.112:0/4115100952 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffbb00b93a0 msgr2=0x7ffbb015d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:31.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.860+0000 7ffbb7539640 1 --2- 172.21.15.112:0/4115100952 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffbb00b93a0 0x7ffbb015d460 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7ffbac00c970 tx=0x7ffbac00ce30 comp rx=0 tx=0).stop 2024-09-21T00:30:31.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.860+0000 7ffbb5ad6640 1 -- 172.21.15.112:0/4115100952 reap_dead start 2024-09-21T00:30:31.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.860+0000 7ffbb7539640 1 -- 172.21.15.112:0/4115100952 shutdown_connections 2024-09-21T00:30:31.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.860+0000 7ffbb7539640 1 -- 172.21.15.112:0/4115100952 >> 172.21.15.112:0/4115100952 conn(0x7ffbb008bfe0 msgr2=0x7ffbb0068d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:31.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.860+0000 7ffbb7539640 1 -- 172.21.15.112:0/4115100952 shutdown_connections 2024-09-21T00:30:31.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.860+0000 7ffbb7539640 1 -- 172.21.15.112:0/4115100952 wait complete. 2024-09-21T00:30:31.873 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-21T00:30:31.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.984+0000 7fb2e7a55640 1 -- 172.21.15.112:0/1630346264 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2e00b37f0 msgr2=0x7fb2e0162ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:31.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.984+0000 7fb2e7a55640 1 --2- 172.21.15.112:0/1630346264 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2e00b37f0 0x7fb2e0162ef0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc009830 tx=0x7fb2dc02a480 comp rx=0 tx=0).stop 2024-09-21T00:30:31.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.984+0000 7fb2e7a55640 1 -- 172.21.15.112:0/1630346264 shutdown_connections 2024-09-21T00:30:31.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.984+0000 7fb2e7a55640 1 --2- 172.21.15.112:0/1630346264 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2e00b37f0 0x7fb2e0162ef0 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:31.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.984+0000 7fb2e7a55640 1 -- 172.21.15.112:0/1630346264 >> 172.21.15.112:0/1630346264 conn(0x7fb2e00b6050 msgr2=0x7fb2e00b6460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:31.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.984+0000 7fb2e7a55640 1 -- 172.21.15.112:0/1630346264 shutdown_connections 2024-09-21T00:30:31.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.984+0000 7fb2e7a55640 1 -- 172.21.15.112:0/1630346264 wait complete. 2024-09-21T00:30:31.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.986+0000 7fb2e7a55640 1 Processor -- start 2024-09-21T00:30:31.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.986+0000 7fb2e7a55640 1 -- start start 2024-09-21T00:30:31.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.987+0000 7fb2e7a55640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2e00b37f0 0x7fb2e015a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:31.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.987+0000 7fb2e7a55640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb2e00cfbc0 0x7fb2e014da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:31.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.987+0000 7fb2e7a55640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb2e015f900 0x7fb2e00ce1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:31.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.987+0000 7fb2e7a55640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e006e0e0 con 0x7fb2e00b37f0 2024-09-21T00:30:31.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.987+0000 7fb2e7a55640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e006df60 con 0x7fb2e015f900 2024-09-21T00:30:31.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.987+0000 7fb2e7a55640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fb2e006e260 con 0x7fb2e00cfbc0 2024-09-21T00:30:31.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.987+0000 7fb2e57f1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2e00b37f0 0x7fb2e015a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:31.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.987+0000 7fb2e57f1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2e00b37f0 0x7fb2e015a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56682/0 (socket says 172.21.15.112:56682) 2024-09-21T00:30:31.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.987+0000 7fb2e57f1640 1 -- 172.21.15.112:0/1739357491 learned_addr learned my addr 172.21.15.112:0/1739357491 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:30:31.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.987+0000 7fb2e4ff0640 1 --2- 172.21.15.112:0/1739357491 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb2e00cfbc0 0x7fb2e014da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:31.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.988+0000 7fb2e5ff2640 1 --2- 172.21.15.112:0/1739357491 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb2e015f900 0x7fb2e00ce1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:31.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.988+0000 7fb2e57f1640 1 -- 172.21.15.112:0/1739357491 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb2e00cfbc0 msgr2=0x7fb2e014da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:31.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.988+0000 7fb2e57f1640 1 --2- 172.21.15.112:0/1739357491 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb2e00cfbc0 0x7fb2e014da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:31.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.988+0000 7fb2e57f1640 1 -- 172.21.15.112:0/1739357491 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb2e015f900 msgr2=0x7fb2e00ce1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:31.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.988+0000 7fb2e57f1640 1 --2- 172.21.15.112:0/1739357491 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb2e015f900 0x7fb2e00ce1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:31.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.988+0000 7fb2e57f1640 1 -- 172.21.15.112:0/1739357491 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2e015ad10 con 0x7fb2e00b37f0 2024-09-21T00:30:31.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.988+0000 7fb2e57f1640 1 --2- 172.21.15.112:0/1739357491 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2e00b37f0 0x7fb2e015a7d0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc02aa70 tx=0x7fb2dc033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:31.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.988+0000 7fb2d5ffb640 1 -- 172.21.15.112:0/1739357491 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2dc031d60 con 0x7fb2e00b37f0 2024-09-21T00:30:31.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.988+0000 7fb2d5ffb640 1 -- 172.21.15.112:0/1739357491 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb2dc033e10 con 0x7fb2e00b37f0 2024-09-21T00:30:31.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.988+0000 7fb2d5ffb640 1 -- 172.21.15.112:0/1739357491 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2dc005260 con 0x7fb2e00b37f0 2024-09-21T00:30:31.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.988+0000 7fb2e7a55640 1 -- 172.21.15.112:0/1739357491 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2e014dfc0 con 0x7fb2e00b37f0 2024-09-21T00:30:31.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.988+0000 7fb2e7a55640 1 -- 172.21.15.112:0/1739357491 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2e0159a10 con 0x7fb2e00b37f0 2024-09-21T00:30:31.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.989+0000 7fb2e7a55640 1 -- 172.21.15.112:0/1739357491 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2e006e770 con 0x7fb2e00b37f0 2024-09-21T00:30:31.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.990+0000 7fb2d5ffb640 1 -- 172.21.15.112:0/1739357491 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb2dc005400 con 0x7fb2e00b37f0 2024-09-21T00:30:31.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.990+0000 7fb2d5ffb640 1 --2- 172.21.15.112:0/1739357491 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fb2c4078bc0 0x7fb2c407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:31.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.990+0000 7fb2d5ffb640 1 -- 172.21.15.112:0/1739357491 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5507+0+0 (secure 0 0 0) 0x7fb2dc0bbed0 con 0x7fb2e00b37f0 2024-09-21T00:30:31.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.990+0000 7fb2e4ff0640 1 --2- 172.21.15.112:0/1739357491 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fb2c4078bc0 0x7fb2c407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:31.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.991+0000 7fb2e4ff0640 1 --2- 172.21.15.112:0/1739357491 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fb2c4078bc0 0x7fb2c407b0d0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fb2d0006fd0 tx=0x7fb2d0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:31.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:31.995+0000 7fb2d5ffb640 1 -- 172.21.15.112:0/1739357491 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb2dc03f030 con 0x7fb2e00b37f0 2024-09-21T00:30:32.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.175+0000 7fb2e7a55640 1 -- 172.21.15.112:0/1739357491 --> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7fb2e00ce6f0 con 0x7fb2c4078bc0 2024-09-21T00:30:32.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.176+0000 7fb2d5ffb640 1 -- 172.21.15.112:0/1739357491 <== mgr.4103 v2:172.21.15.187:6800/2921522906 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fb2e00ce6f0 con 0x7fb2c4078bc0 2024-09-21T00:30:32.178 INFO:teuthology.orchestra.run.smithi112.stderr:instructed osd(s) 3 to deep-scrub 2024-09-21T00:30:32.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.178+0000 7fb2e7a55640 1 -- 172.21.15.112:0/1739357491 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fb2c4078bc0 msgr2=0x7fb2c407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:32.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.178+0000 7fb2e7a55640 1 --2- 172.21.15.112:0/1739357491 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fb2c4078bc0 0x7fb2c407b0d0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fb2d0006fd0 tx=0x7fb2d0008040 comp rx=0 tx=0).stop 2024-09-21T00:30:32.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.178+0000 7fb2e7a55640 1 -- 172.21.15.112:0/1739357491 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2e00b37f0 msgr2=0x7fb2e015a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:32.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.178+0000 7fb2e7a55640 1 --2- 172.21.15.112:0/1739357491 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2e00b37f0 0x7fb2e015a7d0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc02aa70 tx=0x7fb2dc033c20 comp rx=0 tx=0).stop 2024-09-21T00:30:32.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.179+0000 7fb2e7a55640 1 -- 172.21.15.112:0/1739357491 shutdown_connections 2024-09-21T00:30:32.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.179+0000 7fb2e7a55640 1 --2- 172.21.15.112:0/1739357491 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb2e015f900 0x7fb2e00ce1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:32.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.179+0000 7fb2e7a55640 1 --2- 172.21.15.112:0/1739357491 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb2e00cfbc0 0x7fb2e014da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:32.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.179+0000 7fb2e7a55640 1 --2- 172.21.15.112:0/1739357491 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fb2c4078bc0 0x7fb2c407b0d0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:32.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.179+0000 7fb2e7a55640 1 --2- 172.21.15.112:0/1739357491 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2e00b37f0 0x7fb2e015a7d0 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:32.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.179+0000 7fb2e7a55640 1 -- 172.21.15.112:0/1739357491 >> 172.21.15.112:0/1739357491 conn(0x7fb2e00b6050 msgr2=0x7fb2e0160600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:32.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.179+0000 7fb2e7a55640 1 -- 172.21.15.112:0/1739357491 shutdown_connections 2024-09-21T00:30:32.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.179+0000 7fb2e7a55640 1 -- 172.21.15.112:0/1739357491 wait complete. 2024-09-21T00:30:32.192 INFO:tasks.ceph:Scrubbing osd.4 2024-09-21T00:30:32.192 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-09-21T00:30:32.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.307+0000 7fed09f13640 1 -- 172.21.15.112:0/1100726469 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fed040d1450 msgr2=0x7fed040b47d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:32.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.307+0000 7fed09f13640 1 --2- 172.21.15.112:0/1100726469 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fed040d1450 0x7fed040b47d0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7fecfc005fd0 tx=0x7fecfc02a8b0 comp rx=0 tx=0).stop 2024-09-21T00:30:32.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.307+0000 7fed09f13640 1 -- 172.21.15.112:0/1100726469 shutdown_connections 2024-09-21T00:30:32.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.307+0000 7fed09f13640 1 --2- 172.21.15.112:0/1100726469 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fed040d1450 0x7fed040b47d0 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:32.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.307+0000 7fed09f13640 1 --2- 172.21.15.112:0/1100726469 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fed040b80e0 0x7fed040b4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:32.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.307+0000 7fed09f13640 1 -- 172.21.15.112:0/1100726469 >> 172.21.15.112:0/1100726469 conn(0x7fed040cf300 msgr2=0x7fed040cf710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:32.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.307+0000 7fed09f13640 1 -- 172.21.15.112:0/1100726469 shutdown_connections 2024-09-21T00:30:32.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.307+0000 7fed09f13640 1 -- 172.21.15.112:0/1100726469 wait complete. 2024-09-21T00:30:32.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.308+0000 7fed09f13640 1 Processor -- start 2024-09-21T00:30:32.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.309+0000 7fed09f13640 1 -- start start 2024-09-21T00:30:32.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.310+0000 7fed09f13640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fed040b7b90 0x7fed0414f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:32.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.310+0000 7fed09f13640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fed040b80e0 0x7fed041f4550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:32.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.310+0000 7fed09f13640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fed040d1450 0x7fed04068d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:32.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.310+0000 7fed09f13640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fed040b2860 con 0x7fed040d1450 2024-09-21T00:30:32.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.310+0000 7fed09f13640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fed040b26e0 con 0x7fed040b80e0 2024-09-21T00:30:32.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.310+0000 7fed09f13640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fed040b29e0 con 0x7fed040b7b90 2024-09-21T00:30:32.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.310+0000 7fed03fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fed040d1450 0x7fed04068d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:32.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.310+0000 7fed03fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fed040d1450 0x7fed04068d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56694/0 (socket says 172.21.15.112:56694) 2024-09-21T00:30:32.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.310+0000 7fed03fff640 1 -- 172.21.15.112:0/3949175841 learned_addr learned my addr 172.21.15.112:0/3949175841 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:30:32.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.310+0000 7fed03fff640 1 -- 172.21.15.112:0/3949175841 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fed040b7b90 msgr2=0x7fed0414f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:32.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.310+0000 7fed03fff640 1 --2- 172.21.15.112:0/3949175841 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fed040b7b90 0x7fed0414f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:32.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.310+0000 7fed03fff640 1 -- 172.21.15.112:0/3949175841 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fed040b80e0 msgr2=0x7fed041f4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:32.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.310+0000 7fed03fff640 1 --2- 172.21.15.112:0/3949175841 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fed040b80e0 0x7fed041f4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:32.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.310+0000 7fed03fff640 1 -- 172.21.15.112:0/3949175841 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed041504b0 con 0x7fed040d1450 2024-09-21T00:30:32.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.310+0000 7fed03fff640 1 --2- 172.21.15.112:0/3949175841 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fed040d1450 0x7fed04068d00 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7fecfc009cd0 tx=0x7fecfc003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:32.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.311+0000 7fecebfff640 1 -- 172.21.15.112:0/3949175841 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecfc031e60 con 0x7fed040d1450 2024-09-21T00:30:32.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.311+0000 7fecebfff640 1 -- 172.21.15.112:0/3949175841 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fecfc004540 con 0x7fed040d1450 2024-09-21T00:30:32.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.311+0000 7fed09f13640 1 -- 172.21.15.112:0/3949175841 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed04155400 con 0x7fed040d1450 2024-09-21T00:30:32.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.311+0000 7fecebfff640 1 -- 172.21.15.112:0/3949175841 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecfc003e10 con 0x7fed040d1450 2024-09-21T00:30:32.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.311+0000 7fed09f13640 1 -- 172.21.15.112:0/3949175841 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed0414ec20 con 0x7fed040d1450 2024-09-21T00:30:32.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.311+0000 7fed09f13640 1 -- 172.21.15.112:0/3949175841 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fed040b72c0 con 0x7fed040d1450 2024-09-21T00:30:32.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.312+0000 7fecebfff640 1 -- 172.21.15.112:0/3949175841 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fecfc03b410 con 0x7fed040d1450 2024-09-21T00:30:32.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.313+0000 7fecebfff640 1 --2- 172.21.15.112:0/3949175841 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fecf0078b90 0x7fecf007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:32.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.313+0000 7fecebfff640 1 -- 172.21.15.112:0/3949175841 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5507+0+0 (secure 0 0 0) 0x7fecfc0bc8e0 con 0x7fed040d1450 2024-09-21T00:30:32.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.313+0000 7fecebfff640 1 --2- 172.21.15.112:0/3949175841 >> [v2:172.21.15.187:6802/522077560,v1:172.21.15.187:6803/522077560] conn(0x7fecf007f1c0 0x7fecf0081650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:32.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.313+0000 7fecebfff640 1 -- 172.21.15.112:0/3949175841 --> [v2:172.21.15.187:6802/522077560,v1:172.21.15.187:6803/522077560] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fecfc032e50 con 0x7fecf007f1c0 2024-09-21T00:30:32.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.313+0000 7fecebfff640 1 -- 172.21.15.112:0/3949175841 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_get_version_reply(handle=1 version=115) ==== 24+0+0 (secure 0 0 0) 0x7fecfc0bcd70 con 0x7fed040d1450 2024-09-21T00:30:32.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.313+0000 7fed037fe640 1 --2- 172.21.15.112:0/3949175841 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fecf0078b90 0x7fecf007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:32.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.313+0000 7fed02ffd640 1 --2- 172.21.15.112:0/3949175841 >> [v2:172.21.15.187:6802/522077560,v1:172.21.15.187:6803/522077560] conn(0x7fecf007f1c0 0x7fecf0081650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:32.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.314+0000 7fed02ffd640 1 --2- 172.21.15.112:0/3949175841 >> [v2:172.21.15.187:6802/522077560,v1:172.21.15.187:6803/522077560] conn(0x7fecf007f1c0 0x7fecf0081650 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:32.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.314+0000 7fed037fe640 1 --2- 172.21.15.112:0/3949175841 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fecf0078b90 0x7fecf007b0a0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fecf4009670 tx=0x7fecf4006bf0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:32.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.315+0000 7fecebfff640 1 -- 172.21.15.112:0/3949175841 <== osd.4 v2:172.21.15.187:6802/522077560 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fecfc032e50 con 0x7fecf007f1c0 2024-09-21T00:30:32.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.327+0000 7fed09f13640 1 -- 172.21.15.112:0/3949175841 --> [v2:172.21.15.187:6802/522077560,v1:172.21.15.187:6803/522077560] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fed0414ee80 con 0x7fecf007f1c0 2024-09-21T00:30:32.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.327+0000 7fecebfff640 1 -- 172.21.15.112:0/3949175841 <== osd.4 v2:172.21.15.187:6802/522077560 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fed0414ee80 con 0x7fecf007f1c0 2024-09-21T00:30:32.329 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-21T00:30:32.329 INFO:teuthology.orchestra.run.smithi112.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T00:30:32.329 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-21T00:30:32.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.328+0000 7fed09f13640 1 -- 172.21.15.112:0/3949175841 >> [v2:172.21.15.187:6802/522077560,v1:172.21.15.187:6803/522077560] conn(0x7fecf007f1c0 msgr2=0x7fecf0081650 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:32.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.328+0000 7fed09f13640 1 --2- 172.21.15.112:0/3949175841 >> [v2:172.21.15.187:6802/522077560,v1:172.21.15.187:6803/522077560] conn(0x7fecf007f1c0 0x7fecf0081650 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:32.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.328+0000 7fed09f13640 1 -- 172.21.15.112:0/3949175841 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fecf0078b90 msgr2=0x7fecf007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:32.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.328+0000 7fed09f13640 1 --2- 172.21.15.112:0/3949175841 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fecf0078b90 0x7fecf007b0a0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fecf4009670 tx=0x7fecf4006bf0 comp rx=0 tx=0).stop 2024-09-21T00:30:32.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.328+0000 7fed09f13640 1 -- 172.21.15.112:0/3949175841 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fed040d1450 msgr2=0x7fed04068d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:32.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.328+0000 7fed09f13640 1 --2- 172.21.15.112:0/3949175841 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fed040d1450 0x7fed04068d00 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7fecfc009cd0 tx=0x7fecfc003c20 comp rx=0 tx=0).stop 2024-09-21T00:30:32.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.328+0000 7fed03fff640 1 -- 172.21.15.112:0/3949175841 reap_dead start 2024-09-21T00:30:32.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.328+0000 7fed09f13640 1 -- 172.21.15.112:0/3949175841 shutdown_connections 2024-09-21T00:30:32.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.328+0000 7fed09f13640 1 -- 172.21.15.112:0/3949175841 >> 172.21.15.112:0/3949175841 conn(0x7fed040cf300 msgr2=0x7fed04071850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:32.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.328+0000 7fed09f13640 1 -- 172.21.15.112:0/3949175841 shutdown_connections 2024-09-21T00:30:32.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.328+0000 7fed09f13640 1 -- 172.21.15.112:0/3949175841 wait complete. 2024-09-21T00:30:32.340 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-09-21T00:30:32.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.452+0000 7f0be55f1640 1 -- 172.21.15.112:0/3225667924 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0be00cd2f0 msgr2=0x7f0be00b1cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:32.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.452+0000 7f0be55f1640 1 --2- 172.21.15.112:0/3225667924 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0be00cd2f0 0x7f0be00b1cd0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f0bc8005fd0 tx=0x7f0bc802a540 comp rx=0 tx=0).stop 2024-09-21T00:30:32.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.452+0000 7f0be55f1640 1 -- 172.21.15.112:0/3225667924 shutdown_connections 2024-09-21T00:30:32.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.452+0000 7f0be55f1640 1 --2- 172.21.15.112:0/3225667924 >> v2:172.21.15.187:3300/0 conn(0x7f0be00b2210 0x7f0be00b52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:32.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.452+0000 7f0be55f1640 1 --2- 172.21.15.112:0/3225667924 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0be00cd2f0 0x7f0be00b1cd0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:32.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.452+0000 7f0be55f1640 1 -- 172.21.15.112:0/3225667924 >> 172.21.15.112:0/3225667924 conn(0x7f0be00b44c0 msgr2=0x7f0be00d2380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:32.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.452+0000 7f0be55f1640 1 -- 172.21.15.112:0/3225667924 shutdown_connections 2024-09-21T00:30:32.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.452+0000 7f0be55f1640 1 -- 172.21.15.112:0/3225667924 wait complete. 2024-09-21T00:30:32.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.453+0000 7f0be55f1640 1 Processor -- start 2024-09-21T00:30:32.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.454+0000 7f0be55f1640 1 -- start start 2024-09-21T00:30:32.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.455+0000 7f0be55f1640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0be00b2210 0x7f0be01e6f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:32.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.455+0000 7f0be55f1640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0be015c8a0 0x7f0be0147670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:32.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.455+0000 7f0be55f1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0be01e6a60 0x7f0be015f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:32.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.455+0000 7f0be55f1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0be00b5f60 con 0x7f0be01e6a60 2024-09-21T00:30:32.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.455+0000 7f0be55f1640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0be00b5de0 con 0x7f0be00b2210 2024-09-21T00:30:32.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.455+0000 7f0be55f1640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f0be00b60e0 con 0x7f0be015c8a0 2024-09-21T00:30:32.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.455+0000 7f0bdf7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0be01e6a60 0x7f0be015f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:32.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.455+0000 7f0bdf7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0be01e6a60 0x7f0be015f170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56702/0 (socket says 172.21.15.112:56702) 2024-09-21T00:30:32.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.455+0000 7f0bdf7fe640 1 -- 172.21.15.112:0/1019135666 learned_addr learned my addr 172.21.15.112:0/1019135666 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:30:32.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.455+0000 7f0bd67fc640 1 --2- 172.21.15.112:0/1019135666 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0be015c8a0 0x7f0be0147670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:32.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.455+0000 7f0bdeffd640 1 --2- 172.21.15.112:0/1019135666 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0be00b2210 0x7f0be01e6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:32.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.455+0000 7f0bdf7fe640 1 -- 172.21.15.112:0/1019135666 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0be015c8a0 msgr2=0x7f0be0147670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:32.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.455+0000 7f0bdf7fe640 1 --2- 172.21.15.112:0/1019135666 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0be015c8a0 0x7f0be0147670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:32.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.455+0000 7f0bdf7fe640 1 -- 172.21.15.112:0/1019135666 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0be00b2210 msgr2=0x7f0be01e6f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:32.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.455+0000 7f0bdf7fe640 1 --2- 172.21.15.112:0/1019135666 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0be00b2210 0x7f0be01e6f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:32.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.455+0000 7f0bdf7fe640 1 -- 172.21.15.112:0/1019135666 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0be01e6200 con 0x7f0be01e6a60 2024-09-21T00:30:32.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.456+0000 7f0bdf7fe640 1 --2- 172.21.15.112:0/1019135666 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0be01e6a60 0x7f0be015f170 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f0bd800ec20 tx=0x7f0bd800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:32.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.456+0000 7f0bd7fff640 1 -- 172.21.15.112:0/1019135666 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd8003b70 con 0x7f0be01e6a60 2024-09-21T00:30:32.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.456+0000 7f0bd7fff640 1 -- 172.21.15.112:0/1019135666 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0bd8003d10 con 0x7f0be01e6a60 2024-09-21T00:30:32.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.456+0000 7f0be55f1640 1 -- 172.21.15.112:0/1019135666 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0be01e5160 con 0x7f0be01e6a60 2024-09-21T00:30:32.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.456+0000 7f0bd7fff640 1 -- 172.21.15.112:0/1019135666 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd8004280 con 0x7f0be01e6a60 2024-09-21T00:30:32.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.456+0000 7f0be55f1640 1 -- 172.21.15.112:0/1019135666 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0be01e5720 con 0x7f0be01e6a60 2024-09-21T00:30:32.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.457+0000 7f0be55f1640 1 -- 172.21.15.112:0/1019135666 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0be00b6590 con 0x7f0be01e6a60 2024-09-21T00:30:32.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.458+0000 7f0bd7fff640 1 -- 172.21.15.112:0/1019135666 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0bd8004420 con 0x7f0be01e6a60 2024-09-21T00:30:32.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.458+0000 7f0bd7fff640 1 --2- 172.21.15.112:0/1019135666 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f0bc0078b90 0x7f0bc007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:32.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.458+0000 7f0bd7fff640 1 -- 172.21.15.112:0/1019135666 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5507+0+0 (secure 0 0 0) 0x7f0bd809b810 con 0x7f0be01e6a60 2024-09-21T00:30:32.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.459+0000 7f0bdeffd640 1 --2- 172.21.15.112:0/1019135666 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f0bc0078b90 0x7f0bc007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:32.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.459+0000 7f0bdeffd640 1 --2- 172.21.15.112:0/1019135666 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f0bc0078b90 0x7f0bc007b0a0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f0bd0007fd0 tx=0x7f0bd0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:32.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.463+0000 7f0bd7fff640 1 -- 172.21.15.112:0/1019135666 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f0bd8063640 con 0x7f0be01e6a60 2024-09-21T00:30:32.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.639+0000 7f0be55f1640 1 -- 172.21.15.112:0/1019135666 --> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7f0be00cfb40 con 0x7f0bc0078b90 2024-09-21T00:30:32.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.640+0000 7f0bd7fff640 1 -- 172.21.15.112:0/1019135666 <== mgr.4103 v2:172.21.15.187:6800/2921522906 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f0be00cfb40 con 0x7f0bc0078b90 2024-09-21T00:30:32.641 INFO:teuthology.orchestra.run.smithi112.stderr:instructed osd(s) 4 to deep-scrub 2024-09-21T00:30:32.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.642+0000 7f0be55f1640 1 -- 172.21.15.112:0/1019135666 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f0bc0078b90 msgr2=0x7f0bc007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:32.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.642+0000 7f0be55f1640 1 --2- 172.21.15.112:0/1019135666 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f0bc0078b90 0x7f0bc007b0a0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f0bd0007fd0 tx=0x7f0bd0009040 comp rx=0 tx=0).stop 2024-09-21T00:30:32.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.642+0000 7f0be55f1640 1 -- 172.21.15.112:0/1019135666 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0be01e6a60 msgr2=0x7f0be015f170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:32.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.642+0000 7f0be55f1640 1 --2- 172.21.15.112:0/1019135666 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0be01e6a60 0x7f0be015f170 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f0bd800ec20 tx=0x7f0bd800c6a0 comp rx=0 tx=0).stop 2024-09-21T00:30:32.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.642+0000 7f0be55f1640 1 -- 172.21.15.112:0/1019135666 shutdown_connections 2024-09-21T00:30:32.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.642+0000 7f0be55f1640 1 --2- 172.21.15.112:0/1019135666 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f0bc0078b90 0x7f0bc007b0a0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:32.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.642+0000 7f0be55f1640 1 --2- 172.21.15.112:0/1019135666 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0be01e6a60 0x7f0be015f170 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:32.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.642+0000 7f0be55f1640 1 --2- 172.21.15.112:0/1019135666 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0be015c8a0 0x7f0be0147670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:32.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.642+0000 7f0be55f1640 1 --2- 172.21.15.112:0/1019135666 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0be00b2210 0x7f0be01e6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:32.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.642+0000 7f0be55f1640 1 -- 172.21.15.112:0/1019135666 >> 172.21.15.112:0/1019135666 conn(0x7f0be00b44c0 msgr2=0x7f0be0154e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:32.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.642+0000 7f0be55f1640 1 -- 172.21.15.112:0/1019135666 shutdown_connections 2024-09-21T00:30:32.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.642+0000 7f0be55f1640 1 -- 172.21.15.112:0/1019135666 wait complete. 2024-09-21T00:30:32.655 INFO:tasks.ceph:Scrubbing osd.5 2024-09-21T00:30:32.655 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-09-21T00:30:32.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.769+0000 7faae6b0b640 1 -- 172.21.15.112:0/1345650373 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faae0070950 msgr2=0x7faae00cbd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:32.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.769+0000 7faae6b0b640 1 --2- 172.21.15.112:0/1345650373 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faae0070950 0x7faae00cbd50 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7faad8009be0 tx=0x7faad802a8d0 comp rx=0 tx=0).stop 2024-09-21T00:30:32.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.769+0000 7faae6b0b640 1 -- 172.21.15.112:0/1345650373 shutdown_connections 2024-09-21T00:30:32.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.769+0000 7faae6b0b640 1 --2- 172.21.15.112:0/1345650373 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faae0070950 0x7faae00cbd50 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:32.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.769+0000 7faae6b0b640 1 --2- 172.21.15.112:0/1345650373 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faae00cd7b0 0x7faae0070410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:32.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.769+0000 7faae6b0b640 1 -- 172.21.15.112:0/1345650373 >> 172.21.15.112:0/1345650373 conn(0x7faae00d1c30 msgr2=0x7faae00cde50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:32.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.769+0000 7faae6b0b640 1 -- 172.21.15.112:0/1345650373 shutdown_connections 2024-09-21T00:30:32.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.769+0000 7faae6b0b640 1 -- 172.21.15.112:0/1345650373 wait complete. 2024-09-21T00:30:32.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.771+0000 7faae6b0b640 1 Processor -- start 2024-09-21T00:30:32.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.771+0000 7faae6b0b640 1 -- start start 2024-09-21T00:30:32.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.772+0000 7faae6b0b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faae0070950 0x7faae00a9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:32.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.772+0000 7faae6b0b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faae00cd7b0 0x7faae00c72b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:32.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.772+0000 7faae6b0b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faae00ce950 0x7faae00c9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:32.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.772+0000 7faae6b0b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7faae00cc650 con 0x7faae00cd7b0 2024-09-21T00:30:32.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.772+0000 7faae6b0b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7faae00cc4d0 con 0x7faae0070950 2024-09-21T00:30:32.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.772+0000 7faae6b0b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7faae00cc7d0 con 0x7faae00ce950 2024-09-21T00:30:32.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.772+0000 7faadffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faae00cd7b0 0x7faae00c72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:32.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.772+0000 7faadffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faae00cd7b0 0x7faae00c72b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56710/0 (socket says 172.21.15.112:56710) 2024-09-21T00:30:32.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.772+0000 7faadffff640 1 -- 172.21.15.112:0/2306520130 learned_addr learned my addr 172.21.15.112:0/2306520130 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:30:32.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.772+0000 7faae50a8640 1 --2- 172.21.15.112:0/2306520130 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faae00ce950 0x7faae00c9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:32.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.772+0000 7faadffff640 1 -- 172.21.15.112:0/2306520130 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faae00ce950 msgr2=0x7faae00c9dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:32.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.772+0000 7faadffff640 1 --2- 172.21.15.112:0/2306520130 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faae00ce950 0x7faae00c9dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:32.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.772+0000 7faadffff640 1 -- 172.21.15.112:0/2306520130 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faae0070950 msgr2=0x7faae00a9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:32.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.772+0000 7faae48a7640 1 --2- 172.21.15.112:0/2306520130 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faae0070950 0x7faae00a9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:32.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.773+0000 7faadffff640 1 --2- 172.21.15.112:0/2306520130 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faae0070950 0x7faae00a9ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:32.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.773+0000 7faadffff640 1 -- 172.21.15.112:0/2306520130 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faae00aa8b0 con 0x7faae00cd7b0 2024-09-21T00:30:32.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.773+0000 7faae48a7640 1 --2- 172.21.15.112:0/2306520130 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faae0070950 0x7faae00a9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T00:30:32.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.773+0000 7faadffff640 1 --2- 172.21.15.112:0/2306520130 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faae00cd7b0 0x7faae00c72b0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7faad400c970 tx=0x7faad400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:32.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.773+0000 7faadd7fa640 1 -- 172.21.15.112:0/2306520130 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faad4007bb0 con 0x7faae00cd7b0 2024-09-21T00:30:32.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.773+0000 7faadd7fa640 1 -- 172.21.15.112:0/2306520130 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faad4007d50 con 0x7faae00cd7b0 2024-09-21T00:30:32.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.773+0000 7faadd7fa640 1 -- 172.21.15.112:0/2306520130 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faad4005280 con 0x7faae00cd7b0 2024-09-21T00:30:32.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.773+0000 7faae6b0b640 1 -- 172.21.15.112:0/2306520130 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faae00c8480 con 0x7faae00cd7b0 2024-09-21T00:30:32.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.773+0000 7faae6b0b640 1 -- 172.21.15.112:0/2306520130 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faae00c8a40 con 0x7faae00cd7b0 2024-09-21T00:30:32.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.774+0000 7faae6b0b640 1 -- 172.21.15.112:0/2306520130 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7faae00ccd20 con 0x7faae00cd7b0 2024-09-21T00:30:32.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.775+0000 7faadd7fa640 1 -- 172.21.15.112:0/2306520130 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7faad4005420 con 0x7faae00cd7b0 2024-09-21T00:30:32.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.775+0000 7faadd7fa640 1 --2- 172.21.15.112:0/2306520130 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7faacc078bc0 0x7faacc07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:32.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.775+0000 7faadd7fa640 1 -- 172.21.15.112:0/2306520130 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5507+0+0 (secure 0 0 0) 0x7faad409b0f0 con 0x7faae00cd7b0 2024-09-21T00:30:32.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.775+0000 7faadd7fa640 1 --2- 172.21.15.112:0/2306520130 >> [v2:172.21.15.187:6826/3593144721,v1:172.21.15.187:6827/3593144721] conn(0x7faacc07f1f0 0x7faacc081680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:32.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.775+0000 7faae48a7640 1 --2- 172.21.15.112:0/2306520130 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7faacc078bc0 0x7faacc07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:32.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.775+0000 7faadd7fa640 1 -- 172.21.15.112:0/2306520130 --> [v2:172.21.15.187:6826/3593144721,v1:172.21.15.187:6827/3593144721] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7faad4004490 con 0x7faacc07f1f0 2024-09-21T00:30:32.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.775+0000 7faadd7fa640 1 -- 172.21.15.112:0/2306520130 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_get_version_reply(handle=1 version=115) ==== 24+0+0 (secure 0 0 0) 0x7faad40a2050 con 0x7faae00cd7b0 2024-09-21T00:30:32.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.775+0000 7faae50a8640 1 --2- 172.21.15.112:0/2306520130 >> [v2:172.21.15.187:6826/3593144721,v1:172.21.15.187:6827/3593144721] conn(0x7faacc07f1f0 0x7faacc081680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:32.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.776+0000 7faae48a7640 1 --2- 172.21.15.112:0/2306520130 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7faacc078bc0 0x7faacc07b0d0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7faad0000f60 tx=0x7faad0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:32.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.776+0000 7faae50a8640 1 --2- 172.21.15.112:0/2306520130 >> [v2:172.21.15.187:6826/3593144721,v1:172.21.15.187:6827/3593144721] conn(0x7faacc07f1f0 0x7faacc081680 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:32.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.777+0000 7faadd7fa640 1 -- 172.21.15.112:0/2306520130 <== osd.5 v2:172.21.15.187:6826/3593144721 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7faad4004490 con 0x7faacc07f1f0 2024-09-21T00:30:32.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.789+0000 7faae6b0b640 1 -- 172.21.15.112:0/2306520130 --> [v2:172.21.15.187:6826/3593144721,v1:172.21.15.187:6827/3593144721] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7faae00aa660 con 0x7faacc07f1f0 2024-09-21T00:30:32.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.790+0000 7faadd7fa640 1 -- 172.21.15.112:0/2306520130 <== osd.5 v2:172.21.15.187:6826/3593144721 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7faae00aa660 con 0x7faacc07f1f0 2024-09-21T00:30:32.791 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-21T00:30:32.791 INFO:teuthology.orchestra.run.smithi112.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T00:30:32.791 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-21T00:30:32.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.790+0000 7faae6b0b640 1 -- 172.21.15.112:0/2306520130 >> [v2:172.21.15.187:6826/3593144721,v1:172.21.15.187:6827/3593144721] conn(0x7faacc07f1f0 msgr2=0x7faacc081680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:32.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.790+0000 7faae6b0b640 1 --2- 172.21.15.112:0/2306520130 >> [v2:172.21.15.187:6826/3593144721,v1:172.21.15.187:6827/3593144721] conn(0x7faacc07f1f0 0x7faacc081680 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:32.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.790+0000 7faae6b0b640 1 -- 172.21.15.112:0/2306520130 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7faacc078bc0 msgr2=0x7faacc07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:32.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.790+0000 7faae6b0b640 1 --2- 172.21.15.112:0/2306520130 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7faacc078bc0 0x7faacc07b0d0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7faad0000f60 tx=0x7faad0009040 comp rx=0 tx=0).stop 2024-09-21T00:30:32.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.790+0000 7faae6b0b640 1 -- 172.21.15.112:0/2306520130 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faae00cd7b0 msgr2=0x7faae00c72b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:32.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.790+0000 7faae6b0b640 1 --2- 172.21.15.112:0/2306520130 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faae00cd7b0 0x7faae00c72b0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7faad400c970 tx=0x7faad400ce30 comp rx=0 tx=0).stop 2024-09-21T00:30:32.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.790+0000 7faae50a8640 1 -- 172.21.15.112:0/2306520130 reap_dead start 2024-09-21T00:30:32.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.790+0000 7faae6b0b640 1 -- 172.21.15.112:0/2306520130 shutdown_connections 2024-09-21T00:30:32.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.790+0000 7faae6b0b640 1 -- 172.21.15.112:0/2306520130 >> 172.21.15.112:0/2306520130 conn(0x7faae00d1c30 msgr2=0x7faae015e2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:32.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.790+0000 7faae6b0b640 1 -- 172.21.15.112:0/2306520130 shutdown_connections 2024-09-21T00:30:32.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.790+0000 7faae6b0b640 1 -- 172.21.15.112:0/2306520130 wait complete. 2024-09-21T00:30:32.802 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-09-21T00:30:32.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.914+0000 7fb43edc3640 1 -- 172.21.15.112:0/291502668 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb4381602e0 msgr2=0x7fb4381606f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:32.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.914+0000 7fb43edc3640 1 --2- 172.21.15.112:0/291502668 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb4381602e0 0x7fb4381606f0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7fb438157360 tx=0x7fb43002a8b0 comp rx=0 tx=0).stop 2024-09-21T00:30:32.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.914+0000 7fb43edc3640 1 -- 172.21.15.112:0/291502668 shutdown_connections 2024-09-21T00:30:32.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.914+0000 7fb43edc3640 1 --2- 172.21.15.112:0/291502668 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb4381602e0 0x7fb4381606f0 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:32.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.914+0000 7fb43edc3640 1 --2- 172.21.15.112:0/291502668 >> v2:172.21.15.187:3300/0 conn(0x7fb4380b4530 0x7fb4380b3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:32.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.914+0000 7fb43edc3640 1 -- 172.21.15.112:0/291502668 >> 172.21.15.112:0/291502668 conn(0x7fb4380cf020 msgr2=0x7fb4380cf430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:32.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.914+0000 7fb43edc3640 1 -- 172.21.15.112:0/291502668 shutdown_connections 2024-09-21T00:30:32.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.914+0000 7fb43edc3640 1 -- 172.21.15.112:0/291502668 wait complete. 2024-09-21T00:30:32.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.915+0000 7fb43edc3640 1 Processor -- start 2024-09-21T00:30:32.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.916+0000 7fb43edc3640 1 -- start start 2024-09-21T00:30:32.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.917+0000 7fb43edc3640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb4380b4530 0x7fb4380cbee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:32.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.917+0000 7fb43edc3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb4381602e0 0x7fb4380ca5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:32.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.917+0000 7fb43edc3640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb4380b79f0 0x7fb4380caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:32.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.917+0000 7fb43edc3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb4380d0ff0 con 0x7fb4381602e0 2024-09-21T00:30:32.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.917+0000 7fb43edc3640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb4380d0e70 con 0x7fb4380b4530 2024-09-21T00:30:32.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.917+0000 7fb43edc3640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fb4380d1170 con 0x7fb4380b79f0 2024-09-21T00:30:32.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.917+0000 7fb42ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb4381602e0 0x7fb4380ca5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:32.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.917+0000 7fb42ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb4381602e0 0x7fb4380ca5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56734/0 (socket says 172.21.15.112:56734) 2024-09-21T00:30:32.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.917+0000 7fb42ffff640 1 -- 172.21.15.112:0/3857690868 learned_addr learned my addr 172.21.15.112:0/3857690868 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:30:32.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.917+0000 7fb42ffff640 1 -- 172.21.15.112:0/3857690868 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb4380b79f0 msgr2=0x7fb4380caaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:32.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.917+0000 7fb42ffff640 1 --2- 172.21.15.112:0/3857690868 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb4380b79f0 0x7fb4380caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:32.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.917+0000 7fb42ffff640 1 -- 172.21.15.112:0/3857690868 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb4380b4530 msgr2=0x7fb4380cbee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:32.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.917+0000 7fb43cb5f640 1 --2- 172.21.15.112:0/3857690868 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb4380b4530 0x7fb4380cbee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:32.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.917+0000 7fb42ffff640 1 --2- 172.21.15.112:0/3857690868 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb4380b4530 0x7fb4380cbee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:32.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.917+0000 7fb42ffff640 1 -- 172.21.15.112:0/3857690868 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb438158360 con 0x7fb4381602e0 2024-09-21T00:30:32.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.918+0000 7fb42ffff640 1 --2- 172.21.15.112:0/3857690868 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb4381602e0 0x7fb4380ca5b0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7fb42800c970 tx=0x7fb42800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:32.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.918+0000 7fb42d7fa640 1 -- 172.21.15.112:0/3857690868 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb428004220 con 0x7fb4381602e0 2024-09-21T00:30:32.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.918+0000 7fb42d7fa640 1 -- 172.21.15.112:0/3857690868 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb4280044b0 con 0x7fb4381602e0 2024-09-21T00:30:32.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.918+0000 7fb42d7fa640 1 -- 172.21.15.112:0/3857690868 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb428007c10 con 0x7fb4381602e0 2024-09-21T00:30:32.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.918+0000 7fb43edc3640 1 -- 172.21.15.112:0/3857690868 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4380b6950 con 0x7fb4381602e0 2024-09-21T00:30:32.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.918+0000 7fb43edc3640 1 -- 172.21.15.112:0/3857690868 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4380b6f10 con 0x7fb4381602e0 2024-09-21T00:30:32.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.919+0000 7fb43edc3640 1 -- 172.21.15.112:0/3857690868 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4380b66c0 con 0x7fb4381602e0 2024-09-21T00:30:32.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.920+0000 7fb42d7fa640 1 -- 172.21.15.112:0/3857690868 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb428003610 con 0x7fb4381602e0 2024-09-21T00:30:32.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.920+0000 7fb42d7fa640 1 --2- 172.21.15.112:0/3857690868 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fb424078b90 0x7fb42407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:32.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.920+0000 7fb42d7fa640 1 -- 172.21.15.112:0/3857690868 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5507+0+0 (secure 0 0 0) 0x7fb42809b040 con 0x7fb4381602e0 2024-09-21T00:30:32.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.920+0000 7fb43cb5f640 1 --2- 172.21.15.112:0/3857690868 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fb424078b90 0x7fb42407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:32.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.921+0000 7fb43cb5f640 1 --2- 172.21.15.112:0/3857690868 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fb424078b90 0x7fb42407b0a0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fb434001530 tx=0x7fb434009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:32.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:32.925+0000 7fb42d7fa640 1 -- 172.21.15.112:0/3857690868 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb428014030 con 0x7fb4381602e0 2024-09-21T00:30:33.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:33.101+0000 7fb43edc3640 1 -- 172.21.15.112:0/3857690868 --> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7fb4380cc420 con 0x7fb424078b90 2024-09-21T00:30:33.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:33.101+0000 7fb42d7fa640 1 -- 172.21.15.112:0/3857690868 <== mgr.4103 v2:172.21.15.187:6800/2921522906 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fb4380cc420 con 0x7fb424078b90 2024-09-21T00:30:33.103 INFO:teuthology.orchestra.run.smithi112.stderr:instructed osd(s) 5 to deep-scrub 2024-09-21T00:30:33.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:33.103+0000 7fb43edc3640 1 -- 172.21.15.112:0/3857690868 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fb424078b90 msgr2=0x7fb42407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:33.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:33.103+0000 7fb43edc3640 1 --2- 172.21.15.112:0/3857690868 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fb424078b90 0x7fb42407b0a0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fb434001530 tx=0x7fb434009040 comp rx=0 tx=0).stop 2024-09-21T00:30:33.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:33.103+0000 7fb43edc3640 1 -- 172.21.15.112:0/3857690868 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb4381602e0 msgr2=0x7fb4380ca5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:33.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:33.103+0000 7fb43edc3640 1 --2- 172.21.15.112:0/3857690868 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb4381602e0 0x7fb4380ca5b0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7fb42800c970 tx=0x7fb42800ce30 comp rx=0 tx=0).stop 2024-09-21T00:30:33.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:33.103+0000 7fb43edc3640 1 -- 172.21.15.112:0/3857690868 shutdown_connections 2024-09-21T00:30:33.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:33.103+0000 7fb43edc3640 1 --2- 172.21.15.112:0/3857690868 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb4380b79f0 0x7fb4380caaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:33.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:33.103+0000 7fb43edc3640 1 --2- 172.21.15.112:0/3857690868 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7fb424078b90 0x7fb42407b0a0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:33.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:33.103+0000 7fb43edc3640 1 --2- 172.21.15.112:0/3857690868 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb4381602e0 0x7fb4380ca5b0 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:33.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:33.103+0000 7fb43edc3640 1 --2- 172.21.15.112:0/3857690868 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb4380b4530 0x7fb4380cbee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:33.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:33.103+0000 7fb43edc3640 1 -- 172.21.15.112:0/3857690868 >> 172.21.15.112:0/3857690868 conn(0x7fb4380cf020 msgr2=0x7fb4380cf430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:33.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:33.103+0000 7fb43edc3640 1 -- 172.21.15.112:0/3857690868 shutdown_connections 2024-09-21T00:30:33.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:33.103+0000 7fb43edc3640 1 -- 172.21.15.112:0/3857690868 wait complete. 2024-09-21T00:30:33.116 INFO:tasks.ceph:Scrubbing osd.6 2024-09-21T00:30:33.116 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-09-21T00:30:33.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:33.234+0000 7f5230615640 1 -- 172.21.15.112:0/877302093 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52280b4250 msgr2=0x7f52280b4660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:33.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:33.234+0000 7f5230615640 1 --2- 172.21.15.112:0/877302093 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52280b4250 0x7f52280b4660 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f52280d34a0 tx=0x7f521802a560 comp rx=0 tx=0).stop 2024-09-21T00:30:33.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:33.235+0000 7f5230615640 1 -- 172.21.15.112:0/877302093 shutdown_connections 2024-09-21T00:30:33.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:33.235+0000 7f5230615640 1 --2- 172.21.15.112:0/877302093 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f522815ce20 0x7f522815d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:33.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:33.235+0000 7f5230615640 1 --2- 172.21.15.112:0/877302093 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52280b4250 0x7f52280b4660 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:33.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:33.235+0000 7f5230615640 1 --2- 172.21.15.112:0/877302093 >> v2:172.21.15.187:3300/0 conn(0x7f52280b2a90 0x7f522815a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:33.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:33.235+0000 7f5230615640 1 -- 172.21.15.112:0/877302093 >> 172.21.15.112:0/877302093 conn(0x7f52280cf300 msgr2=0x7f52280cf710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:33.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:33.235+0000 7f5230615640 1 -- 172.21.15.112:0/877302093 shutdown_connections 2024-09-21T00:30:33.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:33.235+0000 7f5230615640 1 -- 172.21.15.112:0/877302093 wait complete. 2024-09-21T00:30:33.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:33.236+0000 7f5230615640 1 Processor -- start 2024-09-21T00:30:33.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:33.236+0000 7f5230615640 1 -- start start 2024-09-21T00:30:33.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:33.238+0000 7f5230615640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f52280b2a90 0x7f52280b7da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:33.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:33.238+0000 7f5230615640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52280b4250 0x7f5228156800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:33.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:33.238+0000 7f5230615640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f522815ce20 0x7f52281f6d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:33.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:33.238+0000 7f5230615640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f52281608b0 con 0x7f52280b4250 2024-09-21T00:30:33.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:33.238+0000 7f5230615640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5228160730 con 0x7f52280b2a90 2024-09-21T00:30:33.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:33.238+0000 7f5230615640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f5228160a30 con 0x7f522815ce20 2024-09-21T00:30:33.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:33.238+0000 7f522dbb0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52280b4250 0x7f5228156800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:33.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:33.238+0000 7f522dbb0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52280b4250 0x7f5228156800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56766/0 (socket says 172.21.15.112:56766) 2024-09-21T00:30:33.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:33.238+0000 7f522dbb0640 1 -- 172.21.15.112:0/139946136 learned_addr learned my addr 172.21.15.112:0/139946136 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:30:33.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:33.238+0000 7f522e3b1640 1 --2- 172.21.15.112:0/139946136 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f52280b2a90 0x7f52280b7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:33.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:33.238+0000 7f522ebb2640 1 --2- 172.21.15.112:0/139946136 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f522815ce20 0x7f52281f6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:33.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:33.238+0000 7f522dbb0640 1 -- 172.21.15.112:0/139946136 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f522815ce20 msgr2=0x7f52281f6d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:33.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:33.238+0000 7f522dbb0640 1 --2- 172.21.15.112:0/139946136 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f522815ce20 0x7f52281f6d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:33.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:33.238+0000 7f522dbb0640 1 -- 172.21.15.112:0/139946136 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f52280b2a90 msgr2=0x7f52280b7da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:33.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:33.238+0000 7f522dbb0640 1 --2- 172.21.15.112:0/139946136 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f52280b2a90 0x7f52280b7da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:33.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:33.238+0000 7f522dbb0640 1 -- 172.21.15.112:0/139946136 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52280b5a00 con 0x7f52280b4250 2024-09-21T00:30:33.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:33.238+0000 7f522e3b1640 1 --2- 172.21.15.112:0/139946136 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f52280b2a90 0x7f52280b7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:30:33.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:33.238+0000 7f522dbb0640 1 --2- 172.21.15.112:0/139946136 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52280b4250 0x7f5228156800 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f5218000c00 tx=0x7f52180039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:33.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:33.238+0000 7f521effd640 1 -- 172.21.15.112:0/139946136 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f521803e030 con 0x7f52280b4250 2024-09-21T00:30:33.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:33.238+0000 7f521effd640 1 -- 172.21.15.112:0/139946136 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f52180047a0 con 0x7f52280b4250 2024-09-21T00:30:33.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:33.238+0000 7f5230615640 1 -- 172.21.15.112:0/139946136 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52280b5ce0 con 0x7f52280b4250 2024-09-21T00:30:33.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:33.238+0000 7f521effd640 1 -- 172.21.15.112:0/139946136 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5218035040 con 0x7f52280b4250 2024-09-21T00:30:33.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:33.238+0000 7f5230615640 1 -- 172.21.15.112:0/139946136 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52280b62a0 con 0x7f52280b4250 2024-09-21T00:30:33.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:33.239+0000 7f5230615640 1 -- 172.21.15.112:0/139946136 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f52280ce950 con 0x7f52280b4250 2024-09-21T00:30:33.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:33.240+0000 7f521effd640 1 -- 172.21.15.112:0/139946136 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5218041030 con 0x7f52280b4250 2024-09-21T00:30:33.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:33.240+0000 7f521effd640 1 --2- 172.21.15.112:0/139946136 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f520c078b90 0x7f520c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:33.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:33.241+0000 7f521effd640 1 -- 172.21.15.112:0/139946136 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5507+0+0 (secure 0 0 0) 0x7f52180bbb00 con 0x7f52280b4250 2024-09-21T00:30:33.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:33.241+0000 7f522e3b1640 1 --2- 172.21.15.112:0/139946136 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f520c078b90 0x7f520c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:33.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:33.241+0000 7f521effd640 1 --2- 172.21.15.112:0/139946136 >> [v2:172.21.15.187:6810/1494387657,v1:172.21.15.187:6811/1494387657] conn(0x7f520c07f1c0 0x7f520c081650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:33.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:33.241+0000 7f521effd640 1 -- 172.21.15.112:0/139946136 --> [v2:172.21.15.187:6810/1494387657,v1:172.21.15.187:6811/1494387657] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f521804de50 con 0x7f520c07f1c0 2024-09-21T00:30:33.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:33.241+0000 7f521effd640 1 -- 172.21.15.112:0/139946136 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_get_version_reply(handle=1 version=115) ==== 24+0+0 (secure 0 0 0) 0x7f52180bbf90 con 0x7f52280b4250 2024-09-21T00:30:33.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:33.241+0000 7f522ebb2640 1 --2- 172.21.15.112:0/139946136 >> [v2:172.21.15.187:6810/1494387657,v1:172.21.15.187:6811/1494387657] conn(0x7f520c07f1c0 0x7f520c081650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:33.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:33.241+0000 7f522e3b1640 1 --2- 172.21.15.112:0/139946136 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f520c078b90 0x7f520c07b0a0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f5224005fd0 tx=0x7f5224005e20 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:33.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:33.241+0000 7f522ebb2640 1 --2- 172.21.15.112:0/139946136 >> [v2:172.21.15.187:6810/1494387657,v1:172.21.15.187:6811/1494387657] conn(0x7f520c07f1c0 0x7f520c081650 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:33.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:33.242+0000 7f521effd640 1 -- 172.21.15.112:0/139946136 <== osd.6 v2:172.21.15.187:6810/1494387657 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f521804de50 con 0x7f520c07f1c0 2024-09-21T00:30:33.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:33.254+0000 7f5230615640 1 -- 172.21.15.112:0/139946136 --> [v2:172.21.15.187:6810/1494387657,v1:172.21.15.187:6811/1494387657] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f52280b57b0 con 0x7f520c07f1c0 2024-09-21T00:30:33.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:33.255+0000 7f521effd640 1 -- 172.21.15.112:0/139946136 <== osd.6 v2:172.21.15.187:6810/1494387657 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f52280b57b0 con 0x7f520c07f1c0 2024-09-21T00:30:33.865 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-21T00:30:33.865 INFO:teuthology.orchestra.run.smithi112.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T00:30:33.865 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-21T00:30:33.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:33.256+0000 7f5230615640 1 -- 172.21.15.112:0/139946136 >> [v2:172.21.15.187:6810/1494387657,v1:172.21.15.187:6811/1494387657] conn(0x7f520c07f1c0 msgr2=0x7f520c081650 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:33.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:33.256+0000 7f5230615640 1 --2- 172.21.15.112:0/139946136 >> [v2:172.21.15.187:6810/1494387657,v1:172.21.15.187:6811/1494387657] conn(0x7f520c07f1c0 0x7f520c081650 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:33.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:33.256+0000 7f5230615640 1 -- 172.21.15.112:0/139946136 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f520c078b90 msgr2=0x7f520c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:33.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:33.256+0000 7f5230615640 1 --2- 172.21.15.112:0/139946136 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f520c078b90 0x7f520c07b0a0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f5224005fd0 tx=0x7f5224005e20 comp rx=0 tx=0).stop 2024-09-21T00:30:33.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:33.256+0000 7f5230615640 1 -- 172.21.15.112:0/139946136 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52280b4250 msgr2=0x7f5228156800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:33.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:33.256+0000 7f5230615640 1 --2- 172.21.15.112:0/139946136 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52280b4250 0x7f5228156800 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f5218000c00 tx=0x7f52180039e0 comp rx=0 tx=0).stop 2024-09-21T00:30:33.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:33.256+0000 7f522ebb2640 1 -- 172.21.15.112:0/139946136 reap_dead start 2024-09-21T00:30:33.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:33.256+0000 7f5230615640 1 -- 172.21.15.112:0/139946136 shutdown_connections 2024-09-21T00:30:33.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:33.256+0000 7f5230615640 1 -- 172.21.15.112:0/139946136 >> 172.21.15.112:0/139946136 conn(0x7f52280cf300 msgr2=0x7f5228071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:33.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:33.256+0000 7f5230615640 1 -- 172.21.15.112:0/139946136 shutdown_connections 2024-09-21T00:30:33.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:33.256+0000 7f5230615640 1 -- 172.21.15.112:0/139946136 wait complete. 2024-09-21T00:30:33.868 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-09-21T00:30:33.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:33.993+0000 7ff46fcbd640 1 -- 172.21.15.112:0/1035330142 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff468164530 msgr2=0x7ff468166950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:33.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:33.993+0000 7ff46fcbd640 1 --2- 172.21.15.112:0/1035330142 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff468164530 0x7ff468166950 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7ff4680cedb0 tx=0x7ff46002a820 comp rx=0 tx=0).stop 2024-09-21T00:30:33.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:33.993+0000 7ff46fcbd640 1 -- 172.21.15.112:0/1035330142 shutdown_connections 2024-09-21T00:30:33.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:33.993+0000 7ff46fcbd640 1 --2- 172.21.15.112:0/1035330142 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff468164530 0x7ff468166950 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:33.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:33.993+0000 7ff46fcbd640 1 --2- 172.21.15.112:0/1035330142 >> v2:172.21.15.187:3300/0 conn(0x7ff4681603c0 0x7ff4681607d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:33.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:33.993+0000 7ff46fcbd640 1 -- 172.21.15.112:0/1035330142 >> 172.21.15.112:0/1035330142 conn(0x7ff468073e60 msgr2=0x7ff468074270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:33.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:33.994+0000 7ff46fcbd640 1 -- 172.21.15.112:0/1035330142 shutdown_connections 2024-09-21T00:30:33.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:33.994+0000 7ff46fcbd640 1 -- 172.21.15.112:0/1035330142 wait complete. 2024-09-21T00:30:33.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:33.995+0000 7ff46fcbd640 1 Processor -- start 2024-09-21T00:30:33.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:33.995+0000 7ff46fcbd640 1 -- start start 2024-09-21T00:30:33.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:33.997+0000 7ff46fcbd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4680cb230 0x7ff4681660f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:33.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:33.997+0000 7ff46fcbd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff4681603c0 0x7ff46814b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:33.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:33.997+0000 7ff46fcbd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff468164530 0x7ff4681f7000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:33.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:33.997+0000 7ff46fcbd640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff468160190 con 0x7ff4680cb230 2024-09-21T00:30:33.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:33.997+0000 7ff46fcbd640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff468160010 con 0x7ff468164530 2024-09-21T00:30:34.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:33.997+0000 7ff46fcbd640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff468166e90 con 0x7ff4681603c0 2024-09-21T00:30:34.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:33.997+0000 7ff46da59640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4680cb230 0x7ff4681660f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:34.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:33.997+0000 7ff46da59640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4680cb230 0x7ff4681660f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56774/0 (socket says 172.21.15.112:56774) 2024-09-21T00:30:34.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:33.997+0000 7ff46da59640 1 -- 172.21.15.112:0/1367764095 learned_addr learned my addr 172.21.15.112:0/1367764095 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:30:34.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:33.997+0000 7ff46d258640 1 --2- 172.21.15.112:0/1367764095 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff4681603c0 0x7ff46814b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:34.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:33.997+0000 7ff46da59640 1 -- 172.21.15.112:0/1367764095 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff4681603c0 msgr2=0x7ff46814b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:34.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:33.997+0000 7ff46da59640 1 --2- 172.21.15.112:0/1367764095 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff4681603c0 0x7ff46814b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:34.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:33.997+0000 7ff46e25a640 1 --2- 172.21.15.112:0/1367764095 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff468164530 0x7ff4681f7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:34.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:33.997+0000 7ff46da59640 1 -- 172.21.15.112:0/1367764095 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff468164530 msgr2=0x7ff4681f7000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:34.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:33.997+0000 7ff46da59640 1 --2- 172.21.15.112:0/1367764095 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff468164530 0x7ff4681f7000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:34.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:33.997+0000 7ff46da59640 1 -- 172.21.15.112:0/1367764095 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff46814bf20 con 0x7ff4680cb230 2024-09-21T00:30:34.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:33.997+0000 7ff46da59640 1 --2- 172.21.15.112:0/1367764095 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4680cb230 0x7ff4681660f0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7ff46400c730 tx=0x7ff46400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:34.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:33.997+0000 7ff45e7fc640 1 -- 172.21.15.112:0/1367764095 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff464016020 con 0x7ff4680cb230 2024-09-21T00:30:34.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:33.997+0000 7ff45e7fc640 1 -- 172.21.15.112:0/1367764095 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff46400a950 con 0x7ff4680cb230 2024-09-21T00:30:34.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:33.997+0000 7ff45e7fc640 1 -- 172.21.15.112:0/1367764095 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff46400ada0 con 0x7ff4680cb230 2024-09-21T00:30:34.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:33.997+0000 7ff46fcbd640 1 -- 172.21.15.112:0/1367764095 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff468165090 con 0x7ff4680cb230 2024-09-21T00:30:34.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:33.997+0000 7ff46fcbd640 1 -- 172.21.15.112:0/1367764095 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff468165610 con 0x7ff4680cb230 2024-09-21T00:30:34.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:33.999+0000 7ff46fcbd640 1 -- 172.21.15.112:0/1367764095 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4681607d0 con 0x7ff4680cb230 2024-09-21T00:30:34.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:33.999+0000 7ff45e7fc640 1 -- 172.21.15.112:0/1367764095 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff464019020 con 0x7ff4680cb230 2024-09-21T00:30:34.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.000+0000 7ff45e7fc640 1 --2- 172.21.15.112:0/1367764095 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff44c078b90 0x7ff44c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:34.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.000+0000 7ff45e7fc640 1 -- 172.21.15.112:0/1367764095 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5507+0+0 (secure 0 0 0) 0x7ff46409aa70 con 0x7ff4680cb230 2024-09-21T00:30:34.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.000+0000 7ff46d258640 1 --2- 172.21.15.112:0/1367764095 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff44c078b90 0x7ff44c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:34.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.001+0000 7ff46d258640 1 --2- 172.21.15.112:0/1367764095 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff44c078b90 0x7ff44c07b0a0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7ff468164f80 tx=0x7ff458005c70 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:34.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.005+0000 7ff45e7fc640 1 -- 172.21.15.112:0/1367764095 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ff464020020 con 0x7ff4680cb230 2024-09-21T00:30:34.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.181+0000 7ff46fcbd640 1 -- 172.21.15.112:0/1367764095 --> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7ff46815c9d0 con 0x7ff44c078b90 2024-09-21T00:30:34.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.181+0000 7ff45e7fc640 1 -- 172.21.15.112:0/1367764095 <== mgr.4103 v2:172.21.15.187:6800/2921522906 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7ff46815c9d0 con 0x7ff44c078b90 2024-09-21T00:30:34.183 INFO:teuthology.orchestra.run.smithi112.stderr:instructed osd(s) 6 to deep-scrub 2024-09-21T00:30:34.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.184+0000 7ff46fcbd640 1 -- 172.21.15.112:0/1367764095 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff44c078b90 msgr2=0x7ff44c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:34.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.184+0000 7ff46fcbd640 1 --2- 172.21.15.112:0/1367764095 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff44c078b90 0x7ff44c07b0a0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7ff468164f80 tx=0x7ff458005c70 comp rx=0 tx=0).stop 2024-09-21T00:30:34.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.184+0000 7ff46fcbd640 1 -- 172.21.15.112:0/1367764095 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4680cb230 msgr2=0x7ff4681660f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:34.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.184+0000 7ff46fcbd640 1 --2- 172.21.15.112:0/1367764095 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4680cb230 0x7ff4681660f0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7ff46400c730 tx=0x7ff46400cbf0 comp rx=0 tx=0).stop 2024-09-21T00:30:34.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.184+0000 7ff46fcbd640 1 -- 172.21.15.112:0/1367764095 shutdown_connections 2024-09-21T00:30:34.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.184+0000 7ff46fcbd640 1 --2- 172.21.15.112:0/1367764095 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff468164530 0x7ff4681f7000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:34.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.184+0000 7ff46fcbd640 1 --2- 172.21.15.112:0/1367764095 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff4681603c0 0x7ff46814b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:34.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.184+0000 7ff46fcbd640 1 --2- 172.21.15.112:0/1367764095 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7ff44c078b90 0x7ff44c07b0a0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:34.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.184+0000 7ff46fcbd640 1 --2- 172.21.15.112:0/1367764095 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4680cb230 0x7ff4681660f0 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:34.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.184+0000 7ff46fcbd640 1 -- 172.21.15.112:0/1367764095 >> 172.21.15.112:0/1367764095 conn(0x7ff468073e60 msgr2=0x7ff46815e360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:34.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.184+0000 7ff46fcbd640 1 -- 172.21.15.112:0/1367764095 shutdown_connections 2024-09-21T00:30:34.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.184+0000 7ff46fcbd640 1 -- 172.21.15.112:0/1367764095 wait complete. 2024-09-21T00:30:34.197 INFO:tasks.ceph:Scrubbing osd.7 2024-09-21T00:30:34.197 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-09-21T00:30:34.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.312+0000 7f0c8f8bd640 1 -- 172.21.15.112:0/751428775 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c74005350 msgr2=0x7f0c7400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:34.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.312+0000 7f0c8f8bd640 1 --2- 172.21.15.112:0/751428775 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c74005350 0x7f0c7400b9c0 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f0c800016c0 tx=0x7f0c74005830 comp rx=0 tx=0).stop 2024-09-21T00:30:34.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.312+0000 7f0c8f8bd640 1 -- 172.21.15.112:0/751428775 shutdown_connections 2024-09-21T00:30:34.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.312+0000 7f0c8f8bd640 1 --2- 172.21.15.112:0/751428775 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c74005350 0x7f0c7400b9c0 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:34.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.312+0000 7f0c8f8bd640 1 -- 172.21.15.112:0/751428775 >> 172.21.15.112:0/751428775 conn(0x7f0c880b2a10 msgr2=0x7f0c880b2e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:34.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.312+0000 7f0c8f8bd640 1 -- 172.21.15.112:0/751428775 shutdown_connections 2024-09-21T00:30:34.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.312+0000 7f0c8f8bd640 1 -- 172.21.15.112:0/751428775 wait complete. 2024-09-21T00:30:34.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.314+0000 7f0c8f8bd640 1 Processor -- start 2024-09-21T00:30:34.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.314+0000 7f0c8f8bd640 1 -- start start 2024-09-21T00:30:34.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.315+0000 7f0c8f8bd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c74005350 0x7f0c881558e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:34.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.315+0000 7f0c8f8bd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0c880cad20 0x7f0c881f8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:34.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.315+0000 7f0c8f8bd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0c88156170 0x7f0c881f9030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:34.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.315+0000 7f0c8f8bd640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0c8815ca10 con 0x7f0c74005350 2024-09-21T00:30:34.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.315+0000 7f0c8f8bd640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0c8815c890 con 0x7f0c880cad20 2024-09-21T00:30:34.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.315+0000 7f0c8f8bd640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f0c880cc850 con 0x7f0c88156170 2024-09-21T00:30:34.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.315+0000 7f0c8d659640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c74005350 0x7f0c881558e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:34.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.315+0000 7f0c8d659640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c74005350 0x7f0c881558e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56786/0 (socket says 172.21.15.112:56786) 2024-09-21T00:30:34.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.315+0000 7f0c8d659640 1 -- 172.21.15.112:0/2280669765 learned_addr learned my addr 172.21.15.112:0/2280669765 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:30:34.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.315+0000 7f0c8de5a640 1 --2- 172.21.15.112:0/2280669765 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0c88156170 0x7f0c881f9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:34.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.315+0000 7f0c8ce58640 1 --2- 172.21.15.112:0/2280669765 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0c880cad20 0x7f0c881f8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:34.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.315+0000 7f0c8d659640 1 -- 172.21.15.112:0/2280669765 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0c88156170 msgr2=0x7f0c881f9030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:34.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.315+0000 7f0c8d659640 1 --2- 172.21.15.112:0/2280669765 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0c88156170 0x7f0c881f9030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:34.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.315+0000 7f0c8d659640 1 -- 172.21.15.112:0/2280669765 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0c880cad20 msgr2=0x7f0c881f8af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:34.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.315+0000 7f0c8d659640 1 --2- 172.21.15.112:0/2280669765 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0c880cad20 0x7f0c881f8af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:34.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.315+0000 7f0c8d659640 1 -- 172.21.15.112:0/2280669765 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c881f97c0 con 0x7f0c74005350 2024-09-21T00:30:34.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.315+0000 7f0c8d659640 1 --2- 172.21.15.112:0/2280669765 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c74005350 0x7f0c881558e0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f0c84011c70 tx=0x7f0c8400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:34.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.316+0000 7f0c7dffb640 1 -- 172.21.15.112:0/2280669765 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c84003e40 con 0x7f0c74005350 2024-09-21T00:30:34.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.316+0000 7f0c7dffb640 1 -- 172.21.15.112:0/2280669765 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0c84004b20 con 0x7f0c74005350 2024-09-21T00:30:34.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.316+0000 7f0c8f8bd640 1 -- 172.21.15.112:0/2280669765 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c880cbc00 con 0x7f0c74005350 2024-09-21T00:30:34.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.316+0000 7f0c7dffb640 1 -- 172.21.15.112:0/2280669765 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c84004050 con 0x7f0c74005350 2024-09-21T00:30:34.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.316+0000 7f0c8f8bd640 1 -- 172.21.15.112:0/2280669765 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c880af010 con 0x7f0c74005350 2024-09-21T00:30:34.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.317+0000 7f0c8f8bd640 1 -- 172.21.15.112:0/2280669765 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0c880cbe40 con 0x7f0c74005350 2024-09-21T00:30:34.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.317+0000 7f0c7dffb640 1 -- 172.21.15.112:0/2280669765 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0c840041f0 con 0x7f0c74005350 2024-09-21T00:30:34.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.318+0000 7f0c7dffb640 1 --2- 172.21.15.112:0/2280669765 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f0c6c078bc0 0x7f0c6c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:34.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.318+0000 7f0c7dffb640 1 -- 172.21.15.112:0/2280669765 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5507+0+0 (secure 0 0 0) 0x7f0c8409c5d0 con 0x7f0c74005350 2024-09-21T00:30:34.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.318+0000 7f0c7dffb640 1 --2- 172.21.15.112:0/2280669765 >> [v2:172.21.15.187:6818/1624699842,v1:172.21.15.187:6819/1624699842] conn(0x7f0c6c07f1f0 0x7f0c6c081680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:34.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.318+0000 7f0c7dffb640 1 -- 172.21.15.112:0/2280669765 --> [v2:172.21.15.187:6818/1624699842,v1:172.21.15.187:6819/1624699842] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0c8400be50 con 0x7f0c6c07f1f0 2024-09-21T00:30:34.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.318+0000 7f0c7dffb640 1 -- 172.21.15.112:0/2280669765 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_get_version_reply(handle=1 version=115) ==== 24+0+0 (secure 0 0 0) 0x7f0c8409ca60 con 0x7f0c74005350 2024-09-21T00:30:34.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.318+0000 7f0c8ce58640 1 --2- 172.21.15.112:0/2280669765 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f0c6c078bc0 0x7f0c6c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:34.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.318+0000 7f0c8de5a640 1 --2- 172.21.15.112:0/2280669765 >> [v2:172.21.15.187:6818/1624699842,v1:172.21.15.187:6819/1624699842] conn(0x7f0c6c07f1f0 0x7f0c6c081680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:34.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.319+0000 7f0c8ce58640 1 --2- 172.21.15.112:0/2280669765 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f0c6c078bc0 0x7f0c6c07b0d0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f0c780018a0 tx=0x7f0c780011b0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:34.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.319+0000 7f0c8de5a640 1 --2- 172.21.15.112:0/2280669765 >> [v2:172.21.15.187:6818/1624699842,v1:172.21.15.187:6819/1624699842] conn(0x7f0c6c07f1f0 0x7f0c6c081680 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:34.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.320+0000 7f0c7dffb640 1 -- 172.21.15.112:0/2280669765 <== osd.7 v2:172.21.15.187:6818/1624699842 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0c8400be50 con 0x7f0c6c07f1f0 2024-09-21T00:30:34.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.332+0000 7f0c8f8bd640 1 -- 172.21.15.112:0/2280669765 --> [v2:172.21.15.187:6818/1624699842,v1:172.21.15.187:6819/1624699842] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f0c880cb9f0 con 0x7f0c6c07f1f0 2024-09-21T00:30:34.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.333+0000 7f0c7dffb640 1 -- 172.21.15.112:0/2280669765 <== osd.7 v2:172.21.15.187:6818/1624699842 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f0c880cb9f0 con 0x7f0c6c07f1f0 2024-09-21T00:30:34.334 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-21T00:30:34.334 INFO:teuthology.orchestra.run.smithi112.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T00:30:34.335 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-21T00:30:34.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.333+0000 7f0c8f8bd640 1 -- 172.21.15.112:0/2280669765 >> [v2:172.21.15.187:6818/1624699842,v1:172.21.15.187:6819/1624699842] conn(0x7f0c6c07f1f0 msgr2=0x7f0c6c081680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:34.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.333+0000 7f0c8f8bd640 1 --2- 172.21.15.112:0/2280669765 >> [v2:172.21.15.187:6818/1624699842,v1:172.21.15.187:6819/1624699842] conn(0x7f0c6c07f1f0 0x7f0c6c081680 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:34.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.333+0000 7f0c8f8bd640 1 -- 172.21.15.112:0/2280669765 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f0c6c078bc0 msgr2=0x7f0c6c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:34.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.333+0000 7f0c8f8bd640 1 --2- 172.21.15.112:0/2280669765 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f0c6c078bc0 0x7f0c6c07b0d0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f0c780018a0 tx=0x7f0c780011b0 comp rx=0 tx=0).stop 2024-09-21T00:30:34.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.333+0000 7f0c8f8bd640 1 -- 172.21.15.112:0/2280669765 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c74005350 msgr2=0x7f0c881558e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:34.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.333+0000 7f0c8f8bd640 1 --2- 172.21.15.112:0/2280669765 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c74005350 0x7f0c881558e0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f0c84011c70 tx=0x7f0c8400f790 comp rx=0 tx=0).stop 2024-09-21T00:30:34.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.333+0000 7f0c8de5a640 1 -- 172.21.15.112:0/2280669765 reap_dead start 2024-09-21T00:30:34.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.333+0000 7f0c8f8bd640 1 -- 172.21.15.112:0/2280669765 shutdown_connections 2024-09-21T00:30:34.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.333+0000 7f0c8f8bd640 1 -- 172.21.15.112:0/2280669765 >> 172.21.15.112:0/2280669765 conn(0x7f0c880b2a10 msgr2=0x7f0c880b4260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:34.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.333+0000 7f0c8f8bd640 1 -- 172.21.15.112:0/2280669765 shutdown_connections 2024-09-21T00:30:34.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.333+0000 7f0c8f8bd640 1 -- 172.21.15.112:0/2280669765 wait complete. 2024-09-21T00:30:34.345 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-09-21T00:30:34.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.457+0000 7f4fcdac5640 1 -- 172.21.15.112:0/2815796478 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4fc80b4530 msgr2=0x7f4fc80d1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:34.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.457+0000 7f4fcdac5640 1 --2- 172.21.15.112:0/2815796478 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4fc80b4530 0x7f4fc80d1af0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc009830 tx=0x7f4fbc02a510 comp rx=0 tx=0).stop 2024-09-21T00:30:34.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.457+0000 7f4fcdac5640 1 -- 172.21.15.112:0/2815796478 shutdown_connections 2024-09-21T00:30:34.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.457+0000 7f4fcdac5640 1 --2- 172.21.15.112:0/2815796478 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4fc80b4530 0x7f4fc80d1af0 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:34.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.457+0000 7f4fcdac5640 1 --2- 172.21.15.112:0/2815796478 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4fc80b1cc0 0x7f4fc80b3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:34.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.457+0000 7f4fcdac5640 1 -- 172.21.15.112:0/2815796478 >> 172.21.15.112:0/2815796478 conn(0x7f4fc80cf020 msgr2=0x7f4fc80cf430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:34.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.457+0000 7f4fcdac5640 1 -- 172.21.15.112:0/2815796478 shutdown_connections 2024-09-21T00:30:34.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.457+0000 7f4fcdac5640 1 -- 172.21.15.112:0/2815796478 wait complete. 2024-09-21T00:30:34.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.459+0000 7f4fcdac5640 1 Processor -- start 2024-09-21T00:30:34.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.459+0000 7f4fcdac5640 1 -- start start 2024-09-21T00:30:34.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.460+0000 7f4fcdac5640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4fc80b4530 0x7f4fc80b7c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:34.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.460+0000 7f4fcdac5640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4fc81602e0 0x7f4fc81f2910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:34.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.460+0000 7f4fcdac5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4fc80c7f60 0x7f4fc80c8370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:34.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.460+0000 7f4fcdac5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4fc80d0f50 con 0x7f4fc80c7f60 2024-09-21T00:30:34.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.460+0000 7f4fcdac5640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f4fc80d0dd0 con 0x7f4fc80b4530 2024-09-21T00:30:34.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.460+0000 7f4fcdac5640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f4fc80d10d0 con 0x7f4fc81602e0 2024-09-21T00:30:34.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.460+0000 7f4fc7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4fc80c7f60 0x7f4fc80c8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:34.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.460+0000 7f4fc7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4fc80c7f60 0x7f4fc80c8370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56812/0 (socket says 172.21.15.112:56812) 2024-09-21T00:30:34.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.460+0000 7f4fc7fff640 1 -- 172.21.15.112:0/894116159 learned_addr learned my addr 172.21.15.112:0/894116159 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:30:34.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.460+0000 7f4fc6ffd640 1 --2- 172.21.15.112:0/894116159 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4fc81602e0 0x7f4fc81f2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:34.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.461+0000 7f4fc77fe640 1 --2- 172.21.15.112:0/894116159 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4fc80b4530 0x7f4fc80b7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:34.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.461+0000 7f4fc7fff640 1 -- 172.21.15.112:0/894116159 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4fc81602e0 msgr2=0x7f4fc81f2910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:34.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.461+0000 7f4fc7fff640 1 --2- 172.21.15.112:0/894116159 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4fc81602e0 0x7f4fc81f2910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:34.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.461+0000 7f4fc7fff640 1 -- 172.21.15.112:0/894116159 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4fc80b4530 msgr2=0x7f4fc80b7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:34.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.461+0000 7f4fc7fff640 1 --2- 172.21.15.112:0/894116159 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4fc80b4530 0x7f4fc80b7c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:34.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.461+0000 7f4fc7fff640 1 -- 172.21.15.112:0/894116159 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fc81f2e50 con 0x7f4fc80c7f60 2024-09-21T00:30:34.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.461+0000 7f4fc77fe640 1 --2- 172.21.15.112:0/894116159 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4fc80b4530 0x7f4fc80b7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:30:34.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.461+0000 7f4fc7fff640 1 --2- 172.21.15.112:0/894116159 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4fc80c7f60 0x7f4fc80c8370 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f4fc000c730 tx=0x7f4fc000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:34.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.461+0000 7f4faffff640 1 -- 172.21.15.112:0/894116159 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fc0003b70 con 0x7f4fc80c7f60 2024-09-21T00:30:34.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.461+0000 7f4faffff640 1 -- 172.21.15.112:0/894116159 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4fc0003d30 con 0x7f4fc80c7f60 2024-09-21T00:30:34.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.461+0000 7f4faffff640 1 -- 172.21.15.112:0/894116159 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fc00052d0 con 0x7f4fc80c7f60 2024-09-21T00:30:34.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.461+0000 7f4fcdac5640 1 -- 172.21.15.112:0/894116159 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fc80c88b0 con 0x7f4fc80c7f60 2024-09-21T00:30:34.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.461+0000 7f4fcdac5640 1 -- 172.21.15.112:0/894116159 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fc80c7160 con 0x7f4fc80c7f60 2024-09-21T00:30:34.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.462+0000 7f4fcdac5640 1 -- 172.21.15.112:0/894116159 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4fc80d1580 con 0x7f4fc80c7f60 2024-09-21T00:30:34.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.463+0000 7f4faffff640 1 -- 172.21.15.112:0/894116159 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f4fc0005680 con 0x7f4fc80c7f60 2024-09-21T00:30:34.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.464+0000 7f4faffff640 1 --2- 172.21.15.112:0/894116159 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4fb4078b90 0x7f4fb407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:34.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.464+0000 7f4faffff640 1 -- 172.21.15.112:0/894116159 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5507+0+0 (secure 0 0 0) 0x7f4fc009b530 con 0x7f4fc80c7f60 2024-09-21T00:30:34.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.464+0000 7f4fc77fe640 1 --2- 172.21.15.112:0/894116159 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4fb4078b90 0x7f4fb407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:34.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.464+0000 7f4fc77fe640 1 --2- 172.21.15.112:0/894116159 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4fb4078b90 0x7f4fb407b0a0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f4fb8005860 tx=0x7f4fb8005e10 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:34.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.467+0000 7f4faffff640 1 -- 172.21.15.112:0/894116159 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f4fc80c88b0 con 0x7f4fc80c7f60 2024-09-21T00:30:34.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.645+0000 7f4fcdac5640 1 -- 172.21.15.112:0/894116159 --> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7f4fc80c6c10 con 0x7f4fb4078b90 2024-09-21T00:30:34.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.646+0000 7f4faffff640 1 -- 172.21.15.112:0/894116159 <== mgr.4103 v2:172.21.15.187:6800/2921522906 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f4fc80c6c10 con 0x7f4fb4078b90 2024-09-21T00:30:34.647 INFO:teuthology.orchestra.run.smithi112.stderr:instructed osd(s) 7 to deep-scrub 2024-09-21T00:30:34.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.648+0000 7f4fcdac5640 1 -- 172.21.15.112:0/894116159 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4fb4078b90 msgr2=0x7f4fb407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:34.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.648+0000 7f4fcdac5640 1 --2- 172.21.15.112:0/894116159 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4fb4078b90 0x7f4fb407b0a0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f4fb8005860 tx=0x7f4fb8005e10 comp rx=0 tx=0).stop 2024-09-21T00:30:34.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.648+0000 7f4fcdac5640 1 -- 172.21.15.112:0/894116159 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4fc80c7f60 msgr2=0x7f4fc80c8370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:34.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.648+0000 7f4fcdac5640 1 --2- 172.21.15.112:0/894116159 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4fc80c7f60 0x7f4fc80c8370 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f4fc000c730 tx=0x7f4fc000cbf0 comp rx=0 tx=0).stop 2024-09-21T00:30:34.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.648+0000 7f4fcdac5640 1 -- 172.21.15.112:0/894116159 shutdown_connections 2024-09-21T00:30:34.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.648+0000 7f4fcdac5640 1 --2- 172.21.15.112:0/894116159 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f4fb4078b90 0x7f4fb407b0a0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:34.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.648+0000 7f4fcdac5640 1 --2- 172.21.15.112:0/894116159 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4fc80c7f60 0x7f4fc80c8370 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:34.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.648+0000 7f4fcdac5640 1 --2- 172.21.15.112:0/894116159 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4fc81602e0 0x7f4fc81f2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:34.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.648+0000 7f4fcdac5640 1 --2- 172.21.15.112:0/894116159 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4fc80b4530 0x7f4fc80b7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:34.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.648+0000 7f4fcdac5640 1 -- 172.21.15.112:0/894116159 >> 172.21.15.112:0/894116159 conn(0x7f4fc80cf020 msgr2=0x7f4fc815f490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:34.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.648+0000 7f4fcdac5640 1 -- 172.21.15.112:0/894116159 shutdown_connections 2024-09-21T00:30:34.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.648+0000 7f4fcdac5640 1 -- 172.21.15.112:0/894116159 wait complete. 2024-09-21T00:30:34.661 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T00:30:34.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.770+0000 7f8fc06b4640 1 -- 172.21.15.112:0/2666431715 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8fb806db10 msgr2=0x7f8fb80ca770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:34.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.770+0000 7f8fc06b4640 1 --2- 172.21.15.112:0/2666431715 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8fb806db10 0x7f8fb80ca770 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f8fa800a940 tx=0x7f8fa802b850 comp rx=0 tx=0).stop 2024-09-21T00:30:34.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.770+0000 7f8fc06b4640 1 -- 172.21.15.112:0/2666431715 shutdown_connections 2024-09-21T00:30:34.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.770+0000 7f8fc06b4640 1 --2- 172.21.15.112:0/2666431715 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8fb806db10 0x7f8fb80ca770 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:34.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.770+0000 7f8fc06b4640 1 --2- 172.21.15.112:0/2666431715 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8fb8158950 0x7f8fb806d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:34.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.770+0000 7f8fc06b4640 1 -- 172.21.15.112:0/2666431715 >> 172.21.15.112:0/2666431715 conn(0x7f8fb80b1c10 msgr2=0x7f8fb80b2020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:34.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.770+0000 7f8fc06b4640 1 -- 172.21.15.112:0/2666431715 shutdown_connections 2024-09-21T00:30:34.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.770+0000 7f8fc06b4640 1 -- 172.21.15.112:0/2666431715 wait complete. 2024-09-21T00:30:34.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.772+0000 7f8fc06b4640 1 Processor -- start 2024-09-21T00:30:34.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.772+0000 7f8fc06b4640 1 -- start start 2024-09-21T00:30:34.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.773+0000 7f8fc06b4640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8fb806db10 0x7f8fb81e4b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:34.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.773+0000 7f8fc06b4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8fb80cacb0 0x7f8fb8152ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:34.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.773+0000 7f8fc06b4640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8fb8158950 0x7f8fb8148520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:34.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.773+0000 7f8fc06b4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb806cd40 con 0x7f8fb80cacb0 2024-09-21T00:30:34.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.773+0000 7f8fc06b4640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb8074770 con 0x7f8fb806db10 2024-09-21T00:30:34.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.773+0000 7f8fc06b4640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f8fb806cec0 con 0x7f8fb8158950 2024-09-21T00:30:34.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.773+0000 7f8fbdc4f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8fb80cacb0 0x7f8fb8152ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:34.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.773+0000 7f8fbdc4f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8fb80cacb0 0x7f8fb8152ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56826/0 (socket says 172.21.15.112:56826) 2024-09-21T00:30:34.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.773+0000 7f8fbdc4f640 1 -- 172.21.15.112:0/4188539786 learned_addr learned my addr 172.21.15.112:0/4188539786 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:30:34.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.773+0000 7f8fbec51640 1 --2- 172.21.15.112:0/4188539786 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8fb8158950 0x7f8fb8148520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:34.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.773+0000 7f8fbe450640 1 --2- 172.21.15.112:0/4188539786 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8fb806db10 0x7f8fb81e4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:34.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.773+0000 7f8fbdc4f640 1 -- 172.21.15.112:0/4188539786 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8fb8158950 msgr2=0x7f8fb8148520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:34.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.773+0000 7f8fbdc4f640 1 --2- 172.21.15.112:0/4188539786 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8fb8158950 0x7f8fb8148520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:34.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.773+0000 7f8fbdc4f640 1 -- 172.21.15.112:0/4188539786 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8fb806db10 msgr2=0x7f8fb81e4b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:34.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.773+0000 7f8fbdc4f640 1 --2- 172.21.15.112:0/4188539786 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8fb806db10 0x7f8fb81e4b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:34.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.773+0000 7f8fbdc4f640 1 -- 172.21.15.112:0/4188539786 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fb81e35b0 con 0x7f8fb80cacb0 2024-09-21T00:30:34.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.773+0000 7f8fbec51640 1 --2- 172.21.15.112:0/4188539786 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8fb8158950 0x7f8fb8148520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:30:34.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.773+0000 7f8fbe450640 1 --2- 172.21.15.112:0/4188539786 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8fb806db10 0x7f8fb81e4b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:30:34.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.774+0000 7f8fbdc4f640 1 --2- 172.21.15.112:0/4188539786 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8fb80cacb0 0x7f8fb8152ab0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f8fa800aa30 tx=0x7f8fa8003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:34.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.774+0000 7f8faeffd640 1 -- 172.21.15.112:0/4188539786 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fa8041070 con 0x7f8fb80cacb0 2024-09-21T00:30:34.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.774+0000 7f8faeffd640 1 -- 172.21.15.112:0/4188539786 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8fa8005310 con 0x7f8fb80cacb0 2024-09-21T00:30:34.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.774+0000 7f8fc06b4640 1 -- 172.21.15.112:0/4188539786 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fb81e3ab0 con 0x7f8fb80cacb0 2024-09-21T00:30:34.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.774+0000 7f8faeffd640 1 -- 172.21.15.112:0/4188539786 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fa8003e30 con 0x7f8fb80cacb0 2024-09-21T00:30:34.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.774+0000 7f8fc06b4640 1 -- 172.21.15.112:0/4188539786 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fb81e4070 con 0x7f8fb80cacb0 2024-09-21T00:30:34.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.775+0000 7f8fc06b4640 1 -- 172.21.15.112:0/4188539786 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fb81e3820 con 0x7f8fb80cacb0 2024-09-21T00:30:34.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.776+0000 7f8faeffd640 1 -- 172.21.15.112:0/4188539786 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8fa803b040 con 0x7f8fb80cacb0 2024-09-21T00:30:34.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.776+0000 7f8faeffd640 1 --2- 172.21.15.112:0/4188539786 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f8f9c078b90 0x7f8f9c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:34.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.776+0000 7f8faeffd640 1 -- 172.21.15.112:0/4188539786 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5507+0+0 (secure 0 0 0) 0x7f8fa80bbb40 con 0x7f8fb80cacb0 2024-09-21T00:30:34.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.776+0000 7f8fbe450640 1 --2- 172.21.15.112:0/4188539786 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f8f9c078b90 0x7f8f9c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:34.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.777+0000 7f8fbe450640 1 --2- 172.21.15.112:0/4188539786 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f8f9c078b90 0x7f8f9c07b0a0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f8fb4005fd0 tx=0x7f8fb400f360 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:34.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.781+0000 7f8faeffd640 1 -- 172.21.15.112:0/4188539786 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f8fa8084160 con 0x7f8fb80cacb0 2024-09-21T00:30:34.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.954+0000 7f8fc06b4640 1 -- 172.21.15.112:0/4188539786 --> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8fb806e570 con 0x7f8f9c078b90 2024-09-21T00:30:34.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.955+0000 7f8faeffd640 1 -- 172.21.15.112:0/4188539786 <== mgr.4103 v2:172.21.15.187:6800/2921522906 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13382 (secure 0 0 0) 0x7f8fb806e570 con 0x7f8f9c078b90 2024-09-21T00:30:34.957 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:30:34.957 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2024-09-21T00:30:34.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.957+0000 7f8fc06b4640 1 -- 172.21.15.112:0/4188539786 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f8f9c078b90 msgr2=0x7f8f9c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:34.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.957+0000 7f8fc06b4640 1 --2- 172.21.15.112:0/4188539786 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f8f9c078b90 0x7f8f9c07b0a0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f8fb4005fd0 tx=0x7f8fb400f360 comp rx=0 tx=0).stop 2024-09-21T00:30:34.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.957+0000 7f8fc06b4640 1 -- 172.21.15.112:0/4188539786 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8fb80cacb0 msgr2=0x7f8fb8152ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:34.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.957+0000 7f8fc06b4640 1 --2- 172.21.15.112:0/4188539786 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8fb80cacb0 0x7f8fb8152ab0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f8fa800aa30 tx=0x7f8fa8003c10 comp rx=0 tx=0).stop 2024-09-21T00:30:34.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.957+0000 7f8fc06b4640 1 -- 172.21.15.112:0/4188539786 shutdown_connections 2024-09-21T00:30:34.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.957+0000 7f8fc06b4640 1 --2- 172.21.15.112:0/4188539786 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8fb8158950 0x7f8fb8148520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:34.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.957+0000 7f8fc06b4640 1 --2- 172.21.15.112:0/4188539786 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f8f9c078b90 0x7f8f9c07b0a0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:34.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.957+0000 7f8fc06b4640 1 --2- 172.21.15.112:0/4188539786 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8fb80cacb0 0x7f8fb8152ab0 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:34.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.957+0000 7f8fc06b4640 1 --2- 172.21.15.112:0/4188539786 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8fb806db10 0x7f8fb81e4b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:34.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.957+0000 7f8fc06b4640 1 -- 172.21.15.112:0/4188539786 >> 172.21.15.112:0/4188539786 conn(0x7f8fb80b1c10 msgr2=0x7f8fb8155710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:34.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.957+0000 7f8fc06b4640 1 -- 172.21.15.112:0/4188539786 shutdown_connections 2024-09-21T00:30:34.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:34.957+0000 7f8fc06b4640 1 -- 172.21.15.112:0/4188539786 wait complete. 2024-09-21T00:30:34.969 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":258,"stamp":"2024-09-21T00:30:33.163011+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":22,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":261892,"kb_used_data":12360,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754712828,"statfs":{"total":773094113280,"available":772825935872,"internally_reserved":0,"allocated":12656640,"data_stored":6434976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.014521"},"pg_stats":[{"pgid":"1.0","version":"17'72","reported_seq":283,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-21T00:30:26.177047+0000","last_change":"2024-09-21T00:25:38.725792+0000","last_active":"2024-09-21T00:30:26.177047+0000","last_peered":"2024-09-21T00:30:26.177047+0000","last_clean":"2024-09-21T00:30:26.177047+0000","last_became_active":"2024-09-21T00:25:38.724668+0000","last_became_peered":"2024-09-21T00:25:38.724668+0000","last_unstale":"2024-09-21T00:30:26.177047+0000","last_undegraded":"2024-09-21T00:30:26.177047+0000","last_fullsized":"2024-09-21T00:30:26.177047+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T00:25:37.163216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T00:25:37.163216+0000","last_clean_scrub_stamp":"2024-09-21T00:25:37.163216+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:14:36.588660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509502,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33176,"kb_used_data":1980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338664,"statfs":{"total":96636764160,"available":96602791936,"internally_reserved":0,"allocated":2027520,"data_stored":1247148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509503,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32596,"kb_used_data":1400,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339244,"statfs":{"total":96636764160,"available":96603385856,"internally_reserved":0,"allocated":1433600,"data_stored":656780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509502,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32588,"kb_used_data":1400,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339252,"statfs":{"total":96636764160,"available":96603394048,"internally_reserved":0,"allocated":1433600,"data_stored":656780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509501,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32588,"kb_used_data":1400,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339252,"statfs":{"total":96636764160,"available":96603394048,"internally_reserved":0,"allocated":1433600,"data_stored":656780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509503,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32588,"kb_used_data":1400,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339252,"statfs":{"total":96636764160,"available":96603394048,"internally_reserved":0,"allocated":1433600,"data_stored":656780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509503,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32592,"kb_used_data":1400,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339248,"statfs":{"total":96636764160,"available":96603389952,"internally_reserved":0,"allocated":1433600,"data_stored":656780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509503,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33172,"kb_used_data":1980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338668,"statfs":{"total":96636764160,"available":96602796032,"internally_reserved":0,"allocated":2027520,"data_stored":1247148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574910,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32592,"kb_used_data":1400,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339248,"statfs":{"total":96636764160,"available":96603389952,"internally_reserved":0,"allocated":1433600,"data_stored":656780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T00:30:34.972 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-21T00:25:37.163216+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=0, tm_min=25, tm_sec=37, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=0, tm_min=30, tm_sec=29, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T00:30:34.972 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-21T00:30:45.707 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T00:30:45.711 DEBUG:teuthology.orchestra.run.smithi187:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T00:30:54.973 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T00:30:55.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.099+0000 7f7c7ae15640 1 -- 172.21.15.112:0/3108200111 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c5c004800 msgr2=0x7f7c5c004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:55.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.099+0000 7f7c7ae15640 1 --2- 172.21.15.112:0/3108200111 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c5c004800 0x7f7c5c004c10 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f7c7406e750 tx=0x7f7c5c005830 comp rx=0 tx=0).stop 2024-09-21T00:30:55.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.099+0000 7f7c7ae15640 1 -- 172.21.15.112:0/3108200111 shutdown_connections 2024-09-21T00:30:55.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.099+0000 7f7c7ae15640 1 --2- 172.21.15.112:0/3108200111 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c5c004800 0x7f7c5c004c10 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:55.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.099+0000 7f7c7ae15640 1 -- 172.21.15.112:0/3108200111 >> 172.21.15.112:0/3108200111 conn(0x7f7c740b1c10 msgr2=0x7f7c740b2020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:55.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.099+0000 7f7c7ae15640 1 -- 172.21.15.112:0/3108200111 shutdown_connections 2024-09-21T00:30:55.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.099+0000 7f7c7ae15640 1 -- 172.21.15.112:0/3108200111 wait complete. 2024-09-21T00:30:55.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.101+0000 7f7c7ae15640 1 Processor -- start 2024-09-21T00:30:55.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.101+0000 7f7c7ae15640 1 -- start start 2024-09-21T00:30:55.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.102+0000 7f7c7ae15640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7c5c004800 0x7f7c740b0890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:55.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.102+0000 7f7c7ae15640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7c740b0dd0 0x7f7c74156d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:55.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.102+0000 7f7c7ae15640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c740c71d0 0x7f7c741572b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:55.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.102+0000 7f7c7ae15640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7c7415c000 con 0x7f7c740c71d0 2024-09-21T00:30:55.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.102+0000 7f7c7ae15640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7c7415be80 con 0x7f7c740b0dd0 2024-09-21T00:30:55.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.102+0000 7f7c7ae15640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f7c7415c180 con 0x7f7c5c004800 2024-09-21T00:30:55.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.102+0000 7f7c793b2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c740c71d0 0x7f7c741572b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:55.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.103+0000 7f7c793b2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c740c71d0 0x7f7c741572b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39454/0 (socket says 172.21.15.112:39454) 2024-09-21T00:30:55.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.103+0000 7f7c793b2640 1 -- 172.21.15.112:0/2735787792 learned_addr learned my addr 172.21.15.112:0/2735787792 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:30:55.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.103+0000 7f7c78bb1640 1 --2- 172.21.15.112:0/2735787792 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7c5c004800 0x7f7c740b0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:55.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.103+0000 7f7c6bfff640 1 --2- 172.21.15.112:0/2735787792 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7c740b0dd0 0x7f7c74156d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:55.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.103+0000 7f7c793b2640 1 -- 172.21.15.112:0/2735787792 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7c5c004800 msgr2=0x7f7c740b0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:55.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.103+0000 7f7c793b2640 1 --2- 172.21.15.112:0/2735787792 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7c5c004800 0x7f7c740b0890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:55.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.103+0000 7f7c793b2640 1 -- 172.21.15.112:0/2735787792 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7c740b0dd0 msgr2=0x7f7c74156d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:55.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.103+0000 7f7c793b2640 1 --2- 172.21.15.112:0/2735787792 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7c740b0dd0 0x7f7c74156d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:55.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.103+0000 7f7c793b2640 1 -- 172.21.15.112:0/2735787792 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c740c81b0 con 0x7f7c740c71d0 2024-09-21T00:30:55.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.103+0000 7f7c793b2640 1 --2- 172.21.15.112:0/2735787792 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c740c71d0 0x7f7c741572b0 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f7c740765a0 tx=0x7f7c6c0019b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:55.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.103+0000 7f7c697fa640 1 -- 172.21.15.112:0/2735787792 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c6c011870 con 0x7f7c740c71d0 2024-09-21T00:30:55.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.103+0000 7f7c697fa640 1 -- 172.21.15.112:0/2735787792 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7c6c017070 con 0x7f7c740c71d0 2024-09-21T00:30:55.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.103+0000 7f7c697fa640 1 -- 172.21.15.112:0/2735787792 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c6c011da0 con 0x7f7c740c71d0 2024-09-21T00:30:55.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.103+0000 7f7c7ae15640 1 -- 172.21.15.112:0/2735787792 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c740ca470 con 0x7f7c740c71d0 2024-09-21T00:30:55.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.103+0000 7f7c7ae15640 1 -- 172.21.15.112:0/2735787792 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c740af470 con 0x7f7c740c71d0 2024-09-21T00:30:55.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.104+0000 7f7c7ae15640 1 -- 172.21.15.112:0/2735787792 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c7415f260 con 0x7f7c740c71d0 2024-09-21T00:30:55.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.105+0000 7f7c697fa640 1 -- 172.21.15.112:0/2735787792 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f7c6c01c460 con 0x7f7c740c71d0 2024-09-21T00:30:55.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.106+0000 7f7c697fa640 1 --2- 172.21.15.112:0/2735787792 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f7c60078b90 0x7f7c6007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:55.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.106+0000 7f7c697fa640 1 -- 172.21.15.112:0/2735787792 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5507+0+0 (secure 0 0 0) 0x7f7c6c09b410 con 0x7f7c740c71d0 2024-09-21T00:30:55.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.106+0000 7f7c78bb1640 1 --2- 172.21.15.112:0/2735787792 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f7c60078b90 0x7f7c6007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:55.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.107+0000 7f7c78bb1640 1 --2- 172.21.15.112:0/2735787792 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f7c60078b90 0x7f7c6007b0a0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f7c70009c10 tx=0x7f7c70005f90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:55.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.110+0000 7f7c697fa640 1 -- 172.21.15.112:0/2735787792 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f7c6c0639a0 con 0x7f7c740c71d0 2024-09-21T00:30:55.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.291+0000 7f7c7ae15640 1 -- 172.21.15.112:0/2735787792 --> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7c74070e70 con 0x7f7c60078b90 2024-09-21T00:30:55.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.292+0000 7f7c697fa640 1 -- 172.21.15.112:0/2735787792 <== mgr.4103 v2:172.21.15.187:6800/2921522906 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13364 (secure 0 0 0) 0x7f7c70024450 con 0x7f7c60078b90 2024-09-21T00:30:55.293 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:30:55.293 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2024-09-21T00:30:55.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.294+0000 7f7c7ae15640 1 -- 172.21.15.112:0/2735787792 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f7c60078b90 msgr2=0x7f7c6007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:55.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.294+0000 7f7c7ae15640 1 --2- 172.21.15.112:0/2735787792 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f7c60078b90 0x7f7c6007b0a0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f7c70009c10 tx=0x7f7c70005f90 comp rx=0 tx=0).stop 2024-09-21T00:30:55.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.294+0000 7f7c7ae15640 1 -- 172.21.15.112:0/2735787792 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c740c71d0 msgr2=0x7f7c741572b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:55.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.294+0000 7f7c7ae15640 1 --2- 172.21.15.112:0/2735787792 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c740c71d0 0x7f7c741572b0 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f7c740765a0 tx=0x7f7c6c0019b0 comp rx=0 tx=0).stop 2024-09-21T00:30:55.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.294+0000 7f7c7ae15640 1 -- 172.21.15.112:0/2735787792 shutdown_connections 2024-09-21T00:30:55.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.294+0000 7f7c7ae15640 1 --2- 172.21.15.112:0/2735787792 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f7c60078b90 0x7f7c6007b0a0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:55.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.294+0000 7f7c7ae15640 1 --2- 172.21.15.112:0/2735787792 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c740c71d0 0x7f7c741572b0 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:55.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.294+0000 7f7c7ae15640 1 --2- 172.21.15.112:0/2735787792 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7c740b0dd0 0x7f7c74156d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:55.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.294+0000 7f7c7ae15640 1 --2- 172.21.15.112:0/2735787792 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7c5c004800 0x7f7c740b0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:55.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.294+0000 7f7c7ae15640 1 -- 172.21.15.112:0/2735787792 >> 172.21.15.112:0/2735787792 conn(0x7f7c740b1c10 msgr2=0x7f7c7415d850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:55.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.294+0000 7f7c7ae15640 1 -- 172.21.15.112:0/2735787792 shutdown_connections 2024-09-21T00:30:55.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.294+0000 7f7c7ae15640 1 -- 172.21.15.112:0/2735787792 wait complete. 2024-09-21T00:30:55.306 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":269,"stamp":"2024-09-21T00:30:55.168697+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":261872,"kb_used_data":12360,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754712848,"statfs":{"total":773094113280,"available":772825956352,"internally_reserved":0,"allocated":12656640,"data_stored":6434976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003089"},"pg_stats":[{"pgid":"1.0","version":"115'74","reported_seq":293,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-21T00:30:35.553507+0000","last_change":"2024-09-21T00:30:35.553507+0000","last_active":"2024-09-21T00:30:35.553507+0000","last_peered":"2024-09-21T00:30:35.553507+0000","last_clean":"2024-09-21T00:30:35.553507+0000","last_became_active":"2024-09-21T00:25:38.724668+0000","last_became_peered":"2024-09-21T00:25:38.724668+0000","last_unstale":"2024-09-21T00:30:35.553507+0000","last_undegraded":"2024-09-21T00:30:35.553507+0000","last_fullsized":"2024-09-21T00:30:35.553507+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"115'74","last_scrub_stamp":"2024-09-21T00:30:35.553402+0000","last_deep_scrub":"115'74","last_deep_scrub_stamp":"2024-09-21T00:30:35.553402+0000","last_clean_scrub_stamp":"2024-09-21T00:30:35.553402+0000","objects_scrubbed":2,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:36:20.955329+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509506,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33164,"kb_used_data":1980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338676,"statfs":{"total":96636764160,"available":96602804224,"internally_reserved":0,"allocated":2027520,"data_stored":1247148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509507,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32596,"kb_used_data":1400,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339244,"statfs":{"total":96636764160,"available":96603385856,"internally_reserved":0,"allocated":1433600,"data_stored":656780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509506,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32588,"kb_used_data":1400,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339252,"statfs":{"total":96636764160,"available":96603394048,"internally_reserved":0,"allocated":1433600,"data_stored":656780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509505,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32588,"kb_used_data":1400,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339252,"statfs":{"total":96636764160,"available":96603394048,"internally_reserved":0,"allocated":1433600,"data_stored":656780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509507,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32588,"kb_used_data":1400,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339252,"statfs":{"total":96636764160,"available":96603394048,"internally_reserved":0,"allocated":1433600,"data_stored":656780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509507,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32592,"kb_used_data":1400,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339248,"statfs":{"total":96636764160,"available":96603389952,"internally_reserved":0,"allocated":1433600,"data_stored":656780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509507,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33164,"kb_used_data":1980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338676,"statfs":{"total":96636764160,"available":96602804224,"internally_reserved":0,"allocated":2027520,"data_stored":1247148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574915,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32592,"kb_used_data":1400,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339248,"statfs":{"total":96636764160,"available":96603389952,"internally_reserved":0,"allocated":1433600,"data_stored":656780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T00:30:55.307 DEBUG:teuthology.orchestra.run.smithi112:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-21T00:30:55.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.422+0000 7f61f3ad3640 1 -- 172.21.15.112:0/3375800439 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f61ec0d4ac0 msgr2=0x7f61ec0d1370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:55.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.422+0000 7f61f3ad3640 1 --2- 172.21.15.112:0/3375800439 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f61ec0d4ac0 0x7f61ec0d1370 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f61e4005fd0 tx=0x7f61e402aa90 comp rx=0 tx=0).stop 2024-09-21T00:30:55.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.422+0000 7f61f3ad3640 1 -- 172.21.15.112:0/3375800439 shutdown_connections 2024-09-21T00:30:55.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.422+0000 7f61f3ad3640 1 --2- 172.21.15.112:0/3375800439 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f61ec0d4ac0 0x7f61ec0d1370 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:55.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.422+0000 7f61f3ad3640 1 --2- 172.21.15.112:0/3375800439 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f61ec15f980 0x7f61ec0d4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:55.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.422+0000 7f61f3ad3640 1 -- 172.21.15.112:0/3375800439 >> 172.21.15.112:0/3375800439 conn(0x7f61ec08bfe0 msgr2=0x7f61ec09d8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:55.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.422+0000 7f61f3ad3640 1 -- 172.21.15.112:0/3375800439 shutdown_connections 2024-09-21T00:30:55.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.422+0000 7f61f3ad3640 1 -- 172.21.15.112:0/3375800439 wait complete. 2024-09-21T00:30:55.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.424+0000 7f61f3ad3640 1 Processor -- start 2024-09-21T00:30:55.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.424+0000 7f61f3ad3640 1 -- start start 2024-09-21T00:30:55.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.425+0000 7f61f3ad3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f61ec0b6a20 0x7f61ec06e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:55.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.425+0000 7f61f3ad3640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f61ec0d4ac0 0x7f61ec0c8090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:55.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.425+0000 7f61f3ad3640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f61ec15f980 0x7f61ec0c85d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:55.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.425+0000 7f61f3ad3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f61ec0d1c80 con 0x7f61ec0b6a20 2024-09-21T00:30:55.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.425+0000 7f61f3ad3640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f61ec0d1b00 con 0x7f61ec15f980 2024-09-21T00:30:55.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.425+0000 7f61f3ad3640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f61ec0d1e00 con 0x7f61ec0d4ac0 2024-09-21T00:30:55.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.425+0000 7f61f186f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f61ec0b6a20 0x7f61ec06e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:55.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.425+0000 7f61f186f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f61ec0b6a20 0x7f61ec06e060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39460/0 (socket says 172.21.15.112:39460) 2024-09-21T00:30:55.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.425+0000 7f61f186f640 1 -- 172.21.15.112:0/139739064 learned_addr learned my addr 172.21.15.112:0/139739064 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:30:55.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.425+0000 7f61f106e640 1 --2- 172.21.15.112:0/139739064 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f61ec0d4ac0 0x7f61ec0c8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:55.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.425+0000 7f61f2070640 1 --2- 172.21.15.112:0/139739064 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f61ec15f980 0x7f61ec0c85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:55.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.426+0000 7f61f186f640 1 -- 172.21.15.112:0/139739064 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f61ec0d4ac0 msgr2=0x7f61ec0c8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:55.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.426+0000 7f61f186f640 1 --2- 172.21.15.112:0/139739064 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f61ec0d4ac0 0x7f61ec0c8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:55.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.426+0000 7f61f186f640 1 -- 172.21.15.112:0/139739064 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f61ec15f980 msgr2=0x7f61ec0c85d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:55.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.426+0000 7f61f186f640 1 --2- 172.21.15.112:0/139739064 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f61ec15f980 0x7f61ec0c85d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:55.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.426+0000 7f61f186f640 1 -- 172.21.15.112:0/139739064 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61ec0c8b10 con 0x7f61ec0b6a20 2024-09-21T00:30:55.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.426+0000 7f61f186f640 1 --2- 172.21.15.112:0/139739064 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f61ec0b6a20 0x7f61ec06e060 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f61e800c3d0 tx=0x7f61e800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:55.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.426+0000 7f61e27fc640 1 -- 172.21.15.112:0/139739064 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61e8017070 con 0x7f61ec0b6a20 2024-09-21T00:30:55.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.426+0000 7f61e27fc640 1 -- 172.21.15.112:0/139739064 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f61e80053c0 con 0x7f61ec0b6a20 2024-09-21T00:30:55.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.426+0000 7f61f3ad3640 1 -- 172.21.15.112:0/139739064 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61ec06e990 con 0x7f61ec0b6a20 2024-09-21T00:30:55.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.426+0000 7f61e27fc640 1 -- 172.21.15.112:0/139739064 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61e8003850 con 0x7f61ec0b6a20 2024-09-21T00:30:55.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.426+0000 7f61f3ad3640 1 -- 172.21.15.112:0/139739064 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61ec06ef50 con 0x7f61ec0b6a20 2024-09-21T00:30:55.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.427+0000 7f61f3ad3640 1 -- 172.21.15.112:0/139739064 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61ec0706a0 con 0x7f61ec0b6a20 2024-09-21T00:30:55.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.428+0000 7f61e27fc640 1 -- 172.21.15.112:0/139739064 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f61e801a070 con 0x7f61ec0b6a20 2024-09-21T00:30:55.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.428+0000 7f61e27fc640 1 --2- 172.21.15.112:0/139739064 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f61d0078bc0 0x7f61d007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:55.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.428+0000 7f61e27fc640 1 -- 172.21.15.112:0/139739064 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5507+0+0 (secure 0 0 0) 0x7f61e8099f70 con 0x7f61ec0b6a20 2024-09-21T00:30:55.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.429+0000 7f61f106e640 1 --2- 172.21.15.112:0/139739064 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f61d0078bc0 0x7f61d007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:55.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.429+0000 7f61f106e640 1 --2- 172.21.15.112:0/139739064 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f61d0078bc0 0x7f61d007b0d0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f61ec159980 tx=0x7f61dc00b2e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:55.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.433+0000 7f61e27fc640 1 -- 172.21.15.112:0/139739064 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f61e8061e20 con 0x7f61ec0b6a20 2024-09-21T00:30:55.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.617+0000 7f61f3ad3640 1 -- 172.21.15.112:0/139739064 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7f61ec0b1f80 con 0x7f61ec0b6a20 2024-09-21T00:30:55.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.623+0000 7f61e27fc640 1 -- 172.21.15.112:0/139739064 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v49)=0 v49) ==== 123+0+0 (secure 0 0 0) 0x7f61e8066850 con 0x7f61ec0b6a20 2024-09-21T00:30:55.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.625+0000 7f61e27fc640 1 -- 172.21.15.112:0/139739064 <== mon.0 v2:172.21.15.112:3300/0 8 ==== config(6 keys) ==== 172+0+0 (secure 0 0 0) 0x7f61e8061930 con 0x7f61ec0b6a20 2024-09-21T00:30:55.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.626+0000 7f61f3ad3640 1 -- 172.21.15.112:0/139739064 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f61d0078bc0 msgr2=0x7f61d007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:55.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.626+0000 7f61f3ad3640 1 --2- 172.21.15.112:0/139739064 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f61d0078bc0 0x7f61d007b0d0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f61ec159980 tx=0x7f61dc00b2e0 comp rx=0 tx=0).stop 2024-09-21T00:30:55.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.626+0000 7f61f3ad3640 1 -- 172.21.15.112:0/139739064 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f61ec0b6a20 msgr2=0x7f61ec06e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:55.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.626+0000 7f61f3ad3640 1 --2- 172.21.15.112:0/139739064 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f61ec0b6a20 0x7f61ec06e060 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f61e800c3d0 tx=0x7f61e800c890 comp rx=0 tx=0).stop 2024-09-21T00:30:55.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.626+0000 7f61f3ad3640 1 -- 172.21.15.112:0/139739064 shutdown_connections 2024-09-21T00:30:55.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.626+0000 7f61f3ad3640 1 --2- 172.21.15.112:0/139739064 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f61ec15f980 0x7f61ec0c85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:55.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.626+0000 7f61f3ad3640 1 --2- 172.21.15.112:0/139739064 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f61ec0d4ac0 0x7f61ec0c8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:55.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.626+0000 7f61f3ad3640 1 --2- 172.21.15.112:0/139739064 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f61d0078bc0 0x7f61d007b0d0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:55.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.626+0000 7f61f3ad3640 1 --2- 172.21.15.112:0/139739064 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f61ec0b6a20 0x7f61ec06e060 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:55.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.626+0000 7f61f3ad3640 1 -- 172.21.15.112:0/139739064 >> 172.21.15.112:0/139739064 conn(0x7f61ec08bfe0 msgr2=0x7f61ec0b5340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:55.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.626+0000 7f61f3ad3640 1 -- 172.21.15.112:0/139739064 shutdown_connections 2024-09-21T00:30:55.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.626+0000 7f61f3ad3640 1 -- 172.21.15.112:0/139739064 wait complete. 2024-09-21T00:30:55.638 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-21T00:30:55.638 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T00:30:55.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.754+0000 7f330f932640 1 -- 172.21.15.112:0/710872839 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32f4005270 msgr2=0x7f32f400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:55.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.754+0000 7f330f932640 1 --2- 172.21.15.112:0/710872839 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32f4005270 0x7f32f400b9c0 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f3300009ed0 tx=0x7f32f4005830 comp rx=0 tx=0).stop 2024-09-21T00:30:55.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.754+0000 7f330f932640 1 -- 172.21.15.112:0/710872839 shutdown_connections 2024-09-21T00:30:55.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.754+0000 7f330f932640 1 --2- 172.21.15.112:0/710872839 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32f4005270 0x7f32f400b9c0 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:55.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.754+0000 7f330f932640 1 -- 172.21.15.112:0/710872839 >> 172.21.15.112:0/710872839 conn(0x7f33080735e0 msgr2=0x7f33080739f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:55.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.754+0000 7f330f932640 1 -- 172.21.15.112:0/710872839 shutdown_connections 2024-09-21T00:30:55.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.754+0000 7f330f932640 1 -- 172.21.15.112:0/710872839 wait complete. 2024-09-21T00:30:55.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.755+0000 7f330f932640 1 Processor -- start 2024-09-21T00:30:55.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.756+0000 7f330f932640 1 -- start start 2024-09-21T00:30:55.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.757+0000 7f330f932640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f32f4005270 0x7f330814c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:55.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.757+0000 7f330f932640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f330814bd70 0x7f33081f1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:55.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.757+0000 7f330f932640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f33081f1ff0 0x7f33081564e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:55.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.757+0000 7f330f932640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3308071af0 con 0x7f33081f1ff0 2024-09-21T00:30:55.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.757+0000 7f330f932640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3308071970 con 0x7f330814bd70 2024-09-21T00:30:55.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.757+0000 7f330f932640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3308071c70 con 0x7f32f4005270 2024-09-21T00:30:55.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.757+0000 7f330decf640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f33081f1ff0 0x7f33081564e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:55.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.757+0000 7f330decf640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f33081f1ff0 0x7f33081564e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39482/0 (socket says 172.21.15.112:39482) 2024-09-21T00:30:55.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.757+0000 7f330decf640 1 -- 172.21.15.112:0/1566425682 learned_addr learned my addr 172.21.15.112:0/1566425682 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T00:30:55.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.757+0000 7f330d6ce640 1 --2- 172.21.15.112:0/1566425682 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f32f4005270 0x7f330814c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:55.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.757+0000 7f330cecd640 1 --2- 172.21.15.112:0/1566425682 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f330814bd70 0x7f33081f1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:55.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.757+0000 7f330decf640 1 -- 172.21.15.112:0/1566425682 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f32f4005270 msgr2=0x7f330814c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:55.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.757+0000 7f330decf640 1 --2- 172.21.15.112:0/1566425682 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f32f4005270 0x7f330814c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:55.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.757+0000 7f330decf640 1 -- 172.21.15.112:0/1566425682 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f330814bd70 msgr2=0x7f33081f1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:55.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.757+0000 7f330decf640 1 --2- 172.21.15.112:0/1566425682 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f330814bd70 0x7f33081f1ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:55.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.757+0000 7f330decf640 1 -- 172.21.15.112:0/1566425682 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33081585f0 con 0x7f33081f1ff0 2024-09-21T00:30:55.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.757+0000 7f330cecd640 1 --2- 172.21.15.112:0/1566425682 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f330814bd70 0x7f33081f1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:30:55.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.757+0000 7f330d6ce640 1 --2- 172.21.15.112:0/1566425682 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f32f4005270 0x7f330814c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T00:30:55.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.757+0000 7f330decf640 1 --2- 172.21.15.112:0/1566425682 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f33081f1ff0 0x7f33081564e0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f3300009ed0 tx=0x7f33000042d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:55.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.758+0000 7f32f9ffb640 1 -- 172.21.15.112:0/1566425682 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33000046d0 con 0x7f33081f1ff0 2024-09-21T00:30:55.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.758+0000 7f32f9ffb640 1 -- 172.21.15.112:0/1566425682 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(6 keys) ==== 172+0+0 (secure 0 0 0) 0x7f330003c480 con 0x7f33081f1ff0 2024-09-21T00:30:55.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.758+0000 7f330f932640 1 -- 172.21.15.112:0/1566425682 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3308155a00 con 0x7f33081f1ff0 2024-09-21T00:30:55.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.758+0000 7f32f9ffb640 1 -- 172.21.15.112:0/1566425682 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f330003c890 con 0x7f33081f1ff0 2024-09-21T00:30:55.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.758+0000 7f330f932640 1 -- 172.21.15.112:0/1566425682 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3308155fc0 con 0x7f33081f1ff0 2024-09-21T00:30:55.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.759+0000 7f330f932640 1 -- 172.21.15.112:0/1566425682 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33080b2400 con 0x7f33081f1ff0 2024-09-21T00:30:55.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.760+0000 7f32f9ffb640 1 -- 172.21.15.112:0/1566425682 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3300044020 con 0x7f33081f1ff0 2024-09-21T00:30:55.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.760+0000 7f32f9ffb640 1 --2- 172.21.15.112:0/1566425682 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f32ec078b90 0x7f32ec07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T00:30:55.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.760+0000 7f32f9ffb640 1 -- 172.21.15.112:0/1566425682 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5507+0+0 (secure 0 0 0) 0x7f33000bc440 con 0x7f33081f1ff0 2024-09-21T00:30:55.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.761+0000 7f330d6ce640 1 --2- 172.21.15.112:0/1566425682 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f32ec078b90 0x7f32ec07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T00:30:55.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.761+0000 7f330d6ce640 1 --2- 172.21.15.112:0/1566425682 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f32ec078b90 0x7f32ec07b0a0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f33040035f0 tx=0x7f3304006b70 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T00:30:55.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:55.764+0000 7f32f9ffb640 1 -- 172.21.15.112:0/1566425682 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f33080b2400 con 0x7f33081f1ff0 2024-09-21T00:30:56.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:56.001+0000 7f330f932640 1 -- 172.21.15.112:0/1566425682 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3308158380 con 0x7f33081f1ff0 2024-09-21T00:30:56.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:56.002+0000 7f32f9ffb640 1 -- 172.21.15.112:0/1566425682 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) ==== 94+0+3920 (secure 0 0 0) 0x7f330004b020 con 0x7f33081f1ff0 2024-09-21T00:30:56.003 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T00:30:56.003 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":82,"btime":"2024-09-21T00:30:24:138756+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6113,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/3217596137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":3217596137},{"type":"v1","addr":"172.21.15.187:6835","nonce":3217596137}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":73},{"gid":6649,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6837/776068904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":776068904},{"type":"v1","addr":"172.21.15.187:6837","nonce":776068904}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":73},{"gid":7623,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/3567261033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":3567261033},{"type":"v1","addr":"172.21.15.112:6833","nonce":3567261033}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":82},{"gid":7632,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/3807766291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3807766291},{"type":"v1","addr":"172.21.15.112:6835","nonce":3807766291}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":82}],"filesystems":[]} 2024-09-21T00:30:56.003 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 82 2024-09-21T00:30:56.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:56.004+0000 7f330f932640 1 -- 172.21.15.112:0/1566425682 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f32ec078b90 msgr2=0x7f32ec07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:56.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:56.004+0000 7f330f932640 1 --2- 172.21.15.112:0/1566425682 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f32ec078b90 0x7f32ec07b0a0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f33040035f0 tx=0x7f3304006b70 comp rx=0 tx=0).stop 2024-09-21T00:30:56.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:56.004+0000 7f330f932640 1 -- 172.21.15.112:0/1566425682 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f33081f1ff0 msgr2=0x7f33081564e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T00:30:56.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:56.004+0000 7f330f932640 1 --2- 172.21.15.112:0/1566425682 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f33081f1ff0 0x7f33081564e0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f3300009ed0 tx=0x7f33000042d0 comp rx=0 tx=0).stop 2024-09-21T00:30:56.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:56.004+0000 7f330f932640 1 -- 172.21.15.112:0/1566425682 shutdown_connections 2024-09-21T00:30:56.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:56.004+0000 7f330f932640 1 --2- 172.21.15.112:0/1566425682 >> [v2:172.21.15.187:6800/2921522906,v1:172.21.15.187:6801/2921522906] conn(0x7f32ec078b90 0x7f32ec07b0a0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:56.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:56.004+0000 7f330f932640 1 --2- 172.21.15.112:0/1566425682 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f33081f1ff0 0x7f33081564e0 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:56.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:56.004+0000 7f330f932640 1 --2- 172.21.15.112:0/1566425682 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f330814bd70 0x7f33081f1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:56.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:56.004+0000 7f330f932640 1 --2- 172.21.15.112:0/1566425682 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f32f4005270 0x7f330814c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T00:30:56.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:56.004+0000 7f330f932640 1 -- 172.21.15.112:0/1566425682 >> 172.21.15.112:0/1566425682 conn(0x7f33080735e0 msgr2=0x7f330815ea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T00:30:56.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:56.004+0000 7f330f932640 1 -- 172.21.15.112:0/1566425682 shutdown_connections 2024-09-21T00:30:56.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T00:30:56.004+0000 7f330f932640 1 -- 172.21.15.112:0/1566425682 wait complete. 2024-09-21T00:30:56.016 DEBUG:tasks.cephfs.filesystem:already dead... 2024-09-21T00:30:56.016 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-21T00:30:56.016 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-21T00:30:56.017 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T00:30:56.028 INFO:tasks.ceph.mds.a:Stopped 2024-09-21T00:30:56.028 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-21T00:30:56.029 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T00:30:56.080 INFO:tasks.ceph.mds.c:Stopped 2024-09-21T00:30:56.080 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-21T00:30:56.080 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T00:30:56.093 INFO:tasks.ceph.mds.b:Stopped 2024-09-21T00:30:56.093 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-21T00:30:56.093 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T00:30:56.144 INFO:tasks.ceph.mds.d:Stopped 2024-09-21T00:30:56.144 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-21T00:30:56.144 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-21T00:30:56.145 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T00:30:56.178 INFO:tasks.ceph.osd.0:Stopped 2024-09-21T00:30:56.178 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-21T00:30:56.178 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T00:30:56.257 INFO:tasks.ceph.osd.1:Stopped 2024-09-21T00:30:56.258 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-21T00:30:56.258 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T00:30:56.325 INFO:tasks.ceph.osd.2:Stopped 2024-09-21T00:30:56.326 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-21T00:30:56.326 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T00:30:56.397 INFO:tasks.ceph.osd.3:Stopped 2024-09-21T00:30:56.398 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-21T00:30:56.398 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T00:30:56.438 INFO:tasks.ceph.osd.4:Stopped 2024-09-21T00:30:56.438 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-21T00:30:56.438 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T00:30:56.515 INFO:tasks.ceph.osd.5:Stopped 2024-09-21T00:30:56.516 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-09-21T00:30:56.516 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T00:30:56.584 INFO:tasks.ceph.osd.6:Stopped 2024-09-21T00:30:56.584 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-09-21T00:30:56.584 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T00:30:56.654 INFO:tasks.ceph.osd.7:Stopped 2024-09-21T00:30:56.655 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-21T00:30:56.655 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-09-21T00:30:56.655 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T00:30:56.682 INFO:tasks.ceph.mgr.y:Stopped 2024-09-21T00:30:56.682 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-21T00:30:56.683 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T00:30:56.725 INFO:tasks.ceph.mgr.x:Stopped 2024-09-21T00:30:56.725 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-21T00:30:56.725 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-21T00:30:56.726 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T00:30:56.737 INFO:tasks.ceph.mon.a:Stopped 2024-09-21T00:30:56.738 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-21T00:30:56.738 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T00:30:56.779 INFO:tasks.ceph.mon.b:Stopped 2024-09-21T00:30:56.779 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-21T00:30:56.779 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T00:30:56.831 INFO:tasks.ceph.mon.c:Stopped 2024-09-21T00:30:56.831 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-21T00:30:56.832 DEBUG:teuthology.orchestra.run.smithi112:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v 'fs.*is degraded' | egrep -v 'filesystem is degraded' | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v 'filesystem is offline' | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v 'insufficient standby MDS daemons available' | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'client session with non-allowable root' | head -n 1 2024-09-21T00:30:56.875 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi112.front.sepia.ceph.com 2024-09-21T00:30:56.875 DEBUG:teuthology.orchestra.run.smithi112:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-21T00:30:57.107 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi112.front.sepia.ceph.com 2024-09-21T00:30:57.107 DEBUG:teuthology.orchestra.run.smithi112:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-21T00:30:57.227 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi112.front.sepia.ceph.com 2024-09-21T00:30:57.227 DEBUG:teuthology.orchestra.run.smithi112:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-21T00:30:57.284 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi112.front.sepia.ceph.com 2024-09-21T00:30:57.284 DEBUG:teuthology.orchestra.run.smithi112:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-21T00:30:57.343 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi187.front.sepia.ceph.com 2024-09-21T00:30:57.343 DEBUG:teuthology.orchestra.run.smithi187:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-09-21T00:30:57.537 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi187.front.sepia.ceph.com 2024-09-21T00:30:57.538 DEBUG:teuthology.orchestra.run.smithi187:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-09-21T00:30:57.625 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi187.front.sepia.ceph.com 2024-09-21T00:30:57.625 DEBUG:teuthology.orchestra.run.smithi187:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-09-21T00:30:57.683 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi187.front.sepia.ceph.com 2024-09-21T00:30:57.683 DEBUG:teuthology.orchestra.run.smithi187:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-09-21T00:30:57.740 INFO:tasks.ceph:Archiving mon data... 2024-09-21T00:30:57.741 DEBUG:teuthology.misc:Transferring archived files from smithi112:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913327/data/mon.a.tgz 2024-09-21T00:30:57.741 DEBUG:teuthology.orchestra.run.smithi112:> mktemp 2024-09-21T00:30:57.757 INFO:teuthology.orchestra.run.smithi112.stdout:/tmp/tmp.x6gOHx3vqh 2024-09-21T00:30:57.757 DEBUG:teuthology.orchestra.run.smithi112:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.x6gOHx3vqh 2024-09-21T00:30:57.937 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod 0666 /tmp/tmp.x6gOHx3vqh 2024-09-21T00:30:58.020 DEBUG:teuthology.orchestra.remote:smithi112:/tmp/tmp.x6gOHx3vqh is 551KB 2024-09-21T00:30:58.119 DEBUG:teuthology.orchestra.run.smithi112:> rm -fr /tmp/tmp.x6gOHx3vqh 2024-09-21T00:30:58.134 DEBUG:teuthology.misc:Transferring archived files from smithi187:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913327/data/mon.b.tgz 2024-09-21T00:30:58.134 DEBUG:teuthology.orchestra.run.smithi187:> mktemp 2024-09-21T00:30:58.151 INFO:teuthology.orchestra.run.smithi187.stdout:/tmp/tmp.UMvzg2NLAB 2024-09-21T00:30:58.151 DEBUG:teuthology.orchestra.run.smithi187:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.UMvzg2NLAB 2024-09-21T00:30:58.407 DEBUG:teuthology.orchestra.run.smithi187:> sudo chmod 0666 /tmp/tmp.UMvzg2NLAB 2024-09-21T00:30:58.490 DEBUG:teuthology.orchestra.remote:smithi187:/tmp/tmp.UMvzg2NLAB is 701KB 2024-09-21T00:30:58.589 DEBUG:teuthology.orchestra.run.smithi187:> rm -fr /tmp/tmp.UMvzg2NLAB 2024-09-21T00:30:58.604 DEBUG:teuthology.misc:Transferring archived files from smithi187:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913327/data/mon.c.tgz 2024-09-21T00:30:58.604 DEBUG:teuthology.orchestra.run.smithi187:> mktemp 2024-09-21T00:30:58.659 INFO:teuthology.orchestra.run.smithi187.stdout:/tmp/tmp.4Ga4KGITKH 2024-09-21T00:30:58.659 DEBUG:teuthology.orchestra.run.smithi187:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.4Ga4KGITKH 2024-09-21T00:30:58.864 DEBUG:teuthology.orchestra.run.smithi187:> sudo chmod 0666 /tmp/tmp.4Ga4KGITKH 2024-09-21T00:30:58.946 DEBUG:teuthology.orchestra.remote:smithi187:/tmp/tmp.4Ga4KGITKH is 701KB 2024-09-21T00:30:59.045 DEBUG:teuthology.orchestra.run.smithi187:> rm -fr /tmp/tmp.4Ga4KGITKH 2024-09-21T00:30:59.060 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-21T00:30:59.060 DEBUG:teuthology.orchestra.run.smithi112:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-21T00:30:59.064 DEBUG:teuthology.orchestra.run.smithi187:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-21T00:30:59.145 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-21T00:30:59.179 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-21T00:30:59.179 INFO:tasks.ceph:Archiving crash dumps... 2024-09-21T00:30:59.181 DEBUG:teuthology.misc:Transferring archived files from smithi112:/var/lib/ceph/crash to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913327/remote/smithi112/crash 2024-09-21T00:30:59.182 DEBUG:teuthology.orchestra.run.smithi112:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-21T00:30:59.212 DEBUG:teuthology.misc:Transferring archived files from smithi187:/var/lib/ceph/crash to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913327/remote/smithi187/crash 2024-09-21T00:30:59.213 DEBUG:teuthology.orchestra.run.smithi187:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-21T00:30:59.243 INFO:tasks.ceph:Shutting down logrotate 2024-09-21T00:30:59.243 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T00:30:59.257 DEBUG:teuthology.orchestra.run.smithi187:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T00:30:59.514 DEBUG:teuthology.orchestra.run.smithi112:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-09-21T00:30:59.542 DEBUG:teuthology.orchestra.run.smithi187:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-09-21T00:30:59.567 INFO:tasks.ceph:Compressing logs... 2024-09-21T00:30:59.567 DEBUG:teuthology.orchestra.run.smithi112:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-21T00:30:59.585 DEBUG:teuthology.orchestra.run.smithi187:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-21T00:30:59.609 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-09-21T00:30:59.609 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64112.log 2024-09-21T00:30:59.611 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53503.log 2024-09-21T00:30:59.611 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-osd.0.log: /var/log/ceph/ceph-client.admin.64112.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48282.log 2024-09-21T00:30:59.611 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.53503.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.64112.log.gz 2024-09-21T00:30:59.611 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45568.log 2024-09-21T00:30:59.611 INFO:teuthology.orchestra.run.smithi112.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.53503.log.gz 2024-09-21T00:30:59.629 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.48282.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47452.log 2024-09-21T00:30:59.629 INFO:teuthology.orchestra.run.smithi112.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.48282.log.gz 2024-09-21T00:30:59.629 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.45568.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54268.log 2024-09-21T00:30:59.629 INFO:teuthology.orchestra.run.smithi112.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.45568.log.gz 2024-09-21T00:30:59.629 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.47452.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63258.log 2024-09-21T00:30:59.629 INFO:teuthology.orchestra.run.smithi112.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47452.log.gz 2024-09-21T00:30:59.629 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.54268.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48198.log 2024-09-21T00:30:59.629 INFO:teuthology.orchestra.run.smithi112.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.54268.log.gz 2024-09-21T00:30:59.629 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.63258.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49613.log 2024-09-21T00:30:59.629 INFO:teuthology.orchestra.run.smithi112.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.63258.log.gz 2024-09-21T00:30:59.629 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.48198.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53545.log 2024-09-21T00:30:59.630 INFO:teuthology.orchestra.run.smithi112.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48198.log.gz 2024-09-21T00:30:59.630 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.49613.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50226.log 2024-09-21T00:30:59.630 INFO:teuthology.orchestra.run.smithi112.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49613.log.gz 2024-09-21T00:30:59.630 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.53545.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60530.log 2024-09-21T00:30:59.630 INFO:teuthology.orchestra.run.smithi112.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.53545.log.gz 2024-09-21T00:30:59.630 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.50226.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64850.log 2024-09-21T00:30:59.630 INFO:teuthology.orchestra.run.smithi112.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50226.log.gz 2024-09-21T00:30:59.630 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.60530.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47537.log 2024-09-21T00:30:59.630 INFO:teuthology.orchestra.run.smithi112.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.60530.log.gz 2024-09-21T00:30:59.630 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.64850.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.49951.log 2024-09-21T00:30:59.630 INFO:teuthology.orchestra.run.smithi112.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.64850.log.gz 2024-09-21T00:30:59.630 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.47537.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57586.log 2024-09-21T00:30:59.630 INFO:teuthology.orchestra.run.smithi112.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47537.log.gz 2024-09-21T00:30:59.630 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.0.49951.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67014.log 0.0% -- replaced with /var/log/ceph/ceph-client.0.49951.log.gz 2024-09-21T00:30:59.630 INFO:teuthology.orchestra.run.smithi112.stderr: 2024-09-21T00:30:59.631 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.57586.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62549.log 2024-09-21T00:30:59.631 INFO:teuthology.orchestra.run.smithi112.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.57586.log.gz 2024-09-21T00:30:59.631 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.67014.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37463.log 2024-09-21T00:30:59.631 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.62549.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51053.log 2024-09-21T00:30:59.631 INFO:teuthology.orchestra.run.smithi112.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.67014.log.gz 2024-09-21T00:30:59.631 INFO:teuthology.orchestra.run.smithi112.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.62549.log.gz 2024-09-21T00:30:59.631 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.37463.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48695.log 2024-09-21T00:30:59.631 INFO:teuthology.orchestra.run.smithi112.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.37463.log.gz 2024-09-21T00:30:59.631 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.51053.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54659.log 2024-09-21T00:30:59.631 INFO:teuthology.orchestra.run.smithi112.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51053.log.gz 2024-09-21T00:30:59.631 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.48695.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36735.log 2024-09-21T00:30:59.631 INFO:teuthology.orchestra.run.smithi112.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48695.log.gz 2024-09-21T00:30:59.631 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.54659.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57334.log 2024-09-21T00:30:59.631 INFO:teuthology.orchestra.run.smithi112.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.54659.log.gz 2024-09-21T00:30:59.631 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.36735.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58490.log 2024-09-21T00:30:59.631 INFO:teuthology.orchestra.run.smithi112.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.36735.log.gz 2024-09-21T00:30:59.632 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.57334.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46325.log 2024-09-21T00:30:59.632 INFO:teuthology.orchestra.run.smithi112.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.57334.log.gz/var/log/ceph/ceph-client.admin.58490.log: 2024-09-21T00:30:59.632 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47988.log 2024-09-21T00:30:59.632 INFO:teuthology.orchestra.run.smithi112.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.58490.log.gz 2024-09-21T00:30:59.632 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.46325.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45664.log 2024-09-21T00:30:59.632 INFO:teuthology.orchestra.run.smithi112.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46325.log.gz 2024-09-21T00:30:59.632 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.47988.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43424.log 2024-09-21T00:30:59.632 INFO:teuthology.orchestra.run.smithi112.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47988.log.gz 2024-09-21T00:30:59.632 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.45664.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61934.log 2024-09-21T00:30:59.632 INFO:teuthology.orchestra.run.smithi112.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45664.log.gz 2024-09-21T00:30:59.632 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.43424.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56365.log 2024-09-21T00:30:59.632 INFO:teuthology.orchestra.run.smithi112.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.43424.log.gz 2024-09-21T00:30:59.632 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.61934.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66288.log 2024-09-21T00:30:59.632 INFO:teuthology.orchestra.run.smithi112.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.61934.log.gz 2024-09-21T00:30:59.633 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.56365.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67952.log 2024-09-21T00:30:59.633 INFO:teuthology.orchestra.run.smithi112.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.56365.log.gz 2024-09-21T00:30:59.633 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.66288.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44027.log 2024-09-21T00:30:59.633 INFO:teuthology.orchestra.run.smithi112.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.66288.log.gz 2024-09-21T00:30:59.633 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.67952.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60192.log 2024-09-21T00:30:59.633 INFO:teuthology.orchestra.run.smithi112.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.67952.log.gz 2024-09-21T00:30:59.633 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.44027.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.46650.log 2024-09-21T00:30:59.633 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.60192.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66006.log 2024-09-21T00:30:59.633 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.0.46650.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.60192.log.gz 2024-09-21T00:30:59.633 INFO:teuthology.orchestra.run.smithi112.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.46650.log.gz 2024-09-21T00:30:59.633 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60764.log 2024-09-21T00:30:59.633 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56860.log 2024-09-21T00:30:59.633 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.66006.log: /var/log/ceph/ceph-client.admin.60764.log: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.44027.log.gz 2024-09-21T00:30:59.633 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55605.log 2024-09-21T00:30:59.633 INFO:teuthology.orchestra.run.smithi112.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.66006.log.gz 2024-09-21T00:30:59.634 INFO:teuthology.orchestra.run.smithi112.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.60764.log.gz 2024-09-21T00:30:59.634 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.56860.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43298.log 2024-09-21T00:30:59.634 INFO:teuthology.orchestra.run.smithi112.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.56860.log.gz 2024-09-21T00:30:59.634 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.55605.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53013.log 2024-09-21T00:30:59.634 INFO:teuthology.orchestra.run.smithi112.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.55605.log.gz 2024-09-21T00:30:59.634 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.43298.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62684.log 2024-09-21T00:30:59.634 INFO:teuthology.orchestra.run.smithi112.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.43298.log.gz 2024-09-21T00:30:59.634 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.53013.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44991.log 2024-09-21T00:30:59.634 INFO:teuthology.orchestra.run.smithi112.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53013.log.gz 2024-09-21T00:30:59.634 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.62684.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-09-21T00:30:59.634 INFO:teuthology.orchestra.run.smithi112.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.62684.log.gz 2024-09-21T00:30:59.634 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.44991.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.30141.log 2024-09-21T00:30:59.634 INFO:teuthology.orchestra.run.smithi112.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.44991.log.gz 2024-09-21T00:30:59.634 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67868.log 2024-09-21T00:30:59.634 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph.tmp-client.admin.30141.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64070.log 2024-09-21T00:30:59.634 INFO:teuthology.orchestra.run.smithi112.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.30141.log.gz 2024-09-21T00:30:59.635 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.67868.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57723.log 2024-09-21T00:30:59.635 INFO:teuthology.orchestra.run.smithi112.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.67868.log.gz 2024-09-21T00:30:59.635 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.64070.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60320.log 2024-09-21T00:30:59.635 INFO:teuthology.orchestra.run.smithi112.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.64070.log.gz 2024-09-21T00:30:59.635 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.57723.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63679.log 2024-09-21T00:30:59.635 INFO:teuthology.orchestra.run.smithi112.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.57723.log.gz 2024-09-21T00:30:59.635 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62642.log 2024-09-21T00:30:59.635 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.60320.log: /var/log/ceph/ceph-client.admin.63679.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59182.log 2024-09-21T00:30:59.635 INFO:teuthology.orchestra.run.smithi112.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.60320.log.gz 2024-09-21T00:30:59.635 INFO:teuthology.orchestra.run.smithi112.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.63679.log.gz 2024-09-21T00:30:59.635 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.62642.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44986.log 2024-09-21T00:30:59.635 INFO:teuthology.orchestra.run.smithi112.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.62642.log.gz 2024-09-21T00:30:59.635 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.59182.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68545.log 2024-09-21T00:30:59.635 INFO:teuthology.orchestra.run.smithi112.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.59182.log.gz 2024-09-21T00:30:59.635 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.44986.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63066.log 2024-09-21T00:30:59.635 INFO:teuthology.orchestra.run.smithi112.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.44986.log.gz 2024-09-21T00:30:59.636 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50801.log 2024-09-21T00:30:59.636 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.68545.log: /var/log/ceph/ceph-client.admin.63066.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53190.log 2024-09-21T00:30:59.636 INFO:teuthology.orchestra.run.smithi112.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.68545.log.gz 2024-09-21T00:30:59.636 INFO:teuthology.orchestra.run.smithi112.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.63066.log.gz 2024-09-21T00:30:59.636 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.50801.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59266.log 2024-09-21T00:30:59.636 INFO:teuthology.orchestra.run.smithi112.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.50801.log.gz 2024-09-21T00:30:59.636 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.53190.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48324.log 2024-09-21T00:30:59.636 INFO:teuthology.orchestra.run.smithi112.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.53190.log.gz 2024-09-21T00:30:59.636 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59982.log 2024-09-21T00:30:59.636 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.59266.log: /var/log/ceph/ceph-client.admin.48324.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45668.log 2024-09-21T00:30:59.636 INFO:teuthology.orchestra.run.smithi112.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.59266.log.gz 2024-09-21T00:30:59.636 INFO:teuthology.orchestra.run.smithi112.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48324.log.gz 2024-09-21T00:30:59.636 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.59982.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43466.log 2024-09-21T00:30:59.636 INFO:teuthology.orchestra.run.smithi112.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.59982.log.gz 2024-09-21T00:30:59.636 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63602.log 2024-09-21T00:30:59.663 INFO:teuthology.orchestra.run.smithi187.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.43619.log 2024-09-21T00:30:59.663 INFO:teuthology.orchestra.run.smithi187.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.44714.log 2024-09-21T00:30:59.663 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-client.3.43619.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.46574.log 2024-09-21T00:30:59.663 INFO:teuthology.orchestra.run.smithi187.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.3.43619.log.gz 2024-09-21T00:30:59.663 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-client.1.44714.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36520.log 2024-09-21T00:30:59.663 INFO:teuthology.orchestra.run.smithi187.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.44714.log.gz 2024-09-21T00:30:59.663 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-client.1.46574.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-09-21T00:30:59.663 INFO:teuthology.orchestra.run.smithi187.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.46574.log.gz 2024-09-21T00:30:59.663 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-client.admin.36520.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36280.log 2024-09-21T00:30:59.663 INFO:teuthology.orchestra.run.smithi187.stderr: 86.9%/var/log/ceph/ceph-osd.7.log: -- replaced with /var/log/ceph/ceph-client.admin.36520.log.gz 2024-09-21T00:30:59.663 INFO:teuthology.orchestra.run.smithi187.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36360.log 2024-09-21T00:30:59.663 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-client.admin.36280.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.47130.log 2024-09-21T00:30:59.663 INFO:teuthology.orchestra.run.smithi187.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.36280.log.gz 2024-09-21T00:30:59.664 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-client.admin.36360.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-09-21T00:30:59.664 INFO:teuthology.orchestra.run.smithi187.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.36360.log.gz 2024-09-21T00:30:59.664 INFO:teuthology.orchestra.run.smithi187.stderr:gzip/var/log/ceph/ceph-client.1.47130.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.36561.log 2024-09-21T00:30:59.664 INFO:teuthology.orchestra.run.smithi187.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.47130.log.gz 2024-09-21T00:30:59.664 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36440.log 2024-09-21T00:30:59.664 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-client.admin.36561.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36400.log 2024-09-21T00:30:59.664 INFO:teuthology.orchestra.run.smithi187.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.36561.log.gz 2024-09-21T00:30:59.664 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-client.admin.36440.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.47641.log 2024-09-21T00:30:59.664 INFO:teuthology.orchestra.run.smithi187.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.36440.log.gz 2024-09-21T00:30:59.664 INFO:teuthology.orchestra.run.smithi187.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-09-21T00:30:59.664 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-client.admin.36400.log: /var/log/ceph/ceph-client.1.47641.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-21T00:30:59.664 INFO:teuthology.orchestra.run.smithi187.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.47641.log.gz 2024-09-21T00:30:59.664 INFO:teuthology.orchestra.run.smithi187.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.36400.log.gz 2024-09-21T00:30:59.664 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.44217.log 2024-09-21T00:30:59.665 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36480.log 2024-09-21T00:30:59.665 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-client.1.44217.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.46084.log 2024-09-21T00:30:59.665 INFO:teuthology.orchestra.run.smithi187.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.44217.log.gz 2024-09-21T00:30:59.665 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-client.admin.36480.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.45535.log 2024-09-21T00:30:59.665 INFO:teuthology.orchestra.run.smithi187.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.36480.log.gz 2024-09-21T00:30:59.665 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-client.1.46084.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-09-21T00:30:59.665 INFO:teuthology.orchestra.run.smithi187.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.46084.log.gz 2024-09-21T00:30:59.665 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-client.1.45535.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.48046.log 2024-09-21T00:30:59.665 INFO:teuthology.orchestra.run.smithi187.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.45535.log.gz 2024-09-21T00:30:59.665 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.47645.log 2024-09-21T00:30:59.665 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-client.1.48046.log: gzip 0.0% -5 --verbose -- /var/log/ceph/ceph-client.1.43219.log 2024-09-21T00:30:59.665 INFO:teuthology.orchestra.run.smithi187.stderr: -- replaced with /var/log/ceph/ceph-client.1.48046.log.gz 2024-09-21T00:30:59.665 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-client.1.47645.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36320.log 2024-09-21T00:30:59.665 INFO:teuthology.orchestra.run.smithi187.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.47645.log.gz 2024-09-21T00:30:59.665 INFO:teuthology.orchestra.run.smithi187.stderr:gzip -5/var/log/ceph/ceph-client.1.43219.log: --verbose -- /var/log/ceph/ceph-client.1.45177.log 0.0% -- replaced with /var/log/ceph/ceph-client.1.43219.log.gz 2024-09-21T00:30:59.666 INFO:teuthology.orchestra.run.smithi187.stderr: 2024-09-21T00:30:59.666 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-client.admin.36320.log: gzip -5 --verbose 86.9% -- replaced with /var/log/ceph/ceph-client.admin.36320.log.gz 2024-09-21T00:30:59.666 INFO:teuthology.orchestra.run.smithi187.stderr: -- /var/log/ceph/ceph-osd.6.log 2024-09-21T00:30:59.666 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-client.1.45177.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.45177.log.gz 2024-09-21T00:30:59.666 INFO:teuthology.orchestra.run.smithi187.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.44213.log 2024-09-21T00:30:59.666 INFO:teuthology.orchestra.run.smithi187.stderr: 94.0% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-21T00:30:59.666 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.45531.log 2024-09-21T00:30:59.666 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-client.1.44213.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.1.44213.log.gz 2024-09-21T00:30:59.666 INFO:teuthology.orchestra.run.smithi187.stderr: --verbose -- /var/log/ceph/ceph-client.1.45173.log 2024-09-21T00:30:59.666 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-client.1.45531.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.45531.log.gz 2024-09-21T00:30:59.666 INFO:teuthology.orchestra.run.smithi187.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-09-21T00:30:59.666 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-client.1.45173.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.1.45173.log.gz 2024-09-21T00:30:59.666 INFO:teuthology.orchestra.run.smithi187.stderr: --verbose -- /var/log/ceph/ceph.log 2024-09-21T00:30:59.666 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-09-21T00:30:59.667 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-09-21T00:30:59.667 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-mds.d.log: 92.3% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-21T00:30:59.667 INFO:teuthology.orchestra.run.smithi187.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.44710.log 2024-09-21T00:30:59.667 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-mgr.x.log: 92.2% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-09-21T00:30:59.683 INFO:teuthology.orchestra.run.smithi187.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.48042.log 2024-09-21T00:30:59.683 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-client.1.44710.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.44710.log.gz 2024-09-21T00:30:59.683 INFO:teuthology.orchestra.run.smithi187.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.47134.log 2024-09-21T00:30:59.683 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-client.1.48042.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.48042.log.gz 2024-09-21T00:30:59.683 INFO:teuthology.orchestra.run.smithi187.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-09-21T00:30:59.683 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-client.1.47134.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.47134.log.gz 2024-09-21T00:30:59.684 INFO:teuthology.orchestra.run.smithi187.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.46080.log 2024-09-21T00:30:59.684 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-mon.b.log: 93.0% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-09-21T00:30:59.684 INFO:teuthology.orchestra.run.smithi187.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.43214.log 2024-09-21T00:30:59.684 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-client.1.46080.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.46080.log.gz 2024-09-21T00:30:59.684 INFO:teuthology.orchestra.run.smithi187.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.43615.log 2024-09-21T00:30:59.684 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-client.1.43214.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.43214.log.gz 2024-09-21T00:30:59.684 INFO:teuthology.orchestra.run.smithi187.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.46578.log 2024-09-21T00:30:59.684 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-client.3.43615.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.43615.log.gz 2024-09-21T00:30:59.684 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-client.1.46578.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.46578.log.gz 2024-09-21T00:30:59.749 INFO:teuthology.orchestra.run.smithi187.stderr: 91.4% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-09-21T00:30:59.829 INFO:teuthology.orchestra.run.smithi187.stderr: 91.4% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-09-21T00:30:59.845 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.45668.log: /var/log/ceph/ceph-client.admin.43466.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58013.log 2024-09-21T00:30:59.845 INFO:teuthology.orchestra.run.smithi112.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.45668.log.gz 2024-09-21T00:30:59.845 INFO:teuthology.orchestra.run.smithi112.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43466.log.gz 2024-09-21T00:30:59.845 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.63602.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57170.log 2024-09-21T00:30:59.845 INFO:teuthology.orchestra.run.smithi112.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.63602.log.gz 2024-09-21T00:30:59.845 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49129.log 2024-09-21T00:30:59.846 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.58013.log: /var/log/ceph/ceph-client.admin.57170.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.58013.log.gz 2024-09-21T00:30:59.846 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43508.log 2024-09-21T00:30:59.846 INFO:teuthology.orchestra.run.smithi112.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.57170.log.gz 2024-09-21T00:30:59.846 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57502.log 2024-09-21T00:30:59.846 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.49129.log: /var/log/ceph/ceph-client.admin.43508.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68629.log 2024-09-21T00:30:59.846 INFO:teuthology.orchestra.run.smithi112.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49129.log.gz 2024-09-21T00:30:59.846 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.57502.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43508.log.gz 2024-09-21T00:30:59.846 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54533.log 2024-09-21T00:30:59.846 INFO:teuthology.orchestra.run.smithi112.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.57502.log.gz 2024-09-21T00:30:59.846 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.68629.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65670.log 2024-09-21T00:30:59.846 INFO:teuthology.orchestra.run.smithi112.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.68629.log.gz 2024-09-21T00:30:59.846 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.54533.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53839.log 2024-09-21T00:30:59.846 INFO:teuthology.orchestra.run.smithi112.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.54533.log.gz 2024-09-21T00:30:59.847 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.65670.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36741.log 2024-09-21T00:30:59.847 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49087.log 2024-09-21T00:30:59.847 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.53839.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.65670.log.gz 2024-09-21T00:30:59.847 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.36741.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54575.log 2024-09-21T00:30:59.847 INFO:teuthology.orchestra.run.smithi112.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.53839.log.gz 2024-09-21T00:30:59.847 INFO:teuthology.orchestra.run.smithi112.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.36741.log.gz 2024-09-21T00:30:59.847 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.49087.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45569.log 2024-09-21T00:30:59.847 INFO:teuthology.orchestra.run.smithi112.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49087.log.gz 2024-09-21T00:30:59.847 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50330.log 2024-09-21T00:30:59.847 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.54575.log: /var/log/ceph/ceph-client.admin.45569.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.60956.log 2024-09-21T00:30:59.847 INFO:teuthology.orchestra.run.smithi112.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.54575.log.gz 2024-09-21T00:30:59.847 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.50330.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45569.log.gz 2024-09-21T00:30:59.847 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64910.log 2024-09-21T00:30:59.847 INFO:teuthology.orchestra.run.smithi112.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50330.log.gz 2024-09-21T00:30:59.847 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69007.log 2024-09-21T00:30:59.848 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.0.60956.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.60956.log.gz 2024-09-21T00:30:59.848 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.64910.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63902.log 2024-09-21T00:30:59.848 INFO:teuthology.orchestra.run.smithi112.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.64910.log.gz 2024-09-21T00:30:59.848 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.69007.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64154.log 2024-09-21T00:30:59.848 INFO:teuthology.orchestra.run.smithi112.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.69007.log.gz 2024-09-21T00:30:59.848 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57926.log 2024-09-21T00:30:59.848 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.63902.log: /var/log/ceph/ceph-client.admin.64154.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63150.log 2024-09-21T00:30:59.848 INFO:teuthology.orchestra.run.smithi112.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.63902.log.gz 2024-09-21T00:30:59.848 INFO:teuthology.orchestra.run.smithi112.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.64154.log.gz 2024-09-21T00:30:59.848 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.57926.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62982.log 2024-09-21T00:30:59.848 INFO:teuthology.orchestra.run.smithi112.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.57926.log.gz 2024-09-21T00:30:59.848 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52498.log 2024-09-21T00:30:59.848 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.63150.log: /var/log/ceph/ceph-client.admin.62982.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.63150.log.gz 2024-09-21T00:30:59.848 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50969.log 2024-09-21T00:30:59.849 INFO:teuthology.orchestra.run.smithi112.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.62982.log.gz 2024-09-21T00:30:59.849 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.52498.log: -- /var/log/ceph/ceph-client.admin.64341.log 2024-09-21T00:30:59.849 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.50969.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44219.log 87.0% 2024-09-21T00:30:59.849 INFO:teuthology.orchestra.run.smithi112.stderr: -- replaced with /var/log/ceph/ceph-client.admin.52498.log.gz 2024-09-21T00:30:59.849 INFO:teuthology.orchestra.run.smithi112.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50969.log.gz 2024-09-21T00:30:59.849 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.64341.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58280.log 2024-09-21T00:30:59.849 INFO:teuthology.orchestra.run.smithi112.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.64341.log.gz 2024-09-21T00:30:59.849 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.44219.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65124.log 2024-09-21T00:30:59.849 INFO:teuthology.orchestra.run.smithi112.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44219.log.gz 2024-09-21T00:30:59.849 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60278.log 2024-09-21T00:30:59.849 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.58280.log: /var/log/ceph/ceph-client.admin.65124.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36771.log 2024-09-21T00:30:59.849 INFO:teuthology.orchestra.run.smithi112.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.58280.log.gz 2024-09-21T00:30:59.849 INFO:teuthology.orchestra.run.smithi112.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.65124.log.gz 2024-09-21T00:30:59.849 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.60278.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59902.log 2024-09-21T00:30:59.850 INFO:teuthology.orchestra.run.smithi112.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.60278.log.gz 2024-09-21T00:30:59.850 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.36771.log: /var/log/ceph/ceph-client.admin.47494.log 2024-09-21T00:30:59.850 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.59902.log: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.36771.log.gz 2024-09-21T00:30:59.850 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65292.log 2024-09-21T00:30:59.850 INFO:teuthology.orchestra.run.smithi112.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.59902.log.gz 2024-09-21T00:30:59.850 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36816.log 2024-09-21T00:30:59.850 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.47494.log: /var/log/ceph/ceph-client.admin.65292.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44774.log 2024-09-21T00:30:59.850 INFO:teuthology.orchestra.run.smithi112.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47494.log.gz 2024-09-21T00:30:59.850 INFO:teuthology.orchestra.run.smithi112.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.65292.log.gz 2024-09-21T00:30:59.850 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.36816.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54403.log 2024-09-21T00:30:59.850 INFO:teuthology.orchestra.run.smithi112.stderr: 86.0% -- replaced with /var/log/ceph/ceph-client.admin.36816.log.gz 2024-09-21T00:30:59.850 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.44774.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55874.log 2024-09-21T00:30:59.850 INFO:teuthology.orchestra.run.smithi112.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44774.log.gz 2024-09-21T00:30:59.850 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.54403.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58192.log 2024-09-21T00:30:59.851 INFO:teuthology.orchestra.run.smithi112.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.54403.log.gz 2024-09-21T00:30:59.851 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.55874.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53316.log 2024-09-21T00:30:59.851 INFO:teuthology.orchestra.run.smithi112.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.55874.log.gz 2024-09-21T00:30:59.851 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.55793.log 2024-09-21T00:30:59.851 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.58192.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.58192.log.gz 2024-09-21T00:30:59.851 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.53316.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67607.log 2024-09-21T00:30:59.851 INFO:teuthology.orchestra.run.smithi112.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.53316.log.gz 2024-09-21T00:30:59.851 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.0.55793.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48240.log 2024-09-21T00:30:59.851 INFO:teuthology.orchestra.run.smithi112.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.55793.log.gz 2024-09-21T00:30:59.851 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.67607.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65586.log 2024-09-21T00:30:59.851 INFO:teuthology.orchestra.run.smithi112.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.67607.log.gz 2024-09-21T00:30:59.851 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.48240.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54449.log 2024-09-21T00:30:59.851 INFO:teuthology.orchestra.run.smithi112.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48240.log.gz 2024-09-21T00:30:59.851 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.65586.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60656.log 2024-09-21T00:30:59.851 INFO:teuthology.orchestra.run.smithi112.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.65586.log.gz 2024-09-21T00:30:59.852 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68965.log 2024-09-21T00:30:59.852 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.54449.log: /var/log/ceph/ceph-client.admin.60656.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60404.log 2024-09-21T00:30:59.852 INFO:teuthology.orchestra.run.smithi112.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.54449.log.gz 2024-09-21T00:30:59.852 INFO:teuthology.orchestra.run.smithi112.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.60656.log.gz 2024-09-21T00:30:59.852 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.68965.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36561.log 2024-09-21T00:30:59.852 INFO:teuthology.orchestra.run.smithi112.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.68965.log.gz 2024-09-21T00:30:59.852 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52270.log 2024-09-21T00:30:59.852 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.60404.log: /var/log/ceph/ceph-client.admin.36561.log: gzip -5 87.5% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.60404.log.gz /var/log/ceph/ceph-client.admin.54224.log 2024-09-21T00:30:59.852 INFO:teuthology.orchestra.run.smithi112.stderr: 2024-09-21T00:30:59.852 INFO:teuthology.orchestra.run.smithi112.stderr: 85.9% -- replaced with /var/log/ceph/ceph-client.admin.36561.log.gz 2024-09-21T00:30:59.852 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.52270.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51972.log 2024-09-21T00:30:59.852 INFO:teuthology.orchestra.run.smithi112.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.52270.log.gz 2024-09-21T00:30:59.852 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.54224.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69217.log 2024-09-21T00:30:59.852 INFO:teuthology.orchestra.run.smithi112.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.54224.log.gz 2024-09-21T00:30:59.853 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.51972.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67910.log 2024-09-21T00:30:59.853 INFO:teuthology.orchestra.run.smithi112.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51972.log.gz 2024-09-21T00:30:59.853 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.69217.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47632.log 2024-09-21T00:30:59.853 INFO:teuthology.orchestra.run.smithi112.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.69217.log.gz 2024-09-21T00:30:59.853 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.67910.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49511.log 2024-09-21T00:30:59.853 INFO:teuthology.orchestra.run.smithi112.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.67910.log.gz 2024-09-21T00:30:59.853 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.47632.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59392.log 2024-09-21T00:30:59.853 INFO:teuthology.orchestra.run.smithi112.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47632.log.gz 2024-09-21T00:30:59.853 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.49511.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67994.log 2024-09-21T00:30:59.853 INFO:teuthology.orchestra.run.smithi112.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.49511.log.gz 2024-09-21T00:30:59.853 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48450.log 2024-09-21T00:30:59.853 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.59392.log: /var/log/ceph/ceph-client.admin.67994.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.59392.log.gz 2024-09-21T00:30:59.853 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44732.log 2024-09-21T00:30:59.854 INFO:teuthology.orchestra.run.smithi112.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.67994.log.gz 2024-09-21T00:30:59.854 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.48450.log: /var/log/ceph/ceph-client.admin.49217.log 2024-09-21T00:30:59.854 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.44732.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47284.log 2024-09-21T00:30:59.854 INFO:teuthology.orchestra.run.smithi112.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48450.log.gz 2024-09-21T00:30:59.854 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.49217.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44732.log.gz 2024-09-21T00:30:59.854 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.47091.log 2024-09-21T00:30:59.854 INFO:teuthology.orchestra.run.smithi112.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49217.log.gz 2024-09-21T00:30:59.854 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.47284.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45333.log 2024-09-21T00:30:59.854 INFO:teuthology.orchestra.run.smithi112.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47284.log.gz 2024-09-21T00:30:59.854 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.2.47091.log: 0.0% -- replaced with /var/log/ceph/ceph-client.2.47091.log.gz 2024-09-21T00:30:59.854 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62600.log 2024-09-21T00:30:59.854 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36620.log 2024-09-21T00:30:59.854 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.45333.log: /var/log/ceph/ceph-client.admin.62600.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36386.log 2024-09-21T00:30:59.854 INFO:teuthology.orchestra.run.smithi112.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.45333.log.gz 2024-09-21T00:30:59.855 INFO:teuthology.orchestra.run.smithi112.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.62600.log.gz 2024-09-21T00:30:59.855 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.36620.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55319.log 2024-09-21T00:30:59.855 INFO:teuthology.orchestra.run.smithi112.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.36620.log.gz 2024-09-21T00:30:59.855 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.55001.log 2024-09-21T00:30:59.855 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.36386.log: 79.1% -- replaced with /var/log/ceph/ceph-client.admin.36386.log.gz 2024-09-21T00:30:59.855 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.55319.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43256.log 2024-09-21T00:30:59.855 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.0.55001.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.55001.log.gz 2024-09-21T00:30:59.855 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51408.log 2024-09-21T00:30:59.855 INFO:teuthology.orchestra.run.smithi112.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.55319.log.gz 2024-09-21T00:30:59.855 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43084.log 2024-09-21T00:30:59.855 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.43256.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.43256.log.gz 2024-09-21T00:30:59.855 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.51408.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60446.log 2024-09-21T00:30:59.855 INFO:teuthology.orchestra.run.smithi112.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.51408.log.gz 2024-09-21T00:30:59.856 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44988.log 2024-09-21T00:30:59.856 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.43084.log: /var/log/ceph/ceph-client.admin.60446.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53090.log 2024-09-21T00:30:59.856 INFO:teuthology.orchestra.run.smithi112.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43084.log.gz 2024-09-21T00:30:59.856 INFO:teuthology.orchestra.run.smithi112.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.60446.log.gz 2024-09-21T00:30:59.856 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.44988.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61598.log 2024-09-21T00:30:59.856 INFO:teuthology.orchestra.run.smithi112.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.44988.log.gz 2024-09-21T00:30:59.856 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42679.log 2024-09-21T00:30:59.856 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.53090.log: /var/log/ceph/ceph-client.admin.61598.log: gzip -5 87.7% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.53090.log.gz /var/log/ceph/ceph-client.admin.53358.log 2024-09-21T00:30:59.856 INFO:teuthology.orchestra.run.smithi112.stderr: 2024-09-21T00:30:59.856 INFO:teuthology.orchestra.run.smithi112.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.61598.log.gz 2024-09-21T00:30:59.856 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.42679.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.50518.log 2024-09-21T00:30:59.856 INFO:teuthology.orchestra.run.smithi112.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42679.log.gz 2024-09-21T00:30:59.856 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.53358.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44277.log 2024-09-21T00:30:59.856 INFO:teuthology.orchestra.run.smithi112.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.53358.log.gz 2024-09-21T00:30:59.857 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.0.50518.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.50518.log.gz 2024-09-21T00:30:59.857 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62277.log 2024-09-21T00:30:59.857 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.44277.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49571.log 2024-09-21T00:30:59.857 INFO:teuthology.orchestra.run.smithi112.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.44277.log.gz 2024-09-21T00:30:59.857 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.62277.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59098.log 2024-09-21T00:30:59.857 INFO:teuthology.orchestra.run.smithi112.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.62277.log.gz 2024-09-21T00:30:59.857 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-21T00:30:59.857 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.49571.log: /var/log/ceph/ceph-client.admin.59098.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65082.log 2024-09-21T00:30:59.857 INFO:teuthology.orchestra.run.smithi112.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.49571.log.gz 2024-09-21T00:30:59.857 INFO:teuthology.orchestra.run.smithi112.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.59098.log.gz 2024-09-21T00:30:59.857 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50927.log 2024-09-21T00:30:59.857 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69259.log 2024-09-21T00:30:59.857 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.65082.log: /var/log/ceph/ceph-client.admin.50927.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.65082.log.gz 2024-09-21T00:30:59.857 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69091.log 2024-09-21T00:30:59.857 INFO:teuthology.orchestra.run.smithi112.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50927.log.gz 2024-09-21T00:30:59.858 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.69259.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65736.log 2024-09-21T00:30:59.858 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.69091.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.69259.log.gz 2024-09-21T00:30:59.858 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61682.log 2024-09-21T00:30:59.858 INFO:teuthology.orchestra.run.smithi112.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.69091.log.gz 2024-09-21T00:30:59.858 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.65736.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45667.log 2024-09-21T00:30:59.858 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- 87.4% /var/log/ceph/ceph-client.admin.57544.log 2024-09-21T00:30:59.858 INFO:teuthology.orchestra.run.smithi112.stderr: -- replaced with /var/log/ceph/ceph-client.admin.65736.log.gz 2024-09-21T00:30:59.858 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.61682.log: 89.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-21T00:30:59.858 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.45667.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.61682.log.gz 2024-09-21T00:30:59.858 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55163.log 2024-09-21T00:30:59.858 INFO:teuthology.orchestra.run.smithi112.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45667.log.gz 2024-09-21T00:30:59.858 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64688.log 2024-09-21T00:30:59.858 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.57544.log: /var/log/ceph/ceph-client.admin.55163.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64996.log 2024-09-21T00:30:59.858 INFO:teuthology.orchestra.run.smithi112.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.57544.log.gz 2024-09-21T00:30:59.859 INFO:teuthology.orchestra.run.smithi112.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.55163.log.gz 2024-09-21T00:30:59.859 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.64688.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68839.log 2024-09-21T00:30:59.859 INFO:teuthology.orchestra.run.smithi112.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.64688.log.gz 2024-09-21T00:30:59.859 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67160.log 2024-09-21T00:30:59.859 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.64996.log: /var/log/ceph/ceph-client.admin.68839.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63024.log 2024-09-21T00:30:59.859 INFO:teuthology.orchestra.run.smithi112.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.64996.log.gz 2024-09-21T00:30:59.859 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.67160.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.68839.log.gz 2024-09-21T00:30:59.859 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60572.log 2024-09-21T00:30:59.859 INFO:teuthology.orchestra.run.smithi112.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.67160.log.gz 2024-09-21T00:30:59.859 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.63024.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48114.log 2024-09-21T00:30:59.859 INFO:teuthology.orchestra.run.smithi112.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.63024.log.gz 2024-09-21T00:30:59.859 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.60572.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36271.log 2024-09-21T00:30:59.859 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.48114.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57460.log 2024-09-21T00:30:59.859 INFO:teuthology.orchestra.run.smithi112.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.60572.log.gz 2024-09-21T00:30:59.860 INFO:teuthology.orchestra.run.smithi112.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48114.log.gz 2024-09-21T00:30:59.860 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.36271.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53148.log 2024-09-21T00:30:59.860 INFO:teuthology.orchestra.run.smithi112.stderr: 76.7% -- replaced with /var/log/ceph/ceph-client.admin.36271.log.gz 2024-09-21T00:30:59.860 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.57460.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58963.log 2024-09-21T00:30:59.860 INFO:teuthology.orchestra.run.smithi112.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.57460.log.gz 2024-09-21T00:30:59.860 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.53148.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44998.log 2024-09-21T00:30:59.860 INFO:teuthology.orchestra.run.smithi112.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53148.log.gz 2024-09-21T00:30:59.860 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.58963.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55385.log 2024-09-21T00:30:59.860 INFO:teuthology.orchestra.run.smithi112.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.58963.log.gz 2024-09-21T00:30:59.860 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53587.log 2024-09-21T00:30:59.860 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.44998.log: /var/log/ceph/ceph-client.admin.55385.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.63450.log 2024-09-21T00:30:59.860 INFO:teuthology.orchestra.run.smithi112.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.44998.log.gz 2024-09-21T00:30:59.860 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.53587.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44480.log 2024-09-21T00:30:59.860 INFO:teuthology.orchestra.run.smithi112.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.55385.log.gz 2024-09-21T00:30:59.861 INFO:teuthology.orchestra.run.smithi112.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53587.log.gz 2024-09-21T00:30:59.861 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.0.63450.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61766.log 2024-09-21T00:30:59.861 INFO:teuthology.orchestra.run.smithi112.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.63450.log.gz 2024-09-21T00:30:59.861 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36475.log 2024-09-21T00:30:59.861 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.44480.log: /var/log/ceph/ceph-client.admin.61766.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68671.log 2024-09-21T00:30:59.861 INFO:teuthology.orchestra.run.smithi112.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44480.log.gz 2024-09-21T00:30:59.861 INFO:teuthology.orchestra.run.smithi112.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.61766.log.gz/var/log/ceph/ceph-client.admin.36475.log: 2024-09-21T00:30:59.861 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53400.log 2024-09-21T00:30:59.861 INFO:teuthology.orchestra.run.smithi112.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.36475.log.gz 2024-09-21T00:30:59.861 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43170.log 2024-09-21T00:30:59.861 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.68671.log: /var/log/ceph/ceph-client.admin.53400.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49469.log 2024-09-21T00:30:59.861 INFO:teuthology.orchestra.run.smithi112.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.68671.log.gz 2024-09-21T00:30:59.861 INFO:teuthology.orchestra.run.smithi112.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.53400.log.gz 2024-09-21T00:30:59.861 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.43170.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36774.log 2024-09-21T00:30:59.862 INFO:teuthology.orchestra.run.smithi112.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43170.log.gz 2024-09-21T00:30:59.862 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67529.log 2024-09-21T00:30:59.862 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.49469.log: /var/log/ceph/ceph-client.admin.36774.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49469.log.gz 2024-09-21T00:30:59.862 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46118.log 2024-09-21T00:30:59.862 INFO:teuthology.orchestra.run.smithi112.stderr: 85.9% -- replaced with /var/log/ceph/ceph-client.admin.36774.log.gz 2024-09-21T00:30:59.862 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.58790.log 2024-09-21T00:30:59.862 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.67529.log: /var/log/ceph/ceph-client.admin.46118.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58322.log 2024-09-21T00:30:59.862 INFO:teuthology.orchestra.run.smithi112.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.67529.log.gz 2024-09-21T00:30:59.862 INFO:teuthology.orchestra.run.smithi112.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46118.log.gz 2024-09-21T00:30:59.862 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.0.58790.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.58790.log.gz 2024-09-21T00:30:59.862 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50268.log 2024-09-21T00:30:59.862 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.58322.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51450.log 2024-09-21T00:30:59.862 INFO:teuthology.orchestra.run.smithi112.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.58322.log.gz 2024-09-21T00:30:59.862 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.50268.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49036.log 2024-09-21T00:30:59.863 INFO:teuthology.orchestra.run.smithi112.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50268.log.gz 2024-09-21T00:30:59.863 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52648.log 2024-09-21T00:30:59.863 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.51450.log: /var/log/ceph/ceph-client.admin.49036.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43340.log 2024-09-21T00:30:59.863 INFO:teuthology.orchestra.run.smithi112.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51450.log.gz 2024-09-21T00:30:59.863 INFO:teuthology.orchestra.run.smithi112.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49036.log.gz 2024-09-21T00:30:59.863 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.52648.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64299.log 2024-09-21T00:30:59.863 INFO:teuthology.orchestra.run.smithi112.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.52648.log.gz 2024-09-21T00:30:59.863 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.43340.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54809.log 2024-09-21T00:30:59.863 INFO:teuthology.orchestra.run.smithi112.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43340.log.gz 2024-09-21T00:30:59.863 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52354.log 2024-09-21T00:30:59.863 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.64299.log: /var/log/ceph/ceph-client.admin.54809.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57376.log 2024-09-21T00:30:59.863 INFO:teuthology.orchestra.run.smithi112.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.64299.log.gz 2024-09-21T00:30:59.863 INFO:teuthology.orchestra.run.smithi112.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.54809.log.gz 2024-09-21T00:30:59.863 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.52354.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36299.log 2024-09-21T00:30:59.863 INFO:teuthology.orchestra.run.smithi112.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52354.log.gz 2024-09-21T00:30:59.864 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68312.log 2024-09-21T00:30:59.864 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.57376.log: /var/log/ceph/ceph-client.admin.36299.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.57376.log.gz 2024-09-21T00:30:59.864 INFO:teuthology.orchestra.run.smithi112.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36299.log.gz 2024-09-21T00:30:59.864 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50885.log 2024-09-21T00:30:59.864 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56783.log 2024-09-21T00:30:59.864 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.68312.log: /var/log/ceph/ceph-client.admin.50885.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.50522.log 2024-09-21T00:30:59.864 INFO:teuthology.orchestra.run.smithi112.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.68312.log.gz 2024-09-21T00:30:59.864 INFO:teuthology.orchestra.run.smithi112.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50885.log.gz 2024-09-21T00:30:59.864 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.56783.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42938.log 2024-09-21T00:30:59.864 INFO:teuthology.orchestra.run.smithi112.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.56783.log.gz 2024-09-21T00:30:59.864 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.0.50522.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.50522.log.gz 2024-09-21T00:30:59.864 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45329.log 2024-09-21T00:30:59.864 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36355.log 2024-09-21T00:30:59.864 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.42938.log: /var/log/ceph/ceph-client.admin.45329.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66285.log 2024-09-21T00:30:59.864 INFO:teuthology.orchestra.run.smithi112.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42938.log.gz 2024-09-21T00:30:59.865 INFO:teuthology.orchestra.run.smithi112.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.45329.log.gz 2024-09-21T00:30:59.865 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.36355.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.65928.log 2024-09-21T00:30:59.865 INFO:teuthology.orchestra.run.smithi112.stderr: 76.5% -- replaced with /var/log/ceph/ceph-client.admin.36355.log.gz 2024-09-21T00:30:59.865 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.66285.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59140.log 2024-09-21T00:30:59.865 INFO:teuthology.orchestra.run.smithi112.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.66285.log.gz 2024-09-21T00:30:59.865 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.0.65928.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61850.log 2024-09-21T00:30:59.865 INFO:teuthology.orchestra.run.smithi112.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.65928.log.gz 2024-09-21T00:30:59.865 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.59140.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60698.log 2024-09-21T00:30:59.865 INFO:teuthology.orchestra.run.smithi112.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.59140.log.gz 2024-09-21T00:30:59.865 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65334.log 2024-09-21T00:30:59.865 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.61850.log: /var/log/ceph/ceph-client.admin.60698.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64196.log 2024-09-21T00:30:59.865 INFO:teuthology.orchestra.run.smithi112.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.61850.log.gz 2024-09-21T00:30:59.865 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.65334.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44522.log 2024-09-21T00:30:59.865 INFO:teuthology.orchestra.run.smithi112.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.65334.log.gz 2024-09-21T00:30:59.866 INFO:teuthology.orchestra.run.smithi112.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.60698.log.gz 2024-09-21T00:30:59.866 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.64196.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66158.log 2024-09-21T00:30:59.866 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.44522.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66615.log 2024-09-21T00:30:59.866 INFO:teuthology.orchestra.run.smithi112.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.64196.log.gz 2024-09-21T00:30:59.866 INFO:teuthology.orchestra.run.smithi112.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44522.log.gz 2024-09-21T00:30:59.866 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.66158.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61269.log 2024-09-21T00:30:59.866 INFO:teuthology.orchestra.run.smithi112.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.66158.log.gz 2024-09-21T00:30:59.866 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.66615.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47589.log 2024-09-21T00:30:59.866 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68755.log 2024-09-21T00:30:59.866 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.61269.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.66615.log.gz 2024-09-21T00:30:59.866 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.47589.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51832.log 2024-09-21T00:30:59.866 INFO:teuthology.orchestra.run.smithi112.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.61269.log.gz 2024-09-21T00:30:59.866 INFO:teuthology.orchestra.run.smithi112.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.47589.log.gz 2024-09-21T00:30:59.866 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.68755.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.56635.log 2024-09-21T00:30:59.866 INFO:teuthology.orchestra.run.smithi112.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.68755.log.gz 2024-09-21T00:30:59.867 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.51832.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47368.log 2024-09-21T00:30:59.867 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.0.56635.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46160.log 2024-09-21T00:30:59.867 INFO:teuthology.orchestra.run.smithi112.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.51832.log.gz 2024-09-21T00:30:59.867 INFO:teuthology.orchestra.run.smithi112.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.56635.log.gz 2024-09-21T00:30:59.867 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.47368.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64425.log 2024-09-21T00:30:59.867 INFO:teuthology.orchestra.run.smithi112.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47368.log.gz 2024-09-21T00:30:59.867 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.46160.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-09-21T00:30:59.867 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52186.log 2024-09-21T00:30:59.867 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.64425.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.46160.log.gz 2024-09-21T00:30:59.867 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45908.log 2024-09-21T00:30:59.867 INFO:teuthology.orchestra.run.smithi112.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.64425.log.gz 2024-09-21T00:30:59.867 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51748.log 2024-09-21T00:30:59.867 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.52186.log: /var/log/ceph/ceph-client.admin.45908.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.52186.log.gzgzip 2024-09-21T00:30:59.867 INFO:teuthology.orchestra.run.smithi112.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.48156.log 2024-09-21T00:30:59.868 INFO:teuthology.orchestra.run.smithi112.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.45908.log.gz 2024-09-21T00:30:59.868 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.51748.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51095.log 2024-09-21T00:30:59.868 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.48156.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51748.log.gz 2024-09-21T00:30:59.868 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59639.log 2024-09-21T00:30:59.868 INFO:teuthology.orchestra.run.smithi112.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48156.log.gz 2024-09-21T00:30:59.868 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.51095.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61556.log 2024-09-21T00:30:59.868 INFO:teuthology.orchestra.run.smithi112.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51095.log.gz 2024-09-21T00:30:59.868 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.59639.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.56631.log 2024-09-21T00:30:59.868 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.61556.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.59639.log.gz 2024-09-21T00:30:59.868 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64593.log 2024-09-21T00:30:59.868 INFO:teuthology.orchestra.run.smithi112.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.61556.log.gz 2024-09-21T00:30:59.868 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.0.56631.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.56631.log.gz 2024-09-21T00:30:59.868 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69343.log 2024-09-21T00:30:59.868 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43212.log 2024-09-21T00:30:59.868 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.64593.log: /var/log/ceph/ceph-client.admin.69343.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.64593.log.gz 2024-09-21T00:30:59.869 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65208.log 2024-09-21T00:30:59.869 INFO:teuthology.orchestra.run.smithi112.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.69343.log.gz 2024-09-21T00:30:59.869 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.43212.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59723.log 2024-09-21T00:30:59.869 INFO:teuthology.orchestra.run.smithi112.stderr:gzip 87.3% -5 -- replaced with /var/log/ceph/ceph-client.admin.43212.log.gz --verbose 2024-09-21T00:30:59.869 INFO:teuthology.orchestra.run.smithi112.stderr: -- /var/log/ceph/ceph-client.admin.45332.log 2024-09-21T00:30:59.869 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.65208.log: /var/log/ceph/ceph-client.admin.59723.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.65208.log.gz 2024-09-21T00:30:59.869 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55511.log 87.3% -- replaced with /var/log/ceph/ceph-client.admin.59723.log.gz 2024-09-21T00:30:59.869 INFO:teuthology.orchestra.run.smithi112.stderr: 2024-09-21T00:30:59.869 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.45332.log: gzip -5 --verbose -- 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45332.log.gz 2024-09-21T00:30:59.869 INFO:teuthology.orchestra.run.smithi112.stderr: /var/log/ceph/ceph-client.admin.64028.log 2024-09-21T00:30:59.869 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.55511.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57128.log 2024-09-21T00:30:59.869 INFO:teuthology.orchestra.run.smithi112.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.55511.log.gz 2024-09-21T00:30:59.869 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.64028.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.49955.log 2024-09-21T00:30:59.869 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.57128.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.64028.log.gz 2024-09-21T00:30:59.869 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43040.log 2024-09-21T00:30:59.870 INFO:teuthology.orchestra.run.smithi112.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.57128.log.gz 2024-09-21T00:30:59.870 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52396.log 2024-09-21T00:30:59.870 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.0.49955.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.49955.log.gz 2024-09-21T00:30:59.870 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59765.log 2024-09-21T00:30:59.870 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.43040.log: /var/log/ceph/ceph-client.admin.52396.log: 87.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61724.log 2024-09-21T00:30:59.870 INFO:teuthology.orchestra.run.smithi112.stderr: -- replaced with /var/log/ceph/ceph-client.admin.43040.log.gz 2024-09-21T00:30:59.870 INFO:teuthology.orchestra.run.smithi112.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.52396.log.gz 2024-09-21T00:30:59.870 INFO:teuthology.orchestra.run.smithi112.stderr:gzip/var/log/ceph/ceph-client.admin.59765.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.68036.log 2024-09-21T00:30:59.870 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52456.log 2024-09-21T00:30:59.870 INFO:teuthology.orchestra.run.smithi112.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.59765.log.gz 2024-09-21T00:30:59.870 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.61724.log: /var/log/ceph/ceph-client.admin.68036.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44606.log 2024-09-21T00:30:59.870 INFO:teuthology.orchestra.run.smithi112.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.61724.log.gz 2024-09-21T00:30:59.870 INFO:teuthology.orchestra.run.smithi112.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.68036.log.gz 2024-09-21T00:30:59.871 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.52456.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47821.log 2024-09-21T00:30:59.871 INFO:teuthology.orchestra.run.smithi112.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.52456.log.gz 2024-09-21T00:30:59.871 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.44606.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65544.log 2024-09-21T00:30:59.871 INFO:teuthology.orchestra.run.smithi112.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.44606.log.gz 2024-09-21T00:30:59.871 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.47821.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59453.log 2024-09-21T00:30:59.871 INFO:teuthology.orchestra.run.smithi112.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47821.log.gz 2024-09-21T00:30:59.871 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.65544.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54137.log 2024-09-21T00:30:59.871 INFO:teuthology.orchestra.run.smithi112.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.65544.log.gz 2024-09-21T00:30:59.871 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48653.log 2024-09-21T00:30:59.871 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.59453.log: /var/log/ceph/ceph-client.admin.54137.log: gzip 87.5% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.59453.log.gz -- 2024-09-21T00:30:59.871 INFO:teuthology.orchestra.run.smithi112.stderr: /var/log/ceph/ceph-client.admin.62505.log 2024-09-21T00:30:59.871 INFO:teuthology.orchestra.run.smithi112.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.54137.log.gz 2024-09-21T00:30:59.871 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.48653.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57292.log 2024-09-21T00:30:59.871 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.62505.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48653.log.gz 2024-09-21T00:30:59.871 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61227.log 2024-09-21T00:30:59.872 INFO:teuthology.orchestra.run.smithi112.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.62505.log.gz 2024-09-21T00:30:59.872 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.57292.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54743.log 2024-09-21T00:30:59.872 INFO:teuthology.orchestra.run.smithi112.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.57292.log.gz 2024-09-21T00:30:59.872 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.61227.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66287.log 2024-09-21T00:30:59.872 INFO:teuthology.orchestra.run.smithi112.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.61227.log.gz 2024-09-21T00:30:59.872 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.54743.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52014.log 2024-09-21T00:30:59.872 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.66287.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44996.log 2024-09-21T00:30:59.872 INFO:teuthology.orchestra.run.smithi112.stderr: 90.9% 87.5% -- replaced with /var/log/ceph/ceph-client.admin.66287.log.gz 2024-09-21T00:30:59.872 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44993.log 2024-09-21T00:30:59.872 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.52014.log: -- replaced with /var/log/ceph/ceph-client.admin.54743.log.gz 2024-09-21T00:30:59.872 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.44996.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52014.log.gz 2024-09-21T00:30:59.872 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50651.log 2024-09-21T00:30:59.872 INFO:teuthology.orchestra.run.smithi112.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.44996.log.gz 2024-09-21T00:30:59.872 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56443.log 2024-09-21T00:30:59.873 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.44993.log: /var/log/ceph/ceph-client.admin.50651.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66573.log 2024-09-21T00:30:59.873 INFO:teuthology.orchestra.run.smithi112.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44993.log.gz 2024-09-21T00:30:59.873 INFO:teuthology.orchestra.run.smithi112.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.50651.log.gz 2024-09-21T00:30:59.873 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.56443.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68270.log 2024-09-21T00:30:59.873 INFO:teuthology.orchestra.run.smithi112.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.56443.log.gz 2024-09-21T00:30:59.873 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.66573.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51324.log 2024-09-21T00:30:59.873 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51240.log 2024-09-21T00:30:59.873 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.68270.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.66573.log.gz 2024-09-21T00:30:59.873 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.51324.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.68270.log.gz 2024-09-21T00:30:59.873 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62856.log 2024-09-21T00:30:59.873 INFO:teuthology.orchestra.run.smithi112.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51324.log.gz 2024-09-21T00:30:59.873 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.51240.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55121.log 2024-09-21T00:30:59.873 INFO:teuthology.orchestra.run.smithi112.stderr:gzip 87.7% -5 -- replaced with /var/log/ceph/ceph-client.admin.51240.log.gz --verbose 2024-09-21T00:30:59.873 INFO:teuthology.orchestra.run.smithi112.stderr: -- /var/log/ceph/ceph-client.admin.36775.log 2024-09-21T00:30:59.873 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.62856.log: gzip 87.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.62856.log.gz --verbose 2024-09-21T00:30:59.874 INFO:teuthology.orchestra.run.smithi112.stderr: -- /var/log/ceph/ceph-client.admin.54095.log 2024-09-21T00:30:59.874 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.55121.log: /var/log/ceph/ceph-client.admin.36775.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.55121.log.gz 2024-09-21T00:30:59.874 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51667.log 2024-09-21T00:30:59.874 INFO:teuthology.orchestra.run.smithi112.stderr: 85.7% -- replaced with /var/log/ceph/ceph-client.admin.36775.log.gz 2024-09-21T00:30:59.874 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.54095.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55916.log 2024-09-21T00:30:59.874 INFO:teuthology.orchestra.run.smithi112.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.54095.log.gz 2024-09-21T00:30:59.874 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45950.log 2024-09-21T00:30:59.874 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.51667.log: /var/log/ceph/ceph-client.admin.55916.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51667.log.gz 2024-09-21T00:30:59.874 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68078.log 2024-09-21T00:30:59.874 INFO:teuthology.orchestra.run.smithi112.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.55916.log.gz 2024-09-21T00:30:59.874 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.45950.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68797.log 2024-09-21T00:30:59.874 INFO:teuthology.orchestra.run.smithi112.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45950.log.gz 2024-09-21T00:30:59.874 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49171.log 2024-09-21T00:30:59.875 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.68078.log: /var/log/ceph/ceph-client.admin.68797.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.68078.log.gz 2024-09-21T00:30:59.875 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56960.log 2024-09-21T00:30:59.875 INFO:teuthology.orchestra.run.smithi112.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.68797.log.gz 2024-09-21T00:30:59.875 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.49171.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69301.log 2024-09-21T00:30:59.875 INFO:teuthology.orchestra.run.smithi112.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49171.log.gz 2024-09-21T00:30:59.875 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59555.log 2024-09-21T00:30:59.875 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.56960.log: /var/log/ceph/ceph-client.admin.69301.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.56960.log.gz 2024-09-21T00:30:59.875 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67826.log 2024-09-21T00:30:59.875 INFO:teuthology.orchestra.run.smithi112.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.69301.log.gz 2024-09-21T00:30:59.875 INFO:teuthology.orchestra.run.smithi112.stderr:gzip/var/log/ceph/ceph-client.admin.59555.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.54361.log 2024-09-21T00:30:59.875 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.67826.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.59555.log.gz 2024-09-21T00:30:59.875 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45331.log 2024-09-21T00:30:59.875 INFO:teuthology.orchestra.run.smithi112.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.67826.log.gz 2024-09-21T00:30:59.875 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.54361.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61150.log 2024-09-21T00:30:59.876 INFO:teuthology.orchestra.run.smithi112.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.54361.log.gz 2024-09-21T00:30:59.876 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5/var/log/ceph/ceph-client.admin.45331.log: --verbose -- /var/log/ceph/ceph-client.admin.63763.log 2024-09-21T00:30:59.876 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.61150.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.45331.log.gz 2024-09-21T00:30:59.876 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36758.log 2024-09-21T00:30:59.876 INFO:teuthology.orchestra.run.smithi112.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.61150.log.gz 2024-09-21T00:30:59.876 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.63763.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59681.log 2024-09-21T00:30:59.876 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.36758.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.63763.log.gz 2024-09-21T00:30:59.876 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48530.log 2024-09-21T00:30:59.876 INFO:teuthology.orchestra.run.smithi112.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.36758.log.gz 2024-09-21T00:30:59.876 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.59681.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-09-21T00:30:59.876 INFO:teuthology.orchestra.run.smithi112.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.59681.log.gz 2024-09-21T00:30:59.876 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.48530.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65376.log 2024-09-21T00:30:59.876 INFO:teuthology.orchestra.run.smithi112.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48530.log.gz 2024-09-21T00:30:59.876 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51587.log 2024-09-21T00:30:59.876 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.65376.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64509.log 2024-09-21T00:30:59.877 INFO:teuthology.orchestra.run.smithi112.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.65376.log.gz 2024-09-21T00:30:59.877 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.51587.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51156.log 2024-09-21T00:30:59.877 INFO:teuthology.orchestra.run.smithi112.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51587.log.gz 2024-09-21T00:30:59.877 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.64509.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49385.log 2024-09-21T00:30:59.877 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.51156.log: gzip 87.4% -- replaced with /var/log/ceph/ceph-client.admin.64509.log.gz 2024-09-21T00:30:59.877 INFO:teuthology.orchestra.run.smithi112.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.53797.log 2024-09-21T00:30:59.877 INFO:teuthology.orchestra.run.smithi112.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.51156.log.gz 2024-09-21T00:30:59.877 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.49385.log: gzip -5 --verbose -- 87.3% -- replaced with /var/log/ceph/ceph-client.admin.49385.log.gz 2024-09-21T00:30:59.877 INFO:teuthology.orchestra.run.smithi112.stderr: /var/log/ceph/ceph-client.admin.44816.log 2024-09-21T00:30:59.877 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.53797.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55277.log 2024-09-21T00:30:59.877 INFO:teuthology.orchestra.run.smithi112.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.53797.log.gz 2024-09-21T00:30:59.877 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.44816.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59807.log 2024-09-21T00:30:59.877 INFO:teuthology.orchestra.run.smithi112.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44816.log.gz 2024-09-21T00:30:59.877 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.55277.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69496.log 2024-09-21T00:30:59.878 INFO:teuthology.orchestra.run.smithi112.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.55277.log.gz 2024-09-21T00:30:59.878 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.59807.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48992.log 2024-09-21T00:30:59.878 INFO:teuthology.orchestra.run.smithi112.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.59807.log.gz 2024-09-21T00:30:59.878 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.69496.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69405.log 2024-09-21T00:30:59.878 INFO:teuthology.orchestra.run.smithi112.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.69496.log.gz 2024-09-21T00:30:59.878 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.48992.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.52836.log 2024-09-21T00:30:59.878 INFO:teuthology.orchestra.run.smithi112.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.48992.log.gz 2024-09-21T00:30:59.878 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.69405.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51492.log 2024-09-21T00:30:59.878 INFO:teuthology.orchestra.run.smithi112.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.69405.log.gz 2024-09-21T00:30:59.878 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.0.52836.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51877.log 2024-09-21T00:30:59.878 INFO:teuthology.orchestra.run.smithi112.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.52836.log.gz 2024-09-21T00:30:59.878 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.51492.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66286.log 2024-09-21T00:30:59.878 INFO:teuthology.orchestra.run.smithi112.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.51492.log.gz 2024-09-21T00:30:59.879 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.51877.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57803.log 2024-09-21T00:30:59.879 INFO:teuthology.orchestra.run.smithi112.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.51877.log.gz 2024-09-21T00:30:59.879 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.66286.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.55797.log 2024-09-21T00:30:59.879 INFO:teuthology.orchestra.run.smithi112.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.66286.log.gz 2024-09-21T00:30:59.879 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.57803.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57212.log 2024-09-21T00:30:59.879 INFO:teuthology.orchestra.run.smithi112.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.57803.log.gz 2024-09-21T00:30:59.879 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.0.55797.log: gzip 0.0% -5 --verbose -- replaced with /var/log/ceph/ceph-client.0.55797.log.gz -- 2024-09-21T00:30:59.879 INFO:teuthology.orchestra.run.smithi112.stderr: /var/log/ceph/ceph-client.admin.48408.log 2024-09-21T00:30:59.879 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.57212.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60064.log 2024-09-21T00:30:59.879 INFO:teuthology.orchestra.run.smithi112.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.57212.log.gz 2024-09-21T00:30:59.879 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.48408.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42760.log 2024-09-21T00:30:59.879 INFO:teuthology.orchestra.run.smithi112.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48408.log.gz 2024-09-21T00:30:59.880 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.60064.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67784.log 2024-09-21T00:30:59.880 INFO:teuthology.orchestra.run.smithi112.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.60064.log.gz 2024-09-21T00:30:59.880 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.42760.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60148.log 2024-09-21T00:30:59.880 INFO:teuthology.orchestra.run.smithi112.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42760.log.gz 2024-09-21T00:30:59.880 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.67784.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61076.log 2024-09-21T00:30:59.880 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.60148.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.67784.log.gz 2024-09-21T00:30:59.880 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.66873.log 2024-09-21T00:30:59.880 INFO:teuthology.orchestra.run.smithi112.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.60148.log.gz 2024-09-21T00:30:59.880 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.61076.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54701.log 2024-09-21T00:30:59.880 INFO:teuthology.orchestra.run.smithi112.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.61076.log.gz 2024-09-21T00:30:59.880 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.0.66873.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.66873.log.gz 2024-09-21T00:30:59.880 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.46645.log 2024-09-21T00:30:59.880 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61808.log 2024-09-21T00:30:59.881 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.54701.log: /var/log/ceph/ceph-client.0.46645.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.54701.log.gz 2024-09-21T00:30:59.881 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62197.log 2024-09-21T00:30:59.881 INFO:teuthology.orchestra.run.smithi112.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.46645.log.gz 2024-09-21T00:30:59.881 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.61808.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67670.log 2024-09-21T00:30:59.881 INFO:teuthology.orchestra.run.smithi112.stderr: 87.6%/var/log/ceph/ceph-client.admin.62197.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43616.log 2024-09-21T00:30:59.881 INFO:teuthology.orchestra.run.smithi112.stderr: -- replaced with /var/log/ceph/ceph-client.admin.61808.log.gz 2024-09-21T00:30:59.881 INFO:teuthology.orchestra.run.smithi112.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.62197.log.gz 2024-09-21T00:30:59.881 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.67670.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45334.log 2024-09-21T00:30:59.881 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.43616.log: 91.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47410.log 2024-09-21T00:30:59.881 INFO:teuthology.orchestra.run.smithi112.stderr: -- replaced with /var/log/ceph/ceph-client.admin.67670.log.gz 2024-09-21T00:30:59.881 INFO:teuthology.orchestra.run.smithi112.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43616.log.gz 2024-09-21T00:30:59.881 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.45334.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-09-21T00:30:59.881 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.47410.log: gzip 87.1% -- replaced with /var/log/ceph/ceph-client.admin.45334.log.gz 2024-09-21T00:30:59.882 INFO:teuthology.orchestra.run.smithi112.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.44690.log 2024-09-21T00:30:59.882 INFO:teuthology.orchestra.run.smithi112.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47410.log.gz 2024-09-21T00:30:59.882 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49301.log 2024-09-21T00:30:59.882 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.44690.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45670.log 2024-09-21T00:30:59.882 INFO:teuthology.orchestra.run.smithi112.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44690.log.gz 2024-09-21T00:30:59.882 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.49301.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48780.log 87.3% -- replaced with /var/log/ceph/ceph-client.admin.49301.log.gz 2024-09-21T00:30:59.882 INFO:teuthology.orchestra.run.smithi112.stderr: 2024-09-21T00:30:59.882 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.45670.log: gzip -5 87.2% -- replaced with /var/log/ceph/ceph-client.admin.45670.log.gz 2024-09-21T00:30:59.882 INFO:teuthology.orchestra.run.smithi112.stderr: --verbose -- /var/log/ceph/ceph-client.admin.46783.log 2024-09-21T00:30:59.882 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.48780.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45992.log 2024-09-21T00:30:59.882 INFO:teuthology.orchestra.run.smithi112.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48780.log.gz 2024-09-21T00:30:59.882 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.46783.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48948.log 2024-09-21T00:30:59.882 INFO:teuthology.orchestra.run.smithi112.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46783.log.gz 2024-09-21T00:30:59.882 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.45992.log: gzip 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45992.log.gz 2024-09-21T00:30:59.883 INFO:teuthology.orchestra.run.smithi112.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.51366.log 2024-09-21T00:30:59.883 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.48948.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52312.log 2024-09-21T00:30:59.883 INFO:teuthology.orchestra.run.smithi112.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48948.log.gz/var/log/ceph/ceph-client.admin.51366.log: 2024-09-21T00:30:59.883 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57418.log 86.8% -- replaced with /var/log/ceph/ceph-client.admin.51366.log.gz 2024-09-21T00:30:59.883 INFO:teuthology.orchestra.run.smithi112.stderr: 2024-09-21T00:30:59.883 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.52312.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36815.log 2024-09-21T00:30:59.883 INFO:teuthology.orchestra.run.smithi112.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.52312.log.gz 2024-09-21T00:30:59.883 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.57418.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45570.log 2024-09-21T00:30:59.883 INFO:teuthology.orchestra.run.smithi112.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.57418.log.gz 2024-09-21T00:30:59.883 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.36815.log: -- /var/log/ceph/ceph-client.admin.48072.log 2024-09-21T00:30:59.883 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.45570.log: 86.0% -- replaced with /var/log/ceph/ceph-client.admin.36815.log.gz 2024-09-21T00:30:59.883 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64257.log 2024-09-21T00:30:59.883 INFO:teuthology.orchestra.run.smithi112.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45570.log.gz 2024-09-21T00:30:59.883 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.48072.log: -- /var/log/ceph/ceph-client.admin.51011.log 2024-09-21T00:30:59.884 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.64257.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48072.log.gz 2024-09-21T00:30:59.884 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53934.log 2024-09-21T00:30:59.884 INFO:teuthology.orchestra.run.smithi112.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.64257.log.gz 2024-09-21T00:30:59.884 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50843.log 2024-09-21T00:30:59.884 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.51011.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65166.log 2024-09-21T00:30:59.884 INFO:teuthology.orchestra.run.smithi112.stderr: 87.7%/var/log/ceph/ceph-client.admin.53934.log: -- replaced with /var/log/ceph/ceph-client.admin.51011.log.gz 2024-09-21T00:30:59.884 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.50843.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.53934.log.gz 2024-09-21T00:30:59.884 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose 87.7% -- replaced with /var/log/ceph/ceph-client.admin.50843.log.gz 2024-09-21T00:30:59.884 INFO:teuthology.orchestra.run.smithi112.stderr: -- /var/log/ceph/ceph-mgr.y.log 2024-09-21T00:30:59.884 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.65166.log: gzip -5 --verbose 87.4% -- /var/log/ceph/ceph-client.admin.49655.log 2024-09-21T00:30:59.884 INFO:teuthology.orchestra.run.smithi112.stderr: -- replaced with /var/log/ceph/ceph-client.admin.65166.log.gz 2024-09-21T00:30:59.884 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59040.log 2024-09-21T00:30:59.885 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.49655.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65460.log 2024-09-21T00:30:59.885 INFO:teuthology.orchestra.run.smithi112.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49655.log.gz 2024-09-21T00:30:59.885 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.59040.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60488.log 2024-09-21T00:30:59.885 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.65460.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.59040.log.gz 2024-09-21T00:30:59.885 INFO:teuthology.orchestra.run.smithi112.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.65460.log.gz 2024-09-21T00:30:59.885 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45322.log 2024-09-21T00:30:59.885 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69446.log 2024-09-21T00:30:59.885 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.45322.log: /var/log/ceph/ceph-client.admin.60488.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.60488.log.gz 2024-09-21T00:30:59.885 INFO:teuthology.orchestra.run.smithi112.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.45322.log.gz 2024-09-21T00:30:59.885 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62940.log 2024-09-21T00:30:59.885 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.69446.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45669.log 2024-09-21T00:30:59.885 INFO:teuthology.orchestra.run.smithi112.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.69446.log.gz 2024-09-21T00:30:59.885 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5/var/log/ceph/ceph-client.admin.62940.log: --verbose -- /var/log/ceph/ceph-client.admin.49427.log 2024-09-21T00:30:59.885 INFO:teuthology.orchestra.run.smithi112.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.62940.log.gz 2024-09-21T00:30:59.886 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.45669.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62376.log 2024-09-21T00:30:59.886 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.49427.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.45669.log.gz 2024-09-21T00:30:59.886 INFO:teuthology.orchestra.run.smithi112.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.49427.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52918.log 2024-09-21T00:30:59.886 INFO:teuthology.orchestra.run.smithi112.stderr: 2024-09-21T00:30:59.886 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose 93.6% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-09-21T00:30:59.886 INFO:teuthology.orchestra.run.smithi112.stderr: -- /var/log/ceph/ceph-client.admin.44648.log 2024-09-21T00:30:59.886 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.52918.log: /var/log/ceph/ceph-client.admin.62376.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.54997.log 2024-09-21T00:30:59.886 INFO:teuthology.orchestra.run.smithi112.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.62376.log.gz 2024-09-21T00:30:59.886 INFO:teuthology.orchestra.run.smithi112.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.52918.log.gz 2024-09-21T00:30:59.886 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.44648.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51198.log 2024-09-21T00:30:59.886 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.0.54997.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47911.log 87.4% 2024-09-21T00:30:59.886 INFO:teuthology.orchestra.run.smithi112.stderr: -- replaced with /var/log/ceph/ceph-client.admin.44648.log.gz 2024-09-21T00:30:59.886 INFO:teuthology.orchestra.run.smithi112.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.54997.log.gz 2024-09-21T00:30:59.886 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.51198.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52144.log 2024-09-21T00:30:59.887 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.47911.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49343.log 2024-09-21T00:30:59.887 INFO:teuthology.orchestra.run.smithi112.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51198.log.gz 2024-09-21T00:30:59.887 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.52144.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.47911.log.gz 2024-09-21T00:30:59.887 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-21T00:30:59.887 INFO:teuthology.orchestra.run.smithi112.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52144.log.gz 2024-09-21T00:30:59.887 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54617.log 2024-09-21T00:30:59.887 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.49343.log: /var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63108.log 2024-09-21T00:30:59.887 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.54617.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.49343.log.gz 2024-09-21T00:30:59.887 INFO:teuthology.orchestra.run.smithi112.stderr: 86.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62730.log 2024-09-21T00:30:59.887 INFO:teuthology.orchestra.run.smithi112.stderr: -- replaced with /var/log/ceph/ceph-client.admin.54617.log.gz 2024-09-21T00:30:59.887 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.63108.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.63108.log.gz 2024-09-21T00:30:59.887 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 86.0% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-21T00:30:59.887 INFO:teuthology.orchestra.run.smithi112.stderr: --verbose -- /var/log/ceph/ceph-client.admin.66951.log 2024-09-21T00:30:59.887 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.62730.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61640.log 87.5% 2024-09-21T00:30:59.888 INFO:teuthology.orchestra.run.smithi112.stderr: -- replaced with /var/log/ceph/ceph-client.admin.62730.log.gz 2024-09-21T00:30:59.888 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.66951.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.60952.log 2024-09-21T00:30:59.888 INFO:teuthology.orchestra.run.smithi112.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.66951.log.gz 2024-09-21T00:30:59.888 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.61640.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61472.log 2024-09-21T00:30:59.888 INFO:teuthology.orchestra.run.smithi112.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.61640.log.gz 2024-09-21T00:30:59.888 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.0.60952.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.60952.log.gz 2024-09-21T00:30:59.888 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61430.log 2024-09-21T00:30:59.888 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5/var/log/ceph/ceph-client.admin.61472.log: --verbose -- /var/log/ceph/ceph-client.admin.47224.log 2024-09-21T00:30:59.888 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.61430.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.61472.log.gz 2024-09-21T00:30:59.888 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62460.log 2024-09-21T00:30:59.888 INFO:teuthology.orchestra.run.smithi112.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.61430.log.gz 2024-09-21T00:30:59.888 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.47224.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50149.log 2024-09-21T00:30:59.888 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.62460.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46034.log 2024-09-21T00:30:59.888 INFO:teuthology.orchestra.run.smithi112.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47224.log.gz 2024-09-21T00:30:59.888 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.50149.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.62460.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66081.log 2024-09-21T00:30:59.888 INFO:teuthology.orchestra.run.smithi112.stderr: 2024-09-21T00:30:59.889 INFO:teuthology.orchestra.run.smithi112.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50149.log.gz 2024-09-21T00:30:59.889 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67205.log 2024-09-21T00:30:59.889 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.46034.log: /var/log/ceph/ceph-client.admin.66081.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.65924.log 2024-09-21T00:30:59.889 INFO:teuthology.orchestra.run.smithi112.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46034.log.gz 2024-09-21T00:30:59.889 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.67205.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.66081.log.gz 2024-09-21T00:30:59.889 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 87.6% -- replaced with /var/log/ceph/ceph-client.admin.67205.log.gz 2024-09-21T00:30:59.889 INFO:teuthology.orchestra.run.smithi112.stderr: --verbose -- /var/log/ceph/ceph-client.admin.52102.log 2024-09-21T00:30:59.889 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.0.65924.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.65924.log.gz 2024-09-21T00:30:59.889 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51282.log 2024-09-21T00:30:59.889 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5/var/log/ceph/ceph-client.admin.52102.log: --verbose -- /var/log/ceph/ceph-client.admin.65250.log 2024-09-21T00:30:59.889 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.51282.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.52102.log.gz 2024-09-21T00:30:59.889 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45866.log 2024-09-21T00:30:59.889 INFO:teuthology.orchestra.run.smithi112.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51282.log.gz 2024-09-21T00:30:59.889 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.65250.log: -- /var/log/ceph/ceph-client.admin.48611.log 2024-09-21T00:30:59.890 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.45866.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.65250.log.gz 2024-09-21T00:30:59.890 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67109.log 2024-09-21T00:30:59.890 INFO:teuthology.orchestra.run.smithi112.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.45866.log.gz 2024-09-21T00:30:59.890 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44995.log 2024-09-21T00:30:59.890 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.48611.log: /var/log/ceph/ceph-client.admin.67109.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36243.log 2024-09-21T00:30:59.890 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.44995.log: 87.8% 87.6% -- replaced with /var/log/ceph/ceph-client.admin.67109.log.gz -- replaced with /var/log/ceph/ceph-client.admin.48611.log.gz 2024-09-21T00:30:59.890 INFO:teuthology.orchestra.run.smithi112.stderr: 2024-09-21T00:30:59.890 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44995.log.gz 2024-09-21T00:30:59.890 INFO:teuthology.orchestra.run.smithi112.stderr: --verbose -- /var/log/ceph/ceph-client.admin.68587.log 2024-09-21T00:30:59.890 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.36243.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36243.log.gz 2024-09-21T00:30:59.890 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.66869.log 2024-09-21T00:30:59.890 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5/var/log/ceph/ceph-client.admin.68587.log: --verbose -- /var/log/ceph/ceph-client.admin.61514.log 2024-09-21T00:30:59.890 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.0.66869.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.68587.log.gz 2024-09-21T00:30:59.890 INFO:teuthology.orchestra.run.smithi112.stderr: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42856.log 2024-09-21T00:30:59.891 INFO:teuthology.orchestra.run.smithi112.stderr: -- replaced with /var/log/ceph/ceph-client.0.66869.log.gz 2024-09-21T00:30:59.891 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58868.log 2024-09-21T00:30:59.891 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.61514.log: /var/log/ceph/ceph-client.admin.42856.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60106.log 2024-09-21T00:30:59.891 INFO:teuthology.orchestra.run.smithi112.stderr: 87.6% 87.6% -- replaced with /var/log/ceph/ceph-client.admin.61514.log.gz 2024-09-21T00:30:59.891 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.58868.log: -- replaced with /var/log/ceph/ceph-client.admin.42856.log.gz 2024-09-21T00:30:59.891 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 87.8% -- replaced with /var/log/ceph/ceph-client.admin.58868.log.gz 2024-09-21T00:30:59.891 INFO:teuthology.orchestra.run.smithi112.stderr: --verbose -- /var/log/ceph/ceph-client.admin.51921.log 2024-09-21T00:30:59.891 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.60106.log: gzip -5 --verbose 87.5% -- replaced with /var/log/ceph/ceph-client.admin.60106.log.gz 2024-09-21T00:30:59.891 INFO:teuthology.orchestra.run.smithi112.stderr: -- /var/log/ceph/ceph-client.admin.36814.log 2024-09-21T00:30:59.891 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.51921.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46076.log 2024-09-21T00:30:59.891 INFO:teuthology.orchestra.run.smithi112.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.51921.log.gz 2024-09-21T00:30:59.891 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.36814.log: gzip 86.4% -- replaced with /var/log/ceph/ceph-client.admin.36814.log.gz 2024-09-21T00:30:59.891 INFO:teuthology.orchestra.run.smithi112.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.53274.log 2024-09-21T00:30:59.892 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.46076.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63192.log 2024-09-21T00:30:59.892 INFO:teuthology.orchestra.run.smithi112.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46076.log.gz/var/log/ceph/ceph-client.admin.53274.log: 2024-09-21T00:30:59.892 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63986.log 2024-09-21T00:30:59.892 INFO:teuthology.orchestra.run.smithi112.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.53274.log.gz 2024-09-21T00:30:59.892 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49259.log 2024-09-21T00:30:59.892 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.63192.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48906.log 2024-09-21T00:30:59.892 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.63986.log: /var/log/ceph/ceph-client.admin.49259.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.63192.log.gz 87.7% -- replaced with /var/log/ceph/ceph-client.admin.63986.log.gz 2024-09-21T00:30:59.892 INFO:teuthology.orchestra.run.smithi112.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49259.log.gz 2024-09-21T00:30:59.892 INFO:teuthology.orchestra.run.smithi112.stderr: 2024-09-21T00:30:59.892 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62772.log 2024-09-21T00:30:59.892 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.48906.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49697.log 2024-09-21T00:30:59.892 INFO:teuthology.orchestra.run.smithi112.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48906.log.gz/var/log/ceph/ceph-client.admin.62772.log: 2024-09-21T00:30:59.892 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57002.log 2024-09-21T00:30:59.893 INFO:teuthology.orchestra.run.smithi112.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.62772.log.gz 2024-09-21T00:30:59.893 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.49697.log: /var/log/ceph/ceph-client.admin.36517.log 2024-09-21T00:30:59.893 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.57002.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63944.log 2024-09-21T00:30:59.893 INFO:teuthology.orchestra.run.smithi112.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.57002.log.gz 90.9% -- replaced with /var/log/ceph/ceph-client.admin.49697.log.gz 2024-09-21T00:30:59.893 INFO:teuthology.orchestra.run.smithi112.stderr: 2024-09-21T00:30:59.893 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.36517.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64467.log 2024-09-21T00:30:59.893 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.63944.log: 86.2% -- replaced with /var/log/ceph/ceph-client.admin.36517.log.gz 2024-09-21T00:30:59.893 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66681.log 2024-09-21T00:30:59.893 INFO:teuthology.orchestra.run.smithi112.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.63944.log.gz 2024-09-21T00:30:59.893 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64768.log 2024-09-21T00:30:59.893 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.64467.log: /var/log/ceph/ceph-client.admin.66681.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36819.log 2024-09-21T00:30:59.893 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.64768.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.66681.log.gz 2024-09-21T00:30:59.893 INFO:teuthology.orchestra.run.smithi112.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.64467.log.gz 2024-09-21T00:30:59.894 INFO:teuthology.orchestra.run.smithi112.stderr:gzip 87.7% -- replaced with /var/log/ceph/ceph-client.admin.64768.log.gz 2024-09-21T00:30:59.894 INFO:teuthology.orchestra.run.smithi112.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.52582.log 2024-09-21T00:30:59.894 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.36819.log: gzip -5 86.2% -- replaced with /var/log/ceph/ceph-client.admin.36819.log.gz 2024-09-21T00:30:59.894 INFO:teuthology.orchestra.run.smithi112.stderr: --verbose -- /var/log/ceph/ceph-client.admin.54014.log 2024-09-21T00:30:59.894 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.52582.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68713.log 2024-09-21T00:30:59.894 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.54014.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69049.log 2024-09-21T00:30:59.894 INFO:teuthology.orchestra.run.smithi112.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.52582.log.gz 2024-09-21T00:30:59.894 INFO:teuthology.orchestra.run.smithi112.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.54014.log.gz 2024-09-21T00:30:59.894 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.68713.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48030.log 2024-09-21T00:30:59.894 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.69049.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50075.log 2024-09-21T00:30:59.894 INFO:teuthology.orchestra.run.smithi112.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.68713.log.gz 2024-09-21T00:30:59.894 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.48030.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.69049.log.gz 2024-09-21T00:30:59.895 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51790.log 2024-09-21T00:30:59.895 INFO:teuthology.orchestra.run.smithi112.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48030.log.gz 2024-09-21T00:30:59.895 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69133.log 2024-09-21T00:30:59.895 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.50075.log: /var/log/ceph/ceph-client.admin.51790.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36186.log 2024-09-21T00:30:59.895 INFO:teuthology.orchestra.run.smithi112.stderr: 87.4%/var/log/ceph/ceph-client.admin.69133.log: -- replaced with /var/log/ceph/ceph-client.admin.50075.log.gz 2024-09-21T00:30:59.895 INFO:teuthology.orchestra.run.smithi112.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51790.log.gz 2024-09-21T00:30:59.895 INFO:teuthology.orchestra.run.smithi112.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.69133.log.gz 2024-09-21T00:30:59.895 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44396.log 2024-09-21T00:30:59.895 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43382.log/var/log/ceph/ceph-client.admin.36186.log: 2024-09-21T00:30:59.895 INFO:teuthology.orchestra.run.smithi112.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36186.log.gz/var/log/ceph/ceph-client.admin.44396.log: 2024-09-21T00:30:59.895 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose 87.3% -- replaced with /var/log/ceph/ceph-client.admin.44396.log.gz 2024-09-21T00:30:59.895 INFO:teuthology.orchestra.run.smithi112.stderr: -- /var/log/ceph/ceph-client.admin.47326.log 2024-09-21T00:30:59.896 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.43382.log: gzip -5 --verbose 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43382.log.gz 2024-09-21T00:30:59.896 INFO:teuthology.orchestra.run.smithi112.stderr: -- /var/log/ceph/ceph-client.admin.53671.log 2024-09-21T00:30:59.896 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.47326.log: gzip -5 --verbose -- 87.7% /var/log/ceph/ceph-client.admin.37169.log 2024-09-21T00:30:59.896 INFO:teuthology.orchestra.run.smithi112.stderr: -- replaced with /var/log/ceph/ceph-client.admin.47326.log.gz 2024-09-21T00:30:59.896 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.53671.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54179.log 2024-09-21T00:30:59.896 INFO:teuthology.orchestra.run.smithi112.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53671.log.gz 2024-09-21T00:30:59.896 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48366.log 2024-09-21T00:30:59.896 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.37169.log: /var/log/ceph/ceph-client.admin.54179.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58406.log 2024-09-21T00:30:59.896 INFO:teuthology.orchestra.run.smithi112.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.37169.log.gz 2024-09-21T00:30:59.896 INFO:teuthology.orchestra.run.smithi112.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.54179.log.gz/var/log/ceph/ceph-client.admin.48366.log: 2024-09-21T00:30:59.896 INFO:teuthology.orchestra.run.smithi112.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.48366.log.gz 2024-09-21T00:30:59.896 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53232.log 2024-09-21T00:30:59.896 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.58406.log: gzip -5 --verbose -- 87.6% -- replaced with /var/log/ceph/ceph-client.admin.58406.log.gz 2024-09-21T00:30:59.896 INFO:teuthology.orchestra.run.smithi112.stderr: /var/log/ceph/ceph-client.admin.52228.log 2024-09-21T00:30:59.896 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.53232.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45330.log 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53232.log.gz 2024-09-21T00:30:59.897 INFO:teuthology.orchestra.run.smithi112.stderr: 2024-09-21T00:30:59.897 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.52228.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65418.log 2024-09-21T00:30:59.897 INFO:teuthology.orchestra.run.smithi112.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.52228.log.gz 2024-09-21T00:30:59.897 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.45330.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59350.log 2024-09-21T00:30:59.897 INFO:teuthology.orchestra.run.smithi112.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.45330.log.gz 2024-09-21T00:30:59.897 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.65418.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68881.log 2024-09-21T00:30:59.897 INFO:teuthology.orchestra.run.smithi112.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.65418.log.gz 2024-09-21T00:30:59.897 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.59350.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61034.log 87.3% 2024-09-21T00:30:59.897 INFO:teuthology.orchestra.run.smithi112.stderr: -- replaced with /var/log/ceph/ceph-client.admin.59350.log.gz 2024-09-21T00:30:59.897 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.68881.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58057.log 87.5% 2024-09-21T00:30:59.897 INFO:teuthology.orchestra.run.smithi112.stderr: -- replaced with /var/log/ceph/ceph-client.admin.68881.log.gz 2024-09-21T00:30:59.897 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.61034.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58364.log 2024-09-21T00:30:59.897 INFO:teuthology.orchestra.run.smithi112.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.61034.log.gz 2024-09-21T00:30:59.897 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.58057.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62418.log 2024-09-21T00:30:59.898 INFO:teuthology.orchestra.run.smithi112.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.58057.log.gz 2024-09-21T00:30:59.898 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.58364.log: /var/log/ceph/ceph-client.admin.58532.log 2024-09-21T00:30:59.898 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.62418.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.58364.log.gz 2024-09-21T00:30:59.898 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59597.log 2024-09-21T00:30:59.898 INFO:teuthology.orchestra.run.smithi112.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.62418.log.gz 2024-09-21T00:30:59.898 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43550.log 2024-09-21T00:30:59.898 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.58532.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50033.log 2024-09-21T00:30:59.898 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.59597.log: /var/log/ceph/ceph-client.admin.43550.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.58532.log.gz 2024-09-21T00:30:59.898 INFO:teuthology.orchestra.run.smithi112.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.59597.log.gz 2024-09-21T00:30:59.898 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36327.log 2024-09-21T00:30:59.898 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.50033.log: 91.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.2.47087.log 2024-09-21T00:30:59.898 INFO:teuthology.orchestra.run.smithi112.stderr: -- replaced with /var/log/ceph/ceph-client.admin.43550.log.gz 2024-09-21T00:30:59.898 INFO:teuthology.orchestra.run.smithi112.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.50033.log.gz 2024-09-21T00:30:59.898 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.36327.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68503.log 2024-09-21T00:30:59.899 INFO:teuthology.orchestra.run.smithi112.stderr: 75.5% -- replaced with /var/log/ceph/ceph-client.admin.36327.log.gz 2024-09-21T00:30:59.899 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.2.47087.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48738.log 2024-09-21T00:30:59.899 INFO:teuthology.orchestra.run.smithi112.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.2.47087.log.gz 2024-09-21T00:30:59.899 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.63446.log 2024-09-21T00:30:59.899 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.68503.log: /var/log/ceph/ceph-client.admin.48738.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68200.log 2024-09-21T00:30:59.899 INFO:teuthology.orchestra.run.smithi112.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48738.log.gz 2024-09-21T00:30:59.899 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.0.63446.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.63446.log.gz 2024-09-21T00:30:59.899 INFO:teuthology.orchestra.run.smithi112.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.68503.log.gz 2024-09-21T00:30:59.899 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44997.log 2024-09-21T00:30:59.899 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.68200.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58238.log 2024-09-21T00:30:59.899 INFO:teuthology.orchestra.run.smithi112.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.68200.log.gz 2024-09-21T00:30:59.899 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.44997.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.52840.log 2024-09-21T00:30:59.899 INFO:teuthology.orchestra.run.smithi112.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44997.log.gz 2024-09-21T00:30:59.899 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.58238.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68923.log 2024-09-21T00:30:59.899 INFO:teuthology.orchestra.run.smithi112.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.58238.log.gz 2024-09-21T00:30:59.900 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60362.log 2024-09-21T00:30:59.900 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.0.52840.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.52840.log.gz 2024-09-21T00:30:59.900 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60236.log 2024-09-21T00:30:59.900 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.68923.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58150.log 2024-09-21T00:30:59.900 INFO:teuthology.orchestra.run.smithi112.stderr: 87.3%/var/log/ceph/ceph-client.admin.60362.log: -- replaced with /var/log/ceph/ceph-client.admin.68923.log.gz 2024-09-21T00:30:59.900 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.60236.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.60362.log.gz 2024-09-21T00:30:59.900 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 87.8% -- replaced with /var/log/ceph/ceph-client.admin.60236.log.gz 2024-09-21T00:30:59.900 INFO:teuthology.orchestra.run.smithi112.stderr: --verbose -- /var/log/ceph/ceph-client.admin.61976.log 2024-09-21T00:30:59.900 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.58150.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.58150.log.gz 2024-09-21T00:30:59.900 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54491.log 2024-09-21T00:30:59.900 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.61976.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63528.log 2024-09-21T00:30:59.900 INFO:teuthology.orchestra.run.smithi112.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.61976.log.gz 2024-09-21T00:30:59.900 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.54491.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44564.log 2024-09-21T00:30:59.900 INFO:teuthology.orchestra.run.smithi112.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.54491.log.gz 2024-09-21T00:30:59.900 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.63528.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48822.log 2024-09-21T00:30:59.901 INFO:teuthology.orchestra.run.smithi112.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.63528.log.gz 2024-09-21T00:30:59.901 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.44564.log: gzip -5 --verbose 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44564.log.gz 2024-09-21T00:30:59.901 INFO:teuthology.orchestra.run.smithi112.stderr: -- /var/log/ceph/ceph-client.admin.61311.log 2024-09-21T00:30:59.901 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.48822.log: gzip -5 --verbose -- 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48822.log.gz 2024-09-21T00:30:59.901 INFO:teuthology.orchestra.run.smithi112.stderr: /var/log/ceph/ceph-client.admin.65040.log 2024-09-21T00:30:59.901 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.61311.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64952.log 2024-09-21T00:30:59.901 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.65040.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44354.log 2024-09-21T00:30:59.901 INFO:teuthology.orchestra.run.smithi112.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.61311.log.gz 2024-09-21T00:30:59.901 INFO:teuthology.orchestra.run.smithi112.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.65040.log.gz 2024-09-21T00:30:59.901 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57628.log 2024-09-21T00:30:59.901 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.64952.log: /var/log/ceph/ceph-client.admin.44354.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47743.log 2024-09-21T00:30:59.901 INFO:teuthology.orchestra.run.smithi112.stderr: 87.3%/var/log/ceph/ceph-client.admin.57628.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.44354.log.gz 2024-09-21T00:30:59.901 INFO:teuthology.orchestra.run.smithi112.stderr: -- replaced with /var/log/ceph/ceph-client.admin.64952.log.gz 2024-09-21T00:30:59.901 INFO:teuthology.orchestra.run.smithi112.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.57628.log.gz 2024-09-21T00:30:59.901 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57884.log 2024-09-21T00:30:59.902 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.47743.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.58786.log 2024-09-21T00:30:59.902 INFO:teuthology.orchestra.run.smithi112.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47743.log.gz/var/log/ceph/ceph-client.admin.57884.log: 2024-09-21T00:30:59.902 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53442.log 2024-09-21T00:30:59.902 INFO:teuthology.orchestra.run.smithi112.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.57884.log.gz 2024-09-21T00:30:59.902 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.0.58786.log: -- /var/log/ceph/ceph-client.admin.55079.log 2024-09-21T00:30:59.902 INFO:teuthology.orchestra.run.smithi112.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.58786.log.gz 2024-09-21T00:30:59.902 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.53442.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57086.log 2024-09-21T00:30:59.902 INFO:teuthology.orchestra.run.smithi112.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53442.log.gz 2024-09-21T00:30:59.902 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65502.log 2024-09-21T00:30:59.902 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.55079.log: /var/log/ceph/ceph-client.admin.57086.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42998.log 2024-09-21T00:30:59.902 INFO:teuthology.orchestra.run.smithi112.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.55079.log.gz 2024-09-21T00:30:59.902 INFO:teuthology.orchestra.run.smithi112.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.57086.log.gz 2024-09-21T00:30:59.902 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.65502.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62898.log 2024-09-21T00:30:59.902 INFO:teuthology.orchestra.run.smithi112.stderr: 87.2%/var/log/ceph/ceph-client.admin.42998.log: gzip -- replaced with /var/log/ceph/ceph-client.admin.65502.log.gz -5 --verbose -- /var/log/ceph/ceph-client.admin.64383.log 2024-09-21T00:30:59.902 INFO:teuthology.orchestra.run.smithi112.stderr: 2024-09-21T00:30:59.903 INFO:teuthology.orchestra.run.smithi112.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42998.log.gz 2024-09-21T00:30:59.903 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36772.log 2024-09-21T00:30:59.903 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.62898.log: /var/log/ceph/ceph-client.admin.64383.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54319.log 2024-09-21T00:30:59.903 INFO:teuthology.orchestra.run.smithi112.stderr: 86.9% 87.5% -- replaced with /var/log/ceph/ceph-client.admin.62898.log.gz 2024-09-21T00:30:59.903 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.36772.log: -- replaced with /var/log/ceph/ceph-client.admin.64383.log.gz 2024-09-21T00:30:59.903 INFO:teuthology.orchestra.run.smithi112.stderr: 85.7% -- replaced with /var/log/ceph/ceph-client.admin.36772.log.gz 2024-09-21T00:30:59.903 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59224.log 2024-09-21T00:30:59.903 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.54319.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58598.log 87.8% -- replaced with /var/log/ceph/ceph-client.admin.54319.log.gz 2024-09-21T00:30:59.903 INFO:teuthology.orchestra.run.smithi112.stderr: 2024-09-21T00:30:59.903 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.59224.log: gzip -5 --verbose -- 87.6% -- replaced with /var/log/ceph/ceph-client.admin.59224.log.gz 2024-09-21T00:30:59.903 INFO:teuthology.orchestra.run.smithi112.stderr: /var/log/ceph/ceph-client.admin.64551.log 2024-09-21T00:30:59.903 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.58598.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57968.log 87.8% -- replaced with /var/log/ceph/ceph-client.admin.58598.log.gz 2024-09-21T00:30:59.903 INFO:teuthology.orchestra.run.smithi112.stderr: 2024-09-21T00:30:59.903 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.64551.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65628.log 87.7% -- replaced with /var/log/ceph/ceph-client.admin.64551.log.gz 2024-09-21T00:30:59.904 INFO:teuthology.orchestra.run.smithi112.stderr: 2024-09-21T00:30:59.904 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.57968.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55214.log 2024-09-21T00:30:59.904 INFO:teuthology.orchestra.run.smithi112.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.57968.log.gz 2024-09-21T00:30:59.904 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.65628.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52540.log 2024-09-21T00:30:59.904 INFO:teuthology.orchestra.run.smithi112.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.65628.log.gz 2024-09-21T00:30:59.904 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56918.log 2024-09-21T00:30:59.904 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.55214.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62814.log 2024-09-21T00:30:59.904 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.52540.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.55214.log.gz 2024-09-21T00:30:59.904 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.56918.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52540.log.gz 2024-09-21T00:30:59.904 INFO:teuthology.orchestra.run.smithi112.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.56918.log.gz 2024-09-21T00:30:59.904 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62102.log 2024-09-21T00:30:59.904 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.62814.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53755.log 2024-09-21T00:30:59.904 INFO:teuthology.orchestra.run.smithi112.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.62814.log.gz 2024-09-21T00:30:59.904 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.62102.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43128.log 2024-09-21T00:30:59.904 INFO:teuthology.orchestra.run.smithi112.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.62102.log.gz 2024-09-21T00:30:59.905 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.53755.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62060.log 2024-09-21T00:30:59.905 INFO:teuthology.orchestra.run.smithi112.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.53755.log.gz 2024-09-21T00:30:59.905 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37370.log/var/log/ceph/ceph-client.admin.43128.log: 2024-09-21T00:30:59.905 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.62060.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.43128.log.gz 2024-09-21T00:30:59.905 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52056.log 2024-09-21T00:30:59.905 INFO:teuthology.orchestra.run.smithi112.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.62060.log.gz 2024-09-21T00:30:59.905 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53629.log 2024-09-21T00:30:59.905 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.37370.log: 86.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68158.log 2024-09-21T00:30:59.905 INFO:teuthology.orchestra.run.smithi112.stderr: -- replaced with /var/log/ceph/ceph-client.admin.37370.log.gz 2024-09-21T00:30:59.905 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.52056.log: /var/log/ceph/ceph-client.admin.53629.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60614.log 2024-09-21T00:30:59.905 INFO:teuthology.orchestra.run.smithi112.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.52056.log.gz 2024-09-21T00:30:59.905 INFO:teuthology.orchestra.run.smithi112.stderr: 87.2%/var/log/ceph/ceph-client.admin.68158.log: -- replaced with /var/log/ceph/ceph-client.admin.53629.log.gz 2024-09-21T00:30:59.905 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59495.log 2024-09-21T00:30:59.905 INFO:teuthology.orchestra.run.smithi112.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.68158.log.gz 2024-09-21T00:30:59.906 INFO:teuthology.orchestra.run.smithi112.stderr:gzip/var/log/ceph/ceph-client.admin.60614.log: -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-09-21T00:30:59.906 INFO:teuthology.orchestra.run.smithi112.stderr: 87.3%/var/log/ceph/ceph-client.admin.59495.log: -- replaced with /var/log/ceph/ceph-client.admin.60614.log.gz 2024-09-21T00:30:59.906 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- 87.3% -- replaced with /var/log/ceph/ceph-client.admin.59495.log.gz 2024-09-21T00:30:59.906 INFO:teuthology.orchestra.run.smithi112.stderr: /var/log/ceph/ceph-client.admin.37129.log 2024-09-21T00:30:59.906 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44438.log 2024-09-21T00:30:59.906 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.37129.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58108.log 2024-09-21T00:30:59.906 INFO:teuthology.orchestra.run.smithi112.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.37129.log.gz 2024-09-21T00:30:59.906 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.44438.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.44438.log.gz 2024-09-21T00:30:59.906 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-09-21T00:30:59.906 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69175.log 2024-09-21T00:30:59.906 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.58108.log: /var/log/ceph/ceph-mds.c.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.58108.log.gz 2024-09-21T00:30:59.907 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66527.log 2024-09-21T00:30:59.907 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.69175.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.69175.log.gz 2024-09-21T00:30:59.907 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59308.log 2024-09-21T00:30:59.907 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.66527.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.66527.log.gz 2024-09-21T00:30:59.907 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53713.log 2024-09-21T00:30:59.907 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.59308.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.59308.log.gz 2024-09-21T00:30:59.907 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61892.log 2024-09-21T00:30:59.907 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.53713.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.53713.log.gz 2024-09-21T00:30:59.907 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45324.log 2024-09-21T00:30:59.907 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.61892.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.61892.log.gz 2024-09-21T00:30:59.907 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67313.log 2024-09-21T00:30:59.907 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.45324.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.45324.log.gz 2024-09-21T00:30:59.907 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63721.log 2024-09-21T00:30:59.907 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.67313.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.67313.log.gz 2024-09-21T00:30:59.907 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57044.log 2024-09-21T00:30:59.908 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.63721.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.63721.log.gz 2024-09-21T00:30:59.908 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49763.log 2024-09-21T00:30:59.908 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.57044.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.57044.log.gz 2024-09-21T00:30:59.908 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62018.log 2024-09-21T00:30:59.908 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.49763.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49763.log.gz 2024-09-21T00:30:59.908 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66434.log 2024-09-21T00:30:59.908 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.62018.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.62018.log.gz 2024-09-21T00:30:59.908 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48864.log 2024-09-21T00:30:59.908 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.66434.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.66434.log.gz 2024-09-21T00:30:59.908 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58448.log 2024-09-21T00:30:59.908 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.48864.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48864.log.gz 2024-09-21T00:30:59.908 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67247.log 2024-09-21T00:30:59.908 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.58448.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.58448.log.gz 2024-09-21T00:30:59.909 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.67247.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.67247.log.gz 2024-09-21T00:30:59.909 INFO:teuthology.orchestra.run.smithi112.stderr: 92.4% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-09-21T00:30:59.909 INFO:teuthology.orchestra.run.smithi187.stderr: 91.2% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-09-21T00:30:59.925 INFO:teuthology.orchestra.run.smithi112.stderr: 92.3% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-09-21T00:31:00.165 INFO:teuthology.orchestra.run.smithi112.stderr: 90.8% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-09-21T00:31:00.701 INFO:teuthology.orchestra.run.smithi187.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-09-21T00:31:00.720 INFO:teuthology.orchestra.run.smithi187.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-09-21T00:31:00.756 INFO:teuthology.orchestra.run.smithi112.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-09-21T00:31:00.768 INFO:teuthology.orchestra.run.smithi187.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-09-21T00:31:00.820 INFO:teuthology.orchestra.run.smithi112.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-09-21T00:31:00.830 INFO:teuthology.orchestra.run.smithi187.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-09-21T00:31:00.832 INFO:teuthology.orchestra.run.smithi187.stderr: 2024-09-21T00:31:00.832 INFO:teuthology.orchestra.run.smithi187.stderr:real 0m1.211s 2024-09-21T00:31:00.832 INFO:teuthology.orchestra.run.smithi187.stderr:user 0m4.899s 2024-09-21T00:31:00.832 INFO:teuthology.orchestra.run.smithi187.stderr:sys 0m0.180s 2024-09-21T00:31:00.833 INFO:teuthology.orchestra.run.smithi112.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-09-21T00:31:00.857 INFO:teuthology.orchestra.run.smithi112.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-09-21T00:31:00.859 INFO:teuthology.orchestra.run.smithi112.stderr: 2024-09-21T00:31:00.859 INFO:teuthology.orchestra.run.smithi112.stderr:real 0m1.261s 2024-09-21T00:31:00.859 INFO:teuthology.orchestra.run.smithi112.stderr:user 0m5.324s 2024-09-21T00:31:00.859 INFO:teuthology.orchestra.run.smithi112.stderr:sys 0m0.483s 2024-09-21T00:31:00.859 INFO:tasks.ceph:Archiving logs... 2024-09-21T00:31:00.859 DEBUG:teuthology.misc:Transferring archived files from smithi112:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913327/remote/smithi112/log 2024-09-21T00:31:00.860 DEBUG:teuthology.orchestra.run.smithi112:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-21T00:31:02.032 DEBUG:teuthology.misc:Transferring archived files from smithi187:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913327/remote/smithi187/log 2024-09-21T00:31:02.033 DEBUG:teuthology.orchestra.run.smithi187:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-21T00:31:02.711 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-21T00:31:02.721 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-21T00:31:02.721 DEBUG:teuthology.orchestra.run.smithi112:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-21T00:31:02.723 DEBUG:teuthology.orchestra.run.smithi187:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-21T00:31:02.811 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-21T00:31:02.820 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-21T00:31:02.820 DEBUG:teuthology.orchestra.run.smithi112:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T00:31:02.822 DEBUG:teuthology.orchestra.run.smithi187:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T00:31:02.837 INFO:teuthology.orchestra.run.smithi112.stderr:bash: line 1: ntpq: command not found 2024-09-21T00:31:02.838 INFO:teuthology.orchestra.run.smithi187.stderr:bash: line 1: ntpq: command not found 2024-09-21T00:31:02.842 INFO:teuthology.orchestra.run.smithi112.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T00:31:02.842 INFO:teuthology.orchestra.run.smithi112.stdout:=============================================================================== 2024-09-21T00:31:02.842 INFO:teuthology.orchestra.run.smithi112.stdout:^* hv01.front.sepia.ceph.com 3 6 377 21 +343us[ +347us] +/- 35ms 2024-09-21T00:31:02.842 INFO:teuthology.orchestra.run.smithi112.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 20 -1329us[-1329us] +/- 56ms 2024-09-21T00:31:02.842 INFO:teuthology.orchestra.run.smithi112.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 20 +881us[ +881us] +/- 73ms 2024-09-21T00:31:02.842 INFO:teuthology.orchestra.run.smithi112.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T00:31:02.844 INFO:teuthology.orchestra.run.smithi187.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T00:31:02.844 INFO:teuthology.orchestra.run.smithi187.stdout:=============================================================================== 2024-09-21T00:31:02.844 INFO:teuthology.orchestra.run.smithi187.stdout:^* hv01.front.sepia.ceph.com 3 6 377 21 +374us[ +395us] +/- 35ms 2024-09-21T00:31:02.844 INFO:teuthology.orchestra.run.smithi187.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 21 -1285us[-1285us] +/- 56ms 2024-09-21T00:31:02.844 INFO:teuthology.orchestra.run.smithi187.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 20 +906us[ +906us] +/- 73ms 2024-09-21T00:31:02.844 INFO:teuthology.orchestra.run.smithi187.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T00:31:02.844 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-21T00:31:02.853 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-21T00:31:02.853 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-21T00:31:02.860 DEBUG:teuthology.orchestra.run.smithi112:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-21T00:31:02.968 DEBUG:teuthology.orchestra.run.smithi187:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-21T00:31:03.039 DEBUG:teuthology.orchestra.run.smithi112:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-21T00:31:03.069 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T00:31:03.069 DEBUG:teuthology.orchestra.run.smithi187:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-21T00:31:03.097 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T00:31:03.097 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-21T00:31:03.107 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-21T00:31:03.116 INFO:teuthology.task.internal:Duration was 704.652427 seconds 2024-09-21T00:31:03.116 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-21T00:31:03.124 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-21T00:31:03.125 DEBUG:teuthology.orchestra.run.smithi112:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-21T00:31:03.127 DEBUG:teuthology.orchestra.run.smithi187:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-21T00:31:03.165 INFO:teuthology.orchestra.run.smithi112.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T00:31:03.176 INFO:teuthology.orchestra.run.smithi187.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T00:31:03.459 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-21T00:31:03.459 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi112.front.sepia.ceph.com 2024-09-21T00:31:03.459 DEBUG:teuthology.orchestra.run.smithi112:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-21T00:31:03.480 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi187.front.sepia.ceph.com 2024-09-21T00:31:03.480 DEBUG:teuthology.orchestra.run.smithi187:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-21T00:31:03.519 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-21T00:31:03.520 DEBUG:teuthology.orchestra.run.smithi112:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-21T00:31:03.523 DEBUG:teuthology.orchestra.run.smithi187:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-21T00:31:03.586 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-21T00:31:03.587 DEBUG:teuthology.orchestra.run.smithi112:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-21T00:31:03.590 DEBUG:teuthology.orchestra.run.smithi187:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-21T00:31:03.734 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-21T00:31:03.744 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-21T00:31:03.744 DEBUG:teuthology.orchestra.run.smithi112:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-21T00:31:03.769 DEBUG:teuthology.orchestra.run.smithi187:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-21T00:31:03.794 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-21T00:31:03.805 DEBUG:teuthology.orchestra.run.smithi112:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-21T00:31:03.812 DEBUG:teuthology.orchestra.run.smithi187:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-21T00:31:03.833 INFO:teuthology.orchestra.run.smithi112.stdout:kernel.core_pattern = core 2024-09-21T00:31:03.857 INFO:teuthology.orchestra.run.smithi187.stdout:kernel.core_pattern = core 2024-09-21T00:31:03.870 DEBUG:teuthology.orchestra.run.smithi112:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-21T00:31:03.900 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T00:31:03.901 DEBUG:teuthology.orchestra.run.smithi187:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-21T00:31:03.924 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T00:31:03.924 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-21T00:31:03.934 INFO:teuthology.task.internal:Transferring archived files... 2024-09-21T00:31:03.935 DEBUG:teuthology.misc:Transferring archived files from smithi112:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913327/remote/smithi112 2024-09-21T00:31:03.935 DEBUG:teuthology.orchestra.run.smithi112:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-21T00:31:03.976 DEBUG:teuthology.misc:Transferring archived files from smithi187:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913327/remote/smithi187 2024-09-21T00:31:03.976 DEBUG:teuthology.orchestra.run.smithi187:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-21T00:31:04.009 INFO:teuthology.task.internal:Removing archive directory... 2024-09-21T00:31:04.009 DEBUG:teuthology.orchestra.run.smithi112:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-21T00:31:04.013 DEBUG:teuthology.orchestra.run.smithi187:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-21T00:31:04.059 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-21T00:31:04.072 INFO:teuthology.task.internal:Not uploading archives. 2024-09-21T00:31:04.073 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-21T00:31:04.111 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-21T00:31:04.112 DEBUG:teuthology.orchestra.run.smithi112:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-21T00:31:04.114 DEBUG:teuthology.orchestra.run.smithi187:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-21T00:31:04.126 INFO:teuthology.orchestra.run.smithi112.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 21 00:31 /home/ubuntu/cephtest 2024-09-21T00:31:04.129 INFO:teuthology.orchestra.run.smithi187.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 21 00:31 /home/ubuntu/cephtest 2024-09-21T00:31:04.130 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-21T00:31:04.142 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-21T00:31:04.164 INFO:teuthology.run:Summary data: description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/sessionmap} duration: 704.6524274349213 flavor: default owner: scheduled_pdonnell@teuthology success: true 2024-09-21T00:31:04.164 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-21T00:31:04.253 INFO:teuthology.run:pass